2024-04-25T13:26:45.126 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T13:26:45.127 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/mds-full} --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174 --verbose --name pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/orig.config.yaml 2024-04-25T13:26:45.178 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:26:45.274 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174 branch: wip-pdonnell-testing-20240425.015853-debug description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/mds-full} email: pdonnell@redhat.com first_in_suite: false job_id: '7673174' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-pdonnell-testing-20240425.015853-debug ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240425.015853-debug sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 owner: scheduled_pdonnell@teuthology priority: 85 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1880 sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 sleep_before_teardown: 0 subset: 178/512 suite: fs suite_branch: wip-pdonnell-testing-20240425.015853-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi080.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJD/i+Q2yLx/9ITFeoBQlucVgb4Kk7o+CbCc5cFp1bUHTz+Qpf8VBTdCX4eFUrZcWh3VZIUe3fgHGltJ+DTCnM4= smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/VNH3ueL4MNCM4M0mx2qnHZ4LQHlkeJ18toGZD0Wpr4dpJod6IiLgFe8cNZkoqvM2QKta3eYMfoDwwGEMJcUU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_10:00:23 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T13:26:45.275 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa; will attempt to use it 2024-04-25T13:26:45.275 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa/tasks 2024-04-25T13:26:45.275 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T13:26:45.277 INFO:teuthology.task.internal:Checking packages... 2024-04-25T13:26:45.300 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '994e2f20102a20fa9a27966c14b625cc01884df8' 2024-04-25T13:26:45.300 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T13:26:45.301 INFO:teuthology.packaging:ref: None 2024-04-25T13:26:45.301 INFO:teuthology.packaging:tag: None 2024-04-25T13:26:45.301 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:26:45.301 INFO:teuthology.packaging:sha1: 994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:26:45.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-pdonnell-testing-20240425.015853-debug 2024-04-25T13:26:45.447 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:26:45.448 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T13:26:45.455 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T13:26:45.455 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T13:26:45.462 INFO:teuthology.task.internal:Saving configuration 2024-04-25T13:26:45.480 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T13:26:45.487 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T13:26:45.513 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi080.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:19:34.367804', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIO+L1wVwBi0OoRGxJen6pWVZnE0+YlhIDiCoQ179/XiW'} 2024-04-25T13:26:45.536 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 13:19:34.369126', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJin9jYoowtMl6L+ngA7i3KkJDek2ZbZwIx1mQKzrOkc'} 2024-04-25T13:26:45.537 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T13:26:45.543 INFO:teuthology.task.internal:roles: ubuntu@smithi080.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T13:26:45.543 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T13:26:45.543 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T13:26:45.616 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f3401f6d3a0>, signals=[15]) 2024-04-25T13:26:45.616 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T13:26:45.623 INFO:teuthology.task.internal:Opening connections... 2024-04-25T13:26:45.623 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:26:45.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:26:45.706 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:26:45.707 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:26:45.785 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T13:26:45.791 DEBUG:teuthology.orchestra.run.smithi080:> uname -m 2024-04-25T13:26:45.796 INFO:teuthology.orchestra.run.smithi080.stdout:x86_64 2024-04-25T13:26:45.797 DEBUG:teuthology.orchestra.run.smithi080:> cat /etc/os-release 2024-04-25T13:26:45.846 INFO:teuthology.orchestra.run.smithi080.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T13:26:45.846 INFO:teuthology.orchestra.run.smithi080.stdout:NAME="Ubuntu" 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_ID="22.04" 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:VERSION_CODENAME=jammy 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:ID=ubuntu 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:ID_LIKE=debian 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T13:26:45.847 INFO:teuthology.orchestra.run.smithi080.stdout:UBUNTU_CODENAME=jammy 2024-04-25T13:26:45.848 INFO:teuthology.lock.ops:Updating smithi080.front.sepia.ceph.com on lock server 2024-04-25T13:26:45.870 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-04-25T13:26:45.875 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-04-25T13:26:45.875 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-04-25T13:26:45.921 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-25T13:26:45.921 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="Ubuntu" 2024-04-25T13:26:45.921 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="22.04" 2024-04-25T13:26:45.921 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-25T13:26:45.922 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_CODENAME=jammy 2024-04-25T13:26:45.922 INFO:teuthology.orchestra.run.smithi087.stdout:ID=ubuntu 2024-04-25T13:26:45.922 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE=debian 2024-04-25T13:26:45.922 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-25T13:26:45.922 INFO:teuthology.orchestra.run.smithi087.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-25T13:26:45.922 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-25T13:26:45.922 INFO:teuthology.orchestra.run.smithi087.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-25T13:26:45.922 INFO:teuthology.orchestra.run.smithi087.stdout:UBUNTU_CODENAME=jammy 2024-04-25T13:26:45.922 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-04-25T13:26:45.953 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T13:26:45.965 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T13:26:45.971 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T13:26:45.971 DEBUG:teuthology.orchestra.run.smithi080:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:26:45.974 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-04-25T13:26:45.978 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T13:26:45.985 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T13:26:45.985 DEBUG:teuthology.orchestra.run.smithi080:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:26:46.024 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-04-25T13:26:46.030 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T13:26:46.085 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T13:26:46.102 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T13:26:46.102 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T13:26:46.103 DEBUG:teuthology.orchestra.run.smithi080:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:26:46.103 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T13:26:46.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:46.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:26:46.108 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-04-25T13:26:46.108 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-04-25T13:26:46.154 INFO:teuthology.orchestra.run.smithi087.stdout:5.15.0-56-generic 2024-04-25T13:26:46.154 INFO:teuthology.task.kernel:Running kernel on smithi087: 5.15.0-56-generic 2024-04-25T13:26:46.154 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get clean 2024-04-25T13:26:46.155 INFO:teuthology.orchestra.run.smithi080.stdout:5.15.0-56-generic 2024-04-25T13:26:46.155 INFO:teuthology.task.kernel:Running kernel on smithi080: 5.15.0-56-generic 2024-04-25T13:26:46.155 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2024-04-25T13:26:46.279 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get update 2024-04-25T13:26:46.285 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-04-25T13:26:46.502 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T13:26:46.547 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T13:26:46.548 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-25T13:26:46.569 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T13:26:46.607 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T13:26:46.636 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-25T13:26:46.795 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T13:26:46.894 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T13:26:46.935 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T13:26:46.938 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T13:26:46.938 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T13:26:46.967 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T13:26:46.967 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T13:26:46.968 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,394 kB] 2024-04-25T13:26:46.968 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T13:26:46.969 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T13:26:46.981 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T13:26:46.986 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T13:26:46.994 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T13:26:46.994 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T13:26:46.994 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T13:26:46.994 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T13:26:47.002 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T13:26:47.002 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T13:26:47.002 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-25T13:26:47.109 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T13:26:47.136 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T13:26:47.337 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,610 kB] 2024-04-25T13:26:47.392 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [452 kB] 2024-04-25T13:26:47.409 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [243 kB] 2024-04-25T13:26:47.418 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-25T13:26:47.419 INFO:teuthology.orchestra.run.smithi087.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [619 kB] 2024-04-25T13:26:47.419 INFO:teuthology.orchestra.run.smithi087.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [36.5 kB] 2024-04-25T13:26:47.420 INFO:teuthology.orchestra.run.smithi087.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,773 kB] 2024-04-25T13:26:47.457 INFO:teuthology.orchestra.run.smithi087.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T13:26:47.472 INFO:teuthology.orchestra.run.smithi087.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T13:26:47.473 INFO:teuthology.orchestra.run.smithi087.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T13:26:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [303 kB] 2024-04-25T13:26:47.502 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-25T13:26:47.502 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T13:26:47.504 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,830 kB] 2024-04-25T13:26:47.513 INFO:teuthology.orchestra.run.smithi087.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [300 kB] 2024-04-25T13:26:47.546 INFO:teuthology.orchestra.run.smithi087.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-25T13:26:47.546 INFO:teuthology.orchestra.run.smithi087.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T13:26:47.546 INFO:teuthology.orchestra.run.smithi087.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-25T13:26:47.547 INFO:teuthology.orchestra.run.smithi087.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T13:26:47.554 INFO:teuthology.orchestra.run.smithi087.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T13:26:47.562 INFO:teuthology.orchestra.run.smithi087.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [601 kB] 2024-04-25T13:26:47.567 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [311 kB] 2024-04-25T13:26:47.573 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-25T13:26:47.718 INFO:teuthology.orchestra.run.smithi087.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T13:26:47.738 INFO:teuthology.orchestra.run.smithi087.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-25T13:26:47.738 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-25T13:26:48.142 INFO:teuthology.orchestra.run.smithi087.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-25T13:26:48.147 INFO:teuthology.orchestra.run.smithi087.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-25T13:26:48.162 INFO:teuthology.orchestra.run.smithi087.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-25T13:26:48.162 INFO:teuthology.orchestra.run.smithi087.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-25T13:26:48.165 INFO:teuthology.orchestra.run.smithi087.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-25T13:26:48.347 INFO:teuthology.orchestra.run.smithi087.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T13:26:48.364 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [700 kB] 2024-04-25T13:26:48.384 INFO:teuthology.orchestra.run.smithi087.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T13:26:48.398 INFO:teuthology.orchestra.run.smithi087.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T13:26:48.399 INFO:teuthology.orchestra.run.smithi087.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T13:26:48.399 INFO:teuthology.orchestra.run.smithi087.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T13:26:48.403 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-25T13:26:48.403 INFO:teuthology.orchestra.run.smithi087.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T13:26:48.404 INFO:teuthology.orchestra.run.smithi087.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T13:26:48.404 INFO:teuthology.orchestra.run.smithi087.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-25T13:26:48.415 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-25T13:26:48.416 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-25T13:26:48.416 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-25T13:26:48.418 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-25T13:26:48.419 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-25T13:26:48.419 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-25T13:26:48.430 INFO:teuthology.orchestra.run.smithi087.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.3 kB] 2024-04-25T13:26:48.432 INFO:teuthology.orchestra.run.smithi087.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T13:26:48.445 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-25T13:26:48.448 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-25T13:26:48.596 INFO:teuthology.orchestra.run.smithi087.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T13:26:48.613 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-25T13:26:48.679 INFO:teuthology.orchestra.run.smithi087.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-25T13:26:48.695 INFO:teuthology.orchestra.run.smithi080.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [16.0 kB] 2024-04-25T13:26:48.760 INFO:teuthology.orchestra.run.smithi087.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-25T13:26:48.777 INFO:teuthology.orchestra.run.smithi080.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [27.2 kB] 2024-04-25T13:26:48.814 INFO:teuthology.orchestra.run.smithi087.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T13:26:48.858 INFO:teuthology.orchestra.run.smithi080.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-25T13:26:48.863 INFO:teuthology.orchestra.run.smithi087.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T13:26:48.889 INFO:teuthology.orchestra.run.smithi080.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-25T13:26:53.173 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 13.2 MB in 3s (5,256 kB/s) 2024-04-25T13:26:53.314 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 13.2 MB in 3s (5,257 kB/s) 2024-04-25T13:26:54.628 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T13:26:54.646 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T13:26:54.698 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T13:26:54.709 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T13:26:54.715 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T13:26:54.775 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T13:26:54.907 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-04-25T13:26:54.908 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-04-25T13:26:54.970 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-04-25T13:26:54.971 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-04-25T13:26:55.082 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:26:55.082 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:26:55.082 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:26:55.083 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:26:55.083 INFO:teuthology.orchestra.run.smithi080.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T13:26:55.083 INFO:teuthology.orchestra.run.smithi080.stdout: python2.7-minimal 2024-04-25T13:26:55.083 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:26:55.084 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-04-25T13:26:55.084 INFO:teuthology.orchestra.run.smithi080.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T13:26:55.084 INFO:teuthology.orchestra.run.smithi080.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T13:26:55.084 INFO:teuthology.orchestra.run.smithi080.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T13:26:55.085 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-04-25T13:26:55.085 INFO:teuthology.orchestra.run.smithi080.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T13:26:55.086 INFO:teuthology.orchestra.run.smithi080.stdout:Recommended packages: 2024-04-25T13:26:55.086 INFO:teuthology.orchestra.run.smithi080.stdout: thermald 2024-04-25T13:26:55.134 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-04-25T13:26:55.135 INFO:teuthology.orchestra.run.smithi080.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T13:26:55.135 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T13:26:55.135 INFO:teuthology.orchestra.run.smithi080.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T13:26:55.136 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:26:55.136 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:26:55.136 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:26:55.136 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:26:55.136 INFO:teuthology.orchestra.run.smithi087.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-25T13:26:55.136 INFO:teuthology.orchestra.run.smithi087.stdout: python2.7-minimal 2024-04-25T13:26:55.136 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:26:55.136 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages will be upgraded: 2024-04-25T13:26:55.137 INFO:teuthology.orchestra.run.smithi080.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T13:26:55.137 INFO:teuthology.orchestra.run.smithi087.stdout:The following additional packages will be installed: 2024-04-25T13:26:55.137 INFO:teuthology.orchestra.run.smithi087.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T13:26:55.137 INFO:teuthology.orchestra.run.smithi087.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-25T13:26:55.138 INFO:teuthology.orchestra.run.smithi087.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-25T13:26:55.139 INFO:teuthology.orchestra.run.smithi087.stdout:Suggested packages: 2024-04-25T13:26:55.139 INFO:teuthology.orchestra.run.smithi087.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-25T13:26:55.139 INFO:teuthology.orchestra.run.smithi087.stdout:Recommended packages: 2024-04-25T13:26:55.139 INFO:teuthology.orchestra.run.smithi087.stdout: thermald 2024-04-25T13:26:55.190 INFO:teuthology.orchestra.run.smithi087.stdout:The following NEW packages will be installed: 2024-04-25T13:26:55.190 INFO:teuthology.orchestra.run.smithi087.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-25T13:26:55.190 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-25T13:26:55.190 INFO:teuthology.orchestra.run.smithi087.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-25T13:26:55.191 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages will be upgraded: 2024-04-25T13:26:55.191 INFO:teuthology.orchestra.run.smithi087.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-25T13:26:55.195 INFO:teuthology.orchestra.run.smithi080.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T13:26:55.195 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 113 MB of archives. 2024-04-25T13:26:55.195 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T13:26:55.195 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T13:26:55.371 INFO:teuthology.orchestra.run.smithi087.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T13:26:55.371 INFO:teuthology.orchestra.run.smithi087.stdout:Need to get 113 MB of archives. 2024-04-25T13:26:55.371 INFO:teuthology.orchestra.run.smithi087.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-25T13:26:55.371 INFO:teuthology.orchestra.run.smithi087.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-25T13:26:55.714 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T13:26:55.862 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T13:26:56.895 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-25T13:26:57.136 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T13:26:57.136 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T13:26:57.136 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T13:26:57.312 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-25T13:26:57.450 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T13:26:57.501 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T13:26:57.919 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 113 MB in 2s (48.1 MB/s) 2024-04-25T13:26:58.082 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T13:26:59.490 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-25T13:26:59.490 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-25T13:26:59.491 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-25T13:27:00.155 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T13:27:00.159 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T13:27:00.254 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:00.334 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-25T13:27:00.458 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-25T13:27:00.861 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 113 MB in 5s (21.5 MB/s) 2024-04-25T13:27:01.033 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-25T13:27:02.422 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T13:27:02.443 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T13:27:02.548 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:02.983 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T13:27:03.004 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T13:27:03.033 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:03.194 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-25T13:27:03.198 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T13:27:03.454 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:05.554 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-25T13:27:05.575 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T13:27:05.658 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:06.074 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-25T13:27:06.094 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T13:27:06.124 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:09.932 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T13:27:10.059 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T13:27:10.392 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T13:27:10.521 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T13:27:10.832 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T13:27:10.854 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T13:27:10.890 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T13:27:12.563 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T13:27:12.663 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T13:27:13.020 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T13:27:13.141 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T13:27:13.444 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-25T13:27:13.455 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-25T13:27:13.502 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T13:27:16.883 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T13:27:16.920 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T13:27:16.958 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:18.767 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-25T13:27:18.804 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-25T13:27:18.841 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:19.554 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T13:27:19.677 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T13:27:20.002 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T13:27:20.124 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:20.233 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T13:27:20.338 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:21.373 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-25T13:27:21.494 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-25T13:27:21.660 INFO:teuthology.orchestra.run.smithi080.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:27:21.661 INFO:teuthology.orchestra.run.smithi080.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T13:27:21.661 INFO:teuthology.orchestra.run.smithi080.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T13:27:21.661 INFO:teuthology.orchestra.run.smithi080.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T13:27:21.828 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-25T13:27:21.949 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:22.269 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:22.809 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-25T13:27:22.938 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:24.050 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:24.248 INFO:teuthology.orchestra.run.smithi087.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:27:24.249 INFO:teuthology.orchestra.run.smithi087.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-25T13:27:24.249 INFO:teuthology.orchestra.run.smithi087.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-25T13:27:24.249 INFO:teuthology.orchestra.run.smithi087.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-25T13:27:24.431 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:25.425 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T13:27:25.534 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T13:27:25.669 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:25.735 INFO:teuthology.orchestra.run.smithi080.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T13:27:25.735 INFO:teuthology.orchestra.run.smithi080.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T13:27:25.808 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:27.201 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-25T13:27:27.318 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-25T13:27:27.795 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-25T13:27:28.763 INFO:teuthology.orchestra.run.smithi087.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-25T13:27:28.763 INFO:teuthology.orchestra.run.smithi087.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-25T13:27:39.860 INFO:teuthology.orchestra.run.smithi080.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T13:27:39.860 INFO:teuthology.orchestra.run.smithi080.stdout:Sourcing file `/etc/default/grub' 2024-04-25T13:27:39.877 INFO:teuthology.orchestra.run.smithi080.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T13:27:39.895 INFO:teuthology.orchestra.run.smithi080.stdout:Generating grub configuration file ... 2024-04-25T13:27:40.172 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T13:27:40.192 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T13:27:40.511 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:27:40.516 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T13:27:40.590 INFO:teuthology.orchestra.run.smithi080.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T13:27:40.595 INFO:teuthology.orchestra.run.smithi080.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T13:27:40.755 INFO:teuthology.orchestra.run.smithi080.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T13:27:40.755 INFO:teuthology.orchestra.run.smithi080.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T13:27:40.755 INFO:teuthology.orchestra.run.smithi080.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T13:27:40.806 INFO:teuthology.orchestra.run.smithi080.stdout:done 2024-04-25T13:27:41.301 DEBUG:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 21 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2024-04-25T13:27:41.322 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.15.0.105.102 2024-04-25T13:27:41.323 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T13:27:41.323 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T13:27:41.323 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2024-04-25T13:27:41.323 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2024-04-25T13:27:41.323 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2024-04-25T13:27:41.323 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2024-04-25T13:27:41.323 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T13:27:41.323 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T13:27:41.323 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T13:27:41.323 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-25T13:27:41.324 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi080.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T13:27:41.324 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get clean 2024-04-25T13:27:41.509 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-04-25T13:27:41.777 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T13:27:41.777 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T13:27:41.835 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T13:27:41.917 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T13:27:42.534 INFO:teuthology.orchestra.run.smithi087.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-25T13:27:42.534 INFO:teuthology.orchestra.run.smithi087.stdout:Sourcing file `/etc/default/grub' 2024-04-25T13:27:42.553 INFO:teuthology.orchestra.run.smithi087.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T13:27:42.573 INFO:teuthology.orchestra.run.smithi087.stdout:Generating grub configuration file ... 2024-04-25T13:27:42.844 INFO:teuthology.orchestra.run.smithi087.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T13:27:42.882 INFO:teuthology.orchestra.run.smithi087.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T13:27:43.151 INFO:teuthology.orchestra.run.smithi087.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:27:43.156 INFO:teuthology.orchestra.run.smithi087.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T13:27:43.187 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T13:27:43.205 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T13:27:43.222 INFO:teuthology.orchestra.run.smithi087.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T13:27:43.227 INFO:teuthology.orchestra.run.smithi087.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T13:27:43.264 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T13:27:43.392 INFO:teuthology.orchestra.run.smithi087.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T13:27:43.392 INFO:teuthology.orchestra.run.smithi087.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T13:27:43.392 INFO:teuthology.orchestra.run.smithi087.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T13:27:43.433 INFO:teuthology.orchestra.run.smithi087.stdout:done 2024-04-25T13:27:43.473 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-04-25T13:27:43.474 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-04-25T13:27:43.644 INFO:teuthology.orchestra.run.smithi080.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T13:27:43.644 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:27:43.644 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:27:43.644 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:27:43.644 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:27:43.644 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T13:27:43.645 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T13:27:43.645 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:27:43.683 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T13:27:43.684 DEBUG:teuthology.orchestra.run.smithi080:> dpkg -s linux-image-generic 2024-04-25T13:27:43.700 INFO:teuthology.orchestra.run.smithi080.stdout:Package: linux-image-generic 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Status: install ok installed 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Priority: optional 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Section: kernel 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Installed-Size: 21 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Architecture: amd64 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Source: linux-meta 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Version: 5.15.0.105.102 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Recommends: thermald 2024-04-25T13:27:43.701 INFO:teuthology.orchestra.run.smithi080.stdout:Description: Generic Linux kernel image 2024-04-25T13:27:43.702 INFO:teuthology.orchestra.run.smithi080.stdout: This package will always depend on the latest generic kernel image 2024-04-25T13:27:43.702 INFO:teuthology.orchestra.run.smithi080.stdout: available. 2024-04-25T13:27:43.702 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-04-25T13:27:43.749 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.Gs3vORVwwy 2024-04-25T13:27:43.749 DEBUG:teuthology.orchestra.run.smithi080:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Gs3vORVwwy 2024-04-25T13:27:43.806 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.Gs3vORVwwy 2024-04-25T13:27:43.899 DEBUG:teuthology.orchestra.run.smithi087:> dpkg -s linux-image-generic 2024-04-25T13:27:43.916 INFO:teuthology.orchestra.run.smithi087.stdout:Package: linux-image-generic 2024-04-25T13:27:43.916 INFO:teuthology.orchestra.run.smithi087.stdout:Status: install ok installed 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Priority: optional 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Section: kernel 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Installed-Size: 21 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: amd64 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Source: linux-meta 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Version: 5.15.0.105.102 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Recommends: thermald 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout:Description: Generic Linux kernel image 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout: This package will always depend on the latest generic kernel image 2024-04-25T13:27:43.917 INFO:teuthology.orchestra.run.smithi087.stdout: available. 2024-04-25T13:27:43.918 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-25T13:27:43.918 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-25T13:27:43.918 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-25T13:27:43.918 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-25T13:27:43.918 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2024-04-25T13:27:43.918 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get clean 2024-04-25T13:27:43.930 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.Gs3vORVwwy is 10KB 2024-04-25T13:27:43.943 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.Gs3vORVwwy 2024-04-25T13:27:43.948 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T13:27:44.002 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:27:44.002 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T13:27:44.002 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T13:27:44.068 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T13:27:44.068 DEBUG:teuthology.orchestra.run.smithi080:> sudo update-grub 2024-04-25T13:27:44.102 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get update 2024-04-25T13:27:44.234 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T13:27:44.234 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T13:27:44.256 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T13:27:44.280 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T13:27:45.306 INFO:teuthology.orchestra.run.smithi080.stderr:Sourcing file `/etc/default/grub' 2024-04-25T13:27:45.307 INFO:teuthology.orchestra.run.smithi080.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T13:27:45.310 INFO:teuthology.orchestra.run.smithi080.stderr:Generating grub configuration file ... 2024-04-25T13:27:45.482 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T13:27:45.488 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T13:27:45.597 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T13:27:45.614 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-25T13:27:45.673 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T13:27:45.726 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:27:45.731 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T13:27:45.774 INFO:teuthology.orchestra.run.smithi080.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T13:27:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T13:27:45.871 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-04-25T13:27:45.872 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-04-25T13:27:45.924 INFO:teuthology.orchestra.run.smithi080.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T13:27:45.924 INFO:teuthology.orchestra.run.smithi080.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T13:27:45.924 INFO:teuthology.orchestra.run.smithi080.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T13:27:45.941 INFO:teuthology.orchestra.run.smithi080.stderr:done 2024-04-25T13:27:45.957 DEBUG:teuthology.orchestra.run.smithi080:> sudo shutdown -r now 2024-04-25T13:27:46.031 INFO:teuthology.orchestra.run.smithi087.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-25T13:27:46.031 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:27:46.031 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:27:46.031 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:27:46.032 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:27:46.032 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T13:27:46.032 INFO:teuthology.orchestra.run.smithi087.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T13:27:46.032 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:27:46.072 INFO:teuthology.orchestra.run.smithi087.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-25T13:27:46.074 DEBUG:teuthology.orchestra.run.smithi087:> dpkg -s linux-image-generic 2024-04-25T13:27:46.089 INFO:teuthology.orchestra.run.smithi087.stdout:Package: linux-image-generic 2024-04-25T13:27:46.089 INFO:teuthology.orchestra.run.smithi087.stdout:Status: install ok installed 2024-04-25T13:27:46.089 INFO:teuthology.orchestra.run.smithi087.stdout:Priority: optional 2024-04-25T13:27:46.089 INFO:teuthology.orchestra.run.smithi087.stdout:Section: kernel 2024-04-25T13:27:46.089 INFO:teuthology.orchestra.run.smithi087.stdout:Installed-Size: 21 2024-04-25T13:27:46.089 INFO:teuthology.orchestra.run.smithi087.stdout:Maintainer: Ubuntu Kernel Team 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: amd64 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout:Source: linux-meta 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout:Version: 5.15.0.105.102 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout:Recommends: thermald 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout:Description: Generic Linux kernel image 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout: This package will always depend on the latest generic kernel image 2024-04-25T13:27:46.090 INFO:teuthology.orchestra.run.smithi087.stdout: available. 2024-04-25T13:27:46.090 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-04-25T13:27:46.136 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.EsFRwuKBRn 2024-04-25T13:27:46.137 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub/grub.cfg /tmp/tmp.EsFRwuKBRn 2024-04-25T13:27:46.192 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.EsFRwuKBRn 2024-04-25T13:27:46.305 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.EsFRwuKBRn is 10KB 2024-04-25T13:27:46.318 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.EsFRwuKBRn 2024-04-25T13:27:46.323 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-25T13:27:46.378 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:27:46.378 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-25T13:27:46.378 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-25T13:27:46.443 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-25T13:27:46.444 DEBUG:teuthology.orchestra.run.smithi087:> sudo update-grub 2024-04-25T13:27:47.506 INFO:teuthology.orchestra.run.smithi087.stderr:Sourcing file `/etc/default/grub' 2024-04-25T13:27:47.507 INFO:teuthology.orchestra.run.smithi087.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-25T13:27:47.509 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-04-25T13:27:47.682 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-25T13:27:47.688 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-25T13:27:47.915 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-25T13:27:47.920 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-25T13:27:47.967 INFO:teuthology.orchestra.run.smithi087.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-25T13:27:47.973 INFO:teuthology.orchestra.run.smithi087.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-25T13:27:48.125 INFO:teuthology.orchestra.run.smithi087.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-25T13:27:48.126 INFO:teuthology.orchestra.run.smithi087.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-25T13:27:48.126 INFO:teuthology.orchestra.run.smithi087.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-25T13:27:48.141 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-04-25T13:27:48.149 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2024-04-25T13:28:15.986 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-25T13:28:15.987 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T13:28:15.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:28:18.196 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-25T13:28:18.197 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T13:28:18.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:28:34.339 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-04-25T13:28:36.638 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-04-25T13:28:43.346 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T13:28:43.347 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:28:45.641 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T13:28:45.642 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:28:46.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-04-25T13:28:48.702 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-04-25T13:28:58.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T13:28:58.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:29:00.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T13:29:00.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:29:16.831 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.80 2024-04-25T13:29:19.138 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-04-25T13:29:31.842 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T13:29:31.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:29:32.170 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-04-25T13:29:33.377 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T13:29:33.377 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-25T13:29:33.377 DEBUG:teuthology.orchestra.run.smithi080:> uname -r 2024-04-25T13:29:33.423 INFO:teuthology.orchestra.run.smithi080.stdout:5.15.0-105-generic 2024-04-25T13:29:33.423 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T13:29:33.423 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T13:29:33.423 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T13:29:34.139 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T13:29:34.140 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:29:34.424 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T13:29:34.425 DEBUG:teuthology.orchestra.run.smithi080:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T13:29:34.580 INFO:teuthology.orchestra.run.smithi080.stdout:ttyS1 2024-04-25T13:29:34.606 DEBUG:teuthology.parallel:result is None 2024-04-25T13:29:34.984 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-04-25T13:29:35.551 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T13:29:35.551 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-25T13:29:35.551 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-04-25T13:29:35.598 INFO:teuthology.orchestra.run.smithi087.stdout:5.15.0-105-generic 2024-04-25T13:29:35.599 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-25T13:29:35.599 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T13:29:35.599 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-25T13:29:36.600 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T13:29:36.600 DEBUG:teuthology.orchestra.run.smithi087:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T13:29:36.698 INFO:teuthology.orchestra.run.smithi087.stdout:ttyS1 2024-04-25T13:29:36.732 DEBUG:teuthology.parallel:result is None 2024-04-25T13:29:36.733 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T13:29:36.739 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T13:29:36.740 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:29:36.742 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T13:29:36.748 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T13:29:36.755 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T13:29:36.761 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T13:29:36.761 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:29:36.789 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T13:29:36.812 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T13:29:36.818 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T13:29:36.819 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:29:36.848 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T13:29:36.874 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:29:36.878 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:29:36.883 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:29:36.886 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T13:29:36.888 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T13:29:36.896 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T13:29:36.896 DEBUG:teuthology.orchestra.run.smithi080:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:29:36.932 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T13:29:36.970 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T13:29:36.979 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T13:29:36.979 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:29:36.988 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T13:29:37.019 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:29:37.040 DEBUG:teuthology.orchestra.run.smithi080:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:29:37.089 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:29:37.089 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:29:37.148 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T13:29:37.153 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T13:29:37.199 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:29:37.199 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T13:29:37.271 DEBUG:teuthology.orchestra.run.smithi080:> sudo service rsyslog restart 2024-04-25T13:29:37.274 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-04-25T13:29:37.362 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T13:29:37.369 INFO:teuthology.task.internal:Starting timer... 2024-04-25T13:29:37.369 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T13:29:37.379 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T13:29:37.387 DEBUG:teuthology.task.selinux:Excluding smithi080: OS 'ubuntu' does not support SELinux 2024-04-25T13:29:37.387 DEBUG:teuthology.task.selinux:Excluding smithi087: OS 'ubuntu' does not support SELinux 2024-04-25T13:29:37.388 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T13:29:37.388 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-25T13:29:37.388 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T13:29:37.388 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T13:29:37.396 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T13:29:37.413 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T13:29:37.415 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi080.front.sepia.ceph.com,smithi087.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T13:36:38.771 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi080.front.sepia.ceph.com'), Remote(name='ubuntu@smithi087.front.sepia.ceph.com')] 2024-04-25T13:36:38.773 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T13:36:38.774 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi080.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:36:38.855 DEBUG:teuthology.orchestra.run.smithi080:> true 2024-04-25T13:36:38.930 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi080.front.sepia.ceph.com' 2024-04-25T13:36:38.931 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T13:36:38.931 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T13:36:39.008 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-04-25T13:36:39.086 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-04-25T13:36:39.087 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T13:36:39.097 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T13:36:39.097 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:36:39.098 DEBUG:teuthology.orchestra.run.smithi080:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:36:39.100 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T13:36:39.100 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:36:39.128 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T13:36:39.128 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Command line: ntpd -gq 2024-04-25T13:36:39.128 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: ---------------------------------------------------- 2024-04-25T13:36:39.128 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T13:36:39.128 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T13:36:39.128 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: corporation. Support and training for ntp-4 are 2024-04-25T13:36:39.128 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: available at https://www.nwtime.org/support 2024-04-25T13:36:39.128 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: ---------------------------------------------------- 2024-04-25T13:36:39.129 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: proto: precision = 0.043 usec (-24) 2024-04-25T13:36:39.129 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: basedate set to 2022-02-04 2024-04-25T13:36:39.129 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: gps base set to 2022-02-06 (week 2196) 2024-04-25T13:36:39.129 INFO:teuthology.orchestra.run.smithi080.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T13:36:39.129 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T13:36:39.129 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T13:36:39.130 INFO:teuthology.orchestra.run.smithi080.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T13:36:39.131 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T13:36:39.131 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T13:36:39.131 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T13:36:39.131 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Listen normally on 3 enp3s0f1 172.21.15.80:123 2024-04-25T13:36:39.131 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Listen normally on 4 lo [::1]:123 2024-04-25T13:36:39.131 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:cba9%5]:123 2024-04-25T13:36:39.131 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:39 ntpd[17234]: Listening on routing socket on fd #22 for interface updates 2024-04-25T13:36:39.156 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-25T13:36:39.156 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Command line: ntpd -gq 2024-04-25T13:36:39.156 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: ---------------------------------------------------- 2024-04-25T13:36:39.156 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: ntp-4 is maintained by Network Time Foundation, 2024-04-25T13:36:39.156 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-25T13:36:39.156 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: corporation. Support and training for ntp-4 are 2024-04-25T13:36:39.156 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: available at https://www.nwtime.org/support 2024-04-25T13:36:39.157 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: ---------------------------------------------------- 2024-04-25T13:36:39.157 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: proto: precision = 0.055 usec (-24) 2024-04-25T13:36:39.157 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: basedate set to 2022-02-04 2024-04-25T13:36:39.157 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: gps base set to 2022-02-06 (week 2196) 2024-04-25T13:36:39.157 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T13:36:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-25T13:36:39.158 INFO:teuthology.orchestra.run.smithi087.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-25T13:36:39.159 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: restrict ::: KOD does nothing without LIMITED. 2024-04-25T13:36:39.159 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Listen and drop on 0 v6wildcard [::]:123 2024-04-25T13:36:39.159 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-25T13:36:39.159 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Listen normally on 2 lo 127.0.0.1:123 2024-04-25T13:36:39.160 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Listen normally on 3 enp3s0f1 172.21.15.87:123 2024-04-25T13:36:39.160 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Listen normally on 4 lo [::1]:123 2024-04-25T13:36:39.160 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d5f5%5]:123 2024-04-25T13:36:39.160 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:39 ntpd[17208]: Listening on routing socket on fd #22 for interface updates 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:40.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:40.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:40.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:40.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:40 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:42.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:42.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:42.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:42.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:42 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:42.157 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:42 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:42.157 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:42 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:42.157 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:42 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:42.157 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:42 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:42.157 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:42 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:42.157 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:42 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:43.156 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:43 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:43.157 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:43 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:44.131 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:44 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:44.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:44 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:46.129 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:46.130 INFO:teuthology.orchestra.run.smithi080.stdout:25 Apr 13:36:46 ntpd[17234]: ntpd: time slew -0.003106 s 2024-04-25T13:36:46.130 INFO:teuthology.orchestra.run.smithi080.stdout:ntpd: time slew -0.003106s 2024-04-25T13:36:46.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T13:36:46.130 INFO:teuthology.orchestra.run.smithi080.stderr:25 Apr 13:36:46 ntpd[17234]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:46.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:46.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:46 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:46.187 INFO:teuthology.orchestra.run.smithi080.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T13:36:46.187 INFO:teuthology.orchestra.run.smithi080.stdout:============================================================================== 2024-04-25T13:36:46.187 INFO:teuthology.orchestra.run.smithi080.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:36:46.188 INFO:teuthology.orchestra.run.smithi080.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:36:46.188 INFO:teuthology.orchestra.run.smithi080.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:36:46.188 INFO:teuthology.orchestra.run.smithi080.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:36:48.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:48.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:48.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:48.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:48.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:48.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:48.158 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/rawstats.20240425: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/peerstats.20240425: Permission denied 2024-04-25T13:36:48.159 INFO:teuthology.orchestra.run.smithi087.stdout:25 Apr 13:36:48 ntpd[17208]: ntpd: time slew -0.001529 s 2024-04-25T13:36:48.160 INFO:teuthology.orchestra.run.smithi087.stdout:ntpd: time slew -0.001529s 2024-04-25T13:36:48.160 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-25T13:36:48.160 INFO:teuthology.orchestra.run.smithi087.stderr:25 Apr 13:36:48 ntpd[17208]: can't open /var/log/ntpstats/loopstats.20240425: Permission denied 2024-04-25T13:36:48.211 INFO:teuthology.orchestra.run.smithi087.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T13:36:48.211 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================== 2024-04-25T13:36:48.211 INFO:teuthology.orchestra.run.smithi087.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:36:48.211 INFO:teuthology.orchestra.run.smithi087.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:36:48.211 INFO:teuthology.orchestra.run.smithi087.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:36:48.211 INFO:teuthology.orchestra.run.smithi087.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-25T13:36:48.212 INFO:teuthology.run_tasks:Running task install... 2024-04-25T13:36:48.220 DEBUG:teuthology.task.install:project ceph 2024-04-25T13:36:48.221 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}} 2024-04-25T13:36:48.221 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'} 2024-04-25T13:36:48.221 INFO:teuthology.task.install:Using flavor: default 2024-04-25T13:36:48.226 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T13:36:48.226 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T13:36:48.227 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-key list | grep Ceph 2024-04-25T13:36:48.227 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-key list | grep Ceph 2024-04-25T13:36:48.295 INFO:teuthology.orchestra.run.smithi080.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T13:36:48.311 INFO:teuthology.orchestra.run.smithi087.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-25T13:36:48.325 INFO:teuthology.orchestra.run.smithi080.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T13:36:48.325 INFO:teuthology.orchestra.run.smithi080.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T13:36:48.326 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T13:36:48.326 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T13:36:48.326 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:36:48.337 INFO:teuthology.orchestra.run.smithi087.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-25T13:36:48.338 INFO:teuthology.orchestra.run.smithi087.stdout:uid [ unknown] Ceph.com (release key) 2024-04-25T13:36:48.338 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-04-25T13:36:48.338 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-04-25T13:36:48.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:36:48.559 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default/ 2024-04-25T13:36:48.559 INFO:teuthology.task.install.deb:Package version is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:36:48.562 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default/ 2024-04-25T13:36:48.563 INFO:teuthology.task.install.deb:Package version is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:36:48.671 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:36:48.671 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T13:36:48.673 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:36:48.673 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-25T13:36:48.686 DEBUG:teuthology.orchestra.run.smithi080:> sudo apt-get update 2024-04-25T13:36:48.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo apt-get update 2024-04-25T13:36:48.860 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T13:36:48.860 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T13:36:48.861 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-25T13:36:48.861 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-25T13:36:48.872 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T13:36:48.874 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-25T13:36:48.892 INFO:teuthology.orchestra.run.smithi087.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T13:36:48.894 INFO:teuthology.orchestra.run.smithi080.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-25T13:36:48.906 INFO:teuthology.orchestra.run.smithi087.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T13:36:48.927 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T13:36:48.949 INFO:teuthology.orchestra.run.smithi087.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T13:36:48.969 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy InRelease 2024-04-25T13:36:48.970 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-25T13:36:48.997 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-04-25T13:36:49.005 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-25T13:36:49.042 INFO:teuthology.orchestra.run.smithi080.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-25T13:36:49.079 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2024-04-25T13:36:49.120 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-04-25T13:36:49.198 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 29.2 kB in 0s (87.1 kB/s) 2024-04-25T13:36:49.214 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 29.2 kB in 0s (83.2 kB/s) 2024-04-25T13:36:50.123 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T13:36:50.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3291-g994e2f20-1jammy cephadm=19.0.0-3291-g994e2f20-1jammy ceph-mds=19.0.0-3291-g994e2f20-1jammy ceph-mgr=19.0.0-3291-g994e2f20-1jammy ceph-common=19.0.0-3291-g994e2f20-1jammy ceph-fuse=19.0.0-3291-g994e2f20-1jammy ceph-test=19.0.0-3291-g994e2f20-1jammy ceph-volume=19.0.0-3291-g994e2f20-1jammy radosgw=19.0.0-3291-g994e2f20-1jammy python3-rados=19.0.0-3291-g994e2f20-1jammy python3-rgw=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy python3-rbd=19.0.0-3291-g994e2f20-1jammy libcephfs2=19.0.0-3291-g994e2f20-1jammy libcephfs-dev=19.0.0-3291-g994e2f20-1jammy librados2=19.0.0-3291-g994e2f20-1jammy librbd1=19.0.0-3291-g994e2f20-1jammy rbd-fuse=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy cephfs-shell=19.0.0-3291-g994e2f20-1jammy cephfs-top=19.0.0-3291-g994e2f20-1jammy cephfs-mirror=19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:36:50.178 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T13:36:50.194 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-3291-g994e2f20-1jammy cephadm=19.0.0-3291-g994e2f20-1jammy ceph-mds=19.0.0-3291-g994e2f20-1jammy ceph-mgr=19.0.0-3291-g994e2f20-1jammy ceph-common=19.0.0-3291-g994e2f20-1jammy ceph-fuse=19.0.0-3291-g994e2f20-1jammy ceph-test=19.0.0-3291-g994e2f20-1jammy ceph-volume=19.0.0-3291-g994e2f20-1jammy radosgw=19.0.0-3291-g994e2f20-1jammy python3-rados=19.0.0-3291-g994e2f20-1jammy python3-rgw=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy python3-rbd=19.0.0-3291-g994e2f20-1jammy libcephfs2=19.0.0-3291-g994e2f20-1jammy libcephfs-dev=19.0.0-3291-g994e2f20-1jammy librados2=19.0.0-3291-g994e2f20-1jammy librbd1=19.0.0-3291-g994e2f20-1jammy rbd-fuse=19.0.0-3291-g994e2f20-1jammy python3-cephfs=19.0.0-3291-g994e2f20-1jammy cephfs-shell=19.0.0-3291-g994e2f20-1jammy cephfs-top=19.0.0-3291-g994e2f20-1jammy cephfs-mirror=19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:36:50.200 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T13:36:50.257 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T13:36:50.400 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-04-25T13:36:50.400 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-04-25T13:36:50.456 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-04-25T13:36:50.457 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-04-25T13:36:50.584 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:36:50.584 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:36:50.584 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:36:50.584 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:36:50.584 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T13:36:50.585 INFO:teuthology.orchestra.run.smithi087.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T13:36:50.585 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:36:50.585 INFO:teuthology.orchestra.run.smithi087.stdout:The following additional packages will be installed: 2024-04-25T13:36:50.585 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T13:36:50.586 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T13:36:50.587 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T13:36:50.588 INFO:teuthology.orchestra.run.smithi087.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout:Suggested packages: 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout:Recommended packages: 2024-04-25T13:36:50.589 INFO:teuthology.orchestra.run.smithi087.stdout: btrfs-tools 2024-04-25T13:36:50.644 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:36:50.644 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:36:50.644 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:36:50.644 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:36:50.645 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T13:36:50.645 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T13:36:50.645 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:36:50.646 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-04-25T13:36:50.646 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-25T13:36:50.646 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-25T13:36:50.646 INFO:teuthology.orchestra.run.smithi080.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T13:36:50.646 INFO:teuthology.orchestra.run.smithi080.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-25T13:36:50.647 INFO:teuthology.orchestra.run.smithi080.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-25T13:36:50.647 INFO:teuthology.orchestra.run.smithi080.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-25T13:36:50.647 INFO:teuthology.orchestra.run.smithi080.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-04-25T13:36:50.648 INFO:teuthology.orchestra.run.smithi080.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-25T13:36:50.649 INFO:teuthology.orchestra.run.smithi080.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-25T13:36:50.650 INFO:teuthology.orchestra.run.smithi080.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-25T13:36:50.650 INFO:teuthology.orchestra.run.smithi080.stdout:Recommended packages: 2024-04-25T13:36:50.650 INFO:teuthology.orchestra.run.smithi080.stdout: btrfs-tools 2024-04-25T13:36:50.692 INFO:teuthology.orchestra.run.smithi087.stdout:The following NEW packages will be installed: 2024-04-25T13:36:50.692 INFO:teuthology.orchestra.run.smithi087.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T13:36:50.693 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T13:36:50.693 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T13:36:50.693 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T13:36:50.694 INFO:teuthology.orchestra.run.smithi087.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T13:36:50.694 INFO:teuthology.orchestra.run.smithi087.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T13:36:50.694 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T13:36:50.694 INFO:teuthology.orchestra.run.smithi087.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T13:36:50.694 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T13:36:50.694 INFO:teuthology.orchestra.run.smithi087.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T13:36:50.695 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T13:36:50.696 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T13:36:50.696 INFO:teuthology.orchestra.run.smithi087.stdout: socat unzip xmlstarlet zip 2024-04-25T13:36:50.752 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-04-25T13:36:50.752 INFO:teuthology.orchestra.run.smithi080.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-25T13:36:50.752 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-25T13:36:50.752 INFO:teuthology.orchestra.run.smithi080.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-04-25T13:36:50.752 INFO:teuthology.orchestra.run.smithi080.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-04-25T13:36:50.754 INFO:teuthology.orchestra.run.smithi080.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-25T13:36:50.754 INFO:teuthology.orchestra.run.smithi080.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-04-25T13:36:50.754 INFO:teuthology.orchestra.run.smithi080.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-04-25T13:36:50.754 INFO:teuthology.orchestra.run.smithi080.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-04-25T13:36:50.754 INFO:teuthology.orchestra.run.smithi080.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-04-25T13:36:50.754 INFO:teuthology.orchestra.run.smithi080.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-04-25T13:36:50.754 INFO:teuthology.orchestra.run.smithi080.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-04-25T13:36:50.754 INFO:teuthology.orchestra.run.smithi080.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-25T13:36:50.755 INFO:teuthology.orchestra.run.smithi080.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-25T13:36:50.756 INFO:teuthology.orchestra.run.smithi080.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-04-25T13:36:50.756 INFO:teuthology.orchestra.run.smithi080.stdout: socat unzip xmlstarlet zip 2024-04-25T13:36:50.820 INFO:teuthology.orchestra.run.smithi080.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T13:36:50.820 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 175 MB of archives. 2024-04-25T13:36:50.820 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 887 MB of additional disk space will be used. 2024-04-25T13:36:50.820 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T13:36:50.841 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T13:36:50.853 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T13:36:50.883 INFO:teuthology.orchestra.run.smithi087.stdout:0 upgraded, 124 newly installed, 0 to remove and 302 not upgraded. 2024-04-25T13:36:50.883 INFO:teuthology.orchestra.run.smithi087.stdout:Need to get 175 MB of archives. 2024-04-25T13:36:50.883 INFO:teuthology.orchestra.run.smithi087.stdout:After this operation, 887 MB of additional disk space will be used. 2024-04-25T13:36:50.883 INFO:teuthology.orchestra.run.smithi087.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-04-25T13:36:50.902 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T13:36:50.928 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T13:36:50.932 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T13:36:50.945 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T13:36:50.966 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-25T13:36:50.997 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T13:36:51.001 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3291-g994e2f20-1jammy [3,525 kB] 2024-04-25T13:36:51.002 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T13:36:51.011 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-04-25T13:36:51.013 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T13:36:51.017 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T13:36:51.018 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T13:36:51.026 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T13:36:51.028 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T13:36:51.030 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T13:36:51.032 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T13:36:51.032 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T13:36:51.042 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T13:36:51.044 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T13:36:51.046 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T13:36:51.046 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T13:36:51.063 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T13:36:51.064 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T13:36:51.064 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T13:36:51.064 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T13:36:51.065 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T13:36:51.065 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T13:36:51.083 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T13:36:51.084 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T13:36:51.086 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T13:36:51.087 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T13:36:51.104 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T13:36:51.104 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T13:36:51.105 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T13:36:51.106 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T13:36:51.106 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T13:36:51.107 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T13:36:51.125 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T13:36:51.125 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T13:36:51.130 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-3291-g994e2f20-1jammy [3,525 kB] 2024-04-25T13:36:51.131 INFO:teuthology.orchestra.run.smithi080.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T13:36:51.131 INFO:teuthology.orchestra.run.smithi080.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T13:36:51.146 INFO:teuthology.orchestra.run.smithi080.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T13:36:51.147 INFO:teuthology.orchestra.run.smithi080.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T13:36:51.148 INFO:teuthology.orchestra.run.smithi080.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T13:36:51.150 INFO:teuthology.orchestra.run.smithi080.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T13:36:51.151 INFO:teuthology.orchestra.run.smithi080.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T13:36:51.152 INFO:teuthology.orchestra.run.smithi080.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T13:36:51.167 INFO:teuthology.orchestra.run.smithi080.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T13:36:51.169 INFO:teuthology.orchestra.run.smithi080.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T13:36:51.169 INFO:teuthology.orchestra.run.smithi080.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T13:36:51.170 INFO:teuthology.orchestra.run.smithi080.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T13:36:51.187 INFO:teuthology.orchestra.run.smithi080.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T13:36:51.208 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-25T13:36:51.242 INFO:teuthology.orchestra.run.smithi080.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T13:36:51.244 INFO:teuthology.orchestra.run.smithi080.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T13:36:51.245 INFO:teuthology.orchestra.run.smithi080.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T13:36:51.245 INFO:teuthology.orchestra.run.smithi080.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T13:36:51.246 INFO:teuthology.orchestra.run.smithi080.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T13:36:51.266 INFO:teuthology.orchestra.run.smithi080.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T13:36:51.268 INFO:teuthology.orchestra.run.smithi080.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T13:36:51.268 INFO:teuthology.orchestra.run.smithi080.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T13:36:51.287 INFO:teuthology.orchestra.run.smithi080.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T13:36:51.287 INFO:teuthology.orchestra.run.smithi080.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T13:36:51.299 INFO:teuthology.orchestra.run.smithi080.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T13:36:51.299 INFO:teuthology.orchestra.run.smithi080.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T13:36:51.299 INFO:teuthology.orchestra.run.smithi080.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T13:36:51.299 INFO:teuthology.orchestra.run.smithi080.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T13:36:51.299 INFO:teuthology.orchestra.run.smithi080.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T13:36:51.299 INFO:teuthology.orchestra.run.smithi080.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T13:36:51.303 INFO:teuthology.orchestra.run.smithi080.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T13:36:51.306 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-25T13:36:51.318 INFO:teuthology.orchestra.run.smithi080.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T13:36:51.319 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-25T13:36:51.320 INFO:teuthology.orchestra.run.smithi080.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T13:36:51.320 INFO:teuthology.orchestra.run.smithi080.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T13:36:51.338 INFO:teuthology.orchestra.run.smithi080.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T13:36:51.341 INFO:teuthology.orchestra.run.smithi080.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T13:36:51.341 INFO:teuthology.orchestra.run.smithi080.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T13:36:51.343 INFO:teuthology.orchestra.run.smithi080.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T13:36:51.343 INFO:teuthology.orchestra.run.smithi080.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T13:36:51.348 INFO:teuthology.orchestra.run.smithi080.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T13:36:51.349 INFO:teuthology.orchestra.run.smithi080.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T13:36:51.359 INFO:teuthology.orchestra.run.smithi080.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T13:36:51.359 INFO:teuthology.orchestra.run.smithi080.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T13:36:51.360 INFO:teuthology.orchestra.run.smithi080.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T13:36:51.371 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-25T13:36:51.379 INFO:teuthology.orchestra.run.smithi080.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T13:36:51.381 INFO:teuthology.orchestra.run.smithi080.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T13:36:51.382 INFO:teuthology.orchestra.run.smithi080.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T13:36:51.390 INFO:teuthology.orchestra.run.smithi080.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T13:36:51.390 INFO:teuthology.orchestra.run.smithi080.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T13:36:51.390 INFO:teuthology.orchestra.run.smithi080.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T13:36:51.393 INFO:teuthology.orchestra.run.smithi080.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T13:36:51.400 INFO:teuthology.orchestra.run.smithi080.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T13:36:51.545 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-25T13:36:51.554 INFO:teuthology.orchestra.run.smithi087.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-25T13:36:51.582 INFO:teuthology.orchestra.run.smithi087.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-25T13:36:51.590 INFO:teuthology.orchestra.run.smithi087.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-25T13:36:51.754 INFO:teuthology.orchestra.run.smithi087.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-25T13:36:51.812 INFO:teuthology.orchestra.run.smithi087.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3291-g994e2f20-1jammy [3,287 kB] 2024-04-25T13:36:52.326 INFO:teuthology.orchestra.run.smithi087.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-25T13:36:52.341 INFO:teuthology.orchestra.run.smithi087.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-25T13:36:52.357 INFO:teuthology.orchestra.run.smithi087.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-25T13:36:52.366 INFO:teuthology.orchestra.run.smithi087.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-25T13:36:52.366 INFO:teuthology.orchestra.run.smithi087.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-25T13:36:52.382 INFO:teuthology.orchestra.run.smithi087.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-25T13:36:52.389 INFO:teuthology.orchestra.run.smithi087.stdout:Get:21 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3291-g994e2f20-1jammy [726 kB] 2024-04-25T13:36:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-25T13:36:52.433 INFO:teuthology.orchestra.run.smithi087.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-25T13:36:52.437 INFO:teuthology.orchestra.run.smithi087.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-25T13:36:52.438 INFO:teuthology.orchestra.run.smithi087.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-25T13:36:52.479 INFO:teuthology.orchestra.run.smithi087.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3291-g994e2f20-1jammy [356 kB] 2024-04-25T13:36:52.551 INFO:teuthology.orchestra.run.smithi087.stdout:Get:27 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3291-g994e2f20-1jammy [32.8 kB] 2024-04-25T13:36:52.552 INFO:teuthology.orchestra.run.smithi087.stdout:Get:28 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3291-g994e2f20-1jammy [180 kB] 2024-04-25T13:36:52.608 INFO:teuthology.orchestra.run.smithi087.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-25T13:36:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:Get:30 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3291-g994e2f20-1jammy [67.1 kB] 2024-04-25T13:36:52.618 INFO:teuthology.orchestra.run.smithi087.stdout:Get:31 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3291-g994e2f20-1jammy [330 kB] 2024-04-25T13:36:52.688 INFO:teuthology.orchestra.run.smithi087.stdout:Get:32 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3291-g994e2f20-1jammy [7,189 kB] 2024-04-25T13:36:52.691 INFO:teuthology.orchestra.run.smithi087.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-25T13:36:52.714 INFO:teuthology.orchestra.run.smithi087.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-25T13:36:52.772 INFO:teuthology.orchestra.run.smithi087.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-25T13:36:52.815 INFO:teuthology.orchestra.run.smithi087.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-25T13:36:52.839 INFO:teuthology.orchestra.run.smithi087.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-25T13:36:52.866 INFO:teuthology.orchestra.run.smithi087.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-25T13:36:53.048 INFO:teuthology.orchestra.run.smithi087.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-25T13:36:53.064 INFO:teuthology.orchestra.run.smithi087.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-25T13:36:53.071 INFO:teuthology.orchestra.run.smithi087.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-25T13:36:53.076 INFO:teuthology.orchestra.run.smithi087.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-25T13:36:53.260 INFO:teuthology.orchestra.run.smithi087.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-25T13:36:53.344 INFO:teuthology.orchestra.run.smithi087.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-25T13:36:53.359 INFO:teuthology.orchestra.run.smithi087.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-25T13:36:53.542 INFO:teuthology.orchestra.run.smithi087.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-25T13:36:53.594 INFO:teuthology.orchestra.run.smithi087.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-25T13:36:53.607 INFO:teuthology.orchestra.run.smithi087.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-25T13:36:53.765 INFO:teuthology.orchestra.run.smithi087.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-25T13:36:53.765 INFO:teuthology.orchestra.run.smithi087.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-25T13:36:53.770 INFO:teuthology.orchestra.run.smithi087.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-25T13:36:53.775 INFO:teuthology.orchestra.run.smithi087.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-25T13:36:53.789 INFO:teuthology.orchestra.run.smithi087.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-25T13:36:53.910 INFO:teuthology.orchestra.run.smithi080.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-3291-g994e2f20-1jammy [3,287 kB] 2024-04-25T13:36:53.973 INFO:teuthology.orchestra.run.smithi087.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-25T13:36:54.138 INFO:teuthology.orchestra.run.smithi087.stdout:Get:55 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3291-g994e2f20-1jammy [112 kB] 2024-04-25T13:36:54.190 INFO:teuthology.orchestra.run.smithi087.stdout:Get:56 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3291-g994e2f20-1jammy [471 kB] 2024-04-25T13:36:54.237 INFO:teuthology.orchestra.run.smithi087.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-25T13:36:54.261 INFO:teuthology.orchestra.run.smithi087.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3291-g994e2f20-1jammy [25.6 MB] 2024-04-25T13:36:54.353 INFO:teuthology.orchestra.run.smithi087.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-25T13:36:54.397 INFO:teuthology.orchestra.run.smithi087.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-25T13:36:54.481 INFO:teuthology.orchestra.run.smithi087.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-25T13:36:54.493 INFO:teuthology.orchestra.run.smithi087.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-25T13:36:54.506 INFO:teuthology.orchestra.run.smithi087.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-25T13:36:54.546 INFO:teuthology.orchestra.run.smithi087.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-25T13:36:54.807 INFO:teuthology.orchestra.run.smithi087.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-25T13:36:54.809 INFO:teuthology.orchestra.run.smithi087.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-25T13:36:54.811 INFO:teuthology.orchestra.run.smithi087.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-25T13:36:54.976 INFO:teuthology.orchestra.run.smithi087.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-25T13:36:55.302 INFO:teuthology.orchestra.run.smithi087.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-25T13:36:55.712 INFO:teuthology.orchestra.run.smithi087.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-25T13:36:55.721 INFO:teuthology.orchestra.run.smithi087.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-25T13:36:55.723 INFO:teuthology.orchestra.run.smithi087.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-25T13:36:55.792 INFO:teuthology.orchestra.run.smithi087.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-25T13:36:55.793 INFO:teuthology.orchestra.run.smithi087.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-25T13:36:55.795 INFO:teuthology.orchestra.run.smithi087.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-25T13:36:55.796 INFO:teuthology.orchestra.run.smithi087.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-25T13:36:55.797 INFO:teuthology.orchestra.run.smithi087.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-25T13:36:55.819 INFO:teuthology.orchestra.run.smithi080.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-3291-g994e2f20-1jammy [726 kB] 2024-04-25T13:36:55.963 INFO:teuthology.orchestra.run.smithi087.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-25T13:36:56.058 INFO:teuthology.orchestra.run.smithi087.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-25T13:36:56.155 INFO:teuthology.orchestra.run.smithi087.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-25T13:36:56.288 INFO:teuthology.orchestra.run.smithi080.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-3291-g994e2f20-1jammy [356 kB] 2024-04-25T13:36:56.402 INFO:teuthology.orchestra.run.smithi087.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-25T13:36:56.445 INFO:teuthology.orchestra.run.smithi087.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-25T13:36:56.469 INFO:teuthology.orchestra.run.smithi087.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-25T13:36:56.478 INFO:teuthology.orchestra.run.smithi087.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-25T13:36:56.511 INFO:teuthology.orchestra.run.smithi080.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-3291-g994e2f20-1jammy [32.8 kB] 2024-04-25T13:36:56.522 INFO:teuthology.orchestra.run.smithi087.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-25T13:36:56.546 INFO:teuthology.orchestra.run.smithi080.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-3291-g994e2f20-1jammy [180 kB] 2024-04-25T13:36:56.547 INFO:teuthology.orchestra.run.smithi087.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-04-25T13:36:56.548 INFO:teuthology.orchestra.run.smithi087.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-25T13:36:56.555 INFO:teuthology.orchestra.run.smithi087.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-25T13:36:56.645 INFO:teuthology.orchestra.run.smithi080.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-3291-g994e2f20-1jammy [67.1 kB] 2024-04-25T13:36:56.688 INFO:teuthology.orchestra.run.smithi080.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-3291-g994e2f20-1jammy [330 kB] 2024-04-25T13:36:56.722 INFO:teuthology.orchestra.run.smithi087.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-25T13:36:56.937 INFO:teuthology.orchestra.run.smithi080.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-3291-g994e2f20-1jammy [7,189 kB] 2024-04-25T13:36:57.261 INFO:teuthology.orchestra.run.smithi087.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-25T13:36:57.267 INFO:teuthology.orchestra.run.smithi087.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-25T13:36:57.306 INFO:teuthology.orchestra.run.smithi087.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-25T13:36:57.307 INFO:teuthology.orchestra.run.smithi087.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-25T13:36:57.313 INFO:teuthology.orchestra.run.smithi087.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-25T13:36:57.316 INFO:teuthology.orchestra.run.smithi087.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-25T13:36:57.356 INFO:teuthology.orchestra.run.smithi087.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-25T13:36:57.365 INFO:teuthology.orchestra.run.smithi087.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-25T13:36:57.438 INFO:teuthology.orchestra.run.smithi087.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-25T13:36:57.439 INFO:teuthology.orchestra.run.smithi087.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-25T13:36:57.603 INFO:teuthology.orchestra.run.smithi087.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-25T13:36:58.042 INFO:teuthology.orchestra.run.smithi087.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-25T13:36:58.057 INFO:teuthology.orchestra.run.smithi087.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-25T13:37:01.218 INFO:teuthology.orchestra.run.smithi087.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3291-g994e2f20-1jammy [4,917 kB] 2024-04-25T13:37:02.279 INFO:teuthology.orchestra.run.smithi080.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-3291-g994e2f20-1jammy [112 kB] 2024-04-25T13:37:02.360 INFO:teuthology.orchestra.run.smithi080.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-3291-g994e2f20-1jammy [471 kB] 2024-04-25T13:37:02.449 INFO:teuthology.orchestra.run.smithi087.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3291-g994e2f20-1jammy [244 kB] 2024-04-25T13:37:02.517 INFO:teuthology.orchestra.run.smithi087.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3291-g994e2f20-1jammy [125 kB] 2024-04-25T13:37:02.555 INFO:teuthology.orchestra.run.smithi087.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3291-g994e2f20-1jammy [1,542 kB] 2024-04-25T13:37:02.649 INFO:teuthology.orchestra.run.smithi080.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-3291-g994e2f20-1jammy [25.6 MB] 2024-04-25T13:37:02.977 INFO:teuthology.orchestra.run.smithi087.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3291-g994e2f20-1jammy [6,148 kB] 2024-04-25T13:37:04.666 INFO:teuthology.orchestra.run.smithi087.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3291-g994e2f20-1jammy [22.5 MB] 2024-04-25T13:37:09.992 INFO:teuthology.orchestra.run.smithi087.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3291-g994e2f20-1jammy [14.1 kB] 2024-04-25T13:37:09.993 INFO:teuthology.orchestra.run.smithi087.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3291-g994e2f20-1jammy [856 kB] 2024-04-25T13:37:10.154 INFO:teuthology.orchestra.run.smithi087.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3291-g994e2f20-1jammy [2,367 kB] 2024-04-25T13:37:10.686 INFO:teuthology.orchestra.run.smithi087.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3291-g994e2f20-1jammy [791 kB] 2024-04-25T13:37:10.870 INFO:teuthology.orchestra.run.smithi087.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3291-g994e2f20-1jammy [148 kB] 2024-04-25T13:37:10.919 INFO:teuthology.orchestra.run.smithi087.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3291-g994e2f20-1jammy [2,297 kB] 2024-04-25T13:37:11.464 INFO:teuthology.orchestra.run.smithi087.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3291-g994e2f20-1jammy [8,625 kB] 2024-04-25T13:37:12.798 INFO:teuthology.orchestra.run.smithi080.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-3291-g994e2f20-1jammy [4,917 kB] 2024-04-25T13:37:13.490 INFO:teuthology.orchestra.run.smithi087.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3291-g994e2f20-1jammy [14.2 kB] 2024-04-25T13:37:13.491 INFO:teuthology.orchestra.run.smithi087.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3291-g994e2f20-1jammy [44.9 MB] 2024-04-25T13:37:14.051 INFO:teuthology.orchestra.run.smithi080.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-3291-g994e2f20-1jammy [244 kB] 2024-04-25T13:37:14.103 INFO:teuthology.orchestra.run.smithi080.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-3291-g994e2f20-1jammy [125 kB] 2024-04-25T13:37:14.139 INFO:teuthology.orchestra.run.smithi080.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-3291-g994e2f20-1jammy [1,542 kB] 2024-04-25T13:37:14.537 INFO:teuthology.orchestra.run.smithi080.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-3291-g994e2f20-1jammy [6,148 kB] 2024-04-25T13:37:16.174 INFO:teuthology.orchestra.run.smithi080.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-3291-g994e2f20-1jammy [22.5 MB] 2024-04-25T13:37:22.192 INFO:teuthology.orchestra.run.smithi080.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-3291-g994e2f20-1jammy [14.1 kB] 2024-04-25T13:37:22.193 INFO:teuthology.orchestra.run.smithi080.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-3291-g994e2f20-1jammy [856 kB] 2024-04-25T13:37:22.320 INFO:teuthology.orchestra.run.smithi080.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-3291-g994e2f20-1jammy [2,367 kB] 2024-04-25T13:37:22.707 INFO:teuthology.orchestra.run.smithi080.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-3291-g994e2f20-1jammy [791 kB] 2024-04-25T13:37:22.905 INFO:teuthology.orchestra.run.smithi080.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-3291-g994e2f20-1jammy [148 kB] 2024-04-25T13:37:22.909 INFO:teuthology.orchestra.run.smithi080.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-3291-g994e2f20-1jammy [2,297 kB] 2024-04-25T13:37:23.440 INFO:teuthology.orchestra.run.smithi080.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-3291-g994e2f20-1jammy [8,625 kB] 2024-04-25T13:37:24.224 INFO:teuthology.orchestra.run.smithi087.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3291-g994e2f20-1jammy [124 kB] 2024-04-25T13:37:24.315 INFO:teuthology.orchestra.run.smithi087.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3291-g994e2f20-1jammy [236 kB] 2024-04-25T13:37:24.395 INFO:teuthology.orchestra.run.smithi087.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3291-g994e2f20-1jammy [33.9 kB] 2024-04-25T13:37:24.396 INFO:teuthology.orchestra.run.smithi087.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3291-g994e2f20-1jammy [23.4 kB] 2024-04-25T13:37:24.397 INFO:teuthology.orchestra.run.smithi087.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3291-g994e2f20-1jammy [39.0 kB] 2024-04-25T13:37:24.398 INFO:teuthology.orchestra.run.smithi087.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3291-g994e2f20-1jammy [14.2 MB] 2024-04-25T13:37:25.274 INFO:teuthology.orchestra.run.smithi080.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-3291-g994e2f20-1jammy [14.2 kB] 2024-04-25T13:37:25.326 INFO:teuthology.orchestra.run.smithi080.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-3291-g994e2f20-1jammy [44.9 MB] 2024-04-25T13:37:26.464 INFO:teuthology.orchestra.run.smithi087.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3291-g994e2f20-1jammy [92.6 kB] 2024-04-25T13:37:27.289 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 175 MB in 36s (4,858 kB/s) 2024-04-25T13:37:27.434 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T13:37:27.533 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T13:37:27.538 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T13:37:27.582 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T13:37:27.902 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T13:37:27.928 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T13:37:27.968 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T13:37:28.288 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T13:37:28.303 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T13:37:28.513 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T13:37:28.832 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T13:37:28.858 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T13:37:28.933 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T13:37:29.219 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T13:37:29.245 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T13:37:29.277 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T13:37:29.605 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T13:37:29.632 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T13:37:29.672 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T13:37:30.109 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T13:37:30.135 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:37:30.184 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:37:30.537 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T13:37:30.563 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:37:30.603 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:37:30.932 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T13:37:30.945 INFO:teuthology.orchestra.run.smithi080.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-3291-g994e2f20-1jammy [124 kB] 2024-04-25T13:37:30.948 INFO:teuthology.orchestra.run.smithi080.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.0.0-3291-g994e2f20-1jammy [236 kB] 2024-04-25T13:37:30.952 INFO:teuthology.orchestra.run.smithi080.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.0.0-3291-g994e2f20-1jammy [33.9 kB] 2024-04-25T13:37:30.953 INFO:teuthology.orchestra.run.smithi080.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.0.0-3291-g994e2f20-1jammy [23.4 kB] 2024-04-25T13:37:30.958 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:37:30.998 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:37:31.013 INFO:teuthology.orchestra.run.smithi080.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-3291-g994e2f20-1jammy [39.0 kB] 2024-04-25T13:37:31.014 INFO:teuthology.orchestra.run.smithi080.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-3291-g994e2f20-1jammy [14.2 MB] 2024-04-25T13:37:31.368 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T13:37:31.394 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T13:37:31.435 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T13:37:31.712 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librados2. 2024-04-25T13:37:31.739 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../010-librados2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:31.779 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:32.207 INFO:teuthology.orchestra.run.smithi080.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240425.015853-debug/994e2f20102a20fa9a27966c14b625cc01884df8/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-3291-g994e2f20-1jammy [92.6 kB] 2024-04-25T13:37:32.274 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librbd1. 2024-04-25T13:37:32.300 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../011-librbd1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:32.340 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:32.743 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T13:37:32.758 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:32.793 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:32.819 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 175 MB in 41s (4,218 kB/s) 2024-04-25T13:37:32.959 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyperclip. 2024-04-25T13:37:33.048 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-25T13:37:33.053 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-04-25T13:37:33.070 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rados. 2024-04-25T13:37:33.085 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:33.099 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-04-25T13:37:33.128 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:33.371 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-25T13:37:33.397 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-25T13:37:33.437 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T13:37:33.456 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T13:37:33.485 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:37:33.598 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:33.757 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cmd2. 2024-04-25T13:37:33.783 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-04-25T13:37:33.823 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-04-25T13:37:33.850 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T13:37:33.866 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:33.900 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:34.128 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T13:37:34.154 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:37:34.168 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-25T13:37:34.186 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:34.195 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-25T13:37:34.285 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T13:37:34.439 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T13:37:34.454 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T13:37:34.497 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T13:37:34.630 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-25T13:37:34.657 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-25T13:37:34.697 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T13:37:34.758 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T13:37:34.774 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:34.817 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:35.000 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-25T13:37:35.026 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-25T13:37:35.117 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T13:37:35.170 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T13:37:35.196 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T13:37:35.236 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T13:37:35.489 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T13:37:35.503 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T13:37:35.538 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T13:37:35.579 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-25T13:37:35.605 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:37:35.653 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:37:35.800 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T13:37:35.826 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T13:37:35.867 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T13:37:36.065 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-25T13:37:36.091 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:37:36.132 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:37:36.220 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua5.1. 2024-04-25T13:37:36.246 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T13:37:36.286 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T13:37:36.460 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-25T13:37:36.486 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-25T13:37:36.535 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:37:36.597 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-any. 2024-04-25T13:37:36.612 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T13:37:36.647 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T13:37:36.875 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package zip. 2024-04-25T13:37:36.880 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-25T13:37:36.901 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T13:37:36.906 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-25T13:37:36.942 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T13:37:36.947 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T13:37:37.253 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package unzip. 2024-04-25T13:37:37.266 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librados2. 2024-04-25T13:37:37.279 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T13:37:37.292 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../010-librados2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:37.320 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T13:37:37.333 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:37.707 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package luarocks. 2024-04-25T13:37:37.734 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T13:37:37.774 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T13:37:37.886 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librbd1. 2024-04-25T13:37:37.912 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../011-librbd1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:37.952 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:38.106 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package librgw2. 2024-04-25T13:37:38.122 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../027-librgw2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:38.156 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:38.422 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libcephfs2. 2024-04-25T13:37:38.449 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../012-libcephfs2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:38.489 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:38.808 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rados. 2024-04-25T13:37:38.834 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../013-python3-rados_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:38.875 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:38.884 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T13:37:38.899 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:38.942 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:39.203 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-25T13:37:39.229 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:37:39.269 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:39.287 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T13:37:39.314 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T13:37:39.354 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T13:37:39.514 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cephfs. 2024-04-25T13:37:39.529 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../015-python3-cephfs_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:39.563 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:39.623 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T13:37:39.638 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:39.673 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:39.850 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-25T13:37:39.876 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../016-python3-ceph-common_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:37:39.916 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:40.068 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-common. 2024-04-25T13:37:40.095 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:40.159 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:40.245 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-prettytable. 2024-04-25T13:37:40.271 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-04-25T13:37:40.311 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-25T13:37:40.580 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rbd. 2024-04-25T13:37:40.607 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../018-python3-rbd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:40.647 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:40.967 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-25T13:37:40.993 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-25T13:37:41.033 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T13:37:41.295 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-25T13:37:41.310 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-25T13:37:41.353 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T13:37:41.656 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-25T13:37:41.656 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-base. 2024-04-25T13:37:41.681 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-25T13:37:41.683 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:41.722 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T13:37:41.732 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:42.051 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua5.1. 2024-04-25T13:37:42.077 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-25T13:37:42.117 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-25T13:37:42.269 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T13:37:42.296 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T13:37:42.335 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T13:37:42.437 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-any. 2024-04-25T13:37:42.463 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-04-25T13:37:42.504 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-25T13:37:42.580 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T13:37:42.595 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T13:37:42.630 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T13:37:42.757 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package zip. 2024-04-25T13:37:42.772 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-04-25T13:37:42.815 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking zip (3.0-12build2) ... 2024-04-25T13:37:42.925 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T13:37:42.951 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T13:37:42.991 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T13:37:43.152 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package unzip. 2024-04-25T13:37:43.178 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-25T13:37:43.218 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-25T13:37:43.261 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T13:37:43.287 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T13:37:43.327 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T13:37:43.597 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T13:37:43.624 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T13:37:43.663 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T13:37:43.764 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package luarocks. 2024-04-25T13:37:43.790 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-25T13:37:43.822 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-25T13:37:43.950 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T13:37:43.976 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T13:37:44.018 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T13:37:44.197 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package librgw2. 2024-04-25T13:37:44.223 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../027-librgw2_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:44.327 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-portend. 2024-04-25T13:37:44.354 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T13:37:44.394 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T13:37:44.697 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T13:37:44.723 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T13:37:44.763 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T13:37:44.899 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rgw. 2024-04-25T13:37:44.926 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../028-python3-rgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:44.966 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:45.058 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T13:37:45.085 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T13:37:45.124 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T13:37:45.294 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-25T13:37:45.320 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-25T13:37:45.361 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T13:37:45.470 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T13:37:45.496 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T13:37:45.536 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T13:37:45.656 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libradosstriper1. 2024-04-25T13:37:45.682 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../030-libradosstriper1_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:45.722 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:45.848 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T13:37:45.874 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T13:37:45.914 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T13:37:46.184 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-common. 2024-04-25T13:37:46.192 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T13:37:46.210 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../031-ceph-common_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:46.219 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T13:37:46.250 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:46.258 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T13:37:46.478 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-mako. 2024-04-25T13:37:46.490 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T13:37:46.528 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T13:37:46.831 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T13:37:46.857 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T13:37:46.897 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T13:37:47.209 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T13:37:47.236 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T13:37:47.275 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T13:37:47.595 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-webob. 2024-04-25T13:37:47.621 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T13:37:47.661 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T13:37:47.772 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-base. 2024-04-25T13:37:47.799 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../032-ceph-base_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:47.848 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:48.014 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T13:37:48.040 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T13:37:48.084 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T13:37:48.392 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T13:37:48.419 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T13:37:48.459 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T13:37:48.519 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-25T13:37:48.545 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-25T13:37:48.585 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-25T13:37:48.695 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-paste. 2024-04-25T13:37:48.710 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T13:37:48.753 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T13:37:48.888 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cheroot. 2024-04-25T13:37:48.914 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-25T13:37:48.955 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T13:37:49.283 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-25T13:37:49.309 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-25T13:37:49.349 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-25T13:37:49.540 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T13:37:49.555 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T13:37:49.598 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T13:37:49.644 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-25T13:37:49.670 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-25T13:37:49.711 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-25T13:37:49.894 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T13:37:49.920 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T13:37:49.961 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T13:37:49.972 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-25T13:37:49.998 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-25T13:37:50.038 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-25T13:37:50.264 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T13:37:50.291 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T13:37:50.330 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T13:37:50.341 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-tempora. 2024-04-25T13:37:50.368 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-04-25T13:37:50.408 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-25T13:37:50.642 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T13:37:50.668 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T13:37:50.708 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T13:37:50.711 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-portend. 2024-04-25T13:37:50.737 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-04-25T13:37:50.777 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-25T13:37:51.047 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-25T13:37:51.062 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-25T13:37:51.063 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T13:37:51.090 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T13:37:51.097 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-25T13:37:51.130 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T13:37:51.399 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-idna. 2024-04-25T13:37:51.400 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-25T13:37:51.426 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T13:37:51.426 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-25T13:37:51.466 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T13:37:51.466 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-25T13:37:51.777 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T13:37:51.795 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-natsort. 2024-04-25T13:37:51.803 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T13:37:51.811 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-04-25T13:37:51.843 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T13:37:51.845 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-25T13:37:52.157 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-logutils. 2024-04-25T13:37:52.163 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-requests. 2024-04-25T13:37:52.183 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-04-25T13:37:52.190 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T13:37:52.223 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-25T13:37:52.230 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T13:37:52.526 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-25T13:37:52.552 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-25T13:37:52.574 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T13:37:52.593 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-25T13:37:52.601 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T13:37:52.641 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T13:37:52.896 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-mako. 2024-04-25T13:37:52.911 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T13:37:52.922 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-25T13:37:52.926 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:37:52.962 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T13:37:52.969 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:53.299 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-25T13:37:53.315 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T13:37:53.325 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-25T13:37:53.342 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:53.365 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-25T13:37:53.373 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:53.635 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-25T13:37:53.651 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T13:37:53.661 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-25T13:37:53.678 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T13:37:53.702 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-25T13:37:53.718 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T13:37:53.996 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webob. 2024-04-25T13:37:54.021 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T13:37:54.023 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-25T13:37:54.047 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T13:37:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-25T13:37:54.087 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T13:37:54.416 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-waitress. 2024-04-25T13:37:54.424 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T13:37:54.443 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-25T13:37:54.451 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:54.486 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T13:37:54.490 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:54.802 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-tempita. 2024-04-25T13:37:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-25T13:37:54.869 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T13:37:54.885 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T13:37:54.912 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:54.951 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:55.122 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-paste. 2024-04-25T13:37:55.148 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-25T13:37:55.180 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T13:37:55.500 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-25T13:37:55.515 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-25T13:37:55.558 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T13:37:55.564 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T13:37:55.591 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:55.630 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:55.836 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-25T13:37:55.851 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-25T13:37:55.886 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-25T13:37:56.164 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-webtest. 2024-04-25T13:37:56.190 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-04-25T13:37:56.231 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-25T13:37:56.542 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pecan. 2024-04-25T13:37:56.568 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-25T13:37:56.625 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T13:37:56.666 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph. 2024-04-25T13:37:56.682 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../068-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:56.725 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:56.989 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-certifi. 2024-04-25T13:37:57.015 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-04-25T13:37:57.019 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T13:37:57.046 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:57.055 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-25T13:37:57.077 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:57.392 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-idna. 2024-04-25T13:37:57.418 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../057-python3-idna_3.3-1_all.deb ... 2024-04-25T13:37:57.430 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T13:37:57.457 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:57.458 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-25T13:37:57.497 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:57.728 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-urllib3. 2024-04-25T13:37:57.754 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-25T13:37:57.786 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T13:37:57.900 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephadm. 2024-04-25T13:37:57.926 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../071-cephadm_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:57.966 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:58.081 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-requests. 2024-04-25T13:37:58.108 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-25T13:37:58.139 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T13:37:58.294 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T13:37:58.321 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T13:37:58.361 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T13:37:58.425 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-25T13:37:58.452 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-25T13:37:58.492 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T13:37:58.639 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T13:37:58.665 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T13:37:58.705 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T13:37:58.962 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-25T13:37:58.977 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:37:59.012 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:59.375 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-25T13:37:59.402 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:37:59.442 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:37:59.762 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-25T13:37:59.788 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-25T13:37:59.828 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-25T13:37:59.893 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-babel. 2024-04-25T13:37:59.920 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T13:37:59.951 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T13:38:00.089 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-openssl. 2024-04-25T13:38:00.116 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-04-25T13:38:00.156 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-25T13:38:00.238 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T13:38:00.265 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T13:38:00.295 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T13:38:00.501 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr. 2024-04-25T13:38:00.527 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../065-ceph-mgr_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:00.567 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:00.649 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T13:38:00.676 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:00.707 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:00.954 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mon. 2024-04-25T13:38:00.980 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../066-ceph-mon_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:01.019 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T13:38:01.020 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:01.045 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T13:38:01.077 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T13:38:01.380 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-routes. 2024-04-25T13:38:01.407 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T13:38:01.446 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T13:38:01.616 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-osd. 2024-04-25T13:38:01.642 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../067-ceph-osd_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:01.682 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:01.783 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T13:38:01.810 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:01.849 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:02.769 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph. 2024-04-25T13:38:02.795 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../068-ceph_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:02.835 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:03.154 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T13:38:03.155 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-fuse. 2024-04-25T13:38:03.181 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../069-ceph-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:03.182 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T13:38:03.221 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T13:38:03.221 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:03.541 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mds. 2024-04-25T13:38:03.567 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../070-ceph-mds_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:03.607 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:03.691 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T13:38:03.718 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T13:38:03.757 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T13:38:04.094 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephadm. 2024-04-25T13:38:04.121 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../071-cephadm_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:04.136 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T13:38:04.161 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:04.163 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T13:38:04.288 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T13:38:04.480 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-25T13:38:04.507 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-25T13:38:04.546 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-25T13:38:04.566 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T13:38:04.593 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T13:38:04.632 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T13:38:04.875 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-25T13:38:04.901 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T13:38:04.941 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T13:38:05.533 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T13:38:05.561 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:05.599 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:06.005 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-babel. 2024-04-25T13:38:06.031 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-25T13:38:06.071 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T13:38:06.399 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-jinja2. 2024-04-25T13:38:06.426 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-25T13:38:06.465 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T13:38:06.494 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T13:38:06.521 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T13:38:06.561 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T13:38:06.819 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-25T13:38:06.846 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:06.847 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T13:38:06.874 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T13:38:06.885 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:06.913 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T13:38:07.205 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-25T13:38:07.225 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T13:38:07.233 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-04-25T13:38:07.252 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T13:38:07.272 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-25T13:38:07.291 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T13:38:07.567 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-routes. 2024-04-25T13:38:07.593 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-25T13:38:07.603 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T13:38:07.630 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T13:38:07.633 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T13:38:07.678 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T13:38:07.956 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T13:38:07.972 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T13:38:07.978 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-25T13:38:08.005 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:08.006 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T13:38:08.044 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:08.318 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T13:38:08.345 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T13:38:08.384 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T13:38:08.696 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T13:38:08.723 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T13:38:08.762 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T13:38:09.091 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T13:38:09.118 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T13:38:09.157 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T13:38:09.325 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-25T13:38:09.352 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-25T13:38:09.391 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T13:38:09.854 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T13:38:09.862 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-joblib. 2024-04-25T13:38:09.870 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:09.877 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-25T13:38:09.911 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T13:38:09.912 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:10.232 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T13:38:10.259 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T13:38:10.265 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-25T13:38:10.292 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-25T13:38:10.298 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T13:38:10.409 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-25T13:38:10.593 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T13:38:10.620 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T13:38:10.659 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T13:38:10.720 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-sklearn. 2024-04-25T13:38:10.747 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-25T13:38:10.786 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T13:38:10.921 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package jq. 2024-04-25T13:38:10.936 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T13:38:10.979 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T13:38:11.231 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package socat. 2024-04-25T13:38:11.259 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T13:38:11.298 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T13:38:11.498 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-25T13:38:11.525 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:11.564 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:11.601 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T13:38:11.617 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T13:38:11.659 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T13:38:12.005 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-test. 2024-04-25T13:38:12.033 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:12.072 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:12.526 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-cachetools. 2024-04-25T13:38:12.553 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-04-25T13:38:12.584 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-25T13:38:12.879 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-25T13:38:12.905 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-25T13:38:12.953 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-25T13:38:13.265 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-25T13:38:13.293 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-25T13:38:13.332 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T13:38:13.643 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-rsa. 2024-04-25T13:38:13.670 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-04-25T13:38:13.710 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-25T13:38:14.031 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-google-auth. 2024-04-25T13:38:14.058 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-04-25T13:38:14.097 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-25T13:38:14.326 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T13:38:14.353 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:14.392 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:14.625 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-25T13:38:14.653 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-25T13:38:14.692 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T13:38:14.763 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T13:38:14.791 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:14.830 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:15.011 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-websocket. 2024-04-25T13:38:15.039 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-04-25T13:38:15.078 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-25T13:38:15.124 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T13:38:15.151 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T13:38:15.191 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T13:38:15.373 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-25T13:38:15.401 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-25T13:38:15.439 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T13:38:15.485 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T13:38:15.512 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:15.677 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:15.946 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T13:38:15.962 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:16.004 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:16.020 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-25T13:38:16.037 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:16.070 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:16.248 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T13:38:16.264 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:16.298 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:16.373 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-25T13:38:16.401 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-25T13:38:16.440 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T13:38:16.568 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T13:38:16.592 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T13:38:16.626 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T13:38:16.776 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-25T13:38:16.804 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T13:38:16.843 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T13:38:16.948 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T13:38:16.975 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T13:38:17.022 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T13:38:17.112 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package jq. 2024-04-25T13:38:17.140 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-25T13:38:17.170 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-25T13:38:17.390 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package socat. 2024-04-25T13:38:17.393 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T13:38:17.406 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-25T13:38:17.420 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T13:38:17.448 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-25T13:38:17.459 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T13:38:17.735 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package xmlstarlet. 2024-04-25T13:38:17.751 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-25T13:38:17.784 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-25T13:38:17.890 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package pkg-config. 2024-04-25T13:38:17.917 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T13:38:17.956 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T13:38:18.164 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-test. 2024-04-25T13:38:18.191 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../099-ceph-test_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:18.209 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T13:38:18.225 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T13:38:18.230 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:18.267 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T13:38:18.688 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T13:38:18.714 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T13:38:18.754 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T13:38:19.065 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T13:38:19.092 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T13:38:19.123 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T13:38:19.393 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T13:38:19.420 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T13:38:19.459 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T13:38:19.762 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T13:38:19.790 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T13:38:19.829 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T13:38:20.140 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-py. 2024-04-25T13:38:20.168 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T13:38:20.207 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T13:38:20.535 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T13:38:20.563 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T13:38:20.601 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T13:38:20.726 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ceph-volume. 2024-04-25T13:38:20.754 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../100-ceph-volume_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:20.793 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:21.087 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T13:38:21.115 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T13:38:21.156 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T13:38:21.172 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephfs-mirror. 2024-04-25T13:38:21.200 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../101-cephfs-mirror_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:21.239 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:21.467 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-toml. 2024-04-25T13:38:21.494 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T13:38:21.533 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T13:38:21.558 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-colorama. 2024-04-25T13:38:21.586 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-04-25T13:38:21.625 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-04-25T13:38:21.803 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T13:38:21.830 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T13:38:21.869 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T13:38:21.902 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephfs-shell. 2024-04-25T13:38:21.918 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../103-cephfs-shell_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:21.961 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:22.205 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package cephfs-top. 2024-04-25T13:38:22.231 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T13:38:22.233 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../104-cephfs-top_19.0.0-3291-g994e2f20-1jammy_all.deb ... 2024-04-25T13:38:22.259 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T13:38:22.272 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:22.297 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T13:38:22.524 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-25T13:38:22.540 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../105-libcephfs-dev_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:22.567 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T13:38:22.582 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:22.583 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T13:38:22.625 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T13:38:22.910 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-25T13:38:22.938 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-25T13:38:22.977 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T13:38:23.299 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-25T13:38:23.326 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-04-25T13:38:23.329 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package radosgw. 2024-04-25T13:38:23.357 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../122-radosgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:23.365 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-25T13:38:23.404 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:23.736 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package nvme-cli. 2024-04-25T13:38:23.763 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-25T13:38:23.802 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T13:38:24.224 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package pkg-config. 2024-04-25T13:38:24.252 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-25T13:38:24.290 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T13:38:24.373 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T13:38:24.390 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:24.432 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:24.552 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-25T13:38:24.568 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-25T13:38:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-25T13:38:24.728 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T13:38:24.995 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T13:38:25.039 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-25T13:38:25.066 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-25T13:38:25.105 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-25T13:38:25.129 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T13:38:25.399 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pastescript. 2024-04-25T13:38:25.417 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T13:38:25.427 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-04-25T13:38:25.466 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-25T13:38:25.719 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pluggy. 2024-04-25T13:38:25.730 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T13:38:25.735 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-25T13:38:25.769 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-25T13:38:26.089 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-psutil. 2024-04-25T13:38:26.117 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-25T13:38:26.142 INFO:teuthology.orchestra.run.smithi087.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T13:38:26.156 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-25T13:38:26.223 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:26.484 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-py. 2024-04-25T13:38:26.511 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-04-25T13:38:26.550 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-25T13:38:26.731 INFO:teuthology.orchestra.run.smithi087.stdout:Adding system user cephadm....done 2024-04-25T13:38:26.845 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pygments. 2024-04-25T13:38:26.872 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-25T13:38:26.887 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T13:38:26.911 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T13:38:27.156 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T13:38:27.366 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-25T13:38:27.394 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-25T13:38:27.406 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T13:38:27.424 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-25T13:38:27.533 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T13:38:27.727 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-toml. 2024-04-25T13:38:27.754 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-04-25T13:38:27.784 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T13:38:27.793 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-25T13:38:28.035 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T13:38:28.096 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-pytest. 2024-04-25T13:38:28.124 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-25T13:38:28.144 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T13:38:28.162 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T13:38:28.437 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T13:38:28.474 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package python3-simplejson. 2024-04-25T13:38:28.502 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-25T13:38:28.541 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-25T13:38:28.797 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T13:38:28.802 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-25T13:38:28.818 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-25T13:38:28.885 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-25T13:38:29.057 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T13:38:29.308 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T13:38:29.593 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T13:38:29.623 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package radosgw. 2024-04-25T13:38:29.651 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../122-radosgw_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:29.689 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:29.877 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T13:38:30.120 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T13:38:30.372 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T13:38:30.684 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package rbd-fuse. 2024-04-25T13:38:30.712 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../123-rbd-fuse_19.0.0-3291-g994e2f20-1jammy_amd64.deb ... 2024-04-25T13:38:30.722 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:30.750 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:31.055 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-25T13:38:31.165 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T13:38:31.291 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T13:38:31.298 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-25T13:38:31.417 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T13:38:31.424 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-25T13:38:31.668 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T13:38:31.687 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-25T13:38:31.794 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T13:38:32.043 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-25T13:38:32.047 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T13:38:32.463 INFO:teuthology.orchestra.run.smithi080.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-25T13:38:32.482 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T13:38:32.512 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T13:38:32.534 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:32.567 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T13:38:32.677 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T13:38:33.025 INFO:teuthology.orchestra.run.smithi080.stdout:Adding system user cephadm....done 2024-04-25T13:38:33.027 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T13:38:33.153 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T13:38:33.182 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-25T13:38:33.421 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T13:38:33.458 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-25T13:38:33.689 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T13:38:33.717 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-25T13:38:33.819 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-25T13:38:33.998 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T13:38:34.079 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-25T13:38:34.116 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T13:38:34.321 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-25T13:38:34.448 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-25T13:38:34.749 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-25T13:38:34.810 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T13:38:35.061 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T13:38:35.101 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-25T13:38:35.186 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T13:38:35.329 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T13:38:35.344 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-04-25T13:38:35.604 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-25T13:38:35.630 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T13:38:35.872 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-25T13:38:36.023 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T13:38:36.132 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-25T13:38:36.358 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T13:38:36.391 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-25T13:38:36.659 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-25T13:38:36.734 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T13:38:36.927 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-ceph-argparse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:37.010 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T13:38:37.178 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-25T13:38:37.303 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T13:38:37.304 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-25T13:38:37.430 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-04-25T13:38:37.671 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T13:38:37.681 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-25T13:38:37.807 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-25T13:38:37.922 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T13:38:38.056 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:38.058 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-25T13:38:38.358 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T13:38:38.470 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-25T13:38:38.507 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-25T13:38:38.561 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-25T13:38:38.658 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T13:38:38.687 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-25T13:38:39.047 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-25T13:38:39.173 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-25T13:38:39.441 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-25T13:38:39.717 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-25T13:38:39.781 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T13:38:40.032 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:38:40.043 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up zip (3.0-12build2) ... 2024-04-25T13:38:40.141 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T13:38:40.153 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-25T13:38:40.384 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T13:38:40.510 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T13:38:40.636 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T13:38:40.805 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-25T13:38:40.937 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T13:38:41.056 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-25T13:38:41.182 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-25T13:38:41.196 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T13:38:41.308 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-25T13:38:41.473 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:38:41.600 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-25T13:38:41.624 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T13:38:41.925 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T13:38:41.968 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-25T13:38:42.159 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T13:38:42.212 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-25T13:38:42.410 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T13:38:42.572 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-25T13:38:42.670 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T13:38:42.864 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-04-25T13:38:42.921 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T13:38:43.165 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-25T13:38:43.189 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T13:38:43.323 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T13:38:43.534 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-25T13:38:43.599 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T13:38:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T13:38:43.785 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-25T13:38:43.894 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:43.984 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T13:38:44.204 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-idna (3.3-1) ... 2024-04-25T13:38:44.269 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T13:38:44.513 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-25T13:38:44.528 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T13:38:44.654 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T13:38:44.780 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T13:38:44.869 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T13:38:45.116 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T13:38:45.241 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T13:38:45.626 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T13:38:45.628 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-25T13:38:45.888 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:38:45.910 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T13:38:46.022 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-25T13:38:46.028 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T13:38:46.281 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-25T13:38:46.287 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:38:46.408 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-25T13:38:46.413 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T13:38:46.534 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-25T13:38:46.748 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T13:38:46.826 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-25T13:38:47.049 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T13:38:47.078 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-25T13:38:47.317 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T13:38:47.321 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:38:47.447 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-25T13:38:47.521 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T13:38:47.749 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-25T13:38:47.779 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T13:38:48.008 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-25T13:38:48.038 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T13:38:48.250 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-25T13:38:48.298 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T13:38:48.485 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-25T13:38:48.666 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T13:38:48.736 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-25T13:38:48.943 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T13:38:48.987 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-25T13:38:49.113 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-25T13:38:49.270 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T13:38:49.389 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-25T13:38:49.396 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:49.515 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-25T13:38:49.538 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:49.673 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T13:38:49.791 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-25T13:38:50.025 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:50.068 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-25T13:38:50.151 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:50.277 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T13:38:50.302 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-25T13:38:50.453 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-25T13:38:50.537 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:50.579 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-25T13:38:50.663 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:50.668 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-25T13:38:50.788 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:50.898 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-25T13:38:51.000 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T13:38:51.000 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T13:38:51.024 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-25T13:38:51.417 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-25T13:38:51.473 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:51.600 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:51.676 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-25T13:38:51.709 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:51.803 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-25T13:38:51.852 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:51.978 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T13:38:52.054 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-25T13:38:52.180 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-25T13:38:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-25T13:38:52.790 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-25T13:38:53.058 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-25T13:38:53.135 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:53.253 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-25T13:38:53.261 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:53.387 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:53.512 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:53.529 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-25T13:38:53.650 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:53.773 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:53.797 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-25T13:38:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-25T13:38:54.298 INFO:teuthology.orchestra.run.smithi087.stdout:Adding group ceph....done 2024-04-25T13:38:54.450 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-25T13:38:54.666 INFO:teuthology.orchestra.run.smithi087.stdout:Adding system user ceph....done 2024-04-25T13:38:54.727 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-25T13:38:54.834 INFO:teuthology.orchestra.run.smithi087.stdout:Setting system user ceph properties....done 2024-04-25T13:38:54.848 INFO:teuthology.orchestra.run.smithi087.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T13:38:54.985 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T13:38:55.055 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-25T13:38:55.181 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librados2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:55.307 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libsqlite3-mod-ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:55.307 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T13:38:55.433 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-25T13:38:55.773 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:55.980 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:38:55.980 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:38:56.136 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libcephfs2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:56.254 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libradosstriper1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:56.380 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-25T13:38:56.441 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:56.551 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:56.656 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librbd1 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:56.782 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-modules-core (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:56.892 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:57.015 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:38:57.015 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:38:57.086 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T13:38:57.086 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-25T13:38:57.477 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:57.585 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libcephfs-dev (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:57.711 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rados (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:57.821 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T13:38:57.837 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up librgw2 (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:57.963 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephfs-top (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:58.089 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-25T13:38:58.279 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:58.508 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T13:38:58.508 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T13:38:58.939 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:59.153 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T13:38:59.153 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T13:38:59.259 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rbd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:59.379 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up rbd-fuse (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:59.505 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-rgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:59.631 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up python3-cephfs (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:59.640 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:59.757 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephfs-shell (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:59.883 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-common (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:38:59.895 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T13:38:59.895 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T13:39:00.351 INFO:teuthology.orchestra.run.smithi080.stdout:Adding group ceph....done 2024-04-25T13:39:00.408 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:00.518 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:00.652 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:00.728 INFO:teuthology.orchestra.run.smithi080.stdout:Adding system user ceph....done 2024-04-25T13:39:00.864 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T13:39:00.864 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T13:39:00.897 INFO:teuthology.orchestra.run.smithi080.stdout:Setting system user ceph properties....done 2024-04-25T13:39:00.914 INFO:teuthology.orchestra.run.smithi080.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-25T13:39:01.050 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-25T13:39:01.395 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:01.397 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-25T13:39:01.546 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:01.672 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:01.840 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:02.154 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up cephfs-mirror (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:02.201 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T13:39:02.310 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T13:39:02.454 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T13:39:02.514 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:39:02.514 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-04-25T13:39:02.980 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-test (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:03.106 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up radosgw (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:03.220 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T13:39:03.588 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:39:03.589 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-25T13:39:03.850 INFO:teuthology.orchestra.run.smithi087.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T13:39:03.853 DEBUG:teuthology.orchestra.run.smithi087:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T13:39:03.912 INFO:teuthology.orchestra.run.smithi087.stdout:Reading package lists... 2024-04-25T13:39:04.114 INFO:teuthology.orchestra.run.smithi087.stdout:Building dependency tree... 2024-04-25T13:39:04.114 INFO:teuthology.orchestra.run.smithi087.stdout:Reading state information... 2024-04-25T13:39:04.130 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-base (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:04.304 INFO:teuthology.orchestra.run.smithi087.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T13:39:04.304 INFO:teuthology.orchestra.run.smithi087.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T13:39:04.304 INFO:teuthology.orchestra.run.smithi087.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T13:39:04.304 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:39:04.304 INFO:teuthology.orchestra.run.smithi087.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:39:04.304 INFO:teuthology.orchestra.run.smithi087.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:39:04.304 INFO:teuthology.orchestra.run.smithi087.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:39:04.305 INFO:teuthology.orchestra.run.smithi087.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T13:39:04.305 INFO:teuthology.orchestra.run.smithi087.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T13:39:04.305 INFO:teuthology.orchestra.run.smithi087.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:39:04.306 INFO:teuthology.orchestra.run.smithi087.stdout:The following additional packages will be installed: 2024-04-25T13:39:04.306 INFO:teuthology.orchestra.run.smithi087.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T13:39:04.306 INFO:teuthology.orchestra.run.smithi087.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T13:39:04.307 INFO:teuthology.orchestra.run.smithi087.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T13:39:04.307 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T13:39:04.307 INFO:teuthology.orchestra.run.smithi087.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T13:39:04.308 INFO:teuthology.orchestra.run.smithi087.stdout:Suggested packages: 2024-04-25T13:39:04.308 INFO:teuthology.orchestra.run.smithi087.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T13:39:04.308 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T13:39:04.308 INFO:teuthology.orchestra.run.smithi087.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T13:39:04.346 INFO:teuthology.orchestra.run.smithi087.stdout:The following NEW packages will be installed: 2024-04-25T13:39:04.347 INFO:teuthology.orchestra.run.smithi087.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T13:39:04.347 INFO:teuthology.orchestra.run.smithi087.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T13:39:04.347 INFO:teuthology.orchestra.run.smithi087.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T13:39:04.347 INFO:teuthology.orchestra.run.smithi087.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T13:39:04.348 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T13:39:04.348 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T13:39:04.348 INFO:teuthology.orchestra.run.smithi087.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T13:39:04.349 INFO:teuthology.orchestra.run.smithi087.stdout:The following packages will be upgraded: 2024-04-25T13:39:04.349 INFO:teuthology.orchestra.run.smithi087.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T13:39:04.427 INFO:teuthology.orchestra.run.smithi087.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T13:39:04.427 INFO:teuthology.orchestra.run.smithi087.stdout:Need to get 55.7 MB of archives. 2024-04-25T13:39:04.427 INFO:teuthology.orchestra.run.smithi087.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T13:39:04.427 INFO:teuthology.orchestra.run.smithi087.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T13:39:04.617 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-25T13:39:04.815 INFO:teuthology.orchestra.run.smithi087.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T13:39:04.822 INFO:teuthology.orchestra.run.smithi087.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T13:39:04.868 INFO:teuthology.orchestra.run.smithi087.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T13:39:04.868 INFO:teuthology.orchestra.run.smithi087.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T13:39:04.869 INFO:teuthology.orchestra.run.smithi087.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T13:39:04.882 INFO:teuthology.orchestra.run.smithi087.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T13:39:04.883 INFO:teuthology.orchestra.run.smithi087.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T13:39:04.885 INFO:teuthology.orchestra.run.smithi087.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T13:39:04.887 INFO:teuthology.orchestra.run.smithi087.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T13:39:04.887 INFO:teuthology.orchestra.run.smithi087.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T13:39:04.888 INFO:teuthology.orchestra.run.smithi087.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T13:39:04.893 INFO:teuthology.orchestra.run.smithi087.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T13:39:04.911 INFO:teuthology.orchestra.run.smithi087.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T13:39:04.911 INFO:teuthology.orchestra.run.smithi087.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T13:39:04.912 INFO:teuthology.orchestra.run.smithi087.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T13:39:05.087 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mds (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:05.323 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T13:39:05.323 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-25T13:39:05.373 INFO:teuthology.orchestra.run.smithi087.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T13:39:05.377 INFO:teuthology.orchestra.run.smithi087.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T13:39:05.377 INFO:teuthology.orchestra.run.smithi087.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T13:39:05.381 INFO:teuthology.orchestra.run.smithi087.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T13:39:05.381 INFO:teuthology.orchestra.run.smithi087.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T13:39:05.389 INFO:teuthology.orchestra.run.smithi087.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T13:39:05.391 INFO:teuthology.orchestra.run.smithi087.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T13:39:05.403 INFO:teuthology.orchestra.run.smithi087.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T13:39:05.403 INFO:teuthology.orchestra.run.smithi087.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T13:39:05.435 INFO:teuthology.orchestra.run.smithi087.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T13:39:05.436 INFO:teuthology.orchestra.run.smithi087.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T13:39:05.447 INFO:teuthology.orchestra.run.smithi087.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T13:39:05.469 INFO:teuthology.orchestra.run.smithi087.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T13:39:05.474 INFO:teuthology.orchestra.run.smithi087.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T13:39:05.475 INFO:teuthology.orchestra.run.smithi087.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T13:39:05.475 INFO:teuthology.orchestra.run.smithi087.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T13:39:05.476 INFO:teuthology.orchestra.run.smithi087.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T13:39:05.494 INFO:teuthology.orchestra.run.smithi087.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T13:39:05.496 INFO:teuthology.orchestra.run.smithi087.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T13:39:05.797 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:05.811 INFO:teuthology.orchestra.run.smithi087.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T13:39:05.812 INFO:teuthology.orchestra.run.smithi087.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T13:39:05.812 INFO:teuthology.orchestra.run.smithi087.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T13:39:05.812 INFO:teuthology.orchestra.run.smithi087.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T13:39:05.812 INFO:teuthology.orchestra.run.smithi087.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T13:39:05.812 INFO:teuthology.orchestra.run.smithi087.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T13:39:05.828 INFO:teuthology.orchestra.run.smithi087.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T13:39:05.828 INFO:teuthology.orchestra.run.smithi087.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T13:39:06.024 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T13:39:06.024 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-25T13:39:06.231 INFO:teuthology.orchestra.run.smithi087.stdout:Preconfiguring packages ... 2024-04-25T13:39:06.377 INFO:teuthology.orchestra.run.smithi087.stdout:Fetched 55.7 MB in 1s (37.8 MB/s) 2024-04-25T13:39:06.682 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-osd (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:06.799 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T13:39:06.805 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T13:39:06.933 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T13:39:06.933 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-25T13:39:07.291 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T13:39:07.387 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-k8sevents (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:07.563 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:07.814 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mon (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:08.014 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T13:39:08.028 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T13:39:08.028 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-25T13:39:08.482 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-cephadm (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:08.617 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:08.735 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-mgr-dashboard (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:08.869 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ceph-volume (19.0.0-3291-g994e2f20-1jammy) ... 2024-04-25T13:39:09.003 INFO:teuthology.orchestra.run.smithi087.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T13:39:09.213 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package flex. 2024-04-25T13:39:09.214 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T13:39:09.296 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T13:39:09.301 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T13:39:09.315 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T13:39:09.337 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T13:39:09.467 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T13:39:09.836 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T13:39:10.009 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T13:39:10.168 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T13:39:10.420 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T13:39:10.849 INFO:teuthology.orchestra.run.smithi080.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T13:39:10.851 DEBUG:teuthology.orchestra.run.smithi080:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-04-25T13:39:10.903 INFO:teuthology.orchestra.run.smithi080.stdout:Reading package lists... 2024-04-25T13:39:10.914 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T13:39:10.919 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T13:39:11.041 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T13:39:11.081 INFO:teuthology.orchestra.run.smithi080.stdout:Building dependency tree... 2024-04-25T13:39:11.082 INFO:teuthology.orchestra.run.smithi080.stdout:Reading state information... 2024-04-25T13:39:11.253 INFO:teuthology.orchestra.run.smithi080.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-04-25T13:39:11.253 INFO:teuthology.orchestra.run.smithi080.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-04-25T13:39:11.253 INFO:teuthology.orchestra.run.smithi080.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-04-25T13:39:11.253 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages were automatically installed and are no longer required: 2024-04-25T13:39:11.253 INFO:teuthology.orchestra.run.smithi080.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-25T13:39:11.253 INFO:teuthology.orchestra.run.smithi080.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-25T13:39:11.254 INFO:teuthology.orchestra.run.smithi080.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-25T13:39:11.254 INFO:teuthology.orchestra.run.smithi080.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-25T13:39:11.254 INFO:teuthology.orchestra.run.smithi080.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-25T13:39:11.255 INFO:teuthology.orchestra.run.smithi080.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-25T13:39:11.255 INFO:teuthology.orchestra.run.smithi080.stdout:The following additional packages will be installed: 2024-04-25T13:39:11.255 INFO:teuthology.orchestra.run.smithi080.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-04-25T13:39:11.255 INFO:teuthology.orchestra.run.smithi080.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-04-25T13:39:11.255 INFO:teuthology.orchestra.run.smithi080.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-04-25T13:39:11.255 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-04-25T13:39:11.256 INFO:teuthology.orchestra.run.smithi080.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T13:39:11.256 INFO:teuthology.orchestra.run.smithi080.stdout:Suggested packages: 2024-04-25T13:39:11.257 INFO:teuthology.orchestra.run.smithi080.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-04-25T13:39:11.257 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-04-25T13:39:11.257 INFO:teuthology.orchestra.run.smithi080.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-04-25T13:39:11.297 INFO:teuthology.orchestra.run.smithi080.stdout:The following NEW packages will be installed: 2024-04-25T13:39:11.297 INFO:teuthology.orchestra.run.smithi080.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-04-25T13:39:11.298 INFO:teuthology.orchestra.run.smithi080.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-04-25T13:39:11.298 INFO:teuthology.orchestra.run.smithi080.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-04-25T13:39:11.298 INFO:teuthology.orchestra.run.smithi080.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-04-25T13:39:11.298 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-04-25T13:39:11.299 INFO:teuthology.orchestra.run.smithi080.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-04-25T13:39:11.299 INFO:teuthology.orchestra.run.smithi080.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-04-25T13:39:11.300 INFO:teuthology.orchestra.run.smithi080.stdout:The following packages will be upgraded: 2024-04-25T13:39:11.300 INFO:teuthology.orchestra.run.smithi080.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-04-25T13:39:11.359 INFO:teuthology.orchestra.run.smithi080.stdout:4 upgraded, 39 newly installed, 0 to remove and 298 not upgraded. 2024-04-25T13:39:11.359 INFO:teuthology.orchestra.run.smithi080.stdout:Need to get 55.7 MB of archives. 2024-04-25T13:39:11.359 INFO:teuthology.orchestra.run.smithi080.stdout:After this operation, 202 MB of additional disk space will be used. 2024-04-25T13:39:11.360 INFO:teuthology.orchestra.run.smithi080.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-04-25T13:39:11.399 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T13:39:11.505 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T13:39:11.521 INFO:teuthology.orchestra.run.smithi080.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-04-25T13:39:11.529 INFO:teuthology.orchestra.run.smithi080.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.15 [1,905 kB] 2024-04-25T13:39:11.569 INFO:teuthology.orchestra.run.smithi080.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-04-25T13:39:11.569 INFO:teuthology.orchestra.run.smithi080.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-04-25T13:39:11.570 INFO:teuthology.orchestra.run.smithi080.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-04-25T13:39:11.581 INFO:teuthology.orchestra.run.smithi080.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-04-25T13:39:11.582 INFO:teuthology.orchestra.run.smithi080.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-04-25T13:39:11.584 INFO:teuthology.orchestra.run.smithi080.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-04-25T13:39:11.585 INFO:teuthology.orchestra.run.smithi080.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-04-25T13:39:11.586 INFO:teuthology.orchestra.run.smithi080.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2024-04-25T13:39:11.587 INFO:teuthology.orchestra.run.smithi080.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-04-25T13:39:11.591 INFO:teuthology.orchestra.run.smithi080.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-04-25T13:39:11.609 INFO:teuthology.orchestra.run.smithi080.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-04-25T13:39:11.610 INFO:teuthology.orchestra.run.smithi080.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-04-25T13:39:11.611 INFO:teuthology.orchestra.run.smithi080.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-04-25T13:39:11.841 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T13:39:11.917 INFO:teuthology.orchestra.run.smithi080.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-04-25T13:39:11.919 INFO:teuthology.orchestra.run.smithi080.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-04-25T13:39:11.920 INFO:teuthology.orchestra.run.smithi080.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-04-25T13:39:11.922 INFO:teuthology.orchestra.run.smithi080.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2024-04-25T13:39:11.922 INFO:teuthology.orchestra.run.smithi080.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-04-25T13:39:11.927 INFO:teuthology.orchestra.run.smithi080.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.11-0ubuntu0.22.04.1 [144 kB] 2024-04-25T13:39:11.928 INFO:teuthology.orchestra.run.smithi080.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-04-25T13:39:11.935 INFO:teuthology.orchestra.run.smithi080.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-04-25T13:39:11.949 INFO:teuthology.orchestra.run.smithi080.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.15 [2,376 kB] 2024-04-25T13:39:11.971 INFO:teuthology.orchestra.run.smithi080.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-04-25T13:39:11.971 INFO:teuthology.orchestra.run.smithi080.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-04-25T13:39:11.981 INFO:teuthology.orchestra.run.smithi080.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-04-25T13:39:12.002 INFO:teuthology.orchestra.run.smithi080.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-04-25T13:39:12.007 INFO:teuthology.orchestra.run.smithi080.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-04-25T13:39:12.008 INFO:teuthology.orchestra.run.smithi080.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-04-25T13:39:12.008 INFO:teuthology.orchestra.run.smithi080.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-04-25T13:39:12.008 INFO:teuthology.orchestra.run.smithi080.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.11-0ubuntu0.22.04.1 [1,222 kB] 2024-04-25T13:39:12.021 INFO:teuthology.orchestra.run.smithi080.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-04-25T13:39:12.023 INFO:teuthology.orchestra.run.smithi080.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.11-0ubuntu0.22.04.1 [16.2 MB] 2024-04-25T13:39:12.155 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package bison. 2024-04-25T13:39:12.225 INFO:teuthology.orchestra.run.smithi087.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T13:39:12.230 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T13:39:12.274 INFO:teuthology.orchestra.run.smithi080.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-04-25T13:39:12.274 INFO:teuthology.orchestra.run.smithi080.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-04-25T13:39:12.274 INFO:teuthology.orchestra.run.smithi080.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-04-25T13:39:12.274 INFO:teuthology.orchestra.run.smithi080.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-04-25T13:39:12.275 INFO:teuthology.orchestra.run.smithi080.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-04-25T13:39:12.275 INFO:teuthology.orchestra.run.smithi080.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2024-04-25T13:39:12.275 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T13:39:12.286 INFO:teuthology.orchestra.run.smithi080.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-04-25T13:39:12.286 INFO:teuthology.orchestra.run.smithi080.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-04-25T13:39:12.688 INFO:teuthology.orchestra.run.smithi080.stdout:Preconfiguring packages ... 2024-04-25T13:39:12.760 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T13:39:12.788 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T13:39:12.826 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T13:39:12.847 INFO:teuthology.orchestra.run.smithi080.stdout:Fetched 55.7 MB in 1s (56.9 MB/s) 2024-04-25T13:39:13.137 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package dump. 2024-04-25T13:39:13.165 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T13:39:13.190 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T13:39:13.196 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T13:39:13.204 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T13:39:13.341 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T13:39:13.482 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package indent. 2024-04-25T13:39:13.499 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:13.532 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T13:39:13.785 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T13:39:13.801 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T13:39:13.843 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T13:39:13.973 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-04-25T13:39:14.172 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T13:39:14.200 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T13:39:14.238 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T13:39:14.525 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T13:39:14.541 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T13:39:14.574 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T13:39:14.879 INFO:teuthology.orchestra.run.smithi080.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-04-25T13:39:14.903 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T13:39:14.930 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T13:39:14.969 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T13:39:15.149 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package flex. 2024-04-25T13:39:15.221 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T13:39:15.237 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T13:39:15.244 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 173995 files and directories currently installed.) 2024-04-25T13:39:15.249 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-04-25T13:39:15.280 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T13:39:15.289 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking flex (2.6.4-8build2) ... 2024-04-25T13:39:15.541 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T13:39:15.569 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T13:39:15.607 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T13:39:15.830 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T13:39:15.902 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T13:39:15.930 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T13:39:15.960 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T13:39:16.003 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.15) over (3.0.2-0ubuntu1.7) ... 2024-04-25T13:39:16.423 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T13:39:16.992 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T13:39:16.997 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T13:39:17.128 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T13:39:17.555 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-04-25T13:39:17.588 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T13:39:17.616 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T13:39:17.660 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-04-25T13:39:17.697 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T13:39:17.952 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T13:39:17.969 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T13:39:17.988 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T13:39:18.002 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T13:39:18.304 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T13:39:18.332 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T13:39:18.370 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T13:39:18.469 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package bison. 2024-04-25T13:39:18.534 INFO:teuthology.orchestra.run.smithi080.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 174068 files and directories currently installed.) 2024-04-25T13:39:18.539 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-04-25T13:39:18.582 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T13:39:18.849 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T13:39:18.877 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T13:39:18.915 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T13:39:18.990 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-04-25T13:39:19.018 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-04-25T13:39:19.074 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T13:39:19.251 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T13:39:19.279 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T13:39:19.318 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T13:39:19.394 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package dump. 2024-04-25T13:39:19.421 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-04-25T13:39:19.460 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking dump (0.4b47-3) ... 2024-04-25T13:39:19.629 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T13:39:19.657 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:19.687 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:19.764 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package indent. 2024-04-25T13:39:19.781 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:19.822 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T13:39:20.015 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T13:39:20.043 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:20.081 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T13:39:20.101 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-04-25T13:39:20.128 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-04-25T13:39:20.167 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T13:39:20.384 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T13:39:20.400 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:20.442 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T13:39:20.481 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-04-25T13:39:20.508 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2024-04-25T13:39:20.546 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T13:39:20.694 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T13:39:20.720 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T13:39:20.761 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T13:39:20.799 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-04-25T13:39:20.827 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-04-25T13:39:20.866 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T13:39:21.178 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libfl2:amd64. 2024-04-25T13:39:21.206 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-04-25T13:39:21.223 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T13:39:21.236 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T13:39:21.251 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T13:39:21.289 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T13:39:21.472 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-04-25T13:39:21.489 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-04-25T13:39:21.531 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T13:39:21.642 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package modemmanager. 2024-04-25T13:39:21.670 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T13:39:21.709 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T13:39:21.783 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libjson-perl. 2024-04-25T13:39:21.800 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-04-25T13:39:21.842 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-04-25T13:39:22.162 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libllvm14:amd64. 2024-04-25T13:39:22.190 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-04-25T13:39:22.237 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T13:39:22.349 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package network-manager. 2024-04-25T13:39:22.377 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T13:39:22.423 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T13:39:23.174 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package ppp. 2024-04-25T13:39:23.203 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T13:39:23.487 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T13:39:23.804 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T13:39:23.832 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T13:39:23.870 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T13:39:23.932 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-04-25T13:39:23.959 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T13:39:23.998 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T13:39:24.123 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T13:39:24.139 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T13:39:24.173 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T13:39:24.268 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libmbim-proxy. 2024-04-25T13:39:24.284 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-04-25T13:39:24.326 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T13:39:24.417 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T13:39:24.430 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T13:39:24.467 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T13:39:24.654 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-04-25T13:39:24.682 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T13:39:24.721 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T13:39:24.779 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T13:39:24.807 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:24.845 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:25.032 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libndp0:amd64. 2024-04-25T13:39:25.060 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2024-04-25T13:39:25.099 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T13:39:25.425 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T13:39:25.435 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libnm0:amd64. 2024-04-25T13:39:25.453 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T13:39:25.463 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T13:39:25.502 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T13:39:25.510 INFO:teuthology.orchestra.run.smithi087.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T13:39:25.539 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-common (238) ... 2024-04-25T13:39:25.855 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libpq5:amd64. 2024-04-25T13:39:25.883 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../16-libpq5_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:25.922 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:25.982 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T13:39:26.010 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:26.051 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:26.267 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-04-25T13:39:26.295 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:26.333 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T13:39:26.636 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libqmi-proxy. 2024-04-25T13:39:26.652 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:26.715 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T13:39:26.972 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-04-25T13:39:26.988 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.15_amd64.deb ... 2024-04-25T13:39:27.030 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T13:39:27.434 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql. 2024-04-25T13:39:27.463 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T13:39:27.492 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql (14+238) ... 2024-04-25T13:39:27.526 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-04-25T13:39:27.554 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-04-25T13:39:27.584 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T13:39:27.728 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T13:39:27.744 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T13:39:27.786 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T13:39:27.904 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package modemmanager. 2024-04-25T13:39:27.932 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-04-25T13:39:27.979 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T13:39:28.072 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T13:39:28.100 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T13:39:28.138 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T13:39:28.458 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T13:39:28.486 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T13:39:28.524 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T13:39:28.578 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package network-manager. 2024-04-25T13:39:28.606 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-04-25T13:39:28.671 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-04-25T13:39:28.760 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package tcl. 2024-04-25T13:39:28.776 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T13:39:28.823 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T13:39:29.155 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T13:39:29.183 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T13:39:29.221 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T13:39:29.345 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package ppp. 2024-04-25T13:39:29.373 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-04-25T13:39:29.575 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T13:39:29.591 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T13:39:29.633 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T13:39:29.640 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T13:39:29.967 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package pptp-linux. 2024-04-25T13:39:29.995 INFO:teuthology.orchestra.run.smithi087.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T13:39:29.995 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-04-25T13:39:30.023 INFO:teuthology.orchestra.run.smithi087.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T13:39:30.033 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-04-25T13:39:30.062 INFO:teuthology.orchestra.run.smithi087.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T13:39:30.286 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package network-manager-pptp. 2024-04-25T13:39:30.306 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-04-25T13:39:30.353 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-04-25T13:39:30.358 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T13:39:30.542 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T13:39:30.667 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T13:39:30.723 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-client-common. 2024-04-25T13:39:30.751 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-04-25T13:39:30.798 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-client-common (238) ... 2024-04-25T13:39:30.818 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:30.953 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T13:39:31.079 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T13:39:31.102 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-client-14. 2024-04-25T13:39:31.130 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../27-postgresql-client-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:31.168 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:31.205 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:31.707 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T13:39:31.766 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-common. 2024-04-25T13:39:31.782 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-04-25T13:39:31.793 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T13:39:31.850 INFO:teuthology.orchestra.run.smithi080.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-04-25T13:39:31.880 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-common (238) ... 2024-04-25T13:39:32.181 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T13:39:32.181 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T13:39:32.357 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-14. 2024-04-25T13:39:32.385 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../29-postgresql-14_14.11-0ubuntu0.22.04.1_amd64.deb ... 2024-04-25T13:39:32.426 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:32.688 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T13:39:32.814 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T13:39:32.931 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T13:39:33.066 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T13:39:33.192 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T13:39:33.334 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T13:39:33.460 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T13:39:33.594 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T13:39:33.811 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql. 2024-04-25T13:39:33.827 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-04-25T13:39:33.869 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql (14+238) ... 2024-04-25T13:39:34.122 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-client. 2024-04-25T13:39:34.150 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-04-25T13:39:34.196 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-client (14+238) ... 2024-04-25T13:39:34.208 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T13:39:34.407 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package postgresql-contrib. 2024-04-25T13:39:34.417 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T13:39:34.423 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-04-25T13:39:34.457 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking postgresql-contrib (14+238) ... 2024-04-25T13:39:34.511 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T13:39:34.577 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T13:39:34.711 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T13:39:34.768 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package tcl8.6. 2024-04-25T13:39:34.796 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-04-25T13:39:34.834 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T13:39:34.854 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T13:39:34.971 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T13:39:35.087 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package tcl. 2024-04-25T13:39:35.089 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T13:39:35.104 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-04-25T13:39:35.142 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-04-25T13:39:35.180 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T13:39:35.240 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T13:39:35.366 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T13:39:35.490 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package wpasupplicant. 2024-04-25T13:39:35.518 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2024-04-25T13:39:35.542 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T13:39:35.557 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T13:39:35.668 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T13:39:35.803 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T13:39:35.920 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-04-25T13:39:35.936 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-04-25T13:39:35.971 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T13:39:35.986 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-04-25T13:39:36.080 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T13:39:36.348 INFO:teuthology.orchestra.run.smithi080.stdout:Selecting previously unselected package usb-modeswitch. 2024-04-25T13:39:36.376 INFO:teuthology.orchestra.run.smithi080.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-04-25T13:39:36.423 INFO:teuthology.orchestra.run.smithi080.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T13:39:36.551 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:36.551 INFO:teuthology.orchestra.run.smithi087.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T13:39:36.551 INFO:teuthology.orchestra.run.smithi087.stdout:which means they are currently configured by ifupdown: 2024-04-25T13:39:36.551 INFO:teuthology.orchestra.run.smithi087.stdout:- enp3s0f1 2024-04-25T13:39:36.551 INFO:teuthology.orchestra.run.smithi087.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T13:39:36.551 INFO:teuthology.orchestra.run.smithi087.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T13:39:36.551 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:36.675 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T13:39:36.728 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-client-common (238) ... 2024-04-25T13:39:36.905 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up flex (2.6.4-8build2) ... 2024-04-25T13:39:36.975 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T13:39:37.301 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T13:39:37.531 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-04-25T13:39:37.657 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libpq5:amd64 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:37.783 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-04-25T13:39:37.790 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T13:39:37.909 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-04-25T13:39:37.950 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T13:39:38.027 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-client-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:38.076 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T13:39:38.518 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-04-25T13:39:38.618 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2024-04-25T13:39:38.677 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-common (238) ... 2024-04-25T13:39:38.954 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T13:39:38.954 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-04-25T13:39:39.446 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-04-25T13:39:39.572 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-04-25T13:39:39.706 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.15) ... 2024-04-25T13:39:39.777 INFO:teuthology.orchestra.run.smithi087.stdout:Adding user postgres to group ssl-cert 2024-04-25T13:39:39.832 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-04-25T13:39:39.958 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-04-25T13:39:39.982 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:39.982 INFO:teuthology.orchestra.run.smithi087.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T13:39:40.084 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-04-25T13:39:40.196 INFO:teuthology.orchestra.run.smithi087.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T13:39:40.197 INFO:teuthology.orchestra.run.smithi087.stdout:Removing obsolete dictionary files: 2024-04-25T13:39:40.210 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-04-25T13:39:40.328 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-04-25T13:39:40.791 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T13:39:41.133 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-04-25T13:39:41.145 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:41.259 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up dump (0.4b47-3) ... 2024-04-25T13:39:41.353 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-04-25T13:39:41.419 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T13:39:41.482 INFO:teuthology.orchestra.run.smithi087.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T13:39:41.514 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T13:39:41.528 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2024-04-25T13:39:41.570 INFO:teuthology.orchestra.run.smithi087.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T13:39:41.570 INFO:teuthology.orchestra.run.smithi087.stdout:This user must also own the server process. 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout:The default text search configuration will be set to "english". 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout:Data page checksums are disabled. 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout:creating subdirectories ... ok 2024-04-25T13:39:41.571 INFO:teuthology.orchestra.run.smithi087.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T13:39:41.589 INFO:teuthology.orchestra.run.smithi087.stdout:selecting default max_connections ... 100 2024-04-25T13:39:41.609 INFO:teuthology.orchestra.run.smithi087.stdout:selecting default shared_buffers ... 128MB 2024-04-25T13:39:41.619 INFO:teuthology.orchestra.run.smithi087.stdout:selecting default time zone ... Etc/UTC 2024-04-25T13:39:41.620 INFO:teuthology.orchestra.run.smithi087.stdout:creating configuration files ... ok 2024-04-25T13:39:41.638 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up tcl (8.6.11+1build2) ... 2024-04-25T13:39:41.730 INFO:teuthology.orchestra.run.smithi087.stdout:running bootstrap script ... ok 2024-04-25T13:39:41.763 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libjson-perl (4.04000-1) ... 2024-04-25T13:39:41.898 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-04-25T13:39:41.988 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-04-25T13:39:42.075 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-04-25T13:39:42.404 INFO:teuthology.orchestra.run.smithi087.stdout:performing post-bootstrap initialization ... ok 2024-04-25T13:39:42.626 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-04-25T13:39:42.786 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2024-04-25T13:39:42.895 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T13:39:43.004 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-04-25T13:39:43.058 INFO:teuthology.orchestra.run.smithi087.stdout:syncing data to disk ... ok 2024-04-25T13:39:43.181 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-client (14+238) ... 2024-04-25T13:39:43.307 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-04-25T13:39:43.452 INFO:teuthology.orchestra.run.smithi087.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T13:39:43.788 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:43.788 INFO:teuthology.orchestra.run.smithi080.stdout:The following network interfaces were found in /etc/network/interfaces 2024-04-25T13:39:43.788 INFO:teuthology.orchestra.run.smithi080.stdout:which means they are currently configured by ifupdown: 2024-04-25T13:39:43.788 INFO:teuthology.orchestra.run.smithi080.stdout:- enp3s0f1 2024-04-25T13:39:43.788 INFO:teuthology.orchestra.run.smithi080.stdout:If you want to manage those interfaces with NetworkManager instead 2024-04-25T13:39:43.788 INFO:teuthology.orchestra.run.smithi080.stdout:remove their configuration from /etc/network/interfaces. 2024-04-25T13:39:43.788 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:43.904 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-04-25T13:39:44.240 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-04-25T13:39:44.572 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-04-25T13:39:45.076 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-04-25T13:39:45.219 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-04-25T13:39:45.328 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-04-25T13:39:45.429 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-common (238) ... 2024-04-25T13:39:46.176 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T13:39:46.310 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T13:39:46.339 INFO:teuthology.orchestra.run.smithi080.stdout:Adding user postgres to group ssl-cert 2024-04-25T13:39:46.436 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up postgresql (14+238) ... 2024-04-25T13:39:46.562 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T13:39:46.565 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:46.565 INFO:teuthology.orchestra.run.smithi080.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-04-25T13:39:46.696 INFO:teuthology.orchestra.run.smithi087.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T13:39:46.776 INFO:teuthology.orchestra.run.smithi080.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-04-25T13:39:46.777 INFO:teuthology.orchestra.run.smithi080.stdout:Removing obsolete dictionary files: 2024-04-25T13:39:46.954 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T13:39:46.955 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T13:39:47.315 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-04-25T13:39:47.490 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T13:39:47.591 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T13:39:47.731 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-14 (14.11-0ubuntu0.22.04.1) ... 2024-04-25T13:39:47.910 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T13:39:48.060 INFO:teuthology.orchestra.run.smithi080.stdout:Creating new PostgreSQL cluster 14/main ... 2024-04-25T13:39:48.062 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T13:39:48.097 INFO:teuthology.orchestra.run.smithi080.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-04-25T13:39:48.142 INFO:teuthology.orchestra.run.smithi080.stdout:The files belonging to this database system will be owned by user "postgres". 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout:This user must also own the server process. 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout:The database cluster will be initialized with locale "en_US". 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout:The default text search configuration will be set to "english". 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout:Data page checksums are disabled. 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:48.143 INFO:teuthology.orchestra.run.smithi080.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-04-25T13:39:48.144 INFO:teuthology.orchestra.run.smithi080.stdout:creating subdirectories ... ok 2024-04-25T13:39:48.144 INFO:teuthology.orchestra.run.smithi080.stdout:selecting dynamic shared memory implementation ... posix 2024-04-25T13:39:48.156 INFO:teuthology.orchestra.run.smithi080.stdout:selecting default max_connections ... 100 2024-04-25T13:39:48.173 INFO:teuthology.orchestra.run.smithi080.stdout:selecting default shared_buffers ... 128MB 2024-04-25T13:39:48.183 INFO:teuthology.orchestra.run.smithi080.stdout:selecting default time zone ... Etc/UTC 2024-04-25T13:39:48.184 INFO:teuthology.orchestra.run.smithi080.stdout:creating configuration files ... ok 2024-04-25T13:39:48.295 INFO:teuthology.orchestra.run.smithi080.stdout:running bootstrap script ... ok 2024-04-25T13:39:48.957 INFO:teuthology.orchestra.run.smithi080.stdout:performing post-bootstrap initialization ... ok 2024-04-25T13:39:49.431 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T13:39:49.530 INFO:teuthology.orchestra.run.smithi087.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T13:39:49.811 INFO:teuthology.orchestra.run.smithi080.stdout:syncing data to disk ... ok 2024-04-25T13:39:50.018 INFO:teuthology.orchestra.run.smithi087.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T13:39:50.024 DEBUG:teuthology.parallel:result is None 2024-04-25T13:39:50.190 INFO:teuthology.orchestra.run.smithi080.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-04-25T13:39:52.980 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T13:39:53.106 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql-contrib (14+238) ... 2024-04-25T13:39:53.232 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up postgresql (14+238) ... 2024-04-25T13:39:53.375 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-04-25T13:39:53.526 INFO:teuthology.orchestra.run.smithi080.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-04-25T13:39:53.812 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T13:39:53.812 INFO:teuthology.orchestra.run.smithi080.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-04-25T13:39:54.353 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-25T13:39:54.462 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-04-25T13:39:54.765 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-25T13:39:54.959 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-25T13:39:56.262 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-04-25T13:39:56.353 INFO:teuthology.orchestra.run.smithi080.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-25T13:39:56.806 INFO:teuthology.orchestra.run.smithi080.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-25T13:39:56.811 DEBUG:teuthology.parallel:result is None 2024-04-25T13:39:56.812 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:39:57.037 DEBUG:teuthology.orchestra.run.smithi080:> dpkg-query -W -f '${Version}' ceph 2024-04-25T13:39:57.056 INFO:teuthology.orchestra.run.smithi080.stdout:19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:39:57.056 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:39:57.057 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291-g994e2f20-1jammy is installed. 2024-04-25T13:39:57.058 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=994e2f20102a20fa9a27966c14b625cc01884df8 2024-04-25T13:39:57.193 DEBUG:teuthology.orchestra.run.smithi087:> dpkg-query -W -f '${Version}' ceph 2024-04-25T13:39:57.212 INFO:teuthology.orchestra.run.smithi087.stdout:19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:39:57.212 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3291-g994e2f20-1jammy 2024-04-25T13:39:57.212 INFO:teuthology.task.install:The correct ceph version 19.0.0-3291-g994e2f20-1jammy is installed. 2024-04-25T13:39:57.213 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T13:39:57.214 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:57.214 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:39:57.228 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:39:57.228 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T13:39:57.266 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T13:39:57.267 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:57.267 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:39:57.282 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:39:57.338 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:39:57.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T13:39:57.351 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T13:39:57.405 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T13:39:57.406 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:57.406 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:39:57.419 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:39:57.472 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:39:57.472 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T13:39:57.486 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T13:39:57.541 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T13:39:57.541 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:57.541 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:39:57.554 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:39:57.609 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:39:57.609 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T13:39:57.623 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T13:39:57.676 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T13:39:57.756 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T13:39:57.756 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 777 /var/log/ceph 2024-04-25T13:39:57.759 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /var/log/ceph 2024-04-25T13:39:57.771 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T13:39:57.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T13:39:57.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T13:39:57.827 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T13:39:57.827 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T13:39:57.871 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T13:39:57.889 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T13:39:57.890 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T13:39:57.890 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T13:39:57.890 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:57.890 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T13:39:57.930 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T13:39:58.004 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:39:58.005 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T13:39:58.010 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T13:39:58.088 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T13:39:58.088 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T13:39:58.089 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174', 'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/mds-full}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7673174', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240425.015853-debug'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240425.015853-debug', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 85, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1880, 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'sleep_before_teardown': 0, 'subset': '178/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240425.015853-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_994e2f20102a20fa9a27966c14b625cc01884df8/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi080.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJD/i+Q2yLx/9ITFeoBQlucVgb4Kk7o+CbCc5cFp1bUHTz+Qpf8VBTdCX4eFUrZcWh3VZIUe3fgHGltJ+DTCnM4=', 'smithi087.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL/VNH3ueL4MNCM4M0mx2qnHZ4LQHlkeJ18toGZD0Wpr4dpJod6IiLgFe8cNZkoqvM2QKta3eYMfoDwwGEMJcUU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '994e2f20102a20fa9a27966c14b625cc01884df8', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_10:00:23', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T13:39:58.089 DEBUG:teuthology.orchestra.run.smithi080:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T13:39:58.092 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T13:39:58.130 DEBUG:teuthology.orchestra.run.smithi080:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T13:39:58.139 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T13:39:58.186 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:58.186 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:39:58.198 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:39:58.198 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_1 2024-04-25T13:39:58.246 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:39:58.246 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:39:58.247 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 885 Links: 1 2024-04-25T13:39:58.247 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:39:58.247 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 13:33:22.513228227 +0000 2024-04-25T13:39:58.247 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 13:33:22.221230835 +0000 2024-04-25T13:39:58.247 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 13:33:22.221230835 +0000 2024-04-25T13:39:58.247 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-04-25T13:39:58.247 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:39:58.300 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T13:39:58.300 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T13:39:58.301 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000427694 s, 1.2 MB/s 2024-04-25T13:39:58.302 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:39:58.351 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_2 2024-04-25T13:39:58.401 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:39:58.402 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:39:58.402 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 905 Links: 1 2024-04-25T13:39:58.402 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:39:58.402 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 13:33:23.025223654 +0000 2024-04-25T13:39:58.402 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 13:33:22.789225762 +0000 2024-04-25T13:39:58.402 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 13:33:22.789225762 +0000 2024-04-25T13:39:58.402 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-04-25T13:39:58.402 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:39:58.456 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T13:39:58.456 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T13:39:58.456 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000445841 s, 1.1 MB/s 2024-04-25T13:39:58.457 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:39:58.511 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_3 2024-04-25T13:39:58.558 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:39:58.558 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:39:58.558 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-04-25T13:39:58.558 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:39:58.558 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 13:33:23.257221582 +0000 2024-04-25T13:39:58.558 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 13:33:23.253221616 +0000 2024-04-25T13:39:58.558 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 13:33:23.253221616 +0000 2024-04-25T13:39:58.558 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-04-25T13:39:58.558 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:39:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T13:39:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T13:39:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000409674 s, 1.2 MB/s 2024-04-25T13:39:58.613 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:39:58.663 DEBUG:teuthology.orchestra.run.smithi080:> stat /dev/vg_nvme/lv_4 2024-04-25T13:39:58.710 INFO:teuthology.orchestra.run.smithi080.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:39:58.710 INFO:teuthology.orchestra.run.smithi080.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:39:58.710 INFO:teuthology.orchestra.run.smithi080.stdout:Device: 5h/5d Inode: 942 Links: 1 2024-04-25T13:39:58.710 INFO:teuthology.orchestra.run.smithi080.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:39:58.710 INFO:teuthology.orchestra.run.smithi080.stdout:Access: 2024-04-25 13:33:24.033214650 +0000 2024-04-25T13:39:58.710 INFO:teuthology.orchestra.run.smithi080.stdout:Modify: 2024-04-25 13:33:23.757217115 +0000 2024-04-25T13:39:58.711 INFO:teuthology.orchestra.run.smithi080.stdout:Change: 2024-04-25 13:33:23.757217115 +0000 2024-04-25T13:39:58.711 INFO:teuthology.orchestra.run.smithi080.stdout: Birth: - 2024-04-25T13:39:58.711 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:39:58.768 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T13:39:58.768 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records out 2024-04-25T13:39:58.768 INFO:teuthology.orchestra.run.smithi080.stderr:512 bytes copied, 0.000299579 s, 1.7 MB/s 2024-04-25T13:39:58.769 DEBUG:teuthology.orchestra.run.smithi080:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:39:58.823 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T13:39:58.824 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:39:58.824 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T13:39:58.829 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T13:39:58.829 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-04-25T13:39:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T13:39:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:39:58.879 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-04-25T13:39:58.879 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:39:58.879 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 13:33:22.383720885 +0000 2024-04-25T13:39:58.879 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 13:33:22.379720921 +0000 2024-04-25T13:39:58.879 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 13:33:22.379720921 +0000 2024-04-25T13:39:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-04-25T13:39:58.879 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T13:39:58.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T13:39:58.933 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T13:39:58.933 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000453963 s, 1.1 MB/s 2024-04-25T13:39:58.934 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T13:39:58.983 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-04-25T13:39:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T13:39:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:39:59.029 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 886 Links: 1 2024-04-25T13:39:59.030 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:39:59.030 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 13:33:23.159713962 +0000 2024-04-25T13:39:59.030 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 13:33:22.879716460 +0000 2024-04-25T13:39:59.030 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 13:33:22.879716460 +0000 2024-04-25T13:39:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-04-25T13:39:59.030 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T13:39:59.084 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T13:39:59.084 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T13:39:59.084 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.0003574 s, 1.4 MB/s 2024-04-25T13:39:59.085 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T13:39:59.135 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-04-25T13:39:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T13:39:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:39:59.182 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 907 Links: 1 2024-04-25T13:39:59.182 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:39:59.182 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 13:33:23.683709287 +0000 2024-04-25T13:39:59.182 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 13:33:23.399711822 +0000 2024-04-25T13:39:59.182 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 13:33:23.399711822 +0000 2024-04-25T13:39:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-04-25T13:39:59.182 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T13:39:59.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T13:39:59.237 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T13:39:59.238 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000385457 s, 1.3 MB/s 2024-04-25T13:39:59.239 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T13:39:59.287 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-04-25T13:39:59.334 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T13:39:59.334 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T13:39:59.334 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 928 Links: 1 2024-04-25T13:39:59.334 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T13:39:59.334 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-04-25 13:33:24.159705042 +0000 2024-04-25T13:39:59.334 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-04-25 13:33:23.899707361 +0000 2024-04-25T13:39:59.334 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-04-25 13:33:23.899707361 +0000 2024-04-25T13:39:59.334 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: - 2024-04-25T13:39:59.334 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T13:39:59.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-04-25T13:39:59.388 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-04-25T13:39:59.388 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000280721 s, 1.8 MB/s 2024-04-25T13:39:59.389 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T13:39:59.439 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T13:39:59.439 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T13:39:59.439 INFO:tasks.ceph:Generating config... 2024-04-25T13:39:59.440 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T13:39:59.440 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client] pid file = 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client.0] debug client = 20 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client.0] debug objecter = 20 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client.1] debug client = 20 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2024-04-25T13:39:59.441 INFO:tasks.ceph:[client.1] debug objecter = 20 2024-04-25T13:39:59.441 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T13:39:59.441 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T13:39:59.442 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-25T13:39:59.443 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T13:39:59.443 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2024-04-25T13:39:59.443 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2024-04-25T13:39:59.443 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T13:39:59.443 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2024-04-25T13:39:59.444 INFO:tasks.ceph:[osd] osd mon report interval = 5 2024-04-25T13:39:59.444 INFO:tasks.ceph:[osd] osd objectstore = memstore 2024-04-25T13:39:59.444 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T13:39:59.444 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T13:39:59.444 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T13:39:59.509 INFO:teuthology.orchestra.run.smithi080.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T13:39:59.522 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T13:39:59.587 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T13:39:59.598 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.80'), ('mon.b', '172.21.15.87'), ('mon.c', '[v2:172.21.15.87:3301,v1:172.21.15.87:6790]')] 2024-04-25T13:39:59.598 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.80,172.21.15.87,[v2:172.21.15.87:3301,v1:172.21.15.87:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T13:39:59.599 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:59.599 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T13:39:59.645 DEBUG:teuthology.orchestra.run.smithi080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.80 --add b 172.21.15.87 --addv c '[v2:172.21.15.87:3301,v1:172.21.15.87:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:39:59.759 INFO:teuthology.orchestra.run.smithi080.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:monmaptool: generated fsid 3a629b7a-ce3b-47b8-98ac-844ad072aa94 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:setting min_mon_release = quincy 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:epoch 0 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:fsid 3a629b7a-ce3b-47b8-98ac-844ad072aa94 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:last_changed 2024-04-25T13:39:59.762758+0000 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:created 2024-04-25T13:39:59.762758+0000 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:min_mon_release 17 (quincy) 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:election_strategy: 1 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:0: [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] mon.a 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:1: [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon.b 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:2: [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] mon.c 2024-04-25T13:39:59.760 INFO:teuthology.orchestra.run.smithi080.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T13:39:59.765 DEBUG:teuthology.orchestra.run.smithi080:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T13:39:59.771 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 3a629b7a-ce3b-47b8-98ac-844ad072aa94... 2024-04-25T13:39:59.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T13:39:59.815 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T13:39:59.838 INFO:teuthology.orchestra.run.smithi080.stdout:[global] 2024-04-25T13:39:59.838 INFO:teuthology.orchestra.run.smithi080.stdout: chdir = "" 2024-04-25T13:39:59.838 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T13:39:59.838 INFO:teuthology.orchestra.run.smithi080.stdout: auth supported = cephx 2024-04-25T13:39:59.838 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.838 INFO:teuthology.orchestra.run.smithi080.stdout: filestore xattr use omap = true 2024-04-25T13:39:59.838 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: mon clock drift allowed = 1.000 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: osd crush chooseleaf type = 0 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: auth debug = true 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on old message = true 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: ms die on bug = true 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: mon pg warn max object skew = 0 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.839 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default size = 2 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow primary affinity = true 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd allow pg remap = true 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on legacy crush tunables = false 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on crush straw calc version zero = false 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on no sortbitwise = false 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on osd down out interval zero = false 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon warn on too few osds = false 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: mon_allow_pool_size_one = true 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T13:39:59.840 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: osd default data pool replay window = 5 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: mon allow pool delete = true 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: mon cluster log file level = debug 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: debug asserts on shutdown = true 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: mon health detail to clog = false 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: mon host = "172.21.15.80,172.21.15.87,[v2:172.21.15.87:3301,v1:172.21.15.87:6790]" 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: fsid = 3a629b7a-ce3b-47b8-98ac-844ad072aa94 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout:[osd] 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: osd journal size = 100 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub load threshold = 5.0 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: osd scrub max interval = 600 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: osd mclock profile = high_recovery_ops 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.841 INFO:teuthology.orchestra.run.smithi080.stdout: osd recover clone overlap = true 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd recovery max chunk = 1048576 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug shutdown = true 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug op order = true 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug verify stray on activate = true 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug trim objects = true 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd open classes on start = true 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug pg log writeout = true 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd deep scrub update digest min age = 30 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: osd map max advance = 10 2024-04-25T13:39:59.842 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: journal zero on create = true 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: filestore ondisk finisher threads = 3 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: filestore apply finisher threads = 3 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: bdev debug aio = true 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: osd debug misdirected ops = true 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore block size = 96636764160 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: bluestore fsck on mount = True 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluefs = 20 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: debug bluestore = 20 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: debug osd = 20 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: debug rocksdb = 10 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: memstore device bytes = 200000000 2024-04-25T13:39:59.843 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.9 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.8 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: osd failsafe full ratio = 1.0 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: osd mon report interval = 5 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: osd objectstore = memstore 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr] 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: debug mgr = 20 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T13:39:59.844 INFO:teuthology.orchestra.run.smithi080.stdout: mgr/telemetry/nag = false 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout:[mon] 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: debug mon = 20 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: debug paxos = 20 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: debug auth = 20 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: mon data avail warn = 5 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: mon mgr mkfs grace = 240 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min pgs per osd = 4 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd reporter subtree level = osd 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd prime pg temp = true 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: mon reweight min bytes per osd = 10 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T13:39:59.845 INFO:teuthology.orchestra.run.smithi080.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: # 1m isn't quite enough 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon_down_mkfs_grace = 2m 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon_warn_on_filestore_osds = false 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon down mkfs grace = 300 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon op complaint time = 120 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd backfillfull ratio = 0.6 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd full ratio = 0.7 2024-04-25T13:39:59.846 INFO:teuthology.orchestra.run.smithi080.stdout: mon osd nearfull ratio = 0.6 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout:[client] 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: rgw cache enabled = true 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable ops log = true 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: rgw enable usage log = true 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: client mount timeout = 600 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: pid file = "" 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.a] 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.b] 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout:[mon.c] 2024-04-25T13:39:59.847 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c] 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d] 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug objectcacher = 20 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug objecter = 20 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout:[client.1] 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug client = 20 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug objectcacher = 20 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug objecter = 20 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout:[mds] 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds = 20 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug mds balancer = 20 2024-04-25T13:39:59.848 INFO:teuthology.orchestra.run.smithi080.stdout: debug ms = 1 2024-04-25T13:39:59.849 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug frag = True 2024-04-25T13:39:59.849 INFO:teuthology.orchestra.run.smithi080.stdout: mds debug scatterstat = True 2024-04-25T13:39:59.849 INFO:teuthology.orchestra.run.smithi080.stdout: mds op complaint time = 180 2024-04-25T13:39:59.849 INFO:teuthology.orchestra.run.smithi080.stdout: mds verify scatter = True 2024-04-25T13:39:59.849 INFO:teuthology.orchestra.run.smithi080.stdout: osd op complaint time = 180 2024-04-25T13:39:59.849 INFO:teuthology.orchestra.run.smithi080.stdout: rados mon op timeout = 900 2024-04-25T13:39:59.849 INFO:teuthology.orchestra.run.smithi080.stdout: rados osd op timeout = 900 2024-04-25T13:39:59.849 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: chdir = "" 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: auth supported = cephx 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: filestore xattr use omap = true 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: mon clock drift allowed = 1.000 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: osd crush chooseleaf type = 0 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: auth debug = true 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on old message = true 2024-04-25T13:39:59.850 INFO:teuthology.orchestra.run.smithi087.stdout: ms die on bug = true 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon pg warn max object skew = 0 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default size = 2 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow primary affinity = true 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd allow pg remap = true 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on legacy crush tunables = false 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on crush straw calc version zero = false 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on no sortbitwise = false 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on osd down out interval zero = false 2024-04-25T13:39:59.851 INFO:teuthology.orchestra.run.smithi087.stdout: mon warn on too few osds = false 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: mon_allow_pool_size_one = true 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: osd default data pool replay window = 5 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: mon allow pool delete = true 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: mon cluster log file level = debug 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: debug asserts on shutdown = true 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: mon health detail to clog = false 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: mon host = "172.21.15.80,172.21.15.87,[v2:172.21.15.87:3301,v1:172.21.15.87:6790]" 2024-04-25T13:39:59.852 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 3a629b7a-ce3b-47b8-98ac-844ad072aa94 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout:[osd] 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd journal size = 100 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub load threshold = 5.0 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd scrub max interval = 600 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd mclock profile = high_recovery_ops 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd recover clone overlap = true 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd recovery max chunk = 1048576 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug shutdown = true 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug op order = true 2024-04-25T13:39:59.853 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug verify stray on activate = true 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug trim objects = true 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: osd open classes on start = true 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug pg log writeout = true 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: osd deep scrub update digest min age = 30 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: osd map max advance = 10 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: journal zero on create = true 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: filestore ondisk finisher threads = 3 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: filestore apply finisher threads = 3 2024-04-25T13:39:59.854 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: bdev debug aio = true 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: osd debug misdirected ops = true 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore block size = 96636764160 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: bluestore fsck on mount = True 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluefs = 20 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: debug bluestore = 20 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: debug osd = 20 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: debug rocksdb = 10 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: memstore device bytes = 200000000 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.9 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.8 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: osd failsafe full ratio = 1.0 2024-04-25T13:39:59.855 INFO:teuthology.orchestra.run.smithi087.stdout: osd mon report interval = 5 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: osd objectstore = memstore 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr] 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: debug mgr = 20 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: mgr/telemetry/nag = false 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout:[mon] 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T13:39:59.856 INFO:teuthology.orchestra.run.smithi087.stdout: debug mon = 20 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: debug paxos = 20 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: debug auth = 20 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: mon data avail warn = 5 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: mon mgr mkfs grace = 240 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min pgs per osd = 4 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd reporter subtree level = osd 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd prime pg temp = true 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: mon reweight min bytes per osd = 10 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.857 INFO:teuthology.orchestra.run.smithi087.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: # 1m isn't quite enough 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: mon_down_mkfs_grace = 2m 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: mon_warn_on_filestore_osds = false 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: mon down mkfs grace = 300 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: mon op complaint time = 120 2024-04-25T13:39:59.858 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd backfillfull ratio = 0.6 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd full ratio = 0.7 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: mon osd nearfull ratio = 0.6 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout:[client] 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: rgw cache enabled = true 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable ops log = true 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: rgw enable usage log = true 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: client mount timeout = 600 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: pid file = "" 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T13:39:59.859 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.a] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.b] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[mon.c] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout: debug objectcacher = 20 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout: debug objecter = 20 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout:[client.1] 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout: debug client = 20 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout: debug objectcacher = 20 2024-04-25T13:39:59.860 INFO:teuthology.orchestra.run.smithi087.stdout: debug objecter = 20 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout:[mds] 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds = 20 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: debug mds balancer = 20 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: debug ms = 1 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug frag = True 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: mds debug scatterstat = True 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: mds op complaint time = 180 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: mds verify scatter = True 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: osd op complaint time = 180 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: rados mon op timeout = 900 2024-04-25T13:39:59.861 INFO:teuthology.orchestra.run.smithi087.stdout: rados osd op timeout = 900 2024-04-25T13:39:59.862 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T13:39:59.863 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T13:39:59.984 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T13:39:59.984 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:59.984 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T13:39:59.989 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:39:59.989 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T13:40:00.038 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:40:00.038 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:00.038 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T13:40:00.038 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T13:40:00.101 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:00.101 DEBUG:teuthology.orchestra.run.smithi080:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:00.149 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:40:00.149 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:00.150 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T13:40:00.150 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T13:40:00.173 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:00.174 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:00.222 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T13:40:00.222 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T13:40:00.222 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T13:40:00.294 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T13:40:00.301 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T13:40:00.376 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T13:40:00.389 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T13:40:00.389 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T13:40:00.457 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T13:40:00.464 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T13:40:00.476 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T13:40:00.579 INFO:teuthology.orchestra.run.smithi080.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T13:40:00.585 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T13:40:00.597 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T13:40:00.673 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T13:40:00.680 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T13:40:00.692 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T13:40:00.795 INFO:teuthology.orchestra.run.smithi087.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T13:40:00.802 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T13:40:00.814 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T13:40:00.814 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T13:40:00.880 INFO:teuthology.orchestra.run.smithi080.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T13:40:00.895 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T13:40:00.956 INFO:teuthology.orchestra.run.smithi080.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T13:40:00.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T13:40:01.033 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T13:40:01.048 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T13:40:01.103 INFO:teuthology.orchestra.run.smithi087.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T13:40:01.117 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T13:40:01.117 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi080.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi087.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T13:40:01.117 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T13:40:01.130 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T13:40:01.130 INFO:tasks.ceph:role: osd.0 2024-04-25T13:40:01.130 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:40:01.131 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T13:40:01.194 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T13:40:01.681 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T13:40:01.683 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T13:40:01.683 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T13:40:01.827 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T13:40:01.838 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T13:40:01.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:01.839 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T13:40:01.893 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T13:40:01.894 INFO:tasks.ceph:role: osd.1 2024-04-25T13:40:01.894 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:40:01.894 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T13:40:01.957 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T13:40:02.407 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T13:40:02.409 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T13:40:02.409 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T13:40:02.504 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T13:40:02.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:02.514 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T13:40:02.514 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T13:40:02.567 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T13:40:02.568 INFO:tasks.ceph:role: osd.2 2024-04-25T13:40:02.568 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:40:02.568 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T13:40:02.629 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T13:40:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T13:40:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T13:40:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T13:40:02.629 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T13:40:02.629 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T13:40:02.629 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T13:40:02.629 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T13:40:02.630 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T13:40:02.630 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T13:40:03.094 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T13:40:03.096 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T13:40:03.096 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T13:40:03.207 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T13:40:03.218 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T13:40:03.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:03.219 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T13:40:03.277 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T13:40:03.277 INFO:tasks.ceph:role: osd.3 2024-04-25T13:40:03.277 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:40:03.277 DEBUG:teuthology.orchestra.run.smithi080:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T13:40:03.342 INFO:teuthology.orchestra.run.smithi080.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T13:40:03.342 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T13:40:03.343 INFO:teuthology.orchestra.run.smithi080.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T13:40:03.343 INFO:teuthology.orchestra.run.smithi080.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T13:40:03.343 INFO:teuthology.orchestra.run.smithi080.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T13:40:03.343 INFO:teuthology.orchestra.run.smithi080.stdout: = sunit=0 swidth=0 blks 2024-04-25T13:40:03.343 INFO:teuthology.orchestra.run.smithi080.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T13:40:03.344 INFO:teuthology.orchestra.run.smithi080.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T13:40:03.344 INFO:teuthology.orchestra.run.smithi080.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T13:40:03.344 INFO:teuthology.orchestra.run.smithi080.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T13:40:03.841 INFO:teuthology.orchestra.run.smithi080.stdout:Discarding blocks...Done. 2024-04-25T13:40:03.843 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi080.front.sepia.ceph.com -o noatime 2024-04-25T13:40:03.843 DEBUG:teuthology.orchestra.run.smithi080:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T13:40:03.973 DEBUG:teuthology.orchestra.run.smithi080:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T13:40:03.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:03.985 INFO:teuthology.orchestra.run.smithi080.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T13:40:03.985 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:04.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.126+0000 7f0fb71df7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T13:40:04.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.130+0000 7f0fb71df7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T13:40:04.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.138+0000 7f0fb71df7c0 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2024-04-25T13:40:04.163 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T13:40:04.175 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:04.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.286+0000 7f140826f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T13:40:04.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.286+0000 7f140826f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T13:40:04.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.286+0000 7f140826f7c0 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2024-04-25T13:40:04.305 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T13:40:04.317 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:04.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.430+0000 7fe04da7e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T13:40:04.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.430+0000 7fe04da7e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T13:40:04.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.430+0000 7fe04da7e7c0 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2024-04-25T13:40:04.447 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T13:40:04.459 DEBUG:teuthology.orchestra.run.smithi080:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:04.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.566+0000 7f1c3aa5c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T13:40:04.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.566+0000 7f1c3aa5c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T13:40:04.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:04.566+0000 7f1c3aa5c7c0 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2024-04-25T13:40:04.582 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T13:40:04.592 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T13:40:04.606 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T13:40:04.606 INFO:tasks.ceph:role: osd.4 2024-04-25T13:40:04.607 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:40:04.607 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T13:40:04.670 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T13:40:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T13:40:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T13:40:04.670 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T13:40:04.671 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T13:40:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T13:40:04.671 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T13:40:04.671 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T13:40:04.671 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T13:40:04.671 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T13:40:05.134 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T13:40:05.136 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T13:40:05.136 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T13:40:05.236 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T13:40:05.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:05.246 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T13:40:05.247 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T13:40:05.303 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T13:40:05.303 INFO:tasks.ceph:role: osd.5 2024-04-25T13:40:05.303 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:40:05.303 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T13:40:05.370 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T13:40:05.370 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T13:40:05.370 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T13:40:05.370 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T13:40:05.370 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T13:40:05.371 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T13:40:05.371 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T13:40:05.371 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T13:40:05.371 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T13:40:05.371 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T13:40:05.847 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T13:40:05.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T13:40:05.850 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T13:40:06.010 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T13:40:06.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:06.021 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T13:40:06.021 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T13:40:06.077 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T13:40:06.077 INFO:tasks.ceph:role: osd.6 2024-04-25T13:40:06.077 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:40:06.077 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T13:40:06.142 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T13:40:06.143 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T13:40:06.619 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T13:40:06.621 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T13:40:06.621 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T13:40:06.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T13:40:06.782 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T13:40:06.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:06.783 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T13:40:06.842 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T13:40:06.842 INFO:tasks.ceph:role: osd.7 2024-04-25T13:40:06.842 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:40:06.842 DEBUG:teuthology.orchestra.run.smithi087:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T13:40:06.905 INFO:teuthology.orchestra.run.smithi087.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout: = sunit=0 swidth=0 blks 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T13:40:06.906 INFO:teuthology.orchestra.run.smithi087.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T13:40:07.338 INFO:teuthology.orchestra.run.smithi087.stdout:Discarding blocks...Done. 2024-04-25T13:40:07.341 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi087.front.sepia.ceph.com -o noatime 2024-04-25T13:40:07.341 DEBUG:teuthology.orchestra.run.smithi087:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T13:40:07.439 DEBUG:teuthology.orchestra.run.smithi087:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T13:40:07.449 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: /sbin/restorecon: command not found 2024-04-25T13:40:07.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:07.450 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:07.591 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.588+0000 7f31f29a77c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T13:40:07.595 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.592+0000 7f31f29a77c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T13:40:07.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.592+0000 7f31f29a77c0 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2024-04-25T13:40:07.616 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T13:40:07.627 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:07.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.740+0000 7fcfc20677c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T13:40:07.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.740+0000 7fcfc20677c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T13:40:07.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.744+0000 7fcfc20677c0 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2024-04-25T13:40:07.770 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T13:40:07.782 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:07.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.888+0000 7febaa7387c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T13:40:07.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.892+0000 7febaa7387c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T13:40:07.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:07.900+0000 7febaa7387c0 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2024-04-25T13:40:07.922 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T13:40:07.932 DEBUG:teuthology.orchestra.run.smithi087:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:08.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:08.036+0000 7f69a05867c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T13:40:08.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:08.036+0000 7f69a05867c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T13:40:08.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:08.044+0000 7f69a05867c0 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2024-04-25T13:40:08.071 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T13:40:08.082 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T13:40:08.082 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.082 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T13:40:08.096 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.097 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T13:40:08.149 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.149 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T13:40:08.205 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.205 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T13:40:08.262 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.262 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T13:40:08.317 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.317 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T13:40:08.373 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.501 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T13:40:08.514 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.514 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T13:40:08.527 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.527 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T13:40:08.581 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.581 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T13:40:08.637 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.638 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T13:40:08.692 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.692 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T13:40:08.745 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.745 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T13:40:08.802 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.802 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T13:40:08.857 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.857 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T13:40:08.864 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:08.864 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T13:40:08.910 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.910 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:08.916 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T13:40:08.966 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T13:40:08.966 DEBUG:teuthology.orchestra.run.smithi080:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T13:40:08.968 DEBUG:teuthology.orchestra.run.smithi087:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T13:40:09.011 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.y] 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw== 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.a] 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ== 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.c] 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg== 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.0] 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ== 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.1] 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA== 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.2] 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q== 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.3] 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw== 2024-04-25T13:40:09.012 INFO:teuthology.orchestra.run.smithi080.stdout:[mgr.x] 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ== 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.b] 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg== 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout:[mds.d] 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ== 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.4] 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw== 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.5] 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw== 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.6] 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ== 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout:[osd.7] 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ== 2024-04-25T13:40:09.013 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:40:09.014 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:40:09.014 INFO:teuthology.orchestra.run.smithi080.stdout:[client.2] 2024-04-25T13:40:09.014 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg== 2024-04-25T13:40:09.014 INFO:teuthology.orchestra.run.smithi080.stdout:[client.1] 2024-04-25T13:40:09.014 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w== 2024-04-25T13:40:09.014 INFO:teuthology.orchestra.run.smithi080.stdout:[client.3] 2024-04-25T13:40:09.014 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ== 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.y] 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw== 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a] 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ== 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c] 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg== 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.0] 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ== 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.1] 2024-04-25T13:40:09.020 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA== 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.2] 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q== 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.3] 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw== 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout:[mgr.x] 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ== 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b] 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg== 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d] 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ== 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.4] 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw== 2024-04-25T13:40:09.021 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.5] 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw== 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.6] 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ== 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout:[osd.7] 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ== 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout:[client.2] 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg== 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout:[client.1] 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w== 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout:[client.3] 2024-04-25T13:40:09.022 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ== 2024-04-25T13:40:09.023 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T13:40:09.059 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T13:40:09.145 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T13:40:09.148 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T13:40:09.226 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T13:40:09.229 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T13:40:09.306 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.308 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.377 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.380 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.449 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.451 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.520 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.523 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.592 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T13:40:09.594 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T13:40:09.663 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T13:40:09.665 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T13:40:09.738 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T13:40:09.740 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T13:40:09.811 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.813 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.878 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.880 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.949 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:09.952 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:10.019 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:10.022 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T13:40:10.094 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T13:40:10.096 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T13:40:10.186 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T13:40:10.188 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T13:40:10.263 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T13:40:10.265 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T13:40:10.402 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T13:40:10.405 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T13:40:10.479 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T13:40:10.479 DEBUG:teuthology.orchestra.run.smithi080:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T13:40:10.491 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T13:40:10.615 DEBUG:teuthology.orchestra.run.smithi080:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T13:40:10.625 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T13:40:10.639 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T13:40:10.764 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T13:40:10.774 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T13:40:10.827 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T13:40:10.949 DEBUG:teuthology.orchestra.run.smithi087:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T13:40:10.960 DEBUG:teuthology.orchestra.run.smithi080:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:10.963 DEBUG:teuthology.orchestra.run.smithi087:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T13:40:11.005 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T13:40:11.005 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T13:40:11.006 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T13:40:11.011 INFO:tasks.ceph.mon.a:Started 2024-04-25T13:40:11.011 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T13:40:11.011 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T13:40:11.051 INFO:tasks.ceph.mon.b:Started 2024-04-25T13:40:11.051 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T13:40:11.051 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T13:40:11.053 INFO:tasks.ceph.mon.c:Started 2024-04-25T13:40:11.054 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T13:40:11.054 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T13:40:11.054 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T13:40:11.056 INFO:tasks.ceph.mgr.y:Started 2024-04-25T13:40:11.057 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T13:40:11.057 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T13:40:11.059 INFO:tasks.ceph.mgr.x:Started 2024-04-25T13:40:11.059 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T13:40:11.059 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T13:40:11.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36fb639640 1 Processor -- start 2024-04-25T13:40:11.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36fb639640 1 -- start start 2024-04-25T13:40:11.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36fb639640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:11.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36fb639640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:11.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36fb639640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:11.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36fb639640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f36f40b4ea0 con 0x7f36f40b0dc0 2024-04-25T13:40:11.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36fb639640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f36f40b1370 con 0x7f36f40b09b0 2024-04-25T13:40:11.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36fb639640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36f40b1650 con 0x7f36f4164e80 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f8bd4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f8bd4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48142/0 (socket says 172.21.15.80:48142) 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f8bd4640 1 -- 172.21.15.80:0/3125666958 learned_addr learned my addr 172.21.15.80:0/3125666958 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f8bd4640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 msgr2=0x7f36f40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f8bd4640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 msgr2=0x7f36f40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f93d5640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f8bd4640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f8bd4640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f9bd6640 1 -- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 msgr2=0x7f36f41672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f9bd6640 1 -- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 msgr2=0x7f36f41672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:40:11.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f93d5640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 msgr2=0x7f36f40b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T13:40:11.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f93d5640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 msgr2=0x7f36f40b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T13:40:11.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f93d5640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:40:11.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.201+0000 7f36f93d5640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f8bd4640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f93d5640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f8bd4640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 msgr2=0x7f36f40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f8bd4640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 msgr2=0x7f36f40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f8bd4640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f8bd4640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f9bd6640 1 -- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 msgr2=0x7f36f41672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f9bd6640 1 -- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 msgr2=0x7f36f41672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T13:40:11.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:40:11.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:40:11.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f93d5640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 msgr2=0x7f36f40b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T13:40:11.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f93d5640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 msgr2=0x7f36f40b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T13:40:11.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f93d5640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T13:40:11.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.405+0000 7f36f93d5640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T13:40:11.675 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:11.672+0000 7f983647e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:40:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.809+0000 7f36f8bd4640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.809+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.809+0000 7f36f93d5640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.809+0000 7f36f9bd6640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 msgr2=0x7f36f40b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.809+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.809+0000 7f36f9bd6640 1 -- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 msgr2=0x7f36f40b4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.809+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.809+0000 7f36f9bd6640 1 -- 172.21.15.80:0/3125666958 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f40b1930 con 0x7f36f4164e80 2024-04-25T13:40:11.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.813+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f36f4073920 tx=0x7f36e401ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adc59867d81e0580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:11.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.813+0000 7f36f3fff640 1 -- 172.21.15.80:0/3125666958 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e40ab070 con 0x7f36f4164e80 2024-04-25T13:40:11.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.813+0000 7f36f3fff640 1 -- 172.21.15.80:0/3125666958 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e40a58a0 con 0x7f36f4164e80 2024-04-25T13:40:11.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.813+0000 7f36f3fff640 1 -- 172.21.15.80:0/3125666958 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e400f040 con 0x7f36f4164e80 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 -- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 msgr2=0x7f36f41672a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f36f4073920 tx=0x7f36e401ca30 comp rx=0 tx=0).stop 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 -- 172.21.15.80:0/3125666958 shutdown_connections 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 --2- 172.21.15.80:0/3125666958 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f4164e80 0x7f36f41672a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.80:3300/0 conn(0x7f36f40b0dc0 0x7f36f40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 --2- 172.21.15.80:0/3125666958 >> v2:172.21.15.87:3300/0 conn(0x7f36f40b09b0 0x7f36f40b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 -- 172.21.15.80:0/3125666958 >> 172.21.15.80:0/3125666958 conn(0x7f36f40b56d0 msgr2=0x7f36f40b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 -- 172.21.15.80:0/3125666958 shutdown_connections 2024-04-25T13:40:11.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.817+0000 7f36fb639640 1 -- 172.21.15.80:0/3125666958 wait complete. 2024-04-25T13:40:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.821+0000 7f36fb639640 1 Processor -- start 2024-04-25T13:40:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.821+0000 7f36fb639640 1 -- start start 2024-04-25T13:40:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36fb639640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40b09b0 0x7f36f414c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36fb639640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f40b0dc0 0x7f36f415f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36fb639640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f4164e80 0x7f36f415f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36fb639640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f36f40b69d0 con 0x7f36f4164e80 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36fb639640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f36f40b53c0 con 0x7f36f40b09b0 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36fb639640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36f40b6b50 con 0x7f36f40b0dc0 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f4164e80 0x7f36f415f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f4164e80 0x7f36f415f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48188/0 (socket says 172.21.15.80:48188) 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 -- 172.21.15.80:0/1159382534 learned_addr learned my addr 172.21.15.80:0/1159382534 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f8bd4640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f40b0dc0 0x7f36f415f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f93d5640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40b09b0 0x7f36f414c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 -- 172.21.15.80:0/1159382534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f40b0dc0 msgr2=0x7f36f415f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f40b0dc0 0x7f36f415f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 -- 172.21.15.80:0/1159382534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40b09b0 msgr2=0x7f36f414c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40b09b0 0x7f36f414c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 -- 172.21.15.80:0/1159382534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36f40b4930 con 0x7f36f4164e80 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.825+0000 7f36f9bd6640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f4164e80 0x7f36f415f940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f36e401b0e0 tx=0x7f36e401b270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.829+0000 7f36f1ffb640 1 -- 172.21.15.80:0/1159382534 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e40ab040 con 0x7f36f4164e80 2024-04-25T13:40:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.829+0000 7f36f1ffb640 1 -- 172.21.15.80:0/1159382534 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36e40be070 con 0x7f36f4164e80 2024-04-25T13:40:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.829+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36f40b3db0 con 0x7f36f4164e80 2024-04-25T13:40:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.829+0000 7f36f1ffb640 1 -- 172.21.15.80:0/1159382534 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e40af810 con 0x7f36f4164e80 2024-04-25T13:40:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.829+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36f40b4370 con 0x7f36f4164e80 2024-04-25T13:40:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.829+0000 7f36f1ffb640 1 -- 172.21.15.80:0/1159382534 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f36e40b6070 con 0x7f36f4164e80 2024-04-25T13:40:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.829+0000 7f36f1ffb640 1 -- 172.21.15.80:0/1159382534 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f36e40c3770 con 0x7f36f4164e80 2024-04-25T13:40:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.829+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36f406a3a0 con 0x7f36f4164e80 2024-04-25T13:40:11.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.837+0000 7f36f1ffb640 1 -- 172.21.15.80:0/1159382534 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f36e40c8020 con 0x7f36f4164e80 2024-04-25T13:40:11.843 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:11.841+0000 7fb653c72180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T13:40:11.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.905+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f36f4071d60 con 0x7f36f4164e80 2024-04-25T13:40:11.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36f1ffb640 1 -- 172.21.15.80:0/1159382534 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f36e4016660 con 0x7f36f4164e80 2024-04-25T13:40:11.912 INFO:teuthology.orchestra.run.smithi080.stderr:adjusted tunables profile to default 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f4164e80 msgr2=0x7f36f415f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f4164e80 0x7f36f415f940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f36e401b0e0 tx=0x7f36e401b270 comp rx=0 tx=0).stop 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 shutdown_connections 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36f4164e80 0x7f36f415f940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36f40b0dc0 0x7f36f415f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 --2- 172.21.15.80:0/1159382534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36f40b09b0 0x7f36f414c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 >> 172.21.15.80:0/1159382534 conn(0x7f36f40b56d0 msgr2=0x7f36f4163c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 shutdown_connections 2024-04-25T13:40:11.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:11.909+0000 7f36fb639640 1 -- 172.21.15.80:0/1159382534 wait complete. 2024-04-25T13:40:11.933 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T13:40:11.933 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T13:40:11.933 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:11.933 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T13:40:11.943 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:11.943 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T13:40:11.995 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:11.995 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T13:40:12.052 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:40:12.053 DEBUG:teuthology.orchestra.run.smithi080:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T13:40:12.105 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:12.105 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T13:40:12.118 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:12.118 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T13:40:12.136 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:12.132+0000 7f983647e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:40:12.146 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:12.147 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T13:40:12.204 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:40:12.205 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T13:40:12.256 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2024-04-25T13:40:12.339 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:12.336+0000 7f983647e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:40:12.421 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:12.417+0000 7fb653c72180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T13:40:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c7e15640 1 Processor -- start 2024-04-25T13:40:12.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c7e15640 1 -- start start 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c7e15640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c006d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c7e15640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14c006db40 0x7f14c00d1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c7e15640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14c006df50 con 0x7f14c00d1cc0 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c7e15640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f14c00b1f40 con 0x7f14c006db40 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c7e15640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f14c00701a0 con 0x7f14c006fd90 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c5bb1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c53b0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14c006db40 0x7f14c00d1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c63b2640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f14c00d1cc0 0x7f14c0143c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:46422/0 (socket says 172.21.15.87:46422) 2024-04-25T13:40:12.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c5bb1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c006d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:35076/0 (socket says 172.21.15.87:35076) 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c53b0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14c006db40 0x7f14c00d1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46590/0 (socket says 172.21.15.87:46590) 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c5bb1640 1 -- 172.21.15.87:0/2534016598 learned_addr learned my addr 172.21.15.87:0/2534016598 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c4baf640 1 -- 172.21.15.87:0/2534016598 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 902801698 0 0) 0x7f14bc001120 con 0x7f14c00d1cc0 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c4baf640 1 -- 172.21.15.87:0/2534016598 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f14ac003530 con 0x7f14c00d1cc0 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c53b0640 1 -- 172.21.15.87:0/2534016598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 msgr2=0x7f14c006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c53b0640 1 --2- 172.21.15.87:0/2534016598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c006d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c53b0640 1 -- 172.21.15.87:0/2534016598 >> v1:172.21.15.87:6789/0 conn(0x7f14c00d1cc0 legacy=0x7f14c0143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c53b0640 1 -- 172.21.15.87:0/2534016598 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c00d24d0 con 0x7f14c006db40 2024-04-25T13:40:12.446 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.440+0000 7f14c5bb1640 1 --2- 172.21.15.87:0/2534016598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:12.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c53b0640 1 --2- 172.21.15.87:0/2534016598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c006db40 0x7f14c00d1780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f14b00097c0 tx=0x7f14b001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e5b5042c0ab896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:12.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c4baf640 1 -- 172.21.15.87:0/2534016598 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b00b0070 con 0x7f14c006db40 2024-04-25T13:40:12.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c4baf640 1 -- 172.21.15.87:0/2534016598 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b00ab600 con 0x7f14c006db40 2024-04-25T13:40:12.447 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c4baf640 1 -- 172.21.15.87:0/2534016598 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b00ab9d0 con 0x7f14c006db40 2024-04-25T13:40:12.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c7e15640 1 -- 172.21.15.87:0/2534016598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c006db40 msgr2=0x7f14c00d1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c7e15640 1 --2- 172.21.15.87:0/2534016598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c006db40 0x7f14c00d1780 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f14b00097c0 tx=0x7f14b001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.448 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c7e15640 1 -- 172.21.15.87:0/2534016598 shutdown_connections 2024-04-25T13:40:12.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c7e15640 1 --2- 172.21.15.87:0/2534016598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c006db40 0x7f14c00d1780 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c7e15640 1 --2- 172.21.15.87:0/2534016598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c7e15640 1 -- 172.21.15.87:0/2534016598 >> 172.21.15.87:0/2534016598 conn(0x7f14c0072e30 msgr2=0x7f14c0073240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:12.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c7e15640 1 -- 172.21.15.87:0/2534016598 shutdown_connections 2024-04-25T13:40:12.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.444+0000 7f14c7e15640 1 -- 172.21.15.87:0/2534016598 wait complete. 2024-04-25T13:40:12.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 Processor -- start 2024-04-25T13:40:12.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 -- start start 2024-04-25T13:40:12.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14c006db40 0x7f14c015c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c0142560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c00d1cc0 0x7f14c006cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00d2250 con 0x7f14c006db40 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f14c00d20d0 con 0x7f14c00d1cc0 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f14c01447a0 con 0x7f14c006fd90 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c0142560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c63b2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c00d1cc0 0x7f14c006cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c0142560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:35088/0 (socket says 172.21.15.87:35088) 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 -- 172.21.15.87:0/3008361491 learned_addr learned my addr 172.21.15.87:0/3008361491 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c5bb1640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14c006db40 0x7f14c015c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 -- 172.21.15.87:0/3008361491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c00d1cc0 msgr2=0x7f14c006cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c00d1cc0 0x7f14c006cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 -- 172.21.15.87:0/3008361491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14c006db40 msgr2=0x7f14c015c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14c006db40 0x7f14c015c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 -- 172.21.15.87:0/3008361491 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c015cde0 con 0x7f14c006fd90 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c5bb1640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14c006db40 0x7f14c015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c53b0640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c0142560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f14b001a0d0 tx=0x7f14b0019240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14b67fc640 1 -- 172.21.15.87:0/3008361491 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b00b0040 con 0x7f14c006fd90 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14b67fc640 1 -- 172.21.15.87:0/3008361491 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14b00ba070 con 0x7f14c006fd90 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c01433e0 con 0x7f14c006fd90 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14b67fc640 1 -- 172.21.15.87:0/3008361491 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b0006e20 con 0x7f14c006fd90 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c013afd0 con 0x7f14c006fd90 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14b67fc640 1 -- 172.21.15.87:0/3008361491 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f14b00b4630 con 0x7f14c006fd90 2024-04-25T13:40:12.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14b67fc640 1 -- 172.21.15.87:0/3008361491 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f14b00bf7f0 con 0x7f14c006fd90 2024-04-25T13:40:12.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.448+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c006a3a0 con 0x7f14c006fd90 2024-04-25T13:40:12.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.452+0000 7f14b67fc640 1 -- 172.21.15.87:0/3008361491 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f14b0016090 con 0x7f14c006fd90 2024-04-25T13:40:12.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.520+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) -- 0x7f14c0160850 con 0x7f14c006fd90 2024-04-25T13:40:12.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14b67fc640 1 -- 172.21.15.87:0/3008361491 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f14b00b4ab0 con 0x7f14c006fd90 2024-04-25T13:40:12.531 INFO:teuthology.orchestra.run.smithi087.stdout:0 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 msgr2=0x7f14c0142560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c0142560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f14b001a0d0 tx=0x7f14b0019240 comp rx=0 tx=0).stop 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 shutdown_connections 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14c00d1cc0 0x7f14c006cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14c006fd90 0x7f14c0142560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 --2- 172.21.15.87:0/3008361491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14c006db40 0x7f14c015c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 >> 172.21.15.87:0/3008361491 conn(0x7f14c0072e30 msgr2=0x7f14c00757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 shutdown_connections 2024-04-25T13:40:12.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.528+0000 7f14c7e15640 1 -- 172.21.15.87:0/3008361491 wait complete. 2024-04-25T13:40:12.549 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2024-04-25T13:40:12.646 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:12.645+0000 7fb653c72180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T13:40:12.733 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f6511e7c640 1 Processor -- start 2024-04-25T13:40:12.734 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f6511e7c640 1 -- start start 2024-04-25T13:40:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f6511e7c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f650c0b0dc0 0x7f650c165e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f6511e7c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b4b30 0x7f650c1683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f6511e7c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f650c0b1370 con 0x7f650c0b09b0 2024-04-25T13:40:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f6511e7c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f650c0b1650 con 0x7f650c0b0dc0 2024-04-25T13:40:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f6511e7c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f650c0b1930 con 0x7f650c0b4b30 2024-04-25T13:40:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650b7fe640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f650c0b09b0 0x7f650c0b4420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:46436/0 (socket says 172.21.15.87:46436) 2024-04-25T13:40:12.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650bfff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b4b30 0x7f650c1683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650b7fe640 1 -- 172.21.15.87:0/274758310 learned_addr learned my addr 172.21.15.87:0/274758310 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T13:40:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650affd640 1 --2- 172.21.15.87:0/274758310 >> v2:172.21.15.80:3300/0 conn(0x7f650c0b0dc0 0x7f650c165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650a7fc640 1 -- 172.21.15.87:0/274758310 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1751205592 0 0) 0x7f650c0b1370 con 0x7f650c0b09b0 2024-04-25T13:40:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650a7fc640 1 -- 172.21.15.87:0/274758310 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f64f4003530 con 0x7f650c0b09b0 2024-04-25T13:40:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650bfff640 1 -- 172.21.15.87:0/274758310 >> v2:172.21.15.80:3300/0 conn(0x7f650c0b0dc0 msgr2=0x7f650c165e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650bfff640 1 --2- 172.21.15.87:0/274758310 >> v2:172.21.15.80:3300/0 conn(0x7f650c0b0dc0 0x7f650c165e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650bfff640 1 -- 172.21.15.87:0/274758310 >> v1:172.21.15.87:6789/0 conn(0x7f650c0b09b0 legacy=0x7f650c0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.732+0000 7f650bfff640 1 -- 172.21.15.87:0/274758310 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c0b1c10 con 0x7f650c0b4b30 2024-04-25T13:40:12.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f650bfff640 1 --2- 172.21.15.87:0/274758310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b4b30 0x7f650c1683e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6504009d90 tx=0x7f650401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88174caac2f9bfec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:12.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f650a7fc640 1 -- 172.21.15.87:0/274758310 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040b2070 con 0x7f650c0b4b30 2024-04-25T13:40:12.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f650a7fc640 1 -- 172.21.15.87:0/274758310 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65040a5940 con 0x7f650c0b4b30 2024-04-25T13:40:12.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f650a7fc640 1 -- 172.21.15.87:0/274758310 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65040a5d10 con 0x7f650c0b4b30 2024-04-25T13:40:12.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f6511e7c640 1 -- 172.21.15.87:0/274758310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b4b30 msgr2=0x7f650c1683e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f6511e7c640 1 --2- 172.21.15.87:0/274758310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b4b30 0x7f650c1683e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6504009d90 tx=0x7f650401c970 comp rx=0 tx=0).stop 2024-04-25T13:40:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f6511e7c640 1 -- 172.21.15.87:0/274758310 shutdown_connections 2024-04-25T13:40:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f6511e7c640 1 --2- 172.21.15.87:0/274758310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b4b30 0x7f650c1683e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f6511e7c640 1 --2- 172.21.15.87:0/274758310 >> v2:172.21.15.80:3300/0 conn(0x7f650c0b0dc0 0x7f650c165e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f6511e7c640 1 -- 172.21.15.87:0/274758310 >> 172.21.15.87:0/274758310 conn(0x7f650c0b56d0 msgr2=0x7f650c0b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f6511e7c640 1 -- 172.21.15.87:0/274758310 shutdown_connections 2024-04-25T13:40:12.741 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.736+0000 7f6511e7c640 1 -- 172.21.15.87:0/274758310 wait complete. 2024-04-25T13:40:12.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.740+0000 7f6511e7c640 1 Processor -- start 2024-04-25T13:40:12.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.740+0000 7f6511e7c640 1 -- start start 2024-04-25T13:40:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.740+0000 7f6511e7c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f650c0b09b0 0x7f650c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.740+0000 7f6511e7c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b0dc0 0x7f650c1574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.740+0000 7f6511e7c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f650c0b4b30 0x7f650c15e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.747 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.740+0000 7f6511e7c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f650c16b140 con 0x7f650c0b4b30 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.740+0000 7f6511e7c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f650c0b53c0 con 0x7f650c0b09b0 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.740+0000 7f6511e7c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f650c16b2c0 con 0x7f650c0b0dc0 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f650c0b09b0 0x7f650c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650affd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b0dc0 0x7f650c1574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f650c0b09b0 0x7f650c0c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46612/0 (socket says 172.21.15.87:46612) 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650affd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b0dc0 0x7f650c1574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:35108/0 (socket says 172.21.15.87:35108) 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 -- 172.21.15.87:0/1760251306 learned_addr learned my addr 172.21.15.87:0/1760251306 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650bfff640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f650c0b4b30 0x7f650c15e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 -- 172.21.15.87:0/1760251306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b0dc0 msgr2=0x7f650c1574a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b0dc0 0x7f650c1574a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 -- 172.21.15.87:0/1760251306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f650c0b4b30 msgr2=0x7f650c15e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f650c0b4b30 0x7f650c15e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 -- 172.21.15.87:0/1760251306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c159510 con 0x7f650c0b09b0 2024-04-25T13:40:12.748 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650affd640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b0dc0 0x7f650c1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650bfff640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f650c0b4b30 0x7f650c15e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f650b7fe640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f650c0b09b0 0x7f650c0c75d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c390 tx=0x7f64fc00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f64f3fff640 1 -- 172.21.15.87:0/1760251306 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc017070 con 0x7f650c0b09b0 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f64f3fff640 1 -- 172.21.15.87:0/1760251306 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64fc005d10 con 0x7f650c0b09b0 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f64f3fff640 1 -- 172.21.15.87:0/1760251306 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc007310 con 0x7f650c0b09b0 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c1597f0 con 0x7f650c0b09b0 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c159db0 con 0x7f650c0b09b0 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f64f3fff640 1 -- 172.21.15.87:0/1760251306 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f64fc01a070 con 0x7f650c0b09b0 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c06a3a0 con 0x7f650c0b09b0 2024-04-25T13:40:12.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.744+0000 7f64f3fff640 1 -- 172.21.15.87:0/1760251306 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f64fc0127b0 con 0x7f650c0b09b0 2024-04-25T13:40:12.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.748+0000 7f64f3fff640 1 -- 172.21.15.87:0/1760251306 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f64fc021070 con 0x7f650c0b09b0 2024-04-25T13:40:12.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.816+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) -- 0x7f650c1579e0 con 0x7f650c0b09b0 2024-04-25T13:40:12.825 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f64f3fff640 1 -- 172.21.15.87:0/1760251306 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f64fc007680 con 0x7f650c0b09b0 2024-04-25T13:40:12.826 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f650c0b09b0 msgr2=0x7f650c0c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f650c0b09b0 0x7f650c0c75d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c390 tx=0x7f64fc00c850 comp rx=0 tx=0).stop 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 shutdown_connections 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f650c0b4b30 0x7f650c15e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f650c0b0dc0 0x7f650c1574a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 --2- 172.21.15.87:0/1760251306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f650c0b09b0 0x7f650c0c75d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 >> 172.21.15.87:0/1760251306 conn(0x7f650c0b56d0 msgr2=0x7f650c15d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 shutdown_connections 2024-04-25T13:40:12.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.824+0000 7f6511e7c640 1 -- 172.21.15.87:0/1760251306 wait complete. 2024-04-25T13:40:12.847 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2024-04-25T13:40:12.912 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:12.908+0000 7f983647e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:40:12.968 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:12.964+0000 7f983647e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:40:12.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.984+0000 7fa94ca7c640 1 Processor -- start 2024-04-25T13:40:12.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.984+0000 7fa94ca7c640 1 -- start start 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa94ca7c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa9480b0dc0 0x7fa948165e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa94ca7c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa9480b4b30 0x7fa9481683e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa94ca7c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9480b1370 con 0x7fa9480b09b0 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa94ca7c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa9480b1650 con 0x7fa9480b4b30 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa94ca7c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa9480b1930 con 0x7fa9480b0dc0 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa945d9b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa9480b0dc0 0x7fa948165e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa94659c640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa9480b09b0 0x7fa9480b4420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:46444/0 (socket says 172.21.15.87:46444) 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa94659c640 1 -- 172.21.15.87:0/1175972240 learned_addr learned my addr 172.21.15.87:0/1175972240 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T13:40:12.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa946d9d640 1 --2- 172.21.15.87:0/1175972240 >> v2:172.21.15.80:3300/0 conn(0x7fa9480b4b30 0x7fa9481683e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:12.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa945d9b640 1 -- 172.21.15.87:0/1175972240 >> v2:172.21.15.80:3300/0 conn(0x7fa9480b4b30 msgr2=0x7fa9481683e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:12.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa945d9b640 1 --2- 172.21.15.87:0/1175972240 >> v2:172.21.15.80:3300/0 conn(0x7fa9480b4b30 0x7fa9481683e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa945d9b640 1 -- 172.21.15.87:0/1175972240 >> v1:172.21.15.87:6789/0 conn(0x7fa9480b09b0 legacy=0x7fa9480b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa945d9b640 1 -- 172.21.15.87:0/1175972240 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9480b1c10 con 0x7fa9480b0dc0 2024-04-25T13:40:12.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.988+0000 7fa946d9d640 1 --2- 172.21.15.87:0/1175972240 >> v2:172.21.15.80:3300/0 conn(0x7fa9480b4b30 0x7fa9481683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:12.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa945d9b640 1 --2- 172.21.15.87:0/1175972240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b0dc0 0x7fa948165e90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa948073d80 tx=0x7fa93c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a81615ade3918bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:12.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94559a640 1 -- 172.21.15.87:0/1175972240 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0b1070 con 0x7fa9480b0dc0 2024-04-25T13:40:12.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94559a640 1 -- 172.21.15.87:0/1175972240 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c0ac5f0 con 0x7fa9480b0dc0 2024-04-25T13:40:12.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94559a640 1 -- 172.21.15.87:0/1175972240 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0ac9c0 con 0x7fa9480b0dc0 2024-04-25T13:40:12.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94ca7c640 1 -- 172.21.15.87:0/1175972240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b0dc0 msgr2=0x7fa948165e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:12.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94ca7c640 1 --2- 172.21.15.87:0/1175972240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b0dc0 0x7fa948165e90 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa948073d80 tx=0x7fa93c01c480 comp rx=0 tx=0).stop 2024-04-25T13:40:12.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94ca7c640 1 -- 172.21.15.87:0/1175972240 shutdown_connections 2024-04-25T13:40:12.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94ca7c640 1 --2- 172.21.15.87:0/1175972240 >> v2:172.21.15.80:3300/0 conn(0x7fa9480b4b30 0x7fa9481683e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94ca7c640 1 --2- 172.21.15.87:0/1175972240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b0dc0 0x7fa948165e90 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:12.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94ca7c640 1 -- 172.21.15.87:0/1175972240 >> 172.21.15.87:0/1175972240 conn(0x7fa9480b56d0 msgr2=0x7fa9480b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:12.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94ca7c640 1 -- 172.21.15.87:0/1175972240 shutdown_connections 2024-04-25T13:40:12.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.992+0000 7fa94ca7c640 1 -- 172.21.15.87:0/1175972240 wait complete. 2024-04-25T13:40:12.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 Processor -- start 2024-04-25T13:40:12.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 -- start start 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9480b09b0 0x7fa9481446e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9480b0dc0 0x7fa948144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b4b30 0x7fa948160b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa94816b140 con 0x7fa9480b09b0 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa9480b53c0 con 0x7fa9480b4b30 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa94816b2c0 con 0x7fa9480b0dc0 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa946d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b4b30 0x7fa948160b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa945d9b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9480b0dc0 0x7fa948144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa946d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b4b30 0x7fa948160b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46624/0 (socket says 172.21.15.87:46624) 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa945d9b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9480b0dc0 0x7fa948144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:35112/0 (socket says 172.21.15.87:35112) 2024-04-25T13:40:13.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa946d9d640 1 -- 172.21.15.87:0/3215141748 learned_addr learned my addr 172.21.15.87:0/3215141748 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94659c640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9480b09b0 0x7fa9481446e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa945d9b640 1 -- 172.21.15.87:0/3215141748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b4b30 msgr2=0x7fa948160b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa945d9b640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b4b30 0x7fa948160b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa945d9b640 1 -- 172.21.15.87:0/3215141748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9480b09b0 msgr2=0x7fa9481446e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa945d9b640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9480b09b0 0x7fa9481446e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa945d9b640 1 -- 172.21.15.87:0/3215141748 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9480d0760 con 0x7fa9480b0dc0 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa946d9d640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b4b30 0x7fa948160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa945d9b640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9480b0dc0 0x7fa948144c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9480b1ab0 tx=0x7fa93c019220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa936ffd640 1 -- 172.21.15.87:0/3215141748 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0b1040 con 0x7fa9480b0dc0 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa936ffd640 1 -- 172.21.15.87:0/3215141748 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa93c0ba070 con 0x7fa9480b0dc0 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa936ffd640 1 -- 172.21.15.87:0/3215141748 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0b5690 con 0x7fa9480b0dc0 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9480d0a40 con 0x7fa9480b0dc0 2024-04-25T13:40:13.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9480d1000 con 0x7fa9480b0dc0 2024-04-25T13:40:13.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa936ffd640 1 -- 172.21.15.87:0/3215141748 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa93c0b5830 con 0x7fa9480b0dc0 2024-04-25T13:40:13.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa936ffd640 1 -- 172.21.15.87:0/3215141748 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fa93c0bfce0 con 0x7fa9480b0dc0 2024-04-25T13:40:13.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:12.996+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa94806a3a0 con 0x7fa9480b0dc0 2024-04-25T13:40:13.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.000+0000 7fa936ffd640 1 -- 172.21.15.87:0/3215141748 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa93c0a9070 con 0x7fa9480b0dc0 2024-04-25T13:40:13.036 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:13.032+0000 7f983647e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:40:13.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.076+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) -- 0x7fa9480b4830 con 0x7fa9480b0dc0 2024-04-25T13:40:13.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa936ffd640 1 -- 172.21.15.87:0/3215141748 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fa93c016070 con 0x7fa9480b0dc0 2024-04-25T13:40:13.088 INFO:teuthology.orchestra.run.smithi087.stdout:2 2024-04-25T13:40:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9480b0dc0 msgr2=0x7fa948144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9480b0dc0 0x7fa948144c20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fa9480b1ab0 tx=0x7fa93c019220 comp rx=0 tx=0).stop 2024-04-25T13:40:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 shutdown_connections 2024-04-25T13:40:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9480b4b30 0x7fa948160b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9480b0dc0 0x7fa948144c20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 --2- 172.21.15.87:0/3215141748 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa9480b09b0 0x7fa9481446e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 >> 172.21.15.87:0/3215141748 conn(0x7fa9480b56d0 msgr2=0x7fa9480d16f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 shutdown_connections 2024-04-25T13:40:13.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.084+0000 7fa94ca7c640 1 -- 172.21.15.87:0/3215141748 wait complete. 2024-04-25T13:40:13.107 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2024-04-25T13:40:13.122 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:13.120+0000 7f983647e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:40:13.180 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:13.176+0000 7f983647e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:40:13.225 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.221+0000 7fb653c72180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T13:40:13.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.240+0000 7fe8ecf00640 1 Processor -- start 2024-04-25T13:40:13.244 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.240+0000 7fe8ecf00640 1 -- start start 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8ecf00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8068ca0 0x7fe8e80b6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8ecf00640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe8e80b1680 0x7fe8e806f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8ecf00640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8e80b2110 con 0x7fe8e8160350 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8ecf00640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe8e80b1a90 con 0x7fe8e80b1680 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8ecf00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e80b08c0 con 0x7fe8e8068ca0 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8068ca0 0x7fe8e80b6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8e659c640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe8e8160350 0x7fe8e8068590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:46448/0 (socket says 172.21.15.87:46448) 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8e659c640 1 -- 172.21.15.87:0/2753865644 learned_addr learned my addr 172.21.15.87:0/2753865644 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8e6d9d640 1 --2- 172.21.15.87:0/2753865644 >> v2:172.21.15.80:3300/0 conn(0x7fe8e80b1680 0x7fe8e806f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8e5d9b640 1 -- 172.21.15.87:0/2753865644 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1962560340 0 0) 0x7fe8e80b2110 con 0x7fe8e8160350 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8dffff640 1 -- 172.21.15.87:0/2753865644 >> v2:172.21.15.80:3300/0 conn(0x7fe8e80b1680 msgr2=0x7fe8e806f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8dffff640 1 --2- 172.21.15.87:0/2753865644 >> v2:172.21.15.80:3300/0 conn(0x7fe8e80b1680 0x7fe8e806f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8dffff640 1 -- 172.21.15.87:0/2753865644 >> v1:172.21.15.87:6789/0 conn(0x7fe8e8160350 legacy=0x7fe8e8068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8dffff640 1 -- 172.21.15.87:0/2753865644 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e80b0ba0 con 0x7fe8e8068ca0 2024-04-25T13:40:13.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8dffff640 1 --2- 172.21.15.87:0/2753865644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8068ca0 0x7fe8e80b6450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00097c0 tx=0x7fe8d001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f4295491bbe4da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:13.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8e5d9b640 1 -- 172.21.15.87:0/2753865644 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00b0070 con 0x7fe8e8068ca0 2024-04-25T13:40:13.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8e5d9b640 1 -- 172.21.15.87:0/2753865644 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d00ab600 con 0x7fe8e8068ca0 2024-04-25T13:40:13.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.244+0000 7fe8e5d9b640 1 -- 172.21.15.87:0/2753865644 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00ab9d0 con 0x7fe8e8068ca0 2024-04-25T13:40:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- 172.21.15.87:0/2753865644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8068ca0 msgr2=0x7fe8e80b6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 --2- 172.21.15.87:0/2753865644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8068ca0 0x7fe8e80b6450 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe8d00097c0 tx=0x7fe8d001c410 comp rx=0 tx=0).stop 2024-04-25T13:40:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- 172.21.15.87:0/2753865644 shutdown_connections 2024-04-25T13:40:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 --2- 172.21.15.87:0/2753865644 >> v2:172.21.15.80:3300/0 conn(0x7fe8e80b1680 0x7fe8e806f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 --2- 172.21.15.87:0/2753865644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8068ca0 0x7fe8e80b6450 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- 172.21.15.87:0/2753865644 >> 172.21.15.87:0/2753865644 conn(0x7fe8e80737c0 msgr2=0x7fe8e8073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- 172.21.15.87:0/2753865644 shutdown_connections 2024-04-25T13:40:13.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- 172.21.15.87:0/2753865644 wait complete. 2024-04-25T13:40:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 Processor -- start 2024-04-25T13:40:13.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- start start 2024-04-25T13:40:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8e80b1680 0x7fe8e815b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8160350 0x7fe8e8151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8e8150aa0 0x7fe8e8158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e816d360 con 0x7fe8e8150aa0 2024-04-25T13:40:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe8e816d1e0 con 0x7fe8e80b1680 2024-04-25T13:40:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8ecf00640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe8e816d4e0 con 0x7fe8e8160350 2024-04-25T13:40:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8e80b1680 0x7fe8e815b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8dffff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8160350 0x7fe8e8151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e659c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8e80b1680 0x7fe8e815b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46634/0 (socket says 172.21.15.87:46634) 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e659c640 1 -- 172.21.15.87:0/3565680674 learned_addr learned my addr 172.21.15.87:0/3565680674 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e6d9d640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8e8150aa0 0x7fe8e8158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e659c640 1 -- 172.21.15.87:0/3565680674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8160350 msgr2=0x7fe8e8151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e659c640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8160350 0x7fe8e8151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e659c640 1 -- 172.21.15.87:0/3565680674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8e8150aa0 msgr2=0x7fe8e8158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e659c640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8e8150aa0 0x7fe8e8158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e659c640 1 -- 172.21.15.87:0/3565680674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8e815b9b0 con 0x7fe8e80b1680 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8dffff640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8160350 0x7fe8e8151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.248+0000 7fe8e6d9d640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8e8150aa0 0x7fe8e8158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8e659c640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8e80b1680 0x7fe8e815b470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800c400 tx=0x7fe8d800c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8deffd640 1 -- 172.21.15.87:0/3565680674 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d8017070 con 0x7fe8e80b1680 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8deffd640 1 -- 172.21.15.87:0/3565680674 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8d80074e0 con 0x7fe8e80b1680 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8deffd640 1 -- 172.21.15.87:0/3565680674 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d80059f0 con 0x7fe8e80b1680 2024-04-25T13:40:13.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8e815a6b0 con 0x7fe8e80b1680 2024-04-25T13:40:13.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8e815ac70 con 0x7fe8e80b1680 2024-04-25T13:40:13.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8deffd640 1 -- 172.21.15.87:0/3565680674 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe8d801a070 con 0x7fe8e80b1680 2024-04-25T13:40:13.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8deffd640 1 -- 172.21.15.87:0/3565680674 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fe8d8005b90 con 0x7fe8e80b1680 2024-04-25T13:40:13.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e815a440 con 0x7fe8e80b1680 2024-04-25T13:40:13.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.252+0000 7fe8deffd640 1 -- 172.21.15.87:0/3565680674 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe8d8021070 con 0x7fe8e80b1680 2024-04-25T13:40:13.266 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:13.264+0000 7f983647e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:40:13.279 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.277+0000 7fb653c72180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T13:40:13.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.316+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) -- 0x7fe8e806a3a0 con 0x7fe8e80b1680 2024-04-25T13:40:13.321 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:13.320+0000 7f983647e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:40:13.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8deffd640 1 -- 172.21.15.87:0/3565680674 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fe8d801c440 con 0x7fe8e80b1680 2024-04-25T13:40:13.327 INFO:teuthology.orchestra.run.smithi087.stdout:3 2024-04-25T13:40:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8e80b1680 msgr2=0x7fe8e815b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8e80b1680 0x7fe8e815b470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe8d800c400 tx=0x7fe8d800c8c0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 shutdown_connections 2024-04-25T13:40:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8e8150aa0 0x7fe8e8158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8e8160350 0x7fe8e8151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 --2- 172.21.15.87:0/3565680674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8e80b1680 0x7fe8e815b470 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 >> 172.21.15.87:0/3565680674 conn(0x7fe8e80737c0 msgr2=0x7fe8e8152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:13.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 shutdown_connections 2024-04-25T13:40:13.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.324+0000 7fe8ecf00640 1 -- 172.21.15.87:0/3565680674 wait complete. 2024-04-25T13:40:13.333 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.329+0000 7fb653c72180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T13:40:13.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2024-04-25T13:40:13.409 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.405+0000 7fb653c72180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T13:40:13.460 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.457+0000 7fb653c72180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T13:40:13.479 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:13.476+0000 7f983647e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:40:13.532 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.528+0000 7f140320b640 1 Processor -- start 2024-04-25T13:40:13.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.528+0000 7f140320b640 1 -- start start 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f140320b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0b7690 0x7f13fc0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f140320b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f13fc0d1d60 0x7f13fc160b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f140320b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13fc1614b0 con 0x7f13fc1610a0 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f140320b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f13fc070490 con 0x7f13fc0d1d60 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f140320b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc0b5fb0 con 0x7f13fc0b7690 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f1400fa7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0b7690 0x7f13fc0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f1400fa7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0b7690 0x7f13fc0b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:35144/0 (socket says 172.21.15.87:35144) 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f1400fa7640 1 -- 172.21.15.87:0/3325401611 learned_addr learned my addr 172.21.15.87:0/3325401611 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f13fbfff640 1 --2- 172.21.15.87:0/3325401611 >> v2:172.21.15.80:3300/0 conn(0x7f13fc0d1d60 0x7f13fc160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f1400fa7640 1 -- 172.21.15.87:0/3325401611 >> v2:172.21.15.80:3300/0 conn(0x7f13fc0d1d60 msgr2=0x7f13fc160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:13.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f1400fa7640 1 --2- 172.21.15.87:0/3325401611 >> v2:172.21.15.80:3300/0 conn(0x7f13fc0d1d60 0x7f13fc160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f1400fa7640 1 -- 172.21.15.87:0/3325401611 >> v1:172.21.15.80:6789/0 conn(0x7f13fc1610a0 legacy=0x7f13fc070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.532+0000 7f1400fa7640 1 -- 172.21.15.87:0/3325401611 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc0d2540 con 0x7f13fc0b7690 2024-04-25T13:40:13.537 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.537+0000 7fb653c72180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T13:40:13.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f1400fa7640 1 --2- 172.21.15.87:0/3325401611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0b7690 0x7f13fc0b5a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17729907eb6fae04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:13.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f13fb7fe640 1 -- 172.21.15.87:0/3325401611 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f40ab430 con 0x7f13fc0b7690 2024-04-25T13:40:13.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f13fb7fe640 1 -- 172.21.15.87:0/3325401611 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f40b0070 con 0x7f13fc0b7690 2024-04-25T13:40:13.539 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f13fb7fe640 1 -- 172.21.15.87:0/3325401611 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f40ab7f0 con 0x7f13fc0b7690 2024-04-25T13:40:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f140320b640 1 -- 172.21.15.87:0/3325401611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0b7690 msgr2=0x7f13fc0b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f140320b640 1 --2- 172.21.15.87:0/3325401611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0b7690 0x7f13fc0b5a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009870 tx=0x7f13f401c540 comp rx=0 tx=0).stop 2024-04-25T13:40:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f140320b640 1 -- 172.21.15.87:0/3325401611 shutdown_connections 2024-04-25T13:40:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f140320b640 1 --2- 172.21.15.87:0/3325401611 >> v2:172.21.15.80:3300/0 conn(0x7f13fc0d1d60 0x7f13fc160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f140320b640 1 --2- 172.21.15.87:0/3325401611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0b7690 0x7f13fc0b5a70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f140320b640 1 -- 172.21.15.87:0/3325401611 >> 172.21.15.87:0/3325401611 conn(0x7f13fc0737c0 msgr2=0x7f13fc073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f140320b640 1 -- 172.21.15.87:0/3325401611 shutdown_connections 2024-04-25T13:40:13.541 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.536+0000 7f140320b640 1 -- 172.21.15.87:0/3325401611 wait complete. 2024-04-25T13:40:13.543 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.540+0000 7f140320b640 1 Processor -- start 2024-04-25T13:40:13.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.540+0000 7f140320b640 1 -- start start 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13fc0b7690 0x7f13fc13c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0d1d60 0x7f13fc15c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13fc1610a0 0x7f13fc0b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc071440 con 0x7f13fc1610a0 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13fc0b7aa0 con 0x7f13fc0b7690 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13fc0d2170 con 0x7f13fc0d1d60 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13fbfff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0d1d60 0x7f13fc15c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f1400fa7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13fc0b7690 0x7f13fc13c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13fbfff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0d1d60 0x7f13fc15c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:35158/0 (socket says 172.21.15.87:35158) 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13fbfff640 1 -- 172.21.15.87:0/3570610270 learned_addr learned my addr 172.21.15.87:0/3570610270 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f14017a8640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13fc1610a0 0x7f13fc0b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f1400fa7640 1 -- 172.21.15.87:0/3570610270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0d1d60 msgr2=0x7f13fc15c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f1400fa7640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0d1d60 0x7f13fc15c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f1400fa7640 1 -- 172.21.15.87:0/3570610270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13fc1610a0 msgr2=0x7f13fc0b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f1400fa7640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13fc1610a0 0x7f13fc0b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f1400fa7640 1 -- 172.21.15.87:0/3570610270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13fc150d40 con 0x7f13fc0b7690 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13fbfff640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0d1d60 0x7f13fc15c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f14017a8640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13fc1610a0 0x7f13fc0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f1400fa7640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13fc0b7690 0x7f13fc13c4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0d0860 tx=0x7f13f40192e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13f97fa640 1 -- 172.21.15.87:0/3570610270 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f40039f0 con 0x7f13fc0b7690 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13f97fa640 1 -- 172.21.15.87:0/3570610270 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13f40b8070 con 0x7f13fc0b7690 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13fc151020 con 0x7f13fc0b7690 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13f97fa640 1 -- 172.21.15.87:0/3570610270 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4003db0 con 0x7f13fc0b7690 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13fc1515e0 con 0x7f13fc0b7690 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13f97fa640 1 -- 172.21.15.87:0/3570610270 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f13f40b48b0 con 0x7f13fc0b7690 2024-04-25T13:40:13.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f13f97fa640 1 -- 172.21.15.87:0/3570610270 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f13f40b4ba0 con 0x7f13fc0b7690 2024-04-25T13:40:13.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13fc06a3a0 con 0x7f13fc0b7690 2024-04-25T13:40:13.550 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:13.544+0000 7f983647e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:40:13.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.548+0000 7f13f97fa640 1 -- 172.21.15.87:0/3570610270 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f13f4016130 con 0x7f13fc0b7690 2024-04-25T13:40:13.590 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.589+0000 7fb653c72180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T13:40:13.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.616+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) -- 0x7f13fc167cd0 con 0x7f13fc0b7690 2024-04-25T13:40:13.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.620+0000 7f13f97fa640 1 -- 172.21.15.87:0/3570610270 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f13f40b4dc0 con 0x7f13fc0b7690 2024-04-25T13:40:13.626 INFO:teuthology.orchestra.run.smithi087.stdout:4 2024-04-25T13:40:13.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13fc0b7690 msgr2=0x7f13fc13c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13fc0b7690 0x7f13fc13c4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f13fc0d0860 tx=0x7f13f40192e0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 shutdown_connections 2024-04-25T13:40:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13fc1610a0 0x7f13fc0b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13fc0d1d60 0x7f13fc15c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 --2- 172.21.15.87:0/3570610270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13fc0b7690 0x7f13fc13c4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 >> 172.21.15.87:0/3570610270 conn(0x7f13fc0737c0 msgr2=0x7f13fc15d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 shutdown_connections 2024-04-25T13:40:13.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.624+0000 7f140320b640 1 -- 172.21.15.87:0/3570610270 wait complete. 2024-04-25T13:40:13.645 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2024-04-25T13:40:13.739 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.737+0000 7fb653c72180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T13:40:13.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feeda985640 1 Processor -- start 2024-04-25T13:40:13.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feeda985640 1 -- start start 2024-04-25T13:40:13.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feeda985640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feed40d0010 0x7feed40d0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feeda985640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed406ef80 con 0x7feed406eb70 2024-04-25T13:40:13.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feeda985640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feed40d0ad0 con 0x7feed406e760 2024-04-25T13:40:13.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feeda985640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feed40b6580 con 0x7feed40d0010 2024-04-25T13:40:13.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feed3fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feed40d0010 0x7feed40d0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feed3fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feed40d0010 0x7feed40d0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46648/0 (socket says 172.21.15.87:46648) 2024-04-25T13:40:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feed3fff640 1 -- 172.21.15.87:0/2886767997 learned_addr learned my addr 172.21.15.87:0/2886767997 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feed3fff640 1 -- 172.21.15.87:0/2886767997 >> v1:172.21.15.87:6789/0 conn(0x7feed406e760 legacy=0x7feed40b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feed3fff640 1 -- 172.21.15.87:0/2886767997 >> v1:172.21.15.80:6789/0 conn(0x7feed406eb70 legacy=0x7feed4163f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.780+0000 7feed3fff640 1 -- 172.21.15.87:0/2886767997 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed40d0db0 con 0x7feed40d0010 2024-04-25T13:40:13.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feed3fff640 1 --2- 172.21.15.87:0/2886767997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 0x7feed40d0420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec401c530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78afec8168e265f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:13.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feed2ffd640 1 -- 172.21.15.87:0/2886767997 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec40aa670 con 0x7feed40d0010 2024-04-25T13:40:13.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feed2ffd640 1 -- 172.21.15.87:0/2886767997 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feec40af070 con 0x7feed40d0010 2024-04-25T13:40:13.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feed2ffd640 1 -- 172.21.15.87:0/2886767997 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec40aaa30 con 0x7feed40d0010 2024-04-25T13:40:13.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 -- 172.21.15.87:0/2886767997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 msgr2=0x7feed40d0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 --2- 172.21.15.87:0/2886767997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 0x7feed40d0420 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feec4009870 tx=0x7feec401c530 comp rx=0 tx=0).stop 2024-04-25T13:40:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 -- 172.21.15.87:0/2886767997 shutdown_connections 2024-04-25T13:40:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 --2- 172.21.15.87:0/2886767997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 0x7feed40d0420 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 -- 172.21.15.87:0/2886767997 >> 172.21.15.87:0/2886767997 conn(0x7feed40b3cc0 msgr2=0x7feed40b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 -- 172.21.15.87:0/2886767997 shutdown_connections 2024-04-25T13:40:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 -- 172.21.15.87:0/2886767997 wait complete. 2024-04-25T13:40:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 Processor -- start 2024-04-25T13:40:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.784+0000 7feeda985640 1 -- start start 2024-04-25T13:40:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feed406e760 0x7feed409fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feed406eb70 0x7feed4157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 0x7feed409eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feed40b72a0 con 0x7feed406e760 2024-04-25T13:40:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feed40b7120 con 0x7feed40d0010 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feed4164670 con 0x7feed406eb70 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed37fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feed406eb70 0x7feed4157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed8f22640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 0x7feed409eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed37fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feed406eb70 0x7feed4157b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:35170/0 (socket says 172.21.15.87:35170) 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed37fe640 1 -- 172.21.15.87:0/2070786417 learned_addr learned my addr 172.21.15.87:0/2070786417 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed3fff640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feed406e760 0x7feed409fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed8f22640 1 -- 172.21.15.87:0/2070786417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feed406eb70 msgr2=0x7feed4157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed8f22640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feed406eb70 0x7feed4157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed8f22640 1 -- 172.21.15.87:0/2070786417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feed406e760 msgr2=0x7feed409fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed8f22640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feed406e760 0x7feed409fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed8f22640 1 -- 172.21.15.87:0/2070786417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed4159fa0 con 0x7feed40d0010 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed3fff640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feed406e760 0x7feed409fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed37fe640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feed406eb70 0x7feed4157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed8f22640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 0x7feed409eda0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feed406f150 tx=0x7feecc00c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed0ff9640 1 -- 172.21.15.87:0/2070786417 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc017070 con 0x7feed40d0010 2024-04-25T13:40:13.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed0ff9640 1 -- 172.21.15.87:0/2070786417 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feecc0052d0 con 0x7feed40d0010 2024-04-25T13:40:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed0ff9640 1 -- 172.21.15.87:0/2070786417 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc003840 con 0x7feed40d0010 2024-04-25T13:40:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed40b0d90 con 0x7feed40d0010 2024-04-25T13:40:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed40b1350 con 0x7feed40d0010 2024-04-25T13:40:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed0ff9640 1 -- 172.21.15.87:0/2070786417 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feecc01a070 con 0x7feed40d0010 2024-04-25T13:40:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feed0ff9640 1 -- 172.21.15.87:0/2070786417 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7feecc011a90 con 0x7feed40d0010 2024-04-25T13:40:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.788+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feed406a3a0 con 0x7feed40d0010 2024-04-25T13:40:13.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.792+0000 7feed0ff9640 1 -- 172.21.15.87:0/2070786417 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7feecc021020 con 0x7feed40d0010 2024-04-25T13:40:13.801 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:13.797+0000 7fb653c72180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T13:40:13.893 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.888+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) -- 0x7feed406d6e0 con 0x7feed40d0010 2024-04-25T13:40:13.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.896+0000 7feed0ff9640 1 -- 172.21.15.87:0/2070786417 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7feecc011d60 con 0x7feed40d0010 2024-04-25T13:40:13.900 INFO:teuthology.orchestra.run.smithi087.stdout:5 2024-04-25T13:40:13.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 msgr2=0x7feed409eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:13.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 0x7feed409eda0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7feed406f150 tx=0x7feecc00c820 comp rx=0 tx=0).stop 2024-04-25T13:40:13.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 shutdown_connections 2024-04-25T13:40:13.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feed40d0010 0x7feed409eda0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feed406eb70 0x7feed4157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 --2- 172.21.15.87:0/2070786417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feed406e760 0x7feed409fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:13.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 >> 172.21.15.87:0/2070786417 conn(0x7feed40b3cc0 msgr2=0x7feed4073f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:13.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 shutdown_connections 2024-04-25T13:40:13.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:13.908+0000 7feeda985640 1 -- 172.21.15.87:0/2070786417 wait complete. 2024-04-25T13:40:13.932 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2024-04-25T13:40:14.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.056+0000 7fe9a73c4640 1 Processor -- start 2024-04-25T13:40:14.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.056+0000 7fe9a73c4640 1 -- start start 2024-04-25T13:40:14.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a73c4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe9a00b09b0 0x7fe9a00b4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a73c4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9a00b0dc0 0x7fe9a00b4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a73c4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9a00b1370 con 0x7fe9a0164e80 2024-04-25T13:40:14.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a73c4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a00b4ea0 con 0x7fe9a00b09b0 2024-04-25T13:40:14.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a73c4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe9a00b1650 con 0x7fe9a00b0dc0 2024-04-25T13:40:14.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a495f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9a00b0dc0 0x7fe9a00b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a495f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9a00b0dc0 0x7fe9a00b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46660/0 (socket says 172.21.15.87:46660) 2024-04-25T13:40:14.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a495f640 1 -- 172.21.15.87:0/2439366780 learned_addr learned my addr 172.21.15.87:0/2439366780 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:14.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a5160640 1 --2- 172.21.15.87:0/2439366780 >> v2:172.21.15.80:3300/0 conn(0x7fe9a00b09b0 0x7fe9a00b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a495f640 1 -- 172.21.15.87:0/2439366780 >> v2:172.21.15.80:3300/0 conn(0x7fe9a00b09b0 msgr2=0x7fe9a00b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:14.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a495f640 1 --2- 172.21.15.87:0/2439366780 >> v2:172.21.15.80:3300/0 conn(0x7fe9a00b09b0 0x7fe9a00b4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a495f640 1 -- 172.21.15.87:0/2439366780 >> v1:172.21.15.80:6789/0 conn(0x7fe9a0164e80 legacy=0x7fe9a01672a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:14.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a495f640 1 -- 172.21.15.87:0/2439366780 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a00b1930 con 0x7fe9a00b0dc0 2024-04-25T13:40:14.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe9a495f640 1 --2- 172.21.15.87:0/2439366780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0dc0 0x7fe9a00b4960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0073d80 tx=0x7fe99001c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36f5750ffddbc24c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:14.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe997fff640 1 -- 172.21.15.87:0/2439366780 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9900b1070 con 0x7fe9a00b0dc0 2024-04-25T13:40:14.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe997fff640 1 -- 172.21.15.87:0/2439366780 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9900ac630 con 0x7fe9a00b0dc0 2024-04-25T13:40:14.065 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.060+0000 7fe997fff640 1 -- 172.21.15.87:0/2439366780 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9900a8050 con 0x7fe9a00b0dc0 2024-04-25T13:40:14.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2439366780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0dc0 msgr2=0x7fe9a00b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 --2- 172.21.15.87:0/2439366780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0dc0 0x7fe9a00b4960 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe9a0073d80 tx=0x7fe99001c4e0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2439366780 shutdown_connections 2024-04-25T13:40:14.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 --2- 172.21.15.87:0/2439366780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b0dc0 0x7fe9a00b4960 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 --2- 172.21.15.87:0/2439366780 >> v2:172.21.15.80:3300/0 conn(0x7fe9a00b09b0 0x7fe9a00b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2439366780 >> 172.21.15.87:0/2439366780 conn(0x7fe9a00b56d0 msgr2=0x7fe9a00b5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:14.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2439366780 shutdown_connections 2024-04-25T13:40:14.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2439366780 wait complete. 2024-04-25T13:40:14.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 Processor -- start 2024-04-25T13:40:14.067 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- start start 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b09b0 0x7fe9a0153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a00b0dc0 0x7fe9a0161ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9a0164e80 0x7fe9a0162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00b6b50 con 0x7fe9a00b0dc0 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe9a00b69d0 con 0x7fe9a00b09b0 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a73c4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe9a016b1e0 con 0x7fe9a0164e80 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5160640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b09b0 0x7fe9a0153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5160640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b09b0 0x7fe9a0153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46666/0 (socket says 172.21.15.87:46666) 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5160640 1 -- 172.21.15.87:0/2620206652 learned_addr learned my addr 172.21.15.87:0/2620206652 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:14.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5961640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9a0164e80 0x7fe9a0162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a495f640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a00b0dc0 0x7fe9a0161ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5160640 1 -- 172.21.15.87:0/2620206652 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9a0164e80 msgr2=0x7fe9a0162400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5160640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9a0164e80 0x7fe9a0162400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5160640 1 -- 172.21.15.87:0/2620206652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a00b0dc0 msgr2=0x7fe9a0161ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5160640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a00b0dc0 0x7fe9a0161ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.064+0000 7fe9a5160640 1 -- 172.21.15.87:0/2620206652 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9a0154260 con 0x7fe9a00b09b0 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe9a5160640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b09b0 0x7fe9a0153d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c820 tx=0x7fe99c00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe995ffb640 1 -- 172.21.15.87:0/2620206652 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c004030 con 0x7fe9a00b09b0 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe995ffb640 1 -- 172.21.15.87:0/2620206652 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c0041d0 con 0x7fe9a00b09b0 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9a01639a0 con 0x7fe9a00b09b0 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe995ffb640 1 -- 172.21.15.87:0/2620206652 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c007830 con 0x7fe9a00b09b0 2024-04-25T13:40:14.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9a0163c80 con 0x7fe9a00b09b0 2024-04-25T13:40:14.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe995ffb640 1 -- 172.21.15.87:0/2620206652 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe99c003620 con 0x7fe9a00b09b0 2024-04-25T13:40:14.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe995ffb640 1 -- 172.21.15.87:0/2620206652 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7fe99c019c60 con 0x7fe9a00b09b0 2024-04-25T13:40:14.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9a006a3a0 con 0x7fe9a00b09b0 2024-04-25T13:40:14.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.068+0000 7fe995ffb640 1 -- 172.21.15.87:0/2620206652 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe99c015020 con 0x7fe9a00b09b0 2024-04-25T13:40:14.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.136+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) -- 0x7fe9a0070f50 con 0x7fe9a00b09b0 2024-04-25T13:40:14.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe995ffb640 1 -- 172.21.15.87:0/2620206652 <== mon.1 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7fe99c007d60 con 0x7fe9a00b09b0 2024-04-25T13:40:14.143 INFO:teuthology.orchestra.run.smithi087.stdout:6 2024-04-25T13:40:14.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b09b0 msgr2=0x7fe9a0153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b09b0 0x7fe9a0153d20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c820 tx=0x7fe99c00cce0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 shutdown_connections 2024-04-25T13:40:14.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9a0164e80 0x7fe9a0162400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9a00b0dc0 0x7fe9a0161ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 --2- 172.21.15.87:0/2620206652 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9a00b09b0 0x7fe9a0153d20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 >> 172.21.15.87:0/2620206652 conn(0x7fe9a00b56d0 msgr2=0x7fe9a00b8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:14.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 shutdown_connections 2024-04-25T13:40:14.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.140+0000 7fe9a73c4640 1 -- 172.21.15.87:0/2620206652 wait complete. 2024-04-25T13:40:14.160 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2024-04-25T13:40:14.161 INFO:tasks.ceph.mgr.x.smithi087.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T13:40:14.161 INFO:tasks.ceph.mgr.x.smithi087.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T13:40:14.161 INFO:tasks.ceph.mgr.x.smithi087.stderr: from numpy import show_config as show_numpy_config 2024-04-25T13:40:14.209 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:14.208+0000 7f983647e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:40:14.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.296+0000 7f250f4dc640 1 Processor -- start 2024-04-25T13:40:14.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.296+0000 7f250f4dc640 1 -- start start 2024-04-25T13:40:14.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250f4dc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2508161270 0x7f2508070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250f4dc640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25080d2540 con 0x7f25080b5a70 2024-04-25T13:40:14.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250f4dc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f250806fe40 con 0x7f25080b7690 2024-04-25T13:40:14.299 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250f4dc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2508070490 con 0x7f2508161270 2024-04-25T13:40:14.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250da79640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2508161270 0x7f2508070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250d278640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f25080b7690 0x7f25080d1d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.87:46456/0 (socket says 172.21.15.87:46456) 2024-04-25T13:40:14.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250d278640 1 -- 172.21.15.87:0/2094915272 learned_addr learned my addr 172.21.15.87:0/2094915272 (peer_addr_for_me v1:172.21.15.87:0/0) 2024-04-25T13:40:14.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250da79640 1 -- 172.21.15.87:0/2094915272 >> v1:172.21.15.87:6789/0 conn(0x7f25080b7690 legacy=0x7f25080d1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250da79640 1 -- 172.21.15.87:0/2094915272 >> v1:172.21.15.80:6789/0 conn(0x7f25080b5a70 legacy=0x7f2508160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:14.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250da79640 1 -- 172.21.15.87:0/2094915272 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2508070120 con 0x7f2508161270 2024-04-25T13:40:14.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.300+0000 7f250da79640 1 --2- 172.21.15.87:0/2094915272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2508161270 0x7f2508070920 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2500009dc0 tx=0x7f250001c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db4263875a228d10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:14.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f24fffff640 1 -- 172.21.15.87:0/2094915272 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25000b2070 con 0x7f2508161270 2024-04-25T13:40:14.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f24fffff640 1 -- 172.21.15.87:0/2094915272 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25000a5850 con 0x7f2508161270 2024-04-25T13:40:14.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f24fffff640 1 -- 172.21.15.87:0/2094915272 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25000a5c20 con 0x7f2508161270 2024-04-25T13:40:14.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 -- 172.21.15.87:0/2094915272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2508161270 msgr2=0x7f2508070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 --2- 172.21.15.87:0/2094915272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2508161270 0x7f2508070920 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2500009dc0 tx=0x7f250001c880 comp rx=0 tx=0).stop 2024-04-25T13:40:14.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 -- 172.21.15.87:0/2094915272 shutdown_connections 2024-04-25T13:40:14.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 --2- 172.21.15.87:0/2094915272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2508161270 0x7f2508070920 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 -- 172.21.15.87:0/2094915272 >> 172.21.15.87:0/2094915272 conn(0x7f25080737c0 msgr2=0x7f2508073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:14.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 -- 172.21.15.87:0/2094915272 shutdown_connections 2024-04-25T13:40:14.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 -- 172.21.15.87:0/2094915272 wait complete. 2024-04-25T13:40:14.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 Processor -- start 2024-04-25T13:40:14.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.304+0000 7f250f4dc640 1 -- start start 2024-04-25T13:40:14.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25080b5a70 0x7f250814d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25080b7690 0x7f2508148f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2508150950 0x7f25081f9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f250816b090 con 0x7f2508150950 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f250816af10 con 0x7f25080b5a70 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f250816b210 con 0x7f25080b7690 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250d278640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25080b5a70 0x7f250814d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250ca77640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25080b7690 0x7f2508148f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250d278640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25080b5a70 0x7f250814d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46688/0 (socket says 172.21.15.87:46688) 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250ca77640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25080b7690 0x7f2508148f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.87:35188/0 (socket says 172.21.15.87:35188) 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250d278640 1 -- 172.21.15.87:0/1630363424 learned_addr learned my addr 172.21.15.87:0/1630363424 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250da79640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2508150950 0x7f25081f9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250ca77640 1 -- 172.21.15.87:0/1630363424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25080b5a70 msgr2=0x7f250814d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250ca77640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25080b5a70 0x7f250814d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250ca77640 1 -- 172.21.15.87:0/1630363424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2508150950 msgr2=0x7f25081f9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250ca77640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2508150950 0x7f25081f9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250ca77640 1 -- 172.21.15.87:0/1630363424 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25080687e0 con 0x7f25080b7690 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250d278640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25080b5a70 0x7f250814d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250ca77640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25080b7690 0x7f2508148f40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f24f800b440 tx=0x7f24f800b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f24fdffb640 1 -- 172.21.15.87:0/1630363424 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8011830 con 0x7f25080b7690 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f24fdffb640 1 -- 172.21.15.87:0/1630363424 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24f8016070 con 0x7f25080b7690 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2508068ac0 con 0x7f25080b7690 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f24fdffb640 1 -- 172.21.15.87:0/1630363424 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24f8011c10 con 0x7f25080b7690 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f25081f9f70 con 0x7f25080b7690 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f24fdffb640 1 -- 172.21.15.87:0/1630363424 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f24f8004020 con 0x7f25080b7690 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f24fdffb640 1 -- 172.21.15.87:0/1630363424 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f24f801bac0 con 0x7f25080b7690 2024-04-25T13:40:14.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.308+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f250806a3a0 con 0x7f25080b7690 2024-04-25T13:40:14.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.312+0000 7f24fdffb640 1 -- 172.21.15.87:0/1630363424 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f24f8020020 con 0x7f25080b7690 2024-04-25T13:40:14.349 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:14.348+0000 7f983647e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:40:14.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.372+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) -- 0x7f250814e0a0 con 0x7f25080b7690 2024-04-25T13:40:14.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f24fdffb640 1 -- 172.21.15.87:0/1630363424 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f24f8011db0 con 0x7f25080b7690 2024-04-25T13:40:14.379 INFO:teuthology.orchestra.run.smithi087.stdout:7 2024-04-25T13:40:14.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25080b7690 msgr2=0x7f2508148f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25080b7690 0x7f2508148f40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f24f800b440 tx=0x7f24f800b900 comp rx=0 tx=0).stop 2024-04-25T13:40:14.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 shutdown_connections 2024-04-25T13:40:14.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2508150950 0x7f25081f9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25080b7690 0x7f2508148f40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 --2- 172.21.15.87:0/1630363424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25080b5a70 0x7f250814d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 >> 172.21.15.87:0/1630363424 conn(0x7f25080737c0 msgr2=0x7f250814c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:14.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 shutdown_connections 2024-04-25T13:40:14.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:14.380+0000 7f250f4dc640 1 -- 172.21.15.87:0/1630363424 wait complete. 2024-04-25T13:40:14.397 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T13:40:14.397 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T13:40:14.400 INFO:tasks.ceph.osd.0:Started 2024-04-25T13:40:14.400 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T13:40:14.400 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T13:40:14.402 INFO:tasks.ceph.osd.1:Started 2024-04-25T13:40:14.402 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T13:40:14.403 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T13:40:14.405 INFO:tasks.ceph.osd.2:Started 2024-04-25T13:40:14.405 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T13:40:14.405 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T13:40:14.407 INFO:tasks.ceph.osd.3:Started 2024-04-25T13:40:14.407 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T13:40:14.408 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T13:40:14.409 INFO:tasks.ceph.osd.4:Started 2024-04-25T13:40:14.410 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T13:40:14.410 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T13:40:14.412 INFO:tasks.ceph.osd.5:Started 2024-04-25T13:40:14.412 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T13:40:14.412 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T13:40:14.414 INFO:tasks.ceph.osd.6:Started 2024-04-25T13:40:14.414 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T13:40:14.414 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T13:40:14.416 INFO:tasks.ceph.osd.7:Started 2024-04-25T13:40:14.416 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:14.420 INFO:tasks.ceph.mgr.y.smithi080.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-25T13:40:14.420 INFO:tasks.ceph.mgr.y.smithi080.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-25T13:40:14.420 INFO:tasks.ceph.mgr.y.smithi080.stderr: from numpy import show_config as show_numpy_config 2024-04-25T13:40:14.477 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:14.473+0000 7fb653c72180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T13:40:14.500 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-04-25T13:40:14.500+0000 7fcb566737c0 -1 Falling back to public interface 2024-04-25T13:40:14.502 INFO:tasks.ceph.osd.1.smithi080.stderr:2024-04-25T13:40:14.501+0000 7f08dd7af7c0 -1 Falling back to public interface 2024-04-25T13:40:14.507 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-04-25T13:40:14.508+0000 7f6fbb1d77c0 -1 Falling back to public interface 2024-04-25T13:40:14.508 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-04-25T13:40:14.508+0000 7fcb566737c0 -1 osd.5 0 log_to_monitors true 2024-04-25T13:40:14.512 INFO:tasks.ceph.osd.1.smithi080.stderr:2024-04-25T13:40:14.509+0000 7f08dd7af7c0 -1 osd.1 0 log_to_monitors true 2024-04-25T13:40:14.514 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-04-25T13:40:14.516+0000 7f7bb1dfd7c0 -1 Falling back to public interface 2024-04-25T13:40:14.519 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-04-25T13:40:14.520+0000 7f6fbb1d77c0 -1 osd.7 0 log_to_monitors true 2024-04-25T13:40:14.520 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-04-25T13:40:14.520+0000 7fb04553c7c0 -1 Falling back to public interface 2024-04-25T13:40:14.527 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-04-25T13:40:14.528+0000 7f7bb1dfd7c0 -1 osd.4 0 log_to_monitors true 2024-04-25T13:40:14.527 INFO:tasks.ceph.osd.0.smithi080.stderr:2024-04-25T13:40:14.525+0000 7f91f8ad77c0 -1 Falling back to public interface 2024-04-25T13:40:14.527 INFO:tasks.ceph.osd.2.smithi080.stderr:2024-04-25T13:40:14.525+0000 7f06169167c0 -1 Falling back to public interface 2024-04-25T13:40:14.532 INFO:tasks.ceph.osd.3.smithi080.stderr:2024-04-25T13:40:14.529+0000 7f079f7357c0 -1 Falling back to public interface 2024-04-25T13:40:14.537 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-04-25T13:40:14.536+0000 7fb04553c7c0 -1 osd.6 0 log_to_monitors true 2024-04-25T13:40:14.547 INFO:tasks.ceph.osd.0.smithi080.stderr:2024-04-25T13:40:14.545+0000 7f91f8ad77c0 -1 osd.0 0 log_to_monitors true 2024-04-25T13:40:14.547 INFO:tasks.ceph.osd.3.smithi080.stderr:2024-04-25T13:40:14.545+0000 7f079f7357c0 -1 osd.3 0 log_to_monitors true 2024-04-25T13:40:14.552 INFO:tasks.ceph.osd.2.smithi080.stderr:2024-04-25T13:40:14.549+0000 7f06169167c0 -1 osd.2 0 log_to_monitors true 2024-04-25T13:40:14.553 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:14.556+0000 7f983647e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:40:14.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe69dba4640 1 Processor -- start 2024-04-25T13:40:14.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe69dba4640 1 -- start start 2024-04-25T13:40:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe69dba4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe6981614b0 0x7fe69806dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe69dba4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe6980cfb60 0x7fe6980cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe69dba4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe698163eb0 0x7fe6981642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe69dba4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe6980d04b0 con 0x7fe6981614b0 2024-04-25T13:40:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe69dba4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe6980b2100 con 0x7fe6980cfb60 2024-04-25T13:40:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe69dba4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe6980b23e0 con 0x7fe698163eb0 2024-04-25T13:40:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe6977fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe6981614b0 0x7fe69806dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe6977fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe6981614b0 0x7fe69806dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48286/0 (socket says 172.21.15.80:48286) 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe6977fe640 1 -- 172.21.15.80:0/3292183904 learned_addr learned my addr 172.21.15.80:0/3292183904 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe696ffd640 1 --2- 172.21.15.80:0/3292183904 >> v2:172.21.15.87:3300/0 conn(0x7fe6980cfb60 0x7fe6980cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe697fff640 1 --2- 172.21.15.80:0/3292183904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe698163eb0 0x7fe6981642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.577+0000 7fe6977fe640 1 -- 172.21.15.80:0/3292183904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe698163eb0 msgr2=0x7fe6981642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe6977fe640 1 --2- 172.21.15.80:0/3292183904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe698163eb0 0x7fe6981642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe6977fe640 1 -- 172.21.15.80:0/3292183904 >> v2:172.21.15.87:3300/0 conn(0x7fe6980cfb60 msgr2=0x7fe6980cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe6977fe640 1 --2- 172.21.15.80:0/3292183904 >> v2:172.21.15.87:3300/0 conn(0x7fe6980cfb60 0x7fe6980cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe6977fe640 1 -- 172.21.15.80:0/3292183904 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6981609d0 con 0x7fe6981614b0 2024-04-25T13:40:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe696ffd640 1 --2- 172.21.15.80:0/3292183904 >> v2:172.21.15.87:3300/0 conn(0x7fe6980cfb60 0x7fe6980cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe6977fe640 1 --2- 172.21.15.80:0/3292183904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6981614b0 0x7fe69806dae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe6980d0720 tx=0x7fe68c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67466c94e42a1af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe6967fc640 1 -- 172.21.15.80:0/3292183904 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0b0070 con 0x7fe6981614b0 2024-04-25T13:40:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe6967fc640 1 -- 172.21.15.80:0/3292183904 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68c0ab600 con 0x7fe6981614b0 2024-04-25T13:40:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe6967fc640 1 -- 172.21.15.80:0/3292183904 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0ab9d0 con 0x7fe6981614b0 2024-04-25T13:40:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 -- 172.21.15.80:0/3292183904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6981614b0 msgr2=0x7fe69806dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 --2- 172.21.15.80:0/3292183904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6981614b0 0x7fe69806dae0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe6980d0720 tx=0x7fe68c01c520 comp rx=0 tx=0).stop 2024-04-25T13:40:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 -- 172.21.15.80:0/3292183904 shutdown_connections 2024-04-25T13:40:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 --2- 172.21.15.80:0/3292183904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe698163eb0 0x7fe6981642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 --2- 172.21.15.80:0/3292183904 >> v2:172.21.15.87:3300/0 conn(0x7fe6980cfb60 0x7fe6980cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 --2- 172.21.15.80:0/3292183904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6981614b0 0x7fe69806dae0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 -- 172.21.15.80:0/3292183904 >> 172.21.15.80:0/3292183904 conn(0x7fe6980d33a0 msgr2=0x7fe6980cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 -- 172.21.15.80:0/3292183904 shutdown_connections 2024-04-25T13:40:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.581+0000 7fe69dba4640 1 -- 172.21.15.80:0/3292183904 wait complete. 2024-04-25T13:40:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 Processor -- start 2024-04-25T13:40:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 -- start start 2024-04-25T13:40:14.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6980cfb60 0x7fe698150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6981614b0 0x7fe698154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe698163eb0 0x7fe6980b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:14.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe69806d230 con 0x7fe6980cfb60 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe6980b1d90 con 0x7fe698163eb0 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe69806d3b0 con 0x7fe6981614b0 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6980cfb60 0x7fe698150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6980cfb60 0x7fe698150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48288/0 (socket says 172.21.15.80:48288) 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 -- 172.21.15.80:0/2615603064 learned_addr learned my addr 172.21.15.80:0/2615603064 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe696ffd640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6981614b0 0x7fe698154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe697fff640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe698163eb0 0x7fe6980b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 -- 172.21.15.80:0/2615603064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6981614b0 msgr2=0x7fe698154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6981614b0 0x7fe698154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 -- 172.21.15.80:0/2615603064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe698163eb0 msgr2=0x7fe6980b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe698163eb0 0x7fe6980b25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 -- 172.21.15.80:0/2615603064 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe698151670 con 0x7fe6980cfb60 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe6977fe640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6980cfb60 0x7fe698150bb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe68c01abd0 tx=0x7fe68c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe67ffff640 1 -- 172.21.15.80:0/2615603064 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0ba070 con 0x7fe6980cfb60 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe67ffff640 1 -- 172.21.15.80:0/2615603064 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68c0b4620 con 0x7fe6980cfb60 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe698155bf0 con 0x7fe6980cfb60 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe67ffff640 1 -- 172.21.15.80:0/2615603064 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68c0b0040 con 0x7fe6980cfb60 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6980b0510 con 0x7fe6980cfb60 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe67ffff640 1 -- 172.21.15.80:0/2615603064 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe68c0bd070 con 0x7fe6980cfb60 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.585+0000 7fe67ffff640 1 -- 172.21.15.80:0/2615603064 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fe68c0b3a90 con 0x7fe6980cfb60 2024-04-25T13:40:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.589+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69806a3a0 con 0x7fe6980cfb60 2024-04-25T13:40:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.589+0000 7fe67ffff640 1 -- 172.21.15.80:0/2615603064 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe68c016110 con 0x7fe6980cfb60 2024-04-25T13:40:14.619 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:14.620+0000 7f983647e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:40:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.641+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6981606a0 con 0x7fe6980cfb60 2024-04-25T13:40:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe67ffff640 1 -- 172.21.15.80:0/2615603064 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fe68c0c5090 con 0x7fe6980cfb60 2024-04-25T13:40:14.646 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:14.646 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":10,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:14.377362+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6980cfb60 msgr2=0x7fe698150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6980cfb60 0x7fe698150bb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe68c01abd0 tx=0x7fe68c0192c0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 shutdown_connections 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe698163eb0 0x7fe6980b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6981614b0 0x7fe698154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 --2- 172.21.15.80:0/2615603064 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6980cfb60 0x7fe698150bb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 >> 172.21.15.80:0/2615603064 conn(0x7fe6980d33a0 msgr2=0x7fe698154280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 shutdown_connections 2024-04-25T13:40:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:14.645+0000 7fe69dba4640 1 -- 172.21.15.80:0/2615603064 wait complete. 2024-04-25T13:40:14.657 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:14.653+0000 7fb653c72180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:40:14.664 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T13:40:14.664 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T13:40:14.753 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:14.752+0000 7f983647e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:40:14.824 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:14.821+0000 7fb653c72180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T13:40:14.887 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:14.885+0000 7fb653c72180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T13:40:15.020 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:15.017+0000 7fb653c72180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T13:40:15.370 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:15.368+0000 7f983647e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:40:15.445 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:15.444+0000 7f983647e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:40:15.470 DEBUG:teuthology.orchestra.run.smithi080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:15.599 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62e627640 1 Processor -- start 2024-04-25T13:40:15.600 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62e627640 1 -- start start 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62e627640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff6280d0890 0x7ff6280b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62e627640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6280b77a0 0x7ff6280b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62e627640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6280d49c0 con 0x7ff6280d45b0 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62e627640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff6280b53f0 con 0x7ff6280d0890 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62e627640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff6280cf470 con 0x7ff6280b77a0 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff6277fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff6280d0890 0x7ff6280b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff627fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff6280d45b0 0x7ff6280d0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59896/0 (socket says 172.21.15.80:59896) 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff627fff640 1 -- 172.21.15.80:0/27722189 learned_addr learned my addr 172.21.15.80:0/27722189 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62cbc4640 1 --2- 172.21.15.80:0/27722189 >> v2:172.21.15.87:3300/0 conn(0x7ff6280b77a0 0x7ff6280b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff626ffd640 1 -- 172.21.15.80:0/27722189 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3129737963 0 0) 0x7ff6280d49c0 con 0x7ff6280d45b0 2024-04-25T13:40:15.602 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff626ffd640 1 -- 172.21.15.80:0/27722189 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff614003530 con 0x7ff6280d45b0 2024-04-25T13:40:15.603 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff6277fe640 1 -- 172.21.15.80:0/27722189 >> v2:172.21.15.87:3300/0 conn(0x7ff6280b77a0 msgr2=0x7ff6280b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:15.603 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff6277fe640 1 --2- 172.21.15.80:0/27722189 >> v2:172.21.15.87:3300/0 conn(0x7ff6280b77a0 0x7ff6280b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.603 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff6277fe640 1 -- 172.21.15.80:0/27722189 >> v1:172.21.15.80:6789/0 conn(0x7ff6280d45b0 legacy=0x7ff6280d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.603 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff6277fe640 1 -- 172.21.15.80:0/27722189 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6280cf750 con 0x7ff6280d0890 2024-04-25T13:40:15.603 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.597+0000 7ff62cbc4640 1 --2- 172.21.15.80:0/27722189 >> v2:172.21.15.87:3300/0 conn(0x7ff6280b77a0 0x7ff6280b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:15.604 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff6277fe640 1 --2- 172.21.15.80:0/27722189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6280d0890 0x7ff6280b7260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff6280cfd50 tx=0x7ff61001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790810079c8a7eb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:15.604 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff626ffd640 1 -- 172.21.15.80:0/27722189 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100ab5f0 con 0x7ff6280d0890 2024-04-25T13:40:15.604 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff626ffd640 1 -- 172.21.15.80:0/27722189 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6100b0070 con 0x7ff6280d0890 2024-04-25T13:40:15.604 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff626ffd640 1 -- 172.21.15.80:0/27722189 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6100ab9b0 con 0x7ff6280d0890 2024-04-25T13:40:15.605 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff62e627640 1 -- 172.21.15.80:0/27722189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6280d0890 msgr2=0x7ff6280b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.605 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff62e627640 1 --2- 172.21.15.80:0/27722189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6280d0890 0x7ff6280b7260 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff6280cfd50 tx=0x7ff61001c3f0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.605 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff62e627640 1 -- 172.21.15.80:0/27722189 shutdown_connections 2024-04-25T13:40:15.605 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff62e627640 1 --2- 172.21.15.80:0/27722189 >> v2:172.21.15.87:3300/0 conn(0x7ff6280b77a0 0x7ff6280b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.605 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff62e627640 1 --2- 172.21.15.80:0/27722189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6280d0890 0x7ff6280b7260 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.606 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff62e627640 1 -- 172.21.15.80:0/27722189 >> 172.21.15.80:0/27722189 conn(0x7ff6280d3a90 msgr2=0x7ff6280d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:15.606 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff62e627640 1 -- 172.21.15.80:0/27722189 shutdown_connections 2024-04-25T13:40:15.606 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.601+0000 7ff62e627640 1 -- 172.21.15.80:0/27722189 wait complete. 2024-04-25T13:40:15.607 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 Processor -- start 2024-04-25T13:40:15.607 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 -- start start 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6280b77a0 0x7ff628150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6280d45b0 0x7ff628156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff62814f320 0x7ff6281517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff6280cf210 con 0x7ff62814f320 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff6280cf090 con 0x7ff6280b77a0 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff6280b9300 con 0x7ff6280d45b0 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff62814f320 0x7ff6281517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff62814f320 0x7ff6281517f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48296/0 (socket says 172.21.15.80:48296) 2024-04-25T13:40:15.610 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 -- 172.21.15.80:0/1660584683 learned_addr learned my addr 172.21.15.80:0/1660584683 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff6277fe640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6280d45b0 0x7ff628156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff627fff640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6280b77a0 0x7ff628150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 -- 172.21.15.80:0/1660584683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6280d45b0 msgr2=0x7ff628156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6280d45b0 0x7ff628156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 -- 172.21.15.80:0/1660584683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6280b77a0 msgr2=0x7ff628150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6280b77a0 0x7ff628150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 -- 172.21.15.80:0/1660584683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff62814fc80 con 0x7ff62814f320 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62cbc4640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff62814f320 0x7ff6281517f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff620007e60 tx=0x7ff62000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff624ff9640 1 -- 172.21.15.80:0/1660584683 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62001a070 con 0x7ff62814f320 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff624ff9640 1 -- 172.21.15.80:0/1660584683 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff620015600 con 0x7ff62814f320 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff62814ff60 con 0x7ff62814f320 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff624ff9640 1 -- 172.21.15.80:0/1660584683 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6200159f0 con 0x7ff62814f320 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff628150520 con 0x7ff62814f320 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff624ff9640 1 -- 172.21.15.80:0/1660584683 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff62001d070 con 0x7ff62814f320 2024-04-25T13:40:15.611 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.605+0000 7ff624ff9640 1 -- 172.21.15.80:0/1660584683 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 1883+0+0 (secure 0 0 0) 0x7ff62001eb70 con 0x7ff62814f320 2024-04-25T13:40:15.612 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.609+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62806a3a0 con 0x7ff62814f320 2024-04-25T13:40:15.612 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.609+0000 7ff624ff9640 1 -- 172.21.15.80:0/1660584683 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ff620024070 con 0x7ff62814f320 2024-04-25T13:40:15.629 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:15.628+0000 7f983647e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:40:15.650 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:15.645+0000 7fb653c72180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T13:40:15.666 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff6281574c0 con 0x7ff62814f320 2024-04-25T13:40:15.668 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff624ff9640 1 -- 172.21.15.80:0/1660584683 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+7058 (secure 0 0 0) 0x7ff620010070 con 0x7ff62814f320 2024-04-25T13:40:15.668 INFO:teuthology.misc.health.smithi080.stdout: 2024-04-25T13:40:15.668 INFO:teuthology.misc.health.smithi080.stdout:{"epoch":11,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:15.463836+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:15.669 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff62814f320 msgr2=0x7ff6281517f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:15.669 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff62814f320 0x7ff6281517f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7ff620007e60 tx=0x7ff62000d4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.669 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 shutdown_connections 2024-04-25T13:40:15.669 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff62814f320 0x7ff6281517f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.669 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6280d45b0 0x7ff628156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.669 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 --2- 172.21.15.80:0/1660584683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6280b77a0 0x7ff628150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:15.670 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 >> 172.21.15.80:0/1660584683 conn(0x7ff6280d3a90 msgr2=0x7ff6280d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:15.670 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 shutdown_connections 2024-04-25T13:40:15.670 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:15.665+0000 7ff62e627640 1 -- 172.21.15.80:0/1660584683 wait complete. 2024-04-25T13:40:15.685 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T13:40:15.703 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:15.704+0000 7f983647e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:40:15.727 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:15.725+0000 7fb653c72180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T13:40:15.907 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:15.905+0000 7fb653c72180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T13:40:15.980 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:15.977+0000 7fb653c72180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T13:40:15.991 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:15.992+0000 7f983647e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:40:16.053 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:16.052+0000 7f983647e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:40:16.277 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:16.273+0000 7fb653c72180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T13:40:16.337 INFO:tasks.ceph.mgr.y.smithi080.stderr:2024-04-25T13:40:16.333+0000 7fb653c72180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T13:40:16.483 INFO:tasks.ceph.osd.1.smithi080.stderr:2024-04-25T13:40:16.481+0000 7f08d9756640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T13:40:16.483 INFO:tasks.ceph.osd.0.smithi080.stderr:2024-04-25T13:40:16.481+0000 7f91f4a7e640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T13:40:16.483 INFO:tasks.ceph.osd.3.smithi080.stderr:2024-04-25T13:40:16.481+0000 7f079b6dc640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T13:40:16.492 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-04-25T13:40:16.492+0000 7fcb52e2e640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T13:40:16.492 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-04-25T13:40:16.492+0000 7f6fb717e640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T13:40:16.493 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-04-25T13:40:16.492+0000 7fb0414e3640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T13:40:16.493 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-04-25T13:40:16.492+0000 7f7badda4640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T13:40:16.493 INFO:tasks.ceph.osd.2.smithi080.stderr:2024-04-25T13:40:16.489+0000 7f06128bd640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T13:40:16.494 INFO:tasks.ceph.osd.1.smithi080.stderr:2024-04-25T13:40:16.489+0000 7f08d2d6b640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:40:16.494 INFO:tasks.ceph.osd.0.smithi080.stderr:2024-04-25T13:40:16.489+0000 7f91ee093640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:40:16.494 INFO:tasks.ceph.osd.3.smithi080.stderr:2024-04-25T13:40:16.489+0000 7f0794cf1640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:40:16.496 INFO:tasks.ceph.osd.5.smithi087.stderr:2024-04-25T13:40:16.496+0000 7fcb4bc2f640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:40:16.496 INFO:tasks.ceph.osd.6.smithi087.stderr:2024-04-25T13:40:16.496+0000 7fb03aaf8640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:40:16.497 INFO:tasks.ceph.osd.7.smithi087.stderr:2024-04-25T13:40:16.496+0000 7f6fb0793640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:40:16.498 INFO:tasks.ceph.osd.4.smithi087.stderr:2024-04-25T13:40:16.500+0000 7f7ba73b9640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:40:16.499 INFO:tasks.ceph.osd.2.smithi080.stderr:2024-04-25T13:40:16.497+0000 7f060bed2640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T13:40:17.105 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:17.104+0000 7f97dce30640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T13:40:17.107 INFO:tasks.ceph.mgr.x.smithi087.stderr:2024-04-25T13:40:17.108+0000 7f97dce30640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T13:40:22.491 DEBUG:teuthology.orchestra.run.smithi080:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:22.642 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.637+0000 7f41e4911640 1 Processor -- start 2024-04-25T13:40:22.642 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.637+0000 7f41e4911640 1 -- start start 2024-04-25T13:40:22.644 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41e4911640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00b0ed0 0x7f41e015c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:22.644 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41e4911640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f41e006eba0 0x7f41e00ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:22.644 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41e4911640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41e015cda0 con 0x7f41e00ce730 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41e4911640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f41e0073710 con 0x7f41e006eba0 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41e4911640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f41e00739f0 con 0x7f41e00b0ed0 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41ded9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f41e006eba0 0x7f41e00ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41ded9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f41e006eba0 0x7f41e00ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42538/0 (socket says 172.21.15.80:42538) 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41ded9d640 1 -- 172.21.15.80:0/3398553150 learned_addr learned my addr 172.21.15.80:0/3398553150 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41de59c640 1 --2- 172.21.15.80:0/3398553150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00b0ed0 0x7f41e015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41ded9d640 1 -- 172.21.15.80:0/3398553150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00b0ed0 msgr2=0x7f41e015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41ded9d640 1 --2- 172.21.15.80:0/3398553150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00b0ed0 0x7f41e015c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41ded9d640 1 -- 172.21.15.80:0/3398553150 >> v1:172.21.15.87:6789/0 conn(0x7f41e00ce730 legacy=0x7f41e006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:22.645 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.641+0000 7f41ded9d640 1 -- 172.21.15.80:0/3398553150 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e0073cd0 con 0x7f41e006eba0 2024-04-25T13:40:22.647 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41ded9d640 1 --2- 172.21.15.80:0/3398553150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e006eba0 0x7f41e00ceb40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009e50 tx=0x7f41d801f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d33abd1a217190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:22.647 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41ddd9b640 1 -- 172.21.15.80:0/3398553150 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d80b4070 con 0x7f41e006eba0 2024-04-25T13:40:22.647 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41ddd9b640 1 -- 172.21.15.80:0/3398553150 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d80af400 con 0x7f41e006eba0 2024-04-25T13:40:22.647 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41ddd9b640 1 -- 172.21.15.80:0/3398553150 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d80af7d0 con 0x7f41e006eba0 2024-04-25T13:40:22.648 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 -- 172.21.15.80:0/3398553150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e006eba0 msgr2=0x7f41e00ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:22.648 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 --2- 172.21.15.80:0/3398553150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e006eba0 0x7f41e00ceb40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f41d8009e50 tx=0x7f41d801f230 comp rx=0 tx=0).stop 2024-04-25T13:40:22.648 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 -- 172.21.15.80:0/3398553150 shutdown_connections 2024-04-25T13:40:22.648 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 --2- 172.21.15.80:0/3398553150 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e006eba0 0x7f41e00ceb40 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.648 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 --2- 172.21.15.80:0/3398553150 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00b0ed0 0x7f41e015c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.648 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 -- 172.21.15.80:0/3398553150 >> 172.21.15.80:0/3398553150 conn(0x7f41e006d080 msgr2=0x7f41e00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:22.648 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 -- 172.21.15.80:0/3398553150 shutdown_connections 2024-04-25T13:40:22.648 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 -- 172.21.15.80:0/3398553150 wait complete. 2024-04-25T13:40:22.650 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 Processor -- start 2024-04-25T13:40:22.650 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.645+0000 7f41e4911640 1 -- start start 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e00b0ed0 0x7f41e00cfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41e00ce730 0x7f41e00afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00affc0 0x7f41e014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f41e00733a0 con 0x7f41e00b0ed0 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f41e00b12e0 con 0x7f41e00ce730 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f41e006dc00 con 0x7f41e00affc0 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e00b0ed0 0x7f41e00cfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e00b0ed0 0x7f41e00cfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42554/0 (socket says 172.21.15.80:42554) 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 -- 172.21.15.80:0/2988190104 learned_addr learned my addr 172.21.15.80:0/2988190104 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41ded9d640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00affc0 0x7f41e014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41d5d9b640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41e00ce730 0x7f41e00afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:22.653 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 -- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00affc0 msgr2=0x7f41e014c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00affc0 0x7f41e014c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 -- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41e00ce730 msgr2=0x7f41e00afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41e00ce730 0x7f41e00afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 -- 172.21.15.80:0/2988190104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e015e2b0 con 0x7f41e00b0ed0 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41d5d9b640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41e00ce730 0x7f41e00afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41de59c640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e00b0ed0 0x7f41e00cfbe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f41d000d5f0 tx=0x7f41d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41d77fe640 1 -- 172.21.15.80:0/2988190104 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d00125f0 con 0x7f41e00b0ed0 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41d77fe640 1 -- 172.21.15.80:0/2988190104 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41d0019070 con 0x7f41e00b0ed0 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41d77fe640 1 -- 172.21.15.80:0/2988190104 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d00129d0 con 0x7f41e00b0ed0 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e015e590 con 0x7f41e00b0ed0 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e015eb50 con 0x7f41e00b0ed0 2024-04-25T13:40:22.654 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e006a3a0 con 0x7f41e00b0ed0 2024-04-25T13:40:22.655 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.649+0000 7f41d77fe640 1 -- 172.21.15.80:0/2988190104 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f41d0012b70 con 0x7f41e00b0ed0 2024-04-25T13:40:22.655 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.653+0000 7f41d77fe640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f41c006f400 0x7f41c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:22.655 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.653+0000 7f41d77fe640 1 -- 172.21.15.80:0/2988190104 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f41d008f400 con 0x7f41e00b0ed0 2024-04-25T13:40:22.655 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.653+0000 7f41d5d9b640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f41c006f400 0x7f41c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:22.656 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.653+0000 7f41d5d9b640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f41c006f400 0x7f41c0071910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41e015ffa0 tx=0x7f41c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:22.659 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.657+0000 7f41d77fe640 1 -- 172.21.15.80:0/2988190104 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f41d005c6d0 con 0x7f41e00b0ed0 2024-04-25T13:40:22.820 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.817+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41e006f880 con 0x7f41e00b0ed0 2024-04-25T13:40:22.822 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.817+0000 7f41d77fe640 1 -- 172.21.15.80:0/2988190104 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12812 (secure 0 0 0) 0x7f41d005bfd0 con 0x7f41e00b0ed0 2024-04-25T13:40:22.822 INFO:teuthology.misc.health.smithi080.stdout: 2024-04-25T13:40:22.822 INFO:teuthology.misc.health.smithi080.stdout:{"epoch":16,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:20.533833+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f41c006f400 msgr2=0x7f41c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f41c006f400 0x7f41c0071910 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41e015ffa0 tx=0x7f41c8009040 comp rx=0 tx=0).stop 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e00b0ed0 msgr2=0x7f41e00cfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e00b0ed0 0x7f41e00cfbe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f41d000d5f0 tx=0x7f41d000dab0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 shutdown_connections 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f41c006f400 0x7f41c0071910 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f41e00affc0 0x7f41e014c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f41e00ce730 0x7f41e00afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 --2- 172.21.15.80:0/2988190104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f41e00b0ed0 0x7f41e00cfbe0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 >> 172.21.15.80:0/2988190104 conn(0x7f41e006d080 msgr2=0x7f41e00b2df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 shutdown_connections 2024-04-25T13:40:22.825 INFO:teuthology.misc.health.smithi080.stderr:2024-04-25T13:40:22.821+0000 7f41e4911640 1 -- 172.21.15.80:0/2988190104 wait complete. 2024-04-25T13:40:22.842 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T13:40:22.842 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T13:40:22.842 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:40:22.842 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:40:22.845 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:40:22.845 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:40:22.845 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:40:22.847 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:40:22.847 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:40:22.847 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:40:22.850 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:40:22.850 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:40:22.850 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:40:22.853 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:40:22.853 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T13:40:22.853 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:22.944 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:40:22.944 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:40:22.965 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:40:22.966 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:40:22.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.993+0000 7f02959bb640 1 Processor -- start 2024-04-25T13:40:22.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.993+0000 7f02959bb640 1 -- start start 2024-04-25T13:40:22.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f02900cf000 0x7f029006f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:22.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f029006c810 0x7f029006cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:22.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02900700e0 con 0x7f02900b5860 2024-04-25T13:40:22.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f02900cf410 con 0x7f029006c810 2024-04-25T13:40:22.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f02900703c0 con 0x7f02900cf000 2024-04-25T13:40:22.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f0287fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f02900b5860 0x7f02900b5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59418/0 (socket says 172.21.15.80:59418) 2024-04-25T13:40:22.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f0287fff640 1 -- 172.21.15.80:0/147721130 learned_addr learned my addr 172.21.15.80:0/147721130 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:22.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028f7fe640 1 --2- 172.21.15.80:0/147721130 >> v2:172.21.15.80:3300/0 conn(0x7f029006c810 0x7f029006cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028effd640 1 --2- 172.21.15.80:0/147721130 >> v2:172.21.15.87:3300/0 conn(0x7f02900cf000 0x7f029006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028e7fc640 1 -- 172.21.15.80:0/147721130 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962891723 0 0) 0x7f02900700e0 con 0x7f02900b5860 2024-04-25T13:40:23.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028e7fc640 1 -- 172.21.15.80:0/147721130 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0270003530 con 0x7f02900b5860 2024-04-25T13:40:23.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028f7fe640 1 -- 172.21.15.80:0/147721130 >> v2:172.21.15.87:3300/0 conn(0x7f02900cf000 msgr2=0x7f029006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:23.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028f7fe640 1 --2- 172.21.15.80:0/147721130 >> v2:172.21.15.87:3300/0 conn(0x7f02900cf000 0x7f029006f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028f7fe640 1 -- 172.21.15.80:0/147721130 >> v1:172.21.15.80:6789/0 conn(0x7f02900b5860 legacy=0x7f02900b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028f7fe640 1 -- 172.21.15.80:0/147721130 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02900706a0 con 0x7f029006c810 2024-04-25T13:40:23.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028f7fe640 1 --2- 172.21.15.80:0/147721130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f029006c810 0x7f029006cc20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02900cf590 tx=0x7f028801c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36fca1b7abc8a2e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028e7fc640 1 -- 172.21.15.80:0/147721130 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880b2070 con 0x7f029006c810 2024-04-25T13:40:23.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028e7fc640 1 -- 172.21.15.80:0/147721130 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02880a5930 con 0x7f029006c810 2024-04-25T13:40:23.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f028e7fc640 1 -- 172.21.15.80:0/147721130 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02880a5ce0 con 0x7f029006c810 2024-04-25T13:40:23.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 -- 172.21.15.80:0/147721130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f029006c810 msgr2=0x7f029006cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 --2- 172.21.15.80:0/147721130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f029006c810 0x7f029006cc20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f02900cf590 tx=0x7f028801c870 comp rx=0 tx=0).stop 2024-04-25T13:40:23.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 -- 172.21.15.80:0/147721130 shutdown_connections 2024-04-25T13:40:23.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 --2- 172.21.15.80:0/147721130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f029006c810 0x7f029006cc20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 --2- 172.21.15.80:0/147721130 >> v2:172.21.15.87:3300/0 conn(0x7f02900cf000 0x7f029006f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 -- 172.21.15.80:0/147721130 >> 172.21.15.80:0/147721130 conn(0x7f02900d28f0 msgr2=0x7f02900d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:23.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 -- 172.21.15.80:0/147721130 shutdown_connections 2024-04-25T13:40:23.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:22.997+0000 7f02959bb640 1 -- 172.21.15.80:0/147721130 wait complete. 2024-04-25T13:40:23.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f02959bb640 1 Processor -- start 2024-04-25T13:40:23.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f02959bb640 1 -- start start 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f02959bb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f029006c810 0x7f029015e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f02959bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02900b5860 0x7f0290145230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f02959bb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02900cf000 0x7f02901f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f02959bb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0290169d60 con 0x7f02900b5860 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f02959bb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0290169be0 con 0x7f02900cf000 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f02959bb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0290169ee0 con 0x7f029006c810 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02900b5860 0x7f0290145230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02900b5860 0x7f0290145230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42572/0 (socket says 172.21.15.80:42572) 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 -- 172.21.15.80:0/586389883 learned_addr learned my addr 172.21.15.80:0/586389883 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f028effd640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f029006c810 0x7f029015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f028f7fe640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02900cf000 0x7f02901f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 -- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f029006c810 msgr2=0x7f029015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f029006c810 0x7f029015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 -- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02900cf000 msgr2=0x7f02901f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02900cf000 0x7f02901f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 -- 172.21.15.80:0/586389883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0290153970 con 0x7f02900b5860 2024-04-25T13:40:23.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f0287fff640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02900b5860 0x7f0290145230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f028000c3c0 tx=0x7f028000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.001+0000 7f028cff9640 1 -- 172.21.15.80:0/586389883 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280017070 con 0x7f02900b5860 2024-04-25T13:40:23.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f028cff9640 1 -- 172.21.15.80:0/586389883 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0280005d10 con 0x7f02900b5860 2024-04-25T13:40:23.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f029015d690 con 0x7f02900b5860 2024-04-25T13:40:23.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f028cff9640 1 -- 172.21.15.80:0/586389883 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280007310 con 0x7f02900b5860 2024-04-25T13:40:23.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f029015dc50 con 0x7f02900b5860 2024-04-25T13:40:23.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02900d1dc0 con 0x7f02900b5860 2024-04-25T13:40:23.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f028cff9640 1 -- 172.21.15.80:0/586389883 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f028001a070 con 0x7f02900b5860 2024-04-25T13:40:23.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f028cff9640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f027406f430 0x7f0274071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f028cff9640 1 -- 172.21.15.80:0/586389883 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f028008e1d0 con 0x7f02900b5860 2024-04-25T13:40:23.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f028effd640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f027406f430 0x7f0274071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.005+0000 7f028effd640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f027406f430 0x7f0274071940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f02780079c0 tx=0x7f0278008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.009+0000 7f028cff9640 1 -- 172.21.15.80:0/586389883 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f028005af20 con 0x7f02900b5860 2024-04-25T13:40:23.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.165+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f029006a3a0 con 0x7f02900b5860 2024-04-25T13:40:23.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f028cff9640 1 -- 172.21.15.80:0/586389883 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12812 (secure 0 0 0) 0x7f028005f220 con 0x7f02900b5860 2024-04-25T13:40:23.171 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:23.171 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:20.533833+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:23.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f027406f430 msgr2=0x7f0274071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f027406f430 0x7f0274071940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f02780079c0 tx=0x7f0278008040 comp rx=0 tx=0).stop 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02900b5860 msgr2=0x7f0290145230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02900b5860 0x7f0290145230 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f028000c3c0 tx=0x7f028000c880 comp rx=0 tx=0).stop 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 shutdown_connections 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f027406f430 0x7f0274071940 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02900cf000 0x7f02901f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02900b5860 0x7f0290145230 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 --2- 172.21.15.80:0/586389883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f029006c810 0x7f029015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 >> 172.21.15.80:0/586389883 conn(0x7f02900d28f0 msgr2=0x7f02901608c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 shutdown_connections 2024-04-25T13:40:23.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.169+0000 7f02959bb640 1 -- 172.21.15.80:0/586389883 wait complete. 2024-04-25T13:40:23.189 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:40:23.190 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:40:23.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.341+0000 7f635dcdc640 1 Processor -- start 2024-04-25T13:40:23.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.341+0000 7f635dcdc640 1 -- start start 2024-04-25T13:40:23.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f635dcdc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6358160b10 0x7f6358160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f635dcdc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635806c9b0 0x7f635806cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f635dcdc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6358161460 con 0x7f635806f1d0 2024-04-25T13:40:23.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f635dcdc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f63580710f0 con 0x7f6358160b10 2024-04-25T13:40:23.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f635dcdc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f635806f6d0 con 0x7f635806c9b0 2024-04-25T13:40:23.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f634effd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6358160b10 0x7f6358160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f6357fff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635806c9b0 0x7f635806cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f634effd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6358160b10 0x7f6358160f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:36928/0 (socket says 172.21.15.80:36928) 2024-04-25T13:40:23.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f634effd640 1 -- 172.21.15.80:0/2451155242 learned_addr learned my addr 172.21.15.80:0/2451155242 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:23.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f634effd640 1 -- 172.21.15.80:0/2451155242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635806c9b0 msgr2=0x7f635806cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:23.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f634effd640 1 --2- 172.21.15.80:0/2451155242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635806c9b0 0x7f635806cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f634effd640 1 -- 172.21.15.80:0/2451155242 >> v1:172.21.15.87:6789/0 conn(0x7f635806f1d0 legacy=0x7f63580709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f634effd640 1 -- 172.21.15.80:0/2451155242 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f635806f9b0 con 0x7f6358160b10 2024-04-25T13:40:23.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.345+0000 7f6356ffd640 1 -- 172.21.15.80:0/2451155242 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2047039197 0 0) 0x7f6348001120 con 0x7f635806f1d0 2024-04-25T13:40:23.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f634effd640 1 --2- 172.21.15.80:0/2451155242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6358160b10 0x7f6358160f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6340009870 tx=0x7f634001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=479363039c46b987 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f6356ffd640 1 -- 172.21.15.80:0/2451155242 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400aa600 con 0x7f6358160b10 2024-04-25T13:40:23.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f6356ffd640 1 -- 172.21.15.80:0/2451155242 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63400af070 con 0x7f6358160b10 2024-04-25T13:40:23.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f6356ffd640 1 -- 172.21.15.80:0/2451155242 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400aa9c0 con 0x7f6358160b10 2024-04-25T13:40:23.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 -- 172.21.15.80:0/2451155242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6358160b10 msgr2=0x7f6358160f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 --2- 172.21.15.80:0/2451155242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6358160b10 0x7f6358160f20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6340009870 tx=0x7f634001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 -- 172.21.15.80:0/2451155242 shutdown_connections 2024-04-25T13:40:23.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 --2- 172.21.15.80:0/2451155242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f635806c9b0 0x7f635806cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 --2- 172.21.15.80:0/2451155242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6358160b10 0x7f6358160f20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 -- 172.21.15.80:0/2451155242 >> 172.21.15.80:0/2451155242 conn(0x7f63580cfa50 msgr2=0x7f63580cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:23.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 -- 172.21.15.80:0/2451155242 shutdown_connections 2024-04-25T13:40:23.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 -- 172.21.15.80:0/2451155242 wait complete. 2024-04-25T13:40:23.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 Processor -- start 2024-04-25T13:40:23.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.349+0000 7f635dcdc640 1 -- start start 2024-04-25T13:40:23.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f635806c9b0 0x7f6358161c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f635806f1d0 0x7f6358140240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6358160b10 0x7f635815f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f635816a240 con 0x7f635806c9b0 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f635816a0c0 con 0x7f635806f1d0 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f635816a3c0 con 0x7f6358160b10 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f635806c9b0 0x7f6358161c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f635806c9b0 0x7f6358161c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42576/0 (socket says 172.21.15.80:42576) 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 -- 172.21.15.80:0/1724386709 learned_addr learned my addr 172.21.15.80:0/1724386709 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f634effd640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f635806f1d0 0x7f6358140240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f6357fff640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6358160b10 0x7f635815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 -- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6358160b10 msgr2=0x7f635815f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6358160b10 0x7f635815f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 -- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f635806f1d0 msgr2=0x7f6358140240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f635806f1d0 0x7f6358140240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 -- 172.21.15.80:0/1724386709 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63581626a0 con 0x7f635806c9b0 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f63577fe640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f635806c9b0 0x7f6358161c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f634800c410 tx=0x7f634800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f6354ff9640 1 -- 172.21.15.80:0/1724386709 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348017070 con 0x7f635806c9b0 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f6354ff9640 1 -- 172.21.15.80:0/1724386709 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63480052d0 con 0x7f635806c9b0 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6358162980 con 0x7f635806c9b0 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f6354ff9640 1 -- 172.21.15.80:0/1724386709 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6348003840 con 0x7f635806c9b0 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6358162f40 con 0x7f635806c9b0 2024-04-25T13:40:23.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f635806a3a0 con 0x7f635806c9b0 2024-04-25T13:40:23.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.353+0000 7f6354ff9640 1 -- 172.21.15.80:0/1724386709 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f634801a070 con 0x7f635806c9b0 2024-04-25T13:40:23.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.357+0000 7f6354ff9640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f633806f430 0x7f6338071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.357+0000 7f6354ff9640 1 -- 172.21.15.80:0/1724386709 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f634808e140 con 0x7f635806c9b0 2024-04-25T13:40:23.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.357+0000 7f634effd640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f633806f430 0x7f6338071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.357+0000 7f634effd640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f633806f430 0x7f6338071940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f63400ae000 tx=0x7f63400a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.361+0000 7f6354ff9640 1 -- 172.21.15.80:0/1724386709 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6358161460 con 0x7f635806c9b0 2024-04-25T13:40:23.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f63580d1790 con 0x7f635806c9b0 2024-04-25T13:40:23.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f6354ff9640 1 -- 172.21.15.80:0/1724386709 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f6348005480 con 0x7f635806c9b0 2024-04-25T13:40:23.523 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:40:23.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f633806f430 msgr2=0x7f6338071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f633806f430 0x7f6338071940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f63400ae000 tx=0x7f63400a7000 comp rx=0 tx=0).stop 2024-04-25T13:40:23.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f635806c9b0 msgr2=0x7f6358161c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f635806c9b0 0x7f6358161c20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f634800c410 tx=0x7f634800c8d0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 shutdown_connections 2024-04-25T13:40:23.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f633806f430 0x7f6338071940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6358160b10 0x7f635815f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f635806f1d0 0x7f6358140240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 --2- 172.21.15.80:0/1724386709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f635806c9b0 0x7f6358161c20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 >> 172.21.15.80:0/1724386709 conn(0x7f63580cfa50 msgr2=0x7f635815b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:23.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 shutdown_connections 2024-04-25T13:40:23.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.521+0000 7f635dcdc640 1 -- 172.21.15.80:0/1724386709 wait complete. 2024-04-25T13:40:23.543 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:40:23.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc31df640 1 Processor -- start 2024-04-25T13:40:23.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc31df640 1 -- start start 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc31df640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fecbc06f5a0 0x7fecbc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc31df640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 0x7fecbc161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc31df640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fecbc0b4bd0 0x7fecbc0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc31df640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fecbc1619c0 con 0x7fecbc06f5a0 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc31df640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fecbc073f10 con 0x7fecbc0b4bd0 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc31df640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fecbc06fcf0 con 0x7fecbc161070 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc0f7b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fecbc06f5a0 0x7fecbc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc0f7b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fecbc06f5a0 0x7fecbc160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42586/0 (socket says 172.21.15.80:42586) 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc0f7b640 1 -- 172.21.15.80:0/2631134556 learned_addr learned my addr 172.21.15.80:0/2631134556 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecbbfff640 1 --2- 172.21.15.80:0/2631134556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 0x7fecbc161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc0f7b640 1 -- 172.21.15.80:0/2631134556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 msgr2=0x7fecbc161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:23.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc0f7b640 1 --2- 172.21.15.80:0/2631134556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 0x7fecbc161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc0f7b640 1 -- 172.21.15.80:0/2631134556 >> v2:172.21.15.87:3300/0 conn(0x7fecbc0b4bd0 msgr2=0x7fecbc0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:23.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc177c640 1 --2- 172.21.15.80:0/2631134556 >> v2:172.21.15.87:3300/0 conn(0x7fecbc0b4bd0 0x7fecbc0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc0f7b640 1 --2- 172.21.15.80:0/2631134556 >> v2:172.21.15.87:3300/0 conn(0x7fecbc0b4bd0 0x7fecbc0b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.681+0000 7fecc0f7b640 1 -- 172.21.15.80:0/2631134556 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc0b0c60 con 0x7fecbc06f5a0 2024-04-25T13:40:23.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc0f7b640 1 --2- 172.21.15.80:0/2631134556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc160b30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0ce160 tx=0x7fecb401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901ef2f87f49c4b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecbb7fe640 1 -- 172.21.15.80:0/2631134556 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40b0070 con 0x7fecbc06f5a0 2024-04-25T13:40:23.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecbb7fe640 1 -- 172.21.15.80:0/2631134556 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb40ab600 con 0x7fecbc06f5a0 2024-04-25T13:40:23.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecbb7fe640 1 -- 172.21.15.80:0/2631134556 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40ab9d0 con 0x7fecbc06f5a0 2024-04-25T13:40:23.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 -- 172.21.15.80:0/2631134556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 msgr2=0x7fecbc160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 --2- 172.21.15.80:0/2631134556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc160b30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0ce160 tx=0x7fecb401c490 comp rx=0 tx=0).stop 2024-04-25T13:40:23.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 -- 172.21.15.80:0/2631134556 shutdown_connections 2024-04-25T13:40:23.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 --2- 172.21.15.80:0/2631134556 >> v2:172.21.15.87:3300/0 conn(0x7fecbc0b4bd0 0x7fecbc0b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 --2- 172.21.15.80:0/2631134556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 0x7fecbc161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 --2- 172.21.15.80:0/2631134556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc160b30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 -- 172.21.15.80:0/2631134556 >> 172.21.15.80:0/2631134556 conn(0x7fecbc06d080 msgr2=0x7fecbc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:23.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 -- 172.21.15.80:0/2631134556 shutdown_connections 2024-04-25T13:40:23.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.685+0000 7fecc31df640 1 -- 172.21.15.80:0/2631134556 wait complete. 2024-04-25T13:40:23.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 Processor -- start 2024-04-25T13:40:23.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 -- start start 2024-04-25T13:40:23.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecbc0b4bd0 0x7fecbc14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 0x7fecbc150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc072bb0 con 0x7fecbc06f5a0 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc0b5930 con 0x7fecbc0b4bd0 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fecbc072d30 con 0x7fecbc161070 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42598/0 (socket says 172.21.15.80:42598) 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 -- 172.21.15.80:0/1268702993 learned_addr learned my addr 172.21.15.80:0/1268702993 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecbbfff640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecbc0b4bd0 0x7fecbc14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc177c640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 0x7fecbc150f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 -- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 msgr2=0x7fecbc150f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 0x7fecbc150f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 -- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecbc0b4bd0 msgr2=0x7fecbc14abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecbc0b4bd0 0x7fecbc14abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 -- 172.21.15.80:0/1268702993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc0d1000 con 0x7fecbc06f5a0 2024-04-25T13:40:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc0f7b640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc158790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fecb401cd30 tx=0x7fecb4019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecb97fa640 1 -- 172.21.15.80:0/1268702993 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40b9070 con 0x7fecbc06f5a0 2024-04-25T13:40:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecb97fa640 1 -- 172.21.15.80:0/1268702993 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb40b4640 con 0x7fecbc06f5a0 2024-04-25T13:40:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc14b130 con 0x7fecbc06f5a0 2024-04-25T13:40:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecb97fa640 1 -- 172.21.15.80:0/1268702993 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40b0040 con 0x7fecbc06f5a0 2024-04-25T13:40:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc15d6e0 con 0x7fecbc06f5a0 2024-04-25T13:40:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.689+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecbc159ba0 con 0x7fecbc06f5a0 2024-04-25T13:40:23.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.693+0000 7fecb97fa640 1 -- 172.21.15.80:0/1268702993 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fecb40bc070 con 0x7fecbc06f5a0 2024-04-25T13:40:23.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.693+0000 7fecb97fa640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feca806f430 0x7feca8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:23.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.693+0000 7fecb97fa640 1 -- 172.21.15.80:0/1268702993 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fecb4128020 con 0x7fecbc06f5a0 2024-04-25T13:40:23.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.693+0000 7fecbbfff640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feca806f430 0x7feca8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:23.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.693+0000 7fecbbfff640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feca806f430 0x7feca8071940 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0b05a0 tx=0x7fecb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:23.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.697+0000 7fecb97fa640 1 -- 172.21.15.80:0/1268702993 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fecbc159ba0 con 0x7fecbc06f5a0 2024-04-25T13:40:23.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.877+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecbc06a3a0 con 0x7fecbc06f5a0 2024-04-25T13:40:23.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.877+0000 7fecb97fa640 1 -- 172.21.15.80:0/1268702993 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fecbc14b130 con 0x7fecbc06f5a0 2024-04-25T13:40:23.881 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:23.881 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T13:40:23.882 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 2 2024-04-25T13:40:23.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feca806f430 msgr2=0x7feca8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feca806f430 0x7feca8071940 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0b05a0 tx=0x7fecb0008040 comp rx=0 tx=0).stop 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 msgr2=0x7fecbc158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc158790 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fecb401cd30 tx=0x7fecb4019230 comp rx=0 tx=0).stop 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 shutdown_connections 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feca806f430 0x7feca8071940 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fecbc0b05a0 tx=0x7fecb0008040 comp rx=0 tx=0).stop 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecbc161070 0x7fecbc150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecbc0b4bd0 0x7fecbc14abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 --2- 172.21.15.80:0/1268702993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecbc06f5a0 0x7fecbc158790 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 >> 172.21.15.80:0/1268702993 conn(0x7fecbc06d080 msgr2=0x7fecbc0cf6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 shutdown_connections 2024-04-25T13:40:23.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:23.881+0000 7fecc31df640 1 -- 172.21.15.80:0/1268702993 wait complete. 2024-04-25T13:40:25.902 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:40:26.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb55b889640 1 Processor -- start 2024-04-25T13:40:26.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb55b889640 1 -- start start 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb55b889640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb55413fe90 0x7fb554160a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb55b889640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb554160fb0 0x7fb55415d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb55b889640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5540d12f0 con 0x7fb5540d25d0 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb55b889640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb5540d2140 con 0x7fb55413fe90 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb55b889640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb5541613c0 con 0x7fb554160fb0 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb558e24640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb55413fe90 0x7fb554160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb558e24640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb55413fe90 0x7fb554160a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42610/0 (socket says 172.21.15.80:42610) 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb558e24640 1 -- 172.21.15.80:0/1003270398 learned_addr learned my addr 172.21.15.80:0/1003270398 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:26.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb559e26640 1 --2- 172.21.15.80:0/1003270398 >> v2:172.21.15.87:3300/0 conn(0x7fb554160fb0 0x7fb55415d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb558e24640 1 -- 172.21.15.80:0/1003270398 >> v2:172.21.15.87:3300/0 conn(0x7fb554160fb0 msgr2=0x7fb55415d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:26.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb558e24640 1 --2- 172.21.15.80:0/1003270398 >> v2:172.21.15.87:3300/0 conn(0x7fb554160fb0 0x7fb55415d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb558e24640 1 -- 172.21.15.80:0/1003270398 >> v1:172.21.15.87:6789/0 conn(0x7fb5540d25d0 legacy=0x7fb5540cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:26.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb558e24640 1 -- 172.21.15.80:0/1003270398 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5540d1570 con 0x7fb55413fe90 2024-04-25T13:40:26.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.060+0000 7fb559e26640 1 --2- 172.21.15.80:0/1003270398 >> v2:172.21.15.87:3300/0 conn(0x7fb554160fb0 0x7fb55415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb558e24640 1 --2- 172.21.15.80:0/1003270398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb55413fe90 0x7fb554160a70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb5540d1740 tx=0x7fb54401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39bb64f614d5cfa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb54bfff640 1 -- 172.21.15.80:0/1003270398 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440b0070 con 0x7fb55413fe90 2024-04-25T13:40:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb54bfff640 1 -- 172.21.15.80:0/1003270398 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5440ab600 con 0x7fb55413fe90 2024-04-25T13:40:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb54bfff640 1 -- 172.21.15.80:0/1003270398 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5440ab9d0 con 0x7fb55413fe90 2024-04-25T13:40:26.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb55b889640 1 -- 172.21.15.80:0/1003270398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb55413fe90 msgr2=0x7fb554160a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb55b889640 1 --2- 172.21.15.80:0/1003270398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb55413fe90 0x7fb554160a70 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb5540d1740 tx=0x7fb54401c300 comp rx=0 tx=0).stop 2024-04-25T13:40:26.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb55b889640 1 -- 172.21.15.80:0/1003270398 shutdown_connections 2024-04-25T13:40:26.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb55b889640 1 --2- 172.21.15.80:0/1003270398 >> v2:172.21.15.87:3300/0 conn(0x7fb554160fb0 0x7fb55415d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb55b889640 1 --2- 172.21.15.80:0/1003270398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb55413fe90 0x7fb554160a70 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb55b889640 1 -- 172.21.15.80:0/1003270398 >> 172.21.15.80:0/1003270398 conn(0x7fb55406c7c0 msgr2=0x7fb55406cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb55b889640 1 -- 172.21.15.80:0/1003270398 shutdown_connections 2024-04-25T13:40:26.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.064+0000 7fb55b889640 1 -- 172.21.15.80:0/1003270398 wait complete. 2024-04-25T13:40:26.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 Processor -- start 2024-04-25T13:40:26.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 -- start start 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5540d25d0 0x7fb55415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb55413fe90 0x7fb55413d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb554160fb0 0x7fb554068590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb554169f90 con 0x7fb554160fb0 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb554169e10 con 0x7fb55413fe90 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb55416a110 con 0x7fb5540d25d0 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb554160fb0 0x7fb554068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb554160fb0 0x7fb554068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42622/0 (socket says 172.21.15.80:42622) 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 -- 172.21.15.80:0/4285194941 learned_addr learned my addr 172.21.15.80:0/4285194941 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb558e24640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb55413fe90 0x7fb55413d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559625640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5540d25d0 0x7fb55415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 -- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5540d25d0 msgr2=0x7fb55415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5540d25d0 0x7fb55415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 -- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb55413fe90 msgr2=0x7fb55413d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb55413fe90 0x7fb55413d080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 -- 172.21.15.80:0/4285194941 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55415a020 con 0x7fb554160fb0 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559625640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5540d25d0 0x7fb55415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb559e26640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb554160fb0 0x7fb554068590 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00cdb0 tx=0x7fb54c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb549ffb640 1 -- 172.21.15.80:0/4285194941 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00f680 con 0x7fb554160fb0 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb549ffb640 1 -- 172.21.15.80:0/4285194941 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb54c00f840 con 0x7fb554160fb0 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb549ffb640 1 -- 172.21.15.80:0/4285194941 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54c00fc30 con 0x7fb554160fb0 2024-04-25T13:40:26.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55415a550 con 0x7fb554160fb0 2024-04-25T13:40:26.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.068+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb55415ab10 con 0x7fb554160fb0 2024-04-25T13:40:26.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.072+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb554165170 con 0x7fb554160fb0 2024-04-25T13:40:26.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.072+0000 7fb549ffb640 1 -- 172.21.15.80:0/4285194941 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb54c00e3e0 con 0x7fb554160fb0 2024-04-25T13:40:26.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.072+0000 7fb549ffb640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb53806f430 0x7fb538071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.072+0000 7fb549ffb640 1 -- 172.21.15.80:0/4285194941 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fb54c08fc10 con 0x7fb554160fb0 2024-04-25T13:40:26.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.072+0000 7fb559625640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb53806f430 0x7fb538071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.072+0000 7fb559625640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb53806f430 0x7fb538071940 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb55000a850 tx=0x7fb550008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.076+0000 7fb549ffb640 1 -- 172.21.15.80:0/4285194941 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb554165170 con 0x7fb554160fb0 2024-04-25T13:40:26.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.276+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb55406a3a0 con 0x7fb554160fb0 2024-04-25T13:40:26.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.276+0000 7fb549ffb640 1 -- 172.21.15.80:0/4285194941 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3861 (secure 0 0 0) 0x7fb54c014030 con 0x7fb554160fb0 2024-04-25T13:40:26.281 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:26.281 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4272,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-25T13:40:26.282 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 3 2024-04-25T13:40:26.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb53806f430 msgr2=0x7fb538071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb53806f430 0x7fb538071940 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb55000a850 tx=0x7fb550008040 comp rx=0 tx=0).stop 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb554160fb0 msgr2=0x7fb554068590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb554160fb0 0x7fb554068590 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb54c00cdb0 tx=0x7fb54c00a650 comp rx=0 tx=0).stop 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 shutdown_connections 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb53806f430 0x7fb538071940 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb554160fb0 0x7fb554068590 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb55413fe90 0x7fb55413d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 --2- 172.21.15.80:0/4285194941 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5540d25d0 0x7fb55415c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 >> 172.21.15.80:0/4285194941 conn(0x7fb55406c7c0 msgr2=0x7fb5540cf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 shutdown_connections 2024-04-25T13:40:26.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.280+0000 7fb55b889640 1 -- 172.21.15.80:0/4285194941 wait complete. 2024-04-25T13:40:26.303 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:26.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49f1c68640 1 Processor -- start 2024-04-25T13:40:26.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49f1c68640 1 -- start start 2024-04-25T13:40:26.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49f1c68640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f49ec0d0270 0x7f49ec070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49f1c68640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ec068e90 con 0x7f49ec068590 2024-04-25T13:40:26.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49f1c68640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49ec069170 con 0x7f49ec144680 2024-04-25T13:40:26.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49f1c68640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f49ec15e9c0 con 0x7f49ec0d0270 2024-04-25T13:40:26.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49ebfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f49ec0d0270 0x7f49ec070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49eaffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f49ec068590 0x7f49ec0cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59432/0 (socket says 172.21.15.80:59432) 2024-04-25T13:40:26.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49eaffd640 1 -- 172.21.15.80:0/2872449638 learned_addr learned my addr 172.21.15.80:0/2872449638 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49ebfff640 1 -- 172.21.15.80:0/2872449638 >> v1:172.21.15.87:6789/0 conn(0x7f49ec144680 legacy=0x7f49ec15f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49ebfff640 1 -- 172.21.15.80:0/2872449638 >> v1:172.21.15.80:6789/0 conn(0x7f49ec068590 legacy=0x7f49ec0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.444+0000 7f49ea7fc640 1 -- 172.21.15.80:0/2872449638 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2990082847 0 0) 0x7f49ec068e90 con 0x7f49ec068590 2024-04-25T13:40:26.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49ebfff640 1 -- 172.21.15.80:0/2872449638 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec15eca0 con 0x7f49ec0d0270 2024-04-25T13:40:26.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49ebfff640 1 --2- 172.21.15.80:0/2872449638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec0d0270 0x7f49ec070490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009d90 tx=0x7f49e401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab04f9607b0f3402 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49ea7fc640 1 -- 172.21.15.80:0/2872449638 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40b2070 con 0x7f49ec0d0270 2024-04-25T13:40:26.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49ea7fc640 1 -- 172.21.15.80:0/2872449638 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e40a5760 con 0x7f49ec0d0270 2024-04-25T13:40:26.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49ea7fc640 1 -- 172.21.15.80:0/2872449638 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40a5b30 con 0x7f49ec0d0270 2024-04-25T13:40:26.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49f1c68640 1 -- 172.21.15.80:0/2872449638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec0d0270 msgr2=0x7f49ec070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49f1c68640 1 --2- 172.21.15.80:0/2872449638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec0d0270 0x7f49ec070490 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009d90 tx=0x7f49e401c860 comp rx=0 tx=0).stop 2024-04-25T13:40:26.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49f1c68640 1 -- 172.21.15.80:0/2872449638 shutdown_connections 2024-04-25T13:40:26.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49f1c68640 1 --2- 172.21.15.80:0/2872449638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec0d0270 0x7f49ec070490 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49f1c68640 1 -- 172.21.15.80:0/2872449638 >> 172.21.15.80:0/2872449638 conn(0x7f49ec0d33a0 msgr2=0x7f49ec0cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49f1c68640 1 -- 172.21.15.80:0/2872449638 shutdown_connections 2024-04-25T13:40:26.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.448+0000 7f49f1c68640 1 -- 172.21.15.80:0/2872449638 wait complete. 2024-04-25T13:40:26.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49f1c68640 1 Processor -- start 2024-04-25T13:40:26.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49f1c68640 1 -- start start 2024-04-25T13:40:26.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49f1c68640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec068590 0x7f49ec140d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49f1c68640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec144680 0x7f49ec144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49f1c68640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec151ef0 0x7f49ec152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49f1c68640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec16c190 con 0x7f49ec151ef0 2024-04-25T13:40:26.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49f1c68640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec16c010 con 0x7f49ec144680 2024-04-25T13:40:26.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49f1c68640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f49ec16c310 con 0x7f49ec068590 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec151ef0 0x7f49ec152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec151ef0 0x7f49ec152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42632/0 (socket says 172.21.15.80:42632) 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.452+0000 7f49ebfff640 1 -- 172.21.15.80:0/1500631330 learned_addr learned my addr 172.21.15.80:0/1500631330 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49eaffd640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec144680 0x7f49ec144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49eb7fe640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec068590 0x7f49ec140d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49ebfff640 1 -- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec068590 msgr2=0x7f49ec140d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49ebfff640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec068590 0x7f49ec140d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49ebfff640 1 -- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec144680 msgr2=0x7f49ec144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49ebfff640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec144680 0x7f49ec144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49ebfff640 1 -- 172.21.15.80:0/1500631330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec0d0490 con 0x7f49ec151ef0 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49ebfff640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec151ef0 0x7f49ec152300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49e40b0040 tx=0x7f49e4019600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49d3fff640 1 -- 172.21.15.80:0/1500631330 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40b2040 con 0x7f49ec151ef0 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49d3fff640 1 -- 172.21.15.80:0/1500631330 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e40bf070 con 0x7f49ec151ef0 2024-04-25T13:40:26.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ec152840 con 0x7f49ec151ef0 2024-04-25T13:40:26.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49d3fff640 1 -- 172.21.15.80:0/1500631330 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40add30 con 0x7f49ec151ef0 2024-04-25T13:40:26.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ec1459e0 con 0x7f49ec151ef0 2024-04-25T13:40:26.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ec06a3a0 con 0x7f49ec151ef0 2024-04-25T13:40:26.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49d3fff640 1 -- 172.21.15.80:0/1500631330 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f49e40b8470 con 0x7f49ec151ef0 2024-04-25T13:40:26.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49d3fff640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f49d806f400 0x7f49d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49d3fff640 1 -- 172.21.15.80:0/1500631330 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f49e412a430 con 0x7f49ec151ef0 2024-04-25T13:40:26.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.456+0000 7f49eb7fe640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f49d806f400 0x7f49d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.460+0000 7f49eb7fe640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f49d806f400 0x7f49d8071910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0695c0 tx=0x7f49dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.464+0000 7f49d3fff640 1 -- 172.21.15.80:0/1500631330 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f49ec152840 con 0x7f49ec151ef0 2024-04-25T13:40:26.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.624+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f49ec145490 con 0x7f49ec151ef0 2024-04-25T13:40:26.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.624+0000 7f49d3fff640 1 -- 172.21.15.80:0/1500631330 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12812 (secure 0 0 0) 0x7f49e40164e0 con 0x7f49ec151ef0 2024-04-25T13:40:26.629 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:26.629 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:20.533833+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f49d806f400 msgr2=0x7f49d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f49d806f400 0x7f49d8071910 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f49ec0695c0 tx=0x7f49dc008040 comp rx=0 tx=0).stop 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec151ef0 msgr2=0x7f49ec152300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec151ef0 0x7f49ec152300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f49e40b0040 tx=0x7f49e4019600 comp rx=0 tx=0).stop 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 shutdown_connections 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f49d806f400 0x7f49d8071910 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f49ec151ef0 0x7f49ec152300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f49ec144680 0x7f49ec144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 --2- 172.21.15.80:0/1500631330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49ec068590 0x7f49ec140d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 >> 172.21.15.80:0/1500631330 conn(0x7f49ec0d33a0 msgr2=0x7f49ec156840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 shutdown_connections 2024-04-25T13:40:26.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.628+0000 7f49f1c68640 1 -- 172.21.15.80:0/1500631330 wait complete. 2024-04-25T13:40:26.650 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:40:26.651 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:40:26.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f6462c39640 1 Processor -- start 2024-04-25T13:40:26.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f6462c39640 1 -- start start 2024-04-25T13:40:26.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f6462c39640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 0x7f645c0b4140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f6462c39640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645c0b93f0 con 0x7f645c0d1ec0 2024-04-25T13:40:26.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f6462c39640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f645c0d2660 con 0x7f645c0d1ab0 2024-04-25T13:40:26.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f6462c39640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f645c0b4c60 con 0x7f645c0b60a0 2024-04-25T13:40:26.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f64611d6640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f645c0d1ec0 0x7f645c0cec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59442/0 (socket says 172.21.15.80:59442) 2024-04-25T13:40:26.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f64611d6640 1 -- 172.21.15.80:0/3060452698 learned_addr learned my addr 172.21.15.80:0/3060452698 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f64609d5640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 0x7f645c0b4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2329942046 0 0) 0x7f645c0b93f0 con 0x7f645c0d1ec0 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6444003530 con 0x7f645c0d1ec0 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 685151806 0 0) 0x7f6444003530 con 0x7f645c0d1ec0 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 msgr2=0x7f645c0b4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 0x7f645c0b4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 >> v1:172.21.15.87:6789/0 conn(0x7f645c0d1ab0 legacy=0x7f645c0ce580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c0b4680 con 0x7f645c0d1ec0 2024-04-25T13:40:26.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f6454001a00 con 0x7f645c0d1ec0 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 >> v1:172.21.15.80:6789/0 conn(0x7f645c0d1ec0 legacy=0x7f645c0cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64440039f0 0x7f6444003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f6444004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 0x7f6444005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f645c0b4c60 con 0x7f6444004810 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f644400bb00 con 0x7f64440039f0 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f644400bd00 con 0x7f645c0b60a0 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645bfff640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f6444004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f64611d6640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 0x7f6444005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f64609d5640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64440039f0 0x7f6444003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645bfff640 1 -- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 msgr2=0x7f6444005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645bfff640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 0x7f6444005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645bfff640 1 -- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64440039f0 msgr2=0x7f6444003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645bfff640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64440039f0 0x7f6444003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f645bfff640 1 -- 172.21.15.80:0/3060452698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6444003530 con 0x7f6444004810 2024-04-25T13:40:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f64609d5640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64440039f0 0x7f6444003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:26.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.800+0000 7f64611d6640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b60a0 0x7f6444005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:26.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f645bfff640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f6444004c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f645c0a1bb0 tx=0x7f645001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41fc2f0d1f2729e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64500013a0 con 0x7f6444004810 2024-04-25T13:40:26.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64500052f0 con 0x7f6444004810 2024-04-25T13:40:26.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f645b7fe640 1 -- 172.21.15.80:0/3060452698 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6450003840 con 0x7f6444004810 2024-04-25T13:40:26.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f6462c39640 1 -- 172.21.15.80:0/3060452698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 msgr2=0x7f6444004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f6462c39640 1 --2- 172.21.15.80:0/3060452698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f6444004c20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f645c0a1bb0 tx=0x7f645001c3c0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f64611d6640 1 -- 172.21.15.80:0/3060452698 reap_dead start 2024-04-25T13:40:26.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f6462c39640 1 -- 172.21.15.80:0/3060452698 shutdown_connections 2024-04-25T13:40:26.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f6462c39640 1 -- 172.21.15.80:0/3060452698 >> 172.21.15.80:0/3060452698 conn(0x7f645c0b5580 msgr2=0x7f645c0b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f6462c39640 1 -- 172.21.15.80:0/3060452698 shutdown_connections 2024-04-25T13:40:26.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.804+0000 7f6462c39640 1 -- 172.21.15.80:0/3060452698 wait complete. 2024-04-25T13:40:26.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 Processor -- start 2024-04-25T13:40:26.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 -- start start 2024-04-25T13:40:26.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f645c0c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c1d9b20 0x7f645c0b1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b1ad0 0x7f645c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f645c0d2ac0 con 0x7f6444004810 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f645c0d2940 con 0x7f645c1d9b20 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f645c0b4140 con 0x7f645c0b1ad0 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f645c0c8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f645c0c8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42648/0 (socket says 172.21.15.80:42648) 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 -- 172.21.15.80:0/3009304891 learned_addr learned my addr 172.21.15.80:0/3009304891 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64611d6640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b1ad0 0x7f645c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f645bfff640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c1d9b20 0x7f645c0b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 -- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b1ad0 msgr2=0x7f645c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b1ad0 0x7f645c0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 -- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c1d9b20 msgr2=0x7f645c0b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c1d9b20 0x7f645c0b1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 -- 172.21.15.80:0/3009304891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f645c0c9c90 con 0x7f6444004810 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64609d5640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f645c0c8a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f644c0108e0 tx=0x7f644c010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64597fa640 1 -- 172.21.15.80:0/3009304891 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c00f040 con 0x7f6444004810 2024-04-25T13:40:26.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64597fa640 1 -- 172.21.15.80:0/3009304891 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f644c00ec20 con 0x7f6444004810 2024-04-25T13:40:26.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f645c0c9f70 con 0x7f6444004810 2024-04-25T13:40:26.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f64597fa640 1 -- 172.21.15.80:0/3009304891 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f644c004020 con 0x7f6444004810 2024-04-25T13:40:26.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.808+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f645c0c8510 con 0x7f6444004810 2024-04-25T13:40:26.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.812+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f645c06a3a0 con 0x7f6444004810 2024-04-25T13:40:26.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.812+0000 7f64597fa640 1 -- 172.21.15.80:0/3009304891 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f644c0041c0 con 0x7f6444004810 2024-04-25T13:40:26.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.812+0000 7f64597fa640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f644806f400 0x7f6448071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:26.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.812+0000 7f64597fa640 1 -- 172.21.15.80:0/3009304891 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f644c08f5c0 con 0x7f6444004810 2024-04-25T13:40:26.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.812+0000 7f645bfff640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f644806f400 0x7f6448071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:26.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.812+0000 7f645bfff640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f644806f400 0x7f6448071910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6444004790 tx=0x7f64500011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:26.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.816+0000 7f64597fa640 1 -- 172.21.15.80:0/3009304891 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f644c018070 con 0x7f6444004810 2024-04-25T13:40:26.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.976+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f645c0d10f0 con 0x7f6444004810 2024-04-25T13:40:26.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.976+0000 7f64597fa640 1 -- 172.21.15.80:0/3009304891 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f644c05c800 con 0x7f6444004810 2024-04-25T13:40:26.982 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:40:26.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f644806f400 msgr2=0x7f6448071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f644806f400 0x7f6448071910 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6444004790 tx=0x7f64500011a0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 msgr2=0x7f645c0c8a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:26.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f645c0c8a30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f644c0108e0 tx=0x7f644c010da0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 shutdown_connections 2024-04-25T13:40:26.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f644806f400 0x7f6448071910 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f645c0b1ad0 0x7f645c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f645c1d9b20 0x7f645c0b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 --2- 172.21.15.80:0/3009304891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6444004810 0x7f645c0c8a30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:26.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 >> 172.21.15.80:0/3009304891 conn(0x7f645c0b5580 msgr2=0x7f645c0cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:26.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 shutdown_connections 2024-04-25T13:40:26.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:26.980+0000 7f6462c39640 1 -- 172.21.15.80:0/3009304891 wait complete. 2024-04-25T13:40:27.003 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:40:27.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.140+0000 7f88960b4640 1 Processor -- start 2024-04-25T13:40:27.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.140+0000 7f88960b4640 1 -- start start 2024-04-25T13:40:27.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f88960b4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f88900cf000 0x7f889006f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f88960b4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 0x7f889006cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f88960b4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88900700e0 con 0x7f88900b5860 2024-04-25T13:40:27.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f88960b4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f88900cf410 con 0x7f88900cf000 2024-04-25T13:40:27.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f88960b4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88900703c0 con 0x7f889006c810 2024-04-25T13:40:27.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f888f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f88900cf000 0x7f889006f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f8887fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f88900b5860 0x7f88900b5c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59446/0 (socket says 172.21.15.80:59446) 2024-04-25T13:40:27.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f8887fff640 1 -- 172.21.15.80:0/2074777715 learned_addr learned my addr 172.21.15.80:0/2074777715 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:27.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f888f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f88900cf000 0x7f889006f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42652/0 (socket says 172.21.15.80:42652) 2024-04-25T13:40:27.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f888ffff640 1 --2- 172.21.15.80:0/2074777715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 0x7f889006cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f888f7fe640 1 -- 172.21.15.80:0/2074777715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 msgr2=0x7f889006cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:27.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f888f7fe640 1 --2- 172.21.15.80:0/2074777715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 0x7f889006cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f888f7fe640 1 -- 172.21.15.80:0/2074777715 >> v1:172.21.15.80:6789/0 conn(0x7f88900b5860 legacy=0x7f88900b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f888effd640 1 -- 172.21.15.80:0/2074777715 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044589751 0 0) 0x7f88900700e0 con 0x7f88900b5860 2024-04-25T13:40:27.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.144+0000 7f888f7fe640 1 -- 172.21.15.80:0/2074777715 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88900706a0 con 0x7f88900cf000 2024-04-25T13:40:27.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f888f7fe640 1 --2- 172.21.15.80:0/2074777715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889006f800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f888000ba60 tx=0x7f888001eab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda9e1e3c1b6ce3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f888effd640 1 -- 172.21.15.80:0/2074777715 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88800b4070 con 0x7f88900cf000 2024-04-25T13:40:27.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f888effd640 1 -- 172.21.15.80:0/2074777715 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88800a7b80 con 0x7f88900cf000 2024-04-25T13:40:27.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f888effd640 1 -- 172.21.15.80:0/2074777715 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88800af440 con 0x7f88900cf000 2024-04-25T13:40:27.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 -- 172.21.15.80:0/2074777715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 msgr2=0x7f889006f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 --2- 172.21.15.80:0/2074777715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889006f800 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f888000ba60 tx=0x7f888001eab0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 -- 172.21.15.80:0/2074777715 shutdown_connections 2024-04-25T13:40:27.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 --2- 172.21.15.80:0/2074777715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 0x7f889006cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 --2- 172.21.15.80:0/2074777715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889006f800 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 -- 172.21.15.80:0/2074777715 >> 172.21.15.80:0/2074777715 conn(0x7f88900d28f0 msgr2=0x7f88900d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:27.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 -- 172.21.15.80:0/2074777715 shutdown_connections 2024-04-25T13:40:27.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 -- 172.21.15.80:0/2074777715 wait complete. 2024-04-25T13:40:27.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 Processor -- start 2024-04-25T13:40:27.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.148+0000 7f88960b4640 1 -- start start 2024-04-25T13:40:27.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 0x7f889015bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88900b5860 0x7f889013f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889015e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8890068710 con 0x7f88900cf000 2024-04-25T13:40:27.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8890068590 con 0x7f88900b5860 2024-04-25T13:40:27.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8890068890 con 0x7f889006c810 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889015e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889015e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42656/0 (socket says 172.21.15.80:42656) 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 -- 172.21.15.80:0/3299420879 learned_addr learned my addr 172.21.15.80:0/3299420879 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888f7fe640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 0x7f889015bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f8887fff640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88900b5860 0x7f889013f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 -- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 msgr2=0x7f889015bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 0x7f889015bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 -- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88900b5860 msgr2=0x7f889013f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88900b5860 0x7f889013f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 -- 172.21.15.80:0/3299420879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8890145700 con 0x7f88900cf000 2024-04-25T13:40:27.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888ffff640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889015e5e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8888007d70 tx=0x7f888800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888cff9640 1 -- 172.21.15.80:0/3299420879 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888014420 con 0x7f88900cf000 2024-04-25T13:40:27.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888cff9640 1 -- 172.21.15.80:0/3299420879 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8888019070 con 0x7f88900cf000 2024-04-25T13:40:27.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88901459e0 con 0x7f88900cf000 2024-04-25T13:40:27.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888cff9640 1 -- 172.21.15.80:0/3299420879 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888014820 con 0x7f88900cf000 2024-04-25T13:40:27.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8890145fa0 con 0x7f88900cf000 2024-04-25T13:40:27.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889006a3a0 con 0x7f88900cf000 2024-04-25T13:40:27.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888cff9640 1 -- 172.21.15.80:0/3299420879 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f88880149c0 con 0x7f88900cf000 2024-04-25T13:40:27.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888cff9640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887006f400 0x7f8870071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888cff9640 1 -- 172.21.15.80:0/3299420879 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f888808f6a0 con 0x7f88900cf000 2024-04-25T13:40:27.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.152+0000 7f888f7fe640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887006f400 0x7f8870071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.156+0000 7f888f7fe640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887006f400 0x7f8870071910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f88800b1000 tx=0x7f88800a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.156+0000 7f888cff9640 1 -- 172.21.15.80:0/3299420879 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f888805ca00 con 0x7f88900cf000 2024-04-25T13:40:27.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.312+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8890161c10 con 0x7f88900cf000 2024-04-25T13:40:27.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.312+0000 7f888cff9640 1 -- 172.21.15.80:0/3299420879 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7f888805c300 con 0x7f88900cf000 2024-04-25T13:40:27.317 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:27.317 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:40:27.318 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:40:27.318 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:40:27.318 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:40:27.318 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:40:27.318 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887006f400 msgr2=0x7f8870071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887006f400 0x7f8870071910 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f88800b1000 tx=0x7f88800a9070 comp rx=0 tx=0).stop 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 msgr2=0x7f889015e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889015e5e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8888007d70 tx=0x7f888800e4b0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 shutdown_connections 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887006f400 0x7f8870071910 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88900cf000 0x7f889015e5e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88900b5860 0x7f889013f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 --2- 172.21.15.80:0/3299420879 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f889006c810 0x7f889015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 >> 172.21.15.80:0/3299420879 conn(0x7f88900d28f0 msgr2=0x7f8890143800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 shutdown_connections 2024-04-25T13:40:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.316+0000 7f88960b4640 1 -- 172.21.15.80:0/3299420879 wait complete. 2024-04-25T13:40:27.337 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:40:27.337 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:40:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.468+0000 7f5960091640 1 Processor -- start 2024-04-25T13:40:27.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.468+0000 7f5960091640 1 -- start start 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f5960091640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59580d62f0 0x7f5958163800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f5960091640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f59580b9280 0x7f59580b9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f5960091640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59580d46a0 con 0x7f59580b4e20 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f5960091640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f59580d4980 con 0x7f59580b9280 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f5960091640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f59580d4c60 con 0x7f59580d62f0 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595e62e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f59580b9280 0x7f59580b9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595d62c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59580d62f0 0x7f5958163800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595de2d640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f59580b4e20 0x7f59580d5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:56292/0 (socket says 172.21.15.80:56292) 2024-04-25T13:40:27.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595de2d640 1 -- 172.21.15.80:0/1966550502 learned_addr learned my addr 172.21.15.80:0/1966550502 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595d62c640 1 -- 172.21.15.80:0/1966550502 >> v2:172.21.15.87:3300/0 conn(0x7f59580b9280 msgr2=0x7f59580b9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595d62c640 1 --2- 172.21.15.80:0/1966550502 >> v2:172.21.15.87:3300/0 conn(0x7f59580b9280 0x7f59580b9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595d62c640 1 -- 172.21.15.80:0/1966550502 >> v1:172.21.15.87:6789/0 conn(0x7f59580b4e20 legacy=0x7f59580d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595d62c640 1 -- 172.21.15.80:0/1966550502 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59580d4f40 con 0x7f59580d62f0 2024-04-25T13:40:27.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595d62c640 1 --2- 172.21.15.80:0/1966550502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59580d62f0 0x7f5958163800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f59480097f0 tx=0x7f594801c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81340540f7bbd637 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595ce2b640 1 -- 172.21.15.80:0/1966550502 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480b1070 con 0x7f59580d62f0 2024-04-25T13:40:27.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595ce2b640 1 -- 172.21.15.80:0/1966550502 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59480ac410 con 0x7f59580d62f0 2024-04-25T13:40:27.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.472+0000 7f595ce2b640 1 -- 172.21.15.80:0/1966550502 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59480ac7e0 con 0x7f59580d62f0 2024-04-25T13:40:27.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 -- 172.21.15.80:0/1966550502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59580d62f0 msgr2=0x7f5958163800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 --2- 172.21.15.80:0/1966550502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59580d62f0 0x7f5958163800 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f59480097f0 tx=0x7f594801c4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 -- 172.21.15.80:0/1966550502 shutdown_connections 2024-04-25T13:40:27.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 --2- 172.21.15.80:0/1966550502 >> v2:172.21.15.87:3300/0 conn(0x7f59580b9280 0x7f59580b9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 --2- 172.21.15.80:0/1966550502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59580d62f0 0x7f5958163800 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 -- 172.21.15.80:0/1966550502 >> 172.21.15.80:0/1966550502 conn(0x7f5958073700 msgr2=0x7f595806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:27.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 -- 172.21.15.80:0/1966550502 shutdown_connections 2024-04-25T13:40:27.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 -- 172.21.15.80:0/1966550502 wait complete. 2024-04-25T13:40:27.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 Processor -- start 2024-04-25T13:40:27.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.476+0000 7f5960091640 1 -- start start 2024-04-25T13:40:27.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59580b4e20 0x7f595814e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59580b9280 0x7f5958159870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f595814ebc0 0x7f5958159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f595816d1d0 con 0x7f59580b4e20 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f595816d050 con 0x7f59580b9280 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f595816d350 con 0x7f595814ebc0 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59580b4e20 0x7f595814e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59580b4e20 0x7f595814e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42664/0 (socket says 172.21.15.80:42664) 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 -- 172.21.15.80:0/354986973 learned_addr learned my addr 172.21.15.80:0/354986973 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595d62c640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59580b9280 0x7f5958159870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595e62e640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f595814ebc0 0x7f5958159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 -- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f595814ebc0 msgr2=0x7f5958159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f595814ebc0 0x7f5958159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 -- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59580b9280 msgr2=0x7f5958159870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59580b9280 0x7f5958159870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 -- 172.21.15.80:0/354986973 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f595815d8a0 con 0x7f59580b4e20 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595e62e640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f595814ebc0 0x7f5958159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:27.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f595de2d640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59580b4e20 0x7f595814e040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f595400c430 tx=0x7f595400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f594e7fc640 1 -- 172.21.15.80:0/354986973 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5954017070 con 0x7f59580b4e20 2024-04-25T13:40:27.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f594e7fc640 1 -- 172.21.15.80:0/354986973 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5954005cf0 con 0x7f59580b4e20 2024-04-25T13:40:27.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f594e7fc640 1 -- 172.21.15.80:0/354986973 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59540072d0 con 0x7f59580b4e20 2024-04-25T13:40:27.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f595815d1b0 con 0x7f59580b4e20 2024-04-25T13:40:27.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f595815d490 con 0x7f59580b4e20 2024-04-25T13:40:27.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f595815d6c0 con 0x7f59580b4e20 2024-04-25T13:40:27.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f594e7fc640 1 -- 172.21.15.80:0/354986973 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f595401a070 con 0x7f59580b4e20 2024-04-25T13:40:27.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f594e7fc640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f593c06f430 0x7f593c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:27.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.480+0000 7f594e7fc640 1 -- 172.21.15.80:0/354986973 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f595408e1b0 con 0x7f59580b4e20 2024-04-25T13:40:27.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.484+0000 7f595d62c640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f593c06f430 0x7f593c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:27.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.484+0000 7f595d62c640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f593c06f430 0x7f593c071940 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f59480ae000 tx=0x7f5948008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:27.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.488+0000 7f594e7fc640 1 -- 172.21.15.80:0/354986973 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f595815d6c0 con 0x7f59580b4e20 2024-04-25T13:40:27.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:27.632+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f595806a440 con 0x7f59580b4e20 2024-04-25T13:40:28.090 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:40:28.093 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:40:28.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.140+0000 7f594e7fc640 1 -- 172.21.15.80:0/354986973 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f595405ae30 con 0x7f59580b4e20 2024-04-25T13:40:28.146 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:40:28.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f593c06f430 msgr2=0x7f593c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f593c06f430 0x7f593c071940 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f59480ae000 tx=0x7f5948008040 comp rx=0 tx=0).stop 2024-04-25T13:40:28.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59580b4e20 msgr2=0x7f595814e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59580b4e20 0x7f595814e040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f595400c430 tx=0x7f595400c8f0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 shutdown_connections 2024-04-25T13:40:28.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f593c06f430 0x7f593c071940 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f595814ebc0 0x7f5958159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59580b9280 0x7f5958159870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 --2- 172.21.15.80:0/354986973 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59580b4e20 0x7f595814e040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.144+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 >> 172.21.15.80:0/354986973 conn(0x7f5958073700 msgr2=0x7f59580b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:28.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.148+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 shutdown_connections 2024-04-25T13:40:28.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.148+0000 7f5960091640 1 -- 172.21.15.80:0/354986973 wait complete. 2024-04-25T13:40:28.176 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:40:28.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1d5120640 1 Processor -- start 2024-04-25T13:40:28.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1d5120640 1 -- start start 2024-04-25T13:40:28.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1d5120640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 0x7fe1d00d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1d5120640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1d00b9440 0x7fe1d00b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1d5120640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d00d3510 con 0x7fe1d0160640 2024-04-25T13:40:28.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1d5120640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d00d5630 con 0x7fe1d00b9440 2024-04-25T13:40:28.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1d5120640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d00d4a20 con 0x7fe1d00d2bc0 2024-04-25T13:40:28.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1ced9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe1d0160640 0x7fe1d00d4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59458/0 (socket says 172.21.15.80:59458) 2024-04-25T13:40:28.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1ced9d640 1 -- 172.21.15.80:0/310302056 learned_addr learned my addr 172.21.15.80:0/310302056 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:28.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1c7fff640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 0x7fe1d00d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.324+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2906791607 0 0) 0x7fe1c0001120 con 0x7fe1d0160640 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe1bc003530 con 0x7fe1d0160640 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1cf59e640 1 --2- 172.21.15.80:0/310302056 >> v2:172.21.15.87:3300/0 conn(0x7fe1d00b9440 0x7fe1d00b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1130087146 0 0) 0x7fe1bc003530 con 0x7fe1d0160640 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 msgr2=0x7fe1d00d2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 0x7fe1d00d2fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 >> v2:172.21.15.87:3300/0 conn(0x7fe1d00b9440 msgr2=0x7fe1d00b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 --2- 172.21.15.80:0/310302056 >> v2:172.21.15.87:3300/0 conn(0x7fe1d00b9440 0x7fe1d00b9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d00d4d00 con 0x7fe1d0160640 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fe1c0001870 con 0x7fe1d0160640 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 >> v1:172.21.15.80:6789/0 conn(0x7fe1d0160640 legacy=0x7fe1d00d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc004810 0x7fe1bc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 0x7fe1bc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00d5630 con 0x7fe1bc0039c0 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00d4a20 con 0x7fe1bc004810 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1bc00ba80 con 0x7fe1d00d2bc0 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1cf59e640 1 --2- 172.21.15.80:0/310302056 >> v2:172.21.15.87:3300/0 conn(0x7fe1d00b9440 0x7fe1d00b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ced9d640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1cf59e640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 0x7fe1bc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1c7fff640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc004810 0x7fe1bc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ced9d640 1 -- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 msgr2=0x7fe1bc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ced9d640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 0x7fe1bc005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ced9d640 1 -- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc004810 msgr2=0x7fe1bc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ced9d640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc004810 0x7fe1bc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ced9d640 1 -- 172.21.15.80:0/310302056 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1bc003530 con 0x7fe1bc0039c0 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1cf59e640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d00d2bc0 0x7fe1bc005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:28.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1c7fff640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1bc004810 0x7fe1bc004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ced9d640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1bc003e30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00cccd0 tx=0x7fe1c001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1d4f18fff68d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000c0e0 con 0x7fe1bc0039c0 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c00a9020 con 0x7fe1bc0039c0 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1ce59c640 1 -- 172.21.15.80:0/310302056 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0005b00 con 0x7fe1bc0039c0 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1d5120640 1 -- 172.21.15.80:0/310302056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 msgr2=0x7fe1bc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1d5120640 1 --2- 172.21.15.80:0/310302056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1bc003e30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00cccd0 tx=0x7fe1c001cc60 comp rx=0 tx=0).stop 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1cf59e640 1 -- 172.21.15.80:0/310302056 reap_dead start 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1d5120640 1 -- 172.21.15.80:0/310302056 shutdown_connections 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.328+0000 7fe1d5120640 1 -- 172.21.15.80:0/310302056 >> 172.21.15.80:0/310302056 conn(0x7fe1d006c9f0 msgr2=0x7fe1d006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 -- 172.21.15.80:0/310302056 shutdown_connections 2024-04-25T13:40:28.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 -- 172.21.15.80:0/310302056 wait complete. 2024-04-25T13:40:28.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 Processor -- start 2024-04-25T13:40:28.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 -- start start 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1d00b5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d01fd6a0 0x7fe1d01f91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d01fdab0 0x7fe1d01f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d016b0b0 con 0x7fe1bc0039c0 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d00ba7b0 con 0x7fe1d01fdab0 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d016b230 con 0x7fe1d01fd6a0 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1d00b5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1d00b5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42670/0 (socket says 172.21.15.80:42670) 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 -- 172.21.15.80:0/480251721 learned_addr learned my addr 172.21.15.80:0/480251721 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1c7fff640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d01fd6a0 0x7fe1d01f91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1cf59e640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d01fdab0 0x7fe1d01f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 -- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d01fd6a0 msgr2=0x7fe1d01f91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d01fd6a0 0x7fe1d01f91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 -- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d01fdab0 msgr2=0x7fe1d01f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d01fdab0 0x7fe1d01f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 -- 172.21.15.80:0/480251721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d00b8640 con 0x7fe1bc0039c0 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1ced9d640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1d00b5770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1c001ca90 tx=0x7fe1bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1c77fe640 1 -- 172.21.15.80:0/480251721 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00a58d0 con 0x7fe1bc0039c0 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1c77fe640 1 -- 172.21.15.80:0/480251721 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c00b3030 con 0x7fe1bc0039c0 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d00b8360 con 0x7fe1bc0039c0 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1c77fe640 1 -- 172.21.15.80:0/480251721 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00a5c90 con 0x7fe1bc0039c0 2024-04-25T13:40:28.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.332+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d01651f0 con 0x7fe1bc0039c0 2024-04-25T13:40:28.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.336+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d006a3a0 con 0x7fe1bc0039c0 2024-04-25T13:40:28.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.336+0000 7fe1c77fe640 1 -- 172.21.15.80:0/480251721 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe1c00047f0 con 0x7fe1bc0039c0 2024-04-25T13:40:28.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.336+0000 7fe1c77fe640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1b006f400 0x7fe1b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:28.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.336+0000 7fe1c77fe640 1 -- 172.21.15.80:0/480251721 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7fe1c0128510 con 0x7fe1bc0039c0 2024-04-25T13:40:28.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.336+0000 7fe1c7fff640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1b006f400 0x7fe1b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:28.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.336+0000 7fe1c7fff640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1b006f400 0x7fe1b0071910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009160 tx=0x7fe1b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:28.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.340+0000 7fe1c77fe640 1 -- 172.21.15.80:0/480251721 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe1d00d3510 con 0x7fe1bc0039c0 2024-04-25T13:40:28.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:28.488+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe1d015cba0 con 0x7fe1bc0039c0 2024-04-25T13:40:29.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.148+0000 7fe1c77fe640 1 -- 172.21.15.80:0/480251721 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) ==== 170+0+0 (secure 0 0 0) 0x7fe1c00f5700 con 0x7fe1bc0039c0 2024-04-25T13:40:29.152 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1b006f400 msgr2=0x7fe1b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1b006f400 0x7fe1b0071910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009160 tx=0x7fe1b8008040 comp rx=0 tx=0).stop 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 msgr2=0x7fe1d00b5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1d00b5770 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe1c001ca90 tx=0x7fe1bc005840 comp rx=0 tx=0).stop 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 shutdown_connections 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1b006f400 0x7fe1b0071910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d01fdab0 0x7fe1d01f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d01fd6a0 0x7fe1d01f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 --2- 172.21.15.80:0/480251721 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1bc0039c0 0x7fe1d00b5770 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 >> 172.21.15.80:0/480251721 conn(0x7fe1d006c9f0 msgr2=0x7fe1d015d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:29.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 shutdown_connections 2024-04-25T13:40:29.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.152+0000 7fe1d5120640 1 -- 172.21.15.80:0/480251721 wait complete. 2024-04-25T13:40:29.187 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:40:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff0587ee640 1 Processor -- start 2024-04-25T13:40:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff0587ee640 1 -- start start 2024-04-25T13:40:29.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff0587ee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff0500b5330 0x7ff0500d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:29.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff0587ee640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff0500d0ff0 0x7ff05006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:29.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff0587ee640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0500b5740 con 0x7ff05006f2f0 2024-04-25T13:40:29.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff0587ee640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff05006f700 con 0x7ff0500d0ff0 2024-04-25T13:40:29.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff0587ee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff0501629b0 con 0x7ff0500b5330 2024-04-25T13:40:29.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff055d89640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff0500d0ff0 0x7ff05006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:29.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff05658a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff0500b5330 0x7ff0500d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:29.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff055d89640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff0500d0ff0 0x7ff05006edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:37074/0 (socket says 172.21.15.80:37074) 2024-04-25T13:40:29.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff055d89640 1 -- 172.21.15.80:0/3004589751 learned_addr learned my addr 172.21.15.80:0/3004589751 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:29.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff055d89640 1 -- 172.21.15.80:0/3004589751 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff0500b5330 msgr2=0x7ff0500d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:29.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff055d89640 1 --2- 172.21.15.80:0/3004589751 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff0500b5330 0x7ff0500d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff055d89640 1 -- 172.21.15.80:0/3004589751 >> v1:172.21.15.87:6789/0 conn(0x7ff05006f2f0 legacy=0x7ff050161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:29.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.336+0000 7ff055d89640 1 -- 172.21.15.80:0/3004589751 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05006dd10 con 0x7ff0500d0ff0 2024-04-25T13:40:29.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff055d89640 1 --2- 172.21.15.80:0/3004589751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500d0ff0 0x7ff05006edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff040009870 tx=0x7ff04001c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=471c7be221a1c4de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:29.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff055588640 1 -- 172.21.15.80:0/3004589751 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0400b1070 con 0x7ff0500d0ff0 2024-04-25T13:40:29.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff055588640 1 -- 172.21.15.80:0/3004589751 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0400ac620 con 0x7ff0500d0ff0 2024-04-25T13:40:29.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff055588640 1 -- 172.21.15.80:0/3004589751 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0400ac9f0 con 0x7ff0500d0ff0 2024-04-25T13:40:29.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff0587ee640 1 -- 172.21.15.80:0/3004589751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500d0ff0 msgr2=0x7ff05006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:29.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3004589751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500d0ff0 0x7ff05006edb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff040009870 tx=0x7ff04001c4d0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff0587ee640 1 -- 172.21.15.80:0/3004589751 shutdown_connections 2024-04-25T13:40:29.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3004589751 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500d0ff0 0x7ff05006edb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3004589751 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff0500b5330 0x7ff0500d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff0587ee640 1 -- 172.21.15.80:0/3004589751 >> 172.21.15.80:0/3004589751 conn(0x7ff0500b4160 msgr2=0x7ff0500d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:29.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff0587ee640 1 -- 172.21.15.80:0/3004589751 shutdown_connections 2024-04-25T13:40:29.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.340+0000 7ff0587ee640 1 -- 172.21.15.80:0/3004589751 wait complete. 2024-04-25T13:40:29.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 Processor -- start 2024-04-25T13:40:29.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 -- start start 2024-04-25T13:40:29.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff05006f2f0 0x7ff05015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:29.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500b5330 0x7ff050131fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:29.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0500d0ff0 0x7ff05015fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:29.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff0500b0cd0 con 0x7ff0500d0ff0 2024-04-25T13:40:29.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff05006e890 con 0x7ff0500b5330 2024-04-25T13:40:29.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff0500b0e50 con 0x7ff05006f2f0 2024-04-25T13:40:29.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0500d0ff0 0x7ff05015fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:29.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0500d0ff0 0x7ff05015fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42678/0 (socket says 172.21.15.80:42678) 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 -- 172.21.15.80:0/3390903315 learned_addr learned my addr 172.21.15.80:0/3390903315 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff05658a640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff05006f2f0 0x7ff05015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff055d89640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500b5330 0x7ff050131fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 -- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff05006f2f0 msgr2=0x7ff05015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff05006f2f0 0x7ff05015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 -- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500b5330 msgr2=0x7ff050131fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500b5330 0x7ff050131fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 -- 172.21.15.80:0/3390903315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05015f2d0 con 0x7ff0500d0ff0 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff05658a640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff05006f2f0 0x7ff05015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff056d8b640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0500d0ff0 0x7ff05015fdd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff04800c760 tx=0x7ff04800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff046ffd640 1 -- 172.21.15.80:0/3390903315 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048016020 con 0x7ff0500d0ff0 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff046ffd640 1 -- 172.21.15.80:0/3390903315 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff048006d80 con 0x7ff0500d0ff0 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff050132500 con 0x7ff0500d0ff0 2024-04-25T13:40:29.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff046ffd640 1 -- 172.21.15.80:0/3390903315 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff048003bc0 con 0x7ff0500d0ff0 2024-04-25T13:40:29.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.344+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0501407c0 con 0x7ff0500d0ff0 2024-04-25T13:40:29.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.348+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff050140250 con 0x7ff0500d0ff0 2024-04-25T13:40:29.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.348+0000 7ff046ffd640 1 -- 172.21.15.80:0/3390903315 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff04800a850 con 0x7ff0500d0ff0 2024-04-25T13:40:29.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.348+0000 7ff046ffd640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff03406f400 0x7ff034071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:29.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.348+0000 7ff046ffd640 1 -- 172.21.15.80:0/3390903315 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7ff04808faf0 con 0x7ff0500d0ff0 2024-04-25T13:40:29.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.348+0000 7ff05658a640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff03406f400 0x7ff034071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:29.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.348+0000 7ff05658a640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff03406f400 0x7ff034071910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009b10 tx=0x7ff04c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:29.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.352+0000 7ff046ffd640 1 -- 172.21.15.80:0/3390903315 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff048020020 con 0x7ff0500d0ff0 2024-04-25T13:40:29.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:29.552+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff05006a3a0 con 0x7ff0500d0ff0 2024-04-25T13:40:30.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.200+0000 7ff046ffd640 1 -- 172.21.15.80:0/3390903315 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ff04805cbb0 con 0x7ff0500d0ff0 2024-04-25T13:40:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T13:40:30.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff03406f400 msgr2=0x7ff034071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff03406f400 0x7ff034071910 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff04c009b10 tx=0x7ff04c008040 comp rx=0 tx=0).stop 2024-04-25T13:40:30.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0500d0ff0 msgr2=0x7ff05015fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0500d0ff0 0x7ff05015fdd0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff04800c760 tx=0x7ff04800cc20 comp rx=0 tx=0).stop 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 shutdown_connections 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff03406f400 0x7ff034071910 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0500d0ff0 0x7ff05015fdd0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff0500b5330 0x7ff050131fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 --2- 172.21.15.80:0/3390903315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff05006f2f0 0x7ff05015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 >> 172.21.15.80:0/3390903315 conn(0x7ff0500b4160 msgr2=0x7ff0500d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 shutdown_connections 2024-04-25T13:40:30.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.212+0000 7ff0587ee640 1 -- 172.21.15.80:0/3390903315 wait complete. 2024-04-25T13:40:30.244 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:30.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.384+0000 7f7e571e3640 1 Processor -- start 2024-04-25T13:40:30.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.384+0000 7f7e571e3640 1 -- start start 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e571e3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7e500b4eb0 0x7f7e501603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e571e3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7e500b93f0 0x7f7e500b9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e571e3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e500d48a0 con 0x7f7e500b7780 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e571e3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7e500d4b80 con 0x7f7e500b4eb0 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e571e3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7e500d4e60 con 0x7f7e500b93f0 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e4ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7e500b7780 0x7f7e500b7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59464/0 (socket says 172.21.15.80:59464) 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e4ffff640 1 -- 172.21.15.80:0/1272367834 learned_addr learned my addr 172.21.15.80:0/1272367834 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e54f7f640 1 --2- 172.21.15.80:0/1272367834 >> v2:172.21.15.80:3300/0 conn(0x7f7e500b4eb0 0x7f7e501603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e55780640 1 --2- 172.21.15.80:0/1272367834 >> v2:172.21.15.87:3300/0 conn(0x7f7e500b93f0 0x7f7e500b9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e4f7fe640 1 -- 172.21.15.80:0/1272367834 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4175566143 0 0) 0x7f7e500d48a0 con 0x7f7e500b7780 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e4f7fe640 1 -- 172.21.15.80:0/1272367834 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7e38003530 con 0x7f7e500b7780 2024-04-25T13:40:30.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e54f7f640 1 -- 172.21.15.80:0/1272367834 >> v2:172.21.15.87:3300/0 conn(0x7f7e500b93f0 msgr2=0x7f7e500b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:30.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e54f7f640 1 --2- 172.21.15.80:0/1272367834 >> v2:172.21.15.87:3300/0 conn(0x7f7e500b93f0 0x7f7e500b9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e54f7f640 1 -- 172.21.15.80:0/1272367834 >> v1:172.21.15.80:6789/0 conn(0x7f7e500b7780 legacy=0x7f7e500b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.388+0000 7f7e54f7f640 1 -- 172.21.15.80:0/1272367834 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e500d5140 con 0x7f7e500b4eb0 2024-04-25T13:40:30.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e54f7f640 1 --2- 172.21.15.80:0/1272367834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b4eb0 0x7f7e501603c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7e5008ef70 tx=0x7f7e4001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916769c17304f720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:30.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e4f7fe640 1 -- 172.21.15.80:0/1272367834 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400b0070 con 0x7f7e500b4eb0 2024-04-25T13:40:30.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e4f7fe640 1 -- 172.21.15.80:0/1272367834 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e400ab600 con 0x7f7e500b4eb0 2024-04-25T13:40:30.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e4f7fe640 1 -- 172.21.15.80:0/1272367834 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e400ab9b0 con 0x7f7e500b4eb0 2024-04-25T13:40:30.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 -- 172.21.15.80:0/1272367834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b4eb0 msgr2=0x7f7e501603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 --2- 172.21.15.80:0/1272367834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b4eb0 0x7f7e501603c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7e5008ef70 tx=0x7f7e4001c3c0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 -- 172.21.15.80:0/1272367834 shutdown_connections 2024-04-25T13:40:30.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 --2- 172.21.15.80:0/1272367834 >> v2:172.21.15.87:3300/0 conn(0x7f7e500b93f0 0x7f7e500b9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 --2- 172.21.15.80:0/1272367834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b4eb0 0x7f7e501603c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 -- 172.21.15.80:0/1272367834 >> 172.21.15.80:0/1272367834 conn(0x7f7e5006c9f0 msgr2=0x7f7e5006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:30.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 -- 172.21.15.80:0/1272367834 shutdown_connections 2024-04-25T13:40:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 -- 172.21.15.80:0/1272367834 wait complete. 2024-04-25T13:40:30.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 Processor -- start 2024-04-25T13:40:30.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.392+0000 7f7e571e3640 1 -- start start 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e500b4eb0 0x7f7e50155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b7780 0x7f7e5015a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e500b93f0 0x7f7e500ba610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7e500d3120 con 0x7f7e500b7780 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5016d090 con 0x7f7e500b93f0 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7e500cb640 con 0x7f7e500b4eb0 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b7780 0x7f7e5015a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b7780 0x7f7e5015a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42696/0 (socket says 172.21.15.80:42696) 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 -- 172.21.15.80:0/2919973583 learned_addr learned my addr 172.21.15.80:0/2919973583 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e55780640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e500b93f0 0x7f7e500ba610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e54f7f640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e500b4eb0 0x7f7e50155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 -- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e500b4eb0 msgr2=0x7f7e50155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e500b4eb0 0x7f7e50155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 -- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e500b93f0 msgr2=0x7f7e500ba610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e500b93f0 0x7f7e500ba610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 -- 172.21.15.80:0/2919973583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5015afc0 con 0x7f7e500b7780 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e54f7f640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e500b4eb0 0x7f7e50155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4ffff640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b7780 0x7f7e5015a300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400c3f0 tx=0x7f7e4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4d7fa640 1 -- 172.21.15.80:0/2919973583 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e44017070 con 0x7f7e500b7780 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4d7fa640 1 -- 172.21.15.80:0/2919973583 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e44005cf0 con 0x7f7e500b7780 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5015b7a0 con 0x7f7e500b7780 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4d7fa640 1 -- 172.21.15.80:0/2919973583 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e440072f0 con 0x7f7e500b7780 2024-04-25T13:40:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5015bd60 con 0x7f7e500b7780 2024-04-25T13:40:30.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5015ad30 con 0x7f7e500b7780 2024-04-25T13:40:30.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.396+0000 7f7e4d7fa640 1 -- 172.21.15.80:0/2919973583 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7e4401a070 con 0x7f7e500b7780 2024-04-25T13:40:30.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.400+0000 7f7e4d7fa640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7e3c06f400 0x7f7e3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.400+0000 7f7e4d7fa640 1 -- 172.21.15.80:0/2919973583 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f7e4408e6f0 con 0x7f7e500b7780 2024-04-25T13:40:30.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.400+0000 7f7e54f7f640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7e3c06f400 0x7f7e3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.400+0000 7f7e54f7f640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7e3c06f400 0x7f7e3c071910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e400ad000 tx=0x7f7e400a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.404+0000 7f7e4d7fa640 1 -- 172.21.15.80:0/2919973583 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7e4405b5e0 con 0x7f7e500b7780 2024-04-25T13:40:30.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.564+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7e5006a3a0 con 0x7f7e500b7780 2024-04-25T13:40:30.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.568+0000 7f7e4d7fa640 1 -- 172.21.15.80:0/2919973583 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16546 (secure 0 0 0) 0x7f7e4405aee0 con 0x7f7e500b7780 2024-04-25T13:40:30.573 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:30.573 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":19,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:29.558189+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7e3c06f400 msgr2=0x7f7e3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7e3c06f400 0x7f7e3c071910 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7e400ad000 tx=0x7f7e400a6000 comp rx=0 tx=0).stop 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b7780 msgr2=0x7f7e5015a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b7780 0x7f7e5015a300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7e4400c3f0 tx=0x7f7e4400c8b0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 shutdown_connections 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7e3c06f400 0x7f7e3c071910 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7e500b93f0 0x7f7e500ba610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7e500b7780 0x7f7e5015a300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 --2- 172.21.15.80:0/2919973583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7e500b4eb0 0x7f7e50155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 >> 172.21.15.80:0/2919973583 conn(0x7f7e5006c9f0 msgr2=0x7f7e500b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:30.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 shutdown_connections 2024-04-25T13:40:30.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.572+0000 7f7e571e3640 1 -- 172.21.15.80:0/2919973583 wait complete. 2024-04-25T13:40:30.594 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:30.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa616397640 1 Processor -- start 2024-04-25T13:40:30.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa616397640 1 -- start start 2024-04-25T13:40:30.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa616397640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa6100b6d30 0x7fa6100d0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa616397640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6100b4ad0 con 0x7fa6100d1050 2024-04-25T13:40:30.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa616397640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6100d4840 con 0x7fa6100b6920 2024-04-25T13:40:30.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa616397640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa6100b5340 con 0x7fa6100b6d30 2024-04-25T13:40:30.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa60f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa6100b6d30 0x7fa6100d0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa614934640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa6100d1050 0x7fa6100ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59468/0 (socket says 172.21.15.80:59468) 2024-04-25T13:40:30.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa614934640 1 -- 172.21.15.80:0/3683767971 learned_addr learned my addr 172.21.15.80:0/3683767971 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:30.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa60f7fe640 1 -- 172.21.15.80:0/3683767971 >> v1:172.21.15.87:6789/0 conn(0x7fa6100b6920 legacy=0x7fa6100d1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:30.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa60f7fe640 1 -- 172.21.15.80:0/3683767971 >> v1:172.21.15.80:6789/0 conn(0x7fa6100d1050 legacy=0x7fa6100ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa60f7fe640 1 -- 172.21.15.80:0/3683767971 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6100b5060 con 0x7fa6100b6d30 2024-04-25T13:40:30.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.740+0000 7fa60effd640 1 -- 172.21.15.80:0/3683767971 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2608467040 0 0) 0x7fa6100b4ad0 con 0x7fa6100d1050 2024-04-25T13:40:30.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa60f7fe640 1 --2- 172.21.15.80:0/3683767971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6d30 0x7fa6100d0b10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80097c0 tx=0x7fa5f801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5353b6dbecf90ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:30.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa60effd640 1 -- 172.21.15.80:0/3683767971 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80b0070 con 0x7fa6100b6d30 2024-04-25T13:40:30.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa60effd640 1 -- 172.21.15.80:0/3683767971 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5f80ab600 con 0x7fa6100b6d30 2024-04-25T13:40:30.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa60effd640 1 -- 172.21.15.80:0/3683767971 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f80ab9d0 con 0x7fa6100b6d30 2024-04-25T13:40:30.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 -- 172.21.15.80:0/3683767971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6d30 msgr2=0x7fa6100d0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 --2- 172.21.15.80:0/3683767971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6d30 0x7fa6100d0b10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80097c0 tx=0x7fa5f801c4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 -- 172.21.15.80:0/3683767971 shutdown_connections 2024-04-25T13:40:30.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 --2- 172.21.15.80:0/3683767971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6d30 0x7fa6100d0b10 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 -- 172.21.15.80:0/3683767971 >> 172.21.15.80:0/3683767971 conn(0x7fa6100d3ad0 msgr2=0x7fa6100d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:30.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 -- 172.21.15.80:0/3683767971 shutdown_connections 2024-04-25T13:40:30.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 -- 172.21.15.80:0/3683767971 wait complete. 2024-04-25T13:40:30.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 Processor -- start 2024-04-25T13:40:30.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.744+0000 7fa616397640 1 -- start start 2024-04-25T13:40:30.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6920 0x7fa6100c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6100b6d30 0x7fa61015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100d1050 0x7fa6101620e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100b4e10 con 0x7fa6100b6920 2024-04-25T13:40:30.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa6100d45f0 con 0x7fa6100d1050 2024-04-25T13:40:30.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa6100cf500 con 0x7fa6100b6d30 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6920 0x7fa6100c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6920 0x7fa6100c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42714/0 (socket says 172.21.15.80:42714) 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 -- 172.21.15.80:0/612288358 learned_addr learned my addr 172.21.15.80:0/612288358 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60f7fe640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6100b6d30 0x7fa61015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 -- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6100b6d30 msgr2=0x7fa61015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6100b6d30 0x7fa61015a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 -- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100d1050 msgr2=0x7fa6101620e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100d1050 0x7fa6101620e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 -- 172.21.15.80:0/612288358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61015b390 con 0x7fa6100b6920 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60ffff640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6920 0x7fa6100c8010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa60800c760 tx=0x7fa60800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60cff9640 1 -- 172.21.15.80:0/612288358 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608003b80 con 0x7fa6100b6920 2024-04-25T13:40:30.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60cff9640 1 -- 172.21.15.80:0/612288358 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa608003d20 con 0x7fa6100b6920 2024-04-25T13:40:30.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61015b670 con 0x7fa6100b6920 2024-04-25T13:40:30.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60cff9640 1 -- 172.21.15.80:0/612288358 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60800aa90 con 0x7fa6100b6920 2024-04-25T13:40:30.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61015bc30 con 0x7fa6100b6920 2024-04-25T13:40:30.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61006a3a0 con 0x7fa6100b6920 2024-04-25T13:40:30.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60cff9640 1 -- 172.21.15.80:0/612288358 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa60800ac30 con 0x7fa6100b6920 2024-04-25T13:40:30.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.748+0000 7fa60cff9640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa5f006f400 0x7fa5f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:30.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.752+0000 7fa60cff9640 1 -- 172.21.15.80:0/612288358 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7fa60808f120 con 0x7fa6100b6920 2024-04-25T13:40:30.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.752+0000 7fa60f7fe640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa5f006f400 0x7fa5f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:30.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.752+0000 7fa60f7fe640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa5f006f400 0x7fa5f0071910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80ad000 tx=0x7fa5f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:30.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.756+0000 7fa60cff9640 1 -- 172.21.15.80:0/612288358 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa60805c0a0 con 0x7fa6100b6920 2024-04-25T13:40:30.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.916+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa610076930 con 0x7fa6100b6920 2024-04-25T13:40:30.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa60cff9640 1 -- 172.21.15.80:0/612288358 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+16546 (secure 0 0 0) 0x7fa608064090 con 0x7fa6100b6920 2024-04-25T13:40:30.923 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:30.923 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":19,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:29.558189+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa5f006f400 msgr2=0x7fa5f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa5f006f400 0x7fa5f0071910 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa5f80ad000 tx=0x7fa5f8008040 comp rx=0 tx=0).stop 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6920 msgr2=0x7fa6100c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6920 0x7fa6100c8010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa60800c760 tx=0x7fa60800cc20 comp rx=0 tx=0).stop 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 shutdown_connections 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa5f006f400 0x7fa5f0071910 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6100d1050 0x7fa6101620e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6100b6d30 0x7fa61015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 --2- 172.21.15.80:0/612288358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6100b6920 0x7fa6100c8010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 >> 172.21.15.80:0/612288358 conn(0x7fa6100d3ad0 msgr2=0x7fa610071720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:30.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 shutdown_connections 2024-04-25T13:40:30.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:30.920+0000 7fa616397640 1 -- 172.21.15.80:0/612288358 wait complete. 2024-04-25T13:40:30.944 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:40:31.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.088+0000 7ff840e2d640 1 Processor -- start 2024-04-25T13:40:31.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.088+0000 7ff840e2d640 1 -- start start 2024-04-25T13:40:31.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff840e2d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff83c0d3ed0 0x7ff83c0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:31.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff840e2d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83c0b92a0 con 0x7ff83c0ba770 2024-04-25T13:40:31.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff840e2d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff83c0b84f0 con 0x7ff83c0d2ad0 2024-04-25T13:40:31.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff840e2d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff83c0d4c10 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff83a59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff83c0d3ed0 0x7ff83c0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:31.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff83ad9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff83c0ba770 0x7ff83c0b7de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35566/0 (socket says 172.21.15.80:35566) 2024-04-25T13:40:31.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff83ad9d640 1 -- 172.21.15.80:0/4276315297 learned_addr learned my addr 172.21.15.80:0/4276315297 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:31.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff83959a640 1 -- 172.21.15.80:0/4276315297 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1757196176 0 0) 0x7ff8340014a0 con 0x7ff83c0ba770 2024-04-25T13:40:31.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff83959a640 1 -- 172.21.15.80:0/4276315297 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff824003530 con 0x7ff83c0ba770 2024-04-25T13:40:31.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff83a59c640 1 -- 172.21.15.80:0/4276315297 >> v1:172.21.15.87:6789/0 conn(0x7ff83c0d2ad0 legacy=0x7ff83c0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:31.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff83a59c640 1 -- 172.21.15.80:0/4276315297 >> v1:172.21.15.80:6789/0 conn(0x7ff83c0ba770 legacy=0x7ff83c0b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:31.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.092+0000 7ff83a59c640 1 -- 172.21.15.80:0/4276315297 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c0d4ef0 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff83a59c640 1 --2- 172.21.15.80:0/4276315297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c0d2590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0097c0 tx=0x7ff82c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a64137e170f598 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:31.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff83959a640 1 -- 172.21.15.80:0/4276315297 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0ab610 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff83959a640 1 -- 172.21.15.80:0/4276315297 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff82c0b0070 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff83959a640 1 -- 172.21.15.80:0/4276315297 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0ab9b0 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 -- 172.21.15.80:0/4276315297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 msgr2=0x7ff83c0d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:31.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 --2- 172.21.15.80:0/4276315297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c0d2590 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0097c0 tx=0x7ff82c01c410 comp rx=0 tx=0).stop 2024-04-25T13:40:31.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 -- 172.21.15.80:0/4276315297 shutdown_connections 2024-04-25T13:40:31.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 --2- 172.21.15.80:0/4276315297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c0d2590 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:31.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 -- 172.21.15.80:0/4276315297 >> 172.21.15.80:0/4276315297 conn(0x7ff83c0737b0 msgr2=0x7ff83c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:31.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 -- 172.21.15.80:0/4276315297 shutdown_connections 2024-04-25T13:40:31.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 -- 172.21.15.80:0/4276315297 wait complete. 2024-04-25T13:40:31.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 Processor -- start 2024-04-25T13:40:31.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.096+0000 7ff840e2d640 1 -- start start 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff83c0ba770 0x7ff83c15ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff83c0d2ad0 0x7ff83c156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0ccae0 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff83c0cc960 con 0x7ff83c0d2ad0 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff83c160620 con 0x7ff83c0ba770 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c15ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35926/0 (socket says 172.21.15.80:35926) 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 -- 172.21.15.80:0/1615748733 learned_addr learned my addr 172.21.15.80:0/1615748733 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:31.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83a59c640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff83c0ba770 0x7ff83c15ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff839d9b640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff83c0d2ad0 0x7ff83c156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 -- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff83c0ba770 msgr2=0x7ff83c15ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff83c0ba770 0x7ff83c15ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 -- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff83c0d2ad0 msgr2=0x7ff83c156f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff83c0d2ad0 0x7ff83c156f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 -- 172.21.15.80:0/1615748733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff83c15cf90 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff83ad9d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c15ac90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff83400cb10 tx=0x7ff83400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff82affd640 1 -- 172.21.15.80:0/1615748733 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834017070 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff82affd640 1 -- 172.21.15.80:0/1615748733 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff83400ac10 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83c15e100 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff82affd640 1 -- 172.21.15.80:0/1615748733 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834012400 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83c15e6c0 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff83c06a3a0 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.100+0000 7ff82affd640 1 -- 172.21.15.80:0/1615748733 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff83401a070 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.104+0000 7ff82affd640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff81c06f430 0x7ff81c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:31.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.104+0000 7ff82affd640 1 -- 172.21.15.80:0/1615748733 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7ff83408ef60 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.104+0000 7ff83a59c640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff81c06f430 0x7ff81c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:31.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.104+0000 7ff83a59c640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff81c06f430 0x7ff81c071940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0af000 tx=0x7ff82c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:31.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.108+0000 7ff82affd640 1 -- 172.21.15.80:0/1615748733 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff83c0b92a0 con 0x7ff83c0d3ed0 2024-04-25T13:40:31.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:31.308+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff83c15d940 con 0x7ff83c0d3ed0 2024-04-25T13:40:32.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff82affd640 1 -- 172.21.15.80:0/1615748733 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) ==== 118+0+0 (secure 0 0 0) 0x7ff83405bee0 con 0x7ff83c0d3ed0 2024-04-25T13:40:32.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff81c06f430 msgr2=0x7ff81c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:32.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff81c06f430 0x7ff81c071940 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff82c0af000 tx=0x7ff82c0a7000 comp rx=0 tx=0).stop 2024-04-25T13:40:32.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 msgr2=0x7ff83c15ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:32.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c15ac90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff83400cb10 tx=0x7ff83400a060 comp rx=0 tx=0).stop 2024-04-25T13:40:32.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 shutdown_connections 2024-04-25T13:40:32.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff81c06f430 0x7ff81c071940 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff83c0d3ed0 0x7ff83c15ac90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff83c0d2ad0 0x7ff83c156f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 --2- 172.21.15.80:0/1615748733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff83c0ba770 0x7ff83c15ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 >> 172.21.15.80:0/1615748733 conn(0x7ff83c0737b0 msgr2=0x7ff83c0ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:32.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 shutdown_connections 2024-04-25T13:40:32.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.188+0000 7ff840e2d640 1 -- 172.21.15.80:0/1615748733 wait complete. 2024-04-25T13:40:32.219 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']} 2024-04-25T13:40:32.219 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:40:32.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74df17e640 1 Processor -- start 2024-04-25T13:40:32.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74df17e640 1 -- start start 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74df17e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f74d80b5330 0x7f74d80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74df17e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f74d80d0ff0 0x7f74d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74df17e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d8161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74df17e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f74d806f700 con 0x7f74d80d0ff0 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74df17e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f74d81629b0 con 0x7f74d80b5330 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74df17e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f74d80b5740 con 0x7f74d806f2f0 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74d7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f74d80d0ff0 0x7f74d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74d7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f74d80d0ff0 0x7f74d806edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35930/0 (socket says 172.21.15.80:35930) 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74d7fff640 1 -- 172.21.15.80:0/3482318284 learned_addr learned my addr 172.21.15.80:0/3482318284 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74dcf1a640 1 --2- 172.21.15.80:0/3482318284 >> v2:172.21.15.87:3300/0 conn(0x7f74d80b5330 0x7f74d80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74dd71b640 1 --2- 172.21.15.80:0/3482318284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d8161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74d7fff640 1 -- 172.21.15.80:0/3482318284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 msgr2=0x7f74d8161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74d7fff640 1 --2- 172.21.15.80:0/3482318284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d8161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74d7fff640 1 -- 172.21.15.80:0/3482318284 >> v2:172.21.15.87:3300/0 conn(0x7f74d80b5330 msgr2=0x7f74d80d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:32.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74d7fff640 1 --2- 172.21.15.80:0/3482318284 >> v2:172.21.15.87:3300/0 conn(0x7f74d80b5330 0x7f74d80d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74d7fff640 1 -- 172.21.15.80:0/3482318284 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d806dd10 con 0x7f74d80d0ff0 2024-04-25T13:40:32.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.376+0000 7f74dd71b640 1 --2- 172.21.15.80:0/3482318284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d8161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:32.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74d7fff640 1 --2- 172.21.15.80:0/3482318284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f74d80701f0 tx=0x7f74cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbfb14d3d26649e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:32.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74d77fe640 1 -- 172.21.15.80:0/3482318284 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0b0070 con 0x7f74d80d0ff0 2024-04-25T13:40:32.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74d77fe640 1 -- 172.21.15.80:0/3482318284 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74cc0ab600 con 0x7f74d80d0ff0 2024-04-25T13:40:32.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74d77fe640 1 -- 172.21.15.80:0/3482318284 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0ab9d0 con 0x7f74d80d0ff0 2024-04-25T13:40:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 -- 172.21.15.80:0/3482318284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 msgr2=0x7f74d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 --2- 172.21.15.80:0/3482318284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f74d80701f0 tx=0x7f74cc01c490 comp rx=0 tx=0).stop 2024-04-25T13:40:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 -- 172.21.15.80:0/3482318284 shutdown_connections 2024-04-25T13:40:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 --2- 172.21.15.80:0/3482318284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d8161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 --2- 172.21.15.80:0/3482318284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d806edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 --2- 172.21.15.80:0/3482318284 >> v2:172.21.15.87:3300/0 conn(0x7f74d80b5330 0x7f74d80d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 -- 172.21.15.80:0/3482318284 >> 172.21.15.80:0/3482318284 conn(0x7f74d80b4160 msgr2=0x7f74d80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 -- 172.21.15.80:0/3482318284 shutdown_connections 2024-04-25T13:40:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.380+0000 7f74df17e640 1 -- 172.21.15.80:0/3482318284 wait complete. 2024-04-25T13:40:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 Processor -- start 2024-04-25T13:40:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 -- start start 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d81578b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74d80b5330 0x7f74d815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d815abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f74d80b0cd0 con 0x7f74d80d0ff0 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f74d806e890 con 0x7f74d80b5330 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f74d80b0e50 con 0x7f74d806f2f0 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d815abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d815abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35940/0 (socket says 172.21.15.80:35940) 2024-04-25T13:40:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 -- 172.21.15.80:0/4080672813 learned_addr learned my addr 172.21.15.80:0/4080672813 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74d7fff640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74d80b5330 0x7f74d815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dcf1a640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d81578b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 -- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 msgr2=0x7f74d81578b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d81578b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 -- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74d80b5330 msgr2=0x7f74d815a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74d80b5330 0x7f74d815a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 -- 172.21.15.80:0/4080672813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d8159600 con 0x7f74d80d0ff0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74dd71b640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d815abe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f74d000ccc0 tx=0x7f74d000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74d57fa640 1 -- 172.21.15.80:0/4080672813 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d000f680 con 0x7f74d80d0ff0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74d57fa640 1 -- 172.21.15.80:0/4080672813 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74d000f820 con 0x7f74d80d0ff0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d81598e0 con 0x7f74d80d0ff0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74d57fa640 1 -- 172.21.15.80:0/4080672813 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d000fc10 con 0x7f74d80d0ff0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d8159ea0 con 0x7f74d80d0ff0 2024-04-25T13:40:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.384+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74d806a3a0 con 0x7f74d80d0ff0 2024-04-25T13:40:32.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.388+0000 7f74d57fa640 1 -- 172.21.15.80:0/4080672813 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f74d00213f0 con 0x7f74d80d0ff0 2024-04-25T13:40:32.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.388+0000 7f74d57fa640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f74c406f400 0x7f74c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:32.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.388+0000 7f74d57fa640 1 -- 172.21.15.80:0/4080672813 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f74d008ffc0 con 0x7f74d80d0ff0 2024-04-25T13:40:32.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.388+0000 7f74dcf1a640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f74c406f400 0x7f74c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:32.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.388+0000 7f74dcf1a640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f74c406f400 0x7f74c4071910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f74c800a8d0 tx=0x7f74c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:32.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.392+0000 7f74d57fa640 1 -- 172.21.15.80:0/4080672813 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f74d0014030 con 0x7f74d80d0ff0 2024-04-25T13:40:32.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:32.588+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f74d806fd00 con 0x7f74d80d0ff0 2024-04-25T13:40:33.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.212+0000 7f74d57fa640 1 -- 172.21.15.80:0/4080672813 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) ==== 122+0+0 (secure 0 0 0) 0x7f74d005ceb0 con 0x7f74d80d0ff0 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f74c406f400 msgr2=0x7f74c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f74c406f400 0x7f74c4071910 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f74c800a8d0 tx=0x7f74c8008040 comp rx=0 tx=0).stop 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 msgr2=0x7f74d815abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d815abe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f74d000ccc0 tx=0x7f74d000a650 comp rx=0 tx=0).stop 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 shutdown_connections 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f74c406f400 0x7f74c4071910 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f74d80d0ff0 0x7f74d815abe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f74d80b5330 0x7f74d815a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 --2- 172.21.15.80:0/4080672813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f74d806f2f0 0x7f74d81578b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 >> 172.21.15.80:0/4080672813 conn(0x7f74d80b4160 msgr2=0x7f74d80b2bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:33.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 shutdown_connections 2024-04-25T13:40:33.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.216+0000 7f74df17e640 1 -- 172.21.15.80:0/4080672813 wait complete. 2024-04-25T13:40:33.241 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:40:33.242 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:40:33.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1793cf6640 1 Processor -- start 2024-04-25T13:40:33.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1793cf6640 1 -- start start 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1793cf6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f178c161650 0x7f178c072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1793cf6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f178c06fcf0 con 0x7f178c06f5a0 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1793cf6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f178c0b0c60 con 0x7f178c161240 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1793cf6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f178c073f10 con 0x7f178c161650 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1792293640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f178c161650 0x7f178c072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1792293640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f178c161650 0x7f178c072ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35946/0 (socket says 172.21.15.80:35946) 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1791a92640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f178c06f5a0 0x7f178c160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35574/0 (socket says 172.21.15.80:35574) 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1792293640 1 -- 172.21.15.80:0/3597955130 learned_addr learned my addr 172.21.15.80:0/3597955130 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1792293640 1 -- 172.21.15.80:0/3597955130 >> v1:172.21.15.87:6789/0 conn(0x7f178c161240 legacy=0x7f178c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1792293640 1 -- 172.21.15.80:0/3597955130 >> v1:172.21.15.80:6789/0 conn(0x7f178c06f5a0 legacy=0x7f178c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1792293640 1 -- 172.21.15.80:0/3597955130 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c0b0f40 con 0x7f178c161650 2024-04-25T13:40:33.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.384+0000 7f1790a90640 1 -- 172.21.15.80:0/3597955130 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2973967709 0 0) 0x7f178c06fcf0 con 0x7f178c06f5a0 2024-04-25T13:40:33.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1792293640 1 --2- 172.21.15.80:0/3597955130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c161650 0x7f178c072ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1784009e50 tx=0x7f178401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de498aa9e350957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:33.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1790a90640 1 -- 172.21.15.80:0/3597955130 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17840a56e0 con 0x7f178c161650 2024-04-25T13:40:33.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1790a90640 1 -- 172.21.15.80:0/3597955130 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17840a5880 con 0x7f178c161650 2024-04-25T13:40:33.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1790a90640 1 -- 172.21.15.80:0/3597955130 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17840a5c50 con 0x7f178c161650 2024-04-25T13:40:33.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 -- 172.21.15.80:0/3597955130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c161650 msgr2=0x7f178c072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 --2- 172.21.15.80:0/3597955130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c161650 0x7f178c072ab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1784009e50 tx=0x7f178401c9a0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 -- 172.21.15.80:0/3597955130 shutdown_connections 2024-04-25T13:40:33.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 --2- 172.21.15.80:0/3597955130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c161650 0x7f178c072ab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 -- 172.21.15.80:0/3597955130 >> 172.21.15.80:0/3597955130 conn(0x7f178c06d080 msgr2=0x7f178c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:33.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 -- 172.21.15.80:0/3597955130 shutdown_connections 2024-04-25T13:40:33.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 -- 172.21.15.80:0/3597955130 wait complete. 2024-04-25T13:40:33.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 Processor -- start 2024-04-25T13:40:33.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.388+0000 7f1793cf6640 1 -- start start 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c06f5a0 0x7f178c13f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f178c161240 0x7f178c06ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178c161650 0x7f178c150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f178c16b270 con 0x7f178c06f5a0 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f178c16b0f0 con 0x7f178c161650 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f178c16b3f0 con 0x7f178c161240 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c06f5a0 0x7f178c13f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c06f5a0 0x7f178c13f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35956/0 (socket says 172.21.15.80:35956) 2024-04-25T13:40:33.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 -- 172.21.15.80:0/433963678 learned_addr learned my addr 172.21.15.80:0/433963678 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 -- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f178c161240 msgr2=0x7f178c06ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f178c161240 0x7f178c06ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 -- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178c161650 msgr2=0x7f178c150480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178c161650 0x7f178c150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 -- 172.21.15.80:0/433963678 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178c15fb60 con 0x7f178c06f5a0 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1791a92640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c06f5a0 0x7f178c13f340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f178800c410 tx=0x7f178800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f17827fc640 1 -- 172.21.15.80:0/433963678 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17880074e0 con 0x7f178c06f5a0 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f17827fc640 1 -- 172.21.15.80:0/433963678 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17880047f0 con 0x7f178c06f5a0 2024-04-25T13:40:33.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f17827fc640 1 -- 172.21.15.80:0/433963678 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1788007680 con 0x7f178c06f5a0 2024-04-25T13:40:33.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f178c15fe40 con 0x7f178c06f5a0 2024-04-25T13:40:33.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178c160400 con 0x7f178c06f5a0 2024-04-25T13:40:33.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178c1fa9d0 con 0x7f178c06f5a0 2024-04-25T13:40:33.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.392+0000 7f17827fc640 1 -- 172.21.15.80:0/433963678 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1788011070 con 0x7f178c06f5a0 2024-04-25T13:40:33.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.396+0000 7f17827fc640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177006f400 0x7f1770071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.396+0000 7f17827fc640 1 -- 172.21.15.80:0/433963678 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f178808eeb0 con 0x7f178c06f5a0 2024-04-25T13:40:33.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.396+0000 7f1791291640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177006f400 0x7f1770071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:33.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.396+0000 7f1791291640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177006f400 0x7f1770071910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f177c000f70 tx=0x7f177c0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:33.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.400+0000 7f17827fc640 1 -- 172.21.15.80:0/433963678 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f178805bda0 con 0x7f178c06f5a0 2024-04-25T13:40:33.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.584+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f178c06a3a0 con 0x7f178c06f5a0 2024-04-25T13:40:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.584+0000 7f17827fc640 1 -- 172.21.15.80:0/433963678 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f178805b6a0 con 0x7f178c06f5a0 2024-04-25T13:40:33.585 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:33.585 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:40:29.558137+0000","modified":"2024-04-25T13:40:32.595776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-25T13:40:33.585 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177006f400 msgr2=0x7f1770071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177006f400 0x7f1770071910 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f177c000f70 tx=0x7f177c0079c0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c06f5a0 msgr2=0x7f178c13f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c06f5a0 0x7f178c13f340 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f178800c410 tx=0x7f178800c8d0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 shutdown_connections 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177006f400 0x7f1770071910 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178c161650 0x7f178c150480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f178c161240 0x7f178c06ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 --2- 172.21.15.80:0/433963678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178c06f5a0 0x7f178c13f340 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 >> 172.21.15.80:0/433963678 conn(0x7f178c06d080 msgr2=0x7f178c0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:33.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 shutdown_connections 2024-04-25T13:40:33.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.588+0000 7f1793cf6640 1 -- 172.21.15.80:0/433963678 wait complete. 2024-04-25T13:40:33.604 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:33.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.744+0000 7fad95827640 1 Processor -- start 2024-04-25T13:40:33.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.744+0000 7fad95827640 1 -- start start 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad95827640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 0x7fad900d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad95827640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fad900d4330 0x7fad900d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad95827640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad900d6300 con 0x7fad900d2e80 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad95827640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fad90161060 con 0x7fad900d4330 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad95827640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad90161340 con 0x7fad900d59b0 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fad900d4330 0x7fad900d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8effd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fad900d2e80 0x7fad901607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35578/0 (socket says 172.21.15.80:35578) 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8effd640 1 -- 172.21.15.80:0/1923629192 learned_addr learned my addr 172.21.15.80:0/1923629192 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8e7fc640 1 --2- 172.21.15.80:0/1923629192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 0x7fad900d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8f7fe640 1 -- 172.21.15.80:0/1923629192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 msgr2=0x7fad900d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8f7fe640 1 --2- 172.21.15.80:0/1923629192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 0x7fad900d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8f7fe640 1 -- 172.21.15.80:0/1923629192 >> v1:172.21.15.80:6789/0 conn(0x7fad900d2e80 legacy=0x7fad901607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8f7fe640 1 -- 172.21.15.80:0/1923629192 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad900d2680 con 0x7fad900d4330 2024-04-25T13:40:33.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.748+0000 7fad8dffb640 1 -- 172.21.15.80:0/1923629192 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 209417314 0 0) 0x7fad880014a0 con 0x7fad900d2e80 2024-04-25T13:40:33.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad8f7fe640 1 --2- 172.21.15.80:0/1923629192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad900d4740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad800097c0 tx=0x7fad8001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b467ff35619f9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:33.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad8dffb640 1 -- 172.21.15.80:0/1923629192 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad800b1070 con 0x7fad900d4330 2024-04-25T13:40:33.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad8dffb640 1 -- 172.21.15.80:0/1923629192 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad800ac410 con 0x7fad900d4330 2024-04-25T13:40:33.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad8dffb640 1 -- 172.21.15.80:0/1923629192 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad800ac7e0 con 0x7fad900d4330 2024-04-25T13:40:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 -- 172.21.15.80:0/1923629192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 msgr2=0x7fad900d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 --2- 172.21.15.80:0/1923629192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad900d4740 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fad800097c0 tx=0x7fad8001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 -- 172.21.15.80:0/1923629192 shutdown_connections 2024-04-25T13:40:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 --2- 172.21.15.80:0/1923629192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad900d4740 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 --2- 172.21.15.80:0/1923629192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 0x7fad900d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 -- 172.21.15.80:0/1923629192 >> 172.21.15.80:0/1923629192 conn(0x7fad900737a0 msgr2=0x7fad9006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 -- 172.21.15.80:0/1923629192 shutdown_connections 2024-04-25T13:40:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 -- 172.21.15.80:0/1923629192 wait complete. 2024-04-25T13:40:33.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 Processor -- start 2024-04-25T13:40:33.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.752+0000 7fad95827640 1 -- start start 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad900d2e80 0x7fad900ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad90156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 0x7fad900ba270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad900d3410 con 0x7fad900d4330 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad900d3290 con 0x7fad900d2e80 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad9015df60 con 0x7fad900d59b0 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad90156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad90156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35968/0 (socket says 172.21.15.80:35968) 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 -- 172.21.15.80:0/4018585086 learned_addr learned my addr 172.21.15.80:0/4018585086 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8f7fe640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 0x7fad900ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 -- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 msgr2=0x7fad900ba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 0x7fad900ba270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 -- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad900d2e80 msgr2=0x7fad900ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad900d2e80 0x7fad900ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 -- 172.21.15.80:0/4018585086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad900c97e0 con 0x7fad900d4330 2024-04-25T13:40:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad8e7fc640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad90156dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad7800d5f0 tx=0x7fad7800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad87fff640 1 -- 172.21.15.80:0/4018585086 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780125f0 con 0x7fad900d4330 2024-04-25T13:40:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad87fff640 1 -- 172.21.15.80:0/4018585086 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad78019070 con 0x7fad900d4330 2024-04-25T13:40:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad87fff640 1 -- 172.21.15.80:0/4018585086 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad780129b0 con 0x7fad900d4330 2024-04-25T13:40:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad900cf7b0 con 0x7fad900d4330 2024-04-25T13:40:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad900c9d60 con 0x7fad900d4330 2024-04-25T13:40:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.756+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad900cf520 con 0x7fad900d4330 2024-04-25T13:40:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.760+0000 7fad87fff640 1 -- 172.21.15.80:0/4018585086 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad78012b50 con 0x7fad900d4330 2024-04-25T13:40:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.760+0000 7fad87fff640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad7006f430 0x7fad70071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.760+0000 7fad87fff640 1 -- 172.21.15.80:0/4018585086 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fad7808f640 con 0x7fad900d4330 2024-04-25T13:40:33.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.760+0000 7fad8effd640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad7006f430 0x7fad70071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:33.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.760+0000 7fad8effd640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad7006f430 0x7fad70071940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fad900b8fd0 tx=0x7fad88007cf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.764+0000 7fad87fff640 1 -- 172.21.15.80:0/4018585086 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad7805c5c0 con 0x7fad900d4330 2024-04-25T13:40:33.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.924+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad9006a3a0 con 0x7fad900d4330 2024-04-25T13:40:33.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad87fff640 1 -- 172.21.15.80:0/4018585086 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7fad7805bec0 con 0x7fad900d4330 2024-04-25T13:40:33.927 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:33.927 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:33.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad7006f430 msgr2=0x7fad70071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad7006f430 0x7fad70071940 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fad900b8fd0 tx=0x7fad88007cf0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 msgr2=0x7fad90156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:33.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad90156dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fad7800d5f0 tx=0x7fad7800dab0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 shutdown_connections 2024-04-25T13:40:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad7006f430 0x7fad70071940 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad900d59b0 0x7fad900ba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad900d4330 0x7fad90156dc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 --2- 172.21.15.80:0/4018585086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad900d2e80 0x7fad900ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 >> 172.21.15.80:0/4018585086 conn(0x7fad900737a0 msgr2=0x7fad9015a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 shutdown_connections 2024-04-25T13:40:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:33.928+0000 7fad95827640 1 -- 172.21.15.80:0/4018585086 wait complete. 2024-04-25T13:40:33.947 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:40:33.947 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:40:33.984 INFO:tasks.ceph.mds.d.smithi087.stderr:2024-04-25T13:40:33.983+0000 7f32083a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:40:34.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.096+0000 7f654a40f640 1 Processor -- start 2024-04-25T13:40:34.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.096+0000 7f654a40f640 1 -- start start 2024-04-25T13:40:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f654a40f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f654406fb30 0x7f654415d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f654a40f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f654415dbc0 0x7f65440d00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f654a40f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f654415e420 con 0x7f654415acc0 2024-04-25T13:40:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f654a40f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f65440ce540 con 0x7f654415dbc0 2024-04-25T13:40:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f654a40f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65440ce790 con 0x7f654406fb30 2024-04-25T13:40:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6543fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f654415acc0 0x7f654406f420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35594/0 (socket says 172.21.15.80:35594) 2024-04-25T13:40:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6543fff640 1 -- 172.21.15.80:0/19129356 learned_addr learned my addr 172.21.15.80:0/19129356 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126938796 0 0) 0x7f654415e420 con 0x7f654415acc0 2024-04-25T13:40:34.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f652c003530 con 0x7f654415acc0 2024-04-25T13:40:34.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f65489ac640 1 --2- 172.21.15.80:0/19129356 >> v2:172.21.15.87:3300/0 conn(0x7f654415dbc0 0x7f65440d00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f65437fe640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f654406fb30 0x7f654415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3047734306 0 0) 0x7f652c003530 con 0x7f654415acc0 2024-04-25T13:40:34.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f654406fb30 msgr2=0x7f654415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f654406fb30 0x7f654415d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 >> v2:172.21.15.87:3300/0 conn(0x7f654415dbc0 msgr2=0x7f65440d00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 --2- 172.21.15.80:0/19129356 >> v2:172.21.15.87:3300/0 conn(0x7f654415dbc0 0x7f65440d00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65440cea70 con 0x7f654415acc0 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f6534001680 con 0x7f654415acc0 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 >> v1:172.21.15.80:6789/0 conn(0x7f654415acc0 legacy=0x7f654406f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f652c0039c0 0x7f652c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c004810 0x7f652c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f652c005160 0x7f652c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f65440ce540 con 0x7f652c0039c0 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65440ce790 con 0x7f652c004810 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f652c00bf10 con 0x7f652c005160 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6543fff640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f652c0039c0 0x7f652c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f65437fe640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c004810 0x7f652c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f65489ac640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f652c005160 0x7f652c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6543fff640 1 -- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f652c005160 msgr2=0x7f652c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6543fff640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f652c005160 0x7f652c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6543fff640 1 -- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c004810 msgr2=0x7f652c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6543fff640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c004810 0x7f652c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f6543fff640 1 -- 172.21.15.80:0/19129356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f652c003530 con 0x7f652c0039c0 2024-04-25T13:40:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.100+0000 7f65437fe640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c004810 0x7f652c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f6543fff640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f652c0039c0 0x7f652c003e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65440746e0 tx=0x7f653401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871b9d4daf7c7708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534007d60 con 0x7f652c0039c0 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65340a9020 con 0x7f652c0039c0 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f6542ffd640 1 -- 172.21.15.80:0/19129356 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534005b20 con 0x7f652c0039c0 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f654a40f640 1 -- 172.21.15.80:0/19129356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f652c0039c0 msgr2=0x7f652c003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f654a40f640 1 --2- 172.21.15.80:0/19129356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f652c0039c0 0x7f652c003e50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f65440746e0 tx=0x7f653401cc60 comp rx=0 tx=0).stop 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f65489ac640 1 -- 172.21.15.80:0/19129356 reap_dead start 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f654a40f640 1 -- 172.21.15.80:0/19129356 shutdown_connections 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f654a40f640 1 -- 172.21.15.80:0/19129356 >> 172.21.15.80:0/19129356 conn(0x7f65440d1d10 msgr2=0x7f65440d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f654a40f640 1 -- 172.21.15.80:0/19129356 shutdown_connections 2024-04-25T13:40:34.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f654a40f640 1 -- 172.21.15.80:0/19129356 wait complete. 2024-04-25T13:40:34.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f654a40f640 1 Processor -- start 2024-04-25T13:40:34.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.104+0000 7f654a40f640 1 -- start start 2024-04-25T13:40:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c0039c0 0x7f6544150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65441566d0 0x7f6544156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544159c40 0x7f654415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f654406fb30 con 0x7f6544159c40 2024-04-25T13:40:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f654415e1e0 con 0x7f652c0039c0 2024-04-25T13:40:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f654406fcb0 con 0x7f65441566d0 2024-04-25T13:40:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544159c40 0x7f654415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544159c40 0x7f654415a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35986/0 (socket says 172.21.15.80:35986) 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 -- 172.21.15.80:0/3061746653 learned_addr learned my addr 172.21.15.80:0/3061746653 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f6543fff640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c0039c0 0x7f6544150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65437fe640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65441566d0 0x7f6544156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 -- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65441566d0 msgr2=0x7f6544156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65441566d0 0x7f6544156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 -- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c0039c0 msgr2=0x7f6544150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c0039c0 0x7f6544150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 -- 172.21.15.80:0/3061746653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654415a590 con 0x7f6544159c40 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f65489ac640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544159c40 0x7f654415a050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65440cfae0 tx=0x7f653c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f6540ff9640 1 -- 172.21.15.80:0/3061746653 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c016420 con 0x7f6544159c40 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f6540ff9640 1 -- 172.21.15.80:0/3061746653 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f653c01b070 con 0x7f6544159c40 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f6540ff9640 1 -- 172.21.15.80:0/3061746653 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c016820 con 0x7f6544159c40 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65441f7670 con 0x7f6544159c40 2024-04-25T13:40:34.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65441f7c30 con 0x7f6544159c40 2024-04-25T13:40:34.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.108+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65440d2b40 con 0x7f6544159c40 2024-04-25T13:40:34.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.112+0000 7f6540ff9640 1 -- 172.21.15.80:0/3061746653 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f653c0169c0 con 0x7f6544159c40 2024-04-25T13:40:34.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.112+0000 7f6540ff9640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f653006f400 0x7f6530071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.112+0000 7f6540ff9640 1 -- 172.21.15.80:0/3061746653 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f653c090090 con 0x7f6544159c40 2024-04-25T13:40:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.112+0000 7f6543fff640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f653006f400 0x7f6530071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.112+0000 7f6543fff640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f653006f400 0x7f6530071910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65340aa000 tx=0x7f6534000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.116+0000 7f6540ff9640 1 -- 172.21.15.80:0/3061746653 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f653c025020 con 0x7f6544159c40 2024-04-25T13:40:34.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.264+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f65441f71b0 con 0x7f653006f400 2024-04-25T13:40:34.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.296+0000 7f6540ff9640 1 -- 172.21.15.80:0/3061746653 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247472 (secure 0 0 0) 0x7f6534005c40 con 0x7f653006f400 2024-04-25T13:40:34.299 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:34.300 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:40:34.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f653006f400 msgr2=0x7f6530071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f653006f400 0x7f6530071910 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65340aa000 tx=0x7f6534000ff0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544159c40 msgr2=0x7f654415a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544159c40 0x7f654415a050 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f65440cfae0 tx=0x7f653c00f520 comp rx=0 tx=0).stop 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 shutdown_connections 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f653006f400 0x7f6530071910 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6544159c40 0x7f654415a050 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65441566d0 0x7f6544156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 --2- 172.21.15.80:0/3061746653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f652c0039c0 0x7f6544150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 >> 172.21.15.80:0/3061746653 conn(0x7f65440d1d10 msgr2=0x7f6544168460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 shutdown_connections 2024-04-25T13:40:34.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.300+0000 7f654a40f640 1 -- 172.21.15.80:0/3061746653 wait complete. 2024-04-25T13:40:34.318 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-25T13:40:34.092573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.942372"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348886+0000","last_change":"2024-04-25T13:40:30.221428+0000","last_active":"2024-04-25T13:40:31.348886+0000","last_peered":"2024-04-25T13:40:31.348886+0000","last_clean":"2024-04-25T13:40:31.348886+0000","last_became_active":"2024-04-25T13:40:30.219271+0000","last_became_peered":"2024-04-25T13:40:30.219271+0000","last_unstale":"2024-04-25T13:40:31.348886+0000","last_undegraded":"2024-04-25T13:40:31.348886+0000","last_fullsized":"2024-04-25T13:40:31.348886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.180832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184646+0000","last_change":"2024-04-25T13:40:30.211459+0000","last_active":"2024-04-25T13:40:31.184646+0000","last_peered":"2024-04-25T13:40:31.184646+0000","last_clean":"2024-04-25T13:40:31.184646+0000","last_became_active":"2024-04-25T13:40:30.211073+0000","last_became_peered":"2024-04-25T13:40:30.211073+0000","last_unstale":"2024-04-25T13:40:31.184646+0000","last_undegraded":"2024-04-25T13:40:31.184646+0000","last_fullsized":"2024-04-25T13:40:31.184646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:13.973526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346644+0000","last_change":"2024-04-25T13:40:30.213483+0000","last_active":"2024-04-25T13:40:31.346644+0000","last_peered":"2024-04-25T13:40:31.346644+0000","last_clean":"2024-04-25T13:40:31.346644+0000","last_became_active":"2024-04-25T13:40:30.207846+0000","last_became_peered":"2024-04-25T13:40:30.207846+0000","last_unstale":"2024-04-25T13:40:31.346644+0000","last_undegraded":"2024-04-25T13:40:31.346644+0000","last_fullsized":"2024-04-25T13:40:31.346644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:10.391107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.345909+0000","last_change":"2024-04-25T13:40:30.216562+0000","last_active":"2024-04-25T13:40:31.345909+0000","last_peered":"2024-04-25T13:40:31.345909+0000","last_clean":"2024-04-25T13:40:31.345909+0000","last_became_active":"2024-04-25T13:40:30.215696+0000","last_became_peered":"2024-04-25T13:40:30.215696+0000","last_unstale":"2024-04-25T13:40:31.345909+0000","last_undegraded":"2024-04-25T13:40:31.345909+0000","last_fullsized":"2024-04-25T13:40:31.345909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:42.952152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347520+0000","last_change":"2024-04-25T13:40:30.207884+0000","last_active":"2024-04-25T13:40:31.347520+0000","last_peered":"2024-04-25T13:40:31.347520+0000","last_clean":"2024-04-25T13:40:31.347520+0000","last_became_active":"2024-04-25T13:40:30.207080+0000","last_became_peered":"2024-04-25T13:40:30.207080+0000","last_unstale":"2024-04-25T13:40:31.347520+0000","last_undegraded":"2024-04-25T13:40:31.347520+0000","last_fullsized":"2024-04-25T13:40:31.347520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:07.164518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347177+0000","last_change":"2024-04-25T13:40:30.221017+0000","last_active":"2024-04-25T13:40:31.347177+0000","last_peered":"2024-04-25T13:40:31.347177+0000","last_clean":"2024-04-25T13:40:31.347177+0000","last_became_active":"2024-04-25T13:40:30.219333+0000","last_became_peered":"2024-04-25T13:40:30.219333+0000","last_unstale":"2024-04-25T13:40:31.347177+0000","last_undegraded":"2024-04-25T13:40:31.347177+0000","last_fullsized":"2024-04-25T13:40:31.347177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:29.428442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349282+0000","last_change":"2024-04-25T13:40:30.216405+0000","last_active":"2024-04-25T13:40:31.349282+0000","last_peered":"2024-04-25T13:40:31.349282+0000","last_clean":"2024-04-25T13:40:31.349282+0000","last_became_active":"2024-04-25T13:40:30.215354+0000","last_became_peered":"2024-04-25T13:40:30.215354+0000","last_unstale":"2024-04-25T13:40:31.349282+0000","last_undegraded":"2024-04-25T13:40:31.349282+0000","last_fullsized":"2024-04-25T13:40:31.349282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:22.813674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346103+0000","last_change":"2024-04-25T13:40:30.207820+0000","last_active":"2024-04-25T13:40:31.346103+0000","last_peered":"2024-04-25T13:40:31.346103+0000","last_clean":"2024-04-25T13:40:31.346103+0000","last_became_active":"2024-04-25T13:40:30.207148+0000","last_became_peered":"2024-04-25T13:40:30.207148+0000","last_unstale":"2024-04-25T13:40:31.346103+0000","last_undegraded":"2024-04-25T13:40:31.346103+0000","last_fullsized":"2024-04-25T13:40:31.346103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:05.730270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346696+0000","last_change":"2024-04-25T13:40:30.219574+0000","last_active":"2024-04-25T13:40:31.346696+0000","last_peered":"2024-04-25T13:40:31.346696+0000","last_clean":"2024-04-25T13:40:31.346696+0000","last_became_active":"2024-04-25T13:40:30.216463+0000","last_became_peered":"2024-04-25T13:40:30.216463+0000","last_unstale":"2024-04-25T13:40:31.346696+0000","last_undegraded":"2024-04-25T13:40:31.346696+0000","last_fullsized":"2024-04-25T13:40:31.346696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:33.974714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219726+0000","last_change":"2024-04-25T13:40:30.202005+0000","last_active":"2024-04-25T13:40:31.219726+0000","last_peered":"2024-04-25T13:40:31.219726+0000","last_clean":"2024-04-25T13:40:31.219726+0000","last_became_active":"2024-04-25T13:40:30.201195+0000","last_became_peered":"2024-04-25T13:40:30.201195+0000","last_unstale":"2024-04-25T13:40:31.219726+0000","last_undegraded":"2024-04-25T13:40:31.219726+0000","last_fullsized":"2024-04-25T13:40:31.219726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:20.543637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219380+0000","last_change":"2024-04-25T13:40:30.221626+0000","last_active":"2024-04-25T13:40:31.219380+0000","last_peered":"2024-04-25T13:40:31.219380+0000","last_clean":"2024-04-25T13:40:31.219380+0000","last_became_active":"2024-04-25T13:40:30.221294+0000","last_became_peered":"2024-04-25T13:40:30.221294+0000","last_unstale":"2024-04-25T13:40:31.219380+0000","last_undegraded":"2024-04-25T13:40:31.219380+0000","last_fullsized":"2024-04-25T13:40:31.219380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:49.053408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348097+0000","last_change":"2024-04-25T13:40:30.199635+0000","last_active":"2024-04-25T13:40:31.348097+0000","last_peered":"2024-04-25T13:40:31.348097+0000","last_clean":"2024-04-25T13:40:31.348097+0000","last_became_active":"2024-04-25T13:40:30.198748+0000","last_became_peered":"2024-04-25T13:40:30.198748+0000","last_unstale":"2024-04-25T13:40:31.348097+0000","last_undegraded":"2024-04-25T13:40:31.348097+0000","last_fullsized":"2024-04-25T13:40:31.348097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346436+0000","last_change":"2024-04-25T13:40:30.215629+0000","last_active":"2024-04-25T13:40:31.346436+0000","last_peered":"2024-04-25T13:40:31.346436+0000","last_clean":"2024-04-25T13:40:31.346436+0000","last_became_active":"2024-04-25T13:40:30.214870+0000","last_became_peered":"2024-04-25T13:40:30.214870+0000","last_unstale":"2024-04-25T13:40:31.346436+0000","last_undegraded":"2024-04-25T13:40:31.346436+0000","last_fullsized":"2024-04-25T13:40:31.346436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:21.817644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346237+0000","last_change":"2024-04-25T13:40:30.200380+0000","last_active":"2024-04-25T13:40:31.346237+0000","last_peered":"2024-04-25T13:40:31.346237+0000","last_clean":"2024-04-25T13:40:31.346237+0000","last_became_active":"2024-04-25T13:40:30.198804+0000","last_became_peered":"2024-04-25T13:40:30.198804+0000","last_unstale":"2024-04-25T13:40:31.346237+0000","last_undegraded":"2024-04-25T13:40:31.346237+0000","last_fullsized":"2024-04-25T13:40:31.346237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:02.090517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185879+0000","last_change":"2024-04-25T13:40:30.228782+0000","last_active":"2024-04-25T13:40:31.185879+0000","last_peered":"2024-04-25T13:40:31.185879+0000","last_clean":"2024-04-25T13:40:31.185879+0000","last_became_active":"2024-04-25T13:40:30.228329+0000","last_became_peered":"2024-04-25T13:40:30.228329+0000","last_unstale":"2024-04-25T13:40:31.185879+0000","last_undegraded":"2024-04-25T13:40:31.185879+0000","last_fullsized":"2024-04-25T13:40:31.185879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:39.981195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346736+0000","last_change":"2024-04-25T13:40:30.221267+0000","last_active":"2024-04-25T13:40:31.346736+0000","last_peered":"2024-04-25T13:40:31.346736+0000","last_clean":"2024-04-25T13:40:31.346736+0000","last_became_active":"2024-04-25T13:40:30.219761+0000","last_became_peered":"2024-04-25T13:40:30.219761+0000","last_unstale":"2024-04-25T13:40:31.346736+0000","last_undegraded":"2024-04-25T13:40:31.346736+0000","last_fullsized":"2024-04-25T13:40:31.346736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:16.273351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346948+0000","last_change":"2024-04-25T13:40:30.225471+0000","last_active":"2024-04-25T13:40:31.346948+0000","last_peered":"2024-04-25T13:40:31.346948+0000","last_clean":"2024-04-25T13:40:31.346948+0000","last_became_active":"2024-04-25T13:40:30.224161+0000","last_became_peered":"2024-04-25T13:40:30.224161+0000","last_unstale":"2024-04-25T13:40:31.346948+0000","last_undegraded":"2024-04-25T13:40:31.346948+0000","last_fullsized":"2024-04-25T13:40:31.346948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:16.868301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186576+0000","last_change":"2024-04-25T13:40:30.204030+0000","last_active":"2024-04-25T13:40:31.186576+0000","last_peered":"2024-04-25T13:40:31.186576+0000","last_clean":"2024-04-25T13:40:31.186576+0000","last_became_active":"2024-04-25T13:40:30.203298+0000","last_became_peered":"2024-04-25T13:40:30.203298+0000","last_unstale":"2024-04-25T13:40:31.186576+0000","last_undegraded":"2024-04-25T13:40:31.186576+0000","last_fullsized":"2024-04-25T13:40:31.186576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.572150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347790+0000","last_change":"2024-04-25T13:40:30.220004+0000","last_active":"2024-04-25T13:40:31.347790+0000","last_peered":"2024-04-25T13:40:31.347790+0000","last_clean":"2024-04-25T13:40:31.347790+0000","last_became_active":"2024-04-25T13:40:30.217606+0000","last_became_peered":"2024-04-25T13:40:30.217606+0000","last_unstale":"2024-04-25T13:40:31.347790+0000","last_undegraded":"2024-04-25T13:40:31.347790+0000","last_fullsized":"2024-04-25T13:40:31.347790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:48.795741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220183+0000","last_change":"2024-04-25T13:40:30.228676+0000","last_active":"2024-04-25T13:40:31.220183+0000","last_peered":"2024-04-25T13:40:31.220183+0000","last_clean":"2024-04-25T13:40:31.220183+0000","last_became_active":"2024-04-25T13:40:30.228364+0000","last_became_peered":"2024-04-25T13:40:30.228364+0000","last_unstale":"2024-04-25T13:40:31.220183+0000","last_undegraded":"2024-04-25T13:40:31.220183+0000","last_fullsized":"2024-04-25T13:40:31.220183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:09.381688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347827+0000","last_change":"2024-04-25T13:40:30.220330+0000","last_active":"2024-04-25T13:40:31.347827+0000","last_peered":"2024-04-25T13:40:31.347827+0000","last_clean":"2024-04-25T13:40:31.347827+0000","last_became_active":"2024-04-25T13:40:30.219702+0000","last_became_peered":"2024-04-25T13:40:30.219702+0000","last_unstale":"2024-04-25T13:40:31.347827+0000","last_undegraded":"2024-04-25T13:40:31.347827+0000","last_fullsized":"2024-04-25T13:40:31.347827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:24.416759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349924+0000","last_change":"2024-04-25T13:40:30.224681+0000","last_active":"2024-04-25T13:40:31.349924+0000","last_peered":"2024-04-25T13:40:31.349924+0000","last_clean":"2024-04-25T13:40:31.349924+0000","last_became_active":"2024-04-25T13:40:30.218495+0000","last_became_peered":"2024-04-25T13:40:30.218495+0000","last_unstale":"2024-04-25T13:40:31.349924+0000","last_undegraded":"2024-04-25T13:40:31.349924+0000","last_fullsized":"2024-04-25T13:40:31.349924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:21.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347112+0000","last_change":"2024-04-25T13:40:30.220292+0000","last_active":"2024-04-25T13:40:31.347112+0000","last_peered":"2024-04-25T13:40:31.347112+0000","last_clean":"2024-04-25T13:40:31.347112+0000","last_became_active":"2024-04-25T13:40:30.219473+0000","last_became_peered":"2024-04-25T13:40:30.219473+0000","last_unstale":"2024-04-25T13:40:31.347112+0000","last_undegraded":"2024-04-25T13:40:31.347112+0000","last_fullsized":"2024-04-25T13:40:31.347112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.322464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346401+0000","last_change":"2024-04-25T13:40:30.216234+0000","last_active":"2024-04-25T13:40:31.346401+0000","last_peered":"2024-04-25T13:40:31.346401+0000","last_clean":"2024-04-25T13:40:31.346401+0000","last_became_active":"2024-04-25T13:40:30.215246+0000","last_became_peered":"2024-04-25T13:40:30.215246+0000","last_unstale":"2024-04-25T13:40:31.346401+0000","last_undegraded":"2024-04-25T13:40:31.346401+0000","last_fullsized":"2024-04-25T13:40:31.346401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:09.558215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347524+0000","last_change":"2024-04-25T13:40:30.226582+0000","last_active":"2024-04-25T13:40:31.347524+0000","last_peered":"2024-04-25T13:40:31.347524+0000","last_clean":"2024-04-25T13:40:31.347524+0000","last_became_active":"2024-04-25T13:40:30.226077+0000","last_became_peered":"2024-04-25T13:40:30.226077+0000","last_unstale":"2024-04-25T13:40:31.347524+0000","last_undegraded":"2024-04-25T13:40:31.347524+0000","last_fullsized":"2024-04-25T13:40:31.347524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.951509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219854+0000","last_change":"2024-04-25T13:40:30.220604+0000","last_active":"2024-04-25T13:40:31.219854+0000","last_peered":"2024-04-25T13:40:31.219854+0000","last_clean":"2024-04-25T13:40:31.219854+0000","last_became_active":"2024-04-25T13:40:30.220289+0000","last_became_peered":"2024-04-25T13:40:30.220289+0000","last_unstale":"2024-04-25T13:40:31.219854+0000","last_undegraded":"2024-04-25T13:40:31.219854+0000","last_fullsized":"2024-04-25T13:40:31.219854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:54.152591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184983+0000","last_change":"2024-04-25T13:40:30.204488+0000","last_active":"2024-04-25T13:40:31.184983+0000","last_peered":"2024-04-25T13:40:31.184983+0000","last_clean":"2024-04-25T13:40:31.184983+0000","last_became_active":"2024-04-25T13:40:30.203206+0000","last_became_peered":"2024-04-25T13:40:30.203206+0000","last_unstale":"2024-04-25T13:40:31.184983+0000","last_undegraded":"2024-04-25T13:40:31.184983+0000","last_fullsized":"2024-04-25T13:40:31.184983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:07.299339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346652+0000","last_change":"2024-04-25T13:40:30.219870+0000","last_active":"2024-04-25T13:40:31.346652+0000","last_peered":"2024-04-25T13:40:31.346652+0000","last_clean":"2024-04-25T13:40:31.346652+0000","last_became_active":"2024-04-25T13:40:30.218894+0000","last_became_peered":"2024-04-25T13:40:30.218894+0000","last_unstale":"2024-04-25T13:40:31.346652+0000","last_undegraded":"2024-04-25T13:40:31.346652+0000","last_fullsized":"2024-04-25T13:40:31.346652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:04.359710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346438+0000","last_change":"2024-04-25T13:40:30.212775+0000","last_active":"2024-04-25T13:40:31.346438+0000","last_peered":"2024-04-25T13:40:31.346438+0000","last_clean":"2024-04-25T13:40:31.346438+0000","last_became_active":"2024-04-25T13:40:30.212382+0000","last_became_peered":"2024-04-25T13:40:30.212382+0000","last_unstale":"2024-04-25T13:40:31.346438+0000","last_undegraded":"2024-04-25T13:40:31.346438+0000","last_fullsized":"2024-04-25T13:40:31.346438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.132012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347837+0000","last_change":"2024-04-25T13:40:30.213875+0000","last_active":"2024-04-25T13:40:31.347837+0000","last_peered":"2024-04-25T13:40:31.347837+0000","last_clean":"2024-04-25T13:40:31.347837+0000","last_became_active":"2024-04-25T13:40:30.213530+0000","last_became_peered":"2024-04-25T13:40:30.213530+0000","last_unstale":"2024-04-25T13:40:31.347837+0000","last_undegraded":"2024-04-25T13:40:31.347837+0000","last_fullsized":"2024-04-25T13:40:31.347837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:16.067584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186661+0000","last_change":"2024-04-25T13:40:30.221392+0000","last_active":"2024-04-25T13:40:31.186661+0000","last_peered":"2024-04-25T13:40:31.186661+0000","last_clean":"2024-04-25T13:40:31.186661+0000","last_became_active":"2024-04-25T13:40:30.221056+0000","last_became_peered":"2024-04-25T13:40:30.221056+0000","last_unstale":"2024-04-25T13:40:31.186661+0000","last_undegraded":"2024-04-25T13:40:31.186661+0000","last_fullsized":"2024-04-25T13:40:31.186661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:54.209154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219551+0000","last_change":"2024-04-25T13:40:30.211302+0000","last_active":"2024-04-25T13:40:31.219551+0000","last_peered":"2024-04-25T13:40:31.219551+0000","last_clean":"2024-04-25T13:40:31.219551+0000","last_became_active":"2024-04-25T13:40:30.210970+0000","last_became_peered":"2024-04-25T13:40:30.210970+0000","last_unstale":"2024-04-25T13:40:31.219551+0000","last_undegraded":"2024-04-25T13:40:31.219551+0000","last_fullsized":"2024-04-25T13:40:31.219551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:19.633269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349550+0000","last_change":"2024-04-25T13:40:30.201358+0000","last_active":"2024-04-25T13:40:31.349550+0000","last_peered":"2024-04-25T13:40:31.349550+0000","last_clean":"2024-04-25T13:40:31.349550+0000","last_became_active":"2024-04-25T13:40:30.200493+0000","last_became_peered":"2024-04-25T13:40:30.200493+0000","last_unstale":"2024-04-25T13:40:31.349550+0000","last_undegraded":"2024-04-25T13:40:31.349550+0000","last_fullsized":"2024-04-25T13:40:31.349550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:38.778259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347724+0000","last_change":"2024-04-25T13:40:29.193901+0000","last_active":"2024-04-25T13:40:31.347724+0000","last_peered":"2024-04-25T13:40:31.347724+0000","last_clean":"2024-04-25T13:40:31.347724+0000","last_became_active":"2024-04-25T13:40:29.193719+0000","last_became_peered":"2024-04-25T13:40:29.193719+0000","last_unstale":"2024-04-25T13:40:31.347724+0000","last_undegraded":"2024-04-25T13:40:31.347724+0000","last_fullsized":"2024-04-25T13:40:31.347724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:57.372209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347538+0000","last_change":"2024-04-25T13:40:30.186468+0000","last_active":"2024-04-25T13:40:31.347538+0000","last_peered":"2024-04-25T13:40:31.347538+0000","last_clean":"2024-04-25T13:40:31.347538+0000","last_became_active":"2024-04-25T13:40:30.185965+0000","last_became_peered":"2024-04-25T13:40:30.185965+0000","last_unstale":"2024-04-25T13:40:31.347538+0000","last_undegraded":"2024-04-25T13:40:31.347538+0000","last_fullsized":"2024-04-25T13:40:31.347538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:42.249844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220247+0000","last_change":"2024-04-25T13:40:29.185827+0000","last_active":"2024-04-25T13:40:31.220247+0000","last_peered":"2024-04-25T13:40:31.220247+0000","last_clean":"2024-04-25T13:40:31.220247+0000","last_became_active":"2024-04-25T13:40:29.185130+0000","last_became_peered":"2024-04-25T13:40:29.185130+0000","last_unstale":"2024-04-25T13:40:31.220247+0000","last_undegraded":"2024-04-25T13:40:31.220247+0000","last_fullsized":"2024-04-25T13:40:31.220247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:46.813189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347960+0000","last_change":"2024-04-25T13:40:29.175272+0000","last_active":"2024-04-25T13:40:31.347960+0000","last_peered":"2024-04-25T13:40:31.347960+0000","last_clean":"2024-04-25T13:40:31.347960+0000","last_became_active":"2024-04-25T13:40:29.174962+0000","last_became_peered":"2024-04-25T13:40:29.174962+0000","last_unstale":"2024-04-25T13:40:31.347960+0000","last_undegraded":"2024-04-25T13:40:31.347960+0000","last_fullsized":"2024-04-25T13:40:31.347960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:14.296130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347747+0000","last_change":"2024-04-25T13:40:30.214155+0000","last_active":"2024-04-25T13:40:31.347747+0000","last_peered":"2024-04-25T13:40:31.347747+0000","last_clean":"2024-04-25T13:40:31.347747+0000","last_became_active":"2024-04-25T13:40:30.212880+0000","last_became_peered":"2024-04-25T13:40:30.212880+0000","last_unstale":"2024-04-25T13:40:31.347747+0000","last_undegraded":"2024-04-25T13:40:31.347747+0000","last_fullsized":"2024-04-25T13:40:31.347747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:17.642730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349764+0000","last_change":"2024-04-25T13:40:29.185537+0000","last_active":"2024-04-25T13:40:31.349764+0000","last_peered":"2024-04-25T13:40:31.349764+0000","last_clean":"2024-04-25T13:40:31.349764+0000","last_became_active":"2024-04-25T13:40:29.182804+0000","last_became_peered":"2024-04-25T13:40:29.182804+0000","last_unstale":"2024-04-25T13:40:31.349764+0000","last_undegraded":"2024-04-25T13:40:31.349764+0000","last_fullsized":"2024-04-25T13:40:31.349764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:45.820954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184917+0000","last_change":"2024-04-25T13:40:30.217108+0000","last_active":"2024-04-25T13:40:31.184917+0000","last_peered":"2024-04-25T13:40:31.184917+0000","last_clean":"2024-04-25T13:40:31.184917+0000","last_became_active":"2024-04-25T13:40:30.216260+0000","last_became_peered":"2024-04-25T13:40:30.216260+0000","last_unstale":"2024-04-25T13:40:31.184917+0000","last_undegraded":"2024-04-25T13:40:31.184917+0000","last_fullsized":"2024-04-25T13:40:31.184917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:23.045755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220246+0000","last_change":"2024-04-25T13:40:29.184744+0000","last_active":"2024-04-25T13:40:31.220246+0000","last_peered":"2024-04-25T13:40:31.220246+0000","last_clean":"2024-04-25T13:40:31.220246+0000","last_became_active":"2024-04-25T13:40:29.183985+0000","last_became_peered":"2024-04-25T13:40:29.183985+0000","last_unstale":"2024-04-25T13:40:31.220246+0000","last_undegraded":"2024-04-25T13:40:31.220246+0000","last_fullsized":"2024-04-25T13:40:31.220246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:56.228309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347453+0000","last_change":"2024-04-25T13:40:30.225124+0000","last_active":"2024-04-25T13:40:31.347453+0000","last_peered":"2024-04-25T13:40:31.347453+0000","last_clean":"2024-04-25T13:40:31.347453+0000","last_became_active":"2024-04-25T13:40:30.223455+0000","last_became_peered":"2024-04-25T13:40:30.223455+0000","last_unstale":"2024-04-25T13:40:31.347453+0000","last_undegraded":"2024-04-25T13:40:31.347453+0000","last_fullsized":"2024-04-25T13:40:31.347453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:03.877694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350848+0000","last_change":"2024-04-25T13:40:29.190850+0000","last_active":"2024-04-25T13:40:31.350848+0000","last_peered":"2024-04-25T13:40:31.350848+0000","last_clean":"2024-04-25T13:40:31.350848+0000","last_became_active":"2024-04-25T13:40:29.188098+0000","last_became_peered":"2024-04-25T13:40:29.188098+0000","last_unstale":"2024-04-25T13:40:31.350848+0000","last_undegraded":"2024-04-25T13:40:31.350848+0000","last_fullsized":"2024-04-25T13:40:31.350848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:07.915234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347972+0000","last_change":"2024-04-25T13:40:30.221083+0000","last_active":"2024-04-25T13:40:31.347972+0000","last_peered":"2024-04-25T13:40:31.347972+0000","last_clean":"2024-04-25T13:40:31.347972+0000","last_became_active":"2024-04-25T13:40:30.220819+0000","last_became_peered":"2024-04-25T13:40:30.220819+0000","last_unstale":"2024-04-25T13:40:31.347972+0000","last_undegraded":"2024-04-25T13:40:31.347972+0000","last_fullsized":"2024-04-25T13:40:31.347972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:43.306079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185190+0000","last_change":"2024-04-25T13:40:29.189173+0000","last_active":"2024-04-25T13:40:31.185190+0000","last_peered":"2024-04-25T13:40:31.185190+0000","last_clean":"2024-04-25T13:40:31.185190+0000","last_became_active":"2024-04-25T13:40:29.188914+0000","last_became_peered":"2024-04-25T13:40:29.188914+0000","last_unstale":"2024-04-25T13:40:31.185190+0000","last_undegraded":"2024-04-25T13:40:31.185190+0000","last_fullsized":"2024-04-25T13:40:31.185190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:12.374973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346508+0000","last_change":"2024-04-25T13:40:30.218953+0000","last_active":"2024-04-25T13:40:31.346508+0000","last_peered":"2024-04-25T13:40:31.346508+0000","last_clean":"2024-04-25T13:40:31.346508+0000","last_became_active":"2024-04-25T13:40:30.218568+0000","last_became_peered":"2024-04-25T13:40:30.218568+0000","last_unstale":"2024-04-25T13:40:31.346508+0000","last_undegraded":"2024-04-25T13:40:31.346508+0000","last_fullsized":"2024-04-25T13:40:31.346508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:02.608556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219913+0000","last_change":"2024-04-25T13:40:29.175817+0000","last_active":"2024-04-25T13:40:31.219913+0000","last_peered":"2024-04-25T13:40:31.219913+0000","last_clean":"2024-04-25T13:40:31.219913+0000","last_became_active":"2024-04-25T13:40:29.175066+0000","last_became_peered":"2024-04-25T13:40:29.175066+0000","last_unstale":"2024-04-25T13:40:31.219913+0000","last_undegraded":"2024-04-25T13:40:31.219913+0000","last_fullsized":"2024-04-25T13:40:31.219913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:27.404657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348852+0000","last_change":"2024-04-25T13:40:30.215383+0000","last_active":"2024-04-25T13:40:31.348852+0000","last_peered":"2024-04-25T13:40:31.348852+0000","last_clean":"2024-04-25T13:40:31.348852+0000","last_became_active":"2024-04-25T13:40:30.214994+0000","last_became_peered":"2024-04-25T13:40:30.214994+0000","last_unstale":"2024-04-25T13:40:31.348852+0000","last_undegraded":"2024-04-25T13:40:31.348852+0000","last_fullsized":"2024-04-25T13:40:31.348852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:49.832720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347401+0000","last_change":"2024-04-25T13:40:29.185820+0000","last_active":"2024-04-25T13:40:31.347401+0000","last_peered":"2024-04-25T13:40:31.347401+0000","last_clean":"2024-04-25T13:40:31.347401+0000","last_became_active":"2024-04-25T13:40:29.185576+0000","last_became_peered":"2024-04-25T13:40:29.185576+0000","last_unstale":"2024-04-25T13:40:31.347401+0000","last_undegraded":"2024-04-25T13:40:31.347401+0000","last_fullsized":"2024-04-25T13:40:31.347401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:18.356259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348088+0000","last_change":"2024-04-25T13:40:30.220046+0000","last_active":"2024-04-25T13:40:31.348088+0000","last_peered":"2024-04-25T13:40:31.348088+0000","last_clean":"2024-04-25T13:40:31.348088+0000","last_became_active":"2024-04-25T13:40:30.219244+0000","last_became_peered":"2024-04-25T13:40:30.219244+0000","last_unstale":"2024-04-25T13:40:31.348088+0000","last_undegraded":"2024-04-25T13:40:31.348088+0000","last_fullsized":"2024-04-25T13:40:31.348088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:34.257699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350082+0000","last_change":"2024-04-25T13:40:29.187441+0000","last_active":"2024-04-25T13:40:31.350082+0000","last_peered":"2024-04-25T13:40:31.350082+0000","last_clean":"2024-04-25T13:40:31.350082+0000","last_became_active":"2024-04-25T13:40:29.181693+0000","last_became_peered":"2024-04-25T13:40:29.181693+0000","last_unstale":"2024-04-25T13:40:31.350082+0000","last_undegraded":"2024-04-25T13:40:31.350082+0000","last_fullsized":"2024-04-25T13:40:31.350082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:57.361661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347958+0000","last_change":"2024-04-25T13:40:30.229833+0000","last_active":"2024-04-25T13:40:31.347958+0000","last_peered":"2024-04-25T13:40:31.347958+0000","last_clean":"2024-04-25T13:40:31.347958+0000","last_became_active":"2024-04-25T13:40:30.229011+0000","last_became_peered":"2024-04-25T13:40:30.229011+0000","last_unstale":"2024-04-25T13:40:31.347958+0000","last_undegraded":"2024-04-25T13:40:31.347958+0000","last_fullsized":"2024-04-25T13:40:31.347958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:15.058616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219728+0000","last_change":"2024-04-25T13:40:29.188978+0000","last_active":"2024-04-25T13:40:31.219728+0000","last_peered":"2024-04-25T13:40:31.219728+0000","last_clean":"2024-04-25T13:40:31.219728+0000","last_became_active":"2024-04-25T13:40:29.185846+0000","last_became_peered":"2024-04-25T13:40:29.185846+0000","last_unstale":"2024-04-25T13:40:31.219728+0000","last_undegraded":"2024-04-25T13:40:31.219728+0000","last_fullsized":"2024-04-25T13:40:31.219728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:08.912260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219935+0000","last_change":"2024-04-25T13:40:30.215550+0000","last_active":"2024-04-25T13:40:31.219935+0000","last_peered":"2024-04-25T13:40:31.219935+0000","last_clean":"2024-04-25T13:40:31.219935+0000","last_became_active":"2024-04-25T13:40:30.215250+0000","last_became_peered":"2024-04-25T13:40:30.215250+0000","last_unstale":"2024-04-25T13:40:31.219935+0000","last_undegraded":"2024-04-25T13:40:31.219935+0000","last_fullsized":"2024-04-25T13:40:31.219935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.530072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186915+0000","last_change":"2024-04-25T13:40:29.168980+0000","last_active":"2024-04-25T13:40:31.186915+0000","last_peered":"2024-04-25T13:40:31.186915+0000","last_clean":"2024-04-25T13:40:31.186915+0000","last_became_active":"2024-04-25T13:40:29.168218+0000","last_became_peered":"2024-04-25T13:40:29.168218+0000","last_unstale":"2024-04-25T13:40:31.186915+0000","last_undegraded":"2024-04-25T13:40:31.186915+0000","last_fullsized":"2024-04-25T13:40:31.186915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:25.060887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349934+0000","last_change":"2024-04-25T13:40:30.216707+0000","last_active":"2024-04-25T13:40:31.349934+0000","last_peered":"2024-04-25T13:40:31.349934+0000","last_clean":"2024-04-25T13:40:31.349934+0000","last_became_active":"2024-04-25T13:40:30.215907+0000","last_became_peered":"2024-04-25T13:40:30.215907+0000","last_unstale":"2024-04-25T13:40:31.349934+0000","last_undegraded":"2024-04-25T13:40:31.349934+0000","last_fullsized":"2024-04-25T13:40:31.349934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:22.984851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346871+0000","last_change":"2024-04-25T13:40:29.182090+0000","last_active":"2024-04-25T13:40:31.346871+0000","last_peered":"2024-04-25T13:40:31.346871+0000","last_clean":"2024-04-25T13:40:31.346871+0000","last_became_active":"2024-04-25T13:40:29.181228+0000","last_became_peered":"2024-04-25T13:40:29.181228+0000","last_unstale":"2024-04-25T13:40:31.346871+0000","last_undegraded":"2024-04-25T13:40:31.346871+0000","last_fullsized":"2024-04-25T13:40:31.346871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:49.643409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348554+0000","last_change":"2024-04-25T13:40:30.230355+0000","last_active":"2024-04-25T13:40:31.348554+0000","last_peered":"2024-04-25T13:40:31.348554+0000","last_clean":"2024-04-25T13:40:31.348554+0000","last_became_active":"2024-04-25T13:40:30.229397+0000","last_became_peered":"2024-04-25T13:40:30.229397+0000","last_unstale":"2024-04-25T13:40:31.348554+0000","last_undegraded":"2024-04-25T13:40:31.348554+0000","last_fullsized":"2024-04-25T13:40:31.348554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:23.803621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.187149+0000","last_change":"2024-04-25T13:40:29.160176+0000","last_active":"2024-04-25T13:40:31.187149+0000","last_peered":"2024-04-25T13:40:31.187149+0000","last_clean":"2024-04-25T13:40:31.187149+0000","last_became_active":"2024-04-25T13:40:29.159439+0000","last_became_peered":"2024-04-25T13:40:29.159439+0000","last_unstale":"2024-04-25T13:40:31.187149+0000","last_undegraded":"2024-04-25T13:40:31.187149+0000","last_fullsized":"2024-04-25T13:40:31.187149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:48.998902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347297+0000","last_change":"2024-04-25T13:40:30.224277+0000","last_active":"2024-04-25T13:40:31.347297+0000","last_peered":"2024-04-25T13:40:31.347297+0000","last_clean":"2024-04-25T13:40:31.347297+0000","last_became_active":"2024-04-25T13:40:30.221764+0000","last_became_peered":"2024-04-25T13:40:30.221764+0000","last_unstale":"2024-04-25T13:40:31.347297+0000","last_undegraded":"2024-04-25T13:40:31.347297+0000","last_fullsized":"2024-04-25T13:40:31.347297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:12.238288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185510+0000","last_change":"2024-04-25T13:40:29.182136+0000","last_active":"2024-04-25T13:40:31.185510+0000","last_peered":"2024-04-25T13:40:31.185510+0000","last_clean":"2024-04-25T13:40:31.185510+0000","last_became_active":"2024-04-25T13:40:29.181852+0000","last_became_peered":"2024-04-25T13:40:29.181852+0000","last_unstale":"2024-04-25T13:40:31.185510+0000","last_undegraded":"2024-04-25T13:40:31.185510+0000","last_fullsized":"2024-04-25T13:40:31.185510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:24.681218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350369+0000","last_change":"2024-04-25T13:40:30.217315+0000","last_active":"2024-04-25T13:40:31.350369+0000","last_peered":"2024-04-25T13:40:31.350369+0000","last_clean":"2024-04-25T13:40:31.350369+0000","last_became_active":"2024-04-25T13:40:30.214308+0000","last_became_peered":"2024-04-25T13:40:30.214308+0000","last_unstale":"2024-04-25T13:40:31.350369+0000","last_undegraded":"2024-04-25T13:40:31.350369+0000","last_fullsized":"2024-04-25T13:40:31.350369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:57.857844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186012+0000","last_change":"2024-04-25T13:40:29.189228+0000","last_active":"2024-04-25T13:40:31.186012+0000","last_peered":"2024-04-25T13:40:31.186012+0000","last_clean":"2024-04-25T13:40:31.186012+0000","last_became_active":"2024-04-25T13:40:29.189004+0000","last_became_peered":"2024-04-25T13:40:29.189004+0000","last_unstale":"2024-04-25T13:40:31.186012+0000","last_undegraded":"2024-04-25T13:40:31.186012+0000","last_fullsized":"2024-04-25T13:40:31.186012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:14.763408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349561+0000","last_change":"2024-04-25T13:40:30.223154+0000","last_active":"2024-04-25T13:40:31.349561+0000","last_peered":"2024-04-25T13:40:31.349561+0000","last_clean":"2024-04-25T13:40:31.349561+0000","last_became_active":"2024-04-25T13:40:30.222384+0000","last_became_peered":"2024-04-25T13:40:30.222384+0000","last_unstale":"2024-04-25T13:40:31.349561+0000","last_undegraded":"2024-04-25T13:40:31.349561+0000","last_fullsized":"2024-04-25T13:40:31.349561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:59.286900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350710+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:40:31.350710+0000","last_peered":"2024-04-25T13:40:31.350710+0000","last_clean":"2024-04-25T13:40:31.350710+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:40:31.350710+0000","last_undegraded":"2024-04-25T13:40:31.350710+0000","last_fullsized":"2024-04-25T13:40:31.350710+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350590+0000","last_change":"2024-04-25T13:40:29.186348+0000","last_active":"2024-04-25T13:40:31.350590+0000","last_peered":"2024-04-25T13:40:31.350590+0000","last_clean":"2024-04-25T13:40:31.350590+0000","last_became_active":"2024-04-25T13:40:29.180642+0000","last_became_peered":"2024-04-25T13:40:29.180642+0000","last_unstale":"2024-04-25T13:40:31.350590+0000","last_undegraded":"2024-04-25T13:40:31.350590+0000","last_fullsized":"2024-04-25T13:40:31.350590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:43.636874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347213+0000","last_change":"2024-04-25T13:40:30.209809+0000","last_active":"2024-04-25T13:40:31.347213+0000","last_peered":"2024-04-25T13:40:31.347213+0000","last_clean":"2024-04-25T13:40:31.347213+0000","last_became_active":"2024-04-25T13:40:30.209014+0000","last_became_peered":"2024-04-25T13:40:30.209014+0000","last_unstale":"2024-04-25T13:40:31.347213+0000","last_undegraded":"2024-04-25T13:40:31.347213+0000","last_fullsized":"2024-04-25T13:40:31.347213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:24.945222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350549+0000","last_change":"2024-04-25T13:40:29.197074+0000","last_active":"2024-04-25T13:40:31.350549+0000","last_peered":"2024-04-25T13:40:31.350549+0000","last_clean":"2024-04-25T13:40:31.350549+0000","last_became_active":"2024-04-25T13:40:29.193688+0000","last_became_peered":"2024-04-25T13:40:29.193688+0000","last_unstale":"2024-04-25T13:40:31.350549+0000","last_undegraded":"2024-04-25T13:40:31.350549+0000","last_fullsized":"2024-04-25T13:40:31.350549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:03.325908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347895+0000","last_change":"2024-04-25T13:40:30.187471+0000","last_active":"2024-04-25T13:40:31.347895+0000","last_peered":"2024-04-25T13:40:31.347895+0000","last_clean":"2024-04-25T13:40:31.347895+0000","last_became_active":"2024-04-25T13:40:30.186993+0000","last_became_peered":"2024-04-25T13:40:30.186993+0000","last_unstale":"2024-04-25T13:40:31.347895+0000","last_undegraded":"2024-04-25T13:40:31.347895+0000","last_fullsized":"2024-04-25T13:40:31.347895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.749355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.187216+0000","last_change":"2024-04-25T13:40:29.159566+0000","last_active":"2024-04-25T13:40:31.187216+0000","last_peered":"2024-04-25T13:40:31.187216+0000","last_clean":"2024-04-25T13:40:31.187216+0000","last_became_active":"2024-04-25T13:40:29.159054+0000","last_became_peered":"2024-04-25T13:40:29.159054+0000","last_unstale":"2024-04-25T13:40:31.187216+0000","last_undegraded":"2024-04-25T13:40:31.187216+0000","last_fullsized":"2024-04-25T13:40:31.187216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:03.455177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185373+0000","last_change":"2024-04-25T13:40:30.203352+0000","last_active":"2024-04-25T13:40:31.185373+0000","last_peered":"2024-04-25T13:40:31.185373+0000","last_clean":"2024-04-25T13:40:31.185373+0000","last_became_active":"2024-04-25T13:40:30.202463+0000","last_became_peered":"2024-04-25T13:40:30.202463+0000","last_unstale":"2024-04-25T13:40:31.185373+0000","last_undegraded":"2024-04-25T13:40:31.185373+0000","last_fullsized":"2024-04-25T13:40:31.185373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:01.912451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350985+0000","last_change":"2024-04-25T13:40:29.190061+0000","last_active":"2024-04-25T13:40:31.350985+0000","last_peered":"2024-04-25T13:40:31.350985+0000","last_clean":"2024-04-25T13:40:31.350985+0000","last_became_active":"2024-04-25T13:40:29.182897+0000","last_became_peered":"2024-04-25T13:40:29.182897+0000","last_unstale":"2024-04-25T13:40:31.350985+0000","last_undegraded":"2024-04-25T13:40:31.350985+0000","last_fullsized":"2024-04-25T13:40:31.350985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:31.168555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346748+0000","last_change":"2024-04-25T13:40:30.187160+0000","last_active":"2024-04-25T13:40:31.346748+0000","last_peered":"2024-04-25T13:40:31.346748+0000","last_clean":"2024-04-25T13:40:31.346748+0000","last_became_active":"2024-04-25T13:40:30.186695+0000","last_became_peered":"2024-04-25T13:40:30.186695+0000","last_unstale":"2024-04-25T13:40:31.346748+0000","last_undegraded":"2024-04-25T13:40:31.346748+0000","last_fullsized":"2024-04-25T13:40:31.346748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:57.859568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186430+0000","last_change":"2024-04-25T13:40:29.168378+0000","last_active":"2024-04-25T13:40:31.186430+0000","last_peered":"2024-04-25T13:40:31.186430+0000","last_clean":"2024-04-25T13:40:31.186430+0000","last_became_active":"2024-04-25T13:40:29.167515+0000","last_became_peered":"2024-04-25T13:40:29.167515+0000","last_unstale":"2024-04-25T13:40:31.186430+0000","last_undegraded":"2024-04-25T13:40:31.186430+0000","last_fullsized":"2024-04-25T13:40:31.186430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.417158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346850+0000","last_change":"2024-04-25T13:40:30.214354+0000","last_active":"2024-04-25T13:40:31.346850+0000","last_peered":"2024-04-25T13:40:31.346850+0000","last_clean":"2024-04-25T13:40:31.346850+0000","last_became_active":"2024-04-25T13:40:30.213970+0000","last_became_peered":"2024-04-25T13:40:30.213970+0000","last_unstale":"2024-04-25T13:40:31.346850+0000","last_undegraded":"2024-04-25T13:40:31.346850+0000","last_fullsized":"2024-04-25T13:40:31.346850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:54.278471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348113+0000","last_change":"2024-04-25T13:40:29.197843+0000","last_active":"2024-04-25T13:40:31.348113+0000","last_peered":"2024-04-25T13:40:31.348113+0000","last_clean":"2024-04-25T13:40:31.348113+0000","last_became_active":"2024-04-25T13:40:29.196299+0000","last_became_peered":"2024-04-25T13:40:29.196299+0000","last_unstale":"2024-04-25T13:40:31.348113+0000","last_undegraded":"2024-04-25T13:40:31.348113+0000","last_fullsized":"2024-04-25T13:40:31.348113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:41.785353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185424+0000","last_change":"2024-04-25T13:40:30.212599+0000","last_active":"2024-04-25T13:40:31.185424+0000","last_peered":"2024-04-25T13:40:31.185424+0000","last_clean":"2024-04-25T13:40:31.185424+0000","last_became_active":"2024-04-25T13:40:30.211967+0000","last_became_peered":"2024-04-25T13:40:30.211967+0000","last_unstale":"2024-04-25T13:40:31.185424+0000","last_undegraded":"2024-04-25T13:40:31.185424+0000","last_fullsized":"2024-04-25T13:40:31.185424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:28.946427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219865+0000","last_change":"2024-04-25T13:40:29.175520+0000","last_active":"2024-04-25T13:40:31.219865+0000","last_peered":"2024-04-25T13:40:31.219865+0000","last_clean":"2024-04-25T13:40:31.219865+0000","last_became_active":"2024-04-25T13:40:29.174561+0000","last_became_peered":"2024-04-25T13:40:29.174561+0000","last_unstale":"2024-04-25T13:40:31.219865+0000","last_undegraded":"2024-04-25T13:40:31.219865+0000","last_fullsized":"2024-04-25T13:40:31.219865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:55.788741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346789+0000","last_change":"2024-04-25T13:40:30.217175+0000","last_active":"2024-04-25T13:40:31.346789+0000","last_peered":"2024-04-25T13:40:31.346789+0000","last_clean":"2024-04-25T13:40:31.346789+0000","last_became_active":"2024-04-25T13:40:30.216851+0000","last_became_peered":"2024-04-25T13:40:30.216851+0000","last_unstale":"2024-04-25T13:40:31.346789+0000","last_undegraded":"2024-04-25T13:40:31.346789+0000","last_fullsized":"2024-04-25T13:40:31.346789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:12.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219908+0000","last_change":"2024-04-25T13:40:29.194455+0000","last_active":"2024-04-25T13:40:31.219908+0000","last_peered":"2024-04-25T13:40:31.219908+0000","last_clean":"2024-04-25T13:40:31.219908+0000","last_became_active":"2024-04-25T13:40:29.193686+0000","last_became_peered":"2024-04-25T13:40:29.193686+0000","last_unstale":"2024-04-25T13:40:31.219908+0000","last_undegraded":"2024-04-25T13:40:31.219908+0000","last_fullsized":"2024-04-25T13:40:31.219908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:27.100097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347083+0000","last_change":"2024-04-25T13:40:30.211947+0000","last_active":"2024-04-25T13:40:31.347083+0000","last_peered":"2024-04-25T13:40:31.347083+0000","last_clean":"2024-04-25T13:40:31.347083+0000","last_became_active":"2024-04-25T13:40:30.202151+0000","last_became_peered":"2024-04-25T13:40:30.202151+0000","last_unstale":"2024-04-25T13:40:31.347083+0000","last_undegraded":"2024-04-25T13:40:31.347083+0000","last_fullsized":"2024-04-25T13:40:31.347083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.174571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350372+0000","last_change":"2024-04-25T13:40:29.200572+0000","last_active":"2024-04-25T13:40:31.350372+0000","last_peered":"2024-04-25T13:40:31.350372+0000","last_clean":"2024-04-25T13:40:31.350372+0000","last_became_active":"2024-04-25T13:40:29.199836+0000","last_became_peered":"2024-04-25T13:40:29.199836+0000","last_unstale":"2024-04-25T13:40:31.350372+0000","last_undegraded":"2024-04-25T13:40:31.350372+0000","last_fullsized":"2024-04-25T13:40:31.350372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:38.068886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349670+0000","last_change":"2024-04-25T13:40:30.225985+0000","last_active":"2024-04-25T13:40:31.349670+0000","last_peered":"2024-04-25T13:40:31.349670+0000","last_clean":"2024-04-25T13:40:31.349670+0000","last_became_active":"2024-04-25T13:40:30.224959+0000","last_became_peered":"2024-04-25T13:40:30.224959+0000","last_unstale":"2024-04-25T13:40:31.349670+0000","last_undegraded":"2024-04-25T13:40:31.349670+0000","last_fullsized":"2024-04-25T13:40:31.349670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:13.498711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186159+0000","last_change":"2024-04-25T13:40:29.196670+0000","last_active":"2024-04-25T13:40:31.186159+0000","last_peered":"2024-04-25T13:40:31.186159+0000","last_clean":"2024-04-25T13:40:31.186159+0000","last_became_active":"2024-04-25T13:40:29.196115+0000","last_became_peered":"2024-04-25T13:40:29.196115+0000","last_unstale":"2024-04-25T13:40:31.186159+0000","last_undegraded":"2024-04-25T13:40:31.186159+0000","last_fullsized":"2024-04-25T13:40:31.186159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:35.705063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347672+0000","last_change":"2024-04-25T13:40:30.203777+0000","last_active":"2024-04-25T13:40:31.347672+0000","last_peered":"2024-04-25T13:40:31.347672+0000","last_clean":"2024-04-25T13:40:31.347672+0000","last_became_active":"2024-04-25T13:40:30.202751+0000","last_became_peered":"2024-04-25T13:40:30.202751+0000","last_unstale":"2024-04-25T13:40:31.347672+0000","last_undegraded":"2024-04-25T13:40:31.347672+0000","last_fullsized":"2024-04-25T13:40:31.347672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:09.528068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185653+0000","last_change":"2024-04-25T13:40:29.176333+0000","last_active":"2024-04-25T13:40:31.185653+0000","last_peered":"2024-04-25T13:40:31.185653+0000","last_clean":"2024-04-25T13:40:31.185653+0000","last_became_active":"2024-04-25T13:40:29.176078+0000","last_became_peered":"2024-04-25T13:40:29.176078+0000","last_unstale":"2024-04-25T13:40:31.185653+0000","last_undegraded":"2024-04-25T13:40:31.185653+0000","last_fullsized":"2024-04-25T13:40:31.185653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:19.200730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346905+0000","last_change":"2024-04-25T13:40:30.216029+0000","last_active":"2024-04-25T13:40:31.346905+0000","last_peered":"2024-04-25T13:40:31.346905+0000","last_clean":"2024-04-25T13:40:31.346905+0000","last_became_active":"2024-04-25T13:40:30.215353+0000","last_became_peered":"2024-04-25T13:40:30.215353+0000","last_unstale":"2024-04-25T13:40:31.346905+0000","last_undegraded":"2024-04-25T13:40:31.346905+0000","last_fullsized":"2024-04-25T13:40:31.346905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:10.825968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349163+0000","last_change":"2024-04-25T13:40:29.195078+0000","last_active":"2024-04-25T13:40:31.349163+0000","last_peered":"2024-04-25T13:40:31.349163+0000","last_clean":"2024-04-25T13:40:31.349163+0000","last_became_active":"2024-04-25T13:40:29.193847+0000","last_became_peered":"2024-04-25T13:40:29.193847+0000","last_unstale":"2024-04-25T13:40:31.349163+0000","last_undegraded":"2024-04-25T13:40:31.349163+0000","last_fullsized":"2024-04-25T13:40:31.349163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:33.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219558+0000","last_change":"2024-04-25T13:40:30.205595+0000","last_active":"2024-04-25T13:40:31.219558+0000","last_peered":"2024-04-25T13:40:31.219558+0000","last_clean":"2024-04-25T13:40:31.219558+0000","last_became_active":"2024-04-25T13:40:30.204793+0000","last_became_peered":"2024-04-25T13:40:30.204793+0000","last_unstale":"2024-04-25T13:40:31.219558+0000","last_undegraded":"2024-04-25T13:40:31.219558+0000","last_fullsized":"2024-04-25T13:40:31.219558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:55.486355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349463+0000","last_change":"2024-04-25T13:40:30.202452+0000","last_active":"2024-04-25T13:40:31.349463+0000","last_peered":"2024-04-25T13:40:31.349463+0000","last_clean":"2024-04-25T13:40:31.349463+0000","last_became_active":"2024-04-25T13:40:30.201683+0000","last_became_peered":"2024-04-25T13:40:30.201683+0000","last_unstale":"2024-04-25T13:40:31.349463+0000","last_undegraded":"2024-04-25T13:40:31.349463+0000","last_fullsized":"2024-04-25T13:40:31.349463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.132452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350105+0000","last_change":"2024-04-25T13:40:29.199128+0000","last_active":"2024-04-25T13:40:31.350105+0000","last_peered":"2024-04-25T13:40:31.350105+0000","last_clean":"2024-04-25T13:40:31.350105+0000","last_became_active":"2024-04-25T13:40:29.198449+0000","last_became_peered":"2024-04-25T13:40:29.198449+0000","last_unstale":"2024-04-25T13:40:31.350105+0000","last_undegraded":"2024-04-25T13:40:31.350105+0000","last_fullsized":"2024-04-25T13:40:31.350105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:08.596939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347469+0000","last_change":"2024-04-25T13:40:30.220740+0000","last_active":"2024-04-25T13:40:31.347469+0000","last_peered":"2024-04-25T13:40:31.347469+0000","last_clean":"2024-04-25T13:40:31.347469+0000","last_became_active":"2024-04-25T13:40:30.218927+0000","last_became_peered":"2024-04-25T13:40:30.218927+0000","last_unstale":"2024-04-25T13:40:31.347469+0000","last_undegraded":"2024-04-25T13:40:31.347469+0000","last_fullsized":"2024-04-25T13:40:31.347469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.133104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349098+0000","last_change":"2024-04-25T13:40:29.199401+0000","last_active":"2024-04-25T13:40:31.349098+0000","last_peered":"2024-04-25T13:40:31.349098+0000","last_clean":"2024-04-25T13:40:31.349098+0000","last_became_active":"2024-04-25T13:40:29.198182+0000","last_became_peered":"2024-04-25T13:40:29.198182+0000","last_unstale":"2024-04-25T13:40:31.349098+0000","last_undegraded":"2024-04-25T13:40:31.349098+0000","last_fullsized":"2024-04-25T13:40:31.349098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:31.807935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347497+0000","last_change":"2024-04-25T13:40:30.213026+0000","last_active":"2024-04-25T13:40:31.347497+0000","last_peered":"2024-04-25T13:40:31.347497+0000","last_clean":"2024-04-25T13:40:31.347497+0000","last_became_active":"2024-04-25T13:40:30.203212+0000","last_became_peered":"2024-04-25T13:40:30.203212+0000","last_unstale":"2024-04-25T13:40:31.347497+0000","last_undegraded":"2024-04-25T13:40:31.347497+0000","last_fullsized":"2024-04-25T13:40:31.347497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:24.180245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347687+0000","last_change":"2024-04-25T13:40:29.173054+0000","last_active":"2024-04-25T13:40:31.347687+0000","last_peered":"2024-04-25T13:40:31.347687+0000","last_clean":"2024-04-25T13:40:31.347687+0000","last_became_active":"2024-04-25T13:40:29.172375+0000","last_became_peered":"2024-04-25T13:40:29.172375+0000","last_unstale":"2024-04-25T13:40:31.347687+0000","last_undegraded":"2024-04-25T13:40:31.347687+0000","last_fullsized":"2024-04-25T13:40:31.347687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:26.996154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347229+0000","last_change":"2024-04-25T13:40:30.214823+0000","last_active":"2024-04-25T13:40:31.347229+0000","last_peered":"2024-04-25T13:40:31.347229+0000","last_clean":"2024-04-25T13:40:31.347229+0000","last_became_active":"2024-04-25T13:40:30.210600+0000","last_became_peered":"2024-04-25T13:40:30.210600+0000","last_unstale":"2024-04-25T13:40:31.347229+0000","last_undegraded":"2024-04-25T13:40:31.347229+0000","last_fullsized":"2024-04-25T13:40:31.347229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:14.151771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348125+0000","last_change":"2024-04-25T13:40:29.192636+0000","last_active":"2024-04-25T13:40:31.348125+0000","last_peered":"2024-04-25T13:40:31.348125+0000","last_clean":"2024-04-25T13:40:31.348125+0000","last_became_active":"2024-04-25T13:40:29.192248+0000","last_became_peered":"2024-04-25T13:40:29.192248+0000","last_unstale":"2024-04-25T13:40:31.348125+0000","last_undegraded":"2024-04-25T13:40:31.348125+0000","last_fullsized":"2024-04-25T13:40:31.348125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:25.540341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929100,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929066,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338220,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:40:34.322 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:40:34.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.496+0000 7f0c1c6be640 1 Processor -- start 2024-04-25T13:40:34.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.496+0000 7f0c1c6be640 1 -- start start 2024-04-25T13:40:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1c6be640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c140746d0 0x7f0c1406f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1c6be640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c140cddf0 0x7f0c140d0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1c6be640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c14070150 con 0x7f0c1406fd40 2024-04-25T13:40:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1c6be640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0c14072fe0 con 0x7f0c140746d0 2024-04-25T13:40:34.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1c6be640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c140ccae0 con 0x7f0c140cddf0 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19c59640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0c1406fd40 0x7f0c140cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35600/0 (socket says 172.21.15.80:35600) 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19c59640 1 -- 172.21.15.80:0/1704870098 learned_addr learned my addr 172.21.15.80:0/1704870098 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1a45a640 1 --2- 172.21.15.80:0/1704870098 >> v2:172.21.15.87:3300/0 conn(0x7f0c140746d0 0x7f0c1406f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2586489073 0 0) 0x7f0c04001120 con 0x7f0c1406fd40 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1ac5b640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c140cddf0 0x7f0c140d0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0c00005740 con 0x7f0c1406fd40 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2197862017 0 0) 0x7f0c00005740 con 0x7f0c1406fd40 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c140cddf0 msgr2=0x7f0c140d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c140cddf0 0x7f0c140d0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 >> v2:172.21.15.87:3300/0 conn(0x7f0c140746d0 msgr2=0x7f0c1406f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 --2- 172.21.15.80:0/1704870098 >> v2:172.21.15.87:3300/0 conn(0x7f0c140746d0 0x7f0c1406f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c14073210 con 0x7f0c1406fd40 2024-04-25T13:40:34.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0c04001850 con 0x7f0c1406fd40 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 >> v1:172.21.15.80:6789/0 conn(0x7f0c1406fd40 legacy=0x7f0c140cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c00005bd0 0x7f0c00006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c00006a20 0x7f0c00006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c140cddf0 0x7f0c00007440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c140ccae0 con 0x7f0c00006a20 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0000dd70 con 0x7f0c00005bd0 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0000df40 con 0x7f0c140cddf0 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1a45a640 1 --2- 172.21.15.80:0/1704870098 >> v2:172.21.15.87:3300/0 conn(0x7f0c140746d0 0x7f0c1406f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19c59640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c00006a20 0x7f0c00006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c1ac5b640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c140cddf0 0x7f0c00007440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19c59640 1 -- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c140cddf0 msgr2=0x7f0c00007440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19c59640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c140cddf0 0x7f0c00007440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19c59640 1 -- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c00005bd0 msgr2=0x7f0c00006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19c59640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c00005bd0 0x7f0c00006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.500+0000 7f0c19c59640 1 -- 172.21.15.80:0/1704870098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c00005740 con 0x7f0c00006a20 2024-04-25T13:40:34.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c19c59640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c00006a20 0x7f0c00006e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c440 tx=0x7f0c0401cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e810f17740797193 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0400c0e0 con 0x7f0c00006a20 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c040a9070 con 0x7f0c00006a20 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c19458640 1 -- 172.21.15.80:0/1704870098 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040a5dd0 con 0x7f0c00006a20 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1c6be640 1 -- 172.21.15.80:0/1704870098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c00006a20 msgr2=0x7f0c00006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1c6be640 1 --2- 172.21.15.80:0/1704870098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c00006a20 0x7f0c00006e30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c440 tx=0x7f0c0401cc90 comp rx=0 tx=0).stop 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1ac5b640 1 -- 172.21.15.80:0/1704870098 reap_dead start 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1c6be640 1 -- 172.21.15.80:0/1704870098 shutdown_connections 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1c6be640 1 -- 172.21.15.80:0/1704870098 >> 172.21.15.80:0/1704870098 conn(0x7f0c140d2170 msgr2=0x7f0c140d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1c6be640 1 -- 172.21.15.80:0/1704870098 shutdown_connections 2024-04-25T13:40:34.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1c6be640 1 -- 172.21.15.80:0/1704870098 wait complete. 2024-04-25T13:40:34.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1c6be640 1 Processor -- start 2024-04-25T13:40:34.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.504+0000 7f0c1c6be640 1 -- start start 2024-04-25T13:40:34.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c00006a20 0x7f0c140cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c140af850 0x7f0c140afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c140b4cf0 0x7f0c140b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1415d4b0 con 0x7f0c140af850 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c14073d90 con 0x7f0c140b4cf0 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1406e6f0 con 0x7f0c00006a20 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c140af850 0x7f0c140afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c140af850 0x7f0c140afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36014/0 (socket says 172.21.15.80:36014) 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 -- 172.21.15.80:0/882037291 learned_addr learned my addr 172.21.15.80:0/882037291 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 -- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c00006a20 msgr2=0x7f0c140cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c00006a20 0x7f0c140cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 -- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c140b4cf0 msgr2=0x7f0c140b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c140b4cf0 0x7f0c140b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 -- 172.21.15.80:0/882037291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c140ca1d0 con 0x7f0c140af850 2024-04-25T13:40:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c19c59640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c140af850 0x7f0c140afc60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c3e0 tx=0x7f0c00007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c0affd640 1 -- 172.21.15.80:0/882037291 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c04001090 con 0x7f0c140af850 2024-04-25T13:40:34.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c0affd640 1 -- 172.21.15.80:0/882037291 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c040a5970 con 0x7f0c140af850 2024-04-25T13:40:34.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c0affd640 1 -- 172.21.15.80:0/882037291 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c040a5d60 con 0x7f0c140af850 2024-04-25T13:40:34.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c140b6190 con 0x7f0c140af850 2024-04-25T13:40:34.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c140cc310 con 0x7f0c140af850 2024-04-25T13:40:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1406a3a0 con 0x7f0c140af850 2024-04-25T13:40:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c0affd640 1 -- 172.21.15.80:0/882037291 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0c040b7030 con 0x7f0c140af850 2024-04-25T13:40:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c0affd640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf806f400 0x7f0bf8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c0affd640 1 -- 172.21.15.80:0/882037291 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0c04128390 con 0x7f0c140af850 2024-04-25T13:40:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.508+0000 7f0c1a45a640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf806f400 0x7f0bf8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.512+0000 7f0c1a45a640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf806f400 0x7f0bf8071910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0c14072a70 tx=0x7f0c10008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.512+0000 7f0c0affd640 1 -- 172.21.15.80:0/882037291 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c14070150 con 0x7f0c140af850 2024-04-25T13:40:34.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.668+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0c140c8cf0 con 0x7f0bf806f400 2024-04-25T13:40:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.688+0000 7f0c0affd640 1 -- 172.21.15.80:0/882037291 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247472 (secure 0 0 0) 0x7f0c140c8cf0 con 0x7f0bf806f400 2024-04-25T13:40:34.688 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:40:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf806f400 msgr2=0x7f0bf8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf806f400 0x7f0bf8071910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0c14072a70 tx=0x7f0c10008040 comp rx=0 tx=0).stop 2024-04-25T13:40:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c140af850 msgr2=0x7f0c140afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c140af850 0x7f0c140afc60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c3e0 tx=0x7f0c00007a50 comp rx=0 tx=0).stop 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 shutdown_connections 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf806f400 0x7f0bf8071910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c140b4cf0 0x7f0c140b5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c140af850 0x7f0c140afc60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 --2- 172.21.15.80:0/882037291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c00006a20 0x7f0c140cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 >> 172.21.15.80:0/882037291 conn(0x7f0c140d2170 msgr2=0x7f0c1415c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 shutdown_connections 2024-04-25T13:40:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.692+0000 7f0c1c6be640 1 -- 172.21.15.80:0/882037291 wait complete. 2024-04-25T13:40:34.711 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-04-25T13:40:34.092573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.942372"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348886+0000","last_change":"2024-04-25T13:40:30.221428+0000","last_active":"2024-04-25T13:40:31.348886+0000","last_peered":"2024-04-25T13:40:31.348886+0000","last_clean":"2024-04-25T13:40:31.348886+0000","last_became_active":"2024-04-25T13:40:30.219271+0000","last_became_peered":"2024-04-25T13:40:30.219271+0000","last_unstale":"2024-04-25T13:40:31.348886+0000","last_undegraded":"2024-04-25T13:40:31.348886+0000","last_fullsized":"2024-04-25T13:40:31.348886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.180832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184646+0000","last_change":"2024-04-25T13:40:30.211459+0000","last_active":"2024-04-25T13:40:31.184646+0000","last_peered":"2024-04-25T13:40:31.184646+0000","last_clean":"2024-04-25T13:40:31.184646+0000","last_became_active":"2024-04-25T13:40:30.211073+0000","last_became_peered":"2024-04-25T13:40:30.211073+0000","last_unstale":"2024-04-25T13:40:31.184646+0000","last_undegraded":"2024-04-25T13:40:31.184646+0000","last_fullsized":"2024-04-25T13:40:31.184646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:13.973526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346644+0000","last_change":"2024-04-25T13:40:30.213483+0000","last_active":"2024-04-25T13:40:31.346644+0000","last_peered":"2024-04-25T13:40:31.346644+0000","last_clean":"2024-04-25T13:40:31.346644+0000","last_became_active":"2024-04-25T13:40:30.207846+0000","last_became_peered":"2024-04-25T13:40:30.207846+0000","last_unstale":"2024-04-25T13:40:31.346644+0000","last_undegraded":"2024-04-25T13:40:31.346644+0000","last_fullsized":"2024-04-25T13:40:31.346644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:10.391107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.345909+0000","last_change":"2024-04-25T13:40:30.216562+0000","last_active":"2024-04-25T13:40:31.345909+0000","last_peered":"2024-04-25T13:40:31.345909+0000","last_clean":"2024-04-25T13:40:31.345909+0000","last_became_active":"2024-04-25T13:40:30.215696+0000","last_became_peered":"2024-04-25T13:40:30.215696+0000","last_unstale":"2024-04-25T13:40:31.345909+0000","last_undegraded":"2024-04-25T13:40:31.345909+0000","last_fullsized":"2024-04-25T13:40:31.345909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:42.952152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347520+0000","last_change":"2024-04-25T13:40:30.207884+0000","last_active":"2024-04-25T13:40:31.347520+0000","last_peered":"2024-04-25T13:40:31.347520+0000","last_clean":"2024-04-25T13:40:31.347520+0000","last_became_active":"2024-04-25T13:40:30.207080+0000","last_became_peered":"2024-04-25T13:40:30.207080+0000","last_unstale":"2024-04-25T13:40:31.347520+0000","last_undegraded":"2024-04-25T13:40:31.347520+0000","last_fullsized":"2024-04-25T13:40:31.347520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:07.164518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347177+0000","last_change":"2024-04-25T13:40:30.221017+0000","last_active":"2024-04-25T13:40:31.347177+0000","last_peered":"2024-04-25T13:40:31.347177+0000","last_clean":"2024-04-25T13:40:31.347177+0000","last_became_active":"2024-04-25T13:40:30.219333+0000","last_became_peered":"2024-04-25T13:40:30.219333+0000","last_unstale":"2024-04-25T13:40:31.347177+0000","last_undegraded":"2024-04-25T13:40:31.347177+0000","last_fullsized":"2024-04-25T13:40:31.347177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:29.428442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349282+0000","last_change":"2024-04-25T13:40:30.216405+0000","last_active":"2024-04-25T13:40:31.349282+0000","last_peered":"2024-04-25T13:40:31.349282+0000","last_clean":"2024-04-25T13:40:31.349282+0000","last_became_active":"2024-04-25T13:40:30.215354+0000","last_became_peered":"2024-04-25T13:40:30.215354+0000","last_unstale":"2024-04-25T13:40:31.349282+0000","last_undegraded":"2024-04-25T13:40:31.349282+0000","last_fullsized":"2024-04-25T13:40:31.349282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:22.813674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346103+0000","last_change":"2024-04-25T13:40:30.207820+0000","last_active":"2024-04-25T13:40:31.346103+0000","last_peered":"2024-04-25T13:40:31.346103+0000","last_clean":"2024-04-25T13:40:31.346103+0000","last_became_active":"2024-04-25T13:40:30.207148+0000","last_became_peered":"2024-04-25T13:40:30.207148+0000","last_unstale":"2024-04-25T13:40:31.346103+0000","last_undegraded":"2024-04-25T13:40:31.346103+0000","last_fullsized":"2024-04-25T13:40:31.346103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:05.730270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346696+0000","last_change":"2024-04-25T13:40:30.219574+0000","last_active":"2024-04-25T13:40:31.346696+0000","last_peered":"2024-04-25T13:40:31.346696+0000","last_clean":"2024-04-25T13:40:31.346696+0000","last_became_active":"2024-04-25T13:40:30.216463+0000","last_became_peered":"2024-04-25T13:40:30.216463+0000","last_unstale":"2024-04-25T13:40:31.346696+0000","last_undegraded":"2024-04-25T13:40:31.346696+0000","last_fullsized":"2024-04-25T13:40:31.346696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:33.974714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219726+0000","last_change":"2024-04-25T13:40:30.202005+0000","last_active":"2024-04-25T13:40:31.219726+0000","last_peered":"2024-04-25T13:40:31.219726+0000","last_clean":"2024-04-25T13:40:31.219726+0000","last_became_active":"2024-04-25T13:40:30.201195+0000","last_became_peered":"2024-04-25T13:40:30.201195+0000","last_unstale":"2024-04-25T13:40:31.219726+0000","last_undegraded":"2024-04-25T13:40:31.219726+0000","last_fullsized":"2024-04-25T13:40:31.219726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:20.543637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219380+0000","last_change":"2024-04-25T13:40:30.221626+0000","last_active":"2024-04-25T13:40:31.219380+0000","last_peered":"2024-04-25T13:40:31.219380+0000","last_clean":"2024-04-25T13:40:31.219380+0000","last_became_active":"2024-04-25T13:40:30.221294+0000","last_became_peered":"2024-04-25T13:40:30.221294+0000","last_unstale":"2024-04-25T13:40:31.219380+0000","last_undegraded":"2024-04-25T13:40:31.219380+0000","last_fullsized":"2024-04-25T13:40:31.219380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:49.053408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348097+0000","last_change":"2024-04-25T13:40:30.199635+0000","last_active":"2024-04-25T13:40:31.348097+0000","last_peered":"2024-04-25T13:40:31.348097+0000","last_clean":"2024-04-25T13:40:31.348097+0000","last_became_active":"2024-04-25T13:40:30.198748+0000","last_became_peered":"2024-04-25T13:40:30.198748+0000","last_unstale":"2024-04-25T13:40:31.348097+0000","last_undegraded":"2024-04-25T13:40:31.348097+0000","last_fullsized":"2024-04-25T13:40:31.348097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346436+0000","last_change":"2024-04-25T13:40:30.215629+0000","last_active":"2024-04-25T13:40:31.346436+0000","last_peered":"2024-04-25T13:40:31.346436+0000","last_clean":"2024-04-25T13:40:31.346436+0000","last_became_active":"2024-04-25T13:40:30.214870+0000","last_became_peered":"2024-04-25T13:40:30.214870+0000","last_unstale":"2024-04-25T13:40:31.346436+0000","last_undegraded":"2024-04-25T13:40:31.346436+0000","last_fullsized":"2024-04-25T13:40:31.346436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:21.817644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346237+0000","last_change":"2024-04-25T13:40:30.200380+0000","last_active":"2024-04-25T13:40:31.346237+0000","last_peered":"2024-04-25T13:40:31.346237+0000","last_clean":"2024-04-25T13:40:31.346237+0000","last_became_active":"2024-04-25T13:40:30.198804+0000","last_became_peered":"2024-04-25T13:40:30.198804+0000","last_unstale":"2024-04-25T13:40:31.346237+0000","last_undegraded":"2024-04-25T13:40:31.346237+0000","last_fullsized":"2024-04-25T13:40:31.346237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:02.090517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185879+0000","last_change":"2024-04-25T13:40:30.228782+0000","last_active":"2024-04-25T13:40:31.185879+0000","last_peered":"2024-04-25T13:40:31.185879+0000","last_clean":"2024-04-25T13:40:31.185879+0000","last_became_active":"2024-04-25T13:40:30.228329+0000","last_became_peered":"2024-04-25T13:40:30.228329+0000","last_unstale":"2024-04-25T13:40:31.185879+0000","last_undegraded":"2024-04-25T13:40:31.185879+0000","last_fullsized":"2024-04-25T13:40:31.185879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:39.981195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346736+0000","last_change":"2024-04-25T13:40:30.221267+0000","last_active":"2024-04-25T13:40:31.346736+0000","last_peered":"2024-04-25T13:40:31.346736+0000","last_clean":"2024-04-25T13:40:31.346736+0000","last_became_active":"2024-04-25T13:40:30.219761+0000","last_became_peered":"2024-04-25T13:40:30.219761+0000","last_unstale":"2024-04-25T13:40:31.346736+0000","last_undegraded":"2024-04-25T13:40:31.346736+0000","last_fullsized":"2024-04-25T13:40:31.346736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:16.273351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346948+0000","last_change":"2024-04-25T13:40:30.225471+0000","last_active":"2024-04-25T13:40:31.346948+0000","last_peered":"2024-04-25T13:40:31.346948+0000","last_clean":"2024-04-25T13:40:31.346948+0000","last_became_active":"2024-04-25T13:40:30.224161+0000","last_became_peered":"2024-04-25T13:40:30.224161+0000","last_unstale":"2024-04-25T13:40:31.346948+0000","last_undegraded":"2024-04-25T13:40:31.346948+0000","last_fullsized":"2024-04-25T13:40:31.346948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:16.868301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186576+0000","last_change":"2024-04-25T13:40:30.204030+0000","last_active":"2024-04-25T13:40:31.186576+0000","last_peered":"2024-04-25T13:40:31.186576+0000","last_clean":"2024-04-25T13:40:31.186576+0000","last_became_active":"2024-04-25T13:40:30.203298+0000","last_became_peered":"2024-04-25T13:40:30.203298+0000","last_unstale":"2024-04-25T13:40:31.186576+0000","last_undegraded":"2024-04-25T13:40:31.186576+0000","last_fullsized":"2024-04-25T13:40:31.186576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.572150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347790+0000","last_change":"2024-04-25T13:40:30.220004+0000","last_active":"2024-04-25T13:40:31.347790+0000","last_peered":"2024-04-25T13:40:31.347790+0000","last_clean":"2024-04-25T13:40:31.347790+0000","last_became_active":"2024-04-25T13:40:30.217606+0000","last_became_peered":"2024-04-25T13:40:30.217606+0000","last_unstale":"2024-04-25T13:40:31.347790+0000","last_undegraded":"2024-04-25T13:40:31.347790+0000","last_fullsized":"2024-04-25T13:40:31.347790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:48.795741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220183+0000","last_change":"2024-04-25T13:40:30.228676+0000","last_active":"2024-04-25T13:40:31.220183+0000","last_peered":"2024-04-25T13:40:31.220183+0000","last_clean":"2024-04-25T13:40:31.220183+0000","last_became_active":"2024-04-25T13:40:30.228364+0000","last_became_peered":"2024-04-25T13:40:30.228364+0000","last_unstale":"2024-04-25T13:40:31.220183+0000","last_undegraded":"2024-04-25T13:40:31.220183+0000","last_fullsized":"2024-04-25T13:40:31.220183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:09.381688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347827+0000","last_change":"2024-04-25T13:40:30.220330+0000","last_active":"2024-04-25T13:40:31.347827+0000","last_peered":"2024-04-25T13:40:31.347827+0000","last_clean":"2024-04-25T13:40:31.347827+0000","last_became_active":"2024-04-25T13:40:30.219702+0000","last_became_peered":"2024-04-25T13:40:30.219702+0000","last_unstale":"2024-04-25T13:40:31.347827+0000","last_undegraded":"2024-04-25T13:40:31.347827+0000","last_fullsized":"2024-04-25T13:40:31.347827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:24.416759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349924+0000","last_change":"2024-04-25T13:40:30.224681+0000","last_active":"2024-04-25T13:40:31.349924+0000","last_peered":"2024-04-25T13:40:31.349924+0000","last_clean":"2024-04-25T13:40:31.349924+0000","last_became_active":"2024-04-25T13:40:30.218495+0000","last_became_peered":"2024-04-25T13:40:30.218495+0000","last_unstale":"2024-04-25T13:40:31.349924+0000","last_undegraded":"2024-04-25T13:40:31.349924+0000","last_fullsized":"2024-04-25T13:40:31.349924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:21.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347112+0000","last_change":"2024-04-25T13:40:30.220292+0000","last_active":"2024-04-25T13:40:31.347112+0000","last_peered":"2024-04-25T13:40:31.347112+0000","last_clean":"2024-04-25T13:40:31.347112+0000","last_became_active":"2024-04-25T13:40:30.219473+0000","last_became_peered":"2024-04-25T13:40:30.219473+0000","last_unstale":"2024-04-25T13:40:31.347112+0000","last_undegraded":"2024-04-25T13:40:31.347112+0000","last_fullsized":"2024-04-25T13:40:31.347112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.322464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346401+0000","last_change":"2024-04-25T13:40:30.216234+0000","last_active":"2024-04-25T13:40:31.346401+0000","last_peered":"2024-04-25T13:40:31.346401+0000","last_clean":"2024-04-25T13:40:31.346401+0000","last_became_active":"2024-04-25T13:40:30.215246+0000","last_became_peered":"2024-04-25T13:40:30.215246+0000","last_unstale":"2024-04-25T13:40:31.346401+0000","last_undegraded":"2024-04-25T13:40:31.346401+0000","last_fullsized":"2024-04-25T13:40:31.346401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:09.558215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347524+0000","last_change":"2024-04-25T13:40:30.226582+0000","last_active":"2024-04-25T13:40:31.347524+0000","last_peered":"2024-04-25T13:40:31.347524+0000","last_clean":"2024-04-25T13:40:31.347524+0000","last_became_active":"2024-04-25T13:40:30.226077+0000","last_became_peered":"2024-04-25T13:40:30.226077+0000","last_unstale":"2024-04-25T13:40:31.347524+0000","last_undegraded":"2024-04-25T13:40:31.347524+0000","last_fullsized":"2024-04-25T13:40:31.347524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.951509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219854+0000","last_change":"2024-04-25T13:40:30.220604+0000","last_active":"2024-04-25T13:40:31.219854+0000","last_peered":"2024-04-25T13:40:31.219854+0000","last_clean":"2024-04-25T13:40:31.219854+0000","last_became_active":"2024-04-25T13:40:30.220289+0000","last_became_peered":"2024-04-25T13:40:30.220289+0000","last_unstale":"2024-04-25T13:40:31.219854+0000","last_undegraded":"2024-04-25T13:40:31.219854+0000","last_fullsized":"2024-04-25T13:40:31.219854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:54.152591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184983+0000","last_change":"2024-04-25T13:40:30.204488+0000","last_active":"2024-04-25T13:40:31.184983+0000","last_peered":"2024-04-25T13:40:31.184983+0000","last_clean":"2024-04-25T13:40:31.184983+0000","last_became_active":"2024-04-25T13:40:30.203206+0000","last_became_peered":"2024-04-25T13:40:30.203206+0000","last_unstale":"2024-04-25T13:40:31.184983+0000","last_undegraded":"2024-04-25T13:40:31.184983+0000","last_fullsized":"2024-04-25T13:40:31.184983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:07.299339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346652+0000","last_change":"2024-04-25T13:40:30.219870+0000","last_active":"2024-04-25T13:40:31.346652+0000","last_peered":"2024-04-25T13:40:31.346652+0000","last_clean":"2024-04-25T13:40:31.346652+0000","last_became_active":"2024-04-25T13:40:30.218894+0000","last_became_peered":"2024-04-25T13:40:30.218894+0000","last_unstale":"2024-04-25T13:40:31.346652+0000","last_undegraded":"2024-04-25T13:40:31.346652+0000","last_fullsized":"2024-04-25T13:40:31.346652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:04.359710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346438+0000","last_change":"2024-04-25T13:40:30.212775+0000","last_active":"2024-04-25T13:40:31.346438+0000","last_peered":"2024-04-25T13:40:31.346438+0000","last_clean":"2024-04-25T13:40:31.346438+0000","last_became_active":"2024-04-25T13:40:30.212382+0000","last_became_peered":"2024-04-25T13:40:30.212382+0000","last_unstale":"2024-04-25T13:40:31.346438+0000","last_undegraded":"2024-04-25T13:40:31.346438+0000","last_fullsized":"2024-04-25T13:40:31.346438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.132012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347837+0000","last_change":"2024-04-25T13:40:30.213875+0000","last_active":"2024-04-25T13:40:31.347837+0000","last_peered":"2024-04-25T13:40:31.347837+0000","last_clean":"2024-04-25T13:40:31.347837+0000","last_became_active":"2024-04-25T13:40:30.213530+0000","last_became_peered":"2024-04-25T13:40:30.213530+0000","last_unstale":"2024-04-25T13:40:31.347837+0000","last_undegraded":"2024-04-25T13:40:31.347837+0000","last_fullsized":"2024-04-25T13:40:31.347837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:16.067584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186661+0000","last_change":"2024-04-25T13:40:30.221392+0000","last_active":"2024-04-25T13:40:31.186661+0000","last_peered":"2024-04-25T13:40:31.186661+0000","last_clean":"2024-04-25T13:40:31.186661+0000","last_became_active":"2024-04-25T13:40:30.221056+0000","last_became_peered":"2024-04-25T13:40:30.221056+0000","last_unstale":"2024-04-25T13:40:31.186661+0000","last_undegraded":"2024-04-25T13:40:31.186661+0000","last_fullsized":"2024-04-25T13:40:31.186661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:54.209154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219551+0000","last_change":"2024-04-25T13:40:30.211302+0000","last_active":"2024-04-25T13:40:31.219551+0000","last_peered":"2024-04-25T13:40:31.219551+0000","last_clean":"2024-04-25T13:40:31.219551+0000","last_became_active":"2024-04-25T13:40:30.210970+0000","last_became_peered":"2024-04-25T13:40:30.210970+0000","last_unstale":"2024-04-25T13:40:31.219551+0000","last_undegraded":"2024-04-25T13:40:31.219551+0000","last_fullsized":"2024-04-25T13:40:31.219551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:19.633269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349550+0000","last_change":"2024-04-25T13:40:30.201358+0000","last_active":"2024-04-25T13:40:31.349550+0000","last_peered":"2024-04-25T13:40:31.349550+0000","last_clean":"2024-04-25T13:40:31.349550+0000","last_became_active":"2024-04-25T13:40:30.200493+0000","last_became_peered":"2024-04-25T13:40:30.200493+0000","last_unstale":"2024-04-25T13:40:31.349550+0000","last_undegraded":"2024-04-25T13:40:31.349550+0000","last_fullsized":"2024-04-25T13:40:31.349550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:38.778259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347724+0000","last_change":"2024-04-25T13:40:29.193901+0000","last_active":"2024-04-25T13:40:31.347724+0000","last_peered":"2024-04-25T13:40:31.347724+0000","last_clean":"2024-04-25T13:40:31.347724+0000","last_became_active":"2024-04-25T13:40:29.193719+0000","last_became_peered":"2024-04-25T13:40:29.193719+0000","last_unstale":"2024-04-25T13:40:31.347724+0000","last_undegraded":"2024-04-25T13:40:31.347724+0000","last_fullsized":"2024-04-25T13:40:31.347724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:57.372209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347538+0000","last_change":"2024-04-25T13:40:30.186468+0000","last_active":"2024-04-25T13:40:31.347538+0000","last_peered":"2024-04-25T13:40:31.347538+0000","last_clean":"2024-04-25T13:40:31.347538+0000","last_became_active":"2024-04-25T13:40:30.185965+0000","last_became_peered":"2024-04-25T13:40:30.185965+0000","last_unstale":"2024-04-25T13:40:31.347538+0000","last_undegraded":"2024-04-25T13:40:31.347538+0000","last_fullsized":"2024-04-25T13:40:31.347538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:42.249844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220247+0000","last_change":"2024-04-25T13:40:29.185827+0000","last_active":"2024-04-25T13:40:31.220247+0000","last_peered":"2024-04-25T13:40:31.220247+0000","last_clean":"2024-04-25T13:40:31.220247+0000","last_became_active":"2024-04-25T13:40:29.185130+0000","last_became_peered":"2024-04-25T13:40:29.185130+0000","last_unstale":"2024-04-25T13:40:31.220247+0000","last_undegraded":"2024-04-25T13:40:31.220247+0000","last_fullsized":"2024-04-25T13:40:31.220247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:46.813189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347960+0000","last_change":"2024-04-25T13:40:29.175272+0000","last_active":"2024-04-25T13:40:31.347960+0000","last_peered":"2024-04-25T13:40:31.347960+0000","last_clean":"2024-04-25T13:40:31.347960+0000","last_became_active":"2024-04-25T13:40:29.174962+0000","last_became_peered":"2024-04-25T13:40:29.174962+0000","last_unstale":"2024-04-25T13:40:31.347960+0000","last_undegraded":"2024-04-25T13:40:31.347960+0000","last_fullsized":"2024-04-25T13:40:31.347960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:14.296130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347747+0000","last_change":"2024-04-25T13:40:30.214155+0000","last_active":"2024-04-25T13:40:31.347747+0000","last_peered":"2024-04-25T13:40:31.347747+0000","last_clean":"2024-04-25T13:40:31.347747+0000","last_became_active":"2024-04-25T13:40:30.212880+0000","last_became_peered":"2024-04-25T13:40:30.212880+0000","last_unstale":"2024-04-25T13:40:31.347747+0000","last_undegraded":"2024-04-25T13:40:31.347747+0000","last_fullsized":"2024-04-25T13:40:31.347747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:17.642730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349764+0000","last_change":"2024-04-25T13:40:29.185537+0000","last_active":"2024-04-25T13:40:31.349764+0000","last_peered":"2024-04-25T13:40:31.349764+0000","last_clean":"2024-04-25T13:40:31.349764+0000","last_became_active":"2024-04-25T13:40:29.182804+0000","last_became_peered":"2024-04-25T13:40:29.182804+0000","last_unstale":"2024-04-25T13:40:31.349764+0000","last_undegraded":"2024-04-25T13:40:31.349764+0000","last_fullsized":"2024-04-25T13:40:31.349764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:45.820954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184917+0000","last_change":"2024-04-25T13:40:30.217108+0000","last_active":"2024-04-25T13:40:31.184917+0000","last_peered":"2024-04-25T13:40:31.184917+0000","last_clean":"2024-04-25T13:40:31.184917+0000","last_became_active":"2024-04-25T13:40:30.216260+0000","last_became_peered":"2024-04-25T13:40:30.216260+0000","last_unstale":"2024-04-25T13:40:31.184917+0000","last_undegraded":"2024-04-25T13:40:31.184917+0000","last_fullsized":"2024-04-25T13:40:31.184917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:23.045755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220246+0000","last_change":"2024-04-25T13:40:29.184744+0000","last_active":"2024-04-25T13:40:31.220246+0000","last_peered":"2024-04-25T13:40:31.220246+0000","last_clean":"2024-04-25T13:40:31.220246+0000","last_became_active":"2024-04-25T13:40:29.183985+0000","last_became_peered":"2024-04-25T13:40:29.183985+0000","last_unstale":"2024-04-25T13:40:31.220246+0000","last_undegraded":"2024-04-25T13:40:31.220246+0000","last_fullsized":"2024-04-25T13:40:31.220246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:56.228309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347453+0000","last_change":"2024-04-25T13:40:30.225124+0000","last_active":"2024-04-25T13:40:31.347453+0000","last_peered":"2024-04-25T13:40:31.347453+0000","last_clean":"2024-04-25T13:40:31.347453+0000","last_became_active":"2024-04-25T13:40:30.223455+0000","last_became_peered":"2024-04-25T13:40:30.223455+0000","last_unstale":"2024-04-25T13:40:31.347453+0000","last_undegraded":"2024-04-25T13:40:31.347453+0000","last_fullsized":"2024-04-25T13:40:31.347453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:03.877694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350848+0000","last_change":"2024-04-25T13:40:29.190850+0000","last_active":"2024-04-25T13:40:31.350848+0000","last_peered":"2024-04-25T13:40:31.350848+0000","last_clean":"2024-04-25T13:40:31.350848+0000","last_became_active":"2024-04-25T13:40:29.188098+0000","last_became_peered":"2024-04-25T13:40:29.188098+0000","last_unstale":"2024-04-25T13:40:31.350848+0000","last_undegraded":"2024-04-25T13:40:31.350848+0000","last_fullsized":"2024-04-25T13:40:31.350848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:07.915234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347972+0000","last_change":"2024-04-25T13:40:30.221083+0000","last_active":"2024-04-25T13:40:31.347972+0000","last_peered":"2024-04-25T13:40:31.347972+0000","last_clean":"2024-04-25T13:40:31.347972+0000","last_became_active":"2024-04-25T13:40:30.220819+0000","last_became_peered":"2024-04-25T13:40:30.220819+0000","last_unstale":"2024-04-25T13:40:31.347972+0000","last_undegraded":"2024-04-25T13:40:31.347972+0000","last_fullsized":"2024-04-25T13:40:31.347972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:43.306079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185190+0000","last_change":"2024-04-25T13:40:29.189173+0000","last_active":"2024-04-25T13:40:31.185190+0000","last_peered":"2024-04-25T13:40:31.185190+0000","last_clean":"2024-04-25T13:40:31.185190+0000","last_became_active":"2024-04-25T13:40:29.188914+0000","last_became_peered":"2024-04-25T13:40:29.188914+0000","last_unstale":"2024-04-25T13:40:31.185190+0000","last_undegraded":"2024-04-25T13:40:31.185190+0000","last_fullsized":"2024-04-25T13:40:31.185190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:12.374973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346508+0000","last_change":"2024-04-25T13:40:30.218953+0000","last_active":"2024-04-25T13:40:31.346508+0000","last_peered":"2024-04-25T13:40:31.346508+0000","last_clean":"2024-04-25T13:40:31.346508+0000","last_became_active":"2024-04-25T13:40:30.218568+0000","last_became_peered":"2024-04-25T13:40:30.218568+0000","last_unstale":"2024-04-25T13:40:31.346508+0000","last_undegraded":"2024-04-25T13:40:31.346508+0000","last_fullsized":"2024-04-25T13:40:31.346508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:02.608556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219913+0000","last_change":"2024-04-25T13:40:29.175817+0000","last_active":"2024-04-25T13:40:31.219913+0000","last_peered":"2024-04-25T13:40:31.219913+0000","last_clean":"2024-04-25T13:40:31.219913+0000","last_became_active":"2024-04-25T13:40:29.175066+0000","last_became_peered":"2024-04-25T13:40:29.175066+0000","last_unstale":"2024-04-25T13:40:31.219913+0000","last_undegraded":"2024-04-25T13:40:31.219913+0000","last_fullsized":"2024-04-25T13:40:31.219913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:27.404657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348852+0000","last_change":"2024-04-25T13:40:30.215383+0000","last_active":"2024-04-25T13:40:31.348852+0000","last_peered":"2024-04-25T13:40:31.348852+0000","last_clean":"2024-04-25T13:40:31.348852+0000","last_became_active":"2024-04-25T13:40:30.214994+0000","last_became_peered":"2024-04-25T13:40:30.214994+0000","last_unstale":"2024-04-25T13:40:31.348852+0000","last_undegraded":"2024-04-25T13:40:31.348852+0000","last_fullsized":"2024-04-25T13:40:31.348852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:49.832720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347401+0000","last_change":"2024-04-25T13:40:29.185820+0000","last_active":"2024-04-25T13:40:31.347401+0000","last_peered":"2024-04-25T13:40:31.347401+0000","last_clean":"2024-04-25T13:40:31.347401+0000","last_became_active":"2024-04-25T13:40:29.185576+0000","last_became_peered":"2024-04-25T13:40:29.185576+0000","last_unstale":"2024-04-25T13:40:31.347401+0000","last_undegraded":"2024-04-25T13:40:31.347401+0000","last_fullsized":"2024-04-25T13:40:31.347401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:18.356259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348088+0000","last_change":"2024-04-25T13:40:30.220046+0000","last_active":"2024-04-25T13:40:31.348088+0000","last_peered":"2024-04-25T13:40:31.348088+0000","last_clean":"2024-04-25T13:40:31.348088+0000","last_became_active":"2024-04-25T13:40:30.219244+0000","last_became_peered":"2024-04-25T13:40:30.219244+0000","last_unstale":"2024-04-25T13:40:31.348088+0000","last_undegraded":"2024-04-25T13:40:31.348088+0000","last_fullsized":"2024-04-25T13:40:31.348088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:34.257699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350082+0000","last_change":"2024-04-25T13:40:29.187441+0000","last_active":"2024-04-25T13:40:31.350082+0000","last_peered":"2024-04-25T13:40:31.350082+0000","last_clean":"2024-04-25T13:40:31.350082+0000","last_became_active":"2024-04-25T13:40:29.181693+0000","last_became_peered":"2024-04-25T13:40:29.181693+0000","last_unstale":"2024-04-25T13:40:31.350082+0000","last_undegraded":"2024-04-25T13:40:31.350082+0000","last_fullsized":"2024-04-25T13:40:31.350082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:57.361661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347958+0000","last_change":"2024-04-25T13:40:30.229833+0000","last_active":"2024-04-25T13:40:31.347958+0000","last_peered":"2024-04-25T13:40:31.347958+0000","last_clean":"2024-04-25T13:40:31.347958+0000","last_became_active":"2024-04-25T13:40:30.229011+0000","last_became_peered":"2024-04-25T13:40:30.229011+0000","last_unstale":"2024-04-25T13:40:31.347958+0000","last_undegraded":"2024-04-25T13:40:31.347958+0000","last_fullsized":"2024-04-25T13:40:31.347958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:15.058616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219728+0000","last_change":"2024-04-25T13:40:29.188978+0000","last_active":"2024-04-25T13:40:31.219728+0000","last_peered":"2024-04-25T13:40:31.219728+0000","last_clean":"2024-04-25T13:40:31.219728+0000","last_became_active":"2024-04-25T13:40:29.185846+0000","last_became_peered":"2024-04-25T13:40:29.185846+0000","last_unstale":"2024-04-25T13:40:31.219728+0000","last_undegraded":"2024-04-25T13:40:31.219728+0000","last_fullsized":"2024-04-25T13:40:31.219728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:08.912260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219935+0000","last_change":"2024-04-25T13:40:30.215550+0000","last_active":"2024-04-25T13:40:31.219935+0000","last_peered":"2024-04-25T13:40:31.219935+0000","last_clean":"2024-04-25T13:40:31.219935+0000","last_became_active":"2024-04-25T13:40:30.215250+0000","last_became_peered":"2024-04-25T13:40:30.215250+0000","last_unstale":"2024-04-25T13:40:31.219935+0000","last_undegraded":"2024-04-25T13:40:31.219935+0000","last_fullsized":"2024-04-25T13:40:31.219935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.530072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186915+0000","last_change":"2024-04-25T13:40:29.168980+0000","last_active":"2024-04-25T13:40:31.186915+0000","last_peered":"2024-04-25T13:40:31.186915+0000","last_clean":"2024-04-25T13:40:31.186915+0000","last_became_active":"2024-04-25T13:40:29.168218+0000","last_became_peered":"2024-04-25T13:40:29.168218+0000","last_unstale":"2024-04-25T13:40:31.186915+0000","last_undegraded":"2024-04-25T13:40:31.186915+0000","last_fullsized":"2024-04-25T13:40:31.186915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:25.060887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349934+0000","last_change":"2024-04-25T13:40:30.216707+0000","last_active":"2024-04-25T13:40:31.349934+0000","last_peered":"2024-04-25T13:40:31.349934+0000","last_clean":"2024-04-25T13:40:31.349934+0000","last_became_active":"2024-04-25T13:40:30.215907+0000","last_became_peered":"2024-04-25T13:40:30.215907+0000","last_unstale":"2024-04-25T13:40:31.349934+0000","last_undegraded":"2024-04-25T13:40:31.349934+0000","last_fullsized":"2024-04-25T13:40:31.349934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:22.984851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346871+0000","last_change":"2024-04-25T13:40:29.182090+0000","last_active":"2024-04-25T13:40:31.346871+0000","last_peered":"2024-04-25T13:40:31.346871+0000","last_clean":"2024-04-25T13:40:31.346871+0000","last_became_active":"2024-04-25T13:40:29.181228+0000","last_became_peered":"2024-04-25T13:40:29.181228+0000","last_unstale":"2024-04-25T13:40:31.346871+0000","last_undegraded":"2024-04-25T13:40:31.346871+0000","last_fullsized":"2024-04-25T13:40:31.346871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:49.643409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348554+0000","last_change":"2024-04-25T13:40:30.230355+0000","last_active":"2024-04-25T13:40:31.348554+0000","last_peered":"2024-04-25T13:40:31.348554+0000","last_clean":"2024-04-25T13:40:31.348554+0000","last_became_active":"2024-04-25T13:40:30.229397+0000","last_became_peered":"2024-04-25T13:40:30.229397+0000","last_unstale":"2024-04-25T13:40:31.348554+0000","last_undegraded":"2024-04-25T13:40:31.348554+0000","last_fullsized":"2024-04-25T13:40:31.348554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:23.803621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.187149+0000","last_change":"2024-04-25T13:40:29.160176+0000","last_active":"2024-04-25T13:40:31.187149+0000","last_peered":"2024-04-25T13:40:31.187149+0000","last_clean":"2024-04-25T13:40:31.187149+0000","last_became_active":"2024-04-25T13:40:29.159439+0000","last_became_peered":"2024-04-25T13:40:29.159439+0000","last_unstale":"2024-04-25T13:40:31.187149+0000","last_undegraded":"2024-04-25T13:40:31.187149+0000","last_fullsized":"2024-04-25T13:40:31.187149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:48.998902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347297+0000","last_change":"2024-04-25T13:40:30.224277+0000","last_active":"2024-04-25T13:40:31.347297+0000","last_peered":"2024-04-25T13:40:31.347297+0000","last_clean":"2024-04-25T13:40:31.347297+0000","last_became_active":"2024-04-25T13:40:30.221764+0000","last_became_peered":"2024-04-25T13:40:30.221764+0000","last_unstale":"2024-04-25T13:40:31.347297+0000","last_undegraded":"2024-04-25T13:40:31.347297+0000","last_fullsized":"2024-04-25T13:40:31.347297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:12.238288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185510+0000","last_change":"2024-04-25T13:40:29.182136+0000","last_active":"2024-04-25T13:40:31.185510+0000","last_peered":"2024-04-25T13:40:31.185510+0000","last_clean":"2024-04-25T13:40:31.185510+0000","last_became_active":"2024-04-25T13:40:29.181852+0000","last_became_peered":"2024-04-25T13:40:29.181852+0000","last_unstale":"2024-04-25T13:40:31.185510+0000","last_undegraded":"2024-04-25T13:40:31.185510+0000","last_fullsized":"2024-04-25T13:40:31.185510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:24.681218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350369+0000","last_change":"2024-04-25T13:40:30.217315+0000","last_active":"2024-04-25T13:40:31.350369+0000","last_peered":"2024-04-25T13:40:31.350369+0000","last_clean":"2024-04-25T13:40:31.350369+0000","last_became_active":"2024-04-25T13:40:30.214308+0000","last_became_peered":"2024-04-25T13:40:30.214308+0000","last_unstale":"2024-04-25T13:40:31.350369+0000","last_undegraded":"2024-04-25T13:40:31.350369+0000","last_fullsized":"2024-04-25T13:40:31.350369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:57.857844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186012+0000","last_change":"2024-04-25T13:40:29.189228+0000","last_active":"2024-04-25T13:40:31.186012+0000","last_peered":"2024-04-25T13:40:31.186012+0000","last_clean":"2024-04-25T13:40:31.186012+0000","last_became_active":"2024-04-25T13:40:29.189004+0000","last_became_peered":"2024-04-25T13:40:29.189004+0000","last_unstale":"2024-04-25T13:40:31.186012+0000","last_undegraded":"2024-04-25T13:40:31.186012+0000","last_fullsized":"2024-04-25T13:40:31.186012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:14.763408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349561+0000","last_change":"2024-04-25T13:40:30.223154+0000","last_active":"2024-04-25T13:40:31.349561+0000","last_peered":"2024-04-25T13:40:31.349561+0000","last_clean":"2024-04-25T13:40:31.349561+0000","last_became_active":"2024-04-25T13:40:30.222384+0000","last_became_peered":"2024-04-25T13:40:30.222384+0000","last_unstale":"2024-04-25T13:40:31.349561+0000","last_undegraded":"2024-04-25T13:40:31.349561+0000","last_fullsized":"2024-04-25T13:40:31.349561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:59.286900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350710+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:40:31.350710+0000","last_peered":"2024-04-25T13:40:31.350710+0000","last_clean":"2024-04-25T13:40:31.350710+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:40:31.350710+0000","last_undegraded":"2024-04-25T13:40:31.350710+0000","last_fullsized":"2024-04-25T13:40:31.350710+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350590+0000","last_change":"2024-04-25T13:40:29.186348+0000","last_active":"2024-04-25T13:40:31.350590+0000","last_peered":"2024-04-25T13:40:31.350590+0000","last_clean":"2024-04-25T13:40:31.350590+0000","last_became_active":"2024-04-25T13:40:29.180642+0000","last_became_peered":"2024-04-25T13:40:29.180642+0000","last_unstale":"2024-04-25T13:40:31.350590+0000","last_undegraded":"2024-04-25T13:40:31.350590+0000","last_fullsized":"2024-04-25T13:40:31.350590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:43.636874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347213+0000","last_change":"2024-04-25T13:40:30.209809+0000","last_active":"2024-04-25T13:40:31.347213+0000","last_peered":"2024-04-25T13:40:31.347213+0000","last_clean":"2024-04-25T13:40:31.347213+0000","last_became_active":"2024-04-25T13:40:30.209014+0000","last_became_peered":"2024-04-25T13:40:30.209014+0000","last_unstale":"2024-04-25T13:40:31.347213+0000","last_undegraded":"2024-04-25T13:40:31.347213+0000","last_fullsized":"2024-04-25T13:40:31.347213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:24.945222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350549+0000","last_change":"2024-04-25T13:40:29.197074+0000","last_active":"2024-04-25T13:40:31.350549+0000","last_peered":"2024-04-25T13:40:31.350549+0000","last_clean":"2024-04-25T13:40:31.350549+0000","last_became_active":"2024-04-25T13:40:29.193688+0000","last_became_peered":"2024-04-25T13:40:29.193688+0000","last_unstale":"2024-04-25T13:40:31.350549+0000","last_undegraded":"2024-04-25T13:40:31.350549+0000","last_fullsized":"2024-04-25T13:40:31.350549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:03.325908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347895+0000","last_change":"2024-04-25T13:40:30.187471+0000","last_active":"2024-04-25T13:40:31.347895+0000","last_peered":"2024-04-25T13:40:31.347895+0000","last_clean":"2024-04-25T13:40:31.347895+0000","last_became_active":"2024-04-25T13:40:30.186993+0000","last_became_peered":"2024-04-25T13:40:30.186993+0000","last_unstale":"2024-04-25T13:40:31.347895+0000","last_undegraded":"2024-04-25T13:40:31.347895+0000","last_fullsized":"2024-04-25T13:40:31.347895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.749355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.187216+0000","last_change":"2024-04-25T13:40:29.159566+0000","last_active":"2024-04-25T13:40:31.187216+0000","last_peered":"2024-04-25T13:40:31.187216+0000","last_clean":"2024-04-25T13:40:31.187216+0000","last_became_active":"2024-04-25T13:40:29.159054+0000","last_became_peered":"2024-04-25T13:40:29.159054+0000","last_unstale":"2024-04-25T13:40:31.187216+0000","last_undegraded":"2024-04-25T13:40:31.187216+0000","last_fullsized":"2024-04-25T13:40:31.187216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:03.455177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185373+0000","last_change":"2024-04-25T13:40:30.203352+0000","last_active":"2024-04-25T13:40:31.185373+0000","last_peered":"2024-04-25T13:40:31.185373+0000","last_clean":"2024-04-25T13:40:31.185373+0000","last_became_active":"2024-04-25T13:40:30.202463+0000","last_became_peered":"2024-04-25T13:40:30.202463+0000","last_unstale":"2024-04-25T13:40:31.185373+0000","last_undegraded":"2024-04-25T13:40:31.185373+0000","last_fullsized":"2024-04-25T13:40:31.185373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:01.912451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350985+0000","last_change":"2024-04-25T13:40:29.190061+0000","last_active":"2024-04-25T13:40:31.350985+0000","last_peered":"2024-04-25T13:40:31.350985+0000","last_clean":"2024-04-25T13:40:31.350985+0000","last_became_active":"2024-04-25T13:40:29.182897+0000","last_became_peered":"2024-04-25T13:40:29.182897+0000","last_unstale":"2024-04-25T13:40:31.350985+0000","last_undegraded":"2024-04-25T13:40:31.350985+0000","last_fullsized":"2024-04-25T13:40:31.350985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:31.168555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346748+0000","last_change":"2024-04-25T13:40:30.187160+0000","last_active":"2024-04-25T13:40:31.346748+0000","last_peered":"2024-04-25T13:40:31.346748+0000","last_clean":"2024-04-25T13:40:31.346748+0000","last_became_active":"2024-04-25T13:40:30.186695+0000","last_became_peered":"2024-04-25T13:40:30.186695+0000","last_unstale":"2024-04-25T13:40:31.346748+0000","last_undegraded":"2024-04-25T13:40:31.346748+0000","last_fullsized":"2024-04-25T13:40:31.346748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:57.859568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186430+0000","last_change":"2024-04-25T13:40:29.168378+0000","last_active":"2024-04-25T13:40:31.186430+0000","last_peered":"2024-04-25T13:40:31.186430+0000","last_clean":"2024-04-25T13:40:31.186430+0000","last_became_active":"2024-04-25T13:40:29.167515+0000","last_became_peered":"2024-04-25T13:40:29.167515+0000","last_unstale":"2024-04-25T13:40:31.186430+0000","last_undegraded":"2024-04-25T13:40:31.186430+0000","last_fullsized":"2024-04-25T13:40:31.186430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.417158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346850+0000","last_change":"2024-04-25T13:40:30.214354+0000","last_active":"2024-04-25T13:40:31.346850+0000","last_peered":"2024-04-25T13:40:31.346850+0000","last_clean":"2024-04-25T13:40:31.346850+0000","last_became_active":"2024-04-25T13:40:30.213970+0000","last_became_peered":"2024-04-25T13:40:30.213970+0000","last_unstale":"2024-04-25T13:40:31.346850+0000","last_undegraded":"2024-04-25T13:40:31.346850+0000","last_fullsized":"2024-04-25T13:40:31.346850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:54.278471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348113+0000","last_change":"2024-04-25T13:40:29.197843+0000","last_active":"2024-04-25T13:40:31.348113+0000","last_peered":"2024-04-25T13:40:31.348113+0000","last_clean":"2024-04-25T13:40:31.348113+0000","last_became_active":"2024-04-25T13:40:29.196299+0000","last_became_peered":"2024-04-25T13:40:29.196299+0000","last_unstale":"2024-04-25T13:40:31.348113+0000","last_undegraded":"2024-04-25T13:40:31.348113+0000","last_fullsized":"2024-04-25T13:40:31.348113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:41.785353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185424+0000","last_change":"2024-04-25T13:40:30.212599+0000","last_active":"2024-04-25T13:40:31.185424+0000","last_peered":"2024-04-25T13:40:31.185424+0000","last_clean":"2024-04-25T13:40:31.185424+0000","last_became_active":"2024-04-25T13:40:30.211967+0000","last_became_peered":"2024-04-25T13:40:30.211967+0000","last_unstale":"2024-04-25T13:40:31.185424+0000","last_undegraded":"2024-04-25T13:40:31.185424+0000","last_fullsized":"2024-04-25T13:40:31.185424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:28.946427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219865+0000","last_change":"2024-04-25T13:40:29.175520+0000","last_active":"2024-04-25T13:40:31.219865+0000","last_peered":"2024-04-25T13:40:31.219865+0000","last_clean":"2024-04-25T13:40:31.219865+0000","last_became_active":"2024-04-25T13:40:29.174561+0000","last_became_peered":"2024-04-25T13:40:29.174561+0000","last_unstale":"2024-04-25T13:40:31.219865+0000","last_undegraded":"2024-04-25T13:40:31.219865+0000","last_fullsized":"2024-04-25T13:40:31.219865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:55.788741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346789+0000","last_change":"2024-04-25T13:40:30.217175+0000","last_active":"2024-04-25T13:40:31.346789+0000","last_peered":"2024-04-25T13:40:31.346789+0000","last_clean":"2024-04-25T13:40:31.346789+0000","last_became_active":"2024-04-25T13:40:30.216851+0000","last_became_peered":"2024-04-25T13:40:30.216851+0000","last_unstale":"2024-04-25T13:40:31.346789+0000","last_undegraded":"2024-04-25T13:40:31.346789+0000","last_fullsized":"2024-04-25T13:40:31.346789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:12.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219908+0000","last_change":"2024-04-25T13:40:29.194455+0000","last_active":"2024-04-25T13:40:31.219908+0000","last_peered":"2024-04-25T13:40:31.219908+0000","last_clean":"2024-04-25T13:40:31.219908+0000","last_became_active":"2024-04-25T13:40:29.193686+0000","last_became_peered":"2024-04-25T13:40:29.193686+0000","last_unstale":"2024-04-25T13:40:31.219908+0000","last_undegraded":"2024-04-25T13:40:31.219908+0000","last_fullsized":"2024-04-25T13:40:31.219908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:27.100097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347083+0000","last_change":"2024-04-25T13:40:30.211947+0000","last_active":"2024-04-25T13:40:31.347083+0000","last_peered":"2024-04-25T13:40:31.347083+0000","last_clean":"2024-04-25T13:40:31.347083+0000","last_became_active":"2024-04-25T13:40:30.202151+0000","last_became_peered":"2024-04-25T13:40:30.202151+0000","last_unstale":"2024-04-25T13:40:31.347083+0000","last_undegraded":"2024-04-25T13:40:31.347083+0000","last_fullsized":"2024-04-25T13:40:31.347083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.174571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350372+0000","last_change":"2024-04-25T13:40:29.200572+0000","last_active":"2024-04-25T13:40:31.350372+0000","last_peered":"2024-04-25T13:40:31.350372+0000","last_clean":"2024-04-25T13:40:31.350372+0000","last_became_active":"2024-04-25T13:40:29.199836+0000","last_became_peered":"2024-04-25T13:40:29.199836+0000","last_unstale":"2024-04-25T13:40:31.350372+0000","last_undegraded":"2024-04-25T13:40:31.350372+0000","last_fullsized":"2024-04-25T13:40:31.350372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:38.068886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349670+0000","last_change":"2024-04-25T13:40:30.225985+0000","last_active":"2024-04-25T13:40:31.349670+0000","last_peered":"2024-04-25T13:40:31.349670+0000","last_clean":"2024-04-25T13:40:31.349670+0000","last_became_active":"2024-04-25T13:40:30.224959+0000","last_became_peered":"2024-04-25T13:40:30.224959+0000","last_unstale":"2024-04-25T13:40:31.349670+0000","last_undegraded":"2024-04-25T13:40:31.349670+0000","last_fullsized":"2024-04-25T13:40:31.349670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:13.498711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186159+0000","last_change":"2024-04-25T13:40:29.196670+0000","last_active":"2024-04-25T13:40:31.186159+0000","last_peered":"2024-04-25T13:40:31.186159+0000","last_clean":"2024-04-25T13:40:31.186159+0000","last_became_active":"2024-04-25T13:40:29.196115+0000","last_became_peered":"2024-04-25T13:40:29.196115+0000","last_unstale":"2024-04-25T13:40:31.186159+0000","last_undegraded":"2024-04-25T13:40:31.186159+0000","last_fullsized":"2024-04-25T13:40:31.186159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:35.705063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347672+0000","last_change":"2024-04-25T13:40:30.203777+0000","last_active":"2024-04-25T13:40:31.347672+0000","last_peered":"2024-04-25T13:40:31.347672+0000","last_clean":"2024-04-25T13:40:31.347672+0000","last_became_active":"2024-04-25T13:40:30.202751+0000","last_became_peered":"2024-04-25T13:40:30.202751+0000","last_unstale":"2024-04-25T13:40:31.347672+0000","last_undegraded":"2024-04-25T13:40:31.347672+0000","last_fullsized":"2024-04-25T13:40:31.347672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:09.528068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185653+0000","last_change":"2024-04-25T13:40:29.176333+0000","last_active":"2024-04-25T13:40:31.185653+0000","last_peered":"2024-04-25T13:40:31.185653+0000","last_clean":"2024-04-25T13:40:31.185653+0000","last_became_active":"2024-04-25T13:40:29.176078+0000","last_became_peered":"2024-04-25T13:40:29.176078+0000","last_unstale":"2024-04-25T13:40:31.185653+0000","last_undegraded":"2024-04-25T13:40:31.185653+0000","last_fullsized":"2024-04-25T13:40:31.185653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:19.200730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346905+0000","last_change":"2024-04-25T13:40:30.216029+0000","last_active":"2024-04-25T13:40:31.346905+0000","last_peered":"2024-04-25T13:40:31.346905+0000","last_clean":"2024-04-25T13:40:31.346905+0000","last_became_active":"2024-04-25T13:40:30.215353+0000","last_became_peered":"2024-04-25T13:40:30.215353+0000","last_unstale":"2024-04-25T13:40:31.346905+0000","last_undegraded":"2024-04-25T13:40:31.346905+0000","last_fullsized":"2024-04-25T13:40:31.346905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:10.825968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349163+0000","last_change":"2024-04-25T13:40:29.195078+0000","last_active":"2024-04-25T13:40:31.349163+0000","last_peered":"2024-04-25T13:40:31.349163+0000","last_clean":"2024-04-25T13:40:31.349163+0000","last_became_active":"2024-04-25T13:40:29.193847+0000","last_became_peered":"2024-04-25T13:40:29.193847+0000","last_unstale":"2024-04-25T13:40:31.349163+0000","last_undegraded":"2024-04-25T13:40:31.349163+0000","last_fullsized":"2024-04-25T13:40:31.349163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:33.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219558+0000","last_change":"2024-04-25T13:40:30.205595+0000","last_active":"2024-04-25T13:40:31.219558+0000","last_peered":"2024-04-25T13:40:31.219558+0000","last_clean":"2024-04-25T13:40:31.219558+0000","last_became_active":"2024-04-25T13:40:30.204793+0000","last_became_peered":"2024-04-25T13:40:30.204793+0000","last_unstale":"2024-04-25T13:40:31.219558+0000","last_undegraded":"2024-04-25T13:40:31.219558+0000","last_fullsized":"2024-04-25T13:40:31.219558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:55.486355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349463+0000","last_change":"2024-04-25T13:40:30.202452+0000","last_active":"2024-04-25T13:40:31.349463+0000","last_peered":"2024-04-25T13:40:31.349463+0000","last_clean":"2024-04-25T13:40:31.349463+0000","last_became_active":"2024-04-25T13:40:30.201683+0000","last_became_peered":"2024-04-25T13:40:30.201683+0000","last_unstale":"2024-04-25T13:40:31.349463+0000","last_undegraded":"2024-04-25T13:40:31.349463+0000","last_fullsized":"2024-04-25T13:40:31.349463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.132452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350105+0000","last_change":"2024-04-25T13:40:29.199128+0000","last_active":"2024-04-25T13:40:31.350105+0000","last_peered":"2024-04-25T13:40:31.350105+0000","last_clean":"2024-04-25T13:40:31.350105+0000","last_became_active":"2024-04-25T13:40:29.198449+0000","last_became_peered":"2024-04-25T13:40:29.198449+0000","last_unstale":"2024-04-25T13:40:31.350105+0000","last_undegraded":"2024-04-25T13:40:31.350105+0000","last_fullsized":"2024-04-25T13:40:31.350105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:08.596939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347469+0000","last_change":"2024-04-25T13:40:30.220740+0000","last_active":"2024-04-25T13:40:31.347469+0000","last_peered":"2024-04-25T13:40:31.347469+0000","last_clean":"2024-04-25T13:40:31.347469+0000","last_became_active":"2024-04-25T13:40:30.218927+0000","last_became_peered":"2024-04-25T13:40:30.218927+0000","last_unstale":"2024-04-25T13:40:31.347469+0000","last_undegraded":"2024-04-25T13:40:31.347469+0000","last_fullsized":"2024-04-25T13:40:31.347469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.133104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349098+0000","last_change":"2024-04-25T13:40:29.199401+0000","last_active":"2024-04-25T13:40:31.349098+0000","last_peered":"2024-04-25T13:40:31.349098+0000","last_clean":"2024-04-25T13:40:31.349098+0000","last_became_active":"2024-04-25T13:40:29.198182+0000","last_became_peered":"2024-04-25T13:40:29.198182+0000","last_unstale":"2024-04-25T13:40:31.349098+0000","last_undegraded":"2024-04-25T13:40:31.349098+0000","last_fullsized":"2024-04-25T13:40:31.349098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:31.807935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347497+0000","last_change":"2024-04-25T13:40:30.213026+0000","last_active":"2024-04-25T13:40:31.347497+0000","last_peered":"2024-04-25T13:40:31.347497+0000","last_clean":"2024-04-25T13:40:31.347497+0000","last_became_active":"2024-04-25T13:40:30.203212+0000","last_became_peered":"2024-04-25T13:40:30.203212+0000","last_unstale":"2024-04-25T13:40:31.347497+0000","last_undegraded":"2024-04-25T13:40:31.347497+0000","last_fullsized":"2024-04-25T13:40:31.347497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:24.180245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347687+0000","last_change":"2024-04-25T13:40:29.173054+0000","last_active":"2024-04-25T13:40:31.347687+0000","last_peered":"2024-04-25T13:40:31.347687+0000","last_clean":"2024-04-25T13:40:31.347687+0000","last_became_active":"2024-04-25T13:40:29.172375+0000","last_became_peered":"2024-04-25T13:40:29.172375+0000","last_unstale":"2024-04-25T13:40:31.347687+0000","last_undegraded":"2024-04-25T13:40:31.347687+0000","last_fullsized":"2024-04-25T13:40:31.347687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:26.996154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347229+0000","last_change":"2024-04-25T13:40:30.214823+0000","last_active":"2024-04-25T13:40:31.347229+0000","last_peered":"2024-04-25T13:40:31.347229+0000","last_clean":"2024-04-25T13:40:31.347229+0000","last_became_active":"2024-04-25T13:40:30.210600+0000","last_became_peered":"2024-04-25T13:40:30.210600+0000","last_unstale":"2024-04-25T13:40:31.347229+0000","last_undegraded":"2024-04-25T13:40:31.347229+0000","last_fullsized":"2024-04-25T13:40:31.347229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:14.151771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348125+0000","last_change":"2024-04-25T13:40:29.192636+0000","last_active":"2024-04-25T13:40:31.348125+0000","last_peered":"2024-04-25T13:40:31.348125+0000","last_clean":"2024-04-25T13:40:31.348125+0000","last_became_active":"2024-04-25T13:40:29.192248+0000","last_became_peered":"2024-04-25T13:40:29.192248+0000","last_unstale":"2024-04-25T13:40:31.348125+0000","last_undegraded":"2024-04-25T13:40:31.348125+0000","last_fullsized":"2024-04-25T13:40:31.348125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:25.540341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929100,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929066,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338220,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:40:34.714 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:40:34.715 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T13:40:34.715 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T13:40:34.715 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T13:40:34.715 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T13:40:34.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b8b89640 1 Processor -- start 2024-04-25T13:40:34.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b8b89640 1 -- start start 2024-04-25T13:40:34.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b8b89640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14b40b9240 0x7f14b40cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b8b89640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b40cfb20 con 0x7f14b40d04e0 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b8b89640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14b40b6b70 con 0x7f14b40b7650 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b8b89640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f14b40b6920 con 0x7f14b40b9240 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b1d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f14b40d04e0 0x7f14b40d08f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35604/0 (socket says 172.21.15.80:35604) 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b1d9b640 1 -- 172.21.15.80:0/3361033980 learned_addr learned my addr 172.21.15.80:0/3361033980 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b259c640 1 --2- 172.21.15.80:0/3361033980 >> v2:172.21.15.87:3300/0 conn(0x7f14b40b9240 0x7f14b40cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1635639756 0 0) 0x7f14b40cfb20 con 0x7f14b40d04e0 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f149c003530 con 0x7f14b40d04e0 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1757190496 0 0) 0x7f149c003530 con 0x7f14b40d04e0 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 >> v2:172.21.15.87:3300/0 conn(0x7f14b40b9240 msgr2=0x7f14b40cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 --2- 172.21.15.80:0/3361033980 >> v2:172.21.15.87:3300/0 conn(0x7f14b40b9240 0x7f14b40cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 >> v1:172.21.15.87:6789/0 conn(0x7f14b40b7650 legacy=0x7f14b40b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b40b6e50 con 0x7f14b40d04e0 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f14a8001680 con 0x7f14b40d04e0 2024-04-25T13:40:34.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 >> v1:172.21.15.80:6789/0 conn(0x7f14b40d04e0 legacy=0x7f14b40d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c0039c0 0x7f149c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f149c004810 0x7f149c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f149c005260 0x7f149c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f14b40b6920 con 0x7f149c004810 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f149c00c1f0 con 0x7f149c0039c0 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.888+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f149c00c3f0 con 0x7f149c005260 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b1d9b640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f149c004810 0x7f149c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b259c640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c0039c0 0x7f149c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b1d9b640 1 -- 172.21.15.80:0/3361033980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f149c005260 msgr2=0x7f149c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b1d9b640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f149c005260 0x7f149c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b1d9b640 1 -- 172.21.15.80:0/3361033980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c0039c0 msgr2=0x7f149c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b1d9b640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c0039c0 0x7f149c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b1d9b640 1 -- 172.21.15.80:0/3361033980 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149c003530 con 0x7f149c004810 2024-04-25T13:40:34.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b2d9d640 1 -- 172.21.15.80:0/3361033980 reap_dead start 2024-04-25T13:40:34.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b1d9b640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f149c004810 0x7f149c004c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c210 tx=0x7f14a801cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe15970a7ec3436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8007d40 con 0x7f149c004810 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14a80a9070 con 0x7f149c004810 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b159a640 1 -- 172.21.15.80:0/3361033980 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14a8005d80 con 0x7f149c004810 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b8b89640 1 -- 172.21.15.80:0/3361033980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f149c004810 msgr2=0x7f149c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b8b89640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f149c004810 0x7f149c004c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f14a800c210 tx=0x7f14a801cc70 comp rx=0 tx=0).stop 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b8b89640 1 -- 172.21.15.80:0/3361033980 shutdown_connections 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b8b89640 1 --2- 172.21.15.80:0/3361033980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f149c004810 0x7f149c004c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b8b89640 1 -- 172.21.15.80:0/3361033980 >> 172.21.15.80:0/3361033980 conn(0x7f14b40d3ad0 msgr2=0x7f14b40d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b8b89640 1 -- 172.21.15.80:0/3361033980 shutdown_connections 2024-04-25T13:40:34.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.892+0000 7f14b8b89640 1 -- 172.21.15.80:0/3361033980 wait complete. 2024-04-25T13:40:34.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 Processor -- start 2024-04-25T13:40:34.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 -- start start 2024-04-25T13:40:34.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c004810 0x7f14b415d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14b40b06f0 0x7f14b40ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14b40cd0d0 0x7f14b41fb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f14b40d1df0 con 0x7f14b40cd0d0 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f14b40d1c70 con 0x7f149c004810 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f14b40d1f70 con 0x7f14b40b06f0 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14b40cd0d0 0x7f14b41fb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14b40cd0d0 0x7f14b41fb2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36042/0 (socket says 172.21.15.80:36042) 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 -- 172.21.15.80:0/2313092456 learned_addr learned my addr 172.21.15.80:0/2313092456 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b1d9b640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14b40b06f0 0x7f14b40ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b259c640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c004810 0x7f14b415d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 -- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14b40b06f0 msgr2=0x7f14b40ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14b40b06f0 0x7f14b40ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 -- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c004810 msgr2=0x7f14b415d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:34.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c004810 0x7f14b415d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:34.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 -- 172.21.15.80:0/2313092456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14b40cd4e0 con 0x7f14b40cd0d0 2024-04-25T13:40:34.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b2d9d640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14b40cd0d0 0x7f14b41fb2b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b97d0 tx=0x7f14ac00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14a2ffd640 1 -- 172.21.15.80:0/2313092456 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac005d50 con 0x7f14b40cd0d0 2024-04-25T13:40:34.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14a2ffd640 1 -- 172.21.15.80:0/2313092456 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14ac003980 con 0x7f14b40cd0d0 2024-04-25T13:40:34.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14b40b31b0 con 0x7f14b40cd0d0 2024-04-25T13:40:34.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14a2ffd640 1 -- 172.21.15.80:0/2313092456 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac003d70 con 0x7f14b40cd0d0 2024-04-25T13:40:34.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.896+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14b40b3770 con 0x7f14b40cd0d0 2024-04-25T13:40:34.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.900+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14b40a1730 con 0x7f14b40cd0d0 2024-04-25T13:40:34.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.900+0000 7f14a2ffd640 1 -- 172.21.15.80:0/2313092456 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14ac00a640 con 0x7f14b40cd0d0 2024-04-25T13:40:34.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.900+0000 7f14a2ffd640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f149406f400 0x7f1494071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:34.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.900+0000 7f14a2ffd640 1 -- 172.21.15.80:0/2313092456 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f14ac08e700 con 0x7f14b40cd0d0 2024-04-25T13:40:34.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.900+0000 7f14b259c640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f149406f400 0x7f1494071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:34.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.900+0000 7f14b259c640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f149406f400 0x7f1494071910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14b4076e80 tx=0x7f14a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:34.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:34.904+0000 7f14a2ffd640 1 -- 172.21.15.80:0/2313092456 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14b40cd4e0 con 0x7f14b40cd0d0 2024-04-25T13:40:35.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.088+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f14b4074c30 con 0x7f14b40cd0d0 2024-04-25T13:40:35.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14a2ffd640 1 -- 172.21.15.80:0/2313092456 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+179545 (secure 0 0 0) 0x7f14ac05b5f0 con 0x7f14b40cd0d0 2024-04-25T13:40:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f149406f400 msgr2=0x7f1494071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f149406f400 0x7f1494071910 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f14b4076e80 tx=0x7f14a4008040 comp rx=0 tx=0).stop 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14b40cd0d0 msgr2=0x7f14b41fb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14b40cd0d0 0x7f14b41fb2b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f14b40b97d0 tx=0x7f14ac00cb90 comp rx=0 tx=0).stop 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 shutdown_connections 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f149406f400 0x7f1494071910 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14b40cd0d0 0x7f14b41fb2b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14b40b06f0 0x7f14b40ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 --2- 172.21.15.80:0/2313092456 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f149c004810 0x7f14b415d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.096+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 >> 172.21.15.80:0/2313092456 conn(0x7f14b40d3ad0 msgr2=0x7f14b415df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.100+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 shutdown_connections 2024-04-25T13:40:35.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.100+0000 7f14b8b89640 1 -- 172.21.15.80:0/2313092456 wait complete. 2024-04-25T13:40:35.116 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":1633598483},{"type":"v1","addr":"172.21.15.87:6833","nonce":1633598483}]},"active_addr":"172.21.15.87:6833/1633598483","active_change":"2024-04-25T13:40:16.074264+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":1720457716}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3439659350}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":1477528236}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3272546997}]}]} 2024-04-25T13:40:35.119 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T13:40:35.119 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T13:40:35.119 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:35.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a3236640 1 Processor -- start 2024-04-25T13:40:35.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a3236640 1 -- start start 2024-04-25T13:40:35.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a3236640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd89c161f90 0x7fd89c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a3236640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 0x7fd89c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a3236640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd89c0d0d70 con 0x7fd89c0d0510 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a3236640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd89c0d1050 con 0x7fd89c161f90 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a3236640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd89c06dca0 con 0x7fd89c15ec40 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a0fd2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd89c161f90 0x7fd89c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd89bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd89c0d0510 0x7fd89c15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35608/0 (socket says 172.21.15.80:35608) 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd89bfff640 1 -- 172.21.15.80:0/2225676169 learned_addr learned my addr 172.21.15.80:0/2225676169 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a17d3640 1 --2- 172.21.15.80:0/2225676169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 0x7fd89c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a0fd2640 1 -- 172.21.15.80:0/2225676169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 msgr2=0x7fd89c15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a0fd2640 1 --2- 172.21.15.80:0/2225676169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 0x7fd89c15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a0fd2640 1 -- 172.21.15.80:0/2225676169 >> v1:172.21.15.80:6789/0 conn(0x7fd89c0d0510 legacy=0x7fd89c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd89b7fe640 1 -- 172.21.15.80:0/2225676169 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2923230712 0 0) 0x7fd89c0d0d70 con 0x7fd89c0d0510 2024-04-25T13:40:35.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.256+0000 7fd8a0fd2640 1 -- 172.21.15.80:0/2225676169 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c06df80 con 0x7fd89c161f90 2024-04-25T13:40:35.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a0fd2640 1 --2- 172.21.15.80:0/2225676169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c0cffd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd8940097c0 tx=0x7fd89401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef78eef51b403998 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd89b7fe640 1 -- 172.21.15.80:0/2225676169 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8940b0070 con 0x7fd89c161f90 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd89b7fe640 1 -- 172.21.15.80:0/2225676169 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8940ab600 con 0x7fd89c161f90 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd89b7fe640 1 -- 172.21.15.80:0/2225676169 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8940ab9d0 con 0x7fd89c161f90 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a3236640 1 -- 172.21.15.80:0/2225676169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 msgr2=0x7fd89c0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a3236640 1 --2- 172.21.15.80:0/2225676169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c0cffd0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd8940097c0 tx=0x7fd89401c3b0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a3236640 1 -- 172.21.15.80:0/2225676169 shutdown_connections 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a3236640 1 --2- 172.21.15.80:0/2225676169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 0x7fd89c15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a3236640 1 --2- 172.21.15.80:0/2225676169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c0cffd0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a3236640 1 -- 172.21.15.80:0/2225676169 >> 172.21.15.80:0/2225676169 conn(0x7fd89c06d080 msgr2=0x7fd89c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:35.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a3236640 1 -- 172.21.15.80:0/2225676169 shutdown_connections 2024-04-25T13:40:35.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.260+0000 7fd8a3236640 1 -- 172.21.15.80:0/2225676169 wait complete. 2024-04-25T13:40:35.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 Processor -- start 2024-04-25T13:40:35.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 -- start start 2024-04-25T13:40:35.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c0d0510 0x7fd89c142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 0x7fd89c1430d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c06f4b0 con 0x7fd89c161f90 2024-04-25T13:40:35.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c06f630 con 0x7fd89c0d0510 2024-04-25T13:40:35.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd89c0d0920 con 0x7fd89c15ec40 2024-04-25T13:40:35.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c140930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36066/0 (socket says 172.21.15.80:36066) 2024-04-25T13:40:35.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 -- 172.21.15.80:0/4093243575 learned_addr learned my addr 172.21.15.80:0/4093243575 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:35.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd89bfff640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 0x7fd89c1430d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a0fd2640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c0d0510 0x7fd89c142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 -- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 msgr2=0x7fd89c1430d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 0x7fd89c1430d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 -- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c0d0510 msgr2=0x7fd89c142b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c0d0510 0x7fd89c142b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 -- 172.21.15.80:0/4093243575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c0afa10 con 0x7fd89c161f90 2024-04-25T13:40:35.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a17d3640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c140930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd88c007d80 tx=0x7fd88c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:35.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8997fa640 1 -- 172.21.15.80:0/4093243575 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c014420 con 0x7fd89c161f90 2024-04-25T13:40:35.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8997fa640 1 -- 172.21.15.80:0/4093243575 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd88c019070 con 0x7fd89c161f90 2024-04-25T13:40:35.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89c0acdc0 con 0x7fd89c161f90 2024-04-25T13:40:35.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8997fa640 1 -- 172.21.15.80:0/4093243575 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c014800 con 0x7fd89c161f90 2024-04-25T13:40:35.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89c0ad380 con 0x7fd89c161f90 2024-04-25T13:40:35.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.264+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd89c06a3a0 con 0x7fd89c161f90 2024-04-25T13:40:35.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.268+0000 7fd8997fa640 1 -- 172.21.15.80:0/4093243575 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd88c0149a0 con 0x7fd89c161f90 2024-04-25T13:40:35.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.268+0000 7fd8997fa640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd87c06f400 0x7fd87c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.268+0000 7fd8997fa640 1 -- 172.21.15.80:0/4093243575 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd88c08fb50 con 0x7fd89c161f90 2024-04-25T13:40:35.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.268+0000 7fd8a0fd2640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd87c06f400 0x7fd87c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.268+0000 7fd8a0fd2640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd87c06f400 0x7fd87c071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8940ad000 tx=0x7fd8940a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:35.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.272+0000 7fd8997fa640 1 -- 172.21.15.80:0/4093243575 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd88c05ca40 con 0x7fd89c161f90 2024-04-25T13:40:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.424+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd89c0af7a0 con 0x7fd89c161f90 2024-04-25T13:40:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.424+0000 7fd8997fa640 1 -- 172.21.15.80:0/4093243575 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7fd88c05c340 con 0x7fd89c161f90 2024-04-25T13:40:35.428 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:35.428 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:35.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd87c06f400 msgr2=0x7fd87c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd87c06f400 0x7fd87c071910 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd8940ad000 tx=0x7fd8940a6000 comp rx=0 tx=0).stop 2024-04-25T13:40:35.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 msgr2=0x7fd89c140930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c140930 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd88c007d80 tx=0x7fd88c00e4b0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 shutdown_connections 2024-04-25T13:40:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd87c06f400 0x7fd87c071910 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c161f90 0x7fd89c140930 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd89c15ec40 0x7fd89c1430d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 --2- 172.21.15.80:0/4093243575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c0d0510 0x7fd89c142b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 >> 172.21.15.80:0/4093243575 conn(0x7fd89c06d080 msgr2=0x7fd89c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:35.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 shutdown_connections 2024-04-25T13:40:35.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.428+0000 7fd8a3236640 1 -- 172.21.15.80:0/4093243575 wait complete. 2024-04-25T13:40:35.447 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T13:40:35.447 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:35.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14a2e2640 1 Processor -- start 2024-04-25T13:40:35.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14a2e2640 1 -- start start 2024-04-25T13:40:35.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14a2e2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440b0ed0 0x7fb14415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14a2e2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1440ce730 0x7fb1440ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14a2e2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb1441608f0 0x7fb144160d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14a2e2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb14415cda0 con 0x7fb1440ce730 2024-04-25T13:40:35.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14a2e2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb144073710 con 0x7fb1441608f0 2024-04-25T13:40:35.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14a2e2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1440739f0 con 0x7fb1440b0ed0 2024-04-25T13:40:35.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb1437fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1440ce730 0x7fb1440ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb1437fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1440ce730 0x7fb1440ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36074/0 (socket says 172.21.15.80:36074) 2024-04-25T13:40:35.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb1437fe640 1 -- 172.21.15.80:0/491489538 learned_addr learned my addr 172.21.15.80:0/491489538 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:35.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb14887f640 1 --2- 172.21.15.80:0/491489538 >> v2:172.21.15.87:3300/0 conn(0x7fb1441608f0 0x7fb144160d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb143fff640 1 --2- 172.21.15.80:0/491489538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440b0ed0 0x7fb14415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb1437fe640 1 -- 172.21.15.80:0/491489538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440b0ed0 msgr2=0x7fb14415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb1437fe640 1 --2- 172.21.15.80:0/491489538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440b0ed0 0x7fb14415c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb1437fe640 1 -- 172.21.15.80:0/491489538 >> v2:172.21.15.87:3300/0 conn(0x7fb1441608f0 msgr2=0x7fb144160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb1437fe640 1 --2- 172.21.15.80:0/491489538 >> v2:172.21.15.87:3300/0 conn(0x7fb1441608f0 0x7fb144160d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.580+0000 7fb1437fe640 1 -- 172.21.15.80:0/491489538 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb144073cd0 con 0x7fb1440ce730 2024-04-25T13:40:35.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb1437fe640 1 --2- 172.21.15.80:0/491489538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1440ce730 0x7fb1440ceb40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb1440b4950 tx=0x7fb12c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4870c455a511d21e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:35.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb142ffd640 1 -- 172.21.15.80:0/491489538 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0b5070 con 0x7fb1440ce730 2024-04-25T13:40:35.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb142ffd640 1 -- 172.21.15.80:0/491489538 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb12c0a89d0 con 0x7fb1440ce730 2024-04-25T13:40:35.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb142ffd640 1 -- 172.21.15.80:0/491489538 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0a8da0 con 0x7fb1440ce730 2024-04-25T13:40:35.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 -- 172.21.15.80:0/491489538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1440ce730 msgr2=0x7fb1440ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/491489538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1440ce730 0x7fb1440ceb40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb1440b4950 tx=0x7fb12c01ed10 comp rx=0 tx=0).stop 2024-04-25T13:40:35.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 -- 172.21.15.80:0/491489538 shutdown_connections 2024-04-25T13:40:35.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/491489538 >> v2:172.21.15.87:3300/0 conn(0x7fb1441608f0 0x7fb144160d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/491489538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1440ce730 0x7fb1440ceb40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/491489538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440b0ed0 0x7fb14415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 -- 172.21.15.80:0/491489538 >> 172.21.15.80:0/491489538 conn(0x7fb14406d080 msgr2=0x7fb1440b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:35.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 -- 172.21.15.80:0/491489538 shutdown_connections 2024-04-25T13:40:35.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.584+0000 7fb14a2e2640 1 -- 172.21.15.80:0/491489538 wait complete. 2024-04-25T13:40:35.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 Processor -- start 2024-04-25T13:40:35.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 -- start start 2024-04-25T13:40:35.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1440b0ed0 0x7fb1441392e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440ce730 0x7fb14414a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1441608f0 0x7fb14406dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb1440733a0 con 0x7fb1441608f0 2024-04-25T13:40:35.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb1440b12e0 con 0x7fb1440b0ed0 2024-04-25T13:40:35.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb144161240 con 0x7fb1440ce730 2024-04-25T13:40:35.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1441608f0 0x7fb14406dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1441608f0 0x7fb14406dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36088/0 (socket says 172.21.15.80:36088) 2024-04-25T13:40:35.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 -- 172.21.15.80:0/295286553 learned_addr learned my addr 172.21.15.80:0/295286553 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:35.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb143fff640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1440b0ed0 0x7fb1441392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb1437fe640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440ce730 0x7fb14414a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 -- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440ce730 msgr2=0x7fb14414a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440ce730 0x7fb14414a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 -- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1440b0ed0 msgr2=0x7fb1441392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1440b0ed0 0x7fb1441392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 -- 172.21.15.80:0/295286553 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb144148c50 con 0x7fb1441608f0 2024-04-25T13:40:35.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb1437fe640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440ce730 0x7fb14414a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:35.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14887f640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1441608f0 0x7fb14406dc00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb134007e60 tx=0x7fb13400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:35.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb140ff9640 1 -- 172.21.15.80:0/295286553 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134014420 con 0x7fb1441608f0 2024-04-25T13:40:35.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb140ff9640 1 -- 172.21.15.80:0/295286553 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb134019070 con 0x7fb1441608f0 2024-04-25T13:40:35.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb144148f30 con 0x7fb1441608f0 2024-04-25T13:40:35.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb140ff9640 1 -- 172.21.15.80:0/295286553 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb134014820 con 0x7fb1441608f0 2024-04-25T13:40:35.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.588+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1441494f0 con 0x7fb1441608f0 2024-04-25T13:40:35.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.592+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14406a3a0 con 0x7fb1441608f0 2024-04-25T13:40:35.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.592+0000 7fb140ff9640 1 -- 172.21.15.80:0/295286553 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb1340149c0 con 0x7fb1441608f0 2024-04-25T13:40:35.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.592+0000 7fb140ff9640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb12406f400 0x7fb124071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.592+0000 7fb140ff9640 1 -- 172.21.15.80:0/295286553 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb13408fbd0 con 0x7fb1441608f0 2024-04-25T13:40:35.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.592+0000 7fb143fff640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb12406f400 0x7fb124071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.592+0000 7fb143fff640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb12406f400 0x7fb124071910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b990 tx=0x7fb13c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:35.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.596+0000 7fb140ff9640 1 -- 172.21.15.80:0/295286553 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb13405c470 con 0x7fb1441608f0 2024-04-25T13:40:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.744+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb14406f880 con 0x7fb1441608f0 2024-04-25T13:40:35.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb140ff9640 1 -- 172.21.15.80:0/295286553 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7fb134060770 con 0x7fb1441608f0 2024-04-25T13:40:35.747 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:35.747 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb12406f400 msgr2=0x7fb124071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb12406f400 0x7fb124071910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00b990 tx=0x7fb13c008040 comp rx=0 tx=0).stop 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1441608f0 msgr2=0x7fb14406dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1441608f0 0x7fb14406dc00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb134007e60 tx=0x7fb13400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 shutdown_connections 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb12406f400 0x7fb124071910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1441608f0 0x7fb14406dc00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1440ce730 0x7fb14414a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 --2- 172.21.15.80:0/295286553 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1440b0ed0 0x7fb1441392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 >> 172.21.15.80:0/295286553 conn(0x7fb14406d080 msgr2=0x7fb1440cf080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:35.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 shutdown_connections 2024-04-25T13:40:35.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.748+0000 7fb14a2e2640 1 -- 172.21.15.80:0/295286553 wait complete. 2024-04-25T13:40:35.769 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T13:40:35.769 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T13:40:35.769 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T13:40:35.770 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T13:40:35.770 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T13:40:35.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T13:40:35.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T13:40:35.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T13:40:35.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c7d988640 1 Processor -- start 2024-04-25T13:40:35.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18ffff640 1 Processor -- start 2024-04-25T13:40:35.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18ffff640 1 -- start start 2024-04-25T13:40:35.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f550dd1c640 1 Processor -- start 2024-04-25T13:40:35.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f550dd1c640 1 -- start start 2024-04-25T13:40:35.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4b0da640 1 Processor -- start 2024-04-25T13:40:35.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4b0da640 1 -- start start 2024-04-25T13:40:35.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c7d988640 1 -- start start 2024-04-25T13:40:35.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f90b8800640 1 Processor -- start 2024-04-25T13:40:35.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f90b8800640 1 -- start start 2024-04-25T13:40:35.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18ffff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1900ce270 0x7ff1900d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1900d0f80 0x7ff19006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18ffff640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1900d1390 con 0x7ff1900cde60 2024-04-25T13:40:35.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18ffff640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff1900ce680 con 0x7ff1900d0f80 2024-04-25T13:40:35.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18ffff640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff1900d0280 con 0x7ff1900ce270 2024-04-25T13:40:35.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c7d988640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5c780d0e30 0x7f5c78167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c7d988640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c7806fdc0 0x7f5c7815e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c7d988640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c78168aa0 con 0x7f5c78168520 2024-04-25T13:40:35.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c7d988640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5c78165940 con 0x7f5c780d0e30 2024-04-25T13:40:35.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c7d988640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5c78168d80 con 0x7f5c7806fdc0 2024-04-25T13:40:35.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f550dd1c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f55081619d0 0x7f5508161de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f550dd1c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5508162320 con 0x7f55080b4de0 2024-04-25T13:40:35.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f550dd1c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5508071f00 con 0x7f55080d01c0 2024-04-25T13:40:35.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f550dd1c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f55080b51f0 con 0x7f55081619d0 2024-04-25T13:40:35.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7fb1aa3ea640 1 Processor -- start 2024-04-25T13:40:35.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c76ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5c780d0e30 0x7f5c78167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c76ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5c780d0e30 0x7f5c78167fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36092/0 (socket says 172.21.15.80:36092) 2024-04-25T13:40:35.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c76ffd640 1 -- 172.21.15.80:0/3634982764 learned_addr learned my addr 172.21.15.80:0/3634982764 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:35.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4b0da640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c033b90 0x7f2f3c0e5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4b0da640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2f3c0e5680 0x7f2f3c0e5a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4b0da640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f3c0343a0 con 0x7f2f3c04fed0 2024-04-25T13:40:35.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4b0da640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2f3c033fa0 con 0x7f2f3c0e5680 2024-04-25T13:40:35.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4b0da640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c034150 con 0x7f2f3c033b90 2024-04-25T13:40:35.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18e7fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1900ce270 0x7ff1900d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18e7fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1900ce270 0x7ff1900d0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:55574/0 (socket says 172.21.15.80:55574) 2024-04-25T13:40:35.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7ff18e7fc640 1 -- 172.21.15.80:0/1264445597 learned_addr learned my addr 172.21.15.80:0/1264445597 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:35.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5506ffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f55081619d0 0x7f5508161de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4a0d8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c033b90 0x7f2f3c0e5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5c777fe640 1 --2- 172.21.15.80:0/3634982764 >> v2:172.21.15.87:3300/0 conn(0x7f5c7806fdc0 0x7f5c7815e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4a0d8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c033b90 0x7f2f3c0e5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33310/0 (socket says 172.21.15.80:33310) 2024-04-25T13:40:35.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f4a0d8640 1 -- 172.21.15.80:0/2520616340 learned_addr learned my addr 172.21.15.80:0/2520616340 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:35.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f90b8800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b015e120 0x7f90b015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f90b8800640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b0168aa0 con 0x7f90b006f6b0 2024-04-25T13:40:35.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f90b8800640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f90b0168d80 con 0x7f90b00d0e30 2024-04-25T13:40:35.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f90b8800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f90b0165940 con 0x7f90b015e120 2024-04-25T13:40:35.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f2f498d7640 1 --2- 172.21.15.80:0/2520616340 >> v2:172.21.15.80:3300/0 conn(0x7f2f3c0e5680 0x7f2f3c0e5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5507fff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f55080d01c0 0x7f55080d05d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:47322/0 (socket says 172.21.15.80:47322) 2024-04-25T13:40:35.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7f5507fff640 1 -- 172.21.15.80:0/2557082627 learned_addr learned my addr 172.21.15.80:0/2557082627 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:35.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7fb1aa3ea640 1 -- start start 2024-04-25T13:40:35.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7ff18f7fe640 1 --2- 172.21.15.80:0/1264445597 >> v2:172.21.15.80:3300/0 conn(0x7ff1900d0f80 0x7ff19006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7fed61a3e640 1 Processor -- start 2024-04-25T13:40:35.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.988+0000 7fed61a3e640 1 -- start start 2024-04-25T13:40:35.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f90b6d9d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b015e120 0x7f90b015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:35.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7ff18e7fc640 1 -- 172.21.15.80:0/1264445597 >> v2:172.21.15.80:3300/0 conn(0x7ff1900d0f80 msgr2=0x7ff19006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:35.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7ff18e7fc640 1 --2- 172.21.15.80:0/1264445597 >> v2:172.21.15.80:3300/0 conn(0x7ff1900d0f80 0x7ff19006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7ff18e7fc640 1 -- 172.21.15.80:0/1264445597 >> v1:172.21.15.87:6789/0 conn(0x7ff1900cde60 legacy=0x7ff190072530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7ff18e7fc640 1 -- 172.21.15.80:0/1264445597 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1900ce8e0 con 0x7ff1900ce270 2024-04-25T13:40:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f90b6d9d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b015e120 0x7f90b015e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33322/0 (socket says 172.21.15.80:33322) 2024-04-25T13:40:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f90b6d9d640 1 -- 172.21.15.80:0/1083937452 learned_addr learned my addr 172.21.15.80:0/1083937452 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f2f4a0d8640 1 -- 172.21.15.80:0/2520616340 >> v2:172.21.15.80:3300/0 conn(0x7f2f3c0e5680 msgr2=0x7f2f3c0e5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f2f4a0d8640 1 --2- 172.21.15.80:0/2520616340 >> v2:172.21.15.80:3300/0 conn(0x7f2f3c0e5680 0x7f2f3c0e5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f2f4a0d8640 1 -- 172.21.15.80:0/2520616340 >> v1:172.21.15.80:6789/0 conn(0x7f2f3c04fed0 legacy=0x7f2f3c0502e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f2f4a0d8640 1 -- 172.21.15.80:0/2520616340 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c03a870 con 0x7f2f3c033b90 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f5c76ffd640 1 -- 172.21.15.80:0/3634982764 >> v2:172.21.15.87:3300/0 conn(0x7f5c7806fdc0 msgr2=0x7f5c7815e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f5c76ffd640 1 --2- 172.21.15.80:0/3634982764 >> v2:172.21.15.87:3300/0 conn(0x7f5c7806fdc0 0x7f5c7815e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f5c76ffd640 1 -- 172.21.15.80:0/3634982764 >> v1:172.21.15.87:6789/0 conn(0x7f5c78168520 legacy=0x7f5c7806f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f5c76ffd640 1 -- 172.21.15.80:0/3634982764 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c78164d20 con 0x7f5c780d0e30 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1aa3ea640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb1a40d0e30 0x7fb1a4167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1aa3ea640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1a4168520 0x7fb1a406f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1aa3ea640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1a4070000 con 0x7fb1a406fbf0 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1aa3ea640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb1a4165940 con 0x7fb1a4168520 2024-04-25T13:40:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1aa3ea640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb1a4168aa0 con 0x7fb1a40d0e30 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f55067fc640 1 -- 172.21.15.80:0/2557082627 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3281409448 0 0) 0x7f5508071f00 con 0x7f55080d01c0 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f55067fc640 1 -- 172.21.15.80:0/2557082627 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54f0003530 con 0x7f55080d01c0 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f5506ffd640 1 -- 172.21.15.80:0/2557082627 >> v1:172.21.15.87:6789/0 conn(0x7f55080d01c0 legacy=0x7f55080d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f5506ffd640 1 -- 172.21.15.80:0/2557082627 >> v1:172.21.15.80:6789/0 conn(0x7f55080b4de0 legacy=0x7f5508072cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f5506ffd640 1 -- 172.21.15.80:0/2557082627 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55080721e0 con 0x7f55081619d0 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed61a3e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fed540d9ad0 0x7fed5404f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed61a3e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ee0 0x7fed5404f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed61a3e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fed54050470 0x7fed54050880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed61a3e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fed540541c0 con 0x7fed540d9ad0 2024-04-25T13:40:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed61a3e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fed54050dc0 con 0x7fed54050470 2024-04-25T13:40:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed61a3e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed540e0bf0 con 0x7fed540d9ee0 2024-04-25T13:40:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f90b6d9d640 1 -- 172.21.15.80:0/1083937452 >> v1:172.21.15.87:6789/0 conn(0x7f90b00d0e30 legacy=0x7f90b0167fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f90b6d9d640 1 -- 172.21.15.80:0/1083937452 >> v1:172.21.15.80:6789/0 conn(0x7f90b006f6b0 legacy=0x7f90b006fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f90b6d9d640 1 -- 172.21.15.80:0/1083937452 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b0164d20 con 0x7f90b015e120 2024-04-25T13:40:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1a3fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb1a40d0e30 0x7fb1a4167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1a3fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb1a40d0e30 0x7fb1a4167fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:55614/0 (socket says 172.21.15.80:55614) 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1a3fff640 1 -- 172.21.15.80:0/2574999038 learned_addr learned my addr 172.21.15.80:0/2574999038 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1a37fe640 1 --2- 172.21.15.80:0/2574999038 >> v2:172.21.15.80:3300/0 conn(0x7fb1a4168520 0x7fb1a406f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1a3fff640 1 -- 172.21.15.80:0/2574999038 >> v2:172.21.15.80:3300/0 conn(0x7fb1a4168520 msgr2=0x7fb1a406f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1a3fff640 1 --2- 172.21.15.80:0/2574999038 >> v2:172.21.15.80:3300/0 conn(0x7fb1a4168520 0x7fb1a406f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1a3fff640 1 -- 172.21.15.80:0/2574999038 >> v1:172.21.15.80:6789/0 conn(0x7fb1a406fbf0 legacy=0x7fb1a415e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fb1a3fff640 1 -- 172.21.15.80:0/2574999038 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a4168d80 con 0x7fb1a40d0e30 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fed540d9ad0 0x7fed5404f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fed540d9ad0 0x7fed5404f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36126/0 (socket says 172.21.15.80:36126) 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5affd640 1 -- 172.21.15.80:0/785031162 learned_addr learned my addr 172.21.15.80:0/785031162 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5b7fe640 1 --2- 172.21.15.80:0/785031162 >> v2:172.21.15.87:3300/0 conn(0x7fed54050470 0x7fed54050880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5affd640 1 -- 172.21.15.80:0/785031162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ee0 msgr2=0x7fed5404f890 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T13:40:36.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7f5c76ffd640 1 --2- 172.21.15.80:0/3634982764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c780d0e30 0x7f5c78167fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c780b3ca0 tx=0x7f5c6801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36b2beabc73ddd20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5a7fc640 1 --2- 172.21.15.80:0/785031162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ee0 0x7fed5404f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5affd640 1 --2- 172.21.15.80:0/785031162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ee0 0x7fed5404f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5affd640 1 -- 172.21.15.80:0/785031162 >> v2:172.21.15.87:3300/0 conn(0x7fed54050470 msgr2=0x7fed54050880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5affd640 1 --2- 172.21.15.80:0/785031162 >> v2:172.21.15.87:3300/0 conn(0x7fed54050470 0x7fed54050880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5affd640 1 -- 172.21.15.80:0/785031162 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed54053890 con 0x7fed540d9ad0 2024-04-25T13:40:36.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7fed5b7fe640 1 --2- 172.21.15.80:0/785031162 >> v2:172.21.15.87:3300/0 conn(0x7fed54050470 0x7fed54050880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:40:36.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.992+0000 7ff18e7fc640 1 --2- 172.21.15.80:0/1264445597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900ce270 0x7ff1900d0a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1900c86f0 tx=0x7ff18001c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd59815f89161385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4a0d8640 1 --2- 172.21.15.80:0/2520616340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c033b90 0x7f2f3c0e5140 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009870 tx=0x7f2f3401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dfa0eb33fce8b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18dffb640 1 -- 172.21.15.80:0/1264445597 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800ab600 con 0x7ff1900ce270 2024-04-25T13:40:36.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c75ffb640 1 -- 172.21.15.80:0/3634982764 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c680b1070 con 0x7f5c780d0e30 2024-04-25T13:40:36.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c75ffb640 1 -- 172.21.15.80:0/3634982764 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7815ee70 con 0x7f5c780d0e30 2024-04-25T13:40:36.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c75ffb640 1 -- 172.21.15.80:0/3634982764 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c680a8050 con 0x7f5c780d0e30 2024-04-25T13:40:36.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18dffb640 1 -- 172.21.15.80:0/1264445597 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1800b0070 con 0x7ff1900ce270 2024-04-25T13:40:36.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18dffb640 1 -- 172.21.15.80:0/1264445597 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1800ab9c0 con 0x7ff1900ce270 2024-04-25T13:40:36.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b6d9d640 1 --2- 172.21.15.80:0/1083937452 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b015e120 0x7f90b015e530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f90a40098a0 tx=0x7f90a401c500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f40fe304adeba6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f490d6640 1 -- 172.21.15.80:0/2520616340 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340b0070 con 0x7f2f3c033b90 2024-04-25T13:40:36.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5506ffd640 1 --2- 172.21.15.80:0/2557082627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081619d0 0x7f5508161de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f54f80097c0 tx=0x7f54f801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=92b67e31cb56727d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b559a640 1 -- 172.21.15.80:0/1083937452 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40b0070 con 0x7f90b015e120 2024-04-25T13:40:36.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b559a640 1 -- 172.21.15.80:0/1083937452 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a40ab600 con 0x7f90b015e120 2024-04-25T13:40:36.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b559a640 1 -- 172.21.15.80:0/1083937452 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a40ab9d0 con 0x7f90b015e120 2024-04-25T13:40:36.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f490d6640 1 -- 172.21.15.80:0/2520616340 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f340a4e50 con 0x7f2f3c033b90 2024-04-25T13:40:36.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f55067fc640 1 -- 172.21.15.80:0/2557082627 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80b0070 con 0x7f55081619d0 2024-04-25T13:40:36.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f490d6640 1 -- 172.21.15.80:0/2520616340 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340ab820 con 0x7f2f3c033b90 2024-04-25T13:40:36.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f55067fc640 1 -- 172.21.15.80:0/2557082627 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54f80ab600 con 0x7f55081619d0 2024-04-25T13:40:36.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f55067fc640 1 -- 172.21.15.80:0/2557082627 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54f80ab9d0 con 0x7f55081619d0 2024-04-25T13:40:36.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18ffff640 1 -- 172.21.15.80:0/1264445597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900ce270 msgr2=0x7ff1900d0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1264445597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900ce270 0x7ff1900d0a40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1900c86f0 tx=0x7ff18001c4f0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c7d988640 1 -- 172.21.15.80:0/3634982764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c780d0e30 msgr2=0x7f5c78167fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c7d988640 1 --2- 172.21.15.80:0/3634982764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c780d0e30 0x7f5c78167fe0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c780b3ca0 tx=0x7f5c6801c520 comp rx=0 tx=0).stop 2024-04-25T13:40:36.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fb1a3fff640 1 --2- 172.21.15.80:0/2574999038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 0x7fb1a4167fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4071480 tx=0x7fb19c01ca30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fd3b40de8568e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c7d988640 1 -- 172.21.15.80:0/3634982764 shutdown_connections 2024-04-25T13:40:36.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c7d988640 1 --2- 172.21.15.80:0/3634982764 >> v2:172.21.15.87:3300/0 conn(0x7f5c7806fdc0 0x7f5c7815e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c7d988640 1 --2- 172.21.15.80:0/3634982764 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c780d0e30 0x7f5c78167fe0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c7d988640 1 -- 172.21.15.80:0/3634982764 >> 172.21.15.80:0/3634982764 conn(0x7f5c7806cbf0 msgr2=0x7f5c780b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fb1a2ffd640 1 -- 172.21.15.80:0/2574999038 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0a57a0 con 0x7fb1a40d0e30 2024-04-25T13:40:36.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fb1a2ffd640 1 -- 172.21.15.80:0/2574999038 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb19c0a5940 con 0x7fb1a40d0e30 2024-04-25T13:40:36.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c7d988640 1 -- 172.21.15.80:0/3634982764 shutdown_connections 2024-04-25T13:40:36.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fb1a2ffd640 1 -- 172.21.15.80:0/2574999038 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0a5d10 con 0x7fb1a40d0e30 2024-04-25T13:40:36.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f550dd1c640 1 -- 172.21.15.80:0/2557082627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081619d0 msgr2=0x7f5508161de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f550dd1c640 1 --2- 172.21.15.80:0/2557082627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081619d0 0x7f5508161de0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f54f80097c0 tx=0x7f54f801c450 comp rx=0 tx=0).stop 2024-04-25T13:40:36.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b8800640 1 -- 172.21.15.80:0/1083937452 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b015e120 msgr2=0x7f90b015e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b8800640 1 --2- 172.21.15.80:0/1083937452 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b015e120 0x7f90b015e530 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f90a40098a0 tx=0x7f90a401c500 comp rx=0 tx=0).stop 2024-04-25T13:40:36.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18ffff640 1 -- 172.21.15.80:0/1264445597 shutdown_connections 2024-04-25T13:40:36.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1264445597 >> v2:172.21.15.80:3300/0 conn(0x7ff1900d0f80 0x7ff19006dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1264445597 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900ce270 0x7ff1900d0a40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18ffff640 1 -- 172.21.15.80:0/1264445597 >> 172.21.15.80:0/1264445597 conn(0x7ff1900b3410 msgr2=0x7ff1900b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18ffff640 1 -- 172.21.15.80:0/1264445597 shutdown_connections 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7ff18ffff640 1 -- 172.21.15.80:0/1264445597 wait complete. 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4b0da640 1 -- 172.21.15.80:0/2520616340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c033b90 msgr2=0x7f2f3c0e5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4b0da640 1 --2- 172.21.15.80:0/2520616340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c033b90 0x7f2f3c0e5140 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009870 tx=0x7f2f3401c400 comp rx=0 tx=0).stop 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4b0da640 1 -- 172.21.15.80:0/2520616340 shutdown_connections 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4b0da640 1 --2- 172.21.15.80:0/2520616340 >> v2:172.21.15.80:3300/0 conn(0x7f2f3c0e5680 0x7f2f3c0e5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4b0da640 1 --2- 172.21.15.80:0/2520616340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c033b90 0x7f2f3c0e5140 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4b0da640 1 -- 172.21.15.80:0/2520616340 >> 172.21.15.80:0/2520616340 conn(0x7f2f3c054d00 msgr2=0x7f2f3c055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b8800640 1 -- 172.21.15.80:0/1083937452 shutdown_connections 2024-04-25T13:40:36.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b8800640 1 --2- 172.21.15.80:0/1083937452 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b015e120 0x7f90b015e530 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b8800640 1 -- 172.21.15.80:0/1083937452 >> 172.21.15.80:0/1083937452 conn(0x7f90b006cbf0 msgr2=0x7f90b00b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b8800640 1 -- 172.21.15.80:0/1083937452 shutdown_connections 2024-04-25T13:40:36.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f90b8800640 1 -- 172.21.15.80:0/1083937452 wait complete. 2024-04-25T13:40:36.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f550dd1c640 1 -- 172.21.15.80:0/2557082627 shutdown_connections 2024-04-25T13:40:36.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f550dd1c640 1 --2- 172.21.15.80:0/2557082627 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081619d0 0x7f5508161de0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f550dd1c640 1 -- 172.21.15.80:0/2557082627 >> 172.21.15.80:0/2557082627 conn(0x7f550806cbf0 msgr2=0x7f55080b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f5c7d988640 1 -- 172.21.15.80:0/3634982764 wait complete. 2024-04-25T13:40:36.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4b0da640 1 -- 172.21.15.80:0/2520616340 shutdown_connections 2024-04-25T13:40:36.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fed5affd640 1 --2- 172.21.15.80:0/785031162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed540d9ad0 0x7fed5404f350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed540378c0 tx=0x7fed4c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c608e3f84b3d294f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f550dd1c640 1 -- 172.21.15.80:0/2557082627 shutdown_connections 2024-04-25T13:40:36.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f2f4b0da640 1 -- 172.21.15.80:0/2520616340 wait complete. 2024-04-25T13:40:36.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/2574999038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 msgr2=0x7fb1a4167fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fb1aa3ea640 1 --2- 172.21.15.80:0/2574999038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 0x7fb1a4167fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4071480 tx=0x7fb19c01ca30 comp rx=0 tx=0).stop 2024-04-25T13:40:36.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7ff18ffff640 1 Processor -- start 2024-04-25T13:40:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7ff18ffff640 1 -- start start 2024-04-25T13:40:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7f550dd1c640 1 -- 172.21.15.80:0/2557082627 wait complete. 2024-04-25T13:40:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f550dd1c640 1 Processor -- start 2024-04-25T13:40:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fed59ffb640 1 -- 172.21.15.80:0/785031162 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0b0070 con 0x7fed540d9ad0 2024-04-25T13:40:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fed59ffb640 1 -- 172.21.15.80:0/785031162 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed4c0ab600 con 0x7fed540d9ad0 2024-04-25T13:40:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:35.996+0000 7fed59ffb640 1 -- 172.21.15.80:0/785031162 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4c0ab9d0 con 0x7fed540d9ad0 2024-04-25T13:40:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fed61a3e640 1 -- 172.21.15.80:0/785031162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed540d9ad0 msgr2=0x7fed5404f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fed61a3e640 1 --2- 172.21.15.80:0/785031162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed540d9ad0 0x7fed5404f350 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fed540378c0 tx=0x7fed4c01c4f0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fed61a3e640 1 -- 172.21.15.80:0/785031162 shutdown_connections 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fed61a3e640 1 --2- 172.21.15.80:0/785031162 >> v2:172.21.15.87:3300/0 conn(0x7fed54050470 0x7fed54050880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fed61a3e640 1 --2- 172.21.15.80:0/785031162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ee0 0x7fed5404f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fed61a3e640 1 --2- 172.21.15.80:0/785031162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed540d9ad0 0x7fed5404f350 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fed61a3e640 1 -- 172.21.15.80:0/785031162 >> 172.21.15.80:0/785031162 conn(0x7fed54054d00 msgr2=0x7fed54055110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c7d988640 1 Processor -- start 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c7d988640 1 -- start start 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f90b8800640 1 Processor -- start 2024-04-25T13:40:36.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f90b8800640 1 -- start start 2024-04-25T13:40:36.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f2f4b0da640 1 Processor -- start 2024-04-25T13:40:36.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c7d988640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7806fdc0 0x7f5c7815bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c7d988640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c780d0e30 0x7f5c78163470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c7d988640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c78168520 0x7f5c780b1d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c7d988640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5c780d13c0 con 0x7f5c78168520 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c767fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c780d0e30 0x7f5c78163470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c767fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c780d0e30 0x7f5c78163470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:55620/0 (socket says 172.21.15.80:55620) 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c767fc640 1 -- 172.21.15.80:0/4160788547 learned_addr learned my addr 172.21.15.80:0/4160788547 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/2574999038 shutdown_connections 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fb1aa3ea640 1 --2- 172.21.15.80:0/2574999038 >> v2:172.21.15.80:3300/0 conn(0x7fb1a4168520 0x7fb1a406f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fb1aa3ea640 1 --2- 172.21.15.80:0/2574999038 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 0x7fb1a4167fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/2574999038 >> 172.21.15.80:0/2574999038 conn(0x7fb1a406cbf0 msgr2=0x7fb1a40b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c76ffd640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7806fdc0 0x7f5c7815bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/2574999038 shutdown_connections 2024-04-25T13:40:36.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c7d988640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c780d1240 con 0x7f5c780d0e30 2024-04-25T13:40:36.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c78071cb0 con 0x7f5c7806fdc0 2024-04-25T13:40:36.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f550dd1c640 1 -- start start 2024-04-25T13:40:36.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c777fe640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c78168520 0x7f5c780b1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c767fc640 1 -- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7806fdc0 msgr2=0x7f5c7815bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c767fc640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c7806fdc0 0x7f5c7815bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c767fc640 1 -- 172.21.15.80:0/4160788547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c78168520 msgr2=0x7f5c780b1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c767fc640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c78168520 0x7f5c780b1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/2574999038 wait complete. 2024-04-25T13:40:36.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f5c767fc640 1 -- 172.21.15.80:0/4160788547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c7815c5e0 con 0x7f5c780d0e30 2024-04-25T13:40:36.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b006f6b0 0x7f90b0209240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b00d0e30 0x7f90b0151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b015e120 0x7f90b01611f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f550dd1c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55080b4de0 0x7f55081431a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fed61a3e640 1 -- 172.21.15.80:0/785031162 shutdown_connections 2024-04-25T13:40:36.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.000+0000 7f2f4b0da640 1 -- start start 2024-04-25T13:40:36.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f8ff4a2c640 1 Processor -- start 2024-04-25T13:40:36.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fed61a3e640 1 -- 172.21.15.80:0/785031162 wait complete. 2024-04-25T13:40:36.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b006f6b0 0x7f90b0209240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b006f6b0 0x7f90b0209240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33342/0 (socket says 172.21.15.80:33342) 2024-04-25T13:40:36.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b015e120 0x7f90b01611f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b00d0e30 0x7f90b0151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b015e120 0x7f90b01611f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36148/0 (socket says 172.21.15.80:36148) 2024-04-25T13:40:36.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f8ff4a2c640 1 -- start start 2024-04-25T13:40:36.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5c767fc640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c780d0e30 0x7f5c78163470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c3c0 tx=0x7f5c6c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b5d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b00d0e30 0x7f90b0151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:55622/0 (socket says 172.21.15.80:55622) 2024-04-25T13:40:36.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f550dd1c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080d01c0 0x7f55081436e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4b0da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f3c033b90 0x7f2f3c0ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4b0da640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f3c04fed0 0x7f2f3c0da330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1aa3ea640 1 Processor -- start 2024-04-25T13:40:36.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55080b4de0 0x7f55081431a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55080b4de0 0x7f55081431a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36160/0 (socket says 172.21.15.80:36160) 2024-04-25T13:40:36.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4b0da640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c0e5680 0x7f2f3c0e3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c017070 con 0x7f5c780d0e30 2024-04-25T13:40:36.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f90b01686f0 con 0x7f90b015e120 2024-04-25T13:40:36.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f90b00d1240 con 0x7f90b00d0e30 2024-04-25T13:40:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f90b0206a90 con 0x7f90b006f6b0 2024-04-25T13:40:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 -- 172.21.15.80:0/3788357235 learned_addr learned my addr 172.21.15.80:0/3788357235 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4b0da640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c0ef020 con 0x7f2f3c04fed0 2024-04-25T13:40:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4b0da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c0eeea0 con 0x7f2f3c033b90 2024-04-25T13:40:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 -- 172.21.15.80:0/3645576301 learned_addr learned my addr 172.21.15.80:0/3645576301 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4b0da640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c0ef1a0 con 0x7f2f3c0e5680 2024-04-25T13:40:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f550dd1c640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081619d0 0x7f5508208630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a8d9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c0e5680 0x7f2f3c0e3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a8d9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c0e5680 0x7f2f3c0e3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33352/0 (socket says 172.21.15.80:33352) 2024-04-25T13:40:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a8d9640 1 -- 172.21.15.80:0/3111595259 learned_addr learned my addr 172.21.15.80:0/3111595259 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f550806e640 con 0x7f55080b4de0 2024-04-25T13:40:36.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f550806e4c0 con 0x7f55081619d0 2024-04-25T13:40:36.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5506ffd640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080d01c0 0x7f55081436e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a0d8640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f3c033b90 0x7f2f3c0ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f550806e7c0 con 0x7f55080d01c0 2024-04-25T13:40:36.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1aa3ea640 1 -- start start 2024-04-25T13:40:36.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5507fff640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081619d0 0x7f5508208630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 -- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b00d0e30 msgr2=0x7f90b0151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f90b00d0e30 0x7f90b0151280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 -- 172.21.15.80:0/3788357235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b015e120 msgr2=0x7f90b01611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f90b015e120 0x7f90b01611f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 -- 172.21.15.80:0/3788357235 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b0151f00 con 0x7f90b006f6b0 2024-04-25T13:40:36.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c7815c8c0 con 0x7f5c780d0e30 2024-04-25T13:40:36.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c7815ce80 con 0x7f5c780d0e30 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900cde60 0x7ff1900b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900ce270 0x7ff1900acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900d0f80 0x7ff1900c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18ffff640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff190072dc0 con 0x7ff1900cde60 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18ffff640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff190072c40 con 0x7ff1900d0f80 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18ffff640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff19015ab90 con 0x7ff1900ce270 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 -- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080d01c0 msgr2=0x7f55081436e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f55080d01c0 0x7f55081436e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 -- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081619d0 msgr2=0x7f5508208630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f55081619d0 0x7f5508208630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f550816fea0 con 0x7f55080b4de0 2024-04-25T13:40:36.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900d0f80 0x7ff1900c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900d0f80 0x7ff1900c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:55644/0 (socket says 172.21.15.80:55644) 2024-04-25T13:40:36.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18f7fe640 1 -- 172.21.15.80:0/1925019278 learned_addr learned my addr 172.21.15.80:0/1925019278 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f55077fe640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55080b4de0 0x7f55081431a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00c390 tx=0x7f54fc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6c0052d0 con 0x7f5c780d0e30 2024-04-25T13:40:36.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c003840 con 0x7f5c780d0e30 2024-04-25T13:40:36.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c58001040 con 0x7f5c780d0e30 2024-04-25T13:40:36.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a0d8640 1 -- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c0e5680 msgr2=0x7f2f3c0e3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a0d8640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f3c0e5680 0x7f2f3c0e3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a0d8640 1 -- 172.21.15.80:0/3111595259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f3c04fed0 msgr2=0x7f2f3c0da330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a0d8640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f3c04fed0 0x7f2f3c0da330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a0d8640 1 -- 172.21.15.80:0/3111595259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c0ca600 con 0x7f2f3c033b90 2024-04-25T13:40:36.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b659c640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b006f6b0 0x7f90b0209240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90a800b410 tx=0x7f90a800b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18effd640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900cde60 0x7ff1900b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc011790 con 0x7f55080b4de0 2024-04-25T13:40:36.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54fc017070 con 0x7f55080b4de0 2024-04-25T13:40:36.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc011b30 con 0x7f55080b4de0 2024-04-25T13:40:36.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1aa3ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1a406fbf0 0x7fb1a4153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4a0d8640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f3c033b90 0x7f2f3c0ca0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c03ab90 tx=0x7f2f340191a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1aa3ea640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 0x7fb1a40b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1aa3ea640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1a4168520 0x7fb1a40b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1aa3ea640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a40d1240 con 0x7fb1a406fbf0 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1aa3ea640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a41649f0 con 0x7fb1a40d0e30 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1aa3ea640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a40d13c0 con 0x7fb1a4168520 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fb1a8987640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1a4168520 0x7fb1a40b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18e7fc640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900ce270 0x7ff1900acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18f7fe640 1 -- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900ce270 msgr2=0x7ff1900acaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18f7fe640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900ce270 0x7ff1900acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18f7fe640 1 -- 172.21.15.80:0/1925019278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900cde60 msgr2=0x7ff1900b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18f7fe640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900cde60 0x7ff1900b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7ff18f7fe640 1 -- 172.21.15.80:0/1925019278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1900ad790 con 0x7ff1900d0f80 2024-04-25T13:40:36.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7ff18f7fe640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900d0f80 0x7ff1900c5ef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff18400e820 tx=0x7ff18400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5508207b50 con 0x7f55080b4de0 2024-04-25T13:40:36.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5508208110 con 0x7f55080b4de0 2024-04-25T13:40:36.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fed61a3e640 1 Processor -- start 2024-04-25T13:40:36.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f8ff4a2c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff0164520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f8ff4a2c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8ff00d0970 0x7f8ff00d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f8ff4a2c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ff00d12c0 con 0x7f8ff00cf750 2024-04-25T13:40:36.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8ff4a2c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8ff01680a0 con 0x7f8ff00d0970 2024-04-25T13:40:36.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8ff4a2c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff0168340 con 0x7f8ff006e740 2024-04-25T13:40:36.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8015020 con 0x7f90b006f6b0 2024-04-25T13:40:36.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b01521e0 con 0x7f90b006f6b0 2024-04-25T13:40:36.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b01527a0 con 0x7f90b006f6b0 2024-04-25T13:40:36.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90b006a2c0 con 0x7f90b006f6b0 2024-04-25T13:40:36.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90a8011830 con 0x7f90b006f6b0 2024-04-25T13:40:36.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340b0050 con 0x7f2f3c033b90 2024-04-25T13:40:36.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7f2f4b0da640 1 -- 172.21.15.80:0/3111595259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c036920 con 0x7f2f3c033b90 2024-04-25T13:40:36.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f2f4b0da640 1 -- 172.21.15.80:0/3111595259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c036ee0 con 0x7f2f3c033b90 2024-04-25T13:40:36.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f34000e50 con 0x7f2f3c033b90 2024-04-25T13:40:36.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f340b5440 con 0x7f2f3c033b90 2024-04-25T13:40:36.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f550806a2c0 con 0x7f55080b4de0 2024-04-25T13:40:36.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a8987640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1a4168520 0x7fb1a40b1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33370/0 (socket says 172.21.15.80:33370) 2024-04-25T13:40:36.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a8987640 1 -- 172.21.15.80:0/1607861556 learned_addr learned my addr 172.21.15.80:0/1607861556 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a3fff640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1a406fbf0 0x7fb1a4153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fef7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff0164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fef7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff0164520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33380/0 (socket says 172.21.15.80:33380) 2024-04-25T13:40:36.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a8011c20 con 0x7f90b006f6b0 2024-04-25T13:40:36.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fef7fe640 1 -- 172.21.15.80:0/4260311396 learned_addr learned my addr 172.21.15.80:0/4260311396 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8feeffd640 1 --2- 172.21.15.80:0/4260311396 >> v2:172.21.15.87:3300/0 conn(0x7f8ff00d0970 0x7f8ff00d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.004+0000 7fed61a3e640 1 -- start start 2024-04-25T13:40:36.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a37fe640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 0x7fb1a40b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a3fff640 1 -- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1a4168520 msgr2=0x7fb1a40b1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a3fff640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1a4168520 0x7fb1a40b1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a3fff640 1 -- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 msgr2=0x7fb1a40b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a3fff640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 0x7fb1a40b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a3fff640 1 -- 172.21.15.80:0/1607861556 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a415e3f0 con 0x7fb1a406fbf0 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a3fff640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1a406fbf0 0x7fb1a4153460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4168c20 tx=0x7fb19c0197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a37fe640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1a40d0e30 0x7fb1a40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0b6440 con 0x7fb1a406fbf0 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fee7fc640 1 -- 172.21.15.80:0/4260311396 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549472888 0 0) 0x7f8fe80014a0 con 0x7f8ff00cf750 2024-04-25T13:40:36.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fee7fc640 1 -- 172.21.15.80:0/4260311396 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8fd8003530 con 0x7f8ff00cf750 2024-04-25T13:40:36.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fef7fe640 1 -- 172.21.15.80:0/4260311396 >> v2:172.21.15.87:3300/0 conn(0x7f8ff00d0970 msgr2=0x7f8ff00d0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb19c0b65e0 con 0x7fb1a406fbf0 2024-04-25T13:40:36.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb19c0b69d0 con 0x7fb1a406fbf0 2024-04-25T13:40:36.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fef7fe640 1 --2- 172.21.15.80:0/4260311396 >> v2:172.21.15.87:3300/0 conn(0x7f8ff00d0970 0x7f8ff00d0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5c6c01a070 con 0x7f5c780d0e30 2024-04-25T13:40:36.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f5c5ffff640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c6406f400 0x7f5c64071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5c6c08ed30 con 0x7f5c780d0e30 2024-04-25T13:40:36.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f5c5ffff640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f5c640776d0 0x7f5c64079b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 --> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c6c01dec0 con 0x7f5c640776d0 2024-04-25T13:40:36.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5c6c08f120 con 0x7f5c780d0e30 2024-04-25T13:40:36.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f54fc011cd0 con 0x7f55080b4de0 2024-04-25T13:40:36.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f5c76ffd640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c6406f400 0x7f5c64071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f5c777fe640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f5c640776d0 0x7f5c64079b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a41539a0 con 0x7fb1a406fbf0 2024-04-25T13:40:36.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a415cf50 con 0x7fb1a406fbf0 2024-04-25T13:40:36.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb1a406a2c0 con 0x7fb1a406fbf0 2024-04-25T13:40:36.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f54effff640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f54f406f4d0 0x7f54f40719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f54fc08fa60 con 0x7f55080b4de0 2024-04-25T13:40:36.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f2f4b0da640 1 -- 172.21.15.80:0/3111595259 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2f3c0054b0 con 0x7f2f3c033b90 2024-04-25T13:40:36.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fef7fe640 1 -- 172.21.15.80:0/4260311396 >> v1:172.21.15.87:6789/0 conn(0x7f8ff00cf750 legacy=0x7f8ff00cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f8fef7fe640 1 -- 172.21.15.80:0/4260311396 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff0168620 con 0x7f8ff006e740 2024-04-25T13:40:36.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f54effff640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f54f40777a0 0x7f54f4079bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f54fc010e50 con 0x7f54f40777a0 2024-04-25T13:40:36.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f54fc08fe50 con 0x7f55080b4de0 2024-04-25T13:40:36.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184017070 con 0x7ff1900d0f80 2024-04-25T13:40:36.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1900ada70 con 0x7ff1900d0f80 2024-04-25T13:40:36.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1900ae030 con 0x7ff1900d0f80 2024-04-25T13:40:36.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff18400c8e0 con 0x7ff1900d0f80 2024-04-25T13:40:36.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f5507fff640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f54f40777a0 0x7f54f4079bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f5506ffd640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f54f406f4d0 0x7f54f40719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f5c777fe640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f5c640776d0 0x7f5c64079b20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f340bc030 con 0x7f2f3c033b90 2024-04-25T13:40:36.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f3affd640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f2806f400 0x7f2f28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2f34129a10 con 0x7f2f3c033b90 2024-04-25T13:40:36.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f3affd640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7f2f280775f0 0x7f2f28079a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 --> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2f340b1e90 con 0x7f2f280775f0 2024-04-25T13:40:36.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2f34129ea0 con 0x7f2f3c033b90 2024-04-25T13:40:36.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f498d7640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f2806f400 0x7f2f28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f4a8d9640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7f2f280775f0 0x7f2f28079a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f5c76ffd640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c6406f400 0x7f5c64071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c680ae000 tx=0x7f5c68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f5506ffd640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f54f406f4d0 0x7f54f40719e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54f80ad000 tx=0x7f54f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed61a3e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed54050470 0x7fed540da2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed61a3e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ad0 0x7fed540dc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed61a3e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed540d9ee0 0x7fed540ce030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed61a3e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fed540e07c0 con 0x7fed54050470 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed61a3e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fed540e0640 con 0x7fed540d9ee0 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed61a3e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fed540e0940 con 0x7fed540d9ad0 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed540d9ee0 0x7fed540ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed540d9ee0 0x7fed540ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:55670/0 (socket says 172.21.15.80:55670) 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5b7fe640 1 -- 172.21.15.80:0/3880318423 learned_addr learned my addr 172.21.15.80:0/3880318423 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5a7fc640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ad0 0x7fed540dc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5b7fe640 1 -- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ad0 msgr2=0x7fed540dc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5b7fe640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fed540d9ad0 0x7fed540dc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5b7fe640 1 -- 172.21.15.80:0/3880318423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed54050470 msgr2=0x7fed540da2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5b7fe640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fed54050470 0x7fed540da2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7fed5b7fe640 1 -- 172.21.15.80:0/3880318423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed540dab40 con 0x7fed540d9ee0 2024-04-25T13:40:36.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fed5b7fe640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed540d9ee0 0x7fed540ce030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0754a0 tx=0x7fed5c06af70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.008+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f90a8018020 con 0x7f90b006f6b0 2024-04-25T13:40:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90a2ffd640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f909406f400 0x7f9094071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f90a808f950 con 0x7f90b006f6b0 2024-04-25T13:40:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90a2ffd640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f9094077650 0x7f9094079aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 --> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f909407a170 con 0x7f9094077650 2024-04-25T13:40:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f90a808fde0 con 0x7f90b006f6b0 2024-04-25T13:40:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90b5d9b640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f909406f400 0x7f9094071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18400ccb0 con 0x7ff1900d0f80 2024-04-25T13:40:36.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff170001040 con 0x7ff1900d0f80 2024-04-25T13:40:36.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90b6d9d640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f9094077650 0x7f9094079aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb19c0bf020 con 0x7fb1a406fbf0 2024-04-25T13:40:36.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a0ff9640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb19006f500 0x7fb190071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fb19c12a570 con 0x7fb1a406fbf0 2024-04-25T13:40:36.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a0ff9640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7fb190077770 0x7fb190079bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 --> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb19c0a5940 con 0x7fb190077770 2024-04-25T13:40:36.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb19c12a960 con 0x7fb1a406fbf0 2024-04-25T13:40:36.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a8987640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7fb190077770 0x7fb190079bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f5507fff640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f54f40777a0 0x7f54f4079bf0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c06c7f0 con 0x7fed540d9ee0 2024-04-25T13:40:36.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed5c06c990 con 0x7fed540d9ee0 2024-04-25T13:40:36.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5c06cd40 con 0x7fed540d9ee0 2024-04-25T13:40:36.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f498d7640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f2806f400 0x7f2f28071910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2f40006aa0 tx=0x7f2f40006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f2f4a8d9640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7f2f280775f0 0x7f2f28079a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a37fe640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb19006f500 0x7fb190071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed540dae20 con 0x7fed540d9ee0 2024-04-25T13:40:36.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed540cb2e0 con 0x7fed540d9ee0 2024-04-25T13:40:36.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 <== osd.2 v2:172.21.15.80:6809/968203588 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f54fc010e50 con 0x7f54f40777a0 2024-04-25T13:40:36.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90b5d9b640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f909406f400 0x7f9094071910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f90b01709d0 tx=0x7f90ac0094d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f90b6d9d640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f9094077650 0x7f9094079aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fb1a8987640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7fb190077770 0x7fb190079bc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fed540db340 con 0x7fed540d9ee0 2024-04-25T13:40:36.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f8fef7fe640 1 --2- 172.21.15.80:0/4260311396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff0164520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009870 tx=0x7f8fe001c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3459005aaeb3e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f8fee7fc640 1 -- 172.21.15.80:0/4260311396 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00b2070 con 0x7f8ff006e740 2024-04-25T13:40:36.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.012+0000 7f8fee7fc640 1 -- 172.21.15.80:0/4260311396 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe00a9070 con 0x7f8ff006e740 2024-04-25T13:40:36.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 <== osd.4 v2:172.21.15.87:6816/2235735313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f5c6c01dec0 con 0x7f5c640776d0 2024-04-25T13:40:36.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff18401a070 con 0x7ff1900d0f80 2024-04-25T13:40:36.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7f8fee7fc640 1 -- 172.21.15.80:0/4260311396 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00b2070 con 0x7f8ff006e740 2024-04-25T13:40:36.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff1777fe640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17c06f4d0 0x7ff17c0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7f8ff4a2c640 1 -- 172.21.15.80:0/4260311396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 msgr2=0x7f8ff0164520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7f8ff4a2c640 1 --2- 172.21.15.80:0/4260311396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff0164520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009870 tx=0x7f8fe001c560 comp rx=0 tx=0).stop 2024-04-25T13:40:36.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 <== osd.7 v2:172.21.15.87:6808/710369794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f2f340b1e90 con 0x7f2f280775f0 2024-04-25T13:40:36.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff18408f7c0 con 0x7ff1900d0f80 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff1777fe640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7ff17c077740 0x7ff17c079b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff18effd640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17c06f4d0 0x7ff17c0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff18e7fc640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7ff17c077740 0x7ff17c079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 --> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff184016ec0 con 0x7ff17c077740 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff18408fbb0 con 0x7ff1900d0f80 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff18effd640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17c06f4d0 0x7ff17c0719e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1880012f0 tx=0x7ff188009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7ff18e7fc640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7ff17c077740 0x7ff17c079b90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 <== osd.0 v2:172.21.15.80:6808/2104747008 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fb19c0a5940 con 0x7fb190077770 2024-04-25T13:40:36.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.016+0000 7fb1a37fe640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb19006f500 0x7fb190071a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4167fe0 tx=0x7fb198008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 <== osd.1 v2:172.21.15.80:6800/1560454941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7ff184016ec0 con 0x7ff17c077740 2024-04-25T13:40:36.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 <== mon.1 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fed5c0a03a0 con 0x7fed540d9ee0 2024-04-25T13:40:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed43fff640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fed4806f4d0 0x7fed480719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 <== mon.1 v2:172.21.15.87:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fed5c10bea0 con 0x7fed540d9ee0 2024-04-25T13:40:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed43fff640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7fed48077740 0x7fed48079b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 --> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fed4807a260 con 0x7fed48077740 2024-04-25T13:40:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 <== mon.1 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fed5c10c290 con 0x7fed540d9ee0 2024-04-25T13:40:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed5affd640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fed4806f4d0 0x7fed480719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed5a7fc640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7fed48077740 0x7fed48079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 <== osd.6 v2:172.21.15.87:6824/1131074035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f909407a170 con 0x7f9094077650 2024-04-25T13:40:36.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed5affd640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fed4806f4d0 0x7fed480719e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0ad000 tx=0x7fed4c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.020+0000 7fed5a7fc640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7fed48077740 0x7fed48079b90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.024+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 <== osd.3 v2:172.21.15.80:6823/3781596558 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fed4807a260 con 0x7fed48077740 2024-04-25T13:40:36.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.028+0000 7f8ff4a2c640 1 -- 172.21.15.80:0/4260311396 shutdown_connections 2024-04-25T13:40:36.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.028+0000 7f8ff4a2c640 1 --2- 172.21.15.80:0/4260311396 >> v2:172.21.15.87:3300/0 conn(0x7f8ff00d0970 0x7f8ff00d0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.028+0000 7f8ff4a2c640 1 --2- 172.21.15.80:0/4260311396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff0164520 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.028+0000 7f8ff4a2c640 1 -- 172.21.15.80:0/4260311396 >> 172.21.15.80:0/4260311396 conn(0x7f8ff00b3410 msgr2=0x7f8ff00b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.028+0000 7f8ff4a2c640 1 -- 172.21.15.80:0/4260311396 shutdown_connections 2024-04-25T13:40:36.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.028+0000 7f8ff4a2c640 1 -- 172.21.15.80:0/4260311396 wait complete. 2024-04-25T13:40:36.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8ff4a2c640 1 Processor -- start 2024-04-25T13:40:36.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8ff4a2c640 1 -- start start 2024-04-25T13:40:36.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 --> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f550816f9f0 con 0x7f54f40777a0 2024-04-25T13:40:36.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f54effff640 1 -- 172.21.15.80:0/3645576301 <== osd.2 v2:172.21.15.80:6809/968203588 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f550816f9f0 con 0x7f54f40777a0 2024-04-25T13:40:36.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f54f40777a0 msgr2=0x7f54f4079bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f54f40777a0 0x7f54f4079bf0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f54f406f4d0 msgr2=0x7f54f40719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f54f406f4d0 0x7f54f40719e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f54f80ad000 tx=0x7f54f8008040 comp rx=0 tx=0).stop 2024-04-25T13:40:36.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55080b4de0 msgr2=0x7f55081431a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 --2- 172.21.15.80:0/3645576301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f55080b4de0 0x7f55081431a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00c390 tx=0x7f54fc00c850 comp rx=0 tx=0).stop 2024-04-25T13:40:36.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f5507fff640 1 -- 172.21.15.80:0/3645576301 reap_dead start 2024-04-25T13:40:36.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8ff4a2c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff014e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8ff4a2c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff00cf750 0x7f8ff0151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8ff4a2c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ff00d0970 0x7f8ff0166d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8ff4a2c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0171d70 con 0x7f8ff00cf750 2024-04-25T13:40:36.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8ff4a2c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff0171bf0 con 0x7f8ff00d0970 2024-04-25T13:40:36.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8ff4a2c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8ff0171ef0 con 0x7f8ff006e740 2024-04-25T13:40:36.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 shutdown_connections 2024-04-25T13:40:36.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 >> 172.21.15.80:0/3645576301 conn(0x7f550806cbf0 msgr2=0x7f550815f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff00cf750 0x7f8ff0151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff00cf750 0x7f8ff0151450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36224/0 (socket says 172.21.15.80:36224) 2024-04-25T13:40:36.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 -- 172.21.15.80:0/2154635856 learned_addr learned my addr 172.21.15.80:0/2154635856 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8fef7fe640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff014e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feffff640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ff00d0970 0x7f8ff0166d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 shutdown_connections 2024-04-25T13:40:36.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f550dd1c640 1 -- 172.21.15.80:0/3645576301 wait complete. 2024-04-25T13:40:36.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 -- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 msgr2=0x7f8ff014e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ff006e740 0x7f8ff014e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 -- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ff00d0970 msgr2=0x7f8ff0166d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ff00d0970 0x7f8ff0166d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 -- 172.21.15.80:0/2154635856 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff0162b10 con 0x7f8ff00cf750 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.032+0000 7f8feeffd640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff00cf750 0x7f8ff0151450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d5f0 tx=0x7f8fe400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.036+0000 7f8fd7fff640 1 -- 172.21.15.80:0/2154635856 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40125f0 con 0x7f8ff00cf750 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.036+0000 7f8fd7fff640 1 -- 172.21.15.80:0/2154635856 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fe4019070 con 0x7f8ff00cf750 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.036+0000 7f8ff4a2c640 1 -- 172.21.15.80:0/2154635856 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff0151990 con 0x7f8ff00cf750 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.036+0000 7f8fd7fff640 1 -- 172.21.15.80:0/2154635856 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40129b0 con 0x7f8ff00cf750 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.036+0000 7f8ff4a2c640 1 -- 172.21.15.80:0/2154635856 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff01632f0 con 0x7f8ff00cf750 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.036+0000 7f8fd7fff640 1 -- 172.21.15.80:0/2154635856 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8fe4012b50 con 0x7f8ff00cf750 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.036+0000 7f8fd7fff640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fdc06f4d0 0x7f8fdc0719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f4b0da640 1 -- 172.21.15.80:0/3111595259 --> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2f3c0daaf0 con 0x7f2f280775f0 2024-04-25T13:40:36.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 --> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5c58002d60 con 0x7f5c640776d0 2024-04-25T13:40:36.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f8fd7fff640 1 -- 172.21.15.80:0/2154635856 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f8fe4093170 con 0x7f8ff00cf750 2024-04-25T13:40:36.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f8fef7fe640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fdc06f4d0 0x7f8fdc0719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f8fd5ffb640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f8ff0163530 0x7f8ff00c0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 --> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ff006a2c0 con 0x7f8ff0163530 2024-04-25T13:40:36.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f8feffff640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f8ff0163530 0x7f8ff00c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c5ffff640 1 -- 172.21.15.80:0/4160788547 <== osd.4 v2:172.21.15.87:6816/2235735313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5c58002d60 con 0x7f5c640776d0 2024-04-25T13:40:36.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f3affd640 1 -- 172.21.15.80:0/3111595259 <== osd.7 v2:172.21.15.87:6808/710369794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2f4406a2c0 con 0x7f2f280775f0 2024-04-25T13:40:36.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f8fef7fe640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fdc06f4d0 0x7f8fdc0719e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00af000 tx=0x7f8fe00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f5c640776d0 msgr2=0x7f5c64079b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f5c640776d0 0x7f5c64079b20 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 -- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7f2f280775f0 msgr2=0x7f2f28079a40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c6406f400 msgr2=0x7f5c64071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c6406f400 0x7f5c64071910 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c680ae000 tx=0x7f5c68008040 comp rx=0 tx=0).stop 2024-04-25T13:40:36.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c780d0e30 msgr2=0x7f5c78163470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 --2- 172.21.15.80:0/4160788547 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c780d0e30 0x7f5c78163470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c00c3c0 tx=0x7f5c6c00c880 comp rx=0 tx=0).stop 2024-04-25T13:40:36.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c777fe640 1 -- 172.21.15.80:0/4160788547 reap_dead start 2024-04-25T13:40:36.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 shutdown_connections 2024-04-25T13:40:36.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f8feffff640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f8ff0163530 0x7f8ff00c0250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7f2f280775f0 0x7f2f28079a40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 -- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f2806f400 msgr2=0x7f2f28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f2806f400 0x7f2f28071910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f2f40006aa0 tx=0x7f2f40006a30 comp rx=0 tx=0).stop 2024-04-25T13:40:36.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 >> 172.21.15.80:0/4160788547 conn(0x7f5c7806cbf0 msgr2=0x7f5c78159810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 -- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f3c033b90 msgr2=0x7f2f3c0ca0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 --2- 172.21.15.80:0/3111595259 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f3c033b90 0x7f2f3c0ca0c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c03ab90 tx=0x7f2f340191a0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f4a8d9640 1 -- 172.21.15.80:0/3111595259 reap_dead start 2024-04-25T13:40:36.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 -- 172.21.15.80:0/3111595259 shutdown_connections 2024-04-25T13:40:36.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 -- 172.21.15.80:0/3111595259 >> 172.21.15.80:0/3111595259 conn(0x7f2f3c054d00 msgr2=0x7f2f3c04e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 shutdown_connections 2024-04-25T13:40:36.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 -- 172.21.15.80:0/3111595259 shutdown_connections 2024-04-25T13:40:36.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f2f38ff9640 1 -- 172.21.15.80:0/3111595259 wait complete. 2024-04-25T13:40:36.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f5c7d988640 1 -- 172.21.15.80:0/4160788547 wait complete. 2024-04-25T13:40:36.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.040+0000 7f8fd7fff640 1 -- 172.21.15.80:0/2154635856 <== osd.5 v2:172.21.15.87:6800/149615972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f8ff006a2c0 con 0x7f8ff0163530 2024-04-25T13:40:36.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 --> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb1a4153bd0 con 0x7fb190077770 2024-04-25T13:40:36.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 --> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90b006a2c0 con 0x7f9094077650 2024-04-25T13:40:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90a2ffd640 1 -- 172.21.15.80:0/3788357235 <== osd.6 v2:172.21.15.87:6824/1131074035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f90b006a2c0 con 0x7f9094077650 2024-04-25T13:40:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f9094077650 msgr2=0x7f9094079aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f9094077650 0x7f9094079aa0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f909406f400 msgr2=0x7f9094071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f909406f400 0x7f9094071910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f90b01709d0 tx=0x7f90ac0094d0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b006f6b0 msgr2=0x7f90b0209240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 --2- 172.21.15.80:0/3788357235 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f90b006f6b0 0x7f90b0209240 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f90a800b410 tx=0x7f90a800b8d0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b6d9d640 1 -- 172.21.15.80:0/3788357235 reap_dead start 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 shutdown_connections 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 >> 172.21.15.80:0/3788357235 conn(0x7f90b006cbf0 msgr2=0x7f90b006e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 shutdown_connections 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7f90b8800640 1 -- 172.21.15.80:0/3788357235 wait complete. 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 --> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff170002de0 con 0x7ff17c077740 2024-04-25T13:40:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7ff1777fe640 1 -- 172.21.15.80:0/1925019278 <== osd.1 v2:172.21.15.80:6800/1560454941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff170002de0 con 0x7ff17c077740 2024-04-25T13:40:36.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.044+0000 7fb1a0ff9640 1 -- 172.21.15.80:0/1607861556 <== osd.0 v2:172.21.15.80:6808/2104747008 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb1a4153bd0 con 0x7fb190077770 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7fb190077770 msgr2=0x7fb190079bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7fb190077770 0x7fb190079bc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb19006f500 msgr2=0x7fb190071a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb19006f500 0x7fb190071a10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4167fe0 tx=0x7fb198008040 comp rx=0 tx=0).stop 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1a406fbf0 msgr2=0x7fb1a4153460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 --2- 172.21.15.80:0/1607861556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1a406fbf0 0x7fb1a4153460 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4168c20 tx=0x7fb19c0197d0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1a8987640 1 -- 172.21.15.80:0/1607861556 reap_dead start 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 shutdown_connections 2024-04-25T13:40:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 >> 172.21.15.80:0/1607861556 conn(0x7fb1a406cbf0 msgr2=0x7fb1a4165b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 shutdown_connections 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.048+0000 7fb1aa3ea640 1 -- 172.21.15.80:0/1607861556 wait complete. 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7ff17c077740 msgr2=0x7ff17c079b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7ff17c077740 0x7ff17c079b90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17c06f4d0 msgr2=0x7ff17c0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17c06f4d0 0x7ff17c0719e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1880012f0 tx=0x7ff188009040 comp rx=0 tx=0).stop 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900d0f80 msgr2=0x7ff1900c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1925019278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900d0f80 0x7ff1900c5ef0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff18400e820 tx=0x7ff18400ece0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7ff18f7fe640 1 -- 172.21.15.80:0/1925019278 reap_dead start 2024-04-25T13:40:36.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 --> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fed540cb580 con 0x7fed48077740 2024-04-25T13:40:36.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.052+0000 7fed43fff640 1 -- 172.21.15.80:0/3880318423 <== osd.3 v2:172.21.15.80:6823/3781596558 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fed540cb580 con 0x7fed48077740 2024-04-25T13:40:36.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.056+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 shutdown_connections 2024-04-25T13:40:36.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.056+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 >> 172.21.15.80:0/1925019278 conn(0x7ff1900b3410 msgr2=0x7ff190160450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.056+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 shutdown_connections 2024-04-25T13:40:36.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.056+0000 7ff18ffff640 1 -- 172.21.15.80:0/1925019278 wait complete. 2024-04-25T13:40:36.086 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.086 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T13:40:36.087 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.087 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T13:40:36.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7fed48077740 msgr2=0x7fed48079b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7fed48077740 0x7fed48079b90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 --> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8ff006f6e0 con 0x7f8ff0163530 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fed4806f4d0 msgr2=0x7fed480719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fed4806f4d0 0x7fed480719e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fed4c0ad000 tx=0x7fed4c004740 comp rx=0 tx=0).stop 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed540d9ee0 msgr2=0x7fed540ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 --2- 172.21.15.80:0/3880318423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fed540d9ee0 0x7fed540ce030 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0754a0 tx=0x7fed5c06af70 comp rx=0 tx=0).stop 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed5b7fe640 1 -- 172.21.15.80:0/3880318423 reap_dead start 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 shutdown_connections 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 >> 172.21.15.80:0/3880318423 conn(0x7fed54054d00 msgr2=0x7fed54035a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 shutdown_connections 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7fed61a3e640 1 -- 172.21.15.80:0/3880318423 wait complete. 2024-04-25T13:40:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd7fff640 1 -- 172.21.15.80:0/2154635856 <== osd.5 v2:172.21.15.87:6800/149615972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8ff006f6e0 con 0x7f8ff0163530 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f8ff0163530 msgr2=0x7f8ff00c0250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f8ff0163530 0x7f8ff00c0250 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fdc06f4d0 msgr2=0x7f8fdc0719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fdc06f4d0 0x7f8fdc0719e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00af000 tx=0x7f8fe00a7000 comp rx=0 tx=0).stop 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff00cf750 msgr2=0x7f8ff0151450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 --2- 172.21.15.80:0/2154635856 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ff00cf750 0x7f8ff0151450 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400d5f0 tx=0x7f8fe400dab0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8feffff640 1 -- 172.21.15.80:0/2154635856 reap_dead start 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 shutdown_connections 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 >> 172.21.15.80:0/2154635856 conn(0x7f8ff00b3410 msgr2=0x7f8ff00b69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 shutdown_connections 2024-04-25T13:40:36.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.072+0000 7f8fd5ffb640 1 -- 172.21.15.80:0/2154635856 wait complete. 2024-04-25T13:40:36.092 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.092 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T13:40:36.093 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.093 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T13:40:36.094 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.095 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T13:40:36.095 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.096 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T13:40:36.098 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.098 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T13:40:36.100 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.101 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T13:40:36.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.292+0000 7f7d84683640 1 Processor -- start 2024-04-25T13:40:36.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.292+0000 7f7d84683640 1 -- start start 2024-04-25T13:40:36.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d84683640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7d7c0b6660 0x7f7d7c15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d84683640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7d7c160110 0x7f7d7c071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d84683640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d7c160520 con 0x7f7d7c153130 2024-04-25T13:40:36.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d84683640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7c0b5650 con 0x7f7d7c160110 2024-04-25T13:40:36.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d84683640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7d7c0b58a0 con 0x7f7d7c0b6660 2024-04-25T13:40:36.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d8241f640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7d7c153130 0x7f7d7c0b0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35664/0 (socket says 172.21.15.80:35664) 2024-04-25T13:40:36.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d8241f640 1 -- 172.21.15.80:0/3389933386 learned_addr learned my addr 172.21.15.80:0/3389933386 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:36.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d82c20640 1 --2- 172.21.15.80:0/3389933386 >> v2:172.21.15.80:3300/0 conn(0x7f7d7c160110 0x7f7d7c071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d81c1e640 1 --2- 172.21.15.80:0/3389933386 >> v2:172.21.15.87:3300/0 conn(0x7f7d7c0b6660 0x7f7d7c15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d82c20640 1 -- 172.21.15.80:0/3389933386 >> v2:172.21.15.87:3300/0 conn(0x7f7d7c0b6660 msgr2=0x7f7d7c15fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d82c20640 1 --2- 172.21.15.80:0/3389933386 >> v2:172.21.15.87:3300/0 conn(0x7f7d7c0b6660 0x7f7d7c15fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d82c20640 1 -- 172.21.15.80:0/3389933386 >> v1:172.21.15.80:6789/0 conn(0x7f7d7c153130 legacy=0x7f7d7c0b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.296+0000 7f7d82c20640 1 -- 172.21.15.80:0/3389933386 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c0b5b80 con 0x7f7d7c160110 2024-04-25T13:40:36.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d82c20640 1 --2- 172.21.15.80:0/3389933386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c160110 0x7f7d7c071cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400aea0 tx=0x7f7d7401da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be3c88ba7779c5bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d8141d640 1 -- 172.21.15.80:0/3389933386 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740a6690 con 0x7f7d7c160110 2024-04-25T13:40:36.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d8141d640 1 -- 172.21.15.80:0/3389933386 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d740a6830 con 0x7f7d7c160110 2024-04-25T13:40:36.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d8141d640 1 -- 172.21.15.80:0/3389933386 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740a6c00 con 0x7f7d7c160110 2024-04-25T13:40:36.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d84683640 1 -- 172.21.15.80:0/3389933386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c160110 msgr2=0x7f7d7c071cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d84683640 1 --2- 172.21.15.80:0/3389933386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c160110 0x7f7d7c071cb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400aea0 tx=0x7f7d7401da50 comp rx=0 tx=0).stop 2024-04-25T13:40:36.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d84683640 1 -- 172.21.15.80:0/3389933386 shutdown_connections 2024-04-25T13:40:36.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d84683640 1 --2- 172.21.15.80:0/3389933386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c160110 0x7f7d7c071cb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d84683640 1 --2- 172.21.15.80:0/3389933386 >> v2:172.21.15.87:3300/0 conn(0x7f7d7c0b6660 0x7f7d7c15fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d84683640 1 -- 172.21.15.80:0/3389933386 >> 172.21.15.80:0/3389933386 conn(0x7f7d7c06d270 msgr2=0x7f7d7c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d84683640 1 -- 172.21.15.80:0/3389933386 shutdown_connections 2024-04-25T13:40:36.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.300+0000 7f7d84683640 1 -- 172.21.15.80:0/3389933386 wait complete. 2024-04-25T13:40:36.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.304+0000 7f7d84683640 1 Processor -- start 2024-04-25T13:40:36.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.304+0000 7f7d84683640 1 -- start start 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c153130 0x7f7d7c144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d7c160110 0x7f7d7c15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d7c144a90 0x7f7d7c15ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c1690f0 con 0x7f7d7c153130 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c168f70 con 0x7f7d7c144a90 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c153130 0x7f7d7c144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c153130 0x7f7d7c144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36250/0 (socket says 172.21.15.80:36250) 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 -- 172.21.15.80:0/30387126 learned_addr learned my addr 172.21.15.80:0/30387126 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 -- 172.21.15.80:0/30387126 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7d7c169270 con 0x7f7d7c160110 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d81c1e640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d7c160110 0x7f7d7c15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d82c20640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d7c144a90 0x7f7d7c15ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 -- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d7c160110 msgr2=0x7f7d7c15e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d7c160110 0x7f7d7c15e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 -- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d7c144a90 msgr2=0x7f7d7c15ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d7c144a90 0x7f7d7c15ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 -- 172.21.15.80:0/30387126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c146710 con 0x7f7d7c153130 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d8241f640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c153130 0x7f7d7c144240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800c690 tx=0x7f7d7800cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d72ffd640 1 -- 172.21.15.80:0/30387126 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78007960 con 0x7f7d7c153130 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d72ffd640 1 -- 172.21.15.80:0/30387126 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d78008040 con 0x7f7d7c153130 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d72ffd640 1 -- 172.21.15.80:0/30387126 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78007d40 con 0x7f7d7c153130 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 -- 172.21.15.80:0/30387126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c1469f0 con 0x7f7d7c153130 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 -- 172.21.15.80:0/30387126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c146fb0 con 0x7f7d7c153130 2024-04-25T13:40:36.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.308+0000 7f7d84683640 1 -- 172.21.15.80:0/30387126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c1471b0 con 0x7f7d7c153130 2024-04-25T13:40:36.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7f7d72ffd640 1 -- 172.21.15.80:0/30387126 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7d78018400 con 0x7f7d7c153130 2024-04-25T13:40:36.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7f7d72ffd640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d6006f4d0 0x7f7d600719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7f7d72ffd640 1 -- 172.21.15.80:0/30387126 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d7808f7f0 con 0x7f7d7c153130 2024-04-25T13:40:36.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7f7d81c1e640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d6006f4d0 0x7f7d600719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7f7d81c1e640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d6006f4d0 0x7f7d600719e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0d24e0 tx=0x7f7d6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7f6670ce1640 1 Processor -- start 2024-04-25T13:40:36.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7faa335c7640 1 Processor -- start 2024-04-25T13:40:36.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7faa335c7640 1 -- start start 2024-04-25T13:40:36.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.312+0000 7f6670ce1640 1 -- start start 2024-04-25T13:40:36.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f4924838640 1 Processor -- start 2024-04-25T13:40:36.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de8ddf640 1 Processor -- start 2024-04-25T13:40:36.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de8ddf640 1 -- start start 2024-04-25T13:40:36.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f4924838640 1 -- start start 2024-04-25T13:40:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6670ce1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c0b6660 0x7f666c15fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6670ce1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f666c160110 0x7f666c071cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa335c7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faa2c0b0bc0 0x7faa2c159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa335c7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c06e070 0x7faa2c15c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa335c7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa2c072380 con 0x7faa2c06dc60 2024-04-25T13:40:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa335c7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7faa2c0729d0 con 0x7faa2c0b0bc0 2024-04-25T13:40:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa335c7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faa2c072660 con 0x7faa2c06e070 2024-04-25T13:40:36.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6669d9b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c0b6660 0x7f666c15fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6670ce1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f666c160520 con 0x7f666c153130 2024-04-25T13:40:36.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6669d9b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c0b6660 0x7f666c15fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33404/0 (socket says 172.21.15.80:33404) 2024-04-25T13:40:36.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6669d9b640 1 -- 172.21.15.80:0/1239528989 learned_addr learned my addr 172.21.15.80:0/1239528989 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6670ce1640 1 -- 172.21.15.80:0/1239528989 --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f666c0b5650 con 0x7f666c160110 2024-04-25T13:40:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa31b64640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c06e070 0x7faa2c15c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6670ce1640 1 -- 172.21.15.80:0/1239528989 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f666c0b58a0 con 0x7f666c0b6660 2024-04-25T13:40:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa31b64640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c06e070 0x7faa2c15c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33414/0 (socket says 172.21.15.80:33414) 2024-04-25T13:40:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa31b64640 1 -- 172.21.15.80:0/3049350406 learned_addr learned my addr 172.21.15.80:0/3049350406 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3758d4f640 1 Processor -- start 2024-04-25T13:40:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa31b64640 1 -- 172.21.15.80:0/3049350406 >> v2:172.21.15.80:3300/0 conn(0x7faa2c0b0bc0 msgr2=0x7faa2c159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa31b64640 1 --2- 172.21.15.80:0/3049350406 >> v2:172.21.15.80:3300/0 conn(0x7faa2c0b0bc0 0x7faa2c159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa31b64640 1 -- 172.21.15.80:0/3049350406 >> v1:172.21.15.80:6789/0 conn(0x7faa2c06dc60 legacy=0x7faa2c072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7faa31b64640 1 -- 172.21.15.80:0/3049350406 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c15a010 con 0x7faa2c06e070 2024-04-25T13:40:36.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f4924838640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f49201606f0 0x7f49200ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f4924838640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4920160b00 con 0x7f49200d02d0 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f4924838640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4920071cd0 con 0x7f49200cbcc0 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f4924838640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f49200cee60 con 0x7f49201606f0 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f491f59e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f49200d02d0 0x7f49200d06e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35704/0 (socket says 172.21.15.80:35704) 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f491f59e640 1 -- 172.21.15.80:0/3728882945 learned_addr learned my addr 172.21.15.80:0/3728882945 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2074009518 0 0) 0x7f4920071cd0 con 0x7f49200cbcc0 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f490c003530 con 0x7f49200cbcc0 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 <== mon.1 v1:172.21.15.87:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3067442240 0 0) 0x7f490c003530 con 0x7f49200cbcc0 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 >> v2:172.21.15.80:3300/0 conn(0x7f49201606f0 msgr2=0x7f49200ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 --2- 172.21.15.80:0/3728882945 >> v2:172.21.15.80:3300/0 conn(0x7f49201606f0 0x7f49200ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 >> v1:172.21.15.80:6789/0 conn(0x7f49200d02d0 legacy=0x7f49200d06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 --> v1:172.21.15.87:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200ceb80 con 0x7f49200cbcc0 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 <== mon.1 v1:172.21.15.87:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f49080016a0 con 0x7f49200cbcc0 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 >> v1:172.21.15.87:6789/0 conn(0x7f49200cbcc0 legacy=0x7f49200cc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f490c0039c0 0x7f490c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f490c004810 0x7f490c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f490c005400 0x7f490c007aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f49200cee60 con 0x7f490c005400 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f490c008380 con 0x7f490c0039c0 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f490c008580 con 0x7f490c004810 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491f59e640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f490c005400 0x7f490c007aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de8ddf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de40d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de8ddf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3de40cf490 0x7f3de40d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de8ddf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3de40d1780 0x7f3de40d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de8ddf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3de40b72c0 con 0x7f3de40cf490 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de8ddf640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3de40b6990 con 0x7f3de40d1780 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de8ddf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3de40b6c70 con 0x7f3de40cf080 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de40d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de37fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de40d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33420/0 (socket says 172.21.15.80:33420) 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de37fe640 1 -- 172.21.15.80:0/4173346001 learned_addr learned my addr 172.21.15.80:0/4173346001 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de3fff640 1 --2- 172.21.15.80:0/4173346001 >> v2:172.21.15.87:3300/0 conn(0x7f3de40d1780 0x7f3de40d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3de2ffd640 1 --2- 172.21.15.80:0/4173346001 >> v2:172.21.15.80:3300/0 conn(0x7f3de40cf490 0x7f3de40d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3de3fff640 1 -- 172.21.15.80:0/4173346001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 msgr2=0x7f3de40d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3de3fff640 1 --2- 172.21.15.80:0/4173346001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de40d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3de3fff640 1 -- 172.21.15.80:0/4173346001 >> v2:172.21.15.80:3300/0 conn(0x7f3de40cf490 msgr2=0x7f3de40d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3de3fff640 1 --2- 172.21.15.80:0/4173346001 >> v2:172.21.15.80:3300/0 conn(0x7f3de40cf490 0x7f3de40d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3de3fff640 1 -- 172.21.15.80:0/4173346001 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de40b6f50 con 0x7f3de40d1780 2024-04-25T13:40:36.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3de2ffd640 1 --2- 172.21.15.80:0/4173346001 >> v2:172.21.15.80:3300/0 conn(0x7f3de40cf490 0x7f3de40d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:36.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f3758d4f640 1 -- start start 2024-04-25T13:40:36.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3758d4f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c007390 0x7f374c034a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3758d4f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f374c0e7ee0 0x7f374c034fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3758d4f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f374c0524e0 con 0x7f374c006d10 2024-04-25T13:40:36.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3758d4f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f374c006880 con 0x7f374c0e7ee0 2024-04-25T13:40:36.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f3758d4f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f374c008da0 con 0x7f374c007390 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f4917fff640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f490c004810 0x7f490c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa31b64640 1 --2- 172.21.15.80:0/3049350406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c06e070 0x7faa2c15c670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa24009e50 tx=0x7faa2401c9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7f65b52bdf56de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa23fff640 1 -- 172.21.15.80:0/3049350406 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240a5890 con 0x7faa2c06e070 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa23fff640 1 -- 172.21.15.80:0/3049350406 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa240a5a30 con 0x7faa2c06e070 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa23fff640 1 -- 172.21.15.80:0/3049350406 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240a5e00 con 0x7faa2c06e070 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa335c7640 1 -- 172.21.15.80:0/3049350406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c06e070 msgr2=0x7faa2c15c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa335c7640 1 --2- 172.21.15.80:0/3049350406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c06e070 0x7faa2c15c670 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7faa24009e50 tx=0x7faa2401c9f0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa335c7640 1 -- 172.21.15.80:0/3049350406 shutdown_connections 2024-04-25T13:40:36.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa335c7640 1 --2- 172.21.15.80:0/3049350406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c06e070 0x7faa2c15c670 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa335c7640 1 --2- 172.21.15.80:0/3049350406 >> v2:172.21.15.80:3300/0 conn(0x7faa2c0b0bc0 0x7faa2c159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7faa335c7640 1 -- 172.21.15.80:0/3049350406 >> 172.21.15.80:0/3049350406 conn(0x7faa2c0b3770 msgr2=0x7faa2c0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491ed9d640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f490c0039c0 0x7f490c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6669d9b640 1 -- 172.21.15.80:0/1239528989 >> v2:172.21.15.80:3300/0 conn(0x7f666c160110 msgr2=0x7f666c071cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6669d9b640 1 --2- 172.21.15.80:0/1239528989 >> v2:172.21.15.80:3300/0 conn(0x7f666c160110 0x7f666c071cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6669d9b640 1 -- 172.21.15.80:0/1239528989 >> v1:172.21.15.80:6789/0 conn(0x7f666c153130 legacy=0x7f666c0b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.316+0000 7f6669d9b640 1 -- 172.21.15.80:0/1239528989 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c0b5b80 con 0x7f666c0b6660 2024-04-25T13:40:36.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f6669d9b640 1 --2- 172.21.15.80:0/1239528989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c0b6660 0x7f666c15fbd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f665c0090e0 tx=0x7f665c01e580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43c1ac8a3147c9fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f666959a640 1 -- 172.21.15.80:0/1239528989 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b2070 con 0x7f666c0b6660 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f666959a640 1 -- 172.21.15.80:0/1239528989 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c0ad600 con 0x7f666c0b6660 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f666959a640 1 -- 172.21.15.80:0/1239528989 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0ada10 con 0x7f666c0b6660 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f37537fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c007390 0x7f374c034a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f37537fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c007390 0x7f374c034a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33430/0 (socket says 172.21.15.80:33430) 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f37537fe640 1 -- 172.21.15.80:0/752672273 learned_addr learned my addr 172.21.15.80:0/752672273 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f37537fe640 1 -- 172.21.15.80:0/752672273 >> v2:172.21.15.80:3300/0 conn(0x7f374c0e7ee0 msgr2=0x7f374c034fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f37537fe640 1 --2- 172.21.15.80:0/752672273 >> v2:172.21.15.80:3300/0 conn(0x7f374c0e7ee0 0x7f374c034fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f37537fe640 1 -- 172.21.15.80:0/752672273 >> v1:172.21.15.80:6789/0 conn(0x7f374c006d10 legacy=0x7f374c0543f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f37537fe640 1 -- 172.21.15.80:0/752672273 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c009080 con 0x7f374c007390 2024-04-25T13:40:36.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7fca32161640 1 Processor -- start 2024-04-25T13:40:36.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7faa335c7640 1 -- 172.21.15.80:0/3049350406 shutdown_connections 2024-04-25T13:40:36.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7faa335c7640 1 -- 172.21.15.80:0/3049350406 wait complete. 2024-04-25T13:40:36.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de3fff640 1 --2- 172.21.15.80:0/4173346001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de40d1780 0x7f3de40d1b90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3de406ac50 tx=0x7f3ddc01c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd50cc4bd1e96579 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491f59e640 1 -- 172.21.15.80:0/3728882945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f490c004810 msgr2=0x7f490c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491f59e640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f490c004810 0x7f490c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491f59e640 1 -- 172.21.15.80:0/3728882945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f490c0039c0 msgr2=0x7f490c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491f59e640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f490c0039c0 0x7f490c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.320+0000 7f491f59e640 1 -- 172.21.15.80:0/3728882945 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f490c003530 con 0x7f490c005400 2024-04-25T13:40:36.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f491f59e640 1 -- 172.21.15.80:0/3728882945 reap_dead start 2024-04-25T13:40:36.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f491f59e640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f490c005400 0x7f490c007aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49200b1180 tx=0x7f491801c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9632d1466bc9f77d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de27fc640 1 -- 172.21.15.80:0/4173346001 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0b2070 con 0x7f3de40d1780 2024-04-25T13:40:36.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de27fc640 1 -- 172.21.15.80:0/4173346001 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0a5830 con 0x7f3de40d1780 2024-04-25T13:40:36.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de27fc640 1 -- 172.21.15.80:0/4173346001 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0a5c00 con 0x7f3de40d1780 2024-04-25T13:40:36.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 -- 172.21.15.80:0/4173346001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de40d1780 msgr2=0x7f3de40d1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/4173346001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de40d1780 0x7f3de40d1b90 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3de406ac50 tx=0x7f3ddc01c8a0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 -- 172.21.15.80:0/4173346001 shutdown_connections 2024-04-25T13:40:36.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/4173346001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de40d1780 0x7f3de40d1b90 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/4173346001 >> v2:172.21.15.80:3300/0 conn(0x7f3de40cf490 0x7f3de40d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/4173346001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de40d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 -- 172.21.15.80:0/4173346001 >> 172.21.15.80:0/4173346001 conn(0x7f3de40d39a0 msgr2=0x7f3de40d3db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f37537fe640 1 --2- 172.21.15.80:0/752672273 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c007390 0x7f374c034a70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3748006fd0 tx=0x7f374801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2c2c58d1767aa6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f37527fc640 1 -- 172.21.15.80:0/752672273 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480ab5f0 con 0x7f374c007390 2024-04-25T13:40:36.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 -- 172.21.15.80:0/4173346001 shutdown_connections 2024-04-25T13:40:36.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3de8ddf640 1 -- 172.21.15.80:0/4173346001 wait complete. 2024-04-25T13:40:36.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca32161640 1 -- start start 2024-04-25T13:40:36.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca32161640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca2c06e740 0x7fca2c164520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca32161640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca2c0d0970 0x7fca2c0d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca32161640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca2c0d12c0 con 0x7fca2c0cf750 2024-04-25T13:40:36.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca32161640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fca2c1680a0 con 0x7fca2c06e740 2024-04-25T13:40:36.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca32161640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fca2c168340 con 0x7fca2c0d0970 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7faa335c7640 1 Processor -- start 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49180086d0 con 0x7f490c005400 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49180a9020 con 0x7f490c005400 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f491e59c640 1 -- 172.21.15.80:0/3728882945 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4918008310 con 0x7f490c005400 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f4924838640 1 -- 172.21.15.80:0/3728882945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f490c005400 msgr2=0x7f490c007aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f4924838640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f490c005400 0x7f490c007aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f49200b1180 tx=0x7f491801c860 comp rx=0 tx=0).stop 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f4924838640 1 -- 172.21.15.80:0/3728882945 shutdown_connections 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f4924838640 1 --2- 172.21.15.80:0/3728882945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f490c005400 0x7f490c007aa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f4924838640 1 -- 172.21.15.80:0/3728882945 >> 172.21.15.80:0/3728882945 conn(0x7f49200b3240 msgr2=0x7f49200b3650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f4924838640 1 -- 172.21.15.80:0/3728882945 shutdown_connections 2024-04-25T13:40:36.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f4924838640 1 -- 172.21.15.80:0/3728882945 wait complete. 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f7d72ffd640 1 -- 172.21.15.80:0/30387126 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d78014030 con 0x7f7d7c153130 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f6670ce1640 1 -- 172.21.15.80:0/1239528989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c0b6660 msgr2=0x7f666c15fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1239528989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c0b6660 0x7f666c15fbd0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f665c0090e0 tx=0x7f665c01e580 comp rx=0 tx=0).stop 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f6670ce1640 1 -- 172.21.15.80:0/1239528989 shutdown_connections 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1239528989 >> v2:172.21.15.80:3300/0 conn(0x7f666c160110 0x7f666c071cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1239528989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c0b6660 0x7f666c15fbd0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f6670ce1640 1 -- 172.21.15.80:0/1239528989 >> 172.21.15.80:0/1239528989 conn(0x7f666c06d270 msgr2=0x7f666c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f6670ce1640 1 -- 172.21.15.80:0/1239528989 shutdown_connections 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f6670ce1640 1 -- 172.21.15.80:0/1239528989 wait complete. 2024-04-25T13:40:36.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f6670ce1640 1 Processor -- start 2024-04-25T13:40:36.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3de8ddf640 1 Processor -- start 2024-04-25T13:40:36.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3de8ddf640 1 -- start start 2024-04-25T13:40:36.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca3115f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca2c06e740 0x7fca2c164520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca3095e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca2c0d0970 0x7fca2c0d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca3115f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca2c06e740 0x7fca2c164520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36318/0 (socket says 172.21.15.80:36318) 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca3115f640 1 -- 172.21.15.80:0/2735593944 learned_addr learned my addr 172.21.15.80:0/2735593944 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca3095e640 1 -- 172.21.15.80:0/2735593944 >> v2:172.21.15.80:3300/0 conn(0x7fca2c06e740 msgr2=0x7fca2c164520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca3095e640 1 --2- 172.21.15.80:0/2735593944 >> v2:172.21.15.80:3300/0 conn(0x7fca2c06e740 0x7fca2c164520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca3095e640 1 -- 172.21.15.80:0/2735593944 >> v1:172.21.15.80:6789/0 conn(0x7fca2c0cf750 legacy=0x7fca2c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7fca3095e640 1 -- 172.21.15.80:0/2735593944 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c168620 con 0x7fca2c0d0970 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f37527fc640 1 -- 172.21.15.80:0/752672273 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37480b0070 con 0x7f374c007390 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3758d4f640 1 -- 172.21.15.80:0/752672273 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c007390 msgr2=0x7f374c034a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f37527fc640 1 -- 172.21.15.80:0/752672273 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37480ab5f0 con 0x7f374c007390 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3758d4f640 1 --2- 172.21.15.80:0/752672273 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c007390 0x7f374c034a70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3748006fd0 tx=0x7f374801c410 comp rx=0 tx=0).stop 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3758d4f640 1 -- 172.21.15.80:0/752672273 shutdown_connections 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3758d4f640 1 --2- 172.21.15.80:0/752672273 >> v2:172.21.15.80:3300/0 conn(0x7f374c0e7ee0 0x7f374c034fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3758d4f640 1 --2- 172.21.15.80:0/752672273 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c007390 0x7f374c034a70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3758d4f640 1 -- 172.21.15.80:0/752672273 >> 172.21.15.80:0/752672273 conn(0x7f374c055e10 msgr2=0x7f374c056220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3758d4f640 1 -- 172.21.15.80:0/752672273 shutdown_connections 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.324+0000 7f3758d4f640 1 -- 172.21.15.80:0/752672273 wait complete. 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3758d4f640 1 Processor -- start 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3758d4f640 1 -- start start 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3758d4f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374c006d10 0x7f374c035a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3758d4f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c0e7ee0 0x7f374c0377c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3758d4f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f374c035f80 0x7f374c037d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3758d4f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f374c0f2c70 con 0x7f374c006d10 2024-04-25T13:40:36.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3758d4f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f374c0f2af0 con 0x7f374c035f80 2024-04-25T13:40:36.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3758d4f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f374c0f2df0 con 0x7f374c0e7ee0 2024-04-25T13:40:36.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3753fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f374c035f80 0x7f374c037d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3752ffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c0e7ee0 0x7f374c0377c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3752ffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c0e7ee0 0x7f374c0377c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33440/0 (socket says 172.21.15.80:33440) 2024-04-25T13:40:36.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3752ffd640 1 -- 172.21.15.80:0/3846958799 learned_addr learned my addr 172.21.15.80:0/3846958799 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f6670ce1640 1 -- start start 2024-04-25T13:40:36.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f6670ce1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c153130 0x7f666c1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f6670ce1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666c160110 0x7f666c20a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3752ffd640 1 -- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f374c035f80 msgr2=0x7f374c037d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3752ffd640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f374c035f80 0x7f374c037d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3752ffd640 1 -- 172.21.15.80:0/3846958799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374c006d10 msgr2=0x7f374c035a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3752ffd640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374c006d10 0x7f374c035a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.328+0000 7f3752ffd640 1 -- 172.21.15.80:0/3846958799 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c0cffd0 con 0x7f374c0e7ee0 2024-04-25T13:40:36.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3752ffd640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c0e7ee0 0x7f374c0377c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f374400b410 tx=0x7f374400b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f4924838640 1 Processor -- start 2024-04-25T13:40:36.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f373bfff640 1 -- 172.21.15.80:0/3846958799 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744015020 con 0x7f374c0e7ee0 2024-04-25T13:40:36.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de8ddf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de4159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de8ddf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40cf490 0x7f3de4156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de8ddf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de40d1780 0x7f3de41572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de8ddf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3de40b82f0 con 0x7f3de40cf490 2024-04-25T13:40:36.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f373bfff640 1 -- 172.21.15.80:0/3846958799 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3744011830 con 0x7f374c0e7ee0 2024-04-25T13:40:36.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f373bfff640 1 -- 172.21.15.80:0/3846958799 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744011c20 con 0x7f374c0e7ee0 2024-04-25T13:40:36.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3758d4f640 1 -- 172.21.15.80:0/3846958799 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c0d9250 con 0x7f374c0e7ee0 2024-04-25T13:40:36.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3758d4f640 1 -- 172.21.15.80:0/3846958799 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c0e7390 con 0x7f374c0e7ee0 2024-04-25T13:40:36.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40cf490 0x7f3de4156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40cf490 0x7f3de4156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36334/0 (socket says 172.21.15.80:36334) 2024-04-25T13:40:36.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 -- 172.21.15.80:0/256822499 learned_addr learned my addr 172.21.15.80:0/256822499 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f4924838640 1 -- start start 2024-04-25T13:40:36.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de37fe640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de4159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3de40b8170 con 0x7f3de40d1780 2024-04-25T13:40:36.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3de40b8470 con 0x7f3de40cf080 2024-04-25T13:40:36.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 -- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 msgr2=0x7f3de4159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de4159330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 -- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de40d1780 msgr2=0x7f3de41572a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T13:40:36.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de40d1780 0x7f3de41572a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 -- 172.21.15.80:0/256822499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3de40b4a60 con 0x7f3de40cf490 2024-04-25T13:40:36.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7fca3095e640 1 --2- 172.21.15.80:0/2735593944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c0d0970 0x7fca2c0d0d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0cf680 tx=0x7fca1c01c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=243fcf602190ba3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3758d4f640 1 -- 172.21.15.80:0/3846958799 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f374c005d60 con 0x7f374c0e7ee0 2024-04-25T13:40:36.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7fca23fff640 1 -- 172.21.15.80:0/2735593944 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b0070 con 0x7fca2c0d0970 2024-04-25T13:40:36.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7fca23fff640 1 -- 172.21.15.80:0/2735593944 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c0ab600 con 0x7fca2c0d0970 2024-04-25T13:40:36.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7fca23fff640 1 -- 172.21.15.80:0/2735593944 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0aba10 con 0x7fca2c0d0970 2024-04-25T13:40:36.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa335c7640 1 -- start start 2024-04-25T13:40:36.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa335c7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa2c06dc60 0x7faa2c0aba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f666a59c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c153130 0x7f666c1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f666a59c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c153130 0x7f666c1516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33448/0 (socket says 172.21.15.80:33448) 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f666a59c640 1 -- 172.21.15.80:0/1966258294 learned_addr learned my addr 172.21.15.80:0/1966258294 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f666c154b20 0x7f666c154f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f666c1690f0 con 0x7f666c160110 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f666c168f70 con 0x7f666c154b20 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6669d9b640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666c160110 0x7f666c20a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f666c169270 con 0x7f666c153130 2024-04-25T13:40:36.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f3de2ffd640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40cf490 0x7f3de4156d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c7f0 tx=0x7f3dd400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31363640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa2c06dc60 0x7faa2c0aba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31363640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa2c06dc60 0x7faa2c0aba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36362/0 (socket says 172.21.15.80:36362) 2024-04-25T13:40:36.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f666ad9d640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f666c154b20 0x7f666c154f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31363640 1 -- 172.21.15.80:0/4174539675 learned_addr learned my addr 172.21.15.80:0/4174539675 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa335c7640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa2c06e070 0x7faa2c1431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6669d9b640 1 -- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c153130 msgr2=0x7f666c1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6669d9b640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c153130 0x7f666c1516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6669d9b640 1 -- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f666c154b20 msgr2=0x7f666c154f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6669d9b640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f666c154b20 0x7f666c154f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6669d9b640 1 -- 172.21.15.80:0/1966258294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666c155470 con 0x7f666c160110 2024-04-25T13:40:36.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa335c7640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c0b0bc0 0x7faa2c0ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa335c7640 1 -- 172.21.15.80:0/4174539675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1633a0 con 0x7faa2c06dc60 2024-04-25T13:40:36.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa335c7640 1 -- 172.21.15.80:0/4174539675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c163220 con 0x7faa2c06e070 2024-04-25T13:40:36.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa335c7640 1 -- 172.21.15.80:0/4174539675 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faa2c163520 con 0x7faa2c0b0bc0 2024-04-25T13:40:36.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7f6669d9b640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666c160110 0x7f666c20a710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f666c0b57d0 tx=0x7f665c01cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa30b62640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa2c06e070 0x7faa2c1431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31b64640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c0b0bc0 0x7faa2c0ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31363640 1 -- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c0b0bc0 msgr2=0x7faa2c0ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31363640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c0b0bc0 0x7faa2c0ad280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31363640 1 -- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa2c06e070 msgr2=0x7faa2c1431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31363640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa2c06e070 0x7faa2c1431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa31363640 1 -- 172.21.15.80:0/4174539675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c1409f0 con 0x7faa2c06dc60 2024-04-25T13:40:36.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.332+0000 7faa30b62640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa2c06e070 0x7faa2c1431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:36.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f373bfff640 1 -- 172.21.15.80:0/3846958799 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3744018020 con 0x7f374c0e7ee0 2024-04-25T13:40:36.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7fca32161640 1 -- 172.21.15.80:0/2735593944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c0d0970 msgr2=0x7fca2c0d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7fca32161640 1 --2- 172.21.15.80:0/2735593944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c0d0970 0x7fca2c0d0d80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca2c0cf680 tx=0x7fca1c01c4c0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7faa31363640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa2c06dc60 0x7faa2c0aba20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa2800b3f0 tx=0x7faa2800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7fca32161640 1 -- 172.21.15.80:0/2735593944 shutdown_connections 2024-04-25T13:40:36.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7fca32161640 1 --2- 172.21.15.80:0/2735593944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c0d0970 0x7fca2c0d0d80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7fca32161640 1 --2- 172.21.15.80:0/2735593944 >> v2:172.21.15.80:3300/0 conn(0x7fca2c06e740 0x7fca2c164520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7fca32161640 1 -- 172.21.15.80:0/2735593944 >> 172.21.15.80:0/2735593944 conn(0x7fca2c0b3410 msgr2=0x7fca2c0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f373bfff640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f374006f400 0x7f3740071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f373bfff640 1 -- 172.21.15.80:0/3846958799 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f374408f7e0 con 0x7f374c0e7ee0 2024-04-25T13:40:36.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f37537fe640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f374006f400 0x7f3740071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f4924838640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4920154b30 0x7f49200b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f4924838640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49200b6a00 0x7f492015a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f4924838640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015a8f0 0x7f492015c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f37537fe640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f374006f400 0x7f3740071910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37480af000 tx=0x7f3748004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f4924838640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f49200cfdb0 con 0x7f492015a8f0 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f4924838640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f49200cfad0 con 0x7f4920154b30 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f4924838640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f49200b3d60 con 0x7f49200b6a00 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015a8f0 0x7f492015c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015a8f0 0x7f492015c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36372/0 (socket says 172.21.15.80:36372) 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 -- 172.21.15.80:0/1086229133 learned_addr learned my addr 172.21.15.80:0/1086229133 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f4917fff640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49200b6a00 0x7f492015a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491ed9d640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4920154b30 0x7f49200b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 -- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49200b6a00 msgr2=0x7f492015a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49200b6a00 0x7f492015a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 -- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4920154b30 msgr2=0x7f49200b64c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4920154b30 0x7f49200b64c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 -- 172.21.15.80:0/1086229133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200b6e10 con 0x7f492015a8f0 2024-04-25T13:40:36.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f4917fff640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49200b6a00 0x7f492015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.336+0000 7f491f59e640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015a8f0 0x7f492015c610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f491801d100 tx=0x7f490c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7faa21ffb640 1 -- 172.21.15.80:0/4174539675 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28015020 con 0x7faa2c06dc60 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7faa21ffb640 1 -- 172.21.15.80:0/4174539675 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa28011850 con 0x7faa2c06dc60 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7faa21ffb640 1 -- 172.21.15.80:0/4174539675 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28011c40 con 0x7faa2c06dc60 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f49177fe640 1 -- 172.21.15.80:0/1086229133 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49180a5760 con 0x7f492015a8f0 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f49177fe640 1 -- 172.21.15.80:0/1086229133 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49180a5900 con 0x7f492015a8f0 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f4924838640 1 -- 172.21.15.80:0/1086229133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49201f2380 con 0x7f492015a8f0 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f4924838640 1 -- 172.21.15.80:0/1086229133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4920157270 con 0x7f492015a8f0 2024-04-25T13:40:36.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f49177fe640 1 -- 172.21.15.80:0/1086229133 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49180a5ab0 con 0x7f492015a8f0 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f665affd640 1 -- 172.21.15.80:0/1966258294 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0bd070 con 0x7f666c160110 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f665affd640 1 -- 172.21.15.80:0/1966258294 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f665c0ade50 con 0x7f666c160110 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f665affd640 1 -- 172.21.15.80:0/1966258294 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f665c0b2070 con 0x7f666c160110 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f666c1582b0 con 0x7f666c160110 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3dcbfff640 1 -- 172.21.15.80:0/256822499 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4015070 con 0x7f3de40cf490 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3de4155330 con 0x7f3de40cf490 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3dcbfff640 1 -- 172.21.15.80:0/256822499 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd4007820 con 0x7f3de40cf490 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3dcbfff640 1 -- 172.21.15.80:0/256822499 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd401f060 con 0x7f3de40cf490 2024-04-25T13:40:36.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3de4159d60 con 0x7f3de40cf490 2024-04-25T13:40:36.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666c158870 con 0x7f666c160110 2024-04-25T13:40:36.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7faa335c7640 1 -- 172.21.15.80:0/4174539675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2c0ac1e0 con 0x7faa2c06dc60 2024-04-25T13:40:36.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7faa335c7640 1 -- 172.21.15.80:0/4174539675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2c0ac7a0 con 0x7faa2c06dc60 2024-04-25T13:40:36.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3dcbfff640 1 -- 172.21.15.80:0/256822499 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3dd4018070 con 0x7f3de40cf490 2024-04-25T13:40:36.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f49157fa640 1 -- 172.21.15.80:0/1086229133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f492006a2c0 con 0x7f492015a8f0 2024-04-25T13:40:36.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3dcbfff640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3dd006f400 0x7f3dd0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666c06a2c0 con 0x7f666c160110 2024-04-25T13:40:36.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3de37fe640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3dd006f400 0x7f3dd0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7faa0f7fe640 1 -- 172.21.15.80:0/4174539675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa2c140bc0 con 0x7faa2c06dc60 2024-04-25T13:40:36.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f49177fe640 1 -- 172.21.15.80:0/1086229133 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f49180b5030 con 0x7f492015a8f0 2024-04-25T13:40:36.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.340+0000 7f3dcbfff640 1 -- 172.21.15.80:0/256822499 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f3dd408dc40 con 0x7f3de40cf490 2024-04-25T13:40:36.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f3de37fe640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3dd006f400 0x7f3dd0071910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3de40c74a0 tx=0x7f3dd8009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3de406a2c0 con 0x7f3de40cf490 2024-04-25T13:40:36.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7fca32161640 1 -- 172.21.15.80:0/2735593944 shutdown_connections 2024-04-25T13:40:36.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f49177fe640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f490006f400 0x7f4900071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f49177fe640 1 -- 172.21.15.80:0/1086229133 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4918126480 con 0x7f492015a8f0 2024-04-25T13:40:36.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f491ed9d640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f490006f400 0x7f4900071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f491ed9d640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f490006f400 0x7f4900071910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49200cbbf0 tx=0x7f4910008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f665affd640 1 -- 172.21.15.80:0/1966258294 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f665c0acbc0 con 0x7f666c160110 2024-04-25T13:40:36.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f665affd640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664c06f430 0x7f664c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f666a59c640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664c06f430 0x7f664c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7faa21ffb640 1 -- 172.21.15.80:0/4174539675 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faa28018020 con 0x7faa2c06dc60 2024-04-25T13:40:36.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7faa21ffb640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa1006f400 0x7faa10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7fca32161640 1 -- 172.21.15.80:0/2735593944 wait complete. 2024-04-25T13:40:36.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7faa30b62640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa1006f400 0x7faa10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7fca32161640 1 Processor -- start 2024-04-25T13:40:36.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7f666a59c640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664c06f430 0x7f664c071940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f66600099a0 tx=0x7f66600090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.344+0000 7fca32161640 1 -- start start 2024-04-25T13:40:36.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7faa30b62640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa1006f400 0x7faa10071910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa2c1408c0 tx=0x7faa1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca32161640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c06e740 0x7fca2c0ab920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca32161640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c0cf750 0x7fca2c0b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c0cf750 0x7fca2c0b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c0cf750 0x7fca2c0b6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36388/0 (socket says 172.21.15.80:36388) 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3115f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c06e740 0x7fca2c0ab920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 -- 172.21.15.80:0/4069156100 learned_addr learned my addr 172.21.15.80:0/4069156100 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca32161640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca2c0d0970 0x7fca2c1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c16fd60 con 0x7fca2c0cf750 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca2c16fbe0 con 0x7fca2c06e740 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fca2c16fee0 con 0x7fca2c0d0970 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca31960640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca2c0d0970 0x7fca2c1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7f3dcbfff640 1 -- 172.21.15.80:0/256822499 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3dd405bf30 con 0x7f3de40cf490 2024-04-25T13:40:36.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 -- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca2c0d0970 msgr2=0x7fca2c1506d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca2c0d0970 0x7fca2c1506d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 -- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c06e740 msgr2=0x7fca2c0ab920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c06e740 0x7fca2c0ab920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 -- 172.21.15.80:0/4069156100 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca2c0ae590 con 0x7fca2c0cf750 2024-04-25T13:40:36.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7f665affd640 1 -- 172.21.15.80:0/1966258294 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f665c127a70 con 0x7f666c160110 2024-04-25T13:40:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7f373bfff640 1 -- 172.21.15.80:0/3846958799 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f374401f020 con 0x7f374c0e7ee0 2024-04-25T13:40:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7faa21ffb640 1 -- 172.21.15.80:0/4174539675 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7faa28011de0 con 0x7faa2c06dc60 2024-04-25T13:40:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.348+0000 7fca3095e640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c0cf750 0x7fca2c0b6070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ae040 tx=0x7fca1c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efd1ce640 1 Processor -- start 2024-04-25T13:40:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efd1ce640 1 -- start start 2024-04-25T13:40:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f49177fe640 1 -- 172.21.15.80:0/1086229133 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f49180bc020 con 0x7f492015a8f0 2024-04-25T13:40:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7fca21ffb640 1 -- 172.21.15.80:0/4069156100 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0b0040 con 0x7fca2c0cf750 2024-04-25T13:40:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efd1ce640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ef00e3fc0 0x7f4ef00e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efd1ce640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef0058540 con 0x7f4ef0057ff0 2024-04-25T13:40:36.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efd1ce640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ef0058820 con 0x7f4ef00547d0 2024-04-25T13:40:36.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efd1ce640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4ef00067d0 con 0x7f4ef00e3fc0 2024-04-25T13:40:36.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4ef7fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4ef0057ff0 0x7f4ef00540c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35746/0 (socket says 172.21.15.80:35746) 2024-04-25T13:40:36.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4ef7fff640 1 -- 172.21.15.80:0/1064963148 learned_addr learned my addr 172.21.15.80:0/1064963148 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:36.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efc9cd640 1 --2- 172.21.15.80:0/1064963148 >> v2:172.21.15.87:3300/0 conn(0x7f4ef00e3fc0 0x7f4ef00e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca2c0ae870 con 0x7fca2c0cf750 2024-04-25T13:40:36.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca2c0aee30 con 0x7fca2c0cf750 2024-04-25T13:40:36.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2c0ae300 con 0x7fca2c0cf750 2024-04-25T13:40:36.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efc9cd640 1 -- 172.21.15.80:0/1064963148 >> v1:172.21.15.87:6789/0 conn(0x7f4ef00547d0 legacy=0x7f4ef00e38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efc9cd640 1 -- 172.21.15.80:0/1064963148 >> v1:172.21.15.80:6789/0 conn(0x7f4ef0057ff0 legacy=0x7f4ef00540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.360+0000 7f4efc9cd640 1 -- 172.21.15.80:0/1064963148 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef0058b00 con 0x7f4ef00e3fc0 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.364+0000 7f4efc9cd640 1 --2- 172.21.15.80:0/1064963148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef00e3fc0 0x7f4ef00e43d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80515a0 tx=0x7f4ef8099560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95ecf865199de186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.364+0000 7f4ef6ffd640 1 -- 172.21.15.80:0/1064963148 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef812d070 con 0x7f4ef00e3fc0 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.364+0000 7f4ef6ffd640 1 -- 172.21.15.80:0/1064963148 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef8128600 con 0x7f4ef00e3fc0 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.364+0000 7f4ef6ffd640 1 -- 172.21.15.80:0/1064963148 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef8128a70 con 0x7f4ef00e3fc0 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 -- 172.21.15.80:0/1064963148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef00e3fc0 msgr2=0x7f4ef00e43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 --2- 172.21.15.80:0/1064963148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef00e3fc0 0x7f4ef00e43d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80515a0 tx=0x7f4ef8099560 comp rx=0 tx=0).stop 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 -- 172.21.15.80:0/1064963148 shutdown_connections 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 --2- 172.21.15.80:0/1064963148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef00e3fc0 0x7f4ef00e43d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 -- 172.21.15.80:0/1064963148 >> 172.21.15.80:0/1064963148 conn(0x7f4ef003ac70 msgr2=0x7f4ef003b080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 -- 172.21.15.80:0/1064963148 shutdown_connections 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 -- 172.21.15.80:0/1064963148 wait complete. 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 Processor -- start 2024-04-25T13:40:36.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.368+0000 7f4efd1ce640 1 -- start start 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efd1ce640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ef0057ff0 0x7f4ef004f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efd1ce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef00e3fc0 0x7f4ef004d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efd1ce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef004da20 0x7f4ef004ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efd1ce640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef00f1dd0 con 0x7f4ef00e3fc0 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef00e3fc0 0x7f4ef004d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efd1ce640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef00e7750 con 0x7f4ef004da20 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef00e3fc0 0x7f4ef004d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36396/0 (socket says 172.21.15.80:36396) 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef77fe640 1 -- 172.21.15.80:0/2226392782 learned_addr learned my addr 172.21.15.80:0/2226392782 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ef00f1f50 con 0x7f4ef0057ff0 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef7fff640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ef0057ff0 0x7f4ef004f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efc9cd640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef004da20 0x7f4ef004ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f665affd640 1 -- 172.21.15.80:0/1966258294 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f666c155470 con 0x7f666c160110 2024-04-25T13:40:36.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef7fff640 1 -- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef004da20 msgr2=0x7f4ef004ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef7fff640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef004da20 0x7f4ef004ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef7fff640 1 -- 172.21.15.80:0/2226392782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef00e3fc0 msgr2=0x7f4ef004d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef7fff640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef00e3fc0 0x7f4ef004d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef7fff640 1 -- 172.21.15.80:0/2226392782 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef00e1ba0 con 0x7f4ef0057ff0 2024-04-25T13:40:36.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef7fff640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ef0057ff0 0x7f4ef004f370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0057350 tx=0x7f4eec00c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef4ff9640 1 -- 172.21.15.80:0/2226392782 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec017070 con 0x7f4ef0057ff0 2024-04-25T13:40:36.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4ef4ff9640 1 -- 172.21.15.80:0/2226392782 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eec005d10 con 0x7f4ef0057ff0 2024-04-25T13:40:36.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef00e1e80 con 0x7f4ef0057ff0 2024-04-25T13:40:36.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.372+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef0050be0 con 0x7f4ef0057ff0 2024-04-25T13:40:36.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.376+0000 7f4ef4ff9640 1 -- 172.21.15.80:0/2226392782 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec007310 con 0x7f4ef0057ff0 2024-04-25T13:40:36.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.376+0000 7f4ef4ff9640 1 -- 172.21.15.80:0/2226392782 <== mon.2 v2:172.21.15.87:3301/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4eec01a070 con 0x7f4ef0057ff0 2024-04-25T13:40:36.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.376+0000 7f4ef4ff9640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4edc06f400 0x7f4edc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.376+0000 7f4ef4ff9640 1 -- 172.21.15.80:0/2226392782 <== mon.2 v2:172.21.15.87:3301/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4eec08e4b0 con 0x7f4ef0057ff0 2024-04-25T13:40:36.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.376+0000 7f4ef77fe640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4edc06f400 0x7f4edc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.380+0000 7f4ef77fe640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4edc06f400 0x7f4edc071910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4001810 tx=0x7f4ee4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.380+0000 7faa21ffb640 1 -- 172.21.15.80:0/4174539675 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa2801f020 con 0x7faa2c06dc60 2024-04-25T13:40:36.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.384+0000 7fca21ffb640 1 -- 172.21.15.80:0/4069156100 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca1c0ba070 con 0x7fca2c0cf750 2024-04-25T13:40:36.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.384+0000 7fca21ffb640 1 -- 172.21.15.80:0/4069156100 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca1c006e20 con 0x7fca2c0cf750 2024-04-25T13:40:36.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.384+0000 7fca21ffb640 1 -- 172.21.15.80:0/4069156100 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fca1c0be610 con 0x7fca2c0cf750 2024-04-25T13:40:36.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.384+0000 7fca21ffb640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca1006f400 0x7fca10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.384+0000 7fca21ffb640 1 -- 172.21.15.80:0/4069156100 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fca1c129280 con 0x7fca2c0cf750 2024-04-25T13:40:36.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.384+0000 7fca3115f640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca1006f400 0x7fca10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.388+0000 7fca3115f640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca1006f400 0x7fca10071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca28009ab0 tx=0x7fca28009240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.396+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef0005900 con 0x7f4ef0057ff0 2024-04-25T13:40:36.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.396+0000 7fca21ffb640 1 -- 172.21.15.80:0/4069156100 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca1c0c3020 con 0x7fca2c0cf750 2024-04-25T13:40:36.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.408+0000 7f4ef4ff9640 1 -- 172.21.15.80:0/2226392782 <== mon.2 v2:172.21.15.87:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4eec05ad60 con 0x7f4ef0057ff0 2024-04-25T13:40:36.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.636+0000 7f7d84683640 1 -- 172.21.15.80:0/30387126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7d7c06a2c0 con 0x7f7d7c153130 2024-04-25T13:40:36.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.636+0000 7f7d72ffd640 1 -- 172.21.15.80:0/30387126 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7d7c160520 con 0x7f7d7c153130 2024-04-25T13:40:36.638 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 -- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d6006f4d0 msgr2=0x7f7d600719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d6006f4d0 0x7f7d600719e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c0d24e0 tx=0x7f7d6c008040 comp rx=0 tx=0).stop 2024-04-25T13:40:36.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 -- 172.21.15.80:0/30387126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c153130 msgr2=0x7f7d7c144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c153130 0x7f7d7c144240 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d7800c690 tx=0x7f7d7800cb50 comp rx=0 tx=0).stop 2024-04-25T13:40:36.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 -- 172.21.15.80:0/30387126 shutdown_connections 2024-04-25T13:40:36.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d6006f4d0 0x7f7d600719e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d7c144a90 0x7f7d7c15ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d7c160110 0x7f7d7c15e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 --2- 172.21.15.80:0/30387126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d7c153130 0x7f7d7c144240 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 -- 172.21.15.80:0/30387126 >> 172.21.15.80:0/30387126 conn(0x7f7d7c0b6660 msgr2=0x7f7d7c14b730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 -- 172.21.15.80:0/30387126 shutdown_connections 2024-04-25T13:40:36.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.644+0000 7f7d70ff9640 1 -- 172.21.15.80:0/30387126 wait complete. 2024-04-25T13:40:36.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.652+0000 7f3758d4f640 1 -- 172.21.15.80:0/3846958799 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f374c0dd4b0 con 0x7f374c0e7ee0 2024-04-25T13:40:36.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.652+0000 7f373bfff640 1 -- 172.21.15.80:0/3846958799 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f374405c630 con 0x7f374c0e7ee0 2024-04-25T13:40:36.655 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 -- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f374006f400 msgr2=0x7f3740071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f374006f400 0x7f3740071910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f37480af000 tx=0x7f3748004740 comp rx=0 tx=0).stop 2024-04-25T13:40:36.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 -- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c0e7ee0 msgr2=0x7f374c0377c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c0e7ee0 0x7f374c0377c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f374400b410 tx=0x7f374400b8d0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 -- 172.21.15.80:0/3846958799 shutdown_connections 2024-04-25T13:40:36.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f374006f400 0x7f3740071910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f374c035f80 0x7f374c037d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f374c0e7ee0 0x7f374c0377c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 --2- 172.21.15.80:0/3846958799 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f374c006d10 0x7f374c035a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 -- 172.21.15.80:0/3846958799 >> 172.21.15.80:0/3846958799 conn(0x7f374c055e10 msgr2=0x7f374c04ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 -- 172.21.15.80:0/3846958799 shutdown_connections 2024-04-25T13:40:36.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.660+0000 7f3739ffb640 1 -- 172.21.15.80:0/3846958799 wait complete. 2024-04-25T13:40:36.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.668+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3de4152e50 con 0x7f3de40cf490 2024-04-25T13:40:36.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.668+0000 7f3dcbfff640 1 -- 172.21.15.80:0/256822499 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3dd405e070 con 0x7f3de40cf490 2024-04-25T13:40:36.670 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.671 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.4 2024-04-25T13:40:36.671 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3dd006f400 msgr2=0x7f3dd0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3dd006f400 0x7f3dd0071910 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3de40c74a0 tx=0x7f3dd8009240 comp rx=0 tx=0).stop 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40cf490 msgr2=0x7f3de4156d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40cf490 0x7f3de4156d60 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400c7f0 tx=0x7f3dd400ccb0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 shutdown_connections 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3dd006f400 0x7f3dd0071910 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3de40d1780 0x7f3de41572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3de40cf490 0x7f3de4156d60 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 --2- 172.21.15.80:0/256822499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3de40cf080 0x7f3de4159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 >> 172.21.15.80:0/256822499 conn(0x7f3de40d39a0 msgr2=0x7f3de415b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 shutdown_connections 2024-04-25T13:40:36.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.672+0000 7f3de8ddf640 1 -- 172.21.15.80:0/256822499 wait complete. 2024-04-25T13:40:36.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.680+0000 7f49157fa640 1 -- 172.21.15.80:0/1086229133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f492006fb10 con 0x7f492015a8f0 2024-04-25T13:40:36.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.684+0000 7f49177fe640 1 -- 172.21.15.80:0/1086229133 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f49200b6e10 con 0x7f492015a8f0 2024-04-25T13:40:36.683 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.684+0000 7f49157fa640 1 -- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f490006f400 msgr2=0x7f4900071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.684+0000 7f49157fa640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f490006f400 0x7f4900071910 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49200cbbf0 tx=0x7f4910008040 comp rx=0 tx=0).stop 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.684+0000 7f49157fa640 1 -- 172.21.15.80:0/1086229133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015a8f0 msgr2=0x7f492015c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.684+0000 7f49157fa640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015a8f0 0x7f492015c610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f491801d100 tx=0x7f490c005840 comp rx=0 tx=0).stop 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.688+0000 7f49157fa640 1 -- 172.21.15.80:0/1086229133 shutdown_connections 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.688+0000 7f49157fa640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f490006f400 0x7f4900071910 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.688+0000 7f49157fa640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f492015a8f0 0x7f492015c610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.688+0000 7f49157fa640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f49200b6a00 0x7f492015a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.688+0000 7f49157fa640 1 --2- 172.21.15.80:0/1086229133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4920154b30 0x7f49200b64c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.688+0000 7f49157fa640 1 -- 172.21.15.80:0/1086229133 >> 172.21.15.80:0/1086229133 conn(0x7f49200b3240 msgr2=0x7f49200b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.688+0000 7f49157fa640 1 -- 172.21.15.80:0/1086229133 shutdown_connections 2024-04-25T13:40:36.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.688+0000 7f49157fa640 1 -- 172.21.15.80:0/1086229133 wait complete. 2024-04-25T13:40:36.689 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.1 2024-04-25T13:40:36.689 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:36.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f666c20ac50 con 0x7f666c160110 2024-04-25T13:40:36.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f665affd640 1 -- 172.21.15.80:0/1966258294 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f666c160520 con 0x7f666c160110 2024-04-25T13:40:36.691 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2024-04-25T13:40:36.694 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:36.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664c06f430 msgr2=0x7f664c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664c06f430 0x7f664c071940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f66600099a0 tx=0x7f66600090e0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666c160110 msgr2=0x7f666c20a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666c160110 0x7f666c20a710 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f666c0b57d0 tx=0x7f665c01cdc0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 shutdown_connections 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664c06f430 0x7f664c071940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f666c154b20 0x7f666c154f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666c160110 0x7f666c20a710 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 --2- 172.21.15.80:0/1966258294 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f666c153130 0x7f666c1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 >> 172.21.15.80:0/1966258294 conn(0x7f666c0b6660 msgr2=0x7f666c0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 shutdown_connections 2024-04-25T13:40:36.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.692+0000 7f6670ce1640 1 -- 172.21.15.80:0/1966258294 wait complete. 2024-04-25T13:40:36.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.708+0000 7faa0f7fe640 1 -- 172.21.15.80:0/4174539675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7faa2c06a2c0 con 0x7faa2c06dc60 2024-04-25T13:40:36.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.708+0000 7faa21ffb640 1 -- 172.21.15.80:0/4174539675 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faa2805c750 con 0x7faa2c06dc60 2024-04-25T13:40:36.709 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 -- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa1006f400 msgr2=0x7faa10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa1006f400 0x7faa10071910 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa2c1408c0 tx=0x7faa1c008040 comp rx=0 tx=0).stop 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 -- 172.21.15.80:0/4174539675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa2c06dc60 msgr2=0x7faa2c0aba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa2c06dc60 0x7faa2c0aba20 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7faa2800b3f0 tx=0x7faa2800b8b0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 -- 172.21.15.80:0/4174539675 shutdown_connections 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa1006f400 0x7faa10071910 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa2c0b0bc0 0x7faa2c0ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa2c06e070 0x7faa2c1431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 --2- 172.21.15.80:0/4174539675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa2c06dc60 0x7faa2c0aba20 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 -- 172.21.15.80:0/4174539675 >> 172.21.15.80:0/4174539675 conn(0x7faa2c0b3770 msgr2=0x7faa2c0b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 -- 172.21.15.80:0/4174539675 shutdown_connections 2024-04-25T13:40:36.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.712+0000 7faa0f7fe640 1 -- 172.21.15.80:0/4174539675 wait complete. 2024-04-25T13:40:36.714 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-04-25T13:40:36.714 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:36.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.2 2024-04-25T13:40:36.717 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:36.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.720+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fca2c06a2c0 con 0x7fca2c0cf750 2024-04-25T13:40:36.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.720+0000 7fca21ffb640 1 -- 172.21.15.80:0/4069156100 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fca1c0160b0 con 0x7fca2c0cf750 2024-04-25T13:40:36.721 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4ef004c3d0 con 0x7f4ef0057ff0 2024-04-25T13:40:36.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4ef4ff9640 1 -- 172.21.15.80:0/2226392782 <== mon.2 v2:172.21.15.87:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4eec05f060 con 0x7f4ef0057ff0 2024-04-25T13:40:36.724 INFO:teuthology.orchestra.run.smithi080.stdout:55834574853 2024-04-25T13:40:36.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca1006f400 msgr2=0x7fca10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca1006f400 0x7fca10071910 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fca28009ab0 tx=0x7fca28009240 comp rx=0 tx=0).stop 2024-04-25T13:40:36.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c0cf750 msgr2=0x7fca2c0b6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c0cf750 0x7fca2c0b6070 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fca1c0ae040 tx=0x7fca1c019260 comp rx=0 tx=0).stop 2024-04-25T13:40:36.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 shutdown_connections 2024-04-25T13:40:36.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca1006f400 0x7fca10071910 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca2c0d0970 0x7fca2c1506d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca2c0cf750 0x7fca2c0b6070 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 --2- 172.21.15.80:0/4069156100 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca2c06e740 0x7fca2c0ab920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 >> 172.21.15.80:0/4069156100 conn(0x7fca2c0b3410 msgr2=0x7fca2c0b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 shutdown_connections 2024-04-25T13:40:36.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7fca32161640 1 -- 172.21.15.80:0/4069156100 wait complete. 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4edc06f400 msgr2=0x7f4edc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4edc06f400 0x7f4edc071910 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4001810 tx=0x7f4ee4009040 comp rx=0 tx=0).stop 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ef0057ff0 msgr2=0x7f4ef004f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ef0057ff0 0x7f4ef004f370 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0057350 tx=0x7f4eec00c820 comp rx=0 tx=0).stop 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 shutdown_connections 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4edc06f400 0x7f4edc071910 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ef004da20 0x7f4ef004ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ef00e3fc0 0x7f4ef004d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 --2- 172.21.15.80:0/2226392782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ef0057ff0 0x7f4ef004f370 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 >> 172.21.15.80:0/2226392782 conn(0x7f4ef003ac70 msgr2=0x7f4ef003d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 shutdown_connections 2024-04-25T13:40:36.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.724+0000 7f4efd1ce640 1 -- 172.21.15.80:0/2226392782 wait complete. 2024-04-25T13:40:36.740 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2024-04-25T13:40:36.740 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:36.746 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.5 2024-04-25T13:40:36.746 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:36.749 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2024-04-25T13:40:36.749 DEBUG:teuthology.parallel:result is None 2024-04-25T13:40:36.749 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T13:40:36.749 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:40:36.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbed0ed7640 1 Processor -- start 2024-04-25T13:40:36.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbed0ed7640 1 -- start start 2024-04-25T13:40:36.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbed0ed7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbecc161f90 0x7fbecc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbed0ed7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc0d0510 0x7fbecc15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbed0ed7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbecc0d0d70 con 0x7fbecc15ea70 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbed0ed7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbecc0d1050 con 0x7fbecc161f90 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbed0ed7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc06dca0 con 0x7fbecc0d0510 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbecad9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fbecc15ea70 0x7fbecc15ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35760/0 (socket says 172.21.15.80:35760) 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbecad9d640 1 -- 172.21.15.80:0/31163095 learned_addr learned my addr 172.21.15.80:0/31163095 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec9d9b640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc0d0510 0x7fbecc15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbeca59c640 1 --2- 172.21.15.80:0/31163095 >> v2:172.21.15.87:3300/0 conn(0x7fbecc161f90 0x7fbecc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1237170628 0 0) 0x7fbecc0d0d70 con 0x7fbecc15ea70 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbeb4003530 con 0x7fbecc15ea70 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2679037660 0 0) 0x7fbeb4003530 con 0x7fbecc15ea70 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc0d0510 msgr2=0x7fbecc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc0d0510 0x7fbecc15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 >> v2:172.21.15.87:3300/0 conn(0x7fbecc161f90 msgr2=0x7fbecc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 --2- 172.21.15.80:0/31163095 >> v2:172.21.15.87:3300/0 conn(0x7fbecc161f90 0x7fbecc0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc06df80 con 0x7fbecc15ea70 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fbec4001a00 con 0x7fbecc15ea70 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 >> v1:172.21.15.80:6789/0 conn(0x7fbecc15ea70 legacy=0x7fbecc15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc0d0510 0x7fbeb40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb4003f40 0x7fbeb4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb4004d50 0x7fbeb4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc0d1050 con 0x7fbeb4004d50 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc06dca0 con 0x7fbeb4003f40 2024-04-25T13:40:36.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbeb400bad0 con 0x7fbecc0d0510 2024-04-25T13:40:36.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbecad9d640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb4004d50 0x7fbeb4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbec9d9b640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb4003f40 0x7fbeb4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbeca59c640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc0d0510 0x7fbeb40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbecad9d640 1 -- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc0d0510 msgr2=0x7fbeb40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbecad9d640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc0d0510 0x7fbeb40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbecad9d640 1 -- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb4003f40 msgr2=0x7fbeb4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:36.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbecad9d640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbeb4003f40 0x7fbeb4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.920+0000 7fbecad9d640 1 -- 172.21.15.80:0/31163095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeb4003530 con 0x7fbeb4004d50 2024-04-25T13:40:36.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbecad9d640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb4004d50 0x7fbeb4005160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbecc071e30 tx=0x7fbec401faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4aa2478e9db8e8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec400a4a0 con 0x7fbeb4004d50 2024-04-25T13:40:36.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbec40aa6f0 con 0x7fbeb4004d50 2024-04-25T13:40:36.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbec959a640 1 -- 172.21.15.80:0/31163095 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec40aaae0 con 0x7fbeb4004d50 2024-04-25T13:40:36.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbed0ed7640 1 -- 172.21.15.80:0/31163095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb4004d50 msgr2=0x7fbeb4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbed0ed7640 1 --2- 172.21.15.80:0/31163095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbeb4004d50 0x7fbeb4005160 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbecc071e30 tx=0x7fbec401faa0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbecad9d640 1 -- 172.21.15.80:0/31163095 reap_dead start 2024-04-25T13:40:36.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbed0ed7640 1 -- 172.21.15.80:0/31163095 shutdown_connections 2024-04-25T13:40:36.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbed0ed7640 1 -- 172.21.15.80:0/31163095 >> 172.21.15.80:0/31163095 conn(0x7fbecc06d080 msgr2=0x7fbecc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:36.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbed0ed7640 1 -- 172.21.15.80:0/31163095 shutdown_connections 2024-04-25T13:40:36.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.924+0000 7fbed0ed7640 1 -- 172.21.15.80:0/31163095 wait complete. 2024-04-25T13:40:36.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 Processor -- start 2024-04-25T13:40:36.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 -- start start 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbecc073480 0x7fbecc073890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbecc0afa80 0x7fbecc0afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc144d10 0x7fbecc145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc162520 con 0x7fbecc073480 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc1623a0 con 0x7fbecc0afa80 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbecc144620 con 0x7fbecc144d10 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbecc073480 0x7fbecc073890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbecc073480 0x7fbecc073890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36414/0 (socket says 172.21.15.80:36414) 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 -- 172.21.15.80:0/1369281814 learned_addr learned my addr 172.21.15.80:0/1369281814 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbec9d9b640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbecc0afa80 0x7fbecc0afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbecad9d640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc144d10 0x7fbecc145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 -- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc144d10 msgr2=0x7fbecc145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc144d10 0x7fbecc145120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 -- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbecc0afa80 msgr2=0x7fbecc0afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbecc0afa80 0x7fbecc0afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 -- 172.21.15.80:0/1369281814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc072260 con 0x7fbecc073480 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbecad9d640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc144d10 0x7fbecc145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbeca59c640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbecc073480 0x7fbecc073890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0115e0 tx=0x7fbebc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbebaffd640 1 -- 172.21.15.80:0/1369281814 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc019070 con 0x7fbecc073480 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbebaffd640 1 -- 172.21.15.80:0/1369281814 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbebc004440 con 0x7fbecc073480 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc072540 con 0x7fbecc073480 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbebaffd640 1 -- 172.21.15.80:0/1369281814 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc003860 con 0x7fbecc073480 2024-04-25T13:40:36.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.928+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc072b00 con 0x7fbecc073480 2024-04-25T13:40:36.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.932+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc06a3a0 con 0x7fbecc073480 2024-04-25T13:40:36.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.932+0000 7fbebaffd640 1 -- 172.21.15.80:0/1369281814 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbebc01c070 con 0x7fbecc073480 2024-04-25T13:40:36.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.932+0000 7fbebaffd640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbeac06f400 0x7fbeac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:36.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.932+0000 7fbebaffd640 1 -- 172.21.15.80:0/1369281814 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbebc08f810 con 0x7fbecc073480 2024-04-25T13:40:36.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.932+0000 7fbec9d9b640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbeac06f400 0x7fbeac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:36.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.932+0000 7fbec9d9b640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbeac06f400 0x7fbeac071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbecc15a1e0 tx=0x7fbec0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:36.936+0000 7fbebaffd640 1 -- 172.21.15.80:0/1369281814 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbebc05c780 con 0x7fbecc073480 2024-04-25T13:40:37.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.080+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbecc071fa0 con 0x7fbeac06f400 2024-04-25T13:40:37.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.100+0000 7fbebaffd640 1 -- 172.21.15.80:0/1369281814 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247474 (secure 0 0 0) 0x7fbecc071fa0 con 0x7fbeac06f400 2024-04-25T13:40:37.101 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:37.103 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbeac06f400 msgr2=0x7fbeac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbeac06f400 0x7fbeac071910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbecc15a1e0 tx=0x7fbec0008040 comp rx=0 tx=0).stop 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbecc073480 msgr2=0x7fbecc073890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbecc073480 0x7fbecc073890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0115e0 tx=0x7fbebc011aa0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 shutdown_connections 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbeac06f400 0x7fbeac071910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbecc144d10 0x7fbecc145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbecc0afa80 0x7fbecc0afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 --2- 172.21.15.80:0/1369281814 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbecc073480 0x7fbecc073890 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbebc0115e0 tx=0x7fbebc011aa0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 >> 172.21.15.80:0/1369281814 conn(0x7fbecc06d080 msgr2=0x7fbecc0b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 shutdown_connections 2024-04-25T13:40:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.104+0000 7fbed0ed7640 1 -- 172.21.15.80:0/1369281814 wait complete. 2024-04-25T13:40:37.121 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T13:40:36.093458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.938610"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348886+0000","last_change":"2024-04-25T13:40:30.221428+0000","last_active":"2024-04-25T13:40:31.348886+0000","last_peered":"2024-04-25T13:40:31.348886+0000","last_clean":"2024-04-25T13:40:31.348886+0000","last_became_active":"2024-04-25T13:40:30.219271+0000","last_became_peered":"2024-04-25T13:40:30.219271+0000","last_unstale":"2024-04-25T13:40:31.348886+0000","last_undegraded":"2024-04-25T13:40:31.348886+0000","last_fullsized":"2024-04-25T13:40:31.348886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.180832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184646+0000","last_change":"2024-04-25T13:40:30.211459+0000","last_active":"2024-04-25T13:40:31.184646+0000","last_peered":"2024-04-25T13:40:31.184646+0000","last_clean":"2024-04-25T13:40:31.184646+0000","last_became_active":"2024-04-25T13:40:30.211073+0000","last_became_peered":"2024-04-25T13:40:30.211073+0000","last_unstale":"2024-04-25T13:40:31.184646+0000","last_undegraded":"2024-04-25T13:40:31.184646+0000","last_fullsized":"2024-04-25T13:40:31.184646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:13.973526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346644+0000","last_change":"2024-04-25T13:40:30.213483+0000","last_active":"2024-04-25T13:40:31.346644+0000","last_peered":"2024-04-25T13:40:31.346644+0000","last_clean":"2024-04-25T13:40:31.346644+0000","last_became_active":"2024-04-25T13:40:30.207846+0000","last_became_peered":"2024-04-25T13:40:30.207846+0000","last_unstale":"2024-04-25T13:40:31.346644+0000","last_undegraded":"2024-04-25T13:40:31.346644+0000","last_fullsized":"2024-04-25T13:40:31.346644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:10.391107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.345909+0000","last_change":"2024-04-25T13:40:30.216562+0000","last_active":"2024-04-25T13:40:31.345909+0000","last_peered":"2024-04-25T13:40:31.345909+0000","last_clean":"2024-04-25T13:40:31.345909+0000","last_became_active":"2024-04-25T13:40:30.215696+0000","last_became_peered":"2024-04-25T13:40:30.215696+0000","last_unstale":"2024-04-25T13:40:31.345909+0000","last_undegraded":"2024-04-25T13:40:31.345909+0000","last_fullsized":"2024-04-25T13:40:31.345909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:42.952152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347520+0000","last_change":"2024-04-25T13:40:30.207884+0000","last_active":"2024-04-25T13:40:31.347520+0000","last_peered":"2024-04-25T13:40:31.347520+0000","last_clean":"2024-04-25T13:40:31.347520+0000","last_became_active":"2024-04-25T13:40:30.207080+0000","last_became_peered":"2024-04-25T13:40:30.207080+0000","last_unstale":"2024-04-25T13:40:31.347520+0000","last_undegraded":"2024-04-25T13:40:31.347520+0000","last_fullsized":"2024-04-25T13:40:31.347520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:07.164518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347177+0000","last_change":"2024-04-25T13:40:30.221017+0000","last_active":"2024-04-25T13:40:31.347177+0000","last_peered":"2024-04-25T13:40:31.347177+0000","last_clean":"2024-04-25T13:40:31.347177+0000","last_became_active":"2024-04-25T13:40:30.219333+0000","last_became_peered":"2024-04-25T13:40:30.219333+0000","last_unstale":"2024-04-25T13:40:31.347177+0000","last_undegraded":"2024-04-25T13:40:31.347177+0000","last_fullsized":"2024-04-25T13:40:31.347177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:29.428442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349282+0000","last_change":"2024-04-25T13:40:30.216405+0000","last_active":"2024-04-25T13:40:31.349282+0000","last_peered":"2024-04-25T13:40:31.349282+0000","last_clean":"2024-04-25T13:40:31.349282+0000","last_became_active":"2024-04-25T13:40:30.215354+0000","last_became_peered":"2024-04-25T13:40:30.215354+0000","last_unstale":"2024-04-25T13:40:31.349282+0000","last_undegraded":"2024-04-25T13:40:31.349282+0000","last_fullsized":"2024-04-25T13:40:31.349282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:22.813674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346103+0000","last_change":"2024-04-25T13:40:30.207820+0000","last_active":"2024-04-25T13:40:31.346103+0000","last_peered":"2024-04-25T13:40:31.346103+0000","last_clean":"2024-04-25T13:40:31.346103+0000","last_became_active":"2024-04-25T13:40:30.207148+0000","last_became_peered":"2024-04-25T13:40:30.207148+0000","last_unstale":"2024-04-25T13:40:31.346103+0000","last_undegraded":"2024-04-25T13:40:31.346103+0000","last_fullsized":"2024-04-25T13:40:31.346103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:05.730270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346696+0000","last_change":"2024-04-25T13:40:30.219574+0000","last_active":"2024-04-25T13:40:31.346696+0000","last_peered":"2024-04-25T13:40:31.346696+0000","last_clean":"2024-04-25T13:40:31.346696+0000","last_became_active":"2024-04-25T13:40:30.216463+0000","last_became_peered":"2024-04-25T13:40:30.216463+0000","last_unstale":"2024-04-25T13:40:31.346696+0000","last_undegraded":"2024-04-25T13:40:31.346696+0000","last_fullsized":"2024-04-25T13:40:31.346696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:33.974714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219726+0000","last_change":"2024-04-25T13:40:30.202005+0000","last_active":"2024-04-25T13:40:31.219726+0000","last_peered":"2024-04-25T13:40:31.219726+0000","last_clean":"2024-04-25T13:40:31.219726+0000","last_became_active":"2024-04-25T13:40:30.201195+0000","last_became_peered":"2024-04-25T13:40:30.201195+0000","last_unstale":"2024-04-25T13:40:31.219726+0000","last_undegraded":"2024-04-25T13:40:31.219726+0000","last_fullsized":"2024-04-25T13:40:31.219726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:20.543637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219380+0000","last_change":"2024-04-25T13:40:30.221626+0000","last_active":"2024-04-25T13:40:31.219380+0000","last_peered":"2024-04-25T13:40:31.219380+0000","last_clean":"2024-04-25T13:40:31.219380+0000","last_became_active":"2024-04-25T13:40:30.221294+0000","last_became_peered":"2024-04-25T13:40:30.221294+0000","last_unstale":"2024-04-25T13:40:31.219380+0000","last_undegraded":"2024-04-25T13:40:31.219380+0000","last_fullsized":"2024-04-25T13:40:31.219380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:49.053408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348097+0000","last_change":"2024-04-25T13:40:30.199635+0000","last_active":"2024-04-25T13:40:31.348097+0000","last_peered":"2024-04-25T13:40:31.348097+0000","last_clean":"2024-04-25T13:40:31.348097+0000","last_became_active":"2024-04-25T13:40:30.198748+0000","last_became_peered":"2024-04-25T13:40:30.198748+0000","last_unstale":"2024-04-25T13:40:31.348097+0000","last_undegraded":"2024-04-25T13:40:31.348097+0000","last_fullsized":"2024-04-25T13:40:31.348097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346436+0000","last_change":"2024-04-25T13:40:30.215629+0000","last_active":"2024-04-25T13:40:31.346436+0000","last_peered":"2024-04-25T13:40:31.346436+0000","last_clean":"2024-04-25T13:40:31.346436+0000","last_became_active":"2024-04-25T13:40:30.214870+0000","last_became_peered":"2024-04-25T13:40:30.214870+0000","last_unstale":"2024-04-25T13:40:31.346436+0000","last_undegraded":"2024-04-25T13:40:31.346436+0000","last_fullsized":"2024-04-25T13:40:31.346436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:21.817644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346237+0000","last_change":"2024-04-25T13:40:30.200380+0000","last_active":"2024-04-25T13:40:31.346237+0000","last_peered":"2024-04-25T13:40:31.346237+0000","last_clean":"2024-04-25T13:40:31.346237+0000","last_became_active":"2024-04-25T13:40:30.198804+0000","last_became_peered":"2024-04-25T13:40:30.198804+0000","last_unstale":"2024-04-25T13:40:31.346237+0000","last_undegraded":"2024-04-25T13:40:31.346237+0000","last_fullsized":"2024-04-25T13:40:31.346237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:02.090517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185879+0000","last_change":"2024-04-25T13:40:30.228782+0000","last_active":"2024-04-25T13:40:31.185879+0000","last_peered":"2024-04-25T13:40:31.185879+0000","last_clean":"2024-04-25T13:40:31.185879+0000","last_became_active":"2024-04-25T13:40:30.228329+0000","last_became_peered":"2024-04-25T13:40:30.228329+0000","last_unstale":"2024-04-25T13:40:31.185879+0000","last_undegraded":"2024-04-25T13:40:31.185879+0000","last_fullsized":"2024-04-25T13:40:31.185879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:39.981195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346736+0000","last_change":"2024-04-25T13:40:30.221267+0000","last_active":"2024-04-25T13:40:31.346736+0000","last_peered":"2024-04-25T13:40:31.346736+0000","last_clean":"2024-04-25T13:40:31.346736+0000","last_became_active":"2024-04-25T13:40:30.219761+0000","last_became_peered":"2024-04-25T13:40:30.219761+0000","last_unstale":"2024-04-25T13:40:31.346736+0000","last_undegraded":"2024-04-25T13:40:31.346736+0000","last_fullsized":"2024-04-25T13:40:31.346736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:16.273351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346948+0000","last_change":"2024-04-25T13:40:30.225471+0000","last_active":"2024-04-25T13:40:31.346948+0000","last_peered":"2024-04-25T13:40:31.346948+0000","last_clean":"2024-04-25T13:40:31.346948+0000","last_became_active":"2024-04-25T13:40:30.224161+0000","last_became_peered":"2024-04-25T13:40:30.224161+0000","last_unstale":"2024-04-25T13:40:31.346948+0000","last_undegraded":"2024-04-25T13:40:31.346948+0000","last_fullsized":"2024-04-25T13:40:31.346948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:16.868301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186576+0000","last_change":"2024-04-25T13:40:30.204030+0000","last_active":"2024-04-25T13:40:31.186576+0000","last_peered":"2024-04-25T13:40:31.186576+0000","last_clean":"2024-04-25T13:40:31.186576+0000","last_became_active":"2024-04-25T13:40:30.203298+0000","last_became_peered":"2024-04-25T13:40:30.203298+0000","last_unstale":"2024-04-25T13:40:31.186576+0000","last_undegraded":"2024-04-25T13:40:31.186576+0000","last_fullsized":"2024-04-25T13:40:31.186576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.572150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347790+0000","last_change":"2024-04-25T13:40:30.220004+0000","last_active":"2024-04-25T13:40:31.347790+0000","last_peered":"2024-04-25T13:40:31.347790+0000","last_clean":"2024-04-25T13:40:31.347790+0000","last_became_active":"2024-04-25T13:40:30.217606+0000","last_became_peered":"2024-04-25T13:40:30.217606+0000","last_unstale":"2024-04-25T13:40:31.347790+0000","last_undegraded":"2024-04-25T13:40:31.347790+0000","last_fullsized":"2024-04-25T13:40:31.347790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:48.795741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220183+0000","last_change":"2024-04-25T13:40:30.228676+0000","last_active":"2024-04-25T13:40:31.220183+0000","last_peered":"2024-04-25T13:40:31.220183+0000","last_clean":"2024-04-25T13:40:31.220183+0000","last_became_active":"2024-04-25T13:40:30.228364+0000","last_became_peered":"2024-04-25T13:40:30.228364+0000","last_unstale":"2024-04-25T13:40:31.220183+0000","last_undegraded":"2024-04-25T13:40:31.220183+0000","last_fullsized":"2024-04-25T13:40:31.220183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:09.381688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347827+0000","last_change":"2024-04-25T13:40:30.220330+0000","last_active":"2024-04-25T13:40:31.347827+0000","last_peered":"2024-04-25T13:40:31.347827+0000","last_clean":"2024-04-25T13:40:31.347827+0000","last_became_active":"2024-04-25T13:40:30.219702+0000","last_became_peered":"2024-04-25T13:40:30.219702+0000","last_unstale":"2024-04-25T13:40:31.347827+0000","last_undegraded":"2024-04-25T13:40:31.347827+0000","last_fullsized":"2024-04-25T13:40:31.347827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:24.416759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349924+0000","last_change":"2024-04-25T13:40:30.224681+0000","last_active":"2024-04-25T13:40:31.349924+0000","last_peered":"2024-04-25T13:40:31.349924+0000","last_clean":"2024-04-25T13:40:31.349924+0000","last_became_active":"2024-04-25T13:40:30.218495+0000","last_became_peered":"2024-04-25T13:40:30.218495+0000","last_unstale":"2024-04-25T13:40:31.349924+0000","last_undegraded":"2024-04-25T13:40:31.349924+0000","last_fullsized":"2024-04-25T13:40:31.349924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:21.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347112+0000","last_change":"2024-04-25T13:40:30.220292+0000","last_active":"2024-04-25T13:40:31.347112+0000","last_peered":"2024-04-25T13:40:31.347112+0000","last_clean":"2024-04-25T13:40:31.347112+0000","last_became_active":"2024-04-25T13:40:30.219473+0000","last_became_peered":"2024-04-25T13:40:30.219473+0000","last_unstale":"2024-04-25T13:40:31.347112+0000","last_undegraded":"2024-04-25T13:40:31.347112+0000","last_fullsized":"2024-04-25T13:40:31.347112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.322464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346401+0000","last_change":"2024-04-25T13:40:30.216234+0000","last_active":"2024-04-25T13:40:31.346401+0000","last_peered":"2024-04-25T13:40:31.346401+0000","last_clean":"2024-04-25T13:40:31.346401+0000","last_became_active":"2024-04-25T13:40:30.215246+0000","last_became_peered":"2024-04-25T13:40:30.215246+0000","last_unstale":"2024-04-25T13:40:31.346401+0000","last_undegraded":"2024-04-25T13:40:31.346401+0000","last_fullsized":"2024-04-25T13:40:31.346401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:09.558215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347524+0000","last_change":"2024-04-25T13:40:30.226582+0000","last_active":"2024-04-25T13:40:31.347524+0000","last_peered":"2024-04-25T13:40:31.347524+0000","last_clean":"2024-04-25T13:40:31.347524+0000","last_became_active":"2024-04-25T13:40:30.226077+0000","last_became_peered":"2024-04-25T13:40:30.226077+0000","last_unstale":"2024-04-25T13:40:31.347524+0000","last_undegraded":"2024-04-25T13:40:31.347524+0000","last_fullsized":"2024-04-25T13:40:31.347524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.951509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219854+0000","last_change":"2024-04-25T13:40:30.220604+0000","last_active":"2024-04-25T13:40:31.219854+0000","last_peered":"2024-04-25T13:40:31.219854+0000","last_clean":"2024-04-25T13:40:31.219854+0000","last_became_active":"2024-04-25T13:40:30.220289+0000","last_became_peered":"2024-04-25T13:40:30.220289+0000","last_unstale":"2024-04-25T13:40:31.219854+0000","last_undegraded":"2024-04-25T13:40:31.219854+0000","last_fullsized":"2024-04-25T13:40:31.219854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:54.152591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184983+0000","last_change":"2024-04-25T13:40:30.204488+0000","last_active":"2024-04-25T13:40:31.184983+0000","last_peered":"2024-04-25T13:40:31.184983+0000","last_clean":"2024-04-25T13:40:31.184983+0000","last_became_active":"2024-04-25T13:40:30.203206+0000","last_became_peered":"2024-04-25T13:40:30.203206+0000","last_unstale":"2024-04-25T13:40:31.184983+0000","last_undegraded":"2024-04-25T13:40:31.184983+0000","last_fullsized":"2024-04-25T13:40:31.184983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:07.299339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346652+0000","last_change":"2024-04-25T13:40:30.219870+0000","last_active":"2024-04-25T13:40:31.346652+0000","last_peered":"2024-04-25T13:40:31.346652+0000","last_clean":"2024-04-25T13:40:31.346652+0000","last_became_active":"2024-04-25T13:40:30.218894+0000","last_became_peered":"2024-04-25T13:40:30.218894+0000","last_unstale":"2024-04-25T13:40:31.346652+0000","last_undegraded":"2024-04-25T13:40:31.346652+0000","last_fullsized":"2024-04-25T13:40:31.346652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:04.359710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346438+0000","last_change":"2024-04-25T13:40:30.212775+0000","last_active":"2024-04-25T13:40:31.346438+0000","last_peered":"2024-04-25T13:40:31.346438+0000","last_clean":"2024-04-25T13:40:31.346438+0000","last_became_active":"2024-04-25T13:40:30.212382+0000","last_became_peered":"2024-04-25T13:40:30.212382+0000","last_unstale":"2024-04-25T13:40:31.346438+0000","last_undegraded":"2024-04-25T13:40:31.346438+0000","last_fullsized":"2024-04-25T13:40:31.346438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.132012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347837+0000","last_change":"2024-04-25T13:40:30.213875+0000","last_active":"2024-04-25T13:40:31.347837+0000","last_peered":"2024-04-25T13:40:31.347837+0000","last_clean":"2024-04-25T13:40:31.347837+0000","last_became_active":"2024-04-25T13:40:30.213530+0000","last_became_peered":"2024-04-25T13:40:30.213530+0000","last_unstale":"2024-04-25T13:40:31.347837+0000","last_undegraded":"2024-04-25T13:40:31.347837+0000","last_fullsized":"2024-04-25T13:40:31.347837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:16.067584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186661+0000","last_change":"2024-04-25T13:40:30.221392+0000","last_active":"2024-04-25T13:40:31.186661+0000","last_peered":"2024-04-25T13:40:31.186661+0000","last_clean":"2024-04-25T13:40:31.186661+0000","last_became_active":"2024-04-25T13:40:30.221056+0000","last_became_peered":"2024-04-25T13:40:30.221056+0000","last_unstale":"2024-04-25T13:40:31.186661+0000","last_undegraded":"2024-04-25T13:40:31.186661+0000","last_fullsized":"2024-04-25T13:40:31.186661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:54.209154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219551+0000","last_change":"2024-04-25T13:40:30.211302+0000","last_active":"2024-04-25T13:40:31.219551+0000","last_peered":"2024-04-25T13:40:31.219551+0000","last_clean":"2024-04-25T13:40:31.219551+0000","last_became_active":"2024-04-25T13:40:30.210970+0000","last_became_peered":"2024-04-25T13:40:30.210970+0000","last_unstale":"2024-04-25T13:40:31.219551+0000","last_undegraded":"2024-04-25T13:40:31.219551+0000","last_fullsized":"2024-04-25T13:40:31.219551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:19.633269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349550+0000","last_change":"2024-04-25T13:40:30.201358+0000","last_active":"2024-04-25T13:40:31.349550+0000","last_peered":"2024-04-25T13:40:31.349550+0000","last_clean":"2024-04-25T13:40:31.349550+0000","last_became_active":"2024-04-25T13:40:30.200493+0000","last_became_peered":"2024-04-25T13:40:30.200493+0000","last_unstale":"2024-04-25T13:40:31.349550+0000","last_undegraded":"2024-04-25T13:40:31.349550+0000","last_fullsized":"2024-04-25T13:40:31.349550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:38.778259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347724+0000","last_change":"2024-04-25T13:40:29.193901+0000","last_active":"2024-04-25T13:40:31.347724+0000","last_peered":"2024-04-25T13:40:31.347724+0000","last_clean":"2024-04-25T13:40:31.347724+0000","last_became_active":"2024-04-25T13:40:29.193719+0000","last_became_peered":"2024-04-25T13:40:29.193719+0000","last_unstale":"2024-04-25T13:40:31.347724+0000","last_undegraded":"2024-04-25T13:40:31.347724+0000","last_fullsized":"2024-04-25T13:40:31.347724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:57.372209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347538+0000","last_change":"2024-04-25T13:40:30.186468+0000","last_active":"2024-04-25T13:40:31.347538+0000","last_peered":"2024-04-25T13:40:31.347538+0000","last_clean":"2024-04-25T13:40:31.347538+0000","last_became_active":"2024-04-25T13:40:30.185965+0000","last_became_peered":"2024-04-25T13:40:30.185965+0000","last_unstale":"2024-04-25T13:40:31.347538+0000","last_undegraded":"2024-04-25T13:40:31.347538+0000","last_fullsized":"2024-04-25T13:40:31.347538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:42.249844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220247+0000","last_change":"2024-04-25T13:40:29.185827+0000","last_active":"2024-04-25T13:40:31.220247+0000","last_peered":"2024-04-25T13:40:31.220247+0000","last_clean":"2024-04-25T13:40:31.220247+0000","last_became_active":"2024-04-25T13:40:29.185130+0000","last_became_peered":"2024-04-25T13:40:29.185130+0000","last_unstale":"2024-04-25T13:40:31.220247+0000","last_undegraded":"2024-04-25T13:40:31.220247+0000","last_fullsized":"2024-04-25T13:40:31.220247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:46.813189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347960+0000","last_change":"2024-04-25T13:40:29.175272+0000","last_active":"2024-04-25T13:40:31.347960+0000","last_peered":"2024-04-25T13:40:31.347960+0000","last_clean":"2024-04-25T13:40:31.347960+0000","last_became_active":"2024-04-25T13:40:29.174962+0000","last_became_peered":"2024-04-25T13:40:29.174962+0000","last_unstale":"2024-04-25T13:40:31.347960+0000","last_undegraded":"2024-04-25T13:40:31.347960+0000","last_fullsized":"2024-04-25T13:40:31.347960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:14.296130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347747+0000","last_change":"2024-04-25T13:40:30.214155+0000","last_active":"2024-04-25T13:40:31.347747+0000","last_peered":"2024-04-25T13:40:31.347747+0000","last_clean":"2024-04-25T13:40:31.347747+0000","last_became_active":"2024-04-25T13:40:30.212880+0000","last_became_peered":"2024-04-25T13:40:30.212880+0000","last_unstale":"2024-04-25T13:40:31.347747+0000","last_undegraded":"2024-04-25T13:40:31.347747+0000","last_fullsized":"2024-04-25T13:40:31.347747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:17.642730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349764+0000","last_change":"2024-04-25T13:40:29.185537+0000","last_active":"2024-04-25T13:40:31.349764+0000","last_peered":"2024-04-25T13:40:31.349764+0000","last_clean":"2024-04-25T13:40:31.349764+0000","last_became_active":"2024-04-25T13:40:29.182804+0000","last_became_peered":"2024-04-25T13:40:29.182804+0000","last_unstale":"2024-04-25T13:40:31.349764+0000","last_undegraded":"2024-04-25T13:40:31.349764+0000","last_fullsized":"2024-04-25T13:40:31.349764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:45.820954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184917+0000","last_change":"2024-04-25T13:40:30.217108+0000","last_active":"2024-04-25T13:40:31.184917+0000","last_peered":"2024-04-25T13:40:31.184917+0000","last_clean":"2024-04-25T13:40:31.184917+0000","last_became_active":"2024-04-25T13:40:30.216260+0000","last_became_peered":"2024-04-25T13:40:30.216260+0000","last_unstale":"2024-04-25T13:40:31.184917+0000","last_undegraded":"2024-04-25T13:40:31.184917+0000","last_fullsized":"2024-04-25T13:40:31.184917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:23.045755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220246+0000","last_change":"2024-04-25T13:40:29.184744+0000","last_active":"2024-04-25T13:40:31.220246+0000","last_peered":"2024-04-25T13:40:31.220246+0000","last_clean":"2024-04-25T13:40:31.220246+0000","last_became_active":"2024-04-25T13:40:29.183985+0000","last_became_peered":"2024-04-25T13:40:29.183985+0000","last_unstale":"2024-04-25T13:40:31.220246+0000","last_undegraded":"2024-04-25T13:40:31.220246+0000","last_fullsized":"2024-04-25T13:40:31.220246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:56.228309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347453+0000","last_change":"2024-04-25T13:40:30.225124+0000","last_active":"2024-04-25T13:40:31.347453+0000","last_peered":"2024-04-25T13:40:31.347453+0000","last_clean":"2024-04-25T13:40:31.347453+0000","last_became_active":"2024-04-25T13:40:30.223455+0000","last_became_peered":"2024-04-25T13:40:30.223455+0000","last_unstale":"2024-04-25T13:40:31.347453+0000","last_undegraded":"2024-04-25T13:40:31.347453+0000","last_fullsized":"2024-04-25T13:40:31.347453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:03.877694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350848+0000","last_change":"2024-04-25T13:40:29.190850+0000","last_active":"2024-04-25T13:40:31.350848+0000","last_peered":"2024-04-25T13:40:31.350848+0000","last_clean":"2024-04-25T13:40:31.350848+0000","last_became_active":"2024-04-25T13:40:29.188098+0000","last_became_peered":"2024-04-25T13:40:29.188098+0000","last_unstale":"2024-04-25T13:40:31.350848+0000","last_undegraded":"2024-04-25T13:40:31.350848+0000","last_fullsized":"2024-04-25T13:40:31.350848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:07.915234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347972+0000","last_change":"2024-04-25T13:40:30.221083+0000","last_active":"2024-04-25T13:40:31.347972+0000","last_peered":"2024-04-25T13:40:31.347972+0000","last_clean":"2024-04-25T13:40:31.347972+0000","last_became_active":"2024-04-25T13:40:30.220819+0000","last_became_peered":"2024-04-25T13:40:30.220819+0000","last_unstale":"2024-04-25T13:40:31.347972+0000","last_undegraded":"2024-04-25T13:40:31.347972+0000","last_fullsized":"2024-04-25T13:40:31.347972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:43.306079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185190+0000","last_change":"2024-04-25T13:40:29.189173+0000","last_active":"2024-04-25T13:40:31.185190+0000","last_peered":"2024-04-25T13:40:31.185190+0000","last_clean":"2024-04-25T13:40:31.185190+0000","last_became_active":"2024-04-25T13:40:29.188914+0000","last_became_peered":"2024-04-25T13:40:29.188914+0000","last_unstale":"2024-04-25T13:40:31.185190+0000","last_undegraded":"2024-04-25T13:40:31.185190+0000","last_fullsized":"2024-04-25T13:40:31.185190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:12.374973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346508+0000","last_change":"2024-04-25T13:40:30.218953+0000","last_active":"2024-04-25T13:40:31.346508+0000","last_peered":"2024-04-25T13:40:31.346508+0000","last_clean":"2024-04-25T13:40:31.346508+0000","last_became_active":"2024-04-25T13:40:30.218568+0000","last_became_peered":"2024-04-25T13:40:30.218568+0000","last_unstale":"2024-04-25T13:40:31.346508+0000","last_undegraded":"2024-04-25T13:40:31.346508+0000","last_fullsized":"2024-04-25T13:40:31.346508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:02.608556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219913+0000","last_change":"2024-04-25T13:40:29.175817+0000","last_active":"2024-04-25T13:40:31.219913+0000","last_peered":"2024-04-25T13:40:31.219913+0000","last_clean":"2024-04-25T13:40:31.219913+0000","last_became_active":"2024-04-25T13:40:29.175066+0000","last_became_peered":"2024-04-25T13:40:29.175066+0000","last_unstale":"2024-04-25T13:40:31.219913+0000","last_undegraded":"2024-04-25T13:40:31.219913+0000","last_fullsized":"2024-04-25T13:40:31.219913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:27.404657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348852+0000","last_change":"2024-04-25T13:40:30.215383+0000","last_active":"2024-04-25T13:40:31.348852+0000","last_peered":"2024-04-25T13:40:31.348852+0000","last_clean":"2024-04-25T13:40:31.348852+0000","last_became_active":"2024-04-25T13:40:30.214994+0000","last_became_peered":"2024-04-25T13:40:30.214994+0000","last_unstale":"2024-04-25T13:40:31.348852+0000","last_undegraded":"2024-04-25T13:40:31.348852+0000","last_fullsized":"2024-04-25T13:40:31.348852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:49.832720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347401+0000","last_change":"2024-04-25T13:40:29.185820+0000","last_active":"2024-04-25T13:40:31.347401+0000","last_peered":"2024-04-25T13:40:31.347401+0000","last_clean":"2024-04-25T13:40:31.347401+0000","last_became_active":"2024-04-25T13:40:29.185576+0000","last_became_peered":"2024-04-25T13:40:29.185576+0000","last_unstale":"2024-04-25T13:40:31.347401+0000","last_undegraded":"2024-04-25T13:40:31.347401+0000","last_fullsized":"2024-04-25T13:40:31.347401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:18.356259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348088+0000","last_change":"2024-04-25T13:40:30.220046+0000","last_active":"2024-04-25T13:40:31.348088+0000","last_peered":"2024-04-25T13:40:31.348088+0000","last_clean":"2024-04-25T13:40:31.348088+0000","last_became_active":"2024-04-25T13:40:30.219244+0000","last_became_peered":"2024-04-25T13:40:30.219244+0000","last_unstale":"2024-04-25T13:40:31.348088+0000","last_undegraded":"2024-04-25T13:40:31.348088+0000","last_fullsized":"2024-04-25T13:40:31.348088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:34.257699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350082+0000","last_change":"2024-04-25T13:40:29.187441+0000","last_active":"2024-04-25T13:40:31.350082+0000","last_peered":"2024-04-25T13:40:31.350082+0000","last_clean":"2024-04-25T13:40:31.350082+0000","last_became_active":"2024-04-25T13:40:29.181693+0000","last_became_peered":"2024-04-25T13:40:29.181693+0000","last_unstale":"2024-04-25T13:40:31.350082+0000","last_undegraded":"2024-04-25T13:40:31.350082+0000","last_fullsized":"2024-04-25T13:40:31.350082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:57.361661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347958+0000","last_change":"2024-04-25T13:40:30.229833+0000","last_active":"2024-04-25T13:40:31.347958+0000","last_peered":"2024-04-25T13:40:31.347958+0000","last_clean":"2024-04-25T13:40:31.347958+0000","last_became_active":"2024-04-25T13:40:30.229011+0000","last_became_peered":"2024-04-25T13:40:30.229011+0000","last_unstale":"2024-04-25T13:40:31.347958+0000","last_undegraded":"2024-04-25T13:40:31.347958+0000","last_fullsized":"2024-04-25T13:40:31.347958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:15.058616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219728+0000","last_change":"2024-04-25T13:40:29.188978+0000","last_active":"2024-04-25T13:40:31.219728+0000","last_peered":"2024-04-25T13:40:31.219728+0000","last_clean":"2024-04-25T13:40:31.219728+0000","last_became_active":"2024-04-25T13:40:29.185846+0000","last_became_peered":"2024-04-25T13:40:29.185846+0000","last_unstale":"2024-04-25T13:40:31.219728+0000","last_undegraded":"2024-04-25T13:40:31.219728+0000","last_fullsized":"2024-04-25T13:40:31.219728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:08.912260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219935+0000","last_change":"2024-04-25T13:40:30.215550+0000","last_active":"2024-04-25T13:40:31.219935+0000","last_peered":"2024-04-25T13:40:31.219935+0000","last_clean":"2024-04-25T13:40:31.219935+0000","last_became_active":"2024-04-25T13:40:30.215250+0000","last_became_peered":"2024-04-25T13:40:30.215250+0000","last_unstale":"2024-04-25T13:40:31.219935+0000","last_undegraded":"2024-04-25T13:40:31.219935+0000","last_fullsized":"2024-04-25T13:40:31.219935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.530072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186915+0000","last_change":"2024-04-25T13:40:29.168980+0000","last_active":"2024-04-25T13:40:31.186915+0000","last_peered":"2024-04-25T13:40:31.186915+0000","last_clean":"2024-04-25T13:40:31.186915+0000","last_became_active":"2024-04-25T13:40:29.168218+0000","last_became_peered":"2024-04-25T13:40:29.168218+0000","last_unstale":"2024-04-25T13:40:31.186915+0000","last_undegraded":"2024-04-25T13:40:31.186915+0000","last_fullsized":"2024-04-25T13:40:31.186915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:25.060887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349934+0000","last_change":"2024-04-25T13:40:30.216707+0000","last_active":"2024-04-25T13:40:31.349934+0000","last_peered":"2024-04-25T13:40:31.349934+0000","last_clean":"2024-04-25T13:40:31.349934+0000","last_became_active":"2024-04-25T13:40:30.215907+0000","last_became_peered":"2024-04-25T13:40:30.215907+0000","last_unstale":"2024-04-25T13:40:31.349934+0000","last_undegraded":"2024-04-25T13:40:31.349934+0000","last_fullsized":"2024-04-25T13:40:31.349934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:22.984851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346871+0000","last_change":"2024-04-25T13:40:29.182090+0000","last_active":"2024-04-25T13:40:31.346871+0000","last_peered":"2024-04-25T13:40:31.346871+0000","last_clean":"2024-04-25T13:40:31.346871+0000","last_became_active":"2024-04-25T13:40:29.181228+0000","last_became_peered":"2024-04-25T13:40:29.181228+0000","last_unstale":"2024-04-25T13:40:31.346871+0000","last_undegraded":"2024-04-25T13:40:31.346871+0000","last_fullsized":"2024-04-25T13:40:31.346871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:49.643409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348554+0000","last_change":"2024-04-25T13:40:30.230355+0000","last_active":"2024-04-25T13:40:31.348554+0000","last_peered":"2024-04-25T13:40:31.348554+0000","last_clean":"2024-04-25T13:40:31.348554+0000","last_became_active":"2024-04-25T13:40:30.229397+0000","last_became_peered":"2024-04-25T13:40:30.229397+0000","last_unstale":"2024-04-25T13:40:31.348554+0000","last_undegraded":"2024-04-25T13:40:31.348554+0000","last_fullsized":"2024-04-25T13:40:31.348554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:23.803621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.187149+0000","last_change":"2024-04-25T13:40:29.160176+0000","last_active":"2024-04-25T13:40:31.187149+0000","last_peered":"2024-04-25T13:40:31.187149+0000","last_clean":"2024-04-25T13:40:31.187149+0000","last_became_active":"2024-04-25T13:40:29.159439+0000","last_became_peered":"2024-04-25T13:40:29.159439+0000","last_unstale":"2024-04-25T13:40:31.187149+0000","last_undegraded":"2024-04-25T13:40:31.187149+0000","last_fullsized":"2024-04-25T13:40:31.187149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:48.998902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347297+0000","last_change":"2024-04-25T13:40:30.224277+0000","last_active":"2024-04-25T13:40:31.347297+0000","last_peered":"2024-04-25T13:40:31.347297+0000","last_clean":"2024-04-25T13:40:31.347297+0000","last_became_active":"2024-04-25T13:40:30.221764+0000","last_became_peered":"2024-04-25T13:40:30.221764+0000","last_unstale":"2024-04-25T13:40:31.347297+0000","last_undegraded":"2024-04-25T13:40:31.347297+0000","last_fullsized":"2024-04-25T13:40:31.347297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:12.238288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185510+0000","last_change":"2024-04-25T13:40:29.182136+0000","last_active":"2024-04-25T13:40:31.185510+0000","last_peered":"2024-04-25T13:40:31.185510+0000","last_clean":"2024-04-25T13:40:31.185510+0000","last_became_active":"2024-04-25T13:40:29.181852+0000","last_became_peered":"2024-04-25T13:40:29.181852+0000","last_unstale":"2024-04-25T13:40:31.185510+0000","last_undegraded":"2024-04-25T13:40:31.185510+0000","last_fullsized":"2024-04-25T13:40:31.185510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:24.681218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350369+0000","last_change":"2024-04-25T13:40:30.217315+0000","last_active":"2024-04-25T13:40:31.350369+0000","last_peered":"2024-04-25T13:40:31.350369+0000","last_clean":"2024-04-25T13:40:31.350369+0000","last_became_active":"2024-04-25T13:40:30.214308+0000","last_became_peered":"2024-04-25T13:40:30.214308+0000","last_unstale":"2024-04-25T13:40:31.350369+0000","last_undegraded":"2024-04-25T13:40:31.350369+0000","last_fullsized":"2024-04-25T13:40:31.350369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:57.857844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186012+0000","last_change":"2024-04-25T13:40:29.189228+0000","last_active":"2024-04-25T13:40:31.186012+0000","last_peered":"2024-04-25T13:40:31.186012+0000","last_clean":"2024-04-25T13:40:31.186012+0000","last_became_active":"2024-04-25T13:40:29.189004+0000","last_became_peered":"2024-04-25T13:40:29.189004+0000","last_unstale":"2024-04-25T13:40:31.186012+0000","last_undegraded":"2024-04-25T13:40:31.186012+0000","last_fullsized":"2024-04-25T13:40:31.186012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:14.763408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349561+0000","last_change":"2024-04-25T13:40:30.223154+0000","last_active":"2024-04-25T13:40:31.349561+0000","last_peered":"2024-04-25T13:40:31.349561+0000","last_clean":"2024-04-25T13:40:31.349561+0000","last_became_active":"2024-04-25T13:40:30.222384+0000","last_became_peered":"2024-04-25T13:40:30.222384+0000","last_unstale":"2024-04-25T13:40:31.349561+0000","last_undegraded":"2024-04-25T13:40:31.349561+0000","last_fullsized":"2024-04-25T13:40:31.349561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:59.286900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350710+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:40:31.350710+0000","last_peered":"2024-04-25T13:40:31.350710+0000","last_clean":"2024-04-25T13:40:31.350710+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:40:31.350710+0000","last_undegraded":"2024-04-25T13:40:31.350710+0000","last_fullsized":"2024-04-25T13:40:31.350710+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350590+0000","last_change":"2024-04-25T13:40:29.186348+0000","last_active":"2024-04-25T13:40:31.350590+0000","last_peered":"2024-04-25T13:40:31.350590+0000","last_clean":"2024-04-25T13:40:31.350590+0000","last_became_active":"2024-04-25T13:40:29.180642+0000","last_became_peered":"2024-04-25T13:40:29.180642+0000","last_unstale":"2024-04-25T13:40:31.350590+0000","last_undegraded":"2024-04-25T13:40:31.350590+0000","last_fullsized":"2024-04-25T13:40:31.350590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:43.636874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347213+0000","last_change":"2024-04-25T13:40:30.209809+0000","last_active":"2024-04-25T13:40:31.347213+0000","last_peered":"2024-04-25T13:40:31.347213+0000","last_clean":"2024-04-25T13:40:31.347213+0000","last_became_active":"2024-04-25T13:40:30.209014+0000","last_became_peered":"2024-04-25T13:40:30.209014+0000","last_unstale":"2024-04-25T13:40:31.347213+0000","last_undegraded":"2024-04-25T13:40:31.347213+0000","last_fullsized":"2024-04-25T13:40:31.347213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:24.945222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350549+0000","last_change":"2024-04-25T13:40:29.197074+0000","last_active":"2024-04-25T13:40:31.350549+0000","last_peered":"2024-04-25T13:40:31.350549+0000","last_clean":"2024-04-25T13:40:31.350549+0000","last_became_active":"2024-04-25T13:40:29.193688+0000","last_became_peered":"2024-04-25T13:40:29.193688+0000","last_unstale":"2024-04-25T13:40:31.350549+0000","last_undegraded":"2024-04-25T13:40:31.350549+0000","last_fullsized":"2024-04-25T13:40:31.350549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:03.325908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347895+0000","last_change":"2024-04-25T13:40:30.187471+0000","last_active":"2024-04-25T13:40:31.347895+0000","last_peered":"2024-04-25T13:40:31.347895+0000","last_clean":"2024-04-25T13:40:31.347895+0000","last_became_active":"2024-04-25T13:40:30.186993+0000","last_became_peered":"2024-04-25T13:40:30.186993+0000","last_unstale":"2024-04-25T13:40:31.347895+0000","last_undegraded":"2024-04-25T13:40:31.347895+0000","last_fullsized":"2024-04-25T13:40:31.347895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.749355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.187216+0000","last_change":"2024-04-25T13:40:29.159566+0000","last_active":"2024-04-25T13:40:31.187216+0000","last_peered":"2024-04-25T13:40:31.187216+0000","last_clean":"2024-04-25T13:40:31.187216+0000","last_became_active":"2024-04-25T13:40:29.159054+0000","last_became_peered":"2024-04-25T13:40:29.159054+0000","last_unstale":"2024-04-25T13:40:31.187216+0000","last_undegraded":"2024-04-25T13:40:31.187216+0000","last_fullsized":"2024-04-25T13:40:31.187216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:03.455177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185373+0000","last_change":"2024-04-25T13:40:30.203352+0000","last_active":"2024-04-25T13:40:31.185373+0000","last_peered":"2024-04-25T13:40:31.185373+0000","last_clean":"2024-04-25T13:40:31.185373+0000","last_became_active":"2024-04-25T13:40:30.202463+0000","last_became_peered":"2024-04-25T13:40:30.202463+0000","last_unstale":"2024-04-25T13:40:31.185373+0000","last_undegraded":"2024-04-25T13:40:31.185373+0000","last_fullsized":"2024-04-25T13:40:31.185373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:01.912451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350985+0000","last_change":"2024-04-25T13:40:29.190061+0000","last_active":"2024-04-25T13:40:31.350985+0000","last_peered":"2024-04-25T13:40:31.350985+0000","last_clean":"2024-04-25T13:40:31.350985+0000","last_became_active":"2024-04-25T13:40:29.182897+0000","last_became_peered":"2024-04-25T13:40:29.182897+0000","last_unstale":"2024-04-25T13:40:31.350985+0000","last_undegraded":"2024-04-25T13:40:31.350985+0000","last_fullsized":"2024-04-25T13:40:31.350985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:31.168555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346748+0000","last_change":"2024-04-25T13:40:30.187160+0000","last_active":"2024-04-25T13:40:31.346748+0000","last_peered":"2024-04-25T13:40:31.346748+0000","last_clean":"2024-04-25T13:40:31.346748+0000","last_became_active":"2024-04-25T13:40:30.186695+0000","last_became_peered":"2024-04-25T13:40:30.186695+0000","last_unstale":"2024-04-25T13:40:31.346748+0000","last_undegraded":"2024-04-25T13:40:31.346748+0000","last_fullsized":"2024-04-25T13:40:31.346748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:57.859568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186430+0000","last_change":"2024-04-25T13:40:29.168378+0000","last_active":"2024-04-25T13:40:31.186430+0000","last_peered":"2024-04-25T13:40:31.186430+0000","last_clean":"2024-04-25T13:40:31.186430+0000","last_became_active":"2024-04-25T13:40:29.167515+0000","last_became_peered":"2024-04-25T13:40:29.167515+0000","last_unstale":"2024-04-25T13:40:31.186430+0000","last_undegraded":"2024-04-25T13:40:31.186430+0000","last_fullsized":"2024-04-25T13:40:31.186430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.417158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346850+0000","last_change":"2024-04-25T13:40:30.214354+0000","last_active":"2024-04-25T13:40:31.346850+0000","last_peered":"2024-04-25T13:40:31.346850+0000","last_clean":"2024-04-25T13:40:31.346850+0000","last_became_active":"2024-04-25T13:40:30.213970+0000","last_became_peered":"2024-04-25T13:40:30.213970+0000","last_unstale":"2024-04-25T13:40:31.346850+0000","last_undegraded":"2024-04-25T13:40:31.346850+0000","last_fullsized":"2024-04-25T13:40:31.346850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:54.278471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348113+0000","last_change":"2024-04-25T13:40:29.197843+0000","last_active":"2024-04-25T13:40:31.348113+0000","last_peered":"2024-04-25T13:40:31.348113+0000","last_clean":"2024-04-25T13:40:31.348113+0000","last_became_active":"2024-04-25T13:40:29.196299+0000","last_became_peered":"2024-04-25T13:40:29.196299+0000","last_unstale":"2024-04-25T13:40:31.348113+0000","last_undegraded":"2024-04-25T13:40:31.348113+0000","last_fullsized":"2024-04-25T13:40:31.348113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:41.785353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185424+0000","last_change":"2024-04-25T13:40:30.212599+0000","last_active":"2024-04-25T13:40:31.185424+0000","last_peered":"2024-04-25T13:40:31.185424+0000","last_clean":"2024-04-25T13:40:31.185424+0000","last_became_active":"2024-04-25T13:40:30.211967+0000","last_became_peered":"2024-04-25T13:40:30.211967+0000","last_unstale":"2024-04-25T13:40:31.185424+0000","last_undegraded":"2024-04-25T13:40:31.185424+0000","last_fullsized":"2024-04-25T13:40:31.185424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:28.946427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219865+0000","last_change":"2024-04-25T13:40:29.175520+0000","last_active":"2024-04-25T13:40:31.219865+0000","last_peered":"2024-04-25T13:40:31.219865+0000","last_clean":"2024-04-25T13:40:31.219865+0000","last_became_active":"2024-04-25T13:40:29.174561+0000","last_became_peered":"2024-04-25T13:40:29.174561+0000","last_unstale":"2024-04-25T13:40:31.219865+0000","last_undegraded":"2024-04-25T13:40:31.219865+0000","last_fullsized":"2024-04-25T13:40:31.219865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:55.788741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346789+0000","last_change":"2024-04-25T13:40:30.217175+0000","last_active":"2024-04-25T13:40:31.346789+0000","last_peered":"2024-04-25T13:40:31.346789+0000","last_clean":"2024-04-25T13:40:31.346789+0000","last_became_active":"2024-04-25T13:40:30.216851+0000","last_became_peered":"2024-04-25T13:40:30.216851+0000","last_unstale":"2024-04-25T13:40:31.346789+0000","last_undegraded":"2024-04-25T13:40:31.346789+0000","last_fullsized":"2024-04-25T13:40:31.346789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:12.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219908+0000","last_change":"2024-04-25T13:40:29.194455+0000","last_active":"2024-04-25T13:40:31.219908+0000","last_peered":"2024-04-25T13:40:31.219908+0000","last_clean":"2024-04-25T13:40:31.219908+0000","last_became_active":"2024-04-25T13:40:29.193686+0000","last_became_peered":"2024-04-25T13:40:29.193686+0000","last_unstale":"2024-04-25T13:40:31.219908+0000","last_undegraded":"2024-04-25T13:40:31.219908+0000","last_fullsized":"2024-04-25T13:40:31.219908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:27.100097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347083+0000","last_change":"2024-04-25T13:40:30.211947+0000","last_active":"2024-04-25T13:40:31.347083+0000","last_peered":"2024-04-25T13:40:31.347083+0000","last_clean":"2024-04-25T13:40:31.347083+0000","last_became_active":"2024-04-25T13:40:30.202151+0000","last_became_peered":"2024-04-25T13:40:30.202151+0000","last_unstale":"2024-04-25T13:40:31.347083+0000","last_undegraded":"2024-04-25T13:40:31.347083+0000","last_fullsized":"2024-04-25T13:40:31.347083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.174571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350372+0000","last_change":"2024-04-25T13:40:29.200572+0000","last_active":"2024-04-25T13:40:31.350372+0000","last_peered":"2024-04-25T13:40:31.350372+0000","last_clean":"2024-04-25T13:40:31.350372+0000","last_became_active":"2024-04-25T13:40:29.199836+0000","last_became_peered":"2024-04-25T13:40:29.199836+0000","last_unstale":"2024-04-25T13:40:31.350372+0000","last_undegraded":"2024-04-25T13:40:31.350372+0000","last_fullsized":"2024-04-25T13:40:31.350372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:38.068886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349670+0000","last_change":"2024-04-25T13:40:30.225985+0000","last_active":"2024-04-25T13:40:31.349670+0000","last_peered":"2024-04-25T13:40:31.349670+0000","last_clean":"2024-04-25T13:40:31.349670+0000","last_became_active":"2024-04-25T13:40:30.224959+0000","last_became_peered":"2024-04-25T13:40:30.224959+0000","last_unstale":"2024-04-25T13:40:31.349670+0000","last_undegraded":"2024-04-25T13:40:31.349670+0000","last_fullsized":"2024-04-25T13:40:31.349670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:13.498711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186159+0000","last_change":"2024-04-25T13:40:29.196670+0000","last_active":"2024-04-25T13:40:31.186159+0000","last_peered":"2024-04-25T13:40:31.186159+0000","last_clean":"2024-04-25T13:40:31.186159+0000","last_became_active":"2024-04-25T13:40:29.196115+0000","last_became_peered":"2024-04-25T13:40:29.196115+0000","last_unstale":"2024-04-25T13:40:31.186159+0000","last_undegraded":"2024-04-25T13:40:31.186159+0000","last_fullsized":"2024-04-25T13:40:31.186159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:35.705063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347672+0000","last_change":"2024-04-25T13:40:30.203777+0000","last_active":"2024-04-25T13:40:31.347672+0000","last_peered":"2024-04-25T13:40:31.347672+0000","last_clean":"2024-04-25T13:40:31.347672+0000","last_became_active":"2024-04-25T13:40:30.202751+0000","last_became_peered":"2024-04-25T13:40:30.202751+0000","last_unstale":"2024-04-25T13:40:31.347672+0000","last_undegraded":"2024-04-25T13:40:31.347672+0000","last_fullsized":"2024-04-25T13:40:31.347672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:09.528068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185653+0000","last_change":"2024-04-25T13:40:29.176333+0000","last_active":"2024-04-25T13:40:31.185653+0000","last_peered":"2024-04-25T13:40:31.185653+0000","last_clean":"2024-04-25T13:40:31.185653+0000","last_became_active":"2024-04-25T13:40:29.176078+0000","last_became_peered":"2024-04-25T13:40:29.176078+0000","last_unstale":"2024-04-25T13:40:31.185653+0000","last_undegraded":"2024-04-25T13:40:31.185653+0000","last_fullsized":"2024-04-25T13:40:31.185653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:19.200730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346905+0000","last_change":"2024-04-25T13:40:30.216029+0000","last_active":"2024-04-25T13:40:31.346905+0000","last_peered":"2024-04-25T13:40:31.346905+0000","last_clean":"2024-04-25T13:40:31.346905+0000","last_became_active":"2024-04-25T13:40:30.215353+0000","last_became_peered":"2024-04-25T13:40:30.215353+0000","last_unstale":"2024-04-25T13:40:31.346905+0000","last_undegraded":"2024-04-25T13:40:31.346905+0000","last_fullsized":"2024-04-25T13:40:31.346905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:10.825968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349163+0000","last_change":"2024-04-25T13:40:29.195078+0000","last_active":"2024-04-25T13:40:31.349163+0000","last_peered":"2024-04-25T13:40:31.349163+0000","last_clean":"2024-04-25T13:40:31.349163+0000","last_became_active":"2024-04-25T13:40:29.193847+0000","last_became_peered":"2024-04-25T13:40:29.193847+0000","last_unstale":"2024-04-25T13:40:31.349163+0000","last_undegraded":"2024-04-25T13:40:31.349163+0000","last_fullsized":"2024-04-25T13:40:31.349163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:33.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219558+0000","last_change":"2024-04-25T13:40:30.205595+0000","last_active":"2024-04-25T13:40:31.219558+0000","last_peered":"2024-04-25T13:40:31.219558+0000","last_clean":"2024-04-25T13:40:31.219558+0000","last_became_active":"2024-04-25T13:40:30.204793+0000","last_became_peered":"2024-04-25T13:40:30.204793+0000","last_unstale":"2024-04-25T13:40:31.219558+0000","last_undegraded":"2024-04-25T13:40:31.219558+0000","last_fullsized":"2024-04-25T13:40:31.219558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:55.486355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349463+0000","last_change":"2024-04-25T13:40:30.202452+0000","last_active":"2024-04-25T13:40:31.349463+0000","last_peered":"2024-04-25T13:40:31.349463+0000","last_clean":"2024-04-25T13:40:31.349463+0000","last_became_active":"2024-04-25T13:40:30.201683+0000","last_became_peered":"2024-04-25T13:40:30.201683+0000","last_unstale":"2024-04-25T13:40:31.349463+0000","last_undegraded":"2024-04-25T13:40:31.349463+0000","last_fullsized":"2024-04-25T13:40:31.349463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.132452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350105+0000","last_change":"2024-04-25T13:40:29.199128+0000","last_active":"2024-04-25T13:40:31.350105+0000","last_peered":"2024-04-25T13:40:31.350105+0000","last_clean":"2024-04-25T13:40:31.350105+0000","last_became_active":"2024-04-25T13:40:29.198449+0000","last_became_peered":"2024-04-25T13:40:29.198449+0000","last_unstale":"2024-04-25T13:40:31.350105+0000","last_undegraded":"2024-04-25T13:40:31.350105+0000","last_fullsized":"2024-04-25T13:40:31.350105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:08.596939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347469+0000","last_change":"2024-04-25T13:40:30.220740+0000","last_active":"2024-04-25T13:40:31.347469+0000","last_peered":"2024-04-25T13:40:31.347469+0000","last_clean":"2024-04-25T13:40:31.347469+0000","last_became_active":"2024-04-25T13:40:30.218927+0000","last_became_peered":"2024-04-25T13:40:30.218927+0000","last_unstale":"2024-04-25T13:40:31.347469+0000","last_undegraded":"2024-04-25T13:40:31.347469+0000","last_fullsized":"2024-04-25T13:40:31.347469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.133104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349098+0000","last_change":"2024-04-25T13:40:29.199401+0000","last_active":"2024-04-25T13:40:31.349098+0000","last_peered":"2024-04-25T13:40:31.349098+0000","last_clean":"2024-04-25T13:40:31.349098+0000","last_became_active":"2024-04-25T13:40:29.198182+0000","last_became_peered":"2024-04-25T13:40:29.198182+0000","last_unstale":"2024-04-25T13:40:31.349098+0000","last_undegraded":"2024-04-25T13:40:31.349098+0000","last_fullsized":"2024-04-25T13:40:31.349098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:31.807935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347497+0000","last_change":"2024-04-25T13:40:30.213026+0000","last_active":"2024-04-25T13:40:31.347497+0000","last_peered":"2024-04-25T13:40:31.347497+0000","last_clean":"2024-04-25T13:40:31.347497+0000","last_became_active":"2024-04-25T13:40:30.203212+0000","last_became_peered":"2024-04-25T13:40:30.203212+0000","last_unstale":"2024-04-25T13:40:31.347497+0000","last_undegraded":"2024-04-25T13:40:31.347497+0000","last_fullsized":"2024-04-25T13:40:31.347497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:24.180245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347687+0000","last_change":"2024-04-25T13:40:29.173054+0000","last_active":"2024-04-25T13:40:31.347687+0000","last_peered":"2024-04-25T13:40:31.347687+0000","last_clean":"2024-04-25T13:40:31.347687+0000","last_became_active":"2024-04-25T13:40:29.172375+0000","last_became_peered":"2024-04-25T13:40:29.172375+0000","last_unstale":"2024-04-25T13:40:31.347687+0000","last_undegraded":"2024-04-25T13:40:31.347687+0000","last_fullsized":"2024-04-25T13:40:31.347687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:26.996154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347229+0000","last_change":"2024-04-25T13:40:30.214823+0000","last_active":"2024-04-25T13:40:31.347229+0000","last_peered":"2024-04-25T13:40:31.347229+0000","last_clean":"2024-04-25T13:40:31.347229+0000","last_became_active":"2024-04-25T13:40:30.210600+0000","last_became_peered":"2024-04-25T13:40:30.210600+0000","last_unstale":"2024-04-25T13:40:31.347229+0000","last_undegraded":"2024-04-25T13:40:31.347229+0000","last_fullsized":"2024-04-25T13:40:31.347229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:14.151771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348125+0000","last_change":"2024-04-25T13:40:29.192636+0000","last_active":"2024-04-25T13:40:31.348125+0000","last_peered":"2024-04-25T13:40:31.348125+0000","last_clean":"2024-04-25T13:40:31.348125+0000","last_became_active":"2024-04-25T13:40:29.192248+0000","last_became_peered":"2024-04-25T13:40:29.192248+0000","last_unstale":"2024-04-25T13:40:31.348125+0000","last_undegraded":"2024-04-25T13:40:31.348125+0000","last_fullsized":"2024-04-25T13:40:31.348125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:25.540341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929100,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929066,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338220,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:40:37.124 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:40:37.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd5a21d5640 1 Processor -- start 2024-04-25T13:40:37.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd5a21d5640 1 -- start start 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd5a21d5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd59c0d14a0 0x7fd59c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd5a21d5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd59c0d26d0 0x7fd59c0d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd5a21d5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0b7740 0x7fd59c0b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd5a21d5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd59c0d18b0 con 0x7fd59c0d14a0 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd5a21d5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd59c0b5120 con 0x7fd59c0d26d0 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd5a21d5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd59c0b9210 con 0x7fd59c0b7740 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd59c0d14a0 0x7fd59c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd59c0d14a0 0x7fd59c0b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36428/0 (socket says 172.21.15.80:36428) 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59b7fe640 1 -- 172.21.15.80:0/1591412615 learned_addr learned my addr 172.21.15.80:0/1591412615 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:37.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd592ffd640 1 --2- 172.21.15.80:0/1591412615 >> v2:172.21.15.87:3300/0 conn(0x7fd59c0d26d0 0x7fd59c0d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59bfff640 1 --2- 172.21.15.80:0/1591412615 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0b7740 0x7fd59c0b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59b7fe640 1 -- 172.21.15.80:0/1591412615 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0b7740 msgr2=0x7fd59c0b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:37.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59b7fe640 1 --2- 172.21.15.80:0/1591412615 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0b7740 0x7fd59c0b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59b7fe640 1 -- 172.21.15.80:0/1591412615 >> v2:172.21.15.87:3300/0 conn(0x7fd59c0d26d0 msgr2=0x7fd59c0d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:37.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59b7fe640 1 --2- 172.21.15.80:0/1591412615 >> v2:172.21.15.87:3300/0 conn(0x7fd59c0d26d0 0x7fd59c0d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59b7fe640 1 -- 172.21.15.80:0/1591412615 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c0d3020 con 0x7fd59c0d14a0 2024-04-25T13:40:37.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd592ffd640 1 --2- 172.21.15.80:0/1591412615 >> v2:172.21.15.87:3300/0 conn(0x7fd59c0d26d0 0x7fd59c0d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:37.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.296+0000 7fd59bfff640 1 --2- 172.21.15.80:0/1591412615 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0b7740 0x7fd59c0b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:37.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd59b7fe640 1 --2- 172.21.15.80:0/1591412615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c0b4860 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd59c08e760 tx=0x7fd58c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ab0748182a1558e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:37.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd59affd640 1 -- 172.21.15.80:0/1591412615 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0b0070 con 0x7fd59c0d14a0 2024-04-25T13:40:37.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd59affd640 1 -- 172.21.15.80:0/1591412615 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd58c0ab400 con 0x7fd59c0d14a0 2024-04-25T13:40:37.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd59affd640 1 -- 172.21.15.80:0/1591412615 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c0ab7d0 con 0x7fd59c0d14a0 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 -- 172.21.15.80:0/1591412615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 msgr2=0x7fd59c0b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/1591412615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c0b4860 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd59c08e760 tx=0x7fd58c01c490 comp rx=0 tx=0).stop 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 -- 172.21.15.80:0/1591412615 shutdown_connections 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/1591412615 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0b7740 0x7fd59c0b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/1591412615 >> v2:172.21.15.87:3300/0 conn(0x7fd59c0d26d0 0x7fd59c0d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/1591412615 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c0b4860 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 -- 172.21.15.80:0/1591412615 >> 172.21.15.80:0/1591412615 conn(0x7fd59c073770 msgr2=0x7fd59c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 -- 172.21.15.80:0/1591412615 shutdown_connections 2024-04-25T13:40:37.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.300+0000 7fd5a21d5640 1 -- 172.21.15.80:0/1591412615 wait complete. 2024-04-25T13:40:37.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 Processor -- start 2024-04-25T13:40:37.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 -- start start 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd59c0b7740 0x7fd59c1613d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c161910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0d26d0 0x7fd59c14dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c164d20 con 0x7fd59c0d14a0 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd59c164800 con 0x7fd59c0b7740 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd59c0b84a0 con 0x7fd59c0d26d0 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c161910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c161910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36442/0 (socket says 172.21.15.80:36442) 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 -- 172.21.15.80:0/265946218 learned_addr learned my addr 172.21.15.80:0/265946218 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd59bfff640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0d26d0 0x7fd59c14dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd59b7fe640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd59c0b7740 0x7fd59c1613d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 -- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0d26d0 msgr2=0x7fd59c14dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0d26d0 0x7fd59c14dd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 -- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd59c0b7740 msgr2=0x7fd59c1613d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd59c0b7740 0x7fd59c1613d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 -- 172.21.15.80:0/265946218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd59c14f860 con 0x7fd59c0d14a0 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd592ffd640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c161910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c820 tx=0x7fd58400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd598ff9640 1 -- 172.21.15.80:0/265946218 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584005ba0 con 0x7fd59c0d14a0 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd598ff9640 1 -- 172.21.15.80:0/265946218 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd584005d40 con 0x7fd59c0d14a0 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd59c161f80 con 0x7fd59c0d14a0 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd598ff9640 1 -- 172.21.15.80:0/265946218 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584007a50 con 0x7fd59c0d14a0 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.304+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd59c156c50 con 0x7fd59c0d14a0 2024-04-25T13:40:37.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.308+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd59c14ea70 con 0x7fd59c0d14a0 2024-04-25T13:40:37.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.308+0000 7fd598ff9640 1 -- 172.21.15.80:0/265946218 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd584007bf0 con 0x7fd59c0d14a0 2024-04-25T13:40:37.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.308+0000 7fd598ff9640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd57c06f430 0x7fd57c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.308+0000 7fd598ff9640 1 -- 172.21.15.80:0/265946218 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd58408f2c0 con 0x7fd59c0d14a0 2024-04-25T13:40:37.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.308+0000 7fd59b7fe640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd57c06f430 0x7fd57c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.308+0000 7fd59b7fe640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd57c06f430 0x7fd57c071940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0d6d60 tx=0x7fd58c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:37.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.312+0000 7fd598ff9640 1 -- 172.21.15.80:0/265946218 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd584015020 con 0x7fd59c0d14a0 2024-04-25T13:40:37.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.456+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd59c06a3a0 con 0x7fd57c06f430 2024-04-25T13:40:37.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.472+0000 7fd598ff9640 1 -- 172.21.15.80:0/265946218 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247474 (secure 0 0 0) 0x7fd59c06a3a0 con 0x7fd57c06f430 2024-04-25T13:40:37.475 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:37.477 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd57c06f430 msgr2=0x7fd57c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd57c06f430 0x7fd57c071940 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd59c0d6d60 tx=0x7fd58c008040 comp rx=0 tx=0).stop 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 msgr2=0x7fd59c161910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c161910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c820 tx=0x7fd58400cce0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 shutdown_connections 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd57c06f430 0x7fd57c071940 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd59c0d26d0 0x7fd59c14dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd59c0d14a0 0x7fd59c161910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 --2- 172.21.15.80:0/265946218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd59c0b7740 0x7fd59c1613d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 >> 172.21.15.80:0/265946218 conn(0x7fd59c073770 msgr2=0x7fd59c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 shutdown_connections 2024-04-25T13:40:37.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.476+0000 7fd5a21d5640 1 -- 172.21.15.80:0/265946218 wait complete. 2024-04-25T13:40:37.496 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T13:40:36.093458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248252,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.938610"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348886+0000","last_change":"2024-04-25T13:40:30.221428+0000","last_active":"2024-04-25T13:40:31.348886+0000","last_peered":"2024-04-25T13:40:31.348886+0000","last_clean":"2024-04-25T13:40:31.348886+0000","last_became_active":"2024-04-25T13:40:30.219271+0000","last_became_peered":"2024-04-25T13:40:30.219271+0000","last_unstale":"2024-04-25T13:40:31.348886+0000","last_undegraded":"2024-04-25T13:40:31.348886+0000","last_fullsized":"2024-04-25T13:40:31.348886+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:01.180832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184646+0000","last_change":"2024-04-25T13:40:30.211459+0000","last_active":"2024-04-25T13:40:31.184646+0000","last_peered":"2024-04-25T13:40:31.184646+0000","last_clean":"2024-04-25T13:40:31.184646+0000","last_became_active":"2024-04-25T13:40:30.211073+0000","last_became_peered":"2024-04-25T13:40:30.211073+0000","last_unstale":"2024-04-25T13:40:31.184646+0000","last_undegraded":"2024-04-25T13:40:31.184646+0000","last_fullsized":"2024-04-25T13:40:31.184646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:57:13.973526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346644+0000","last_change":"2024-04-25T13:40:30.213483+0000","last_active":"2024-04-25T13:40:31.346644+0000","last_peered":"2024-04-25T13:40:31.346644+0000","last_clean":"2024-04-25T13:40:31.346644+0000","last_became_active":"2024-04-25T13:40:30.207846+0000","last_became_peered":"2024-04-25T13:40:30.207846+0000","last_unstale":"2024-04-25T13:40:31.346644+0000","last_undegraded":"2024-04-25T13:40:31.346644+0000","last_fullsized":"2024-04-25T13:40:31.346644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:10.391107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.345909+0000","last_change":"2024-04-25T13:40:30.216562+0000","last_active":"2024-04-25T13:40:31.345909+0000","last_peered":"2024-04-25T13:40:31.345909+0000","last_clean":"2024-04-25T13:40:31.345909+0000","last_became_active":"2024-04-25T13:40:30.215696+0000","last_became_peered":"2024-04-25T13:40:30.215696+0000","last_unstale":"2024-04-25T13:40:31.345909+0000","last_undegraded":"2024-04-25T13:40:31.345909+0000","last_fullsized":"2024-04-25T13:40:31.345909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:42.952152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347520+0000","last_change":"2024-04-25T13:40:30.207884+0000","last_active":"2024-04-25T13:40:31.347520+0000","last_peered":"2024-04-25T13:40:31.347520+0000","last_clean":"2024-04-25T13:40:31.347520+0000","last_became_active":"2024-04-25T13:40:30.207080+0000","last_became_peered":"2024-04-25T13:40:30.207080+0000","last_unstale":"2024-04-25T13:40:31.347520+0000","last_undegraded":"2024-04-25T13:40:31.347520+0000","last_fullsized":"2024-04-25T13:40:31.347520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:07.164518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347177+0000","last_change":"2024-04-25T13:40:30.221017+0000","last_active":"2024-04-25T13:40:31.347177+0000","last_peered":"2024-04-25T13:40:31.347177+0000","last_clean":"2024-04-25T13:40:31.347177+0000","last_became_active":"2024-04-25T13:40:30.219333+0000","last_became_peered":"2024-04-25T13:40:30.219333+0000","last_unstale":"2024-04-25T13:40:31.347177+0000","last_undegraded":"2024-04-25T13:40:31.347177+0000","last_fullsized":"2024-04-25T13:40:31.347177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:29.428442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349282+0000","last_change":"2024-04-25T13:40:30.216405+0000","last_active":"2024-04-25T13:40:31.349282+0000","last_peered":"2024-04-25T13:40:31.349282+0000","last_clean":"2024-04-25T13:40:31.349282+0000","last_became_active":"2024-04-25T13:40:30.215354+0000","last_became_peered":"2024-04-25T13:40:30.215354+0000","last_unstale":"2024-04-25T13:40:31.349282+0000","last_undegraded":"2024-04-25T13:40:31.349282+0000","last_fullsized":"2024-04-25T13:40:31.349282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:22.813674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346103+0000","last_change":"2024-04-25T13:40:30.207820+0000","last_active":"2024-04-25T13:40:31.346103+0000","last_peered":"2024-04-25T13:40:31.346103+0000","last_clean":"2024-04-25T13:40:31.346103+0000","last_became_active":"2024-04-25T13:40:30.207148+0000","last_became_peered":"2024-04-25T13:40:30.207148+0000","last_unstale":"2024-04-25T13:40:31.346103+0000","last_undegraded":"2024-04-25T13:40:31.346103+0000","last_fullsized":"2024-04-25T13:40:31.346103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:05.730270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346696+0000","last_change":"2024-04-25T13:40:30.219574+0000","last_active":"2024-04-25T13:40:31.346696+0000","last_peered":"2024-04-25T13:40:31.346696+0000","last_clean":"2024-04-25T13:40:31.346696+0000","last_became_active":"2024-04-25T13:40:30.216463+0000","last_became_peered":"2024-04-25T13:40:30.216463+0000","last_unstale":"2024-04-25T13:40:31.346696+0000","last_undegraded":"2024-04-25T13:40:31.346696+0000","last_fullsized":"2024-04-25T13:40:31.346696+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:33.974714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219726+0000","last_change":"2024-04-25T13:40:30.202005+0000","last_active":"2024-04-25T13:40:31.219726+0000","last_peered":"2024-04-25T13:40:31.219726+0000","last_clean":"2024-04-25T13:40:31.219726+0000","last_became_active":"2024-04-25T13:40:30.201195+0000","last_became_peered":"2024-04-25T13:40:30.201195+0000","last_unstale":"2024-04-25T13:40:31.219726+0000","last_undegraded":"2024-04-25T13:40:31.219726+0000","last_fullsized":"2024-04-25T13:40:31.219726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:20.543637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219380+0000","last_change":"2024-04-25T13:40:30.221626+0000","last_active":"2024-04-25T13:40:31.219380+0000","last_peered":"2024-04-25T13:40:31.219380+0000","last_clean":"2024-04-25T13:40:31.219380+0000","last_became_active":"2024-04-25T13:40:30.221294+0000","last_became_peered":"2024-04-25T13:40:30.221294+0000","last_unstale":"2024-04-25T13:40:31.219380+0000","last_undegraded":"2024-04-25T13:40:31.219380+0000","last_fullsized":"2024-04-25T13:40:31.219380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:49.053408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348097+0000","last_change":"2024-04-25T13:40:30.199635+0000","last_active":"2024-04-25T13:40:31.348097+0000","last_peered":"2024-04-25T13:40:31.348097+0000","last_clean":"2024-04-25T13:40:31.348097+0000","last_became_active":"2024-04-25T13:40:30.198748+0000","last_became_peered":"2024-04-25T13:40:30.198748+0000","last_unstale":"2024-04-25T13:40:31.348097+0000","last_undegraded":"2024-04-25T13:40:31.348097+0000","last_fullsized":"2024-04-25T13:40:31.348097+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:09.065327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346436+0000","last_change":"2024-04-25T13:40:30.215629+0000","last_active":"2024-04-25T13:40:31.346436+0000","last_peered":"2024-04-25T13:40:31.346436+0000","last_clean":"2024-04-25T13:40:31.346436+0000","last_became_active":"2024-04-25T13:40:30.214870+0000","last_became_peered":"2024-04-25T13:40:30.214870+0000","last_unstale":"2024-04-25T13:40:31.346436+0000","last_undegraded":"2024-04-25T13:40:31.346436+0000","last_fullsized":"2024-04-25T13:40:31.346436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:21.817644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346237+0000","last_change":"2024-04-25T13:40:30.200380+0000","last_active":"2024-04-25T13:40:31.346237+0000","last_peered":"2024-04-25T13:40:31.346237+0000","last_clean":"2024-04-25T13:40:31.346237+0000","last_became_active":"2024-04-25T13:40:30.198804+0000","last_became_peered":"2024-04-25T13:40:30.198804+0000","last_unstale":"2024-04-25T13:40:31.346237+0000","last_undegraded":"2024-04-25T13:40:31.346237+0000","last_fullsized":"2024-04-25T13:40:31.346237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:02.090517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185879+0000","last_change":"2024-04-25T13:40:30.228782+0000","last_active":"2024-04-25T13:40:31.185879+0000","last_peered":"2024-04-25T13:40:31.185879+0000","last_clean":"2024-04-25T13:40:31.185879+0000","last_became_active":"2024-04-25T13:40:30.228329+0000","last_became_peered":"2024-04-25T13:40:30.228329+0000","last_unstale":"2024-04-25T13:40:31.185879+0000","last_undegraded":"2024-04-25T13:40:31.185879+0000","last_fullsized":"2024-04-25T13:40:31.185879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:39.981195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346736+0000","last_change":"2024-04-25T13:40:30.221267+0000","last_active":"2024-04-25T13:40:31.346736+0000","last_peered":"2024-04-25T13:40:31.346736+0000","last_clean":"2024-04-25T13:40:31.346736+0000","last_became_active":"2024-04-25T13:40:30.219761+0000","last_became_peered":"2024-04-25T13:40:30.219761+0000","last_unstale":"2024-04-25T13:40:31.346736+0000","last_undegraded":"2024-04-25T13:40:31.346736+0000","last_fullsized":"2024-04-25T13:40:31.346736+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:16.273351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346948+0000","last_change":"2024-04-25T13:40:30.225471+0000","last_active":"2024-04-25T13:40:31.346948+0000","last_peered":"2024-04-25T13:40:31.346948+0000","last_clean":"2024-04-25T13:40:31.346948+0000","last_became_active":"2024-04-25T13:40:30.224161+0000","last_became_peered":"2024-04-25T13:40:30.224161+0000","last_unstale":"2024-04-25T13:40:31.346948+0000","last_undegraded":"2024-04-25T13:40:31.346948+0000","last_fullsized":"2024-04-25T13:40:31.346948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:16.868301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186576+0000","last_change":"2024-04-25T13:40:30.204030+0000","last_active":"2024-04-25T13:40:31.186576+0000","last_peered":"2024-04-25T13:40:31.186576+0000","last_clean":"2024-04-25T13:40:31.186576+0000","last_became_active":"2024-04-25T13:40:30.203298+0000","last_became_peered":"2024-04-25T13:40:30.203298+0000","last_unstale":"2024-04-25T13:40:31.186576+0000","last_undegraded":"2024-04-25T13:40:31.186576+0000","last_fullsized":"2024-04-25T13:40:31.186576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:51.572150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347790+0000","last_change":"2024-04-25T13:40:30.220004+0000","last_active":"2024-04-25T13:40:31.347790+0000","last_peered":"2024-04-25T13:40:31.347790+0000","last_clean":"2024-04-25T13:40:31.347790+0000","last_became_active":"2024-04-25T13:40:30.217606+0000","last_became_peered":"2024-04-25T13:40:30.217606+0000","last_unstale":"2024-04-25T13:40:31.347790+0000","last_undegraded":"2024-04-25T13:40:31.347790+0000","last_fullsized":"2024-04-25T13:40:31.347790+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:26:48.795741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220183+0000","last_change":"2024-04-25T13:40:30.228676+0000","last_active":"2024-04-25T13:40:31.220183+0000","last_peered":"2024-04-25T13:40:31.220183+0000","last_clean":"2024-04-25T13:40:31.220183+0000","last_became_active":"2024-04-25T13:40:30.228364+0000","last_became_peered":"2024-04-25T13:40:30.228364+0000","last_unstale":"2024-04-25T13:40:31.220183+0000","last_undegraded":"2024-04-25T13:40:31.220183+0000","last_fullsized":"2024-04-25T13:40:31.220183+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:09.381688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347827+0000","last_change":"2024-04-25T13:40:30.220330+0000","last_active":"2024-04-25T13:40:31.347827+0000","last_peered":"2024-04-25T13:40:31.347827+0000","last_clean":"2024-04-25T13:40:31.347827+0000","last_became_active":"2024-04-25T13:40:30.219702+0000","last_became_peered":"2024-04-25T13:40:30.219702+0000","last_unstale":"2024-04-25T13:40:31.347827+0000","last_undegraded":"2024-04-25T13:40:31.347827+0000","last_fullsized":"2024-04-25T13:40:31.347827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:24.416759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349924+0000","last_change":"2024-04-25T13:40:30.224681+0000","last_active":"2024-04-25T13:40:31.349924+0000","last_peered":"2024-04-25T13:40:31.349924+0000","last_clean":"2024-04-25T13:40:31.349924+0000","last_became_active":"2024-04-25T13:40:30.218495+0000","last_became_peered":"2024-04-25T13:40:30.218495+0000","last_unstale":"2024-04-25T13:40:31.349924+0000","last_undegraded":"2024-04-25T13:40:31.349924+0000","last_fullsized":"2024-04-25T13:40:31.349924+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:21.085624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347112+0000","last_change":"2024-04-25T13:40:30.220292+0000","last_active":"2024-04-25T13:40:31.347112+0000","last_peered":"2024-04-25T13:40:31.347112+0000","last_clean":"2024-04-25T13:40:31.347112+0000","last_became_active":"2024-04-25T13:40:30.219473+0000","last_became_peered":"2024-04-25T13:40:30.219473+0000","last_unstale":"2024-04-25T13:40:31.347112+0000","last_undegraded":"2024-04-25T13:40:31.347112+0000","last_fullsized":"2024-04-25T13:40:31.347112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:03.322464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346401+0000","last_change":"2024-04-25T13:40:30.216234+0000","last_active":"2024-04-25T13:40:31.346401+0000","last_peered":"2024-04-25T13:40:31.346401+0000","last_clean":"2024-04-25T13:40:31.346401+0000","last_became_active":"2024-04-25T13:40:30.215246+0000","last_became_peered":"2024-04-25T13:40:30.215246+0000","last_unstale":"2024-04-25T13:40:31.346401+0000","last_undegraded":"2024-04-25T13:40:31.346401+0000","last_fullsized":"2024-04-25T13:40:31.346401+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:09.558215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347524+0000","last_change":"2024-04-25T13:40:30.226582+0000","last_active":"2024-04-25T13:40:31.347524+0000","last_peered":"2024-04-25T13:40:31.347524+0000","last_clean":"2024-04-25T13:40:31.347524+0000","last_became_active":"2024-04-25T13:40:30.226077+0000","last_became_peered":"2024-04-25T13:40:30.226077+0000","last_unstale":"2024-04-25T13:40:31.347524+0000","last_undegraded":"2024-04-25T13:40:31.347524+0000","last_fullsized":"2024-04-25T13:40:31.347524+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:08.951509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219854+0000","last_change":"2024-04-25T13:40:30.220604+0000","last_active":"2024-04-25T13:40:31.219854+0000","last_peered":"2024-04-25T13:40:31.219854+0000","last_clean":"2024-04-25T13:40:31.219854+0000","last_became_active":"2024-04-25T13:40:30.220289+0000","last_became_peered":"2024-04-25T13:40:30.220289+0000","last_unstale":"2024-04-25T13:40:31.219854+0000","last_undegraded":"2024-04-25T13:40:31.219854+0000","last_fullsized":"2024-04-25T13:40:31.219854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:54.152591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184983+0000","last_change":"2024-04-25T13:40:30.204488+0000","last_active":"2024-04-25T13:40:31.184983+0000","last_peered":"2024-04-25T13:40:31.184983+0000","last_clean":"2024-04-25T13:40:31.184983+0000","last_became_active":"2024-04-25T13:40:30.203206+0000","last_became_peered":"2024-04-25T13:40:30.203206+0000","last_unstale":"2024-04-25T13:40:31.184983+0000","last_undegraded":"2024-04-25T13:40:31.184983+0000","last_fullsized":"2024-04-25T13:40:31.184983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:07.299339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346652+0000","last_change":"2024-04-25T13:40:30.219870+0000","last_active":"2024-04-25T13:40:31.346652+0000","last_peered":"2024-04-25T13:40:31.346652+0000","last_clean":"2024-04-25T13:40:31.346652+0000","last_became_active":"2024-04-25T13:40:30.218894+0000","last_became_peered":"2024-04-25T13:40:30.218894+0000","last_unstale":"2024-04-25T13:40:31.346652+0000","last_undegraded":"2024-04-25T13:40:31.346652+0000","last_fullsized":"2024-04-25T13:40:31.346652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:04.359710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346438+0000","last_change":"2024-04-25T13:40:30.212775+0000","last_active":"2024-04-25T13:40:31.346438+0000","last_peered":"2024-04-25T13:40:31.346438+0000","last_clean":"2024-04-25T13:40:31.346438+0000","last_became_active":"2024-04-25T13:40:30.212382+0000","last_became_peered":"2024-04-25T13:40:30.212382+0000","last_unstale":"2024-04-25T13:40:31.346438+0000","last_undegraded":"2024-04-25T13:40:31.346438+0000","last_fullsized":"2024-04-25T13:40:31.346438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:12.132012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347837+0000","last_change":"2024-04-25T13:40:30.213875+0000","last_active":"2024-04-25T13:40:31.347837+0000","last_peered":"2024-04-25T13:40:31.347837+0000","last_clean":"2024-04-25T13:40:31.347837+0000","last_became_active":"2024-04-25T13:40:30.213530+0000","last_became_peered":"2024-04-25T13:40:30.213530+0000","last_unstale":"2024-04-25T13:40:31.347837+0000","last_undegraded":"2024-04-25T13:40:31.347837+0000","last_fullsized":"2024-04-25T13:40:31.347837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:26:16.067584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186661+0000","last_change":"2024-04-25T13:40:30.221392+0000","last_active":"2024-04-25T13:40:31.186661+0000","last_peered":"2024-04-25T13:40:31.186661+0000","last_clean":"2024-04-25T13:40:31.186661+0000","last_became_active":"2024-04-25T13:40:30.221056+0000","last_became_peered":"2024-04-25T13:40:30.221056+0000","last_unstale":"2024-04-25T13:40:31.186661+0000","last_undegraded":"2024-04-25T13:40:31.186661+0000","last_fullsized":"2024-04-25T13:40:31.186661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:54.209154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219551+0000","last_change":"2024-04-25T13:40:30.211302+0000","last_active":"2024-04-25T13:40:31.219551+0000","last_peered":"2024-04-25T13:40:31.219551+0000","last_clean":"2024-04-25T13:40:31.219551+0000","last_became_active":"2024-04-25T13:40:30.210970+0000","last_became_peered":"2024-04-25T13:40:30.210970+0000","last_unstale":"2024-04-25T13:40:31.219551+0000","last_undegraded":"2024-04-25T13:40:31.219551+0000","last_fullsized":"2024-04-25T13:40:31.219551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:19.633269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349550+0000","last_change":"2024-04-25T13:40:30.201358+0000","last_active":"2024-04-25T13:40:31.349550+0000","last_peered":"2024-04-25T13:40:31.349550+0000","last_clean":"2024-04-25T13:40:31.349550+0000","last_became_active":"2024-04-25T13:40:30.200493+0000","last_became_peered":"2024-04-25T13:40:30.200493+0000","last_unstale":"2024-04-25T13:40:31.349550+0000","last_undegraded":"2024-04-25T13:40:31.349550+0000","last_fullsized":"2024-04-25T13:40:31.349550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:38.778259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347724+0000","last_change":"2024-04-25T13:40:29.193901+0000","last_active":"2024-04-25T13:40:31.347724+0000","last_peered":"2024-04-25T13:40:31.347724+0000","last_clean":"2024-04-25T13:40:31.347724+0000","last_became_active":"2024-04-25T13:40:29.193719+0000","last_became_peered":"2024-04-25T13:40:29.193719+0000","last_unstale":"2024-04-25T13:40:31.347724+0000","last_undegraded":"2024-04-25T13:40:31.347724+0000","last_fullsized":"2024-04-25T13:40:31.347724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:57.372209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347538+0000","last_change":"2024-04-25T13:40:30.186468+0000","last_active":"2024-04-25T13:40:31.347538+0000","last_peered":"2024-04-25T13:40:31.347538+0000","last_clean":"2024-04-25T13:40:31.347538+0000","last_became_active":"2024-04-25T13:40:30.185965+0000","last_became_peered":"2024-04-25T13:40:30.185965+0000","last_unstale":"2024-04-25T13:40:31.347538+0000","last_undegraded":"2024-04-25T13:40:31.347538+0000","last_fullsized":"2024-04-25T13:40:31.347538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:42.249844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220247+0000","last_change":"2024-04-25T13:40:29.185827+0000","last_active":"2024-04-25T13:40:31.220247+0000","last_peered":"2024-04-25T13:40:31.220247+0000","last_clean":"2024-04-25T13:40:31.220247+0000","last_became_active":"2024-04-25T13:40:29.185130+0000","last_became_peered":"2024-04-25T13:40:29.185130+0000","last_unstale":"2024-04-25T13:40:31.220247+0000","last_undegraded":"2024-04-25T13:40:31.220247+0000","last_fullsized":"2024-04-25T13:40:31.220247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:46.813189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347960+0000","last_change":"2024-04-25T13:40:29.175272+0000","last_active":"2024-04-25T13:40:31.347960+0000","last_peered":"2024-04-25T13:40:31.347960+0000","last_clean":"2024-04-25T13:40:31.347960+0000","last_became_active":"2024-04-25T13:40:29.174962+0000","last_became_peered":"2024-04-25T13:40:29.174962+0000","last_unstale":"2024-04-25T13:40:31.347960+0000","last_undegraded":"2024-04-25T13:40:31.347960+0000","last_fullsized":"2024-04-25T13:40:31.347960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:14.296130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347747+0000","last_change":"2024-04-25T13:40:30.214155+0000","last_active":"2024-04-25T13:40:31.347747+0000","last_peered":"2024-04-25T13:40:31.347747+0000","last_clean":"2024-04-25T13:40:31.347747+0000","last_became_active":"2024-04-25T13:40:30.212880+0000","last_became_peered":"2024-04-25T13:40:30.212880+0000","last_unstale":"2024-04-25T13:40:31.347747+0000","last_undegraded":"2024-04-25T13:40:31.347747+0000","last_fullsized":"2024-04-25T13:40:31.347747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:17.642730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349764+0000","last_change":"2024-04-25T13:40:29.185537+0000","last_active":"2024-04-25T13:40:31.349764+0000","last_peered":"2024-04-25T13:40:31.349764+0000","last_clean":"2024-04-25T13:40:31.349764+0000","last_became_active":"2024-04-25T13:40:29.182804+0000","last_became_peered":"2024-04-25T13:40:29.182804+0000","last_unstale":"2024-04-25T13:40:31.349764+0000","last_undegraded":"2024-04-25T13:40:31.349764+0000","last_fullsized":"2024-04-25T13:40:31.349764+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:45.820954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.184917+0000","last_change":"2024-04-25T13:40:30.217108+0000","last_active":"2024-04-25T13:40:31.184917+0000","last_peered":"2024-04-25T13:40:31.184917+0000","last_clean":"2024-04-25T13:40:31.184917+0000","last_became_active":"2024-04-25T13:40:30.216260+0000","last_became_peered":"2024-04-25T13:40:30.216260+0000","last_unstale":"2024-04-25T13:40:31.184917+0000","last_undegraded":"2024-04-25T13:40:31.184917+0000","last_fullsized":"2024-04-25T13:40:31.184917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:23.045755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.220246+0000","last_change":"2024-04-25T13:40:29.184744+0000","last_active":"2024-04-25T13:40:31.220246+0000","last_peered":"2024-04-25T13:40:31.220246+0000","last_clean":"2024-04-25T13:40:31.220246+0000","last_became_active":"2024-04-25T13:40:29.183985+0000","last_became_peered":"2024-04-25T13:40:29.183985+0000","last_unstale":"2024-04-25T13:40:31.220246+0000","last_undegraded":"2024-04-25T13:40:31.220246+0000","last_fullsized":"2024-04-25T13:40:31.220246+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:56.228309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347453+0000","last_change":"2024-04-25T13:40:30.225124+0000","last_active":"2024-04-25T13:40:31.347453+0000","last_peered":"2024-04-25T13:40:31.347453+0000","last_clean":"2024-04-25T13:40:31.347453+0000","last_became_active":"2024-04-25T13:40:30.223455+0000","last_became_peered":"2024-04-25T13:40:30.223455+0000","last_unstale":"2024-04-25T13:40:31.347453+0000","last_undegraded":"2024-04-25T13:40:31.347453+0000","last_fullsized":"2024-04-25T13:40:31.347453+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:03.877694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350848+0000","last_change":"2024-04-25T13:40:29.190850+0000","last_active":"2024-04-25T13:40:31.350848+0000","last_peered":"2024-04-25T13:40:31.350848+0000","last_clean":"2024-04-25T13:40:31.350848+0000","last_became_active":"2024-04-25T13:40:29.188098+0000","last_became_peered":"2024-04-25T13:40:29.188098+0000","last_unstale":"2024-04-25T13:40:31.350848+0000","last_undegraded":"2024-04-25T13:40:31.350848+0000","last_fullsized":"2024-04-25T13:40:31.350848+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:07.915234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347972+0000","last_change":"2024-04-25T13:40:30.221083+0000","last_active":"2024-04-25T13:40:31.347972+0000","last_peered":"2024-04-25T13:40:31.347972+0000","last_clean":"2024-04-25T13:40:31.347972+0000","last_became_active":"2024-04-25T13:40:30.220819+0000","last_became_peered":"2024-04-25T13:40:30.220819+0000","last_unstale":"2024-04-25T13:40:31.347972+0000","last_undegraded":"2024-04-25T13:40:31.347972+0000","last_fullsized":"2024-04-25T13:40:31.347972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:35:43.306079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185190+0000","last_change":"2024-04-25T13:40:29.189173+0000","last_active":"2024-04-25T13:40:31.185190+0000","last_peered":"2024-04-25T13:40:31.185190+0000","last_clean":"2024-04-25T13:40:31.185190+0000","last_became_active":"2024-04-25T13:40:29.188914+0000","last_became_peered":"2024-04-25T13:40:29.188914+0000","last_unstale":"2024-04-25T13:40:31.185190+0000","last_undegraded":"2024-04-25T13:40:31.185190+0000","last_fullsized":"2024-04-25T13:40:31.185190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:12.374973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346508+0000","last_change":"2024-04-25T13:40:30.218953+0000","last_active":"2024-04-25T13:40:31.346508+0000","last_peered":"2024-04-25T13:40:31.346508+0000","last_clean":"2024-04-25T13:40:31.346508+0000","last_became_active":"2024-04-25T13:40:30.218568+0000","last_became_peered":"2024-04-25T13:40:30.218568+0000","last_unstale":"2024-04-25T13:40:31.346508+0000","last_undegraded":"2024-04-25T13:40:31.346508+0000","last_fullsized":"2024-04-25T13:40:31.346508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:02.608556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219913+0000","last_change":"2024-04-25T13:40:29.175817+0000","last_active":"2024-04-25T13:40:31.219913+0000","last_peered":"2024-04-25T13:40:31.219913+0000","last_clean":"2024-04-25T13:40:31.219913+0000","last_became_active":"2024-04-25T13:40:29.175066+0000","last_became_peered":"2024-04-25T13:40:29.175066+0000","last_unstale":"2024-04-25T13:40:31.219913+0000","last_undegraded":"2024-04-25T13:40:31.219913+0000","last_fullsized":"2024-04-25T13:40:31.219913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:27.404657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348852+0000","last_change":"2024-04-25T13:40:30.215383+0000","last_active":"2024-04-25T13:40:31.348852+0000","last_peered":"2024-04-25T13:40:31.348852+0000","last_clean":"2024-04-25T13:40:31.348852+0000","last_became_active":"2024-04-25T13:40:30.214994+0000","last_became_peered":"2024-04-25T13:40:30.214994+0000","last_unstale":"2024-04-25T13:40:31.348852+0000","last_undegraded":"2024-04-25T13:40:31.348852+0000","last_fullsized":"2024-04-25T13:40:31.348852+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:49.832720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347401+0000","last_change":"2024-04-25T13:40:29.185820+0000","last_active":"2024-04-25T13:40:31.347401+0000","last_peered":"2024-04-25T13:40:31.347401+0000","last_clean":"2024-04-25T13:40:31.347401+0000","last_became_active":"2024-04-25T13:40:29.185576+0000","last_became_peered":"2024-04-25T13:40:29.185576+0000","last_unstale":"2024-04-25T13:40:31.347401+0000","last_undegraded":"2024-04-25T13:40:31.347401+0000","last_fullsized":"2024-04-25T13:40:31.347401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:18.356259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348088+0000","last_change":"2024-04-25T13:40:30.220046+0000","last_active":"2024-04-25T13:40:31.348088+0000","last_peered":"2024-04-25T13:40:31.348088+0000","last_clean":"2024-04-25T13:40:31.348088+0000","last_became_active":"2024-04-25T13:40:30.219244+0000","last_became_peered":"2024-04-25T13:40:30.219244+0000","last_unstale":"2024-04-25T13:40:31.348088+0000","last_undegraded":"2024-04-25T13:40:31.348088+0000","last_fullsized":"2024-04-25T13:40:31.348088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:34.257699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350082+0000","last_change":"2024-04-25T13:40:29.187441+0000","last_active":"2024-04-25T13:40:31.350082+0000","last_peered":"2024-04-25T13:40:31.350082+0000","last_clean":"2024-04-25T13:40:31.350082+0000","last_became_active":"2024-04-25T13:40:29.181693+0000","last_became_peered":"2024-04-25T13:40:29.181693+0000","last_unstale":"2024-04-25T13:40:31.350082+0000","last_undegraded":"2024-04-25T13:40:31.350082+0000","last_fullsized":"2024-04-25T13:40:31.350082+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:57.361661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347958+0000","last_change":"2024-04-25T13:40:30.229833+0000","last_active":"2024-04-25T13:40:31.347958+0000","last_peered":"2024-04-25T13:40:31.347958+0000","last_clean":"2024-04-25T13:40:31.347958+0000","last_became_active":"2024-04-25T13:40:30.229011+0000","last_became_peered":"2024-04-25T13:40:30.229011+0000","last_unstale":"2024-04-25T13:40:31.347958+0000","last_undegraded":"2024-04-25T13:40:31.347958+0000","last_fullsized":"2024-04-25T13:40:31.347958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:15.058616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219728+0000","last_change":"2024-04-25T13:40:29.188978+0000","last_active":"2024-04-25T13:40:31.219728+0000","last_peered":"2024-04-25T13:40:31.219728+0000","last_clean":"2024-04-25T13:40:31.219728+0000","last_became_active":"2024-04-25T13:40:29.185846+0000","last_became_peered":"2024-04-25T13:40:29.185846+0000","last_unstale":"2024-04-25T13:40:31.219728+0000","last_undegraded":"2024-04-25T13:40:31.219728+0000","last_fullsized":"2024-04-25T13:40:31.219728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:08.912260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219935+0000","last_change":"2024-04-25T13:40:30.215550+0000","last_active":"2024-04-25T13:40:31.219935+0000","last_peered":"2024-04-25T13:40:31.219935+0000","last_clean":"2024-04-25T13:40:31.219935+0000","last_became_active":"2024-04-25T13:40:30.215250+0000","last_became_peered":"2024-04-25T13:40:30.215250+0000","last_unstale":"2024-04-25T13:40:31.219935+0000","last_undegraded":"2024-04-25T13:40:31.219935+0000","last_fullsized":"2024-04-25T13:40:31.219935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:59.530072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186915+0000","last_change":"2024-04-25T13:40:29.168980+0000","last_active":"2024-04-25T13:40:31.186915+0000","last_peered":"2024-04-25T13:40:31.186915+0000","last_clean":"2024-04-25T13:40:31.186915+0000","last_became_active":"2024-04-25T13:40:29.168218+0000","last_became_peered":"2024-04-25T13:40:29.168218+0000","last_unstale":"2024-04-25T13:40:31.186915+0000","last_undegraded":"2024-04-25T13:40:31.186915+0000","last_fullsized":"2024-04-25T13:40:31.186915+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:25.060887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349934+0000","last_change":"2024-04-25T13:40:30.216707+0000","last_active":"2024-04-25T13:40:31.349934+0000","last_peered":"2024-04-25T13:40:31.349934+0000","last_clean":"2024-04-25T13:40:31.349934+0000","last_became_active":"2024-04-25T13:40:30.215907+0000","last_became_peered":"2024-04-25T13:40:30.215907+0000","last_unstale":"2024-04-25T13:40:31.349934+0000","last_undegraded":"2024-04-25T13:40:31.349934+0000","last_fullsized":"2024-04-25T13:40:31.349934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:22.984851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346871+0000","last_change":"2024-04-25T13:40:29.182090+0000","last_active":"2024-04-25T13:40:31.346871+0000","last_peered":"2024-04-25T13:40:31.346871+0000","last_clean":"2024-04-25T13:40:31.346871+0000","last_became_active":"2024-04-25T13:40:29.181228+0000","last_became_peered":"2024-04-25T13:40:29.181228+0000","last_unstale":"2024-04-25T13:40:31.346871+0000","last_undegraded":"2024-04-25T13:40:31.346871+0000","last_fullsized":"2024-04-25T13:40:31.346871+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:49.643409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348554+0000","last_change":"2024-04-25T13:40:30.230355+0000","last_active":"2024-04-25T13:40:31.348554+0000","last_peered":"2024-04-25T13:40:31.348554+0000","last_clean":"2024-04-25T13:40:31.348554+0000","last_became_active":"2024-04-25T13:40:30.229397+0000","last_became_peered":"2024-04-25T13:40:30.229397+0000","last_unstale":"2024-04-25T13:40:31.348554+0000","last_undegraded":"2024-04-25T13:40:31.348554+0000","last_fullsized":"2024-04-25T13:40:31.348554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:12:23.803621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.187149+0000","last_change":"2024-04-25T13:40:29.160176+0000","last_active":"2024-04-25T13:40:31.187149+0000","last_peered":"2024-04-25T13:40:31.187149+0000","last_clean":"2024-04-25T13:40:31.187149+0000","last_became_active":"2024-04-25T13:40:29.159439+0000","last_became_peered":"2024-04-25T13:40:29.159439+0000","last_unstale":"2024-04-25T13:40:31.187149+0000","last_undegraded":"2024-04-25T13:40:31.187149+0000","last_fullsized":"2024-04-25T13:40:31.187149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:48.998902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347297+0000","last_change":"2024-04-25T13:40:30.224277+0000","last_active":"2024-04-25T13:40:31.347297+0000","last_peered":"2024-04-25T13:40:31.347297+0000","last_clean":"2024-04-25T13:40:31.347297+0000","last_became_active":"2024-04-25T13:40:30.221764+0000","last_became_peered":"2024-04-25T13:40:30.221764+0000","last_unstale":"2024-04-25T13:40:31.347297+0000","last_undegraded":"2024-04-25T13:40:31.347297+0000","last_fullsized":"2024-04-25T13:40:31.347297+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:12.238288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185510+0000","last_change":"2024-04-25T13:40:29.182136+0000","last_active":"2024-04-25T13:40:31.185510+0000","last_peered":"2024-04-25T13:40:31.185510+0000","last_clean":"2024-04-25T13:40:31.185510+0000","last_became_active":"2024-04-25T13:40:29.181852+0000","last_became_peered":"2024-04-25T13:40:29.181852+0000","last_unstale":"2024-04-25T13:40:31.185510+0000","last_undegraded":"2024-04-25T13:40:31.185510+0000","last_fullsized":"2024-04-25T13:40:31.185510+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:24.681218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350369+0000","last_change":"2024-04-25T13:40:30.217315+0000","last_active":"2024-04-25T13:40:31.350369+0000","last_peered":"2024-04-25T13:40:31.350369+0000","last_clean":"2024-04-25T13:40:31.350369+0000","last_became_active":"2024-04-25T13:40:30.214308+0000","last_became_peered":"2024-04-25T13:40:30.214308+0000","last_unstale":"2024-04-25T13:40:31.350369+0000","last_undegraded":"2024-04-25T13:40:31.350369+0000","last_fullsized":"2024-04-25T13:40:31.350369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:57.857844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186012+0000","last_change":"2024-04-25T13:40:29.189228+0000","last_active":"2024-04-25T13:40:31.186012+0000","last_peered":"2024-04-25T13:40:31.186012+0000","last_clean":"2024-04-25T13:40:31.186012+0000","last_became_active":"2024-04-25T13:40:29.189004+0000","last_became_peered":"2024-04-25T13:40:29.189004+0000","last_unstale":"2024-04-25T13:40:31.186012+0000","last_undegraded":"2024-04-25T13:40:31.186012+0000","last_fullsized":"2024-04-25T13:40:31.186012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:14.763408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349561+0000","last_change":"2024-04-25T13:40:30.223154+0000","last_active":"2024-04-25T13:40:31.349561+0000","last_peered":"2024-04-25T13:40:31.349561+0000","last_clean":"2024-04-25T13:40:31.349561+0000","last_became_active":"2024-04-25T13:40:30.222384+0000","last_became_peered":"2024-04-25T13:40:30.222384+0000","last_unstale":"2024-04-25T13:40:31.349561+0000","last_undegraded":"2024-04-25T13:40:31.349561+0000","last_fullsized":"2024-04-25T13:40:31.349561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:59.286900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":110,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350710+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:40:31.350710+0000","last_peered":"2024-04-25T13:40:31.350710+0000","last_clean":"2024-04-25T13:40:31.350710+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:40:31.350710+0000","last_undegraded":"2024-04-25T13:40:31.350710+0000","last_fullsized":"2024-04-25T13:40:31.350710+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350590+0000","last_change":"2024-04-25T13:40:29.186348+0000","last_active":"2024-04-25T13:40:31.350590+0000","last_peered":"2024-04-25T13:40:31.350590+0000","last_clean":"2024-04-25T13:40:31.350590+0000","last_became_active":"2024-04-25T13:40:29.180642+0000","last_became_peered":"2024-04-25T13:40:29.180642+0000","last_unstale":"2024-04-25T13:40:31.350590+0000","last_undegraded":"2024-04-25T13:40:31.350590+0000","last_fullsized":"2024-04-25T13:40:31.350590+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:43.636874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347213+0000","last_change":"2024-04-25T13:40:30.209809+0000","last_active":"2024-04-25T13:40:31.347213+0000","last_peered":"2024-04-25T13:40:31.347213+0000","last_clean":"2024-04-25T13:40:31.347213+0000","last_became_active":"2024-04-25T13:40:30.209014+0000","last_became_peered":"2024-04-25T13:40:30.209014+0000","last_unstale":"2024-04-25T13:40:31.347213+0000","last_undegraded":"2024-04-25T13:40:31.347213+0000","last_fullsized":"2024-04-25T13:40:31.347213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:24.945222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350549+0000","last_change":"2024-04-25T13:40:29.197074+0000","last_active":"2024-04-25T13:40:31.350549+0000","last_peered":"2024-04-25T13:40:31.350549+0000","last_clean":"2024-04-25T13:40:31.350549+0000","last_became_active":"2024-04-25T13:40:29.193688+0000","last_became_peered":"2024-04-25T13:40:29.193688+0000","last_unstale":"2024-04-25T13:40:31.350549+0000","last_undegraded":"2024-04-25T13:40:31.350549+0000","last_fullsized":"2024-04-25T13:40:31.350549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:03.325908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347895+0000","last_change":"2024-04-25T13:40:30.187471+0000","last_active":"2024-04-25T13:40:31.347895+0000","last_peered":"2024-04-25T13:40:31.347895+0000","last_clean":"2024-04-25T13:40:31.347895+0000","last_became_active":"2024-04-25T13:40:30.186993+0000","last_became_peered":"2024-04-25T13:40:30.186993+0000","last_unstale":"2024-04-25T13:40:31.347895+0000","last_undegraded":"2024-04-25T13:40:31.347895+0000","last_fullsized":"2024-04-25T13:40:31.347895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:17.749355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.187216+0000","last_change":"2024-04-25T13:40:29.159566+0000","last_active":"2024-04-25T13:40:31.187216+0000","last_peered":"2024-04-25T13:40:31.187216+0000","last_clean":"2024-04-25T13:40:31.187216+0000","last_became_active":"2024-04-25T13:40:29.159054+0000","last_became_peered":"2024-04-25T13:40:29.159054+0000","last_unstale":"2024-04-25T13:40:31.187216+0000","last_undegraded":"2024-04-25T13:40:31.187216+0000","last_fullsized":"2024-04-25T13:40:31.187216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:03.455177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185373+0000","last_change":"2024-04-25T13:40:30.203352+0000","last_active":"2024-04-25T13:40:31.185373+0000","last_peered":"2024-04-25T13:40:31.185373+0000","last_clean":"2024-04-25T13:40:31.185373+0000","last_became_active":"2024-04-25T13:40:30.202463+0000","last_became_peered":"2024-04-25T13:40:30.202463+0000","last_unstale":"2024-04-25T13:40:31.185373+0000","last_undegraded":"2024-04-25T13:40:31.185373+0000","last_fullsized":"2024-04-25T13:40:31.185373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:01.912451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350985+0000","last_change":"2024-04-25T13:40:29.190061+0000","last_active":"2024-04-25T13:40:31.350985+0000","last_peered":"2024-04-25T13:40:31.350985+0000","last_clean":"2024-04-25T13:40:31.350985+0000","last_became_active":"2024-04-25T13:40:29.182897+0000","last_became_peered":"2024-04-25T13:40:29.182897+0000","last_unstale":"2024-04-25T13:40:31.350985+0000","last_undegraded":"2024-04-25T13:40:31.350985+0000","last_fullsized":"2024-04-25T13:40:31.350985+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:31.168555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346748+0000","last_change":"2024-04-25T13:40:30.187160+0000","last_active":"2024-04-25T13:40:31.346748+0000","last_peered":"2024-04-25T13:40:31.346748+0000","last_clean":"2024-04-25T13:40:31.346748+0000","last_became_active":"2024-04-25T13:40:30.186695+0000","last_became_peered":"2024-04-25T13:40:30.186695+0000","last_unstale":"2024-04-25T13:40:31.346748+0000","last_undegraded":"2024-04-25T13:40:31.346748+0000","last_fullsized":"2024-04-25T13:40:31.346748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:57.859568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186430+0000","last_change":"2024-04-25T13:40:29.168378+0000","last_active":"2024-04-25T13:40:31.186430+0000","last_peered":"2024-04-25T13:40:31.186430+0000","last_clean":"2024-04-25T13:40:31.186430+0000","last_became_active":"2024-04-25T13:40:29.167515+0000","last_became_peered":"2024-04-25T13:40:29.167515+0000","last_unstale":"2024-04-25T13:40:31.186430+0000","last_undegraded":"2024-04-25T13:40:31.186430+0000","last_fullsized":"2024-04-25T13:40:31.186430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:03.417158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346850+0000","last_change":"2024-04-25T13:40:30.214354+0000","last_active":"2024-04-25T13:40:31.346850+0000","last_peered":"2024-04-25T13:40:31.346850+0000","last_clean":"2024-04-25T13:40:31.346850+0000","last_became_active":"2024-04-25T13:40:30.213970+0000","last_became_peered":"2024-04-25T13:40:30.213970+0000","last_unstale":"2024-04-25T13:40:31.346850+0000","last_undegraded":"2024-04-25T13:40:31.346850+0000","last_fullsized":"2024-04-25T13:40:31.346850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:54.278471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348113+0000","last_change":"2024-04-25T13:40:29.197843+0000","last_active":"2024-04-25T13:40:31.348113+0000","last_peered":"2024-04-25T13:40:31.348113+0000","last_clean":"2024-04-25T13:40:31.348113+0000","last_became_active":"2024-04-25T13:40:29.196299+0000","last_became_peered":"2024-04-25T13:40:29.196299+0000","last_unstale":"2024-04-25T13:40:31.348113+0000","last_undegraded":"2024-04-25T13:40:31.348113+0000","last_fullsized":"2024-04-25T13:40:31.348113+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:41.785353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185424+0000","last_change":"2024-04-25T13:40:30.212599+0000","last_active":"2024-04-25T13:40:31.185424+0000","last_peered":"2024-04-25T13:40:31.185424+0000","last_clean":"2024-04-25T13:40:31.185424+0000","last_became_active":"2024-04-25T13:40:30.211967+0000","last_became_peered":"2024-04-25T13:40:30.211967+0000","last_unstale":"2024-04-25T13:40:31.185424+0000","last_undegraded":"2024-04-25T13:40:31.185424+0000","last_fullsized":"2024-04-25T13:40:31.185424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:28.946427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219865+0000","last_change":"2024-04-25T13:40:29.175520+0000","last_active":"2024-04-25T13:40:31.219865+0000","last_peered":"2024-04-25T13:40:31.219865+0000","last_clean":"2024-04-25T13:40:31.219865+0000","last_became_active":"2024-04-25T13:40:29.174561+0000","last_became_peered":"2024-04-25T13:40:29.174561+0000","last_unstale":"2024-04-25T13:40:31.219865+0000","last_undegraded":"2024-04-25T13:40:31.219865+0000","last_fullsized":"2024-04-25T13:40:31.219865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:55.788741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346789+0000","last_change":"2024-04-25T13:40:30.217175+0000","last_active":"2024-04-25T13:40:31.346789+0000","last_peered":"2024-04-25T13:40:31.346789+0000","last_clean":"2024-04-25T13:40:31.346789+0000","last_became_active":"2024-04-25T13:40:30.216851+0000","last_became_peered":"2024-04-25T13:40:30.216851+0000","last_unstale":"2024-04-25T13:40:31.346789+0000","last_undegraded":"2024-04-25T13:40:31.346789+0000","last_fullsized":"2024-04-25T13:40:31.346789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:12.967853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219908+0000","last_change":"2024-04-25T13:40:29.194455+0000","last_active":"2024-04-25T13:40:31.219908+0000","last_peered":"2024-04-25T13:40:31.219908+0000","last_clean":"2024-04-25T13:40:31.219908+0000","last_became_active":"2024-04-25T13:40:29.193686+0000","last_became_peered":"2024-04-25T13:40:29.193686+0000","last_unstale":"2024-04-25T13:40:31.219908+0000","last_undegraded":"2024-04-25T13:40:31.219908+0000","last_fullsized":"2024-04-25T13:40:31.219908+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:25:27.100097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347083+0000","last_change":"2024-04-25T13:40:30.211947+0000","last_active":"2024-04-25T13:40:31.347083+0000","last_peered":"2024-04-25T13:40:31.347083+0000","last_clean":"2024-04-25T13:40:31.347083+0000","last_became_active":"2024-04-25T13:40:30.202151+0000","last_became_peered":"2024-04-25T13:40:30.202151+0000","last_unstale":"2024-04-25T13:40:31.347083+0000","last_undegraded":"2024-04-25T13:40:31.347083+0000","last_fullsized":"2024-04-25T13:40:31.347083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:54.174571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350372+0000","last_change":"2024-04-25T13:40:29.200572+0000","last_active":"2024-04-25T13:40:31.350372+0000","last_peered":"2024-04-25T13:40:31.350372+0000","last_clean":"2024-04-25T13:40:31.350372+0000","last_became_active":"2024-04-25T13:40:29.199836+0000","last_became_peered":"2024-04-25T13:40:29.199836+0000","last_unstale":"2024-04-25T13:40:31.350372+0000","last_undegraded":"2024-04-25T13:40:31.350372+0000","last_fullsized":"2024-04-25T13:40:31.350372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:38.068886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349670+0000","last_change":"2024-04-25T13:40:30.225985+0000","last_active":"2024-04-25T13:40:31.349670+0000","last_peered":"2024-04-25T13:40:31.349670+0000","last_clean":"2024-04-25T13:40:31.349670+0000","last_became_active":"2024-04-25T13:40:30.224959+0000","last_became_peered":"2024-04-25T13:40:30.224959+0000","last_unstale":"2024-04-25T13:40:31.349670+0000","last_undegraded":"2024-04-25T13:40:31.349670+0000","last_fullsized":"2024-04-25T13:40:31.349670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:13.498711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.186159+0000","last_change":"2024-04-25T13:40:29.196670+0000","last_active":"2024-04-25T13:40:31.186159+0000","last_peered":"2024-04-25T13:40:31.186159+0000","last_clean":"2024-04-25T13:40:31.186159+0000","last_became_active":"2024-04-25T13:40:29.196115+0000","last_became_peered":"2024-04-25T13:40:29.196115+0000","last_unstale":"2024-04-25T13:40:31.186159+0000","last_undegraded":"2024-04-25T13:40:31.186159+0000","last_fullsized":"2024-04-25T13:40:31.186159+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:35.705063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347672+0000","last_change":"2024-04-25T13:40:30.203777+0000","last_active":"2024-04-25T13:40:31.347672+0000","last_peered":"2024-04-25T13:40:31.347672+0000","last_clean":"2024-04-25T13:40:31.347672+0000","last_became_active":"2024-04-25T13:40:30.202751+0000","last_became_peered":"2024-04-25T13:40:30.202751+0000","last_unstale":"2024-04-25T13:40:31.347672+0000","last_undegraded":"2024-04-25T13:40:31.347672+0000","last_fullsized":"2024-04-25T13:40:31.347672+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:09.528068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.185653+0000","last_change":"2024-04-25T13:40:29.176333+0000","last_active":"2024-04-25T13:40:31.185653+0000","last_peered":"2024-04-25T13:40:31.185653+0000","last_clean":"2024-04-25T13:40:31.185653+0000","last_became_active":"2024-04-25T13:40:29.176078+0000","last_became_peered":"2024-04-25T13:40:29.176078+0000","last_unstale":"2024-04-25T13:40:31.185653+0000","last_undegraded":"2024-04-25T13:40:31.185653+0000","last_fullsized":"2024-04-25T13:40:31.185653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:44:19.200730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.346905+0000","last_change":"2024-04-25T13:40:30.216029+0000","last_active":"2024-04-25T13:40:31.346905+0000","last_peered":"2024-04-25T13:40:31.346905+0000","last_clean":"2024-04-25T13:40:31.346905+0000","last_became_active":"2024-04-25T13:40:30.215353+0000","last_became_peered":"2024-04-25T13:40:30.215353+0000","last_unstale":"2024-04-25T13:40:31.346905+0000","last_undegraded":"2024-04-25T13:40:31.346905+0000","last_fullsized":"2024-04-25T13:40:31.346905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:10.825968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349163+0000","last_change":"2024-04-25T13:40:29.195078+0000","last_active":"2024-04-25T13:40:31.349163+0000","last_peered":"2024-04-25T13:40:31.349163+0000","last_clean":"2024-04-25T13:40:31.349163+0000","last_became_active":"2024-04-25T13:40:29.193847+0000","last_became_peered":"2024-04-25T13:40:29.193847+0000","last_unstale":"2024-04-25T13:40:31.349163+0000","last_undegraded":"2024-04-25T13:40:31.349163+0000","last_fullsized":"2024-04-25T13:40:31.349163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:33.205545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.219558+0000","last_change":"2024-04-25T13:40:30.205595+0000","last_active":"2024-04-25T13:40:31.219558+0000","last_peered":"2024-04-25T13:40:31.219558+0000","last_clean":"2024-04-25T13:40:31.219558+0000","last_became_active":"2024-04-25T13:40:30.204793+0000","last_became_peered":"2024-04-25T13:40:30.204793+0000","last_unstale":"2024-04-25T13:40:31.219558+0000","last_undegraded":"2024-04-25T13:40:31.219558+0000","last_fullsized":"2024-04-25T13:40:31.219558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:55.486355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349463+0000","last_change":"2024-04-25T13:40:30.202452+0000","last_active":"2024-04-25T13:40:31.349463+0000","last_peered":"2024-04-25T13:40:31.349463+0000","last_clean":"2024-04-25T13:40:31.349463+0000","last_became_active":"2024-04-25T13:40:30.201683+0000","last_became_peered":"2024-04-25T13:40:30.201683+0000","last_unstale":"2024-04-25T13:40:31.349463+0000","last_undegraded":"2024-04-25T13:40:31.349463+0000","last_fullsized":"2024-04-25T13:40:31.349463+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:26.132452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.350105+0000","last_change":"2024-04-25T13:40:29.199128+0000","last_active":"2024-04-25T13:40:31.350105+0000","last_peered":"2024-04-25T13:40:31.350105+0000","last_clean":"2024-04-25T13:40:31.350105+0000","last_became_active":"2024-04-25T13:40:29.198449+0000","last_became_peered":"2024-04-25T13:40:29.198449+0000","last_unstale":"2024-04-25T13:40:31.350105+0000","last_undegraded":"2024-04-25T13:40:31.350105+0000","last_fullsized":"2024-04-25T13:40:31.350105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:08.596939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347469+0000","last_change":"2024-04-25T13:40:30.220740+0000","last_active":"2024-04-25T13:40:31.347469+0000","last_peered":"2024-04-25T13:40:31.347469+0000","last_clean":"2024-04-25T13:40:31.347469+0000","last_became_active":"2024-04-25T13:40:30.218927+0000","last_became_peered":"2024-04-25T13:40:30.218927+0000","last_unstale":"2024-04-25T13:40:31.347469+0000","last_undegraded":"2024-04-25T13:40:31.347469+0000","last_fullsized":"2024-04-25T13:40:31.347469+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:29.133104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.349098+0000","last_change":"2024-04-25T13:40:29.199401+0000","last_active":"2024-04-25T13:40:31.349098+0000","last_peered":"2024-04-25T13:40:31.349098+0000","last_clean":"2024-04-25T13:40:31.349098+0000","last_became_active":"2024-04-25T13:40:29.198182+0000","last_became_peered":"2024-04-25T13:40:29.198182+0000","last_unstale":"2024-04-25T13:40:31.349098+0000","last_undegraded":"2024-04-25T13:40:31.349098+0000","last_fullsized":"2024-04-25T13:40:31.349098+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:31.807935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347497+0000","last_change":"2024-04-25T13:40:30.213026+0000","last_active":"2024-04-25T13:40:31.347497+0000","last_peered":"2024-04-25T13:40:31.347497+0000","last_clean":"2024-04-25T13:40:31.347497+0000","last_became_active":"2024-04-25T13:40:30.203212+0000","last_became_peered":"2024-04-25T13:40:30.203212+0000","last_unstale":"2024-04-25T13:40:31.347497+0000","last_undegraded":"2024-04-25T13:40:31.347497+0000","last_fullsized":"2024-04-25T13:40:31.347497+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:24.180245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347687+0000","last_change":"2024-04-25T13:40:29.173054+0000","last_active":"2024-04-25T13:40:31.347687+0000","last_peered":"2024-04-25T13:40:31.347687+0000","last_clean":"2024-04-25T13:40:31.347687+0000","last_became_active":"2024-04-25T13:40:29.172375+0000","last_became_peered":"2024-04-25T13:40:29.172375+0000","last_unstale":"2024-04-25T13:40:31.347687+0000","last_undegraded":"2024-04-25T13:40:31.347687+0000","last_fullsized":"2024-04-25T13:40:31.347687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:26.996154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.347229+0000","last_change":"2024-04-25T13:40:30.214823+0000","last_active":"2024-04-25T13:40:31.347229+0000","last_peered":"2024-04-25T13:40:31.347229+0000","last_clean":"2024-04-25T13:40:31.347229+0000","last_became_active":"2024-04-25T13:40:30.210600+0000","last_became_peered":"2024-04-25T13:40:30.210600+0000","last_unstale":"2024-04-25T13:40:31.347229+0000","last_undegraded":"2024-04-25T13:40:31.347229+0000","last_fullsized":"2024-04-25T13:40:31.347229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:29.147778+0000","last_clean_scrub_stamp":"2024-04-25T13:40:29.147778+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:14.151771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T13:40:31.348125+0000","last_change":"2024-04-25T13:40:29.192636+0000","last_active":"2024-04-25T13:40:31.348125+0000","last_peered":"2024-04-25T13:40:31.348125+0000","last_clean":"2024-04-25T13:40:31.348125+0000","last_became_active":"2024-04-25T13:40:29.192248+0000","last_became_peered":"2024-04-25T13:40:29.192248+0000","last_unstale":"2024-04-25T13:40:31.348125+0000","last_undegraded":"2024-04-25T13:40:31.348125+0000","last_fullsized":"2024-04-25T13:40:31.348125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:28.144987+0000","last_clean_scrub_stamp":"2024-04-25T13:40:28.144987+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:25.540341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338276,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929100,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929226,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929066,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":646,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338220,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:40:37.500 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T13:40:37.501 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T13:40:37.501 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T13:40:37.501 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T13:40:37.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5eecdb7640 1 Processor -- start 2024-04-25T13:40:37.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5eecdb7640 1 -- start start 2024-04-25T13:40:37.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5eecdb7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ee80745a0 0x7f5ee806ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5eecdb7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee80cd6c0 con 0x7f5ee80cc720 2024-04-25T13:40:37.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5eecdb7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee80729e0 con 0x7f5ee806f170 2024-04-25T13:40:37.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5eecdb7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee815ea50 con 0x7f5ee80745a0 2024-04-25T13:40:37.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5ee5d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ee80745a0 0x7f5ee806ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5ee659c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5ee80cc720 0x7f5ee8073e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35772/0 (socket says 172.21.15.80:35772) 2024-04-25T13:40:37.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5ee659c640 1 -- 172.21.15.80:0/1139880454 learned_addr learned my addr 172.21.15.80:0/1139880454 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:37.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5ee5d9b640 1 -- 172.21.15.80:0/1139880454 >> v1:172.21.15.87:6789/0 conn(0x7f5ee806f170 legacy=0x7f5ee806f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:37.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5ee5d9b640 1 -- 172.21.15.80:0/1139880454 >> v1:172.21.15.80:6789/0 conn(0x7f5ee80cc720 legacy=0x7f5ee8073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5ee5d9b640 1 -- 172.21.15.80:0/1139880454 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee80cce20 con 0x7f5ee80745a0 2024-04-25T13:40:37.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.672+0000 7f5ee559a640 1 -- 172.21.15.80:0/1139880454 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 796193955 0 0) 0x7f5ee80cd6c0 con 0x7f5ee80cc720 2024-04-25T13:40:37.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5ee5d9b640 1 --2- 172.21.15.80:0/1139880454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80745a0 0x7f5ee806ec30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0097c0 tx=0x7f5edc01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4634590d170a35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:37.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5ee559a640 1 -- 172.21.15.80:0/1139880454 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0b0070 con 0x7f5ee80745a0 2024-04-25T13:40:37.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5ee559a640 1 -- 172.21.15.80:0/1139880454 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5edc0ab600 con 0x7f5ee80745a0 2024-04-25T13:40:37.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5ee559a640 1 -- 172.21.15.80:0/1139880454 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0ab9d0 con 0x7f5ee80745a0 2024-04-25T13:40:37.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5eecdb7640 1 -- 172.21.15.80:0/1139880454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80745a0 msgr2=0x7f5ee806ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5eecdb7640 1 --2- 172.21.15.80:0/1139880454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80745a0 0x7f5ee806ec30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5edc0097c0 tx=0x7f5edc01c3d0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5eecdb7640 1 -- 172.21.15.80:0/1139880454 shutdown_connections 2024-04-25T13:40:37.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5eecdb7640 1 --2- 172.21.15.80:0/1139880454 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80745a0 0x7f5ee806ec30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5eecdb7640 1 -- 172.21.15.80:0/1139880454 >> 172.21.15.80:0/1139880454 conn(0x7f5ee80d20d0 msgr2=0x7f5ee80d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:37.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5eecdb7640 1 -- 172.21.15.80:0/1139880454 shutdown_connections 2024-04-25T13:40:37.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.676+0000 7f5eecdb7640 1 -- 172.21.15.80:0/1139880454 wait complete. 2024-04-25T13:40:37.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 Processor -- start 2024-04-25T13:40:37.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 -- start start 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee806f170 0x7f5ee80c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee80745a0 0x7f5ee8169180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80cc720 0x7f5ee809d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee815f490 con 0x7f5ee80cc720 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee80ccb30 con 0x7f5ee80745a0 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee815f610 con 0x7f5ee806f170 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80cc720 0x7f5ee809d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80cc720 0x7f5ee809d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36466/0 (socket says 172.21.15.80:36466) 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 -- 172.21.15.80:0/948489380 learned_addr learned my addr 172.21.15.80:0/948489380 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:37.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee659c640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee806f170 0x7f5ee80c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 -- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee806f170 msgr2=0x7f5ee80c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee5d9b640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee80745a0 0x7f5ee8169180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee806f170 0x7f5ee80c6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 -- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee80745a0 msgr2=0x7f5ee8169180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee80745a0 0x7f5ee8169180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 -- 172.21.15.80:0/948489380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee80c76e0 con 0x7f5ee80cc720 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ee6d9d640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80cc720 0x7f5ee809d170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000b9c0 tx=0x7f5ee000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ed6ffd640 1 -- 172.21.15.80:0/948489380 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0019070 con 0x7f5ee80cc720 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ed6ffd640 1 -- 172.21.15.80:0/948489380 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee000cba0 con 0x7f5ee80cc720 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee80c79c0 con 0x7f5ee80cc720 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ed6ffd640 1 -- 172.21.15.80:0/948489380 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0014440 con 0x7f5ee80cc720 2024-04-25T13:40:37.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee80c7f80 con 0x7f5ee80cc720 2024-04-25T13:40:37.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee80c7450 con 0x7f5ee80cc720 2024-04-25T13:40:37.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.680+0000 7f5ed6ffd640 1 -- 172.21.15.80:0/948489380 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ee0007370 con 0x7f5ee80cc720 2024-04-25T13:40:37.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.684+0000 7f5ed6ffd640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ec806f430 0x7f5ec8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:37.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.684+0000 7f5ed6ffd640 1 -- 172.21.15.80:0/948489380 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5ee008edd0 con 0x7f5ee80cc720 2024-04-25T13:40:37.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.684+0000 7f5ee659c640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ec806f430 0x7f5ec8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:37.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.684+0000 7f5ee659c640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ec806f430 0x7f5ec8071940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000f70 tx=0x7f5ed8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:37.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.688+0000 7f5ed6ffd640 1 -- 172.21.15.80:0/948489380 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ee005bd50 con 0x7f5ee80cc720 2024-04-25T13:40:37.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f5ee806a3a0 con 0x7f5ee80cc720 2024-04-25T13:40:37.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5ed6ffd640 1 -- 172.21.15.80:0/948489380 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5ee005b650 con 0x7f5ee80cc720 2024-04-25T13:40:37.868 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:37.868 INFO:teuthology.orchestra.run.smithi080.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ec806f430 msgr2=0x7f5ec8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ec806f430 0x7f5ec8071940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8000f70 tx=0x7f5ed8009040 comp rx=0 tx=0).stop 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80cc720 msgr2=0x7f5ee809d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80cc720 0x7f5ee809d170 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000b9c0 tx=0x7f5ee000be80 comp rx=0 tx=0).stop 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 shutdown_connections 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ec806f430 0x7f5ec8071940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee80cc720 0x7f5ee809d170 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee80745a0 0x7f5ee8169180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 --2- 172.21.15.80:0/948489380 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee806f170 0x7f5ee80c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 >> 172.21.15.80:0/948489380 conn(0x7f5ee80d20d0 msgr2=0x7f5ee806f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.868+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 shutdown_connections 2024-04-25T13:40:37.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:37.872+0000 7f5eecdb7640 1 -- 172.21.15.80:0/948489380 wait complete. 2024-04-25T13:40:37.889 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T13:40:37.889 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:38.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99fb4a1640 1 Processor -- start 2024-04-25T13:40:38.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99fb4a1640 1 -- start start 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99fb4a1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f99f40d26d0 0x7f99f40d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99fb4a1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f99f40d46a0 0x7f99f40d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99fb4a1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99f40b9210 con 0x7f99f40d14a0 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99fb4a1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f99f40d18b0 con 0x7f99f40d46a0 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99fb4a1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f99f40b5120 con 0x7f99f40d26d0 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99f9a3e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f99f40d46a0 0x7f99f40d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99f9a3e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f99f40d46a0 0x7f99f40d4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36472/0 (socket says 172.21.15.80:36472) 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99f9a3e640 1 -- 172.21.15.80:0/3170463876 learned_addr learned my addr 172.21.15.80:0/3170463876 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99f8a3c640 1 --2- 172.21.15.80:0/3170463876 >> v2:172.21.15.87:3300/0 conn(0x7f99f40d26d0 0x7f99f40d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99f9a3e640 1 -- 172.21.15.80:0/3170463876 >> v2:172.21.15.87:3300/0 conn(0x7f99f40d26d0 msgr2=0x7f99f40d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:38.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99f9a3e640 1 --2- 172.21.15.80:0/3170463876 >> v2:172.21.15.87:3300/0 conn(0x7f99f40d26d0 0x7f99f40d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99f9a3e640 1 -- 172.21.15.80:0/3170463876 >> v1:172.21.15.87:6789/0 conn(0x7f99f40d14a0 legacy=0x7f99f40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:38.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.024+0000 7f99f9a3e640 1 -- 172.21.15.80:0/3170463876 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f40d3020 con 0x7f99f40d46a0 2024-04-25T13:40:38.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99f9a3e640 1 --2- 172.21.15.80:0/3170463876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d46a0 0x7f99f40d4ab0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99f408e760 tx=0x7f99ec01c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722b76e63e6a0f7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99ebfff640 1 -- 172.21.15.80:0/3170463876 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0b0070 con 0x7f99f40d46a0 2024-04-25T13:40:38.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99ebfff640 1 -- 172.21.15.80:0/3170463876 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ec0ab600 con 0x7f99f40d46a0 2024-04-25T13:40:38.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99ebfff640 1 -- 172.21.15.80:0/3170463876 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec0ab9d0 con 0x7f99f40d46a0 2024-04-25T13:40:38.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99fb4a1640 1 -- 172.21.15.80:0/3170463876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d46a0 msgr2=0x7f99f40d4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/3170463876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d46a0 0x7f99f40d4ab0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f99f408e760 tx=0x7f99ec01c270 comp rx=0 tx=0).stop 2024-04-25T13:40:38.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99fb4a1640 1 -- 172.21.15.80:0/3170463876 shutdown_connections 2024-04-25T13:40:38.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/3170463876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d46a0 0x7f99f40d4ab0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/3170463876 >> v2:172.21.15.87:3300/0 conn(0x7f99f40d26d0 0x7f99f40d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99fb4a1640 1 -- 172.21.15.80:0/3170463876 >> 172.21.15.80:0/3170463876 conn(0x7f99f4073770 msgr2=0x7f99f406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:38.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99fb4a1640 1 -- 172.21.15.80:0/3170463876 shutdown_connections 2024-04-25T13:40:38.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.028+0000 7f99fb4a1640 1 -- 172.21.15.80:0/3170463876 wait complete. 2024-04-25T13:40:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 Processor -- start 2024-04-25T13:40:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 -- start start 2024-04-25T13:40:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d14a0 0x7f99f4153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99f40d26d0 0x7f99f40b7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99f40d46a0 0x7f99f40b6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4164d20 con 0x7f99f40d14a0 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f99f4164800 con 0x7f99f40d26d0 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f99f416d1d0 con 0x7f99f40d46a0 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d14a0 0x7f99f4153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d14a0 0x7f99f4153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36486/0 (socket says 172.21.15.80:36486) 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 -- 172.21.15.80:0/2049170490 learned_addr learned my addr 172.21.15.80:0/2049170490 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f9a3e640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99f40d46a0 0x7f99f40b6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f8a3c640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99f40d26d0 0x7f99f40b7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 -- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99f40d46a0 msgr2=0x7f99f40b6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99f40d46a0 0x7f99f40b6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 -- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99f40d26d0 msgr2=0x7f99f40b7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99f40d26d0 0x7f99f40b7ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 -- 172.21.15.80:0/2049170490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f40b8430 con 0x7f99f40d14a0 2024-04-25T13:40:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f9a3e640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99f40d46a0 0x7f99f40b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99f923d640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d14a0 0x7f99f4153290 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f99f000ba80 tx=0x7f99f000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99e9ffb640 1 -- 172.21.15.80:0/2049170490 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f000cc00 con 0x7f99f40d14a0 2024-04-25T13:40:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99e9ffb640 1 -- 172.21.15.80:0/2049170490 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f000cda0 con 0x7f99f40d14a0 2024-04-25T13:40:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99e9ffb640 1 -- 172.21.15.80:0/2049170490 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f0003bc0 con 0x7f99f40d14a0 2024-04-25T13:40:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f40d0ad0 con 0x7f99f40d14a0 2024-04-25T13:40:38.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f4161380 con 0x7f99f40d14a0 2024-04-25T13:40:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.032+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f406a3a0 con 0x7f99f40d14a0 2024-04-25T13:40:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.036+0000 7f99e9ffb640 1 -- 172.21.15.80:0/2049170490 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f99f0006d10 con 0x7f99f40d14a0 2024-04-25T13:40:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.036+0000 7f99e9ffb640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99d806f400 0x7f99d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.036+0000 7f99e9ffb640 1 -- 172.21.15.80:0/2049170490 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f99f008ee00 con 0x7f99f40d14a0 2024-04-25T13:40:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.036+0000 7f99f8a3c640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99d806f400 0x7f99d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.036+0000 7f99f8a3c640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99d806f400 0x7f99d8071910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f99e4006fd0 tx=0x7f99e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.040+0000 7f99e9ffb640 1 -- 172.21.15.80:0/2049170490 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f99f40b8430 con 0x7f99f40d14a0 2024-04-25T13:40:38.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.188+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99f40d4130 con 0x7f99f40d14a0 2024-04-25T13:40:38.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99e9ffb640 1 -- 172.21.15.80:0/2049170490 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f99f005be10 con 0x7f99f40d14a0 2024-04-25T13:40:38.192 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:38.192 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:38.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99d806f400 msgr2=0x7f99d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99d806f400 0x7f99d8071910 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f99e4006fd0 tx=0x7f99e4008040 comp rx=0 tx=0).stop 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d14a0 msgr2=0x7f99f4153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d14a0 0x7f99f4153290 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f99f000ba80 tx=0x7f99f000bf40 comp rx=0 tx=0).stop 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 shutdown_connections 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99d806f400 0x7f99d8071910 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99f40d46a0 0x7f99f40b6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99f40d26d0 0x7f99f40b7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 --2- 172.21.15.80:0/2049170490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99f40d14a0 0x7f99f4153290 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 >> 172.21.15.80:0/2049170490 conn(0x7f99f4073770 msgr2=0x7f99f4157d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 shutdown_connections 2024-04-25T13:40:38.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.192+0000 7f99fb4a1640 1 -- 172.21.15.80:0/2049170490 wait complete. 2024-04-25T13:40:38.212 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:40:38.212 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:40:38.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f84381f5640 1 Processor -- start 2024-04-25T13:40:38.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f84381f5640 1 -- start start 2024-04-25T13:40:38.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f84381f5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8430071e60 0x7f84301574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f84381f5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8430161170 0x7f84300b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f84381f5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f843006fbf0 con 0x7f8430073c40 2024-04-25T13:40:38.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f84381f5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f84300b1350 con 0x7f8430071e60 2024-04-25T13:40:38.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f84381f5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f843006f9a0 con 0x7f8430161170 2024-04-25T13:40:38.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f8435790640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8430073c40 0x7f8430160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35776/0 (socket says 172.21.15.80:35776) 2024-04-25T13:40:38.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f8435790640 1 -- 172.21.15.80:0/1853365845 learned_addr learned my addr 172.21.15.80:0/1853365845 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:38.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f8435f91640 1 --2- 172.21.15.80:0/1853365845 >> v2:172.21.15.87:3300/0 conn(0x7f8430071e60 0x7f84301574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f8436792640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8430161170 0x7f84300b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.348+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 404953248 0 0) 0x7f843006fbf0 con 0x7f8430073c40 2024-04-25T13:40:38.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f841c005740 con 0x7f8430073c40 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4146411202 0 0) 0x7f841c005740 con 0x7f8430073c40 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8430161170 msgr2=0x7f84300b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8430161170 0x7f84300b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 >> v2:172.21.15.87:3300/0 conn(0x7f8430071e60 msgr2=0x7f84301574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 --2- 172.21.15.80:0/1853365845 >> v2:172.21.15.87:3300/0 conn(0x7f8430071e60 0x7f84301574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84300b0a20 con 0x7f8430073c40 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f8420001680 con 0x7f8430073c40 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 >> v1:172.21.15.80:6789/0 conn(0x7f8430073c40 legacy=0x7f8430160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f841c005bd0 0x7f841c006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c006a20 0x7f841c006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f841c007370 0x7f841c00dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f84300b1350 con 0x7f841c005bd0 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f843006f9a0 con 0x7f841c006a20 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f841c00e120 con 0x7f841c007370 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8435f91640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f841c005bd0 0x7f841c006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8435790640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c006a20 0x7f841c006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8436792640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f841c007370 0x7f841c00dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8435f91640 1 -- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f841c007370 msgr2=0x7f841c00dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8435f91640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f841c007370 0x7f841c00dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8435f91640 1 -- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c006a20 msgr2=0x7f841c006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8435f91640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c006a20 0x7f841c006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8435f91640 1 -- 172.21.15.80:0/1853365845 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c005740 con 0x7f841c005bd0 2024-04-25T13:40:38.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8436792640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f841c007370 0x7f841c00dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:38.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8435f91640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f841c005bd0 0x7f841c006040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f842c009ae0 tx=0x7f842c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75434400513faa2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0b2070 con 0x7f841c005bd0 2024-04-25T13:40:38.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842c0a5570 con 0x7f841c005bd0 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8434f8f640 1 -- 172.21.15.80:0/1853365845 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0096b0 con 0x7f841c005bd0 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f84381f5640 1 -- 172.21.15.80:0/1853365845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f841c005bd0 msgr2=0x7f841c006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f84381f5640 1 --2- 172.21.15.80:0/1853365845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f841c005bd0 0x7f841c006040 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f842c009ae0 tx=0x7f842c01cab0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f8436792640 1 -- 172.21.15.80:0/1853365845 reap_dead start 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f84381f5640 1 -- 172.21.15.80:0/1853365845 shutdown_connections 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f84381f5640 1 -- 172.21.15.80:0/1853365845 >> 172.21.15.80:0/1853365845 conn(0x7f84300b4160 msgr2=0x7f84300d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f84381f5640 1 -- 172.21.15.80:0/1853365845 shutdown_connections 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.352+0000 7f84381f5640 1 -- 172.21.15.80:0/1853365845 wait complete. 2024-04-25T13:40:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84381f5640 1 Processor -- start 2024-04-25T13:40:38.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84381f5640 1 -- start start 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84381f5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c005bd0 0x7f8430151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84381f5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84301f4a50 0x7f843014e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84381f5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f843014f050 0x7f843015cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84381f5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8430161170 con 0x7f843014f050 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84381f5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8430074050 con 0x7f841c005bd0 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84381f5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f84301612f0 con 0x7f84301f4a50 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f843014f050 0x7f843015cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f843014f050 0x7f843015cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36504/0 (socket says 172.21.15.80:36504) 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 -- 172.21.15.80:0/4177466372 learned_addr learned my addr 172.21.15.80:0/4177466372 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8435790640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84301f4a50 0x7f843014e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8435f91640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c005bd0 0x7f8430151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 -- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84301f4a50 msgr2=0x7f843014e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84301f4a50 0x7f843014e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 -- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c005bd0 msgr2=0x7f8430151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c005bd0 0x7f8430151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 -- 172.21.15.80:0/4177466372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84300b2290 con 0x7f843014f050 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f8436792640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f843014f050 0x7f843015cfb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84300d21b0 tx=0x7f842800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84267fc640 1 -- 172.21.15.80:0/4177466372 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842801c070 con 0x7f843014f050 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84267fc640 1 -- 172.21.15.80:0/4177466372 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8428017400 con 0x7f843014f050 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.356+0000 7f84267fc640 1 -- 172.21.15.80:0/4177466372 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280177f0 con 0x7f843014f050 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.360+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f843014fe50 con 0x7f843014f050 2024-04-25T13:40:38.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.360+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84301f4eb0 con 0x7f843014f050 2024-04-25T13:40:38.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.360+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8430150510 con 0x7f843014f050 2024-04-25T13:40:38.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.360+0000 7f84267fc640 1 -- 172.21.15.80:0/4177466372 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f842801f070 con 0x7f843014f050 2024-04-25T13:40:38.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.360+0000 7f84267fc640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f841406f430 0x7f8414071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.360+0000 7f84267fc640 1 -- 172.21.15.80:0/4177466372 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f842808f810 con 0x7f843014f050 2024-04-25T13:40:38.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.360+0000 7f8435f91640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f841406f430 0x7f8414071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.364+0000 7f8435f91640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f841406f430 0x7f8414071940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f842c0af000 tx=0x7f842c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.364+0000 7f84267fc640 1 -- 172.21.15.80:0/4177466372 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8428026080 con 0x7f843014f050 2024-04-25T13:40:38.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f84301f5110 con 0x7f843014f050 2024-04-25T13:40:38.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84267fc640 1 -- 172.21.15.80:0/4177466372 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f8428012070 con 0x7f843014f050 2024-04-25T13:40:38.512 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:40:38.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f841406f430 msgr2=0x7f8414071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f841406f430 0x7f8414071940 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f842c0af000 tx=0x7f842c0046d0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f843014f050 msgr2=0x7f843015cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f843014f050 0x7f843015cfb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f84300d21b0 tx=0x7f842800f4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 shutdown_connections 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f841406f430 0x7f8414071940 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f843014f050 0x7f843015cfb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84301f4a50 0x7f843014e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 --2- 172.21.15.80:0/4177466372 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f841c005bd0 0x7f8430151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 >> 172.21.15.80:0/4177466372 conn(0x7f84300b4160 msgr2=0x7f84300b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 shutdown_connections 2024-04-25T13:40:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.512+0000 7f84381f5640 1 -- 172.21.15.80:0/4177466372 wait complete. 2024-04-25T13:40:38.532 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:40:38.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.680+0000 7f433fc0c640 1 Processor -- start 2024-04-25T13:40:38.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.680+0000 7f433fc0c640 1 -- start start 2024-04-25T13:40:38.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433fc0c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f43381574c0 0x7f4338073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433fc0c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f433806fbf0 con 0x7f4338071e60 2024-04-25T13:40:38.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433fc0c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43380b0a20 con 0x7f4338161170 2024-04-25T13:40:38.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433fc0c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f43380b1350 con 0x7f43381574c0 2024-04-25T13:40:38.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433e1a9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f43381574c0 0x7f4338073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433d9a8640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4338071e60 0x7f4338160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35788/0 (socket says 172.21.15.80:35788) 2024-04-25T13:40:38.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433d9a8640 1 -- 172.21.15.80:0/1895066672 learned_addr learned my addr 172.21.15.80:0/1895066672 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:38.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433e1a9640 1 -- 172.21.15.80:0/1895066672 >> v1:172.21.15.87:6789/0 conn(0x7f4338161170 legacy=0x7f43380b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:38.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433e1a9640 1 -- 172.21.15.80:0/1895066672 >> v1:172.21.15.80:6789/0 conn(0x7f4338071e60 legacy=0x7f4338160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433e1a9640 1 -- 172.21.15.80:0/1895066672 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43380b0d00 con 0x7f43381574c0 2024-04-25T13:40:38.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.684+0000 7f433c9a6640 1 -- 172.21.15.80:0/1895066672 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 643783376 0 0) 0x7f433806fbf0 con 0x7f4338071e60 2024-04-25T13:40:38.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433e1a9640 1 --2- 172.21.15.80:0/1895066672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338073c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43380b14d0 tx=0x7f432801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0372ba66b4c37b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433c9a6640 1 -- 172.21.15.80:0/1895066672 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280b2070 con 0x7f43381574c0 2024-04-25T13:40:38.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433c9a6640 1 -- 172.21.15.80:0/1895066672 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43280a57d0 con 0x7f43381574c0 2024-04-25T13:40:38.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433c9a6640 1 -- 172.21.15.80:0/1895066672 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43280a5ba0 con 0x7f43381574c0 2024-04-25T13:40:38.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 -- 172.21.15.80:0/1895066672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 msgr2=0x7f4338073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 --2- 172.21.15.80:0/1895066672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338073c40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43380b14d0 tx=0x7f432801c910 comp rx=0 tx=0).stop 2024-04-25T13:40:38.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 -- 172.21.15.80:0/1895066672 shutdown_connections 2024-04-25T13:40:38.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 --2- 172.21.15.80:0/1895066672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338073c40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 -- 172.21.15.80:0/1895066672 >> 172.21.15.80:0/1895066672 conn(0x7f43380b4160 msgr2=0x7f43380d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:38.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 -- 172.21.15.80:0/1895066672 shutdown_connections 2024-04-25T13:40:38.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 -- 172.21.15.80:0/1895066672 wait complete. 2024-04-25T13:40:38.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 Processor -- start 2024-04-25T13:40:38.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.688+0000 7f433fc0c640 1 -- start start 2024-04-25T13:40:38.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4338071e60 0x7f433814ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f433814a480 0x7f433814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43380b56c0 con 0x7f43381574c0 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f433816b7e0 con 0x7f433814a480 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4338151840 con 0x7f4338071e60 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338149cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36524/0 (socket says 172.21.15.80:36524) 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 -- 172.21.15.80:0/976590026 learned_addr learned my addr 172.21.15.80:0/976590026 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d9a8640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4338071e60 0x7f433814ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433e1a9640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f433814a480 0x7f433814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 -- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4338071e60 msgr2=0x7f433814ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4338071e60 0x7f433814ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 -- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f433814a480 msgr2=0x7f433814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f433814a480 0x7f433814e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 -- 172.21.15.80:0/976590026 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f433814b750 con 0x7f43381574c0 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d9a8640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4338071e60 0x7f433814ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:38.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433e1a9640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f433814a480 0x7f433814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:38.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d1a7640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338149cf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f432c00b410 tx=0x7f432c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f43367fc640 1 -- 172.21.15.80:0/976590026 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c015030 con 0x7f43381574c0 2024-04-25T13:40:38.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f43367fc640 1 -- 172.21.15.80:0/976590026 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f432c004020 con 0x7f43381574c0 2024-04-25T13:40:38.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f43367fc640 1 -- 172.21.15.80:0/976590026 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432c004410 con 0x7f43381574c0 2024-04-25T13:40:38.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f433814ba30 con 0x7f43381574c0 2024-04-25T13:40:38.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f433814bff0 con 0x7f43381574c0 2024-04-25T13:40:38.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f433806a3a0 con 0x7f43381574c0 2024-04-25T13:40:38.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f43367fc640 1 -- 172.21.15.80:0/976590026 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f432c018070 con 0x7f43381574c0 2024-04-25T13:40:38.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f43367fc640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f432406f400 0x7f4324071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:38.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f43367fc640 1 -- 172.21.15.80:0/976590026 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f432c08ed50 con 0x7f43381574c0 2024-04-25T13:40:38.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.692+0000 7f433d9a8640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f432406f400 0x7f4324071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:38.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.696+0000 7f433d9a8640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f432406f400 0x7f4324071910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4330007fd0 tx=0x7f4330009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:38.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.700+0000 7f43367fc640 1 -- 172.21.15.80:0/976590026 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f432c092030 con 0x7f43381574c0 2024-04-25T13:40:38.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.856+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4338073d40 con 0x7f43381574c0 2024-04-25T13:40:38.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f43367fc640 1 -- 172.21.15.80:0/976590026 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f432c05bd60 con 0x7f43381574c0 2024-04-25T13:40:38.859 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:40:38.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f432406f400 msgr2=0x7f4324071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f432406f400 0x7f4324071910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4330007fd0 tx=0x7f4330009040 comp rx=0 tx=0).stop 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 msgr2=0x7f4338149cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338149cf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f432c00b410 tx=0x7f432c00b8d0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 shutdown_connections 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f432406f400 0x7f4324071910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f433814a480 0x7f433814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43381574c0 0x7f4338149cf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 --2- 172.21.15.80:0/976590026 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4338071e60 0x7f433814ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 >> 172.21.15.80:0/976590026 conn(0x7f43380b4160 msgr2=0x7f433815ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 shutdown_connections 2024-04-25T13:40:38.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:38.860+0000 7f433fc0c640 1 -- 172.21.15.80:0/976590026 wait complete. 2024-04-25T13:40:38.881 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:40:39.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa582184640 1 Processor -- start 2024-04-25T13:40:39.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa582184640 1 -- start start 2024-04-25T13:40:39.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa582184640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa57c0b7010 0x7fa57c162750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa582184640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c0b4cc0 con 0x7fa57c162c90 2024-04-25T13:40:39.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa582184640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa57c0cfd50 con 0x7fa57c0cf940 2024-04-25T13:40:39.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa582184640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa57c0b4fe0 con 0x7fa57c0b7010 2024-04-25T13:40:39.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa57bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa57c162c90 0x7fa57c0d1c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35794/0 (socket says 172.21.15.80:35794) 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa57bfff640 1 -- 172.21.15.80:0/4021083396 learned_addr learned my addr 172.21.15.80:0/4021083396 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.036+0000 7fa57affd640 1 --2- 172.21.15.80:0/4021083396 >> v2:172.21.15.87:3300/0 conn(0x7fa57c0b7010 0x7fa57c162750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3690611696 0 0) 0x7fa57c0b4cc0 con 0x7fa57c162c90 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa564003530 con 0x7fa57c162c90 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3073882 0 0) 0x7fa564003530 con 0x7fa57c162c90 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 >> v2:172.21.15.87:3300/0 conn(0x7fa57c0b7010 msgr2=0x7fa57c162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 --2- 172.21.15.80:0/4021083396 >> v2:172.21.15.87:3300/0 conn(0x7fa57c0b7010 0x7fa57c162750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 >> v1:172.21.15.87:6789/0 conn(0x7fa57c0cf940 legacy=0x7fa57c0b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c0b5230 con 0x7fa57c162c90 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fa574001a20 con 0x7fa57c162c90 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 >> v1:172.21.15.80:6789/0 conn(0x7fa57c162c90 legacy=0x7fa57c0d1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5640039c0 0x7fa564003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa564004810 0x7fa564004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa564005260 0x7fa56400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c0b4fe0 con 0x7fa5640039c0 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa56400c1f0 con 0x7fa564004810 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa56400c3f0 con 0x7fa564005260 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57b7fe640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5640039c0 0x7fa564003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57affd640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa564004810 0x7fa564004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57b7fe640 1 -- 172.21.15.80:0/4021083396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa564005260 msgr2=0x7fa56400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57bfff640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa564005260 0x7fa56400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57b7fe640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa564005260 0x7fa56400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57b7fe640 1 -- 172.21.15.80:0/4021083396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa564004810 msgr2=0x7fa564004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57b7fe640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa564004810 0x7fa564004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57b7fe640 1 -- 172.21.15.80:0/4021083396 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa564003530 con 0x7fa5640039c0 2024-04-25T13:40:39.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57bfff640 1 -- 172.21.15.80:0/4021083396 reap_dead start 2024-04-25T13:40:39.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57b7fe640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5640039c0 0x7fa564003e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa56c001780 tx=0x7fa56c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8129e37b45860296 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c0013a0 con 0x7fa5640039c0 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa56c0052f0 con 0x7fa5640039c0 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.040+0000 7fa57a7fc640 1 -- 172.21.15.80:0/4021083396 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa56c003840 con 0x7fa5640039c0 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- 172.21.15.80:0/4021083396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5640039c0 msgr2=0x7fa564003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5640039c0 0x7fa564003e30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa56c001780 tx=0x7fa56c01c400 comp rx=0 tx=0).stop 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- 172.21.15.80:0/4021083396 shutdown_connections 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 --2- 172.21.15.80:0/4021083396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5640039c0 0x7fa564003e30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- 172.21.15.80:0/4021083396 >> 172.21.15.80:0/4021083396 conn(0x7fa57c0d3ab0 msgr2=0x7fa57c0d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- 172.21.15.80:0/4021083396 shutdown_connections 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- 172.21.15.80:0/4021083396 wait complete. 2024-04-25T13:40:39.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 Processor -- start 2024-04-25T13:40:39.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- start start 2024-04-25T13:40:39.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5640039c0 0x7fa57c152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa57c15df40 0x7fa57c15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa57c15bdb0 0x7fa57c15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c0b7420 con 0x7fa57c15bdb0 2024-04-25T13:40:39.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa57c0cf1d0 con 0x7fa57c15df40 2024-04-25T13:40:39.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa582184640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa57c0b75a0 con 0x7fa5640039c0 2024-04-25T13:40:39.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa57c15bdb0 0x7fa57c15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa57c15bdb0 0x7fa57c15f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36534/0 (socket says 172.21.15.80:36534) 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57bfff640 1 -- 172.21.15.80:0/2067674162 learned_addr learned my addr 172.21.15.80:0/2067674162 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57b7fe640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5640039c0 0x7fa57c152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57affd640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa57c15df40 0x7fa57c15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57bfff640 1 -- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5640039c0 msgr2=0x7fa57c152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57bfff640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5640039c0 0x7fa57c152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57bfff640 1 -- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa57c15df40 msgr2=0x7fa57c15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57bfff640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa57c15df40 0x7fa57c15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.044+0000 7fa57bfff640 1 -- 172.21.15.80:0/2067674162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57c15cea0 con 0x7fa57c15bdb0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa57bfff640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa57c15bdb0 0x7fa57c15f5e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa574000f90 tx=0x7fa57400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa563fff640 1 -- 172.21.15.80:0/2067674162 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa57400bb80 con 0x7fa57c15bdb0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa563fff640 1 -- 172.21.15.80:0/2067674162 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa57400bd20 con 0x7fa57c15bdb0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa57c15d180 con 0x7fa57c15bdb0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa563fff640 1 -- 172.21.15.80:0/2067674162 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5740072b0 con 0x7fa57c15bdb0 2024-04-25T13:40:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa57c15d740 con 0x7fa57c15bdb0 2024-04-25T13:40:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa57c15cc10 con 0x7fa57c15bdb0 2024-04-25T13:40:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa563fff640 1 -- 172.21.15.80:0/2067674162 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa57401b060 con 0x7fa57c15bdb0 2024-04-25T13:40:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa563fff640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa56806f400 0x7fa568071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa563fff640 1 -- 172.21.15.80:0/2067674162 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa57408f220 con 0x7fa57c15bdb0 2024-04-25T13:40:39.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa57b7fe640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa56806f400 0x7fa568071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.048+0000 7fa57b7fe640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa56806f400 0x7fa568071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa56c001750 tx=0x7fa56c0011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.052+0000 7fa563fff640 1 -- 172.21.15.80:0/2067674162 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa57405c1a0 con 0x7fa57c15bdb0 2024-04-25T13:40:39.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.208+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa57c070260 con 0x7fa57c15bdb0 2024-04-25T13:40:39.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa563fff640 1 -- 172.21.15.80:0/2067674162 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fa574064190 con 0x7fa57c15bdb0 2024-04-25T13:40:39.218 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:40:39.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa56806f400 msgr2=0x7fa568071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa56806f400 0x7fa568071910 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa56c001750 tx=0x7fa56c0011a0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa57c15bdb0 msgr2=0x7fa57c15f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa57c15bdb0 0x7fa57c15f5e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa574000f90 tx=0x7fa57400b4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 shutdown_connections 2024-04-25T13:40:39.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa56806f400 0x7fa568071910 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa57c15bdb0 0x7fa57c15f5e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa57c15df40 0x7fa57c15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 --2- 172.21.15.80:0/2067674162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5640039c0 0x7fa57c152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 >> 172.21.15.80:0/2067674162 conn(0x7fa57c0d3ab0 msgr2=0x7fa57c161f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 shutdown_connections 2024-04-25T13:40:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.212+0000 7fa582184640 1 -- 172.21.15.80:0/2067674162 wait complete. 2024-04-25T13:40:39.231 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:40:39.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39bbe29640 1 Processor -- start 2024-04-25T13:40:39.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39bbe29640 1 -- start start 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39bbe29640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b406f5a0 0x7f39b4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39bbe29640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b406fcf0 con 0x7f39b4161070 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39bbe29640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39b40b59f0 con 0x7f39b40733b0 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39bbe29640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39b4073f10 con 0x7f39b406f5a0 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b93c4640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f39b4161070 0x7f39b40b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35802/0 (socket says 172.21.15.80:35802) 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b93c4640 1 -- 172.21.15.80:0/1486075922 learned_addr learned my addr 172.21.15.80:0/1486075922 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 423771902 0 0) 0x7f39b406fcf0 con 0x7f39b4161070 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39a0003530 con 0x7f39b4161070 2024-04-25T13:40:39.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b9bc5640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b406f5a0 0x7f39b4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3019074221 0 0) 0x7f39a0003530 con 0x7f39b4161070 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b406f5a0 msgr2=0x7f39b4160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b406f5a0 0x7f39b4160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 >> v1:172.21.15.87:6789/0 conn(0x7f39b40733b0 legacy=0x7f39b40b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b40b0c60 con 0x7f39b4161070 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f39a4001680 con 0x7f39b4161070 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 >> v1:172.21.15.80:6789/0 conn(0x7f39b4161070 legacy=0x7f39b40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a00039f0 0x7f39a0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b406f5a0 0x7f39a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a0004d50 0x7f39a0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39b4073f10 con 0x7f39a00039f0 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39a000bb90 con 0x7f39a0004d50 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39a000bd90 con 0x7f39b406f5a0 2024-04-25T13:40:39.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b9bc5640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a00039f0 0x7f39a0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b93c4640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b406f5a0 0x7f39a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b9bc5640 1 -- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b406f5a0 msgr2=0x7f39a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b9bc5640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39b406f5a0 0x7f39a0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39ba3c6640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a0004d50 0x7f39a0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b9bc5640 1 -- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a0004d50 msgr2=0x7f39a0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b9bc5640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39a0004d50 0x7f39a0005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.364+0000 7f39b9bc5640 1 -- 172.21.15.80:0/1486075922 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a0003530 con 0x7f39a00039f0 2024-04-25T13:40:39.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39b9bc5640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a00039f0 0x7f39a0003e60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009ab0 tx=0x7f39ac01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8033588355e063 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0096b0 con 0x7f39a00039f0 2024-04-25T13:40:39.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39ac0a6450 con 0x7f39a00039f0 2024-04-25T13:40:39.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39b8bc3640 1 -- 172.21.15.80:0/1486075922 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ac0a6800 con 0x7f39a00039f0 2024-04-25T13:40:39.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39bbe29640 1 -- 172.21.15.80:0/1486075922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a00039f0 msgr2=0x7f39a0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39bbe29640 1 --2- 172.21.15.80:0/1486075922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39a00039f0 0x7f39a0003e60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009ab0 tx=0x7f39ac01ca70 comp rx=0 tx=0).stop 2024-04-25T13:40:39.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39ba3c6640 1 -- 172.21.15.80:0/1486075922 reap_dead start 2024-04-25T13:40:39.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39bbe29640 1 -- 172.21.15.80:0/1486075922 shutdown_connections 2024-04-25T13:40:39.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39bbe29640 1 -- 172.21.15.80:0/1486075922 >> 172.21.15.80:0/1486075922 conn(0x7f39b406d080 msgr2=0x7f39b40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:39.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39bbe29640 1 -- 172.21.15.80:0/1486075922 shutdown_connections 2024-04-25T13:40:39.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.368+0000 7f39bbe29640 1 -- 172.21.15.80:0/1486075922 wait complete. 2024-04-25T13:40:39.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 Processor -- start 2024-04-25T13:40:39.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 -- start start 2024-04-25T13:40:39.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a00039f0 0x7f39b40d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b41526e0 0x7f39b414afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b4156180 0x7f39b4156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39b41696e0 con 0x7f39b4156180 2024-04-25T13:40:39.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39b4161890 con 0x7f39b41526e0 2024-04-25T13:40:39.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39b4169860 con 0x7f39a00039f0 2024-04-25T13:40:39.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b4156180 0x7f39b4156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b4156180 0x7f39b4156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36560/0 (socket says 172.21.15.80:36560) 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 -- 172.21.15.80:0/677443342 learned_addr learned my addr 172.21.15.80:0/677443342 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39b93c4640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b41526e0 0x7f39b414afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39b9bc5640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a00039f0 0x7f39b40d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 -- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a00039f0 msgr2=0x7f39b40d0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a00039f0 0x7f39b40d0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 -- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b41526e0 msgr2=0x7f39b414afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b41526e0 0x7f39b414afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 -- 172.21.15.80:0/677443342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b4156ad0 con 0x7f39b4156180 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39ba3c6640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b4156180 0x7f39b4156590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f39b40cdef0 tx=0x7f39b000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39aa7fc640 1 -- 172.21.15.80:0/677443342 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0018070 con 0x7f39b4156180 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39aa7fc640 1 -- 172.21.15.80:0/677443342 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39b0003b80 con 0x7f39b4156180 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b414f040 con 0x7f39b4156180 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39aa7fc640 1 -- 172.21.15.80:0/677443342 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b0005070 con 0x7f39b4156180 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b40b2a10 con 0x7f39b4156180 2024-04-25T13:40:39.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.372+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b414f210 con 0x7f39b4156180 2024-04-25T13:40:39.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.376+0000 7f39aa7fc640 1 -- 172.21.15.80:0/677443342 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f39b001b070 con 0x7f39b4156180 2024-04-25T13:40:39.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.376+0000 7f39aa7fc640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f399806f430 0x7f3998071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.376+0000 7f39aa7fc640 1 -- 172.21.15.80:0/677443342 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f39b008eb00 con 0x7f39b4156180 2024-04-25T13:40:39.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.376+0000 7f39b9bc5640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f399806f430 0x7f3998071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.376+0000 7f39b9bc5640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f399806f430 0x7f3998071940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0ad000 tx=0x7f39ac0b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.380+0000 7f39aa7fc640 1 -- 172.21.15.80:0/677443342 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f39b4156ad0 con 0x7f39b4156180 2024-04-25T13:40:39.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.560+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39b406a3a0 con 0x7f39b4156180 2024-04-25T13:40:39.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.560+0000 7f39aa7fc640 1 -- 172.21.15.80:0/677443342 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f39b005ba80 con 0x7f39b4156180 2024-04-25T13:40:39.561 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:39.561 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:40:29.558137+0000","modified":"2024-04-25T13:40:32.595776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-25T13:40:39.562 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f399806f430 msgr2=0x7f3998071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f399806f430 0x7f3998071940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39ac0ad000 tx=0x7f39ac0b2040 comp rx=0 tx=0).stop 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b4156180 msgr2=0x7f39b4156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b4156180 0x7f39b4156590 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f39b40cdef0 tx=0x7f39b000dc00 comp rx=0 tx=0).stop 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 shutdown_connections 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f399806f430 0x7f3998071940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39b4156180 0x7f39b4156590 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39b41526e0 0x7f39b414afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 --2- 172.21.15.80:0/677443342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39a00039f0 0x7f39b40d0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 >> 172.21.15.80:0/677443342 conn(0x7f39b406d080 msgr2=0x7f39b40b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 shutdown_connections 2024-04-25T13:40:39.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.564+0000 7f39bbe29640 1 -- 172.21.15.80:0/677443342 wait complete. 2024-04-25T13:40:39.582 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:39.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f1891442640 1 Processor -- start 2024-04-25T13:40:39.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f1891442640 1 -- start start 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f1891442640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 0x7f188c0b5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f1891442640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f188c0b4c30 0x7f188c0b5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f1891442640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f188c0b6530 con 0x7f188c071e80 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f1891442640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f188c15d870 con 0x7f188c0b4c30 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f1891442640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f188c15db50 con 0x7f188c0b5be0 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f188c071e80 0x7f188c06f5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35810/0 (socket says 172.21.15.80:35810) 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188affd640 1 -- 172.21.15.80:0/1363648844 learned_addr learned my addr 172.21.15.80:0/1363648844 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188b7fe640 1 --2- 172.21.15.80:0/1363648844 >> v2:172.21.15.87:3300/0 conn(0x7f188c0b4c30 0x7f188c0b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1701449174 0 0) 0x7f187c001120 con 0x7f188c071e80 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f18827fc640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 0x7f188c0b5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1878005740 con 0x7f188c071e80 2024-04-25T13:40:39.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3310639039 0 0) 0x7f1878005740 con 0x7f188c071e80 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 msgr2=0x7f188c0b5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 0x7f188c0b5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 >> v2:172.21.15.87:3300/0 conn(0x7f188c0b4c30 msgr2=0x7f188c0b5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 --2- 172.21.15.80:0/1363648844 >> v2:172.21.15.87:3300/0 conn(0x7f188c0b4c30 0x7f188c0b5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c15de30 con 0x7f188c071e80 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f187c001850 con 0x7f188c071e80 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f18827fc640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 0x7f188c0b5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 >> v1:172.21.15.80:6789/0 conn(0x7f188c071e80 legacy=0x7f188c06f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1878005bd0 0x7f1878006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 0x7f1878006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f1878007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f188c15d870 con 0x7f1878006f60 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f188c15db50 con 0x7f1878005bd0 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f187800dd10 con 0x7f188c0b5be0 2024-04-25T13:40:39.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188b7fe640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f1878007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f18827fc640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 0x7f1878006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188affd640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1878005bd0 0x7f1878006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188b7fe640 1 -- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 msgr2=0x7f1878006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188b7fe640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0b5be0 0x7f1878006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188b7fe640 1 -- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1878005bd0 msgr2=0x7f1878006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:39.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188b7fe640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1878005bd0 0x7f1878006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.724+0000 7f188b7fe640 1 -- 172.21.15.80:0/1363648844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1878005740 con 0x7f1878006f60 2024-04-25T13:40:39.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f188b7fe640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f1878007370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1884009fc0 tx=0x7f188401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c07d4993c44084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18840b2070 con 0x7f1878006f60 2024-04-25T13:40:39.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18840a5cd0 con 0x7f1878006f60 2024-04-25T13:40:39.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f188a7fc640 1 -- 172.21.15.80:0/1363648844 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1884009c10 con 0x7f1878006f60 2024-04-25T13:40:39.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f1891442640 1 -- 172.21.15.80:0/1363648844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 msgr2=0x7f1878007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f1891442640 1 --2- 172.21.15.80:0/1363648844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f1878007370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1884009fc0 tx=0x7f188401ccc0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f188b7fe640 1 -- 172.21.15.80:0/1363648844 reap_dead start 2024-04-25T13:40:39.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f1891442640 1 -- 172.21.15.80:0/1363648844 shutdown_connections 2024-04-25T13:40:39.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f1891442640 1 -- 172.21.15.80:0/1363648844 >> 172.21.15.80:0/1363648844 conn(0x7f188c06d080 msgr2=0x7f188c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:39.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f1891442640 1 -- 172.21.15.80:0/1363648844 shutdown_connections 2024-04-25T13:40:39.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.728+0000 7f1891442640 1 -- 172.21.15.80:0/1363648844 wait complete. 2024-04-25T13:40:39.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 Processor -- start 2024-04-25T13:40:39.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 -- start start 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f188c0cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c0d1830 0x7f188c0d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c154840 0x7f188c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f188c15d490 con 0x7f1878006f60 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f188c06fcb0 con 0x7f188c0d1830 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f188c15d610 con 0x7f188c154840 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f188c0cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f188c0cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36576/0 (socket says 172.21.15.80:36576) 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 -- 172.21.15.80:0/3335106733 learned_addr learned my addr 172.21.15.80:0/3335106733 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f18827fc640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c0d1830 0x7f188c0d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188b7fe640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c154840 0x7f188c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 -- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c154840 msgr2=0x7f188c154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c154840 0x7f188c154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 -- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c0d1830 msgr2=0x7f188c0d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c0d1830 0x7f188c0d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 -- 172.21.15.80:0/3335106733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c155190 con 0x7f1878006f60 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f188affd640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f188c0cfc30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f187c011870 tx=0x7f187c011d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1883fff640 1 -- 172.21.15.80:0/3335106733 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c010040 con 0x7f1878006f60 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1883fff640 1 -- 172.21.15.80:0/3335106733 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f187c01a070 con 0x7f1878006f60 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188c13fa70 con 0x7f1878006f60 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1883fff640 1 -- 172.21.15.80:0/3335106733 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00fdc0 con 0x7f1878006f60 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f188c158af0 con 0x7f1878006f60 2024-04-25T13:40:39.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.732+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f188c13fc40 con 0x7f1878006f60 2024-04-25T13:40:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.736+0000 7f1883fff640 1 -- 172.21.15.80:0/3335106733 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f187c01f460 con 0x7f1878006f60 2024-04-25T13:40:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.736+0000 7f1883fff640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f186c06f400 0x7f186c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.736+0000 7f1883fff640 1 -- 172.21.15.80:0/3335106733 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f187c0923b0 con 0x7f1878006f60 2024-04-25T13:40:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.736+0000 7f18827fc640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f186c06f400 0x7f186c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:39.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.736+0000 7f18827fc640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f186c06f400 0x7f186c071910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f188c15fb90 tx=0x7f1874008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:39.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.740+0000 7f1883fff640 1 -- 172.21.15.80:0/3335106733 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f188c155190 con 0x7f1878006f60 2024-04-25T13:40:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.884+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f188c06a3a0 con 0x7f1878006f60 2024-04-25T13:40:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.888+0000 7f1883fff640 1 -- 172.21.15.80:0/3335106733 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f188c0b6530 con 0x7f1878006f60 2024-04-25T13:40:39.889 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:39.889 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f186c06f400 msgr2=0x7f186c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f186c06f400 0x7f186c071910 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f188c15fb90 tx=0x7f1874008040 comp rx=0 tx=0).stop 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 msgr2=0x7f188c0cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f188c0cfc30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f187c011870 tx=0x7f187c011d30 comp rx=0 tx=0).stop 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 shutdown_connections 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f186c06f400 0x7f186c071910 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c154840 0x7f188c154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c0d1830 0x7f188c0d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 --2- 172.21.15.80:0/3335106733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1878006f60 0x7f188c0cfc30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:39.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 >> 172.21.15.80:0/3335106733 conn(0x7f188c06d080 msgr2=0x7f188c156020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:39.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 shutdown_connections 2024-04-25T13:40:39.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:39.892+0000 7f1891442640 1 -- 172.21.15.80:0/3335106733 wait complete. 2024-04-25T13:40:39.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:40:39.909 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:40:40.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69e55ab640 1 Processor -- start 2024-04-25T13:40:40.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69e55ab640 1 -- start start 2024-04-25T13:40:40.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69e55ab640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f69e006edb0 0x7f69e006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69e55ab640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f69e0161b60 0x7f69e0161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69e55ab640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e01624b0 con 0x7f69e00b5330 2024-04-25T13:40:40.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69e55ab640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f69e006f700 con 0x7f69e0161b60 2024-04-25T13:40:40.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69e55ab640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f69e01629b0 con 0x7f69e006edb0 2024-04-25T13:40:40.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69df7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f69e0161b60 0x7f69e0161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69df7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f69e0161b60 0x7f69e0161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36584/0 (socket says 172.21.15.80:36584) 2024-04-25T13:40:40.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69df7fe640 1 -- 172.21.15.80:0/821393047 learned_addr learned my addr 172.21.15.80:0/821393047 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:40.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69d7fff640 1 --2- 172.21.15.80:0/821393047 >> v2:172.21.15.87:3300/0 conn(0x7f69e006edb0 0x7f69e006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69df7fe640 1 -- 172.21.15.80:0/821393047 >> v2:172.21.15.87:3300/0 conn(0x7f69e006edb0 msgr2=0x7f69e006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:40.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69df7fe640 1 --2- 172.21.15.80:0/821393047 >> v2:172.21.15.87:3300/0 conn(0x7f69e006edb0 0x7f69e006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69df7fe640 1 -- 172.21.15.80:0/821393047 >> v1:172.21.15.87:6789/0 conn(0x7f69e00b5330 legacy=0x7f69e00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.064+0000 7f69df7fe640 1 -- 172.21.15.80:0/821393047 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e00b5740 con 0x7f69e0161b60 2024-04-25T13:40:40.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69df7fe640 1 --2- 172.21.15.80:0/821393047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0161f70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f69e0074290 tx=0x7f69d801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5f664370e876b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69de7fc640 1 -- 172.21.15.80:0/821393047 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80a57f0 con 0x7f69e0161b60 2024-04-25T13:40:40.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69de7fc640 1 -- 172.21.15.80:0/821393047 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d80a5990 con 0x7f69e0161b60 2024-04-25T13:40:40.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69de7fc640 1 -- 172.21.15.80:0/821393047 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80a5d60 con 0x7f69e0161b60 2024-04-25T13:40:40.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69e55ab640 1 -- 172.21.15.80:0/821393047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 msgr2=0x7f69e0161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69e55ab640 1 --2- 172.21.15.80:0/821393047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0161f70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f69e0074290 tx=0x7f69d801c930 comp rx=0 tx=0).stop 2024-04-25T13:40:40.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69e55ab640 1 -- 172.21.15.80:0/821393047 shutdown_connections 2024-04-25T13:40:40.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69e55ab640 1 --2- 172.21.15.80:0/821393047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0161f70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69e55ab640 1 --2- 172.21.15.80:0/821393047 >> v2:172.21.15.87:3300/0 conn(0x7f69e006edb0 0x7f69e006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69e55ab640 1 -- 172.21.15.80:0/821393047 >> 172.21.15.80:0/821393047 conn(0x7f69e00b4160 msgr2=0x7f69e00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:40.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69e55ab640 1 -- 172.21.15.80:0/821393047 shutdown_connections 2024-04-25T13:40:40.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.068+0000 7f69e55ab640 1 -- 172.21.15.80:0/821393047 wait complete. 2024-04-25T13:40:40.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 Processor -- start 2024-04-25T13:40:40.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 -- start start 2024-04-25T13:40:40.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69e006edb0 0x7f69e0159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00b5330 0x7f69e01556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0155c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f69e006e890 con 0x7f69e0161b60 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f69e006e5b0 con 0x7f69e00b5330 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f69e00b0cd0 con 0x7f69e006edb0 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36586/0 (socket says 172.21.15.80:36586) 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 -- 172.21.15.80:0/45124536 learned_addr learned my addr 172.21.15.80:0/45124536 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69deffd640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69e006edb0 0x7f69e0159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69d7fff640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00b5330 0x7f69e01556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 -- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69e006edb0 msgr2=0x7f69e0159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69e006edb0 0x7f69e0159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 -- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00b5330 msgr2=0x7f69e01556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00b5330 0x7f69e01556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 -- 172.21.15.80:0/45124536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e014fd20 con 0x7f69e0161b60 2024-04-25T13:40:40.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69deffd640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69e006edb0 0x7f69e0159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:40.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69d7fff640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00b5330 0x7f69e01556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:40.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69df7fe640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0155c10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f69d80af040 tx=0x7f69d80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69d77fe640 1 -- 172.21.15.80:0/45124536 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80b6640 con 0x7f69e0161b60 2024-04-25T13:40:40.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69d77fe640 1 -- 172.21.15.80:0/45124536 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d80bd070 con 0x7f69e0161b60 2024-04-25T13:40:40.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e0150000 con 0x7f69e0161b60 2024-04-25T13:40:40.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69d77fe640 1 -- 172.21.15.80:0/45124536 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d80b6a40 con 0x7f69e0161b60 2024-04-25T13:40:40.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.072+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e01505c0 con 0x7f69e0161b60 2024-04-25T13:40:40.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.076+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e006a3a0 con 0x7f69e0161b60 2024-04-25T13:40:40.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.076+0000 7f69d77fe640 1 -- 172.21.15.80:0/45124536 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f69d80b6be0 con 0x7f69e0161b60 2024-04-25T13:40:40.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.076+0000 7f69d77fe640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f69c006f400 0x7f69c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.076+0000 7f69d77fe640 1 -- 172.21.15.80:0/45124536 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f69d812af90 con 0x7f69e0161b60 2024-04-25T13:40:40.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.076+0000 7f69deffd640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f69c006f400 0x7f69c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.076+0000 7f69deffd640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f69c006f400 0x7f69c0071910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69d0001650 tx=0x7f69d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.080+0000 7f69d77fe640 1 -- 172.21.15.80:0/45124536 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69d80165e0 con 0x7f69e0161b60 2024-04-25T13:40:40.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.232+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69e006fd00 con 0x7f69e0161b60 2024-04-25T13:40:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.232+0000 7f69d77fe640 1 -- 172.21.15.80:0/45124536 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f69d80f7e80 con 0x7f69e0161b60 2024-04-25T13:40:40.234 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:40:40.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f69c006f400 msgr2=0x7f69c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f69c006f400 0x7f69c0071910 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69d0001650 tx=0x7f69d0009040 comp rx=0 tx=0).stop 2024-04-25T13:40:40.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 msgr2=0x7f69e0155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0155c10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f69d80af040 tx=0x7f69d80196d0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 shutdown_connections 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f69c006f400 0x7f69c0071910 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69e0161b60 0x7f69e0155c10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69e00b5330 0x7f69e01556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 --2- 172.21.15.80:0/45124536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69e006edb0 0x7f69e0159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 >> 172.21.15.80:0/45124536 conn(0x7f69e00b4160 msgr2=0x7f69e0157d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 shutdown_connections 2024-04-25T13:40:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.236+0000 7f69e55ab640 1 -- 172.21.15.80:0/45124536 wait complete. 2024-04-25T13:40:40.255 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:40:40.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.396+0000 7f7d14fd3640 1 Processor -- start 2024-04-25T13:40:40.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.396+0000 7f7d14fd3640 1 -- start start 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d14fd3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d1006edb0 0x7f7d1006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d14fd3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7d10161b60 0x7f7d10161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d14fd3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d101624b0 con 0x7f7d100b5330 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d14fd3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7d1006f700 con 0x7f7d10161b60 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d14fd3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7d101629b0 con 0x7f7d1006edb0 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d0ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7d10161b60 0x7f7d10161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d0ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7d10161b60 0x7f7d10161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36598/0 (socket says 172.21.15.80:36598) 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d0ed9d640 1 -- 172.21.15.80:0/492286668 learned_addr learned my addr 172.21.15.80:0/492286668 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d0dd9b640 1 --2- 172.21.15.80:0/492286668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d1006edb0 0x7f7d1006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d0ed9d640 1 -- 172.21.15.80:0/492286668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d1006edb0 msgr2=0x7f7d1006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d0ed9d640 1 --2- 172.21.15.80:0/492286668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d1006edb0 0x7f7d1006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d0ed9d640 1 -- 172.21.15.80:0/492286668 >> v1:172.21.15.87:6789/0 conn(0x7f7d100b5330 legacy=0x7f7d100d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:40.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.400+0000 7f7d0ed9d640 1 -- 172.21.15.80:0/492286668 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d100b5740 con 0x7f7d10161b60 2024-04-25T13:40:40.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d0ed9d640 1 --2- 172.21.15.80:0/492286668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d10161b60 0x7f7d10161f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009e50 tx=0x7f7d0801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78d579a02b27db2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d0d59a640 1 -- 172.21.15.80:0/492286668 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080a55d0 con 0x7f7d10161b60 2024-04-25T13:40:40.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d0d59a640 1 -- 172.21.15.80:0/492286668 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d080a5770 con 0x7f7d10161b60 2024-04-25T13:40:40.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d0d59a640 1 -- 172.21.15.80:0/492286668 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d080a5b40 con 0x7f7d10161b60 2024-04-25T13:40:40.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 -- 172.21.15.80:0/492286668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d10161b60 msgr2=0x7f7d10161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/492286668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d10161b60 0x7f7d10161f70 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7d08009e50 tx=0x7f7d0801c750 comp rx=0 tx=0).stop 2024-04-25T13:40:40.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 -- 172.21.15.80:0/492286668 shutdown_connections 2024-04-25T13:40:40.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/492286668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d10161b60 0x7f7d10161f70 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/492286668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d1006edb0 0x7f7d1006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 -- 172.21.15.80:0/492286668 >> 172.21.15.80:0/492286668 conn(0x7f7d100b4160 msgr2=0x7f7d100d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:40.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 -- 172.21.15.80:0/492286668 shutdown_connections 2024-04-25T13:40:40.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 -- 172.21.15.80:0/492286668 wait complete. 2024-04-25T13:40:40.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 Processor -- start 2024-04-25T13:40:40.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.404+0000 7f7d14fd3640 1 -- start start 2024-04-25T13:40:40.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d1006edb0 0x7f7d101594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d100b5330 0x7f7d10158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d10161b60 0x7f7d10147590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1006e890 con 0x7f7d1006edb0 2024-04-25T13:40:40.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1006e5b0 con 0x7f7d10161b60 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7d100b0cd0 con 0x7f7d100b5330 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d1006edb0 0x7f7d101594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d1006edb0 0x7f7d101594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36612/0 (socket says 172.21.15.80:36612) 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 -- 172.21.15.80:0/1354748725 learned_addr learned my addr 172.21.15.80:0/1354748725 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0dd9b640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d100b5330 0x7f7d10158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0ed9d640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d10161b60 0x7f7d10147590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 -- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d100b5330 msgr2=0x7f7d10158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d100b5330 0x7f7d10158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 -- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d10161b60 msgr2=0x7f7d10147590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d10161b60 0x7f7d10147590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 -- 172.21.15.80:0/1354748725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d10147ad0 con 0x7f7d1006edb0 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d0e59c640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d1006edb0 0x7f7d101594d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000b440 tx=0x7f7d0000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7cfeffd640 1 -- 172.21.15.80:0/1354748725 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00015020 con 0x7f7d1006edb0 2024-04-25T13:40:40.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7cfeffd640 1 -- 172.21.15.80:0/1354748725 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d00011830 con 0x7f7d1006edb0 2024-04-25T13:40:40.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7cfeffd640 1 -- 172.21.15.80:0/1354748725 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d00011c20 con 0x7f7d1006edb0 2024-04-25T13:40:40.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1015b0f0 con 0x7f7d1006edb0 2024-04-25T13:40:40.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1015b390 con 0x7f7d1006edb0 2024-04-25T13:40:40.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d1015ae50 con 0x7f7d1006edb0 2024-04-25T13:40:40.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7cfeffd640 1 -- 172.21.15.80:0/1354748725 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7d00018020 con 0x7f7d1006edb0 2024-04-25T13:40:40.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7cfeffd640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7cf006f400 0x7f7cf0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.408+0000 7f7cfeffd640 1 -- 172.21.15.80:0/1354748725 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7d0008ef30 con 0x7f7d1006edb0 2024-04-25T13:40:40.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.412+0000 7f7d0dd9b640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7cf006f400 0x7f7cf0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.412+0000 7f7d0dd9b640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7cf006f400 0x7f7cf0071910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d10168ec0 tx=0x7f7d04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.416+0000 7f7cfeffd640 1 -- 172.21.15.80:0/1354748725 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7d0001f030 con 0x7f7d1006edb0 2024-04-25T13:40:40.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.560+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7d1006a3a0 con 0x7f7d1006edb0 2024-04-25T13:40:40.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7cfeffd640 1 -- 172.21.15.80:0/1354748725 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f7d10147ad0 con 0x7f7d1006edb0 2024-04-25T13:40:40.563 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:40:40.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7cf006f400 msgr2=0x7f7cf0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7cf006f400 0x7f7cf0071910 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7d10168ec0 tx=0x7f7d04008040 comp rx=0 tx=0).stop 2024-04-25T13:40:40.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d1006edb0 msgr2=0x7f7d101594d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d1006edb0 0x7f7d101594d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7d0000b440 tx=0x7f7d0000b900 comp rx=0 tx=0).stop 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 shutdown_connections 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7cf006f400 0x7f7cf0071910 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d10161b60 0x7f7d10147590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d100b5330 0x7f7d10158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 --2- 172.21.15.80:0/1354748725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d1006edb0 0x7f7d101594d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 >> 172.21.15.80:0/1354748725 conn(0x7f7d100b4160 msgr2=0x7f7d1006cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 shutdown_connections 2024-04-25T13:40:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.564+0000 7f7d14fd3640 1 -- 172.21.15.80:0/1354748725 wait complete. 2024-04-25T13:40:40.583 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:40:40.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f44832640 1 Processor -- start 2024-04-25T13:40:40.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f44832640 1 -- start start 2024-04-25T13:40:40.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f44832640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5f400d3ed0 0x7f5f400d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f44832640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5f400ba770 0x7f5f400b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f44832640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f400b92a0 con 0x7f5f400d2ad0 2024-04-25T13:40:40.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f44832640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5f400d4c10 con 0x7f5f400ba770 2024-04-25T13:40:40.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f44832640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5f400d4ef0 con 0x7f5f400d3ed0 2024-04-25T13:40:40.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3dd9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5f400d2ad0 0x7f5f400b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35814/0 (socket says 172.21.15.80:35814) 2024-04-25T13:40:40.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3dd9b640 1 -- 172.21.15.80:0/47432567 learned_addr learned my addr 172.21.15.80:0/47432567 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:40.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/47432567 >> v2:172.21.15.80:3300/0 conn(0x7f5f400ba770 0x7f5f400b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3d59a640 1 -- 172.21.15.80:0/47432567 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 529707210 0 0) 0x7f5f34001120 con 0x7f5f400d2ad0 2024-04-25T13:40:40.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3d59a640 1 -- 172.21.15.80:0/47432567 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5f28003530 con 0x7f5f400d2ad0 2024-04-25T13:40:40.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3e59c640 1 --2- 172.21.15.80:0/47432567 >> v2:172.21.15.87:3300/0 conn(0x7f5f400d3ed0 0x7f5f400d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/47432567 >> v2:172.21.15.87:3300/0 conn(0x7f5f400d3ed0 msgr2=0x7f5f400d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:40.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/47432567 >> v2:172.21.15.87:3300/0 conn(0x7f5f400d3ed0 0x7f5f400d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/47432567 >> v1:172.21.15.80:6789/0 conn(0x7f5f400d2ad0 legacy=0x7f5f400b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.720+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/47432567 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f400d51d0 con 0x7f5f400ba770 2024-04-25T13:40:40.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/47432567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f400b7de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5f4008e910 tx=0x7f5f3801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3632858b04f17612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f3d59a640 1 -- 172.21.15.80:0/47432567 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b2070 con 0x7f5f400ba770 2024-04-25T13:40:40.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f3d59a640 1 -- 172.21.15.80:0/47432567 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f380a58c0 con 0x7f5f400ba770 2024-04-25T13:40:40.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f3d59a640 1 -- 172.21.15.80:0/47432567 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380a5c90 con 0x7f5f400ba770 2024-04-25T13:40:40.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f44832640 1 -- 172.21.15.80:0/47432567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 msgr2=0x7f5f400b7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f44832640 1 --2- 172.21.15.80:0/47432567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f400b7de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5f4008e910 tx=0x7f5f3801c7e0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f44832640 1 -- 172.21.15.80:0/47432567 shutdown_connections 2024-04-25T13:40:40.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f44832640 1 --2- 172.21.15.80:0/47432567 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f400b7de0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f44832640 1 --2- 172.21.15.80:0/47432567 >> v2:172.21.15.87:3300/0 conn(0x7f5f400d3ed0 0x7f5f400d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f44832640 1 -- 172.21.15.80:0/47432567 >> 172.21.15.80:0/47432567 conn(0x7f5f400737b0 msgr2=0x7f5f4006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:40.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f44832640 1 -- 172.21.15.80:0/47432567 shutdown_connections 2024-04-25T13:40:40.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.724+0000 7f5f44832640 1 -- 172.21.15.80:0/47432567 wait complete. 2024-04-25T13:40:40.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 Processor -- start 2024-04-25T13:40:40.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 -- start start 2024-04-25T13:40:40.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f401432a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f400d2ad0 0x7f5f4015c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400d3ed0 0x7f5f4015ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5f400b8320 con 0x7f5f400ba770 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f400b84a0 con 0x7f5f400d2ad0 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f40160620 con 0x7f5f400d3ed0 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f401432a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f401432a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36630/0 (socket says 172.21.15.80:36630) 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 -- 172.21.15.80:0/1909225687 learned_addr learned my addr 172.21.15.80:0/1909225687 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3dd9b640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f400d2ad0 0x7f5f4015c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 -- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400d3ed0 msgr2=0x7f5f4015ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400d3ed0 0x7f5f4015ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400d3ed0 0x7f5f4015ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 -- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f400d2ad0 msgr2=0x7f5f4015c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f400d2ad0 0x7f5f4015c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 -- 172.21.15.80:0/1909225687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f401409e0 con 0x7f5f400ba770 2024-04-25T13:40:40.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f3e59c640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f401432a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000d5f0 tx=0x7f5f3000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f2effd640 1 -- 172.21.15.80:0/1909225687 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300125f0 con 0x7f5f400ba770 2024-04-25T13:40:40.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f2effd640 1 -- 172.21.15.80:0/1909225687 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f30019070 con 0x7f5f400ba770 2024-04-25T13:40:40.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f4015d6d0 con 0x7f5f400ba770 2024-04-25T13:40:40.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f2effd640 1 -- 172.21.15.80:0/1909225687 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f300129b0 con 0x7f5f400ba770 2024-04-25T13:40:40.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.728+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f4015b760 con 0x7f5f400ba770 2024-04-25T13:40:40.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.732+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f40140bb0 con 0x7f5f400ba770 2024-04-25T13:40:40.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.732+0000 7f5f2effd640 1 -- 172.21.15.80:0/1909225687 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5f30012b50 con 0x7f5f400ba770 2024-04-25T13:40:40.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.732+0000 7f5f2effd640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f400 0x7f5f20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:40.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.732+0000 7f5f2effd640 1 -- 172.21.15.80:0/1909225687 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f5f3008f7a0 con 0x7f5f400ba770 2024-04-25T13:40:40.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.732+0000 7f5f3dd9b640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f400 0x7f5f20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:40.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.732+0000 7f5f3dd9b640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f400 0x7f5f20071910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f340012c0 tx=0x7f5f34009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:40.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.736+0000 7f5f2effd640 1 -- 172.21.15.80:0/1909225687 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5f3005c690 con 0x7f5f400ba770 2024-04-25T13:40:40.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.884+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5f4006a3a0 con 0x7f5f400ba770 2024-04-25T13:40:40.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.884+0000 7f5f2effd640 1 -- 172.21.15.80:0/1909225687 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f5f3005bf90 con 0x7f5f400ba770 2024-04-25T13:40:40.885 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.884+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f400 msgr2=0x7f5f20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.884+0000 7f5f44832640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f400 0x7f5f20071910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5f340012c0 tx=0x7f5f34009040 comp rx=0 tx=0).stop 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.884+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 msgr2=0x7f5f401432a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.884+0000 7f5f44832640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f401432a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000d5f0 tx=0x7f5f3000dab0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.884+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 shutdown_connections 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.888+0000 7f5f44832640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f400 0x7f5f20071910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.888+0000 7f5f44832640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400d3ed0 0x7f5f4015ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.888+0000 7f5f44832640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f400d2ad0 0x7f5f4015c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.888+0000 7f5f44832640 1 --2- 172.21.15.80:0/1909225687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f400ba770 0x7f5f401432a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.888+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 >> 172.21.15.80:0/1909225687 conn(0x7f5f400737b0 msgr2=0x7f5f400ced10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:40.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.888+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 shutdown_connections 2024-04-25T13:40:40.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:40.888+0000 7f5f44832640 1 -- 172.21.15.80:0/1909225687 wait complete. 2024-04-25T13:40:40.905 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:40:41.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb982a9640 1 Processor -- start 2024-04-25T13:40:41.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb982a9640 1 -- start start 2024-04-25T13:40:41.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb982a9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feb900d06e0 0x7feb9006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb982a9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb9006e9e0 0x7feb9006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb982a9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb900d0af0 con 0x7feb90161f90 2024-04-25T13:40:41.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb982a9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feb900d0d70 con 0x7feb900d06e0 2024-04-25T13:40:41.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb982a9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feb900d1050 con 0x7feb9006e9e0 2024-04-25T13:40:41.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb95844640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feb900d06e0 0x7feb9006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb96846640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb9006e9e0 0x7feb9006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb96045640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7feb90161f90 0x7feb900cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:47440/0 (socket says 172.21.15.80:47440) 2024-04-25T13:40:41.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb96045640 1 -- 172.21.15.80:0/1982141189 learned_addr learned my addr 172.21.15.80:0/1982141189 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:41.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb95043640 1 -- 172.21.15.80:0/1982141189 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3214653119 0 0) 0x7feb8c001120 con 0x7feb90161f90 2024-04-25T13:40:41.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb95043640 1 -- 172.21.15.80:0/1982141189 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb7c003530 con 0x7feb90161f90 2024-04-25T13:40:41.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb96846640 1 -- 172.21.15.80:0/1982141189 >> v2:172.21.15.87:3300/0 conn(0x7feb900d06e0 msgr2=0x7feb9006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:41.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb96846640 1 --2- 172.21.15.80:0/1982141189 >> v2:172.21.15.87:3300/0 conn(0x7feb900d06e0 0x7feb9006e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb96846640 1 -- 172.21.15.80:0/1982141189 >> v1:172.21.15.87:6789/0 conn(0x7feb90161f90 legacy=0x7feb900cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.048+0000 7feb96846640 1 -- 172.21.15.80:0/1982141189 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9006dca0 con 0x7feb9006e9e0 2024-04-25T13:40:41.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb96846640 1 --2- 172.21.15.80:0/1982141189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb9006e9e0 0x7feb9006edf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb88009d90 tx=0x7feb8801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25bc569a48c5111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb95043640 1 -- 172.21.15.80:0/1982141189 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880b2070 con 0x7feb9006e9e0 2024-04-25T13:40:41.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb95043640 1 -- 172.21.15.80:0/1982141189 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb880a5880 con 0x7feb9006e9e0 2024-04-25T13:40:41.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb95043640 1 -- 172.21.15.80:0/1982141189 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880a5c30 con 0x7feb9006e9e0 2024-04-25T13:40:41.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb982a9640 1 -- 172.21.15.80:0/1982141189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb9006e9e0 msgr2=0x7feb9006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb982a9640 1 --2- 172.21.15.80:0/1982141189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb9006e9e0 0x7feb9006edf0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feb88009d90 tx=0x7feb8801c8b0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb982a9640 1 -- 172.21.15.80:0/1982141189 shutdown_connections 2024-04-25T13:40:41.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb982a9640 1 --2- 172.21.15.80:0/1982141189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb9006e9e0 0x7feb9006edf0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb982a9640 1 --2- 172.21.15.80:0/1982141189 >> v2:172.21.15.87:3300/0 conn(0x7feb900d06e0 0x7feb9006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb982a9640 1 -- 172.21.15.80:0/1982141189 >> 172.21.15.80:0/1982141189 conn(0x7feb9006d080 msgr2=0x7feb900b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:41.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb982a9640 1 -- 172.21.15.80:0/1982141189 shutdown_connections 2024-04-25T13:40:41.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.052+0000 7feb982a9640 1 -- 172.21.15.80:0/1982141189 wait complete. 2024-04-25T13:40:41.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 Processor -- start 2024-04-25T13:40:41.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 -- start start 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb9006e9e0 0x7feb900ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb900d06e0 0x7feb90140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb90161f90 0x7feb901fda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feb901447a0 con 0x7feb90161f90 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feb90144620 con 0x7feb9006e9e0 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feb9016b510 con 0x7feb900d06e0 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb90161f90 0x7feb901fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb90161f90 0x7feb901fda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36642/0 (socket says 172.21.15.80:36642) 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 -- 172.21.15.80:0/2661120103 learned_addr learned my addr 172.21.15.80:0/2661120103 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96045640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb9006e9e0 0x7feb900ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb95844640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb900d06e0 0x7feb90140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 -- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb900d06e0 msgr2=0x7feb90140f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb900d06e0 0x7feb90140f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 -- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb9006e9e0 msgr2=0x7feb900ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb9006e9e0 0x7feb900ab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 -- 172.21.15.80:0/2661120103 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb90144d20 con 0x7feb90161f90 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb96846640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb90161f90 0x7feb901fda20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb88009d90 tx=0x7feb88019650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb86ffd640 1 -- 172.21.15.80:0/2661120103 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880b2040 con 0x7feb90161f90 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb86ffd640 1 -- 172.21.15.80:0/2661120103 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb880bf070 con 0x7feb90161f90 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb90145000 con 0x7feb90161f90 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb86ffd640 1 -- 172.21.15.80:0/2661120103 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb880b86b0 con 0x7feb90161f90 2024-04-25T13:40:41.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb901fd500 con 0x7feb90161f90 2024-04-25T13:40:41.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.056+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb9006a3a0 con 0x7feb90161f90 2024-04-25T13:40:41.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.060+0000 7feb86ffd640 1 -- 172.21.15.80:0/2661120103 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb880b8850 con 0x7feb90161f90 2024-04-25T13:40:41.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.060+0000 7feb86ffd640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb7406f430 0x7feb74071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.060+0000 7feb86ffd640 1 -- 172.21.15.80:0/2661120103 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7feb88129c10 con 0x7feb90161f90 2024-04-25T13:40:41.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.060+0000 7feb96045640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb7406f430 0x7feb74071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.060+0000 7feb96045640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb7406f430 0x7feb74071940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb8c005890 tx=0x7feb8c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.064+0000 7feb86ffd640 1 -- 172.21.15.80:0/2661120103 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feb880c8020 con 0x7feb90161f90 2024-04-25T13:40:41.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.244+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb901451d0 con 0x7feb90161f90 2024-04-25T13:40:41.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.244+0000 7feb86ffd640 1 -- 172.21.15.80:0/2661120103 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7feb8812d090 con 0x7feb90161f90 2024-04-25T13:40:41.246 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:41.246 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:40:29.558137+0000","modified":"2024-04-25T13:40:32.595776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-25T13:40:41.246 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb7406f430 msgr2=0x7feb74071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb7406f430 0x7feb74071940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feb8c005890 tx=0x7feb8c009040 comp rx=0 tx=0).stop 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb90161f90 msgr2=0x7feb901fda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb90161f90 0x7feb901fda20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7feb88009d90 tx=0x7feb88019650 comp rx=0 tx=0).stop 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 shutdown_connections 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb7406f430 0x7feb74071940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feb90161f90 0x7feb901fda20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feb900d06e0 0x7feb90140f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 --2- 172.21.15.80:0/2661120103 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feb9006e9e0 0x7feb900ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 >> 172.21.15.80:0/2661120103 conn(0x7feb9006d080 msgr2=0x7feb900b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:41.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 shutdown_connections 2024-04-25T13:40:41.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.248+0000 7feb982a9640 1 -- 172.21.15.80:0/2661120103 wait complete. 2024-04-25T13:40:41.266 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:41.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.412+0000 7f673d68e640 1 Processor -- start 2024-04-25T13:40:41.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.412+0000 7f673d68e640 1 -- start start 2024-04-25T13:40:41.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f673d68e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67380b4eb0 0x7f67381603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f673d68e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67380d48a0 con 0x7f67380b7780 2024-04-25T13:40:41.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f673d68e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67380d4b80 con 0x7f67380b93f0 2024-04-25T13:40:41.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f673d68e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f67380d4e60 con 0x7f67380b4eb0 2024-04-25T13:40:41.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f67367fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f67380b7780 0x7f67380b7b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45032/0 (socket says 172.21.15.80:45032) 2024-04-25T13:40:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f6736ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67380b4eb0 0x7f67381603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f67367fc640 1 -- 172.21.15.80:0/1424388060 learned_addr learned my addr 172.21.15.80:0/1424388060 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f6735ffb640 1 -- 172.21.15.80:0/1424388060 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3052189410 0 0) 0x7f67380d48a0 con 0x7f67380b7780 2024-04-25T13:40:41.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f6735ffb640 1 -- 172.21.15.80:0/1424388060 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6720003530 con 0x7f67380b7780 2024-04-25T13:40:41.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f6736ffd640 1 -- 172.21.15.80:0/1424388060 >> v1:172.21.15.87:6789/0 conn(0x7f67380b93f0 legacy=0x7f67380b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f6736ffd640 1 -- 172.21.15.80:0/1424388060 >> v1:172.21.15.80:6789/0 conn(0x7f67380b7780 legacy=0x7f67380b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.416+0000 7f6736ffd640 1 -- 172.21.15.80:0/1424388060 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67380d5140 con 0x7f67380b4eb0 2024-04-25T13:40:41.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f6736ffd640 1 --2- 172.21.15.80:0/1424388060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b4eb0 0x7f67381603c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6730009d90 tx=0x7f673001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79bfea166680f4e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f6735ffb640 1 -- 172.21.15.80:0/1424388060 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67300b2070 con 0x7f67380b4eb0 2024-04-25T13:40:41.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f6735ffb640 1 -- 172.21.15.80:0/1424388060 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67300a5920 con 0x7f67380b4eb0 2024-04-25T13:40:41.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f6735ffb640 1 -- 172.21.15.80:0/1424388060 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67300a5cf0 con 0x7f67380b4eb0 2024-04-25T13:40:41.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 -- 172.21.15.80:0/1424388060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b4eb0 msgr2=0x7f67381603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 --2- 172.21.15.80:0/1424388060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b4eb0 0x7f67381603c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6730009d90 tx=0x7f673001c970 comp rx=0 tx=0).stop 2024-04-25T13:40:41.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 -- 172.21.15.80:0/1424388060 shutdown_connections 2024-04-25T13:40:41.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 --2- 172.21.15.80:0/1424388060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b4eb0 0x7f67381603c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 -- 172.21.15.80:0/1424388060 >> 172.21.15.80:0/1424388060 conn(0x7f673806c9f0 msgr2=0x7f673806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:41.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 -- 172.21.15.80:0/1424388060 shutdown_connections 2024-04-25T13:40:41.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 -- 172.21.15.80:0/1424388060 wait complete. 2024-04-25T13:40:41.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 Processor -- start 2024-04-25T13:40:41.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.420+0000 7f673d68e640 1 -- start start 2024-04-25T13:40:41.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67380b4eb0 0x7f67380d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67380b7780 0x7f6738158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b93f0 0x7f67380d0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67380d5940 con 0x7f67380b93f0 2024-04-25T13:40:41.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67380d3120 con 0x7f67380b7780 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f67380cae40 con 0x7f67380b4eb0 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b93f0 0x7f67380d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b93f0 0x7f67380d0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44306/0 (socket says 172.21.15.80:44306) 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 -- 172.21.15.80:0/3640843305 learned_addr learned my addr 172.21.15.80:0/3640843305 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f6736ffd640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67380b4eb0 0x7f67380d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67367fc640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67380b7780 0x7f6738158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 -- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67380b4eb0 msgr2=0x7f67380d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67380b4eb0 0x7f67380d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 -- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67380b7780 msgr2=0x7f6738158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67380b7780 0x7f6738158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 -- 172.21.15.80:0/3640843305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67381fa550 con 0x7f67380b93f0 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f6736ffd640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67380b4eb0 0x7f67380d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67367fc640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67380b7780 0x7f6738158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f67377fe640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b93f0 0x7f67380d0020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f672800c340 tx=0x7f672800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f671f7fe640 1 -- 172.21.15.80:0/3640843305 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728017070 con 0x7f67380b93f0 2024-04-25T13:40:41.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f671f7fe640 1 -- 172.21.15.80:0/3640843305 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67280052d0 con 0x7f67380b93f0 2024-04-25T13:40:41.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67381fa830 con 0x7f67380b93f0 2024-04-25T13:40:41.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f671f7fe640 1 -- 172.21.15.80:0/3640843305 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728003800 con 0x7f67380b93f0 2024-04-25T13:40:41.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67380c98a0 con 0x7f67380b93f0 2024-04-25T13:40:41.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673806a3a0 con 0x7f67380b93f0 2024-04-25T13:40:41.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f671f7fe640 1 -- 172.21.15.80:0/3640843305 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f672801a070 con 0x7f67380b93f0 2024-04-25T13:40:41.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f671f7fe640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f672406f430 0x7f6724071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f671f7fe640 1 -- 172.21.15.80:0/3640843305 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f672808e470 con 0x7f67380b93f0 2024-04-25T13:40:41.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.424+0000 7f6736ffd640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f672406f430 0x7f6724071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.428+0000 7f6736ffd640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f672406f430 0x7f6724071940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67300af000 tx=0x7f67300a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.432+0000 7f671f7fe640 1 -- 172.21.15.80:0/3640843305 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6728005480 con 0x7f67380b93f0 2024-04-25T13:40:41.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.584+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6738160450 con 0x7f67380b93f0 2024-04-25T13:40:41.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f671f7fe640 1 -- 172.21.15.80:0/3640843305 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f67280633e0 con 0x7f67380b93f0 2024-04-25T13:40:41.588 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:41.588 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:41.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f672406f430 msgr2=0x7f6724071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f672406f430 0x7f6724071940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67300af000 tx=0x7f67300a7070 comp rx=0 tx=0).stop 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b93f0 msgr2=0x7f67380d0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b93f0 0x7f67380d0020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f672800c340 tx=0x7f672800c800 comp rx=0 tx=0).stop 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 shutdown_connections 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f672406f430 0x7f6724071940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67380b93f0 0x7f67380d0020 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67380b7780 0x7f6738158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 --2- 172.21.15.80:0/3640843305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67380b4eb0 0x7f67380d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 >> 172.21.15.80:0/3640843305 conn(0x7f673806c9f0 msgr2=0x7f67380b6b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 shutdown_connections 2024-04-25T13:40:41.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.588+0000 7f673d68e640 1 -- 172.21.15.80:0/3640843305 wait complete. 2024-04-25T13:40:41.608 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:40:41.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc012044640 1 Processor -- start 2024-04-25T13:40:41.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc012044640 1 -- start start 2024-04-25T13:40:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc012044640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c161650 0x7fc00c072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc012044640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00c06fcf0 con 0x7fc00c161240 2024-04-25T13:40:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc012044640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc00c0b0c60 con 0x7fc00c06f5a0 2024-04-25T13:40:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc012044640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc00c073f10 con 0x7fc00c161650 2024-04-25T13:40:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc00affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc00c161240 0x7fc00c0723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45044/0 (socket says 172.21.15.80:45044) 2024-04-25T13:40:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc00affd640 1 -- 172.21.15.80:0/2040287038 learned_addr learned my addr 172.21.15.80:0/2040287038 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc00bfff640 1 --2- 172.21.15.80:0/2040287038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c161650 0x7fc00c072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc00a7fc640 1 -- 172.21.15.80:0/2040287038 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4017529575 0 0) 0x7fc00c06fcf0 con 0x7fc00c161240 2024-04-25T13:40:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc00a7fc640 1 -- 172.21.15.80:0/2040287038 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbff4003530 con 0x7fc00c161240 2024-04-25T13:40:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc00a7fc640 1 -- 172.21.15.80:0/2040287038 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1893007331 0 0) 0x7fc00c0b0c60 con 0x7fc00c06f5a0 2024-04-25T13:40:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc00a7fc640 1 -- 172.21.15.80:0/2040287038 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc00c06fcf0 con 0x7fc00c06f5a0 2024-04-25T13:40:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.740+0000 7fc00bfff640 1 -- 172.21.15.80:0/2040287038 >> v1:172.21.15.87:6789/0 conn(0x7fc00c06f5a0 legacy=0x7fc00c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc00bfff640 1 -- 172.21.15.80:0/2040287038 >> v1:172.21.15.80:6789/0 conn(0x7fc00c161240 legacy=0x7fc00c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc00a7fc640 1 -- 172.21.15.80:0/2040287038 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4046029288 0 0) 0x7fbff4003530 con 0x7fc00c161240 2024-04-25T13:40:41.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc00bfff640 1 -- 172.21.15.80:0/2040287038 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c0b0f40 con 0x7fc00c161650 2024-04-25T13:40:41.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc00bfff640 1 --2- 172.21.15.80:0/2040287038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c161650 0x7fc00c072ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009dc0 tx=0x7fbffc01c820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3ff395d0282c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc00a7fc640 1 -- 172.21.15.80:0/2040287038 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc000001680 con 0x7fc00c161650 2024-04-25T13:40:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc00a7fc640 1 -- 172.21.15.80:0/2040287038 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc0a64f0 con 0x7fc00c161650 2024-04-25T13:40:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc00a7fc640 1 -- 172.21.15.80:0/2040287038 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0a68a0 con 0x7fc00c161650 2024-04-25T13:40:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc012044640 1 -- 172.21.15.80:0/2040287038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c161650 msgr2=0x7fc00c072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc012044640 1 --2- 172.21.15.80:0/2040287038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c161650 0x7fc00c072ab0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009dc0 tx=0x7fbffc01c820 comp rx=0 tx=0).stop 2024-04-25T13:40:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc012044640 1 -- 172.21.15.80:0/2040287038 shutdown_connections 2024-04-25T13:40:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc012044640 1 --2- 172.21.15.80:0/2040287038 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c161650 0x7fc00c072ab0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc012044640 1 -- 172.21.15.80:0/2040287038 >> 172.21.15.80:0/2040287038 conn(0x7fc00c06d080 msgr2=0x7fc00c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc012044640 1 -- 172.21.15.80:0/2040287038 shutdown_connections 2024-04-25T13:40:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.744+0000 7fc012044640 1 -- 172.21.15.80:0/2040287038 wait complete. 2024-04-25T13:40:41.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 Processor -- start 2024-04-25T13:40:41.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 -- start start 2024-04-25T13:40:41.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c06f5a0 0x7fc00c142250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc00c161240 0x7fc00c15e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc00c161650 0x7fc00c1f5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c06e860 con 0x7fc00c161650 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c06e6e0 con 0x7fc00c161240 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc00c06e9e0 con 0x7fc00c06f5a0 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc00c161650 0x7fc00c1f5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc00c161650 0x7fc00c1f5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44308/0 (socket says 172.21.15.80:44308) 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 -- 172.21.15.80:0/444806920 learned_addr learned my addr 172.21.15.80:0/444806920 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00affd640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc00c161240 0x7fc00c15e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 -- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c06f5a0 msgr2=0x7fc00c142250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00b7fe640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c06f5a0 0x7fc00c142250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c06f5a0 0x7fc00c142250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 -- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc00c161240 msgr2=0x7fc00c15e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc00c161240 0x7fc00c15e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 -- 172.21.15.80:0/444806920 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c142790 con 0x7fc00c161650 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00affd640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc00c161240 0x7fc00c15e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc00bfff640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc00c161650 0x7fc00c1f5320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbffc01a450 tx=0x7fc00c073800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fbff3fff640 1 -- 172.21.15.80:0/444806920 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc0a6970 con 0x7fc00c161650 2024-04-25T13:40:41.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fbff3fff640 1 -- 172.21.15.80:0/444806920 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbffc0a6b10 con 0x7fc00c161650 2024-04-25T13:40:41.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c15f0c0 con 0x7fc00c161650 2024-04-25T13:40:41.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fbff3fff640 1 -- 172.21.15.80:0/444806920 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc004040 con 0x7fc00c161650 2024-04-25T13:40:41.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.748+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c15f680 con 0x7fc00c161650 2024-04-25T13:40:41.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.752+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c06a3a0 con 0x7fc00c161650 2024-04-25T13:40:41.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.752+0000 7fbff3fff640 1 -- 172.21.15.80:0/444806920 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbffc0c1020 con 0x7fc00c161650 2024-04-25T13:40:41.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.752+0000 7fbff3fff640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbff806f430 0x7fbff8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:41.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.752+0000 7fbff3fff640 1 -- 172.21.15.80:0/444806920 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbffc129b80 con 0x7fc00c161650 2024-04-25T13:40:41.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.752+0000 7fc00b7fe640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbff806f430 0x7fbff8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:41.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.752+0000 7fc00b7fe640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbff806f430 0x7fbff8071940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0040010f0 tx=0x7fc004009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:41.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.756+0000 7fbff3fff640 1 -- 172.21.15.80:0/444806920 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbffc0164a0 con 0x7fc00c161650 2024-04-25T13:40:41.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.932+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc00c15ee10 con 0x7fc00c161650 2024-04-25T13:40:41.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fbff3fff640 1 -- 172.21.15.80:0/444806920 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7fbffc0f6b00 con 0x7fc00c161650 2024-04-25T13:40:41.936 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:41.936 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:40:29.558137+0000","modified":"2024-04-25T13:40:32.595776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-25T13:40:41.936 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-25T13:40:41.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbff806f430 msgr2=0x7fbff8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbff806f430 0x7fbff8071940 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc0040010f0 tx=0x7fc004009040 comp rx=0 tx=0).stop 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc00c161650 msgr2=0x7fc00c1f5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc00c161650 0x7fc00c1f5320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbffc01a450 tx=0x7fc00c073800 comp rx=0 tx=0).stop 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 shutdown_connections 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbff806f430 0x7fbff8071940 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc00c161650 0x7fc00c1f5320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc00c161240 0x7fc00c15e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 --2- 172.21.15.80:0/444806920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc00c06f5a0 0x7fc00c142250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 >> 172.21.15.80:0/444806920 conn(0x7fc00c06d080 msgr2=0x7fc00c0b17d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 shutdown_connections 2024-04-25T13:40:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:41.936+0000 7fc012044640 1 -- 172.21.15.80:0/444806920 wait complete. 2024-04-25T13:40:42.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:40:29.558137+0000', 'modified': '2024-04-25T13:40:32.595776+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4272}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4272': {'gid': 4272, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6835/3340293864', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 3340293864}, {'type': 'v1', 'addr': '172.21.15.87:6835', 'nonce': 3340293864}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4272, 'qdb_cluster': [4272]} 2024-04-25T13:40:42.358 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:40:42.358 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T13:40:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58dabcb640 1 Processor -- start 2024-04-25T13:40:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58dabcb640 1 -- start start 2024-04-25T13:40:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58dabcb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58d40d2e80 0x7f58d41607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58dabcb640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d4161060 con 0x7f58d40d4330 2024-04-25T13:40:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58dabcb640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58d4161340 con 0x7f58d40d59b0 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58dabcb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f58d40d2680 con 0x7f58d40d2e80 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58d9168640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f58d40d4330 0x7f58d40d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45056/0 (socket says 172.21.15.80:45056) 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58d9168640 1 -- 172.21.15.80:0/1498018820 learned_addr learned my addr 172.21.15.80:0/1498018820 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58d8967640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58d40d2e80 0x7f58d41607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 322585522 0 0) 0x7f58d4161060 con 0x7f58d40d4330 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.516+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58bc003530 con 0x7f58d40d4330 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 330235957 0 0) 0x7f58bc003530 con 0x7f58d40d4330 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58d40d2e80 msgr2=0x7f58d41607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58d40d2e80 0x7f58d41607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 >> v1:172.21.15.87:6789/0 conn(0x7f58d40d59b0 legacy=0x7f58d40d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d40d2960 con 0x7f58d40d4330 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f58cc001a00 con 0x7f58d40d4330 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 >> v1:172.21.15.80:6789/0 conn(0x7f58d40d4330 legacy=0x7f58d40d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58bc0039f0 0x7f58bc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58d40d2e80 0x7f58bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58bc004d50 0x7f58bc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40d2680 con 0x7f58bc0039f0 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f58bc00bb90 con 0x7f58bc004d50 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f58bc00bd90 con 0x7f58d40d2e80 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d8967640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58bc0039f0 0x7f58bc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d9168640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58bc004d50 0x7f58bc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cbfff640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58d40d2e80 0x7f58bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d8967640 1 -- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58d40d2e80 msgr2=0x7f58bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d8967640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58d40d2e80 0x7f58bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d8967640 1 -- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58bc004d50 msgr2=0x7f58bc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d8967640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58bc004d50 0x7f58bc005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d8967640 1 -- 172.21.15.80:0/1498018820 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58bc003530 con 0x7f58bc0039f0 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d8967640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58bc0039f0 0x7f58bc003e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009ab0 tx=0x7f58d001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01993305db9d3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00b2070 con 0x7f58bc0039f0 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58d00a5550 con 0x7f58bc0039f0 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58cb7fe640 1 -- 172.21.15.80:0/1498018820 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00096b0 con 0x7f58bc0039f0 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58dabcb640 1 -- 172.21.15.80:0/1498018820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58bc0039f0 msgr2=0x7f58bc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58dabcb640 1 --2- 172.21.15.80:0/1498018820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58bc0039f0 0x7f58bc003e60 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f58d0009ab0 tx=0x7f58d001ca70 comp rx=0 tx=0).stop 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58d9168640 1 -- 172.21.15.80:0/1498018820 reap_dead start 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58dabcb640 1 -- 172.21.15.80:0/1498018820 shutdown_connections 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58dabcb640 1 -- 172.21.15.80:0/1498018820 >> 172.21.15.80:0/1498018820 conn(0x7f58d40737a0 msgr2=0x7f58d406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58dabcb640 1 -- 172.21.15.80:0/1498018820 shutdown_connections 2024-04-25T13:40:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.520+0000 7f58dabcb640 1 -- 172.21.15.80:0/1498018820 wait complete. 2024-04-25T13:40:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 Processor -- start 2024-04-25T13:40:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 -- start start 2024-04-25T13:40:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58bc0039f0 0x7f58d4158710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58d4158fd0 0x7f58d4168690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58d41f9770 0x7f58d4168bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f58d416c2b0 con 0x7f58d41f9770 2024-04-25T13:40:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f58d416c130 con 0x7f58d4158fd0 2024-04-25T13:40:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f58d416c430 con 0x7f58bc0039f0 2024-04-25T13:40:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58d41f9770 0x7f58d4168bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58d41f9770 0x7f58d4168bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44330/0 (socket says 172.21.15.80:44330) 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 -- 172.21.15.80:0/1302860193 learned_addr learned my addr 172.21.15.80:0/1302860193 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d8967640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58bc0039f0 0x7f58d4158710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58cbfff640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58d4158fd0 0x7f58d4168690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 -- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58bc0039f0 msgr2=0x7f58d4158710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58bc0039f0 0x7f58d4158710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 -- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58d4158fd0 msgr2=0x7f58d4168690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58d4158fd0 0x7f58d4168690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 -- 172.21.15.80:0/1302860193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58d4169930 con 0x7f58d41f9770 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58d9168640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58d41f9770 0x7f58d4168bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58d40d36f0 tx=0x7f58cc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58c97fa640 1 -- 172.21.15.80:0/1302860193 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00ec10 con 0x7f58d41f9770 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58c97fa640 1 -- 172.21.15.80:0/1302860193 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc018070 con 0x7f58d41f9770 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58dabcb640 1 -- 172.21.15.80:0/1302860193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58d4167bb0 con 0x7f58d41f9770 2024-04-25T13:40:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.524+0000 7f58c97fa640 1 -- 172.21.15.80:0/1302860193 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc013420 con 0x7f58d41f9770 2024-04-25T13:40:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.528+0000 7f58dabcb640 1 -- 172.21.15.80:0/1302860193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58d40cf860 con 0x7f58d41f9770 2024-04-25T13:40:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.528+0000 7f58c97fa640 1 -- 172.21.15.80:0/1302860193 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f58cc0135c0 con 0x7f58d41f9770 2024-04-25T13:40:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.528+0000 7f58c97fa640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58c006f400 0x7f58c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.528+0000 7f58c97fa640 1 -- 172.21.15.80:0/1302860193 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58cc08f690 con 0x7f58d41f9770 2024-04-25T13:40:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.528+0000 7f58d8967640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58c006f400 0x7f58c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.528+0000 7f58e1f28000 1 Processor -- start 2024-04-25T13:40:42.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.528+0000 7f58e1f28000 1 -- start start 2024-04-25T13:40:42.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.528+0000 7f58d8967640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58c006f400 0x7f58c0071910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58d00af000 tx=0x7f58d00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b682ec20 0x5592b698cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b681aea0 0x5592b698d420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b6413280 0x5592b698d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5592b67fc160 con 0x5592b681aea0 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5592b6791e20 con 0x5592b682ec20 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5592b674aa20 con 0x5592b6413280 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b681aea0 0x5592b698d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b681aea0 0x5592b698d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44346/0 (socket says 172.21.15.80:44346) 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 -- 172.21.15.80:0/3983726471 learned_addr learned my addr 172.21.15.80:0/3983726471 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58d8967640 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b682ec20 0x5592b698cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58d9168640 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b6413280 0x5592b698d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 -- 172.21.15.80:0/3983726471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b6413280 msgr2=0x5592b698d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b6413280 0x5592b698d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 -- 172.21.15.80:0/3983726471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b682ec20 msgr2=0x5592b698cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b682ec20 0x5592b698cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 -- 172.21.15.80:0/3983726471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b674f100 con 0x5592b681aea0 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58cbfff640 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b681aea0 0x5592b698d420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58c40053d0 tx=0x7f58c40039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312faeec9e4c4b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58ba7fc640 1 -- 172.21.15.80:0/3983726471 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4001360 con 0x5592b681aea0 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58ba7fc640 1 -- 172.21.15.80:0/3983726471 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c4019070 con 0x5592b681aea0 2024-04-25T13:40:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58ba7fc640 1 -- 172.21.15.80:0/3983726471 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c4001720 con 0x5592b681aea0 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- 172.21.15.80:0/3983726471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b681aea0 msgr2=0x5592b698d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b681aea0 0x5592b698d420 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f58c40053d0 tx=0x7f58c40039d0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- 172.21.15.80:0/3983726471 shutdown_connections 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b6413280 0x5592b698d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b681aea0 0x5592b698d420 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3983726471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b682ec20 0x5592b698cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- 172.21.15.80:0/3983726471 >> 172.21.15.80:0/3983726471 conn(0x5592b6452ff0 msgr2=0x5592b69424c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- 172.21.15.80:0/3983726471 shutdown_connections 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- 172.21.15.80:0/3983726471 wait complete. 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 -- start start 2024-04-25T13:40:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.532+0000 7f58e1f28000 1 Processor -- start 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b6413280 0x5592b6993e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b681aea0 0x5592b69943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b682ec20 0x5592b694f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5592b679f220 con 0x5592b6413280 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5592b6735ea0 con 0x5592b681aea0 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5592b678cb00 con 0x5592b682ec20 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 10 client.0 _ll_register_callbacks cb 0x5592b679edc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b6413280 0x5592b6993e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b6413280 0x5592b6993e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44358/0 (socket says 172.21.15.80:44358) 2024-04-25T13:40:42.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 -- 172.21.15.80:0/3461733155 learned_addr learned my addr 172.21.15.80:0/3461733155 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d9168640 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b682ec20 0x5592b694f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58cbfff640 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b681aea0 0x5592b69943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 -- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b682ec20 msgr2=0x5592b694f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b682ec20 0x5592b694f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 -- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b681aea0 msgr2=0x5592b69943a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b681aea0 0x5592b69943a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 -- 172.21.15.80:0/3461733155 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b6731500 con 0x5592b6413280 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58d8967640 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b6413280 0x5592b6993e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f58d00b0040 tx=0x7f58d001b2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a92935fafffb5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58b37fe640 1 -- 172.21.15.80:0/3461733155 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d00c8060 con 0x5592b6413280 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58b37fe640 1 -- 172.21.15.80:0/3461733155 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58d00b7660 con 0x5592b6413280 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 -- 172.21.15.80:0/3461733155 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58c4019070 con 0x5592b6413280 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58b37fe640 1 -- 172.21.15.80:0/3461733155 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f58d00ce050 con 0x5592b6413280 2024-04-25T13:40:42.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 10 client.4599 fetch_fsmap learned FSMap version 7 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 -- 172.21.15.80:0/3461733155 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5592b6717ea0 con 0x5592b6413280 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58b37fe640 1 -- 172.21.15.80:0/3461733155 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58c4019070 con 0x5592b6413280 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58b37fe640 1 -- 172.21.15.80:0/3461733155 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f58d00d1020 con 0x5592b6413280 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 10 client.4599 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 10 client.4599 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 20 client.4599 populate_metadata read hostname 'smithi080' 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] conn(0x5592b64377d0 0x5592b694fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 4 client.4599 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58e1f28000 1 -- 172.21.15.80:0/3461733155 --> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592b672abe0 con 0x5592b64377d0 2024-04-25T13:40:42.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58cbfff640 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] conn(0x5592b64377d0 0x5592b694fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58b37fe640 10 client.4599 ms_handle_connect on v2:172.21.15.87:6834/3340293864 2024-04-25T13:40:42.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.536+0000 7f58cbfff640 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] conn(0x5592b64377d0 0x5592b694fe00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbec8ac8dd67c70e server_cookie=84a90d79dac5653d in_seq=0 out_seq=0 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58b37fe640 1 -- 172.21.15.80:0/3461733155 <== mds.0 v2:172.21.15.87:6834/3340293864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592b672abe0 con 0x5592b64377d0 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58b37fe640 10 client.4599 handle_command_reply: tid=0 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 client.4599 shutdown 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 -- 172.21.15.80:0/3461733155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b6413280 msgr2=0x5592b6993e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b6413280 0x5592b6993e60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f58d00b0040 tx=0x7f58d001b2b0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 -- 172.21.15.80:0/3461733155 shutdown_connections 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b682ec20 0x5592b694f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] conn(0x5592b64377d0 0x5592b694fe00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b681aea0 0x5592b69943a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3461733155 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b6413280 0x5592b6993e60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58b37fe640 0 client.4599 ms_handle_reset on v2:172.21.15.87:6834/3340293864 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 -- 172.21.15.80:0/3461733155 >> 172.21.15.80:0/3461733155 conn(0x5592b6452ff0 msgr2=0x5592b6942c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 -- 172.21.15.80:0/3461733155 shutdown_connections 2024-04-25T13:40:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 1 -- 172.21.15.80:0/3461733155 wait complete. 2024-04-25T13:40:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.540+0000 7f58e1f28000 20 client.4599 trim_cache size 0 max 16384 2024-04-25T13:40:42.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.556+0000 7f58e1f28000 1 Processor -- start 2024-04-25T13:40:42.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.556+0000 7f58e1f28000 1 -- start start 2024-04-25T13:40:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 0x5592b69990b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 0x5592b69a2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b69a3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f58d4205640 con 0x5592b682ec20 2024-04-25T13:40:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f58d40ba900 con 0x5592b6413280 2024-04-25T13:40:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f58d4205050 con 0x5592b681aea0 2024-04-25T13:40:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b69a3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b69a3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44364/0 (socket says 172.21.15.80:44364) 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 -- 172.21.15.80:0/3699179387 learned_addr learned my addr 172.21.15.80:0/3699179387 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58cbfff640 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 0x5592b69a2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d8967640 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 0x5592b69990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 -- 172.21.15.80:0/3699179387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 msgr2=0x5592b69a2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 0x5592b69a2ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 -- 172.21.15.80:0/3699179387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 msgr2=0x5592b69990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 0x5592b69990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 -- 172.21.15.80:0/3699179387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b694f430 con 0x5592b682ec20 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58d9168640 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b69a3530 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f58cc007c50 tx=0x7f58cc05b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f82b9ae40cfc2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58b9ffb640 1 -- 172.21.15.80:0/3699179387 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc05bf10 con 0x5592b682ec20 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58b9ffb640 1 -- 172.21.15.80:0/3699179387 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc0602c0 con 0x5592b682ec20 2024-04-25T13:40:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58b9ffb640 1 -- 172.21.15.80:0/3699179387 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc05b390 con 0x5592b682ec20 2024-04-25T13:40:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- 172.21.15.80:0/3699179387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 msgr2=0x5592b69a3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b69a3530 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f58cc007c50 tx=0x7f58cc05b950 comp rx=0 tx=0).stop 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- 172.21.15.80:0/3699179387 shutdown_connections 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b69a3530 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 0x5592b69a2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 --2- 172.21.15.80:0/3699179387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 0x5592b69990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- 172.21.15.80:0/3699179387 >> 172.21.15.80:0/3699179387 conn(0x5592b6452ff0 msgr2=0x5592b694a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- 172.21.15.80:0/3699179387 shutdown_connections 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- 172.21.15.80:0/3699179387 wait complete. 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 -- start start 2024-04-25T13:40:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.560+0000 7f58e1f28000 1 Processor -- start 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 0x5592b69ff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 0x5592b69ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b6997920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5592b6791130 con 0x5592b682ec20 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5592b67277d0 con 0x5592b6413280 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5592b6796450 con 0x5592b681aea0 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 10 client.0 _ll_register_callbacks cb 0x7f58d00b0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b6997920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b6997920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44380/0 (socket says 172.21.15.80:44380) 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 -- 172.21.15.80:0/516772043 learned_addr learned my addr 172.21.15.80:0/516772043 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58cbfff640 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 0x5592b69ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d8967640 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 0x5592b69ff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 -- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 msgr2=0x5592b69ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 0x5592b69ffad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 -- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 msgr2=0x5592b69ff590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 0x5592b69ff590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 -- 172.21.15.80:0/516772043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b674ac70 con 0x5592b682ec20 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d9168640 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b6997920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0012f0 tx=0x7f58cc000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25338f942362703 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 1 -- 172.21.15.80:0/516772043 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58cc064ce0 con 0x5592b682ec20 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 1 -- 172.21.15.80:0/516772043 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f58cc076d20 con 0x5592b682ec20 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- 172.21.15.80:0/516772043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58cc0602c0 con 0x5592b682ec20 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 1 -- 172.21.15.80:0/516772043 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f58cc05bf10 con 0x5592b682ec20 2024-04-25T13:40:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 10 client.4605 fetch_fsmap learned FSMap version 7 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- 172.21.15.80:0/516772043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5592b6814180 con 0x5592b682ec20 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 1 -- 172.21.15.80:0/516772043 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58cc0602c0 con 0x5592b682ec20 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 1 -- 172.21.15.80:0/516772043 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 7) ==== 3494+0+0 (secure 0 0 0) 0x7f58cc057880 con 0x5592b682ec20 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 10 client.4605 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 10 client.4605 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 20 client.4605 populate_metadata read hostname 'smithi080' 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] conn(0x5592b6425830 0x5592b69982e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 4 client.4605 mds_command: new command op to 4272 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- 172.21.15.80:0/516772043 --> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] -- command(tid 0: {"prefix": "status"}) -- 0x5592b679fa30 con 0x5592b6425830 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d8967640 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] conn(0x5592b6425830 0x5592b69982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 10 client.4605 ms_handle_connect on v2:172.21.15.87:6834/3340293864 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58d8967640 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] conn(0x5592b6425830 0x5592b69982e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38a185ab86182376 server_cookie=70f491e54d5c58f3 in_seq=0 out_seq=0 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 1 -- 172.21.15.80:0/516772043 <== mds.0 v2:172.21.15.87:6834/3340293864 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5592b679fa30 con 0x5592b6425830 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 10 client.4605 handle_command_reply: tid=0 2024-04-25T13:40:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 client.4605 shutdown 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- 172.21.15.80:0/516772043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 msgr2=0x5592b6997920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b6997920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f58cc0012f0 tx=0x7f58cc000f50 comp rx=0 tx=0).stop 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- 172.21.15.80:0/516772043 shutdown_connections 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5592b682ec20 0x5592b6997920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5592b681aea0 0x5592b69ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:6834/3340293864,v1:172.21.15.87:6835/3340293864] conn(0x5592b6425830 0x5592b69982e0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 --2- 172.21.15.80:0/516772043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5592b6413280 0x5592b69ff590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58b37fe640 0 client.4605 ms_handle_reset on v2:172.21.15.87:6834/3340293864 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- 172.21.15.80:0/516772043 >> 172.21.15.80:0/516772043 conn(0x5592b6452ff0 msgr2=0x5592b6992010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- 172.21.15.80:0/516772043 shutdown_connections 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 1 -- 172.21.15.80:0/516772043 wait complete. 2024-04-25T13:40:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58e1f28000 20 client.4605 trim_cache size 0 max 16384 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 4272, 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 12.359047091000001, 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 7, 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 20, 2024-04-25T13:40:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T13:40:42.572 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 19.598677393999999 2024-04-25T13:40:42.572 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:40:42.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58dabcb640 1 -- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58c006f400 msgr2=0x7f58c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58dabcb640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58c006f400 0x7f58c0071910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f58d00af000 tx=0x7f58d00046d0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58dabcb640 1 -- 172.21.15.80:0/1302860193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58d41f9770 msgr2=0x7f58d4168bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.564+0000 7f58dabcb640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58d41f9770 0x7f58d4168bd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f58d40d36f0 tx=0x7f58cc00e530 comp rx=0 tx=0).stop 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.568+0000 7f58dabcb640 1 -- 172.21.15.80:0/1302860193 shutdown_connections 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.568+0000 7f58dabcb640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58c006f400 0x7f58c0071910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.568+0000 7f58dabcb640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58d41f9770 0x7f58d4168bd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.568+0000 7f58dabcb640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58d4158fd0 0x7f58d4168690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.568+0000 7f58dabcb640 1 --2- 172.21.15.80:0/1302860193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f58bc0039f0 0x7f58d4158710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.568+0000 7f58dabcb640 1 -- 172.21.15.80:0/1302860193 >> 172.21.15.80:0/1302860193 conn(0x7f58d40737a0 msgr2=0x7f58d40d5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.568+0000 7f58dabcb640 1 -- 172.21.15.80:0/1302860193 shutdown_connections 2024-04-25T13:40:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:42.568+0000 7f58dabcb640 1 -- 172.21.15.80:0/1302860193 wait complete. 2024-04-25T13:40:42.584 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T13:40:42.593 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T13:40:42.593 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T13:40:42.594 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T13:40:42.594 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T13:40:42.732 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T13:40:42.733 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T13:40:42.733 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T13:40:42.733 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:40:42.733 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:42.733 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T13:40:42.733 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:40:42.733 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T13:40:42.733 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T13:40:42.733 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:40:42.733 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:42.734 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T13:40:42.734 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T13:40:42.734 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T13:40:42.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:42.734 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:40:42.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:42.740 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link delete ceph-brx 2024-04-25T13:40:42.797 INFO:teuthology.orchestra.run.smithi080.stderr:Cannot find device "ceph-brx" 2024-04-25T13:40:42.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:42.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:42.799 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:40:42.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:42.806 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T13:40:42.867 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-04-25T13:40:42.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:40:42.869 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T13:40:42.869 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:40:43.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44fdd73640 1 Processor -- start 2024-04-25T13:40:43.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44fdd73640 1 -- start start 2024-04-25T13:40:43.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44fdd73640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44f8068ca0 0x7f44f8070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:43.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44fdd73640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f44f80ccf60 0x7f44f8164100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:43.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44fdd73640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44f80cd370 con 0x7f44f8160400 2024-04-25T13:40:43.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44fdd73640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f44f80c7a60 con 0x7f44f8068ca0 2024-04-25T13:40:43.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44fdd73640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f44f806d580 con 0x7f44f80ccf60 2024-04-25T13:40:43.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44f6ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44f8068ca0 0x7f44f8070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:43.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44f6ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44f8068ca0 0x7f44f8070920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44394/0 (socket says 172.21.15.80:44394) 2024-04-25T13:40:43.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44f6ffd640 1 -- 172.21.15.80:0/4077725409 learned_addr learned my addr 172.21.15.80:0/4077725409 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:43.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44f7fff640 1 --2- 172.21.15.80:0/4077725409 >> v2:172.21.15.87:3300/0 conn(0x7f44f80ccf60 0x7f44f8164100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:43.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44f6ffd640 1 -- 172.21.15.80:0/4077725409 >> v2:172.21.15.87:3300/0 conn(0x7f44f80ccf60 msgr2=0x7f44f8164100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:43.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44f6ffd640 1 --2- 172.21.15.80:0/4077725409 >> v2:172.21.15.87:3300/0 conn(0x7f44f80ccf60 0x7f44f8164100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44f6ffd640 1 -- 172.21.15.80:0/4077725409 >> v1:172.21.15.87:6789/0 conn(0x7f44f8160400 legacy=0x7f44f8068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:43.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.024+0000 7f44f6ffd640 1 -- 172.21.15.80:0/4077725409 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f806d000 con 0x7f44f8068ca0 2024-04-25T13:40:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44f6ffd640 1 --2- 172.21.15.80:0/4077725409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8068ca0 0x7f44f8070920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f44f80cf340 tx=0x7f44ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792818668a1545a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44f67fc640 1 -- 172.21.15.80:0/4077725409 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0b2070 con 0x7f44f8068ca0 2024-04-25T13:40:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44f67fc640 1 -- 172.21.15.80:0/4077725409 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44ec0ad680 con 0x7f44f8068ca0 2024-04-25T13:40:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44f67fc640 1 -- 172.21.15.80:0/4077725409 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec0ada50 con 0x7f44f8068ca0 2024-04-25T13:40:43.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44fdd73640 1 -- 172.21.15.80:0/4077725409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8068ca0 msgr2=0x7f44f8070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:43.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44fdd73640 1 --2- 172.21.15.80:0/4077725409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8068ca0 0x7f44f8070920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f44f80cf340 tx=0x7f44ec01c490 comp rx=0 tx=0).stop 2024-04-25T13:40:43.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44fdd73640 1 -- 172.21.15.80:0/4077725409 shutdown_connections 2024-04-25T13:40:43.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44fdd73640 1 --2- 172.21.15.80:0/4077725409 >> v2:172.21.15.87:3300/0 conn(0x7f44f80ccf60 0x7f44f8164100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44fdd73640 1 --2- 172.21.15.80:0/4077725409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8068ca0 0x7f44f8070920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44fdd73640 1 -- 172.21.15.80:0/4077725409 >> 172.21.15.80:0/4077725409 conn(0x7f44f80cf390 msgr2=0x7f44f80cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:43.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44fdd73640 1 -- 172.21.15.80:0/4077725409 shutdown_connections 2024-04-25T13:40:43.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.028+0000 7f44fdd73640 1 -- 172.21.15.80:0/4077725409 wait complete. 2024-04-25T13:40:43.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 Processor -- start 2024-04-25T13:40:43.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 -- start start 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80ccf60 0x7f44f813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8160400 0x7f44f813d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44f815bf00 0x7f44f815c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f44f816d260 con 0x7f44f80ccf60 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f44f806f310 con 0x7f44f8160400 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f44f816d3e0 con 0x7f44f815bf00 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80ccf60 0x7f44f813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80ccf60 0x7f44f813add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44408/0 (socket says 172.21.15.80:44408) 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 -- 172.21.15.80:0/3055986875 learned_addr learned my addr 172.21.15.80:0/3055986875 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f6ffd640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8160400 0x7f44f813d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f7fff640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44f815bf00 0x7f44f815c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:43.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 -- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44f815bf00 msgr2=0x7f44f815c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44f815bf00 0x7f44f815c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 -- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8160400 msgr2=0x7f44f813d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8160400 0x7f44f813d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 -- 172.21.15.80:0/3055986875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f815c850 con 0x7f44f80ccf60 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44f77fe640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80ccf60 0x7f44f813add0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c340 tx=0x7f44e800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44dffff640 1 -- 172.21.15.80:0/3055986875 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e80052d0 con 0x7f44f80ccf60 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44dffff640 1 -- 172.21.15.80:0/3055986875 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44e8005470 con 0x7f44f80ccf60 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44dffff640 1 -- 172.21.15.80:0/3055986875 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8007260 con 0x7f44f80ccf60 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f813dfe0 con 0x7f44f80ccf60 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f813b6a0 con 0x7f44f80ccf60 2024-04-25T13:40:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.032+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f813b490 con 0x7f44f80ccf60 2024-04-25T13:40:43.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.036+0000 7f44dffff640 1 -- 172.21.15.80:0/3055986875 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f44e8007400 con 0x7f44f80ccf60 2024-04-25T13:40:43.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.036+0000 7f44dffff640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44e406f400 0x7f44e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:43.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.036+0000 7f44dffff640 1 -- 172.21.15.80:0/3055986875 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f44e808eaf0 con 0x7f44f80ccf60 2024-04-25T13:40:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.036+0000 7f44f6ffd640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44e406f400 0x7f44e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.036+0000 7f44f6ffd640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44e406f400 0x7f44e4071910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0af000 tx=0x7f44ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:43.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.040+0000 7f44dffff640 1 -- 172.21.15.80:0/3055986875 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44f815c850 con 0x7f44f80ccf60 2024-04-25T13:40:43.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.228+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f44f806a3a0 con 0x7f44f80ccf60 2024-04-25T13:40:43.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44dffff640 1 -- 172.21.15.80:0/3055986875 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f44f80cd370 con 0x7f44f80ccf60 2024-04-25T13:40:43.231 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:40:43.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44e406f400 msgr2=0x7f44e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:43.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44e406f400 0x7f44e4071910 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f44ec0af000 tx=0x7f44ec008040 comp rx=0 tx=0).stop 2024-04-25T13:40:43.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80ccf60 msgr2=0x7f44f813add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:43.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80ccf60 0x7f44f813add0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f44e800c340 tx=0x7f44e800c800 comp rx=0 tx=0).stop 2024-04-25T13:40:43.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 shutdown_connections 2024-04-25T13:40:43.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44e406f400 0x7f44e4071910 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44f815bf00 0x7f44f815c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8160400 0x7f44f813d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 --2- 172.21.15.80:0/3055986875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f80ccf60 0x7f44f813add0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:43.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 >> 172.21.15.80:0/3055986875 conn(0x7f44f80cf390 msgr2=0x7f44f80b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:43.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 shutdown_connections 2024-04-25T13:40:43.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43.232+0000 7f44fdd73640 1 -- 172.21.15.80:0/3055986875 wait complete. 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:40:43.251 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:40:43.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:43.252 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:40:43.257 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:40:43.257 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:40:43.258 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:43.259 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:40:43.259 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42525sec preferred_lft 42525sec 2024-04-25T13:40:43.259 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:40:43.259 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:43.259 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:40:43.259 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:43.259 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:40:43.259 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:43.259 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:40:43.260 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:43.260 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:43.260 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:40:43.260 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip addr flush dev ceph-brx 2024-04-25T13:40:43.260 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set ceph-brx up 2024-04-25T13:40:43.260 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:40:43.260 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:43.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:43.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:43.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:43.381 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:40:43.434 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:40:43.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:43.435 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:40:43.498 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:40:43.498 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:40:43.498 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:40:43.498 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:40:43.499 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:40:43.499 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:40:43.499 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:40:43.499 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:43.499 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:43.499 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:43.499 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:40:43.499 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:43.499 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:43.499 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:40:43.500 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:40:43.500 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:40:43.500 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:43.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:43.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:43.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:43.614 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:40:43.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:43.659 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list-id 2024-04-25T13:40:43.707 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:43.707 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:43.707 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:43.707 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T13:40:43.707 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:43.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:43.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:43.813 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T13:40:43.813 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:43.813 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:43.813 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T13:40:43.813 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:40:43.813 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T13:40:43.814 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T13:40:43.814 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T13:40:43.814 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:43.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:43.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:43.962 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:43.962 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:43.962 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:40:43.962 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:40:43.962 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:44.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:44.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:44.063 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:40:44.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:40:44.064 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:44.111 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:40:44.111 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:40:44.111 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:44.158 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:40:44.214 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:44.262 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:40:44.262 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:40:44.262 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:40:44.262 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:40:44.262 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:40:44.262 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:40:44.262 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:40:44.262 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:44.263 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:44.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:44.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:44.266 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:40:44.312 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:40:44.312 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:40:44.356 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:40:44.369 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:40:44.416 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:40:44.417 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:44.418 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:44.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:44.420 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:40:44.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:40:44.460+0000 7f6de0ef9580 -1 init, newargv = 0x7f6dcc005870 newargc=15 2024-04-25T13:40:44.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[25104]: starting ceph client 2024-04-25T13:40:44.468 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[25104]: starting fuse 2024-04-25T13:40:45.567 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:40:45.584 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:45.632 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:40:45.633 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:40:45.634 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:40:45.635 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:40:45.635 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:40:45.635 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:45.635 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:45.635 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:45.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:45.636 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:40:45.683 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:40:45.683 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:40:45.684 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:40:45.685 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:40:45.807 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.25104.asok 2024-04-25T13:40:45.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:45.818 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.25104.asok 2024-04-25T13:40:45.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:45.818 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.25104.asok status 2024-04-25T13:40:45.998 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:40:45.998 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:40:45.998 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:40:45.998 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "25104", 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 4620, 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 4620 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:40:45.999 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 122796286 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 122796286 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.4620 192.168.0.1:0/122796286", 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/122796286", 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 7, 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 20, 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:40:46.000 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:40:46.001 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:40:46.015 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:40:46.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.164+0000 7f404b5af640 1 Processor -- start 2024-04-25T13:40:46.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404b5af640 1 -- start start 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404b5af640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f404406f5a0 0x7f4044160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404b5af640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 0x7f4044161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404b5af640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f404406fcf0 con 0x7f40440b4bd0 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404b5af640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f40441619c0 con 0x7f404406f5a0 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404b5af640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4044073f10 con 0x7f4044161070 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4049b4c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f40440b4bd0 0x7f40440b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45062/0 (socket says 172.21.15.80:45062) 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4049b4c640 1 -- 172.21.15.80:0/2459133985 learned_addr learned my addr 172.21.15.80:0/2459133985 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404934b640 1 --2- 172.21.15.80:0/2459133985 >> v2:172.21.15.87:3300/0 conn(0x7f404406f5a0 0x7f4044160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4048b4a640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 0x7f4044161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 759284283 0 0) 0x7f404406fcf0 con 0x7f40440b4bd0 2024-04-25T13:40:46.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4030003530 con 0x7f40440b4bd0 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3404833076 0 0) 0x7f4030003530 con 0x7f40440b4bd0 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 msgr2=0x7f4044161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 0x7f4044161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 >> v2:172.21.15.87:3300/0 conn(0x7f404406f5a0 msgr2=0x7f4044160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 --2- 172.21.15.80:0/2459133985 >> v2:172.21.15.87:3300/0 conn(0x7f404406f5a0 0x7f4044160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440b0c60 con 0x7f40440b4bd0 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f403c001a00 con 0x7f40440b4bd0 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 >> v1:172.21.15.80:6789/0 conn(0x7f40440b4bd0 legacy=0x7f40440b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 0x7f40300039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f4030004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4030004d50 0x7f4030005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f40441619c0 con 0x7f4030003f70 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4044073f10 con 0x7f4030004d50 2024-04-25T13:40:46.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f403000dd10 con 0x7f4044161070 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4048b4a640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f4030004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404934b640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 0x7f40300039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4049b4c640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4030004d50 0x7f4030005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4048b4a640 1 -- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 msgr2=0x7f40300039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4048b4a640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 0x7f40300039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4048b4a640 1 -- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4030004d50 msgr2=0x7f4030005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4048b4a640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4030004d50 0x7f4030005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f4048b4a640 1 -- 172.21.15.80:0/2459133985 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4030003530 con 0x7f4030003f70 2024-04-25T13:40:46.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.168+0000 7f404934b640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044161070 0x7f40300039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:46.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f4048b4a640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f4030004810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4034009ae0 tx=0x7f403401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c02cb8c0720e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340096b0 con 0x7f4030003f70 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40340a6460 con 0x7f4030003f70 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f403bfff640 1 -- 172.21.15.80:0/2459133985 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40340a6850 con 0x7f4030003f70 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f404b5af640 1 -- 172.21.15.80:0/2459133985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 msgr2=0x7f4030004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f404b5af640 1 --2- 172.21.15.80:0/2459133985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f4030004810 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4034009ae0 tx=0x7f403401ca90 comp rx=0 tx=0).stop 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f4049b4c640 1 -- 172.21.15.80:0/2459133985 reap_dead start 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f404b5af640 1 -- 172.21.15.80:0/2459133985 shutdown_connections 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f404b5af640 1 -- 172.21.15.80:0/2459133985 >> 172.21.15.80:0/2459133985 conn(0x7f404406d080 msgr2=0x7f40440b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f404b5af640 1 -- 172.21.15.80:0/2459133985 shutdown_connections 2024-04-25T13:40:46.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.172+0000 7f404b5af640 1 -- 172.21.15.80:0/2459133985 wait complete. 2024-04-25T13:40:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 Processor -- start 2024-04-25T13:40:46.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 -- start start 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f40440ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4044159920 0x7f40441451a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044147bd0 0x7f4044147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4044161600 con 0x7f4030003f70 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4044161480 con 0x7f4044159920 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4044161780 con 0x7f4044147bd0 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f40440ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f40440ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44440/0 (socket says 172.21.15.80:44440) 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 -- 172.21.15.80:0/3222573636 learned_addr learned my addr 172.21.15.80:0/3222573636 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f4048b4a640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4044159920 0x7f40441451a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f4049b4c640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044147bd0 0x7f4044147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 -- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044147bd0 msgr2=0x7f4044147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044147bd0 0x7f4044147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 -- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4044159920 msgr2=0x7f40441451a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4044159920 0x7f40441451a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 -- 172.21.15.80:0/3222573636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40440ab9e0 con 0x7f4030003f70 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f4048b4a640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4044159920 0x7f40441451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f4049b4c640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044147bd0 0x7f4044147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404934b640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f40440ab4a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4040010920 tx=0x7f4040010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f4039ffb640 1 -- 172.21.15.80:0/3222573636 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404000f040 con 0x7f4030003f70 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f4039ffb640 1 -- 172.21.15.80:0/3222573636 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404000ebc0 con 0x7f4030003f70 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4044148520 con 0x7f4030003f70 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f4039ffb640 1 -- 172.21.15.80:0/3222573636 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404001c400 con 0x7f4030003f70 2024-04-25T13:40:46.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40441461b0 con 0x7f4030003f70 2024-04-25T13:40:46.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.176+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f404406a3a0 con 0x7f4030003f70 2024-04-25T13:40:46.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.180+0000 7f4039ffb640 1 -- 172.21.15.80:0/3222573636 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f404001c5a0 con 0x7f4030003f70 2024-04-25T13:40:46.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.180+0000 7f4039ffb640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f402806f400 0x7f4028071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:46.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.180+0000 7f4039ffb640 1 -- 172.21.15.80:0/3222573636 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f404008f9f0 con 0x7f4030003f70 2024-04-25T13:40:46.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.180+0000 7f4048b4a640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f402806f400 0x7f4028071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:46.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.180+0000 7f4048b4a640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f402806f400 0x7f4028071910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40340ad000 tx=0x7f40340a9040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:46.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.184+0000 7f4039ffb640 1 -- 172.21.15.80:0/3222573636 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4040018030 con 0x7f4030003f70 2024-04-25T13:40:46.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.384+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f40440739c0 con 0x7f4030003f70 2024-04-25T13:40:46.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.384+0000 7f4039ffb640 1 -- 172.21.15.80:0/3222573636 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f404005c8e0 con 0x7f4030003f70 2024-04-25T13:40:46.386 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f402806f400 msgr2=0x7f4028071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f402806f400 0x7f4028071910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f40340ad000 tx=0x7f40340a9040 comp rx=0 tx=0).stop 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 msgr2=0x7f40440ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f40440ab4a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4040010920 tx=0x7f4040010de0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 shutdown_connections 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f402806f400 0x7f4028071910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4044147bd0 0x7f4044147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4044159920 0x7f40441451a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 --2- 172.21.15.80:0/3222573636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4030003f70 0x7f40440ab4a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 >> 172.21.15.80:0/3222573636 conn(0x7f404406d080 msgr2=0x7f40440d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:46.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 shutdown_connections 2024-04-25T13:40:46.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46.388+0000 7f404b5af640 1 -- 172.21.15.80:0/3222573636 wait complete. 2024-04-25T13:40:46.407 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:40:46.407 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:40:46.407 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:40:46.407 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T13:40:46.407 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T13:40:46.407 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:40:46.408 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:40:46.408 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:40:46.408 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T13:40:46.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:46.408 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:46.414 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42522sec preferred_lft 42522sec 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:40:46.415 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:46.417 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:40:46.465 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:40:46.466 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:46.466 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:40:46.514 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:40:46.514 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:40:46.514 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:40:46.514 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:40:46.514 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:40:46.514 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:40:46.515 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:40:46.515 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:46.515 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:46.515 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:46.515 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:40:46.515 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:46.515 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:46.515 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:40:46.515 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:40:46.515 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:40:46.515 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:46.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:46.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:46.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:46.638 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:40:46.644 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:40:46.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:46.644 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list-id 2024-04-25T13:40:46.692 INFO:teuthology.orchestra.run.smithi080.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T13:40:46.693 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:46.693 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:46.693 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:40:46.693 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T13:40:46.693 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:46.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:46.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:46.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:46.808 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 4e:8f:1a:a8:f5:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T13:40:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::4c8f:1aff:fea8:f502/64 scope link 2024-04-25T13:40:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:46.821 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 4e:8f:1a:a8:f5:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.872 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::4c8f:1aff:fea8:f502/64 scope link 2024-04-25T13:40:46.873 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:46.874 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T13:40:46.874 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:46.874 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:46.874 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T13:40:46.874 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T13:40:46.874 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T13:40:46.874 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T13:40:46.874 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T13:40:46.874 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:46.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:47.010 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:47.011 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:40:47.011 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.1 up 2024-04-25T13:40:47.011 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T13:40:47.011 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:40:47.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:47.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:47.123 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T13:40:47.123 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:40:47.123 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T13:40:47.174 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T13:40:47.174 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:40:47.175 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T13:40:47.223 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:40:47.281 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:47.328 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:40:47.328 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:40:47.328 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:40:47.328 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:47.329 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:40:47.330 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:40:47.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:47.331 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:40:47.379 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:40:47.379 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T13:40:47.379 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T13:40:47.424 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:40:47.438 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:47.487 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:47.488 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:40:47.489 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:47.490 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:47.490 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:47.490 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:40:47.490 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:40:47.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:47.491 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:40:47.534 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi080.stderr:2024-04-25T13:40:47.532+0000 7f6f759db580 -1 init, newargv = 0x555a66fa2120 newargc=15 2024-04-25T13:40:47.534 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi080.stderr:ceph-fuse[25249]: starting ceph client 2024-04-25T13:40:47.538 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:40:47.541 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi080.stderr:ceph-fuse[25249]: starting fuse 2024-04-25T13:40:48.639 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:40:48.654 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:48.705 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:40:48.705 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:40:48.705 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:40:48.705 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:40:48.705 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:40:48.705 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:40:48.705 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:40:48.705 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:40:48.706 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:48.707 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:40:48.708 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:40:48.708 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.2 2024-04-25T13:40:48.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:48.709 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:40:48.755 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:40:48.756 INFO:teuthology.orchestra.run.smithi080.stdout:46 2024-04-25T13:40:48.756 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T13:40:48.756 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:40:48.756 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:40:48.756 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:40:48.756 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:40:48.756 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:40:48.756 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:40:48.756 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:40:48.756 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:40:48.757 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:40:48.758 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:40:48.758 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.2")) 2024-04-25T13:40:48.758 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:40:48.878 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.2.25249.asok 2024-04-25T13:40:48.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:48.889 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.25249.asok 2024-04-25T13:40:48.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:48.890 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.25249.asok status 2024-04-25T13:40:49.063 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:40:49.063 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:40:49.063 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:40:49.063 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:40:49.063 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "2", 2024-04-25T13:40:49.063 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:40:49.063 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T13:40:49.063 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "25249", 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 4632, 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 4632 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.2:0", 2024-04-25T13:40:49.064 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3029322498 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.2:0", 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3029322498 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.4632 192.168.0.2:0/3029322498", 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.2:0/3029322498", 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 7, 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 20, 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:40:49.065 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:40:49.066 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:40:49.066 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:40:49.079 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:40:49.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ceed3640 1 Processor -- start 2024-04-25T13:40:49.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ceed3640 1 -- start start 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ceed3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f24c80d2bc0 0x7f24c80d2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ceed3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c80b9440 0x7f24c80b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ceed3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24c80d3510 con 0x7f24c8160640 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ceed3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f24c80d5630 con 0x7f24c80d2bc0 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ceed3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f24c80d4a20 con 0x7f24c80b9440 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24c7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f24c80d2bc0 0x7f24c80d2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ccc6f640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f24c8160640 0x7f24c80d4140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45064/0 (socket says 172.21.15.80:45064) 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24ccc6f640 1 -- 172.21.15.80:0/505921568 learned_addr learned my addr 172.21.15.80:0/505921568 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24c7fff640 1 -- 172.21.15.80:0/505921568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c80b9440 msgr2=0x7f24c80b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24cd470640 1 --2- 172.21.15.80:0/505921568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c80b9440 0x7f24c80b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24c77fe640 1 -- 172.21.15.80:0/505921568 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 938132011 0 0) 0x7f24b0001120 con 0x7f24c8160640 2024-04-25T13:40:49.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24c7fff640 1 --2- 172.21.15.80:0/505921568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c80b9440 0x7f24c80b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24c7fff640 1 -- 172.21.15.80:0/505921568 >> v1:172.21.15.80:6789/0 conn(0x7f24c8160640 legacy=0x7f24c80d4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.227+0000 7f24c7fff640 1 -- 172.21.15.80:0/505921568 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80d4d00 con 0x7f24c80d2bc0 2024-04-25T13:40:49.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24c7fff640 1 --2- 172.21.15.80:0/505921568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80d2bc0 0x7f24c80d2fd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009d90 tx=0x7f24c001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c434847f97111a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:49.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24c77fe640 1 -- 172.21.15.80:0/505921568 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00b2070 con 0x7f24c80d2bc0 2024-04-25T13:40:49.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24c77fe640 1 -- 172.21.15.80:0/505921568 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24c00a5940 con 0x7f24c80d2bc0 2024-04-25T13:40:49.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24c77fe640 1 -- 172.21.15.80:0/505921568 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c00a5d10 con 0x7f24c80d2bc0 2024-04-25T13:40:49.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24ceed3640 1 -- 172.21.15.80:0/505921568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80d2bc0 msgr2=0x7f24c80d2fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24ceed3640 1 --2- 172.21.15.80:0/505921568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80d2bc0 0x7f24c80d2fd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009d90 tx=0x7f24c001c970 comp rx=0 tx=0).stop 2024-04-25T13:40:49.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24ceed3640 1 -- 172.21.15.80:0/505921568 shutdown_connections 2024-04-25T13:40:49.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24ceed3640 1 --2- 172.21.15.80:0/505921568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c80b9440 0x7f24c80b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24ceed3640 1 --2- 172.21.15.80:0/505921568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80d2bc0 0x7f24c80d2fd0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24c0009d90 tx=0x7f24c001c970 comp rx=0 tx=0).stop 2024-04-25T13:40:49.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24ceed3640 1 -- 172.21.15.80:0/505921568 >> 172.21.15.80:0/505921568 conn(0x7f24c806c9f0 msgr2=0x7f24c806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:49.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24ceed3640 1 -- 172.21.15.80:0/505921568 shutdown_connections 2024-04-25T13:40:49.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.231+0000 7f24ceed3640 1 -- 172.21.15.80:0/505921568 wait complete. 2024-04-25T13:40:49.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 Processor -- start 2024-04-25T13:40:49.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 -- start start 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80b9440 0x7f24c8153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c8160640 0x7f24c815d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80ceff0 0x7f24c815a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f24c816d0c0 con 0x7f24c80ceff0 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f24c80ba7b0 con 0x7f24c80b9440 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f24c816d240 con 0x7f24c8160640 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80ceff0 0x7f24c815a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80ceff0 0x7f24c815a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44460/0 (socket says 172.21.15.80:44460) 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 -- 172.21.15.80:0/3995231148 learned_addr learned my addr 172.21.15.80:0/3995231148 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ccc6f640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80b9440 0x7f24c8153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24c7fff640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c8160640 0x7f24c815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 -- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c8160640 msgr2=0x7f24c815d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c8160640 0x7f24c815d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 -- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80b9440 msgr2=0x7f24c8153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80b9440 0x7f24c8153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 -- 172.21.15.80:0/3995231148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c80cdf50 con 0x7f24c80ceff0 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24cd470640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80ceff0 0x7f24c815a730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c510 tx=0x7f24b800c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24c57fa640 1 -- 172.21.15.80:0/3995231148 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8015070 con 0x7f24c80ceff0 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24c57fa640 1 -- 172.21.15.80:0/3995231148 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24b8004220 con 0x7f24c80ceff0 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c80ce230 con 0x7f24c80ceff0 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24c57fa640 1 -- 172.21.15.80:0/3995231148 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8003620 con 0x7f24c80ceff0 2024-04-25T13:40:49.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.235+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c80ce7f0 con 0x7f24c80ceff0 2024-04-25T13:40:49.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.239+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24c80cdcc0 con 0x7f24c80ceff0 2024-04-25T13:40:49.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.239+0000 7f24c57fa640 1 -- 172.21.15.80:0/3995231148 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f24b80037c0 con 0x7f24c80ceff0 2024-04-25T13:40:49.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.239+0000 7f24c57fa640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f24a806f400 0x7f24a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:49.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.239+0000 7f24c57fa640 1 -- 172.21.15.80:0/3995231148 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f24b808f030 con 0x7f24c80ceff0 2024-04-25T13:40:49.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.239+0000 7f24ccc6f640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f24a806f400 0x7f24a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:49.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.239+0000 7f24ccc6f640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f24a806f400 0x7f24a8071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24b0007fd0 tx=0x7f24b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:49.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.243+0000 7f24c57fa640 1 -- 172.21.15.80:0/3995231148 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24b805bf20 con 0x7f24c80ceff0 2024-04-25T13:40:49.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f24c806a3a0 con 0x7f24c80ceff0 2024-04-25T13:40:49.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24c57fa640 1 -- 172.21.15.80:0/3995231148 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7f24b805b820 con 0x7f24c80ceff0 2024-04-25T13:40:49.444 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:40:49.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f24a806f400 msgr2=0x7f24a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f24a806f400 0x7f24a8071910 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f24b0007fd0 tx=0x7f24b0009040 comp rx=0 tx=0).stop 2024-04-25T13:40:49.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80ceff0 msgr2=0x7f24c815a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:49.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80ceff0 0x7f24c815a730 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c510 tx=0x7f24b800c9d0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 shutdown_connections 2024-04-25T13:40:49.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f24a806f400 0x7f24a8071910 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24c80ceff0 0x7f24c815a730 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24c8160640 0x7f24c815d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 --2- 172.21.15.80:0/3995231148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24c80b9440 0x7f24c8153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:49.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.443+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 >> 172.21.15.80:0/3995231148 conn(0x7f24c806c9f0 msgr2=0x7f24c8160e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:49.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.447+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 shutdown_connections 2024-04-25T13:40:49.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:49.447+0000 7f24ceed3640 1 -- 172.21.15.80:0/3995231148 wait complete. 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:40:49.464 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:40:49.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:49.465 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:49.472 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42519sec preferred_lft 42519sec 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:40:49.473 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:49.474 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T13:40:49.474 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:49.474 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:49.474 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-04-25T13:40:49.474 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-04-25T13:40:49.474 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-04-25T13:40:49.474 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T13:40:49.474 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:49.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:49.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:49.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:49.604 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:40:49.659 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:40:49.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:49.661 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:40:49.733 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:40:49.733 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:40:49.733 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:40:49.733 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:40:49.734 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:40:49.734 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:40:49.734 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:40:49.734 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:49.734 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:49.734 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:49.734 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:40:49.734 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:49.734 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:49.735 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:40:49.735 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:40:49.735 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:40:49.735 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:49.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:49.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:49.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:49.855 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:40:49.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:49.860 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-04-25T13:40:49.906 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:49.906 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:49.906 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:49.906 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T13:40:49.907 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:49.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:50.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:50.024 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T13:40:50.024 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:50.024 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:50.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T13:40:50.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T13:40:50.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T13:40:50.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T13:40:50.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T13:40:50.024 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:50.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:50.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:50.171 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:50.171 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:50.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:40:50.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T13:40:50.171 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:50.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:50.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:50.273 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:40:50.273 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:40:50.273 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:50.318 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:40:50.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:40:50.319 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:50.366 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:40:50.424 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:50.472 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:40:50.472 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:40:50.472 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:40:50.472 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:40:50.472 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:40:50.472 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:40:50.472 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:40:50.472 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:50.473 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:50.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:50.475 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:40:50.523 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:40:50.523 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:40:50.568 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:40:50.582 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:50.631 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:40:50.631 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:40:50.631 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:40:50.631 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:40:50.631 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:40:50.632 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:50.633 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:50.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:50.634 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:40:50.678 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:40:50.678+0000 7f55aea47580 -1 init, newargv = 0x7f5590005ec0 newargc=15 2024-04-25T13:40:50.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[23928]: starting ceph client 2024-04-25T13:40:50.684 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[23928]: starting fuse 2024-04-25T13:40:51.782 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:40:51.799 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:51.848 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:40:51.849 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:40:51.850 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:40:51.851 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:40:51.851 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:40:51.851 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:51.851 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:51.851 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:51.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:51.851 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:40:51.895 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:40:51.895 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:40:51.895 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:40:51.895 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:40:51.895 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:40:51.895 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:40:51.896 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:40:51.897 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:40:51.897 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:40:51.897 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:40:51.897 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:40:51.897 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:40:51.897 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:40:52.018 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.23928.asok 2024-04-25T13:40:52.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:52.029 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.23928.asok 2024-04-25T13:40:52.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:52.029 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.23928.asok status 2024-04-25T13:40:52.208 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:40:52.208 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "23928", 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4534, 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:40:52.209 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 4534 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 734655816 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 734655816 2024-04-25T13:40:52.210 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.4534 192.168.0.1:0/734655816", 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/734655816", 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 7, 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 20, 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:40:52.211 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:40:52.224 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:40:52.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7feaa1404640 1 Processor -- start 2024-04-25T13:40:52.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7feaa1404640 1 -- start start 2024-04-25T13:40:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7feaa1404640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c0d19a0 0x7fea9c157d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7feaa1404640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fea9c1582c0 0x7fea9c1586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7feaa1404640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea9c15eef0 con 0x7fea9c15eae0 2024-04-25T13:40:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7feaa1404640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fea9c06ec70 con 0x7fea9c1582c0 2024-04-25T13:40:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7feaa1404640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fea9c15f1b0 con 0x7fea9c0d19a0 2024-04-25T13:40:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7fea9affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fea9c15eae0 0x7fea9c0d1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45136/0 (socket says 172.21.15.80:45136) 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7fea9affd640 1 -- 172.21.15.80:0/1765417767 learned_addr learned my addr 172.21.15.80:0/1765417767 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1538133273 0 0) 0x7fea9c15eef0 con 0x7fea9c15eae0 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fea88003530 con 0x7fea9c15eae0 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7fea9b7fe640 1 --2- 172.21.15.80:0/1765417767 >> v2:172.21.15.87:3300/0 conn(0x7fea9c1582c0 0x7fea9c1586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.383+0000 7fea9a7fc640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c0d19a0 0x7fea9c157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4052572974 0 0) 0x7fea88003530 con 0x7fea9c15eae0 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c0d19a0 msgr2=0x7fea9c157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c0d19a0 0x7fea9c157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 >> v2:172.21.15.87:3300/0 conn(0x7fea9c1582c0 msgr2=0x7fea9c1586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 --2- 172.21.15.80:0/1765417767 >> v2:172.21.15.87:3300/0 conn(0x7fea9c1582c0 0x7fea9c1586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c15f490 con 0x7fea9c15eae0 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fea94001a00 con 0x7fea9c15eae0 2024-04-25T13:40:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 >> v1:172.21.15.80:6789/0 conn(0x7fea9c15eae0 legacy=0x7fea9c0d1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c0d19a0 0x7fea880039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea88003f40 0x7fea88004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea88004d50 0x7fea88005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c06ec70 con 0x7fea88004d50 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c15f1b0 con 0x7fea88003f40 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fea8800bad0 con 0x7fea9c0d19a0 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9b7fe640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea88004d50 0x7fea88005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9affd640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c0d19a0 0x7fea880039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9b7fe640 1 -- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c0d19a0 msgr2=0x7fea880039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9b7fe640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c0d19a0 0x7fea880039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9b7fe640 1 -- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea88003f40 msgr2=0x7fea88004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9b7fe640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea88003f40 0x7fea88004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9b7fe640 1 -- 172.21.15.80:0/1765417767 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea88003530 con 0x7fea88004d50 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9b7fe640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea88004d50 0x7fea88005160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009ab0 tx=0x7fea8c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee0285dd173e8f28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0b2070 con 0x7fea88004d50 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8c0a5570 con 0x7fea88004d50 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea99ffb640 1 -- 172.21.15.80:0/1765417767 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c0096b0 con 0x7fea88004d50 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7feaa1404640 1 -- 172.21.15.80:0/1765417767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea88004d50 msgr2=0x7fea88005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7feaa1404640 1 --2- 172.21.15.80:0/1765417767 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea88004d50 0x7fea88005160 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009ab0 tx=0x7fea8c01ca90 comp rx=0 tx=0).stop 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7fea9b7fe640 1 -- 172.21.15.80:0/1765417767 reap_dead start 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7feaa1404640 1 -- 172.21.15.80:0/1765417767 shutdown_connections 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7feaa1404640 1 -- 172.21.15.80:0/1765417767 >> 172.21.15.80:0/1765417767 conn(0x7fea9c0d2170 msgr2=0x7fea9c0d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7feaa1404640 1 -- 172.21.15.80:0/1765417767 shutdown_connections 2024-04-25T13:40:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.387+0000 7feaa1404640 1 -- 172.21.15.80:0/1765417767 wait complete. 2024-04-25T13:40:52.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7feaa1404640 1 Processor -- start 2024-04-25T13:40:52.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7feaa1404640 1 -- start start 2024-04-25T13:40:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7feaa1404640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c149eb0 0x7fea9c156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7feaa1404640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c156a40 0x7fea9c155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7feaa1404640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea9c1557b0 0x7fea9c0b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7feaa1404640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c076e00 con 0x7fea9c156a40 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7feaa1404640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fea9c076c80 con 0x7fea9c1557b0 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7feaa1404640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fea9c0b1910 con 0x7fea9c149eb0 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c156a40 0x7fea9c155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c156a40 0x7fea9c155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48690/0 (socket says 172.21.15.80:48690) 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 -- 172.21.15.80:0/186439983 learned_addr learned my addr 172.21.15.80:0/186439983 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9affd640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c149eb0 0x7fea9c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9b7fe640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea9c1557b0 0x7fea9c0b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 -- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c149eb0 msgr2=0x7fea9c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c149eb0 0x7fea9c156500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 -- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea9c1557b0 msgr2=0x7fea9c0b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea9c1557b0 0x7fea9c0b5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 -- 172.21.15.80:0/186439983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea9c1541d0 con 0x7fea9c156a40 2024-04-25T13:40:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9b7fe640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea9c1557b0 0x7fea9c0b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea9a7fc640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c156a40 0x7fea9c155270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea8400e7f0 tx=0x7fea8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea937fe640 1 -- 172.21.15.80:0/186439983 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84017070 con 0x7fea9c156a40 2024-04-25T13:40:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea937fe640 1 -- 172.21.15.80:0/186439983 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea8400c940 con 0x7fea9c156a40 2024-04-25T13:40:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.391+0000 7fea937fe640 1 -- 172.21.15.80:0/186439983 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8400eea0 con 0x7fea9c156a40 2024-04-25T13:40:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.395+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea9c1544b0 con 0x7fea9c156a40 2024-04-25T13:40:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.395+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea9c154a70 con 0x7fea9c156a40 2024-04-25T13:40:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.395+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea9c06a3a0 con 0x7fea9c156a40 2024-04-25T13:40:52.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.395+0000 7fea937fe640 1 -- 172.21.15.80:0/186439983 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fea8401a070 con 0x7fea9c156a40 2024-04-25T13:40:52.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.395+0000 7fea937fe640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fea7c06f400 0x7fea7c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:52.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.395+0000 7fea937fe640 1 -- 172.21.15.80:0/186439983 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fea8408f420 con 0x7fea9c156a40 2024-04-25T13:40:52.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.395+0000 7fea9affd640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fea7c06f400 0x7fea7c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:52.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.395+0000 7fea9affd640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fea7c06f400 0x7fea7c071910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea9c153e10 tx=0x7fea94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:52.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.399+0000 7fea937fe640 1 -- 172.21.15.80:0/186439983 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fea8405c430 con 0x7fea9c156a40 2024-04-25T13:40:52.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.591+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fea9c06d670 con 0x7fea9c156a40 2024-04-25T13:40:52.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7fea937fe640 1 -- 172.21.15.80:0/186439983 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+73 (secure 0 0 0) 0x7fea84064420 con 0x7fea9c156a40 2024-04-25T13:40:52.595 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:40:52.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fea7c06f400 msgr2=0x7fea7c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fea7c06f400 0x7fea7c071910 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea9c153e10 tx=0x7fea94009040 comp rx=0 tx=0).stop 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c156a40 msgr2=0x7fea9c155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c156a40 0x7fea9c155270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fea8400e7f0 tx=0x7fea8400ecb0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 shutdown_connections 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fea7c06f400 0x7fea7c071910 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fea9c1557b0 0x7fea9c0b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fea9c156a40 0x7fea9c155270 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 --2- 172.21.15.80:0/186439983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fea9c149eb0 0x7fea9c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 >> 172.21.15.80:0/186439983 conn(0x7fea9c0d2170 msgr2=0x7fea9c0b2c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 shutdown_connections 2024-04-25T13:40:52.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:52.595+0000 7feaa1404640 1 -- 172.21.15.80:0/186439983 wait complete. 2024-04-25T13:40:52.614 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:40:52.614 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:40:52.614 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:40:52.614 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T13:40:52.614 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T13:40:52.614 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:40:52.614 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:40:52.614 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:40:52.615 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T13:40:52.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:52.615 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:40:52.622 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:40:52.622 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:40:52.622 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:40:52.622 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:52.622 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:40:52.622 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:52.622 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:52.622 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42516sec preferred_lft 42516sec 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:52.623 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:52.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:52.624 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:40:52.678 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:40:52.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:52.679 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:40:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:40:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:40:52.730 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:40:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:40:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:40:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:40:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:40:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:40:52.731 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:40:52.731 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:52.731 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:52.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:40:52.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:40:52.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:40:52.732 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:52.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:52.848 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:52.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:52.856 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:40:52.903 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:40:52.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:52.904 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-04-25T13:40:52.951 INFO:teuthology.orchestra.run.smithi087.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T13:40:52.951 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:52.951 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:52.951 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:40:52.952 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T13:40:52.952 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:53.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:53.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:53.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:53.067 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T13:40:53.079 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5a:33:99:39:df:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5833:99ff:fe39:df5a/64 scope link 2024-04-25T13:40:53.080 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:53.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:53.081 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5a:33:99:39:df:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T13:40:53.138 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T13:40:53.139 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:53.139 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5833:99ff:fe39:df5a/64 scope link 2024-04-25T13:40:53.139 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:40:53.140 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T13:40:53.140 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:53.140 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:53.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T13:40:53.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T13:40:53.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T13:40:53.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T13:40:53.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T13:40:53.140 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:53.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:53.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:53.284 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:40:53.284 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:40:53.284 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.1 up 2024-04-25T13:40:53.284 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T13:40:53.284 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:40:53.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:40:53.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:53.343 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T13:40:53.344 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:40:53.344 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T13:40:53.390 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T13:40:53.390 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:40:53.390 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T13:40:53.442 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:40:53.500 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:53.547 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:40:53.547 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:40:53.548 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:40:53.548 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:40:53.548 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:40:53.548 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:40:53.548 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:40:53.548 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:40:53.548 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:40:53.549 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:40:53.550 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:40:53.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:53.551 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:40:53.599 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:40:53.599 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2024-04-25T13:40:53.599 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T13:40:53.646 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:40:53.660 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:40:53.710 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:40:53.711 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:40:53.712 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:40:53.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:53.713 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:40:53.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi087.stderr:2024-04-25T13:40:53.746+0000 7f4380d81580 -1 init, newargv = 0x7f4370001220 newargc=15 2024-04-25T13:40:53.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi087.stderr:ceph-fuse[24047]: starting ceph client 2024-04-25T13:40:53.754 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:40:53.755 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi087.stderr:ceph-fuse[24047]: starting fuse 2024-04-25T13:40:54.855 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:40:54.872 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:40:54.920 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:40:54.920 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:40:54.920 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:40:54.920 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:40:54.920 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:40:54.920 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:40:54.920 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:40:54.921 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:40:54.922 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.3 2024-04-25T13:40:54.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:54.923 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:40:54.971 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:40:54.971 INFO:teuthology.orchestra.run.smithi087.stdout:46 2024-04-25T13:40:54.971 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2024-04-25T13:40:54.971 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:40:54.971 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:40:54.971 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:40:54.972 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:40:54.973 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:40:54.973 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:40:54.973 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:40:54.973 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:40:54.973 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:40:54.973 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:40:54.973 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.3")) 2024-04-25T13:40:54.973 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:40:55.096 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.3.24047.asok 2024-04-25T13:40:55.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:40:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:40:55.108 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.24047.asok 2024-04-25T13:40:55.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.109 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.24047.asok status 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "3", 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "24047", 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:40:55.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4546, 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 4546 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.2:0", 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1376108049 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:40:55.289 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.2:0", 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1376108049 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.4546 192.168.0.2:0/1376108049", 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.2:0/1376108049", 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 7, 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 20, 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:40:55.290 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:40:55.291 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:40:55.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.306 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:40:55.306 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:55.313 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:40:55.313 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:55.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.314 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:40:55.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.371 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:40:55.371 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T13:40:55.421 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:40:55.421 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T13:40:55.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.421 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T13:40:55.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.480 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:40:55.480 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:55.487 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:40:55.488 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:55.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.488 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:40:55.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.543 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:40:55.544 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T13:40:55.592 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:40:55.592 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T13:40:55.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:40:55.592 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T13:40:55.647 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T13:40:55.662 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:55.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.819+0000 7fc2b8bee640 1 Processor -- start 2024-04-25T13:40:55.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.819+0000 7fc2b8bee640 1 -- start start 2024-04-25T13:40:55.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b8bee640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc2b40b0ed0 0x7fc2b415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:55.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b8bee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b406eba0 0x7fc2b40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:55.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b8bee640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2b415cda0 con 0x7fc2b40ce730 2024-04-25T13:40:55.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b8bee640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc2b4073710 con 0x7fc2b40b0ed0 2024-04-25T13:40:55.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b8bee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b40739f0 con 0x7fc2b406eba0 2024-04-25T13:40:55.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc2b40b0ed0 0x7fc2b415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:55.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc2b40b0ed0 0x7fc2b415c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48706/0 (socket says 172.21.15.80:48706) 2024-04-25T13:40:55.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b259c640 1 -- 172.21.15.80:0/1588018589 learned_addr learned my addr 172.21.15.80:0/1588018589 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:55.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b2d9d640 1 --2- 172.21.15.80:0/1588018589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b406eba0 0x7fc2b40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:55.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b259c640 1 -- 172.21.15.80:0/1588018589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b406eba0 msgr2=0x7fc2b40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:55.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b259c640 1 --2- 172.21.15.80:0/1588018589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b406eba0 0x7fc2b40ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b259c640 1 -- 172.21.15.80:0/1588018589 >> v1:172.21.15.87:6789/0 conn(0x7fc2b40ce730 legacy=0x7fc2b406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:55.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b259c640 1 -- 172.21.15.80:0/1588018589 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b4073cd0 con 0x7fc2b40b0ed0 2024-04-25T13:40:55.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.823+0000 7fc2b2d9d640 1 --2- 172.21.15.80:0/1588018589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b406eba0 0x7fc2b40ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:55.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b259c640 1 --2- 172.21.15.80:0/1588018589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40b0ed0 0x7fc2b415c860 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4071650 tx=0x7fc2a401ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1884f57d200e8995 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:55.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b159a640 1 -- 172.21.15.80:0/1588018589 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40b5070 con 0x7fc2b40b0ed0 2024-04-25T13:40:55.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b159a640 1 -- 172.21.15.80:0/1588018589 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a40a8990 con 0x7fc2b40b0ed0 2024-04-25T13:40:55.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b159a640 1 -- 172.21.15.80:0/1588018589 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40a8d60 con 0x7fc2b40b0ed0 2024-04-25T13:40:55.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1588018589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40b0ed0 msgr2=0x7fc2b415c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:55.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1588018589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40b0ed0 0x7fc2b415c860 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc2b4071650 tx=0x7fc2a401ecd0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1588018589 shutdown_connections 2024-04-25T13:40:55.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1588018589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b406eba0 0x7fc2b40ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1588018589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40b0ed0 0x7fc2b415c860 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1588018589 >> 172.21.15.80:0/1588018589 conn(0x7fc2b406d080 msgr2=0x7fc2b40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:55.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1588018589 shutdown_connections 2024-04-25T13:40:55.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.827+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1588018589 wait complete. 2024-04-25T13:40:55.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 Processor -- start 2024-04-25T13:40:55.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 -- start start 2024-04-25T13:40:55.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40b0ed0 0x7fc2b41f8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:55.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40ce730 0x7fc2b4140020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b415f170 0x7fc2b415f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b40733a0 con 0x7fc2b40ce730 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b40b12e0 con 0x7fc2b40b0ed0 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc2b406dc00 con 0x7fc2b415f170 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40ce730 0x7fc2b4140020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40ce730 0x7fc2b4140020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48722/0 (socket says 172.21.15.80:48722) 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 -- 172.21.15.80:0/1750243063 learned_addr learned my addr 172.21.15.80:0/1750243063 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b259c640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40b0ed0 0x7fc2b41f8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b2d9d640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b415f170 0x7fc2b415f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 -- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b415f170 msgr2=0x7fc2b415f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:55.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b415f170 0x7fc2b415f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 -- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40b0ed0 msgr2=0x7fc2b41f8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:55.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40b0ed0 0x7fc2b41f8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:55.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 -- 172.21.15.80:0/1750243063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b415fac0 con 0x7fc2b40ce730 2024-04-25T13:40:55.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b1d9b640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40ce730 0x7fc2b4140020 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800b410 tx=0x7fc2a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:55.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2a2ffd640 1 -- 172.21.15.80:0/1750243063 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8015030 con 0x7fc2b40ce730 2024-04-25T13:40:55.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2a2ffd640 1 -- 172.21.15.80:0/1750243063 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a8004020 con 0x7fc2b40ce730 2024-04-25T13:40:55.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2a2ffd640 1 -- 172.21.15.80:0/1750243063 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a8004450 con 0x7fc2b40ce730 2024-04-25T13:40:55.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b41461d0 con 0x7fc2b40ce730 2024-04-25T13:40:55.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.831+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b4146790 con 0x7fc2b40ce730 2024-04-25T13:40:55.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.835+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2b406a3a0 con 0x7fc2b40ce730 2024-04-25T13:40:55.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.835+0000 7fc2a2ffd640 1 -- 172.21.15.80:0/1750243063 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc2a8018070 con 0x7fc2b40ce730 2024-04-25T13:40:55.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.835+0000 7fc2a2ffd640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc29406f400 0x7fc294071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:55.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.835+0000 7fc2a2ffd640 1 -- 172.21.15.80:0/1750243063 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fc2a808ee10 con 0x7fc2b40ce730 2024-04-25T13:40:55.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.835+0000 7fc2b259c640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc29406f400 0x7fc294071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:55.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.835+0000 7fc2b259c640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc29406f400 0x7fc294071910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40b2000 tx=0x7fc2a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:55.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.839+0000 7fc2a2ffd640 1 -- 172.21.15.80:0/1750243063 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2b415fac0 con 0x7fc2b40ce730 2024-04-25T13:40:55.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.999+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2b415c210 con 0x7fc2b40ce730 2024-04-25T13:40:56.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:55.999+0000 7fc2a2ffd640 1 -- 172.21.15.80:0/1750243063 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7fc2a805bd90 con 0x7fc2b40ce730 2024-04-25T13:40:56.001 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:56.001 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:56.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc29406f400 msgr2=0x7fc294071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc29406f400 0x7fc294071910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc2a40b2000 tx=0x7fc2a4008040 comp rx=0 tx=0).stop 2024-04-25T13:40:56.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40ce730 msgr2=0x7fc2b4140020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40ce730 0x7fc2b4140020 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc2a800b410 tx=0x7fc2a800b8d0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 shutdown_connections 2024-04-25T13:40:56.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc29406f400 0x7fc294071910 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2b415f170 0x7fc2b415f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2b40ce730 0x7fc2b4140020 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 --2- 172.21.15.80:0/1750243063 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2b40b0ed0 0x7fc2b41f8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 >> 172.21.15.80:0/1750243063 conn(0x7fc2b406d080 msgr2=0x7fc2b40d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:56.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 shutdown_connections 2024-04-25T13:40:56.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.003+0000 7fc2b8bee640 1 -- 172.21.15.80:0/1750243063 wait complete. 2024-04-25T13:40:56.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:40:56.022 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:40:56.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a7c32640 1 Processor -- start 2024-04-25T13:40:56.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a7c32640 1 -- start start 2024-04-25T13:40:56.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a7c32640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2a0073ad0 0x7fa2a006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a7c32640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa2a006e9b0 0x7fa2a00ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a7c32640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa2a006edc0 con 0x7fa2a0073080 2024-04-25T13:40:56.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a7c32640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa2a00b53d0 con 0x7fa2a006e9b0 2024-04-25T13:40:56.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a7c32640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a0160b20 con 0x7fa2a0073ad0 2024-04-25T13:40:56.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a59ce640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa2a0073080 0x7fa2a015c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45152/0 (socket says 172.21.15.80:45152) 2024-04-25T13:40:56.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a59ce640 1 -- 172.21.15.80:0/501717801 learned_addr learned my addr 172.21.15.80:0/501717801 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:56.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4088004354 0 0) 0x7fa298001120 con 0x7fa2a0073080 2024-04-25T13:40:56.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a51cd640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2a0073ad0 0x7fa2a006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a61cf640 1 --2- 172.21.15.80:0/501717801 >> v2:172.21.15.87:3300/0 conn(0x7fa2a006e9b0 0x7fa2a00ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa28c003450 con 0x7fa2a0073080 2024-04-25T13:40:56.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3469230140 0 0) 0x7fa28c003450 con 0x7fa2a0073080 2024-04-25T13:40:56.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2a0073ad0 msgr2=0x7fa2a006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2a0073ad0 0x7fa2a006e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 >> v2:172.21.15.87:3300/0 conn(0x7fa2a006e9b0 msgr2=0x7fa2a00ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 --2- 172.21.15.80:0/501717801 >> v2:172.21.15.87:3300/0 conn(0x7fa2a006e9b0 0x7fa2a00ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a00737f0 con 0x7fa2a0073080 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fa298001850 con 0x7fa2a0073080 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a61cf640 1 --2- 172.21.15.80:0/501717801 >> v2:172.21.15.87:3300/0 conn(0x7fa2a006e9b0 0x7fa2a00ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 >> v1:172.21.15.80:6789/0 conn(0x7fa2a0073080 legacy=0x7fa2a015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa28c003ab0 0x7fa28c003f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa28c008860 0x7fa28c00ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2a0073ad0 0x7fa28c00d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a00b53d0 con 0x7fa28c008860 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0160b20 con 0x7fa28c003ab0 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa28c00dd10 con 0x7fa2a0073ad0 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a51cd640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa28c008860 0x7fa28c00ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a59ce640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa28c003ab0 0x7fa28c003f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a61cf640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2a0073ad0 0x7fa28c00d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a51cd640 1 -- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2a0073ad0 msgr2=0x7fa28c00d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:56.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a51cd640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa2a0073ad0 0x7fa28c00d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a51cd640 1 -- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa28c003ab0 msgr2=0x7fa28c003f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:56.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a51cd640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa28c003ab0 0x7fa28c003f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.159+0000 7fa2a51cd640 1 -- 172.21.15.80:0/501717801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa28c003450 con 0x7fa28c008860 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a51cd640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa28c008860 0x7fa28c00ad20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa290009ae0 tx=0x7fa29001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44d7daa05b7bb66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900b2070 con 0x7fa28c008860 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2900a5630 con 0x7fa28c008860 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a49cc640 1 -- 172.21.15.80:0/501717801 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900096b0 con 0x7fa28c008860 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a7c32640 1 -- 172.21.15.80:0/501717801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa28c008860 msgr2=0x7fa28c00ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a7c32640 1 --2- 172.21.15.80:0/501717801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa28c008860 0x7fa28c00ad20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa290009ae0 tx=0x7fa29001ca60 comp rx=0 tx=0).stop 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a61cf640 1 -- 172.21.15.80:0/501717801 reap_dead start 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a7c32640 1 -- 172.21.15.80:0/501717801 shutdown_connections 2024-04-25T13:40:56.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a7c32640 1 -- 172.21.15.80:0/501717801 >> 172.21.15.80:0/501717801 conn(0x7fa2a00b4160 msgr2=0x7fa2a00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:56.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a7c32640 1 -- 172.21.15.80:0/501717801 shutdown_connections 2024-04-25T13:40:56.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.163+0000 7fa2a7c32640 1 -- 172.21.15.80:0/501717801 wait complete. 2024-04-25T13:40:56.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 Processor -- start 2024-04-25T13:40:56.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 -- start start 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa28c008860 0x7fa2a014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2a014b320 0x7fa2a0140150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2a0151610 0x7fa2a01f57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a00cf380 con 0x7fa2a014b320 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a0073ee0 con 0x7fa2a0151610 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa2a00cf500 con 0x7fa28c008860 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2a014b320 0x7fa2a0140150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2a014b320 0x7fa2a0140150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48748/0 (socket says 172.21.15.80:48748) 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 -- 172.21.15.80:0/829683118 learned_addr learned my addr 172.21.15.80:0/829683118 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a59ce640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa28c008860 0x7fa2a014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a61cf640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2a0151610 0x7fa2a01f57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 -- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa28c008860 msgr2=0x7fa2a014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa28c008860 0x7fa2a014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 -- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2a0151610 msgr2=0x7fa2a01f57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2a0151610 0x7fa2a01f57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 -- 172.21.15.80:0/829683118 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a0150570 con 0x7fa2a014b320 2024-04-25T13:40:56.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a51cd640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2a014b320 0x7fa2a0140150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa290009ae0 tx=0x7fa28c003740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2967fc640 1 -- 172.21.15.80:0/829683118 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900bd070 con 0x7fa2a014b320 2024-04-25T13:40:56.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2967fc640 1 -- 172.21.15.80:0/829683118 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa29000f000 con 0x7fa2a014b320 2024-04-25T13:40:56.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2967fc640 1 -- 172.21.15.80:0/829683118 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2900b2040 con 0x7fa2a014b320 2024-04-25T13:40:56.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a0150850 con 0x7fa2a014b320 2024-04-25T13:40:56.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a0150e10 con 0x7fa2a014b320 2024-04-25T13:40:56.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.167+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a01502e0 con 0x7fa2a014b320 2024-04-25T13:40:56.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.171+0000 7fa2967fc640 1 -- 172.21.15.80:0/829683118 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa2900c0070 con 0x7fa2a014b320 2024-04-25T13:40:56.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.171+0000 7fa2967fc640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa28406f430 0x7fa284071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.171+0000 7fa2967fc640 1 -- 172.21.15.80:0/829683118 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa290128ff0 con 0x7fa2a014b320 2024-04-25T13:40:56.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.171+0000 7fa2a59ce640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa28406f430 0x7fa284071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.171+0000 7fa2a59ce640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa28406f430 0x7fa284071940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa298001300 tx=0x7fa298009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.175+0000 7fa2967fc640 1 -- 172.21.15.80:0/829683118 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa2900167a0 con 0x7fa2a014b320 2024-04-25T13:40:56.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.323+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa2a006a3a0 con 0x7fa2a014b320 2024-04-25T13:40:56.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.323+0000 7fa2967fc640 1 -- 172.21.15.80:0/829683118 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7fa2900f5890 con 0x7fa2a014b320 2024-04-25T13:40:56.326 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:40:56.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa28406f430 msgr2=0x7fa284071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa28406f430 0x7fa284071940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa298001300 tx=0x7fa298009040 comp rx=0 tx=0).stop 2024-04-25T13:40:56.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2a014b320 msgr2=0x7fa2a0140150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2a014b320 0x7fa2a0140150 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa290009ae0 tx=0x7fa28c003740 comp rx=0 tx=0).stop 2024-04-25T13:40:56.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 shutdown_connections 2024-04-25T13:40:56.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa28406f430 0x7fa284071940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2a0151610 0x7fa2a01f57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2a014b320 0x7fa2a0140150 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 --2- 172.21.15.80:0/829683118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa28c008860 0x7fa2a014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 >> 172.21.15.80:0/829683118 conn(0x7fa2a00b4160 msgr2=0x7fa2a0162e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:56.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 shutdown_connections 2024-04-25T13:40:56.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.327+0000 7fa2a7c32640 1 -- 172.21.15.80:0/829683118 wait complete. 2024-04-25T13:40:56.345 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:40:56.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6540b2d640 1 Processor -- start 2024-04-25T13:40:56.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6540b2d640 1 -- start start 2024-04-25T13:40:56.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6540b2d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f653c06edb0 0x7f653c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6540b2d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f653c161b60 0x7f653c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6540b2d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f653c1624b0 con 0x7f653c0b5330 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6540b2d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f653c06f700 con 0x7f653c06edb0 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6540b2d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f653c1629b0 con 0x7f653c161b60 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6539d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f653c06edb0 0x7f653c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6539d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f653c06edb0 0x7f653c06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48762/0 (socket says 172.21.15.80:48762) 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6539d9b640 1 -- 172.21.15.80:0/2135063983 learned_addr learned my addr 172.21.15.80:0/2135063983 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f653ad9d640 1 --2- 172.21.15.80:0/2135063983 >> v2:172.21.15.87:3300/0 conn(0x7f653c161b60 0x7f653c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6539d9b640 1 -- 172.21.15.80:0/2135063983 >> v2:172.21.15.87:3300/0 conn(0x7f653c161b60 msgr2=0x7f653c161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6539d9b640 1 --2- 172.21.15.80:0/2135063983 >> v2:172.21.15.87:3300/0 conn(0x7f653c161b60 0x7f653c161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6539d9b640 1 -- 172.21.15.80:0/2135063983 >> v1:172.21.15.87:6789/0 conn(0x7f653c0b5330 legacy=0x7f653c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:56.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.495+0000 7f6539d9b640 1 -- 172.21.15.80:0/2135063983 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c0b5740 con 0x7f653c06edb0 2024-04-25T13:40:56.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6539d9b640 1 --2- 172.21.15.80:0/2135063983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c06edb0 0x7f653c06f1c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f653c147cb0 tx=0x7f653001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a21500ba19741736 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f653959a640 1 -- 172.21.15.80:0/2135063983 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300b0070 con 0x7f653c06edb0 2024-04-25T13:40:56.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f653959a640 1 -- 172.21.15.80:0/2135063983 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65300ab600 con 0x7f653c06edb0 2024-04-25T13:40:56.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f653959a640 1 -- 172.21.15.80:0/2135063983 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300ab9d0 con 0x7f653c06edb0 2024-04-25T13:40:56.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6540b2d640 1 -- 172.21.15.80:0/2135063983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c06edb0 msgr2=0x7f653c06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6540b2d640 1 --2- 172.21.15.80:0/2135063983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c06edb0 0x7f653c06f1c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f653c147cb0 tx=0x7f653001c240 comp rx=0 tx=0).stop 2024-04-25T13:40:56.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6540b2d640 1 -- 172.21.15.80:0/2135063983 shutdown_connections 2024-04-25T13:40:56.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6540b2d640 1 --2- 172.21.15.80:0/2135063983 >> v2:172.21.15.87:3300/0 conn(0x7f653c161b60 0x7f653c161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6540b2d640 1 --2- 172.21.15.80:0/2135063983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c06edb0 0x7f653c06f1c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6540b2d640 1 -- 172.21.15.80:0/2135063983 >> 172.21.15.80:0/2135063983 conn(0x7f653c0b4160 msgr2=0x7f653c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:56.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6540b2d640 1 -- 172.21.15.80:0/2135063983 shutdown_connections 2024-04-25T13:40:56.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.499+0000 7f6540b2d640 1 -- 172.21.15.80:0/2135063983 wait complete. 2024-04-25T13:40:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 Processor -- start 2024-04-25T13:40:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 -- start start 2024-04-25T13:40:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f653c06edb0 0x7f653c14dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c0b5330 0x7f653c14c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f653c161b60 0x7f653c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f653c06e890 con 0x7f653c0b5330 2024-04-25T13:40:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f653c06e5b0 con 0x7f653c161b60 2024-04-25T13:40:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f653c0b0cd0 con 0x7f653c06edb0 2024-04-25T13:40:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c0b5330 0x7f653c14c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c0b5330 0x7f653c14c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48774/0 (socket says 172.21.15.80:48774) 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 -- 172.21.15.80:0/1105760768 learned_addr learned my addr 172.21.15.80:0/1105760768 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f653ad9d640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f653c161b60 0x7f653c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f653a59c640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f653c06edb0 0x7f653c14dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 -- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f653c06edb0 msgr2=0x7f653c14dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f653c06edb0 0x7f653c14dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 -- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f653c161b60 msgr2=0x7f653c14ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f653c161b60 0x7f653c14ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 -- 172.21.15.80:0/1105760768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f653c0b2190 con 0x7f653c0b5330 2024-04-25T13:40:56.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6539d9b640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c0b5330 0x7f653c14c4e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6530019e70 tx=0x7f6530018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f652affd640 1 -- 172.21.15.80:0/1105760768 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300b0040 con 0x7f653c0b5330 2024-04-25T13:40:56.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f652affd640 1 -- 172.21.15.80:0/1105760768 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65300b4620 con 0x7f653c0b5330 2024-04-25T13:40:56.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f653c0b2470 con 0x7f653c0b5330 2024-04-25T13:40:56.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f652affd640 1 -- 172.21.15.80:0/1105760768 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300b4a10 con 0x7f653c0b5330 2024-04-25T13:40:56.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.503+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f653c14d450 con 0x7f653c0b5330 2024-04-25T13:40:56.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.507+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f653c06a3a0 con 0x7f653c0b5330 2024-04-25T13:40:56.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.507+0000 7f652affd640 1 -- 172.21.15.80:0/1105760768 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f65300ab4c0 con 0x7f653c0b5330 2024-04-25T13:40:56.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.507+0000 7f652affd640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f651c06f400 0x7f651c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.507+0000 7f652affd640 1 -- 172.21.15.80:0/1105760768 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6530128e80 con 0x7f653c0b5330 2024-04-25T13:40:56.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.507+0000 7f653a59c640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f651c06f400 0x7f651c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.507+0000 7f653a59c640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f651c06f400 0x7f651c071910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f652c00b810 tx=0x7f652c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.511+0000 7f652affd640 1 -- 172.21.15.80:0/1105760768 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f653c1624b0 con 0x7f653c0b5330 2024-04-25T13:40:56.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f653c06fd00 con 0x7f653c0b5330 2024-04-25T13:40:56.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f652affd640 1 -- 172.21.15.80:0/1105760768 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f65300f5e00 con 0x7f653c0b5330 2024-04-25T13:40:56.668 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:40:56.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f651c06f400 msgr2=0x7f651c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f651c06f400 0x7f651c071910 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f652c00b810 tx=0x7f652c008040 comp rx=0 tx=0).stop 2024-04-25T13:40:56.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c0b5330 msgr2=0x7f653c14c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c0b5330 0x7f653c14c4e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6530019e70 tx=0x7f6530018fe0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 shutdown_connections 2024-04-25T13:40:56.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f651c06f400 0x7f651c071910 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f653c161b60 0x7f653c14ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f653c0b5330 0x7f653c14c4e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 --2- 172.21.15.80:0/1105760768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f653c06edb0 0x7f653c14dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 >> 172.21.15.80:0/1105760768 conn(0x7f653c0b4160 msgr2=0x7f653c1542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:56.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 shutdown_connections 2024-04-25T13:40:56.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.667+0000 7f6540b2d640 1 -- 172.21.15.80:0/1105760768 wait complete. 2024-04-25T13:40:56.687 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:40:56.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe9548b640 1 Processor -- start 2024-04-25T13:40:56.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe9548b640 1 -- start start 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe9548b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe9006edb0 0x7fbe9006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe9548b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbe90161b60 0x7fbe90161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe9548b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe901624b0 con 0x7fbe900b5330 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe9548b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbe9006f700 con 0x7fbe9006edb0 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe9548b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbe901629b0 con 0x7fbe90161b60 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe8e7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe9006edb0 0x7fbe9006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe8e7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe9006edb0 0x7fbe9006f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48784/0 (socket says 172.21.15.80:48784) 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe8e7fc640 1 -- 172.21.15.80:0/2239260506 learned_addr learned my addr 172.21.15.80:0/2239260506 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe8f7fe640 1 --2- 172.21.15.80:0/2239260506 >> v2:172.21.15.87:3300/0 conn(0x7fbe90161b60 0x7fbe90161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe8e7fc640 1 -- 172.21.15.80:0/2239260506 >> v2:172.21.15.87:3300/0 conn(0x7fbe90161b60 msgr2=0x7fbe90161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:56.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe8e7fc640 1 --2- 172.21.15.80:0/2239260506 >> v2:172.21.15.87:3300/0 conn(0x7fbe90161b60 0x7fbe90161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe8e7fc640 1 -- 172.21.15.80:0/2239260506 >> v1:172.21.15.87:6789/0 conn(0x7fbe900b5330 legacy=0x7fbe900d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:56.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.839+0000 7fbe8e7fc640 1 -- 172.21.15.80:0/2239260506 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe900b5740 con 0x7fbe9006edb0 2024-04-25T13:40:56.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe8e7fc640 1 --2- 172.21.15.80:0/2239260506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9006f1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe90147cb0 tx=0x7fbe7801c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f704931dbe9eb030 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe8dffb640 1 -- 172.21.15.80:0/2239260506 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe780b0070 con 0x7fbe9006edb0 2024-04-25T13:40:56.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe8dffb640 1 -- 172.21.15.80:0/2239260506 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe780ab600 con 0x7fbe9006edb0 2024-04-25T13:40:56.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe8dffb640 1 -- 172.21.15.80:0/2239260506 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe780ab9d0 con 0x7fbe9006edb0 2024-04-25T13:40:56.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe9548b640 1 -- 172.21.15.80:0/2239260506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 msgr2=0x7fbe9006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe9548b640 1 --2- 172.21.15.80:0/2239260506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9006f1c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbe90147cb0 tx=0x7fbe7801c240 comp rx=0 tx=0).stop 2024-04-25T13:40:56.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe9548b640 1 -- 172.21.15.80:0/2239260506 shutdown_connections 2024-04-25T13:40:56.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe9548b640 1 --2- 172.21.15.80:0/2239260506 >> v2:172.21.15.87:3300/0 conn(0x7fbe90161b60 0x7fbe90161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe9548b640 1 --2- 172.21.15.80:0/2239260506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9006f1c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe9548b640 1 -- 172.21.15.80:0/2239260506 >> 172.21.15.80:0/2239260506 conn(0x7fbe900b4160 msgr2=0x7fbe900d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:56.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe9548b640 1 -- 172.21.15.80:0/2239260506 shutdown_connections 2024-04-25T13:40:56.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.843+0000 7fbe9548b640 1 -- 172.21.15.80:0/2239260506 wait complete. 2024-04-25T13:40:56.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 Processor -- start 2024-04-25T13:40:56.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 -- start start 2024-04-25T13:40:56.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9014dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5330 0x7fbe9014c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe90161b60 0x7fbe9014ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9006e890 con 0x7fbe9006edb0 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe9006e5b0 con 0x7fbe90161b60 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe900b0cd0 con 0x7fbe900b5330 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9014dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9014dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48798/0 (socket says 172.21.15.80:48798) 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 -- 172.21.15.80:0/1270312232 learned_addr learned my addr 172.21.15.80:0/1270312232 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8f7fe640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe90161b60 0x7fbe9014ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8e7fc640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5330 0x7fbe9014c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 -- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5330 msgr2=0x7fbe9014c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5330 0x7fbe9014c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 -- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe90161b60 msgr2=0x7fbe9014ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe90161b60 0x7fbe9014ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 -- 172.21.15.80:0/1270312232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe900b2190 con 0x7fbe9006edb0 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8e7fc640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5330 0x7fbe9014c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:56.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe8effd640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9014dfd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800b440 tx=0x7fbe8800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe877fe640 1 -- 172.21.15.80:0/1270312232 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88015020 con 0x7fbe9006edb0 2024-04-25T13:40:56.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe877fe640 1 -- 172.21.15.80:0/1270312232 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe88011890 con 0x7fbe9006edb0 2024-04-25T13:40:56.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe900b2470 con 0x7fbe9006edb0 2024-04-25T13:40:56.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe877fe640 1 -- 172.21.15.80:0/1270312232 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe88011c80 con 0x7fbe9006edb0 2024-04-25T13:40:56.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe9014d450 con 0x7fbe9006edb0 2024-04-25T13:40:56.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.847+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe9006a3a0 con 0x7fbe9006edb0 2024-04-25T13:40:56.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.851+0000 7fbe877fe640 1 -- 172.21.15.80:0/1270312232 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbe88018020 con 0x7fbe9006edb0 2024-04-25T13:40:56.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.851+0000 7fbe877fe640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7006f400 0x7fbe70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:56.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.851+0000 7fbe877fe640 1 -- 172.21.15.80:0/1270312232 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbe8808f0c0 con 0x7fbe9006edb0 2024-04-25T13:40:56.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.851+0000 7fbe8e7fc640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7006f400 0x7fbe70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:56.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.851+0000 7fbe8e7fc640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7006f400 0x7fbe70071910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe780ad000 tx=0x7fbe78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:56.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:56.855+0000 7fbe877fe640 1 -- 172.21.15.80:0/1270312232 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe8801f030 con 0x7fbe9006edb0 2024-04-25T13:40:57.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.007+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbe9006fd00 con 0x7fbe9006edb0 2024-04-25T13:40:57.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.007+0000 7fbe877fe640 1 -- 172.21.15.80:0/1270312232 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fbe8800d040 con 0x7fbe9006edb0 2024-04-25T13:40:57.009 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:40:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7006f400 msgr2=0x7fbe70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7006f400 0x7fbe70071910 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbe780ad000 tx=0x7fbe78008040 comp rx=0 tx=0).stop 2024-04-25T13:40:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 msgr2=0x7fbe9014dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9014dfd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800b440 tx=0x7fbe8800b900 comp rx=0 tx=0).stop 2024-04-25T13:40:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 shutdown_connections 2024-04-25T13:40:57.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7006f400 0x7fbe70071910 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe90161b60 0x7fbe9014ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe900b5330 0x7fbe9014c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 --2- 172.21.15.80:0/1270312232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe9006edb0 0x7fbe9014dfd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 >> 172.21.15.80:0/1270312232 conn(0x7fbe900b4160 msgr2=0x7fbe901542a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 shutdown_connections 2024-04-25T13:40:57.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.011+0000 7fbe9548b640 1 -- 172.21.15.80:0/1270312232 wait complete. 2024-04-25T13:40:57.030 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:57.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea84d7640 1 Processor -- start 2024-04-25T13:40:57.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea84d7640 1 -- start start 2024-04-25T13:40:57.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea84d7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4ea006ebb0 0x7f4ea006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea84d7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea00d0af0 con 0x7f4ea0161f90 2024-04-25T13:40:57.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea84d7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ea00d0d70 con 0x7f4ea00d06e0 2024-04-25T13:40:57.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea84d7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4ea00d1050 con 0x7f4ea006ebb0 2024-04-25T13:40:57.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea6a74640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4ea006ebb0 0x7f4ea006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea6273640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4ea0161f90 0x7f4ea00cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45158/0 (socket says 172.21.15.80:45158) 2024-04-25T13:40:57.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea6273640 1 -- 172.21.15.80:0/3391852111 learned_addr learned my addr 172.21.15.80:0/3391852111 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:57.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea6a74640 1 -- 172.21.15.80:0/3391852111 >> v1:172.21.15.87:6789/0 conn(0x7f4ea00d06e0 legacy=0x7f4ea006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:57.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea6a74640 1 -- 172.21.15.80:0/3391852111 >> v1:172.21.15.80:6789/0 conn(0x7f4ea0161f90 legacy=0x7f4ea00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.175+0000 7f4ea6a74640 1 -- 172.21.15.80:0/3391852111 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea006dca0 con 0x7f4ea006ebb0 2024-04-25T13:40:57.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea6a74640 1 --2- 172.21.15.80:0/3391852111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea006ebb0 0x7f4ea006efc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009d90 tx=0x7f4e9801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195f1a2affe1ed1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea5271640 1 -- 172.21.15.80:0/3391852111 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e980b2070 con 0x7f4ea006ebb0 2024-04-25T13:40:57.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea5271640 1 -- 172.21.15.80:0/3391852111 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e980a5860 con 0x7f4ea006ebb0 2024-04-25T13:40:57.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea5271640 1 -- 172.21.15.80:0/3391852111 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e980a5c30 con 0x7f4ea006ebb0 2024-04-25T13:40:57.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 -- 172.21.15.80:0/3391852111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea006ebb0 msgr2=0x7f4ea006efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 --2- 172.21.15.80:0/3391852111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea006ebb0 0x7f4ea006efc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009d90 tx=0x7f4e9801c7e0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 -- 172.21.15.80:0/3391852111 shutdown_connections 2024-04-25T13:40:57.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 --2- 172.21.15.80:0/3391852111 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea006ebb0 0x7f4ea006efc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 -- 172.21.15.80:0/3391852111 >> 172.21.15.80:0/3391852111 conn(0x7f4ea006d080 msgr2=0x7f4ea00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:57.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 -- 172.21.15.80:0/3391852111 shutdown_connections 2024-04-25T13:40:57.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 -- 172.21.15.80:0/3391852111 wait complete. 2024-04-25T13:40:57.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 Processor -- start 2024-04-25T13:40:57.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.179+0000 7f4ea84d7640 1 -- start start 2024-04-25T13:40:57.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea006ebb0 0x7f4ea013d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea00d06e0 0x7f4ea01413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea0161f90 0x7f4ea013c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea006f680 con 0x7f4ea0161f90 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea006f500 con 0x7f4ea00d06e0 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea0144620 con 0x7f4ea006ebb0 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea0161f90 0x7f4ea013c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea0161f90 0x7f4ea013c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48822/0 (socket says 172.21.15.80:48822) 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 -- 172.21.15.80:0/4039033104 learned_addr learned my addr 172.21.15.80:0/4039033104 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6273640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea006ebb0 0x7f4ea013d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 -- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea006ebb0 msgr2=0x7f4ea013d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea5a72640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea00d06e0 0x7f4ea01413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea006ebb0 0x7f4ea013d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 -- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea00d06e0 msgr2=0x7f4ea01413d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea00d06e0 0x7f4ea01413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 -- 172.21.15.80:0/4039033104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea01399b0 con 0x7f4ea0161f90 2024-04-25T13:40:57.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea6a74640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea0161f90 0x7f4ea013c340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009d90 tx=0x7f4e98019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4e96ffd640 1 -- 172.21.15.80:0/4039033104 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e980bd020 con 0x7f4ea0161f90 2024-04-25T13:40:57.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4e96ffd640 1 -- 172.21.15.80:0/4039033104 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e980b8670 con 0x7f4ea0161f90 2024-04-25T13:40:57.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea0139c90 con 0x7f4ea0161f90 2024-04-25T13:40:57.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4e96ffd640 1 -- 172.21.15.80:0/4039033104 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e980b2040 con 0x7f4ea0161f90 2024-04-25T13:40:57.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea013a250 con 0x7f4ea0161f90 2024-04-25T13:40:57.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea0139720 con 0x7f4ea0161f90 2024-04-25T13:40:57.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4e96ffd640 1 -- 172.21.15.80:0/4039033104 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4e980c0020 con 0x7f4ea0161f90 2024-04-25T13:40:57.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4e96ffd640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8406f400 0x7f4e84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.183+0000 7f4e96ffd640 1 -- 172.21.15.80:0/4039033104 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4e9812a460 con 0x7f4ea0161f90 2024-04-25T13:40:57.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.187+0000 7f4ea6273640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8406f400 0x7f4e84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.187+0000 7f4ea6273640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8406f400 0x7f4e84071910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c000f70 tx=0x7f4e9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.191+0000 7f4e96ffd640 1 -- 172.21.15.80:0/4039033104 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4e980c7030 con 0x7f4ea0161f90 2024-04-25T13:40:57.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.339+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ea006a3a0 con 0x7f4ea0161f90 2024-04-25T13:40:57.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4e96ffd640 1 -- 172.21.15.80:0/4039033104 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f4e98016520 con 0x7f4ea0161f90 2024-04-25T13:40:57.344 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:57.344 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:57.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8406f400 msgr2=0x7f4e84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8406f400 0x7f4e84071910 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c000f70 tx=0x7f4e9c009040 comp rx=0 tx=0).stop 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea0161f90 msgr2=0x7f4ea013c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea0161f90 0x7f4ea013c340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4e98009d90 tx=0x7f4e98019580 comp rx=0 tx=0).stop 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 shutdown_connections 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8406f400 0x7f4e84071910 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea0161f90 0x7f4ea013c340 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea00d06e0 0x7f4ea01413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 --2- 172.21.15.80:0/4039033104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea006ebb0 0x7f4ea013d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 >> 172.21.15.80:0/4039033104 conn(0x7f4ea006d080 msgr2=0x7f4ea00b2280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 shutdown_connections 2024-04-25T13:40:57.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.343+0000 7f4ea84d7640 1 -- 172.21.15.80:0/4039033104 wait complete. 2024-04-25T13:40:57.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:40:57.364 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:40:57.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6320b4e640 1 Processor -- start 2024-04-25T13:40:57.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6320b4e640 1 -- start start 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6320b4e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f631c0b5330 0x7f631c0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6320b4e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f631c0d0ff0 0x7f631c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6320b4e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f631c0b5740 con 0x7f631c06f2f0 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6320b4e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f631c06f700 con 0x7f631c0d0ff0 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6320b4e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f631c1629b0 con 0x7f631c0b5330 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6319d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f631c0d0ff0 0x7f631c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6319d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f631c0d0ff0 0x7f631c06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48836/0 (socket says 172.21.15.80:48836) 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6319d9b640 1 -- 172.21.15.80:0/2873505271 learned_addr learned my addr 172.21.15.80:0/2873505271 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:57.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f631a59c640 1 --2- 172.21.15.80:0/2873505271 >> v2:172.21.15.87:3300/0 conn(0x7f631c0b5330 0x7f631c0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6319d9b640 1 -- 172.21.15.80:0/2873505271 >> v2:172.21.15.87:3300/0 conn(0x7f631c0b5330 msgr2=0x7f631c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6319d9b640 1 --2- 172.21.15.80:0/2873505271 >> v2:172.21.15.87:3300/0 conn(0x7f631c0b5330 0x7f631c0d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6319d9b640 1 -- 172.21.15.80:0/2873505271 >> v1:172.21.15.87:6789/0 conn(0x7f631c06f2f0 legacy=0x7f631c161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:57.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.503+0000 7f6319d9b640 1 -- 172.21.15.80:0/2873505271 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c06dd10 con 0x7f631c0d0ff0 2024-04-25T13:40:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6319d9b640 1 --2- 172.21.15.80:0/2873505271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0d0ff0 0x7f631c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f631c0701f0 tx=0x7f631001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb13a50a01259628 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f631959a640 1 -- 172.21.15.80:0/2873505271 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63100b0070 con 0x7f631c0d0ff0 2024-04-25T13:40:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f631959a640 1 -- 172.21.15.80:0/2873505271 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63100ab600 con 0x7f631c0d0ff0 2024-04-25T13:40:57.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f631959a640 1 -- 172.21.15.80:0/2873505271 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63100ab9d0 con 0x7f631c0d0ff0 2024-04-25T13:40:57.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6320b4e640 1 -- 172.21.15.80:0/2873505271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0d0ff0 msgr2=0x7f631c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6320b4e640 1 --2- 172.21.15.80:0/2873505271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0d0ff0 0x7f631c06edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f631c0701f0 tx=0x7f631001c450 comp rx=0 tx=0).stop 2024-04-25T13:40:57.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6320b4e640 1 -- 172.21.15.80:0/2873505271 shutdown_connections 2024-04-25T13:40:57.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6320b4e640 1 --2- 172.21.15.80:0/2873505271 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0d0ff0 0x7f631c06edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6320b4e640 1 --2- 172.21.15.80:0/2873505271 >> v2:172.21.15.87:3300/0 conn(0x7f631c0b5330 0x7f631c0d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6320b4e640 1 -- 172.21.15.80:0/2873505271 >> 172.21.15.80:0/2873505271 conn(0x7f631c0b4160 msgr2=0x7f631c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:57.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6320b4e640 1 -- 172.21.15.80:0/2873505271 shutdown_connections 2024-04-25T13:40:57.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.507+0000 7f6320b4e640 1 -- 172.21.15.80:0/2873505271 wait complete. 2024-04-25T13:40:57.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 Processor -- start 2024-04-25T13:40:57.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 -- start start 2024-04-25T13:40:57.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f631c06f2f0 0x7f631c15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0b5330 0x7f631c135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f631c0d0ff0 0x7f631c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f631c0b0cd0 con 0x7f631c0b5330 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f631c06e890 con 0x7f631c0d0ff0 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f631c0b0e50 con 0x7f631c06f2f0 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0b5330 0x7f631c135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0b5330 0x7f631c135080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48840/0 (socket says 172.21.15.80:48840) 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 -- 172.21.15.80:0/1265832811 learned_addr learned my addr 172.21.15.80:0/1265832811 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f631ad9d640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f631c0d0ff0 0x7f631c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f631a59c640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f631c06f2f0 0x7f631c15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 -- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f631c06f2f0 msgr2=0x7f631c15bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f631c06f2f0 0x7f631c15bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 -- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f631c0d0ff0 msgr2=0x7f631c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f631c0d0ff0 0x7f631c0af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 -- 172.21.15.80:0/1265832811 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631c15d6a0 con 0x7f631c0b5330 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f631ad9d640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f631c0d0ff0 0x7f631c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6319d9b640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0b5330 0x7f631c135080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f631001a080 tx=0x7f63100191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f630affd640 1 -- 172.21.15.80:0/1265832811 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63100b9070 con 0x7f631c0b5330 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f630affd640 1 -- 172.21.15.80:0/1265832811 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63100b4640 con 0x7f631c0b5330 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f630affd640 1 -- 172.21.15.80:0/1265832811 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63100b0040 con 0x7f631c0b5330 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631c15c600 con 0x7f631c0b5330 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631c15cbc0 con 0x7f631c0b5330 2024-04-25T13:40:57.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.511+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f631c15c370 con 0x7f631c0b5330 2024-04-25T13:40:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.515+0000 7f630affd640 1 -- 172.21.15.80:0/1265832811 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f63100bc070 con 0x7f631c0b5330 2024-04-25T13:40:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.515+0000 7f630affd640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62fc06f400 0x7f62fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.515+0000 7f630affd640 1 -- 172.21.15.80:0/1265832811 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f63101286a0 con 0x7f631c0b5330 2024-04-25T13:40:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.515+0000 7f631a59c640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62fc06f400 0x7f62fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.515+0000 7f631a59c640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62fc06f400 0x7f62fc071910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b7c0 tx=0x7f630c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.519+0000 7f630affd640 1 -- 172.21.15.80:0/1265832811 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63100c3080 con 0x7f631c0b5330 2024-04-25T13:40:57.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.667+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f631c06a3a0 con 0x7f631c0b5330 2024-04-25T13:40:57.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.667+0000 7f630affd640 1 -- 172.21.15.80:0/1265832811 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f63100160d0 con 0x7f631c0b5330 2024-04-25T13:40:57.670 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:40:57.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62fc06f400 msgr2=0x7f62fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62fc06f400 0x7f62fc071910 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f630c00b7c0 tx=0x7f630c008040 comp rx=0 tx=0).stop 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0b5330 msgr2=0x7f631c135080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0b5330 0x7f631c135080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f631001a080 tx=0x7f63100191f0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 shutdown_connections 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62fc06f400 0x7f62fc071910 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f631c0d0ff0 0x7f631c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f631c0b5330 0x7f631c135080 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 --2- 172.21.15.80:0/1265832811 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f631c06f2f0 0x7f631c15bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 >> 172.21.15.80:0/1265832811 conn(0x7f631c0b4160 msgr2=0x7f631c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 shutdown_connections 2024-04-25T13:40:57.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.671+0000 7f6320b4e640 1 -- 172.21.15.80:0/1265832811 wait complete. 2024-04-25T13:40:57.690 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:40:57.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f260d57d640 1 Processor -- start 2024-04-25T13:40:57.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f260d57d640 1 -- start start 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f260d57d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f26080b7690 0x7f26080b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f260d57d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2608161270 0x7f2608070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f260d57d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26080d2540 con 0x7f26080d1d60 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f260d57d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2608070490 con 0x7f26080b7690 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f260d57d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26080b5fb0 con 0x7f2608161270 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f2606ffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f26080b7690 0x7f26080b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.839+0000 7f26077fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2608161270 0x7f2608070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f26067fc640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f26080d1d60 0x7f2608160b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:59738/0 (socket says 172.21.15.80:59738) 2024-04-25T13:40:57.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f26067fc640 1 -- 172.21.15.80:0/1474988843 learned_addr learned my addr 172.21.15.80:0/1474988843 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:57.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f26077fe640 1 -- 172.21.15.80:0/1474988843 >> v2:172.21.15.87:3300/0 conn(0x7f26080b7690 msgr2=0x7f26080b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:57.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f26077fe640 1 --2- 172.21.15.80:0/1474988843 >> v2:172.21.15.87:3300/0 conn(0x7f26080b7690 0x7f26080b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f26077fe640 1 -- 172.21.15.80:0/1474988843 >> v1:172.21.15.87:6789/0 conn(0x7f26080d1d60 legacy=0x7f2608160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f26077fe640 1 -- 172.21.15.80:0/1474988843 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260806fe40 con 0x7f2608161270 2024-04-25T13:40:57.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f26077fe640 1 --2- 172.21.15.80:0/1474988843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2608161270 0x7f2608070920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25f00097c0 tx=0x7f25f001c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e9fd2e1231dc158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f2605ffb640 1 -- 172.21.15.80:0/1474988843 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00b0070 con 0x7f2608161270 2024-04-25T13:40:57.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f2605ffb640 1 -- 172.21.15.80:0/1474988843 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f00ab600 con 0x7f2608161270 2024-04-25T13:40:57.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.843+0000 7f2605ffb640 1 -- 172.21.15.80:0/1474988843 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00ab9d0 con 0x7f2608161270 2024-04-25T13:40:57.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- 172.21.15.80:0/1474988843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2608161270 msgr2=0x7f2608070920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 --2- 172.21.15.80:0/1474988843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2608161270 0x7f2608070920 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f25f00097c0 tx=0x7f25f001c410 comp rx=0 tx=0).stop 2024-04-25T13:40:57.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- 172.21.15.80:0/1474988843 shutdown_connections 2024-04-25T13:40:57.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 --2- 172.21.15.80:0/1474988843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2608161270 0x7f2608070920 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 --2- 172.21.15.80:0/1474988843 >> v2:172.21.15.87:3300/0 conn(0x7f26080b7690 0x7f26080b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- 172.21.15.80:0/1474988843 >> 172.21.15.80:0/1474988843 conn(0x7f26080737c0 msgr2=0x7f2608073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:57.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- 172.21.15.80:0/1474988843 shutdown_connections 2024-04-25T13:40:57.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- 172.21.15.80:0/1474988843 wait complete. 2024-04-25T13:40:57.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 Processor -- start 2024-04-25T13:40:57.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- start start 2024-04-25T13:40:57.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26080b7690 0x7f26081420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26080d1d60 0x7f260815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f260815b150 0x7f260813fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26080d22f0 con 0x7f260815b150 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f26080d2170 con 0x7f26080b7690 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f260d57d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2608169f00 con 0x7f26080d1d60 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f26077fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f260815b150 0x7f260813fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f26077fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f260815b150 0x7f260813fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48856/0 (socket says 172.21.15.80:48856) 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f26077fe640 1 -- 172.21.15.80:0/1465181534 learned_addr learned my addr 172.21.15.80:0/1465181534 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f26067fc640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26080d1d60 0x7f260815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.847+0000 7f2606ffd640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26080b7690 0x7f26081420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f26077fe640 1 -- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26080d1d60 msgr2=0x7f260815ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f26077fe640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26080d1d60 0x7f260815ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f26077fe640 1 -- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26080b7690 msgr2=0x7f26081420f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:57.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f26077fe640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26080b7690 0x7f26081420f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:57.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f26077fe640 1 -- 172.21.15.80:0/1465181534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2608142630 con 0x7f260815b150 2024-04-25T13:40:57.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f26077fe640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f260815b150 0x7f260813fe50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f25f00097c0 tx=0x7f25f00191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f25ff7fe640 1 -- 172.21.15.80:0/1465181534 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f00b0040 con 0x7f260815b150 2024-04-25T13:40:57.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f25ff7fe640 1 -- 172.21.15.80:0/1465181534 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25f00bc070 con 0x7f260815b150 2024-04-25T13:40:57.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2608142b40 con 0x7f260815b150 2024-04-25T13:40:57.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f25ff7fe640 1 -- 172.21.15.80:0/1465181534 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0006e20 con 0x7f260815b150 2024-04-25T13:40:57.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f260815a130 con 0x7f260815b150 2024-04-25T13:40:57.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26081428b0 con 0x7f260815b150 2024-04-25T13:40:57.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f25ff7fe640 1 -- 172.21.15.80:0/1465181534 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f25f00b3070 con 0x7f260815b150 2024-04-25T13:40:57.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f25ff7fe640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f25e806f400 0x7f25e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:57.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f25ff7fe640 1 -- 172.21.15.80:0/1465181534 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f25f0129db0 con 0x7f260815b150 2024-04-25T13:40:57.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.851+0000 7f2606ffd640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f25e806f400 0x7f25e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:57.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.855+0000 7f2606ffd640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f25e806f400 0x7f25e8071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2600007440 tx=0x7f260000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:57.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:57.855+0000 7f25ff7fe640 1 -- 172.21.15.80:0/1465181534 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25f0016090 con 0x7f260815b150 2024-04-25T13:40:58.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.003+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2608167cd0 con 0x7f260815b150 2024-04-25T13:40:58.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.003+0000 7f25ff7fe640 1 -- 172.21.15.80:0/1465181534 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f25f00f6dc0 con 0x7f260815b150 2024-04-25T13:40:58.006 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f25e806f400 msgr2=0x7f25e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f25e806f400 0x7f25e8071910 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2600007440 tx=0x7f260000b040 comp rx=0 tx=0).stop 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f260815b150 msgr2=0x7f260813fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f260815b150 0x7f260813fe50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f25f00097c0 tx=0x7f25f00191b0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 shutdown_connections 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f25e806f400 0x7f25e8071910 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f260815b150 0x7f260813fe50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26080d1d60 0x7f260815ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 --2- 172.21.15.80:0/1465181534 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26080b7690 0x7f26081420f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 >> 172.21.15.80:0/1465181534 conn(0x7f26080737c0 msgr2=0x7f2608160e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:58.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 shutdown_connections 2024-04-25T13:40:58.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.007+0000 7f260d57d640 1 -- 172.21.15.80:0/1465181534 wait complete. 2024-04-25T13:40:58.025 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:40:58.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.175+0000 7fd31c7f0640 1 Processor -- start 2024-04-25T13:40:58.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.175+0000 7fd31c7f0640 1 -- start start 2024-04-25T13:40:58.180 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:40:58.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31c7f0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd3140b5330 0x7fd3140d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31c7f0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3140b5740 con 0x7fd3140d0ff0 2024-04-25T13:40:58.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31c7f0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd31406dd10 con 0x7fd314161b60 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31c7f0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd3141629b0 con 0x7fd3140b5330 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd319d8b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd3140d0ff0 0x7fd31406edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45162/0 (socket says 172.21.15.80:45162) 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd319d8b640 1 -- 172.21.15.80:0/124512108 learned_addr learned my addr 172.21.15.80:0/124512108 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31a58c640 1 --2- 172.21.15.80:0/124512108 >> v2:172.21.15.87:3300/0 conn(0x7fd3140b5330 0x7fd3140d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1760419847 0 0) 0x7fd3140b5740 con 0x7fd3140d0ff0 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd300003530 con 0x7fd3140d0ff0 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1336322688 0 0) 0x7fd300003530 con 0x7fd3140d0ff0 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 >> v2:172.21.15.87:3300/0 conn(0x7fd3140b5330 msgr2=0x7fd3140d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 --2- 172.21.15.80:0/124512108 >> v2:172.21.15.87:3300/0 conn(0x7fd3140b5330 0x7fd3140d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 >> v1:172.21.15.87:6789/0 conn(0x7fd314161b60 legacy=0x7fd314161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31406dff0 con 0x7fd3140d0ff0 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fd304001680 con 0x7fd3140d0ff0 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 >> v1:172.21.15.80:6789/0 conn(0x7fd3140d0ff0 legacy=0x7fd31406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3000039c0 0x7fd300003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd300004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd300005260 0x7fd30000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141629b0 con 0x7fd300004810 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd30000c1f0 con 0x7fd3000039c0 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd30000c3f0 con 0x7fd300005260 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd319d8b640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd300004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31a58c640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3000039c0 0x7fd300003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd319d8b640 1 -- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd300005260 msgr2=0x7fd30000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31ad8d640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd300005260 0x7fd30000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd319d8b640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd300005260 0x7fd30000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd319d8b640 1 -- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3000039c0 msgr2=0x7fd300003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd319d8b640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd3000039c0 0x7fd300003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd319d8b640 1 -- 172.21.15.80:0/124512108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd300003530 con 0x7fd300004810 2024-04-25T13:40:58.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31ad8d640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd300005260 0x7fd30000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:58.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.179+0000 7fd31ad8d640 1 -- 172.21.15.80:0/124512108 reap_dead start 2024-04-25T13:40:58.186 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd319d8b640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd300004c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd30400c240 tx=0x7fd30401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41aecc7f6f2d63ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304007d20 con 0x7fd300004810 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3040a9070 con 0x7fd300004810 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31958a640 1 -- 172.21.15.80:0/124512108 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304005b00 con 0x7fd300004810 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31c7f0640 1 -- 172.21.15.80:0/124512108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 msgr2=0x7fd300004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31c7f0640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd300004c20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd30400c240 tx=0x7fd30401cc80 comp rx=0 tx=0).stop 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31c7f0640 1 -- 172.21.15.80:0/124512108 shutdown_connections 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31c7f0640 1 --2- 172.21.15.80:0/124512108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd300004c20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31c7f0640 1 -- 172.21.15.80:0/124512108 >> 172.21.15.80:0/124512108 conn(0x7fd3140b4160 msgr2=0x7fd3140d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31c7f0640 1 -- 172.21.15.80:0/124512108 shutdown_connections 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.183+0000 7fd31c7f0640 1 -- 172.21.15.80:0/124512108 wait complete. 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 Processor -- start 2024-04-25T13:40:58.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 -- start start 2024-04-25T13:40:58.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd3141f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31415efe0 0x7fd314155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd314150d20 0x7fd314151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd3140b0cd0 con 0x7fd300004810 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd31406f5c0 con 0x7fd314150d20 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3140b0e50 con 0x7fd31415efe0 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd3141f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd3141f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48878/0 (socket says 172.21.15.80:48878) 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 -- 172.21.15.80:0/1512893996 learned_addr learned my addr 172.21.15.80:0/1512893996 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31ad8d640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd314150d20 0x7fd314151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd319d8b640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31415efe0 0x7fd314155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 -- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31415efe0 msgr2=0x7fd314155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31415efe0 0x7fd314155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 -- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd314150d20 msgr2=0x7fd314151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd314150d20 0x7fd314151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 -- 172.21.15.80:0/1512893996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd314151670 con 0x7fd300004810 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd319d8b640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31415efe0 0x7fd314155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31ad8d640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd314150d20 0x7fd314151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31a58c640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd3141f62c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd310011690 tx=0x7fd310011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd30affd640 1 -- 172.21.15.80:0/1512893996 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31000f9c0 con 0x7fd300004810 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd30affd640 1 -- 172.21.15.80:0/1512893996 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd31000fb60 con 0x7fd300004810 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd31414f200 con 0x7fd300004810 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd30affd640 1 -- 172.21.15.80:0/1512893996 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd310015460 con 0x7fd300004810 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd31414f7c0 con 0x7fd300004810 2024-04-25T13:40:58.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.187+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd31406a3a0 con 0x7fd300004810 2024-04-25T13:40:58.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.191+0000 7fd30affd640 1 -- 172.21.15.80:0/1512893996 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd310015600 con 0x7fd300004810 2024-04-25T13:40:58.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.191+0000 7fd30affd640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2f806f400 0x7fd2f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.191+0000 7fd30affd640 1 -- 172.21.15.80:0/1512893996 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fd31008fe50 con 0x7fd300004810 2024-04-25T13:40:58.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.191+0000 7fd319d8b640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2f806f400 0x7fd2f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.191+0000 7fd319d8b640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2f806f400 0x7fd2f8071910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd3040ab000 tx=0x7fd3040a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.195+0000 7fd30affd640 1 -- 172.21.15.80:0/1512893996 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd314151670 con 0x7fd300004810 2024-04-25T13:40:58.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.343+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd31406fd00 con 0x7fd300004810 2024-04-25T13:40:58.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.343+0000 7fd30affd640 1 -- 172.21.15.80:0/1512893996 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fd310019030 con 0x7fd300004810 2024-04-25T13:40:58.347 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:40:58.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2f806f400 msgr2=0x7fd2f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2f806f400 0x7fd2f8071910 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd3040ab000 tx=0x7fd3040a7070 comp rx=0 tx=0).stop 2024-04-25T13:40:58.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 msgr2=0x7fd3141f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd3141f62c0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd310011690 tx=0x7fd310011b50 comp rx=0 tx=0).stop 2024-04-25T13:40:58.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 shutdown_connections 2024-04-25T13:40:58.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2f806f400 0x7fd2f8071910 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd314150d20 0x7fd314151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31415efe0 0x7fd314155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 --2- 172.21.15.80:0/1512893996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd300004810 0x7fd3141f62c0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 >> 172.21.15.80:0/1512893996 conn(0x7fd3140b4160 msgr2=0x7fd3140b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:58.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 shutdown_connections 2024-04-25T13:40:58.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.347+0000 7fd31c7f0640 1 -- 172.21.15.80:0/1512893996 wait complete. 2024-04-25T13:40:58.366 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:58.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed2773640 1 Processor -- start 2024-04-25T13:40:58.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed2773640 1 -- start start 2024-04-25T13:40:58.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed2773640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7ecc0d4330 0x7f7ecc0d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed2773640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ecc161060 con 0x7f7ecc0d59b0 2024-04-25T13:40:58.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed2773640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ecc161340 con 0x7f7ecc0d2e80 2024-04-25T13:40:58.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed2773640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7ecc0d2680 con 0x7f7ecc0d4330 2024-04-25T13:40:58.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ec37fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7ecc0d59b0 0x7f7ecc0d5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45164/0 (socket says 172.21.15.80:45164) 2024-04-25T13:40:58.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ec37fe640 1 -- 172.21.15.80:0/2337506173 learned_addr learned my addr 172.21.15.80:0/2337506173 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:58.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed0d10640 1 --2- 172.21.15.80:0/2337506173 >> v2:172.21.15.87:3300/0 conn(0x7f7ecc0d4330 0x7f7ecc0d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1925912051 0 0) 0x7f7ecc161060 con 0x7f7ecc0d59b0 2024-04-25T13:40:58.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7eb8003530 con 0x7f7ecc0d59b0 2024-04-25T13:40:58.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 692194446 0 0) 0x7f7eb8003530 con 0x7f7ecc0d59b0 2024-04-25T13:40:58.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 >> v2:172.21.15.87:3300/0 conn(0x7f7ecc0d4330 msgr2=0x7f7ecc0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:58.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 --2- 172.21.15.80:0/2337506173 >> v2:172.21.15.87:3300/0 conn(0x7f7ecc0d4330 0x7f7ecc0d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 >> v1:172.21.15.87:6789/0 conn(0x7f7ecc0d2e80 legacy=0x7f7ecc1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc0d2960 con 0x7f7ecc0d59b0 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7eb40016a0 con 0x7f7ecc0d59b0 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 >> v1:172.21.15.80:6789/0 conn(0x7f7ecc0d59b0 legacy=0x7f7ecc0d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7eb8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8004810 0x7f7eb8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7eb8005260 0x7f7eb800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d2680 con 0x7f7eb80039c0 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb800c1f0 con 0x7f7eb8004810 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7eb800c3f0 con 0x7f7eb8005260 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecbfff640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7eb8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ec37fe640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8004810 0x7f7eb8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed0d10640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7eb8005260 0x7f7eb800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecbfff640 1 -- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7eb8005260 msgr2=0x7f7eb800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecbfff640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7eb8005260 0x7f7eb800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecbfff640 1 -- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8004810 msgr2=0x7f7eb8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecbfff640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8004810 0x7f7eb8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ecbfff640 1 -- 172.21.15.80:0/2337506173 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb8003530 con 0x7f7eb80039c0 2024-04-25T13:40:58.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ec37fe640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7eb8004810 0x7f7eb8004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:58.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.555+0000 7f7ed0d10640 1 -- 172.21.15.80:0/2337506173 reap_dead start 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ecbfff640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7eb8003e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc001750 tx=0x7f7ebc01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06d973874131687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0013a0 con 0x7f7eb80039c0 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc0052f0 con 0x7f7eb80039c0 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ecb7fe640 1 -- 172.21.15.80:0/2337506173 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc003840 con 0x7f7eb80039c0 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ed2773640 1 -- 172.21.15.80:0/2337506173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 msgr2=0x7f7eb8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ed2773640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7eb8003e30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc001750 tx=0x7f7ebc01c3c0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ed2773640 1 -- 172.21.15.80:0/2337506173 shutdown_connections 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ed2773640 1 --2- 172.21.15.80:0/2337506173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7eb8003e30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ed2773640 1 -- 172.21.15.80:0/2337506173 >> 172.21.15.80:0/2337506173 conn(0x7f7ecc0737a0 msgr2=0x7f7ecc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ed2773640 1 -- 172.21.15.80:0/2337506173 shutdown_connections 2024-04-25T13:40:58.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.559+0000 7f7ed2773640 1 -- 172.21.15.80:0/2337506173 wait complete. 2024-04-25T13:40:58.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 Processor -- start 2024-04-25T13:40:58.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 -- start start 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7ecc0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ecc14dcf0 0x7f7ecc1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ecc14e100 0x7f7ecc158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc15df60 con 0x7f7eb80039c0 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc0d34a0 con 0x7f7ecc14dcf0 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7ecc15e0e0 con 0x7f7ecc14e100 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7ecc0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7ecc0cb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48898/0 (socket says 172.21.15.80:48898) 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 -- 172.21.15.80:0/3063365790 learned_addr learned my addr 172.21.15.80:0/3063365790 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ec37fe640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ecc14dcf0 0x7f7ecc1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed0d10640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ecc14e100 0x7f7ecc158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 -- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ecc14e100 msgr2=0x7f7ecc158620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ecc14e100 0x7f7ecc158620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 -- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ecc14dcf0 msgr2=0x7f7ecc1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ecc14dcf0 0x7f7ecc1580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 -- 172.21.15.80:0/3063365790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc0cbc70 con 0x7f7eb80039c0 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed0d10640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ecc14e100 0x7f7ecc158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ecbfff640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7ecc0cb730 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc01c1f0 tx=0x7f7eb8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ec97fa640 1 -- 172.21.15.80:0/3063365790 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc007a40 con 0x7f7eb80039c0 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ec97fa640 1 -- 172.21.15.80:0/3063365790 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ebc0ba070 con 0x7f7eb80039c0 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc0cdcb0 con 0x7f7eb80039c0 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ec97fa640 1 -- 172.21.15.80:0/3063365790 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0054a0 con 0x7f7eb80039c0 2024-04-25T13:40:58.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc0ce270 con 0x7f7eb80039c0 2024-04-25T13:40:58.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.563+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ecc06a3a0 con 0x7f7eb80039c0 2024-04-25T13:40:58.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.567+0000 7f7ec97fa640 1 -- 172.21.15.80:0/3063365790 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7ebc007030 con 0x7f7eb80039c0 2024-04-25T13:40:58.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.567+0000 7f7ec97fa640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7eac06f400 0x7f7eac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.567+0000 7f7ec97fa640 1 -- 172.21.15.80:0/3063365790 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ebc129340 con 0x7f7eb80039c0 2024-04-25T13:40:58.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.567+0000 7f7ec37fe640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7eac06f400 0x7f7eac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.567+0000 7f7ec37fe640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7eac06f400 0x7f7eac071910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4000fe0 tx=0x7f7eb4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.571+0000 7f7ec97fa640 1 -- 172.21.15.80:0/3063365790 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ebc0160d0 con 0x7f7eb80039c0 2024-04-25T13:40:58.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.731+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ecc0bbb90 con 0x7f7eb80039c0 2024-04-25T13:40:58.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ec97fa640 1 -- 172.21.15.80:0/3063365790 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f7ebc0f6230 con 0x7f7eb80039c0 2024-04-25T13:40:58.736 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:40:58.736 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:40:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7eac06f400 msgr2=0x7f7eac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7eac06f400 0x7f7eac071910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7eb4000fe0 tx=0x7f7eb4009040 comp rx=0 tx=0).stop 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 msgr2=0x7f7ecc0cb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7ecc0cb730 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc01c1f0 tx=0x7f7eb8005840 comp rx=0 tx=0).stop 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 shutdown_connections 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7eac06f400 0x7f7eac071910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ecc14e100 0x7f7ecc158620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ecc14dcf0 0x7f7ecc1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 --2- 172.21.15.80:0/3063365790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7eb80039c0 0x7f7ecc0cb730 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 >> 172.21.15.80:0/3063365790 conn(0x7f7ecc0737a0 msgr2=0x7f7ecc1571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 shutdown_connections 2024-04-25T13:40:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.735+0000 7f7ed2773640 1 -- 172.21.15.80:0/3063365790 wait complete. 2024-04-25T13:40:58.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:40:58.759 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:40:58.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.911+0000 7f03a7ea4640 1 Processor -- start 2024-04-25T13:40:58.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.911+0000 7f03a7ea4640 1 -- start start 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.911+0000 7f03a7ea4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0161170 0x7f03a01574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.911+0000 7f03a7ea4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f03a0073c40 0x7f03a00b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.911+0000 7f03a7ea4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03a006fbf0 con 0x7f03a0071e60 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.911+0000 7f03a7ea4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f03a00b1350 con 0x7f03a0073c40 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.911+0000 7f03a7ea4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f03a006f9a0 con 0x7f03a0161170 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a543f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0161170 0x7f03a01574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a6441640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f03a0073c40 0x7f03a00b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a6441640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f03a0073c40 0x7f03a00b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:58140/0 (socket says 172.21.15.80:58140) 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a543f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0161170 0x7f03a01574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:44900/0 (socket says 172.21.15.80:44900) 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a6441640 1 -- 172.21.15.80:0/1443148187 learned_addr learned my addr 172.21.15.80:0/1443148187 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:58.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a5c40640 1 --1- 172.21.15.80:0/1443148187 >> v1:172.21.15.87:6789/0 conn(0x7f03a0071e60 0x7f03a0160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:59772/0 (socket says 172.21.15.80:59772) 2024-04-25T13:40:58.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a543f640 1 -- 172.21.15.80:0/1443148187 >> v2:172.21.15.87:3300/0 conn(0x7f03a0073c40 msgr2=0x7f03a00b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:58.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a543f640 1 --2- 172.21.15.80:0/1443148187 >> v2:172.21.15.87:3300/0 conn(0x7f03a0073c40 0x7f03a00b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a543f640 1 -- 172.21.15.80:0/1443148187 >> v1:172.21.15.87:6789/0 conn(0x7f03a0071e60 legacy=0x7f03a0160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a543f640 1 -- 172.21.15.80:0/1443148187 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a00b0a20 con 0x7f03a0161170 2024-04-25T13:40:58.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a543f640 1 --2- 172.21.15.80:0/1443148187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0161170 0x7f03a01574c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03900097c0 tx=0x7f039001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b69a7a4bd54a4f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a4c3e640 1 -- 172.21.15.80:0/1443148187 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900b2070 con 0x7f03a0161170 2024-04-25T13:40:58.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a4c3e640 1 -- 172.21.15.80:0/1443148187 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03900ad640 con 0x7f03a0161170 2024-04-25T13:40:58.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a4c3e640 1 -- 172.21.15.80:0/1443148187 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900ada10 con 0x7f03a0161170 2024-04-25T13:40:58.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1443148187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0161170 msgr2=0x7f03a01574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1443148187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0161170 0x7f03a01574c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03900097c0 tx=0x7f039001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1443148187 shutdown_connections 2024-04-25T13:40:58.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1443148187 >> v2:172.21.15.87:3300/0 conn(0x7f03a0073c40 0x7f03a00b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1443148187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0161170 0x7f03a01574c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.915+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1443148187 >> 172.21.15.80:0/1443148187 conn(0x7f03a00b4160 msgr2=0x7f03a00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:58.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1443148187 shutdown_connections 2024-04-25T13:40:58.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1443148187 wait complete. 2024-04-25T13:40:58.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 Processor -- start 2024-04-25T13:40:58.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 -- start start 2024-04-25T13:40:58.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03a0071e60 0x7f03a0156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03a0073c40 0x7f03a0146340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0165060 0x7f03a0072480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f03a006e470 con 0x7f03a0073c40 2024-04-25T13:40:58.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f03a0074050 con 0x7f03a0071e60 2024-04-25T13:40:58.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a7ea4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f03a006e5f0 con 0x7f03a0165060 2024-04-25T13:40:58.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a543f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03a0073c40 0x7f03a0146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a543f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03a0073c40 0x7f03a0146340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48904/0 (socket says 172.21.15.80:48904) 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a543f640 1 -- 172.21.15.80:0/1077294888 learned_addr learned my addr 172.21.15.80:0/1077294888 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a6441640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0165060 0x7f03a0072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a5c40640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03a0071e60 0x7f03a0156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a543f640 1 -- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0165060 msgr2=0x7f03a0072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a543f640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0165060 0x7f03a0072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a543f640 1 -- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03a0071e60 msgr2=0x7f03a0156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a543f640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03a0071e60 0x7f03a0156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.919+0000 7f03a543f640 1 -- 172.21.15.80:0/1077294888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03a0165840 con 0x7f03a0073c40 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03a543f640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03a0073c40 0x7f03a0146340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f039001ab50 tx=0x7f0390019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03967fc640 1 -- 172.21.15.80:0/1077294888 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900bc020 con 0x7f03a0073c40 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03967fc640 1 -- 172.21.15.80:0/1077294888 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03900b8420 con 0x7f03a0073c40 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03a0165b20 con 0x7f03a0073c40 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03967fc640 1 -- 172.21.15.80:0/1077294888 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03900b2040 con 0x7f03a0073c40 2024-04-25T13:40:58.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03a01660e0 con 0x7f03a0073c40 2024-04-25T13:40:58.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03a006a3a0 con 0x7f03a0073c40 2024-04-25T13:40:58.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03967fc640 1 -- 172.21.15.80:0/1077294888 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f03900bf020 con 0x7f03a0073c40 2024-04-25T13:40:58.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03967fc640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f038406f400 0x7f0384071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:58.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03967fc640 1 -- 172.21.15.80:0/1077294888 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0390129710 con 0x7f03a0073c40 2024-04-25T13:40:58.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03a5c40640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f038406f400 0x7f0384071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:58.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.923+0000 7f03a5c40640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f038406f400 0x7f0384071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f039c000f70 tx=0x7f039c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:58.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:58.927+0000 7f03967fc640 1 -- 172.21.15.80:0/1077294888 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f03900c6030 con 0x7f03a0073c40 2024-04-25T13:40:59.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.087+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f03a0165560 con 0x7f03a0073c40 2024-04-25T13:40:59.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03967fc640 1 -- 172.21.15.80:0/1077294888 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0390016090 con 0x7f03a0073c40 2024-04-25T13:40:59.091 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f038406f400 msgr2=0x7f0384071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f038406f400 0x7f0384071910 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f039c000f70 tx=0x7f039c009040 comp rx=0 tx=0).stop 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03a0073c40 msgr2=0x7f03a0146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03a0073c40 0x7f03a0146340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f039001ab50 tx=0x7f0390019240 comp rx=0 tx=0).stop 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 shutdown_connections 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f038406f400 0x7f0384071910 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03a0165060 0x7f03a0072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03a0073c40 0x7f03a0146340 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 --2- 172.21.15.80:0/1077294888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03a0071e60 0x7f03a0156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 >> 172.21.15.80:0/1077294888 conn(0x7f03a00b4160 msgr2=0x7f03a0160f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 shutdown_connections 2024-04-25T13:40:59.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.091+0000 7f03a7ea4640 1 -- 172.21.15.80:0/1077294888 wait complete. 2024-04-25T13:40:59.112 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:40:59.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e4180c640 1 Processor -- start 2024-04-25T13:40:59.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e4180c640 1 -- start start 2024-04-25T13:40:59.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e4180c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2e3c0d1e90 0x7f2e3c0b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e4180c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 0x7f2e3c0d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e4180c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2e3c0d4a30 con 0x7f2e3c0d1a80 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e4180c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2e3c0d2b40 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e4180c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c0b4490 con 0x7f2e3c0b6e60 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e3a7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2e3c0d1e90 0x7f2e3c0b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e3affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2e3c0d1a80 0x7f2e3c0cf4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45168/0 (socket says 172.21.15.80:45168) 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e3affd640 1 -- 172.21.15.80:0/1141280163 learned_addr learned my addr 172.21.15.80:0/1141280163 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e3b7fe640 1 --2- 172.21.15.80:0/1141280163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 0x7f2e3c0d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e3a7fc640 1 -- 172.21.15.80:0/1141280163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 msgr2=0x7f2e3c0d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e3a7fc640 1 --2- 172.21.15.80:0/1141280163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 0x7f2e3c0d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e3a7fc640 1 -- 172.21.15.80:0/1141280163 >> v1:172.21.15.80:6789/0 conn(0x7f2e3c0d1a80 legacy=0x7f2e3c0cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e39ffb640 1 -- 172.21.15.80:0/1141280163 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3459339090 0 0) 0x7f2e3c0d4a30 con 0x7f2e3c0d1a80 2024-04-25T13:40:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.259+0000 7f2e3a7fc640 1 -- 172.21.15.80:0/1141280163 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c0b4cd0 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e3a7fc640 1 --2- 172.21.15.80:0/1141280163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0b6920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e300097c0 tx=0x7f2e3001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60941f81b7265b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e39ffb640 1 -- 172.21.15.80:0/1141280163 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300b0070 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e39ffb640 1 -- 172.21.15.80:0/1141280163 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e300ab600 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e39ffb640 1 -- 172.21.15.80:0/1141280163 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300ab9d0 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e4180c640 1 -- 172.21.15.80:0/1141280163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 msgr2=0x7f2e3c0b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e4180c640 1 --2- 172.21.15.80:0/1141280163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0b6920 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2e300097c0 tx=0x7f2e3001c410 comp rx=0 tx=0).stop 2024-04-25T13:40:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e4180c640 1 -- 172.21.15.80:0/1141280163 shutdown_connections 2024-04-25T13:40:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e4180c640 1 --2- 172.21.15.80:0/1141280163 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 0x7f2e3c0d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e4180c640 1 --2- 172.21.15.80:0/1141280163 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0b6920 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e4180c640 1 -- 172.21.15.80:0/1141280163 >> 172.21.15.80:0/1141280163 conn(0x7f2e3c0d3ad0 msgr2=0x7f2e3c0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e4180c640 1 -- 172.21.15.80:0/1141280163 shutdown_connections 2024-04-25T13:40:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.263+0000 7f2e4180c640 1 -- 172.21.15.80:0/1141280163 wait complete. 2024-04-25T13:40:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 Processor -- start 2024-04-25T13:40:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 -- start start 2024-04-25T13:40:59.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 0x7f2e3c0b3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3c0d1a80 0x7f2e3c157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0c8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c0cfbc0 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3c0d2760 con 0x7f2e3c0d1a80 2024-04-25T13:40:59.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2e3c0cfd40 con 0x7f2e3c0b6e60 2024-04-25T13:40:59.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0c8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0c8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48914/0 (socket says 172.21.15.80:48914) 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 -- 172.21.15.80:0/3698743852 learned_addr learned my addr 172.21.15.80:0/3698743852 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3a7fc640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3c0d1a80 0x7f2e3c157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3affd640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 0x7f2e3c0b3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 -- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 msgr2=0x7f2e3c0b3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 0x7f2e3c0b3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 -- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3c0d1a80 msgr2=0x7f2e3c157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3c0d1a80 0x7f2e3c157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 -- 172.21.15.80:0/3698743852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3c159d80 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e3b7fe640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0c8020 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e34007d80 tx=0x7f2e3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e23fff640 1 -- 172.21.15.80:0/3698743852 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34014420 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e23fff640 1 -- 172.21.15.80:0/3698743852 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e34019070 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3c15a060 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e23fff640 1 -- 172.21.15.80:0/3698743852 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34014800 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.267+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e3c15a620 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.271+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e3c06a3a0 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.271+0000 7f2e23fff640 1 -- 172.21.15.80:0/3698743852 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2e340149a0 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.271+0000 7f2e23fff640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e2806f400 0x7f2e28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.271+0000 7f2e23fff640 1 -- 172.21.15.80:0/3698743852 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2e3408fb50 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.271+0000 7f2e3affd640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e2806f400 0x7f2e28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.271+0000 7f2e3affd640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e2806f400 0x7f2e28071910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000f70 tx=0x7f2e2c007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.275+0000 7f2e23fff640 1 -- 172.21.15.80:0/3698743852 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2e3405ca40 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2e3c069cb0 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e23fff640 1 -- 172.21.15.80:0/3698743852 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f2e3405c340 con 0x7f2e3c0d1e90 2024-04-25T13:40:59.440 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e2806f400 msgr2=0x7f2e28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e2806f400 0x7f2e28071910 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2e2c000f70 tx=0x7f2e2c007c10 comp rx=0 tx=0).stop 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 msgr2=0x7f2e3c0c8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0c8020 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2e34007d80 tx=0x7f2e3400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 shutdown_connections 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e2806f400 0x7f2e28071910 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2e3c0d1e90 0x7f2e3c0c8020 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2e3c0d1a80 0x7f2e3c157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 --2- 172.21.15.80:0/3698743852 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2e3c0b6e60 0x7f2e3c0b3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 >> 172.21.15.80:0/3698743852 conn(0x7f2e3c0d3ad0 msgr2=0x7f2e3c0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 shutdown_connections 2024-04-25T13:40:59.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.439+0000 7f2e4180c640 1 -- 172.21.15.80:0/3698743852 wait complete. 2024-04-25T13:40:59.462 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:40:59.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32cb885640 1 Processor -- start 2024-04-25T13:40:59.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32cb885640 1 -- start start 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32cb885640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32c4071e60 0x7f32c41574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32cb885640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4073c40 0x7f32c4160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32cb885640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32c4160fa0 0x7f32c40b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32cb885640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f32c40b1350 con 0x7f32c4071e60 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32cb885640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f32c406f9a0 con 0x7f32c4160fa0 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32cb885640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f32c41613b0 con 0x7f32c4073c40 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9621640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32c4071e60 0x7f32c41574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9621640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32c4071e60 0x7f32c41574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48916/0 (socket says 172.21.15.80:48916) 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9621640 1 -- 172.21.15.80:0/3286716672 learned_addr learned my addr 172.21.15.80:0/3286716672 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c8e20640 1 --2- 172.21.15.80:0/3286716672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4073c40 0x7f32c4160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9e22640 1 --2- 172.21.15.80:0/3286716672 >> v2:172.21.15.87:3300/0 conn(0x7f32c4160fa0 0x7f32c40b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9621640 1 -- 172.21.15.80:0/3286716672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4073c40 msgr2=0x7f32c4160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:59.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9621640 1 --2- 172.21.15.80:0/3286716672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4073c40 0x7f32c4160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9621640 1 -- 172.21.15.80:0/3286716672 >> v2:172.21.15.87:3300/0 conn(0x7f32c4160fa0 msgr2=0x7f32c40b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:59.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9621640 1 --2- 172.21.15.80:0/3286716672 >> v2:172.21.15.87:3300/0 conn(0x7f32c4160fa0 0x7f32c40b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9621640 1 -- 172.21.15.80:0/3286716672 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c406fbf0 con 0x7f32c4071e60 2024-04-25T13:40:59.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.611+0000 7f32c9e22640 1 --2- 172.21.15.80:0/3286716672 >> v2:172.21.15.87:3300/0 conn(0x7f32c4160fa0 0x7f32c40b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:59.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32c9621640 1 --2- 172.21.15.80:0/3286716672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4071e60 0x7f32c41574c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f32c4071df0 tx=0x7f32c001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5590edb265886b77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32bbfff640 1 -- 172.21.15.80:0/3286716672 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00b1070 con 0x7f32c4071e60 2024-04-25T13:40:59.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32bbfff640 1 -- 172.21.15.80:0/3286716672 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32c00ac630 con 0x7f32c4071e60 2024-04-25T13:40:59.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32bbfff640 1 -- 172.21.15.80:0/3286716672 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c00a8050 con 0x7f32c4071e60 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 -- 172.21.15.80:0/3286716672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4071e60 msgr2=0x7f32c41574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 --2- 172.21.15.80:0/3286716672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4071e60 0x7f32c41574c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f32c4071df0 tx=0x7f32c001c520 comp rx=0 tx=0).stop 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 -- 172.21.15.80:0/3286716672 shutdown_connections 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 --2- 172.21.15.80:0/3286716672 >> v2:172.21.15.87:3300/0 conn(0x7f32c4160fa0 0x7f32c40b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 --2- 172.21.15.80:0/3286716672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4073c40 0x7f32c4160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 --2- 172.21.15.80:0/3286716672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4071e60 0x7f32c41574c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 -- 172.21.15.80:0/3286716672 >> 172.21.15.80:0/3286716672 conn(0x7f32c40b4160 msgr2=0x7f32c40d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 -- 172.21.15.80:0/3286716672 shutdown_connections 2024-04-25T13:40:59.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.615+0000 7f32cb885640 1 -- 172.21.15.80:0/3286716672 wait complete. 2024-04-25T13:40:59.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 Processor -- start 2024-04-25T13:40:59.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 -- start start 2024-04-25T13:40:59.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4071e60 0x7f32c41509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4073c40 0x7f32c4151ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32c4160fa0 0x7f32c4152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f32c4074050 con 0x7f32c4073c40 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f32c4072270 con 0x7f32c4160fa0 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f32c415d470 con 0x7f32c4071e60 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4073c40 0x7f32c4151ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4073c40 0x7f32c4151ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48922/0 (socket says 172.21.15.80:48922) 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 -- 172.21.15.80:0/857139548 learned_addr learned my addr 172.21.15.80:0/857139548 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c9e22640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32c4160fa0 0x7f32c4152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c9621640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4071e60 0x7f32c41509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 -- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4071e60 msgr2=0x7f32c41509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4071e60 0x7f32c41509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 -- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32c4160fa0 msgr2=0x7f32c4152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32c4160fa0 0x7f32c4152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 -- 172.21.15.80:0/857139548 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32c40ab780 con 0x7f32c4073c40 2024-04-25T13:40:59.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c9621640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4071e60 0x7f32c41509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:59.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32c8e20640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4073c40 0x7f32c4151ac0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e870 tx=0x7f32b400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32b9ffb640 1 -- 172.21.15.80:0/857139548 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b4017070 con 0x7f32c4073c40 2024-04-25T13:40:59.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32b9ffb640 1 -- 172.21.15.80:0/857139548 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32b400ee60 con 0x7f32c4073c40 2024-04-25T13:40:59.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32c40aba60 con 0x7f32c4073c40 2024-04-25T13:40:59.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32b9ffb640 1 -- 172.21.15.80:0/857139548 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b400cb80 con 0x7f32c4073c40 2024-04-25T13:40:59.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.619+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32c40ac020 con 0x7f32c4073c40 2024-04-25T13:40:59.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.623+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32c406a3a0 con 0x7f32c4073c40 2024-04-25T13:40:59.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.623+0000 7f32b9ffb640 1 -- 172.21.15.80:0/857139548 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f32b401a070 con 0x7f32c4073c40 2024-04-25T13:40:59.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.623+0000 7f32b9ffb640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32a8071660 0x7f32a8073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.623+0000 7f32b9ffb640 1 -- 172.21.15.80:0/857139548 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f32b408f7d0 con 0x7f32c4073c40 2024-04-25T13:40:59.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.623+0000 7f32c9621640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32a8071660 0x7f32a8073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.623+0000 7f32c9621640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32a8071660 0x7f32a8073b70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32c00ae000 tx=0x7f32c00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.627+0000 7f32b9ffb640 1 -- 172.21.15.80:0/857139548 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32b405c7a0 con 0x7f32c4073c40 2024-04-25T13:40:59.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.779+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f32c40ab4d0 con 0x7f32c4073c40 2024-04-25T13:40:59.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.779+0000 7f32b9ffb640 1 -- 172.21.15.80:0/857139548 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f32b405c0a0 con 0x7f32c4073c40 2024-04-25T13:40:59.782 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32a8071660 msgr2=0x7f32a8073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32a8071660 0x7f32a8073b70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f32c00ae000 tx=0x7f32c00a7000 comp rx=0 tx=0).stop 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4073c40 msgr2=0x7f32c4151ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4073c40 0x7f32c4151ac0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f32b400e870 tx=0x7f32b400ed30 comp rx=0 tx=0).stop 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 shutdown_connections 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32a8071660 0x7f32a8073b70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32c4160fa0 0x7f32c4152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32c4073c40 0x7f32c4151ac0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 --2- 172.21.15.80:0/857139548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32c4071e60 0x7f32c41509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 >> 172.21.15.80:0/857139548 conn(0x7f32c40b4160 msgr2=0x7f32c415e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:59.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 shutdown_connections 2024-04-25T13:40:59.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.783+0000 7f32cb885640 1 -- 172.21.15.80:0/857139548 wait complete. 2024-04-25T13:40:59.802 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:40:59.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.947+0000 7f6067992640 1 Processor -- start 2024-04-25T13:40:59.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.947+0000 7f6067992640 1 -- start start 2024-04-25T13:40:59.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6067992640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d2e80 0x7f60601607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6067992640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60600d4330 0x7f60600d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6067992640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6060161060 con 0x7f60600d59b0 2024-04-25T13:40:59.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6067992640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6060161340 con 0x7f60600d4330 2024-04-25T13:40:59.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6067992640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f60600d2680 con 0x7f60600d2e80 2024-04-25T13:40:59.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6065f2f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60600d4330 0x7f60600d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6065f2f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60600d4330 0x7f60600d4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48932/0 (socket says 172.21.15.80:48932) 2024-04-25T13:40:59.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6065f2f640 1 -- 172.21.15.80:0/972294552 learned_addr learned my addr 172.21.15.80:0/972294552 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:59.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f606572e640 1 --2- 172.21.15.80:0/972294552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d2e80 0x7f60601607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6065f2f640 1 -- 172.21.15.80:0/972294552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d2e80 msgr2=0x7f60601607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:59.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6065f2f640 1 --2- 172.21.15.80:0/972294552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d2e80 0x7f60601607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6065f2f640 1 -- 172.21.15.80:0/972294552 >> v1:172.21.15.87:6789/0 conn(0x7f60600d59b0 legacy=0x7f60600d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:40:59.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.951+0000 7f6065f2f640 1 -- 172.21.15.80:0/972294552 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60600d2960 con 0x7f60600d4330 2024-04-25T13:40:59.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6065f2f640 1 --2- 172.21.15.80:0/972294552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60600d4740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6050009870 tx=0x7f605001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3701f6c6e7d9e63f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f605ffff640 1 -- 172.21.15.80:0/972294552 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500a7070 con 0x7f60600d4330 2024-04-25T13:40:59.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f605ffff640 1 -- 172.21.15.80:0/972294552 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60500b0070 con 0x7f60600d4330 2024-04-25T13:40:59.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f605ffff640 1 -- 172.21.15.80:0/972294552 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60500ab890 con 0x7f60600d4330 2024-04-25T13:40:59.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 -- 172.21.15.80:0/972294552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 msgr2=0x7f60600d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 --2- 172.21.15.80:0/972294552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60600d4740 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6050009870 tx=0x7f605001c490 comp rx=0 tx=0).stop 2024-04-25T13:40:59.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 -- 172.21.15.80:0/972294552 shutdown_connections 2024-04-25T13:40:59.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 --2- 172.21.15.80:0/972294552 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60600d4740 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 --2- 172.21.15.80:0/972294552 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d2e80 0x7f60601607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 -- 172.21.15.80:0/972294552 >> 172.21.15.80:0/972294552 conn(0x7f60600737a0 msgr2=0x7f606006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:40:59.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 -- 172.21.15.80:0/972294552 shutdown_connections 2024-04-25T13:40:59.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 -- 172.21.15.80:0/972294552 wait complete. 2024-04-25T13:40:59.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 Processor -- start 2024-04-25T13:40:59.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.955+0000 7f6067992640 1 -- start start 2024-04-25T13:40:59.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60600d2e80 0x7f6060156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60601659e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d59b0 0x7f60601547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f60600d3410 con 0x7f60600d4330 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f60600d3290 con 0x7f60600d2e80 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f606015df60 con 0x7f60600d59b0 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60601659e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60601659e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48938/0 (socket says 172.21.15.80:48938) 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 -- 172.21.15.80:0/2025502788 learned_addr learned my addr 172.21.15.80:0/2025502788 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6065f2f640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d59b0 0x7f60601547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f606572e640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60600d2e80 0x7f6060156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 -- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d59b0 msgr2=0x7f60601547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d59b0 0x7f60601547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 -- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60600d2e80 msgr2=0x7f6060156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60600d2e80 0x7f6060156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 -- 172.21.15.80:0/2025502788 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6060155260 con 0x7f60600d4330 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f606572e640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60600d2e80 0x7f6060156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6064f2d640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60601659e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f605400b440 tx=0x7f605400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f605dffb640 1 -- 172.21.15.80:0/2025502788 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054015020 con 0x7f60600d4330 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f605dffb640 1 -- 172.21.15.80:0/2025502788 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6054011890 con 0x7f60600d4330 2024-04-25T13:40:59.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f605dffb640 1 -- 172.21.15.80:0/2025502788 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6054011c80 con 0x7f60600d4330 2024-04-25T13:40:59.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6060155540 con 0x7f60600d4330 2024-04-25T13:40:59.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6060155b00 con 0x7f60600d4330 2024-04-25T13:40:59.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f606006a3a0 con 0x7f60600d4330 2024-04-25T13:40:59.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.959+0000 7f605dffb640 1 -- 172.21.15.80:0/2025502788 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6054018020 con 0x7f60600d4330 2024-04-25T13:40:59.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.963+0000 7f605dffb640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f604c06f400 0x7f604c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:40:59.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.963+0000 7f605dffb640 1 -- 172.21.15.80:0/2025502788 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f605408f0f0 con 0x7f60600d4330 2024-04-25T13:40:59.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.963+0000 7f606572e640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f604c06f400 0x7f604c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:40:59.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.963+0000 7f606572e640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f604c06f400 0x7f604c071910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f60600b8fd0 tx=0x7f605800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:40:59.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:40:59.967+0000 7f605dffb640 1 -- 172.21.15.80:0/2025502788 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f605401f030 con 0x7f60600d4330 2024-04-25T13:41:00.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.115+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6060166350 con 0x7f60600d4330 2024-04-25T13:41:00.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f605dffb640 1 -- 172.21.15.80:0/2025502788 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f605400d040 con 0x7f60600d4330 2024-04-25T13:41:00.120 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:00.120 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:00.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f604c06f400 msgr2=0x7f604c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f604c06f400 0x7f604c071910 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f60600b8fd0 tx=0x7f605800b040 comp rx=0 tx=0).stop 2024-04-25T13:41:00.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 msgr2=0x7f60601659e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60601659e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f605400b440 tx=0x7f605400b900 comp rx=0 tx=0).stop 2024-04-25T13:41:00.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 shutdown_connections 2024-04-25T13:41:00.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f604c06f400 0x7f604c071910 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60600d59b0 0x7f60601547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60600d4330 0x7f60601659e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 --2- 172.21.15.80:0/2025502788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60600d2e80 0x7f6060156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 >> 172.21.15.80:0/2025502788 conn(0x7f60600737a0 msgr2=0x7f6060163010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:00.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 shutdown_connections 2024-04-25T13:41:00.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.119+0000 7f6067992640 1 -- 172.21.15.80:0/2025502788 wait complete. 2024-04-25T13:41:00.140 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:00.140 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:00.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f662754a640 1 Processor -- start 2024-04-25T13:41:00.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f662754a640 1 -- start start 2024-04-25T13:41:00.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f662754a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f66200d3ed0 0x7f66200d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f662754a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f662754a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66200b92a0 con 0x7f66200d2ad0 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f662754a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f66200d4c10 con 0x7f66200d3ed0 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f662754a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66200d4ef0 con 0x7f66200ba770 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f66252e6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f66200d3ed0 0x7f66200d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f6624ae5640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f66200d2ad0 0x7f66200b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45174/0 (socket says 172.21.15.80:45174) 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f6624ae5640 1 -- 172.21.15.80:0/4188727291 learned_addr learned my addr 172.21.15.80:0/4188727291 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f6617fff640 1 -- 172.21.15.80:0/4188727291 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081944331 0 0) 0x7f6610001120 con 0x7f66200d2ad0 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f6617fff640 1 -- 172.21.15.80:0/4188727291 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f660c003530 con 0x7f66200d2ad0 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f6625ae7640 1 --2- 172.21.15.80:0/4188727291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f66252e6640 1 -- 172.21.15.80:0/4188727291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 msgr2=0x7f66200b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:00.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f66252e6640 1 --2- 172.21.15.80:0/4188727291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200b7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f66252e6640 1 -- 172.21.15.80:0/4188727291 >> v1:172.21.15.80:6789/0 conn(0x7f66200d2ad0 legacy=0x7f66200b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.291+0000 7f66252e6640 1 -- 172.21.15.80:0/4188727291 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66200d51d0 con 0x7f66200d3ed0 2024-04-25T13:41:00.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f66252e6640 1 --2- 172.21.15.80:0/4188727291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66200d2590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661c0097c0 tx=0x7f661c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9d1ca38112a1cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f6617fff640 1 -- 172.21.15.80:0/4188727291 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c0b0070 con 0x7f66200d3ed0 2024-04-25T13:41:00.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f6617fff640 1 -- 172.21.15.80:0/4188727291 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f661c0ab600 con 0x7f66200d3ed0 2024-04-25T13:41:00.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f6617fff640 1 -- 172.21.15.80:0/4188727291 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661c0ab9d0 con 0x7f66200d3ed0 2024-04-25T13:41:00.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 -- 172.21.15.80:0/4188727291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 msgr2=0x7f66200d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 --2- 172.21.15.80:0/4188727291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66200d2590 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f661c0097c0 tx=0x7f661c01c350 comp rx=0 tx=0).stop 2024-04-25T13:41:00.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 -- 172.21.15.80:0/4188727291 shutdown_connections 2024-04-25T13:41:00.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 --2- 172.21.15.80:0/4188727291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 --2- 172.21.15.80:0/4188727291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66200d2590 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 -- 172.21.15.80:0/4188727291 >> 172.21.15.80:0/4188727291 conn(0x7f66200737b0 msgr2=0x7f662006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:00.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 -- 172.21.15.80:0/4188727291 shutdown_connections 2024-04-25T13:41:00.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 -- 172.21.15.80:0/4188727291 wait complete. 2024-04-25T13:41:00.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 Processor -- start 2024-04-25T13:41:00.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.295+0000 7f662754a640 1 -- start start 2024-04-25T13:41:00.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200ce780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66200d2ad0 0x7f66200cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66201597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f66200b8320 con 0x7f66200d3ed0 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66200b84a0 con 0x7f66200d2ad0 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6620160620 con 0x7f66200ba770 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66201597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66201597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48960/0 (socket says 172.21.15.80:48960) 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 -- 172.21.15.80:0/1797210883 learned_addr learned my addr 172.21.15.80:0/1797210883 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f66252e6640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6624ae5640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66200d2ad0 0x7f66200cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 -- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 msgr2=0x7f66200ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 -- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66200d2ad0 msgr2=0x7f66200cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66200d2ad0 0x7f66200cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 -- 172.21.15.80:0/1797210883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66200cdbf0 con 0x7f66200d3ed0 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6624ae5640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66200d2ad0 0x7f66200cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f66252e6640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6625ae7640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66201597e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f661800cc30 tx=0x7f661800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6615ffb640 1 -- 172.21.15.80:0/1797210883 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f661800ce40 con 0x7f66200d3ed0 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6615ffb640 1 -- 172.21.15.80:0/1797210883 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66180039a0 con 0x7f66200d3ed0 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6615ffb640 1 -- 172.21.15.80:0/1797210883 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6618003d50 con 0x7f66200d3ed0 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66200cd070 con 0x7f66200d3ed0 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66200cd630 con 0x7f66200d3ed0 2024-04-25T13:41:00.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66200ccb70 con 0x7f66200d3ed0 2024-04-25T13:41:00.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6615ffb640 1 -- 172.21.15.80:0/1797210883 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6618003ef0 con 0x7f66200d3ed0 2024-04-25T13:41:00.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.299+0000 7f6615ffb640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f660406f400 0x7f6604071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.303+0000 7f6615ffb640 1 -- 172.21.15.80:0/1797210883 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f661808f760 con 0x7f66200d3ed0 2024-04-25T13:41:00.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.303+0000 7f66252e6640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f660406f400 0x7f6604071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.303+0000 7f66252e6640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f660406f400 0x7f6604071910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f661c0ad000 tx=0x7f661c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.307+0000 7f6615ffb640 1 -- 172.21.15.80:0/1797210883 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6618014030 con 0x7f66200d3ed0 2024-04-25T13:41:00.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.467+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f66200d1480 con 0x7f66200d3ed0 2024-04-25T13:41:00.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f6615ffb640 1 -- 172.21.15.80:0/1797210883 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f661805c650 con 0x7f66200d3ed0 2024-04-25T13:41:00.471 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f660406f400 msgr2=0x7f6604071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f660406f400 0x7f6604071910 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f661c0ad000 tx=0x7f661c0a6000 comp rx=0 tx=0).stop 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 msgr2=0x7f66201597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66201597e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f661800cc30 tx=0x7f661800a5c0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 shutdown_connections 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f660406f400 0x7f6604071910 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66200d3ed0 0x7f66201597e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66200d2ad0 0x7f66200cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 --2- 172.21.15.80:0/1797210883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66200ba770 0x7f66200ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 >> 172.21.15.80:0/1797210883 conn(0x7f66200737b0 msgr2=0x7f662015e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:00.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 shutdown_connections 2024-04-25T13:41:00.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.471+0000 7f662754a640 1 -- 172.21.15.80:0/1797210883 wait complete. 2024-04-25T13:41:00.493 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:00.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f788a06b640 1 Processor -- start 2024-04-25T13:41:00.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f788a06b640 1 -- start start 2024-04-25T13:41:00.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f788a06b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7884068ca0 0x7f788406f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f788a06b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7884160690 con 0x7f78840b7d10 2024-04-25T13:41:00.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f788a06b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78840b1440 con 0x7f78840b7900 2024-04-25T13:41:00.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f788a06b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f788406f4c0 con 0x7f7884068ca0 2024-04-25T13:41:00.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7883fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f78840b7d10 0x7f788406fe90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45178/0 (socket says 172.21.15.80:45178) 2024-04-25T13:41:00.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7883fff640 1 -- 172.21.15.80:0/3229843808 learned_addr learned my addr 172.21.15.80:0/3229843808 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:00.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7882ffd640 1 --2- 172.21.15.80:0/3229843808 >> v2:172.21.15.87:3300/0 conn(0x7f7884068ca0 0x7f788406f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3284814240 0 0) 0x7f7884160690 con 0x7f78840b7d10 2024-04-25T13:41:00.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f786c003530 con 0x7f78840b7d10 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 624760893 0 0) 0x7f786c003530 con 0x7f78840b7d10 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 >> v2:172.21.15.87:3300/0 conn(0x7f7884068ca0 msgr2=0x7f788406f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 --2- 172.21.15.80:0/3229843808 >> v2:172.21.15.87:3300/0 conn(0x7f7884068ca0 0x7f788406f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 >> v1:172.21.15.87:6789/0 conn(0x7f78840b7900 legacy=0x7f7884068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78840b1720 con 0x7f78840b7d10 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f787c001a00 con 0x7f78840b7d10 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 >> v1:172.21.15.80:6789/0 conn(0x7f78840b7d10 legacy=0x7f788406fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f786c0039c0 0x7f786c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c004810 0x7f786c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f786c005260 0x7f786c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f788406f4c0 con 0x7f786c005260 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f786c00c1f0 con 0x7f786c004810 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f786c00c3f0 con 0x7f786c0039c0 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7883fff640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f786c005260 0x7f786c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f78837fe640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f786c0039c0 0x7f786c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7882ffd640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c004810 0x7f786c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7883fff640 1 -- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f786c0039c0 msgr2=0x7f786c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7883fff640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f786c0039c0 0x7f786c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7883fff640 1 -- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c004810 msgr2=0x7f786c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7883fff640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c004810 0x7f786c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7883fff640 1 -- 172.21.15.80:0/3229843808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786c003530 con 0x7f786c005260 2024-04-25T13:41:00.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.639+0000 7f7882ffd640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c004810 0x7f786c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f7883fff640 1 -- 172.21.15.80:0/3229843808 reap_dead start 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f7883fff640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f786c005260 0x7f786c00b9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7884072ec0 tx=0x7f787c01fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63da2e3e82bd083c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c00a460 con 0x7f786c005260 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787c0aa6b0 con 0x7f786c005260 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f78827fc640 1 -- 172.21.15.80:0/3229843808 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c0aaaa0 con 0x7f786c005260 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f788a06b640 1 -- 172.21.15.80:0/3229843808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f786c005260 msgr2=0x7f786c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f788a06b640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f786c005260 0x7f786c00b9d0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7884072ec0 tx=0x7f787c01fa60 comp rx=0 tx=0).stop 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f788a06b640 1 -- 172.21.15.80:0/3229843808 shutdown_connections 2024-04-25T13:41:00.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f788a06b640 1 --2- 172.21.15.80:0/3229843808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f786c005260 0x7f786c00b9d0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f788a06b640 1 -- 172.21.15.80:0/3229843808 >> 172.21.15.80:0/3229843808 conn(0x7f7884073730 msgr2=0x7f7884073b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:00.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f788a06b640 1 -- 172.21.15.80:0/3229843808 shutdown_connections 2024-04-25T13:41:00.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.643+0000 7f788a06b640 1 -- 172.21.15.80:0/3229843808 wait complete. 2024-04-25T13:41:00.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 Processor -- start 2024-04-25T13:41:00.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 -- start start 2024-04-25T13:41:00.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c005260 0x7f788415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7884144160 0x7f78840cf1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78840cf720 0x7f78841f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f78840b0fa0 con 0x7f78840cf720 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f788406f150 con 0x7f786c005260 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f78840b1120 con 0x7f7884144160 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78840cf720 0x7f78841f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78840cf720 0x7f78841f97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48978/0 (socket says 172.21.15.80:48978) 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 -- 172.21.15.80:0/863759541 learned_addr learned my addr 172.21.15.80:0/863759541 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f78837fe640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c005260 0x7f788415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7882ffd640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7884144160 0x7f78840cf1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 -- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7884144160 msgr2=0x7f78840cf1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7884144160 0x7f78840cf1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 -- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c005260 msgr2=0x7f788415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c005260 0x7f788415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 -- 172.21.15.80:0/863759541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7884159660 con 0x7f78840cf720 2024-04-25T13:41:00.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f78837fe640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c005260 0x7f788415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:00.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7882ffd640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7884144160 0x7f78840cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:00.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f7883fff640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78840cf720 0x7f78841f97e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f787c01f890 tx=0x7f786c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f786bfff640 1 -- 172.21.15.80:0/863759541 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c0bb070 con 0x7f78840cf720 2024-04-25T13:41:00.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f786bfff640 1 -- 172.21.15.80:0/863759541 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f787c001260 con 0x7f78840cf720 2024-04-25T13:41:00.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f786bfff640 1 -- 172.21.15.80:0/863759541 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f787c00a460 con 0x7f78840cf720 2024-04-25T13:41:00.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788415ab10 con 0x7f78840cf720 2024-04-25T13:41:00.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788415b850 con 0x7f78840cf720 2024-04-25T13:41:00.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.647+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78840d1eb0 con 0x7f78840cf720 2024-04-25T13:41:00.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.651+0000 7f786bfff640 1 -- 172.21.15.80:0/863759541 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f787c0be070 con 0x7f78840cf720 2024-04-25T13:41:00.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.651+0000 7f786bfff640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f787006f430 0x7f7870071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.651+0000 7f786bfff640 1 -- 172.21.15.80:0/863759541 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f787c12c130 con 0x7f78840cf720 2024-04-25T13:41:00.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.651+0000 7f78837fe640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f787006f430 0x7f7870071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.651+0000 7f78837fe640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f787006f430 0x7f7870071940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7878007fd0 tx=0x7f7878009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.655+0000 7f786bfff640 1 -- 172.21.15.80:0/863759541 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f787c019770 con 0x7f78840cf720 2024-04-25T13:41:00.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.807+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f788406a3a0 con 0x7f78840cf720 2024-04-25T13:41:00.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f786bfff640 1 -- 172.21.15.80:0/863759541 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f787c0f9220 con 0x7f78840cf720 2024-04-25T13:41:00.811 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:00.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f787006f430 msgr2=0x7f7870071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f787006f430 0x7f7870071940 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7878007fd0 tx=0x7f7878009040 comp rx=0 tx=0).stop 2024-04-25T13:41:00.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78840cf720 msgr2=0x7f78841f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78840cf720 0x7f78841f97e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f787c01f890 tx=0x7f786c005840 comp rx=0 tx=0).stop 2024-04-25T13:41:00.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 shutdown_connections 2024-04-25T13:41:00.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f787006f430 0x7f7870071940 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78840cf720 0x7f78841f97e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7884144160 0x7f78840cf1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 --2- 172.21.15.80:0/863759541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f786c005260 0x7f788415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 >> 172.21.15.80:0/863759541 conn(0x7f7884073730 msgr2=0x7f78840b3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:00.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 shutdown_connections 2024-04-25T13:41:00.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.811+0000 7f788a06b640 1 -- 172.21.15.80:0/863759541 wait complete. 2024-04-25T13:41:00.831 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:00.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d85c1640 1 Processor -- start 2024-04-25T13:41:00.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d85c1640 1 -- start start 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d85c1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01574c0 0x7f43d0073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d85c1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d006fbf0 con 0x7f43d0161170 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d85c1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43d00b0a20 con 0x7f43d0071e60 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d85c1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f43d00b1350 con 0x7f43d01574c0 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d5b5c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f43d0161170 0x7f43d00b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45186/0 (socket says 172.21.15.80:45186) 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d5b5c640 1 -- 172.21.15.80:0/3664072761 learned_addr learned my addr 172.21.15.80:0/3664072761 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:00.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d6b5e640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01574c0 0x7f43d0073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4116233144 0 0) 0x7f43d006fbf0 con 0x7f43d0161170 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f43bc003530 con 0x7f43d0161170 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 722850770 0 0) 0x7f43bc003530 con 0x7f43d0161170 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01574c0 msgr2=0x7f43d0073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01574c0 0x7f43d0073c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 >> v1:172.21.15.87:6789/0 conn(0x7f43d0071e60 legacy=0x7f43d0160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d00b0d00 con 0x7f43d0161170 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f43c0001680 con 0x7f43d0161170 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 >> v1:172.21.15.80:6789/0 conn(0x7f43d0161170 legacy=0x7f43d00b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc0039f0 0x7f43bc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43bc004810 0x7f43bc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01574c0 0x7f43bc005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43d00b1350 con 0x7f43bc004810 2024-04-25T13:41:00.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f43bc00bb60 con 0x7f43bc0039f0 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f43bc00bd30 con 0x7f43d01574c0 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d5b5c640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43bc004810 0x7f43bc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d635d640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc0039f0 0x7f43bc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d6b5e640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01574c0 0x7f43bc005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d5b5c640 1 -- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01574c0 msgr2=0x7f43bc005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d5b5c640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01574c0 0x7f43bc005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d5b5c640 1 -- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc0039f0 msgr2=0x7f43bc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d5b5c640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc0039f0 0x7f43bc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.979+0000 7f43d5b5c640 1 -- 172.21.15.80:0/3664072761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43bc003530 con 0x7f43bc004810 2024-04-25T13:41:00.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d5b5c640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43bc004810 0x7f43bc004c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c1f0 tx=0x7f43c001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ae3145499185e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0007d00 con 0x7f43bc004810 2024-04-25T13:41:00.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c0005b00 con 0x7f43bc004810 2024-04-25T13:41:00.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d535b640 1 -- 172.21.15.80:0/3664072761 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0007070 con 0x7f43bc004810 2024-04-25T13:41:00.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d85c1640 1 -- 172.21.15.80:0/3664072761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43bc004810 msgr2=0x7f43bc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d85c1640 1 --2- 172.21.15.80:0/3664072761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43bc004810 0x7f43bc004c20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c1f0 tx=0x7f43c001f3b0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d6b5e640 1 -- 172.21.15.80:0/3664072761 reap_dead start 2024-04-25T13:41:00.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d85c1640 1 -- 172.21.15.80:0/3664072761 shutdown_connections 2024-04-25T13:41:00.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d85c1640 1 -- 172.21.15.80:0/3664072761 >> 172.21.15.80:0/3664072761 conn(0x7f43d00b4160 msgr2=0x7f43d00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:00.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d85c1640 1 -- 172.21.15.80:0/3664072761 shutdown_connections 2024-04-25T13:41:00.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d85c1640 1 -- 172.21.15.80:0/3664072761 wait complete. 2024-04-25T13:41:00.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d85c1640 1 Processor -- start 2024-04-25T13:41:00.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.983+0000 7f43d85c1640 1 -- start start 2024-04-25T13:41:00.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc004810 0x7f43d00d0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d0150d90 0x7f43d014d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01fa490 0x7f43d015f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43d0072270 con 0x7f43d0150d90 2024-04-25T13:41:00.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f43d006f9a0 con 0x7f43bc004810 2024-04-25T13:41:00.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f43d01578d0 con 0x7f43d01fa490 2024-04-25T13:41:00.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d0150d90 0x7f43d014d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d0150d90 0x7f43d014d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48992/0 (socket says 172.21.15.80:48992) 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 -- 172.21.15.80:0/3357893084 learned_addr learned my addr 172.21.15.80:0/3357893084 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d635d640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc004810 0x7f43d00d0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d6b5e640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01fa490 0x7f43d015f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 -- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01fa490 msgr2=0x7f43d015f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01fa490 0x7f43d015f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 -- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc004810 msgr2=0x7f43d00d0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc004810 0x7f43d00d0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 -- 172.21.15.80:0/3357893084 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43d00d0d70 con 0x7f43d0150d90 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d635d640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc004810 0x7f43d00d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d5b5c640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d0150d90 0x7f43d014d290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c1f0 tx=0x7f43bc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43c6ffd640 1 -- 172.21.15.80:0/3357893084 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0001c40 con 0x7f43d0150d90 2024-04-25T13:41:00.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43c6ffd640 1 -- 172.21.15.80:0/3357893084 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43c0005cb0 con 0x7f43d0150d90 2024-04-25T13:41:00.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43d01f99b0 con 0x7f43d0150d90 2024-04-25T13:41:00.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43c6ffd640 1 -- 172.21.15.80:0/3357893084 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0005e60 con 0x7f43d0150d90 2024-04-25T13:41:00.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43d01f9f70 con 0x7f43d0150d90 2024-04-25T13:41:00.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43d006a3a0 con 0x7f43d0150d90 2024-04-25T13:41:00.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.987+0000 7f43c6ffd640 1 -- 172.21.15.80:0/3357893084 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f43c00049f0 con 0x7f43d0150d90 2024-04-25T13:41:00.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.991+0000 7f43c6ffd640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f43b406f400 0x7f43b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:00.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.991+0000 7f43c6ffd640 1 -- 172.21.15.80:0/3357893084 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f43c012c530 con 0x7f43d0150d90 2024-04-25T13:41:00.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.991+0000 7f43d635d640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f43b406f400 0x7f43b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:00.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.991+0000 7f43d635d640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f43b406f400 0x7f43b4071910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0055b0 tx=0x7f43cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:00.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:00.995+0000 7f43c6ffd640 1 -- 172.21.15.80:0/3357893084 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f43c00190c0 con 0x7f43d0150d90 2024-04-25T13:41:01.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f43d014d7d0 con 0x7f43d0150d90 2024-04-25T13:41:01.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43c6ffd640 1 -- 172.21.15.80:0/3357893084 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f43c00f9540 con 0x7f43d0150d90 2024-04-25T13:41:01.156 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:01.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f43b406f400 msgr2=0x7f43b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f43b406f400 0x7f43b4071910 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f43cc0055b0 tx=0x7f43cc009040 comp rx=0 tx=0).stop 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d0150d90 msgr2=0x7f43d014d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d0150d90 0x7f43d014d290 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c1f0 tx=0x7f43bc005840 comp rx=0 tx=0).stop 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 shutdown_connections 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f43b406f400 0x7f43b4071910 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f43d01fa490 0x7f43d015f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43d0150d90 0x7f43d014d290 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 --2- 172.21.15.80:0/3357893084 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43bc004810 0x7f43d00d0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 >> 172.21.15.80:0/3357893084 conn(0x7f43d00b4160 msgr2=0x7f43d015d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 shutdown_connections 2024-04-25T13:41:01.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.155+0000 7f43d85c1640 1 -- 172.21.15.80:0/3357893084 wait complete. 2024-04-25T13:41:01.224 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-04-25T13:41:01.224 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2024-04-25T13:41:01.225 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:01.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.375+0000 7f7bb5cd0640 1 Processor -- start 2024-04-25T13:41:01.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.375+0000 7f7bb5cd0640 1 -- start start 2024-04-25T13:41:01.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bb5cd0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7bb00d2e80 0x7f7bb01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bb5cd0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb0161060 con 0x7f7bb00d4330 2024-04-25T13:41:01.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bb5cd0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bb0161340 con 0x7f7bb00d59b0 2024-04-25T13:41:01.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bb5cd0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7bb00d2680 con 0x7f7bb00d2e80 2024-04-25T13:41:01.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7bb00d4330 0x7f7bb00d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41754/0 (socket says 172.21.15.80:41754) 2024-04-25T13:41:01.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baffff640 1 -- 172.21.15.80:0/2391042263 learned_addr learned my addr 172.21.15.80:0/2391042263 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:01.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3388029597 0 0) 0x7f7bb0161060 con 0x7f7bb00d4330 2024-04-25T13:41:01.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baf7fe640 1 --2- 172.21.15.80:0/2391042263 >> v2:172.21.15.87:3300/0 conn(0x7f7bb00d2e80 0x7f7bb01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b98003530 con 0x7f7bb00d4330 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2836599984 0 0) 0x7f7b98003530 con 0x7f7bb00d4330 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 >> v2:172.21.15.87:3300/0 conn(0x7f7bb00d2e80 msgr2=0x7f7bb01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 --2- 172.21.15.80:0/2391042263 >> v2:172.21.15.87:3300/0 conn(0x7f7bb00d2e80 0x7f7bb01607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 >> v1:172.21.15.87:6789/0 conn(0x7f7bb00d59b0 legacy=0x7f7bb00d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb00d2960 con 0x7f7bb00d4330 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7ba8001a20 con 0x7f7bb00d4330 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 >> v1:172.21.15.80:6789/0 conn(0x7f7bb00d4330 legacy=0x7f7bb00d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b980039c0 0x7f7b98003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b98004810 0x7f7b98004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b98005260 0x7f7b9800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb00d2680 con 0x7f7b980039c0 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9800c1f0 con 0x7f7b98005260 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9800c3f0 con 0x7f7b98004810 2024-04-25T13:41:01.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baf7fe640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b980039c0 0x7f7b98003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baffff640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b98005260 0x7f7b9800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baeffd640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b98004810 0x7f7b98004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baf7fe640 1 -- 172.21.15.80:0/2391042263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b98004810 msgr2=0x7f7b98004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baf7fe640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b98004810 0x7f7b98004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baf7fe640 1 -- 172.21.15.80:0/2391042263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b98005260 msgr2=0x7f7b9800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baf7fe640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b98005260 0x7f7b9800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baf7fe640 1 -- 172.21.15.80:0/2391042263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b98003530 con 0x7f7b980039c0 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.379+0000 7f7baffff640 1 -- 172.21.15.80:0/2391042263 reap_dead start 2024-04-25T13:41:01.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7baf7fe640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b980039c0 0x7f7b98003e30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009ab0 tx=0x7f7ba001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1fe5b121c234cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:01.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00b2070 con 0x7f7b980039c0 2024-04-25T13:41:01.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba00a5640 con 0x7f7b980039c0 2024-04-25T13:41:01.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bae7fc640 1 -- 172.21.15.80:0/2391042263 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba00096b0 con 0x7f7b980039c0 2024-04-25T13:41:01.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2391042263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b980039c0 msgr2=0x7f7b98003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bb5cd0640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b980039c0 0x7f7b98003e30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0009ab0 tx=0x7f7ba001ca50 comp rx=0 tx=0).stop 2024-04-25T13:41:01.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2391042263 shutdown_connections 2024-04-25T13:41:01.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bb5cd0640 1 --2- 172.21.15.80:0/2391042263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b980039c0 0x7f7b98003e30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2391042263 >> 172.21.15.80:0/2391042263 conn(0x7f7bb00737a0 msgr2=0x7f7bb006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:01.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2391042263 shutdown_connections 2024-04-25T13:41:01.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.383+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2391042263 wait complete. 2024-04-25T13:41:01.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 Processor -- start 2024-04-25T13:41:01.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 -- start start 2024-04-25T13:41:01.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b980039c0 0x7f7bb00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bb014e300 0x7f7bb00cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bb00cb2f0 0x7f7bb00ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb015df60 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb00d34a0 con 0x7f7b980039c0 2024-04-25T13:41:01.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb015e0e0 con 0x7f7bb014e300 2024-04-25T13:41:01.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bb00cb2f0 0x7f7bb00ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bb00cb2f0 0x7f7bb00ccd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59254/0 (socket says 172.21.15.80:59254) 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 -- 172.21.15.80:0/2682723356 learned_addr learned my addr 172.21.15.80:0/2682723356 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baeffd640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bb014e300 0x7f7bb00cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baf7fe640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b980039c0 0x7f7bb00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 -- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bb014e300 msgr2=0x7f7bb00cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bb014e300 0x7f7bb00cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 -- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b980039c0 msgr2=0x7f7bb00cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b980039c0 0x7f7bb00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 -- 172.21.15.80:0/2682723356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb00cb700 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baeffd640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bb014e300 0x7f7bb00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7baffff640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bb00cb2f0 0x7f7bb00ccd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00d36f0 tx=0x7f7ba8007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7b97fff640 1 -- 172.21.15.80:0/2682723356 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba800aaa0 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7b97fff640 1 -- 172.21.15.80:0/2682723356 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ba8019070 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb00c9a20 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7b97fff640 1 -- 172.21.15.80:0/2682723356 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba80133f0 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb00c9fe0 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.387+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb006a3a0 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.391+0000 7f7b97fff640 1 -- 172.21.15.80:0/2682723356 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7ba8013590 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.391+0000 7f7b97fff640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b9c06f400 0x7f7b9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.391+0000 7f7b97fff640 1 -- 172.21.15.80:0/2682723356 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7ba808f7a0 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.391+0000 7f7baf7fe640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b9c06f400 0x7f7b9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.391+0000 7f7baf7fe640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b9c06f400 0x7f7b9c071910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00af000 tx=0x7f7ba0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:01.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.395+0000 7f7b97fff640 1 -- 172.21.15.80:0/2682723356 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7bb00cb700 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.555+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bb00bbb90 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.559+0000 7f7b97fff640 1 -- 172.21.15.80:0/2682723356 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f7ba805c690 con 0x7f7bb00cb2f0 2024-04-25T13:41:01.561 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:01.561 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.559+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b9c06f400 msgr2=0x7f7b9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.559+0000 7f7bb5cd0640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b9c06f400 0x7f7b9c071910 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7ba00af000 tx=0x7f7ba0008040 comp rx=0 tx=0).stop 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.559+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bb00cb2f0 msgr2=0x7f7bb00ccd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.559+0000 7f7bb5cd0640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bb00cb2f0 0x7f7bb00ccd50 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00d36f0 tx=0x7f7ba8007a90 comp rx=0 tx=0).stop 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.563+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 shutdown_connections 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.563+0000 7f7bb5cd0640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b9c06f400 0x7f7b9c071910 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.563+0000 7f7bb5cd0640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bb00cb2f0 0x7f7bb00ccd50 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.563+0000 7f7bb5cd0640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bb014e300 0x7f7bb00cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.563+0000 7f7bb5cd0640 1 --2- 172.21.15.80:0/2682723356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b980039c0 0x7f7bb00cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.563+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 >> 172.21.15.80:0/2682723356 conn(0x7f7bb00737a0 msgr2=0x7f7bb015e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:01.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.563+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 shutdown_connections 2024-04-25T13:41:01.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.563+0000 7f7bb5cd0640 1 -- 172.21.15.80:0/2682723356 wait complete. 2024-04-25T13:41:01.582 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:01.582 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:01.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.731+0000 7f1ce9471640 1 Processor -- start 2024-04-25T13:41:01.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.731+0000 7f1ce9471640 1 -- start start 2024-04-25T13:41:01.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.731+0000 7f1ce9471640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1ce406eb80 0x7f1ce4167660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.731+0000 7f1ce9471640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce40737f0 con 0x7f1ce4073ad0 2024-04-25T13:41:01.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.731+0000 7f1ce9471640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ce41604d0 con 0x7f1ce4073080 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.731+0000 7f1ce9471640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1ce40b53d0 con 0x7f1ce406eb80 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce27fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1ce4073ad0 0x7f1ce406e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41768/0 (socket says 172.21.15.80:41768) 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce27fc640 1 -- 172.21.15.80:0/4043250006 learned_addr learned my addr 172.21.15.80:0/4043250006 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 --2- 172.21.15.80:0/4043250006 >> v2:172.21.15.87:3300/0 conn(0x7f1ce406eb80 0x7f1ce4167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 22692123 0 0) 0x7f1ce40737f0 con 0x7f1ce4073ad0 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1cd0003530 con 0x7f1ce4073ad0 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3660675159 0 0) 0x7f1cd0003530 con 0x7f1ce4073ad0 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 >> v2:172.21.15.87:3300/0 conn(0x7f1ce406eb80 msgr2=0x7f1ce4167660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 --2- 172.21.15.80:0/4043250006 >> v2:172.21.15.87:3300/0 conn(0x7f1ce406eb80 0x7f1ce4167660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 >> v1:172.21.15.87:6789/0 conn(0x7f1ce4073080 legacy=0x7f1ce415c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce41607b0 con 0x7f1ce4073ad0 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1ccc001680 con 0x7f1ce4073ad0 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 >> v1:172.21.15.80:6789/0 conn(0x7f1ce4073ad0 legacy=0x7f1ce406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd00039c0 0x7f1cd0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd0004810 0x7f1cd0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cd0005260 0x7f1cd000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce40b53d0 con 0x7f1cd0005260 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd000c1f0 con 0x7f1cd0004810 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1cd000c3f0 con 0x7f1cd00039c0 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cd0005260 0x7f1cd000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce27fc640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd0004810 0x7f1cd0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce2ffd640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd00039c0 0x7f1cd0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 -- 172.21.15.80:0/4043250006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd00039c0 msgr2=0x7f1cd0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd00039c0 0x7f1cd0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 -- 172.21.15.80:0/4043250006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd0004810 msgr2=0x7f1cd0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cd0004810 0x7f1cd0004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 -- 172.21.15.80:0/4043250006 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd0003530 con 0x7f1cd0005260 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 -- 172.21.15.80:0/4043250006 reap_dead start 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce37fe640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cd0005260 0x7f1cd000b9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4160ad0 tx=0x7f1cdc01dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7071a204a6f9bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00ad20 con 0x7f1cd0005260 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cdc0a6a50 con 0x7f1cd0005260 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.735+0000 7f1ce1ffb640 1 -- 172.21.15.80:0/4043250006 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0a6e20 con 0x7f1cd0005260 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- 172.21.15.80:0/4043250006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cd0005260 msgr2=0x7f1cd000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cd0005260 0x7f1cd000b9d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4160ad0 tx=0x7f1cdc01dd60 comp rx=0 tx=0).stop 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- 172.21.15.80:0/4043250006 shutdown_connections 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 --2- 172.21.15.80:0/4043250006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cd0005260 0x7f1cd000b9d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- 172.21.15.80:0/4043250006 >> 172.21.15.80:0/4043250006 conn(0x7f1ce40b4160 msgr2=0x7f1ce40d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- 172.21.15.80:0/4043250006 shutdown_connections 2024-04-25T13:41:01.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- 172.21.15.80:0/4043250006 wait complete. 2024-04-25T13:41:01.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 Processor -- start 2024-04-25T13:41:01.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- start start 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd0005260 0x7f1ce414a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce4151c60 0x7f1ce4152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ce414fb50 0x7f1ce414ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4073490 con 0x7f1ce414fb50 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4160b20 con 0x7f1ce4151c60 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.739+0000 7f1ce9471640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce4073610 con 0x7f1cd0005260 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ce414fb50 0x7f1ce414ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ce414fb50 0x7f1ce414ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59274/0 (socket says 172.21.15.80:59274) 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 -- 172.21.15.80:0/2501002832 learned_addr learned my addr 172.21.15.80:0/2501002832 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce2ffd640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd0005260 0x7f1ce414a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce27fc640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce4151c60 0x7f1ce4152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 -- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd0005260 msgr2=0x7f1ce414a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd0005260 0x7f1ce414a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 -- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce4151c60 msgr2=0x7f1ce4152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce4151c60 0x7f1ce4152070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 -- 172.21.15.80:0/2501002832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce40cf320 con 0x7f1ce414fb50 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce27fc640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce4151c60 0x7f1ce4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce37fe640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ce414fb50 0x7f1ce414ff60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00a040 tx=0x7f1cdc00a860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1cdb7fe640 1 -- 172.21.15.80:0/2501002832 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00ad20 con 0x7f1ce414fb50 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1cdb7fe640 1 -- 172.21.15.80:0/2501002832 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1cdc008640 con 0x7f1ce414fb50 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce40cf600 con 0x7f1ce414fb50 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1cdb7fe640 1 -- 172.21.15.80:0/2501002832 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc00a2e0 con 0x7f1ce414fb50 2024-04-25T13:41:01.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce40cfbc0 con 0x7f1ce414fb50 2024-04-25T13:41:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce40cffa0 con 0x7f1ce414fb50 2024-04-25T13:41:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1cdb7fe640 1 -- 172.21.15.80:0/2501002832 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1cdc0bd060 con 0x7f1ce414fb50 2024-04-25T13:41:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1cdb7fe640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1cc406f400 0x7f1cc4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1cdb7fe640 1 -- 172.21.15.80:0/2501002832 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f1cdc129ad0 con 0x7f1ce414fb50 2024-04-25T13:41:01.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.743+0000 7f1ce2ffd640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1cc406f400 0x7f1cc4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:01.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.747+0000 7f1ce2ffd640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1cc406f400 0x7f1cc4071910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c10 tx=0x7f1cd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:01.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.747+0000 7f1cdb7fe640 1 -- 172.21.15.80:0/2501002832 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ce40cffa0 con 0x7f1ce414fb50 2024-04-25T13:41:01.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.907+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1ce40cb3c0 con 0x7f1ce414fb50 2024-04-25T13:41:01.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.907+0000 7f1cdb7fe640 1 -- 172.21.15.80:0/2501002832 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f1ce40cb3c0 con 0x7f1ce414fb50 2024-04-25T13:41:01.911 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:01.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1cc406f400 msgr2=0x7f1cc4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1cc406f400 0x7f1cc4071910 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4000c10 tx=0x7f1cd4009040 comp rx=0 tx=0).stop 2024-04-25T13:41:01.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ce414fb50 msgr2=0x7f1ce414ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:01.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ce414fb50 0x7f1ce414ff60 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc00a040 tx=0x7f1cdc00a860 comp rx=0 tx=0).stop 2024-04-25T13:41:01.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 shutdown_connections 2024-04-25T13:41:01.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1cc406f400 0x7f1cc4071910 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ce414fb50 0x7f1ce414ff60 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ce4151c60 0x7f1ce4152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 --2- 172.21.15.80:0/2501002832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cd0005260 0x7f1ce414a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:01.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 >> 172.21.15.80:0/2501002832 conn(0x7f1ce40b4160 msgr2=0x7f1ce406dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:01.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 shutdown_connections 2024-04-25T13:41:01.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:01.911+0000 7f1ce9471640 1 -- 172.21.15.80:0/2501002832 wait complete. 2024-04-25T13:41:01.931 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:02.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.079+0000 7f94ced64640 1 Processor -- start 2024-04-25T13:41:02.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94ced64640 1 -- start start 2024-04-25T13:41:02.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94ced64640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c8073080 0x7f94c8073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94ced64640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f94c806eb80 0x7f94c80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94ced64640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94c80737f0 con 0x7f94c815c710 2024-04-25T13:41:02.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94ced64640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f94c80b53d0 con 0x7f94c806eb80 2024-04-25T13:41:02.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94ced64640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f94c8160b20 con 0x7f94c8073080 2024-04-25T13:41:02.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94cd301640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f94c806eb80 0x7f94c80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94c7fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f94c815c710 0x7f94c806e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41778/0 (socket says 172.21.15.80:41778) 2024-04-25T13:41:02.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94c7fff640 1 -- 172.21.15.80:0/2770061388 learned_addr learned my addr 172.21.15.80:0/2770061388 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:02.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94ccb00640 1 --2- 172.21.15.80:0/2770061388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c8073080 0x7f94c8073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94cd301640 1 -- 172.21.15.80:0/2770061388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c8073080 msgr2=0x7f94c8073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:02.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94cd301640 1 --2- 172.21.15.80:0/2770061388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c8073080 0x7f94c8073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94cd301640 1 -- 172.21.15.80:0/2770061388 >> v1:172.21.15.80:6789/0 conn(0x7f94c815c710 legacy=0x7f94c806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94cd301640 1 -- 172.21.15.80:0/2770061388 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c81604d0 con 0x7f94c806eb80 2024-04-25T13:41:02.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.083+0000 7f94ccb00640 1 --2- 172.21.15.80:0/2770061388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c8073080 0x7f94c8073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:02.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94cd301640 1 --2- 172.21.15.80:0/2770061388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c806eb80 0x7f94c80ce630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f94c000aea0 tx=0x7f94c001d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c68f95175ecf39cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94c77fe640 1 -- 172.21.15.80:0/2770061388 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c00a6750 con 0x7f94c806eb80 2024-04-25T13:41:02.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94c77fe640 1 -- 172.21.15.80:0/2770061388 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c00a68f0 con 0x7f94c806eb80 2024-04-25T13:41:02.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94c77fe640 1 -- 172.21.15.80:0/2770061388 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c00a6cc0 con 0x7f94c806eb80 2024-04-25T13:41:02.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94ced64640 1 -- 172.21.15.80:0/2770061388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c806eb80 msgr2=0x7f94c80ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94ced64640 1 --2- 172.21.15.80:0/2770061388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c806eb80 0x7f94c80ce630 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f94c000aea0 tx=0x7f94c001d9e0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94ced64640 1 -- 172.21.15.80:0/2770061388 shutdown_connections 2024-04-25T13:41:02.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94ced64640 1 --2- 172.21.15.80:0/2770061388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c806eb80 0x7f94c80ce630 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94ced64640 1 --2- 172.21.15.80:0/2770061388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c8073080 0x7f94c8073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94ced64640 1 -- 172.21.15.80:0/2770061388 >> 172.21.15.80:0/2770061388 conn(0x7f94c80b4160 msgr2=0x7f94c80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:02.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94ced64640 1 -- 172.21.15.80:0/2770061388 shutdown_connections 2024-04-25T13:41:02.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.087+0000 7f94ced64640 1 -- 172.21.15.80:0/2770061388 wait complete. 2024-04-25T13:41:02.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 Processor -- start 2024-04-25T13:41:02.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 -- start start 2024-04-25T13:41:02.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c8073080 0x7f94c81516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c815c710 0x7f94c8151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c813c620 0x7f94c8152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f94c80cecf0 con 0x7f94c813c620 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f94c80ceb70 con 0x7f94c8073080 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f94c80cee70 con 0x7f94c815c710 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c813c620 0x7f94c8152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c813c620 0x7f94c8152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59288/0 (socket says 172.21.15.80:59288) 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 -- 172.21.15.80:0/672044232 learned_addr learned my addr 172.21.15.80:0/672044232 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ccb00640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c8073080 0x7f94c81516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94c7fff640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c815c710 0x7f94c8151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 -- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c815c710 msgr2=0x7f94c8151c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c815c710 0x7f94c8151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 -- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c8073080 msgr2=0x7f94c81516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c8073080 0x7f94c81516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 -- 172.21.15.80:0/672044232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c8158710 con 0x7f94c813c620 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ccb00640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c8073080 0x7f94c81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94cd301640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c813c620 0x7f94c8152150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f94c001b610 tx=0x7f94c001a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94c57fa640 1 -- 172.21.15.80:0/672044232 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c00b7640 con 0x7f94c813c620 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94c57fa640 1 -- 172.21.15.80:0/672044232 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94c00bd070 con 0x7f94c813c620 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c81589f0 con 0x7f94c813c620 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94c57fa640 1 -- 172.21.15.80:0/672044232 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c00b7a60 con 0x7f94c813c620 2024-04-25T13:41:02.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c8150ed0 con 0x7f94c813c620 2024-04-25T13:41:02.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.091+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c806a3a0 con 0x7f94c813c620 2024-04-25T13:41:02.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.095+0000 7f94c57fa640 1 -- 172.21.15.80:0/672044232 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f94c00b7c00 con 0x7f94c813c620 2024-04-25T13:41:02.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.095+0000 7f94c57fa640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f94b406f430 0x7f94b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.095+0000 7f94c57fa640 1 -- 172.21.15.80:0/672044232 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f94c012aaf0 con 0x7f94c813c620 2024-04-25T13:41:02.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.095+0000 7f94ccb00640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f94b406f430 0x7f94b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.095+0000 7f94ccb00640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f94b406f430 0x7f94b4071940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94c8160290 tx=0x7f94b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.099+0000 7f94c57fa640 1 -- 172.21.15.80:0/672044232 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f94c0017660 con 0x7f94c813c620 2024-04-25T13:41:02.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.263+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94c80ce1b0 con 0x7f94c813c620 2024-04-25T13:41:02.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.263+0000 7f94c57fa640 1 -- 172.21.15.80:0/672044232 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f94c00f7b90 con 0x7f94c813c620 2024-04-25T13:41:02.265 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f94b406f430 msgr2=0x7f94b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f94b406f430 0x7f94b4071940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94c8160290 tx=0x7f94b8008040 comp rx=0 tx=0).stop 2024-04-25T13:41:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c813c620 msgr2=0x7f94c8152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c813c620 0x7f94c8152150 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f94c001b610 tx=0x7f94c001a780 comp rx=0 tx=0).stop 2024-04-25T13:41:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 shutdown_connections 2024-04-25T13:41:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f94b406f430 0x7f94b4071940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f94c813c620 0x7f94c8152150 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f94c815c710 0x7f94c8151c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 --2- 172.21.15.80:0/672044232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f94c8073080 0x7f94c81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 >> 172.21.15.80:0/672044232 conn(0x7f94c80b4160 msgr2=0x7f94c806e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 shutdown_connections 2024-04-25T13:41:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.267+0000 7f94ced64640 1 -- 172.21.15.80:0/672044232 wait complete. 2024-04-25T13:41:02.286 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:02.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff6094af640 1 Processor -- start 2024-04-25T13:41:02.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff6094af640 1 -- start start 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff6094af640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6040d2ca0 0x7ff6040b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff6094af640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6040b92a0 con 0x7ff6040d3ed0 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff6094af640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6040d4c10 con 0x7ff6040b9ca0 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff6094af640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff6040d4ef0 con 0x7ff6040d2ca0 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff602ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff6040d3ed0 0x7ff6040d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41788/0 (socket says 172.21.15.80:41788) 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff602ffd640 1 -- 172.21.15.80:0/1613828594 learned_addr learned my addr 172.21.15.80:0/1613828594 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff6027fc640 1 --2- 172.21.15.80:0/1613828594 >> v2:172.21.15.87:3300/0 conn(0x7ff6040d2ca0 0x7ff6040b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1084051810 0 0) 0x7ff5f4001120 con 0x7ff6040d3ed0 2024-04-25T13:41:02.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5ec003530 con 0x7ff6040d3ed0 2024-04-25T13:41:02.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2719874281 0 0) 0x7ff5ec003530 con 0x7ff6040d3ed0 2024-04-25T13:41:02.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 >> v2:172.21.15.87:3300/0 conn(0x7ff6040d2ca0 msgr2=0x7ff6040b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:02.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 --2- 172.21.15.80:0/1613828594 >> v2:172.21.15.87:3300/0 conn(0x7ff6040d2ca0 0x7ff6040b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 >> v1:172.21.15.87:6789/0 conn(0x7ff6040b9ca0 legacy=0x7ff6040ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6040d51d0 con 0x7ff6040d3ed0 2024-04-25T13:41:02.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff5f4001850 con 0x7ff6040d3ed0 2024-04-25T13:41:02.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 >> v1:172.21.15.80:6789/0 conn(0x7ff6040d3ed0 legacy=0x7ff6040d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5ec004810 0x7ff5ec004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5ec005360 0x7ff5ec00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff6040d4ef0 con 0x7ff5ec004810 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ec00c290 con 0x7ff5ec0039c0 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ec00c490 con 0x7ff5ec005360 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.439+0000 7ff6027fc640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5ec004810 0x7ff5ec004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff602ffd640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6037fe640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5ec005360 0x7ff5ec00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6027fc640 1 -- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5ec005360 msgr2=0x7ff5ec00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6027fc640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5ec005360 0x7ff5ec00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6027fc640 1 -- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec0039c0 msgr2=0x7ff5ec003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6027fc640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6027fc640 1 -- 172.21.15.80:0/1613828594 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ec003530 con 0x7ff5ec004810 2024-04-25T13:41:02.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff602ffd640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec0039c0 0x7ff5ec003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:02.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6037fe640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5ec005360 0x7ff5ec00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:02.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6037fe640 1 -- 172.21.15.80:0/1613828594 reap_dead start 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6027fc640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5ec004810 0x7ff5ec004c20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009ae0 tx=0x7ff5f801cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75666a77584d04a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80b2070 con 0x7ff5ec004810 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5f80a55f0 con 0x7ff5ec004810 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff601ffb640 1 -- 172.21.15.80:0/1613828594 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5f80096b0 con 0x7ff5ec004810 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6094af640 1 -- 172.21.15.80:0/1613828594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5ec004810 msgr2=0x7ff5ec004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6094af640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5ec004810 0x7ff5ec004c20 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8009ae0 tx=0x7ff5f801cb00 comp rx=0 tx=0).stop 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6094af640 1 -- 172.21.15.80:0/1613828594 shutdown_connections 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6094af640 1 --2- 172.21.15.80:0/1613828594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5ec004810 0x7ff5ec004c20 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6094af640 1 -- 172.21.15.80:0/1613828594 >> 172.21.15.80:0/1613828594 conn(0x7ff6040737b0 msgr2=0x7ff60406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6094af640 1 -- 172.21.15.80:0/1613828594 shutdown_connections 2024-04-25T13:41:02.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.443+0000 7ff6094af640 1 -- 172.21.15.80:0/1613828594 wait complete. 2024-04-25T13:41:02.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 Processor -- start 2024-04-25T13:41:02.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 -- start start 2024-04-25T13:41:02.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec004810 0x7ff6041fa0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6040c7db0 0x7ff6041edfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6041ee510 0x7ff6041ec1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff6040ba8c0 con 0x7ff6041ee510 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff6040baa40 con 0x7ff5ec004810 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff604160620 con 0x7ff6040c7db0 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6041ee510 0x7ff6041ec1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6041ee510 0x7ff6041ec1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59302/0 (socket says 172.21.15.80:59302) 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 -- 172.21.15.80:0/2632206838 learned_addr learned my addr 172.21.15.80:0/2632206838 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6027fc640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6040c7db0 0x7ff6041edfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff602ffd640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec004810 0x7ff6041fa0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 -- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6040c7db0 msgr2=0x7ff6041edfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6040c7db0 0x7ff6041edfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 -- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec004810 msgr2=0x7ff6041fa0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec004810 0x7ff6041fa0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 -- 172.21.15.80:0/2632206838 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6041ecf30 con 0x7ff6041ee510 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff602ffd640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec004810 0x7ff6041fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6027fc640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6040c7db0 0x7ff6041edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:02.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6037fe640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6041ee510 0x7ff6041ec1e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff60408ebf0 tx=0x7ff5fc00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff5eb7fe640 1 -- 172.21.15.80:0/2632206838 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc018070 con 0x7ff6041ee510 2024-04-25T13:41:02.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff5eb7fe640 1 -- 172.21.15.80:0/2632206838 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5fc003ba0 con 0x7ff6041ee510 2024-04-25T13:41:02.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6041ed210 con 0x7ff6041ee510 2024-04-25T13:41:02.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff5eb7fe640 1 -- 172.21.15.80:0/2632206838 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc005070 con 0x7ff6041ee510 2024-04-25T13:41:02.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.447+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6041ed7d0 con 0x7ff6041ee510 2024-04-25T13:41:02.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.451+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff60406a3a0 con 0x7ff6041ee510 2024-04-25T13:41:02.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.451+0000 7ff5eb7fe640 1 -- 172.21.15.80:0/2632206838 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff5fc01b070 con 0x7ff6041ee510 2024-04-25T13:41:02.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.451+0000 7ff5eb7fe640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5f006f400 0x7ff5f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.451+0000 7ff5eb7fe640 1 -- 172.21.15.80:0/2632206838 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff5fc08ec80 con 0x7ff6041ee510 2024-04-25T13:41:02.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.451+0000 7ff602ffd640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5f006f400 0x7ff5f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.451+0000 7ff602ffd640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5f006f400 0x7ff5f0071910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4001d40 tx=0x7ff5f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.455+0000 7ff5eb7fe640 1 -- 172.21.15.80:0/2632206838 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff5fc022020 con 0x7ff6041ee510 2024-04-25T13:41:02.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.615+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff6040cc410 con 0x7ff6041ee510 2024-04-25T13:41:02.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.615+0000 7ff5eb7fe640 1 -- 172.21.15.80:0/2632206838 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff5fc05bce0 con 0x7ff6041ee510 2024-04-25T13:41:02.618 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:02.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5f006f400 msgr2=0x7ff5f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5f006f400 0x7ff5f0071910 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4001d40 tx=0x7ff5f4009040 comp rx=0 tx=0).stop 2024-04-25T13:41:02.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6041ee510 msgr2=0x7ff6041ec1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6041ee510 0x7ff6041ec1e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff60408ebf0 tx=0x7ff5fc00db80 comp rx=0 tx=0).stop 2024-04-25T13:41:02.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 shutdown_connections 2024-04-25T13:41:02.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5f006f400 0x7ff5f0071910 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6041ee510 0x7ff6041ec1e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6040c7db0 0x7ff6041edfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 --2- 172.21.15.80:0/2632206838 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5ec004810 0x7ff6041fa0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 >> 172.21.15.80:0/2632206838 conn(0x7ff6040737b0 msgr2=0x7ff60415dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:02.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 shutdown_connections 2024-04-25T13:41:02.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.619+0000 7ff6094af640 1 -- 172.21.15.80:0/2632206838 wait complete. 2024-04-25T13:41:02.641 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-04-25T13:41:02.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.783+0000 7f006280c640 1 Processor -- start 2024-04-25T13:41:02.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.783+0000 7f006280c640 1 -- start start 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f006280c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c161f90 0x7f005c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f006280c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f005c15ec40 0x7f005c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f006280c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f005c0d0d70 con 0x7f005c0d0510 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f006280c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f005c0d1050 con 0x7f005c15ec40 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f006280c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f005c06dca0 con 0x7f005c161f90 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f0060da9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f005c15ec40 0x7f005c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f005b7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f005c0d0510 0x7f005c15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41790/0 (socket says 172.21.15.80:41790) 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f005b7fe640 1 -- 172.21.15.80:0/4007217900 learned_addr learned my addr 172.21.15.80:0/4007217900 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:02.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f005bfff640 1 --2- 172.21.15.80:0/4007217900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c161f90 0x7f005c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f0060da9640 1 -- 172.21.15.80:0/4007217900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c161f90 msgr2=0x7f005c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f0060da9640 1 --2- 172.21.15.80:0/4007217900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c161f90 0x7f005c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f0060da9640 1 -- 172.21.15.80:0/4007217900 >> v1:172.21.15.80:6789/0 conn(0x7f005c0d0510 legacy=0x7f005c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f0060da9640 1 -- 172.21.15.80:0/4007217900 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c06df80 con 0x7f005c15ec40 2024-04-25T13:41:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.787+0000 7f005affd640 1 -- 172.21.15.80:0/4007217900 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 809650320 0 0) 0x7f005c0d0d70 con 0x7f005c0d0510 2024-04-25T13:41:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f0060da9640 1 --2- 172.21.15.80:0/4007217900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c15ec40 0x7f005c15f050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0054009d90 tx=0x7f005401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6696de53db62b01c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f005affd640 1 -- 172.21.15.80:0/4007217900 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00540b2070 con 0x7f005c15ec40 2024-04-25T13:41:02.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f005affd640 1 -- 172.21.15.80:0/4007217900 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00540a5890 con 0x7f005c15ec40 2024-04-25T13:41:02.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f005affd640 1 -- 172.21.15.80:0/4007217900 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00540a5c60 con 0x7f005c15ec40 2024-04-25T13:41:02.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 -- 172.21.15.80:0/4007217900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c15ec40 msgr2=0x7f005c15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 --2- 172.21.15.80:0/4007217900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c15ec40 0x7f005c15f050 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0054009d90 tx=0x7f005401c8a0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 -- 172.21.15.80:0/4007217900 shutdown_connections 2024-04-25T13:41:02.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 --2- 172.21.15.80:0/4007217900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c15ec40 0x7f005c15f050 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 --2- 172.21.15.80:0/4007217900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c161f90 0x7f005c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 -- 172.21.15.80:0/4007217900 >> 172.21.15.80:0/4007217900 conn(0x7f005c06d080 msgr2=0x7f005c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:02.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 -- 172.21.15.80:0/4007217900 shutdown_connections 2024-04-25T13:41:02.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 -- 172.21.15.80:0/4007217900 wait complete. 2024-04-25T13:41:02.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 Processor -- start 2024-04-25T13:41:02.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.791+0000 7f006280c640 1 -- start start 2024-04-25T13:41:02.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c0d0510 0x7f005c072b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f005c15ec40 0x7f005c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c161f90 0x7f005c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f005c06f630 con 0x7f005c161f90 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f005c06f4b0 con 0x7f005c15ec40 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f005c0d0920 con 0x7f005c0d0510 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c161f90 0x7f005c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c161f90 0x7f005c0afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59320/0 (socket says 172.21.15.80:59320) 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 -- 172.21.15.80:0/63256883 learned_addr learned my addr 172.21.15.80:0/63256883 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f005b7fe640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f005c15ec40 0x7f005c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f005bfff640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c0d0510 0x7f005c072b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 -- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c0d0510 msgr2=0x7f005c072b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c0d0510 0x7f005c072b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 -- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f005c15ec40 msgr2=0x7f005c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f005c15ec40 0x7f005c0af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:02.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 -- 172.21.15.80:0/63256883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c073430 con 0x7f005c161f90 2024-04-25T13:41:02.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f005b7fe640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f005c15ec40 0x7f005c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:02.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0060da9640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c161f90 0x7f005c0afdf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f00540b0040 tx=0x7f00540196a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0058ff9640 1 -- 172.21.15.80:0/63256883 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00540b2040 con 0x7f005c161f90 2024-04-25T13:41:02.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0058ff9640 1 -- 172.21.15.80:0/63256883 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00540a5690 con 0x7f005c161f90 2024-04-25T13:41:02.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c073710 con 0x7f005c161f90 2024-04-25T13:41:02.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0058ff9640 1 -- 172.21.15.80:0/63256883 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00540b8670 con 0x7f005c161f90 2024-04-25T13:41:02.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c073cd0 con 0x7f005c161f90 2024-04-25T13:41:02.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005c0731a0 con 0x7f005c161f90 2024-04-25T13:41:02.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0058ff9640 1 -- 172.21.15.80:0/63256883 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f00540c1020 con 0x7f005c161f90 2024-04-25T13:41:02.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0058ff9640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f004806f400 0x7f0048071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:02.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.795+0000 7f0058ff9640 1 -- 172.21.15.80:0/63256883 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0054129c50 con 0x7f005c161f90 2024-04-25T13:41:02.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.799+0000 7f005bfff640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f004806f400 0x7f0048071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:02.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.799+0000 7f005bfff640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f004806f400 0x7f0048071910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f005c06fa20 tx=0x7f004c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:02.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:02.803+0000 7f0058ff9640 1 -- 172.21.15.80:0/63256883 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f005412f020 con 0x7f005c161f90 2024-04-25T13:41:03.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.003+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) -- 0x7f005c06a3a0 con 0x7f005c161f90 2024-04-25T13:41:03.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.347+0000 7f0058ff9640 1 -- 172.21.15.80:0/63256883 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v47) ==== 133+0+0 (secure 0 0 0) 0x7f0054016580 con 0x7f005c161f90 2024-04-25T13:41:03.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f004806f400 msgr2=0x7f0048071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f004806f400 0x7f0048071910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f005c06fa20 tx=0x7f004c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:03.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c161f90 msgr2=0x7f005c0afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c161f90 0x7f005c0afdf0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f00540b0040 tx=0x7f00540196a0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 shutdown_connections 2024-04-25T13:41:03.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f004806f400 0x7f0048071910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f005c161f90 0x7f005c0afdf0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f005c15ec40 0x7f005c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 --2- 172.21.15.80:0/63256883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f005c0d0510 0x7f005c072b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 >> 172.21.15.80:0/63256883 conn(0x7f005c06d080 msgr2=0x7f005c0b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:03.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 shutdown_connections 2024-04-25T13:41:03.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.351+0000 7f006280c640 1 -- 172.21.15.80:0/63256883 wait complete. 2024-04-25T13:41:03.377 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-25T13:41:03.540 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_objectstore":"memstore"} 2024-04-25T13:41:03.541 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-25T13:41:03.541 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:41:03.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d2de8640 1 Processor -- start 2024-04-25T13:41:03.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d2de8640 1 -- start start 2024-04-25T13:41:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d2de8640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f11cc1574c0 0x7f11cc073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d2de8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc06fbf0 con 0x7f11cc161170 2024-04-25T13:41:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d2de8640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11cc0b0a20 con 0x7f11cc071e60 2024-04-25T13:41:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d2de8640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f11cc0b1350 con 0x7f11cc1574c0 2024-04-25T13:41:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cbfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f11cc161170 0x7f11cc0b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41806/0 (socket says 172.21.15.80:41806) 2024-04-25T13:41:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cbfff640 1 -- 172.21.15.80:0/962231557 learned_addr learned my addr 172.21.15.80:0/962231557 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d1385640 1 --2- 172.21.15.80:0/962231557 >> v2:172.21.15.87:3300/0 conn(0x7f11cc1574c0 0x7f11cc073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 551142383 0 0) 0x7f11cc06fbf0 con 0x7f11cc161170 2024-04-25T13:41:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f11b4005740 con 0x7f11cc161170 2024-04-25T13:41:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1259815717 0 0) 0x7f11b4005740 con 0x7f11cc161170 2024-04-25T13:41:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 >> v2:172.21.15.87:3300/0 conn(0x7f11cc1574c0 msgr2=0x7f11cc073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 --2- 172.21.15.80:0/962231557 >> v2:172.21.15.87:3300/0 conn(0x7f11cc1574c0 0x7f11cc073c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 >> v1:172.21.15.87:6789/0 conn(0x7f11cc071e60 legacy=0x7f11cc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc0b0d00 con 0x7f11cc161170 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f11c0001680 con 0x7f11cc161170 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 >> v1:172.21.15.80:6789/0 conn(0x7f11cc161170 legacy=0x7f11cc0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b4005bd0 0x7f11b4006060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11b4006a20 0x7f11b4006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4007490 0x7f11b400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0b1350 con 0x7f11b4005bd0 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d0b84640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b4005bd0 0x7f11b4006060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f11b400e3b0 con 0x7f11b4007490 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f11b400e5b0 con 0x7f11b4006a20 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11cbfff640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11b4006a20 0x7f11b4006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d0b84640 1 -- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11b4006a20 msgr2=0x7f11b4006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d1385640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4007490 0x7f11b400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d0b84640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11b4006a20 0x7f11b4006e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d0b84640 1 -- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4007490 msgr2=0x7f11b400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:03.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d0b84640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4007490 0x7f11b400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d0b84640 1 -- 172.21.15.80:0/962231557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b4005740 con 0x7f11b4005bd0 2024-04-25T13:41:03.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d1385640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4007490 0x7f11b400dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:03.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.691+0000 7f11d1385640 1 -- 172.21.15.80:0/962231557 reap_dead start 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11d0b84640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b4005bd0 0x7f11b4006060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f11bc001780 tx=0x7f11bc01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c352e1fbe0ec90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0013a0 con 0x7f11b4005bd0 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11bc0052f0 con 0x7f11b4005bd0 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11cb7fe640 1 -- 172.21.15.80:0/962231557 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11bc003840 con 0x7f11b4005bd0 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11d2de8640 1 -- 172.21.15.80:0/962231557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b4005bd0 msgr2=0x7f11b4006060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11d2de8640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b4005bd0 0x7f11b4006060 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f11bc001780 tx=0x7f11bc01c400 comp rx=0 tx=0).stop 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11d2de8640 1 -- 172.21.15.80:0/962231557 shutdown_connections 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11d2de8640 1 --2- 172.21.15.80:0/962231557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11b4005bd0 0x7f11b4006060 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11d2de8640 1 -- 172.21.15.80:0/962231557 >> 172.21.15.80:0/962231557 conn(0x7f11cc0b4160 msgr2=0x7f11cc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11d2de8640 1 -- 172.21.15.80:0/962231557 shutdown_connections 2024-04-25T13:41:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.695+0000 7f11d2de8640 1 -- 172.21.15.80:0/962231557 wait complete. 2024-04-25T13:41:03.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d2de8640 1 Processor -- start 2024-04-25T13:41:03.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d2de8640 1 -- start start 2024-04-25T13:41:03.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d2de8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4005bd0 0x7f11cc154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d2de8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11cc152370 0x7f11cc1e8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d2de8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11cc1e8dd0 0x7f11cc1f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d2de8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc0b4ba0 con 0x7f11cc152370 2024-04-25T13:41:03.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d2de8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f11cc074050 con 0x7f11b4005bd0 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d2de8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f11cc0b4d20 con 0x7f11cc1e8dd0 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11cbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11cc152370 0x7f11cc1e8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11cbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11cc152370 0x7f11cc1e8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59340/0 (socket says 172.21.15.80:59340) 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11cbfff640 1 -- 172.21.15.80:0/4026933148 learned_addr learned my addr 172.21.15.80:0/4026933148 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d1385640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11cc1e8dd0 0x7f11cc1f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d0b84640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4005bd0 0x7f11cc154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11cbfff640 1 -- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11cc1e8dd0 msgr2=0x7f11cc1f8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11cbfff640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11cc1e8dd0 0x7f11cc1f8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11cbfff640 1 -- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4005bd0 msgr2=0x7f11cc154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11cbfff640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4005bd0 0x7f11cc154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11cbfff640 1 -- 172.21.15.80:0/4026933148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11cc1f9d90 con 0x7f11cc152370 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d1385640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11cc1e8dd0 0x7f11cc1f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.699+0000 7f11d0b84640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4005bd0 0x7f11cc154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11cbfff640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11cc152370 0x7f11cc1e8890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f11c000d770 tx=0x7f11c000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11c97fa640 1 -- 172.21.15.80:0/4026933148 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0019070 con 0x7f11cc152370 2024-04-25T13:41:03.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11c97fa640 1 -- 172.21.15.80:0/4026933148 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11c00074e0 con 0x7f11cc152370 2024-04-25T13:41:03.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11cc1f9ab0 con 0x7f11cc152370 2024-04-25T13:41:03.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11c97fa640 1 -- 172.21.15.80:0/4026933148 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c0004a10 con 0x7f11cc152370 2024-04-25T13:41:03.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11cc154330 con 0x7f11cc152370 2024-04-25T13:41:03.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11cc1f9820 con 0x7f11cc152370 2024-04-25T13:41:03.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11c97fa640 1 -- 172.21.15.80:0/4026933148 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f11c001c070 con 0x7f11cc152370 2024-04-25T13:41:03.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11c97fa640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f11b806f400 0x7f11b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:03.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11c97fa640 1 -- 172.21.15.80:0/4026933148 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f11c008ec50 con 0x7f11cc152370 2024-04-25T13:41:03.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11d0b84640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f11b806f400 0x7f11b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:03.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.703+0000 7f11d0b84640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f11b806f400 0x7f11b8071910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f11bc001750 tx=0x7f11bc0011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:03.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.707+0000 7f11c97fa640 1 -- 172.21.15.80:0/4026933148 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f11c0094020 con 0x7f11cc152370 2024-04-25T13:41:03.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.871+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f11cc06a3a0 con 0x7f11cc152370 2024-04-25T13:41:03.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.875+0000 7f11c97fa640 1 -- 172.21.15.80:0/4026933148 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) ==== 122+0+0 (secure 0 0 0) 0x7f11c005bb40 con 0x7f11cc152370 2024-04-25T13:41:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f11b806f400 msgr2=0x7f11b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f11b806f400 0x7f11b8071910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f11bc001750 tx=0x7f11bc0011a0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11cc152370 msgr2=0x7f11cc1e8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11cc152370 0x7f11cc1e8890 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f11c000d770 tx=0x7f11c000dc30 comp rx=0 tx=0).stop 2024-04-25T13:41:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 shutdown_connections 2024-04-25T13:41:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f11b806f400 0x7f11b8071910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f11cc1e8dd0 0x7f11cc1f8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11cc152370 0x7f11cc1e8890 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 --2- 172.21.15.80:0/4026933148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11b4005bd0 0x7f11cc154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:03.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 >> 172.21.15.80:0/4026933148 conn(0x7f11cc0b4160 msgr2=0x7f11cc15e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:03.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 shutdown_connections 2024-04-25T13:41:03.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:03.879+0000 7f11d2de8640 1 -- 172.21.15.80:0/4026933148 wait complete. 2024-04-25T13:41:03.899 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:41:03.899 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:41:03.940 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:41:03.996 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:41:03.996 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:03.996 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:04.002 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:04.002 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:04.047 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:04.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff9718e2640 1 Processor -- start 2024-04-25T13:41:04.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff9718e2640 1 -- start start 2024-04-25T13:41:04.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff9718e2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c06f340 0x7ff96c06f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff9718e2640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96c0cd6c0 con 0x7ff96c0cc720 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff9718e2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff96c0729e0 con 0x7ff96c0745a0 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff9718e2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c15ea50 con 0x7ff96c06f340 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff96affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff96c0cc720 0x7ff96c073e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41808/0 (socket says 172.21.15.80:41808) 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff96affd640 1 -- 172.21.15.80:0/320135044 learned_addr learned my addr 172.21.15.80:0/320135044 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff96b7fe640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c06f340 0x7ff96c06f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1009102568 0 0) 0x7ff96c0cd6c0 con 0x7ff96c0cc720 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff958003530 con 0x7ff96c0cc720 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1671792789 0 0) 0x7ff958003530 con 0x7ff96c0cc720 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c06f340 msgr2=0x7ff96c06f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c06f340 0x7ff96c06f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 >> v1:172.21.15.87:6789/0 conn(0x7ff96c0745a0 legacy=0x7ff96c06ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c0cce20 con 0x7ff96c0cc720 2024-04-25T13:41:04.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff964001a20 con 0x7ff96c0cc720 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 >> v1:172.21.15.80:6789/0 conn(0x7ff96c0cc720 legacy=0x7ff96c073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9580039f0 0x7ff958003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c06f340 0x7ff958004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff958004d50 0x7ff958005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c15ea50 con 0x7ff9580039f0 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff95800bb90 con 0x7ff958004d50 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.243+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff95800bd90 con 0x7ff96c06f340 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96affd640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9580039f0 0x7ff958003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96a7fc640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c06f340 0x7ff958004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96b7fe640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff958004d50 0x7ff958005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96affd640 1 -- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c06f340 msgr2=0x7ff958004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96affd640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c06f340 0x7ff958004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96affd640 1 -- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff958004d50 msgr2=0x7ff958005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:04.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96affd640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff958004d50 0x7ff958005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96affd640 1 -- 172.21.15.80:0/320135044 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff958003530 con 0x7ff9580039f0 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96affd640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9580039f0 0x7ff958003e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff96c06df30 tx=0x7ff96401d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a23404ed81ac95d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96400a4a0 con 0x7ff9580039f0 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9640ad630 con 0x7ff9580039f0 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff969ffb640 1 -- 172.21.15.80:0/320135044 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c06ff60 con 0x7ff9580039f0 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff9718e2640 1 -- 172.21.15.80:0/320135044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9580039f0 msgr2=0x7ff958003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff9718e2640 1 --2- 172.21.15.80:0/320135044 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9580039f0 0x7ff958003e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff96c06df30 tx=0x7ff96401d300 comp rx=0 tx=0).stop 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff96b7fe640 1 -- 172.21.15.80:0/320135044 reap_dead start 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff9718e2640 1 -- 172.21.15.80:0/320135044 shutdown_connections 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff9718e2640 1 -- 172.21.15.80:0/320135044 >> 172.21.15.80:0/320135044 conn(0x7ff96c0d20d0 msgr2=0x7ff96c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff9718e2640 1 -- 172.21.15.80:0/320135044 shutdown_connections 2024-04-25T13:41:04.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.247+0000 7ff9718e2640 1 -- 172.21.15.80:0/320135044 wait complete. 2024-04-25T13:41:04.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 Processor -- start 2024-04-25T13:41:04.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 -- start start 2024-04-25T13:41:04.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9580039f0 0x7ff96c15d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c15df20 0x7ff96c09dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff96c09e100 0x7ff96c09e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c15f490 con 0x7ff96c09e100 2024-04-25T13:41:04.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff96c0ccb30 con 0x7ff9580039f0 2024-04-25T13:41:04.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff96c15f610 con 0x7ff96c15df20 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff96c09e100 0x7ff96c09e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff96c09e100 0x7ff96c09e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59360/0 (socket says 172.21.15.80:59360) 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 -- 172.21.15.80:0/2721062016 learned_addr learned my addr 172.21.15.80:0/2721062016 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96affd640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9580039f0 0x7ff96c15d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96a7fc640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c15df20 0x7ff96c09dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 -- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c15df20 msgr2=0x7ff96c09dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c15df20 0x7ff96c09dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 -- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9580039f0 msgr2=0x7ff96c15d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9580039f0 0x7ff96c15d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 -- 172.21.15.80:0/2721062016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff96c15e330 con 0x7ff96c09e100 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96affd640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9580039f0 0x7ff96c15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96a7fc640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c15df20 0x7ff96c09dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff96b7fe640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff96c09e100 0x7ff96c09e510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff96c06e050 tx=0x7ff95c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff963fff640 1 -- 172.21.15.80:0/2721062016 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c003620 con 0x7ff96c09e100 2024-04-25T13:41:04.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff963fff640 1 -- 172.21.15.80:0/2721062016 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff95c01b070 con 0x7ff96c09e100 2024-04-25T13:41:04.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff96c0c5e10 con 0x7ff96c09e100 2024-04-25T13:41:04.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff963fff640 1 -- 172.21.15.80:0/2721062016 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff95c003a40 con 0x7ff96c09e100 2024-04-25T13:41:04.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.251+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff96c0b5c20 con 0x7ff96c09e100 2024-04-25T13:41:04.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.255+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff96c0c5b80 con 0x7ff96c09e100 2024-04-25T13:41:04.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.255+0000 7ff963fff640 1 -- 172.21.15.80:0/2721062016 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff95c003be0 con 0x7ff96c09e100 2024-04-25T13:41:04.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.255+0000 7ff963fff640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff94c06f430 0x7ff94c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.255+0000 7ff963fff640 1 -- 172.21.15.80:0/2721062016 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff95c08f610 con 0x7ff96c09e100 2024-04-25T13:41:04.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.255+0000 7ff96affd640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff94c06f430 0x7ff94c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.255+0000 7ff96affd640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff94c06f430 0x7ff94c071940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9640ad000 tx=0x7ff9640af040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.259+0000 7ff963fff640 1 -- 172.21.15.80:0/2721062016 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff95c024020 con 0x7ff96c09e100 2024-04-25T13:41:04.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.419+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff96c06a3a0 con 0x7ff96c09e100 2024-04-25T13:41:04.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff963fff640 1 -- 172.21.15.80:0/2721062016 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7ff96c15e330 con 0x7ff96c09e100 2024-04-25T13:41:04.424 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:04.425 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff94c06f430 msgr2=0x7ff94c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff94c06f430 0x7ff94c071940 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff9640ad000 tx=0x7ff9640af040 comp rx=0 tx=0).stop 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff96c09e100 msgr2=0x7ff96c09e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff96c09e100 0x7ff96c09e510 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff96c06e050 tx=0x7ff95c011920 comp rx=0 tx=0).stop 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 shutdown_connections 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff94c06f430 0x7ff94c071940 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff96c09e100 0x7ff96c09e510 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff96c15df20 0x7ff96c09dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 --2- 172.21.15.80:0/2721062016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9580039f0 0x7ff96c15d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 >> 172.21.15.80:0/2721062016 conn(0x7ff96c0d20d0 msgr2=0x7ff96c0b3490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:04.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 shutdown_connections 2024-04-25T13:41:04.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.423+0000 7ff9718e2640 1 -- 172.21.15.80:0/2721062016 wait complete. 2024-04-25T13:41:04.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:04.447 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:04.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8ce76640 1 Processor -- start 2024-04-25T13:41:04.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8ce76640 1 -- start start 2024-04-25T13:41:04.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8ce76640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd880d22e0 0x7efd88144580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8ce76640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd88068c90 con 0x7efd8815efd0 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8ce76640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd88068f70 con 0x7efd880d0270 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8ce76640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efd88069250 con 0x7efd880d22e0 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd85d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7efd8815efd0 0x7efd880cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41814/0 (socket says 172.21.15.80:41814) 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd85d9b640 1 -- 172.21.15.80:0/3978616770 learned_addr learned my addr 172.21.15.80:0/3978616770 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8659c640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd880d22e0 0x7efd88144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1887678576 0 0) 0x7efd88068c90 con 0x7efd8815efd0 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efd70003530 con 0x7efd8815efd0 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1985712526 0 0) 0x7efd70003530 con 0x7efd8815efd0 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd880d22e0 msgr2=0x7efd88144580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd880d22e0 0x7efd88144580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 >> v1:172.21.15.87:6789/0 conn(0x7efd880d0270 legacy=0x7efd88161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8815eab0 con 0x7efd8815efd0 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7efd7c001680 con 0x7efd8815efd0 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 >> v1:172.21.15.80:6789/0 conn(0x7efd8815efd0 legacy=0x7efd880cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd700039f0 0x7efd70003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd880d22e0 0x7efd70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd70004d50 0x7efd70005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efd88069250 con 0x7efd70004d50 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efd7000bb00 con 0x7efd700039f0 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efd7000bd00 con 0x7efd880d22e0 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.599+0000 7efd86d9d640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd70004d50 0x7efd70005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd85d9b640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd880d22e0 0x7efd70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8659c640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd700039f0 0x7efd70003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd86d9d640 1 -- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd880d22e0 msgr2=0x7efd70004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:04.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd86d9d640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd880d22e0 0x7efd70004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd86d9d640 1 -- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd700039f0 msgr2=0x7efd70003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:04.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd86d9d640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd700039f0 0x7efd70003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd86d9d640 1 -- 172.21.15.80:0/3978616770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd70003530 con 0x7efd70004d50 2024-04-25T13:41:04.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd86d9d640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd70004d50 0x7efd70005160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd80001e40 tx=0x7efd8001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0b4e6c3867fa66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80001a90 con 0x7efd70004d50 2024-04-25T13:41:04.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd80003b80 con 0x7efd70004d50 2024-04-25T13:41:04.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8559a640 1 -- 172.21.15.80:0/3978616770 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80005080 con 0x7efd70004d50 2024-04-25T13:41:04.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8ce76640 1 -- 172.21.15.80:0/3978616770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd70004d50 msgr2=0x7efd70005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8ce76640 1 --2- 172.21.15.80:0/3978616770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd70004d50 0x7efd70005160 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efd80001e40 tx=0x7efd8001c760 comp rx=0 tx=0).stop 2024-04-25T13:41:04.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd86d9d640 1 -- 172.21.15.80:0/3978616770 reap_dead start 2024-04-25T13:41:04.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8ce76640 1 -- 172.21.15.80:0/3978616770 shutdown_connections 2024-04-25T13:41:04.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8ce76640 1 -- 172.21.15.80:0/3978616770 >> 172.21.15.80:0/3978616770 conn(0x7efd880d3370 msgr2=0x7efd880cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:04.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8ce76640 1 -- 172.21.15.80:0/3978616770 shutdown_connections 2024-04-25T13:41:04.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.603+0000 7efd8ce76640 1 -- 172.21.15.80:0/3978616770 wait complete. 2024-04-25T13:41:04.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 Processor -- start 2024-04-25T13:41:04.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 -- start start 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd88157470 0x7efd881589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd88155ee0 0x7efd88159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd881562f0 0x7efd881415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efd8815e7a0 con 0x7efd881562f0 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efd880d26f0 con 0x7efd88155ee0 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efd881624d0 con 0x7efd88157470 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd881562f0 0x7efd881415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd881562f0 0x7efd881415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59378/0 (socket says 172.21.15.80:59378) 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 -- 172.21.15.80:0/2356165705 learned_addr learned my addr 172.21.15.80:0/2356165705 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:04.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8659c640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd88157470 0x7efd881589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd85d9b640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd88155ee0 0x7efd88159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 -- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd88157470 msgr2=0x7efd881589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd88157470 0x7efd881589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 -- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd88155ee0 msgr2=0x7efd88159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd88155ee0 0x7efd88159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 -- 172.21.15.80:0/2356165705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd88157950 con 0x7efd881562f0 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd86d9d640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd881562f0 0x7efd881415b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd8001ccd0 tx=0x7efd70005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd76ffd640 1 -- 172.21.15.80:0/2356165705 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800b7070 con 0x7efd881562f0 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd76ffd640 1 -- 172.21.15.80:0/2356165705 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd80005830 con 0x7efd881562f0 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd88157c30 con 0x7efd881562f0 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd76ffd640 1 -- 172.21.15.80:0/2356165705 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80001a90 con 0x7efd881562f0 2024-04-25T13:41:04.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.607+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd881581f0 con 0x7efd881562f0 2024-04-25T13:41:04.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.611+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd8806a3a0 con 0x7efd881562f0 2024-04-25T13:41:04.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.611+0000 7efd76ffd640 1 -- 172.21.15.80:0/2356165705 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efd800ba070 con 0x7efd881562f0 2024-04-25T13:41:04.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.611+0000 7efd76ffd640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd6806f400 0x7efd68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.611+0000 7efd76ffd640 1 -- 172.21.15.80:0/2356165705 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7efd801283c0 con 0x7efd881562f0 2024-04-25T13:41:04.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.611+0000 7efd8659c640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd6806f400 0x7efd68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.611+0000 7efd8659c640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd6806f400 0x7efd68071910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd88162880 tx=0x7efd78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.615+0000 7efd76ffd640 1 -- 172.21.15.80:0/2356165705 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efd800ac020 con 0x7efd881562f0 2024-04-25T13:41:04.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.779+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd880713b0 con 0x7efd881562f0 2024-04-25T13:41:04.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.779+0000 7efd76ffd640 1 -- 172.21.15.80:0/2356165705 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7efd80016470 con 0x7efd881562f0 2024-04-25T13:41:04.783 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:04.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd6806f400 msgr2=0x7efd68071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd6806f400 0x7efd68071910 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7efd88162880 tx=0x7efd78008040 comp rx=0 tx=0).stop 2024-04-25T13:41:04.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd881562f0 msgr2=0x7efd881415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd881562f0 0x7efd881415b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efd8001ccd0 tx=0x7efd70005840 comp rx=0 tx=0).stop 2024-04-25T13:41:04.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 shutdown_connections 2024-04-25T13:41:04.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd6806f400 0x7efd68071910 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd881562f0 0x7efd881415b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd88155ee0 0x7efd88159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 --2- 172.21.15.80:0/2356165705 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd88157470 0x7efd881589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 >> 172.21.15.80:0/2356165705 conn(0x7efd880d3370 msgr2=0x7efd880758a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:04.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 shutdown_connections 2024-04-25T13:41:04.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.783+0000 7efd8ce76640 1 -- 172.21.15.80:0/2356165705 wait complete. 2024-04-25T13:41:04.805 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:04.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e2f3b640 1 Processor -- start 2024-04-25T13:41:04.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e2f3b640 1 -- start start 2024-04-25T13:41:04.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e2f3b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f79dc0d3ed0 0x7f79dc0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e2f3b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f79dc0d2ad0 0x7f79dc0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e2f3b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79dc0b92a0 con 0x7f79dc0b7c10 2024-04-25T13:41:04.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e2f3b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f79dc0d4c30 con 0x7f79dc0d3ed0 2024-04-25T13:41:04.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e2f3b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f79dc0d4ef0 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e0cd7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f79dc0d3ed0 0x7f79dc0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e14d8640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f79dc0b7c10 0x7f79dc0b9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41820/0 (socket says 172.21.15.80:41820) 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e0cd7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f79dc0d3ed0 0x7f79dc0d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59388/0 (socket says 172.21.15.80:59388) 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e14d8640 1 -- 172.21.15.80:0/1278923790 learned_addr learned my addr 172.21.15.80:0/1278923790 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79dbfff640 1 --2- 172.21.15.80:0/1278923790 >> v2:172.21.15.87:3300/0 conn(0x7f79dc0d2ad0 0x7f79dc0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e0cd7640 1 -- 172.21.15.80:0/1278923790 >> v2:172.21.15.87:3300/0 conn(0x7f79dc0d2ad0 msgr2=0x7f79dc0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e0cd7640 1 --2- 172.21.15.80:0/1278923790 >> v2:172.21.15.87:3300/0 conn(0x7f79dc0d2ad0 0x7f79dc0b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e0cd7640 1 -- 172.21.15.80:0/1278923790 >> v1:172.21.15.80:6789/0 conn(0x7f79dc0b7c10 legacy=0x7f79dc0b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79e0cd7640 1 -- 172.21.15.80:0/1278923790 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc0d51d0 con 0x7f79dc0d3ed0 2024-04-25T13:41:04.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.943+0000 7f79dbfff640 1 --2- 172.21.15.80:0/1278923790 >> v2:172.21.15.87:3300/0 conn(0x7f79dc0d2ad0 0x7f79dc0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:04.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e0cd7640 1 --2- 172.21.15.80:0/1278923790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d3ed0 0x7f79dc0d2590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0b8dc0 tx=0x7f79cc01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5734824b93dbac5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79db7fe640 1 -- 172.21.15.80:0/1278923790 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0b0070 con 0x7f79dc0d3ed0 2024-04-25T13:41:04.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79db7fe640 1 -- 172.21.15.80:0/1278923790 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79cc004230 con 0x7f79dc0d3ed0 2024-04-25T13:41:04.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79db7fe640 1 -- 172.21.15.80:0/1278923790 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79cc003620 con 0x7f79dc0d3ed0 2024-04-25T13:41:04.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1278923790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d3ed0 msgr2=0x7f79dc0d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1278923790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d3ed0 0x7f79dc0d2590 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f79dc0b8dc0 tx=0x7f79cc01c330 comp rx=0 tx=0).stop 2024-04-25T13:41:04.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1278923790 shutdown_connections 2024-04-25T13:41:04.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1278923790 >> v2:172.21.15.87:3300/0 conn(0x7f79dc0d2ad0 0x7f79dc0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1278923790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d3ed0 0x7f79dc0d2590 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1278923790 >> 172.21.15.80:0/1278923790 conn(0x7f79dc0737b0 msgr2=0x7f79dc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:04.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1278923790 shutdown_connections 2024-04-25T13:41:04.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1278923790 wait complete. 2024-04-25T13:41:04.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 Processor -- start 2024-04-25T13:41:04.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.947+0000 7f79e2f3b640 1 -- start start 2024-04-25T13:41:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79dc0b7c10 0x7f79dc14f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d2ad0 0x7f79dc0ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79dc0d3ed0 0x7f79dc0cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc0ccae0 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f79dc0cc960 con 0x7f79dc0d3ed0 2024-04-25T13:41:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f79dc160620 con 0x7f79dc0b7c10 2024-04-25T13:41:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d2ad0 0x7f79dc0ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d2ad0 0x7f79dc0ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59394/0 (socket says 172.21.15.80:59394) 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 -- 172.21.15.80:0/1921208399 learned_addr learned my addr 172.21.15.80:0/1921208399 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 -- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79dc0b7c10 msgr2=0x7f79dc14f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79dc0b7c10 0x7f79dc14f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 -- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79dc0d3ed0 msgr2=0x7f79dc0cd1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e14d8640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79dc0d3ed0 0x7f79dc0cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79dc0d3ed0 0x7f79dc0cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 -- 172.21.15.80:0/1921208399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79dc06c800 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79dbfff640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d2ad0 0x7f79dc0ccc60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c7f0 tx=0x7f79d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79d97fa640 1 -- 172.21.15.80:0/1921208399 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0015070 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79d97fa640 1 -- 172.21.15.80:0/1921208399 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79d0007820 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79dc0cd830 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79d97fa640 1 -- 172.21.15.80:0/1921208399 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0007c10 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79dc0cdad0 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79dc06a3a0 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.951+0000 7f79d97fa640 1 -- 172.21.15.80:0/1921208399 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f79d0018070 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.955+0000 7f79d97fa640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79c806f430 0x7f79c8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.955+0000 7f79d97fa640 1 -- 172.21.15.80:0/1921208399 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f79d008f3d0 con 0x7f79dc0d2ad0 2024-04-25T13:41:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.955+0000 7f79e0cd7640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79c806f430 0x7f79c8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:04.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.955+0000 7f79e0cd7640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79c806f430 0x7f79c8071940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0ad000 tx=0x7f79cc004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:04.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:04.959+0000 7f79d97fa640 1 -- 172.21.15.80:0/1921208399 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79d005c2c0 con 0x7f79dc0d2ad0 2024-04-25T13:41:05.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f79dc0cc210 con 0x7f79dc0d2ad0 2024-04-25T13:41:05.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79d97fa640 1 -- 172.21.15.80:0/1921208399 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f79d005bbc0 con 0x7f79dc0d2ad0 2024-04-25T13:41:05.110 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:05.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79c806f430 msgr2=0x7f79c8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79c806f430 0x7f79c8071940 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f79cc0ad000 tx=0x7f79cc004740 comp rx=0 tx=0).stop 2024-04-25T13:41:05.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d2ad0 msgr2=0x7f79dc0ccc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d2ad0 0x7f79dc0ccc60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c7f0 tx=0x7f79d000ccb0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 shutdown_connections 2024-04-25T13:41:05.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79c806f430 0x7f79c8071940 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79dc0d3ed0 0x7f79dc0cd1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79dc0d2ad0 0x7f79dc0ccc60 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 --2- 172.21.15.80:0/1921208399 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79dc0b7c10 0x7f79dc14f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 >> 172.21.15.80:0/1921208399 conn(0x7f79dc0737b0 msgr2=0x7f79dc15dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:05.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 shutdown_connections 2024-04-25T13:41:05.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.107+0000 7f79e2f3b640 1 -- 172.21.15.80:0/1921208399 wait complete. 2024-04-25T13:41:05.128 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:05.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc25f6d640 1 Processor -- start 2024-04-25T13:41:05.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc25f6d640 1 -- start start 2024-04-25T13:41:05.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc25f6d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbc200b5330 0x7fbc200d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc25f6d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc200b5740 con 0x7fbc20161b60 2024-04-25T13:41:05.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc25f6d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc2006dd10 con 0x7fbc200d0ff0 2024-04-25T13:41:05.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc25f6d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbc201629b0 con 0x7fbc200b5330 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fbc20161b60 0x7fbc20161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41834/0 (socket says 172.21.15.80:41834) 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1ffff640 1 -- 172.21.15.80:0/3268098671 learned_addr learned my addr 172.21.15.80:0/3268098671 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1f7fe640 1 --2- 172.21.15.80:0/3268098671 >> v2:172.21.15.87:3300/0 conn(0x7fbc200b5330 0x7fbc200d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 682494290 0 0) 0x7fbc200b5740 con 0x7fbc20161b60 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbc08003530 con 0x7fbc20161b60 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3559652557 0 0) 0x7fbc08003530 con 0x7fbc20161b60 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 >> v2:172.21.15.87:3300/0 conn(0x7fbc200b5330 msgr2=0x7fbc200d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 --2- 172.21.15.80:0/3268098671 >> v2:172.21.15.87:3300/0 conn(0x7fbc200b5330 0x7fbc200d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 >> v1:172.21.15.87:6789/0 conn(0x7fbc200d0ff0 legacy=0x7fbc2006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc2006dff0 con 0x7fbc20161b60 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fbc18001a00 con 0x7fbc20161b60 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 >> v1:172.21.15.80:6789/0 conn(0x7fbc20161b60 legacy=0x7fbc20161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc08003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc08004810 0x7fbc08004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc08005260 0x7fbc0800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc201629b0 con 0x7fbc080039c0 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0800c1f0 con 0x7fbc08005260 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0800c3f0 con 0x7fbc08004810 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.279+0000 7fbc1f7fe640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc08003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1ffff640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc08005260 0x7fbc0800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1effd640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc08004810 0x7fbc08004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1f7fe640 1 -- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc08004810 msgr2=0x7fbc08004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1f7fe640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc08004810 0x7fbc08004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1f7fe640 1 -- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc08005260 msgr2=0x7fbc0800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1f7fe640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc08005260 0x7fbc0800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1f7fe640 1 -- 172.21.15.80:0/3268098671 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc08003530 con 0x7fbc080039c0 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1ffff640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc08005260 0x7fbc0800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1effd640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc08004810 0x7fbc08004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:05.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1ffff640 1 -- 172.21.15.80:0/3268098671 reap_dead start 2024-04-25T13:41:05.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1f7fe640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc08003e30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009ae0 tx=0x7fbc1001ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2533e145cd46580c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:05.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100096b0 con 0x7fbc080039c0 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc100a6450 con 0x7fbc080039c0 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc1e7fc640 1 -- 172.21.15.80:0/3268098671 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a6840 con 0x7fbc080039c0 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc25f6d640 1 -- 172.21.15.80:0/3268098671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 msgr2=0x7fbc08003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc25f6d640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc08003e30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbc10009ae0 tx=0x7fbc1001ca70 comp rx=0 tx=0).stop 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc25f6d640 1 -- 172.21.15.80:0/3268098671 shutdown_connections 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc25f6d640 1 --2- 172.21.15.80:0/3268098671 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc08003e30 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc25f6d640 1 -- 172.21.15.80:0/3268098671 >> 172.21.15.80:0/3268098671 conn(0x7fbc200b4160 msgr2=0x7fbc200d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc25f6d640 1 -- 172.21.15.80:0/3268098671 shutdown_connections 2024-04-25T13:41:05.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.283+0000 7fbc25f6d640 1 -- 172.21.15.80:0/3268098671 wait complete. 2024-04-25T13:41:05.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 Processor -- start 2024-04-25T13:41:05.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 -- start start 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc20146520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc201fbbf0 0x7fbc2015ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc2015efd0 0x7fbc20140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc20162380 con 0x7fbc080039c0 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2006f5c0 con 0x7fbc2015efd0 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc200adee0 con 0x7fbc201fbbf0 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc20146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc20146520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59404/0 (socket says 172.21.15.80:59404) 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 -- 172.21.15.80:0/2670811683 learned_addr learned my addr 172.21.15.80:0/2670811683 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1effd640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc201fbbf0 0x7fbc2015ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1ffff640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc2015efd0 0x7fbc20140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 -- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc201fbbf0 msgr2=0x7fbc2015ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc201fbbf0 0x7fbc2015ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 -- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc2015efd0 msgr2=0x7fbc20140210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc2015efd0 0x7fbc20140210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 -- 172.21.15.80:0/2670811683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc200abc80 con 0x7fbc080039c0 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc1f7fe640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc20146520 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc1001c8a0 tx=0x7fbc10019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc07fff640 1 -- 172.21.15.80:0/2670811683 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100a6d00 con 0x7fbc080039c0 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc07fff640 1 -- 172.21.15.80:0/2670811683 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc100b9440 con 0x7fbc080039c0 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc200abf60 con 0x7fbc080039c0 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc07fff640 1 -- 172.21.15.80:0/2670811683 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100b9810 con 0x7fbc080039c0 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.287+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc20145210 con 0x7fbc080039c0 2024-04-25T13:41:05.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.291+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc2006a3a0 con 0x7fbc080039c0 2024-04-25T13:41:05.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.291+0000 7fbc07fff640 1 -- 172.21.15.80:0/2670811683 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbc100c2430 con 0x7fbc080039c0 2024-04-25T13:41:05.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.291+0000 7fbc07fff640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbc0c06f400 0x7fbc0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.291+0000 7fbc07fff640 1 -- 172.21.15.80:0/2670811683 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbc10129540 con 0x7fbc080039c0 2024-04-25T13:41:05.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.291+0000 7fbc1effd640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbc0c06f400 0x7fbc0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.291+0000 7fbc1effd640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbc0c06f400 0x7fbc0c071910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc2015f940 tx=0x7fbc14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:05.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.295+0000 7fbc07fff640 1 -- 172.21.15.80:0/2670811683 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbc100ac020 con 0x7fbc080039c0 2024-04-25T13:41:05.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.455+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbc2006fd00 con 0x7fbc080039c0 2024-04-25T13:41:05.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc07fff640 1 -- 172.21.15.80:0/2670811683 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7fbc10016780 con 0x7fbc080039c0 2024-04-25T13:41:05.459 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbc0c06f400 msgr2=0x7fbc0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbc0c06f400 0x7fbc0c071910 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc2015f940 tx=0x7fbc14009040 comp rx=0 tx=0).stop 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 msgr2=0x7fbc20146520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc20146520 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbc1001c8a0 tx=0x7fbc10019810 comp rx=0 tx=0).stop 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 shutdown_connections 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbc0c06f400 0x7fbc0c071910 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc2015efd0 0x7fbc20140210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc201fbbf0 0x7fbc2015ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 --2- 172.21.15.80:0/2670811683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080039c0 0x7fbc20146520 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 >> 172.21.15.80:0/2670811683 conn(0x7fbc200b4160 msgr2=0x7fbc2015fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:05.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 shutdown_connections 2024-04-25T13:41:05.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.459+0000 7fbc25f6d640 1 -- 172.21.15.80:0/2670811683 wait complete. 2024-04-25T13:41:05.481 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:05.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8cbd4a640 1 Processor -- start 2024-04-25T13:41:05.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8cbd4a640 1 -- start start 2024-04-25T13:41:05.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8cbd4a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe8c41614b0 0x7fe8c406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8cbd4a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c4163eb0 0x7fe8c41642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8cbd4a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8c40b2100 con 0x7fe8c40cfb60 2024-04-25T13:41:05.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8cbd4a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe8c40b23e0 con 0x7fe8c41614b0 2024-04-25T13:41:05.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8cbd4a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c41609d0 con 0x7fe8c4163eb0 2024-04-25T13:41:05.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8c9ae6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe8c41614b0 0x7fe8c406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8c9ae6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe8c41614b0 0x7fe8c406dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:51938/0 (socket says 172.21.15.80:51938) 2024-04-25T13:41:05.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8c92e5640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe8c40cfb60 0x7fe8c40cff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:51640/0 (socket says 172.21.15.80:51640) 2024-04-25T13:41:05.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8ca2e7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c4163eb0 0x7fe8c41642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.619+0000 7fe8c9ae6640 1 -- 172.21.15.80:0/863301812 learned_addr learned my addr 172.21.15.80:0/863301812 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:05.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.623+0000 7fe8ca2e7640 1 -- 172.21.15.80:0/863301812 >> v2:172.21.15.87:3300/0 conn(0x7fe8c41614b0 msgr2=0x7fe8c406dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:05.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.623+0000 7fe8ca2e7640 1 --2- 172.21.15.80:0/863301812 >> v2:172.21.15.87:3300/0 conn(0x7fe8c41614b0 0x7fe8c406dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.623+0000 7fe8ca2e7640 1 -- 172.21.15.80:0/863301812 >> v1:172.21.15.87:6789/0 conn(0x7fe8c40cfb60 legacy=0x7fe8c40cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.623+0000 7fe8ca2e7640 1 -- 172.21.15.80:0/863301812 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c4160cb0 con 0x7fe8c4163eb0 2024-04-25T13:41:05.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.623+0000 7fe8ca2e7640 1 --2- 172.21.15.80:0/863301812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c4163eb0 0x7fe8c41642c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009d90 tx=0x7fe8bc01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a2b0b87d758ac47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:05.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.623+0000 7fe8c8ae4640 1 -- 172.21.15.80:0/863301812 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0b2070 con 0x7fe8c4163eb0 2024-04-25T13:41:05.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.623+0000 7fe8c8ae4640 1 -- 172.21.15.80:0/863301812 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8bc0a5830 con 0x7fe8c4163eb0 2024-04-25T13:41:05.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.623+0000 7fe8c8ae4640 1 -- 172.21.15.80:0/863301812 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0a5c00 con 0x7fe8c4163eb0 2024-04-25T13:41:05.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/863301812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c4163eb0 msgr2=0x7fe8c41642c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/863301812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c4163eb0 0x7fe8c41642c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc009d90 tx=0x7fe8bc01c970 comp rx=0 tx=0).stop 2024-04-25T13:41:05.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/863301812 shutdown_connections 2024-04-25T13:41:05.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/863301812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c4163eb0 0x7fe8c41642c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/863301812 >> v2:172.21.15.87:3300/0 conn(0x7fe8c41614b0 0x7fe8c406dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/863301812 >> 172.21.15.80:0/863301812 conn(0x7fe8c40d33a0 msgr2=0x7fe8c40cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:05.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/863301812 shutdown_connections 2024-04-25T13:41:05.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/863301812 wait complete. 2024-04-25T13:41:05.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 Processor -- start 2024-04-25T13:41:05.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.627+0000 7fe8cbd4a640 1 -- start start 2024-04-25T13:41:05.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c40cfb60 0x7fe8c4155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c41614b0 0x7fe8c415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8c4163eb0 0x7fe8c4160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c406d3b0 con 0x7fe8c4163eb0 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe8c406d230 con 0x7fe8c40cfb60 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe8c416d810 con 0x7fe8c41614b0 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8c4163eb0 0x7fe8c4160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8c4163eb0 0x7fe8c4160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59412/0 (socket says 172.21.15.80:59412) 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 -- 172.21.15.80:0/492507114 learned_addr learned my addr 172.21.15.80:0/492507114 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8c92e5640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c41614b0 0x7fe8c415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8c9ae6640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c40cfb60 0x7fe8c4155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 -- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c41614b0 msgr2=0x7fe8c415fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c41614b0 0x7fe8c415fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 -- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c40cfb60 msgr2=0x7fe8c4155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c40cfb60 0x7fe8c4155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 -- 172.21.15.80:0/492507114 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8c41552d0 con 0x7fe8c4163eb0 2024-04-25T13:41:05.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ca2e7640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8c4163eb0 0x7fe8c4160000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0b0040 tx=0x7fe8bc019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:05.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ba7fc640 1 -- 172.21.15.80:0/492507114 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0b2050 con 0x7fe8c4163eb0 2024-04-25T13:41:05.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ba7fc640 1 -- 172.21.15.80:0/492507114 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe8bc0be070 con 0x7fe8c4163eb0 2024-04-25T13:41:05.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ba7fc640 1 -- 172.21.15.80:0/492507114 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0ad870 con 0x7fe8c4163eb0 2024-04-25T13:41:05.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8c4162e70 con 0x7fe8c4163eb0 2024-04-25T13:41:05.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8c41633f0 con 0x7fe8c4163eb0 2024-04-25T13:41:05.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8c415f700 con 0x7fe8c4163eb0 2024-04-25T13:41:05.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ba7fc640 1 -- 172.21.15.80:0/492507114 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe8bc0ada10 con 0x7fe8c4163eb0 2024-04-25T13:41:05.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.631+0000 7fe8ba7fc640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe8a806f400 0x7fe8a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:05.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.635+0000 7fe8ba7fc640 1 -- 172.21.15.80:0/492507114 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe8bc12a450 con 0x7fe8c4163eb0 2024-04-25T13:41:05.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.635+0000 7fe8c9ae6640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe8a806f400 0x7fe8a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:05.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.635+0000 7fe8c9ae6640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe8a806f400 0x7fe8a8071910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40d3a20 tx=0x7fe8c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:05.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.639+0000 7fe8ba7fc640 1 -- 172.21.15.80:0/492507114 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe8bc016560 con 0x7fe8c4163eb0 2024-04-25T13:41:05.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.795+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe8c41606a0 con 0x7fe8c4163eb0 2024-04-25T13:41:05.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.795+0000 7fe8ba7fc640 1 -- 172.21.15.80:0/492507114 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fe8bc0f72c0 con 0x7fe8c4163eb0 2024-04-25T13:41:05.797 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:05.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe8a806f400 msgr2=0x7fe8a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe8a806f400 0x7fe8a8071910 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe8c40d3a20 tx=0x7fe8c0008040 comp rx=0 tx=0).stop 2024-04-25T13:41:05.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8c4163eb0 msgr2=0x7fe8c4160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:05.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8c4163eb0 0x7fe8c4160000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0b0040 tx=0x7fe8bc019710 comp rx=0 tx=0).stop 2024-04-25T13:41:05.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 shutdown_connections 2024-04-25T13:41:05.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe8a806f400 0x7fe8a8071910 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe8c4163eb0 0x7fe8c4160000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe8c41614b0 0x7fe8c415fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 --2- 172.21.15.80:0/492507114 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe8c40cfb60 0x7fe8c4155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:05.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 >> 172.21.15.80:0/492507114 conn(0x7fe8c40d33a0 msgr2=0x7fe8c4153ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:05.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 shutdown_connections 2024-04-25T13:41:05.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:05.799+0000 7fe8cbd4a640 1 -- 172.21.15.80:0/492507114 wait complete. 2024-04-25T13:41:05.817 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:05.818 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:05.822 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:06.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.007+0000 7ff002350640 1 Processor -- start 2024-04-25T13:41:06.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7ff002350640 1 -- start start 2024-04-25T13:41:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7ff002350640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 0x7feffc0734e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7ff002350640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7feffc15fe50 0x7feffc160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7ff002350640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feffc1607a0 con 0x7feffc06ff70 2024-04-25T13:41:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7ff002350640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7feffc0b1320 con 0x7feffc15fe50 2024-04-25T13:41:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7ff002350640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feffc0b1040 con 0x7feffc159f10 2024-04-25T13:41:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7feffb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7feffc15fe50 0x7feffc160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7ff0008ed640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7feffc06ff70 0x7feffc15c800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41836/0 (socket says 172.21.15.80:41836) 2024-04-25T13:41:06.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7ff0008ed640 1 -- 172.21.15.80:0/3711333758 learned_addr learned my addr 172.21.15.80:0/3711333758 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:06.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7feffbfff640 1 --2- 172.21.15.80:0/3711333758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 0x7feffc0734e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7feffb7fe640 1 -- 172.21.15.80:0/3711333758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 msgr2=0x7feffc0734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:06.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7feffb7fe640 1 --2- 172.21.15.80:0/3711333758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 0x7feffc0734e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7feffb7fe640 1 -- 172.21.15.80:0/3711333758 >> v1:172.21.15.80:6789/0 conn(0x7feffc06ff70 legacy=0x7feffc15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.011+0000 7feffb7fe640 1 -- 172.21.15.80:0/3711333758 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc0b5450 con 0x7feffc15fe50 2024-04-25T13:41:06.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7feffb7fe640 1 --2- 172.21.15.80:0/3711333758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc15fe50 0x7feffc160260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097c0 tx=0x7fefe401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360961876b9f4265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7feffaffd640 1 -- 172.21.15.80:0/3711333758 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe40b0070 con 0x7feffc15fe50 2024-04-25T13:41:06.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7feffaffd640 1 -- 172.21.15.80:0/3711333758 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefe40ab600 con 0x7feffc15fe50 2024-04-25T13:41:06.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7feffaffd640 1 -- 172.21.15.80:0/3711333758 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefe40ab9d0 con 0x7feffc15fe50 2024-04-25T13:41:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 -- 172.21.15.80:0/3711333758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc15fe50 msgr2=0x7feffc160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 --2- 172.21.15.80:0/3711333758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc15fe50 0x7feffc160260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fefe40097c0 tx=0x7fefe401c410 comp rx=0 tx=0).stop 2024-04-25T13:41:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 -- 172.21.15.80:0/3711333758 shutdown_connections 2024-04-25T13:41:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 --2- 172.21.15.80:0/3711333758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc15fe50 0x7feffc160260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 --2- 172.21.15.80:0/3711333758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 0x7feffc0734e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 -- 172.21.15.80:0/3711333758 >> 172.21.15.80:0/3711333758 conn(0x7feffc06d080 msgr2=0x7feffc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:06.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 -- 172.21.15.80:0/3711333758 shutdown_connections 2024-04-25T13:41:06.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 -- 172.21.15.80:0/3711333758 wait complete. 2024-04-25T13:41:06.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 Processor -- start 2024-04-25T13:41:06.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.015+0000 7ff002350640 1 -- start start 2024-04-25T13:41:06.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc06ff70 0x7feffc15ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 0x7feffc150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feffc15fe50 0x7feffc151320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feffc0b0ca0 con 0x7feffc06ff70 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feffc073a20 con 0x7feffc15fe50 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feffc0b0e20 con 0x7feffc159f10 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc06ff70 0x7feffc15ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc06ff70 0x7feffc15ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59438/0 (socket says 172.21.15.80:59438) 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 -- 172.21.15.80:0/3549390277 learned_addr learned my addr 172.21.15.80:0/3549390277 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 -- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 msgr2=0x7feffc150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 0x7feffc150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 -- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feffc15fe50 msgr2=0x7feffc151320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feffc15fe50 0x7feffc151320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 -- 172.21.15.80:0/3549390277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feffc15b680 con 0x7feffc06ff70 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffbfff640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc06ff70 0x7feffc15ae50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefec00d600 tx=0x7fefec00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feff8ff9640 1 -- 172.21.15.80:0/3549390277 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec004240 con 0x7feffc06ff70 2024-04-25T13:41:06.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feff8ff9640 1 -- 172.21.15.80:0/3549390277 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fefec0043e0 con 0x7feffc06ff70 2024-04-25T13:41:06.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feffc15b960 con 0x7feffc06ff70 2024-04-25T13:41:06.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feff8ff9640 1 -- 172.21.15.80:0/3549390277 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefec003840 con 0x7feffc06ff70 2024-04-25T13:41:06.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feffc15bf20 con 0x7feffc06ff70 2024-04-25T13:41:06.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feffc06a3a0 con 0x7feffc06ff70 2024-04-25T13:41:06.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feff8ff9640 1 -- 172.21.15.80:0/3549390277 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fefec0039e0 con 0x7feffc06ff70 2024-04-25T13:41:06.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feff8ff9640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fefdc06f400 0x7fefdc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feff8ff9640 1 -- 172.21.15.80:0/3549390277 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fefec08f570 con 0x7feffc06ff70 2024-04-25T13:41:06.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.019+0000 7feffb7fe640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fefdc06f400 0x7fefdc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.023+0000 7feffb7fe640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fefdc06f400 0x7fefdc071910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fefe40ad000 tx=0x7fefe4004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.027+0000 7feff8ff9640 1 -- 172.21.15.80:0/3549390277 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fefec05c4f0 con 0x7feffc06ff70 2024-04-25T13:41:06.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.175+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feffc072390 con 0x7feffc06ff70 2024-04-25T13:41:06.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.175+0000 7feff8ff9640 1 -- 172.21.15.80:0/3549390277 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7fefec05bdf0 con 0x7feffc06ff70 2024-04-25T13:41:06.178 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:06.178 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:06.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fefdc06f400 msgr2=0x7fefdc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fefdc06f400 0x7fefdc071910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fefe40ad000 tx=0x7fefe4004740 comp rx=0 tx=0).stop 2024-04-25T13:41:06.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc06ff70 msgr2=0x7feffc15ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc06ff70 0x7feffc15ae50 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fefec00d600 tx=0x7fefec00dac0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 shutdown_connections 2024-04-25T13:41:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fefdc06f400 0x7fefdc071910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feffc15fe50 0x7feffc151320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feffc159f10 0x7feffc150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 --2- 172.21.15.80:0/3549390277 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feffc06ff70 0x7feffc15ae50 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 >> 172.21.15.80:0/3549390277 conn(0x7feffc06d080 msgr2=0x7feffc15c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 shutdown_connections 2024-04-25T13:41:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.179+0000 7ff002350640 1 -- 172.21.15.80:0/3549390277 wait complete. 2024-04-25T13:41:06.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:06.199 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:06.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8b9bf640 1 Processor -- start 2024-04-25T13:41:06.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8b9bf640 1 -- start start 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8b9bf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4d84071e60 0x7f4d841574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8b9bf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4d84073c40 0x7f4d84160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8b9bf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d841613b0 con 0x7f4d84160fa0 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8b9bf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4d840b1350 con 0x7f4d84071e60 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8b9bf640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4d8406f9a0 con 0x7f4d84073c40 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8975b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4d84071e60 0x7f4d841574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d89f5c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4d84160fa0 0x7f4d840b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41838/0 (socket says 172.21.15.80:41838) 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d89f5c640 1 -- 172.21.15.80:0/3213934005 learned_addr learned my addr 172.21.15.80:0/3213934005 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d88f5a640 1 --2- 172.21.15.80:0/3213934005 >> v2:172.21.15.87:3300/0 conn(0x7f4d84073c40 0x7f4d84160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8975b640 1 -- 172.21.15.80:0/3213934005 >> v2:172.21.15.87:3300/0 conn(0x7f4d84073c40 msgr2=0x7f4d84160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8975b640 1 --2- 172.21.15.80:0/3213934005 >> v2:172.21.15.87:3300/0 conn(0x7f4d84073c40 0x7f4d84160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8975b640 1 -- 172.21.15.80:0/3213934005 >> v1:172.21.15.80:6789/0 conn(0x7f4d84160fa0 legacy=0x7f4d840b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d7bfff640 1 -- 172.21.15.80:0/3213934005 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3677090185 0 0) 0x7f4d7c0014a0 con 0x7f4d84160fa0 2024-04-25T13:41:06.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d8975b640 1 -- 172.21.15.80:0/3213934005 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8406fbf0 con 0x7f4d84071e60 2024-04-25T13:41:06.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.343+0000 7f4d88f5a640 1 --2- 172.21.15.80:0/3213934005 >> v2:172.21.15.87:3300/0 conn(0x7f4d84073c40 0x7f4d84160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:06.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8975b640 1 --2- 172.21.15.80:0/3213934005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84071e60 0x7f4d841574c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4d840cecc0 tx=0x7f4d8001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1025d5ba074e47fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d7bfff640 1 -- 172.21.15.80:0/3213934005 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800b0070 con 0x7f4d84071e60 2024-04-25T13:41:06.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d7bfff640 1 -- 172.21.15.80:0/3213934005 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d800ab600 con 0x7f4d84071e60 2024-04-25T13:41:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d7bfff640 1 -- 172.21.15.80:0/3213934005 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d800ab9d0 con 0x7f4d84071e60 2024-04-25T13:41:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/3213934005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84071e60 msgr2=0x7f4d841574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/3213934005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84071e60 0x7f4d841574c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4d840cecc0 tx=0x7f4d8001c480 comp rx=0 tx=0).stop 2024-04-25T13:41:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/3213934005 shutdown_connections 2024-04-25T13:41:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/3213934005 >> v2:172.21.15.87:3300/0 conn(0x7f4d84073c40 0x7f4d84160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/3213934005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84071e60 0x7f4d841574c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/3213934005 >> 172.21.15.80:0/3213934005 conn(0x7f4d840b4160 msgr2=0x7f4d840d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/3213934005 shutdown_connections 2024-04-25T13:41:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/3213934005 wait complete. 2024-04-25T13:41:06.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 Processor -- start 2024-04-25T13:41:06.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.351+0000 7f4d8b9bf640 1 -- start start 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84071e60 0x7f4d8414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d84073c40 0x7f4d84150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84160fa0 0x7f4d841513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84074050 con 0x7f4d84160fa0 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4d84072270 con 0x7f4d84071e60 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4d840b56c0 con 0x7f4d84073c40 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84160fa0 0x7f4d841513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84160fa0 0x7f4d841513f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59454/0 (socket says 172.21.15.80:59454) 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 -- 172.21.15.80:0/2302878306 learned_addr learned my addr 172.21.15.80:0/2302878306 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8975b640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84071e60 0x7f4d8414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d88f5a640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d84073c40 0x7f4d84150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 -- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d84073c40 msgr2=0x7f4d84150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d84073c40 0x7f4d84150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 -- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84071e60 msgr2=0x7f4d8414fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84071e60 0x7f4d8414fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 -- 172.21.15.80:0/2302878306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d8414ffb0 con 0x7f4d84160fa0 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d88f5a640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d84073c40 0x7f4d84150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d89f5c640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84160fa0 0x7f4d841513f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00c730 tx=0x7f4d7c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d79ffb640 1 -- 172.21.15.80:0/2302878306 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c018070 con 0x7f4d84160fa0 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d79ffb640 1 -- 172.21.15.80:0/2302878306 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00ce40 con 0x7f4d84160fa0 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d841504c0 con 0x7f4d84160fa0 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d79ffb640 1 -- 172.21.15.80:0/2302878306 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c00a880 con 0x7f4d84160fa0 2024-04-25T13:41:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d84154420 con 0x7f4d84160fa0 2024-04-25T13:41:06.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d8406a3a0 con 0x7f4d84160fa0 2024-04-25T13:41:06.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.355+0000 7f4d79ffb640 1 -- 172.21.15.80:0/2302878306 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4d7c00aa20 con 0x7f4d84160fa0 2024-04-25T13:41:06.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.359+0000 7f4d79ffb640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4d6806f400 0x7f4d68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.359+0000 7f4d8975b640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4d6806f400 0x7f4d68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.359+0000 7f4d79ffb640 1 -- 172.21.15.80:0/2302878306 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4d7c08cf60 con 0x7f4d84160fa0 2024-04-25T13:41:06.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.359+0000 7f4d8975b640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4d6806f400 0x7f4d68071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d800ad000 tx=0x7f4d800a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.363+0000 7f4d79ffb640 1 -- 172.21.15.80:0/2302878306 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4d841613b0 con 0x7f4d84160fa0 2024-04-25T13:41:06.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4d84150250 con 0x7f4d84160fa0 2024-04-25T13:41:06.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d79ffb640 1 -- 172.21.15.80:0/2302878306 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f4d7c05b790 con 0x7f4d84160fa0 2024-04-25T13:41:06.516 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:06.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4d6806f400 msgr2=0x7f4d68071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4d6806f400 0x7f4d68071910 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4d800ad000 tx=0x7f4d800a6000 comp rx=0 tx=0).stop 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84160fa0 msgr2=0x7f4d841513f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84160fa0 0x7f4d841513f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00c730 tx=0x7f4d7c00cbf0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 shutdown_connections 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4d6806f400 0x7f4d68071910 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4d84160fa0 0x7f4d841513f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4d84073c40 0x7f4d84150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 --2- 172.21.15.80:0/2302878306 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d84071e60 0x7f4d8414fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 >> 172.21.15.80:0/2302878306 conn(0x7f4d840b4160 msgr2=0x7f4d8415fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 shutdown_connections 2024-04-25T13:41:06.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.515+0000 7f4d8b9bf640 1 -- 172.21.15.80:0/2302878306 wait complete. 2024-04-25T13:41:06.539 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:06.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50d5816640 1 Processor -- start 2024-04-25T13:41:06.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50d5816640 1 -- start start 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50d5816640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f50d00d3ed0 0x7f50d00d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50d5816640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f50d00d2ad0 0x7f50d00b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50d5816640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50d00b92a0 con 0x7f50d00b7c10 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50d5816640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f50d00d4c30 con 0x7f50d00d2ad0 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50d5816640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f50d00d4ef0 con 0x7f50d00d3ed0 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50ce7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f50d00d2ad0 0x7f50d00b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50cf7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f50d00b7c10 0x7f50d00b9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41844/0 (socket says 172.21.15.80:41844) 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50cf7fe640 1 -- 172.21.15.80:0/962881802 learned_addr learned my addr 172.21.15.80:0/962881802 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50ceffd640 1 --2- 172.21.15.80:0/962881802 >> v2:172.21.15.87:3300/0 conn(0x7f50d00d3ed0 0x7f50d00d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50ce7fc640 1 -- 172.21.15.80:0/962881802 >> v2:172.21.15.87:3300/0 conn(0x7f50d00d3ed0 msgr2=0x7f50d00d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50ce7fc640 1 --2- 172.21.15.80:0/962881802 >> v2:172.21.15.87:3300/0 conn(0x7f50d00d3ed0 0x7f50d00d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50ce7fc640 1 -- 172.21.15.80:0/962881802 >> v1:172.21.15.80:6789/0 conn(0x7f50d00b7c10 legacy=0x7f50d00b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.675+0000 7f50ce7fc640 1 -- 172.21.15.80:0/962881802 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d00d51d0 con 0x7f50d00d2ad0 2024-04-25T13:41:06.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50ce7fc640 1 --2- 172.21.15.80:0/962881802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00d2ad0 0x7f50d00b76d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50d008e910 tx=0x7f50b801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0e2b88465fded3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50cdffb640 1 -- 172.21.15.80:0/962881802 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b80b0070 con 0x7f50d00d2ad0 2024-04-25T13:41:06.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50cdffb640 1 -- 172.21.15.80:0/962881802 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50b8004230 con 0x7f50d00d2ad0 2024-04-25T13:41:06.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50cdffb640 1 -- 172.21.15.80:0/962881802 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8003620 con 0x7f50d00d2ad0 2024-04-25T13:41:06.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 -- 172.21.15.80:0/962881802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00d2ad0 msgr2=0x7f50d00b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 --2- 172.21.15.80:0/962881802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00d2ad0 0x7f50d00b76d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f50d008e910 tx=0x7f50b801c330 comp rx=0 tx=0).stop 2024-04-25T13:41:06.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 -- 172.21.15.80:0/962881802 shutdown_connections 2024-04-25T13:41:06.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 --2- 172.21.15.80:0/962881802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00d2ad0 0x7f50d00b76d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 --2- 172.21.15.80:0/962881802 >> v2:172.21.15.87:3300/0 conn(0x7f50d00d3ed0 0x7f50d00d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 -- 172.21.15.80:0/962881802 >> 172.21.15.80:0/962881802 conn(0x7f50d00737b0 msgr2=0x7f50d006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:06.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 -- 172.21.15.80:0/962881802 shutdown_connections 2024-04-25T13:41:06.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 -- 172.21.15.80:0/962881802 wait complete. 2024-04-25T13:41:06.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 Processor -- start 2024-04-25T13:41:06.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.679+0000 7f50d5816640 1 -- start start 2024-04-25T13:41:06.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00b7c10 0x7f50d01500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d00d2ad0 0x7f50d0158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d00d3ed0 0x7f50d01f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f50d00ccae0 con 0x7f50d00b7c10 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f50d00cc960 con 0x7f50d00d3ed0 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f50d0160620 con 0x7f50d00d2ad0 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00b7c10 0x7f50d01500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00b7c10 0x7f50d01500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59466/0 (socket says 172.21.15.80:59466) 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 -- 172.21.15.80:0/1508204095 learned_addr learned my addr 172.21.15.80:0/1508204095 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ce7fc640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d00d2ad0 0x7f50d0158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50cf7fe640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d00d3ed0 0x7f50d01f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 -- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d00d2ad0 msgr2=0x7f50d0158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d00d2ad0 0x7f50d0158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 -- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d00d3ed0 msgr2=0x7f50d01f99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d00d3ed0 0x7f50d01f99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 -- 172.21.15.80:0/1508204095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d01562c0 con 0x7f50d00b7c10 2024-04-25T13:41:06.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50ceffd640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00b7c10 0x7f50d01500f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c820 tx=0x7f50c000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50c7fff640 1 -- 172.21.15.80:0/1508204095 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0015070 con 0x7f50d00b7c10 2024-04-25T13:41:06.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50c7fff640 1 -- 172.21.15.80:0/1508204095 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50c0007a30 con 0x7f50d00b7c10 2024-04-25T13:41:06.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d01565a0 con 0x7f50d00b7c10 2024-04-25T13:41:06.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50c7fff640 1 -- 172.21.15.80:0/1508204095 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0007e20 con 0x7f50d00b7c10 2024-04-25T13:41:06.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d0156b60 con 0x7f50d00b7c10 2024-04-25T13:41:06.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d006a3a0 con 0x7f50d00b7c10 2024-04-25T13:41:06.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50c7fff640 1 -- 172.21.15.80:0/1508204095 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f50c0018070 con 0x7f50d00b7c10 2024-04-25T13:41:06.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50c7fff640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50b006f430 0x7f50b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:06.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.683+0000 7f50c7fff640 1 -- 172.21.15.80:0/1508204095 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f50c008f0b0 con 0x7f50d00b7c10 2024-04-25T13:41:06.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.687+0000 7f50ce7fc640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50b006f430 0x7f50b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:06.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.687+0000 7f50ce7fc640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50b006f430 0x7f50b0071940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f50b80ad000 tx=0x7f50b80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:06.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.691+0000 7f50c7fff640 1 -- 172.21.15.80:0/1508204095 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f50c005c030 con 0x7f50d00b7c10 2024-04-25T13:41:06.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f50d00cc500 con 0x7f50d00b7c10 2024-04-25T13:41:06.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50c7fff640 1 -- 172.21.15.80:0/1508204095 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f50c005b930 con 0x7f50d00b7c10 2024-04-25T13:41:06.840 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:06.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50b006f430 msgr2=0x7f50b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50b006f430 0x7f50b0071940 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f50b80ad000 tx=0x7f50b80a6000 comp rx=0 tx=0).stop 2024-04-25T13:41:06.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00b7c10 msgr2=0x7f50d01500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00b7c10 0x7f50d01500f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f50c000c820 tx=0x7f50c000cce0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 shutdown_connections 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50b006f430 0x7f50b0071940 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50d00d3ed0 0x7f50d01f99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50d00d2ad0 0x7f50d0158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 --2- 172.21.15.80:0/1508204095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50d00b7c10 0x7f50d01500f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 >> 172.21.15.80:0/1508204095 conn(0x7f50d00737b0 msgr2=0x7f50d00cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 shutdown_connections 2024-04-25T13:41:06.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:06.839+0000 7f50d5816640 1 -- 172.21.15.80:0/1508204095 wait complete. 2024-04-25T13:41:06.861 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:07.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.011+0000 7f1337544640 1 Processor -- start 2024-04-25T13:41:07.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.011+0000 7f1337544640 1 -- start start 2024-04-25T13:41:07.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1337544640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13300d2ca0 0x7f13300b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1337544640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1337544640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13300b92a0 con 0x7f13300d3ed0 2024-04-25T13:41:07.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1337544640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f13300d4c10 con 0x7f13300d2ca0 2024-04-25T13:41:07.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1337544640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13300d4ef0 con 0x7f13300b9ca0 2024-04-25T13:41:07.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1334adf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13300d2ca0 0x7f13300b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1335ae1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f13352e0640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f13300d3ed0 0x7f13300d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:51656/0 (socket says 172.21.15.80:51656) 2024-04-25T13:41:07.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f13352e0640 1 -- 172.21.15.80:0/2181310641 learned_addr learned my addr 172.21.15.80:0/2181310641 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:07.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1334adf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13300d2ca0 0x7f13300b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:52012/0 (socket says 172.21.15.80:52012) 2024-04-25T13:41:07.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1335ae1640 1 -- 172.21.15.80:0/2181310641 >> v2:172.21.15.87:3300/0 conn(0x7f13300d2ca0 msgr2=0x7f13300b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:07.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1335ae1640 1 --2- 172.21.15.80:0/2181310641 >> v2:172.21.15.87:3300/0 conn(0x7f13300d2ca0 0x7f13300b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1335ae1640 1 -- 172.21.15.80:0/2181310641 >> v1:172.21.15.87:6789/0 conn(0x7f13300d3ed0 legacy=0x7f13300d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.015+0000 7f1335ae1640 1 -- 172.21.15.80:0/2181310641 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13300d51d0 con 0x7f13300b9ca0 2024-04-25T13:41:07.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1335ae1640 1 --2- 172.21.15.80:0/2181310641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300ba0b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1328009d90 tx=0x7f132801c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca8bf1d02936405 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1327fff640 1 -- 172.21.15.80:0/2181310641 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280b2070 con 0x7f13300b9ca0 2024-04-25T13:41:07.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1327fff640 1 -- 172.21.15.80:0/2181310641 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13280a5910 con 0x7f13300b9ca0 2024-04-25T13:41:07.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1327fff640 1 -- 172.21.15.80:0/2181310641 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280a5ce0 con 0x7f13300b9ca0 2024-04-25T13:41:07.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1337544640 1 -- 172.21.15.80:0/2181310641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 msgr2=0x7f13300ba0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1337544640 1 --2- 172.21.15.80:0/2181310641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300ba0b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1328009d90 tx=0x7f132801c850 comp rx=0 tx=0).stop 2024-04-25T13:41:07.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1337544640 1 -- 172.21.15.80:0/2181310641 shutdown_connections 2024-04-25T13:41:07.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1337544640 1 --2- 172.21.15.80:0/2181310641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300ba0b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1337544640 1 --2- 172.21.15.80:0/2181310641 >> v2:172.21.15.87:3300/0 conn(0x7f13300d2ca0 0x7f13300b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1337544640 1 -- 172.21.15.80:0/2181310641 >> 172.21.15.80:0/2181310641 conn(0x7f13300737b0 msgr2=0x7f133006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:07.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1337544640 1 -- 172.21.15.80:0/2181310641 shutdown_connections 2024-04-25T13:41:07.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.019+0000 7f1337544640 1 -- 172.21.15.80:0/2181310641 wait complete. 2024-04-25T13:41:07.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 Processor -- start 2024-04-25T13:41:07.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 -- start start 2024-04-25T13:41:07.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300d2ca0 0x7f13301530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300d3ed0 0x7f133014f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1330160620 con 0x7f13300d2ca0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13301607a0 con 0x7f13300d3ed0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f133016d290 con 0x7f13300b9ca0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300d2ca0 0x7f13301530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300d2ca0 0x7f13301530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59474/0 (socket says 172.21.15.80:59474) 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 -- 172.21.15.80:0/2457164191 learned_addr learned my addr 172.21.15.80:0/2457164191 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f13352e0640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1335ae1640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300d3ed0 0x7f133014f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 -- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 msgr2=0x7f13300c85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300c85c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 -- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300d3ed0 msgr2=0x7f133014f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300d3ed0 0x7f133014f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 -- 172.21.15.80:0/2457164191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13300cda80 con 0x7f13300d2ca0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f13352e0640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1334adf640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300d2ca0 0x7f13301530c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f132000e870 tx=0x7f132000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1325ffb640 1 -- 172.21.15.80:0/2457164191 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320017070 con 0x7f13300d2ca0 2024-04-25T13:41:07.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1325ffb640 1 -- 172.21.15.80:0/2457164191 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132000c8e0 con 0x7f13300d2ca0 2024-04-25T13:41:07.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13300cdd60 con 0x7f13300d2ca0 2024-04-25T13:41:07.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1325ffb640 1 -- 172.21.15.80:0/2457164191 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132000cc90 con 0x7f13300d2ca0 2024-04-25T13:41:07.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.023+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13300ce320 con 0x7f13300d2ca0 2024-04-25T13:41:07.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.027+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133006a3a0 con 0x7f13300d2ca0 2024-04-25T13:41:07.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.027+0000 7f1325ffb640 1 -- 172.21.15.80:0/2457164191 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f132001a070 con 0x7f13300d2ca0 2024-04-25T13:41:07.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.027+0000 7f1325ffb640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.027+0000 7f1325ffb640 1 -- 172.21.15.80:0/2457164191 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f132008f4f0 con 0x7f13300d2ca0 2024-04-25T13:41:07.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.027+0000 7f13352e0640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.027+0000 7f13352e0640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f132c000f70 tx=0x7f132c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.031+0000 7f1325ffb640 1 -- 172.21.15.80:0/2457164191 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f132005c3e0 con 0x7f13300d2ca0 2024-04-25T13:41:07.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.191+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f13300caa60 con 0x7f13300d2ca0 2024-04-25T13:41:07.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1325ffb640 1 -- 172.21.15.80:0/2457164191 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f132005bce0 con 0x7f13300d2ca0 2024-04-25T13:41:07.195 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:07.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 msgr2=0x7f1314071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f132c000f70 tx=0x7f132c009040 comp rx=0 tx=0).stop 2024-04-25T13:41:07.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300d2ca0 msgr2=0x7f13301530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300d2ca0 0x7f13301530c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f132000e870 tx=0x7f132000ec70 comp rx=0 tx=0).stop 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 shutdown_connections 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300d3ed0 0x7f133014f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300d2ca0 0x7f13301530c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 --2- 172.21.15.80:0/2457164191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b9ca0 0x7f13300c85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 >> 172.21.15.80:0/2457164191 conn(0x7f13300737b0 msgr2=0x7f13300c9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 shutdown_connections 2024-04-25T13:41:07.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.195+0000 7f1337544640 1 -- 172.21.15.80:0/2457164191 wait complete. 2024-04-25T13:41:07.218 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:07.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe821de4640 1 Processor -- start 2024-04-25T13:41:07.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe821de4640 1 -- start start 2024-04-25T13:41:07.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe821de4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d2e80 0x7fe81c1607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe821de4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe81c0d4330 0x7fe81c0d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe821de4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe81c161060 con 0x7fe81c0d59b0 2024-04-25T13:41:07.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe821de4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe81c161340 con 0x7fe81c0d4330 2024-04-25T13:41:07.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe821de4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe81c0d2680 con 0x7fe81c0d2e80 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe81bfff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe81c0d4330 0x7fe81c0d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d2e80 0x7fe81c1607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe81bfff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe81c0d4330 0x7fe81c0d4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:52040/0 (socket says 172.21.15.80:52040) 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe81affd640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe81c0d59b0 0x7fe81c0d5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:51670/0 (socket says 172.21.15.80:51670) 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d2e80 0x7fe81c1607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:60134/0 (socket says 172.21.15.80:60134) 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.363+0000 7fe81bfff640 1 -- 172.21.15.80:0/1862328991 learned_addr learned my addr 172.21.15.80:0/1862328991 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.367+0000 7fe81bfff640 1 -- 172.21.15.80:0/1862328991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d2e80 msgr2=0x7fe81c1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.367+0000 7fe81bfff640 1 --2- 172.21.15.80:0/1862328991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d2e80 0x7fe81c1607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.367+0000 7fe81bfff640 1 -- 172.21.15.80:0/1862328991 >> v1:172.21.15.87:6789/0 conn(0x7fe81c0d59b0 legacy=0x7fe81c0d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.367+0000 7fe81bfff640 1 -- 172.21.15.80:0/1862328991 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c0d2960 con 0x7fe81c0d4330 2024-04-25T13:41:07.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.367+0000 7fe81bfff640 1 --2- 172.21.15.80:0/1862328991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 0x7fe81c0d4740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009870 tx=0x7fe80c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f014bfba841fa3cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.371+0000 7fe81a7fc640 1 -- 172.21.15.80:0/1862328991 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c0aa650 con 0x7fe81c0d4330 2024-04-25T13:41:07.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.371+0000 7fe81a7fc640 1 -- 172.21.15.80:0/1862328991 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe80c0af070 con 0x7fe81c0d4330 2024-04-25T13:41:07.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.371+0000 7fe81a7fc640 1 -- 172.21.15.80:0/1862328991 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe80c0aaa10 con 0x7fe81c0d4330 2024-04-25T13:41:07.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.371+0000 7fe821de4640 1 -- 172.21.15.80:0/1862328991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 msgr2=0x7fe81c0d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.371+0000 7fe821de4640 1 --2- 172.21.15.80:0/1862328991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 0x7fe81c0d4740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009870 tx=0x7fe80c01c4d0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.371+0000 7fe821de4640 1 -- 172.21.15.80:0/1862328991 shutdown_connections 2024-04-25T13:41:07.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.371+0000 7fe821de4640 1 --2- 172.21.15.80:0/1862328991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 0x7fe81c0d4740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.371+0000 7fe821de4640 1 --2- 172.21.15.80:0/1862328991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d2e80 0x7fe81c1607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 -- 172.21.15.80:0/1862328991 >> 172.21.15.80:0/1862328991 conn(0x7fe81c0737a0 msgr2=0x7fe81c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:07.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 -- 172.21.15.80:0/1862328991 shutdown_connections 2024-04-25T13:41:07.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 -- 172.21.15.80:0/1862328991 wait complete. 2024-04-25T13:41:07.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 Processor -- start 2024-04-25T13:41:07.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 -- start start 2024-04-25T13:41:07.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe81c0d2e80 0x7fe81c16a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 0x7fe81c1664a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d59b0 0x7fe81c164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c0d3410 con 0x7fe81c0d2e80 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe81c0d3290 con 0x7fe81c0d4330 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe81c15df60 con 0x7fe81c0d59b0 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe81c0d2e80 0x7fe81c16a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe81c0d2e80 0x7fe81c16a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59490/0 (socket says 172.21.15.80:59490) 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 -- 172.21.15.80:0/3745370832 learned_addr learned my addr 172.21.15.80:0/3745370832 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81affd640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 0x7fe81c1664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81bfff640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d59b0 0x7fe81c164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 -- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d59b0 msgr2=0x7fe81c164ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d59b0 0x7fe81c164ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 -- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 msgr2=0x7fe81c1664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 0x7fe81c1664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 -- 172.21.15.80:0/3745370832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe81c167210 con 0x7fe81c0d2e80 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81bfff640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d59b0 0x7fe81c164ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe81b7fe640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe81c0d2e80 0x7fe81c16a1b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe81400e870 tx=0x7fe81400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe813fff640 1 -- 172.21.15.80:0/3745370832 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe814017070 con 0x7fe81c0d2e80 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe813fff640 1 -- 172.21.15.80:0/3745370832 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe81400c940 con 0x7fe81c0d2e80 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe81c1674f0 con 0x7fe81c0d2e80 2024-04-25T13:41:07.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.375+0000 7fe813fff640 1 -- 172.21.15.80:0/3745370832 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81400cd10 con 0x7fe81c0d2e80 2024-04-25T13:41:07.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.379+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe81c0b9220 con 0x7fe81c0d2e80 2024-04-25T13:41:07.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.379+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe81c06a3a0 con 0x7fe81c0d2e80 2024-04-25T13:41:07.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.379+0000 7fe813fff640 1 -- 172.21.15.80:0/3745370832 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe81401a070 con 0x7fe81c0d2e80 2024-04-25T13:41:07.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.379+0000 7fe813fff640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7fc06f400 0x7fe7fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.379+0000 7fe813fff640 1 -- 172.21.15.80:0/3745370832 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fe81408f500 con 0x7fe81c0d2e80 2024-04-25T13:41:07.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.379+0000 7fe81affd640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7fc06f400 0x7fe7fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.379+0000 7fe81affd640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7fc06f400 0x7fe7fc071910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe804005fd0 tx=0x7fe804005b80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.383+0000 7fe813fff640 1 -- 172.21.15.80:0/3745370832 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe81405c400 con 0x7fe81c0d2e80 2024-04-25T13:41:07.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe81c0bbb90 con 0x7fe81c0d2e80 2024-04-25T13:41:07.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe813fff640 1 -- 172.21.15.80:0/3745370832 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7fe81405bd00 con 0x7fe81c0d2e80 2024-04-25T13:41:07.544 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:07.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7fc06f400 msgr2=0x7fe7fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7fc06f400 0x7fe7fc071910 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe804005fd0 tx=0x7fe804005b80 comp rx=0 tx=0).stop 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe81c0d2e80 msgr2=0x7fe81c16a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe81c0d2e80 0x7fe81c16a1b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe81400e870 tx=0x7fe81400ec70 comp rx=0 tx=0).stop 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 shutdown_connections 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7fc06f400 0x7fe7fc071910 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe81c0d59b0 0x7fe81c164ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe81c0d4330 0x7fe81c1664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 --2- 172.21.15.80:0/3745370832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe81c0d2e80 0x7fe81c16a1b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 >> 172.21.15.80:0/3745370832 conn(0x7fe81c0737a0 msgr2=0x7fe81c0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 shutdown_connections 2024-04-25T13:41:07.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.543+0000 7fe821de4640 1 -- 172.21.15.80:0/3745370832 wait complete. 2024-04-25T13:41:07.565 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:41:07.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:07.565 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:07.590 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:41:07.590 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:41:07.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:07.590 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:41:07.640 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:07.640 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:07.647 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:07.647 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:07.696 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:07.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec63bb0640 1 Processor -- start 2024-04-25T13:41:07.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec63bb0640 1 -- start start 2024-04-25T13:41:07.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec63bb0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c161f90 0x7fec5c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec63bb0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec5c0d0510 0x7fec5c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec63bb0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fec5c15ea70 0x7fec5c15ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec63bb0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fec5c0d0d70 con 0x7fec5c15ea70 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec63bb0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fec5c0d1050 con 0x7fec5c0d0510 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec63bb0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec5c06dca0 con 0x7fec5c161f90 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec6214d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fec5c15ea70 0x7fec5c15ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.843+0000 7fec6214d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fec5c15ea70 0x7fec5c15ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59494/0 (socket says 172.21.15.80:59494) 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6214d640 1 -- 172.21.15.80:0/1670657176 learned_addr learned my addr 172.21.15.80:0/1670657176 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6114b640 1 --2- 172.21.15.80:0/1670657176 >> v2:172.21.15.87:3300/0 conn(0x7fec5c0d0510 0x7fec5c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6194c640 1 --2- 172.21.15.80:0/1670657176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c161f90 0x7fec5c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6214d640 1 -- 172.21.15.80:0/1670657176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c161f90 msgr2=0x7fec5c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6214d640 1 --2- 172.21.15.80:0/1670657176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c161f90 0x7fec5c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6214d640 1 -- 172.21.15.80:0/1670657176 >> v2:172.21.15.87:3300/0 conn(0x7fec5c0d0510 msgr2=0x7fec5c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6214d640 1 --2- 172.21.15.80:0/1670657176 >> v2:172.21.15.87:3300/0 conn(0x7fec5c0d0510 0x7fec5c15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6214d640 1 -- 172.21.15.80:0/1670657176 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c06df80 con 0x7fec5c15ea70 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6114b640 1 --2- 172.21.15.80:0/1670657176 >> v2:172.21.15.87:3300/0 conn(0x7fec5c0d0510 0x7fec5c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:07.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6194c640 1 --2- 172.21.15.80:0/1670657176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c161f90 0x7fec5c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:07.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6214d640 1 --2- 172.21.15.80:0/1670657176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c15ee80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0d1260 tx=0x7fec5401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4497dda60a5350f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6094a640 1 -- 172.21.15.80:0/1670657176 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec540a5770 con 0x7fec5c15ea70 2024-04-25T13:41:07.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6094a640 1 -- 172.21.15.80:0/1670657176 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec540a5910 con 0x7fec5c15ea70 2024-04-25T13:41:07.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec6094a640 1 -- 172.21.15.80:0/1670657176 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec540a5ce0 con 0x7fec5c15ea70 2024-04-25T13:41:07.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 -- 172.21.15.80:0/1670657176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 msgr2=0x7fec5c15ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 --2- 172.21.15.80:0/1670657176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c15ee80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fec5c0d1260 tx=0x7fec5401c930 comp rx=0 tx=0).stop 2024-04-25T13:41:07.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 -- 172.21.15.80:0/1670657176 shutdown_connections 2024-04-25T13:41:07.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 --2- 172.21.15.80:0/1670657176 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c15ee80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 --2- 172.21.15.80:0/1670657176 >> v2:172.21.15.87:3300/0 conn(0x7fec5c0d0510 0x7fec5c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 --2- 172.21.15.80:0/1670657176 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c161f90 0x7fec5c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 -- 172.21.15.80:0/1670657176 >> 172.21.15.80:0/1670657176 conn(0x7fec5c06d080 msgr2=0x7fec5c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:07.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 -- 172.21.15.80:0/1670657176 shutdown_connections 2024-04-25T13:41:07.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.847+0000 7fec63bb0640 1 -- 172.21.15.80:0/1670657176 wait complete. 2024-04-25T13:41:07.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 Processor -- start 2024-04-25T13:41:07.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 -- start start 2024-04-25T13:41:07.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c0d0510 0x7fec5c141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c142c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec5c161f90 0x7fec5c144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c1447a0 con 0x7fec5c15ea70 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec5c144620 con 0x7fec5c161f90 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec5c0d0920 con 0x7fec5c0d0510 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c142c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c142c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59500/0 (socket says 172.21.15.80:59500) 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 -- 172.21.15.80:0/366038218 learned_addr learned my addr 172.21.15.80:0/366038218 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6194c640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c0d0510 0x7fec5c141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6214d640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec5c161f90 0x7fec5c144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 -- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c0d0510 msgr2=0x7fec5c141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c0d0510 0x7fec5c141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 -- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec5c161f90 msgr2=0x7fec5c144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec5c161f90 0x7fec5c144920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:07.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 -- 172.21.15.80:0/366038218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5c144e60 con 0x7fec5c15ea70 2024-04-25T13:41:07.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec6114b640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c142c30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c5a0 tx=0x7fec5800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec527fc640 1 -- 172.21.15.80:0/366038218 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58015070 con 0x7fec5c15ea70 2024-04-25T13:41:07.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec527fc640 1 -- 172.21.15.80:0/366038218 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec58004220 con 0x7fec5c15ea70 2024-04-25T13:41:07.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5c1fafb0 con 0x7fec5c15ea70 2024-04-25T13:41:07.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec527fc640 1 -- 172.21.15.80:0/366038218 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58003620 con 0x7fec5c15ea70 2024-04-25T13:41:07.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.851+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec5c1fb570 con 0x7fec5c15ea70 2024-04-25T13:41:07.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.855+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5c1fad20 con 0x7fec5c15ea70 2024-04-25T13:41:07.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.855+0000 7fec527fc640 1 -- 172.21.15.80:0/366038218 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fec580037c0 con 0x7fec5c15ea70 2024-04-25T13:41:07.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.855+0000 7fec527fc640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4006f400 0x7fec40071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:07.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.855+0000 7fec527fc640 1 -- 172.21.15.80:0/366038218 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fec5808efa0 con 0x7fec5c15ea70 2024-04-25T13:41:07.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.855+0000 7fec6194c640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4006f400 0x7fec40071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:07.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.855+0000 7fec6194c640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4006f400 0x7fec40071910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec4c006fd0 tx=0x7fec4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:07.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:07.859+0000 7fec527fc640 1 -- 172.21.15.80:0/366038218 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fec5805be90 con 0x7fec5c15ea70 2024-04-25T13:41:08.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.023+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec5c06a3a0 con 0x7fec5c15ea70 2024-04-25T13:41:08.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.023+0000 7fec527fc640 1 -- 172.21.15.80:0/366038218 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7fec5805b790 con 0x7fec5c15ea70 2024-04-25T13:41:08.027 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:08.027 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:08.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4006f400 msgr2=0x7fec40071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4006f400 0x7fec40071910 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fec4c006fd0 tx=0x7fec4c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:08.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 msgr2=0x7fec5c142c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c142c30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c5a0 tx=0x7fec5800ca60 comp rx=0 tx=0).stop 2024-04-25T13:41:08.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 shutdown_connections 2024-04-25T13:41:08.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4006f400 0x7fec40071910 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec5c161f90 0x7fec5c144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec5c15ea70 0x7fec5c142c30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 --2- 172.21.15.80:0/366038218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec5c0d0510 0x7fec5c141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 >> 172.21.15.80:0/366038218 conn(0x7fec5c06d080 msgr2=0x7fec5c0ce260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 shutdown_connections 2024-04-25T13:41:08.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.027+0000 7fec63bb0640 1 -- 172.21.15.80:0/366038218 wait complete. 2024-04-25T13:41:08.050 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:08.050 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:08.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0eedd4b640 1 Processor -- start 2024-04-25T13:41:08.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0eedd4b640 1 -- start start 2024-04-25T13:41:08.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0eedd4b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0ee80b08b0 0x7f0ee81655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0eedd4b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ee806dbe0 0x7f0ee806dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0eedd4b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ee80b0cc0 con 0x7f0ee80b4bd0 2024-04-25T13:41:08.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0eedd4b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0ee80b0f40 con 0x7f0ee806dbe0 2024-04-25T13:41:08.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0eedd4b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0ee80b1220 con 0x7f0ee80b08b0 2024-04-25T13:41:08.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0ee6ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ee806dbe0 0x7f0ee806dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0ee6ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ee806dbe0 0x7f0ee806dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59502/0 (socket says 172.21.15.80:59502) 2024-04-25T13:41:08.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0ee6ffd640 1 -- 172.21.15.80:0/1753611512 learned_addr learned my addr 172.21.15.80:0/1753611512 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:08.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0ee77fe640 1 --2- 172.21.15.80:0/1753611512 >> v2:172.21.15.87:3300/0 conn(0x7f0ee80b08b0 0x7f0ee81655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0ee6ffd640 1 -- 172.21.15.80:0/1753611512 >> v2:172.21.15.87:3300/0 conn(0x7f0ee80b08b0 msgr2=0x7f0ee81655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:08.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0ee6ffd640 1 --2- 172.21.15.80:0/1753611512 >> v2:172.21.15.87:3300/0 conn(0x7f0ee80b08b0 0x7f0ee81655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0ee6ffd640 1 -- 172.21.15.80:0/1753611512 >> v1:172.21.15.87:6789/0 conn(0x7f0ee80b4bd0 legacy=0x7f0ee80b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:08.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.207+0000 7f0ee6ffd640 1 -- 172.21.15.80:0/1753611512 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee80726a0 con 0x7f0ee806dbe0 2024-04-25T13:41:08.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0ee6ffd640 1 --2- 172.21.15.80:0/1753611512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee806dff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80d22f0 tx=0x7f0edc01c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a956603429c613 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0ee67fc640 1 -- 172.21.15.80:0/1753611512 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0b0070 con 0x7f0ee806dbe0 2024-04-25T13:41:08.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0ee67fc640 1 -- 172.21.15.80:0/1753611512 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0edc0ab600 con 0x7f0ee806dbe0 2024-04-25T13:41:08.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0ee67fc640 1 -- 172.21.15.80:0/1753611512 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0edc0ab9d0 con 0x7f0ee806dbe0 2024-04-25T13:41:08.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0eedd4b640 1 -- 172.21.15.80:0/1753611512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 msgr2=0x7f0ee806dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/1753611512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee806dff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0ee80d22f0 tx=0x7f0edc01c300 comp rx=0 tx=0).stop 2024-04-25T13:41:08.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0eedd4b640 1 -- 172.21.15.80:0/1753611512 shutdown_connections 2024-04-25T13:41:08.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/1753611512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee806dff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/1753611512 >> v2:172.21.15.87:3300/0 conn(0x7f0ee80b08b0 0x7f0ee81655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0eedd4b640 1 -- 172.21.15.80:0/1753611512 >> 172.21.15.80:0/1753611512 conn(0x7f0ee80b4160 msgr2=0x7f0ee80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0eedd4b640 1 -- 172.21.15.80:0/1753611512 shutdown_connections 2024-04-25T13:41:08.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.211+0000 7f0eedd4b640 1 -- 172.21.15.80:0/1753611512 wait complete. 2024-04-25T13:41:08.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 Processor -- start 2024-04-25T13:41:08.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 -- start start 2024-04-25T13:41:08.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee8200fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ee80b08b0 0x7f0ee8163df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee80b4bd0 0x7f0ee80b1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8171880 con 0x7f0ee806dbe0 2024-04-25T13:41:08.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0ee8171700 con 0x7f0ee80b4bd0 2024-04-25T13:41:08.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0ee8171a00 con 0x7f0ee80b08b0 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee8200fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee8200fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59512/0 (socket says 172.21.15.80:59512) 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 -- 172.21.15.80:0/2263134138 learned_addr learned my addr 172.21.15.80:0/2263134138 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 -- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ee80b08b0 msgr2=0x7f0ee8163df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ee80b08b0 0x7f0ee8163df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 -- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee80b4bd0 msgr2=0x7f0ee80b1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee7fff640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee80b4bd0 0x7f0ee80b1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee80b4bd0 0x7f0ee80b1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 -- 172.21.15.80:0/2263134138 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ee8201c40 con 0x7f0ee806dbe0 2024-04-25T13:41:08.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ee77fe640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee8200fe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800d5d0 tx=0x7f0ed800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ecffff640 1 -- 172.21.15.80:0/2263134138 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8004240 con 0x7f0ee806dbe0 2024-04-25T13:41:08.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ecffff640 1 -- 172.21.15.80:0/2263134138 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ed80043e0 con 0x7f0ee806dbe0 2024-04-25T13:41:08.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ee8201f20 con 0x7f0ee806dbe0 2024-04-25T13:41:08.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0ecffff640 1 -- 172.21.15.80:0/2263134138 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ed8003840 con 0x7f0ee806dbe0 2024-04-25T13:41:08.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ee82024e0 con 0x7f0ee806dbe0 2024-04-25T13:41:08.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.215+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ee806a3a0 con 0x7f0ee806dbe0 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.219+0000 7f0ecffff640 1 -- 172.21.15.80:0/2263134138 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0ed80039e0 con 0x7f0ee806dbe0 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.219+0000 7f0ecffff640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ed406f400 0x7f0ed4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.219+0000 7f0ecffff640 1 -- 172.21.15.80:0/2263134138 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0ed808f580 con 0x7f0ee806dbe0 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.219+0000 7f0ee6ffd640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ed406f400 0x7f0ed4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.219+0000 7f0ee6ffd640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ed406f400 0x7f0ed4071910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0ad000 tx=0x7f0edc004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.223+0000 7f0ecffff640 1 -- 172.21.15.80:0/2263134138 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ed805c470 con 0x7f0ee806dbe0 2024-04-25T13:41:08.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.383+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ee815c070 con 0x7f0ee806dbe0 2024-04-25T13:41:08.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0ecffff640 1 -- 172.21.15.80:0/2263134138 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f0ed805bd70 con 0x7f0ee806dbe0 2024-04-25T13:41:08.387 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ed406f400 msgr2=0x7f0ed4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ed406f400 0x7f0ed4071910 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0edc0ad000 tx=0x7f0edc004740 comp rx=0 tx=0).stop 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 msgr2=0x7f0ee8200fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee8200fe0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0ed800d5d0 tx=0x7f0ed800da90 comp rx=0 tx=0).stop 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 shutdown_connections 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ed406f400 0x7f0ed4071910 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ee80b4bd0 0x7f0ee80b1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ee80b08b0 0x7f0ee8163df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 --2- 172.21.15.80:0/2263134138 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ee806dbe0 0x7f0ee8200fe0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 >> 172.21.15.80:0/2263134138 conn(0x7f0ee80b4160 msgr2=0x7f0ee806f110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 shutdown_connections 2024-04-25T13:41:08.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.387+0000 7f0eedd4b640 1 -- 172.21.15.80:0/2263134138 wait complete. 2024-04-25T13:41:08.409 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:08.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfe655640 1 Processor -- start 2024-04-25T13:41:08.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfe655640 1 -- start start 2024-04-25T13:41:08.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfe655640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 0x7fbbf80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfe655640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf80b5740 con 0x7fbbf80d0ff0 2024-04-25T13:41:08.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfe655640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbbf806dd10 con 0x7fbbf8161b60 2024-04-25T13:41:08.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfe655640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf81629b0 con 0x7fbbf80b5330 2024-04-25T13:41:08.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbef7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fbbf80d0ff0 0x7fbbf806edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41850/0 (socket says 172.21.15.80:41850) 2024-04-25T13:41:08.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbef7fe640 1 -- 172.21.15.80:0/3079939104 learned_addr learned my addr 172.21.15.80:0/3079939104 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf7fff640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 0x7fbbf80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2740899736 0 0) 0x7fbbf80b5740 con 0x7fbbf80d0ff0 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbe4003530 con 0x7fbbf80d0ff0 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 628924310 0 0) 0x7fbbe4003530 con 0x7fbbf80d0ff0 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 msgr2=0x7fbbf80d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 0x7fbbf80d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 >> v1:172.21.15.87:6789/0 conn(0x7fbbf8161b60 legacy=0x7fbbf8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf806dff0 con 0x7fbbf80d0ff0 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fbbe0001680 con 0x7fbbf80d0ff0 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 >> v1:172.21.15.80:6789/0 conn(0x7fbbf80d0ff0 legacy=0x7fbbf806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbe40039f0 0x7fbbe4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 0x7fbbe4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbe4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81629b0 con 0x7fbbe4004d50 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe400bb90 con 0x7fbbe40039f0 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbbe400bd90 con 0x7fbbf80b5330 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfcbf2640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbe4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbef7fe640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 0x7fbbe4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbf7fff640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbe40039f0 0x7fbbe4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfcbf2640 1 -- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 msgr2=0x7fbbe4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfcbf2640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 0x7fbbe4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfcbf2640 1 -- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbe40039f0 msgr2=0x7fbbe4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfcbf2640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbe40039f0 0x7fbbe4003e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbfcbf2640 1 -- 172.21.15.80:0/3079939104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe4003530 con 0x7fbbe4004d50 2024-04-25T13:41:08.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.555+0000 7fbbef7fe640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80b5330 0x7fbbe4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbfcbf2640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbe4005160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0001e40 tx=0x7fbbf001c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a4871406aea1af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0001a90 con 0x7fbbe4004d50 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf0003b80 con 0x7fbbe4004d50 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbf77fe640 1 -- 172.21.15.80:0/3079939104 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0001d50 con 0x7fbbe4004d50 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbfe655640 1 -- 172.21.15.80:0/3079939104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 msgr2=0x7fbbe4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbfe655640 1 --2- 172.21.15.80:0/3079939104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbe4005160 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0001e40 tx=0x7fbbf001c760 comp rx=0 tx=0).stop 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbfcbf2640 1 -- 172.21.15.80:0/3079939104 reap_dead start 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbfe655640 1 -- 172.21.15.80:0/3079939104 shutdown_connections 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbfe655640 1 -- 172.21.15.80:0/3079939104 >> 172.21.15.80:0/3079939104 conn(0x7fbbf80b4160 msgr2=0x7fbbf80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbfe655640 1 -- 172.21.15.80:0/3079939104 shutdown_connections 2024-04-25T13:41:08.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.559+0000 7fbbfe655640 1 -- 172.21.15.80:0/3079939104 wait complete. 2024-04-25T13:41:08.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 Processor -- start 2024-04-25T13:41:08.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 -- start start 2024-04-25T13:41:08.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbf815b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf80af790 0x7fbbf8159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80ad260 0x7fbbf80ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf806e890 con 0x7fbbe4004d50 2024-04-25T13:41:08.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf806e5b0 con 0x7fbbf80af790 2024-04-25T13:41:08.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf80b0cd0 con 0x7fbbf80ad260 2024-04-25T13:41:08.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbf815b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbf815b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59536/0 (socket says 172.21.15.80:59536) 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 -- 172.21.15.80:0/1519008410 learned_addr learned my addr 172.21.15.80:0/1519008410 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbef7fe640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf80af790 0x7fbbf8159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfcbf2640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80ad260 0x7fbbf80ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 -- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80ad260 msgr2=0x7fbbf80ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80ad260 0x7fbbf80ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 -- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf80af790 msgr2=0x7fbbf8159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf80af790 0x7fbbf8159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 -- 172.21.15.80:0/1519008410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf80adbb0 con 0x7fbbe4004d50 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfcbf2640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80ad260 0x7fbbf80ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf7fff640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbf815b6c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80108c0 tx=0x7fbbe8010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf57fa640 1 -- 172.21.15.80:0/1519008410 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe800f040 con 0x7fbbe4004d50 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf57fa640 1 -- 172.21.15.80:0/1519008410 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbe800eb60 con 0x7fbbe4004d50 2024-04-25T13:41:08.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbf57fa640 1 -- 172.21.15.80:0/1519008410 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe801c400 con 0x7fbbe4004d50 2024-04-25T13:41:08.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf815a620 con 0x7fbbe4004d50 2024-04-25T13:41:08.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.563+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf815abe0 con 0x7fbbe4004d50 2024-04-25T13:41:08.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.567+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf806a3a0 con 0x7fbbe4004d50 2024-04-25T13:41:08.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.567+0000 7fbbf57fa640 1 -- 172.21.15.80:0/1519008410 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbbe801c5a0 con 0x7fbbe4004d50 2024-04-25T13:41:08.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.567+0000 7fbbf57fa640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbd806f430 0x7fbbd8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.567+0000 7fbbf57fa640 1 -- 172.21.15.80:0/1519008410 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fbbe808f8c0 con 0x7fbbe4004d50 2024-04-25T13:41:08.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.567+0000 7fbbef7fe640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbd806f430 0x7fbbd8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.567+0000 7fbbef7fe640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbd806f430 0x7fbbd8071940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8158580 tx=0x7fbbe0001db0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.571+0000 7fbbf57fa640 1 -- 172.21.15.80:0/1519008410 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbbe8018030 con 0x7fbbe4004d50 2024-04-25T13:41:08.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.719+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbbf806fd00 con 0x7fbbe4004d50 2024-04-25T13:41:08.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.719+0000 7fbbf57fa640 1 -- 172.21.15.80:0/1519008410 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7fbbe805c840 con 0x7fbbe4004d50 2024-04-25T13:41:08.721 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.719+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbd806f430 msgr2=0x7fbbd8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.719+0000 7fbbfe655640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbd806f430 0x7fbbd8071940 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8158580 tx=0x7fbbe0001db0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.719+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 msgr2=0x7fbbf815b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.719+0000 7fbbfe655640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbf815b6c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbbe80108c0 tx=0x7fbbe8010d80 comp rx=0 tx=0).stop 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.723+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 shutdown_connections 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.723+0000 7fbbfe655640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbd806f430 0x7fbbd8071940 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.723+0000 7fbbfe655640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbbf80ad260 0x7fbbf80ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.723+0000 7fbbfe655640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf80af790 0x7fbbf8159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.723+0000 7fbbfe655640 1 --2- 172.21.15.80:0/1519008410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbe4004d50 0x7fbbf815b6c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.723+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 >> 172.21.15.80:0/1519008410 conn(0x7fbbf80b4160 msgr2=0x7fbbf80b66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.723+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 shutdown_connections 2024-04-25T13:41:08.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.723+0000 7fbbfe655640 1 -- 172.21.15.80:0/1519008410 wait complete. 2024-04-25T13:41:08.741 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:08.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34abc44640 1 Processor -- start 2024-04-25T13:41:08.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34abc44640 1 -- start start 2024-04-25T13:41:08.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34abc44640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d3ed0 0x7f34a40d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34abc44640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f34a40ba770 0x7f34a40b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34abc44640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34a40b92a0 con 0x7f34a40d2ad0 2024-04-25T13:41:08.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34abc44640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f34a40d4c10 con 0x7f34a40ba770 2024-04-25T13:41:08.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34abc44640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34a40d4ef0 con 0x7f34a40d3ed0 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34aa1e1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f34a40ba770 0x7f34a40b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34a91df640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f34a40d2ad0 0x7f34a40b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41860/0 (socket says 172.21.15.80:41860) 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34a91df640 1 -- 172.21.15.80:0/1993110474 learned_addr learned my addr 172.21.15.80:0/1993110474 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34a99e0640 1 --2- 172.21.15.80:0/1993110474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d3ed0 0x7f34a40d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34aa1e1640 1 -- 172.21.15.80:0/1993110474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d3ed0 msgr2=0x7f34a40d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34aa1e1640 1 --2- 172.21.15.80:0/1993110474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d3ed0 0x7f34a40d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34aa1e1640 1 -- 172.21.15.80:0/1993110474 >> v1:172.21.15.80:6789/0 conn(0x7f34a40d2ad0 legacy=0x7f34a40b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34aa1e1640 1 -- 172.21.15.80:0/1993110474 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a40d51d0 con 0x7f34a40ba770 2024-04-25T13:41:08.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.879+0000 7f34a89de640 1 -- 172.21.15.80:0/1993110474 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2589830879 0 0) 0x7f3494001120 con 0x7f34a40d2ad0 2024-04-25T13:41:08.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34aa1e1640 1 --2- 172.21.15.80:0/1993110474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40ba770 0x7f34a40b7de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f349c009d90 tx=0x7f349c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247d36709f0df6bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34a89de640 1 -- 172.21.15.80:0/1993110474 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0b2070 con 0x7f34a40ba770 2024-04-25T13:41:08.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34a89de640 1 -- 172.21.15.80:0/1993110474 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c0a59c0 con 0x7f34a40ba770 2024-04-25T13:41:08.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34a89de640 1 -- 172.21.15.80:0/1993110474 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0a5d90 con 0x7f34a40ba770 2024-04-25T13:41:08.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34abc44640 1 -- 172.21.15.80:0/1993110474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40ba770 msgr2=0x7f34a40b7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34abc44640 1 --2- 172.21.15.80:0/1993110474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40ba770 0x7f34a40b7de0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f349c009d90 tx=0x7f349c01c8e0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34abc44640 1 -- 172.21.15.80:0/1993110474 shutdown_connections 2024-04-25T13:41:08.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34abc44640 1 --2- 172.21.15.80:0/1993110474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40ba770 0x7f34a40b7de0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34abc44640 1 --2- 172.21.15.80:0/1993110474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d3ed0 0x7f34a40d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34abc44640 1 -- 172.21.15.80:0/1993110474 >> 172.21.15.80:0/1993110474 conn(0x7f34a40737b0 msgr2=0x7f34a406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:08.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34abc44640 1 -- 172.21.15.80:0/1993110474 shutdown_connections 2024-04-25T13:41:08.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.883+0000 7f34abc44640 1 -- 172.21.15.80:0/1993110474 wait complete. 2024-04-25T13:41:08.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 Processor -- start 2024-04-25T13:41:08.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 -- start start 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34a40ba770 0x7f34a40cda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d2ad0 0x7f34a414ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40d3ed0 0x7f34a414b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f34a40b8320 con 0x7f34a40d3ed0 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f34a40b84a0 con 0x7f34a40ba770 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34a4160620 con 0x7f34a40d2ad0 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40d3ed0 0x7f34a414b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40d3ed0 0x7f34a414b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59550/0 (socket says 172.21.15.80:59550) 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 -- 172.21.15.80:0/1262351326 learned_addr learned my addr 172.21.15.80:0/1262351326 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34a91df640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d2ad0 0x7f34a414ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34a99e0640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34a40ba770 0x7f34a40cda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 -- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d2ad0 msgr2=0x7f34a414ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d2ad0 0x7f34a414ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 -- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34a40ba770 msgr2=0x7f34a40cda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34a40ba770 0x7f34a40cda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 -- 172.21.15.80:0/1262351326 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a40cdfc0 con 0x7f34a40d3ed0 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34aa1e1640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40d3ed0 0x7f34a414b5b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f349c0b0040 tx=0x7f349c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f349a7fc640 1 -- 172.21.15.80:0/1262351326 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0b2050 con 0x7f34a40d3ed0 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f349a7fc640 1 -- 172.21.15.80:0/1262351326 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f349c0be070 con 0x7f34a40d3ed0 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a40ccd00 con 0x7f34a40d3ed0 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f349a7fc640 1 -- 172.21.15.80:0/1262351326 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c0adaf0 con 0x7f34a40d3ed0 2024-04-25T13:41:08.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a40ccfa0 con 0x7f34a40d3ed0 2024-04-25T13:41:08.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.887+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34a406a3a0 con 0x7f34a40d3ed0 2024-04-25T13:41:08.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.891+0000 7f349a7fc640 1 -- 172.21.15.80:0/1262351326 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f349c0adc90 con 0x7f34a40d3ed0 2024-04-25T13:41:08.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.891+0000 7f349a7fc640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f348806f430 0x7f3488071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:08.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.891+0000 7f349a7fc640 1 -- 172.21.15.80:0/1262351326 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f349c129f00 con 0x7f34a40d3ed0 2024-04-25T13:41:08.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.891+0000 7f34a99e0640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f348806f430 0x7f3488071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:08.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.891+0000 7f34a99e0640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f348806f430 0x7f3488071940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34a000b930 tx=0x7f34a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:08.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:08.895+0000 7f349a7fc640 1 -- 172.21.15.80:0/1262351326 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f349c0c7020 con 0x7f34a40d3ed0 2024-04-25T13:41:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.039+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f34a414b350 con 0x7f34a40d3ed0 2024-04-25T13:41:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f349a7fc640 1 -- 172.21.15.80:0/1262351326 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f34a40cdfc0 con 0x7f34a40d3ed0 2024-04-25T13:41:09.044 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f348806f430 msgr2=0x7f3488071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f348806f430 0x7f3488071940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f34a000b930 tx=0x7f34a0008040 comp rx=0 tx=0).stop 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40d3ed0 msgr2=0x7f34a414b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40d3ed0 0x7f34a414b5b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f349c0b0040 tx=0x7f349c019680 comp rx=0 tx=0).stop 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 shutdown_connections 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f348806f430 0x7f3488071940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34a40d3ed0 0x7f34a414b5b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34a40d2ad0 0x7f34a414ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 --2- 172.21.15.80:0/1262351326 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34a40ba770 0x7f34a40cda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 >> 172.21.15.80:0/1262351326 conn(0x7f34a40737b0 msgr2=0x7f34a40b9760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 shutdown_connections 2024-04-25T13:41:09.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.043+0000 7f34abc44640 1 -- 172.21.15.80:0/1262351326 wait complete. 2024-04-25T13:41:09.064 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:09.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec7913640 1 Processor -- start 2024-04-25T13:41:09.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec7913640 1 -- start start 2024-04-25T13:41:09.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec7913640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2ec006f5a0 0x7f2ec0160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec7913640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 0x7f2ec0161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec7913640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ec00b4bd0 0x7f2ec00b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec7913640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec01619c0 con 0x7f2ec00b4bd0 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec7913640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec0073f10 con 0x7f2ec006f5a0 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec7913640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec006fcf0 con 0x7f2ec0161070 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec5eb0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ec00b4bd0 0x7f2ec00b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec5eb0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ec00b4bd0 0x7f2ec00b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59554/0 (socket says 172.21.15.80:59554) 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec5eb0640 1 -- 172.21.15.80:0/4247192130 learned_addr learned my addr 172.21.15.80:0/4247192130 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec4eae640 1 --2- 172.21.15.80:0/4247192130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 0x7f2ec0161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec56af640 1 --2- 172.21.15.80:0/4247192130 >> v2:172.21.15.87:3300/0 conn(0x7f2ec006f5a0 0x7f2ec0160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec5eb0640 1 -- 172.21.15.80:0/4247192130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 msgr2=0x7f2ec0161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec5eb0640 1 --2- 172.21.15.80:0/4247192130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 0x7f2ec0161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec5eb0640 1 -- 172.21.15.80:0/4247192130 >> v2:172.21.15.87:3300/0 conn(0x7f2ec006f5a0 msgr2=0x7f2ec0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:09.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec5eb0640 1 --2- 172.21.15.80:0/4247192130 >> v2:172.21.15.87:3300/0 conn(0x7f2ec006f5a0 0x7f2ec0160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.211+0000 7f2ec5eb0640 1 -- 172.21.15.80:0/4247192130 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec00b0c60 con 0x7f2ec00b4bd0 2024-04-25T13:41:09.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec5eb0640 1 --2- 172.21.15.80:0/4247192130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec00b4bd0 0x7f2ec00b4fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0071e30 tx=0x7f2eb801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf439f4b84233b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2eb7fff640 1 -- 172.21.15.80:0/4247192130 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80a56d0 con 0x7f2ec00b4bd0 2024-04-25T13:41:09.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2eb7fff640 1 -- 172.21.15.80:0/4247192130 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eb80a5870 con 0x7f2ec00b4bd0 2024-04-25T13:41:09.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2eb7fff640 1 -- 172.21.15.80:0/4247192130 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80a5c40 con 0x7f2ec00b4bd0 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 -- 172.21.15.80:0/4247192130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec00b4bd0 msgr2=0x7f2ec00b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 --2- 172.21.15.80:0/4247192130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec00b4bd0 0x7f2ec00b4fe0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0071e30 tx=0x7f2eb801c960 comp rx=0 tx=0).stop 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 -- 172.21.15.80:0/4247192130 shutdown_connections 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 --2- 172.21.15.80:0/4247192130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec00b4bd0 0x7f2ec00b4fe0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 --2- 172.21.15.80:0/4247192130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 0x7f2ec0161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 --2- 172.21.15.80:0/4247192130 >> v2:172.21.15.87:3300/0 conn(0x7f2ec006f5a0 0x7f2ec0160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 -- 172.21.15.80:0/4247192130 >> 172.21.15.80:0/4247192130 conn(0x7f2ec006d080 msgr2=0x7f2ec00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 -- 172.21.15.80:0/4247192130 shutdown_connections 2024-04-25T13:41:09.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.215+0000 7f2ec7913640 1 -- 172.21.15.80:0/4247192130 wait complete. 2024-04-25T13:41:09.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec7913640 1 Processor -- start 2024-04-25T13:41:09.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec7913640 1 -- start start 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec7913640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec006f5a0 0x7f2ec014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec7913640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec00b4bd0 0x7f2ec0157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec7913640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 0x7f2ec0157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec7913640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec0072bb0 con 0x7f2ec006f5a0 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec7913640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec00b5930 con 0x7f2ec00b4bd0 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec7913640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec0072d30 con 0x7f2ec0161070 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec006f5a0 0x7f2ec014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec006f5a0 0x7f2ec014da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59560/0 (socket says 172.21.15.80:59560) 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 -- 172.21.15.80:0/3710288532 learned_addr learned my addr 172.21.15.80:0/3710288532 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:09.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec4eae640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec00b4bd0 0x7f2ec0157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec5eb0640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 0x7f2ec0157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 -- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 msgr2=0x7f2ec0157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 0x7f2ec0157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 -- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec00b4bd0 msgr2=0x7f2ec0157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec00b4bd0 0x7f2ec0157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 -- 172.21.15.80:0/3710288532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec015dfc0 con 0x7f2ec006f5a0 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2ec56af640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec006f5a0 0x7f2ec014da00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00d5f0 tx=0x7f2ebc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2eb5ffb640 1 -- 172.21.15.80:0/3710288532 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc004240 con 0x7f2ec006f5a0 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2eb5ffb640 1 -- 172.21.15.80:0/3710288532 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ebc0043e0 con 0x7f2ec006f5a0 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.219+0000 7f2eb5ffb640 1 -- 172.21.15.80:0/3710288532 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc003620 con 0x7f2ec006f5a0 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.223+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec015e4c0 con 0x7f2ec006f5a0 2024-04-25T13:41:09.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.223+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec015ea80 con 0x7f2ec006f5a0 2024-04-25T13:41:09.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.223+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec0153e30 con 0x7f2ec006f5a0 2024-04-25T13:41:09.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.223+0000 7f2eb5ffb640 1 -- 172.21.15.80:0/3710288532 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ebc01b020 con 0x7f2ec006f5a0 2024-04-25T13:41:09.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.223+0000 7f2eb5ffb640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f400 0x7f2ea4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.223+0000 7f2eb5ffb640 1 -- 172.21.15.80:0/3710288532 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f2ebc08f970 con 0x7f2ec006f5a0 2024-04-25T13:41:09.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.223+0000 7f2ec4eae640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f400 0x7f2ea4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.223+0000 7f2ec4eae640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f400 0x7f2ea4071910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ec015f710 tx=0x7f2eb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.227+0000 7f2eb5ffb640 1 -- 172.21.15.80:0/3710288532 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ebc05ca60 con 0x7f2ec006f5a0 2024-04-25T13:41:09.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.387+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ec006a3a0 con 0x7f2ec006f5a0 2024-04-25T13:41:09.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.387+0000 7f2eb5ffb640 1 -- 172.21.15.80:0/3710288532 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f2ebc05c360 con 0x7f2ec006f5a0 2024-04-25T13:41:09.388 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:09.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f400 msgr2=0x7f2ea4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f400 0x7f2ea4071910 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2ec015f710 tx=0x7f2eb0008040 comp rx=0 tx=0).stop 2024-04-25T13:41:09.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec006f5a0 msgr2=0x7f2ec014da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec006f5a0 0x7f2ec014da00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00d5f0 tx=0x7f2ebc00dab0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 shutdown_connections 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f400 0x7f2ea4071910 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec0161070 0x7f2ec0157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec00b4bd0 0x7f2ec0157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 --2- 172.21.15.80:0/3710288532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec006f5a0 0x7f2ec014da00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 >> 172.21.15.80:0/3710288532 conn(0x7f2ec006d080 msgr2=0x7f2ec00cf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 shutdown_connections 2024-04-25T13:41:09.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.391+0000 7f2ec7913640 1 -- 172.21.15.80:0/3710288532 wait complete. 2024-04-25T13:41:09.409 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:09.409 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:09.416 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:09.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff471df0640 1 Processor -- start 2024-04-25T13:41:09.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff471df0640 1 -- start start 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff471df0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff46c06ebb0 0x7ff46c06efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff471df0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff46c0d0af0 con 0x7ff46c0d06e0 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff471df0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff46c0d0d70 con 0x7ff46c161f90 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff471df0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff46c0d1050 con 0x7ff46c06ebb0 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff462ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff46c0d06e0 0x7ff46c06e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41872/0 (socket says 172.21.15.80:41872) 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff462ffd640 1 -- 172.21.15.80:0/2695756710 learned_addr learned my addr 172.21.15.80:0/2695756710 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609966385 0 0) 0x7ff454001120 con 0x7ff46c0d06e0 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff458003530 con 0x7ff46c0d06e0 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.555+0000 7ff46bfff640 1 --2- 172.21.15.80:0/2695756710 >> v2:172.21.15.87:3300/0 conn(0x7ff46c06ebb0 0x7ff46c06efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1912067730 0 0) 0x7ff458003530 con 0x7ff46c0d06e0 2024-04-25T13:41:09.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 >> v2:172.21.15.87:3300/0 conn(0x7ff46c06ebb0 msgr2=0x7ff46c06efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 --2- 172.21.15.80:0/2695756710 >> v2:172.21.15.87:3300/0 conn(0x7ff46c06ebb0 0x7ff46c06efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 >> v1:172.21.15.87:6789/0 conn(0x7ff46c161f90 legacy=0x7ff46c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c06dca0 con 0x7ff46c0d06e0 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff454001850 con 0x7ff46c0d06e0 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 >> v1:172.21.15.80:6789/0 conn(0x7ff46c0d06e0 legacy=0x7ff46c06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4580039c0 0x7ff458003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458004810 0x7ff458004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff458005360 0x7ff45800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c0d1050 con 0x7ff4580039c0 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff45800c290 con 0x7ff458004810 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff45800c490 con 0x7ff458005360 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46b7fe640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4580039c0 0x7ff458003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff462ffd640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458004810 0x7ff458004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46b7fe640 1 -- 172.21.15.80:0/2695756710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff458005360 msgr2=0x7ff45800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46b7fe640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff458005360 0x7ff45800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46b7fe640 1 -- 172.21.15.80:0/2695756710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458004810 msgr2=0x7ff458004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46b7fe640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff458004810 0x7ff458004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46bfff640 1 -- 172.21.15.80:0/2695756710 reap_dead start 2024-04-25T13:41:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46b7fe640 1 -- 172.21.15.80:0/2695756710 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff458003530 con 0x7ff4580039c0 2024-04-25T13:41:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46b7fe640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4580039c0 0x7ff458003e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff46c147e20 tx=0x7ff45c01c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2946a3c11462db6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c0013a0 con 0x7ff4580039c0 2024-04-25T13:41:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff45c0052f0 con 0x7ff4580039c0 2024-04-25T13:41:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.559+0000 7ff46affd640 1 -- 172.21.15.80:0/2695756710 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff45c003840 con 0x7ff4580039c0 2024-04-25T13:41:09.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- 172.21.15.80:0/2695756710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4580039c0 msgr2=0x7ff458003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4580039c0 0x7ff458003e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff46c147e20 tx=0x7ff45c01c390 comp rx=0 tx=0).stop 2024-04-25T13:41:09.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- 172.21.15.80:0/2695756710 shutdown_connections 2024-04-25T13:41:09.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 --2- 172.21.15.80:0/2695756710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4580039c0 0x7ff458003e30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- 172.21.15.80:0/2695756710 >> 172.21.15.80:0/2695756710 conn(0x7ff46c06d080 msgr2=0x7ff46c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- 172.21.15.80:0/2695756710 shutdown_connections 2024-04-25T13:41:09.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- 172.21.15.80:0/2695756710 wait complete. 2024-04-25T13:41:09.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 Processor -- start 2024-04-25T13:41:09.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- start start 2024-04-25T13:41:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46c137df0 0x7ff46c15fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff46c1600c0 0x7ff46c1604d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff46c156430 0x7ff46c156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c162520 con 0x7ff46c156430 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff46c1623a0 con 0x7ff46c1600c0 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.563+0000 7ff471df0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff46c06e8b0 con 0x7ff46c137df0 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff46c156430 0x7ff46c156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff46c156430 0x7ff46c156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59570/0 (socket says 172.21.15.80:59570) 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 -- 172.21.15.80:0/2854623378 learned_addr learned my addr 172.21.15.80:0/2854623378 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff462ffd640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff46c1600c0 0x7ff46c1604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46b7fe640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46c137df0 0x7ff46c15fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 -- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46c137df0 msgr2=0x7ff46c15fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46c137df0 0x7ff46c15fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 -- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff46c1600c0 msgr2=0x7ff46c1604d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff46c1600c0 0x7ff46c1604d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 -- 172.21.15.80:0/2854623378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff46c160c30 con 0x7ff46c156430 2024-04-25T13:41:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46bfff640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff46c156430 0x7ff46c156840 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0cf270 tx=0x7ff46400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff468ff9640 1 -- 172.21.15.80:0/2854623378 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464018020 con 0x7ff46c156430 2024-04-25T13:41:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff468ff9640 1 -- 172.21.15.80:0/2854623378 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff46400bce0 con 0x7ff46c156430 2024-04-25T13:41:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff46c160f10 con 0x7ff46c156430 2024-04-25T13:41:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff468ff9640 1 -- 172.21.15.80:0/2854623378 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff464014640 con 0x7ff46c156430 2024-04-25T13:41:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff46c1614d0 con 0x7ff46c156430 2024-04-25T13:41:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff46c06a3a0 con 0x7ff46c156430 2024-04-25T13:41:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff468ff9640 1 -- 172.21.15.80:0/2854623378 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff46401b020 con 0x7ff46c156430 2024-04-25T13:41:09.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff468ff9640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44c06f400 0x7ff44c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff468ff9640 1 -- 172.21.15.80:0/2854623378 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff46408fca0 con 0x7ff46c156430 2024-04-25T13:41:09.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.567+0000 7ff46b7fe640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44c06f400 0x7ff44c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.571+0000 7ff46b7fe640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44c06f400 0x7ff44c071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff46c06fa20 tx=0x7ff45c0011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.571+0000 7ff468ff9640 1 -- 172.21.15.80:0/2854623378 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff464022030 con 0x7ff46c156430 2024-04-25T13:41:09.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.735+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff46c0b1ec0 con 0x7ff46c156430 2024-04-25T13:41:09.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.735+0000 7ff468ff9640 1 -- 172.21.15.80:0/2854623378 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7ff46405cd90 con 0x7ff46c156430 2024-04-25T13:41:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:09.738 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:09.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44c06f400 msgr2=0x7ff44c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44c06f400 0x7ff44c071910 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff46c06fa20 tx=0x7ff45c0011a0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff46c156430 msgr2=0x7ff46c156840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff46c156430 0x7ff46c156840 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0cf270 tx=0x7ff46400b480 comp rx=0 tx=0).stop 2024-04-25T13:41:09.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 shutdown_connections 2024-04-25T13:41:09.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44c06f400 0x7ff44c071910 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff46c156430 0x7ff46c156840 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff46c1600c0 0x7ff46c1604d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 --2- 172.21.15.80:0/2854623378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff46c137df0 0x7ff46c15fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 >> 172.21.15.80:0/2854623378 conn(0x7ff46c06d080 msgr2=0x7ff46c0b6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 shutdown_connections 2024-04-25T13:41:09.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.739+0000 7ff471df0640 1 -- 172.21.15.80:0/2854623378 wait complete. 2024-04-25T13:41:09.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:09.761 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:09.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f0656279640 1 Processor -- start 2024-04-25T13:41:09.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f0656279640 1 -- start start 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f0656279640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f06500b5330 0x7f06500d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f0656279640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0650161b60 0x7f0650161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f0656279640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06501624b0 con 0x7f06500d0ff0 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f0656279640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f06501629b0 con 0x7f06500b5330 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f0656279640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f06500b5740 con 0x7f0650161b60 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f064ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f06500b5330 0x7f06500d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f064ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f06500b5330 0x7f06500d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59572/0 (socket says 172.21.15.80:59572) 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f064ffff640 1 -- 172.21.15.80:0/2755153376 learned_addr learned my addr 172.21.15.80:0/2755153376 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f0654816640 1 --2- 172.21.15.80:0/2755153376 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0650161b60 0x7f0650161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f064ffff640 1 -- 172.21.15.80:0/2755153376 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0650161b60 msgr2=0x7f0650161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:09.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f064ffff640 1 --2- 172.21.15.80:0/2755153376 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0650161b60 0x7f0650161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f064ffff640 1 -- 172.21.15.80:0/2755153376 >> v1:172.21.15.87:6789/0 conn(0x7f06500d0ff0 legacy=0x7f065006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:09.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.911+0000 7f064ffff640 1 -- 172.21.15.80:0/2755153376 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065006dd10 con 0x7f06500b5330 2024-04-25T13:41:09.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f064ffff640 1 --2- 172.21.15.80:0/2755153376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06500b5330 0x7f06500d0ab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0640009870 tx=0x7f064001c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81d8258523ec7b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f064effd640 1 -- 172.21.15.80:0/2755153376 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400b0070 con 0x7f06500b5330 2024-04-25T13:41:09.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f064effd640 1 -- 172.21.15.80:0/2755153376 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06400ab600 con 0x7f06500b5330 2024-04-25T13:41:09.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f064effd640 1 -- 172.21.15.80:0/2755153376 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06400ab9d0 con 0x7f06500b5330 2024-04-25T13:41:09.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f0656279640 1 -- 172.21.15.80:0/2755153376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06500b5330 msgr2=0x7f06500d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f0656279640 1 --2- 172.21.15.80:0/2755153376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06500b5330 0x7f06500d0ab0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0640009870 tx=0x7f064001c2b0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f0656279640 1 -- 172.21.15.80:0/2755153376 shutdown_connections 2024-04-25T13:41:09.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f0656279640 1 --2- 172.21.15.80:0/2755153376 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0650161b60 0x7f0650161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f0656279640 1 --2- 172.21.15.80:0/2755153376 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f06500b5330 0x7f06500d0ab0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f0656279640 1 -- 172.21.15.80:0/2755153376 >> 172.21.15.80:0/2755153376 conn(0x7f06500b4160 msgr2=0x7f06500d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:09.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f0656279640 1 -- 172.21.15.80:0/2755153376 shutdown_connections 2024-04-25T13:41:09.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.915+0000 7f0656279640 1 -- 172.21.15.80:0/2755153376 wait complete. 2024-04-25T13:41:09.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 Processor -- start 2024-04-25T13:41:09.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 -- start start 2024-04-25T13:41:09.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06500b5330 0x7f0650142070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06500d0ff0 0x7f0650072910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0650161b60 0x7f065015d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f06500b0cd0 con 0x7f0650161b60 2024-04-25T13:41:09.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f065006e890 con 0x7f06500d0ff0 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f06500b0e50 con 0x7f06500b5330 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0650161b60 0x7f065015d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0650161b60 0x7f065015d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59586/0 (socket says 172.21.15.80:59586) 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 -- 172.21.15.80:0/1842547169 learned_addr learned my addr 172.21.15.80:0/1842547169 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f064f7fe640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06500d0ff0 0x7f0650072910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f064ffff640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06500b5330 0x7f0650142070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 -- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06500b5330 msgr2=0x7f0650142070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06500b5330 0x7f0650142070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 -- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06500d0ff0 msgr2=0x7f0650072910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06500d0ff0 0x7f0650072910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 -- 172.21.15.80:0/1842547169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065015c570 con 0x7f0650161b60 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f064f7fe640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06500d0ff0 0x7f0650072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f064ffff640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06500b5330 0x7f0650142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:09.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0654816640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0650161b60 0x7f065015d610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0648007d80 tx=0x7f064800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f064cff9640 1 -- 172.21.15.80:0/1842547169 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648014420 con 0x7f0650161b60 2024-04-25T13:41:09.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f064cff9640 1 -- 172.21.15.80:0/1842547169 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0648019070 con 0x7f0650161b60 2024-04-25T13:41:09.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065015c850 con 0x7f0650161b60 2024-04-25T13:41:09.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f064cff9640 1 -- 172.21.15.80:0/1842547169 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0648014800 con 0x7f0650161b60 2024-04-25T13:41:09.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065015ce10 con 0x7f0650161b60 2024-04-25T13:41:09.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.919+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065006a3a0 con 0x7f0650161b60 2024-04-25T13:41:09.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.923+0000 7f064cff9640 1 -- 172.21.15.80:0/1842547169 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f06480149a0 con 0x7f0650161b60 2024-04-25T13:41:09.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.923+0000 7f064cff9640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f063c06f400 0x7f063c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:09.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.923+0000 7f064cff9640 1 -- 172.21.15.80:0/1842547169 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f064808fb70 con 0x7f0650161b60 2024-04-25T13:41:09.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.923+0000 7f064ffff640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f063c06f400 0x7f063c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:09.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.923+0000 7f064ffff640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f063c06f400 0x7f063c071910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f06400ad000 tx=0x7f0640008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:09.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:09.927+0000 7f064cff9640 1 -- 172.21.15.80:0/1842547169 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f064805ca60 con 0x7f0650161b60 2024-04-25T13:41:10.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.087+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f065006fd00 con 0x7f0650161b60 2024-04-25T13:41:10.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.087+0000 7f064cff9640 1 -- 172.21.15.80:0/1842547169 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f064805c360 con 0x7f0650161b60 2024-04-25T13:41:10.089 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:10.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f063c06f400 msgr2=0x7f063c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f063c06f400 0x7f063c071910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f06400ad000 tx=0x7f0640008040 comp rx=0 tx=0).stop 2024-04-25T13:41:10.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0650161b60 msgr2=0x7f065015d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0650161b60 0x7f065015d610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0648007d80 tx=0x7f064800e4b0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 shutdown_connections 2024-04-25T13:41:10.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f063c06f400 0x7f063c071910 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0650161b60 0x7f065015d610 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06500d0ff0 0x7f0650072910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 --2- 172.21.15.80:0/1842547169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f06500b5330 0x7f0650142070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 >> 172.21.15.80:0/1842547169 conn(0x7f06500b4160 msgr2=0x7f065015fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 shutdown_connections 2024-04-25T13:41:10.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.091+0000 7f0656279640 1 -- 172.21.15.80:0/1842547169 wait complete. 2024-04-25T13:41:10.112 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:10.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18b5ae2640 1 Processor -- start 2024-04-25T13:41:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18b5ae2640 1 -- start start 2024-04-25T13:41:10.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18b5ae2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f18b0164ad0 0x7f18b00b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18b5ae2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18b00b7d00 0x7f18b00d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18b5ae2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18b00d4380 con 0x7f18b00d2990 2024-04-25T13:41:10.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18b5ae2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f18b00b8110 con 0x7f18b00b7d00 2024-04-25T13:41:10.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18b5ae2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f18b0164640 con 0x7f18b0164ad0 2024-04-25T13:41:10.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18aeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18b00b7d00 0x7f18b00d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18aeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18b00b7d00 0x7f18b00d2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59602/0 (socket says 172.21.15.80:59602) 2024-04-25T13:41:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.267+0000 7f18aeffd640 1 -- 172.21.15.80:0/3208851917 learned_addr learned my addr 172.21.15.80:0/3208851917 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18af7fe640 1 --2- 172.21.15.80:0/3208851917 >> v2:172.21.15.87:3300/0 conn(0x7f18b0164ad0 0x7f18b00b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18aeffd640 1 -- 172.21.15.80:0/3208851917 >> v2:172.21.15.87:3300/0 conn(0x7f18b0164ad0 msgr2=0x7f18b00b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18aeffd640 1 --2- 172.21.15.80:0/3208851917 >> v2:172.21.15.87:3300/0 conn(0x7f18b0164ad0 0x7f18b00b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18aeffd640 1 -- 172.21.15.80:0/3208851917 >> v1:172.21.15.87:6789/0 conn(0x7f18b00d2990 legacy=0x7f18b0160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18aeffd640 1 -- 172.21.15.80:0/3208851917 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b00b8610 con 0x7f18b00b7d00 2024-04-25T13:41:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18af7fe640 1 --2- 172.21.15.80:0/3208851917 >> v2:172.21.15.87:3300/0 conn(0x7f18b0164ad0 0x7f18b00b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18aeffd640 1 --2- 172.21.15.80:0/3208851917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b00b7d00 0x7f18b00d2450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f18b01647c0 tx=0x7f18a401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75beff38c60438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18ae7fc640 1 -- 172.21.15.80:0/3208851917 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40b0070 con 0x7f18b00b7d00 2024-04-25T13:41:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18ae7fc640 1 -- 172.21.15.80:0/3208851917 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a40ab600 con 0x7f18b00b7d00 2024-04-25T13:41:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18ae7fc640 1 -- 172.21.15.80:0/3208851917 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a40ab9d0 con 0x7f18b00b7d00 2024-04-25T13:41:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18b5ae2640 1 -- 172.21.15.80:0/3208851917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b00b7d00 msgr2=0x7f18b00d2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/3208851917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b00b7d00 0x7f18b00d2450 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f18b01647c0 tx=0x7f18a401c450 comp rx=0 tx=0).stop 2024-04-25T13:41:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18b5ae2640 1 -- 172.21.15.80:0/3208851917 shutdown_connections 2024-04-25T13:41:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/3208851917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b00b7d00 0x7f18b00d2450 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/3208851917 >> v2:172.21.15.87:3300/0 conn(0x7f18b0164ad0 0x7f18b00b77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18b5ae2640 1 -- 172.21.15.80:0/3208851917 >> 172.21.15.80:0/3208851917 conn(0x7f18b0073570 msgr2=0x7f18b006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18b5ae2640 1 -- 172.21.15.80:0/3208851917 shutdown_connections 2024-04-25T13:41:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.271+0000 7f18b5ae2640 1 -- 172.21.15.80:0/3208851917 wait complete. 2024-04-25T13:41:10.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.275+0000 7f18b5ae2640 1 Processor -- start 2024-04-25T13:41:10.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.275+0000 7f18b5ae2640 1 -- start start 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.275+0000 7f18b5ae2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b00b7d00 0x7f18b0159330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.275+0000 7f18b5ae2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b00d2990 0x7f18b00ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.275+0000 7f18b5ae2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b0164ad0 0x7f18b00ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.275+0000 7f18b5ae2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18b016d0e0 con 0x7f18b0164ad0 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.275+0000 7f18b5ae2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18b00d2da0 con 0x7f18b00d2990 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.275+0000 7f18b5ae2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18b016d260 con 0x7f18b00b7d00 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b0164ad0 0x7f18b00ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b0164ad0 0x7f18b00ce390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59604/0 (socket says 172.21.15.80:59604) 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 -- 172.21.15.80:0/760464888 learned_addr learned my addr 172.21.15.80:0/760464888 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:10.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18aeffd640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b00d2990 0x7f18b00ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18af7fe640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b00b7d00 0x7f18b0159330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 -- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b00b7d00 msgr2=0x7f18b0159330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b00b7d00 0x7f18b0159330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 -- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b00d2990 msgr2=0x7f18b00ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b00d2990 0x7f18b00ca8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 -- 172.21.15.80:0/760464888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b00ccdf0 con 0x7f18b0164ad0 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18affff640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b0164ad0 0x7f18b00ce390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b440 tx=0x7f18a000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f1897fff640 1 -- 172.21.15.80:0/760464888 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0015020 con 0x7f18b0164ad0 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f1897fff640 1 -- 172.21.15.80:0/760464888 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18a0011850 con 0x7f18b0164ad0 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b00cd2f0 con 0x7f18b0164ad0 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f1897fff640 1 -- 172.21.15.80:0/760464888 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0011c40 con 0x7f18b0164ad0 2024-04-25T13:41:10.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b00cd8b0 con 0x7f18b0164ad0 2024-04-25T13:41:10.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18b00cd060 con 0x7f18b0164ad0 2024-04-25T13:41:10.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f1897fff640 1 -- 172.21.15.80:0/760464888 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f18a0018020 con 0x7f18b0164ad0 2024-04-25T13:41:10.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f1897fff640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f189c06f430 0x7f189c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f1897fff640 1 -- 172.21.15.80:0/760464888 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f18a008f130 con 0x7f18b0164ad0 2024-04-25T13:41:10.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.279+0000 7f18af7fe640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f189c06f430 0x7f189c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.283+0000 7f18af7fe640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f189c06f430 0x7f189c071940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18b00d23a0 tx=0x7f18a8008450 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.287+0000 7f1897fff640 1 -- 172.21.15.80:0/760464888 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f18a001f020 con 0x7f18b0164ad0 2024-04-25T13:41:10.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.447+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f18b006c960 con 0x7f18b0164ad0 2024-04-25T13:41:10.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.447+0000 7f1897fff640 1 -- 172.21.15.80:0/760464888 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f18a005c0b0 con 0x7f18b0164ad0 2024-04-25T13:41:10.451 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:10.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f189c06f430 msgr2=0x7f189c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f189c06f430 0x7f189c071940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f18b00d23a0 tx=0x7f18a8008450 comp rx=0 tx=0).stop 2024-04-25T13:41:10.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b0164ad0 msgr2=0x7f18b00ce390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b0164ad0 0x7f18b00ce390 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b440 tx=0x7f18a000b900 comp rx=0 tx=0).stop 2024-04-25T13:41:10.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 shutdown_connections 2024-04-25T13:41:10.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f189c06f430 0x7f189c071940 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b0164ad0 0x7f18b00ce390 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b00d2990 0x7f18b00ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 --2- 172.21.15.80:0/760464888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b00b7d00 0x7f18b0159330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 >> 172.21.15.80:0/760464888 conn(0x7f18b0073570 msgr2=0x7f18b015d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 shutdown_connections 2024-04-25T13:41:10.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.451+0000 7f18b5ae2640 1 -- 172.21.15.80:0/760464888 wait complete. 2024-04-25T13:41:10.473 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:10.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.627+0000 7ff9a6141640 1 Processor -- start 2024-04-25T13:41:10.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.627+0000 7ff9a6141640 1 -- start start 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff9a6141640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff9a006e760 0x7ff9a00d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff9a6141640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9a015e250 0x7ff9a00d1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff9a6141640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9a00b72d0 con 0x7ff9a00d1660 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff9a6141640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff9a00b1590 con 0x7ff9a006e760 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff9a6141640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff9a00d0c90 con 0x7ff9a015e250 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff9a006e760 0x7ff9a00d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99effd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff9a00d1660 0x7ff9a0160c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41884/0 (socket says 172.21.15.80:41884) 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99effd640 1 -- 172.21.15.80:0/1586010735 learned_addr learned my addr 172.21.15.80:0/1586010735 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99ffff640 1 --2- 172.21.15.80:0/1586010735 >> v2:172.21.15.87:3300/0 conn(0x7ff9a015e250 0x7ff9a00d1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99e7fc640 1 -- 172.21.15.80:0/1586010735 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1618000126 0 0) 0x7ff9a00b72d0 con 0x7ff9a00d1660 2024-04-25T13:41:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99e7fc640 1 -- 172.21.15.80:0/1586010735 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff988003530 con 0x7ff9a00d1660 2024-04-25T13:41:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99f7fe640 1 -- 172.21.15.80:0/1586010735 >> v2:172.21.15.87:3300/0 conn(0x7ff9a015e250 msgr2=0x7ff9a00d1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99f7fe640 1 --2- 172.21.15.80:0/1586010735 >> v2:172.21.15.87:3300/0 conn(0x7ff9a015e250 0x7ff9a00d1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99f7fe640 1 -- 172.21.15.80:0/1586010735 >> v1:172.21.15.80:6789/0 conn(0x7ff9a00d1660 legacy=0x7ff9a0160c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99f7fe640 1 -- 172.21.15.80:0/1586010735 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a00b17e0 con 0x7ff9a006e760 2024-04-25T13:41:10.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99f7fe640 1 --2- 172.21.15.80:0/1586010735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006e760 0x7ff9a00d1120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00747b0 tx=0x7ff99001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b1e254403cb1d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99e7fc640 1 -- 172.21.15.80:0/1586010735 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900b0070 con 0x7ff9a006e760 2024-04-25T13:41:10.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99e7fc640 1 -- 172.21.15.80:0/1586010735 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9900ab600 con 0x7ff9a006e760 2024-04-25T13:41:10.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff99e7fc640 1 -- 172.21.15.80:0/1586010735 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9900ab9d0 con 0x7ff9a006e760 2024-04-25T13:41:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff9a6141640 1 -- 172.21.15.80:0/1586010735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006e760 msgr2=0x7ff9a00d1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.631+0000 7ff9a6141640 1 --2- 172.21.15.80:0/1586010735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006e760 0x7ff9a00d1120 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff9a00747b0 tx=0x7ff99001c360 comp rx=0 tx=0).stop 2024-04-25T13:41:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 -- 172.21.15.80:0/1586010735 shutdown_connections 2024-04-25T13:41:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 --2- 172.21.15.80:0/1586010735 >> v2:172.21.15.87:3300/0 conn(0x7ff9a015e250 0x7ff9a00d1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 --2- 172.21.15.80:0/1586010735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a006e760 0x7ff9a00d1120 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 -- 172.21.15.80:0/1586010735 >> 172.21.15.80:0/1586010735 conn(0x7ff9a00b3cc0 msgr2=0x7ff9a00b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 -- 172.21.15.80:0/1586010735 shutdown_connections 2024-04-25T13:41:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 -- 172.21.15.80:0/1586010735 wait complete. 2024-04-25T13:41:10.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 Processor -- start 2024-04-25T13:41:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 -- start start 2024-04-25T13:41:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9a006e760 0x7ff9a0149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a00d1660 0x7ff9a0160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9a015e250 0x7ff9a0073750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a016a210 con 0x7ff9a00d1660 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a006eb70 con 0x7ff9a015e250 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff9a6141640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9a016a390 con 0x7ff9a006e760 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a00d1660 0x7ff9a0160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a00d1660 0x7ff9a0160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59616/0 (socket says 172.21.15.80:59616) 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99effd640 1 -- 172.21.15.80:0/3144606761 learned_addr learned my addr 172.21.15.80:0/3144606761 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99effd640 1 -- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9a006e760 msgr2=0x7ff9a0149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99ffff640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9a015e250 0x7ff9a0073750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99effd640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9a006e760 0x7ff9a0149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99effd640 1 -- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9a015e250 msgr2=0x7ff9a0073750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99effd640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9a015e250 0x7ff9a0073750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.635+0000 7ff99effd640 1 -- 172.21.15.80:0/3144606761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a015edd0 con 0x7ff9a00d1660 2024-04-25T13:41:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff99effd640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a00d1660 0x7ff9a0160210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c3c0 tx=0x7ff99400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff987fff640 1 -- 172.21.15.80:0/3144606761 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994017070 con 0x7ff9a00d1660 2024-04-25T13:41:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff987fff640 1 -- 172.21.15.80:0/3144606761 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff994005cf0 con 0x7ff9a00d1660 2024-04-25T13:41:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff987fff640 1 -- 172.21.15.80:0/3144606761 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9940072f0 con 0x7ff9a00d1660 2024-04-25T13:41:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a015f0b0 con 0x7ff9a00d1660 2024-04-25T13:41:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a015f670 con 0x7ff9a00d1660 2024-04-25T13:41:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9a006a3a0 con 0x7ff9a00d1660 2024-04-25T13:41:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff987fff640 1 -- 172.21.15.80:0/3144606761 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff99401a070 con 0x7ff9a00d1660 2024-04-25T13:41:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff987fff640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff98c06f400 0x7ff98c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff987fff640 1 -- 172.21.15.80:0/3144606761 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff99408e5d0 con 0x7ff9a00d1660 2024-04-25T13:41:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff99f7fe640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff98c06f400 0x7ff98c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.639+0000 7ff99f7fe640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff98c06f400 0x7ff98c071910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9900ad000 tx=0x7ff990004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.643+0000 7ff987fff640 1 -- 172.21.15.80:0/3144606761 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff99405b5e0 con 0x7ff9a00d1660 2024-04-25T13:41:10.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.803+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9a0076bb0 con 0x7ff9a00d1660 2024-04-25T13:41:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.803+0000 7ff987fff640 1 -- 172.21.15.80:0/3144606761 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7ff99405aee0 con 0x7ff9a00d1660 2024-04-25T13:41:10.805 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff98c06f400 msgr2=0x7ff98c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff98c06f400 0x7ff98c071910 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9900ad000 tx=0x7ff990004740 comp rx=0 tx=0).stop 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a00d1660 msgr2=0x7ff9a0160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a00d1660 0x7ff9a0160210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff99400c3c0 tx=0x7ff99400c880 comp rx=0 tx=0).stop 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 shutdown_connections 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff98c06f400 0x7ff98c071910 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9a015e250 0x7ff9a0073750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9a00d1660 0x7ff9a0160210 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 --2- 172.21.15.80:0/3144606761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9a006e760 0x7ff9a0149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 >> 172.21.15.80:0/3144606761 conn(0x7ff9a00b3cc0 msgr2=0x7ff9a0153c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 shutdown_connections 2024-04-25T13:41:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.807+0000 7ff9a6141640 1 -- 172.21.15.80:0/3144606761 wait complete. 2024-04-25T13:41:10.826 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f007a16a640 1 Processor -- start 2024-04-25T13:41:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f007a16a640 1 -- start start 2024-04-25T13:41:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f007a16a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0074161240 0x7f00740723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f007a16a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0074161650 0x7f00740728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f007a16a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f007406fcf0 con 0x7f007406f5a0 2024-04-25T13:41:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f007a16a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0074073f10 con 0x7f0074161650 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f007a16a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f00740b0c60 con 0x7f0074161240 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0073fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0074161650 0x7f00740728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0073fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0074161650 0x7f00740728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59632/0 (socket says 172.21.15.80:59632) 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0073fff640 1 -- 172.21.15.80:0/969235019 learned_addr learned my addr 172.21.15.80:0/969235019 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f00737fe640 1 --1- 172.21.15.80:0/969235019 >> v1:172.21.15.80:6789/0 conn(0x7f007406f5a0 0x7f0074160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41886/0 (socket says 172.21.15.80:41886) 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0072ffd640 1 --2- 172.21.15.80:0/969235019 >> v2:172.21.15.87:3300/0 conn(0x7f0074161240 0x7f00740723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0073fff640 1 -- 172.21.15.80:0/969235019 >> v2:172.21.15.87:3300/0 conn(0x7f0074161240 msgr2=0x7f00740723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0073fff640 1 --2- 172.21.15.80:0/969235019 >> v2:172.21.15.87:3300/0 conn(0x7f0074161240 0x7f00740723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0073fff640 1 -- 172.21.15.80:0/969235019 >> v1:172.21.15.80:6789/0 conn(0x7f007406f5a0 legacy=0x7f0074160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f00727fc640 1 -- 172.21.15.80:0/969235019 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1323843607 0 0) 0x7f007406fcf0 con 0x7f007406f5a0 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0073fff640 1 -- 172.21.15.80:0/969235019 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00740b0f40 con 0x7f0074161650 2024-04-25T13:41:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.975+0000 7f0072ffd640 1 --2- 172.21.15.80:0/969235019 >> v2:172.21.15.87:3300/0 conn(0x7f0074161240 0x7f00740723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f0073fff640 1 --2- 172.21.15.80:0/969235019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740728e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f007406ff70 tx=0x7f006401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b22630db0b799d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f00727fc640 1 -- 172.21.15.80:0/969235019 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640a56a0 con 0x7f0074161650 2024-04-25T13:41:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f00727fc640 1 -- 172.21.15.80:0/969235019 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00640a5840 con 0x7f0074161650 2024-04-25T13:41:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f00727fc640 1 -- 172.21.15.80:0/969235019 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640a5c10 con 0x7f0074161650 2024-04-25T13:41:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f007a16a640 1 -- 172.21.15.80:0/969235019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 msgr2=0x7f00740728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f007a16a640 1 --2- 172.21.15.80:0/969235019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740728e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f007406ff70 tx=0x7f006401c960 comp rx=0 tx=0).stop 2024-04-25T13:41:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f007a16a640 1 -- 172.21.15.80:0/969235019 shutdown_connections 2024-04-25T13:41:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f007a16a640 1 --2- 172.21.15.80:0/969235019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740728e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f007a16a640 1 --2- 172.21.15.80:0/969235019 >> v2:172.21.15.87:3300/0 conn(0x7f0074161240 0x7f00740723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f007a16a640 1 -- 172.21.15.80:0/969235019 >> 172.21.15.80:0/969235019 conn(0x7f007406d080 msgr2=0x7f00740b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f007a16a640 1 -- 172.21.15.80:0/969235019 shutdown_connections 2024-04-25T13:41:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.979+0000 7f007a16a640 1 -- 172.21.15.80:0/969235019 wait complete. 2024-04-25T13:41:10.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 Processor -- start 2024-04-25T13:41:10.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 -- start start 2024-04-25T13:41:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f007406f5a0 0x7f00741460d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0074161240 0x7f00740abc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f007416a260 con 0x7f0074161650 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f007416a0e0 con 0x7f007406f5a0 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f007416a3e0 con 0x7f0074161240 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740ac160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59638/0 (socket says 172.21.15.80:59638) 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 -- 172.21.15.80:0/602524256 learned_addr learned my addr 172.21.15.80:0/602524256 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f00737fe640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f007406f5a0 0x7f00741460d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0072ffd640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0074161240 0x7f00740abc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 -- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0074161240 msgr2=0x7f00740abc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0074161240 0x7f00740abc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 -- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f007406f5a0 msgr2=0x7f00741460d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f007406f5a0 0x7f00741460d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 -- 172.21.15.80:0/602524256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00740add20 con 0x7f0074161650 2024-04-25T13:41:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f0073fff640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740ac160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0064019570 tx=0x7f0064019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f006bfff640 1 -- 172.21.15.80:0/602524256 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640b8670 con 0x7f0074161650 2024-04-25T13:41:10.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f006bfff640 1 -- 172.21.15.80:0/602524256 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00640be070 con 0x7f0074161650 2024-04-25T13:41:10.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f006bfff640 1 -- 172.21.15.80:0/602524256 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00640b8a70 con 0x7f0074161650 2024-04-25T13:41:10.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0074157770 con 0x7f0074161650 2024-04-25T13:41:10.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0074157d30 con 0x7f0074161650 2024-04-25T13:41:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.983+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f007406a3a0 con 0x7f0074161650 2024-04-25T13:41:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.987+0000 7f006bfff640 1 -- 172.21.15.80:0/602524256 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f00640b8c10 con 0x7f0074161650 2024-04-25T13:41:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.987+0000 7f006bfff640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f005406f400 0x7f0054071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.987+0000 7f006bfff640 1 -- 172.21.15.80:0/602524256 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f006412a440 con 0x7f0074161650 2024-04-25T13:41:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.987+0000 7f00737fe640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f005406f400 0x7f0054071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.987+0000 7f00737fe640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f005406f400 0x7f0054071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f006c000f70 tx=0x7f006c007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:10.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:10.991+0000 7f006bfff640 1 -- 172.21.15.80:0/602524256 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00640165e0 con 0x7f0074161650 2024-04-25T13:41:11.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0074157500 con 0x7f0074161650 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f006bfff640 1 -- 172.21.15.80:0/602524256 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f00640f7330 con 0x7f0074161650 2024-04-25T13:41:11.168 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f005406f400 msgr2=0x7f0054071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f005406f400 0x7f0054071910 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f006c000f70 tx=0x7f006c007c10 comp rx=0 tx=0).stop 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 msgr2=0x7f00740ac160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740ac160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0064019570 tx=0x7f0064019700 comp rx=0 tx=0).stop 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 shutdown_connections 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f005406f400 0x7f0054071910 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0074161650 0x7f00740ac160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0074161240 0x7f00740abc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 --2- 172.21.15.80:0/602524256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f007406f5a0 0x7f00741460d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 >> 172.21.15.80:0/602524256 conn(0x7f007406d080 msgr2=0x7f0074156000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 shutdown_connections 2024-04-25T13:41:11.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.167+0000 7f007a16a640 1 -- 172.21.15.80:0/602524256 wait complete. 2024-04-25T13:41:11.189 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:41:11.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:11.190 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:11.215 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:41:11.215 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:41:11.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:11.215 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:41:11.270 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:11.270 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:11.276 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:11.276 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:11.324 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:11.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.515+0000 7fa03b946640 1 Processor -- start 2024-04-25T13:41:11.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.515+0000 7fa03b946640 1 -- start start 2024-04-25T13:41:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa03b946640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0340d14a0 0x7fa0340b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa03b946640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0340d18b0 con 0x7fa0340d2c70 2024-04-25T13:41:11.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa03b946640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0340d43f0 con 0x7fa0340d5490 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa03b946640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa0340d46d0 con 0x7fa0340d14a0 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa038ee1640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa0340d2c70 0x7fa0340b92a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36856/0 (socket says 172.21.15.80:36856) 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa038ee1640 1 -- 172.21.15.80:0/793076460 learned_addr learned my addr 172.21.15.80:0/793076460 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa039ee3640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0340d14a0 0x7fa0340b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 392959507 0 0) 0x7fa0340d18b0 con 0x7fa0340d2c70 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa020003530 con 0x7fa0340d2c70 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3011915750 0 0) 0x7fa020003530 con 0x7fa0340d2c70 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0340d14a0 msgr2=0x7fa0340b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0340d14a0 0x7fa0340b99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 >> v1:172.21.15.87:6789/0 conn(0x7fa0340d5490 legacy=0x7fa0340d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0340d49b0 con 0x7fa0340d2c70 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fa028001680 con 0x7fa0340d2c70 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 >> v1:172.21.15.80:6789/0 conn(0x7fa0340d2c70 legacy=0x7fa0340b92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0200039f0 0x7fa020003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0340d14a0 0x7fa020004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa020004d50 0x7fa020005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa0340d46d0 con 0x7fa020004d50 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa02000bb00 con 0x7fa0200039f0 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa02000bd00 con 0x7fa0340d14a0 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa039ee3640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa020004d50 0x7fa020005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa0396e2640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0200039f0 0x7fa020003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa038ee1640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0340d14a0 0x7fa020004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa039ee3640 1 -- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0340d14a0 msgr2=0x7fa020004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa039ee3640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0340d14a0 0x7fa020004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa039ee3640 1 -- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0200039f0 msgr2=0x7fa020003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa039ee3640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa0200039f0 0x7fa020003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.519+0000 7fa039ee3640 1 -- 172.21.15.80:0/793076460 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa020003530 con 0x7fa020004d50 2024-04-25T13:41:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa039ee3640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa020004d50 0x7fa020005160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009f90 tx=0x7fa02c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e8235bd7335526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0b2070 con 0x7fa020004d50 2024-04-25T13:41:11.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa02c0a5c70 con 0x7fa020004d50 2024-04-25T13:41:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa027fff640 1 -- 172.21.15.80:0/793076460 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c009c10 con 0x7fa020004d50 2024-04-25T13:41:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa03b946640 1 -- 172.21.15.80:0/793076460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa020004d50 msgr2=0x7fa020005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa03b946640 1 --2- 172.21.15.80:0/793076460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa020004d50 0x7fa020005160 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa02c009f90 tx=0x7fa02c01cc60 comp rx=0 tx=0).stop 2024-04-25T13:41:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa039ee3640 1 -- 172.21.15.80:0/793076460 reap_dead start 2024-04-25T13:41:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa03b946640 1 -- 172.21.15.80:0/793076460 shutdown_connections 2024-04-25T13:41:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa03b946640 1 -- 172.21.15.80:0/793076460 >> 172.21.15.80:0/793076460 conn(0x7fa034073740 msgr2=0x7fa03406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa03b946640 1 -- 172.21.15.80:0/793076460 shutdown_connections 2024-04-25T13:41:11.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa03b946640 1 -- 172.21.15.80:0/793076460 wait complete. 2024-04-25T13:41:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.523+0000 7fa03b946640 1 Processor -- start 2024-04-25T13:41:11.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 -- start start 2024-04-25T13:41:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa020004d50 0x7fa0340b7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0341f6040 0x7fa03414b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0341f6450 0x7fa03414bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa034160640 con 0x7fa0341f6450 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa0340d4f70 con 0x7fa020004d50 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa0340b4f30 con 0x7fa0341f6040 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0341f6450 0x7fa03414bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0341f6450 0x7fa03414bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52892/0 (socket says 172.21.15.80:52892) 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 -- 172.21.15.80:0/3757058189 learned_addr learned my addr 172.21.15.80:0/3757058189 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa0396e2640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa020004d50 0x7fa0340b7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 -- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0341f6040 msgr2=0x7fa03414b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa038ee1640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0341f6040 0x7fa03414b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0341f6040 0x7fa03414b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 -- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa020004d50 msgr2=0x7fa0340b7ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa020004d50 0x7fa0340b7ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 -- 172.21.15.80:0/3757058189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0340b62e0 con 0x7fa0341f6450 2024-04-25T13:41:11.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa0396e2640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa020004d50 0x7fa0340b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:11.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa039ee3640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0341f6450 0x7fa03414bdd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa02c002240 tx=0x7fa02c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa025ffb640 1 -- 172.21.15.80:0/3757058189 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c0b2040 con 0x7fa0341f6450 2024-04-25T13:41:11.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa025ffb640 1 -- 172.21.15.80:0/3757058189 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa02c0a5a70 con 0x7fa0341f6450 2024-04-25T13:41:11.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa025ffb640 1 -- 172.21.15.80:0/3757058189 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02c009c10 con 0x7fa0341f6450 2024-04-25T13:41:11.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0340b65c0 con 0x7fa0341f6450 2024-04-25T13:41:11.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0340b6b80 con 0x7fa0341f6450 2024-04-25T13:41:11.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.527+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0340b6030 con 0x7fa0341f6450 2024-04-25T13:41:11.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.531+0000 7fa025ffb640 1 -- 172.21.15.80:0/3757058189 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa02c0be020 con 0x7fa0341f6450 2024-04-25T13:41:11.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.531+0000 7fa025ffb640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01806f400 0x7fa018071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.531+0000 7fa025ffb640 1 -- 172.21.15.80:0/3757058189 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fa02c129780 con 0x7fa0341f6450 2024-04-25T13:41:11.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.531+0000 7fa0396e2640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01806f400 0x7fa018071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.531+0000 7fa0396e2640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01806f400 0x7fa018071910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0300016d0 tx=0x7fa030009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.535+0000 7fa025ffb640 1 -- 172.21.15.80:0/3757058189 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa02c0a5e20 con 0x7fa0341f6450 2024-04-25T13:41:11.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa03415f200 con 0x7fa0341f6450 2024-04-25T13:41:11.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa025ffb640 1 -- 172.21.15.80:0/3757058189 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7fa02c016970 con 0x7fa0341f6450 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:11.701 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01806f400 msgr2=0x7fa018071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01806f400 0x7fa018071910 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa0300016d0 tx=0x7fa030009040 comp rx=0 tx=0).stop 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0341f6450 msgr2=0x7fa03414bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0341f6450 0x7fa03414bdd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa02c002240 tx=0x7fa02c009750 comp rx=0 tx=0).stop 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 shutdown_connections 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01806f400 0x7fa018071910 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0341f6450 0x7fa03414bdd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.699+0000 7fa03b946640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0341f6040 0x7fa03414b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.703+0000 7fa03b946640 1 --2- 172.21.15.80:0/3757058189 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa020004d50 0x7fa0340b7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.703+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 >> 172.21.15.80:0/3757058189 conn(0x7fa034073740 msgr2=0x7fa03415e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.703+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 shutdown_connections 2024-04-25T13:41:11.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.703+0000 7fa03b946640 1 -- 172.21.15.80:0/3757058189 wait complete. 2024-04-25T13:41:11.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:11.723 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:11.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f050411e640 1 Processor -- start 2024-04-25T13:41:11.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f050411e640 1 -- start start 2024-04-25T13:41:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f050411e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b08b0 0x7f04fc1655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f050411e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04fc06dbe0 0x7f04fc06dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f050411e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04fc0b0cc0 con 0x7f04fc0b4bd0 2024-04-25T13:41:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f050411e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f04fc0b0f40 con 0x7f04fc06dbe0 2024-04-25T13:41:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f050411e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc0b1220 con 0x7f04fc0b08b0 2024-04-25T13:41:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f05016b9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04fc06dbe0 0x7f04fc06dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f05016b9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04fc06dbe0 0x7f04fc06dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52898/0 (socket says 172.21.15.80:52898) 2024-04-25T13:41:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f05016b9640 1 -- 172.21.15.80:0/2924760251 learned_addr learned my addr 172.21.15.80:0/2924760251 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.863+0000 7f0501eba640 1 --2- 172.21.15.80:0/2924760251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b08b0 0x7f04fc1655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f05016b9640 1 -- 172.21.15.80:0/2924760251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b08b0 msgr2=0x7f04fc1655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f05016b9640 1 --2- 172.21.15.80:0/2924760251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b08b0 0x7f04fc1655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f05016b9640 1 -- 172.21.15.80:0/2924760251 >> v1:172.21.15.87:6789/0 conn(0x7f04fc0b4bd0 legacy=0x7f04fc0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f05016b9640 1 -- 172.21.15.80:0/2924760251 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc0726a0 con 0x7f04fc06dbe0 2024-04-25T13:41:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f0501eba640 1 --2- 172.21.15.80:0/2924760251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b08b0 0x7f04fc1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:11.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f05016b9640 1 --2- 172.21.15.80:0/2924760251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc06dff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009870 tx=0x7f04ec01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23e00dc605d178b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f0500eb8640 1 -- 172.21.15.80:0/2924760251 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0b0070 con 0x7f04fc06dbe0 2024-04-25T13:41:11.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f0500eb8640 1 -- 172.21.15.80:0/2924760251 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec0ab600 con 0x7f04fc06dbe0 2024-04-25T13:41:11.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f0500eb8640 1 -- 172.21.15.80:0/2924760251 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0ab9d0 con 0x7f04fc06dbe0 2024-04-25T13:41:11.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f050411e640 1 -- 172.21.15.80:0/2924760251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 msgr2=0x7f04fc06dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f050411e640 1 --2- 172.21.15.80:0/2924760251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc06dff0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f04ec009870 tx=0x7f04ec01c340 comp rx=0 tx=0).stop 2024-04-25T13:41:11.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f050411e640 1 -- 172.21.15.80:0/2924760251 shutdown_connections 2024-04-25T13:41:11.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f050411e640 1 --2- 172.21.15.80:0/2924760251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc06dff0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f050411e640 1 --2- 172.21.15.80:0/2924760251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b08b0 0x7f04fc1655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f050411e640 1 -- 172.21.15.80:0/2924760251 >> 172.21.15.80:0/2924760251 conn(0x7f04fc0b4160 msgr2=0x7f04fc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:11.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f050411e640 1 -- 172.21.15.80:0/2924760251 shutdown_connections 2024-04-25T13:41:11.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.867+0000 7f050411e640 1 -- 172.21.15.80:0/2924760251 wait complete. 2024-04-25T13:41:11.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 Processor -- start 2024-04-25T13:41:11.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 -- start start 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc162f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc0b08b0 0x7f04fc150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b4bd0 0x7f04fc154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc171880 con 0x7f04fc06dbe0 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc171700 con 0x7f04fc0b08b0 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc171a00 con 0x7f04fc0b4bd0 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc162f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52912/0 (socket says 172.21.15.80:52912) 2024-04-25T13:41:11.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 -- 172.21.15.80:0/2623380511 learned_addr learned my addr 172.21.15.80:0/2623380511 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f05016b9640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc0b08b0 0x7f04fc150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 -- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b4bd0 msgr2=0x7f04fc154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b4bd0 0x7f04fc154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 -- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc0b08b0 msgr2=0x7f04fc150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc0b08b0 0x7f04fc150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 -- 172.21.15.80:0/2623380511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc06fe10 con 0x7f04fc06dbe0 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f0501eba640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc162f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c7f0 tx=0x7f04f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f04f27fc640 1 -- 172.21.15.80:0/2623380511 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8015070 con 0x7f04fc06dbe0 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f04f27fc640 1 -- 172.21.15.80:0/2623380511 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f8007820 con 0x7f04fc06dbe0 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f04f27fc640 1 -- 172.21.15.80:0/2623380511 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8007bd0 con 0x7f04fc06dbe0 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc151b50 con 0x7f04fc06dbe0 2024-04-25T13:41:11.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc152110 con 0x7f04fc06dbe0 2024-04-25T13:41:11.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.871+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc06a3a0 con 0x7f04fc06dbe0 2024-04-25T13:41:11.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.875+0000 7f04f27fc640 1 -- 172.21.15.80:0/2623380511 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04f8018070 con 0x7f04fc06dbe0 2024-04-25T13:41:11.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.875+0000 7f04f27fc640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f430 0x7f04e0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:11.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.875+0000 7f04f27fc640 1 -- 172.21.15.80:0/2623380511 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f04f808f3b0 con 0x7f04fc06dbe0 2024-04-25T13:41:11.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.875+0000 7f05016b9640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f430 0x7f04e0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:11.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.875+0000 7f05016b9640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f430 0x7f04e0071940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0ad000 tx=0x7f04ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:11.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:11.879+0000 7f04f27fc640 1 -- 172.21.15.80:0/2623380511 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04f805c2a0 con 0x7f04fc06dbe0 2024-04-25T13:41:12.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f04fc16f2c0 con 0x7f04fc06dbe0 2024-04-25T13:41:12.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f04f27fc640 1 -- 172.21.15.80:0/2623380511 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f04f805bba0 con 0x7f04fc06dbe0 2024-04-25T13:41:12.028 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:12.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f430 msgr2=0x7f04e0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f430 0x7f04e0071940 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0ad000 tx=0x7f04ec008040 comp rx=0 tx=0).stop 2024-04-25T13:41:12.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 msgr2=0x7f04fc162f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc162f50 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c7f0 tx=0x7f04f800ccb0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 shutdown_connections 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f430 0x7f04e0071940 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc0b4bd0 0x7f04fc154820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc0b08b0 0x7f04fc150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 --2- 172.21.15.80:0/2623380511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc06dbe0 0x7f04fc162f50 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 >> 172.21.15.80:0/2623380511 conn(0x7f04fc0b4160 msgr2=0x7f04fc1674d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 shutdown_connections 2024-04-25T13:41:12.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.027+0000 7f050411e640 1 -- 172.21.15.80:0/2623380511 wait complete. 2024-04-25T13:41:12.048 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:12.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff189fc1640 1 Processor -- start 2024-04-25T13:41:12.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff189fc1640 1 -- start start 2024-04-25T13:41:12.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff189fc1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1840ce7d0 0x7ff1840cebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff189fc1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff18406d560 con 0x7ff1840cfbb0 2024-04-25T13:41:12.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff189fc1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1840d28d0 con 0x7ff18406cf80 2024-04-25T13:41:12.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff189fc1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff1840d2bb0 con 0x7ff1840ce7d0 2024-04-25T13:41:12.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff17affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff1840cfbb0 0x7ff1840ce0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36858/0 (socket says 172.21.15.80:36858) 2024-04-25T13:41:12.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff17affd640 1 -- 172.21.15.80:0/438669577 learned_addr learned my addr 172.21.15.80:0/438669577 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:12.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1047029844 0 0) 0x7ff18406d560 con 0x7ff1840cfbb0 2024-04-25T13:41:12.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff183fff640 1 --2- 172.21.15.80:0/438669577 >> v2:172.21.15.87:3300/0 conn(0x7ff1840ce7d0 0x7ff1840cebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff170003530 con 0x7ff1840cfbb0 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3843389551 0 0) 0x7ff170003530 con 0x7ff1840cfbb0 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 >> v2:172.21.15.87:3300/0 conn(0x7ff1840ce7d0 msgr2=0x7ff1840cebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 --2- 172.21.15.80:0/438669577 >> v2:172.21.15.87:3300/0 conn(0x7ff1840ce7d0 0x7ff1840cebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 >> v1:172.21.15.87:6789/0 conn(0x7ff18406cf80 legacy=0x7ff1840cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1840b6240 con 0x7ff1840cfbb0 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff16c001680 con 0x7ff1840cfbb0 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 >> v1:172.21.15.80:6789/0 conn(0x7ff1840cfbb0 legacy=0x7ff1840ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1700039c0 0x7ff170003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff170004810 0x7ff170004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff170005280 0x7ff17000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff1840d2bb0 con 0x7ff170005280 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff17000c210 con 0x7ff170004810 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff17000c410 con 0x7ff1700039c0 2024-04-25T13:41:12.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff183fff640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff170005280 0x7ff17000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff1837fe640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1700039c0 0x7ff170003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff17affd640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff170004810 0x7ff170004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff183fff640 1 -- 172.21.15.80:0/438669577 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1700039c0 msgr2=0x7ff170003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff183fff640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1700039c0 0x7ff170003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff183fff640 1 -- 172.21.15.80:0/438669577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff170004810 msgr2=0x7ff170004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff183fff640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff170004810 0x7ff170004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.199+0000 7ff183fff640 1 -- 172.21.15.80:0/438669577 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff170003530 con 0x7ff170005280 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff183fff640 1 -- 172.21.15.80:0/438669577 reap_dead start 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff183fff640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff170005280 0x7ff17000b9d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff17c007360 tx=0x7ff17c01ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67652654a6190c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c009be0 con 0x7ff170005280 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17c0a5ce0 con 0x7ff170005280 2024-04-25T13:41:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff182ffd640 1 -- 172.21.15.80:0/438669577 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17c003be0 con 0x7ff170005280 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 -- 172.21.15.80:0/438669577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff170005280 msgr2=0x7ff17000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff170005280 0x7ff17000b9d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff17c007360 tx=0x7ff17c01ce80 comp rx=0 tx=0).stop 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 -- 172.21.15.80:0/438669577 shutdown_connections 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 --2- 172.21.15.80:0/438669577 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff170005280 0x7ff17000b9d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 -- 172.21.15.80:0/438669577 >> 172.21.15.80:0/438669577 conn(0x7ff1840b5580 msgr2=0x7ff1840b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 -- 172.21.15.80:0/438669577 shutdown_connections 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 -- 172.21.15.80:0/438669577 wait complete. 2024-04-25T13:41:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 Processor -- start 2024-04-25T13:41:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.203+0000 7ff189fc1640 1 -- start start 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff170005280 0x7ff1841ea590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1840b2a70 0x7ff1840b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1840ca920 0x7ff1840cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff1840b60a0 con 0x7ff1840b2a70 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff1840b4df0 con 0x7ff1840ca920 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff1840d30d0 con 0x7ff170005280 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1840b2a70 0x7ff1840b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1840b2a70 0x7ff1840b2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52930/0 (socket says 172.21.15.80:52930) 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 -- 172.21.15.80:0/2718276305 learned_addr learned my addr 172.21.15.80:0/2718276305 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff1837fe640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff170005280 0x7ff1841ea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff183fff640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1840ca920 0x7ff1840cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 -- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff170005280 msgr2=0x7ff1841ea590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff170005280 0x7ff1841ea590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 -- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1840ca920 msgr2=0x7ff1840cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1840ca920 0x7ff1840cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 -- 172.21.15.80:0/2718276305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1840cb830 con 0x7ff1840b2a70 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff17affd640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1840b2a70 0x7ff1840b2e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e7b0 tx=0x7ff16c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff180ff9640 1 -- 172.21.15.80:0/2718276305 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00c8d0 con 0x7ff1840b2a70 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff180ff9640 1 -- 172.21.15.80:0/2718276305 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff16c018070 con 0x7ff1840b2a70 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1840cbb10 con 0x7ff1840b2a70 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff180ff9640 1 -- 172.21.15.80:0/2718276305 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00ccb0 con 0x7ff1840b2a70 2024-04-25T13:41:12.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18409e900 con 0x7ff1840b2a70 2024-04-25T13:41:12.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1840cb5a0 con 0x7ff1840b2a70 2024-04-25T13:41:12.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.207+0000 7ff180ff9640 1 -- 172.21.15.80:0/2718276305 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff16c01d460 con 0x7ff1840b2a70 2024-04-25T13:41:12.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.211+0000 7ff180ff9640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16406f430 0x7ff164071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.211+0000 7ff180ff9640 1 -- 172.21.15.80:0/2718276305 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff16c08f0f0 con 0x7ff1840b2a70 2024-04-25T13:41:12.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.211+0000 7ff1837fe640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16406f430 0x7ff164071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.211+0000 7ff1837fe640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16406f430 0x7ff164071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff174000f00 tx=0x7ff174009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.215+0000 7ff180ff9640 1 -- 172.21.15.80:0/2718276305 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff16c05c070 con 0x7ff1840b2a70 2024-04-25T13:41:12.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.359+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff18406a3a0 con 0x7ff1840b2a70 2024-04-25T13:41:12.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff180ff9640 1 -- 172.21.15.80:0/2718276305 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7ff16c05b970 con 0x7ff1840b2a70 2024-04-25T13:41:12.363 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:12.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16406f430 msgr2=0x7ff164071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16406f430 0x7ff164071940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff174000f00 tx=0x7ff174009040 comp rx=0 tx=0).stop 2024-04-25T13:41:12.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1840b2a70 msgr2=0x7ff1840b2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1840b2a70 0x7ff1840b2e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e7b0 tx=0x7ff16c00ec70 comp rx=0 tx=0).stop 2024-04-25T13:41:12.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 shutdown_connections 2024-04-25T13:41:12.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16406f430 0x7ff164071940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1840ca920 0x7ff1840cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1840b2a70 0x7ff1840b2e80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 --2- 172.21.15.80:0/2718276305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff170005280 0x7ff1841ea590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 >> 172.21.15.80:0/2718276305 conn(0x7ff1840b5580 msgr2=0x7ff1840719f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 shutdown_connections 2024-04-25T13:41:12.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.363+0000 7ff189fc1640 1 -- 172.21.15.80:0/2718276305 wait complete. 2024-04-25T13:41:12.383 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:12.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be86b6640 1 Processor -- start 2024-04-25T13:41:12.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be86b6640 1 -- start start 2024-04-25T13:41:12.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be86b6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4be01574c0 0x7f4be0073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be86b6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be006fbf0 con 0x7f4be0161170 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be86b6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4be00b0a20 con 0x7f4be0071e60 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be86b6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4be00b1350 con 0x7f4be01574c0 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be6c53640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4be01574c0 0x7f4be0073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be5c51640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4be0161170 0x7f4be00b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36870/0 (socket says 172.21.15.80:36870) 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be5c51640 1 -- 172.21.15.80:0/2484669449 learned_addr learned my addr 172.21.15.80:0/2484669449 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be6c53640 1 -- 172.21.15.80:0/2484669449 >> v1:172.21.15.87:6789/0 conn(0x7f4be0071e60 legacy=0x7f4be0160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be6c53640 1 -- 172.21.15.80:0/2484669449 >> v1:172.21.15.80:6789/0 conn(0x7f4be0161170 legacy=0x7f4be00b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.527+0000 7f4be6c53640 1 -- 172.21.15.80:0/2484669449 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be00b0d00 con 0x7f4be01574c0 2024-04-25T13:41:12.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be6c53640 1 --2- 172.21.15.80:0/2484669449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be0073c40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4be00b14d0 tx=0x7f4bd801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ebb96564028ce37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be5450640 1 -- 172.21.15.80:0/2484669449 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80b2070 con 0x7f4be01574c0 2024-04-25T13:41:12.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be5450640 1 -- 172.21.15.80:0/2484669449 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd80a57d0 con 0x7f4be01574c0 2024-04-25T13:41:12.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be5450640 1 -- 172.21.15.80:0/2484669449 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd80a5ba0 con 0x7f4be01574c0 2024-04-25T13:41:12.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be86b6640 1 -- 172.21.15.80:0/2484669449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 msgr2=0x7f4be0073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be86b6640 1 --2- 172.21.15.80:0/2484669449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be0073c40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4be00b14d0 tx=0x7f4bd801c910 comp rx=0 tx=0).stop 2024-04-25T13:41:12.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be86b6640 1 -- 172.21.15.80:0/2484669449 shutdown_connections 2024-04-25T13:41:12.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be86b6640 1 --2- 172.21.15.80:0/2484669449 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be0073c40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be86b6640 1 -- 172.21.15.80:0/2484669449 >> 172.21.15.80:0/2484669449 conn(0x7f4be00b4160 msgr2=0x7f4be00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be86b6640 1 -- 172.21.15.80:0/2484669449 shutdown_connections 2024-04-25T13:41:12.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.531+0000 7f4be86b6640 1 -- 172.21.15.80:0/2484669449 wait complete. 2024-04-25T13:41:12.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 Processor -- start 2024-04-25T13:41:12.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 -- start start 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4be0071e60 0x7f4be014e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be014fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4be0151100 0x7f4be0151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4be00b56c0 con 0x7f4be01574c0 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4be016b7e0 con 0x7f4be0071e60 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4be01f9360 con 0x7f4be0151100 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be014fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be014fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52958/0 (socket says 172.21.15.80:52958) 2024-04-25T13:41:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 -- 172.21.15.80:0/2932833606 learned_addr learned my addr 172.21.15.80:0/2932833606 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be6c53640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4be0151100 0x7f4be0151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be6452640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4be0071e60 0x7f4be014e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 -- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4be0151100 msgr2=0x7f4be0151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4be0151100 0x7f4be0151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 -- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4be0071e60 msgr2=0x7f4be014e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4be0071e60 0x7f4be014e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 -- 172.21.15.80:0/2932833606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4be0151a50 con 0x7f4be01574c0 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be6452640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4be0071e60 0x7f4be014e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be5c51640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be014fa30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c370 tx=0x7f4bd000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4bd6ffd640 1 -- 172.21.15.80:0/2932833606 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd0017070 con 0x7f4be01574c0 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4bd6ffd640 1 -- 172.21.15.80:0/2932833606 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bd0005cf0 con 0x7f4be01574c0 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4be01532e0 con 0x7f4be01574c0 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4bd6ffd640 1 -- 172.21.15.80:0/2932833606 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bd00072f0 con 0x7f4be01574c0 2024-04-25T13:41:12.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4be0153ac0 con 0x7f4be01574c0 2024-04-25T13:41:12.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.535+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4be01f7990 con 0x7f4be01574c0 2024-04-25T13:41:12.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.539+0000 7f4bd6ffd640 1 -- 172.21.15.80:0/2932833606 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4bd001a070 con 0x7f4be01574c0 2024-04-25T13:41:12.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.539+0000 7f4bd6ffd640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bc406f400 0x7f4bc4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.539+0000 7f4bd6ffd640 1 -- 172.21.15.80:0/2932833606 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4bd008e980 con 0x7f4be01574c0 2024-04-25T13:41:12.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.539+0000 7f4be6452640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bc406f400 0x7f4bc4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.539+0000 7f4be6452640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bc406f400 0x7f4bc4071910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00a850 tx=0x7f4bdc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.543+0000 7f4bd6ffd640 1 -- 172.21.15.80:0/2932833606 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4be01f7990 con 0x7f4be01574c0 2024-04-25T13:41:12.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.687+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4be006a3a0 con 0x7f4be01574c0 2024-04-25T13:41:12.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.687+0000 7f4bd6ffd640 1 -- 172.21.15.80:0/2932833606 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f4bd0021020 con 0x7f4be01574c0 2024-04-25T13:41:12.691 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bc406f400 msgr2=0x7f4bc4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bc406f400 0x7f4bc4071910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00a850 tx=0x7f4bdc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 msgr2=0x7f4be014fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be014fa30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4bd000c370 tx=0x7f4bd000c830 comp rx=0 tx=0).stop 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 shutdown_connections 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bc406f400 0x7f4bc4071910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4be0151100 0x7f4be0151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4be01574c0 0x7f4be014fa30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 --2- 172.21.15.80:0/2932833606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4be0071e60 0x7f4be014e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 >> 172.21.15.80:0/2932833606 conn(0x7f4be00b4160 msgr2=0x7f4be00b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 shutdown_connections 2024-04-25T13:41:12.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.691+0000 7f4be86b6640 1 -- 172.21.15.80:0/2932833606 wait complete. 2024-04-25T13:41:12.711 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:12.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f372d3d2640 1 Processor -- start 2024-04-25T13:41:12.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f372d3d2640 1 -- start start 2024-04-25T13:41:12.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f372d3d2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f372806f5a0 0x7f3728160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f372d3d2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f37280733b0 0x7f3728161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f372d3d2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f372806fcf0 con 0x7f3728161070 2024-04-25T13:41:12.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f372d3d2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f37281619c0 con 0x7f37280733b0 2024-04-25T13:41:12.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f372d3d2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3728073f10 con 0x7f372806f5a0 2024-04-25T13:41:12.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f37277fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f37280733b0 0x7f3728161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f37277fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f37280733b0 0x7f3728161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52974/0 (socket says 172.21.15.80:52974) 2024-04-25T13:41:12.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f37277fe640 1 -- 172.21.15.80:0/3651255006 learned_addr learned my addr 172.21.15.80:0/3651255006 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:12.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f3726ffd640 1 --2- 172.21.15.80:0/3651255006 >> v2:172.21.15.87:3300/0 conn(0x7f372806f5a0 0x7f3728160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f37277fe640 1 -- 172.21.15.80:0/3651255006 >> v2:172.21.15.87:3300/0 conn(0x7f372806f5a0 msgr2=0x7f3728160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f37277fe640 1 --2- 172.21.15.80:0/3651255006 >> v2:172.21.15.87:3300/0 conn(0x7f372806f5a0 0x7f3728160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f37277fe640 1 -- 172.21.15.80:0/3651255006 >> v1:172.21.15.87:6789/0 conn(0x7f3728161070 legacy=0x7f37280b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:12.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.855+0000 7f37277fe640 1 -- 172.21.15.80:0/3651255006 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37280b0c60 con 0x7f37280733b0 2024-04-25T13:41:12.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f37277fe640 1 --2- 172.21.15.80:0/3651255006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37280733b0 0x7f3728161480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3728071e30 tx=0x7f372001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5f2e371c2b3a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f3725ffb640 1 -- 172.21.15.80:0/3651255006 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200a55f0 con 0x7f37280733b0 2024-04-25T13:41:12.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f3725ffb640 1 -- 172.21.15.80:0/3651255006 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37200a5790 con 0x7f37280733b0 2024-04-25T13:41:12.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f3725ffb640 1 -- 172.21.15.80:0/3651255006 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37200a5b60 con 0x7f37280733b0 2024-04-25T13:41:12.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 -- 172.21.15.80:0/3651255006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37280733b0 msgr2=0x7f3728161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 --2- 172.21.15.80:0/3651255006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37280733b0 0x7f3728161480 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3728071e30 tx=0x7f372001c920 comp rx=0 tx=0).stop 2024-04-25T13:41:12.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 -- 172.21.15.80:0/3651255006 shutdown_connections 2024-04-25T13:41:12.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 --2- 172.21.15.80:0/3651255006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37280733b0 0x7f3728161480 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 --2- 172.21.15.80:0/3651255006 >> v2:172.21.15.87:3300/0 conn(0x7f372806f5a0 0x7f3728160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 -- 172.21.15.80:0/3651255006 >> 172.21.15.80:0/3651255006 conn(0x7f372806d080 msgr2=0x7f37280b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:12.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 -- 172.21.15.80:0/3651255006 shutdown_connections 2024-04-25T13:41:12.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 -- 172.21.15.80:0/3651255006 wait complete. 2024-04-25T13:41:12.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 Processor -- start 2024-04-25T13:41:12.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.859+0000 7f372d3d2640 1 -- start start 2024-04-25T13:41:12.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f372806f5a0 0x7f3728148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37280733b0 0x7f37280afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3728161070 0x7f37280affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f372816a050 con 0x7f372806f5a0 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3728169ed0 con 0x7f37280733b0 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f372816a1d0 con 0x7f3728161070 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f372806f5a0 0x7f3728148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f372806f5a0 0x7f3728148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52980/0 (socket says 172.21.15.80:52980) 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 -- 172.21.15.80:0/2057379557 learned_addr learned my addr 172.21.15.80:0/2057379557 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 -- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3728161070 msgr2=0x7f37280affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3728161070 0x7f37280affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 -- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37280733b0 msgr2=0x7f37280afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37280733b0 0x7f37280afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 -- 172.21.15.80:0/2057379557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37281f7720 con 0x7f372806f5a0 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f3726ffd640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f372806f5a0 0x7f3728148240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f371800c7f0 tx=0x7f371800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f370f7fe640 1 -- 172.21.15.80:0/2057379557 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718004230 con 0x7f372806f5a0 2024-04-25T13:41:12.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f370f7fe640 1 -- 172.21.15.80:0/2057379557 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37180043d0 con 0x7f372806f5a0 2024-04-25T13:41:12.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f370f7fe640 1 -- 172.21.15.80:0/2057379557 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718003860 con 0x7f372806f5a0 2024-04-25T13:41:12.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37280ac290 con 0x7f372806f5a0 2024-04-25T13:41:12.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37280ac850 con 0x7f372806f5a0 2024-04-25T13:41:12.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f372806a3a0 con 0x7f372806f5a0 2024-04-25T13:41:12.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.863+0000 7f370f7fe640 1 -- 172.21.15.80:0/2057379557 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3718003c10 con 0x7f372806f5a0 2024-04-25T13:41:12.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.867+0000 7f370f7fe640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f371406f400 0x7f3714071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:12.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.867+0000 7f370f7fe640 1 -- 172.21.15.80:0/2057379557 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f371808f360 con 0x7f372806f5a0 2024-04-25T13:41:12.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.867+0000 7f37267fc640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f371406f400 0x7f3714071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:12.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.867+0000 7f37267fc640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f371406f400 0x7f3714071910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f372815cbf0 tx=0x7f371c006bb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:12.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:12.871+0000 7f370f7fe640 1 -- 172.21.15.80:0/2057379557 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3718015020 con 0x7f372806f5a0 2024-04-25T13:41:13.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.039+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f37280739c0 con 0x7f372806f5a0 2024-04-25T13:41:13.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.039+0000 7f370f7fe640 1 -- 172.21.15.80:0/2057379557 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f371805c2e0 con 0x7f372806f5a0 2024-04-25T13:41:13.042 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:13.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f371406f400 msgr2=0x7f3714071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f371406f400 0x7f3714071910 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f372815cbf0 tx=0x7f371c006bb0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f372806f5a0 msgr2=0x7f3728148240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f372806f5a0 0x7f3728148240 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f371800c7f0 tx=0x7f371800ccb0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 shutdown_connections 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f371406f400 0x7f3714071910 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3728161070 0x7f37280affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37280733b0 0x7f37280afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 --2- 172.21.15.80:0/2057379557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f372806f5a0 0x7f3728148240 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 >> 172.21.15.80:0/2057379557 conn(0x7f372806d080 msgr2=0x7f37280b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 shutdown_connections 2024-04-25T13:41:13.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.043+0000 7f372d3d2640 1 -- 172.21.15.80:0/2057379557 wait complete. 2024-04-25T13:41:13.063 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:13.063 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:13.068 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bcd8e3640 1 Processor -- start 2024-04-25T13:41:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bcd8e3640 1 -- start start 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bcd8e3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0bc8071ea0 0x7f0bc80b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bcd8e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 0x7f0bc80d2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bcd8e3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0bc80d2a60 con 0x7f0bc80d2650 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bcd8e3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0bc80b7c10 con 0x7f0bc8071ea0 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bcd8e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc80d2ce0 con 0x7f0bc815e380 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bc77fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0bc80d2650 0x7f0bc80afbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36886/0 (socket says 172.21.15.80:36886) 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.255+0000 7f0bc77fe640 1 -- 172.21.15.80:0/916712337 learned_addr learned my addr 172.21.15.80:0/916712337 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc67fc640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 0x7f0bc80d2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc6ffd640 1 --2- 172.21.15.80:0/916712337 >> v2:172.21.15.87:3300/0 conn(0x7f0bc8071ea0 0x7f0bc80b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2572389306 0 0) 0x7f0bc0001120 con 0x7f0bc80d2650 2024-04-25T13:41:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bb4003530 con 0x7f0bc80d2650 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2687082129 0 0) 0x7f0bb4003530 con 0x7f0bc80d2650 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 msgr2=0x7f0bc80d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 0x7f0bc80d2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 >> v2:172.21.15.87:3300/0 conn(0x7f0bc8071ea0 msgr2=0x7f0bc80b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 --2- 172.21.15.80:0/916712337 >> v2:172.21.15.87:3300/0 conn(0x7f0bc8071ea0 0x7f0bc80b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc80d2f80 con 0x7f0bc80d2650 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0bc0001850 con 0x7f0bc80d2650 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 >> v1:172.21.15.80:6789/0 conn(0x7f0bc80d2650 legacy=0x7f0bc80afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb40039c0 0x7f0bb4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 0x7f0bb4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bb4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc80b7c10 con 0x7f0bb4004d50 2024-04-25T13:41:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc80d2ce0 con 0x7f0bb40039c0 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb400ba50 con 0x7f0bc815e380 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc77fe640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bb4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc67fc640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 0x7f0bb4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc6ffd640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb40039c0 0x7f0bb4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc77fe640 1 -- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 msgr2=0x7f0bb4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc77fe640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 0x7f0bb4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc77fe640 1 -- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb40039c0 msgr2=0x7f0bb4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc77fe640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb40039c0 0x7f0bb4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc77fe640 1 -- 172.21.15.80:0/916712337 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb4003530 con 0x7f0bb4004d50 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc6ffd640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bb40039c0 0x7f0bb4003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc67fc640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815e380 0x7f0bb4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc77fe640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bb4005160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8074bd0 tx=0x7f0bc001f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e84b3be9a58ce4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000c0e0 con 0x7f0bb4004d50 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bc0005ae0 con 0x7f0bb4004d50 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.259+0000 7f0bc5ffb640 1 -- 172.21.15.80:0/916712337 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0007070 con 0x7f0bb4004d50 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/916712337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 msgr2=0x7f0bb4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bcd8e3640 1 --2- 172.21.15.80:0/916712337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bb4005160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8074bd0 tx=0x7f0bc001f3e0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bc77fe640 1 -- 172.21.15.80:0/916712337 reap_dead start 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/916712337 shutdown_connections 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/916712337 >> 172.21.15.80:0/916712337 conn(0x7f0bc80b3ed0 msgr2=0x7f0bc80b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/916712337 shutdown_connections 2024-04-25T13:41:13.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/916712337 wait complete. 2024-04-25T13:41:13.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bcd8e3640 1 Processor -- start 2024-04-25T13:41:13.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.263+0000 7f0bcd8e3640 1 -- start start 2024-04-25T13:41:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bc80d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bc80cfd50 0x7f0bc81505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815c3d0 0x7f0bc815c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc80b4d40 con 0x7f0bb4004d50 2024-04-25T13:41:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0bc8076ba0 con 0x7f0bc80cfd50 2024-04-25T13:41:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0bc80b49f0 con 0x7f0bc815c3d0 2024-04-25T13:41:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bc80d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bc80d18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52992/0 (socket says 172.21.15.80:52992) 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 -- 172.21.15.80:0/435925565 learned_addr learned my addr 172.21.15.80:0/435925565 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc67fc640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bc80cfd50 0x7f0bc81505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc77fe640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815c3d0 0x7f0bc815c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 -- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815c3d0 msgr2=0x7f0bc815c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815c3d0 0x7f0bc815c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 -- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bc80cfd50 msgr2=0x7f0bc81505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bc80cfd50 0x7f0bc81505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 -- 172.21.15.80:0/435925565 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bc815cd20 con 0x7f0bb4004d50 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc67fc640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bc80cfd50 0x7f0bc81505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bc6ffd640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bc80d18a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8007f30 tx=0x7f0bb800f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bbffff640 1 -- 172.21.15.80:0/435925565 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80039a0 con 0x7f0bb4004d50 2024-04-25T13:41:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bbffff640 1 -- 172.21.15.80:0/435925565 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bb8003b40 con 0x7f0bb4004d50 2024-04-25T13:41:13.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bc80d02f0 con 0x7f0bb4004d50 2024-04-25T13:41:13.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bbffff640 1 -- 172.21.15.80:0/435925565 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb8004020 con 0x7f0bb4004d50 2024-04-25T13:41:13.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bc80d08b0 con 0x7f0bb4004d50 2024-04-25T13:41:13.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bc806a3a0 con 0x7f0bb4004d50 2024-04-25T13:41:13.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bbffff640 1 -- 172.21.15.80:0/435925565 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0bb801d030 con 0x7f0bb4004d50 2024-04-25T13:41:13.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bbffff640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ba806f400 0x7f0ba8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.267+0000 7f0bbffff640 1 -- 172.21.15.80:0/435925565 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f0bb808fd90 con 0x7f0bb4004d50 2024-04-25T13:41:13.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.271+0000 7f0bc67fc640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ba806f400 0x7f0ba8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.271+0000 7f0bc67fc640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ba806f400 0x7f0ba8071910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0bc815cf50 tx=0x7f0bb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.275+0000 7f0bbffff640 1 -- 172.21.15.80:0/435925565 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0bb8024020 con 0x7f0bb4004d50 2024-04-25T13:41:13.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.423+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0bc815ef70 con 0x7f0bb4004d50 2024-04-25T13:41:13.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bbffff640 1 -- 172.21.15.80:0/435925565 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f0bc815cd20 con 0x7f0bb4004d50 2024-04-25T13:41:13.428 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:13.428 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ba806f400 msgr2=0x7f0ba8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ba806f400 0x7f0ba8071910 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0bc815cf50 tx=0x7f0bb0008040 comp rx=0 tx=0).stop 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 msgr2=0x7f0bc80d18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bc80d18a0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8007f30 tx=0x7f0bb800f5e0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 shutdown_connections 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ba806f400 0x7f0ba8071910 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0bc815c3d0 0x7f0bc815c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0bc80cfd50 0x7f0bc81505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 --2- 172.21.15.80:0/435925565 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0bb4004d50 0x7f0bc80d18a0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 >> 172.21.15.80:0/435925565 conn(0x7f0bc80b3ed0 msgr2=0x7f0bc8151500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:13.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 shutdown_connections 2024-04-25T13:41:13.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.427+0000 7f0bcd8e3640 1 -- 172.21.15.80:0/435925565 wait complete. 2024-04-25T13:41:13.448 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:13.449 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:13.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.595+0000 7f48175b9640 1 Processor -- start 2024-04-25T13:41:13.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.595+0000 7f48175b9640 1 -- start start 2024-04-25T13:41:13.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f48175b9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f48101574c0 0x7f4810073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f48175b9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f481006fbf0 con 0x7f4810071e60 2024-04-25T13:41:13.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f48175b9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48100b0a20 con 0x7f4810161170 2024-04-25T13:41:13.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f48175b9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f48100b1350 con 0x7f48101574c0 2024-04-25T13:41:13.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f4815355640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4810071e60 0x7f4810160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36902/0 (socket says 172.21.15.80:36902) 2024-04-25T13:41:13.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f4815355640 1 -- 172.21.15.80:0/3341323514 learned_addr learned my addr 172.21.15.80:0/3341323514 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:13.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f4815b56640 1 --2- 172.21.15.80:0/3341323514 >> v2:172.21.15.80:3300/0 conn(0x7f48101574c0 0x7f4810073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f4807fff640 1 -- 172.21.15.80:0/3341323514 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3046993845 0 0) 0x7f481006fbf0 con 0x7f4810071e60 2024-04-25T13:41:13.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f4807fff640 1 -- 172.21.15.80:0/3341323514 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f47fc003530 con 0x7f4810071e60 2024-04-25T13:41:13.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f4815b56640 1 -- 172.21.15.80:0/3341323514 >> v1:172.21.15.87:6789/0 conn(0x7f4810161170 legacy=0x7f48100b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f4815b56640 1 -- 172.21.15.80:0/3341323514 >> v1:172.21.15.80:6789/0 conn(0x7f4810071e60 legacy=0x7f4810160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.599+0000 7f4815b56640 1 -- 172.21.15.80:0/3341323514 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48100b0d00 con 0x7f48101574c0 2024-04-25T13:41:13.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f4815b56640 1 --2- 172.21.15.80:0/3341323514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f4810073c40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f48100b14d0 tx=0x7f480801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dd6f8526137c8ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f4807fff640 1 -- 172.21.15.80:0/3341323514 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48080b2070 con 0x7f48101574c0 2024-04-25T13:41:13.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f4807fff640 1 -- 172.21.15.80:0/3341323514 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48080a5810 con 0x7f48101574c0 2024-04-25T13:41:13.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f4807fff640 1 -- 172.21.15.80:0/3341323514 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48080a5be0 con 0x7f48101574c0 2024-04-25T13:41:13.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 -- 172.21.15.80:0/3341323514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 msgr2=0x7f4810073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 --2- 172.21.15.80:0/3341323514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f4810073c40 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f48100b14d0 tx=0x7f480801c950 comp rx=0 tx=0).stop 2024-04-25T13:41:13.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 -- 172.21.15.80:0/3341323514 shutdown_connections 2024-04-25T13:41:13.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 --2- 172.21.15.80:0/3341323514 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f4810073c40 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 -- 172.21.15.80:0/3341323514 >> 172.21.15.80:0/3341323514 conn(0x7f48100b4160 msgr2=0x7f48100d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:13.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 -- 172.21.15.80:0/3341323514 shutdown_connections 2024-04-25T13:41:13.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 -- 172.21.15.80:0/3341323514 wait complete. 2024-04-25T13:41:13.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 Processor -- start 2024-04-25T13:41:13.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.603+0000 7f48175b9640 1 -- start start 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4810071e60 0x7f481013f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f481014d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f481014d7c0 0x7f481014f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f48100b56c0 con 0x7f48101574c0 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f481016b7e0 con 0x7f4810071e60 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f48101542d0 con 0x7f481014d7c0 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f481014d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f481014d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53022/0 (socket says 172.21.15.80:53022) 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 -- 172.21.15.80:0/3860654608 learned_addr learned my addr 172.21.15.80:0/3860654608 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:13.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4815355640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4810071e60 0x7f481013f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4815b56640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f481014d7c0 0x7f481014f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 -- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f481014d7c0 msgr2=0x7f481014f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f481014d7c0 0x7f481014f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 -- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4810071e60 msgr2=0x7f481013f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4810071e60 0x7f481013f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 -- 172.21.15.80:0/3860654608 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481014dfd0 con 0x7f48101574c0 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4814b54640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f481014d280 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f480000c3a0 tx=0x7f480000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4805ffb640 1 -- 172.21.15.80:0/3860654608 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48000052d0 con 0x7f48101574c0 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4805ffb640 1 -- 172.21.15.80:0/3860654608 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4800005470 con 0x7f48101574c0 2024-04-25T13:41:13.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f4805ffb640 1 -- 172.21.15.80:0/3860654608 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4800007260 con 0x7f48101574c0 2024-04-25T13:41:13.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481014e2b0 con 0x7f48101574c0 2024-04-25T13:41:13.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4810152a70 con 0x7f48101574c0 2024-04-25T13:41:13.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.607+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f481006a3a0 con 0x7f48101574c0 2024-04-25T13:41:13.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.611+0000 7f4805ffb640 1 -- 172.21.15.80:0/3860654608 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4800007400 con 0x7f48101574c0 2024-04-25T13:41:13.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.611+0000 7f4805ffb640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47f406f400 0x7f47f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.611+0000 7f4805ffb640 1 -- 172.21.15.80:0/3860654608 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f480008e780 con 0x7f48101574c0 2024-04-25T13:41:13.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.611+0000 7f4815355640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47f406f400 0x7f47f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.611+0000 7f4815355640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47f406f400 0x7f47f4071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f480c0011f0 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.615+0000 7f4805ffb640 1 -- 172.21.15.80:0/3860654608 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f480005b660 con 0x7f48101574c0 2024-04-25T13:41:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.775+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f481015dc40 con 0x7f48101574c0 2024-04-25T13:41:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.775+0000 7f4805ffb640 1 -- 172.21.15.80:0/3860654608 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f480005af60 con 0x7f48101574c0 2024-04-25T13:41:13.777 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.775+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47f406f400 msgr2=0x7f47f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.775+0000 7f48175b9640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47f406f400 0x7f47f4071910 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f480c0011f0 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-04-25T13:41:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 msgr2=0x7f481014d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f481014d280 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f480000c3a0 tx=0x7f480000c860 comp rx=0 tx=0).stop 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 shutdown_connections 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47f406f400 0x7f47f4071910 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f481014d7c0 0x7f481014f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48101574c0 0x7f481014d280 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 --2- 172.21.15.80:0/3860654608 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4810071e60 0x7f481013f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 >> 172.21.15.80:0/3860654608 conn(0x7f48100b4160 msgr2=0x7f48100b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 shutdown_connections 2024-04-25T13:41:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.779+0000 7f48175b9640 1 -- 172.21.15.80:0/3860654608 wait complete. 2024-04-25T13:41:13.799 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:13.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b4498f640 1 Processor -- start 2024-04-25T13:41:13.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b4498f640 1 -- start start 2024-04-25T13:41:13.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b4498f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6b400b5e20 0x7f6b4006cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b4498f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b4015e7d0 con 0x7f6b40068ca0 2024-04-25T13:41:13.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b4498f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6b4015eab0 con 0x7f6b400cecf0 2024-04-25T13:41:13.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b4498f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6b4015ed90 con 0x7f6b400b5e20 2024-04-25T13:41:13.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b3ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6b400b5e20 0x7f6b4006cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b3dd9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6b40068ca0 0x7f6b4006c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36914/0 (socket says 172.21.15.80:36914) 2024-04-25T13:41:13.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.951+0000 7f6b3dd9b640 1 -- 172.21.15.80:0/2710399175 learned_addr learned my addr 172.21.15.80:0/2710399175 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:13.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b3ed9d640 1 -- 172.21.15.80:0/2710399175 >> v1:172.21.15.87:6789/0 conn(0x7f6b400cecf0 legacy=0x7f6b40068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:13.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b3d59a640 1 -- 172.21.15.80:0/2710399175 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088619310 0 0) 0x7f6b4015e7d0 con 0x7f6b40068ca0 2024-04-25T13:41:13.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b3ed9d640 1 -- 172.21.15.80:0/2710399175 >> v1:172.21.15.80:6789/0 conn(0x7f6b40068ca0 legacy=0x7f6b4006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b3ed9d640 1 -- 172.21.15.80:0/2710399175 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b4015f070 con 0x7f6b400b5e20 2024-04-25T13:41:13.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b3ed9d640 1 --2- 172.21.15.80:0/2710399175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400b5e20 0x7f6b4006cd00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009d90 tx=0x7f6b3801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcabffaccd90e1da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b3d59a640 1 -- 172.21.15.80:0/2710399175 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380b2070 con 0x7f6b400b5e20 2024-04-25T13:41:13.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b3d59a640 1 -- 172.21.15.80:0/2710399175 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b380a5830 con 0x7f6b400b5e20 2024-04-25T13:41:13.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b3d59a640 1 -- 172.21.15.80:0/2710399175 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b380a5c00 con 0x7f6b400b5e20 2024-04-25T13:41:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b4498f640 1 -- 172.21.15.80:0/2710399175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400b5e20 msgr2=0x7f6b4006cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b4498f640 1 --2- 172.21.15.80:0/2710399175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400b5e20 0x7f6b4006cd00 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b38009d90 tx=0x7f6b3801c8e0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b4498f640 1 -- 172.21.15.80:0/2710399175 shutdown_connections 2024-04-25T13:41:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b4498f640 1 --2- 172.21.15.80:0/2710399175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400b5e20 0x7f6b4006cd00 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b4498f640 1 -- 172.21.15.80:0/2710399175 >> 172.21.15.80:0/2710399175 conn(0x7f6b400cfa50 msgr2=0x7f6b400cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b4498f640 1 -- 172.21.15.80:0/2710399175 shutdown_connections 2024-04-25T13:41:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.955+0000 7f6b4498f640 1 -- 172.21.15.80:0/2710399175 wait complete. 2024-04-25T13:41:13.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 Processor -- start 2024-04-25T13:41:13.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 -- start start 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400b5e20 0x7f6b401fc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400cecf0 0x7f6b400d1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b400b3e10 0x7f6b400d1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4016cc00 con 0x7f6b400cecf0 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6b4016ca80 con 0x7f6b400b5e20 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6b4016cd80 con 0x7f6b400b3e10 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400cecf0 0x7f6b400d1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400cecf0 0x7f6b400d1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53042/0 (socket says 172.21.15.80:53042) 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 -- 172.21.15.80:0/1269954635 learned_addr learned my addr 172.21.15.80:0/1269954635 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3e59c640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400b5e20 0x7f6b401fc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3ed9d640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b400b3e10 0x7f6b400d1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 -- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b400b3e10 msgr2=0x7f6b400d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b400b3e10 0x7f6b400d1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 -- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400b5e20 msgr2=0x7f6b401fc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400b5e20 0x7f6b401fc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 -- 172.21.15.80:0/1269954635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b40154f00 con 0x7f6b400cecf0 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b3dd9b640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400cecf0 0x7f6b400d1a90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400c410 tx=0x7f6b3400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b2effd640 1 -- 172.21.15.80:0/1269954635 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b340074e0 con 0x7f6b400cecf0 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b2effd640 1 -- 172.21.15.80:0/1269954635 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6b340047f0 con 0x7f6b400cecf0 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b400b3a40 con 0x7f6b400cecf0 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b2effd640 1 -- 172.21.15.80:0/1269954635 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34007680 con 0x7f6b400cecf0 2024-04-25T13:41:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.959+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b40152ac0 con 0x7f6b400cecf0 2024-04-25T13:41:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.963+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b401525d0 con 0x7f6b400cecf0 2024-04-25T13:41:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.963+0000 7f6b2effd640 1 -- 172.21.15.80:0/1269954635 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6b34011070 con 0x7f6b400cecf0 2024-04-25T13:41:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.963+0000 7f6b2effd640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6b2006f430 0x7f6b20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.963+0000 7f6b2effd640 1 -- 172.21.15.80:0/1269954635 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f6b3408ed70 con 0x7f6b400cecf0 2024-04-25T13:41:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.963+0000 7f6b3e59c640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6b2006f430 0x7f6b20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:13.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.963+0000 7f6b3e59c640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6b2006f430 0x7f6b20071940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000f70 tx=0x7f6b30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:13.967+0000 7f6b2effd640 1 -- 172.21.15.80:0/1269954635 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6b3405bcf0 con 0x7f6b400cecf0 2024-04-25T13:41:14.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.115+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b4006a3a0 con 0x7f6b400cecf0 2024-04-25T13:41:14.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b2effd640 1 -- 172.21.15.80:0/1269954635 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f6b3405b5f0 con 0x7f6b400cecf0 2024-04-25T13:41:14.119 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:14.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6b2006f430 msgr2=0x7f6b20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6b2006f430 0x7f6b20071940 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6b30000f70 tx=0x7f6b30009040 comp rx=0 tx=0).stop 2024-04-25T13:41:14.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400cecf0 msgr2=0x7f6b400d1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400cecf0 0x7f6b400d1a90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400c410 tx=0x7f6b3400c8d0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 shutdown_connections 2024-04-25T13:41:14.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6b2006f430 0x7f6b20071940 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6b400b3e10 0x7f6b400d1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6b400cecf0 0x7f6b400d1a90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 --2- 172.21.15.80:0/1269954635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6b400b5e20 0x7f6b401fc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 >> 172.21.15.80:0/1269954635 conn(0x7f6b400cfa50 msgr2=0x7f6b40138310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:14.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 shutdown_connections 2024-04-25T13:41:14.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.119+0000 7f6b4498f640 1 -- 172.21.15.80:0/1269954635 wait complete. 2024-04-25T13:41:14.139 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:14.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.287+0000 7f8090551640 1 Processor -- start 2024-04-25T13:41:14.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.287+0000 7f8090551640 1 -- start start 2024-04-25T13:41:14.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f8090551640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80880d4690 0x7f80880d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f8090551640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f80880d0940 0x7f80880b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f8090551640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80880d0d50 con 0x7f80880b7350 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f8090551640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f80880d2100 con 0x7f80880d0940 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f8090551640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f80880d23a0 con 0x7f80880d4690 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808eaee640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f80880b7350 0x7f80880b7760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36916/0 (socket says 172.21.15.80:36916) 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808eaee640 1 -- 172.21.15.80:0/3154163535 learned_addr learned my addr 172.21.15.80:0/3154163535 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808daec640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f80880d0940 0x7f80880b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808e2ed640 1 --2- 172.21.15.80:0/3154163535 >> v2:172.21.15.87:3300/0 conn(0x7f80880d4690 0x7f80880d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808daec640 1 -- 172.21.15.80:0/3154163535 >> v2:172.21.15.87:3300/0 conn(0x7f80880d4690 msgr2=0x7f80880d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808daec640 1 --2- 172.21.15.80:0/3154163535 >> v2:172.21.15.87:3300/0 conn(0x7f80880d4690 0x7f80880d0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808daec640 1 -- 172.21.15.80:0/3154163535 >> v1:172.21.15.80:6789/0 conn(0x7f80880b7350 legacy=0x7f80880b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808d2eb640 1 -- 172.21.15.80:0/3154163535 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 579336539 0 0) 0x7f80800014a0 con 0x7f80880b7350 2024-04-25T13:41:14.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808daec640 1 -- 172.21.15.80:0/3154163535 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80880d2680 con 0x7f80880d0940 2024-04-25T13:41:14.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808daec640 1 --2- 172.21.15.80:0/3154163535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d0940 0x7f80880b6e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f808806ff60 tx=0x7f807801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fbdecd69d1a466d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:14.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.291+0000 7f808d2eb640 1 -- 172.21.15.80:0/3154163535 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80780b0070 con 0x7f80880d0940 2024-04-25T13:41:14.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f808d2eb640 1 -- 172.21.15.80:0/3154163535 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80780ab600 con 0x7f80880d0940 2024-04-25T13:41:14.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f808d2eb640 1 -- 172.21.15.80:0/3154163535 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80780ab9d0 con 0x7f80880d0940 2024-04-25T13:41:14.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 -- 172.21.15.80:0/3154163535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d0940 msgr2=0x7f80880b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 --2- 172.21.15.80:0/3154163535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d0940 0x7f80880b6e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f808806ff60 tx=0x7f807801c480 comp rx=0 tx=0).stop 2024-04-25T13:41:14.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 -- 172.21.15.80:0/3154163535 shutdown_connections 2024-04-25T13:41:14.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 --2- 172.21.15.80:0/3154163535 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d0940 0x7f80880b6e10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 --2- 172.21.15.80:0/3154163535 >> v2:172.21.15.87:3300/0 conn(0x7f80880d4690 0x7f80880d0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 -- 172.21.15.80:0/3154163535 >> 172.21.15.80:0/3154163535 conn(0x7f80880d3b70 msgr2=0x7f80880d3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:14.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 -- 172.21.15.80:0/3154163535 shutdown_connections 2024-04-25T13:41:14.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 -- 172.21.15.80:0/3154163535 wait complete. 2024-04-25T13:41:14.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 Processor -- start 2024-04-25T13:41:14.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.295+0000 7f8090551640 1 -- start start 2024-04-25T13:41:14.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80880b7350 0x7f808806cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80880d0940 0x7f8088153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d4690 0x7f808815f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f80880b5470 con 0x7f80880d4690 2024-04-25T13:41:14.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f80880cfaf0 con 0x7f80880b7350 2024-04-25T13:41:14.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f808816b7b0 con 0x7f80880d0940 2024-04-25T13:41:14.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d4690 0x7f808815f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d4690 0x7f808815f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53074/0 (socket says 172.21.15.80:53074) 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 -- 172.21.15.80:0/3405625664 learned_addr learned my addr 172.21.15.80:0/3405625664 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808e2ed640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80880b7350 0x7f808806cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808daec640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80880d0940 0x7f8088153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 -- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80880d0940 msgr2=0x7f8088153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80880d0940 0x7f8088153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 -- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80880b7350 msgr2=0x7f808806cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80880b7350 0x7f808806cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 -- 172.21.15.80:0/3405625664 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8088161000 con 0x7f80880d4690 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808eaee640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d4690 0x7f808815f6d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f808000c700 tx=0x7f808000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f807effd640 1 -- 172.21.15.80:0/3405625664 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8080018070 con 0x7f80880d4690 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f807effd640 1 -- 172.21.15.80:0/3405625664 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f808000ce10 con 0x7f80880d4690 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80881612e0 con 0x7f80880d4690 2024-04-25T13:41:14.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f807effd640 1 -- 172.21.15.80:0/3405625664 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f808000a860 con 0x7f80880d4690 2024-04-25T13:41:14.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80880ccb80 con 0x7f80880d4690 2024-04-25T13:41:14.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808806a3a0 con 0x7f80880d4690 2024-04-25T13:41:14.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f807effd640 1 -- 172.21.15.80:0/3405625664 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f808001b070 con 0x7f80880d4690 2024-04-25T13:41:14.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f807effd640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f806c06f400 0x7f806c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f807effd640 1 -- 172.21.15.80:0/3405625664 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f808008e7e0 con 0x7f80880d4690 2024-04-25T13:41:14.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.299+0000 7f808e2ed640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f806c06f400 0x7f806c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.303+0000 7f808e2ed640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f806c06f400 0x7f806c071910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8084006fd0 tx=0x7f8084008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:14.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.303+0000 7f807effd640 1 -- 172.21.15.80:0/3405625664 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f80880d0d50 con 0x7f80880d4690 2024-04-25T13:41:14.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.455+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f80880cffd0 con 0x7f80880d4690 2024-04-25T13:41:14.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.455+0000 7f807effd640 1 -- 172.21.15.80:0/3405625664 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f808005b760 con 0x7f80880d4690 2024-04-25T13:41:14.459 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:14.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f806c06f400 msgr2=0x7f806c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f806c06f400 0x7f806c071910 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8084006fd0 tx=0x7f8084008040 comp rx=0 tx=0).stop 2024-04-25T13:41:14.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d4690 msgr2=0x7f808815f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d4690 0x7f808815f6d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f808000c700 tx=0x7f808000cbc0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 shutdown_connections 2024-04-25T13:41:14.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f806c06f400 0x7f806c071910 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80880d4690 0x7f808815f6d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80880d0940 0x7f8088153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 --2- 172.21.15.80:0/3405625664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80880b7350 0x7f808806cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 >> 172.21.15.80:0/3405625664 conn(0x7f80880d3b70 msgr2=0x7f8088157dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:14.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 shutdown_connections 2024-04-25T13:41:14.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.459+0000 7f8090551640 1 -- 172.21.15.80:0/3405625664 wait complete. 2024-04-25T13:41:14.479 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:14.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.611+0000 7f91f3fe6640 1 Processor -- start 2024-04-25T13:41:14.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.611+0000 7f91f3fe6640 1 -- start start 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f3fe6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f91ec06f5a0 0x7f91ec160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f3fe6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec0733b0 0x7f91ec161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f3fe6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91ec06fcf0 con 0x7f91ec161070 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f3fe6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f91ec1619c0 con 0x7f91ec06f5a0 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f3fe6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec073f10 con 0x7f91ec0733b0 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f1581640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f91ec161070 0x7f91ec0b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36928/0 (socket says 172.21.15.80:36928) 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f1581640 1 -- 172.21.15.80:0/823028959 learned_addr learned my addr 172.21.15.80:0/823028959 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f1d82640 1 --2- 172.21.15.80:0/823028959 >> v2:172.21.15.80:3300/0 conn(0x7f91ec06f5a0 0x7f91ec160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f2583640 1 --2- 172.21.15.80:0/823028959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec0733b0 0x7f91ec161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f0d80640 1 -- 172.21.15.80:0/823028959 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3297267755 0 0) 0x7f91ec06fcf0 con 0x7f91ec161070 2024-04-25T13:41:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f0d80640 1 -- 172.21.15.80:0/823028959 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91d8003530 con 0x7f91ec161070 2024-04-25T13:41:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f0d80640 1 -- 172.21.15.80:0/823028959 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4015081640 0 0) 0x7f91d8003530 con 0x7f91ec161070 2024-04-25T13:41:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f1d82640 1 -- 172.21.15.80:0/823028959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec0733b0 msgr2=0x7f91ec161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f1d82640 1 --2- 172.21.15.80:0/823028959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec0733b0 0x7f91ec161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f1d82640 1 -- 172.21.15.80:0/823028959 >> v1:172.21.15.80:6789/0 conn(0x7f91ec161070 legacy=0x7f91ec0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f1d82640 1 -- 172.21.15.80:0/823028959 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec0b0c60 con 0x7f91ec06f5a0 2024-04-25T13:41:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f1d82640 1 --2- 172.21.15.80:0/823028959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec06f5a0 0x7f91ec160b30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f91e80097f0 tx=0x7f91e801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2982fea2091bb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f0d80640 1 -- 172.21.15.80:0/823028959 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e80b0070 con 0x7f91ec06f5a0 2024-04-25T13:41:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f0d80640 1 -- 172.21.15.80:0/823028959 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e80ab400 con 0x7f91ec06f5a0 2024-04-25T13:41:14.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.615+0000 7f91f0d80640 1 -- 172.21.15.80:0/823028959 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91dc001680 con 0x7f91ec06f5a0 2024-04-25T13:41:14.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- 172.21.15.80:0/823028959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec06f5a0 msgr2=0x7f91ec160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/823028959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec06f5a0 0x7f91ec160b30 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f91e80097f0 tx=0x7f91e801c330 comp rx=0 tx=0).stop 2024-04-25T13:41:14.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- 172.21.15.80:0/823028959 shutdown_connections 2024-04-25T13:41:14.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/823028959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec0733b0 0x7f91ec161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/823028959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec06f5a0 0x7f91ec160b30 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- 172.21.15.80:0/823028959 >> 172.21.15.80:0/823028959 conn(0x7f91ec06d080 msgr2=0x7f91ec0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:14.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- 172.21.15.80:0/823028959 shutdown_connections 2024-04-25T13:41:14.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- 172.21.15.80:0/823028959 wait complete. 2024-04-25T13:41:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 Processor -- start 2024-04-25T13:41:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- start start 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec06f5a0 0x7f91ec151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ec0733b0 0x7f91ec13f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec161070 0x7f91ec158a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec16a050 con 0x7f91ec161070 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f91ec169ed0 con 0x7f91ec0733b0 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f3fe6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91ec16a1d0 con 0x7f91ec06f5a0 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.619+0000 7f91f2583640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec161070 0x7f91ec158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f2583640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec161070 0x7f91ec158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53084/0 (socket says 172.21.15.80:53084) 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f2583640 1 -- 172.21.15.80:0/2544734997 learned_addr learned my addr 172.21.15.80:0/2544734997 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f1d82640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec06f5a0 0x7f91ec151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f1581640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ec0733b0 0x7f91ec13f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f2583640 1 -- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec06f5a0 msgr2=0x7f91ec151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f2583640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec06f5a0 0x7f91ec151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f2583640 1 -- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ec0733b0 msgr2=0x7f91ec13f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f2583640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ec0733b0 0x7f91ec13f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f2583640 1 -- 172.21.15.80:0/2544734997 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ec156720 con 0x7f91ec161070 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f2583640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec161070 0x7f91ec158a00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f91e400ec10 tx=0x7f91e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91e27fc640 1 -- 172.21.15.80:0/2544734997 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e4018020 con 0x7f91ec161070 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91e27fc640 1 -- 172.21.15.80:0/2544734997 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91e400cce0 con 0x7f91ec161070 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ec13f780 con 0x7f91ec161070 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91e27fc640 1 -- 172.21.15.80:0/2544734997 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91e40146c0 con 0x7f91ec161070 2024-04-25T13:41:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ec153f40 con 0x7f91ec161070 2024-04-25T13:41:14.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91ec06a3a0 con 0x7f91ec161070 2024-04-25T13:41:14.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91e27fc640 1 -- 172.21.15.80:0/2544734997 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f91e401b020 con 0x7f91ec161070 2024-04-25T13:41:14.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91e27fc640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91d006f430 0x7f91d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:14.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91e27fc640 1 -- 172.21.15.80:0/2544734997 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f91e408fd90 con 0x7f91ec161070 2024-04-25T13:41:14.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.623+0000 7f91f1d82640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91d006f430 0x7f91d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:14.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.627+0000 7f91f1d82640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91d006f430 0x7f91d0071940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91e80ad000 tx=0x7f91e80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:14.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.627+0000 7f91e27fc640 1 -- 172.21.15.80:0/2544734997 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91e4022030 con 0x7f91ec161070 2024-04-25T13:41:14.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.787+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f91ec0b1530 con 0x7f91ec161070 2024-04-25T13:41:14.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.787+0000 7f91e27fc640 1 -- 172.21.15.80:0/2544734997 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) ==== 80+0+0 (secure 0 0 0) 0x7f91e4010040 con 0x7f91ec161070 2024-04-25T13:41:14.789 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:14.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91d006f430 msgr2=0x7f91d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91d006f430 0x7f91d0071940 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91e80ad000 tx=0x7f91e80a6000 comp rx=0 tx=0).stop 2024-04-25T13:41:14.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec161070 msgr2=0x7f91ec158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec161070 0x7f91ec158a00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f91e400ec10 tx=0x7f91e400c490 comp rx=0 tx=0).stop 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 shutdown_connections 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91d006f430 0x7f91d0071940 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f91e80ad000 tx=0x7f91e80a6000 comp rx=0 tx=0).stop 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ec161070 0x7f91ec158a00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ec0733b0 0x7f91ec13f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 --2- 172.21.15.80:0/2544734997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ec06f5a0 0x7f91ec151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 >> 172.21.15.80:0/2544734997 conn(0x7f91ec06d080 msgr2=0x7f91ec0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 shutdown_connections 2024-04-25T13:41:14.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:14.791+0000 7f91f3fe6640 1 -- 172.21.15.80:0/2544734997 wait complete. 2024-04-25T13:41:14.809 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:41:14.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:14.809 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T13:41:14.820 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:41:14.821 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:41:14.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:14.821 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T13:41:14.871 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:14.871 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:14.878 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:14.878 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:14.924 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4e6b6640 1 Processor -- start 2024-04-25T13:41:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4e6b6640 1 -- start start 2024-04-25T13:41:15.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4e6b6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a480d59b0 0x7f4a480d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4e6b6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4a480d4330 0x7f4a480d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4e6b6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a480d6300 con 0x7f4a480d2e80 2024-04-25T13:41:15.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4e6b6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4a48161060 con 0x7f4a480d4330 2024-04-25T13:41:15.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4e6b6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a48161340 con 0x7f4a480d59b0 2024-04-25T13:41:15.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a47fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4a480d2e80 0x7f4a481607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36930/0 (socket says 172.21.15.80:36930) 2024-04-25T13:41:15.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a47fff640 1 -- 172.21.15.80:0/2820727538 learned_addr learned my addr 172.21.15.80:0/2820727538 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3413433271 0 0) 0x7f4a380014a0 con 0x7f4a480d2e80 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a477fe640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a480d59b0 0x7f4a480d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a30003530 con 0x7f4a480d2e80 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4cc53640 1 --2- 172.21.15.80:0/2820727538 >> v2:172.21.15.87:3300/0 conn(0x7f4a480d4330 0x7f4a480d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2908145594 0 0) 0x7f4a30003530 con 0x7f4a480d2e80 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a480d59b0 msgr2=0x7f4a480d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a480d59b0 0x7f4a480d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 >> v2:172.21.15.87:3300/0 conn(0x7f4a480d4330 msgr2=0x7f4a480d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 --2- 172.21.15.80:0/2820727538 >> v2:172.21.15.87:3300/0 conn(0x7f4a480d4330 0x7f4a480d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a480d2680 con 0x7f4a480d2e80 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4a38001bd0 con 0x7f4a480d2e80 2024-04-25T13:41:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 >> v1:172.21.15.80:6789/0 conn(0x7f4a480d2e80 legacy=0x7f4a481607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a30003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a30004810 0x7f4a30004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a480d59b0 0x7f4a300051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48161060 con 0x7f4a300039c0 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a48161340 con 0x7f4a30004810 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3000bb30 con 0x7f4a480d59b0 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4cc53640 1 --2- 172.21.15.80:0/2820727538 >> v2:172.21.15.87:3300/0 conn(0x7f4a480d4330 0x7f4a480d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a47fff640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a30003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a477fe640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a30004810 0x7f4a30004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a4cc53640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a480d59b0 0x7f4a300051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a47fff640 1 -- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a480d59b0 msgr2=0x7f4a300051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a47fff640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a480d59b0 0x7f4a300051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a47fff640 1 -- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a30004810 msgr2=0x7f4a30004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a47fff640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a30004810 0x7f4a30004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.075+0000 7f4a47fff640 1 -- 172.21.15.80:0/2820727538 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a30003530 con 0x7f4a300039c0 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a47fff640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a30003e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800aa40 tx=0x7f4a3801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c247e720f660e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3800a690 con 0x7f4a300039c0 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a380072f0 con 0x7f4a300039c0 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a46ffd640 1 -- 172.21.15.80:0/2820727538 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a380076e0 con 0x7f4a300039c0 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/2820727538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 msgr2=0x7f4a30003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a4e6b6640 1 --2- 172.21.15.80:0/2820727538 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a30003e30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4a3800aa40 tx=0x7f4a3801d380 comp rx=0 tx=0).stop 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a4cc53640 1 -- 172.21.15.80:0/2820727538 reap_dead start 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/2820727538 shutdown_connections 2024-04-25T13:41:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/2820727538 >> 172.21.15.80:0/2820727538 conn(0x7f4a480737a0 msgr2=0x7f4a4806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:15.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/2820727538 shutdown_connections 2024-04-25T13:41:15.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.079+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/2820727538 wait complete. 2024-04-25T13:41:15.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 Processor -- start 2024-04-25T13:41:15.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 -- start start 2024-04-25T13:41:15.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a481f9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a48159cc0 0x7f4a4815ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4815f3a0 0x7f4a481658d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480d3410 con 0x7f4a300039c0 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a480d3290 con 0x7f4a4815f3a0 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4815df60 con 0x7f4a48159cc0 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a481f9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a481f9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53100/0 (socket says 172.21.15.80:53100) 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 -- 172.21.15.80:0/3572730296 learned_addr learned my addr 172.21.15.80:0/3572730296 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4cc53640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4815f3a0 0x7f4a481658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 -- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a48159cc0 msgr2=0x7f4a4815ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a48159cc0 0x7f4a4815ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 -- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4815f3a0 msgr2=0x7f4a481658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4815f3a0 0x7f4a481658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 -- 172.21.15.80:0/3572730296 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4815f7b0 con 0x7f4a300039c0 2024-04-25T13:41:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4cc53640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4815f3a0 0x7f4a481658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:15.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a47fff640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a481f9e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4a38007040 tx=0x7f4a30005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a44ff9640 1 -- 172.21.15.80:0/3572730296 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a380b2040 con 0x7f4a300039c0 2024-04-25T13:41:15.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a44ff9640 1 -- 172.21.15.80:0/3572730296 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a380014a0 con 0x7f4a300039c0 2024-04-25T13:41:15.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a48164e30 con 0x7f4a300039c0 2024-04-25T13:41:15.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a44ff9640 1 -- 172.21.15.80:0/3572730296 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a38001990 con 0x7f4a300039c0 2024-04-25T13:41:15.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.083+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a480bb530 con 0x7f4a300039c0 2024-04-25T13:41:15.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.087+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4806a3a0 con 0x7f4a300039c0 2024-04-25T13:41:15.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.087+0000 7f4a44ff9640 1 -- 172.21.15.80:0/3572730296 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4a380bf020 con 0x7f4a300039c0 2024-04-25T13:41:15.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.087+0000 7f4a44ff9640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a3406f400 0x7f4a34071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.087+0000 7f4a44ff9640 1 -- 172.21.15.80:0/3572730296 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f4a38129bc0 con 0x7f4a300039c0 2024-04-25T13:41:15.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.087+0000 7f4a477fe640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a3406f400 0x7f4a34071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.087+0000 7f4a477fe640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a3406f400 0x7f4a34071910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c006aa0 tx=0x7f4a3c006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.091+0000 7f4a44ff9640 1 -- 172.21.15.80:0/3572730296 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a380c6020 con 0x7f4a300039c0 2024-04-25T13:41:15.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.251+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a48160610 con 0x7f4a300039c0 2024-04-25T13:41:15.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a44ff9640 1 -- 172.21.15.80:0/3572730296 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16533 (secure 0 0 0) 0x7f4a4815f7b0 con 0x7f4a300039c0 2024-04-25T13:41:15.256 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:15.257 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":20,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:40:31.176828+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a3406f400 msgr2=0x7f4a34071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a3406f400 0x7f4a34071910 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c006aa0 tx=0x7f4a3c006a30 comp rx=0 tx=0).stop 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 msgr2=0x7f4a481f9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a481f9e00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f4a38007040 tx=0x7f4a30005840 comp rx=0 tx=0).stop 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 shutdown_connections 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a3406f400 0x7f4a34071910 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a4815f3a0 0x7f4a481658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a48159cc0 0x7f4a4815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 --2- 172.21.15.80:0/3572730296 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a300039c0 0x7f4a481f9e00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 >> 172.21.15.80:0/3572730296 conn(0x7f4a480737a0 msgr2=0x7f4a480757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:15.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 shutdown_connections 2024-04-25T13:41:15.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.255+0000 7f4a4e6b6640 1 -- 172.21.15.80:0/3572730296 wait complete. 2024-04-25T13:41:15.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:15.279 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:15.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8ebb0640 1 Processor -- start 2024-04-25T13:41:15.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8ebb0640 1 -- start start 2024-04-25T13:41:15.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8ebb0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f88162270 0x7f7f8806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8ebb0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f880b5740 con 0x7f7f880b5330 2024-04-25T13:41:15.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8ebb0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f8806dd10 con 0x7f7f8806edb0 2024-04-25T13:41:15.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8ebb0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f8806f700 con 0x7f7f88162270 2024-04-25T13:41:15.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8c94c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7f880b5330 0x7f7f880d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36940/0 (socket says 172.21.15.80:36940) 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8c94c640 1 -- 172.21.15.80:0/1369146647 learned_addr learned my addr 172.21.15.80:0/1369146647 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 549831096 0 0) 0x7f7f880b5740 con 0x7f7f880b5330 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f78003530 con 0x7f7f880b5330 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8d14d640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f88162270 0x7f7f8806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4097638628 0 0) 0x7f7f78003530 con 0x7f7f880b5330 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f88162270 msgr2=0x7f7f8806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f88162270 0x7f7f8806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 >> v1:172.21.15.87:6789/0 conn(0x7f7f8806edb0 legacy=0x7f7f88161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f8806dff0 con 0x7f7f880b5330 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7f7c001680 con 0x7f7f880b5330 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 >> v1:172.21.15.80:6789/0 conn(0x7f7f880b5330 legacy=0x7f7f880d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f780039f0 0x7f7f78003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f780048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f780050d0 0x7f7f7800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7f8806f700 con 0x7f7f78004380 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7800c020 con 0x7f7f780050d0 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7800c220 con 0x7f7f780039f0 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f87fff640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f780048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f8c94c640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f780039f0 0x7f7f78003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f87fff640 1 -- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f780039f0 msgr2=0x7f7f78003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f87fff640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f780039f0 0x7f7f78003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.431+0000 7f7f87fff640 1 -- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f780050d0 msgr2=0x7f7f7800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f87fff640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f780050d0 0x7f7f7800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f87fff640 1 -- 172.21.15.80:0/1369146647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f78003530 con 0x7f7f78004380 2024-04-25T13:41:15.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f87fff640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f780048d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7f74001780 tx=0x7f7f7401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac716569dfb5745 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f740013a0 con 0x7f7f78004380 2024-04-25T13:41:15.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f740052f0 con 0x7f7f78004380 2024-04-25T13:41:15.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f877fe640 1 -- 172.21.15.80:0/1369146647 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f74003840 con 0x7f7f78004380 2024-04-25T13:41:15.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1369146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 msgr2=0x7f7f780048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f8ebb0640 1 --2- 172.21.15.80:0/1369146647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f780048d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7f74001780 tx=0x7f7f7401c420 comp rx=0 tx=0).stop 2024-04-25T13:41:15.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f8d14d640 1 -- 172.21.15.80:0/1369146647 reap_dead start 2024-04-25T13:41:15.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1369146647 shutdown_connections 2024-04-25T13:41:15.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1369146647 >> 172.21.15.80:0/1369146647 conn(0x7f7f880b4160 msgr2=0x7f7f880d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:15.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1369146647 shutdown_connections 2024-04-25T13:41:15.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.435+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1369146647 wait complete. 2024-04-25T13:41:15.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 Processor -- start 2024-04-25T13:41:15.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 -- start start 2024-04-25T13:41:15.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f8813df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f8813e8b0 0x7f7f881442e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f88144820 0x7f7f88140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7f880b0e50 con 0x7f7f78004380 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f880b0cd0 con 0x7f7f88144820 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f880d11c0 con 0x7f7f8813e8b0 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f8813df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f8813df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53128/0 (socket says 172.21.15.80:53128) 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 -- 172.21.15.80:0/1680687716 learned_addr learned my addr 172.21.15.80:0/1680687716 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f87fff640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f8813e8b0 0x7f7f881442e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8d14d640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f88144820 0x7f7f88140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 -- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f8813e8b0 msgr2=0x7f7f881442e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f8813e8b0 0x7f7f881442e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 -- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f88144820 msgr2=0x7f7f88140d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f88144820 0x7f7f88140d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 -- 172.21.15.80:0/1680687716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f88142fe0 con 0x7f7f78004380 2024-04-25T13:41:15.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8c94c640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f8813df50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c007fd0 tx=0x7f7f7c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f857fa640 1 -- 172.21.15.80:0/1680687716 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c019070 con 0x7f7f78004380 2024-04-25T13:41:15.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f857fa640 1 -- 172.21.15.80:0/1680687716 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f7c0072d0 con 0x7f7f78004380 2024-04-25T13:41:15.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f88144c30 con 0x7f7f78004380 2024-04-25T13:41:15.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f857fa640 1 -- 172.21.15.80:0/1680687716 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f7c0047f0 con 0x7f7f78004380 2024-04-25T13:41:15.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.439+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f8815c190 con 0x7f7f78004380 2024-04-25T13:41:15.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.443+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f88159450 con 0x7f7f78004380 2024-04-25T13:41:15.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.443+0000 7f7f857fa640 1 -- 172.21.15.80:0/1680687716 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7f7c01c070 con 0x7f7f78004380 2024-04-25T13:41:15.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.443+0000 7f7f857fa640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f6c06f430 0x7f7f6c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.443+0000 7f7f857fa640 1 -- 172.21.15.80:0/1680687716 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f7f7c08ed20 con 0x7f7f78004380 2024-04-25T13:41:15.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.443+0000 7f7f87fff640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f6c06f430 0x7f7f6c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.443+0000 7f7f87fff640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f6c06f430 0x7f7f6c071940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7f74001750 tx=0x7f7f740011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.447+0000 7f7f857fa640 1 -- 172.21.15.80:0/1680687716 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f88144c30 con 0x7f7f78004380 2024-04-25T13:41:15.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.607+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7f8806a3a0 con 0x7f7f78004380 2024-04-25T13:41:15.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.607+0000 7f7f857fa640 1 -- 172.21.15.80:0/1680687716 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) ==== 93+0+10 (secure 0 0 0) 0x7f7f7c05bb70 con 0x7f7f78004380 2024-04-25T13:41:15.611 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:15.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f6c06f430 msgr2=0x7f7f6c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f6c06f430 0x7f7f6c071940 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7f74001750 tx=0x7f7f740011a0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 msgr2=0x7f7f8813df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f8813df50 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c007fd0 tx=0x7f7f7c00dc00 comp rx=0 tx=0).stop 2024-04-25T13:41:15.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 shutdown_connections 2024-04-25T13:41:15.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f6c06f430 0x7f7f6c071940 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f88144820 0x7f7f88140d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f8813e8b0 0x7f7f881442e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 --2- 172.21.15.80:0/1680687716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f78004380 0x7f7f8813df50 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 >> 172.21.15.80:0/1680687716 conn(0x7f7f880b4160 msgr2=0x7f7f880b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:15.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 shutdown_connections 2024-04-25T13:41:15.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.611+0000 7f7f8ebb0640 1 -- 172.21.15.80:0/1680687716 wait complete. 2024-04-25T13:41:15.632 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:15.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dc62e640 1 Processor -- start 2024-04-25T13:41:15.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dc62e640 1 -- start start 2024-04-25T13:41:15.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dc62e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f51d40d3ed0 0x7f51d40d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dc62e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d40d2ad0 0x7f51d40b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dc62e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51d40b92a0 con 0x7f51d40b7c10 2024-04-25T13:41:15.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dc62e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f51d40d4c10 con 0x7f51d40d3ed0 2024-04-25T13:41:15.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dc62e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51d40d4ef0 con 0x7f51d40d2ad0 2024-04-25T13:41:15.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dabcb640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f51d40b7c10 0x7f51d40b9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36944/0 (socket says 172.21.15.80:36944) 2024-04-25T13:41:15.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dabcb640 1 -- 172.21.15.80:0/2412591104 learned_addr learned my addr 172.21.15.80:0/2412591104 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d9bc9640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d40d2ad0 0x7f51d40b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820339641 0 0) 0x7f51cc0014a0 con 0x7f51d40b7c10 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51da3ca640 1 --2- 172.21.15.80:0/2412591104 >> v2:172.21.15.87:3300/0 conn(0x7f51d40d3ed0 0x7f51d40d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51c0003530 con 0x7f51d40b7c10 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1486716331 0 0) 0x7f51c0003530 con 0x7f51d40b7c10 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d40d2ad0 msgr2=0x7f51d40b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d40d2ad0 0x7f51d40b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 >> v2:172.21.15.87:3300/0 conn(0x7f51d40d3ed0 msgr2=0x7f51d40d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 --2- 172.21.15.80:0/2412591104 >> v2:172.21.15.87:3300/0 conn(0x7f51d40d3ed0 0x7f51d40d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d40d51d0 con 0x7f51d40b7c10 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51da3ca640 1 --2- 172.21.15.80:0/2412591104 >> v2:172.21.15.87:3300/0 conn(0x7f51d40d3ed0 0x7f51d40d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f51cc001bd0 con 0x7f51d40b7c10 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 >> v1:172.21.15.80:6789/0 conn(0x7f51d40b7c10 legacy=0x7f51d40b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51c00039c0 0x7f51c0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d40d2ad0 0x7f51c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51c0004d50 0x7f51c0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40d4c10 con 0x7f51c0004d50 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40d4ef0 con 0x7f51c00039c0 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51c000bb30 con 0x7f51d40d2ad0 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dabcb640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51c0004d50 0x7f51c0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51d9bc9640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d40d2ad0 0x7f51c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51da3ca640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51c00039c0 0x7f51c0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dabcb640 1 -- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d40d2ad0 msgr2=0x7f51c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dabcb640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d40d2ad0 0x7f51c0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dabcb640 1 -- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51c00039c0 msgr2=0x7f51c0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dabcb640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51c00039c0 0x7f51c0003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.767+0000 7f51dabcb640 1 -- 172.21.15.80:0/2412591104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51c0003530 con 0x7f51c0004d50 2024-04-25T13:41:15.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51dabcb640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51c0004d50 0x7f51c0005160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00aa40 tx=0x7f51cc01fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aaac04675c201a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00a690 con 0x7f51c0004d50 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc0aa710 con 0x7f51c0004d50 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51d93c8640 1 -- 172.21.15.80:0/2412591104 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0aab00 con 0x7f51c0004d50 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51dc62e640 1 -- 172.21.15.80:0/2412591104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51c0004d50 msgr2=0x7f51c0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51dc62e640 1 --2- 172.21.15.80:0/2412591104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51c0004d50 0x7f51c0005160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00aa40 tx=0x7f51cc01fae0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51dabcb640 1 -- 172.21.15.80:0/2412591104 reap_dead start 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51dc62e640 1 -- 172.21.15.80:0/2412591104 shutdown_connections 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51dc62e640 1 -- 172.21.15.80:0/2412591104 >> 172.21.15.80:0/2412591104 conn(0x7f51d40737b0 msgr2=0x7f51d406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51dc62e640 1 -- 172.21.15.80:0/2412591104 shutdown_connections 2024-04-25T13:41:15.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.771+0000 7f51dc62e640 1 -- 172.21.15.80:0/2412591104 wait complete. 2024-04-25T13:41:15.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 Processor -- start 2024-04-25T13:41:15.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 -- start start 2024-04-25T13:41:15.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d4154530 0x7f51d415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51d41f6c40 0x7f51d41f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51d4158550 0x7f51d4158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40cc960 con 0x7f51d4158550 2024-04-25T13:41:15.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f51d40d2ee0 con 0x7f51d41f6c40 2024-04-25T13:41:15.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51d40ccae0 con 0x7f51d4154530 2024-04-25T13:41:15.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51d4158550 0x7f51d4158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51d4158550 0x7f51d4158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53146/0 (socket says 172.21.15.80:53146) 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 -- 172.21.15.80:0/2967790024 learned_addr learned my addr 172.21.15.80:0/2967790024 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51d9bc9640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51d41f6c40 0x7f51d41f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51da3ca640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d4154530 0x7f51d415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 -- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d4154530 msgr2=0x7f51d415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d4154530 0x7f51d415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 -- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51d41f6c40 msgr2=0x7f51d41f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51d41f6c40 0x7f51d41f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 -- 172.21.15.80:0/2967790024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d41f79e0 con 0x7f51d4158550 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dabcb640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51d4158550 0x7f51d4158960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00ab30 tx=0x7f51c0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51caffd640 1 -- 172.21.15.80:0/2967790024 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00a690 con 0x7f51d4158550 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51caffd640 1 -- 172.21.15.80:0/2967790024 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51cc0014a0 con 0x7f51d4158550 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d41f7cc0 con 0x7f51d4158550 2024-04-25T13:41:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51caffd640 1 -- 172.21.15.80:0/2967790024 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0aaac0 con 0x7f51d4158550 2024-04-25T13:41:15.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d40c8450 con 0x7f51d4158550 2024-04-25T13:41:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.775+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d406a3a0 con 0x7f51d4158550 2024-04-25T13:41:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.779+0000 7f51caffd640 1 -- 172.21.15.80:0/2967790024 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f51cc0aac60 con 0x7f51d4158550 2024-04-25T13:41:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.779+0000 7f51caffd640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f51b806f400 0x7f51b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.779+0000 7f51caffd640 1 -- 172.21.15.80:0/2967790024 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f51cc12c2b0 con 0x7f51d4158550 2024-04-25T13:41:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.779+0000 7f51da3ca640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f51b806f400 0x7f51b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.779+0000 7f51da3ca640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f51b806f400 0x7f51b8071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f51d4149b60 tx=0x7f51c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:15.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.783+0000 7f51caffd640 1 -- 172.21.15.80:0/2967790024 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f51cc0ae030 con 0x7f51d4158550 2024-04-25T13:41:15.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.939+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f51d40ca430 con 0x7f51d4158550 2024-04-25T13:41:15.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.939+0000 7f51caffd640 1 -- 172.21.15.80:0/2967790024 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) ==== 104+0+11 (secure 0 0 0) 0x7f51d40b92a0 con 0x7f51d4158550 2024-04-25T13:41:15.941 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:15.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f51b806f400 msgr2=0x7f51b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f51b806f400 0x7f51b8071910 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f51d4149b60 tx=0x7f51c4008040 comp rx=0 tx=0).stop 2024-04-25T13:41:15.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51d4158550 msgr2=0x7f51d4158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:15.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51d4158550 0x7f51d4158960 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f51cc00ab30 tx=0x7f51c0005840 comp rx=0 tx=0).stop 2024-04-25T13:41:15.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 shutdown_connections 2024-04-25T13:41:15.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f51b806f400 0x7f51b8071910 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51d4158550 0x7f51d4158960 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51d41f6c40 0x7f51d41f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 --2- 172.21.15.80:0/2967790024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51d4154530 0x7f51d415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:15.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 >> 172.21.15.80:0/2967790024 conn(0x7f51d40737b0 msgr2=0x7f51d415f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:15.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 shutdown_connections 2024-04-25T13:41:15.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:15.943+0000 7f51dc62e640 1 -- 172.21.15.80:0/2967790024 wait complete. 2024-04-25T13:41:15.963 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:16.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f8461a83640 1 Processor -- start 2024-04-25T13:41:16.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f8461a83640 1 -- start start 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f8461a83640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f845c0d1e90 0x7f845c0b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f8461a83640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f845c0d4a30 con 0x7f845c0d1a80 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f8461a83640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f845c0b4490 con 0x7f845c0b6e60 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f8461a83640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f845c0d2b40 con 0x7f845c0d1e90 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845b7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f845c0d1a80 0x7f845c0cf4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36954/0 (socket says 172.21.15.80:36954) 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845b7fe640 1 -- 172.21.15.80:0/3324107742 learned_addr learned my addr 172.21.15.80:0/3324107742 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4016330105 0 0) 0x7f845c0d4a30 con 0x7f845c0d1a80 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8444003530 con 0x7f845c0d1a80 2024-04-25T13:41:16.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845affd640 1 --2- 172.21.15.80:0/3324107742 >> v2:172.21.15.87:3300/0 conn(0x7f845c0d1e90 0x7f845c0b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2458557797 0 0) 0x7f8444003530 con 0x7f845c0d1a80 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 >> v2:172.21.15.87:3300/0 conn(0x7f845c0d1e90 msgr2=0x7f845c0b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 --2- 172.21.15.80:0/3324107742 >> v2:172.21.15.87:3300/0 conn(0x7f845c0d1e90 0x7f845c0b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 >> v1:172.21.15.87:6789/0 conn(0x7f845c0b6e60 legacy=0x7f845c0d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c0b4cd0 con 0x7f845c0d1a80 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f8454001a00 con 0x7f845c0d1a80 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845affd640 1 --2- 172.21.15.80:0/3324107742 >> v2:172.21.15.87:3300/0 conn(0x7f845c0d1e90 0x7f845c0b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 >> v1:172.21.15.80:6789/0 conn(0x7f845c0d1a80 legacy=0x7f845c0cf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84440039c0 0x7f8444003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8444004810 0x7f8444004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8444005310 0x7f844400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f845c0d2b40 con 0x7f84440039c0 2024-04-25T13:41:16.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f844400c2b0 con 0x7f8444004810 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f844400c4b0 con 0x7f8444005310 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845b7fe640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84440039c0 0x7f8444003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845affd640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8444004810 0x7f8444004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845b7fe640 1 -- 172.21.15.80:0/3324107742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8444005310 msgr2=0x7f844400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845bfff640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8444005310 0x7f844400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845b7fe640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8444005310 0x7f844400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845b7fe640 1 -- 172.21.15.80:0/3324107742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8444004810 msgr2=0x7f8444004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845b7fe640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8444004810 0x7f8444004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845b7fe640 1 -- 172.21.15.80:0/3324107742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8444003530 con 0x7f84440039c0 2024-04-25T13:41:16.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.111+0000 7f845bfff640 1 -- 172.21.15.80:0/3324107742 reap_dead start 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f845b7fe640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84440039c0 0x7f8444003e30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f845400a830 tx=0x7f845401d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897aba130ab3db0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845400a480 con 0x7f84440039c0 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84540ad430 con 0x7f84440039c0 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f845a7fc640 1 -- 172.21.15.80:0/3324107742 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84540ad8a0 con 0x7f84440039c0 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 -- 172.21.15.80:0/3324107742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84440039c0 msgr2=0x7f8444003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84440039c0 0x7f8444003e30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f845400a830 tx=0x7f845401d360 comp rx=0 tx=0).stop 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 -- 172.21.15.80:0/3324107742 shutdown_connections 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 --2- 172.21.15.80:0/3324107742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f84440039c0 0x7f8444003e30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 -- 172.21.15.80:0/3324107742 >> 172.21.15.80:0/3324107742 conn(0x7f845c0d3ad0 msgr2=0x7f845c0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 -- 172.21.15.80:0/3324107742 shutdown_connections 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 -- 172.21.15.80:0/3324107742 wait complete. 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 Processor -- start 2024-04-25T13:41:16.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.115+0000 7f8461a83640 1 -- start start 2024-04-25T13:41:16.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84440039c0 0x7f845c0cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f845c1faba0 0x7f845c1fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f845c1fd770 0x7f845c1fc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f845c162770 con 0x7f845c1fd770 2024-04-25T13:41:16.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f845c0d2760 con 0x7f84440039c0 2024-04-25T13:41:16.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f845c1628f0 con 0x7f845c1faba0 2024-04-25T13:41:16.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f845c1fd770 0x7f845c1fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f845c1fd770 0x7f845c1fc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53166/0 (socket says 172.21.15.80:53166) 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 -- 172.21.15.80:0/4018548466 learned_addr learned my addr 172.21.15.80:0/4018548466 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845affd640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f845c1faba0 0x7f845c1fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845b7fe640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84440039c0 0x7f845c0cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 -- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f845c1faba0 msgr2=0x7f845c1fd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f845c1faba0 0x7f845c1fd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 -- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84440039c0 msgr2=0x7f845c0cce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84440039c0 0x7f845c0cce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 -- 172.21.15.80:0/4018548466 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f845c0cd9c0 con 0x7f845c1fd770 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845b7fe640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84440039c0 0x7f845c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f845bfff640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f845c1fd770 0x7f845c1fc440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f845c0d3aa0 tx=0x7f844c00d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8443fff640 1 -- 172.21.15.80:0/4018548466 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c018070 con 0x7f845c1fd770 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8443fff640 1 -- 172.21.15.80:0/4018548466 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f844c003600 con 0x7f845c1fd770 2024-04-25T13:41:16.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f845c06c720 con 0x7f845c1fd770 2024-04-25T13:41:16.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8443fff640 1 -- 172.21.15.80:0/4018548466 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f844c0039d0 con 0x7f845c1fd770 2024-04-25T13:41:16.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f845c06cca0 con 0x7f845c1fd770 2024-04-25T13:41:16.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f845c0cd730 con 0x7f845c1fd770 2024-04-25T13:41:16.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.119+0000 7f8443fff640 1 -- 172.21.15.80:0/4018548466 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f844c01b070 con 0x7f845c1fd770 2024-04-25T13:41:16.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.123+0000 7f8443fff640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f844806f400 0x7f8448071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.123+0000 7f8443fff640 1 -- 172.21.15.80:0/4018548466 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f844c08e760 con 0x7f845c1fd770 2024-04-25T13:41:16.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.123+0000 7f845b7fe640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f844806f400 0x7f8448071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.123+0000 7f845b7fe640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f844806f400 0x7f8448071910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f84540ad000 tx=0x7f84540a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.127+0000 7f8443fff640 1 -- 172.21.15.80:0/4018548466 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f844c05b7a0 con 0x7f845c1fd770 2024-04-25T13:41:16.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f845c06a3a0 con 0x7f845c1fd770 2024-04-25T13:41:16.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8443fff640 1 -- 172.21.15.80:0/4018548466 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) ==== 100+0+11 (secure 0 0 0) 0x7f844c05b0a0 con 0x7f845c1fd770 2024-04-25T13:41:16.284 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:16.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f844806f400 msgr2=0x7f8448071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f844806f400 0x7f8448071910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f84540ad000 tx=0x7f84540a7000 comp rx=0 tx=0).stop 2024-04-25T13:41:16.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f845c1fd770 msgr2=0x7f845c1fc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f845c1fd770 0x7f845c1fc440 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f845c0d3aa0 tx=0x7f844c00d810 comp rx=0 tx=0).stop 2024-04-25T13:41:16.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 shutdown_connections 2024-04-25T13:41:16.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f844806f400 0x7f8448071910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f845c1fd770 0x7f845c1fc440 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f845c1faba0 0x7f845c1fd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 --2- 172.21.15.80:0/4018548466 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84440039c0 0x7f845c0cce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 >> 172.21.15.80:0/4018548466 conn(0x7f845c0d3ad0 msgr2=0x7f845c0700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:16.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 shutdown_connections 2024-04-25T13:41:16.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.283+0000 7f8461a83640 1 -- 172.21.15.80:0/4018548466 wait complete. 2024-04-25T13:41:16.304 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:16.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.447+0000 7f658d9bf640 1 Processor -- start 2024-04-25T13:41:16.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.447+0000 7f658d9bf640 1 -- start start 2024-04-25T13:41:16.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f658d9bf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f65880d2820 0x7f65880b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f658d9bf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658815e380 0x7f65880afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f658d9bf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65880d2ce0 con 0x7f6588071ea0 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f658d9bf640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f65880b7c10 con 0x7f65880d2820 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f658d9bf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65880d2f80 con 0x7f658815e380 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6586ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6588071ea0 0x7f65880d2110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36968/0 (socket says 172.21.15.80:36968) 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6586ffd640 1 -- 172.21.15.80:0/2236110090 learned_addr learned my addr 172.21.15.80:0/2236110090 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2865452518 0 0) 0x7f65880d2ce0 con 0x7f6588071ea0 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6570003530 con 0x7f6588071ea0 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65877fe640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658815e380 0x7f65880afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65867fc640 1 --2- 172.21.15.80:0/2236110090 >> v2:172.21.15.87:3300/0 conn(0x7f65880d2820 0x7f65880b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 845093426 0 0) 0x7f6570003530 con 0x7f6588071ea0 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658815e380 msgr2=0x7f65880afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:16.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658815e380 0x7f65880afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 >> v2:172.21.15.87:3300/0 conn(0x7f65880d2820 msgr2=0x7f65880b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 --2- 172.21.15.80:0/2236110090 >> v2:172.21.15.87:3300/0 conn(0x7f65880d2820 0x7f65880b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65880768c0 con 0x7f6588071ea0 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f65780016a0 con 0x7f6588071ea0 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 >> v1:172.21.15.80:6789/0 conn(0x7f6588071ea0 legacy=0x7f65880d2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658815e380 0x7f65700039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6570003ff0 0x7f6570004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6570004d50 0x7f6570005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f65880b7c10 con 0x7f6570003ff0 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f65880d2f80 con 0x7f6570004d50 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f657000bc40 con 0x7f658815e380 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65867fc640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6570003ff0 0x7f6570004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f6586ffd640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658815e380 0x7f65700039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65877fe640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6570004d50 0x7f6570005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65867fc640 1 -- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658815e380 msgr2=0x7f65700039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:16.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65867fc640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658815e380 0x7f65700039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65867fc640 1 -- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6570004d50 msgr2=0x7f6570005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:16.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65867fc640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6570004d50 0x7f6570005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.451+0000 7f65867fc640 1 -- 172.21.15.80:0/2236110090 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6570003530 con 0x7f6570003ff0 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f65867fc640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6570003ff0 0x7f6570004810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6588070c70 tx=0x7f657c01d0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55b59116aff1ee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c00b610 con 0x7f6570003ff0 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657c0b1070 con 0x7f6570003ff0 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f6585ffb640 1 -- 172.21.15.80:0/2236110090 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0ac860 con 0x7f6570003ff0 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f658d9bf640 1 -- 172.21.15.80:0/2236110090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6570003ff0 msgr2=0x7f6570004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f658d9bf640 1 --2- 172.21.15.80:0/2236110090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6570003ff0 0x7f6570004810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6588070c70 tx=0x7f657c01d0e0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f65877fe640 1 -- 172.21.15.80:0/2236110090 reap_dead start 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f658d9bf640 1 -- 172.21.15.80:0/2236110090 shutdown_connections 2024-04-25T13:41:16.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f658d9bf640 1 -- 172.21.15.80:0/2236110090 >> 172.21.15.80:0/2236110090 conn(0x7f65880b3ed0 msgr2=0x7f65880b42e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:16.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f658d9bf640 1 -- 172.21.15.80:0/2236110090 shutdown_connections 2024-04-25T13:41:16.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.455+0000 7f658d9bf640 1 -- 172.21.15.80:0/2236110090 wait complete. 2024-04-25T13:41:16.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 Processor -- start 2024-04-25T13:41:16.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 -- start start 2024-04-25T13:41:16.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6570003ff0 0x7f65881f9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658814a200 0x7f65881660a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65881665e0 0x7f6588152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f65880b03d0 con 0x7f658814a200 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f658815e790 con 0x7f65881665e0 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65880b0550 con 0x7f6570003ff0 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658814a200 0x7f65881660a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658814a200 0x7f65881660a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53184/0 (socket says 172.21.15.80:53184) 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 -- 172.21.15.80:0/1516888406 learned_addr learned my addr 172.21.15.80:0/1516888406 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f6586ffd640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6570003ff0 0x7f65881f9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 -- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6570003ff0 msgr2=0x7f65881f9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65877fe640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65881665e0 0x7f6588152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6570003ff0 0x7f65881f9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 -- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65881665e0 msgr2=0x7f6588152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65881665e0 0x7f6588152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 -- 172.21.15.80:0/1516888406 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6588070d70 con 0x7f658814a200 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f65867fc640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658814a200 0x7f65881660a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f657c097f80 tx=0x7f6570005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f656ffff640 1 -- 172.21.15.80:0/1516888406 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0bb070 con 0x7f658814a200 2024-04-25T13:41:16.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f656ffff640 1 -- 172.21.15.80:0/1516888406 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f657c0ac760 con 0x7f658814a200 2024-04-25T13:41:16.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65880af960 con 0x7f658814a200 2024-04-25T13:41:16.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f656ffff640 1 -- 172.21.15.80:0/1516888406 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657c0acb30 con 0x7f658814a200 2024-04-25T13:41:16.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658814eb40 con 0x7f658814a200 2024-04-25T13:41:16.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.459+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658806a3a0 con 0x7f658814a200 2024-04-25T13:41:16.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.463+0000 7f656ffff640 1 -- 172.21.15.80:0/1516888406 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f657c0accd0 con 0x7f658814a200 2024-04-25T13:41:16.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.463+0000 7f656ffff640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657406f400 0x7f6574071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.463+0000 7f656ffff640 1 -- 172.21.15.80:0/1516888406 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f657c129ee0 con 0x7f658814a200 2024-04-25T13:41:16.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.463+0000 7f6586ffd640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657406f400 0x7f6574071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.463+0000 7f6586ffd640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657406f400 0x7f6574071910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6588070a30 tx=0x7f6578001de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.467+0000 7f656ffff640 1 -- 172.21.15.80:0/1516888406 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f657c0a7070 con 0x7f658814a200 2024-04-25T13:41:16.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.627+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f65880b4660 con 0x7f658814a200 2024-04-25T13:41:16.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.627+0000 7f656ffff640 1 -- 172.21.15.80:0/1516888406 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f657c016390 con 0x7f658814a200 2024-04-25T13:41:16.630 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657406f400 msgr2=0x7f6574071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657406f400 0x7f6574071910 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6588070a30 tx=0x7f6578001de0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658814a200 msgr2=0x7f65881660a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658814a200 0x7f65881660a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f657c097f80 tx=0x7f6570005840 comp rx=0 tx=0).stop 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 shutdown_connections 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657406f400 0x7f6574071910 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65881665e0 0x7f6588152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658814a200 0x7f65881660a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 --2- 172.21.15.80:0/1516888406 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6570003ff0 0x7f65881f9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 >> 172.21.15.80:0/1516888406 conn(0x7f65880b3ed0 msgr2=0x7f65880b6960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 shutdown_connections 2024-04-25T13:41:16.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.631+0000 7f658d9bf640 1 -- 172.21.15.80:0/1516888406 wait complete. 2024-04-25T13:41:16.650 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:16.650 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:16.657 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:16.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb2388640 1 Processor -- start 2024-04-25T13:41:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb2388640 1 -- start start 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb2388640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac161f90 0x7f9bac0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb2388640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9bac15ec40 0x7f9bac15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb2388640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bac0d0d70 con 0x7f9bac0d0510 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb2388640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9bac0d1050 con 0x7f9bac15ec40 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb2388640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac06dca0 con 0x7f9bac161f90 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bab7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9bac0d0510 0x7f9bac15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36984/0 (socket says 172.21.15.80:36984) 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bab7fe640 1 -- 172.21.15.80:0/2769077940 learned_addr learned my addr 172.21.15.80:0/2769077940 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb0925640 1 --2- 172.21.15.80:0/2769077940 >> v2:172.21.15.80:3300/0 conn(0x7f9bac15ec40 0x7f9bac15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9babfff640 1 --2- 172.21.15.80:0/2769077940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac161f90 0x7f9bac0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb0925640 1 -- 172.21.15.80:0/2769077940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac161f90 msgr2=0x7f9bac0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb0925640 1 --2- 172.21.15.80:0/2769077940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac161f90 0x7f9bac0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb0925640 1 -- 172.21.15.80:0/2769077940 >> v1:172.21.15.80:6789/0 conn(0x7f9bac0d0510 legacy=0x7f9bac15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.787+0000 7f9bb0925640 1 -- 172.21.15.80:0/2769077940 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac06df80 con 0x7f9bac15ec40 2024-04-25T13:41:16.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb0925640 1 --2- 172.21.15.80:0/2769077940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac15f050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009d90 tx=0x7f9ba401c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221d30e74c76fe10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9baaffd640 1 -- 172.21.15.80:0/2769077940 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba40ac070 con 0x7f9bac15ec40 2024-04-25T13:41:16.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9baaffd640 1 -- 172.21.15.80:0/2769077940 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba40a6650 con 0x7f9bac15ec40 2024-04-25T13:41:16.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9baaffd640 1 -- 172.21.15.80:0/2769077940 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba40a6a20 con 0x7f9bac15ec40 2024-04-25T13:41:16.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb2388640 1 -- 172.21.15.80:0/2769077940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 msgr2=0x7f9bac15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb2388640 1 --2- 172.21.15.80:0/2769077940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac15f050 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4009d90 tx=0x7f9ba401c710 comp rx=0 tx=0).stop 2024-04-25T13:41:16.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb2388640 1 -- 172.21.15.80:0/2769077940 shutdown_connections 2024-04-25T13:41:16.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb2388640 1 --2- 172.21.15.80:0/2769077940 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac15f050 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb2388640 1 --2- 172.21.15.80:0/2769077940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac161f90 0x7f9bac0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb2388640 1 -- 172.21.15.80:0/2769077940 >> 172.21.15.80:0/2769077940 conn(0x7f9bac06d080 msgr2=0x7f9bac0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:16.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb2388640 1 -- 172.21.15.80:0/2769077940 shutdown_connections 2024-04-25T13:41:16.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.791+0000 7f9bb2388640 1 -- 172.21.15.80:0/2769077940 wait complete. 2024-04-25T13:41:16.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 Processor -- start 2024-04-25T13:41:16.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 -- start start 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac0d0510 0x7f9bac1fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac146970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bac161f90 0x7f9bac146eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac06f630 con 0x7f9bac15ec40 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac06f4b0 con 0x7f9bac161f90 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac0d0920 con 0x7f9bac0d0510 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac146970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53204/0 (socket says 172.21.15.80:53204) 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 -- 172.21.15.80:0/1509206541 learned_addr learned my addr 172.21.15.80:0/1509206541 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:16.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 -- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac0d0510 msgr2=0x7f9bac1fa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9babfff640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac0d0510 0x7f9bac1fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb0925640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bac161f90 0x7f9bac146eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac0d0510 0x7f9bac1fa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 -- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bac161f90 msgr2=0x7f9bac146eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bac161f90 0x7f9bac146eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 -- 172.21.15.80:0/1509206541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac145b00 con 0x7f9bac15ec40 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bab7fe640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac146970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000c530 tx=0x7f9ba000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9ba8ff9640 1 -- 172.21.15.80:0/1509206541 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0018070 con 0x7f9bac15ec40 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9ba8ff9640 1 -- 172.21.15.80:0/1509206541 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ba00074e0 con 0x7f9bac15ec40 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac15b340 con 0x7f9bac15ec40 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9ba8ff9640 1 -- 172.21.15.80:0/1509206541 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0004a10 con 0x7f9bac15ec40 2024-04-25T13:41:16.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac1fac10 con 0x7f9bac15ec40 2024-04-25T13:41:16.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.795+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac06a3a0 con 0x7f9bac15ec40 2024-04-25T13:41:16.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.799+0000 7f9ba8ff9640 1 -- 172.21.15.80:0/1509206541 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9ba001b070 con 0x7f9bac15ec40 2024-04-25T13:41:16.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.799+0000 7f9ba8ff9640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b9806f400 0x7f9b98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:16.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.799+0000 7f9ba8ff9640 1 -- 172.21.15.80:0/1509206541 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f9ba008ebd0 con 0x7f9bac15ec40 2024-04-25T13:41:16.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.799+0000 7f9babfff640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b9806f400 0x7f9b98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:16.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.799+0000 7f9babfff640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b9806f400 0x7f9b98071910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9bac06fa20 tx=0x7f9b9c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:16.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.803+0000 7f9ba8ff9640 1 -- 172.21.15.80:0/1509206541 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ba0022020 con 0x7f9bac15ec40 2024-04-25T13:41:16.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.951+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bac168ab0 con 0x7f9bac15ec40 2024-04-25T13:41:16.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9ba8ff9640 1 -- 172.21.15.80:0/1509206541 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16781 (secure 0 0 0) 0x7f9ba005bb20 con 0x7f9bac15ec40 2024-04-25T13:41:16.956 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:16.957 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":21,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:16.416390+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b9806f400 msgr2=0x7f9b98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b9806f400 0x7f9b98071910 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9bac06fa20 tx=0x7f9b9c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 msgr2=0x7f9bac146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac146970 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f9ba000c530 tx=0x7f9ba000c9f0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 shutdown_connections 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b9806f400 0x7f9b98071910 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bac161f90 0x7f9bac146eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bac15ec40 0x7f9bac146970 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 --2- 172.21.15.80:0/1509206541 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bac0d0510 0x7f9bac1fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 >> 172.21.15.80:0/1509206541 conn(0x7f9bac06d080 msgr2=0x7f9bac0b2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 shutdown_connections 2024-04-25T13:41:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:16.955+0000 7f9bb2388640 1 -- 172.21.15.80:0/1509206541 wait complete. 2024-04-25T13:41:16.977 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:16.977 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:17.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a03ce8640 1 Processor -- start 2024-04-25T13:41:17.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a03ce8640 1 -- start start 2024-04-25T13:41:17.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a03ce8640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f99fc0b5330 0x7f99fc0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a03ce8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99fc0b5740 con 0x7f99fc161b60 2024-04-25T13:41:17.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a03ce8640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99fc06dd10 con 0x7f99fc0d0ff0 2024-04-25T13:41:17.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a03ce8640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f99fc1629b0 con 0x7f99fc0b5330 2024-04-25T13:41:17.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a02285640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f99fc161b60 0x7f99fc161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36986/0 (socket says 172.21.15.80:36986) 2024-04-25T13:41:17.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a02285640 1 -- 172.21.15.80:0/3320045467 learned_addr learned my addr 172.21.15.80:0/3320045467 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a01a84640 1 --2- 172.21.15.80:0/3320045467 >> v2:172.21.15.87:3300/0 conn(0x7f99fc0b5330 0x7f99fc0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1491404210 0 0) 0x7f99fc0b5740 con 0x7f99fc161b60 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f99e4003530 con 0x7f99fc161b60 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 206428816 0 0) 0x7f99e4003530 con 0x7f99fc161b60 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 >> v2:172.21.15.87:3300/0 conn(0x7f99fc0b5330 msgr2=0x7f99fc0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 --2- 172.21.15.80:0/3320045467 >> v2:172.21.15.87:3300/0 conn(0x7f99fc0b5330 0x7f99fc0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 >> v1:172.21.15.87:6789/0 conn(0x7f99fc0d0ff0 legacy=0x7f99fc06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc06dff0 con 0x7f99fc161b60 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f99ec001680 con 0x7f99fc161b60 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 >> v1:172.21.15.80:6789/0 conn(0x7f99fc161b60 legacy=0x7f99fc161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99e40039c0 0x7f99e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99e4004810 0x7f99e4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99e400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc1629b0 con 0x7f99e4005260 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f99e400c1f0 con 0x7f99e4004810 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f99e400c3f0 con 0x7f99e40039c0 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a02285640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99e400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a01a84640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99e40039c0 0x7f99e4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a01283640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99e4004810 0x7f99e4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a02285640 1 -- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99e40039c0 msgr2=0x7f99e4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a02285640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99e40039c0 0x7f99e4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a02285640 1 -- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99e4004810 msgr2=0x7f99e4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a02285640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99e4004810 0x7f99e4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.127+0000 7f9a02285640 1 -- 172.21.15.80:0/3320045467 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e4003530 con 0x7f99e4005260 2024-04-25T13:41:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a01283640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99e4004810 0x7f99e4004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:17.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a02285640 1 -- 172.21.15.80:0/3320045467 reap_dead start 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a02285640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99e400b9d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c240 tx=0x7f99ec01f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8edf1ac7776905e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec007d20 con 0x7f99e4005260 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99ec005ae0 con 0x7f99e4005260 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a00a82640 1 -- 172.21.15.80:0/3320045467 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ec007070 con 0x7f99e4005260 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a03ce8640 1 -- 172.21.15.80:0/3320045467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 msgr2=0x7f99e400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a03ce8640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99e400b9d0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00c240 tx=0x7f99ec01f3f0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a03ce8640 1 -- 172.21.15.80:0/3320045467 shutdown_connections 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a03ce8640 1 --2- 172.21.15.80:0/3320045467 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99e400b9d0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a03ce8640 1 -- 172.21.15.80:0/3320045467 >> 172.21.15.80:0/3320045467 conn(0x7f99fc0b4160 msgr2=0x7f99fc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a03ce8640 1 -- 172.21.15.80:0/3320045467 shutdown_connections 2024-04-25T13:41:17.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.131+0000 7f9a03ce8640 1 -- 172.21.15.80:0/3320045467 wait complete. 2024-04-25T13:41:17.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 Processor -- start 2024-04-25T13:41:17.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 -- start start 2024-04-25T13:41:17.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99fc1f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99fc072130 0x7f99fc0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99fc0afdf0 0x7f99fc073200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc06e890 con 0x7f99e4005260 2024-04-25T13:41:17.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f99fc06e5b0 con 0x7f99fc0afdf0 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f99fc0b0cd0 con 0x7f99fc072130 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99fc1f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99fc1f62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53218/0 (socket says 172.21.15.80:53218) 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 -- 172.21.15.80:0/1733050606 learned_addr learned my addr 172.21.15.80:0/1733050606 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a02285640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99fc0afdf0 0x7f99fc073200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01283640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99fc072130 0x7f99fc0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 -- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99fc072130 msgr2=0x7f99fc0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99fc072130 0x7f99fc0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 -- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99fc0afdf0 msgr2=0x7f99fc073200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99fc0afdf0 0x7f99fc073200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 -- 172.21.15.80:0/1733050606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99fc0b0200 con 0x7f99e4005260 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a01a84640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99fc1f62f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009f40 tx=0x7f99f40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f99fa7fc640 1 -- 172.21.15.80:0/1733050606 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f401a020 con 0x7f99e4005260 2024-04-25T13:41:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f99fa7fc640 1 -- 172.21.15.80:0/1733050606 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f4016470 con 0x7f99e4005260 2024-04-25T13:41:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99fc139cd0 con 0x7f99e4005260 2024-04-25T13:41:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f99fa7fc640 1 -- 172.21.15.80:0/1733050606 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4016880 con 0x7f99e4005260 2024-04-25T13:41:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.135+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99fc15b600 con 0x7f99e4005260 2024-04-25T13:41:17.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.139+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99fc06a3a0 con 0x7f99e4005260 2024-04-25T13:41:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.139+0000 7f99fa7fc640 1 -- 172.21.15.80:0/1733050606 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f99f401d020 con 0x7f99e4005260 2024-04-25T13:41:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.139+0000 7f99fa7fc640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99e806f400 0x7f99e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.139+0000 7f99fa7fc640 1 -- 172.21.15.80:0/1733050606 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5825+0+0 (secure 0 0 0) 0x7f99f408fda0 con 0x7f99e4005260 2024-04-25T13:41:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.139+0000 7f9a01283640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99e806f400 0x7f99e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.139+0000 7f9a01283640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99e806f400 0x7f99e8071910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99fc15c090 tx=0x7f99f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.143+0000 7f99fa7fc640 1 -- 172.21.15.80:0/1733050606 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f99f4024070 con 0x7f99e4005260 2024-04-25T13:41:17.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.299+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f99fc06fd00 con 0x7f99e4005260 2024-04-25T13:41:17.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.299+0000 7f99fa7fc640 1 -- 172.21.15.80:0/1733050606 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f99fc0b0200 con 0x7f99e4005260 2024-04-25T13:41:17.300 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:17.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.299+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99e806f400 msgr2=0x7f99e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.299+0000 7f9a03ce8640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99e806f400 0x7f99e8071910 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f99fc15c090 tx=0x7f99f0009040 comp rx=0 tx=0).stop 2024-04-25T13:41:17.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.299+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 msgr2=0x7f99fc1f62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.299+0000 7f9a03ce8640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99fc1f62f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f99f4009f40 tx=0x7f99f40115a0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.303+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 shutdown_connections 2024-04-25T13:41:17.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.303+0000 7f9a03ce8640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99e806f400 0x7f99e8071910 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.303+0000 7f9a03ce8640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99fc0afdf0 0x7f99fc073200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.303+0000 7f9a03ce8640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99fc072130 0x7f99fc0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.303+0000 7f9a03ce8640 1 --2- 172.21.15.80:0/1733050606 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99e4005260 0x7f99fc1f62f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.303+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 >> 172.21.15.80:0/1733050606 conn(0x7f99fc0b4160 msgr2=0x7f99fc15e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:17.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.303+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 shutdown_connections 2024-04-25T13:41:17.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.303+0000 7f9a03ce8640 1 -- 172.21.15.80:0/1733050606 wait complete. 2024-04-25T13:41:17.323 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:17.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f64418d3640 1 Processor -- start 2024-04-25T13:41:17.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f64418d3640 1 -- start start 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f64418d3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f643c0d3ed0 0x7f643c0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f64418d3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f643c0ba770 0x7f643c0b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f64418d3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f643c0b92a0 con 0x7f643c0d2ad0 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f64418d3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f643c0d4c10 con 0x7f643c0d3ed0 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f64418d3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f643c0d4ef0 con 0x7f643c0ba770 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f643c0d3ed0 0x7f643c0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643a7fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f643c0d2ad0 0x7f643c0b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36990/0 (socket says 172.21.15.80:36990) 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643a7fc640 1 -- 172.21.15.80:0/3128048024 learned_addr learned my addr 172.21.15.80:0/3128048024 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f643c0d3ed0 0x7f643c0d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53224/0 (socket says 172.21.15.80:53224) 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643b7fe640 1 --2- 172.21.15.80:0/3128048024 >> v2:172.21.15.87:3300/0 conn(0x7f643c0ba770 0x7f643c0b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f6439ffb640 1 -- 172.21.15.80:0/3128048024 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 178373207 0 0) 0x7f6430001120 con 0x7f643c0d2ad0 2024-04-25T13:41:17.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f6439ffb640 1 -- 172.21.15.80:0/3128048024 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6424003530 con 0x7f643c0d2ad0 2024-04-25T13:41:17.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643affd640 1 -- 172.21.15.80:0/3128048024 >> v2:172.21.15.87:3300/0 conn(0x7f643c0ba770 msgr2=0x7f643c0b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:17.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643affd640 1 --2- 172.21.15.80:0/3128048024 >> v2:172.21.15.87:3300/0 conn(0x7f643c0ba770 0x7f643c0b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643affd640 1 -- 172.21.15.80:0/3128048024 >> v1:172.21.15.80:6789/0 conn(0x7f643c0d2ad0 legacy=0x7f643c0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.463+0000 7f643affd640 1 -- 172.21.15.80:0/3128048024 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c0d51d0 con 0x7f643c0d3ed0 2024-04-25T13:41:17.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f643affd640 1 --2- 172.21.15.80:0/3128048024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0d3ed0 0x7f643c0d2590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f643c0d1d20 tx=0x7f642c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6c09d13a411cee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f6439ffb640 1 -- 172.21.15.80:0/3128048024 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0b0070 con 0x7f643c0d3ed0 2024-04-25T13:41:17.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f6439ffb640 1 -- 172.21.15.80:0/3128048024 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c004230 con 0x7f643c0d3ed0 2024-04-25T13:41:17.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f6439ffb640 1 -- 172.21.15.80:0/3128048024 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c003620 con 0x7f643c0d3ed0 2024-04-25T13:41:17.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f64418d3640 1 -- 172.21.15.80:0/3128048024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0d3ed0 msgr2=0x7f643c0d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f64418d3640 1 --2- 172.21.15.80:0/3128048024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0d3ed0 0x7f643c0d2590 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f643c0d1d20 tx=0x7f642c01c330 comp rx=0 tx=0).stop 2024-04-25T13:41:17.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f64418d3640 1 -- 172.21.15.80:0/3128048024 shutdown_connections 2024-04-25T13:41:17.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f64418d3640 1 --2- 172.21.15.80:0/3128048024 >> v2:172.21.15.87:3300/0 conn(0x7f643c0ba770 0x7f643c0b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f64418d3640 1 --2- 172.21.15.80:0/3128048024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0d3ed0 0x7f643c0d2590 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f64418d3640 1 -- 172.21.15.80:0/3128048024 >> 172.21.15.80:0/3128048024 conn(0x7f643c0737b0 msgr2=0x7f643c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:17.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f64418d3640 1 -- 172.21.15.80:0/3128048024 shutdown_connections 2024-04-25T13:41:17.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.467+0000 7f64418d3640 1 -- 172.21.15.80:0/3128048024 wait complete. 2024-04-25T13:41:17.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 Processor -- start 2024-04-25T13:41:17.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 -- start start 2024-04-25T13:41:17.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0ba770 0x7f643c156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643c0d2ad0 0x7f643c0b78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643c0d3ed0 0x7f643c1fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f643c0b8320 con 0x7f643c0ba770 2024-04-25T13:41:17.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f643c0b84a0 con 0x7f643c0d2ad0 2024-04-25T13:41:17.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f643c160620 con 0x7f643c0d3ed0 2024-04-25T13:41:17.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0ba770 0x7f643c156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0ba770 0x7f643c156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53236/0 (socket says 172.21.15.80:53236) 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 -- 172.21.15.80:0/3823237046 learned_addr learned my addr 172.21.15.80:0/3823237046 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 -- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643c0d3ed0 msgr2=0x7f643c1fcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643b7fe640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643c0d3ed0 0x7f643c1fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643c0d3ed0 0x7f643c1fcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 -- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643c0d2ad0 msgr2=0x7f643c0b78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643a7fc640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643c0d2ad0 0x7f643c0b78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643c0d2ad0 0x7f643c0b78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 -- 172.21.15.80:0/3823237046 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f643c0b7e20 con 0x7f643c0ba770 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f643affd640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0ba770 0x7f643c156fd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ae040 tx=0x7f642c0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f6423fff640 1 -- 172.21.15.80:0/3823237046 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0b9070 con 0x7f643c0ba770 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f6423fff640 1 -- 172.21.15.80:0/3823237046 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f642c003db0 con 0x7f643c0ba770 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f643c15dc50 con 0x7f643c0ba770 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f6423fff640 1 -- 172.21.15.80:0/3823237046 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642c0b0040 con 0x7f643c0ba770 2024-04-25T13:41:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f643c0ced20 con 0x7f643c0ba770 2024-04-25T13:41:17.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.471+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f643c06a3a0 con 0x7f643c0ba770 2024-04-25T13:41:17.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.475+0000 7f6423fff640 1 -- 172.21.15.80:0/3823237046 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f642c0bc070 con 0x7f643c0ba770 2024-04-25T13:41:17.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.475+0000 7f6423fff640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f642806f430 0x7f6428071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.475+0000 7f6423fff640 1 -- 172.21.15.80:0/3823237046 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f642c1290c0 con 0x7f643c0ba770 2024-04-25T13:41:17.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.475+0000 7f643a7fc640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f642806f430 0x7f6428071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.475+0000 7f643a7fc640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f642806f430 0x7f6428071940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64300012c0 tx=0x7f6430009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.479+0000 7f6423fff640 1 -- 172.21.15.80:0/3823237046 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f642c016070 con 0x7f643c0ba770 2024-04-25T13:41:17.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.639+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f643c0cc740 con 0x7f643c0ba770 2024-04-25T13:41:17.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f6423fff640 1 -- 172.21.15.80:0/3823237046 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f642c0f5fc0 con 0x7f643c0ba770 2024-04-25T13:41:17.643 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f642806f430 msgr2=0x7f6428071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f642806f430 0x7f6428071940 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f64300012c0 tx=0x7f6430009040 comp rx=0 tx=0).stop 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0ba770 msgr2=0x7f643c156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0ba770 0x7f643c156fd0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f642c0ae040 tx=0x7f642c0190d0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 shutdown_connections 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f642806f430 0x7f6428071940 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f643c0d3ed0 0x7f643c1fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f643c0d2ad0 0x7f643c0b78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 --2- 172.21.15.80:0/3823237046 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f643c0ba770 0x7f643c156fd0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 >> 172.21.15.80:0/3823237046 conn(0x7f643c0737b0 msgr2=0x7f643c0cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 shutdown_connections 2024-04-25T13:41:17.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.643+0000 7f64418d3640 1 -- 172.21.15.80:0/3823237046 wait complete. 2024-04-25T13:41:17.663 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:17.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e708d2640 1 Processor -- start 2024-04-25T13:41:17.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e708d2640 1 -- start start 2024-04-25T13:41:17.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e708d2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c0b8af0 0x7f1e6c0b8f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e708d2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e6c0d0090 0x7f1e6c0d04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e708d2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e6c0ce570 con 0x7f1e6c161da0 2024-04-25T13:41:17.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e708d2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1e6c0d2250 con 0x7f1e6c0d0090 2024-04-25T13:41:17.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e708d2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c0d09e0 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e6a59c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c0b8af0 0x7f1e6c0b8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e69d9b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1e6c0d0090 0x7f1e6c0d04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e6ad9d640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1e6c161da0 0x7f1e6c1621b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:37980/0 (socket says 172.21.15.80:37980) 2024-04-25T13:41:17.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.799+0000 7f1e6ad9d640 1 -- 172.21.15.80:0/2298332133 learned_addr learned my addr 172.21.15.80:0/2298332133 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:17.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e69d9b640 1 -- 172.21.15.80:0/2298332133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c0b8af0 msgr2=0x7f1e6c0b8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:17.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e69d9b640 1 --2- 172.21.15.80:0/2298332133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c0b8af0 0x7f1e6c0b8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e69d9b640 1 -- 172.21.15.80:0/2298332133 >> v1:172.21.15.87:6789/0 conn(0x7f1e6c161da0 legacy=0x7f1e6c1621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e69d9b640 1 -- 172.21.15.80:0/2298332133 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c0d4580 con 0x7f1e6c0d0090 2024-04-25T13:41:17.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e6a59c640 1 --2- 172.21.15.80:0/2298332133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c0b8af0 0x7f1e6c0b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:41:17.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e69d9b640 1 --2- 172.21.15.80:0/2298332133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 0x7f1e6c0d04a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1e600097c0 tx=0x7f1e6001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c76d5138962f0fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e6959a640 1 -- 172.21.15.80:0/2298332133 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e600b0070 con 0x7f1e6c0d0090 2024-04-25T13:41:17.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e6959a640 1 -- 172.21.15.80:0/2298332133 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e600ab600 con 0x7f1e6c0d0090 2024-04-25T13:41:17.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e6959a640 1 -- 172.21.15.80:0/2298332133 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e600ab9d0 con 0x7f1e6c0d0090 2024-04-25T13:41:17.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e708d2640 1 -- 172.21.15.80:0/2298332133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 msgr2=0x7f1e6c0d04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e708d2640 1 --2- 172.21.15.80:0/2298332133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 0x7f1e6c0d04a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1e600097c0 tx=0x7f1e6001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e708d2640 1 -- 172.21.15.80:0/2298332133 shutdown_connections 2024-04-25T13:41:17.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e708d2640 1 --2- 172.21.15.80:0/2298332133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 0x7f1e6c0d04a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e708d2640 1 --2- 172.21.15.80:0/2298332133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c0b8af0 0x7f1e6c0b8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e708d2640 1 -- 172.21.15.80:0/2298332133 >> 172.21.15.80:0/2298332133 conn(0x7f1e6c0d3a60 msgr2=0x7f1e6c0d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:17.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e708d2640 1 -- 172.21.15.80:0/2298332133 shutdown_connections 2024-04-25T13:41:17.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.803+0000 7f1e708d2640 1 -- 172.21.15.80:0/2298332133 wait complete. 2024-04-25T13:41:17.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 Processor -- start 2024-04-25T13:41:17.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 -- start start 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e6c0b8af0 0x7f1e6c1592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 0x7f1e6c0caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c161da0 0x7f1e6c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0b6a30 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c0b68b0 con 0x7f1e6c0d0090 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e6c16b8f0 con 0x7f1e6c161da0 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e6c0b8af0 0x7f1e6c1592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e6c0b8af0 0x7f1e6c1592d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53244/0 (socket says 172.21.15.80:53244) 2024-04-25T13:41:17.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 -- 172.21.15.80:0/618879982 learned_addr learned my addr 172.21.15.80:0/618879982 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6ad9d640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c161da0 0x7f1e6c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e69d9b640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 0x7f1e6c0caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 -- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c161da0 msgr2=0x7f1e6c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c161da0 0x7f1e6c0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 -- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 msgr2=0x7f1e6c0caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 0x7f1e6c0caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 -- 172.21.15.80:0/618879982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c159e10 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e6a59c640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e6c0b8af0 0x7f1e6c1592d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00e930 tx=0x7f1e5c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e5affd640 1 -- 172.21.15.80:0/618879982 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c017070 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e5affd640 1 -- 172.21.15.80:0/618879982 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e5c00ca00 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c15a0f0 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e5affd640 1 -- 172.21.15.80:0/618879982 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00cdb0 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.807+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c15a6b0 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.811+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c06a3a0 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.811+0000 7f1e5affd640 1 -- 172.21.15.80:0/618879982 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1e5c01a070 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.811+0000 7f1e5affd640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e4c06f400 0x7f1e4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:17.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.811+0000 7f1e5affd640 1 -- 172.21.15.80:0/618879982 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f1e5c08f210 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.811+0000 7f1e69d9b640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e4c06f400 0x7f1e4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:17.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.811+0000 7f1e69d9b640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e4c06f400 0x7f1e4c071910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e600ad000 tx=0x7f1e60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:17.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.815+0000 7f1e5affd640 1 -- 172.21.15.80:0/618879982 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e5c05c080 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1e6c0b9590 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e5affd640 1 -- 172.21.15.80:0/618879982 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7f1e5c05b980 con 0x7f1e6c0b8af0 2024-04-25T13:41:17.964 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e4c06f400 msgr2=0x7f1e4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e4c06f400 0x7f1e4c071910 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1e600ad000 tx=0x7f1e60008040 comp rx=0 tx=0).stop 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e6c0b8af0 msgr2=0x7f1e6c1592d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e6c0b8af0 0x7f1e6c1592d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00e930 tx=0x7f1e5c00edf0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 shutdown_connections 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e4c06f400 0x7f1e4c071910 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e6c161da0 0x7f1e6c0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e6c0d0090 0x7f1e6c0caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 --2- 172.21.15.80:0/618879982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e6c0b8af0 0x7f1e6c1592d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 >> 172.21.15.80:0/618879982 conn(0x7f1e6c0d3a60 msgr2=0x7f1e6c0b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 shutdown_connections 2024-04-25T13:41:17.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:17.963+0000 7f1e708d2640 1 -- 172.21.15.80:0/618879982 wait complete. 2024-04-25T13:41:17.984 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e842a640 1 Processor -- start 2024-04-25T13:41:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e842a640 1 -- start start 2024-04-25T13:41:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e842a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1e00d06e0 0x7fe1e006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e842a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1e006e9e0 0x7fe1e006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e842a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1e00d0af0 con 0x7fe1e0161f90 2024-04-25T13:41:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e842a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e00d0d70 con 0x7fe1e006e9e0 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e842a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe1e00d1050 con 0x7fe1e00d06e0 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e69c7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1e006e9e0 0x7fe1e006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e61c6640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe1e0161f90 0x7fe1e00cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36996/0 (socket says 172.21.15.80:36996) 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e61c6640 1 -- 172.21.15.80:0/2786695543 learned_addr learned my addr 172.21.15.80:0/2786695543 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e59c5640 1 --2- 172.21.15.80:0/2786695543 >> v2:172.21.15.87:3300/0 conn(0x7fe1e00d06e0 0x7fe1e006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e69c7640 1 -- 172.21.15.80:0/2786695543 >> v2:172.21.15.87:3300/0 conn(0x7fe1e00d06e0 msgr2=0x7fe1e006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e69c7640 1 --2- 172.21.15.80:0/2786695543 >> v2:172.21.15.87:3300/0 conn(0x7fe1e00d06e0 0x7fe1e006e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e69c7640 1 -- 172.21.15.80:0/2786695543 >> v1:172.21.15.80:6789/0 conn(0x7fe1e0161f90 legacy=0x7fe1e00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.135+0000 7fe1e69c7640 1 -- 172.21.15.80:0/2786695543 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e006dca0 con 0x7fe1e006e9e0 2024-04-25T13:41:18.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e69c7640 1 --2- 172.21.15.80:0/2786695543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e006e9e0 0x7fe1e006edf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0074290 tx=0x7fe1d001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a30dbc8e2977df7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e51c4640 1 -- 172.21.15.80:0/2786695543 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00b2070 con 0x7fe1e006e9e0 2024-04-25T13:41:18.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e51c4640 1 -- 172.21.15.80:0/2786695543 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d00a57b0 con 0x7fe1e006e9e0 2024-04-25T13:41:18.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e51c4640 1 -- 172.21.15.80:0/2786695543 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00a5b80 con 0x7fe1e006e9e0 2024-04-25T13:41:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e842a640 1 -- 172.21.15.80:0/2786695543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e006e9e0 msgr2=0x7fe1e006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e842a640 1 --2- 172.21.15.80:0/2786695543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e006e9e0 0x7fe1e006edf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0074290 tx=0x7fe1d001c7e0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e842a640 1 -- 172.21.15.80:0/2786695543 shutdown_connections 2024-04-25T13:41:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e842a640 1 --2- 172.21.15.80:0/2786695543 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e006e9e0 0x7fe1e006edf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e842a640 1 --2- 172.21.15.80:0/2786695543 >> v2:172.21.15.87:3300/0 conn(0x7fe1e00d06e0 0x7fe1e006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e842a640 1 -- 172.21.15.80:0/2786695543 >> 172.21.15.80:0/2786695543 conn(0x7fe1e006d080 msgr2=0x7fe1e00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e842a640 1 -- 172.21.15.80:0/2786695543 shutdown_connections 2024-04-25T13:41:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.139+0000 7fe1e842a640 1 -- 172.21.15.80:0/2786695543 wait complete. 2024-04-25T13:41:18.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 Processor -- start 2024-04-25T13:41:18.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 -- start start 2024-04-25T13:41:18.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e006e9e0 0x7fe1e01320d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1e00d06e0 0x7fe1e0167080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e0161f90 0x7fe1e01675c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e01447a0 con 0x7fe1e0161f90 2024-04-25T13:41:18.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe1e0144620 con 0x7fe1e006e9e0 2024-04-25T13:41:18.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1e016b510 con 0x7fe1e00d06e0 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e0161f90 0x7fe1e01675c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e0161f90 0x7fe1e01675c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53254/0 (socket says 172.21.15.80:53254) 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 -- 172.21.15.80:0/1905634785 learned_addr learned my addr 172.21.15.80:0/1905634785 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e61c6640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e006e9e0 0x7fe1e01320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e59c5640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1e00d06e0 0x7fe1e0167080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 -- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1e00d06e0 msgr2=0x7fe1e0167080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1e00d06e0 0x7fe1e0167080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 -- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e006e9e0 msgr2=0x7fe1e01320d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e006e9e0 0x7fe1e01320d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 -- 172.21.15.80:0/1905634785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1e0146730 con 0x7fe1e0161f90 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e61c6640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e006e9e0 0x7fe1e01320d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e69c7640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e0161f90 0x7fe1e01675c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00193f0 tx=0x7fe1d0019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1deffd640 1 -- 172.21.15.80:0/1905634785 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00bd020 con 0x7fe1e0161f90 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1deffd640 1 -- 172.21.15.80:0/1905634785 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1d00b8670 con 0x7fe1e0161f90 2024-04-25T13:41:18.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1e0146a10 con 0x7fe1e0161f90 2024-04-25T13:41:18.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1deffd640 1 -- 172.21.15.80:0/1905634785 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00b2040 con 0x7fe1e0161f90 2024-04-25T13:41:18.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1e0146fd0 con 0x7fe1e0161f90 2024-04-25T13:41:18.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.143+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1e006a3a0 con 0x7fe1e0161f90 2024-04-25T13:41:18.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.147+0000 7fe1deffd640 1 -- 172.21.15.80:0/1905634785 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe1d00c0020 con 0x7fe1e0161f90 2024-04-25T13:41:18.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.147+0000 7fe1deffd640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1cc06f430 0x7fe1cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.147+0000 7fe1deffd640 1 -- 172.21.15.80:0/1905634785 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fe1d012a2c0 con 0x7fe1e0161f90 2024-04-25T13:41:18.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.147+0000 7fe1e61c6640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1cc06f430 0x7fe1cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.147+0000 7fe1e61c6640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1cc06f430 0x7fe1cc071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007fd0 tx=0x7fe1d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.151+0000 7fe1deffd640 1 -- 172.21.15.80:0/1905634785 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe1d00c7030 con 0x7fe1e0161f90 2024-04-25T13:41:18.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe1e00b4f80 con 0x7fe1e0161f90 2024-04-25T13:41:18.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1deffd640 1 -- 172.21.15.80:0/1905634785 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) ==== 80+0+0 (secure 0 0 0) 0x7fe1d0016520 con 0x7fe1e0161f90 2024-04-25T13:41:18.328 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:18.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1cc06f430 msgr2=0x7fe1cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1cc06f430 0x7fe1cc071940 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8007fd0 tx=0x7fe1d8009040 comp rx=0 tx=0).stop 2024-04-25T13:41:18.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e0161f90 msgr2=0x7fe1e01675c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e0161f90 0x7fe1e01675c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00193f0 tx=0x7fe1d0019580 comp rx=0 tx=0).stop 2024-04-25T13:41:18.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 shutdown_connections 2024-04-25T13:41:18.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1cc06f430 0x7fe1cc071940 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1e0161f90 0x7fe1e01675c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1e00d06e0 0x7fe1e0167080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 --2- 172.21.15.80:0/1905634785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1e006e9e0 0x7fe1e01320d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 >> 172.21.15.80:0/1905634785 conn(0x7fe1e006d080 msgr2=0x7fe1e00757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:18.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.327+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 shutdown_connections 2024-04-25T13:41:18.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.331+0000 7fe1e842a640 1 -- 172.21.15.80:0/1905634785 wait complete. 2024-04-25T13:41:18.350 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:41:18.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:18.351 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T13:41:18.365 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:41:18.365 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:41:18.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:18.365 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T13:41:18.416 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:41:18.416 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:18.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544f42a640 1 Processor -- start 2024-04-25T13:41:18.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544f42a640 1 -- start start 2024-04-25T13:41:18.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544f42a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f54480b7780 0x7f54480b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544f42a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54480b80d0 con 0x7f54480b4eb0 2024-04-25T13:41:18.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544f42a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54480d48a0 con 0x7f54480b93f0 2024-04-25T13:41:18.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544f42a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f54480d4b80 con 0x7f54480b7780 2024-04-25T13:41:18.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544d1c6640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f54480b4eb0 0x7f54481603c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:37008/0 (socket says 172.21.15.80:37008) 2024-04-25T13:41:18.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544d1c6640 1 -- 172.21.15.80:0/4073888991 learned_addr learned my addr 172.21.15.80:0/4073888991 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:18.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2232200934 0 0) 0x7f5444001120 con 0x7f54480b4eb0 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544c9c5640 1 --2- 172.21.15.80:0/4073888991 >> v2:172.21.15.87:3300/0 conn(0x7f54480b7780 0x7f54480b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5434003530 con 0x7f54480b4eb0 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4077561610 0 0) 0x7f5434003530 con 0x7f54480b4eb0 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 >> v2:172.21.15.87:3300/0 conn(0x7f54480b7780 msgr2=0x7f54480b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 --2- 172.21.15.80:0/4073888991 >> v2:172.21.15.87:3300/0 conn(0x7f54480b7780 0x7f54480b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 >> v1:172.21.15.87:6789/0 conn(0x7f54480b93f0 legacy=0x7f54480b9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54480d4e60 con 0x7f54480b4eb0 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f5444001850 con 0x7f54480b4eb0 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 >> v1:172.21.15.80:6789/0 conn(0x7f54480b4eb0 legacy=0x7f54481603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54340039c0 0x7f5434003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5434004810 0x7f5434004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54340051d0 0x7f543400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f54480d4b80 con 0x7f5434004810 2024-04-25T13:41:18.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f543400c170 con 0x7f54340039c0 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f543400c370 con 0x7f54340051d0 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544c9c5640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5434004810 0x7f5434004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544d1c6640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54340039c0 0x7f5434003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544c9c5640 1 -- 172.21.15.80:0/4073888991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54340051d0 msgr2=0x7f543400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544d9c7640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54340051d0 0x7f543400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544c9c5640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f54340051d0 0x7f543400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544c9c5640 1 -- 172.21.15.80:0/4073888991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54340039c0 msgr2=0x7f5434003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544c9c5640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54340039c0 0x7f5434003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544c9c5640 1 -- 172.21.15.80:0/4073888991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5434003530 con 0x7f5434004810 2024-04-25T13:41:18.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.571+0000 7f544d9c7640 1 -- 172.21.15.80:0/4073888991 reap_dead start 2024-04-25T13:41:18.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f544c9c5640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5434004810 0x7f5434004c20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5438009ae0 tx=0x7f543801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b79bfaa7e476daf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380b2070 con 0x7f5434004810 2024-04-25T13:41:18.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54380a5660 con 0x7f5434004810 2024-04-25T13:41:18.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f543ffff640 1 -- 172.21.15.80:0/4073888991 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54380096b0 con 0x7f5434004810 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f544f42a640 1 -- 172.21.15.80:0/4073888991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5434004810 msgr2=0x7f5434004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f544f42a640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5434004810 0x7f5434004c20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5438009ae0 tx=0x7f543801ca70 comp rx=0 tx=0).stop 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f544f42a640 1 -- 172.21.15.80:0/4073888991 shutdown_connections 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f544f42a640 1 --2- 172.21.15.80:0/4073888991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5434004810 0x7f5434004c20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f544f42a640 1 -- 172.21.15.80:0/4073888991 >> 172.21.15.80:0/4073888991 conn(0x7f544806c9f0 msgr2=0x7f544806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f544f42a640 1 -- 172.21.15.80:0/4073888991 shutdown_connections 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.575+0000 7f544f42a640 1 -- 172.21.15.80:0/4073888991 wait complete. 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 Processor -- start 2024-04-25T13:41:18.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 -- start start 2024-04-25T13:41:18.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5434004810 0x7f544814f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480ce810 0x7f54480cec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5448162b50 0x7f5448162f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f54480d5940 con 0x7f5448162b50 2024-04-25T13:41:18.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f54480d5420 con 0x7f54480ce810 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5448150fe0 con 0x7f5434004810 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5448162b50 0x7f5448162f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5448162b50 0x7f5448162f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53274/0 (socket says 172.21.15.80:53274) 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 -- 172.21.15.80:0/734689732 learned_addr learned my addr 172.21.15.80:0/734689732 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d1c6640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5434004810 0x7f544814f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544c9c5640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480ce810 0x7f54480cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 -- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5434004810 msgr2=0x7f544814f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5434004810 0x7f544814f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 -- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480ce810 msgr2=0x7f54480cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480ce810 0x7f54480cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 -- 172.21.15.80:0/734689732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54481634a0 con 0x7f5448162b50 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d1c6640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5434004810 0x7f544814f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544c9c5640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480ce810 0x7f54480cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544d9c7640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5448162b50 0x7f5448162f60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5448160b10 tx=0x7f544000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f543dffb640 1 -- 172.21.15.80:0/734689732 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440003b80 con 0x7f5448162b50 2024-04-25T13:41:18.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f543dffb640 1 -- 172.21.15.80:0/734689732 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5440017070 con 0x7f5448162b50 2024-04-25T13:41:18.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54480cf570 con 0x7f5448162b50 2024-04-25T13:41:18.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f543dffb640 1 -- 172.21.15.80:0/734689732 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5440005070 con 0x7f5448162b50 2024-04-25T13:41:18.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.579+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54480cfb30 con 0x7f5448162b50 2024-04-25T13:41:18.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.583+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f544806a3a0 con 0x7f5448162b50 2024-04-25T13:41:18.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.583+0000 7f543dffb640 1 -- 172.21.15.80:0/734689732 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5440005210 con 0x7f5448162b50 2024-04-25T13:41:18.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.583+0000 7f543dffb640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f542c06f430 0x7f542c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.583+0000 7f543dffb640 1 -- 172.21.15.80:0/734689732 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f544008f0d0 con 0x7f5448162b50 2024-04-25T13:41:18.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.583+0000 7f544d1c6640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f542c06f430 0x7f542c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.583+0000 7f544d1c6640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f542c06f430 0x7f542c071940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f544806b930 tx=0x7f5444005630 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.587+0000 7f543dffb640 1 -- 172.21.15.80:0/734689732 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f54481634a0 con 0x7f5448162b50 2024-04-25T13:41:18.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.783+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54480b9a50 con 0x7f5448162b50 2024-04-25T13:41:18.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.783+0000 7f543dffb640 1 -- 172.21.15.80:0/734689732 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f544005bf40 con 0x7f5448162b50 2024-04-25T13:41:18.786 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:18.787 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:40:29.558137+0000","modified":"2024-04-25T13:40:32.595776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-25T13:41:18.787 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f542c06f430 msgr2=0x7f542c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f542c06f430 0x7f542c071940 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f544806b930 tx=0x7f5444005630 comp rx=0 tx=0).stop 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5448162b50 msgr2=0x7f5448162f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5448162b50 0x7f5448162f60 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f5448160b10 tx=0x7f544000dc00 comp rx=0 tx=0).stop 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 shutdown_connections 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f542c06f430 0x7f542c071940 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5448162b50 0x7f5448162f60 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f54480ce810 0x7f54480cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 --2- 172.21.15.80:0/734689732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5434004810 0x7f544814f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 >> 172.21.15.80:0/734689732 conn(0x7f544806c9f0 msgr2=0x7f54480bb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 shutdown_connections 2024-04-25T13:41:18.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.787+0000 7f544f42a640 1 -- 172.21.15.80:0/734689732 wait complete. 2024-04-25T13:41:18.808 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:18.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc4101f1640 1 Processor -- start 2024-04-25T13:41:18.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc4101f1640 1 -- start start 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc4101f1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 0x7fc4080cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc4101f1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc40815ec40 0x7fc40815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc4101f1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4080d0d70 con 0x7fc4080d0510 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc4101f1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc4080d1050 con 0x7fc40815ec40 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc4101f1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc40806dca0 con 0x7fc408161f90 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40d78c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc4080d0510 0x7fc40815e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:37014/0 (socket says 172.21.15.80:37014) 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40d78c640 1 -- 172.21.15.80:0/2947359058 learned_addr learned my addr 172.21.15.80:0/2947359058 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40df8d640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 0x7fc4080cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40e78e640 1 --2- 172.21.15.80:0/2947359058 >> v2:172.21.15.87:3300/0 conn(0x7fc40815ec40 0x7fc40815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1072539577 0 0) 0x7fc4080d0d70 con 0x7fc4080d0510 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc3f4003530 con 0x7fc4080d0510 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3394795268 0 0) 0x7fc3f4003530 con 0x7fc4080d0510 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 msgr2=0x7fc4080cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:18.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 0x7fc4080cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 >> v2:172.21.15.87:3300/0 conn(0x7fc40815ec40 msgr2=0x7fc40815f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 --2- 172.21.15.80:0/2947359058 >> v2:172.21.15.87:3300/0 conn(0x7fc40815ec40 0x7fc40815f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40806df80 con 0x7fc4080d0510 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc3f8001680 con 0x7fc4080d0510 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40df8d640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 0x7fc4080cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 >> v1:172.21.15.80:6789/0 conn(0x7fc4080d0510 legacy=0x7fc40815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 0x7fc3f40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc3f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f4004d50 0x7fc3f4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc4080d1050 con 0x7fc3f4003f20 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc40806dca0 con 0x7fc3f4004d50 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3f400bb60 con 0x7fc408161f90 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.943+0000 7fc40d78c640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc3f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40df8d640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 0x7fc3f40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40d78c640 1 -- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 msgr2=0x7fc3f40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:18.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40e78e640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f4004d50 0x7fc3f4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40d78c640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408161f90 0x7fc3f40039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40d78c640 1 -- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f4004d50 msgr2=0x7fc3f4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:18.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40d78c640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3f4004d50 0x7fc3f4005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40d78c640 1 -- 172.21.15.80:0/2947359058 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f4003530 con 0x7fc3f4003f20 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40d78c640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc3f4004810 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc408074290 tx=0x7fc3f801cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04727687dbef1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8007d60 con 0x7fc3f4003f20 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3f80a9020 con 0x7fc3f4003f20 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40cf8b640 1 -- 172.21.15.80:0/2947359058 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8005ae0 con 0x7fc3f4003f20 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc4101f1640 1 -- 172.21.15.80:0/2947359058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 msgr2=0x7fc3f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc4101f1640 1 --2- 172.21.15.80:0/2947359058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc3f4004810 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc408074290 tx=0x7fc3f801cba0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc40e78e640 1 -- 172.21.15.80:0/2947359058 reap_dead start 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc4101f1640 1 -- 172.21.15.80:0/2947359058 shutdown_connections 2024-04-25T13:41:18.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc4101f1640 1 -- 172.21.15.80:0/2947359058 >> 172.21.15.80:0/2947359058 conn(0x7fc40806d080 msgr2=0x7fc4080b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc4101f1640 1 -- 172.21.15.80:0/2947359058 shutdown_connections 2024-04-25T13:41:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.947+0000 7fc4101f1640 1 -- 172.21.15.80:0/2947359058 wait complete. 2024-04-25T13:41:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 Processor -- start 2024-04-25T13:41:18.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 -- start start 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc408146470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408141580 0x7fc4080b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4080b2d10 0x7fc408165d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc40806f4b0 con 0x7fc3f4003f20 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc40806f630 con 0x7fc4080b2d10 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc4080d0920 con 0x7fc408141580 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc408146470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc408146470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53280/0 (socket says 172.21.15.80:53280) 2024-04-25T13:41:18.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 -- 172.21.15.80:0/3483472607 learned_addr learned my addr 172.21.15.80:0/3483472607 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40e78e640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4080b2d10 0x7fc408165d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40d78c640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408141580 0x7fc4080b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 -- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408141580 msgr2=0x7fc4080b27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408141580 0x7fc4080b27d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 -- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4080b2d10 msgr2=0x7fc408165d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4080b2d10 0x7fc408165d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 -- 172.21.15.80:0/3483472607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4080b3120 con 0x7fc3f4003f20 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40d78c640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408141580 0x7fc4080b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc40df8d640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc408146470 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc400010890 tx=0x7fc400010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc3fe7fc640 1 -- 172.21.15.80:0/3483472607 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40001a070 con 0x7fc3f4003f20 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc3fe7fc640 1 -- 172.21.15.80:0/3483472607 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc40000c8c0 con 0x7fc3f4003f20 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc408146dc0 con 0x7fc3f4003f20 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc3fe7fc640 1 -- 172.21.15.80:0/3483472607 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40000f040 con 0x7fc3f4003f20 2024-04-25T13:41:18.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.951+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc408147380 con 0x7fc3f4003f20 2024-04-25T13:41:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.955+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc40806a3a0 con 0x7fc3f4003f20 2024-04-25T13:41:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.955+0000 7fc3fe7fc640 1 -- 172.21.15.80:0/3483472607 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc40001d070 con 0x7fc3f4003f20 2024-04-25T13:41:18.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.955+0000 7fc3fe7fc640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc3ec06f430 0x7fc3ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:18.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.955+0000 7fc3fe7fc640 1 -- 172.21.15.80:0/3483472607 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fc40008fb10 con 0x7fc3f4003f20 2024-04-25T13:41:18.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.955+0000 7fc40d78c640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc3ec06f430 0x7fc3ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:18.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.955+0000 7fc40d78c640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc3ec06f430 0x7fc3ec071940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80aa000 tx=0x7fc4080d0f40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:18.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:18.959+0000 7fc3fe7fc640 1 -- 172.21.15.80:0/3483472607 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4080b3120 con 0x7fc3f4003f20 2024-04-25T13:41:19.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.107+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc40815f720 con 0x7fc3f4003f20 2024-04-25T13:41:19.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc3fe7fc640 1 -- 172.21.15.80:0/3483472607 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16781 (secure 0 0 0) 0x7fc40008f020 con 0x7fc3f4003f20 2024-04-25T13:41:19.112 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:19.112 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":22,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:17.425119+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc3ec06f430 msgr2=0x7fc3ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc3ec06f430 0x7fc3ec071940 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc3f80aa000 tx=0x7fc4080d0f40 comp rx=0 tx=0).stop 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 msgr2=0x7fc408146470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc408146470 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc400010890 tx=0x7fc400010d50 comp rx=0 tx=0).stop 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 shutdown_connections 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc3ec06f430 0x7fc3ec071940 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4080b2d10 0x7fc408165d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc408141580 0x7fc4080b27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 --2- 172.21.15.80:0/3483472607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3f4003f20 0x7fc408146470 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 >> 172.21.15.80:0/3483472607 conn(0x7fc40806d080 msgr2=0x7fc408160730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:19.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 shutdown_connections 2024-04-25T13:41:19.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.111+0000 7fc4101f1640 1 -- 172.21.15.80:0/3483472607 wait complete. 2024-04-25T13:41:19.131 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:40:27.648187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:40:28.506496+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:19.132 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:19.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7888a640 1 Processor -- start 2024-04-25T13:41:19.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7888a640 1 -- start start 2024-04-25T13:41:19.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7888a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3f741473a0 0x7f3f740d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7888a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 0x7f3f74068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7888a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f74068ee0 con 0x7f3f74068ad0 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7888a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3f74071640 con 0x7f3f741473a0 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7888a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7406f2f0 con 0x7f3f7406fab0 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f72d9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3f74068ad0 0x7f3f74144d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:37022/0 (socket says 172.21.15.80:37022) 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3f741473a0 0x7f3f740d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f72d9d640 1 -- 172.21.15.80:0/2592618715 learned_addr learned my addr 172.21.15.80:0/2592618715 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f71d9b640 1 --2- 172.21.15.80:0/2592618715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 0x7f3f74068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7259c640 1 -- 172.21.15.80:0/2592618715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 msgr2=0x7f3f74068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7259c640 1 --2- 172.21.15.80:0/2592618715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 0x7f3f74068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7259c640 1 -- 172.21.15.80:0/2592618715 >> v1:172.21.15.80:6789/0 conn(0x7f3f74068ad0 legacy=0x7f3f74144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.267+0000 7f3f7259c640 1 -- 172.21.15.80:0/2592618715 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f74071080 con 0x7f3f741473a0 2024-04-25T13:41:19.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7259c640 1 --2- 172.21.15.80:0/2592618715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f741473a0 0x7f3f740d24c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0097c0 tx=0x7f3f6c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af80e9b232dc64aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7159a640 1 -- 172.21.15.80:0/2592618715 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b0070 con 0x7f3f741473a0 2024-04-25T13:41:19.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7159a640 1 -- 172.21.15.80:0/2592618715 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0ab600 con 0x7f3f741473a0 2024-04-25T13:41:19.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7159a640 1 -- 172.21.15.80:0/2592618715 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0ab9d0 con 0x7f3f741473a0 2024-04-25T13:41:19.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7888a640 1 -- 172.21.15.80:0/2592618715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f741473a0 msgr2=0x7f3f740d24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7888a640 1 --2- 172.21.15.80:0/2592618715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f741473a0 0x7f3f740d24c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0097c0 tx=0x7f3f6c01c410 comp rx=0 tx=0).stop 2024-04-25T13:41:19.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7888a640 1 -- 172.21.15.80:0/2592618715 shutdown_connections 2024-04-25T13:41:19.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7888a640 1 --2- 172.21.15.80:0/2592618715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 0x7f3f74068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7888a640 1 --2- 172.21.15.80:0/2592618715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f741473a0 0x7f3f740d24c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7888a640 1 -- 172.21.15.80:0/2592618715 >> 172.21.15.80:0/2592618715 conn(0x7f3f74074000 msgr2=0x7f3f7406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:19.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7888a640 1 -- 172.21.15.80:0/2592618715 shutdown_connections 2024-04-25T13:41:19.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.271+0000 7f3f7888a640 1 -- 172.21.15.80:0/2592618715 wait complete. 2024-04-25T13:41:19.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 Processor -- start 2024-04-25T13:41:19.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 -- start start 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f74068ad0 0x7f3f7414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 0x7f3f7415c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f741473a0 0x7f3f7415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3f74070040 con 0x7f3f74068ad0 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f7406fec0 con 0x7f3f741473a0 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f7416b850 con 0x7f3f7406fab0 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f74068ad0 0x7f3f7414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f74068ad0 0x7f3f7414de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53298/0 (socket says 172.21.15.80:53298) 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 -- 172.21.15.80:0/1976532802 learned_addr learned my addr 172.21.15.80:0/1976532802 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 -- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 msgr2=0x7f3f7415c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 0x7f3f7415c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 -- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f741473a0 msgr2=0x7f3f7415ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f741473a0 0x7f3f7415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 -- 172.21.15.80:0/1976532802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f74167d50 con 0x7f3f74068ad0 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7259c640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f74068ad0 0x7f3f7414de30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0ae040 tx=0x7f3f6c0191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f6affd640 1 -- 172.21.15.80:0/1976532802 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b0040 con 0x7f3f74068ad0 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f6affd640 1 -- 172.21.15.80:0/1976532802 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0ab400 con 0x7f3f74068ad0 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f74168030 con 0x7f3f74068ad0 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f6affd640 1 -- 172.21.15.80:0/1976532802 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c0b4630 con 0x7f3f74068ad0 2024-04-25T13:41:19.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f741685f0 con 0x7f3f74068ad0 2024-04-25T13:41:19.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.275+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f7406a3a0 con 0x7f3f74068ad0 2024-04-25T13:41:19.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.279+0000 7f3f6affd640 1 -- 172.21.15.80:0/1976532802 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3f6c0bc020 con 0x7f3f74068ad0 2024-04-25T13:41:19.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.279+0000 7f3f6affd640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f5406f430 0x7f3f54071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.279+0000 7f3f6affd640 1 -- 172.21.15.80:0/1976532802 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f3f6c128490 con 0x7f3f74068ad0 2024-04-25T13:41:19.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.279+0000 7f3f71d9b640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f5406f430 0x7f3f54071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.279+0000 7f3f71d9b640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f5406f430 0x7f3f54071940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f7415d970 tx=0x7f3f5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.283+0000 7f3f6affd640 1 -- 172.21.15.80:0/1976532802 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3f6c0c3060 con 0x7f3f74068ad0 2024-04-25T13:41:19.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.427+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3f74161a10 con 0x7f3f74068ad0 2024-04-25T13:41:19.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f6affd640 1 -- 172.21.15.80:0/1976532802 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) ==== 93+0+10 (secure 0 0 0) 0x7f3f6c016090 con 0x7f3f74068ad0 2024-04-25T13:41:19.432 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f5406f430 msgr2=0x7f3f54071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f5406f430 0x7f3f54071940 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3f7415d970 tx=0x7f3f5c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f74068ad0 msgr2=0x7f3f7414de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f74068ad0 0x7f3f7414de30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c0ae040 tx=0x7f3f6c0191b0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 shutdown_connections 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f5406f430 0x7f3f54071940 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f741473a0 0x7f3f7415ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f7406fab0 0x7f3f7415c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 --2- 172.21.15.80:0/1976532802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f74068ad0 0x7f3f7414de30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 >> 172.21.15.80:0/1976532802 conn(0x7f3f74074000 msgr2=0x7f3f7406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:19.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 shutdown_connections 2024-04-25T13:41:19.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.431+0000 7f3f7888a640 1 -- 172.21.15.80:0/1976532802 wait complete. 2024-04-25T13:41:19.452 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5ce69640 1 Processor -- start 2024-04-25T13:41:19.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5ce69640 1 -- start start 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5ce69640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c580d26b0 0x7f7c580d2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5ce69640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d09c0 0x7f7c580d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5ce69640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c580d18b0 con 0x7f7c580b9260 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5ce69640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7c580d3000 con 0x7f7c580d26b0 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5ce69640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c580d56c0 con 0x7f7c580d09c0 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5659c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c580d26b0 0x7f7c580d2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5659c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c580d26b0 0x7f7c580d2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53304/0 (socket says 172.21.15.80:53304) 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c56d9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7c580b9260 0x7f7c580b9670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:37026/0 (socket says 172.21.15.80:37026) 2024-04-25T13:41:19.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5659c640 1 -- 172.21.15.80:0/3504922332 learned_addr learned my addr 172.21.15.80:0/3504922332 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:19.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c4ffff640 1 --2- 172.21.15.80:0/3504922332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d09c0 0x7f7c580d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5659c640 1 -- 172.21.15.80:0/3504922332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d09c0 msgr2=0x7f7c580d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:19.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5659c640 1 --2- 172.21.15.80:0/3504922332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d09c0 0x7f7c580d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5659c640 1 -- 172.21.15.80:0/3504922332 >> v1:172.21.15.80:6789/0 conn(0x7f7c580b9260 legacy=0x7f7c580b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c5659c640 1 -- 172.21.15.80:0/3504922332 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c580d1310 con 0x7f7c580d26b0 2024-04-25T13:41:19.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.603+0000 7f7c4ffff640 1 --2- 172.21.15.80:0/3504922332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d09c0 0x7f7c580d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:19.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5659c640 1 --2- 172.21.15.80:0/3504922332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d26b0 0x7f7c580d2ac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009870 tx=0x7f7c4801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbdda590b9dfd04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c55d9b640 1 -- 172.21.15.80:0/3504922332 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480ab650 con 0x7f7c580d26b0 2024-04-25T13:41:19.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c55d9b640 1 -- 172.21.15.80:0/3504922332 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c480b0070 con 0x7f7c580d26b0 2024-04-25T13:41:19.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c55d9b640 1 -- 172.21.15.80:0/3504922332 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480aba10 con 0x7f7c580d26b0 2024-04-25T13:41:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 -- 172.21.15.80:0/3504922332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d26b0 msgr2=0x7f7c580d2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/3504922332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d26b0 0x7f7c580d2ac0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7c48009870 tx=0x7f7c4801c560 comp rx=0 tx=0).stop 2024-04-25T13:41:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 -- 172.21.15.80:0/3504922332 shutdown_connections 2024-04-25T13:41:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/3504922332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d09c0 0x7f7c580d0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/3504922332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d26b0 0x7f7c580d2ac0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 -- 172.21.15.80:0/3504922332 >> 172.21.15.80:0/3504922332 conn(0x7f7c580b77b0 msgr2=0x7f7c580b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 -- 172.21.15.80:0/3504922332 shutdown_connections 2024-04-25T13:41:19.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 -- 172.21.15.80:0/3504922332 wait complete. 2024-04-25T13:41:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 Processor -- start 2024-04-25T13:41:19.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.607+0000 7f7c5ce69640 1 -- start start 2024-04-25T13:41:19.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c580b9260 0x7f7c58144b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d09c0 0x7f7c5815dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d26b0 0x7f7c581553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7c581604b0 con 0x7f7c580d09c0 2024-04-25T13:41:19.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c580d4d80 con 0x7f7c580b9260 2024-04-25T13:41:19.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c58160630 con 0x7f7c580d26b0 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d09c0 0x7f7c5815dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d09c0 0x7f7c5815dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53314/0 (socket says 172.21.15.80:53314) 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 -- 172.21.15.80:0/1214522443 learned_addr learned my addr 172.21.15.80:0/1214522443 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 -- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d26b0 msgr2=0x7f7c581553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c56d9d640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d26b0 0x7f7c581553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5659c640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c580b9260 0x7f7c58144b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d26b0 0x7f7c581553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 -- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c580b9260 msgr2=0x7f7c58144b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c580b9260 0x7f7c58144b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 -- 172.21.15.80:0/1214522443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c5815c2b0 con 0x7f7c580d09c0 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4ffff640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d09c0 0x7f7c5815dfb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c820 tx=0x7f7c4000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4effd640 1 -- 172.21.15.80:0/1214522443 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40015070 con 0x7f7c580d09c0 2024-04-25T13:41:19.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4effd640 1 -- 172.21.15.80:0/1214522443 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c40007a30 con 0x7f7c580d09c0 2024-04-25T13:41:19.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c5815c590 con 0x7f7c580d09c0 2024-04-25T13:41:19.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4effd640 1 -- 172.21.15.80:0/1214522443 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c40007e40 con 0x7f7c580d09c0 2024-04-25T13:41:19.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c5815cb50 con 0x7f7c580d09c0 2024-04-25T13:41:19.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c5806a3a0 con 0x7f7c580d09c0 2024-04-25T13:41:19.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.611+0000 7f7c4effd640 1 -- 172.21.15.80:0/1214522443 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7c40018070 con 0x7f7c580d09c0 2024-04-25T13:41:19.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.615+0000 7f7c4effd640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c3806f400 0x7f7c38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.615+0000 7f7c4effd640 1 -- 172.21.15.80:0/1214522443 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f7c4008f5d0 con 0x7f7c580d09c0 2024-04-25T13:41:19.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.615+0000 7f7c5659c640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c3806f400 0x7f7c38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.615+0000 7f7c5659c640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c3806f400 0x7f7c38071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c480af000 tx=0x7f7c48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.619+0000 7f7c4effd640 1 -- 172.21.15.80:0/1214522443 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c4005c520 con 0x7f7c580d09c0 2024-04-25T13:41:19.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f7c580c90b0 con 0x7f7c580d09c0 2024-04-25T13:41:19.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c4effd640 1 -- 172.21.15.80:0/1214522443 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) ==== 104+0+11 (secure 0 0 0) 0x7f7c4005be20 con 0x7f7c580d09c0 2024-04-25T13:41:19.768 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:19.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c3806f400 msgr2=0x7f7c38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c3806f400 0x7f7c38071910 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7c480af000 tx=0x7f7c48008040 comp rx=0 tx=0).stop 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d09c0 msgr2=0x7f7c5815dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d09c0 0x7f7c5815dfb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f7c4000c820 tx=0x7f7c4000cce0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 shutdown_connections 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c3806f400 0x7f7c38071910 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c580d26b0 0x7f7c581553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c580d09c0 0x7f7c5815dfb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 --2- 172.21.15.80:0/1214522443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c580b9260 0x7f7c58144b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 >> 172.21.15.80:0/1214522443 conn(0x7f7c580b77b0 msgr2=0x7f7c580d3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 shutdown_connections 2024-04-25T13:41:19.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.767+0000 7f7c5ce69640 1 -- 172.21.15.80:0/1214522443 wait complete. 2024-04-25T13:41:19.788 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47cd4a640 1 Processor -- start 2024-04-25T13:41:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47cd4a640 1 -- start start 2024-04-25T13:41:19.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47cd4a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa478160290 0x7fa4780b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47cd4a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa478068ca0 0x7fa47806f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47cd4a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4780b6910 con 0x7fa4780b1880 2024-04-25T13:41:19.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47cd4a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa4781606a0 con 0x7fa478160290 2024-04-25T13:41:19.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47cd4a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780b0eb0 con 0x7fa478068ca0 2024-04-25T13:41:19.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47659c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa478160290 0x7fa4780b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47659c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa478160290 0x7fa4780b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53316/0 (socket says 172.21.15.80:53316) 2024-04-25T13:41:19.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47659c640 1 -- 172.21.15.80:0/3383998630 learned_addr learned my addr 172.21.15.80:0/3383998630 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:19.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa476d9d640 1 --2- 172.21.15.80:0/3383998630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa478068ca0 0x7fa47806f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47659c640 1 -- 172.21.15.80:0/3383998630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa478068ca0 msgr2=0x7fa47806f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:19.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47659c640 1 --2- 172.21.15.80:0/3383998630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa478068ca0 0x7fa47806f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47659c640 1 -- 172.21.15.80:0/3383998630 >> v1:172.21.15.87:6789/0 conn(0x7fa4780b1880 legacy=0x7fa478068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:19.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.923+0000 7fa47659c640 1 -- 172.21.15.80:0/3383998630 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4780b0860 con 0x7fa478160290 2024-04-25T13:41:19.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47659c640 1 --2- 172.21.15.80:0/3383998630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa478160290 0x7fa4780b1340 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa468009870 tx=0x7fa46801c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d95fd92ce57db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa475d9b640 1 -- 172.21.15.80:0/3383998630 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4680a9610 con 0x7fa478160290 2024-04-25T13:41:19.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa475d9b640 1 -- 172.21.15.80:0/3383998630 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4680a97b0 con 0x7fa478160290 2024-04-25T13:41:19.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa475d9b640 1 -- 172.21.15.80:0/3383998630 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4680a9b80 con 0x7fa478160290 2024-04-25T13:41:19.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47cd4a640 1 -- 172.21.15.80:0/3383998630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa478160290 msgr2=0x7fa4780b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/3383998630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa478160290 0x7fa4780b1340 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fa468009870 tx=0x7fa46801c530 comp rx=0 tx=0).stop 2024-04-25T13:41:19.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47cd4a640 1 -- 172.21.15.80:0/3383998630 shutdown_connections 2024-04-25T13:41:19.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/3383998630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa478068ca0 0x7fa47806f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/3383998630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa478160290 0x7fa4780b1340 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47cd4a640 1 -- 172.21.15.80:0/3383998630 >> 172.21.15.80:0/3383998630 conn(0x7fa4780736a0 msgr2=0x7fa478073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:19.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47cd4a640 1 -- 172.21.15.80:0/3383998630 shutdown_connections 2024-04-25T13:41:19.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.927+0000 7fa47cd4a640 1 -- 172.21.15.80:0/3383998630 wait complete. 2024-04-25T13:41:19.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 Processor -- start 2024-04-25T13:41:19.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 -- start start 2024-04-25T13:41:19.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4780b1880 0x7fa47814abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa478160290 0x7fa47815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47815aa20 0x7fa47814b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780b6600 con 0x7fa47815aa20 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa4780b6480 con 0x7fa478160290 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa4780b6d80 con 0x7fa4780b1880 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47815aa20 0x7fa47814b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47815aa20 0x7fa47814b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53322/0 (socket says 172.21.15.80:53322) 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 -- 172.21.15.80:0/2279418969 learned_addr learned my addr 172.21.15.80:0/2279418969 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47659c640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4780b1880 0x7fa47814abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 -- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4780b1880 msgr2=0x7fa47814abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4780b1880 0x7fa47814abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa46dd9b640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa478160290 0x7fa47815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 -- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa478160290 msgr2=0x7fa47815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa478160290 0x7fa47815c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 -- 172.21.15.80:0/2279418969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa478153390 con 0x7fa47815aa20 2024-04-25T13:41:19.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa476d9d640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47815aa20 0x7fa47814b2b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa470007d40 tx=0x7fa47000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa46f7fe640 1 -- 172.21.15.80:0/2279418969 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470014420 con 0x7fa47815aa20 2024-04-25T13:41:19.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa46f7fe640 1 -- 172.21.15.80:0/2279418969 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa470019070 con 0x7fa47815aa20 2024-04-25T13:41:19.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa478153670 con 0x7fa47815aa20 2024-04-25T13:41:19.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa46f7fe640 1 -- 172.21.15.80:0/2279418969 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa470014820 con 0x7fa47815aa20 2024-04-25T13:41:19.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.931+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa478153c30 con 0x7fa47815aa20 2024-04-25T13:41:19.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.935+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa47806a3a0 con 0x7fa47815aa20 2024-04-25T13:41:19.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.935+0000 7fa46f7fe640 1 -- 172.21.15.80:0/2279418969 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa4700149c0 con 0x7fa47815aa20 2024-04-25T13:41:19.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.935+0000 7fa46f7fe640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45c06f430 0x7fa45c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:19.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.935+0000 7fa46f7fe640 1 -- 172.21.15.80:0/2279418969 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fa47008fb70 con 0x7fa47815aa20 2024-04-25T13:41:19.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.935+0000 7fa47659c640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45c06f430 0x7fa45c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:19.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.935+0000 7fa47659c640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45c06f430 0x7fa45c071940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa4680ae000 tx=0x7fa468009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:19.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:19.939+0000 7fa46f7fe640 1 -- 172.21.15.80:0/2279418969 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa47005c940 con 0x7fa47815aa20 2024-04-25T13:41:20.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.087+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa478155d80 con 0x7fa47815aa20 2024-04-25T13:41:20.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.087+0000 7fa46f7fe640 1 -- 172.21.15.80:0/2279418969 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) ==== 100+0+11 (secure 0 0 0) 0x7fa47005c240 con 0x7fa47815aa20 2024-04-25T13:41:20.090 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:20.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45c06f430 msgr2=0x7fa45c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45c06f430 0x7fa45c071940 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa4680ae000 tx=0x7fa468009040 comp rx=0 tx=0).stop 2024-04-25T13:41:20.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47815aa20 msgr2=0x7fa47814b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47815aa20 0x7fa47814b2b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa470007d40 tx=0x7fa47000e4b0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 shutdown_connections 2024-04-25T13:41:20.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45c06f430 0x7fa45c071940 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa47815aa20 0x7fa47814b2b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa478160290 0x7fa47815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 --2- 172.21.15.80:0/2279418969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa4780b1880 0x7fa47814abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 >> 172.21.15.80:0/2279418969 conn(0x7fa4780736a0 msgr2=0x7fa47815f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:20.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 shutdown_connections 2024-04-25T13:41:20.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.091+0000 7fa47cd4a640 1 -- 172.21.15.80:0/2279418969 wait complete. 2024-04-25T13:41:20.111 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:20.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e16030640 1 Processor -- start 2024-04-25T13:41:20.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e16030640 1 -- start start 2024-04-25T13:41:20.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e16030640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9e1015f1c0 0x7f9e10068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e16030640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e100cfb60 0x7f9e100cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e16030640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e100d04b0 con 0x7f9e10070490 2024-04-25T13:41:20.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e16030640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9e10068f00 con 0x7f9e1015f1c0 2024-04-25T13:41:20.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e16030640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9e100691e0 con 0x7f9e100cfb60 2024-04-25T13:41:20.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e0f7fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9e1015f1c0 0x7f9e10068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e0effd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e100cfb60 0x7f9e100cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e0ffff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f9e10070490 0x7f9e100708a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:38002/0 (socket says 172.21.15.80:38002) 2024-04-25T13:41:20.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.251+0000 7f9e0ffff640 1 -- 172.21.15.80:0/3832523050 learned_addr learned my addr 172.21.15.80:0/3832523050 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:20.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0effd640 1 -- 172.21.15.80:0/3832523050 >> v2:172.21.15.87:3300/0 conn(0x7f9e1015f1c0 msgr2=0x7f9e10068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:20.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0effd640 1 --2- 172.21.15.80:0/3832523050 >> v2:172.21.15.87:3300/0 conn(0x7f9e1015f1c0 0x7f9e10068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0effd640 1 -- 172.21.15.80:0/3832523050 >> v1:172.21.15.87:6789/0 conn(0x7f9e10070490 legacy=0x7f9e100708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0effd640 1 -- 172.21.15.80:0/3832523050 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1015e9c0 con 0x7f9e100cfb60 2024-04-25T13:41:20.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0f7fe640 1 --2- 172.21.15.80:0/3832523050 >> v2:172.21.15.87:3300/0 conn(0x7f9e1015f1c0 0x7f9e10068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:41:20.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0effd640 1 --2- 172.21.15.80:0/3832523050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e100cfb60 0x7f9e100cff70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9e040097c0 tx=0x7f9e0401c380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a57cf09ec544327 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0e7fc640 1 -- 172.21.15.80:0/3832523050 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e040b0070 con 0x7f9e100cfb60 2024-04-25T13:41:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0e7fc640 1 -- 172.21.15.80:0/3832523050 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e040ab600 con 0x7f9e100cfb60 2024-04-25T13:41:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e0e7fc640 1 -- 172.21.15.80:0/3832523050 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e040ab9d0 con 0x7f9e100cfb60 2024-04-25T13:41:20.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e16030640 1 -- 172.21.15.80:0/3832523050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e100cfb60 msgr2=0x7f9e100cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e16030640 1 --2- 172.21.15.80:0/3832523050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e100cfb60 0x7f9e100cff70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9e040097c0 tx=0x7f9e0401c380 comp rx=0 tx=0).stop 2024-04-25T13:41:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e16030640 1 -- 172.21.15.80:0/3832523050 shutdown_connections 2024-04-25T13:41:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e16030640 1 --2- 172.21.15.80:0/3832523050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e100cfb60 0x7f9e100cff70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e16030640 1 --2- 172.21.15.80:0/3832523050 >> v2:172.21.15.87:3300/0 conn(0x7f9e1015f1c0 0x7f9e10068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e16030640 1 -- 172.21.15.80:0/3832523050 >> 172.21.15.80:0/3832523050 conn(0x7f9e100d33a0 msgr2=0x7f9e100cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e16030640 1 -- 172.21.15.80:0/3832523050 shutdown_connections 2024-04-25T13:41:20.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.255+0000 7f9e16030640 1 -- 172.21.15.80:0/3832523050 wait complete. 2024-04-25T13:41:20.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 Processor -- start 2024-04-25T13:41:20.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 -- start start 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e10070490 0x7f9e100b5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e100cfb60 0x7f9e1015adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e1015f1c0 0x7f9e10134d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10068c50 con 0x7f9e100cfb60 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9e10068ad0 con 0x7f9e1015f1c0 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9e1006d820 con 0x7f9e10070490 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e100cfb60 0x7f9e1015adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e100cfb60 0x7f9e1015adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53332/0 (socket says 172.21.15.80:53332) 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 -- 172.21.15.80:0/1462568667 learned_addr learned my addr 172.21.15.80:0/1462568667 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:20.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0ffff640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e1015f1c0 0x7f9e10134d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0f7fe640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e10070490 0x7f9e100b5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 -- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e10070490 msgr2=0x7f9e100b5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e10070490 0x7f9e100b5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 -- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e1015f1c0 msgr2=0x7f9e10134d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e1015f1c0 0x7f9e10134d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 -- 172.21.15.80:0/1462568667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e100b65c0 con 0x7f9e100cfb60 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e0effd640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e100cfb60 0x7f9e1015adc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9e04019fb0 tx=0x7f9e04019120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9df7fff640 1 -- 172.21.15.80:0/1462568667 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e040b0040 con 0x7f9e100cfb60 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9df7fff640 1 -- 172.21.15.80:0/1462568667 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e040b43f0 con 0x7f9e100cfb60 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e100b68a0 con 0x7f9e100cfb60 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9df7fff640 1 -- 172.21.15.80:0/1462568667 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e040b47c0 con 0x7f9e100cfb60 2024-04-25T13:41:20.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.259+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e100b6e60 con 0x7f9e100cfb60 2024-04-25T13:41:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.263+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e1006a3a0 con 0x7f9e100cfb60 2024-04-25T13:41:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.263+0000 7f9df7fff640 1 -- 172.21.15.80:0/1462568667 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e040b4960 con 0x7f9e100cfb60 2024-04-25T13:41:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.263+0000 7f9df7fff640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dfc06f430 0x7f9dfc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.263+0000 7f9df7fff640 1 -- 172.21.15.80:0/1462568667 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f9e041299e0 con 0x7f9e100cfb60 2024-04-25T13:41:20.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.263+0000 7f9e0f7fe640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dfc06f430 0x7f9dfc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.263+0000 7f9e0f7fe640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dfc06f430 0x7f9dfc071940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e1006c8e0 tx=0x7f9e0800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.267+0000 7f9df7fff640 1 -- 172.21.15.80:0/1462568667 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9e040c2020 con 0x7f9e100cfb60 2024-04-25T13:41:20.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.451+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e100b6350 con 0x7f9e100cfb60 2024-04-25T13:41:20.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.451+0000 7f9df7fff640 1 -- 172.21.15.80:0/1462568667 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f9e04016090 con 0x7f9e100cfb60 2024-04-25T13:41:20.455 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:20.455 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:40:29.558137+0000","modified":"2024-04-25T13:40:32.595776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-25T13:41:20.455 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-25T13:41:20.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dfc06f430 msgr2=0x7f9dfc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dfc06f430 0x7f9dfc071940 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9e1006c8e0 tx=0x7f9e0800b040 comp rx=0 tx=0).stop 2024-04-25T13:41:20.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e100cfb60 msgr2=0x7f9e1015adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e100cfb60 0x7f9e1015adc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9e04019fb0 tx=0x7f9e04019120 comp rx=0 tx=0).stop 2024-04-25T13:41:20.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 shutdown_connections 2024-04-25T13:41:20.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dfc06f430 0x7f9dfc071940 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e1015f1c0 0x7f9e10134d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e100cfb60 0x7f9e1015adc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 --2- 172.21.15.80:0/1462568667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e10070490 0x7f9e100b5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 >> 172.21.15.80:0/1462568667 conn(0x7f9e100d33a0 msgr2=0x7f9e100b2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:20.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 shutdown_connections 2024-04-25T13:41:20.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.455+0000 7f9e16030640 1 -- 172.21.15.80:0/1462568667 wait complete. 2024-04-25T13:41:20.475 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:20.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.618+0000 7fb614db4640 1 Processor -- start 2024-04-25T13:41:20.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.618+0000 7fb614db4640 1 -- start start 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb614db4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb6100d06e0 0x7fb61006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb614db4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb61006e9e0 0x7fb61006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb614db4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6100d0af0 con 0x7fb610161f90 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb614db4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb6100d0d70 con 0x7fb6100d06e0 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb614db4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb6100d1050 con 0x7fb61006e9e0 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb60dd9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb6100d06e0 0x7fb61006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb60dd9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb6100d06e0 0x7fb61006e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53334/0 (socket says 172.21.15.80:53334) 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb60dd9b640 1 -- 172.21.15.80:0/3595157039 learned_addr learned my addr 172.21.15.80:0/3595157039 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:20.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb60ed9d640 1 --2- 172.21.15.80:0/3595157039 >> v2:172.21.15.87:3300/0 conn(0x7fb61006e9e0 0x7fb61006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb60dd9b640 1 -- 172.21.15.80:0/3595157039 >> v2:172.21.15.87:3300/0 conn(0x7fb61006e9e0 msgr2=0x7fb61006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:20.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb60dd9b640 1 --2- 172.21.15.80:0/3595157039 >> v2:172.21.15.87:3300/0 conn(0x7fb61006e9e0 0x7fb61006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb60dd9b640 1 -- 172.21.15.80:0/3595157039 >> v1:172.21.15.87:6789/0 conn(0x7fb610161f90 legacy=0x7fb6100cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.622+0000 7fb60dd9b640 1 -- 172.21.15.80:0/3595157039 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61006dca0 con 0x7fb6100d06e0 2024-04-25T13:41:20.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb60dd9b640 1 --2- 172.21.15.80:0/3595157039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6100d06e0 0x7fb61006e4a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb6100b4b80 tx=0x7fb60401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d8056cf13bb2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb60d59a640 1 -- 172.21.15.80:0/3595157039 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6040b0070 con 0x7fb6100d06e0 2024-04-25T13:41:20.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb60d59a640 1 -- 172.21.15.80:0/3595157039 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6040ab600 con 0x7fb6100d06e0 2024-04-25T13:41:20.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb60d59a640 1 -- 172.21.15.80:0/3595157039 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6040ab9d0 con 0x7fb6100d06e0 2024-04-25T13:41:20.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 -- 172.21.15.80:0/3595157039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6100d06e0 msgr2=0x7fb61006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 --2- 172.21.15.80:0/3595157039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6100d06e0 0x7fb61006e4a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb6100b4b80 tx=0x7fb60401c3d0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 -- 172.21.15.80:0/3595157039 shutdown_connections 2024-04-25T13:41:20.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 --2- 172.21.15.80:0/3595157039 >> v2:172.21.15.87:3300/0 conn(0x7fb61006e9e0 0x7fb61006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 --2- 172.21.15.80:0/3595157039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6100d06e0 0x7fb61006e4a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 -- 172.21.15.80:0/3595157039 >> 172.21.15.80:0/3595157039 conn(0x7fb61006d080 msgr2=0x7fb6100b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:20.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 -- 172.21.15.80:0/3595157039 shutdown_connections 2024-04-25T13:41:20.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 -- 172.21.15.80:0/3595157039 wait complete. 2024-04-25T13:41:20.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 Processor -- start 2024-04-25T13:41:20.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.626+0000 7fb614db4640 1 -- start start 2024-04-25T13:41:20.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb61006e9e0 0x7fb6101636a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6100d06e0 0x7fb61013fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb610161f90 0x7fb6101403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb6101447a0 con 0x7fb610161f90 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb610144620 con 0x7fb6100d06e0 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb61016b510 con 0x7fb61006e9e0 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb610161f90 0x7fb6101403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb610161f90 0x7fb6101403d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53342/0 (socket says 172.21.15.80:53342) 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 -- 172.21.15.80:0/1659823075 learned_addr learned my addr 172.21.15.80:0/1659823075 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60dd9b640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6100d06e0 0x7fb61013fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60e59c640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb61006e9e0 0x7fb6101636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 -- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb61006e9e0 msgr2=0x7fb6101636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb61006e9e0 0x7fb6101636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 -- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6100d06e0 msgr2=0x7fb61013fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6100d06e0 0x7fb61013fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 -- 172.21.15.80:0/1659823075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb61015cea0 con 0x7fb610161f90 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb60ed9d640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb610161f90 0x7fb6101403d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb608007d80 tx=0x7fb60800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb5feffd640 1 -- 172.21.15.80:0/1659823075 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608014420 con 0x7fb610161f90 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb5feffd640 1 -- 172.21.15.80:0/1659823075 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb608019070 con 0x7fb610161f90 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb61015d180 con 0x7fb610161f90 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb5feffd640 1 -- 172.21.15.80:0/1659823075 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb608014800 con 0x7fb610161f90 2024-04-25T13:41:20.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb61015d740 con 0x7fb610161f90 2024-04-25T13:41:20.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.630+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb61015cc10 con 0x7fb610161f90 2024-04-25T13:41:20.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.634+0000 7fb5feffd640 1 -- 172.21.15.80:0/1659823075 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb6080149a0 con 0x7fb610161f90 2024-04-25T13:41:20.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.634+0000 7fb5feffd640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f006f400 0x7fb5f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.634+0000 7fb5feffd640 1 -- 172.21.15.80:0/1659823075 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fb60808fbd0 con 0x7fb610161f90 2024-04-25T13:41:20.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.634+0000 7fb60e59c640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f006f400 0x7fb5f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.634+0000 7fb60e59c640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f006f400 0x7fb5f0071910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb61006fa20 tx=0x7fb600009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.638+0000 7fb5feffd640 1 -- 172.21.15.80:0/1659823075 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb60805ca40 con 0x7fb610161f90 2024-04-25T13:41:20.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.798+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb61006a3a0 con 0x7fb610161f90 2024-04-25T13:41:20.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.802+0000 7fb5feffd640 1 -- 172.21.15.80:0/1659823075 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16781 (secure 0 0 0) 0x7fb60805c340 con 0x7fb610161f90 2024-04-25T13:41:20.805 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:20.805 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":22,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:17.425119+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f006f400 msgr2=0x7fb5f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f006f400 0x7fb5f0071910 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb61006fa20 tx=0x7fb600009040 comp rx=0 tx=0).stop 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb610161f90 msgr2=0x7fb6101403d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb610161f90 0x7fb6101403d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb608007d80 tx=0x7fb60800e4b0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 shutdown_connections 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f006f400 0x7fb5f0071910 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb610161f90 0x7fb6101403d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6100d06e0 0x7fb61013fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 --2- 172.21.15.80:0/1659823075 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb61006e9e0 0x7fb6101636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 >> 172.21.15.80:0/1659823075 conn(0x7fb61006d080 msgr2=0x7fb6100b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:20.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 shutdown_connections 2024-04-25T13:41:20.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.806+0000 7fb614db4640 1 -- 172.21.15.80:0/1659823075 wait complete. 2024-04-25T13:41:20.827 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:41:20.828 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:20.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8ef31640 1 Processor -- start 2024-04-25T13:41:20.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8ef31640 1 -- start start 2024-04-25T13:41:20.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8ef31640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d88161f90 0x7f2d880cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8ef31640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d880d0510 0x7f2d8815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8ef31640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2d8815ea70 0x7f2d8815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8ef31640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2d880d0d70 con 0x7f2d88161f90 2024-04-25T13:41:20.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8ef31640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2d880d1050 con 0x7f2d8815ea70 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8ef31640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d8806dca0 con 0x7f2d880d0510 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8cccd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d88161f90 0x7f2d880cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8cccd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2d88161f90 0x7f2d880cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53346/0 (socket says 172.21.15.80:53346) 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8cccd640 1 -- 172.21.15.80:0/700866863 learned_addr learned my addr 172.21.15.80:0/700866863 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8d4ce640 1 --2- 172.21.15.80:0/700866863 >> v2:172.21.15.87:3300/0 conn(0x7f2d8815ea70 0x7f2d8815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d87fff640 1 --2- 172.21.15.80:0/700866863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d880d0510 0x7f2d8815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8cccd640 1 -- 172.21.15.80:0/700866863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d880d0510 msgr2=0x7f2d8815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8cccd640 1 --2- 172.21.15.80:0/700866863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d880d0510 0x7f2d8815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8cccd640 1 -- 172.21.15.80:0/700866863 >> v2:172.21.15.87:3300/0 conn(0x7f2d8815ea70 msgr2=0x7f2d8815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8cccd640 1 --2- 172.21.15.80:0/700866863 >> v2:172.21.15.87:3300/0 conn(0x7f2d8815ea70 0x7f2d8815ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d8cccd640 1 -- 172.21.15.80:0/700866863 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8806df80 con 0x7f2d88161f90 2024-04-25T13:41:20.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.962+0000 7f2d87fff640 1 --2- 172.21.15.80:0/700866863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d880d0510 0x7f2d8815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8cccd640 1 --2- 172.21.15.80:0/700866863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d88161f90 0x7f2d880cffd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2d88071090 tx=0x7f2d8001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9eca08b34ac7a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d877fe640 1 -- 172.21.15.80:0/700866863 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d800b1070 con 0x7f2d88161f90 2024-04-25T13:41:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d877fe640 1 -- 172.21.15.80:0/700866863 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d800a8050 con 0x7f2d88161f90 2024-04-25T13:41:20.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d877fe640 1 -- 172.21.15.80:0/700866863 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d800ac860 con 0x7f2d88161f90 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 -- 172.21.15.80:0/700866863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d88161f90 msgr2=0x7f2d880cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/700866863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d88161f90 0x7f2d880cffd0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2d88071090 tx=0x7f2d8001c460 comp rx=0 tx=0).stop 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 -- 172.21.15.80:0/700866863 shutdown_connections 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/700866863 >> v2:172.21.15.87:3300/0 conn(0x7f2d8815ea70 0x7f2d8815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/700866863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d880d0510 0x7f2d8815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/700866863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d88161f90 0x7f2d880cffd0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 -- 172.21.15.80:0/700866863 >> 172.21.15.80:0/700866863 conn(0x7f2d8806d080 msgr2=0x7f2d880b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 -- 172.21.15.80:0/700866863 shutdown_connections 2024-04-25T13:41:20.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.966+0000 7f2d8ef31640 1 -- 172.21.15.80:0/700866863 wait complete. 2024-04-25T13:41:20.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 Processor -- start 2024-04-25T13:41:20.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 -- start start 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d880d0510 0x7f2d880b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d8815ea70 0x7f2d8813fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d88161f90 0x7f2d881403e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2d881447a0 con 0x7f2d880d0510 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2d88144620 con 0x7f2d8815ea70 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2d880d0920 con 0x7f2d88161f90 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d880d0510 0x7f2d880b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d880d0510 0x7f2d880b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53360/0 (socket says 172.21.15.80:53360) 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 -- 172.21.15.80:0/1119437429 learned_addr learned my addr 172.21.15.80:0/1119437429 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:20.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d87fff640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d8815ea70 0x7f2d8813fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 -- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d88161f90 msgr2=0x7f2d881403e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d88161f90 0x7f2d881403e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 -- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d8815ea70 msgr2=0x7f2d8813fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d8815ea70 0x7f2d8813fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 -- 172.21.15.80:0/1119437429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d8815a9c0 con 0x7f2d880d0510 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8cccd640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d880d0510 0x7f2d880b1fd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2d8001cd00 tx=0x7f2d8001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d857fa640 1 -- 172.21.15.80:0/1119437429 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d800b1040 con 0x7f2d880d0510 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d857fa640 1 -- 172.21.15.80:0/1119437429 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d800bd070 con 0x7f2d880d0510 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d8815aca0 con 0x7f2d880d0510 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d857fa640 1 -- 172.21.15.80:0/1119437429 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d800b7620 con 0x7f2d880d0510 2024-04-25T13:41:20.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d8815b260 con 0x7f2d880d0510 2024-04-25T13:41:20.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.970+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8806a3a0 con 0x7f2d880d0510 2024-04-25T13:41:20.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.974+0000 7f2d857fa640 1 -- 172.21.15.80:0/1119437429 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2d800b77c0 con 0x7f2d880d0510 2024-04-25T13:41:20.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.974+0000 7f2d857fa640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2d7406f400 0x7f2d74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:20.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.974+0000 7f2d857fa640 1 -- 172.21.15.80:0/1119437429 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f2d8012a130 con 0x7f2d880d0510 2024-04-25T13:41:20.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.974+0000 7f2d87fff640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2d7406f400 0x7f2d74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:20.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.974+0000 7f2d87fff640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2d7406f400 0x7f2d74071910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006fd0 tx=0x7f2d7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:20.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:20.978+0000 7f2d857fa640 1 -- 172.21.15.80:0/1119437429 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2d80016110 con 0x7f2d880d0510 2024-04-25T13:41:21.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2d88161520 con 0x7f2d880d0510 2024-04-25T13:41:21.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d857fa640 1 -- 172.21.15.80:0/1119437429 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f2d800f7080 con 0x7f2d880d0510 2024-04-25T13:41:21.168 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:21.168 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:40:29.558137+0000","modified":"2024-04-25T13:40:32.595776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-25T13:41:21.169 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-25T13:41:21.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2d7406f400 msgr2=0x7f2d74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2d7406f400 0x7f2d74071910 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c006fd0 tx=0x7f2d7c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d880d0510 msgr2=0x7f2d880b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d880d0510 0x7f2d880b1fd0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2d8001cd00 tx=0x7f2d8001aca0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 shutdown_connections 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2d7406f400 0x7f2d74071910 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2d88161f90 0x7f2d881403e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2d8815ea70 0x7f2d8813fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 --2- 172.21.15.80:0/1119437429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2d880d0510 0x7f2d880b1fd0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 >> 172.21.15.80:0/1119437429 conn(0x7f2d8806d080 msgr2=0x7f2d8815f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 shutdown_connections 2024-04-25T13:41:21.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.166+0000 7f2d8ef31640 1 -- 172.21.15.80:0/1119437429 wait complete. 2024-04-25T13:41:21.189 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:21.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.330+0000 7f607f353640 1 Processor -- start 2024-04-25T13:41:21.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.330+0000 7f607f353640 1 -- start start 2024-04-25T13:41:21.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607f353640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d2e80 0x7f60781607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607f353640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f60780d59b0 0x7f60780d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607f353640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60780d6300 con 0x7f60780d4330 2024-04-25T13:41:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607f353640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6078161060 con 0x7f60780d59b0 2024-04-25T13:41:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607f353640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6078161340 con 0x7f60780d2e80 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607c8ee640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f60780d59b0 0x7f60780d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607d0ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d2e80 0x7f60781607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607d8f0640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f60780d4330 0x7f60780d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:36348/0 (socket says 172.21.15.80:36348) 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607d8f0640 1 -- 172.21.15.80:0/60684547 learned_addr learned my addr 172.21.15.80:0/60684547 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607d0ef640 1 -- 172.21.15.80:0/60684547 >> v2:172.21.15.87:3300/0 conn(0x7f60780d59b0 msgr2=0x7f60780d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607d0ef640 1 --2- 172.21.15.80:0/60684547 >> v2:172.21.15.87:3300/0 conn(0x7f60780d59b0 0x7f60780d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607d0ef640 1 -- 172.21.15.80:0/60684547 >> v1:172.21.15.87:6789/0 conn(0x7f60780d4330 legacy=0x7f60780d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607d0ef640 1 -- 172.21.15.80:0/60684547 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60780d2680 con 0x7f60780d2e80 2024-04-25T13:41:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.334+0000 7f607c8ee640 1 --2- 172.21.15.80:0/60684547 >> v2:172.21.15.87:3300/0 conn(0x7f60780d59b0 0x7f60780d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:41:21.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607d0ef640 1 --2- 172.21.15.80:0/60684547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d2e80 0x7f60781607a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f60740097c0 tx=0x7f607401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dee3a53eeb83569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:21.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f606ffff640 1 -- 172.21.15.80:0/60684547 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60740b0070 con 0x7f60780d2e80 2024-04-25T13:41:21.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f606ffff640 1 -- 172.21.15.80:0/60684547 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60740ab600 con 0x7f60780d2e80 2024-04-25T13:41:21.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f606ffff640 1 -- 172.21.15.80:0/60684547 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60740ab9d0 con 0x7f60780d2e80 2024-04-25T13:41:21.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607f353640 1 -- 172.21.15.80:0/60684547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d2e80 msgr2=0x7f60781607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607f353640 1 --2- 172.21.15.80:0/60684547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d2e80 0x7f60781607a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f60740097c0 tx=0x7f607401c410 comp rx=0 tx=0).stop 2024-04-25T13:41:21.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607f353640 1 -- 172.21.15.80:0/60684547 shutdown_connections 2024-04-25T13:41:21.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607f353640 1 --2- 172.21.15.80:0/60684547 >> v2:172.21.15.87:3300/0 conn(0x7f60780d59b0 0x7f60780d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607f353640 1 --2- 172.21.15.80:0/60684547 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d2e80 0x7f60781607a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607f353640 1 -- 172.21.15.80:0/60684547 >> 172.21.15.80:0/60684547 conn(0x7f60780737a0 msgr2=0x7f607806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:21.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607f353640 1 -- 172.21.15.80:0/60684547 shutdown_connections 2024-04-25T13:41:21.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.338+0000 7f607f353640 1 -- 172.21.15.80:0/60684547 wait complete. 2024-04-25T13:41:21.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 Processor -- start 2024-04-25T13:41:21.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 -- start start 2024-04-25T13:41:21.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60780d2e80 0x7f607815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60780d4330 0x7f6078156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d59b0 0x7f60781ff060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f60780d3410 con 0x7f60780d4330 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f60780d3290 con 0x7f60780d2e80 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f607815df60 con 0x7f60780d59b0 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60780d4330 0x7f6078156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60780d4330 0x7f6078156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48202/0 (socket says 172.21.15.80:48202) 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 -- 172.21.15.80:0/4053905260 learned_addr learned my addr 172.21.15.80:0/4053905260 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607d8f0640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d59b0 0x7f60781ff060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607d0ef640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60780d2e80 0x7f607815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 -- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d59b0 msgr2=0x7f60781ff060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d59b0 0x7f60781ff060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 -- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60780d2e80 msgr2=0x7f607815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60780d2e80 0x7f607815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 -- 172.21.15.80:0/4053905260 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6078163720 con 0x7f60780d4330 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607c8ee640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60780d4330 0x7f6078156bd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f606800e9b0 tx=0x7f606800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f606dffb640 1 -- 172.21.15.80:0/4053905260 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6068017070 con 0x7f60780d4330 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f606dffb640 1 -- 172.21.15.80:0/4053905260 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f606800ca00 con 0x7f60780d4330 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f607815ccc0 con 0x7f60780d4330 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f606dffb640 1 -- 172.21.15.80:0/4053905260 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606800cdd0 con 0x7f60780d4330 2024-04-25T13:41:21.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.342+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f607815b200 con 0x7f60780d4330 2024-04-25T13:41:21.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.346+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f607806a3a0 con 0x7f60780d4330 2024-04-25T13:41:21.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.346+0000 7f606dffb640 1 -- 172.21.15.80:0/4053905260 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f606801a070 con 0x7f60780d4330 2024-04-25T13:41:21.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.346+0000 7f606dffb640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f605c06f400 0x7f605c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.346+0000 7f606dffb640 1 -- 172.21.15.80:0/4053905260 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f606808f260 con 0x7f60780d4330 2024-04-25T13:41:21.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.346+0000 7f607d0ef640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f605c06f400 0x7f605c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.346+0000 7f607d0ef640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f605c06f400 0x7f605c071910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60740ad000 tx=0x7f6074008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:21.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.350+0000 7f606dffb640 1 -- 172.21.15.80:0/4053905260 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f606805c0d0 con 0x7f60780d4330 2024-04-25T13:41:21.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.546+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f607815cf60 con 0x7f60780d4330 2024-04-25T13:41:21.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.546+0000 7f606dffb640 1 -- 172.21.15.80:0/4053905260 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5172 (secure 0 0 0) 0x7f606805b9d0 con 0x7f60780d4330 2024-04-25T13:41:21.550 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:21.550 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/119838745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":119838745},{"type":"v1","addr":"172.21.15.87:6837","nonce":119838745}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4260,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2208359696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2208359696},{"type":"v1","addr":"172.21.15.80:6834","nonce":2208359696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4263,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2722255059","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2722255059},{"type":"v1","addr":"172.21.15.80:6835","nonce":2722255059}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:40:29.558137+0000","modified":"2024-04-25T13:40:32.595776+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4272},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4272":{"gid":4272,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/3340293864","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":3340293864},{"type":"v1","addr":"172.21.15.87:6835","nonce":3340293864}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4272,"qdb_cluster":[4272]},"id":1}]} 2024-04-25T13:41:21.550 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 7 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f605c06f400 msgr2=0x7f605c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f605c06f400 0x7f605c071910 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f60740ad000 tx=0x7f6074008040 comp rx=0 tx=0).stop 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60780d4330 msgr2=0x7f6078156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60780d4330 0x7f6078156bd0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f606800e9b0 tx=0x7f606800ee70 comp rx=0 tx=0).stop 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 shutdown_connections 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f605c06f400 0x7f605c071910 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60780d59b0 0x7f60781ff060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60780d4330 0x7f6078156bd0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 --2- 172.21.15.80:0/4053905260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60780d2e80 0x7f607815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 >> 172.21.15.80:0/4053905260 conn(0x7f60780737a0 msgr2=0x7f6078162200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 shutdown_connections 2024-04-25T13:41:21.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.550+0000 7f607f353640 1 -- 172.21.15.80:0/4053905260 wait complete. 2024-04-25T13:41:21.572 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:21.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.702+0000 7fd2bdd29640 1 Processor -- start 2024-04-25T13:41:21.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.702+0000 7fd2bdd29640 1 -- start start 2024-04-25T13:41:21.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2bdd29640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd2b80d06e0 0x7fd2b806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2bdd29640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2b806e9e0 0x7fd2b806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2bdd29640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2b80d0af0 con 0x7fd2b8161f90 2024-04-25T13:41:21.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2bdd29640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd2b80d0d70 con 0x7fd2b80d06e0 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2bdd29640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd2b80d1050 con 0x7fd2b806e9e0 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b77fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd2b8161f90 0x7fd2b80cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43300/0 (socket says 172.21.15.80:43300) 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b77fe640 1 -- 172.21.15.80:0/442745603 learned_addr learned my addr 172.21.15.80:0/442745603 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b6ffd640 1 --2- 172.21.15.80:0/442745603 >> v2:172.21.15.80:3300/0 conn(0x7fd2b80d06e0 0x7fd2b806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b67fc640 1 -- 172.21.15.80:0/442745603 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1855187668 0 0) 0x7fd2a8001120 con 0x7fd2b8161f90 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b67fc640 1 -- 172.21.15.80:0/442745603 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd2a0003530 con 0x7fd2b8161f90 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b7fff640 1 --2- 172.21.15.80:0/442745603 >> v2:172.21.15.87:3300/0 conn(0x7fd2b806e9e0 0x7fd2b806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b6ffd640 1 -- 172.21.15.80:0/442745603 >> v2:172.21.15.87:3300/0 conn(0x7fd2b806e9e0 msgr2=0x7fd2b806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b6ffd640 1 --2- 172.21.15.80:0/442745603 >> v2:172.21.15.87:3300/0 conn(0x7fd2b806e9e0 0x7fd2b806edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b6ffd640 1 -- 172.21.15.80:0/442745603 >> v1:172.21.15.80:6789/0 conn(0x7fd2b8161f90 legacy=0x7fd2b80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.706+0000 7fd2b6ffd640 1 -- 172.21.15.80:0/442745603 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b806dca0 con 0x7fd2b80d06e0 2024-04-25T13:41:21.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2b6ffd640 1 --2- 172.21.15.80:0/442745603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b806e4a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80b4b80 tx=0x7fd2ac01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec300440d84443b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:21.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2b67fc640 1 -- 172.21.15.80:0/442745603 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0b0070 con 0x7fd2b80d06e0 2024-04-25T13:41:21.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2b67fc640 1 -- 172.21.15.80:0/442745603 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0ab600 con 0x7fd2b80d06e0 2024-04-25T13:41:21.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2b67fc640 1 -- 172.21.15.80:0/442745603 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0ab9b0 con 0x7fd2b80d06e0 2024-04-25T13:41:21.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 -- 172.21.15.80:0/442745603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 msgr2=0x7fd2b806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/442745603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b806e4a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80b4b80 tx=0x7fd2ac01c330 comp rx=0 tx=0).stop 2024-04-25T13:41:21.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 -- 172.21.15.80:0/442745603 shutdown_connections 2024-04-25T13:41:21.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/442745603 >> v2:172.21.15.87:3300/0 conn(0x7fd2b806e9e0 0x7fd2b806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/442745603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b806e4a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 -- 172.21.15.80:0/442745603 >> 172.21.15.80:0/442745603 conn(0x7fd2b806d080 msgr2=0x7fd2b80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:21.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 -- 172.21.15.80:0/442745603 shutdown_connections 2024-04-25T13:41:21.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 -- 172.21.15.80:0/442745603 wait complete. 2024-04-25T13:41:21.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 Processor -- start 2024-04-25T13:41:21.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.710+0000 7fd2bdd29640 1 -- start start 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b806e9e0 0x7fd2b80736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b80af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2b8161f90 0x7fd2b80afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b81447a0 con 0x7fd2b80d06e0 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2b8144620 con 0x7fd2b806e9e0 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd2b816b510 con 0x7fd2b8161f90 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b80af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b80af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48220/0 (socket says 172.21.15.80:48220) 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 -- 172.21.15.80:0/2411320991 learned_addr learned my addr 172.21.15.80:0/2411320991 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b77fe640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b806e9e0 0x7fd2b80736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b7fff640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2b8161f90 0x7fd2b80afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 -- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2b8161f90 msgr2=0x7fd2b80afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2b8161f90 0x7fd2b80afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 -- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b806e9e0 msgr2=0x7fd2b80736f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b806e9e0 0x7fd2b80736f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 -- 172.21.15.80:0/2411320991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2b81460b0 con 0x7fd2b80d06e0 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b7fff640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2b8161f90 0x7fd2b80afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2b6ffd640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b80af8b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0ae040 tx=0x7fd2ac0190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd29ffff640 1 -- 172.21.15.80:0/2411320991 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0b0040 con 0x7fd2b80d06e0 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd29ffff640 1 -- 172.21.15.80:0/2411320991 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2ac0ab9b0 con 0x7fd2b80d06e0 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2b8146390 con 0x7fd2b80d06e0 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd29ffff640 1 -- 172.21.15.80:0/2411320991 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac0abd60 con 0x7fd2b80d06e0 2024-04-25T13:41:21.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2b8146950 con 0x7fd2b80d06e0 2024-04-25T13:41:21.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2b8073d60 con 0x7fd2b80d06e0 2024-04-25T13:41:21.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd29ffff640 1 -- 172.21.15.80:0/2411320991 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd2ac003620 con 0x7fd2b80d06e0 2024-04-25T13:41:21.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.714+0000 7fd29ffff640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2a406f400 0x7fd2a4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:21.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.718+0000 7fd29ffff640 1 -- 172.21.15.80:0/2411320991 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7fd2ac128f00 con 0x7fd2b80d06e0 2024-04-25T13:41:21.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.718+0000 7fd2b77fe640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2a406f400 0x7fd2a4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:21.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.718+0000 7fd2b77fe640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2a406f400 0x7fd2a4071910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8005890 tx=0x7fd2a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:21.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.722+0000 7fd29ffff640 1 -- 172.21.15.80:0/2411320991 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2ac016070 con 0x7fd2b80d06e0 2024-04-25T13:41:21.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.866+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2b806a3a0 con 0x7fd2b80d06e0 2024-04-25T13:41:21.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd29ffff640 1 -- 172.21.15.80:0/2411320991 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16781 (secure 0 0 0) 0x7fd2ac0f5e00 con 0x7fd2b80d06e0 2024-04-25T13:41:21.873 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:21.873 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":22,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:17.425119+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:40:27.648187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T13:40:28.506496+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":2}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:21.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2a406f400 msgr2=0x7fd2a4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2a406f400 0x7fd2a4071910 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8005890 tx=0x7fd2a8009040 comp rx=0 tx=0).stop 2024-04-25T13:41:21.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 msgr2=0x7fd2b80af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:21.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b80af8b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0ae040 tx=0x7fd2ac0190d0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 shutdown_connections 2024-04-25T13:41:21.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2a406f400 0x7fd2a4071910 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2b8161f90 0x7fd2b80afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2b80d06e0 0x7fd2b80af8b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 --2- 172.21.15.80:0/2411320991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2b806e9e0 0x7fd2b80736f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:21.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.870+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 >> 172.21.15.80:0/2411320991 conn(0x7fd2b806d080 msgr2=0x7fd2b8160fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:21.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.874+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 shutdown_connections 2024-04-25T13:41:21.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:21.874+0000 7fd2bdd29640 1 -- 172.21.15.80:0/2411320991 wait complete. 2024-04-25T13:41:21.893 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:41:22.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.022+0000 7f239e4cb640 1 Processor -- start 2024-04-25T13:41:22.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.022+0000 7f239e4cb640 1 -- start start 2024-04-25T13:41:22.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239e4cb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239806eba0 0x7f23980ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239e4cb640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2398073710 con 0x7f23980ce730 2024-04-25T13:41:22.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239e4cb640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23980739f0 con 0x7f23980b0ed0 2024-04-25T13:41:22.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239e4cb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2398073cd0 con 0x7f239806eba0 2024-04-25T13:41:22.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f238ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f23980ce730 0x7f239806e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43316/0 (socket says 172.21.15.80:43316) 2024-04-25T13:41:22.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f238ffff640 1 -- 172.21.15.80:0/4172917428 learned_addr learned my addr 172.21.15.80:0/4172917428 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239ca68640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239806eba0 0x7f23980ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3736136061 0 0) 0x7f2398073710 con 0x7f23980ce730 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2384003530 con 0x7f23980ce730 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 486376391 0 0) 0x7f2384003530 con 0x7f23980ce730 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239806eba0 msgr2=0x7f23980ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239806eba0 0x7f23980ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 >> v1:172.21.15.87:6789/0 conn(0x7f23980b0ed0 legacy=0x7f239815c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2398073fb0 con 0x7f23980ce730 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2380001680 con 0x7f23980ce730 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 >> v1:172.21.15.80:6789/0 conn(0x7f23980ce730 legacy=0x7f239806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23840039f0 0x7f2384003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384004810 0x7f2384004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2384005160 0x7f238400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2398073cd0 con 0x7f2384005160 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f238400bfa0 con 0x7f23840039f0 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f238400c1a0 con 0x7f2384004810 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239ca68640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2384005160 0x7f238400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f2397fff640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23840039f0 0x7f2384003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f238ffff640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384004810 0x7f2384004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239ca68640 1 -- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384004810 msgr2=0x7f2384004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239ca68640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384004810 0x7f2384004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239ca68640 1 -- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23840039f0 msgr2=0x7f2384003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239ca68640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23840039f0 0x7f2384003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.026+0000 7f239ca68640 1 -- 172.21.15.80:0/4172917428 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2384003530 con 0x7f2384005160 2024-04-25T13:41:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239ca68640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2384005160 0x7f238400b9d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f23980b4970 tx=0x7f239001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bb0af55cb03a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900b2070 con 0x7f2384005160 2024-04-25T13:41:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23900a5c50 con 0x7f2384005160 2024-04-25T13:41:22.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f23977fe640 1 -- 172.21.15.80:0/4172917428 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2390009c10 con 0x7f2384005160 2024-04-25T13:41:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239e4cb640 1 -- 172.21.15.80:0/4172917428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2384005160 msgr2=0x7f238400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239e4cb640 1 --2- 172.21.15.80:0/4172917428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2384005160 0x7f238400b9d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f23980b4970 tx=0x7f239001cc40 comp rx=0 tx=0).stop 2024-04-25T13:41:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239ca68640 1 -- 172.21.15.80:0/4172917428 reap_dead start 2024-04-25T13:41:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239e4cb640 1 -- 172.21.15.80:0/4172917428 shutdown_connections 2024-04-25T13:41:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239e4cb640 1 -- 172.21.15.80:0/4172917428 >> 172.21.15.80:0/4172917428 conn(0x7f239806d080 msgr2=0x7f23980b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239e4cb640 1 -- 172.21.15.80:0/4172917428 shutdown_connections 2024-04-25T13:41:22.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239e4cb640 1 -- 172.21.15.80:0/4172917428 wait complete. 2024-04-25T13:41:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239e4cb640 1 Processor -- start 2024-04-25T13:41:22.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.030+0000 7f239e4cb640 1 -- start start 2024-04-25T13:41:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384005160 0x7f2398155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2398159530 0x7f2398155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f239815b480 0x7f239815b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f239806eba0 con 0x7f239815b480 2024-04-25T13:41:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f23980733a0 con 0x7f2398159530 2024-04-25T13:41:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f239806ed20 con 0x7f2384005160 2024-04-25T13:41:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f239815b480 0x7f239815b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f239815b480 0x7f239815b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48240/0 (socket says 172.21.15.80:48240) 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 -- 172.21.15.80:0/881313131 learned_addr learned my addr 172.21.15.80:0/881313131 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f2397fff640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384005160 0x7f2398155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f238ffff640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2398159530 0x7f2398155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 -- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384005160 msgr2=0x7f2398155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384005160 0x7f2398155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 -- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2398159530 msgr2=0x7f2398155b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2398159530 0x7f2398155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 -- 172.21.15.80:0/881313131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239815da90 con 0x7f239815b480 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239ca68640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f239815b480 0x7f239815b890 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f23900b0040 tx=0x7f2390009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f23957fa640 1 -- 172.21.15.80:0/881313131 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900c0070 con 0x7f239815b480 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f23957fa640 1 -- 172.21.15.80:0/881313131 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23900adb50 con 0x7f239815b480 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239815d510 con 0x7f239815b480 2024-04-25T13:41:22.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f23957fa640 1 -- 172.21.15.80:0/881313131 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23900b2040 con 0x7f239815b480 2024-04-25T13:41:22.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239815bdd0 con 0x7f239815b480 2024-04-25T13:41:22.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2398159940 con 0x7f239815b480 2024-04-25T13:41:22.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f23957fa640 1 -- 172.21.15.80:0/881313131 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f23900c3070 con 0x7f239815b480 2024-04-25T13:41:22.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f23957fa640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237806f430 0x7f2378071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f23957fa640 1 -- 172.21.15.80:0/881313131 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5825+0+0 (secure 0 0 0) 0x7f239012ba90 con 0x7f239815b480 2024-04-25T13:41:22.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.034+0000 7f2397fff640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237806f430 0x7f2378071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.038+0000 7f2397fff640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237806f430 0x7f2378071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c10 tx=0x7f2388009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:22.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.038+0000 7f23957fa640 1 -- 172.21.15.80:0/881313131 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2390016950 con 0x7f239815b480 2024-04-25T13:41:22.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.218+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f239806a3a0 con 0x7f239815b480 2024-04-25T13:41:22.223 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:41:22.222+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:41:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.222+0000 7f23957fa640 1 -- 172.21.15.80:0/881313131 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) ==== 161+0+0 (secure 0 0 0) 0x7f23900f82b0 con 0x7f239815b480 2024-04-25T13:41:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:41:22.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237806f430 msgr2=0x7f2378071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237806f430 0x7f2378071940 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2388000c10 tx=0x7f2388009040 comp rx=0 tx=0).stop 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f239815b480 msgr2=0x7f239815b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f239815b480 0x7f239815b890 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f23900b0040 tx=0x7f2390009750 comp rx=0 tx=0).stop 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 shutdown_connections 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237806f430 0x7f2378071940 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f239815b480 0x7f239815b890 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2398159530 0x7f2398155b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 --2- 172.21.15.80:0/881313131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2384005160 0x7f2398155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 >> 172.21.15.80:0/881313131 conn(0x7f239806d080 msgr2=0x7f23980d0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 shutdown_connections 2024-04-25T13:41:22.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.226+0000 7f239e4cb640 1 -- 172.21.15.80:0/881313131 wait complete. 2024-04-25T13:41:22.240 INFO:tasks.ceph.mds.d.smithi087.stderr: -3407> 2024-04-25T13:40:33.983+0000 7f32083a1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:41:22.249 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:41:22.341 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:41:22.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.386+0000 7fbe6cee0640 1 Processor -- start 2024-04-25T13:41:22.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.386+0000 7fbe6cee0640 1 -- start start 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe68162270 0x7fbe6806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe680b5740 con 0x7fbe6806edb0 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe6806dd10 con 0x7fbe680b5330 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbe6806f700 con 0x7fbe68162270 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe66d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe68162270 0x7fbe6806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe65d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fbe6806edb0 0x7fbe68161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43332/0 (socket says 172.21.15.80:43332) 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe66d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe68162270 0x7fbe6806f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48256/0 (socket says 172.21.15.80:48256) 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe65d9b640 1 -- 172.21.15.80:0/1011407451 learned_addr learned my addr 172.21.15.80:0/1011407451 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe66d9d640 1 -- 172.21.15.80:0/1011407451 >> v1:172.21.15.87:6789/0 conn(0x7fbe680b5330 legacy=0x7fbe680d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:22.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe66d9d640 1 -- 172.21.15.80:0/1011407451 >> v1:172.21.15.80:6789/0 conn(0x7fbe6806edb0 legacy=0x7fbe68161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe66d9d640 1 -- 172.21.15.80:0/1011407451 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6806dff0 con 0x7fbe68162270 2024-04-25T13:41:22.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe66d9d640 1 --2- 172.21.15.80:0/1011407451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe68162270 0x7fbe6806f1c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009d90 tx=0x7fbe6001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54af363423a1302 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:22.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6559a640 1 -- 172.21.15.80:0/1011407451 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600b2070 con 0x7fbe68162270 2024-04-25T13:41:22.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6559a640 1 -- 172.21.15.80:0/1011407451 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe600a5850 con 0x7fbe68162270 2024-04-25T13:41:22.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6559a640 1 -- 172.21.15.80:0/1011407451 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe600a5c20 con 0x7fbe68162270 2024-04-25T13:41:22.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1011407451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe68162270 msgr2=0x7fbe6806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 --2- 172.21.15.80:0/1011407451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe68162270 0x7fbe6806f1c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbe60009d90 tx=0x7fbe6001c8a0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1011407451 shutdown_connections 2024-04-25T13:41:22.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 --2- 172.21.15.80:0/1011407451 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe68162270 0x7fbe6806f1c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1011407451 >> 172.21.15.80:0/1011407451 conn(0x7fbe680b4160 msgr2=0x7fbe680d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:22.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1011407451 shutdown_connections 2024-04-25T13:41:22.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.390+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1011407451 wait complete. 2024-04-25T13:41:22.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 Processor -- start 2024-04-25T13:41:22.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 -- start start 2024-04-25T13:41:22.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe6806edb0 0x7fbe6815ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe680b5330 0x7fbe681609b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe68147b30 0x7fbe68160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbe680b0e50 con 0x7fbe6806edb0 2024-04-25T13:41:22.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe680b0cd0 con 0x7fbe680b5330 2024-04-25T13:41:22.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe680d11c0 con 0x7fbe68147b30 2024-04-25T13:41:22.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe6806edb0 0x7fbe6815ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe6806edb0 0x7fbe6815ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48270/0 (socket says 172.21.15.80:48270) 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 -- 172.21.15.80:0/1227617062 learned_addr learned my addr 172.21.15.80:0/1227617062 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 -- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe68147b30 msgr2=0x7fbe68160ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe68147b30 0x7fbe68160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 -- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe680b5330 msgr2=0x7fbe681609b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe680b5330 0x7fbe681609b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 -- 172.21.15.80:0/1227617062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe6814f3f0 con 0x7fbe6806edb0 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6659c640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe6806edb0 0x7fbe6815ec70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800b410 tx=0x7fbe5800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe56ffd640 1 -- 172.21.15.80:0/1227617062 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58015030 con 0x7fbe6806edb0 2024-04-25T13:41:22.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe56ffd640 1 -- 172.21.15.80:0/1227617062 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe58004020 con 0x7fbe6806edb0 2024-04-25T13:41:22.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe6814f6d0 con 0x7fbe6806edb0 2024-04-25T13:41:22.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe56ffd640 1 -- 172.21.15.80:0/1227617062 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58004450 con 0x7fbe6806edb0 2024-04-25T13:41:22.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.394+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe68158e10 con 0x7fbe6806edb0 2024-04-25T13:41:22.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.398+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6806a3a0 con 0x7fbe6806edb0 2024-04-25T13:41:22.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.398+0000 7fbe56ffd640 1 -- 172.21.15.80:0/1227617062 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbe58018070 con 0x7fbe6806edb0 2024-04-25T13:41:22.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.398+0000 7fbe56ffd640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe4806f400 0x7fbe48071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:22.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.398+0000 7fbe56ffd640 1 -- 172.21.15.80:0/1227617062 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7fbe5808eeb0 con 0x7fbe6806edb0 2024-04-25T13:41:22.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.398+0000 7fbe65d9b640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe4806f400 0x7fbe48071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:22.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.398+0000 7fbe65d9b640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe4806f400 0x7fbe48071910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe680d1590 tx=0x7fbe5c007a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:22.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.402+0000 7fbe56ffd640 1 -- 172.21.15.80:0/1227617062 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe5805bd60 con 0x7fbe6806edb0 2024-04-25T13:41:22.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:22.598+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbe6806fd00 con 0x7fbe6806edb0 2024-04-25T13:41:23.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.226+0000 7fbe56ffd640 1 -- 172.21.15.80:0/1227617062 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7fbe5805b660 con 0x7fbe6806edb0 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe4806f400 msgr2=0x7fbe48071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe4806f400 0x7fbe48071910 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe680d1590 tx=0x7fbe5c007a50 comp rx=0 tx=0).stop 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe6806edb0 msgr2=0x7fbe6815ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe6806edb0 0x7fbe6815ec70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800b410 tx=0x7fbe5800b8d0 comp rx=0 tx=0).stop 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 shutdown_connections 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe4806f400 0x7fbe48071910 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe68147b30 0x7fbe68160ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe680b5330 0x7fbe681609b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 --2- 172.21.15.80:0/1227617062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe6806edb0 0x7fbe6815ec70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 >> 172.21.15.80:0/1227617062 conn(0x7fbe680b4160 msgr2=0x7fbe6814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:23.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 shutdown_connections 2024-04-25T13:41:23.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.230+0000 7fbe6cee0640 1 -- 172.21.15.80:0/1227617062 wait complete. 2024-04-25T13:41:23.260 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:41:23.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fd115f640 1 Processor -- start 2024-04-25T13:41:23.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fd115f640 1 -- start start 2024-04-25T13:41:23.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fd115f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9fcc0ce2a0 0x7f9fcc0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:23.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fd115f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fcc0b1320 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fd115f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fcc0cebf0 con 0x7f9fcc161d10 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fd115f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9fcc0b5820 con 0x7f9fcc0ce2a0 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fc259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9fcc0ce2a0 0x7f9fcc0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fcad9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9fcc0b0f10 0x7f9fcc0d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43346/0 (socket says 172.21.15.80:43346) 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.410+0000 7f9fcad9d640 1 -- 172.21.15.80:0/3866592152 learned_addr learned my addr 172.21.15.80:0/3866592152 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fc259c640 1 -- 172.21.15.80:0/3866592152 >> v1:172.21.15.87:6789/0 conn(0x7f9fcc161d10 legacy=0x7f9fcc162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fc259c640 1 -- 172.21.15.80:0/3866592152 >> v1:172.21.15.80:6789/0 conn(0x7f9fcc0b0f10 legacy=0x7f9fcc0d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fca59c640 1 -- 172.21.15.80:0/3866592152 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 954609953 0 0) 0x7f9fbc001120 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fc259c640 1 -- 172.21.15.80:0/3866592152 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc0b4a60 con 0x7f9fcc0ce2a0 2024-04-25T13:41:23.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fc259c640 1 --2- 172.21.15.80:0/3866592152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0ce2a0 0x7f9fcc0ce6b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40097f0 tx=0x7f9fb401c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aab13a70082e2c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:23.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fca59c640 1 -- 172.21.15.80:0/3866592152 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40b0070 con 0x7f9fcc0ce2a0 2024-04-25T13:41:23.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fca59c640 1 -- 172.21.15.80:0/3866592152 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fb40ab600 con 0x7f9fcc0ce2a0 2024-04-25T13:41:23.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fca59c640 1 -- 172.21.15.80:0/3866592152 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb40ab9d0 con 0x7f9fcc0ce2a0 2024-04-25T13:41:23.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fd115f640 1 -- 172.21.15.80:0/3866592152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0ce2a0 msgr2=0x7f9fcc0ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:23.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fd115f640 1 --2- 172.21.15.80:0/3866592152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0ce2a0 0x7f9fcc0ce6b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40097f0 tx=0x7f9fb401c280 comp rx=0 tx=0).stop 2024-04-25T13:41:23.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fd115f640 1 -- 172.21.15.80:0/3866592152 shutdown_connections 2024-04-25T13:41:23.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fd115f640 1 --2- 172.21.15.80:0/3866592152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0ce2a0 0x7f9fcc0ce6b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:23.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fd115f640 1 -- 172.21.15.80:0/3866592152 >> 172.21.15.80:0/3866592152 conn(0x7f9fcc0b35c0 msgr2=0x7f9fcc0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:23.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fd115f640 1 -- 172.21.15.80:0/3866592152 shutdown_connections 2024-04-25T13:41:23.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.414+0000 7f9fd115f640 1 -- 172.21.15.80:0/3866592152 wait complete. 2024-04-25T13:41:23.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 Processor -- start 2024-04-25T13:41:23.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 -- start start 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0b0f10 0x7f9fcc0734a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fcc0ce2a0 0x7f9fcc0739e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fcc161d10 0x7f9fcc1fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc15e720 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc162c40 con 0x7f9fcc0ce2a0 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9fcc16c8d0 con 0x7f9fcc161d10 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0b0f10 0x7f9fcc0734a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0b0f10 0x7f9fcc0734a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48286/0 (socket says 172.21.15.80:48286) 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 -- 172.21.15.80:0/3263122107 learned_addr learned my addr 172.21.15.80:0/3263122107 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:23.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fc259c640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fcc0ce2a0 0x7f9fcc0739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcb59e640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fcc161d10 0x7f9fcc1fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 -- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fcc161d10 msgr2=0x7f9fcc1fbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fcc161d10 0x7f9fcc1fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 -- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fcc0ce2a0 msgr2=0x7f9fcc0739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fcc0ce2a0 0x7f9fcc0739e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 -- 172.21.15.80:0/3263122107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc151ef0 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fcad9d640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0b0f10 0x7f9fcc0734a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00c3e0 tx=0x7f9fbc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fc3fff640 1 -- 172.21.15.80:0/3263122107 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc0074e0 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc152aa0 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fc3fff640 1 -- 172.21.15.80:0/3263122107 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fbc0047f0 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc0b27c0 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.418+0000 7f9fc3fff640 1 -- 172.21.15.80:0/3263122107 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc004be0 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.422+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fcc06a440 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.422+0000 7f9fc3fff640 1 -- 172.21.15.80:0/3263122107 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9fbc011070 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.422+0000 7f9fc3fff640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9fac06f400 0x7f9fac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:23.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.422+0000 7f9fc3fff640 1 -- 172.21.15.80:0/3263122107 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5911+0+0 (secure 0 0 0) 0x7f9fbc08efc0 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.422+0000 7f9fc259c640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9fac06f400 0x7f9fac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:23.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.422+0000 7f9fc259c640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9fac06f400 0x7f9fac071910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40ad000 tx=0x7f9fb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:23.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.426+0000 7f9fc3fff640 1 -- 172.21.15.80:0/3263122107 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fcc0b1320 con 0x7f9fcc0b0f10 2024-04-25T13:41:23.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:23.582+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9fcc0b05d0 con 0x7f9fcc0b0f10 2024-04-25T13:41:24.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.478+0000 7f9fc3fff640 1 -- 172.21.15.80:0/3263122107 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) ==== 181+0+0 (secure 0 0 0) 0x7f9fbc05bdf0 con 0x7f9fcc0b0f10 2024-04-25T13:41:24.480 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:41:24.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9fac06f400 msgr2=0x7f9fac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:24.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9fac06f400 0x7f9fac071910 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9fb40ad000 tx=0x7f9fb4008040 comp rx=0 tx=0).stop 2024-04-25T13:41:24.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0b0f10 msgr2=0x7f9fcc0734a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:24.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0b0f10 0x7f9fcc0734a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00c3e0 tx=0x7f9fbc00c8a0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 shutdown_connections 2024-04-25T13:41:24.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9fac06f400 0x7f9fac071910 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fcc161d10 0x7f9fcc1fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fcc0ce2a0 0x7f9fcc0739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 --2- 172.21.15.80:0/3263122107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fcc0b0f10 0x7f9fcc0734a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 >> 172.21.15.80:0/3263122107 conn(0x7f9fcc0b35c0 msgr2=0x7f9fcc15f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:24.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 shutdown_connections 2024-04-25T13:41:24.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.482+0000 7f9fd115f640 1 -- 172.21.15.80:0/3263122107 wait complete. 2024-04-25T13:41:24.510 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:41:24.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.662+0000 7f57a8df1640 1 Processor -- start 2024-04-25T13:41:24.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.662+0000 7f57a8df1640 1 -- start start 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.662+0000 7f57a8df1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57a40b09a0 0x7f57a41602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.662+0000 7f57a8df1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 0x7f57a40d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.662+0000 7f57a8df1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a40b4a60 con 0x7f57a406fb30 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.662+0000 7f57a8df1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f57a4161270 con 0x7f57a40b09a0 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.662+0000 7f57a8df1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57a40b0db0 con 0x7f57a4160820 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f57a406fb30 0x7f57a40b6640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43360/0 (socket says 172.21.15.80:43360) 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 -- 172.21.15.80:0/3402451151 learned_addr learned my addr 172.21.15.80:0/3402451151 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a2d9d640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 0x7f57a40d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a1d9b640 1 --2- 172.21.15.80:0/3402451151 >> v2:172.21.15.87:3300/0 conn(0x7f57a40b09a0 0x7f57a41602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2471967931 0 0) 0x7f57a40b4a60 con 0x7f57a406fb30 2024-04-25T13:41:24.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5790005740 con 0x7f57a406fb30 2024-04-25T13:41:24.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1744645646 0 0) 0x7f5790005740 con 0x7f57a406fb30 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 msgr2=0x7f57a40d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 0x7f57a40d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 >> v2:172.21.15.87:3300/0 conn(0x7f57a40b09a0 msgr2=0x7f57a41602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 --2- 172.21.15.80:0/3402451151 >> v2:172.21.15.87:3300/0 conn(0x7f57a40b09a0 0x7f57a41602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a4160f90 con 0x7f57a406fb30 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f5794001a00 con 0x7f57a406fb30 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 >> v1:172.21.15.80:6789/0 conn(0x7f57a406fb30 legacy=0x7f57a40b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5790005bd0 0x7f5790006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790006a20 0x7f5790006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 0x7f5790007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57a4161270 con 0x7f5790005bd0 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40b0db0 con 0x7f5790006a20 2024-04-25T13:41:24.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f579000dbe0 con 0x7f57a4160820 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5790005bd0 0x7f5790006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a1d9b640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790006a20 0x7f5790006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a2d9d640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 0x7f5790007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 -- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 msgr2=0x7f5790007370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 0x7f5790007370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 -- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790006a20 msgr2=0x7f5790006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790006a20 0x7f5790006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 -- 172.21.15.80:0/3402451151 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5790005740 con 0x7f5790005bd0 2024-04-25T13:41:24.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a2d9d640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4160820 0x7f5790007370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:24.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a259c640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5790005bd0 0x7f5790006040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f579400a850 tx=0x7f579401d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b448b0df4f9c043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:24.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579400a4a0 con 0x7f5790005bd0 2024-04-25T13:41:24.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57940ad630 con 0x7f5790005bd0 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.666+0000 7f57a159a640 1 -- 172.21.15.80:0/3402451151 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57940ada20 con 0x7f5790005bd0 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- 172.21.15.80:0/3402451151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5790005bd0 msgr2=0x7f5790006040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 --2- 172.21.15.80:0/3402451151 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5790005bd0 0x7f5790006040 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f579400a850 tx=0x7f579401d380 comp rx=0 tx=0).stop 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a2d9d640 1 -- 172.21.15.80:0/3402451151 reap_dead start 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- 172.21.15.80:0/3402451151 shutdown_connections 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- 172.21.15.80:0/3402451151 >> 172.21.15.80:0/3402451151 conn(0x7f57a40b35c0 msgr2=0x7f57a40b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- 172.21.15.80:0/3402451151 shutdown_connections 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- 172.21.15.80:0/3402451151 wait complete. 2024-04-25T13:41:24.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 Processor -- start 2024-04-25T13:41:24.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- start start 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790005bd0 0x7f57a4151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4154830 0x7f57a4154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a4157b50 0x7f57a4157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40d04d0 con 0x7f57a4157b50 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40d0350 con 0x7f5790005bd0 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a8df1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57a40d0650 con 0x7f57a4154830 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a4157b50 0x7f57a4157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a4157b50 0x7f57a4157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48306/0 (socket says 172.21.15.80:48306) 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a2d9d640 1 -- 172.21.15.80:0/3587553368 learned_addr learned my addr 172.21.15.80:0/3587553368 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a1d9b640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4154830 0x7f57a4154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.670+0000 7f57a259c640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790005bd0 0x7f57a4151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a2d9d640 1 -- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4154830 msgr2=0x7f57a4154c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:24.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a2d9d640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4154830 0x7f57a4154c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a2d9d640 1 -- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790005bd0 msgr2=0x7f57a4151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a2d9d640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790005bd0 0x7f57a4151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a2d9d640 1 -- 172.21.15.80:0/3587553368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a4155180 con 0x7f57a4157b50 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a2d9d640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a4157b50 0x7f57a4157f60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f57a406ff40 tx=0x7f579c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f579affd640 1 -- 172.21.15.80:0/3587553368 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c00f040 con 0x7f57a4157b50 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f579affd640 1 -- 172.21.15.80:0/3587553368 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579c01c070 con 0x7f57a4157b50 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a4159b20 con 0x7f57a4157b50 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f579affd640 1 -- 172.21.15.80:0/3587553368 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579c00ea60 con 0x7f57a4157b50 2024-04-25T13:41:24.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a4159210 con 0x7f57a4157b50 2024-04-25T13:41:24.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a4159630 con 0x7f57a4157b50 2024-04-25T13:41:24.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f579affd640 1 -- 172.21.15.80:0/3587553368 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f579c00ec00 con 0x7f57a4157b50 2024-04-25T13:41:24.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f579affd640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578406f400 0x7f5784071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:24.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f579affd640 1 -- 172.21.15.80:0/3587553368 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5473+0+0 (secure 0 0 0) 0x7f579c08ecb0 con 0x7f57a4157b50 2024-04-25T13:41:24.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.674+0000 7f57a259c640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578406f400 0x7f5784071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:24.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.678+0000 7f57a259c640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578406f400 0x7f5784071910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f57940ad000 tx=0x7f57a40b4d00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:24.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.678+0000 7f579affd640 1 -- 172.21.15.80:0/3587553368 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f579c026020 con 0x7f57a4157b50 2024-04-25T13:41:24.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:24.830+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f57a406a440 con 0x7f57a4157b50 2024-04-25T13:41:25.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.478+0000 7f579affd640 1 -- 172.21.15.80:0/3587553368 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) ==== 169+0+0 (secure 0 0 0) 0x7f57a4155180 con 0x7f57a4157b50 2024-04-25T13:41:25.483 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:41:25.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578406f400 msgr2=0x7f5784071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578406f400 0x7f5784071910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f57940ad000 tx=0x7f57a40b4d00 comp rx=0 tx=0).stop 2024-04-25T13:41:25.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a4157b50 msgr2=0x7f57a4157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a4157b50 0x7f57a4157f60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f57a406ff40 tx=0x7f579c010c20 comp rx=0 tx=0).stop 2024-04-25T13:41:25.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 shutdown_connections 2024-04-25T13:41:25.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578406f400 0x7f5784071910 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a4157b50 0x7f57a4157f60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a4154830 0x7f57a4154c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 --2- 172.21.15.80:0/3587553368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5790005bd0 0x7f57a4151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 >> 172.21.15.80:0/3587553368 conn(0x7f57a40b35c0 msgr2=0x7f57a415a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:25.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.482+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 shutdown_connections 2024-04-25T13:41:25.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.486+0000 7f57a8df1640 1 -- 172.21.15.80:0/3587553368 wait complete. 2024-04-25T13:41:25.509 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:41:25.510 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:41:25.510 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:41:25.510 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:41:25.510 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:41:25.510 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:41:25.510 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:41:25.510 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:41:25.511 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:41:25.511 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:41:25.511 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:41:25.511 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:41:25.511 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:41:25.511 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:41:25.511 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:41:25.512 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:41:25.524 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:41:25.524 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:41:25.525 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:41:25.525 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:41:25.532 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:41:25.532 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:41:25.535 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:41:25.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:41:25.572 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:41:25.573 DEBUG:teuthology.parallel:result is None 2024-04-25T13:41:25.573 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:41:25.573 DEBUG:teuthology.parallel:result is None 2024-04-25T13:41:25.579 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:41:25.580 DEBUG:teuthology.parallel:result is None 2024-04-25T13:41:25.581 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:41:25.581 DEBUG:teuthology.parallel:result is None 2024-04-25T13:41:25.581 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:41:25.662 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:41:25.667 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:41:25.679 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:41:25.682 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:41:25.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.714+0000 7fc030684640 1 Processor -- start 2024-04-25T13:41:25.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.714+0000 7fc030684640 1 -- start start 2024-04-25T13:41:25.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc030684640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc028160b20 0x7fc02815de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc030684640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc028160f30 0x7fc028153d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc030684640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc028161450 con 0x7fc028073580 2024-04-25T13:41:25.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc030684640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc0280b2340 con 0x7fc028160b20 2024-04-25T13:41:25.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc030684640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc028161730 con 0x7fc028160f30 2024-04-25T13:41:25.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02ec21640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc028073580 0x7fc028073990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43366/0 (socket says 172.21.15.80:43366) 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02e420640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc028160b20 0x7fc02815de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02ec21640 1 -- 172.21.15.80:0/1228493607 learned_addr learned my addr 172.21.15.80:0/1228493607 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02dc1f640 1 --2- 172.21.15.80:0/1228493607 >> v2:172.21.15.87:3300/0 conn(0x7fc028160f30 0x7fc028153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02d41e640 1 -- 172.21.15.80:0/1228493607 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3182469647 0 0) 0x7fc028161450 con 0x7fc028073580 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02d41e640 1 -- 172.21.15.80:0/1228493607 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc014003530 con 0x7fc028073580 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02e420640 1 -- 172.21.15.80:0/1228493607 >> v2:172.21.15.87:3300/0 conn(0x7fc028160f30 msgr2=0x7fc028153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02e420640 1 --2- 172.21.15.80:0/1228493607 >> v2:172.21.15.87:3300/0 conn(0x7fc028160f30 0x7fc028153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02e420640 1 -- 172.21.15.80:0/1228493607 >> v1:172.21.15.80:6789/0 conn(0x7fc028073580 legacy=0x7fc028073990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02e420640 1 -- 172.21.15.80:0/1228493607 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc028161a10 con 0x7fc028160b20 2024-04-25T13:41:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02e420640 1 --2- 172.21.15.80:0/1228493607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160b20 0x7fc02815de40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc02806e100 tx=0x7fc02401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7cbf4053ed6c7ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02d41e640 1 -- 172.21.15.80:0/1228493607 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240b0070 con 0x7fc028160b20 2024-04-25T13:41:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02d41e640 1 -- 172.21.15.80:0/1228493607 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0240ab600 con 0x7fc028160b20 2024-04-25T13:41:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.718+0000 7fc02d41e640 1 -- 172.21.15.80:0/1228493607 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0240ab9b0 con 0x7fc028160b20 2024-04-25T13:41:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- 172.21.15.80:0/1228493607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160b20 msgr2=0x7fc02815de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 --2- 172.21.15.80:0/1228493607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160b20 0x7fc02815de40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fc02806e100 tx=0x7fc02401c330 comp rx=0 tx=0).stop 2024-04-25T13:41:25.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- 172.21.15.80:0/1228493607 shutdown_connections 2024-04-25T13:41:25.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 --2- 172.21.15.80:0/1228493607 >> v2:172.21.15.87:3300/0 conn(0x7fc028160f30 0x7fc028153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 --2- 172.21.15.80:0/1228493607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160b20 0x7fc02815de40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- 172.21.15.80:0/1228493607 >> 172.21.15.80:0/1228493607 conn(0x7fc0280d3500 msgr2=0x7fc028072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:25.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- 172.21.15.80:0/1228493607 shutdown_connections 2024-04-25T13:41:25.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- 172.21.15.80:0/1228493607 wait complete. 2024-04-25T13:41:25.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 Processor -- start 2024-04-25T13:41:25.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- start start 2024-04-25T13:41:25.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc028073580 0x7fc02814a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc028160b20 0x7fc028156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160f30 0x7fc028156f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc0281543f0 con 0x7fc028160f30 2024-04-25T13:41:25.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc028154270 con 0x7fc028073580 2024-04-25T13:41:25.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc030684640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc02806f400 con 0x7fc028160b20 2024-04-25T13:41:25.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc02ec21640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160f30 0x7fc028156f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc02ec21640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160f30 0x7fc028156f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48346/0 (socket says 172.21.15.80:48346) 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc02e420640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc028073580 0x7fc02814a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc02ec21640 1 -- 172.21.15.80:0/1951857218 learned_addr learned my addr 172.21.15.80:0/1951857218 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.722+0000 7fc02dc1f640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc028160b20 0x7fc028156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc02ec21640 1 -- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc028160b20 msgr2=0x7fc028156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc02ec21640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc028160b20 0x7fc028156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc02ec21640 1 -- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc028073580 msgr2=0x7fc02814a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc02ec21640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc028073580 0x7fc02814a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc02ec21640 1 -- 172.21.15.80:0/1951857218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc02814e2e0 con 0x7fc028160f30 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc02ec21640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160f30 0x7fc028156f70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc02000ca70 tx=0x7fc02000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:25.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc01affd640 1 -- 172.21.15.80:0/1951857218 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020017070 con 0x7fc028160f30 2024-04-25T13:41:25.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc01affd640 1 -- 172.21.15.80:0/1951857218 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc02000ab70 con 0x7fc028160f30 2024-04-25T13:41:25.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc02814e5c0 con 0x7fc028160f30 2024-04-25T13:41:25.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc01affd640 1 -- 172.21.15.80:0/1951857218 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc020012400 con 0x7fc028160f30 2024-04-25T13:41:25.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc02814eb80 con 0x7fc028160f30 2024-04-25T13:41:25.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc02806a3a0 con 0x7fc028160f30 2024-04-25T13:41:25.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc01affd640 1 -- 172.21.15.80:0/1951857218 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc02001a070 con 0x7fc028160f30 2024-04-25T13:41:25.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc01affd640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc00c06f400 0x7fc00c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:25.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc01affd640 1 -- 172.21.15.80:0/1951857218 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 4943+0+0 (secure 0 0 0) 0x7fc02008eb80 con 0x7fc028160f30 2024-04-25T13:41:25.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc02e420640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc00c06f400 0x7fc00c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:25.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.726+0000 7fc02e420640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc00c06f400 0x7fc00c071910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc0240ad000 tx=0x7fc0240a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:25.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.730+0000 7fc01affd640 1 -- 172.21.15.80:0/1951857218 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc02005b720 con 0x7fc028160f30 2024-04-25T13:41:25.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:25.898+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc02814fb80 con 0x7fc028160f30 2024-04-25T13:41:26.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.510+0000 7fc01affd640 1 -- 172.21.15.80:0/1951857218 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) ==== 97+0+0 (secure 0 0 0) 0x7fc02005fa20 con 0x7fc028160f30 2024-04-25T13:41:26.515 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:41:26.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc00c06f400 msgr2=0x7fc00c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc00c06f400 0x7fc00c071910 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc0240ad000 tx=0x7fc0240a6000 comp rx=0 tx=0).stop 2024-04-25T13:41:26.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160f30 msgr2=0x7fc028156f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160f30 0x7fc028156f70 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc02000ca70 tx=0x7fc02000cf30 comp rx=0 tx=0).stop 2024-04-25T13:41:26.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 shutdown_connections 2024-04-25T13:41:26.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc00c06f400 0x7fc00c071910 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc028160f30 0x7fc028156f70 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc028160b20 0x7fc028156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 --2- 172.21.15.80:0/1951857218 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc028073580 0x7fc02814a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.514+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 >> 172.21.15.80:0/1951857218 conn(0x7fc0280d3500 msgr2=0x7fc0280b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:26.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.518+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 shutdown_connections 2024-04-25T13:41:26.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.518+0000 7fc030684640 1 -- 172.21.15.80:0/1951857218 wait complete. 2024-04-25T13:41:26.542 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:41:26.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bdcff6640 1 Processor -- start 2024-04-25T13:41:26.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bdcff6640 1 -- start start 2024-04-25T13:41:26.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bdcff6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7bd8161240 0x7f7bd80723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:26.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bdcff6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd806fcf0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bdcff6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bd80b0c60 con 0x7f7bd8161650 2024-04-25T13:41:26.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bdcff6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7bd8073f10 con 0x7f7bd8161240 2024-04-25T13:41:26.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bd5d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7bd8161240 0x7f7bd80723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:26.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bd659c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7bd806f5a0 0x7f7bd8160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43374/0 (socket says 172.21.15.80:43374) 2024-04-25T13:41:26.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bd659c640 1 -- 172.21.15.80:0/3653673284 learned_addr learned my addr 172.21.15.80:0/3653673284 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bd5d9b640 1 -- 172.21.15.80:0/3653673284 >> v1:172.21.15.87:6789/0 conn(0x7f7bd8161650 legacy=0x7f7bd80728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bd5d9b640 1 -- 172.21.15.80:0/3653673284 >> v1:172.21.15.80:6789/0 conn(0x7f7bd806f5a0 legacy=0x7f7bd8160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bd559a640 1 -- 172.21.15.80:0/3653673284 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2437063962 0 0) 0x7f7bd806fcf0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.686+0000 7f7bd5d9b640 1 -- 172.21.15.80:0/3653673284 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd80b0f40 con 0x7f7bd8161240 2024-04-25T13:41:26.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bd5d9b640 1 --2- 172.21.15.80:0/3653673284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd8161240 0x7f7bd80723a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0097c0 tx=0x7f7bcc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdbec18d76322b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:26.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bd559a640 1 -- 172.21.15.80:0/3653673284 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0b0070 con 0x7f7bd8161240 2024-04-25T13:41:26.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bd559a640 1 -- 172.21.15.80:0/3653673284 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bcc0ab600 con 0x7f7bd8161240 2024-04-25T13:41:26.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bd559a640 1 -- 172.21.15.80:0/3653673284 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc0ab9d0 con 0x7f7bd8161240 2024-04-25T13:41:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3653673284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd8161240 msgr2=0x7f7bd80723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bdcff6640 1 --2- 172.21.15.80:0/3653673284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd8161240 0x7f7bd80723a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0097c0 tx=0x7f7bcc01c410 comp rx=0 tx=0).stop 2024-04-25T13:41:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3653673284 shutdown_connections 2024-04-25T13:41:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bdcff6640 1 --2- 172.21.15.80:0/3653673284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd8161240 0x7f7bd80723a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3653673284 >> 172.21.15.80:0/3653673284 conn(0x7f7bd806d080 msgr2=0x7f7bd80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3653673284 shutdown_connections 2024-04-25T13:41:26.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.690+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3653673284 wait complete. 2024-04-25T13:41:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.694+0000 7f7bdcff6640 1 Processor -- start 2024-04-25T13:41:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.694+0000 7f7bdcff6640 1 -- start start 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd806f5a0 0x7f7bd8156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bd8161650 0x7f7bd814ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bd814df30 0x7f7bd8137dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd816b270 con 0x7f7bd806f5a0 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd816b0f0 con 0x7f7bd814df30 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd816b3f0 con 0x7f7bd8161650 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd806f5a0 0x7f7bd8156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd806f5a0 0x7f7bd8156ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48360/0 (socket says 172.21.15.80:48360) 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 -- 172.21.15.80:0/3523047001 learned_addr learned my addr 172.21.15.80:0/3523047001 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd6d9d640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bd814df30 0x7f7bd8137dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:26.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd5d9b640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bd8161650 0x7f7bd814ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 -- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bd8161650 msgr2=0x7f7bd814ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bd8161650 0x7f7bd814ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 -- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bd814df30 msgr2=0x7f7bd8137dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bd814df30 0x7f7bd8137dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 -- 172.21.15.80:0/3523047001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd81571e0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd6d9d640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bd814df30 0x7f7bd8137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd659c640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd806f5a0 0x7f7bd8156ca0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c410 tx=0x7f7bc800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bc6ffd640 1 -- 172.21.15.80:0/3523047001 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc80074e0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bc6ffd640 1 -- 172.21.15.80:0/3523047001 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc80047f0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd814f2c0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bc6ffd640 1 -- 172.21.15.80:0/3523047001 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8004be0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd8151f50 con 0x7f7bd806f5a0 2024-04-25T13:41:26.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd806a3a0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bc6ffd640 1 -- 172.21.15.80:0/3523047001 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7bc8011070 con 0x7f7bd806f5a0 2024-04-25T13:41:26.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bc6ffd640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7bb806f430 0x7f7bb8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:26.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bc6ffd640 1 -- 172.21.15.80:0/3523047001 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f7bc808eff0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.698+0000 7f7bd5d9b640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7bb806f430 0x7f7bb8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:26.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.702+0000 7f7bd5d9b640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7bb806f430 0x7f7bb8071940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0ad000 tx=0x7f7bcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:26.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.702+0000 7f7bc6ffd640 1 -- 172.21.15.80:0/3523047001 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7bd81571e0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.922+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f7bd80739c0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.922+0000 7f7bc6ffd640 1 -- 172.21.15.80:0/3523047001 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) ==== 80+0+6645 (secure 0 0 0) 0x7f7bc805c3a0 con 0x7f7bd806f5a0 2024-04-25T13:41:26.926 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:26.926 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:41:26.926 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:41:26.926 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.926 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:41:26.926 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:41:26.927 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.928 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:41:26.929 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.930 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:41:26.931 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:41:26.932 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:41:26.933 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:41:26.934 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.935 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.936 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.937 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:41:26.938 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.bootstrap-mds", 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA7XSpmzQQXGhAAeWhN26gBRnzPH2ajvlHgMQ==", 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA7XSpm1hsXGhAAAE4JYoCqww74TEhO20sUtQ==", 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.939 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.bootstrap-osd", 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA7XSpmwS8XGhAAPGF/jueVONzSyQFh2OKe4g==", 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA7XSpmL0MXGhAA3akPX9r0SwT8MbcZIbLYaQ==", 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.940 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA7XSpm9lYXGhAAA8Aqh1qdN4Q8i9iFbhns0Q==", 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T13:41:26.941 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA7XSpmU2sXGhAApG4/leFE8lR7LFy9xx9+5A==", 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:26.942 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:41:26.943 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:41:26.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7bb806f430 msgr2=0x7f7bb8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7bb806f430 0x7f7bb8071940 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0ad000 tx=0x7f7bcc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:26.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd806f5a0 msgr2=0x7f7bd8156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:26.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd806f5a0 0x7f7bd8156ca0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c410 tx=0x7f7bc800c8d0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 shutdown_connections 2024-04-25T13:41:26.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7bb806f430 0x7f7bb8071940 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bd814df30 0x7f7bd8137dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bd8161650 0x7f7bd814ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 --2- 172.21.15.80:0/3523047001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bd806f5a0 0x7f7bd8156ca0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:26.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 >> 172.21.15.80:0/3523047001 conn(0x7f7bd806d080 msgr2=0x7f7bd8154750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:26.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 shutdown_connections 2024-04-25T13:41:26.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:26.926+0000 7f7bdcff6640 1 -- 172.21.15.80:0/3523047001 wait complete. 2024-04-25T13:41:26.947 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T13:41:27.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.094+0000 7f17b00ba640 1 Processor -- start 2024-04-25T13:41:27.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.094+0000 7f17b00ba640 1 -- start start 2024-04-25T13:41:27.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.094+0000 7f17b00ba640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17a816d390 0x7f17a816f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.094+0000 7f17b00ba640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a8072120 con 0x7f17a806eb40 2024-04-25T13:41:27.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.094+0000 7f17b00ba640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17a806dd80 con 0x7f17a81622d0 2024-04-25T13:41:27.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.094+0000 7f17b00ba640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f17a806e060 con 0x7f17a816d390 2024-04-25T13:41:27.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.094+0000 7f17ad655640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f17a806eb40 0x7f17a806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43384/0 (socket says 172.21.15.80:43384) 2024-04-25T13:41:27.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.094+0000 7f17ad655640 1 -- 172.21.15.80:0/4162907809 learned_addr learned my addr 172.21.15.80:0/4162907809 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:27.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ae657640 1 --2- 172.21.15.80:0/4162907809 >> v2:172.21.15.87:3300/0 conn(0x7f17a816d390 0x7f17a816f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1584380022 0 0) 0x7f17a8072120 con 0x7f17a806eb40 2024-04-25T13:41:27.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1794003530 con 0x7f17a806eb40 2024-04-25T13:41:27.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2036620208 0 0) 0x7f1794003530 con 0x7f17a806eb40 2024-04-25T13:41:27.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 >> v2:172.21.15.87:3300/0 conn(0x7f17a816d390 msgr2=0x7f17a816f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 --2- 172.21.15.80:0/4162907809 >> v2:172.21.15.87:3300/0 conn(0x7f17a816d390 0x7f17a816f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 >> v1:172.21.15.87:6789/0 conn(0x7f17a81622d0 legacy=0x7f17a80725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a806e340 con 0x7f17a806eb40 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1798001680 con 0x7f17a806eb40 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 >> v1:172.21.15.80:6789/0 conn(0x7f17a806eb40 legacy=0x7f17a806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17940039c0 0x7f1794003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1794004810 0x7f1794004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1794005360 0x7f179400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f17a806e060 con 0x7f17940039c0 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f179400c290 con 0x7f1794005360 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f179400c490 con 0x7f1794004810 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ade56640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17940039c0 0x7f1794003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ad655640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1794004810 0x7f1794004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ae657640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1794005360 0x7f179400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ade56640 1 -- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1794004810 msgr2=0x7f1794004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ade56640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1794004810 0x7f1794004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ade56640 1 -- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1794005360 msgr2=0x7f179400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ade56640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1794005360 0x7f179400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ade56640 1 -- 172.21.15.80:0/4162907809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1794003530 con 0x7f17940039c0 2024-04-25T13:41:27.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ae657640 1 -- 172.21.15.80:0/4162907809 reap_dead start 2024-04-25T13:41:27.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ad655640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1794004810 0x7f1794004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:27.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ade56640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17940039c0 0x7f1794003e30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f17a4001780 tx=0x7f17a401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd07ec5af2405684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a40013a0 con 0x7f17940039c0 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17a40052f0 con 0x7f17940039c0 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.098+0000 7f17ace54640 1 -- 172.21.15.80:0/4162907809 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4003840 con 0x7f17940039c0 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- 172.21.15.80:0/4162907809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17940039c0 msgr2=0x7f1794003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17940039c0 0x7f1794003e30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f17a4001780 tx=0x7f17a401c400 comp rx=0 tx=0).stop 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- 172.21.15.80:0/4162907809 shutdown_connections 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 --2- 172.21.15.80:0/4162907809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17940039c0 0x7f1794003e30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- 172.21.15.80:0/4162907809 >> 172.21.15.80:0/4162907809 conn(0x7f17a806d080 msgr2=0x7f17a80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- 172.21.15.80:0/4162907809 shutdown_connections 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- 172.21.15.80:0/4162907809 wait complete. 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 Processor -- start 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- start start 2024-04-25T13:41:27.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17940039c0 0x7f17a8158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a814ae70 0x7f17a81fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17a8156510 0x7f17a8156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8165040 con 0x7f17a814ae70 2024-04-25T13:41:27.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17a8164ec0 con 0x7f17940039c0 2024-04-25T13:41:27.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17b00ba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17a81651c0 con 0x7f17a8156510 2024-04-25T13:41:27.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ad655640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a814ae70 0x7f17a81fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ad655640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a814ae70 0x7f17a81fdb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48380/0 (socket says 172.21.15.80:48380) 2024-04-25T13:41:27.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ad655640 1 -- 172.21.15.80:0/2668940497 learned_addr learned my addr 172.21.15.80:0/2668940497 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:27.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ade56640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17940039c0 0x7f17a8158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ae657640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17a8156510 0x7f17a8156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ad655640 1 -- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17a8156510 msgr2=0x7f17a8156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ad655640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17a8156510 0x7f17a8156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ad655640 1 -- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17940039c0 msgr2=0x7f17a8158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ad655640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17940039c0 0x7f17a8158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ad655640 1 -- 172.21.15.80:0/2668940497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a8156e60 con 0x7f17a814ae70 2024-04-25T13:41:27.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.102+0000 7f17ade56640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17940039c0 0x7f17a8158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:27.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f17ad655640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a814ae70 0x7f17a81fdb60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f179800d7d0 tx=0x7f179800dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f179e7fc640 1 -- 172.21.15.80:0/2668940497 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798019070 con 0x7f17a814ae70 2024-04-25T13:41:27.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f179e7fc640 1 -- 172.21.15.80:0/2668940497 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17980074e0 con 0x7f17a814ae70 2024-04-25T13:41:27.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a8145270 con 0x7f17a814ae70 2024-04-25T13:41:27.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f179e7fc640 1 -- 172.21.15.80:0/2668940497 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1798004a10 con 0x7f17a814ae70 2024-04-25T13:41:27.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a8159b30 con 0x7f17a814ae70 2024-04-25T13:41:27.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a806a3a0 con 0x7f17a814ae70 2024-04-25T13:41:27.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f179e7fc640 1 -- 172.21.15.80:0/2668940497 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f179801c070 con 0x7f17a814ae70 2024-04-25T13:41:27.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f179e7fc640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f178c06f400 0x7f178c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f179e7fc640 1 -- 172.21.15.80:0/2668940497 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f179808e920 con 0x7f17a814ae70 2024-04-25T13:41:27.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f17ade56640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f178c06f400 0x7f178c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.106+0000 7f17ade56640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f178c06f400 0x7f178c071910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17a4004000 tx=0x7f17a40011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.110+0000 7f179e7fc640 1 -- 172.21.15.80:0/2668940497 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f17a8156e60 con 0x7f17a814ae70 2024-04-25T13:41:27.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.334+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f17a81595b0 con 0x7f17a814ae70 2024-04-25T13:41:27.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.338+0000 7f179e7fc640 1 -- 172.21.15.80:0/2668940497 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f179805bb60 con 0x7f17a814ae70 2024-04-25T13:41:27.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f178c06f400 msgr2=0x7f178c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f178c06f400 0x7f178c071910 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f17a4004000 tx=0x7f17a40011a0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a814ae70 msgr2=0x7f17a81fdb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a814ae70 0x7f17a81fdb60 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f179800d7d0 tx=0x7f179800dbd0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 shutdown_connections 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f178c06f400 0x7f178c071910 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17a8156510 0x7f17a8156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17a814ae70 0x7f17a81fdb60 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 --2- 172.21.15.80:0/2668940497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17940039c0 0x7f17a8158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 >> 172.21.15.80:0/2668940497 conn(0x7f17a806d080 msgr2=0x7f17a815dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 shutdown_connections 2024-04-25T13:41:27.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.342+0000 7f17b00ba640 1 -- 172.21.15.80:0/2668940497 wait complete. 2024-04-25T13:41:27.363 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T13:41:27.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05cca89640 1 Processor -- start 2024-04-25T13:41:27.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05cca89640 1 -- start start 2024-04-25T13:41:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05cca89640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f05c80d2e80 0x7f05c81607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05cca89640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f05c80d4330 0x7f05c80d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05cca89640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05c8161060 con 0x7f05c80d59b0 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05cca89640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f05c8161340 con 0x7f05c80d4330 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05cca89640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f05c80d2680 con 0x7f05c80d2e80 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05c6d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f05c80d4330 0x7f05c80d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05c6d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f05c80d4330 0x7f05c80d4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48384/0 (socket says 172.21.15.80:48384) 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05c6d9d640 1 -- 172.21.15.80:0/333329056 learned_addr learned my addr 172.21.15.80:0/333329056 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05c6d9d640 1 -- 172.21.15.80:0/333329056 >> v2:172.21.15.87:3300/0 conn(0x7f05c80d2e80 msgr2=0x7f05c81607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05c6d9d640 1 --2- 172.21.15.80:0/333329056 >> v2:172.21.15.87:3300/0 conn(0x7f05c80d2e80 0x7f05c81607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05c6d9d640 1 -- 172.21.15.80:0/333329056 >> v1:172.21.15.87:6789/0 conn(0x7f05c80d59b0 legacy=0x7f05c80d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.510+0000 7f05c6d9d640 1 -- 172.21.15.80:0/333329056 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c80d2960 con 0x7f05c80d4330 2024-04-25T13:41:27.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.514+0000 7f05c6d9d640 1 --2- 172.21.15.80:0/333329056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d4330 0x7f05c80d4740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f05c808ec00 tx=0x7f05c001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f4fbc0bfa54476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.514+0000 7f05c559a640 1 -- 172.21.15.80:0/333329056 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c00a5840 con 0x7f05c80d4330 2024-04-25T13:41:27.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.514+0000 7f05c559a640 1 -- 172.21.15.80:0/333329056 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05c00a59e0 con 0x7f05c80d4330 2024-04-25T13:41:27.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.514+0000 7f05c559a640 1 -- 172.21.15.80:0/333329056 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c00a5db0 con 0x7f05c80d4330 2024-04-25T13:41:27.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.514+0000 7f05cca89640 1 -- 172.21.15.80:0/333329056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d4330 msgr2=0x7f05c80d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.514+0000 7f05cca89640 1 --2- 172.21.15.80:0/333329056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d4330 0x7f05c80d4740 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f05c808ec00 tx=0x7f05c001c960 comp rx=0 tx=0).stop 2024-04-25T13:41:27.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 -- 172.21.15.80:0/333329056 shutdown_connections 2024-04-25T13:41:27.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 --2- 172.21.15.80:0/333329056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d4330 0x7f05c80d4740 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 --2- 172.21.15.80:0/333329056 >> v2:172.21.15.87:3300/0 conn(0x7f05c80d2e80 0x7f05c81607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 -- 172.21.15.80:0/333329056 >> 172.21.15.80:0/333329056 conn(0x7f05c80737a0 msgr2=0x7f05c806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:27.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 -- 172.21.15.80:0/333329056 shutdown_connections 2024-04-25T13:41:27.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 -- 172.21.15.80:0/333329056 wait complete. 2024-04-25T13:41:27.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 Processor -- start 2024-04-25T13:41:27.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 -- start start 2024-04-25T13:41:27.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d2e80 0x7f05c815c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05c80d4330 0x7f05c81580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05c80d59b0 0x7f05c81f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f05c80d3410 con 0x7f05c80d2e80 2024-04-25T13:41:27.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05c80d3290 con 0x7f05c80d4330 2024-04-25T13:41:27.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05cca89640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f05c815df60 con 0x7f05c80d59b0 2024-04-25T13:41:27.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05c659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d2e80 0x7f05c815c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05c659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d2e80 0x7f05c815c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48400/0 (socket says 172.21.15.80:48400) 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05c659c640 1 -- 172.21.15.80:0/477201668 learned_addr learned my addr 172.21.15.80:0/477201668 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.518+0000 7f05c5d9b640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05c80d4330 0x7f05c81580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05c6d9d640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05c80d59b0 0x7f05c81f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05c659c640 1 -- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05c80d59b0 msgr2=0x7f05c81f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05c659c640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05c80d59b0 0x7f05c81f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05c659c640 1 -- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05c80d4330 msgr2=0x7f05c81580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05c659c640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05c80d4330 0x7f05c81580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05c659c640 1 -- 172.21.15.80:0/477201668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c815cb60 con 0x7f05c80d2e80 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05c659c640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d2e80 0x7f05c815c620 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b3f0 tx=0x7f05bc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05b6ffd640 1 -- 172.21.15.80:0/477201668 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc015020 con 0x7f05c80d2e80 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05b6ffd640 1 -- 172.21.15.80:0/477201668 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f05bc011850 con 0x7f05c80d2e80 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c81650f0 con 0x7f05c80d2e80 2024-04-25T13:41:27.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05b6ffd640 1 -- 172.21.15.80:0/477201668 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05bc011c40 con 0x7f05c80d2e80 2024-04-25T13:41:27.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c81656b0 con 0x7f05c80d2e80 2024-04-25T13:41:27.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c8165d70 con 0x7f05c80d2e80 2024-04-25T13:41:27.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05b6ffd640 1 -- 172.21.15.80:0/477201668 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f05bc018020 con 0x7f05c80d2e80 2024-04-25T13:41:27.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05b6ffd640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f05a806f400 0x7f05a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05b6ffd640 1 -- 172.21.15.80:0/477201668 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f05bc08ee40 con 0x7f05c80d2e80 2024-04-25T13:41:27.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.522+0000 7f05c5d9b640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f05a806f400 0x7f05a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.526+0000 7f05c5d9b640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f05a806f400 0x7f05a8071910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f05c81fcd40 tx=0x7f05b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.526+0000 7f05b6ffd640 1 -- 172.21.15.80:0/477201668 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f05bc01f020 con 0x7f05c80d2e80 2024-04-25T13:41:27.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.746+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f05c806a3a0 con 0x7f05c80d2e80 2024-04-25T13:41:27.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05b6ffd640 1 -- 172.21.15.80:0/477201668 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f05bc05c080 con 0x7f05c80d2e80 2024-04-25T13:41:27.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f05a806f400 msgr2=0x7f05a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f05a806f400 0x7f05a8071910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f05c81fcd40 tx=0x7f05b8009040 comp rx=0 tx=0).stop 2024-04-25T13:41:27.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d2e80 msgr2=0x7f05c815c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d2e80 0x7f05c815c620 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f05bc00b3f0 tx=0x7f05bc00b8b0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 shutdown_connections 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f05a806f400 0x7f05a8071910 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05c80d59b0 0x7f05c81f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05c80d4330 0x7f05c81580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.754+0000 7f05cca89640 1 --2- 172.21.15.80:0/477201668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05c80d2e80 0x7f05c815c620 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.758+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 >> 172.21.15.80:0/477201668 conn(0x7f05c80737a0 msgr2=0x7f05c8162db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.758+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 shutdown_connections 2024-04-25T13:41:27.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.758+0000 7f05cca89640 1 -- 172.21.15.80:0/477201668 wait complete. 2024-04-25T13:41:27.778 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T13:41:27.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97f360640 1 Processor -- start 2024-04-25T13:41:27.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97f360640 1 -- start start 2024-04-25T13:41:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97f360640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb978161f90 0x7fb9780cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97f360640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb9780d0510 0x7fb97815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97f360640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9780d0d70 con 0x7fb97815ea70 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97f360640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb9780d1050 con 0x7fb9780d0510 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97f360640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb97806dca0 con 0x7fb978161f90 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97d8fd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb97815ea70 0x7fb97815ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43390/0 (socket says 172.21.15.80:43390) 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97d8fd640 1 -- 172.21.15.80:0/1759537468 learned_addr learned my addr 172.21.15.80:0/1759537468 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97c8fb640 1 --2- 172.21.15.80:0/1759537468 >> v2:172.21.15.87:3300/0 conn(0x7fb9780d0510 0x7fb97815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb97d0fc640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb978161f90 0x7fb9780cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.926+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531093225 0 0) 0x7fb9780d0d70 con 0x7fb97815ea70 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb964003530 con 0x7fb97815ea70 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2758905324 0 0) 0x7fb964003530 con 0x7fb97815ea70 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb978161f90 msgr2=0x7fb9780cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb978161f90 0x7fb9780cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 >> v2:172.21.15.87:3300/0 conn(0x7fb9780d0510 msgr2=0x7fb97815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 --2- 172.21.15.80:0/1759537468 >> v2:172.21.15.87:3300/0 conn(0x7fb9780d0510 0x7fb97815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97806df80 con 0x7fb97815ea70 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb970001a00 con 0x7fb97815ea70 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 >> v1:172.21.15.80:6789/0 conn(0x7fb97815ea70 legacy=0x7fb97815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb978161f90 0x7fb9640039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964003f40 0x7fb964004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964004d50 0x7fb964005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb9780d1050 con 0x7fb964003f40 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb97806dca0 con 0x7fb964004d50 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb96400bad0 con 0x7fb978161f90 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97c8fb640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964003f40 0x7fb964004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97d0fc640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb978161f90 0x7fb9640039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97d8fd640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964004d50 0x7fb964005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97c8fb640 1 -- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb978161f90 msgr2=0x7fb9640039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97c8fb640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb978161f90 0x7fb9640039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97c8fb640 1 -- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964004d50 msgr2=0x7fb964005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97c8fb640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964004d50 0x7fb964005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97c8fb640 1 -- 172.21.15.80:0/1759537468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb964003530 con 0x7fb964003f40 2024-04-25T13:41:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97d8fd640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb964004d50 0x7fb964005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97c8fb640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964003f40 0x7fb964004810 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb978074290 tx=0x7fb96801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18da7140f74ef38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680096b0 con 0x7fb964003f40 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9680a6480 con 0x7fb964003f40 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb96ffff640 1 -- 172.21.15.80:0/1759537468 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680a6870 con 0x7fb964003f40 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97f360640 1 -- 172.21.15.80:0/1759537468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964003f40 msgr2=0x7fb964004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97f360640 1 --2- 172.21.15.80:0/1759537468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb964003f40 0x7fb964004810 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb978074290 tx=0x7fb96801c990 comp rx=0 tx=0).stop 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97d8fd640 1 -- 172.21.15.80:0/1759537468 reap_dead start 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97f360640 1 -- 172.21.15.80:0/1759537468 shutdown_connections 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.930+0000 7fb97f360640 1 -- 172.21.15.80:0/1759537468 >> 172.21.15.80:0/1759537468 conn(0x7fb97806d080 msgr2=0x7fb9780b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 -- 172.21.15.80:0/1759537468 shutdown_connections 2024-04-25T13:41:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 -- 172.21.15.80:0/1759537468 wait complete. 2024-04-25T13:41:27.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 Processor -- start 2024-04-25T13:41:27.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 -- start start 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb964003f40 0x7fb9780b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb97815d230 0x7fb9780b2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb978164cd0 0x7fb9781650e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb97806f4b0 con 0x7fb97815d230 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb97806f630 con 0x7fb978164cd0 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97f360640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb9780d0920 con 0x7fb964003f40 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb97815d230 0x7fb9780b2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb97815d230 0x7fb9780b2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48430/0 (socket says 172.21.15.80:48430) 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 -- 172.21.15.80:0/2337091710 learned_addr learned my addr 172.21.15.80:0/2337091710 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97d8fd640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb978164cd0 0x7fb9781650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97d0fc640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb964003f40 0x7fb9780b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 -- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb964003f40 msgr2=0x7fb9780b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb964003f40 0x7fb9780b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 -- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb978164cd0 msgr2=0x7fb9781650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb978164cd0 0x7fb9781650e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 -- 172.21.15.80:0/2337091710 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb97815f6e0 con 0x7fb97815d230 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.934+0000 7fb97c8fb640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb97815d230 0x7fb9780b2640 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb96801c7c0 tx=0x7fb964005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb96dffb640 1 -- 172.21.15.80:0/2337091710 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680a6c30 con 0x7fb97815d230 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb96dffb640 1 -- 172.21.15.80:0/2337091710 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9680a6dd0 con 0x7fb97815d230 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb97815fbe0 con 0x7fb97815d230 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb96dffb640 1 -- 172.21.15.80:0/2337091710 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9680a76a0 con 0x7fb97815d230 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9781601a0 con 0x7fb97815d230 2024-04-25T13:41:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb97815f950 con 0x7fb97815d230 2024-04-25T13:41:27.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb96dffb640 1 -- 172.21.15.80:0/2337091710 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb9680a7840 con 0x7fb97815d230 2024-04-25T13:41:27.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb96dffb640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb95c06f400 0x7fb95c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:27.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb96dffb640 1 -- 172.21.15.80:0/2337091710 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fb968129530 con 0x7fb97815d230 2024-04-25T13:41:27.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb97d0fc640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb95c06f400 0x7fb95c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:27.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.938+0000 7fb97d0fc640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb95c06f400 0x7fb95c071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb974009160 tx=0x7fb974008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:27.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:27.942+0000 7fb96dffb640 1 -- 172.21.15.80:0/2337091710 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb9680ac020 con 0x7fb97815d230 2024-04-25T13:41:28.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.146+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fb97806a3a0 con 0x7fb97815d230 2024-04-25T13:41:28.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.150+0000 7fb96dffb640 1 -- 172.21.15.80:0/2337091710 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7fb968016700 con 0x7fb97815d230 2024-04-25T13:41:28.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb95c06f400 msgr2=0x7fb95c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb95c06f400 0x7fb95c071910 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb974009160 tx=0x7fb974008040 comp rx=0 tx=0).stop 2024-04-25T13:41:28.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb97815d230 msgr2=0x7fb9780b2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb97815d230 0x7fb9780b2640 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb96801c7c0 tx=0x7fb964005840 comp rx=0 tx=0).stop 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 shutdown_connections 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb95c06f400 0x7fb95c071910 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb978164cd0 0x7fb9781650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb97815d230 0x7fb9780b2640 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 --2- 172.21.15.80:0/2337091710 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb964003f40 0x7fb9780b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 >> 172.21.15.80:0/2337091710 conn(0x7fb97806d080 msgr2=0x7fb978141520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 shutdown_connections 2024-04-25T13:41:28.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.154+0000 7fb97f360640 1 -- 172.21.15.80:0/2337091710 wait complete. 2024-04-25T13:41:28.175 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T13:41:28.310 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:41:28.312 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:41:28.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f19471f9640 1 Processor -- start 2024-04-25T13:41:28.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f19471f9640 1 -- start start 2024-04-25T13:41:28.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f19471f9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1940161240 0x7f19400723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f19471f9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 0x7f19400728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f19471f9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f194006fcf0 con 0x7f194006f5a0 2024-04-25T13:41:28.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f19471f9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1940073f10 con 0x7f1940161240 2024-04-25T13:41:28.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f19471f9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f19400b0c60 con 0x7f1940161650 2024-04-25T13:41:28.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f1944f95640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f194006f5a0 0x7f1940160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43406/0 (socket says 172.21.15.80:43406) 2024-04-25T13:41:28.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f1944f95640 1 -- 172.21.15.80:0/1300841462 learned_addr learned my addr 172.21.15.80:0/1300841462 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:28.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 296394577 0 0) 0x7f194006fcf0 con 0x7f194006f5a0 2024-04-25T13:41:28.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f1945796640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 0x7f19400728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193ffff640 1 --2- 172.21.15.80:0/1300841462 >> v2:172.21.15.87:3300/0 conn(0x7f1940161240 0x7f19400723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1930005740 con 0x7f194006f5a0 2024-04-25T13:41:28.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2728480030 0 0) 0x7f1930005740 con 0x7f194006f5a0 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 msgr2=0x7f19400728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 0x7f19400728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 >> v2:172.21.15.87:3300/0 conn(0x7f1940161240 msgr2=0x7f19400723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 --2- 172.21.15.80:0/1300841462 >> v2:172.21.15.87:3300/0 conn(0x7f1940161240 0x7f19400723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19400b0f40 con 0x7f194006f5a0 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1934001680 con 0x7f194006f5a0 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f1945796640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 0x7f19400728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 >> v1:172.21.15.80:6789/0 conn(0x7f194006f5a0 legacy=0x7f1940160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930005bd0 0x7f1930006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1930006a20 0x7f1930006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 0x7f19300074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1940073f10 con 0x7f1930006a20 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f19400b0c60 con 0x7f1930005bd0 2024-04-25T13:41:28.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f193000dda0 con 0x7f1940161650 2024-04-25T13:41:28.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.310+0000 7f193ffff640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1930006a20 0x7f1930006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f1945796640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 0x7f19300074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f1944f95640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930005bd0 0x7f1930006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193ffff640 1 -- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 msgr2=0x7f19300074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:28.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193ffff640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1940161650 0x7f19300074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193ffff640 1 -- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930005bd0 msgr2=0x7f1930006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:28.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193ffff640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930005bd0 0x7f1930006040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193ffff640 1 -- 172.21.15.80:0/1300841462 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1930005740 con 0x7f1930006a20 2024-04-25T13:41:28.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193ffff640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1930006a20 0x7f1930006e30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f192c00ba30 tx=0x7f192c01c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98889dde0072f43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:28.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c00b600 con 0x7f1930006a20 2024-04-25T13:41:28.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f192c0b1070 con 0x7f1930006a20 2024-04-25T13:41:28.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f193f7fe640 1 -- 172.21.15.80:0/1300841462 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f192c0ac830 con 0x7f1930006a20 2024-04-25T13:41:28.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f19471f9640 1 -- 172.21.15.80:0/1300841462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1930006a20 msgr2=0x7f1930006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f19471f9640 1 --2- 172.21.15.80:0/1300841462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1930006a20 0x7f1930006e30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f192c00ba30 tx=0x7f192c01c6e0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f1945796640 1 -- 172.21.15.80:0/1300841462 reap_dead start 2024-04-25T13:41:28.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f19471f9640 1 -- 172.21.15.80:0/1300841462 shutdown_connections 2024-04-25T13:41:28.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f19471f9640 1 -- 172.21.15.80:0/1300841462 >> 172.21.15.80:0/1300841462 conn(0x7f194006d080 msgr2=0x7f19400b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:28.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f19471f9640 1 -- 172.21.15.80:0/1300841462 shutdown_connections 2024-04-25T13:41:28.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.314+0000 7f19471f9640 1 -- 172.21.15.80:0/1300841462 wait complete. 2024-04-25T13:41:28.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 Processor -- start 2024-04-25T13:41:28.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 -- start start 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930006a20 0x7f1940158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f194015fff0 0x7f194014d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1940159b50 0x7f19400af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1940073480 con 0x7f1940159b50 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1940073300 con 0x7f1930006a20 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1940073600 con 0x7f194015fff0 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1940159b50 0x7f19400af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1940159b50 0x7f19400af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48454/0 (socket says 172.21.15.80:48454) 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 -- 172.21.15.80:0/3093865868 learned_addr learned my addr 172.21.15.80:0/3093865868 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1944f95640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930006a20 0x7f1940158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f193ffff640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f194015fff0 0x7f194014d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 -- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f194015fff0 msgr2=0x7f194014d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f194015fff0 0x7f194014d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 -- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930006a20 msgr2=0x7f1940158260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930006a20 0x7f1940158260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 -- 172.21.15.80:0/3093865868 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1940158ab0 con 0x7f1940159b50 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1944f95640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930006a20 0x7f1940158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f1945796640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1940159b50 0x7f19400af8b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1940070290 tx=0x7f1938004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f193d7fa640 1 -- 172.21.15.80:0/3093865868 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938018de0 con 0x7f1940159b50 2024-04-25T13:41:28.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f193d7fa640 1 -- 172.21.15.80:0/3093865868 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f193801c070 con 0x7f1940159b50 2024-04-25T13:41:28.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1940158d90 con 0x7f1940159b50 2024-04-25T13:41:28.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f193d7fa640 1 -- 172.21.15.80:0/3093865868 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938007ac0 con 0x7f1940159b50 2024-04-25T13:41:28.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.318+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1940159350 con 0x7f1940159b50 2024-04-25T13:41:28.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.322+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f194006a3a0 con 0x7f1940159b50 2024-04-25T13:41:28.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.322+0000 7f193d7fa640 1 -- 172.21.15.80:0/3093865868 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1938007c60 con 0x7f1940159b50 2024-04-25T13:41:28.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.322+0000 7f193d7fa640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f192406f430 0x7f1924071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.322+0000 7f193d7fa640 1 -- 172.21.15.80:0/3093865868 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f1938097810 con 0x7f1940159b50 2024-04-25T13:41:28.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.322+0000 7f1944f95640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f192406f430 0x7f1924071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.322+0000 7f1944f95640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f192406f430 0x7f1924071940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19400b6e70 tx=0x7f1934009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:28.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.326+0000 7f193d7fa640 1 -- 172.21.15.80:0/3093865868 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1938064ae0 con 0x7f1940159b50 2024-04-25T13:41:28.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.534+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f19400739c0 con 0x7f1940159b50 2024-04-25T13:41:28.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.538+0000 7f193d7fa640 1 -- 172.21.15.80:0/3093865868 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f19380643e0 con 0x7f1940159b50 2024-04-25T13:41:28.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f192406f430 msgr2=0x7f1924071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f192406f430 0x7f1924071940 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f19400b6e70 tx=0x7f1934009040 comp rx=0 tx=0).stop 2024-04-25T13:41:28.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1940159b50 msgr2=0x7f19400af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1940159b50 0x7f19400af8b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f1940070290 tx=0x7f1938004700 comp rx=0 tx=0).stop 2024-04-25T13:41:28.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 shutdown_connections 2024-04-25T13:41:28.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f192406f430 0x7f1924071940 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1940159b50 0x7f19400af8b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f194015fff0 0x7f194014d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 --2- 172.21.15.80:0/3093865868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1930006a20 0x7f1940158260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 >> 172.21.15.80:0/3093865868 conn(0x7f194006d080 msgr2=0x7f19400b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:28.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 shutdown_connections 2024-04-25T13:41:28.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.542+0000 7f19471f9640 1 -- 172.21.15.80:0/3093865868 wait complete. 2024-04-25T13:41:28.562 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T13:41:28.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0e02a97640 1 Processor -- start 2024-04-25T13:41:28.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0e02a97640 1 -- start start 2024-04-25T13:41:28.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0e02a97640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc068590 0x7f0dfc0689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0e02a97640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dfc0d0970 con 0x7f0dfc160a60 2024-04-25T13:41:28.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0e02a97640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0dfc0b2180 con 0x7f0dfc070990 2024-04-25T13:41:28.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0e02a97640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc068ee0 con 0x7f0dfc068590 2024-04-25T13:41:28.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0e01034640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0dfc160a60 0x7f0dfc160e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43414/0 (socket says 172.21.15.80:43414) 2024-04-25T13:41:28.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0e01034640 1 -- 172.21.15.80:0/3838971047 learned_addr learned my addr 172.21.15.80:0/3838971047 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0dfbfff640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc068590 0x7f0dfc0689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1354000696 0 0) 0x7f0dfc0d0970 con 0x7f0dfc160a60 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.710+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0de8003530 con 0x7f0dfc160a60 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1902599620 0 0) 0x7f0de8003530 con 0x7f0dfc160a60 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc068590 msgr2=0x7f0dfc0689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc068590 0x7f0dfc0689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 >> v1:172.21.15.87:6789/0 conn(0x7f0dfc070990 legacy=0x7f0dfc06f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc0d0ec0 con 0x7f0dfc160a60 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0de4001680 con 0x7f0dfc160a60 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 >> v1:172.21.15.80:6789/0 conn(0x7f0dfc160a60 legacy=0x7f0dfc160e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc068590 0x7f0de80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0de8003f50 0x7f0de8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8004d50 0x7f0de8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc068ee0 con 0x7f0de8003f50 2024-04-25T13:41:28.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0de800bb90 con 0x7f0de8004d50 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0de800bd90 con 0x7f0dfc068590 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfbfff640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0de8003f50 0x7f0de8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e01034640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8004d50 0x7f0de8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e00833640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc068590 0x7f0de80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfbfff640 1 -- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc068590 msgr2=0x7f0de80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfbfff640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc068590 0x7f0de80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfbfff640 1 -- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8004d50 msgr2=0x7f0de8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfbfff640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8004d50 0x7f0de8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfbfff640 1 -- 172.21.15.80:0/3838971047 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de8003530 con 0x7f0de8003f50 2024-04-25T13:41:28.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfbfff640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0de8003f50 0x7f0de8004810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009ae0 tx=0x7f0dec01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6075e7443b106c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0b2070 con 0x7f0de8003f50 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec0a5680 con 0x7f0de8003f50 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0dfb7fe640 1 -- 172.21.15.80:0/3838971047 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0096b0 con 0x7f0de8003f50 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e02a97640 1 -- 172.21.15.80:0/3838971047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0de8003f50 msgr2=0x7f0de8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e02a97640 1 --2- 172.21.15.80:0/3838971047 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0de8003f50 0x7f0de8004810 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009ae0 tx=0x7f0dec01ca90 comp rx=0 tx=0).stop 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e01034640 1 -- 172.21.15.80:0/3838971047 reap_dead start 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e02a97640 1 -- 172.21.15.80:0/3838971047 shutdown_connections 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e02a97640 1 -- 172.21.15.80:0/3838971047 >> 172.21.15.80:0/3838971047 conn(0x7f0dfc0cf450 msgr2=0x7f0dfc0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e02a97640 1 -- 172.21.15.80:0/3838971047 shutdown_connections 2024-04-25T13:41:28.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.714+0000 7f0e02a97640 1 -- 172.21.15.80:0/3838971047 wait complete. 2024-04-25T13:41:28.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.718+0000 7f0e02a97640 1 Processor -- start 2024-04-25T13:41:28.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.718+0000 7f0e02a97640 1 -- start start 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.718+0000 7f0e02a97640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8003f50 0x7f0dfc15c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.718+0000 7f0e02a97640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc1f71f0 0x7f0dfc1eb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.718+0000 7f0e02a97640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc15b060 0x7f0dfc1ebe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.718+0000 7f0e02a97640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc164e30 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.718+0000 7f0e02a97640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc070da0 con 0x7f0de8003f50 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.718+0000 7f0e02a97640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc06ff20 con 0x7f0dfc15b060 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc1f71f0 0x7f0dfc1eb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc1f71f0 0x7f0dfc1eb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48472/0 (socket says 172.21.15.80:48472) 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 -- 172.21.15.80:0/1666777796 learned_addr learned my addr 172.21.15.80:0/1666777796 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0e00833640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8003f50 0x7f0dfc15c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0e01034640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc15b060 0x7f0dfc1ebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 -- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc15b060 msgr2=0x7f0dfc1ebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc15b060 0x7f0dfc1ebe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 -- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8003f50 msgr2=0x7f0dfc15c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8003f50 0x7f0dfc15c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 -- 172.21.15.80:0/1666777796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc15dc90 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0e00833640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8003f50 0x7f0dfc15c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0dfbfff640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc1f71f0 0x7f0dfc1eb190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009ae0 tx=0x7f0de8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0df97fa640 1 -- 172.21.15.80:0/1666777796 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0b2040 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0df97fa640 1 -- 172.21.15.80:0/1666777796 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0dec00f000 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dfc1eb6d0 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0df97fa640 1 -- 172.21.15.80:0/1666777796 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec004260 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dfc1ea3d0 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dfc15df00 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0df97fa640 1 -- 172.21.15.80:0/1666777796 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0dec0c0020 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0df97fa640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ddc06f430 0x7f0ddc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:28.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0df97fa640 1 -- 172.21.15.80:0/1666777796 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f0dec1292d0 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.722+0000 7f0e00833640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ddc06f430 0x7f0ddc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:28.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.726+0000 7f0e00833640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ddc06f430 0x7f0ddc071940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0df40012f0 tx=0x7f0df4001260 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:28.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.726+0000 7f0df97fa640 1 -- 172.21.15.80:0/1666777796 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0dec0167a0 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.934+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f0dfc06a3a0 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.938+0000 7f0df97fa640 1 -- 172.21.15.80:0/1666777796 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) ==== 97+0+0 (secure 0 0 0) 0x7f0dec0f6510 con 0x7f0dfc1f71f0 2024-04-25T13:41:28.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.938+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ddc06f430 msgr2=0x7f0ddc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.938+0000 7f0e02a97640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ddc06f430 0x7f0ddc071940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0df40012f0 tx=0x7f0df4001260 comp rx=0 tx=0).stop 2024-04-25T13:41:28.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc1f71f0 msgr2=0x7f0dfc1eb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc1f71f0 0x7f0dfc1eb190 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009ae0 tx=0x7f0de8005840 comp rx=0 tx=0).stop 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 shutdown_connections 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ddc06f430 0x7f0ddc071940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0dfc15b060 0x7f0dfc1ebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0dfc1f71f0 0x7f0dfc1eb190 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 --2- 172.21.15.80:0/1666777796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0de8003f50 0x7f0dfc15c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 >> 172.21.15.80:0/1666777796 conn(0x7f0dfc0cf450 msgr2=0x7f0dfc0b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 shutdown_connections 2024-04-25T13:41:28.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:28.942+0000 7f0e02a97640 1 -- 172.21.15.80:0/1666777796 wait complete. 2024-04-25T13:41:28.961 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T13:41:29.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f44eb5640 1 Processor -- start 2024-04-25T13:41:29.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f44eb5640 1 -- start start 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f44eb5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f4015bcc0 0x7f5f400b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f44eb5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5f40161d20 0x7f5f40162130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f44eb5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f4006c9b0 con 0x7f5f400b7d10 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f44eb5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5f400b2210 con 0x7f5f40161d20 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f44eb5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f400b2440 con 0x7f5f4015bcc0 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5f40161d20 0x7f5f40162130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5f40161d20 0x7f5f40162130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48486/0 (socket says 172.21.15.80:48486) 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/1819054488 learned_addr learned my addr 172.21.15.80:0/1819054488 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:29.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3e59c640 1 --2- 172.21.15.80:0/1819054488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f4015bcc0 0x7f5f400b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/1819054488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f4015bcc0 msgr2=0x7f5f400b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:29.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/1819054488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f4015bcc0 0x7f5f400b1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/1819054488 >> v1:172.21.15.87:6789/0 conn(0x7f5f400b7d10 legacy=0x7f5f400b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:29.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/1819054488 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f4006f890 con 0x7f5f40161d20 2024-04-25T13:41:29.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.106+0000 7f5f3e59c640 1 --2- 172.21.15.80:0/1819054488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f4015bcc0 0x7f5f400b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:29.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/1819054488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f40162130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009870 tx=0x7f5f3801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c5cdb46b74a604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f3d59a640 1 -- 172.21.15.80:0/1819054488 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b0070 con 0x7f5f40161d20 2024-04-25T13:41:29.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f3d59a640 1 -- 172.21.15.80:0/1819054488 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f380ab600 con 0x7f5f40161d20 2024-04-25T13:41:29.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f3d59a640 1 -- 172.21.15.80:0/1819054488 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380ab9d0 con 0x7f5f40161d20 2024-04-25T13:41:29.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f44eb5640 1 -- 172.21.15.80:0/1819054488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 msgr2=0x7f5f40162130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/1819054488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f40162130 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009870 tx=0x7f5f3801c340 comp rx=0 tx=0).stop 2024-04-25T13:41:29.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f44eb5640 1 -- 172.21.15.80:0/1819054488 shutdown_connections 2024-04-25T13:41:29.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/1819054488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f40162130 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/1819054488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f4015bcc0 0x7f5f400b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f44eb5640 1 -- 172.21.15.80:0/1819054488 >> 172.21.15.80:0/1819054488 conn(0x7f5f400cf450 msgr2=0x7f5f400cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:29.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f44eb5640 1 -- 172.21.15.80:0/1819054488 shutdown_connections 2024-04-25T13:41:29.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.110+0000 7f5f44eb5640 1 -- 172.21.15.80:0/1819054488 wait complete. 2024-04-25T13:41:29.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 Processor -- start 2024-04-25T13:41:29.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 -- start start 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400b7d10 0x7f5f400b76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4015bcc0 0x7f5f40158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f4014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5f40162a80 con 0x7f5f40161d20 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5f4015c0d0 con 0x7f5f4015bcc0 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5f40162c00 con 0x7f5f400b7d10 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f4014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f4014fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48494/0 (socket says 172.21.15.80:48494) 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/2065727384 learned_addr learned my addr 172.21.15.80:0/2065727384 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400b7d10 msgr2=0x7f5f400b76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400b7d10 0x7f5f400b76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3dd9b640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4015bcc0 0x7f5f40158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4015bcc0 msgr2=0x7f5f40158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4015bcc0 0x7f5f40158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 -- 172.21.15.80:0/2065727384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f401593a0 con 0x7f5f40161d20 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3dd9b640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4015bcc0 0x7f5f40158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f3ed9d640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f4014fbe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5f3801a9f0 tx=0x7f5f380190e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f2effd640 1 -- 172.21.15.80:0/2065727384 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b9070 con 0x7f5f40161d20 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f2effd640 1 -- 172.21.15.80:0/2065727384 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5f380b4610 con 0x7f5f40161d20 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f400b6610 con 0x7f5f40161d20 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f2effd640 1 -- 172.21.15.80:0/2065727384 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f380b0040 con 0x7f5f40161d20 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f400b6bd0 con 0x7f5f40161d20 2024-04-25T13:41:29.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.114+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f4006a3a0 con 0x7f5f40161d20 2024-04-25T13:41:29.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.118+0000 7f5f2effd640 1 -- 172.21.15.80:0/2065727384 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5f380bc070 con 0x7f5f40161d20 2024-04-25T13:41:29.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.118+0000 7f5f2effd640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f430 0x7f5f20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.118+0000 7f5f2effd640 1 -- 172.21.15.80:0/2065727384 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f5f38127f80 con 0x7f5f40161d20 2024-04-25T13:41:29.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.118+0000 7f5f3e59c640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f430 0x7f5f20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.118+0000 7f5f3e59c640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f430 0x7f5f20071940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000a890 tx=0x7f5f30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.122+0000 7f5f2effd640 1 -- 172.21.15.80:0/2065727384 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5f4006f890 con 0x7f5f40161d20 2024-04-25T13:41:29.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.326+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f5f40161ab0 con 0x7f5f40161d20 2024-04-25T13:41:29.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f2effd640 1 -- 172.21.15.80:0/2065727384 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f5f4006c9b0 con 0x7f5f40161d20 2024-04-25T13:41:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f430 msgr2=0x7f5f20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f430 0x7f5f20071940 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000a890 tx=0x7f5f30008040 comp rx=0 tx=0).stop 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 msgr2=0x7f5f4014fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f4014fbe0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5f3801a9f0 tx=0x7f5f380190e0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 shutdown_connections 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5f2006f430 0x7f5f20071940 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5f40161d20 0x7f5f4014fbe0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f4015bcc0 0x7f5f40158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 --2- 172.21.15.80:0/2065727384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5f400b7d10 0x7f5f400b76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 >> 172.21.15.80:0/2065727384 conn(0x7f5f400cf450 msgr2=0x7f5f400b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 shutdown_connections 2024-04-25T13:41:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.334+0000 7f5f44eb5640 1 -- 172.21.15.80:0/2065727384 wait complete. 2024-04-25T13:41:29.357 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:29.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc290a9640 1 Processor -- start 2024-04-25T13:41:29.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc290a9640 1 -- start start 2024-04-25T13:41:29.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc290a9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdc24161650 0x7fdc24072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc290a9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc2406fcf0 con 0x7fdc24161240 2024-04-25T13:41:29.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc290a9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc240b0c60 con 0x7fdc2406f5a0 2024-04-25T13:41:29.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc290a9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdc24073f10 con 0x7fdc24161650 2024-04-25T13:41:29.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc2259c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fdc24161240 0x7fdc240723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43424/0 (socket says 172.21.15.80:43424) 2024-04-25T13:41:29.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc2259c640 1 -- 172.21.15.80:0/3296828130 learned_addr learned my addr 172.21.15.80:0/3296828130 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:29.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc2359e640 1 --2- 172.21.15.80:0/3296828130 >> v2:172.21.15.87:3300/0 conn(0x7fdc24161650 0x7fdc24072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3554646952 0 0) 0x7fdc2406fcf0 con 0x7fdc24161240 2024-04-25T13:41:29.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdc0c003530 con 0x7fdc24161240 2024-04-25T13:41:29.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2162791172 0 0) 0x7fdc0c003530 con 0x7fdc24161240 2024-04-25T13:41:29.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 >> v2:172.21.15.87:3300/0 conn(0x7fdc24161650 msgr2=0x7fdc24072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:29.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 --2- 172.21.15.80:0/3296828130 >> v2:172.21.15.87:3300/0 conn(0x7fdc24161650 0x7fdc24072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 >> v1:172.21.15.87:6789/0 conn(0x7fdc2406f5a0 legacy=0x7fdc24160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc240b0f40 con 0x7fdc24161240 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fdc18001680 con 0x7fdc24161240 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 >> v1:172.21.15.80:6789/0 conn(0x7fdc24161240 legacy=0x7fdc240723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c0039c0 0x7fdc0c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc0c004810 0x7fdc0c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c005310 0x7fdc0c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24073f10 con 0x7fdc0c0039c0 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0c00c2b0 con 0x7fdc0c004810 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdc0c00c4b0 con 0x7fdc0c005310 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc22d9d640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c0039c0 0x7fdc0c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc2259c640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc0c004810 0x7fdc0c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc2359e640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c005310 0x7fdc0c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc22d9d640 1 -- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c005310 msgr2=0x7fdc0c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:29.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc22d9d640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c005310 0x7fdc0c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc22d9d640 1 -- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc0c004810 msgr2=0x7fdc0c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:29.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc22d9d640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc0c004810 0x7fdc0c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc22d9d640 1 -- 172.21.15.80:0/3296828130 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc0c003530 con 0x7fdc0c0039c0 2024-04-25T13:41:29.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc2359e640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c005310 0x7fdc0c00b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:29.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.502+0000 7fdc2359e640 1 -- 172.21.15.80:0/3296828130 reap_dead start 2024-04-25T13:41:29.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc22d9d640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c0039c0 0x7fdc0c003e30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001750 tx=0x7fdc1401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547d57738ffcbf26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc140013a0 con 0x7fdc0c0039c0 2024-04-25T13:41:29.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc140052f0 con 0x7fdc0c0039c0 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc21d9b640 1 -- 172.21.15.80:0/3296828130 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14003840 con 0x7fdc0c0039c0 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc290a9640 1 -- 172.21.15.80:0/3296828130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c0039c0 msgr2=0x7fdc0c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc290a9640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c0039c0 0x7fdc0c003e30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdc14001750 tx=0x7fdc1401c3c0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc290a9640 1 -- 172.21.15.80:0/3296828130 shutdown_connections 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc290a9640 1 --2- 172.21.15.80:0/3296828130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc0c0039c0 0x7fdc0c003e30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc290a9640 1 -- 172.21.15.80:0/3296828130 >> 172.21.15.80:0/3296828130 conn(0x7fdc2406d080 msgr2=0x7fdc240b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc290a9640 1 -- 172.21.15.80:0/3296828130 shutdown_connections 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.506+0000 7fdc290a9640 1 -- 172.21.15.80:0/3296828130 wait complete. 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 Processor -- start 2024-04-25T13:41:29.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 -- start start 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c0039c0 0x7fdc24142a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc241f61a0 0x7fdc24144ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc24145000 0x7fdc241592a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdc240b5560 con 0x7fdc24145000 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdc240b53e0 con 0x7fdc241f61a0 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdc240b56e0 con 0x7fdc0c0039c0 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc24145000 0x7fdc241592a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc24145000 0x7fdc241592a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48508/0 (socket says 172.21.15.80:48508) 2024-04-25T13:41:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 -- 172.21.15.80:0/4035311290 learned_addr learned my addr 172.21.15.80:0/4035311290 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2259c640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc241f61a0 0x7fdc24144ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc22d9d640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c0039c0 0x7fdc24142a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 -- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c0039c0 msgr2=0x7fdc24142a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c0039c0 0x7fdc24142a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 -- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc241f61a0 msgr2=0x7fdc24144ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc241f61a0 0x7fdc24144ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 -- 172.21.15.80:0/4035311290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc24145a20 con 0x7fdc24145000 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc22d9d640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c0039c0 0x7fdc24142a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc2359e640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc24145000 0x7fdc241592a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0057f0 tx=0x7fdc1c009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/4035311290 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c019070 con 0x7fdc24145000 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/4035311290 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc1c007cb0 con 0x7fdc24145000 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc24145740 con 0x7fdc24145000 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/4035311290 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c007e60 con 0x7fdc24145000 2024-04-25T13:41:29.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.510+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc24158aa0 con 0x7fdc24145000 2024-04-25T13:41:29.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.514+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2406a3a0 con 0x7fdc24145000 2024-04-25T13:41:29.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.514+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/4035311290 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdc1c01c070 con 0x7fdc24145000 2024-04-25T13:41:29.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.514+0000 7fdc0b7fe640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdc1006f400 0x7fdc10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.514+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/4035311290 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fdc1c08ead0 con 0x7fdc24145000 2024-04-25T13:41:29.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.514+0000 7fdc22d9d640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdc1006f400 0x7fdc10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.514+0000 7fdc22d9d640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdc1006f400 0x7fdc10071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc240b6e70 tx=0x7fdc140011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.518+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/4035311290 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdc1c05be30 con 0x7fdc24145000 2024-04-25T13:41:29.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.678+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc240739c0 con 0x7fdc24145000 2024-04-25T13:41:29.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc0b7fe640 1 -- 172.21.15.80:0/4035311290 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+12819 (secure 0 0 0) 0x7fdc1c05b730 con 0x7fdc24145000 2024-04-25T13:41:29.683 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:29.684 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":29,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:26.526559+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdc1006f400 msgr2=0x7fdc10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdc1006f400 0x7fdc10071910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdc240b6e70 tx=0x7fdc140011a0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc24145000 msgr2=0x7fdc241592a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc24145000 0x7fdc241592a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c0057f0 tx=0x7fdc1c009d40 comp rx=0 tx=0).stop 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 shutdown_connections 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdc1006f400 0x7fdc10071910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdc24145000 0x7fdc241592a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdc241f61a0 0x7fdc24144ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 --2- 172.21.15.80:0/4035311290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdc0c0039c0 0x7fdc24142a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 >> 172.21.15.80:0/4035311290 conn(0x7fdc2406d080 msgr2=0x7fdc240d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:29.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 shutdown_connections 2024-04-25T13:41:29.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.682+0000 7fdc290a9640 1 -- 172.21.15.80:0/4035311290 wait complete. 2024-04-25T13:41:29.704 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:29.704 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:29.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f346a3fa640 1 Processor -- start 2024-04-25T13:41:29.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f346a3fa640 1 -- start start 2024-04-25T13:41:29.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f346a3fa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f346406edb0 0x7f346406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f346a3fa640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3464161b60 0x7f3464161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f346a3fa640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34641624b0 con 0x7f34640b5330 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f346a3fa640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f346406f700 con 0x7f3464161b60 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f346a3fa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34641629b0 con 0x7f346406edb0 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3463fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f34640b5330 0x7f34640d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43434/0 (socket says 172.21.15.80:43434) 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3468997640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3464161b60 0x7f3464161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3463fff640 1 -- 172.21.15.80:0/4285998736 learned_addr learned my addr 172.21.15.80:0/4285998736 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f34637fe640 1 --2- 172.21.15.80:0/4285998736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f346406edb0 0x7f346406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3462ffd640 1 -- 172.21.15.80:0/4285998736 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2488506117 0 0) 0x7f34540014a0 con 0x7f34640b5330 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3462ffd640 1 -- 172.21.15.80:0/4285998736 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f344c003530 con 0x7f34640b5330 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3468997640 1 -- 172.21.15.80:0/4285998736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f346406edb0 msgr2=0x7f346406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3468997640 1 --2- 172.21.15.80:0/4285998736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f346406edb0 0x7f346406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3468997640 1 -- 172.21.15.80:0/4285998736 >> v1:172.21.15.80:6789/0 conn(0x7f34640b5330 legacy=0x7f34640d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f3468997640 1 -- 172.21.15.80:0/4285998736 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34640b5740 con 0x7f3464161b60 2024-04-25T13:41:29.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.854+0000 7f34637fe640 1 --2- 172.21.15.80:0/4285998736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f346406edb0 0x7f346406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:29.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f3468997640 1 --2- 172.21.15.80:0/4285998736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3464161b60 0x7f3464161f70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f345c0097c0 tx=0x7f345c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c04ee4f9f6ab52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f3462ffd640 1 -- 172.21.15.80:0/4285998736 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0b1070 con 0x7f3464161b60 2024-04-25T13:41:29.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f3462ffd640 1 -- 172.21.15.80:0/4285998736 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345c0a8050 con 0x7f3464161b60 2024-04-25T13:41:29.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f3462ffd640 1 -- 172.21.15.80:0/4285998736 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f345c0ac840 con 0x7f3464161b60 2024-04-25T13:41:29.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f346a3fa640 1 -- 172.21.15.80:0/4285998736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3464161b60 msgr2=0x7f3464161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f346a3fa640 1 --2- 172.21.15.80:0/4285998736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3464161b60 0x7f3464161f70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f345c0097c0 tx=0x7f345c01c3e0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f346a3fa640 1 -- 172.21.15.80:0/4285998736 shutdown_connections 2024-04-25T13:41:29.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f346a3fa640 1 --2- 172.21.15.80:0/4285998736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3464161b60 0x7f3464161f70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f346a3fa640 1 --2- 172.21.15.80:0/4285998736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f346406edb0 0x7f346406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f346a3fa640 1 -- 172.21.15.80:0/4285998736 >> 172.21.15.80:0/4285998736 conn(0x7f34640b4160 msgr2=0x7f34640d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:29.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f346a3fa640 1 -- 172.21.15.80:0/4285998736 shutdown_connections 2024-04-25T13:41:29.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.858+0000 7f346a3fa640 1 -- 172.21.15.80:0/4285998736 wait complete. 2024-04-25T13:41:29.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 Processor -- start 2024-04-25T13:41:29.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 -- start start 2024-04-25T13:41:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f346406edb0 0x7f346414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34640b5330 0x7f3464148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3464161b60 0x7f3464147ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f346406e890 con 0x7f346406edb0 2024-04-25T13:41:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f346406e5b0 con 0x7f34640b5330 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34640b0cd0 con 0x7f3464161b60 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f346406edb0 0x7f346414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f346406edb0 0x7f346414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48528/0 (socket says 172.21.15.80:48528) 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 -- 172.21.15.80:0/1617269945 learned_addr learned my addr 172.21.15.80:0/1617269945 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f34637fe640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34640b5330 0x7f3464148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3468997640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3464161b60 0x7f3464147ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 -- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3464161b60 msgr2=0x7f3464147ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3464161b60 0x7f3464147ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 -- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34640b5330 msgr2=0x7f3464148fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34640b5330 0x7f3464148fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 -- 172.21.15.80:0/1617269945 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3464149870 con 0x7f346406edb0 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3468997640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3464161b60 0x7f3464147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3463fff640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f346406edb0 0x7f346414df20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f345400cb40 tx=0x7f345400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3460ff9640 1 -- 172.21.15.80:0/1617269945 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454017070 con 0x7f346406edb0 2024-04-25T13:41:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3460ff9640 1 -- 172.21.15.80:0/1617269945 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f345400ac40 con 0x7f346406edb0 2024-04-25T13:41:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f346414ce80 con 0x7f346406edb0 2024-04-25T13:41:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f3460ff9640 1 -- 172.21.15.80:0/1617269945 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3454012400 con 0x7f346406edb0 2024-04-25T13:41:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f346414d440 con 0x7f346406edb0 2024-04-25T13:41:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.862+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f346406a3a0 con 0x7f346406edb0 2024-04-25T13:41:29.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.866+0000 7f3460ff9640 1 -- 172.21.15.80:0/1617269945 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f345401a070 con 0x7f346406edb0 2024-04-25T13:41:29.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.866+0000 7f3460ff9640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f345006f430 0x7f3450071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:29.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.866+0000 7f3460ff9640 1 -- 172.21.15.80:0/1617269945 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7f345408ecc0 con 0x7f346406edb0 2024-04-25T13:41:29.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.866+0000 7f34637fe640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f345006f430 0x7f3450071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:29.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.866+0000 7f34637fe640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f345006f430 0x7f3450071940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f34640b3000 tx=0x7f3458008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:29.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:29.870+0000 7f3460ff9640 1 -- 172.21.15.80:0/1617269945 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34641624b0 con 0x7f346406edb0 2024-04-25T13:41:30.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.030+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f346406fd00 con 0x7f346406edb0 2024-04-25T13:41:30.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.030+0000 7f3460ff9640 1 -- 172.21.15.80:0/1617269945 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f345405bf90 con 0x7f346406edb0 2024-04-25T13:41:30.034 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:30.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f345006f430 msgr2=0x7f3450071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f345006f430 0x7f3450071940 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f34640b3000 tx=0x7f3458008040 comp rx=0 tx=0).stop 2024-04-25T13:41:30.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f346406edb0 msgr2=0x7f346414df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f346406edb0 0x7f346414df20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f345400cb40 tx=0x7f345400a060 comp rx=0 tx=0).stop 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 shutdown_connections 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f345006f430 0x7f3450071940 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3464161b60 0x7f3464147ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34640b5330 0x7f3464148fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 --2- 172.21.15.80:0/1617269945 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f346406edb0 0x7f346414df20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 >> 172.21.15.80:0/1617269945 conn(0x7f34640b4160 msgr2=0x7f3464154080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 shutdown_connections 2024-04-25T13:41:30.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.034+0000 7f346a3fa640 1 -- 172.21.15.80:0/1617269945 wait complete. 2024-04-25T13:41:30.056 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:41:30.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0f6ed640 1 Processor -- start 2024-04-25T13:41:30.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0f6ed640 1 -- start start 2024-04-25T13:41:30.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0f6ed640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efc081619b0 0x7efc08073580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0f6ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc08073ac0 0x7efc080afc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0f6ed640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efc080b01b0 0x7efc080b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0f6ed640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efc080b61c0 con 0x7efc081619b0 2024-04-25T13:41:30.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0f6ed640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efc08073ed0 con 0x7efc080b01b0 2024-04-25T13:41:30.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0f6ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efc080b0910 con 0x7efc08073ac0 2024-04-25T13:41:30.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0d489640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efc081619b0 0x7efc08073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0d489640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efc081619b0 0x7efc08073580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48540/0 (socket says 172.21.15.80:48540) 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0d489640 1 -- 172.21.15.80:0/844155997 learned_addr learned my addr 172.21.15.80:0/844155997 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0dc8a640 1 --2- 172.21.15.80:0/844155997 >> v2:172.21.15.87:3300/0 conn(0x7efc080b01b0 0x7efc080b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0cc88640 1 --2- 172.21.15.80:0/844155997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc08073ac0 0x7efc080afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0d489640 1 -- 172.21.15.80:0/844155997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc08073ac0 msgr2=0x7efc080afc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0d489640 1 --2- 172.21.15.80:0/844155997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc08073ac0 0x7efc080afc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0d489640 1 -- 172.21.15.80:0/844155997 >> v2:172.21.15.87:3300/0 conn(0x7efc080b01b0 msgr2=0x7efc080b6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0d489640 1 --2- 172.21.15.80:0/844155997 >> v2:172.21.15.87:3300/0 conn(0x7efc080b01b0 0x7efc080b6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0d489640 1 -- 172.21.15.80:0/844155997 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc080b0bf0 con 0x7efc081619b0 2024-04-25T13:41:30.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.194+0000 7efc0cc88640 1 --2- 172.21.15.80:0/844155997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc08073ac0 0x7efc080afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:30.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0d489640 1 --2- 172.21.15.80:0/844155997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc081619b0 0x7efc08073580 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efc08074180 tx=0x7efc0001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3774b2e0bf48a81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:30.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efbfffff640 1 -- 172.21.15.80:0/844155997 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc000a5670 con 0x7efc081619b0 2024-04-25T13:41:30.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efbfffff640 1 -- 172.21.15.80:0/844155997 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc000a5810 con 0x7efc081619b0 2024-04-25T13:41:30.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efbfffff640 1 -- 172.21.15.80:0/844155997 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc000a5be0 con 0x7efc081619b0 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 -- 172.21.15.80:0/844155997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc081619b0 msgr2=0x7efc08073580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/844155997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc081619b0 0x7efc08073580 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7efc08074180 tx=0x7efc0001c9f0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 -- 172.21.15.80:0/844155997 shutdown_connections 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/844155997 >> v2:172.21.15.87:3300/0 conn(0x7efc080b01b0 0x7efc080b6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/844155997 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc08073ac0 0x7efc080afc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/844155997 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc081619b0 0x7efc08073580 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 -- 172.21.15.80:0/844155997 >> 172.21.15.80:0/844155997 conn(0x7efc080d3500 msgr2=0x7efc08072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 -- 172.21.15.80:0/844155997 shutdown_connections 2024-04-25T13:41:30.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.198+0000 7efc0f6ed640 1 -- 172.21.15.80:0/844155997 wait complete. 2024-04-25T13:41:30.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 Processor -- start 2024-04-25T13:41:30.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 -- start start 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc08073ac0 0x7efc0814ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc080b01b0 0x7efc0814f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc081619b0 0x7efc0814f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efc08068710 con 0x7efc08073ac0 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efc08068590 con 0x7efc080b01b0 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efc08068890 con 0x7efc081619b0 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc08073ac0 0x7efc0814ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc08073ac0 0x7efc0814ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48548/0 (socket says 172.21.15.80:48548) 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 -- 172.21.15.80:0/3403426284 learned_addr learned my addr 172.21.15.80:0/3403426284 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0dc8a640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc081619b0 0x7efc0814f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0cc88640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc080b01b0 0x7efc0814f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 -- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc081619b0 msgr2=0x7efc0814f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc081619b0 0x7efc0814f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 -- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc080b01b0 msgr2=0x7efc0814f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc080b01b0 0x7efc0814f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 -- 172.21.15.80:0/3403426284 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc080b42f0 con 0x7efc08073ac0 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0cc88640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc080b01b0 0x7efc0814f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0d489640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc08073ac0 0x7efc0814ca10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efc000af040 tx=0x7efc00019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efbfdffb640 1 -- 172.21.15.80:0/3403426284 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc00009510 con 0x7efc08073ac0 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efbfdffb640 1 -- 172.21.15.80:0/3403426284 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc000be070 con 0x7efc08073ac0 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc0814cf50 con 0x7efc08073ac0 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efbfdffb640 1 -- 172.21.15.80:0/3403426284 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc000b88d0 con 0x7efc08073ac0 2024-04-25T13:41:30.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.202+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc08154140 con 0x7efc08073ac0 2024-04-25T13:41:30.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.206+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc0806a3a0 con 0x7efc08073ac0 2024-04-25T13:41:30.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.206+0000 7efbfdffb640 1 -- 172.21.15.80:0/3403426284 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efc000b8a70 con 0x7efc08073ac0 2024-04-25T13:41:30.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.206+0000 7efbfdffb640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efbec06f400 0x7efbec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.206+0000 7efbfdffb640 1 -- 172.21.15.80:0/3403426284 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7efc0012a070 con 0x7efc08073ac0 2024-04-25T13:41:30.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.206+0000 7efc0cc88640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efbec06f400 0x7efbec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.206+0000 7efc0cc88640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efbec06f400 0x7efbec071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efbf8006fd0 tx=0x7efbf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:30.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.210+0000 7efbfdffb640 1 -- 172.21.15.80:0/3403426284 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efc000165e0 con 0x7efc08073ac0 2024-04-25T13:41:30.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.362+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efc08157840 con 0x7efc08073ac0 2024-04-25T13:41:30.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.362+0000 7efbfdffb640 1 -- 172.21.15.80:0/3403426284 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) ==== 84+0+65 (secure 0 0 0) 0x7efc000f7210 con 0x7efc08073ac0 2024-04-25T13:41:30.367 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:30.367 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:41:30.367 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:30.367 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:41:30.367 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:41:30.367 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:30.367 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:41:30.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efbec06f400 msgr2=0x7efbec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efbec06f400 0x7efbec071910 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7efbf8006fd0 tx=0x7efbf8008040 comp rx=0 tx=0).stop 2024-04-25T13:41:30.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc08073ac0 msgr2=0x7efc0814ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc08073ac0 0x7efc0814ca10 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efc000af040 tx=0x7efc00019790 comp rx=0 tx=0).stop 2024-04-25T13:41:30.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 shutdown_connections 2024-04-25T13:41:30.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efbec06f400 0x7efbec071910 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efc081619b0 0x7efc0814f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efc080b01b0 0x7efc0814f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 --2- 172.21.15.80:0/3403426284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efc08073ac0 0x7efc0814ca10 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 >> 172.21.15.80:0/3403426284 conn(0x7efc080d3500 msgr2=0x7efc08157d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:30.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 shutdown_connections 2024-04-25T13:41:30.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.366+0000 7efc0f6ed640 1 -- 172.21.15.80:0/3403426284 wait complete. 2024-04-25T13:41:30.387 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:41:30.387 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:41:30.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f8505640 1 Processor -- start 2024-04-25T13:41:30.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f8505640 1 -- start start 2024-04-25T13:41:30.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f8505640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b14c0 0x7fd4f00b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f8505640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4f00b5710 0x7fd4f00aff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f8505640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4f00b04b0 0x7fd4f00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f8505640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f00b0e00 con 0x7fd4f00b04b0 2024-04-25T13:41:30.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f8505640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd4f00b5b20 con 0x7fd4f00b5710 2024-04-25T13:41:30.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f8505640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f01618e0 con 0x7fd4f00b14c0 2024-04-25T13:41:30.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f6aa2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4f00b04b0 0x7fd4f00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f6aa2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd4f00b04b0 0x7fd4f00b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48562/0 (socket says 172.21.15.80:48562) 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f6aa2640 1 -- 172.21.15.80:0/4282368905 learned_addr learned my addr 172.21.15.80:0/4282368905 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f62a1640 1 --2- 172.21.15.80:0/4282368905 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b14c0 0x7fd4f00b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f5aa0640 1 --2- 172.21.15.80:0/4282368905 >> v2:172.21.15.87:3300/0 conn(0x7fd4f00b5710 0x7fd4f00aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f6aa2640 1 -- 172.21.15.80:0/4282368905 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b14c0 msgr2=0x7fd4f00b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f6aa2640 1 --2- 172.21.15.80:0/4282368905 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b14c0 0x7fd4f00b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f6aa2640 1 -- 172.21.15.80:0/4282368905 >> v2:172.21.15.87:3300/0 conn(0x7fd4f00b5710 msgr2=0x7fd4f00aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f6aa2640 1 --2- 172.21.15.80:0/4282368905 >> v2:172.21.15.87:3300/0 conn(0x7fd4f00b5710 0x7fd4f00aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f6aa2640 1 -- 172.21.15.80:0/4282368905 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f00af1b0 con 0x7fd4f00b04b0 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f62a1640 1 --2- 172.21.15.80:0/4282368905 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b14c0 0x7fd4f00b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:30.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.534+0000 7fd4f5aa0640 1 --2- 172.21.15.80:0/4282368905 >> v2:172.21.15.87:3300/0 conn(0x7fd4f00b5710 0x7fd4f00aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:30.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f6aa2640 1 --2- 172.21.15.80:0/4282368905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f00b08c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0071a90 tx=0x7fd4e001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d23e7fccc17e5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:30.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f529f640 1 -- 172.21.15.80:0/4282368905 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00a56d0 con 0x7fd4f00b04b0 2024-04-25T13:41:30.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f529f640 1 -- 172.21.15.80:0/4282368905 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e00a5870 con 0x7fd4f00b04b0 2024-04-25T13:41:30.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f529f640 1 -- 172.21.15.80:0/4282368905 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e00a5c40 con 0x7fd4f00b04b0 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 -- 172.21.15.80:0/4282368905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 msgr2=0x7fd4f00b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 --2- 172.21.15.80:0/4282368905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f00b08c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0071a90 tx=0x7fd4e001c960 comp rx=0 tx=0).stop 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 -- 172.21.15.80:0/4282368905 shutdown_connections 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 --2- 172.21.15.80:0/4282368905 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f00b08c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 --2- 172.21.15.80:0/4282368905 >> v2:172.21.15.87:3300/0 conn(0x7fd4f00b5710 0x7fd4f00aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 --2- 172.21.15.80:0/4282368905 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b14c0 0x7fd4f00b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 -- 172.21.15.80:0/4282368905 >> 172.21.15.80:0/4282368905 conn(0x7fd4f0073b30 msgr2=0x7fd4f0073f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 -- 172.21.15.80:0/4282368905 shutdown_connections 2024-04-25T13:41:30.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.538+0000 7fd4f8505640 1 -- 172.21.15.80:0/4282368905 wait complete. 2024-04-25T13:41:30.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 Processor -- start 2024-04-25T13:41:30.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 -- start start 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f0159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00b14c0 0x7fd4f0152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b5710 0x7fd4f0153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f00b7420 con 0x7fd4f00b04b0 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f00b18d0 con 0x7fd4f00b14c0 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f00b75a0 con 0x7fd4f00b5710 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f0159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f0159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48568/0 (socket says 172.21.15.80:48568) 2024-04-25T13:41:30.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 -- 172.21.15.80:0/1951757895 learned_addr learned my addr 172.21.15.80:0/1951757895 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f5aa0640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00b14c0 0x7fd4f0152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f6aa2640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b5710 0x7fd4f0153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 -- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b5710 msgr2=0x7fd4f0153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b5710 0x7fd4f0153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 -- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00b14c0 msgr2=0x7fd4f0152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00b14c0 0x7fd4f0152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 -- 172.21.15.80:0/1951757895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f01f9b20 con 0x7fd4f00b04b0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f62a1640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f0159730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c5e0 tx=0x7fd4e800caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4eeffd640 1 -- 172.21.15.80:0/1951757895 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8015070 con 0x7fd4f00b04b0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4eeffd640 1 -- 172.21.15.80:0/1951757895 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e8011820 con 0x7fd4f00b04b0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f015a1f0 con 0x7fd4f00b04b0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4eeffd640 1 -- 172.21.15.80:0/1951757895 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8011c10 con 0x7fd4f00b04b0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f015b310 con 0x7fd4f00b04b0 2024-04-25T13:41:30.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f01f9cf0 con 0x7fd4f00b04b0 2024-04-25T13:41:30.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.542+0000 7fd4eeffd640 1 -- 172.21.15.80:0/1951757895 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4e8018070 con 0x7fd4f00b04b0 2024-04-25T13:41:30.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.546+0000 7fd4eeffd640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4dc06f400 0x7fd4dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:30.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.546+0000 7fd4eeffd640 1 -- 172.21.15.80:0/1951757895 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 4857+0+0 (secure 0 0 0) 0x7fd4e808eda0 con 0x7fd4f00b04b0 2024-04-25T13:41:30.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.546+0000 7fd4f5aa0640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4dc06f400 0x7fd4dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:30.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.546+0000 7fd4f5aa0640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4dc06f400 0x7fd4dc071910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4006fd0 tx=0x7fd4e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:30.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.550+0000 7fd4eeffd640 1 -- 172.21.15.80:0/1951757895 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4e805bfe0 con 0x7fd4f00b04b0 2024-04-25T13:41:30.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:30.698+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd4f015b100 con 0x7fd4f00b04b0 2024-04-25T13:41:31.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.542+0000 7fd4eeffd640 1 -- 172.21.15.80:0/1951757895 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) ==== 138+0+0 (secure 0 0 0) 0x7fd4e805b8e0 con 0x7fd4f00b04b0 2024-04-25T13:41:31.547 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:41:31.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.546+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4dc06f400 msgr2=0x7fd4dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.546+0000 7fd4f8505640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4dc06f400 0x7fd4dc071910 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4006fd0 tx=0x7fd4e4008040 comp rx=0 tx=0).stop 2024-04-25T13:41:31.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.546+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 msgr2=0x7fd4f0159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f0159730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fd4e800c5e0 tx=0x7fd4e800caa0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 shutdown_connections 2024-04-25T13:41:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4dc06f400 0x7fd4dc071910 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4f00b5710 0x7fd4f0153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4f00b14c0 0x7fd4f0152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 --2- 172.21.15.80:0/1951757895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4f00b04b0 0x7fd4f0159730 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 >> 172.21.15.80:0/1951757895 conn(0x7fd4f0073b30 msgr2=0x7fd4f0154670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 shutdown_connections 2024-04-25T13:41:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.550+0000 7fd4f8505640 1 -- 172.21.15.80:0/1951757895 wait complete. 2024-04-25T13:41:31.576 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:41:31.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.730+0000 7fe302227640 1 Processor -- start 2024-04-25T13:41:31.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.730+0000 7fe302227640 1 -- start start 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe302227640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc161240 0x7fe2fc0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe302227640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2fc161650 0x7fe2fc0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe302227640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2fc06fcf0 con 0x7fe2fc06f5a0 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe302227640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe2fc073f10 con 0x7fe2fc161650 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe302227640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc0b0c60 con 0x7fe2fc161240 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fbfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2fc161650 0x7fe2fc0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fb7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe2fc06f5a0 0x7fe2fc160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36374/0 (socket says 172.21.15.80:36374) 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fb7fe640 1 -- 172.21.15.80:0/3261764158 learned_addr learned my addr 172.21.15.80:0/3261764158 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2faffd640 1 --2- 172.21.15.80:0/3261764158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc161240 0x7fe2fc0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fbfff640 1 -- 172.21.15.80:0/3261764158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc161240 msgr2=0x7fe2fc0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:31.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fbfff640 1 --2- 172.21.15.80:0/3261764158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc161240 0x7fe2fc0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fbfff640 1 -- 172.21.15.80:0/3261764158 >> v1:172.21.15.80:6789/0 conn(0x7fe2fc06f5a0 legacy=0x7fe2fc160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fbfff640 1 -- 172.21.15.80:0/3261764158 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc0b0f40 con 0x7fe2fc161650 2024-04-25T13:41:31.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2faffd640 1 --2- 172.21.15.80:0/3261764158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc161240 0x7fe2fc0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:31.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fbfff640 1 --2- 172.21.15.80:0/3261764158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0728e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40097f0 tx=0x7fe2f401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e752dd96455b8ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:31.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.734+0000 7fe2fa7fc640 1 -- 172.21.15.80:0/3261764158 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40b0070 con 0x7fe2fc161650 2024-04-25T13:41:31.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe2fa7fc640 1 -- 172.21.15.80:0/3261764158 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f40ab600 con 0x7fe2fc161650 2024-04-25T13:41:31.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe2fa7fc640 1 -- 172.21.15.80:0/3261764158 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40ab9d0 con 0x7fe2fc161650 2024-04-25T13:41:31.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- 172.21.15.80:0/3261764158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 msgr2=0x7fe2fc0728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 --2- 172.21.15.80:0/3261764158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0728e0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40097f0 tx=0x7fe2f401c410 comp rx=0 tx=0).stop 2024-04-25T13:41:31.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- 172.21.15.80:0/3261764158 shutdown_connections 2024-04-25T13:41:31.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 --2- 172.21.15.80:0/3261764158 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0728e0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 --2- 172.21.15.80:0/3261764158 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc161240 0x7fe2fc0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- 172.21.15.80:0/3261764158 >> 172.21.15.80:0/3261764158 conn(0x7fe2fc06d080 msgr2=0x7fe2fc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:31.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- 172.21.15.80:0/3261764158 shutdown_connections 2024-04-25T13:41:31.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- 172.21.15.80:0/3261764158 wait complete. 2024-04-25T13:41:31.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 Processor -- start 2024-04-25T13:41:31.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- start start 2024-04-25T13:41:31.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc06f5a0 0x7fe2fc14ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2fc161240 0x7fe2fc14a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc168250 con 0x7fe2fc161650 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe2fc1680d0 con 0x7fe2fc161240 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.738+0000 7fe302227640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe2fc1683d0 con 0x7fe2fc06f5a0 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38556/0 (socket says 172.21.15.80:38556) 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 -- 172.21.15.80:0/516607280 learned_addr learned my addr 172.21.15.80:0/516607280 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2faffd640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2fc161240 0x7fe2fc14a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fb7fe640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc06f5a0 0x7fe2fc14ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 -- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc06f5a0 msgr2=0x7fe2fc14ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc06f5a0 0x7fe2fc14ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 -- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2fc161240 msgr2=0x7fe2fc14a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2fc161240 0x7fe2fc14a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 -- 172.21.15.80:0/516607280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2fc14a9f0 con 0x7fe2fc161650 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fbfff640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0d0510 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40ae040 tx=0x7fe2f401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2f3fff640 1 -- 172.21.15.80:0/516607280 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40b0050 con 0x7fe2fc161650 2024-04-25T13:41:31.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2f3fff640 1 -- 172.21.15.80:0/516607280 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f40bc070 con 0x7fe2fc161650 2024-04-25T13:41:31.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2f3fff640 1 -- 172.21.15.80:0/516607280 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40b5850 con 0x7fe2fc161650 2024-04-25T13:41:31.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2fc14fe80 con 0x7fe2fc161650 2024-04-25T13:41:31.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2fc152c40 con 0x7fe2fc161650 2024-04-25T13:41:31.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2fc06a3a0 con 0x7fe2fc161650 2024-04-25T13:41:31.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2f3fff640 1 -- 172.21.15.80:0/516607280 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2f40b59f0 con 0x7fe2fc161650 2024-04-25T13:41:31.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2f3fff640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2dc06f400 0x7fe2dc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:31.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2f3fff640 1 -- 172.21.15.80:0/516607280 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5227+0+0 (secure 0 0 0) 0x7fe2f4129720 con 0x7fe2fc161650 2024-04-25T13:41:31.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.742+0000 7fe2fb7fe640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2dc06f400 0x7fe2dc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:31.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.746+0000 7fe2fb7fe640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2dc06f400 0x7fe2dc071910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0012f0 tx=0x7fe2ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:31.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.746+0000 7fe2f3fff640 1 -- 172.21.15.80:0/516607280 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe2f4016090 con 0x7fe2fc161650 2024-04-25T13:41:31.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:31.894+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe2fc1603c0 con 0x7fe2fc161650 2024-04-25T13:41:32.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.558+0000 7fe2f3fff640 1 -- 172.21.15.80:0/516607280 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) ==== 170+0+0 (secure 0 0 0) 0x7fe2f40f67f0 con 0x7fe2fc161650 2024-04-25T13:41:32.563 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:41:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2dc06f400 msgr2=0x7fe2dc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2dc06f400 0x7fe2dc071910 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0012f0 tx=0x7fe2ec009040 comp rx=0 tx=0).stop 2024-04-25T13:41:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 msgr2=0x7fe2fc0d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0d0510 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe2f40ae040 tx=0x7fe2f401ac50 comp rx=0 tx=0).stop 2024-04-25T13:41:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 shutdown_connections 2024-04-25T13:41:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2dc06f400 0x7fe2dc071910 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2fc161650 0x7fe2fc0d0510 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2fc161240 0x7fe2fc14a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 --2- 172.21.15.80:0/516607280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2fc06f5a0 0x7fe2fc14ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 >> 172.21.15.80:0/516607280 conn(0x7fe2fc06d080 msgr2=0x7fe2fc168790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 shutdown_connections 2024-04-25T13:41:32.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.566+0000 7fe302227640 1 -- 172.21.15.80:0/516607280 wait complete. 2024-04-25T13:41:32.610 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:41:32.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69feda640 1 Processor -- start 2024-04-25T13:41:32.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69feda640 1 -- start start 2024-04-25T13:41:32.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69feda640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb698072bd0 0x7fb6980af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69feda640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb698161680 con 0x7fb698072fe0 2024-04-25T13:41:32.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69feda640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb698163de0 con 0x7fb698161270 2024-04-25T13:41:32.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69feda640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb6980d12e0 con 0x7fb698072bd0 2024-04-25T13:41:32.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69d475640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb698072bd0 0x7fb6980af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69d475640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb698072bd0 0x7fb6980af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38564/0 (socket says 172.21.15.80:38564) 2024-04-25T13:41:32.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69d475640 1 -- 172.21.15.80:0/3360844785 learned_addr learned my addr 172.21.15.80:0/3360844785 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:32.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69d475640 1 -- 172.21.15.80:0/3360844785 >> v1:172.21.15.87:6789/0 conn(0x7fb698161270 legacy=0x7fb6980724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:32.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69cc74640 1 -- 172.21.15.80:0/3360844785 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1422925773 0 0) 0x7fb698161680 con 0x7fb698072fe0 2024-04-25T13:41:32.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69d475640 1 -- 172.21.15.80:0/3360844785 >> v1:172.21.15.80:6789/0 conn(0x7fb698072fe0 legacy=0x7fb6980afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.758+0000 7fb69d475640 1 -- 172.21.15.80:0/3360844785 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6981640c0 con 0x7fb698072bd0 2024-04-25T13:41:32.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69d475640 1 --2- 172.21.15.80:0/3360844785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072bd0 0x7fb6980af8b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb688009870 tx=0x7fb68801c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1abc91aa3946ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69cc74640 1 -- 172.21.15.80:0/3360844785 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6880b2070 con 0x7fb698072bd0 2024-04-25T13:41:32.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69cc74640 1 -- 172.21.15.80:0/3360844785 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6880a9070 con 0x7fb698072bd0 2024-04-25T13:41:32.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69cc74640 1 -- 172.21.15.80:0/3360844785 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6880ad880 con 0x7fb698072bd0 2024-04-25T13:41:32.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69feda640 1 -- 172.21.15.80:0/3360844785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072bd0 msgr2=0x7fb6980af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69feda640 1 --2- 172.21.15.80:0/3360844785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072bd0 0x7fb6980af8b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb688009870 tx=0x7fb68801c560 comp rx=0 tx=0).stop 2024-04-25T13:41:32.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69feda640 1 -- 172.21.15.80:0/3360844785 shutdown_connections 2024-04-25T13:41:32.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69feda640 1 --2- 172.21.15.80:0/3360844785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072bd0 0x7fb6980af8b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69feda640 1 -- 172.21.15.80:0/3360844785 >> 172.21.15.80:0/3360844785 conn(0x7fb6980b4160 msgr2=0x7fb6980d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:32.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69feda640 1 -- 172.21.15.80:0/3360844785 shutdown_connections 2024-04-25T13:41:32.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.762+0000 7fb69feda640 1 -- 172.21.15.80:0/3360844785 wait complete. 2024-04-25T13:41:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 Processor -- start 2024-04-25T13:41:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 -- start start 2024-04-25T13:41:32.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb698072bd0 0x7fb69815f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072fe0 0x7fb698147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb698161270 0x7fb6981479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb69816eb20 con 0x7fb698072fe0 2024-04-25T13:41:32.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb69816e9a0 con 0x7fb698072bd0 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb69816eca0 con 0x7fb698161270 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072fe0 0x7fb698147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072fe0 0x7fb698147480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38580/0 (socket says 172.21.15.80:38580) 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 -- 172.21.15.80:0/3280790274 learned_addr learned my addr 172.21.15.80:0/3280790274 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69dc76640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb698072bd0 0x7fb69815f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69e477640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb698161270 0x7fb6981479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 -- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb698161270 msgr2=0x7fb6981479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb698161270 0x7fb6981479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 -- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb698072bd0 msgr2=0x7fb69815f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb698072bd0 0x7fb69815f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 -- 172.21.15.80:0/3280790274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69814b3c0 con 0x7fb698072fe0 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69e477640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb698161270 0x7fb6981479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69d475640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072fe0 0x7fb698147480 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb688009870 tx=0x7fb68801ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb68e7fc640 1 -- 172.21.15.80:0/3280790274 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6880be070 con 0x7fb698072fe0 2024-04-25T13:41:32.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb68e7fc640 1 -- 172.21.15.80:0/3280790274 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6880addb0 con 0x7fb698072fe0 2024-04-25T13:41:32.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69814b6a0 con 0x7fb698072fe0 2024-04-25T13:41:32.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb68e7fc640 1 -- 172.21.15.80:0/3280790274 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6880b2040 con 0x7fb698072fe0 2024-04-25T13:41:32.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69814bc60 con 0x7fb698072fe0 2024-04-25T13:41:32.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.766+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb69814b110 con 0x7fb698072fe0 2024-04-25T13:41:32.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.770+0000 7fb68e7fc640 1 -- 172.21.15.80:0/3280790274 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb6880ab070 con 0x7fb698072fe0 2024-04-25T13:41:32.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.770+0000 7fb68e7fc640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb67c06f400 0x7fb67c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:32.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.770+0000 7fb68e7fc640 1 -- 172.21.15.80:0/3280790274 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5609+0+0 (secure 0 0 0) 0x7fb688128bf0 con 0x7fb698072fe0 2024-04-25T13:41:32.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.770+0000 7fb69dc76640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb67c06f400 0x7fb67c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:32.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.770+0000 7fb69dc76640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb67c06f400 0x7fb67c071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb694000f70 tx=0x7fb694009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:32.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.774+0000 7fb68e7fc640 1 -- 172.21.15.80:0/3280790274 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb688016150 con 0x7fb698072fe0 2024-04-25T13:41:32.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:32.970+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb69806a3a0 con 0x7fb698072fe0 2024-04-25T13:41:33.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.562+0000 7fb68e7fc640 1 -- 172.21.15.80:0/3280790274 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) ==== 172+0+0 (secure 0 0 0) 0x7fb6880f5450 con 0x7fb698072fe0 2024-04-25T13:41:33.566 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb67c06f400 msgr2=0x7fb67c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb67c06f400 0x7fb67c071910 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb694000f70 tx=0x7fb694009040 comp rx=0 tx=0).stop 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072fe0 msgr2=0x7fb698147480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072fe0 0x7fb698147480 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fb688009870 tx=0x7fb68801ada0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 shutdown_connections 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb67c06f400 0x7fb67c071910 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb698161270 0x7fb6981479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb698072fe0 0x7fb698147480 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 --2- 172.21.15.80:0/3280790274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb698072bd0 0x7fb69815f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 >> 172.21.15.80:0/3280790274 conn(0x7fb6980b4160 msgr2=0x7fb6980759b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 shutdown_connections 2024-04-25T13:41:33.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.566+0000 7fb69feda640 1 -- 172.21.15.80:0/3280790274 wait complete. 2024-04-25T13:41:33.609 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c13c69640 1 Processor -- start 2024-04-25T13:41:33.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c13c69640 1 -- start start 2024-04-25T13:41:33.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c13c69640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d2e80 0x7f0c0c1607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c13c69640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c0c0d59b0 0x7f0c0c0d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c13c69640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c0c0d6300 con 0x7f0c0c0d4330 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c13c69640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0c0c161060 con 0x7f0c0c0d59b0 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c13c69640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0c161340 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11204640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c0c0d59b0 0x7f0c0c0d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11204640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c0c0d59b0 0x7f0c0c0d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38590/0 (socket says 172.21.15.80:38590) 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11204640 1 -- 172.21.15.80:0/141676130 learned_addr learned my addr 172.21.15.80:0/141676130 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11a05640 1 --2- 172.21.15.80:0/141676130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d2e80 0x7f0c0c1607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11204640 1 -- 172.21.15.80:0/141676130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d2e80 msgr2=0x7f0c0c1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11204640 1 --2- 172.21.15.80:0/141676130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d2e80 0x7f0c0c1607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11204640 1 -- 172.21.15.80:0/141676130 >> v1:172.21.15.87:6789/0 conn(0x7f0c0c0d4330 legacy=0x7f0c0c0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11204640 1 -- 172.21.15.80:0/141676130 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c0d2680 con 0x7f0c0c0d59b0 2024-04-25T13:41:33.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.750+0000 7f0c11a05640 1 --2- 172.21.15.80:0/141676130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d2e80 0x7f0c0c1607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c11204640 1 --2- 172.21.15.80:0/141676130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d59b0 0x7f0c0c0d5dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009870 tx=0x7f0c0801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a4829d486453821 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c10a03640 1 -- 172.21.15.80:0/141676130 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080b1070 con 0x7f0c0c0d59b0 2024-04-25T13:41:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c10a03640 1 -- 172.21.15.80:0/141676130 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c080a8050 con 0x7f0c0c0d59b0 2024-04-25T13:41:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c10a03640 1 -- 172.21.15.80:0/141676130 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c080ac860 con 0x7f0c0c0d59b0 2024-04-25T13:41:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c13c69640 1 -- 172.21.15.80:0/141676130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d59b0 msgr2=0x7f0c0c0d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c13c69640 1 --2- 172.21.15.80:0/141676130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d59b0 0x7f0c0c0d5dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0c08009870 tx=0x7f0c0801c4d0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c13c69640 1 -- 172.21.15.80:0/141676130 shutdown_connections 2024-04-25T13:41:33.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c13c69640 1 --2- 172.21.15.80:0/141676130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d59b0 0x7f0c0c0d5dc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c13c69640 1 --2- 172.21.15.80:0/141676130 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d2e80 0x7f0c0c1607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c13c69640 1 -- 172.21.15.80:0/141676130 >> 172.21.15.80:0/141676130 conn(0x7f0c0c0737a0 msgr2=0x7f0c0c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c13c69640 1 -- 172.21.15.80:0/141676130 shutdown_connections 2024-04-25T13:41:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.754+0000 7f0c13c69640 1 -- 172.21.15.80:0/141676130 wait complete. 2024-04-25T13:41:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 Processor -- start 2024-04-25T13:41:33.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 -- start start 2024-04-25T13:41:33.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d2e80 0x7f0c0c16a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d4330 0x7f0c0c158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c0c0d59b0 0x7f0c0c159360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c0d3410 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0c0d3290 con 0x7f0c0c0d59b0 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0c15df60 con 0x7f0c0c0d4330 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d2e80 0x7f0c0c16a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d2e80 0x7f0c0c16a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38602/0 (socket says 172.21.15.80:38602) 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 -- 172.21.15.80:0/119287385 learned_addr learned my addr 172.21.15.80:0/119287385 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11204640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d4330 0x7f0c0c158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c12206640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c0c0d59b0 0x7f0c0c159360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 -- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d4330 msgr2=0x7f0c0c158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d4330 0x7f0c0c158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 -- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c0c0d59b0 msgr2=0x7f0c0c159360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c0c0d59b0 0x7f0c0c159360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 -- 172.21.15.80:0/119287385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0c1fd930 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c11a05640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d2e80 0x7f0c0c16a0a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00d800 tx=0x7f0bfc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c027fc640 1 -- 172.21.15.80:0/119287385 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00b6c0 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c027fc640 1 -- 172.21.15.80:0/119287385 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bfc019070 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0c1fdc10 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c027fc640 1 -- 172.21.15.80:0/119287385 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00baa0 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0c1fe1d0 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.758+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0c06a3a0 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.762+0000 7f0c027fc640 1 -- 172.21.15.80:0/119287385 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0bfc00bc40 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.762+0000 7f0c027fc640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:33.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.762+0000 7f0c027fc640 1 -- 172.21.15.80:0/119287385 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0bfc0901e0 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.762+0000 7f0c11204640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:33.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.762+0000 7f0c11204640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ae000 tx=0x7f0c08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:33.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.766+0000 7f0c027fc640 1 -- 172.21.15.80:0/119287385 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0bfc05d140 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.926+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0c0c15fe50 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.930+0000 7f0c027fc640 1 -- 172.21.15.80:0/119287385 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16610 (secure 0 0 0) 0x7f0bfc05ca40 con 0x7f0c0c0d2e80 2024-04-25T13:41:33.934 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:33.934 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":32,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:32.974509+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 msgr2=0x7f0bf0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0c080ae000 tx=0x7f0c08008040 comp rx=0 tx=0).stop 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d2e80 msgr2=0x7f0c0c16a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d2e80 0x7f0c0c16a0a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00d800 tx=0x7f0bfc00dcc0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 shutdown_connections 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c0c0d59b0 0x7f0c0c159360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0c0d4330 0x7f0c0c158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 --2- 172.21.15.80:0/119287385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c0c0d2e80 0x7f0c0c16a0a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 >> 172.21.15.80:0/119287385 conn(0x7f0c0c0737a0 msgr2=0x7f0c0c0d51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:33.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 shutdown_connections 2024-04-25T13:41:33.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:33.934+0000 7f0c13c69640 1 -- 172.21.15.80:0/119287385 wait complete. 2024-04-25T13:41:33.955 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:34.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f9b651640 1 Processor -- start 2024-04-25T13:41:34.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f9b651640 1 -- start start 2024-04-25T13:41:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f9b651640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940d4590 0x7f2f940cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f9b651640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2f940d0430 0x7f2f940d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f9b651640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f940d49a0 con 0x7f2f940b68c0 2024-04-25T13:41:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f9b651640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2f940b4d70 con 0x7f2f940d0430 2024-04-25T13:41:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f9b651640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f940b5000 con 0x7f2f940d4590 2024-04-25T13:41:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f98bec640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2f940d0430 0x7f2f940d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f98bec640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2f940d0430 0x7f2f940d0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38606/0 (socket says 172.21.15.80:38606) 2024-04-25T13:41:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f98bec640 1 -- 172.21.15.80:0/3024935460 learned_addr learned my addr 172.21.15.80:0/3024935460 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f993ed640 1 --2- 172.21.15.80:0/3024935460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940d4590 0x7f2f940cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f98bec640 1 -- 172.21.15.80:0/3024935460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940d4590 msgr2=0x7f2f940cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f98bec640 1 --2- 172.21.15.80:0/3024935460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940d4590 0x7f2f940cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f98bec640 1 -- 172.21.15.80:0/3024935460 >> v1:172.21.15.87:6789/0 conn(0x7f2f940b68c0 legacy=0x7f2f940b6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.098+0000 7f2f98bec640 1 -- 172.21.15.80:0/3024935460 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f940d0f30 con 0x7f2f940d0430 2024-04-25T13:41:34.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f98bec640 1 --2- 172.21.15.80:0/3024935460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d0430 0x7f2f940d0840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009e50 tx=0x7f2f8801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2889837a16081c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f87fff640 1 -- 172.21.15.80:0/3024935460 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f880a5810 con 0x7f2f940d0430 2024-04-25T13:41:34.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f87fff640 1 -- 172.21.15.80:0/3024935460 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f880a59b0 con 0x7f2f940d0430 2024-04-25T13:41:34.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f87fff640 1 -- 172.21.15.80:0/3024935460 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f880a5d80 con 0x7f2f940d0430 2024-04-25T13:41:34.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f9b651640 1 -- 172.21.15.80:0/3024935460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d0430 msgr2=0x7f2f940d0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f9b651640 1 --2- 172.21.15.80:0/3024935460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d0430 0x7f2f940d0840 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2f88009e50 tx=0x7f2f8801ca30 comp rx=0 tx=0).stop 2024-04-25T13:41:34.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f9b651640 1 -- 172.21.15.80:0/3024935460 shutdown_connections 2024-04-25T13:41:34.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f9b651640 1 --2- 172.21.15.80:0/3024935460 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d0430 0x7f2f940d0840 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f9b651640 1 --2- 172.21.15.80:0/3024935460 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940d4590 0x7f2f940cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f9b651640 1 -- 172.21.15.80:0/3024935460 >> 172.21.15.80:0/3024935460 conn(0x7f2f940d3a70 msgr2=0x7f2f940d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f9b651640 1 -- 172.21.15.80:0/3024935460 shutdown_connections 2024-04-25T13:41:34.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.102+0000 7f2f9b651640 1 -- 172.21.15.80:0/3024935460 wait complete. 2024-04-25T13:41:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f9b651640 1 Processor -- start 2024-04-25T13:41:34.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f9b651640 1 -- start start 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f9b651640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940b68c0 0x7f2f940cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f9b651640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f940d0430 0x7f2f9415ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f9b651640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d4590 0x7f2f940cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f9b651640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9416b170 con 0x7f2f940d4590 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f9b651640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f940cfcc0 con 0x7f2f940d0430 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f9b651640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9416b2f0 con 0x7f2f940b68c0 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d4590 0x7f2f940cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d4590 0x7f2f940cc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38620/0 (socket says 172.21.15.80:38620) 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 -- 172.21.15.80:0/1156207809 learned_addr learned my addr 172.21.15.80:0/1156207809 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f993ed640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940b68c0 0x7f2f940cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f98bec640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f940d0430 0x7f2f9415ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 -- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940b68c0 msgr2=0x7f2f940cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940b68c0 0x7f2f940cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 -- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f940d0430 msgr2=0x7f2f9415ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f940d0430 0x7f2f9415ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 -- 172.21.15.80:0/1156207809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9414df60 con 0x7f2f940d4590 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f993ed640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940b68c0 0x7f2f940cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f98bec640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f940d0430 0x7f2f9415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f99bee640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d4590 0x7f2f940cc490 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c310 tx=0x7f2f9000c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.106+0000 7f2f85ffb640 1 -- 172.21.15.80:0/1156207809 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90017070 con 0x7f2f940d4590 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f85ffb640 1 -- 172.21.15.80:0/1156207809 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f900052d0 con 0x7f2f940d4590 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f85ffb640 1 -- 172.21.15.80:0/1156207809 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90003800 con 0x7f2f940d4590 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9415f460 con 0x7f2f940d4590 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f94160480 con 0x7f2f940d4590 2024-04-25T13:41:34.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9406a3a0 con 0x7f2f940d4590 2024-04-25T13:41:34.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f85ffb640 1 -- 172.21.15.80:0/1156207809 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f9001a070 con 0x7f2f940d4590 2024-04-25T13:41:34.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f85ffb640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f7806f400 0x7f2f78071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f85ffb640 1 -- 172.21.15.80:0/1156207809 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f2f9008e870 con 0x7f2f940d4590 2024-04-25T13:41:34.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f993ed640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f7806f400 0x7f2f78071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.110+0000 7f2f993ed640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f7806f400 0x7f2f78071910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2f94072a70 tx=0x7f2f8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.114+0000 7f2f85ffb640 1 -- 172.21.15.80:0/1156207809 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f90005480 con 0x7f2f940d4590 2024-04-25T13:41:34.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.262+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f94165730 con 0x7f2f940d4590 2024-04-25T13:41:34.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.262+0000 7f2f85ffb640 1 -- 172.21.15.80:0/1156207809 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16610 (secure 0 0 0) 0x7f2f9005b1f0 con 0x7f2f940d4590 2024-04-25T13:41:34.267 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:34.267 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":32,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:32.974509+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:34.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f7806f400 msgr2=0x7f2f78071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f7806f400 0x7f2f78071910 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2f94072a70 tx=0x7f2f8c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:34.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d4590 msgr2=0x7f2f940cc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d4590 0x7f2f940cc490 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000c310 tx=0x7f2f9000c7d0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 shutdown_connections 2024-04-25T13:41:34.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f7806f400 0x7f2f78071910 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f940d4590 0x7f2f940cc490 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f940d0430 0x7f2f9415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 --2- 172.21.15.80:0/1156207809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f940b68c0 0x7f2f940cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 >> 172.21.15.80:0/1156207809 conn(0x7f2f940d3a70 msgr2=0x7f2f94070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 shutdown_connections 2024-04-25T13:41:34.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.266+0000 7f2f9b651640 1 -- 172.21.15.80:0/1156207809 wait complete. 2024-04-25T13:41:34.287 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:41:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db84af640 1 Processor -- start 2024-04-25T13:41:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db84af640 1 -- start start 2024-04-25T13:41:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db84af640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7db0161270 0x7f7db00724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db84af640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db0161680 con 0x7f7db0072a00 2024-04-25T13:41:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db84af640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7db006f9f0 con 0x7f7db00b08c0 2024-04-25T13:41:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db84af640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7db00d12e0 con 0x7f7db0161270 2024-04-25T13:41:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db624b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7db0161270 0x7f7db00724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db5a4a640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7db0072a00 0x7f7db006ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36394/0 (socket says 172.21.15.80:36394) 2024-04-25T13:41:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db5a4a640 1 -- 172.21.15.80:0/1574952870 learned_addr learned my addr 172.21.15.80:0/1574952870 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db624b640 1 -- 172.21.15.80:0/1574952870 >> v1:172.21.15.87:6789/0 conn(0x7f7db00b08c0 legacy=0x7f7db006f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db5249640 1 -- 172.21.15.80:0/1574952870 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2988234340 0 0) 0x7f7db0161680 con 0x7f7db0072a00 2024-04-25T13:41:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db624b640 1 -- 172.21.15.80:0/1574952870 >> v1:172.21.15.80:6789/0 conn(0x7f7db0072a00 legacy=0x7f7db006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.438+0000 7f7db624b640 1 -- 172.21.15.80:0/1574952870 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db0163de0 con 0x7f7db0161270 2024-04-25T13:41:34.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.442+0000 7f7db624b640 1 --2- 172.21.15.80:0/1574952870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db0161270 0x7f7db00724c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0097c0 tx=0x7f7dac01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3117e8316cf55eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.442+0000 7f7db5249640 1 -- 172.21.15.80:0/1574952870 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0b0070 con 0x7f7db0161270 2024-04-25T13:41:34.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.442+0000 7f7db5249640 1 -- 172.21.15.80:0/1574952870 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7dac0ab600 con 0x7f7db0161270 2024-04-25T13:41:34.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.442+0000 7f7db5249640 1 -- 172.21.15.80:0/1574952870 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dac0ab9d0 con 0x7f7db0161270 2024-04-25T13:41:34.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- 172.21.15.80:0/1574952870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db0161270 msgr2=0x7f7db00724c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 --2- 172.21.15.80:0/1574952870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db0161270 0x7f7db00724c0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0097c0 tx=0x7f7dac01c4a0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- 172.21.15.80:0/1574952870 shutdown_connections 2024-04-25T13:41:34.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 --2- 172.21.15.80:0/1574952870 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db0161270 0x7f7db00724c0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- 172.21.15.80:0/1574952870 >> 172.21.15.80:0/1574952870 conn(0x7f7db00b4160 msgr2=0x7f7db00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:34.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- 172.21.15.80:0/1574952870 shutdown_connections 2024-04-25T13:41:34.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- 172.21.15.80:0/1574952870 wait complete. 2024-04-25T13:41:34.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 Processor -- start 2024-04-25T13:41:34.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- start start 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7db0072a00 0x7f7db0143580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7db0161270 0x7f7db0155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db01535d0 0x7f7db01539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7db016e9a0 con 0x7f7db01535d0 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7db0073220 con 0x7f7db0072a00 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db84af640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7db016eb20 con 0x7f7db0161270 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db6a4c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db01535d0 0x7f7db01539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db6a4c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db01535d0 0x7f7db01539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38632/0 (socket says 172.21.15.80:38632) 2024-04-25T13:41:34.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db6a4c640 1 -- 172.21.15.80:0/3197828701 learned_addr learned my addr 172.21.15.80:0/3197828701 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db624b640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7db0072a00 0x7f7db0143580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.446+0000 7f7db5a4a640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7db0161270 0x7f7db0155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db6a4c640 1 -- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7db0161270 msgr2=0x7f7db0155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db6a4c640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7db0161270 0x7f7db0155170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db6a4c640 1 -- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7db0072a00 msgr2=0x7f7db0143580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db6a4c640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7db0072a00 0x7f7db0143580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db6a4c640 1 -- 172.21.15.80:0/3197828701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7db0147760 con 0x7f7db01535d0 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db624b640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7db0072a00 0x7f7db0143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db6a4c640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db01535d0 0x7f7db01539e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7da800c310 tx=0x7f7da800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7da2ffd640 1 -- 172.21.15.80:0/3197828701 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8017070 con 0x7f7db01535d0 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7da2ffd640 1 -- 172.21.15.80:0/3197828701 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7da80052d0 con 0x7f7db01535d0 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7da2ffd640 1 -- 172.21.15.80:0/3197828701 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7da8003800 con 0x7f7db01535d0 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7db01543a0 con 0x7f7db01535d0 2024-04-25T13:41:34.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7db01471f0 con 0x7f7db01535d0 2024-04-25T13:41:34.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7db01474d0 con 0x7f7db01535d0 2024-04-25T13:41:34.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7da2ffd640 1 -- 172.21.15.80:0/3197828701 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7da801a070 con 0x7f7db01535d0 2024-04-25T13:41:34.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7da2ffd640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d9406f430 0x7f7d94071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:34.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7da2ffd640 1 -- 172.21.15.80:0/3197828701 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f7da808e440 con 0x7f7db01535d0 2024-04-25T13:41:34.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.450+0000 7f7db624b640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d9406f430 0x7f7d94071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:34.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.454+0000 7f7db624b640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d9406f430 0x7f7d94071940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0ad000 tx=0x7f7dac0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:34.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.454+0000 7f7da2ffd640 1 -- 172.21.15.80:0/3197828701 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7da8005480 con 0x7f7db01535d0 2024-04-25T13:41:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:34.650+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7db006a3a0 con 0x7f7db01535d0 2024-04-25T13:41:35.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.614+0000 7f7da2ffd640 1 -- 172.21.15.80:0/3197828701 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f7da80633b0 con 0x7f7db01535d0 2024-04-25T13:41:35.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d9406f430 msgr2=0x7f7d94071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d9406f430 0x7f7d94071940 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7dac0ad000 tx=0x7f7dac0a6000 comp rx=0 tx=0).stop 2024-04-25T13:41:35.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db01535d0 msgr2=0x7f7db01539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db01535d0 0x7f7db01539e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7da800c310 tx=0x7f7da800c7d0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 shutdown_connections 2024-04-25T13:41:35.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7d9406f430 0x7f7d94071940 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7db01535d0 0x7f7db01539e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7db0161270 0x7f7db0155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 --2- 172.21.15.80:0/3197828701 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7db0072a00 0x7f7db0143580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 >> 172.21.15.80:0/3197828701 conn(0x7f7db00b4160 msgr2=0x7f7db00b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 shutdown_connections 2024-04-25T13:41:35.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.618+0000 7f7db84af640 1 -- 172.21.15.80:0/3197828701 wait complete. 2024-04-25T13:41:35.648 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:41:35.648 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:41:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd557f14640 1 Processor -- start 2024-04-25T13:41:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd557f14640 1 -- start start 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd557f14640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd5500d3ed0 0x7fd5500d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd557f14640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500d2ad0 0x7fd5500b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd557f14640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5500b92a0 con 0x7fd5500b7c10 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd557f14640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd5500d4c10 con 0x7fd5500d3ed0 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd557f14640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5500d4ef0 con 0x7fd5500d2ad0 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd555cb0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd5500d3ed0 0x7fd5500d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd5564b1640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd5500b7c10 0x7fd5500b9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36404/0 (socket says 172.21.15.80:36404) 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd5564b1640 1 -- 172.21.15.80:0/1672665668 learned_addr learned my addr 172.21.15.80:0/1672665668 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd5554af640 1 --2- 172.21.15.80:0/1672665668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500d2ad0 0x7fd5500b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd555cb0640 1 -- 172.21.15.80:0/1672665668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500d2ad0 msgr2=0x7fd5500b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd555cb0640 1 --2- 172.21.15.80:0/1672665668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500d2ad0 0x7fd5500b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd555cb0640 1 -- 172.21.15.80:0/1672665668 >> v1:172.21.15.80:6789/0 conn(0x7fd5500b7c10 legacy=0x7fd5500b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd555cb0640 1 -- 172.21.15.80:0/1672665668 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5500d51d0 con 0x7fd5500d3ed0 2024-04-25T13:41:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd554cae640 1 -- 172.21.15.80:0/1672665668 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 865407386 0 0) 0x7fd5480014a0 con 0x7fd5500b7c10 2024-04-25T13:41:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.806+0000 7fd5554af640 1 --2- 172.21.15.80:0/1672665668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500d2ad0 0x7fd5500b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:35.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd555cb0640 1 --2- 172.21.15.80:0/1672665668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d3ed0 0x7fd5500d2590 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5400097c0 tx=0x7fd54001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b7dd2bdb5383de7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd554cae640 1 -- 172.21.15.80:0/1672665668 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400aa620 con 0x7fd5500d3ed0 2024-04-25T13:41:35.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd554cae640 1 -- 172.21.15.80:0/1672665668 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5400af070 con 0x7fd5500d3ed0 2024-04-25T13:41:35.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd554cae640 1 -- 172.21.15.80:0/1672665668 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5400aa9e0 con 0x7fd5500d3ed0 2024-04-25T13:41:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 -- 172.21.15.80:0/1672665668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d3ed0 msgr2=0x7fd5500d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 --2- 172.21.15.80:0/1672665668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d3ed0 0x7fd5500d2590 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fd5400097c0 tx=0x7fd54001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 -- 172.21.15.80:0/1672665668 shutdown_connections 2024-04-25T13:41:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 --2- 172.21.15.80:0/1672665668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500d2ad0 0x7fd5500b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 --2- 172.21.15.80:0/1672665668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d3ed0 0x7fd5500d2590 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 -- 172.21.15.80:0/1672665668 >> 172.21.15.80:0/1672665668 conn(0x7fd5500737b0 msgr2=0x7fd55006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 -- 172.21.15.80:0/1672665668 shutdown_connections 2024-04-25T13:41:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 -- 172.21.15.80:0/1672665668 wait complete. 2024-04-25T13:41:35.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 Processor -- start 2024-04-25T13:41:35.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.810+0000 7fd557f14640 1 -- start start 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500b7c10 0x7fd5500c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d2ad0 0x7fd5500c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5500d3ed0 0x7fd55015cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd5500ccae0 con 0x7fd5500d2ad0 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5500cc960 con 0x7fd5500d3ed0 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd550160620 con 0x7fd5500b7c10 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d2ad0 0x7fd5500c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d2ad0 0x7fd5500c9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38654/0 (socket says 172.21.15.80:38654) 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 -- 172.21.15.80:0/2845780818 learned_addr learned my addr 172.21.15.80:0/2845780818 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd555cb0640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500b7c10 0x7fd5500c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5564b1640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5500d3ed0 0x7fd55015cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 -- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500b7c10 msgr2=0x7fd5500c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500b7c10 0x7fd5500c8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 -- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5500d3ed0 msgr2=0x7fd55015cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5500d3ed0 0x7fd55015cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 -- 172.21.15.80:0/2845780818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd550150250 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5554af640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d2ad0 0x7fd5500c9be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00c7f0 tx=0x7fd54c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5467fc640 1 -- 172.21.15.80:0/2845780818 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c015070 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5467fc640 1 -- 172.21.15.80:0/2845780818 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd54c004220 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd550150530 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5467fc640 1 -- 172.21.15.80:0/2845780818 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54c003620 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd550150af0 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd55006a3a0 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.814+0000 7fd5467fc640 1 -- 172.21.15.80:0/2845780818 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd54c0039d0 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.818+0000 7fd5467fc640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd53406f400 0x7fd534071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.818+0000 7fd5467fc640 1 -- 172.21.15.80:0/2845780818 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd54c08f2e0 con 0x7fd5500d2ad0 2024-04-25T13:41:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.818+0000 7fd555cb0640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd53406f400 0x7fd534071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:35.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.818+0000 7fd555cb0640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd53406f400 0x7fd534071910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd5400ae000 tx=0x7fd540008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:35.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:35.822+0000 7fd5467fc640 1 -- 172.21.15.80:0/2845780818 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd54c05c260 con 0x7fd5500d2ad0 2024-04-25T13:41:36.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.010+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd55014ffe0 con 0x7fd5500d2ad0 2024-04-25T13:41:36.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.626+0000 7fd5467fc640 1 -- 172.21.15.80:0/2845780818 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fd54c05bb60 con 0x7fd5500d2ad0 2024-04-25T13:41:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd53406f400 msgr2=0x7fd534071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd53406f400 0x7fd534071910 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd5400ae000 tx=0x7fd540008040 comp rx=0 tx=0).stop 2024-04-25T13:41:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d2ad0 msgr2=0x7fd5500c9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d2ad0 0x7fd5500c9be0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fd54c00c7f0 tx=0x7fd54c00ccb0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 shutdown_connections 2024-04-25T13:41:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd53406f400 0x7fd534071910 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5500d3ed0 0x7fd55015cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5500d2ad0 0x7fd5500c9be0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 --2- 172.21.15.80:0/2845780818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5500b7c10 0x7fd5500c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 >> 172.21.15.80:0/2845780818 conn(0x7fd5500737b0 msgr2=0x7fd5500cc210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 shutdown_connections 2024-04-25T13:41:36.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.630+0000 7fd557f14640 1 -- 172.21.15.80:0/2845780818 wait complete. 2024-04-25T13:41:36.659 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:41:36.659 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:36.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3cb98640 1 Processor -- start 2024-04-25T13:41:36.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3cb98640 1 -- start start 2024-04-25T13:41:36.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3cb98640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3815e280 0x7efe380d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3cb98640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe380b3af0 0x7efe380b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3cb98640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe380d13d0 con 0x7efe380d0800 2024-04-25T13:41:36.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3cb98640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efe380b3660 con 0x7efe380b3af0 2024-04-25T13:41:36.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3cb98640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe380d1700 con 0x7efe3815e280 2024-04-25T13:41:36.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe35d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7efe380d0800 0x7efe380d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36418/0 (socket says 172.21.15.80:36418) 2024-04-25T13:41:36.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe35d9b640 1 -- 172.21.15.80:0/4035981528 learned_addr learned my addr 172.21.15.80:0/4035981528 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:36.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe36d9d640 1 --2- 172.21.15.80:0/4035981528 >> v2:172.21.15.87:3300/0 conn(0x7efe380b3af0 0x7efe380b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3659c640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3815e280 0x7efe380d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 246250775 0 0) 0x7efe380d13d0 con 0x7efe380d0800 2024-04-25T13:41:36.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe20003530 con 0x7efe380d0800 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2155778271 0 0) 0x7efe20003530 con 0x7efe380d0800 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3815e280 msgr2=0x7efe380d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3815e280 0x7efe380d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 >> v2:172.21.15.87:3300/0 conn(0x7efe380b3af0 msgr2=0x7efe380b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 --2- 172.21.15.80:0/4035981528 >> v2:172.21.15.87:3300/0 conn(0x7efe380b3af0 0x7efe380b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe380d19e0 con 0x7efe380d0800 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7efe2c001680 con 0x7efe380d0800 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 >> v1:172.21.15.80:6789/0 conn(0x7efe380d0800 legacy=0x7efe380d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe200039c0 0x7efe20003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe20004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3815e280 0x7efe20005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efe380b3660 con 0x7efe20004810 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe380d1700 con 0x7efe200039c0 2024-04-25T13:41:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe2000ba80 con 0x7efe3815e280 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.810+0000 7efe35d9b640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe20004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3659c640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe200039c0 0x7efe20003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe35d9b640 1 -- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3815e280 msgr2=0x7efe20005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe36d9d640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3815e280 0x7efe20005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe35d9b640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3815e280 0x7efe20005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe35d9b640 1 -- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe200039c0 msgr2=0x7efe20003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe35d9b640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe200039c0 0x7efe20003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe35d9b640 1 -- 172.21.15.80:0/4035981528 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe20003530 con 0x7efe20004810 2024-04-25T13:41:36.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3659c640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe200039c0 0x7efe20003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:36.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe35d9b640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe20004c20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efe38069040 tx=0x7efe2c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7baa3ddfdd0b169 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c007d60 con 0x7efe20004810 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c0a9020 con 0x7efe20004810 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3559a640 1 -- 172.21.15.80:0/4035981528 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2c005b20 con 0x7efe20004810 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3cb98640 1 -- 172.21.15.80:0/4035981528 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 msgr2=0x7efe20004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3cb98640 1 --2- 172.21.15.80:0/4035981528 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe20004c20 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efe38069040 tx=0x7efe2c01cc60 comp rx=0 tx=0).stop 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe36d9d640 1 -- 172.21.15.80:0/4035981528 reap_dead start 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3cb98640 1 -- 172.21.15.80:0/4035981528 shutdown_connections 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3cb98640 1 -- 172.21.15.80:0/4035981528 >> 172.21.15.80:0/4035981528 conn(0x7efe380b5740 msgr2=0x7efe380b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3cb98640 1 -- 172.21.15.80:0/4035981528 shutdown_connections 2024-04-25T13:41:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.814+0000 7efe3cb98640 1 -- 172.21.15.80:0/4035981528 wait complete. 2024-04-25T13:41:36.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 Processor -- start 2024-04-25T13:41:36.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 -- start start 2024-04-25T13:41:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe3809da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3814d880 0x7efe3814aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3814a340 0x7efe38160810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efe381690b0 con 0x7efe20004810 2024-04-25T13:41:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe38168f30 con 0x7efe3814a340 2024-04-25T13:41:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe38169230 con 0x7efe3814d880 2024-04-25T13:41:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe3809da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe3809da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38660/0 (socket says 172.21.15.80:38660) 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 -- 172.21.15.80:0/1702524910 learned_addr learned my addr 172.21.15.80:0/1702524910 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 -- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3814d880 msgr2=0x7efe3814aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3814d880 0x7efe3814aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 -- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3814a340 msgr2=0x7efe38160810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3814a340 0x7efe38160810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 -- 172.21.15.80:0/1702524910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3815f770 con 0x7efe20004810 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3659c640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe3809da30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efe28009fd0 tx=0x7efe28010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe26ffd640 1 -- 172.21.15.80:0/1702524910 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2800f040 con 0x7efe20004810 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe26ffd640 1 -- 172.21.15.80:0/1702524910 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2800ea80 con 0x7efe20004810 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe3815fa50 con 0x7efe20004810 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe26ffd640 1 -- 172.21.15.80:0/1702524910 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe2800ee50 con 0x7efe20004810 2024-04-25T13:41:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.818+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe38160010 con 0x7efe20004810 2024-04-25T13:41:36.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.822+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe3806a3a0 con 0x7efe20004810 2024-04-25T13:41:36.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.822+0000 7efe26ffd640 1 -- 172.21.15.80:0/1702524910 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe280253f0 con 0x7efe20004810 2024-04-25T13:41:36.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.822+0000 7efe26ffd640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe1806f430 0x7efe18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:36.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.822+0000 7efe26ffd640 1 -- 172.21.15.80:0/1702524910 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7efe2808f930 con 0x7efe20004810 2024-04-25T13:41:36.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.822+0000 7efe35d9b640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe1806f430 0x7efe18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:36.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.822+0000 7efe35d9b640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe1806f430 0x7efe18071940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efe38069040 tx=0x7efe380d15a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:36.826+0000 7efe26ffd640 1 -- 172.21.15.80:0/1702524910 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe28018030 con 0x7efe20004810 2024-04-25T13:41:37.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe380b8da0 con 0x7efe20004810 2024-04-25T13:41:37.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe26ffd640 1 -- 172.21.15.80:0/1702524910 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5175 (secure 0 0 0) 0x7efe2805c780 con 0x7efe20004810 2024-04-25T13:41:37.013 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:37.013 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:41:32.974439+0000","modified":"2024-04-25T13:41:36.017164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/966995060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":966995060},{"type":"v1","addr":"172.21.15.87:6835","nonce":966995060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5190,"qdb_cluster":[5190]},"id":2}]} 2024-04-25T13:41:37.014 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 16 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe1806f430 msgr2=0x7efe18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe1806f430 0x7efe18071940 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7efe38069040 tx=0x7efe380d15a0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 msgr2=0x7efe3809da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe3809da30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7efe28009fd0 tx=0x7efe28010d60 comp rx=0 tx=0).stop 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 shutdown_connections 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe1806f430 0x7efe18071940 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe3814a340 0x7efe38160810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe3814d880 0x7efe3814aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.010+0000 7efe3cb98640 1 --2- 172.21.15.80:0/1702524910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe20004810 0x7efe3809da30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.014+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 >> 172.21.15.80:0/1702524910 conn(0x7efe380b5740 msgr2=0x7efe380716e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.014+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 shutdown_connections 2024-04-25T13:41:37.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.014+0000 7efe3cb98640 1 -- 172.21.15.80:0/1702524910 wait complete. 2024-04-25T13:41:37.033 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.178+0000 7f426c62a640 1 Processor -- start 2024-04-25T13:41:37.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.178+0000 7f426c62a640 1 -- start start 2024-04-25T13:41:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f426c62a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42640d06e0 0x7f426406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f426c62a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42640d0af0 con 0x7f426406e9e0 2024-04-25T13:41:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f426c62a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42640d0d70 con 0x7f4264161f90 2024-04-25T13:41:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f426c62a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f42640d1050 con 0x7f42640d06e0 2024-04-25T13:41:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f4269bc5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42640d06e0 0x7f426406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f4269bc5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42640d06e0 0x7f426406e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38674/0 (socket says 172.21.15.80:38674) 2024-04-25T13:41:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f4269bc5640 1 -- 172.21.15.80:0/3263108123 learned_addr learned my addr 172.21.15.80:0/3263108123 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:37.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f4269bc5640 1 -- 172.21.15.80:0/3263108123 >> v1:172.21.15.87:6789/0 conn(0x7f4264161f90 legacy=0x7f42640cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f4269bc5640 1 -- 172.21.15.80:0/3263108123 >> v1:172.21.15.80:6789/0 conn(0x7f426406e9e0 legacy=0x7f426406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f4269bc5640 1 -- 172.21.15.80:0/3263108123 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426406dca0 con 0x7f42640d06e0 2024-04-25T13:41:37.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.182+0000 7f42693c4640 1 -- 172.21.15.80:0/3263108123 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3320332129 0 0) 0x7f425c0014a0 con 0x7f426406e9e0 2024-04-25T13:41:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f4269bc5640 1 --2- 172.21.15.80:0/3263108123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42640d06e0 0x7f426406e4a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4258009870 tx=0x7f425801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16881aa8a8fc889e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f42693c4640 1 -- 172.21.15.80:0/3263108123 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42580b1070 con 0x7f42640d06e0 2024-04-25T13:41:37.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f42693c4640 1 -- 172.21.15.80:0/3263108123 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42580ac630 con 0x7f42640d06e0 2024-04-25T13:41:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f42693c4640 1 -- 172.21.15.80:0/3263108123 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42580a8050 con 0x7f42640d06e0 2024-04-25T13:41:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 -- 172.21.15.80:0/3263108123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42640d06e0 msgr2=0x7f426406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 --2- 172.21.15.80:0/3263108123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42640d06e0 0x7f426406e4a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4258009870 tx=0x7f425801c410 comp rx=0 tx=0).stop 2024-04-25T13:41:37.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 -- 172.21.15.80:0/3263108123 shutdown_connections 2024-04-25T13:41:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 --2- 172.21.15.80:0/3263108123 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42640d06e0 0x7f426406e4a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 -- 172.21.15.80:0/3263108123 >> 172.21.15.80:0/3263108123 conn(0x7f426406d080 msgr2=0x7f42640b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 -- 172.21.15.80:0/3263108123 shutdown_connections 2024-04-25T13:41:37.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 -- 172.21.15.80:0/3263108123 wait complete. 2024-04-25T13:41:37.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 Processor -- start 2024-04-25T13:41:37.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.186+0000 7f426c62a640 1 -- start start 2024-04-25T13:41:37.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f426406e9e0 0x7f426415d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42640d06e0 0x7f4264141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4264161f90 0x7f426413c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f426406f680 con 0x7f4264161f90 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f426406f500 con 0x7f42640d06e0 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4264144620 con 0x7f426406e9e0 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4264161f90 0x7f426413c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4264161f90 0x7f426413c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38686/0 (socket says 172.21.15.80:38686) 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 -- 172.21.15.80:0/2941685704 learned_addr learned my addr 172.21.15.80:0/2941685704 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f4269bc5640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42640d06e0 0x7f4264141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426a3c6640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f426406e9e0 0x7f426415d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 -- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f426406e9e0 msgr2=0x7f426415d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f426406e9e0 0x7f426415d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 -- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42640d06e0 msgr2=0x7f4264141560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42640d06e0 0x7f4264141560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 -- 172.21.15.80:0/2941685704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426415d850 con 0x7f4264161f90 2024-04-25T13:41:37.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426abc7640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4264161f90 0x7f426413c6c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c780 tx=0x7f425c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f4262ffd640 1 -- 172.21.15.80:0/2941685704 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c003db0 con 0x7f4264161f90 2024-04-25T13:41:37.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f4262ffd640 1 -- 172.21.15.80:0/2941685704 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f425c00ce30 con 0x7f4264161f90 2024-04-25T13:41:37.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4264141aa0 con 0x7f4264161f90 2024-04-25T13:41:37.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f4262ffd640 1 -- 172.21.15.80:0/2941685704 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c005290 con 0x7f4264161f90 2024-04-25T13:41:37.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426415fd70 con 0x7f4264161f90 2024-04-25T13:41:37.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426406a3a0 con 0x7f4264161f90 2024-04-25T13:41:37.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f4262ffd640 1 -- 172.21.15.80:0/2941685704 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f425c005430 con 0x7f4264161f90 2024-04-25T13:41:37.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f4262ffd640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f425006f430 0x7f4250071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f4262ffd640 1 -- 172.21.15.80:0/2941685704 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f425c08f190 con 0x7f4264161f90 2024-04-25T13:41:37.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.190+0000 7f426a3c6640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f425006f430 0x7f4250071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.194+0000 7f426a3c6640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f425006f430 0x7f4250071940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4254000f70 tx=0x7f4254009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.198+0000 7f4262ffd640 1 -- 172.21.15.80:0/2941685704 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f42640d0af0 con 0x7f4264161f90 2024-04-25T13:41:37.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.358+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42640ce620 con 0x7f4264161f90 2024-04-25T13:41:37.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.358+0000 7f4262ffd640 1 -- 172.21.15.80:0/2941685704 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16597 (secure 0 0 0) 0x7f425c05c080 con 0x7f4264161f90 2024-04-25T13:41:37.362 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:37.362 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":33,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:34.566242+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:37.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f425006f430 msgr2=0x7f4250071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f425006f430 0x7f4250071940 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4254000f70 tx=0x7f4254009040 comp rx=0 tx=0).stop 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4264161f90 msgr2=0x7f426413c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4264161f90 0x7f426413c6c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f425c00c780 tx=0x7f425c00cc40 comp rx=0 tx=0).stop 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 shutdown_connections 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f425006f430 0x7f4250071940 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4264161f90 0x7f426413c6c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42640d06e0 0x7f4264141560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 --2- 172.21.15.80:0/2941685704 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f426406e9e0 0x7f426415d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 >> 172.21.15.80:0/2941685704 conn(0x7f426406d080 msgr2=0x7f42640b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 shutdown_connections 2024-04-25T13:41:37.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.362+0000 7f426c62a640 1 -- 172.21.15.80:0/2941685704 wait complete. 2024-04-25T13:41:37.384 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:41:37.384 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:41:37.524 INFO:tasks.ceph.mds.b.smithi087.stderr:2024-04-25T13:41:37.521+0000 7f00ad614640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:41:37.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b236c7640 1 Processor -- start 2024-04-25T13:41:37.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b236c7640 1 -- start start 2024-04-25T13:41:37.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b236c7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b1c06f6f0 0x7f9b1c06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b236c7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9b1c0d0f30 0x7f9b1c0d1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b236c7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b1c070040 con 0x7f9b1c0cfca0 2024-04-25T13:41:37.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b236c7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1c0cdd80 con 0x7f9b1c0d0f30 2024-04-25T13:41:37.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b236c7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9b1c073940 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b20c62640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9b1c0d0f30 0x7f9b1c0d1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b20c62640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9b1c0d0f30 0x7f9b1c0d1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38700/0 (socket says 172.21.15.80:38700) 2024-04-25T13:41:37.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b20c62640 1 -- 172.21.15.80:0/3533664954 learned_addr learned my addr 172.21.15.80:0/3533664954 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:37.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.526+0000 7f9b21463640 1 --2- 172.21.15.80:0/3533664954 >> v2:172.21.15.87:3300/0 conn(0x7f9b1c06f6f0 0x7f9b1c06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b20c62640 1 -- 172.21.15.80:0/3533664954 >> v2:172.21.15.87:3300/0 conn(0x7f9b1c06f6f0 msgr2=0x7f9b1c06fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:37.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b20c62640 1 --2- 172.21.15.80:0/3533664954 >> v2:172.21.15.87:3300/0 conn(0x7f9b1c06f6f0 0x7f9b1c06fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b20c62640 1 -- 172.21.15.80:0/3533664954 >> v1:172.21.15.87:6789/0 conn(0x7f9b1c0cfca0 legacy=0x7f9b1c0d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b20c62640 1 -- 172.21.15.80:0/3533664954 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c074430 con 0x7f9b1c0d0f30 2024-04-25T13:41:37.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b20c62640 1 --2- 172.21.15.80:0/3533664954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c0d0f30 0x7f9b1c0d1340 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0723a0 tx=0x7f9b0c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a86e44db1d45299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b13fff640 1 -- 172.21.15.80:0/3533664954 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0b2070 con 0x7f9b1c0d0f30 2024-04-25T13:41:37.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b13fff640 1 -- 172.21.15.80:0/3533664954 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b0c0ad680 con 0x7f9b1c0d0f30 2024-04-25T13:41:37.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b13fff640 1 -- 172.21.15.80:0/3533664954 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b0c0ada50 con 0x7f9b1c0d0f30 2024-04-25T13:41:37.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b236c7640 1 -- 172.21.15.80:0/3533664954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c0d0f30 msgr2=0x7f9b1c0d1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b236c7640 1 --2- 172.21.15.80:0/3533664954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c0d0f30 0x7f9b1c0d1340 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c0723a0 tx=0x7f9b0c01c490 comp rx=0 tx=0).stop 2024-04-25T13:41:37.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b236c7640 1 -- 172.21.15.80:0/3533664954 shutdown_connections 2024-04-25T13:41:37.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b236c7640 1 --2- 172.21.15.80:0/3533664954 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c0d0f30 0x7f9b1c0d1340 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b236c7640 1 --2- 172.21.15.80:0/3533664954 >> v2:172.21.15.87:3300/0 conn(0x7f9b1c06f6f0 0x7f9b1c06fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b236c7640 1 -- 172.21.15.80:0/3533664954 >> 172.21.15.80:0/3533664954 conn(0x7f9b1c0d20d0 msgr2=0x7f9b1c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:37.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b236c7640 1 -- 172.21.15.80:0/3533664954 shutdown_connections 2024-04-25T13:41:37.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.530+0000 7f9b236c7640 1 -- 172.21.15.80:0/3533664954 wait complete. 2024-04-25T13:41:37.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 Processor -- start 2024-04-25T13:41:37.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 -- start start 2024-04-25T13:41:37.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c06f6f0 0x7f9b1c0b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b1c0cfca0 0x7f9b1c0c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b1c0d0f30 0x7f9b1c09f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c0aeec0 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b1c0d1880 con 0x7f9b1c0cfca0 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1c0af040 con 0x7f9b1c0d0f30 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c06f6f0 0x7f9b1c0b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c06f6f0 0x7f9b1c0b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38712/0 (socket says 172.21.15.80:38712) 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 -- 172.21.15.80:0/2306496182 learned_addr learned my addr 172.21.15.80:0/2306496182 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b20c62640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b1c0cfca0 0x7f9b1c0c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21c64640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b1c0d0f30 0x7f9b1c09f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 -- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b1c0d0f30 msgr2=0x7f9b1c09f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b1c0d0f30 0x7f9b1c09f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 -- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b1c0cfca0 msgr2=0x7f9b1c0c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b1c0cfca0 0x7f9b1c0c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 -- 172.21.15.80:0/2306496182 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b1c156540 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21c64640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b1c0d0f30 0x7f9b1c09f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:37.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b21463640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c06f6f0 0x7f9b1c0b4cf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9b18007e60 tx=0x7f9b1800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b11ffb640 1 -- 172.21.15.80:0/2306496182 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1801a070 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b11ffb640 1 -- 172.21.15.80:0/2306496182 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b18003ba0 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b1c0b5230 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b11ffb640 1 -- 172.21.15.80:0/2306496182 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004030 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.534+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b1c156820 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.538+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b1c06a3a0 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.538+0000 7f9b11ffb640 1 -- 172.21.15.80:0/2306496182 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b1801d070 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.538+0000 7f9b11ffb640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0006f400 0x7f9b00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.538+0000 7f9b11ffb640 1 -- 172.21.15.80:0/2306496182 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f9b1808f3f0 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.538+0000 7f9b20c62640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0006f400 0x7f9b00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.538+0000 7f9b20c62640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0006f400 0x7f9b00071910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0af000 tx=0x7f9b0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.542+0000 7f9b11ffb640 1 -- 172.21.15.80:0/2306496182 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b1805bc90 con 0x7f9b1c06f6f0 2024-04-25T13:41:37.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.690+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b1c1585f0 con 0x7f9b0006f400 2024-04-25T13:41:37.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.710+0000 7f9b11ffb640 1 -- 172.21.15.80:0/2306496182 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242206 (secure 0 0 0) 0x7f9b1c1585f0 con 0x7f9b0006f400 2024-04-25T13:41:37.714 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:37.716 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:41:37.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0006f400 msgr2=0x7f9b00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0006f400 0x7f9b00071910 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9b0c0af000 tx=0x7f9b0c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:37.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c06f6f0 msgr2=0x7f9b1c0b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c06f6f0 0x7f9b1c0b4cf0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9b18007e60 tx=0x7f9b1800e4b0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 shutdown_connections 2024-04-25T13:41:37.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0006f400 0x7f9b00071910 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b1c0d0f30 0x7f9b1c09f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b1c0cfca0 0x7f9b1c0c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 --2- 172.21.15.80:0/2306496182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b1c06f6f0 0x7f9b1c0b4cf0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 >> 172.21.15.80:0/2306496182 conn(0x7f9b1c0d20d0 msgr2=0x7f9b1c15e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:37.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 shutdown_connections 2024-04-25T13:41:37.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.714+0000 7f9b236c7640 1 -- 172.21.15.80:0/2306496182 wait complete. 2024-04-25T13:41:37.733 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-25T13:41:36.116599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2176,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560323,"statfs":{"total":1600000000,"available":1597771017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.567085"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583136+0000","last_change":"2024-04-25T13:41:32.583136+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583136+0000","last_undegraded":"2024-04-25T13:41:32.583136+0000","last_fullsized":"2024-04-25T13:41:32.583136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.586323+0000","last_change":"2024-04-25T13:41:32.586323+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.586323+0000","last_undegraded":"2024-04-25T13:41:32.586323+0000","last_fullsized":"2024-04-25T13:41:32.586323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573531+0000","last_change":"2024-04-25T13:41:32.573531+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573531+0000","last_undegraded":"2024-04-25T13:41:32.573531+0000","last_fullsized":"2024-04-25T13:41:32.573531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583817+0000","last_change":"2024-04-25T13:41:32.583817+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583817+0000","last_undegraded":"2024-04-25T13:41:32.583817+0000","last_fullsized":"2024-04-25T13:41:32.583817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.571611+0000","last_change":"2024-04-25T13:41:32.571611+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.571611+0000","last_undegraded":"2024-04-25T13:41:32.571611+0000","last_fullsized":"2024-04-25T13:41:32.571611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585490+0000","last_change":"2024-04-25T13:41:32.585490+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585490+0000","last_undegraded":"2024-04-25T13:41:32.585490+0000","last_fullsized":"2024-04-25T13:41:32.585490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576011+0000","last_change":"2024-04-25T13:41:32.576011+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576011+0000","last_undegraded":"2024-04-25T13:41:32.576011+0000","last_fullsized":"2024-04-25T13:41:32.576011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585019+0000","last_change":"2024-04-25T13:41:32.585019+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585019+0000","last_undegraded":"2024-04-25T13:41:32.585019+0000","last_fullsized":"2024-04-25T13:41:32.585019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576129+0000","last_change":"2024-04-25T13:41:32.576129+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576129+0000","last_undegraded":"2024-04-25T13:41:32.576129+0000","last_fullsized":"2024-04-25T13:41:32.576129+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583916+0000","last_change":"2024-04-25T13:41:32.583916+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583916+0000","last_undegraded":"2024-04-25T13:41:32.583916+0000","last_fullsized":"2024-04-25T13:41:32.583916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.579893+0000","last_change":"2024-04-25T13:41:32.579893+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.579893+0000","last_undegraded":"2024-04-25T13:41:32.579893+0000","last_fullsized":"2024-04-25T13:41:32.579893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581018+0000","last_change":"2024-04-25T13:41:32.581018+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581018+0000","last_undegraded":"2024-04-25T13:41:32.581018+0000","last_fullsized":"2024-04-25T13:41:32.581018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.571707+0000","last_change":"2024-04-25T13:41:32.571707+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.571707+0000","last_undegraded":"2024-04-25T13:41:32.571707+0000","last_fullsized":"2024-04-25T13:41:32.571707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585982+0000","last_change":"2024-04-25T13:41:32.585982+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585982+0000","last_undegraded":"2024-04-25T13:41:32.585982+0000","last_fullsized":"2024-04-25T13:41:32.585982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572059+0000","last_change":"2024-04-25T13:41:32.572059+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572059+0000","last_undegraded":"2024-04-25T13:41:32.572059+0000","last_fullsized":"2024-04-25T13:41:32.572059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578159+0000","last_change":"2024-04-25T13:41:32.578159+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578159+0000","last_undegraded":"2024-04-25T13:41:32.578159+0000","last_fullsized":"2024-04-25T13:41:32.578159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.580079+0000","last_change":"2024-04-25T13:41:32.580079+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.580079+0000","last_undegraded":"2024-04-25T13:41:32.580079+0000","last_fullsized":"2024-04-25T13:41:32.580079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585445+0000","last_change":"2024-04-25T13:41:32.585445+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585445+0000","last_undegraded":"2024-04-25T13:41:32.585445+0000","last_fullsized":"2024-04-25T13:41:32.585445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585538+0000","last_change":"2024-04-25T13:41:32.585538+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585538+0000","last_undegraded":"2024-04-25T13:41:32.585538+0000","last_fullsized":"2024-04-25T13:41:32.585538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570881+0000","last_change":"2024-04-25T13:41:32.570881+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570881+0000","last_undegraded":"2024-04-25T13:41:32.570881+0000","last_fullsized":"2024-04-25T13:41:32.570881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.574825+0000","last_change":"2024-04-25T13:41:32.574825+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.574825+0000","last_undegraded":"2024-04-25T13:41:32.574825+0000","last_fullsized":"2024-04-25T13:41:32.574825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.589854+0000","last_change":"2024-04-25T13:41:32.589854+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.589854+0000","last_undegraded":"2024-04-25T13:41:32.589854+0000","last_fullsized":"2024-04-25T13:41:32.589854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577095+0000","last_change":"2024-04-25T13:41:32.577095+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577095+0000","last_undegraded":"2024-04-25T13:41:32.577095+0000","last_fullsized":"2024-04-25T13:41:32.577095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573074+0000","last_change":"2024-04-25T13:41:32.573074+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573074+0000","last_undegraded":"2024-04-25T13:41:32.573074+0000","last_fullsized":"2024-04-25T13:41:32.573074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583161+0000","last_change":"2024-04-25T13:41:32.583161+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583161+0000","last_undegraded":"2024-04-25T13:41:32.583161+0000","last_fullsized":"2024-04-25T13:41:32.583161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584676+0000","last_change":"2024-04-25T13:41:32.584676+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584676+0000","last_undegraded":"2024-04-25T13:41:32.584676+0000","last_fullsized":"2024-04-25T13:41:32.584676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582937+0000","last_change":"2024-04-25T13:41:32.582937+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582937+0000","last_undegraded":"2024-04-25T13:41:32.582937+0000","last_fullsized":"2024-04-25T13:41:32.582937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577565+0000","last_change":"2024-04-25T13:41:32.577565+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577565+0000","last_undegraded":"2024-04-25T13:41:32.577565+0000","last_fullsized":"2024-04-25T13:41:32.577565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.575137+0000","last_change":"2024-04-25T13:41:32.575137+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.575137+0000","last_undegraded":"2024-04-25T13:41:32.575137+0000","last_fullsized":"2024-04-25T13:41:32.575137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578736+0000","last_change":"2024-04-25T13:41:32.578736+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578736+0000","last_undegraded":"2024-04-25T13:41:32.578736+0000","last_fullsized":"2024-04-25T13:41:32.578736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582191+0000","last_change":"2024-04-25T13:41:32.582191+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582191+0000","last_undegraded":"2024-04-25T13:41:32.582191+0000","last_fullsized":"2024-04-25T13:41:32.582191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577621+0000","last_change":"2024-04-25T13:41:32.577621+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577621+0000","last_undegraded":"2024-04-25T13:41:32.577621+0000","last_fullsized":"2024-04-25T13:41:32.577621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577142+0000","last_change":"2024-04-25T13:41:32.577142+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577142+0000","last_undegraded":"2024-04-25T13:41:32.577142+0000","last_fullsized":"2024-04-25T13:41:32.577142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591696+0000","last_change":"2024-04-25T13:41:32.591289+0000","last_active":"2024-04-25T13:41:32.591696+0000","last_peered":"2024-04-25T13:41:32.591696+0000","last_clean":"2024-04-25T13:41:32.591696+0000","last_became_active":"2024-04-25T13:41:32.590480+0000","last_became_peered":"2024-04-25T13:41:32.590480+0000","last_unstale":"2024-04-25T13:41:32.591696+0000","last_undegraded":"2024-04-25T13:41:32.591696+0000","last_fullsized":"2024-04-25T13:41:32.591696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:43.229224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578440+0000","last_change":"2024-04-25T13:41:32.578440+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578440+0000","last_undegraded":"2024-04-25T13:41:32.578440+0000","last_fullsized":"2024-04-25T13:41:32.578440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590389+0000","last_change":"2024-04-25T13:41:32.590216+0000","last_active":"2024-04-25T13:41:32.590389+0000","last_peered":"2024-04-25T13:41:32.590389+0000","last_clean":"2024-04-25T13:41:32.590389+0000","last_became_active":"2024-04-25T13:41:32.589855+0000","last_became_peered":"2024-04-25T13:41:32.589855+0000","last_unstale":"2024-04-25T13:41:32.590389+0000","last_undegraded":"2024-04-25T13:41:32.590389+0000","last_fullsized":"2024-04-25T13:41:32.590389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:29.322913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573601+0000","last_change":"2024-04-25T13:41:32.573601+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573601+0000","last_undegraded":"2024-04-25T13:41:32.573601+0000","last_fullsized":"2024-04-25T13:41:32.573601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579266+0000","last_change":"2024-04-25T13:41:32.578789+0000","last_active":"2024-04-25T13:41:32.579266+0000","last_peered":"2024-04-25T13:41:32.579266+0000","last_clean":"2024-04-25T13:41:32.579266+0000","last_became_active":"2024-04-25T13:41:32.578278+0000","last_became_peered":"2024-04-25T13:41:32.578278+0000","last_unstale":"2024-04-25T13:41:32.579266+0000","last_undegraded":"2024-04-25T13:41:32.579266+0000","last_fullsized":"2024-04-25T13:41:32.579266+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:04.613101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582460+0000","last_change":"2024-04-25T13:41:32.582460+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582460+0000","last_undegraded":"2024-04-25T13:41:32.582460+0000","last_fullsized":"2024-04-25T13:41:32.582460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.577933+0000","last_change":"2024-04-25T13:41:32.577475+0000","last_active":"2024-04-25T13:41:32.577933+0000","last_peered":"2024-04-25T13:41:32.577933+0000","last_clean":"2024-04-25T13:41:32.577933+0000","last_became_active":"2024-04-25T13:41:32.576661+0000","last_became_peered":"2024-04-25T13:41:32.576661+0000","last_unstale":"2024-04-25T13:41:32.577933+0000","last_undegraded":"2024-04-25T13:41:32.577933+0000","last_fullsized":"2024-04-25T13:41:32.577933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:23.408800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583853+0000","last_change":"2024-04-25T13:41:32.583853+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583853+0000","last_undegraded":"2024-04-25T13:41:32.583853+0000","last_fullsized":"2024-04-25T13:41:32.583853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.603280+0000","last_change":"2024-04-25T13:41:32.602990+0000","last_active":"2024-04-25T13:41:32.603280+0000","last_peered":"2024-04-25T13:41:32.603280+0000","last_clean":"2024-04-25T13:41:32.603280+0000","last_became_active":"2024-04-25T13:41:32.602174+0000","last_became_peered":"2024-04-25T13:41:32.602174+0000","last_unstale":"2024-04-25T13:41:32.603280+0000","last_undegraded":"2024-04-25T13:41:32.603280+0000","last_fullsized":"2024-04-25T13:41:32.603280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:20.977995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.575940+0000","last_change":"2024-04-25T13:41:32.575940+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.575940+0000","last_undegraded":"2024-04-25T13:41:32.575940+0000","last_fullsized":"2024-04-25T13:41:32.575940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.606395+0000","last_change":"2024-04-25T13:41:32.606199+0000","last_active":"2024-04-25T13:41:32.606395+0000","last_peered":"2024-04-25T13:41:32.606395+0000","last_clean":"2024-04-25T13:41:32.606395+0000","last_became_active":"2024-04-25T13:41:32.604933+0000","last_became_peered":"2024-04-25T13:41:32.604933+0000","last_unstale":"2024-04-25T13:41:32.606395+0000","last_undegraded":"2024-04-25T13:41:32.606395+0000","last_fullsized":"2024-04-25T13:41:32.606395+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:50.753222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.602644+0000","last_change":"2024-04-25T13:41:32.602644+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.602644+0000","last_undegraded":"2024-04-25T13:41:32.602644+0000","last_fullsized":"2024-04-25T13:41:32.602644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.586454+0000","last_change":"2024-04-25T13:41:32.586286+0000","last_active":"2024-04-25T13:41:32.586454+0000","last_peered":"2024-04-25T13:41:32.586454+0000","last_clean":"2024-04-25T13:41:32.586454+0000","last_became_active":"2024-04-25T13:41:32.584859+0000","last_became_peered":"2024-04-25T13:41:32.584859+0000","last_unstale":"2024-04-25T13:41:32.586454+0000","last_undegraded":"2024-04-25T13:41:32.586454+0000","last_fullsized":"2024-04-25T13:41:32.586454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:21.284984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581729+0000","last_change":"2024-04-25T13:41:32.581729+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581729+0000","last_undegraded":"2024-04-25T13:41:32.581729+0000","last_fullsized":"2024-04-25T13:41:32.581729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.610432+0000","last_change":"2024-04-25T13:41:32.610199+0000","last_active":"2024-04-25T13:41:32.610432+0000","last_peered":"2024-04-25T13:41:32.610432+0000","last_clean":"2024-04-25T13:41:32.610432+0000","last_became_active":"2024-04-25T13:41:32.607495+0000","last_became_peered":"2024-04-25T13:41:32.607495+0000","last_unstale":"2024-04-25T13:41:32.610432+0000","last_undegraded":"2024-04-25T13:41:32.610432+0000","last_fullsized":"2024-04-25T13:41:32.610432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:34.040738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572876+0000","last_change":"2024-04-25T13:41:32.572876+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572876+0000","last_undegraded":"2024-04-25T13:41:32.572876+0000","last_fullsized":"2024-04-25T13:41:32.572876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591561+0000","last_change":"2024-04-25T13:41:32.591386+0000","last_active":"2024-04-25T13:41:32.591561+0000","last_peered":"2024-04-25T13:41:32.591561+0000","last_clean":"2024-04-25T13:41:32.591561+0000","last_became_active":"2024-04-25T13:41:32.591004+0000","last_became_peered":"2024-04-25T13:41:32.591004+0000","last_unstale":"2024-04-25T13:41:32.591561+0000","last_undegraded":"2024-04-25T13:41:32.591561+0000","last_fullsized":"2024-04-25T13:41:32.591561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:55.459670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584067+0000","last_change":"2024-04-25T13:41:32.584067+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584067+0000","last_undegraded":"2024-04-25T13:41:32.584067+0000","last_fullsized":"2024-04-25T13:41:32.584067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.577987+0000","last_change":"2024-04-25T13:41:32.577747+0000","last_active":"2024-04-25T13:41:32.577987+0000","last_peered":"2024-04-25T13:41:32.577987+0000","last_clean":"2024-04-25T13:41:32.577987+0000","last_became_active":"2024-04-25T13:41:32.577358+0000","last_became_peered":"2024-04-25T13:41:32.577358+0000","last_unstale":"2024-04-25T13:41:32.577987+0000","last_undegraded":"2024-04-25T13:41:32.577987+0000","last_fullsized":"2024-04-25T13:41:32.577987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:06.339192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584147+0000","last_change":"2024-04-25T13:41:32.584147+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584147+0000","last_undegraded":"2024-04-25T13:41:32.584147+0000","last_fullsized":"2024-04-25T13:41:32.584147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591771+0000","last_change":"2024-04-25T13:41:32.591430+0000","last_active":"2024-04-25T13:41:32.591771+0000","last_peered":"2024-04-25T13:41:32.591771+0000","last_clean":"2024-04-25T13:41:32.591771+0000","last_became_active":"2024-04-25T13:41:32.590686+0000","last_became_peered":"2024-04-25T13:41:32.590686+0000","last_unstale":"2024-04-25T13:41:32.591771+0000","last_undegraded":"2024-04-25T13:41:32.591771+0000","last_fullsized":"2024-04-25T13:41:32.591771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.287292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576542+0000","last_change":"2024-04-25T13:41:32.576542+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576542+0000","last_undegraded":"2024-04-25T13:41:32.576542+0000","last_fullsized":"2024-04-25T13:41:32.576542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.580413+0000","last_change":"2024-04-25T13:41:32.580227+0000","last_active":"2024-04-25T13:41:32.580413+0000","last_peered":"2024-04-25T13:41:32.580413+0000","last_clean":"2024-04-25T13:41:32.580413+0000","last_became_active":"2024-04-25T13:41:32.579922+0000","last_became_peered":"2024-04-25T13:41:32.579922+0000","last_unstale":"2024-04-25T13:41:32.580413+0000","last_undegraded":"2024-04-25T13:41:32.580413+0000","last_fullsized":"2024-04-25T13:41:32.580413+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:33.424560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581800+0000","last_change":"2024-04-25T13:41:32.581800+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581800+0000","last_undegraded":"2024-04-25T13:41:32.581800+0000","last_fullsized":"2024-04-25T13:41:32.581800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.609913+0000","last_change":"2024-04-25T13:41:32.609679+0000","last_active":"2024-04-25T13:41:32.609913+0000","last_peered":"2024-04-25T13:41:32.609913+0000","last_clean":"2024-04-25T13:41:32.609913+0000","last_became_active":"2024-04-25T13:41:32.606623+0000","last_became_peered":"2024-04-25T13:41:32.606623+0000","last_unstale":"2024-04-25T13:41:32.609913+0000","last_undegraded":"2024-04-25T13:41:32.609913+0000","last_fullsized":"2024-04-25T13:41:32.609913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.383684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582909+0000","last_change":"2024-04-25T13:41:32.582909+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582909+0000","last_undegraded":"2024-04-25T13:41:32.582909+0000","last_fullsized":"2024-04-25T13:41:32.582909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590846+0000","last_change":"2024-04-25T13:41:32.590694+0000","last_active":"2024-04-25T13:41:32.590846+0000","last_peered":"2024-04-25T13:41:32.590846+0000","last_clean":"2024-04-25T13:41:32.590846+0000","last_became_active":"2024-04-25T13:41:32.590377+0000","last_became_peered":"2024-04-25T13:41:32.590377+0000","last_unstale":"2024-04-25T13:41:32.590846+0000","last_undegraded":"2024-04-25T13:41:32.590846+0000","last_fullsized":"2024-04-25T13:41:32.590846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:33.114772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582801+0000","last_change":"2024-04-25T13:41:32.582801+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582801+0000","last_undegraded":"2024-04-25T13:41:32.582801+0000","last_fullsized":"2024-04-25T13:41:32.582801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590381+0000","last_change":"2024-04-25T13:41:32.590186+0000","last_active":"2024-04-25T13:41:32.590381+0000","last_peered":"2024-04-25T13:41:32.590381+0000","last_clean":"2024-04-25T13:41:32.590381+0000","last_became_active":"2024-04-25T13:41:32.589854+0000","last_became_peered":"2024-04-25T13:41:32.589854+0000","last_unstale":"2024-04-25T13:41:32.590381+0000","last_undegraded":"2024-04-25T13:41:32.590381+0000","last_fullsized":"2024-04-25T13:41:32.590381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:48.433156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.580608+0000","last_change":"2024-04-25T13:41:32.580608+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.580608+0000","last_undegraded":"2024-04-25T13:41:32.580608+0000","last_fullsized":"2024-04-25T13:41:32.580608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.578994+0000","last_change":"2024-04-25T13:41:32.578797+0000","last_active":"2024-04-25T13:41:32.578994+0000","last_peered":"2024-04-25T13:41:32.578994+0000","last_clean":"2024-04-25T13:41:32.578994+0000","last_became_active":"2024-04-25T13:41:32.578411+0000","last_became_peered":"2024-04-25T13:41:32.578411+0000","last_unstale":"2024-04-25T13:41:32.578994+0000","last_undegraded":"2024-04-25T13:41:32.578994+0000","last_fullsized":"2024-04-25T13:41:32.578994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:46.073052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.600550+0000","last_change":"2024-04-25T13:41:32.600550+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.600550+0000","last_undegraded":"2024-04-25T13:41:32.600550+0000","last_fullsized":"2024-04-25T13:41:32.600550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579352+0000","last_change":"2024-04-25T13:41:32.579069+0000","last_active":"2024-04-25T13:41:32.579352+0000","last_peered":"2024-04-25T13:41:32.579352+0000","last_clean":"2024-04-25T13:41:32.579352+0000","last_became_active":"2024-04-25T13:41:32.578548+0000","last_became_peered":"2024-04-25T13:41:32.578548+0000","last_unstale":"2024-04-25T13:41:32.579352+0000","last_undegraded":"2024-04-25T13:41:32.579352+0000","last_fullsized":"2024-04-25T13:41:32.579352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:28.487849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572005+0000","last_change":"2024-04-25T13:41:32.572005+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572005+0000","last_undegraded":"2024-04-25T13:41:32.572005+0000","last_fullsized":"2024-04-25T13:41:32.572005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.609401+0000","last_change":"2024-04-25T13:41:32.609157+0000","last_active":"2024-04-25T13:41:32.609401+0000","last_peered":"2024-04-25T13:41:32.609401+0000","last_clean":"2024-04-25T13:41:32.609401+0000","last_became_active":"2024-04-25T13:41:32.605222+0000","last_became_peered":"2024-04-25T13:41:32.605222+0000","last_unstale":"2024-04-25T13:41:32.609401+0000","last_undegraded":"2024-04-25T13:41:32.609401+0000","last_fullsized":"2024-04-25T13:41:32.609401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:02.704888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581520+0000","last_change":"2024-04-25T13:41:32.581520+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581520+0000","last_undegraded":"2024-04-25T13:41:32.581520+0000","last_fullsized":"2024-04-25T13:41:32.581520+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.596477+0000","last_change":"2024-04-25T13:41:32.595845+0000","last_active":"2024-04-25T13:41:32.596477+0000","last_peered":"2024-04-25T13:41:32.596477+0000","last_clean":"2024-04-25T13:41:32.596477+0000","last_became_active":"2024-04-25T13:41:32.594815+0000","last_became_peered":"2024-04-25T13:41:32.594815+0000","last_unstale":"2024-04-25T13:41:32.596477+0000","last_undegraded":"2024-04-25T13:41:32.596477+0000","last_fullsized":"2024-04-25T13:41:32.596477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:02.015421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570714+0000","last_change":"2024-04-25T13:41:32.570714+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570714+0000","last_undegraded":"2024-04-25T13:41:32.570714+0000","last_fullsized":"2024-04-25T13:41:32.570714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.602883+0000","last_change":"2024-04-25T13:41:32.602637+0000","last_active":"2024-04-25T13:41:32.602883+0000","last_peered":"2024-04-25T13:41:32.602883+0000","last_clean":"2024-04-25T13:41:32.602883+0000","last_became_active":"2024-04-25T13:41:32.601334+0000","last_became_peered":"2024-04-25T13:41:32.601334+0000","last_unstale":"2024-04-25T13:41:32.602883+0000","last_undegraded":"2024-04-25T13:41:32.602883+0000","last_fullsized":"2024-04-25T13:41:32.602883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:17.292465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569941+0000","last_change":"2024-04-25T13:41:32.569941+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569941+0000","last_undegraded":"2024-04-25T13:41:32.569941+0000","last_fullsized":"2024-04-25T13:41:32.569941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.581327+0000","last_change":"2024-04-25T13:41:32.581137+0000","last_active":"2024-04-25T13:41:32.581327+0000","last_peered":"2024-04-25T13:41:32.581327+0000","last_clean":"2024-04-25T13:41:32.581327+0000","last_became_active":"2024-04-25T13:41:32.580368+0000","last_became_peered":"2024-04-25T13:41:32.580368+0000","last_unstale":"2024-04-25T13:41:32.581327+0000","last_undegraded":"2024-04-25T13:41:32.581327+0000","last_fullsized":"2024-04-25T13:41:32.581327+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:25.222450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570534+0000","last_change":"2024-04-25T13:41:32.570534+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570534+0000","last_undegraded":"2024-04-25T13:41:32.570534+0000","last_fullsized":"2024-04-25T13:41:32.570534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.588429+0000","last_change":"2024-04-25T13:41:32.588053+0000","last_active":"2024-04-25T13:41:32.588429+0000","last_peered":"2024-04-25T13:41:32.588429+0000","last_clean":"2024-04-25T13:41:32.588429+0000","last_became_active":"2024-04-25T13:41:32.587211+0000","last_became_peered":"2024-04-25T13:41:32.587211+0000","last_unstale":"2024-04-25T13:41:32.588429+0000","last_undegraded":"2024-04-25T13:41:32.588429+0000","last_fullsized":"2024-04-25T13:41:32.588429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:56.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569593+0000","last_change":"2024-04-25T13:41:32.569593+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569593+0000","last_undegraded":"2024-04-25T13:41:32.569593+0000","last_fullsized":"2024-04-25T13:41:32.569593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.608840+0000","last_change":"2024-04-25T13:41:32.608569+0000","last_active":"2024-04-25T13:41:32.608840+0000","last_peered":"2024-04-25T13:41:32.608840+0000","last_clean":"2024-04-25T13:41:32.608840+0000","last_became_active":"2024-04-25T13:41:32.604150+0000","last_became_peered":"2024-04-25T13:41:32.604150+0000","last_unstale":"2024-04-25T13:41:32.608840+0000","last_undegraded":"2024-04-25T13:41:32.608840+0000","last_fullsized":"2024-04-25T13:41:32.608840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:49.689785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577294+0000","last_change":"2024-04-25T13:41:32.577294+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577294+0000","last_undegraded":"2024-04-25T13:41:32.577294+0000","last_fullsized":"2024-04-25T13:41:32.577294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.592391+0000","last_change":"2024-04-25T13:41:32.591996+0000","last_active":"2024-04-25T13:41:32.592391+0000","last_peered":"2024-04-25T13:41:32.592391+0000","last_clean":"2024-04-25T13:41:32.592391+0000","last_became_active":"2024-04-25T13:41:32.591149+0000","last_became_peered":"2024-04-25T13:41:32.591149+0000","last_unstale":"2024-04-25T13:41:32.592391+0000","last_undegraded":"2024-04-25T13:41:32.592391+0000","last_fullsized":"2024-04-25T13:41:32.592391+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:47.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569832+0000","last_change":"2024-04-25T13:41:32.569832+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569832+0000","last_undegraded":"2024-04-25T13:41:32.569832+0000","last_fullsized":"2024-04-25T13:41:32.569832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.587494+0000","last_change":"2024-04-25T13:41:32.587338+0000","last_active":"2024-04-25T13:41:32.587494+0000","last_peered":"2024-04-25T13:41:32.587494+0000","last_clean":"2024-04-25T13:41:32.587494+0000","last_became_active":"2024-04-25T13:41:32.587021+0000","last_became_peered":"2024-04-25T13:41:32.587021+0000","last_unstale":"2024-04-25T13:41:32.587494+0000","last_undegraded":"2024-04-25T13:41:32.587494+0000","last_fullsized":"2024-04-25T13:41:32.587494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:02.289278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.574871+0000","last_change":"2024-04-25T13:41:32.574871+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.574871+0000","last_undegraded":"2024-04-25T13:41:32.574871+0000","last_fullsized":"2024-04-25T13:41:32.574871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.581004+0000","last_change":"2024-04-25T13:41:32.580819+0000","last_active":"2024-04-25T13:41:32.581004+0000","last_peered":"2024-04-25T13:41:32.581004+0000","last_clean":"2024-04-25T13:41:32.581004+0000","last_became_active":"2024-04-25T13:41:32.579887+0000","last_became_peered":"2024-04-25T13:41:32.579887+0000","last_unstale":"2024-04-25T13:41:32.581004+0000","last_undegraded":"2024-04-25T13:41:32.581004+0000","last_fullsized":"2024-04-25T13:41:32.581004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:56.380154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576518+0000","last_change":"2024-04-25T13:41:32.576518+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576518+0000","last_undegraded":"2024-04-25T13:41:32.576518+0000","last_fullsized":"2024-04-25T13:41:32.576518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579600+0000","last_change":"2024-04-25T13:41:32.579441+0000","last_active":"2024-04-25T13:41:32.579600+0000","last_peered":"2024-04-25T13:41:32.579600+0000","last_clean":"2024-04-25T13:41:32.579600+0000","last_became_active":"2024-04-25T13:41:32.579097+0000","last_became_peered":"2024-04-25T13:41:32.579097+0000","last_unstale":"2024-04-25T13:41:32.579600+0000","last_undegraded":"2024-04-25T13:41:32.579600+0000","last_fullsized":"2024-04-25T13:41:32.579600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:57.842622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.579544+0000","last_change":"2024-04-25T13:41:32.579544+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.579544+0000","last_undegraded":"2024-04-25T13:41:32.579544+0000","last_fullsized":"2024-04-25T13:41:32.579544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590013+0000","last_change":"2024-04-25T13:41:32.589797+0000","last_active":"2024-04-25T13:41:32.590013+0000","last_peered":"2024-04-25T13:41:32.590013+0000","last_clean":"2024-04-25T13:41:32.590013+0000","last_became_active":"2024-04-25T13:41:32.589337+0000","last_became_peered":"2024-04-25T13:41:32.589337+0000","last_unstale":"2024-04-25T13:41:32.590013+0000","last_undegraded":"2024-04-25T13:41:32.590013+0000","last_fullsized":"2024-04-25T13:41:32.590013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.774287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577739+0000","last_change":"2024-04-25T13:41:32.577739+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577739+0000","last_undegraded":"2024-04-25T13:41:32.577739+0000","last_fullsized":"2024-04-25T13:41:32.577739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590236+0000","last_change":"2024-04-25T13:41:32.590039+0000","last_active":"2024-04-25T13:41:32.590236+0000","last_peered":"2024-04-25T13:41:32.590236+0000","last_clean":"2024-04-25T13:41:32.590236+0000","last_became_active":"2024-04-25T13:41:32.589673+0000","last_became_peered":"2024-04-25T13:41:32.589673+0000","last_unstale":"2024-04-25T13:41:32.590236+0000","last_undegraded":"2024-04-25T13:41:32.590236+0000","last_fullsized":"2024-04-25T13:41:32.590236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:36.047945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583410+0000","last_change":"2024-04-25T13:41:32.583410+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583410+0000","last_undegraded":"2024-04-25T13:41:32.583410+0000","last_fullsized":"2024-04-25T13:41:32.583410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.605796+0000","last_change":"2024-04-25T13:41:32.605521+0000","last_active":"2024-04-25T13:41:32.605796+0000","last_peered":"2024-04-25T13:41:32.605796+0000","last_clean":"2024-04-25T13:41:32.605796+0000","last_became_active":"2024-04-25T13:41:32.603724+0000","last_became_peered":"2024-04-25T13:41:32.603724+0000","last_unstale":"2024-04-25T13:41:32.605796+0000","last_undegraded":"2024-04-25T13:41:32.605796+0000","last_fullsized":"2024-04-25T13:41:32.605796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:22.787359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.596171+0000","last_change":"2024-04-25T13:41:32.596171+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.596171+0000","last_undegraded":"2024-04-25T13:41:32.596171+0000","last_fullsized":"2024-04-25T13:41:32.596171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590930+0000","last_change":"2024-04-25T13:41:32.590768+0000","last_active":"2024-04-25T13:41:32.590930+0000","last_peered":"2024-04-25T13:41:32.590930+0000","last_clean":"2024-04-25T13:41:32.590930+0000","last_became_active":"2024-04-25T13:41:32.589798+0000","last_became_peered":"2024-04-25T13:41:32.589798+0000","last_unstale":"2024-04-25T13:41:32.590930+0000","last_undegraded":"2024-04-25T13:41:32.590930+0000","last_fullsized":"2024-04-25T13:41:32.590930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:15.102750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576040+0000","last_change":"2024-04-25T13:41:32.576040+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576040+0000","last_undegraded":"2024-04-25T13:41:32.576040+0000","last_fullsized":"2024-04-25T13:41:32.576040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.597276+0000","last_change":"2024-04-25T13:41:32.596829+0000","last_active":"2024-04-25T13:41:32.597276+0000","last_peered":"2024-04-25T13:41:32.597276+0000","last_clean":"2024-04-25T13:41:32.597276+0000","last_became_active":"2024-04-25T13:41:32.596092+0000","last_became_peered":"2024-04-25T13:41:32.596092+0000","last_unstale":"2024-04-25T13:41:32.597276+0000","last_undegraded":"2024-04-25T13:41:32.597276+0000","last_fullsized":"2024-04-25T13:41:32.597276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:55.603960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":129,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.585989+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:41:32.585989+0000","last_peered":"2024-04-25T13:41:32.585989+0000","last_clean":"2024-04-25T13:41:32.585989+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:41:32.585989+0000","last_undegraded":"2024-04-25T13:41:32.585989+0000","last_fullsized":"2024-04-25T13:41:32.585989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:41:37.737 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:41:37.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7ff5782640 1 Processor -- start 2024-04-25T13:41:37.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7ff5782640 1 -- start start 2024-04-25T13:41:37.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7ff5782640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7ff0161f90 0x7f7ff00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7ff5782640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7ff5782640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ff00d0d70 con 0x7f7ff015ea70 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7ff5782640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7ff00d1050 con 0x7f7ff0161f90 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7ff5782640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff006dca0 con 0x7f7ff00d0510 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7feeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7ff0161f90 0x7f7ff00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7fef7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7ff015ea70 0x7f7ff015ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36444/0 (socket says 172.21.15.80:36444) 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7fef7fe640 1 -- 172.21.15.80:0/500872289 learned_addr learned my addr 172.21.15.80:0/500872289 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7fee7fc640 1 --2- 172.21.15.80:0/500872289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7feeffd640 1 -- 172.21.15.80:0/500872289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 msgr2=0x7f7ff015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7feeffd640 1 --2- 172.21.15.80:0/500872289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7feeffd640 1 -- 172.21.15.80:0/500872289 >> v1:172.21.15.80:6789/0 conn(0x7f7ff015ea70 legacy=0x7f7ff015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7fedffb640 1 -- 172.21.15.80:0/500872289 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 940871350 0 0) 0x7f7ff00d0d70 con 0x7f7ff015ea70 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7feeffd640 1 -- 172.21.15.80:0/500872289 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff006df80 con 0x7f7ff0161f90 2024-04-25T13:41:37.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.922+0000 7f7fee7fc640 1 --2- 172.21.15.80:0/500872289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff015e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:37.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7feeffd640 1 --2- 172.21.15.80:0/500872289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff00cffd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00097c0 tx=0x7f7fe001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4710be681c47918e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7fedffb640 1 -- 172.21.15.80:0/500872289 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00b0070 con 0x7f7ff0161f90 2024-04-25T13:41:37.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7fedffb640 1 -- 172.21.15.80:0/500872289 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe00ab600 con 0x7f7ff0161f90 2024-04-25T13:41:37.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7fedffb640 1 -- 172.21.15.80:0/500872289 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe00ab9d0 con 0x7f7ff0161f90 2024-04-25T13:41:37.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7ff5782640 1 -- 172.21.15.80:0/500872289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 msgr2=0x7f7ff00cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7ff5782640 1 --2- 172.21.15.80:0/500872289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff00cffd0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00097c0 tx=0x7f7fe001c3b0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7ff5782640 1 -- 172.21.15.80:0/500872289 shutdown_connections 2024-04-25T13:41:37.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7ff5782640 1 --2- 172.21.15.80:0/500872289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff015e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7ff5782640 1 --2- 172.21.15.80:0/500872289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff00cffd0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7ff5782640 1 -- 172.21.15.80:0/500872289 >> 172.21.15.80:0/500872289 conn(0x7f7ff006d080 msgr2=0x7f7ff00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:37.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7ff5782640 1 -- 172.21.15.80:0/500872289 shutdown_connections 2024-04-25T13:41:37.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.926+0000 7f7ff5782640 1 -- 172.21.15.80:0/500872289 wait complete. 2024-04-25T13:41:37.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 Processor -- start 2024-04-25T13:41:37.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 -- start start 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff0165a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff015ea70 0x7f7ff0140930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff015a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff006f4b0 con 0x7f7ff0161f90 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff006f630 con 0x7f7ff015ea70 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7ff00d0920 con 0x7f7ff00d0510 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff015a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff015a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38726/0 (socket says 172.21.15.80:38726) 2024-04-25T13:41:37.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 -- 172.21.15.80:0/1663820655 learned_addr learned my addr 172.21.15.80:0/1663820655 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7feeffd640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff0165a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fee7fc640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff015ea70 0x7f7ff0140930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 -- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 msgr2=0x7f7ff0165a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff0165a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 -- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff015ea70 msgr2=0x7f7ff0140930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff015ea70 0x7f7ff0140930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 -- 172.21.15.80:0/1663820655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff0166390 con 0x7f7ff0161f90 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fef7fe640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff015a1e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800c6d0 tx=0x7f7fe800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fe77fe640 1 -- 172.21.15.80:0/1663820655 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8018070 con 0x7f7ff0161f90 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fe77fe640 1 -- 172.21.15.80:0/1663820655 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7fe8004d70 con 0x7f7ff0161f90 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7fe77fe640 1 -- 172.21.15.80:0/1663820655 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800a880 con 0x7f7ff0161f90 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff01608d0 con 0x7f7ff0161f90 2024-04-25T13:41:37.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff0160e90 con 0x7f7ff0161f90 2024-04-25T13:41:37.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.930+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff006a3a0 con 0x7f7ff0161f90 2024-04-25T13:41:37.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.934+0000 7f7fe77fe640 1 -- 172.21.15.80:0/1663820655 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7fe800ac30 con 0x7f7ff0161f90 2024-04-25T13:41:37.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.934+0000 7f7fe77fe640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7fd006f430 0x7f7fd0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:37.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.934+0000 7f7fe77fe640 1 -- 172.21.15.80:0/1663820655 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f7fe808e8c0 con 0x7f7ff0161f90 2024-04-25T13:41:37.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.934+0000 7f7feeffd640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7fd006f430 0x7f7fd0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:37.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.934+0000 7f7feeffd640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7fd006f430 0x7f7fd0071940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00ad000 tx=0x7f7fe00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:37.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:37.938+0000 7f7fe77fe640 1 -- 172.21.15.80:0/1663820655 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7fe805b150 con 0x7f7ff0161f90 2024-04-25T13:41:38.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.094+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ff00b0be0 con 0x7f7fd006f430 2024-04-25T13:41:38.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.114+0000 7f7fe77fe640 1 -- 172.21.15.80:0/1663820655 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242206 (secure 0 0 0) 0x7f7ff00b0be0 con 0x7f7fd006f430 2024-04-25T13:41:38.120 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:38.122 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:41:38.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7fd006f430 msgr2=0x7f7fd0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7fd006f430 0x7f7fd0071940 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7fe00ad000 tx=0x7f7fe00a6000 comp rx=0 tx=0).stop 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 msgr2=0x7f7ff015a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff015a1e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800c6d0 tx=0x7f7fe800cb90 comp rx=0 tx=0).stop 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 shutdown_connections 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7fd006f430 0x7f7fd0071940 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ff0161f90 0x7f7ff015a1e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ff015ea70 0x7f7ff0140930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 --2- 172.21.15.80:0/1663820655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ff00d0510 0x7f7ff0165a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 >> 172.21.15.80:0/1663820655 conn(0x7f7ff006d080 msgr2=0x7f7ff00b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:38.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 shutdown_connections 2024-04-25T13:41:38.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.118+0000 7f7ff5782640 1 -- 172.21.15.80:0/1663820655 wait complete. 2024-04-25T13:41:38.141 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-04-25T13:41:36.116599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2176,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560323,"statfs":{"total":1600000000,"available":1597771017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.567085"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583136+0000","last_change":"2024-04-25T13:41:32.583136+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583136+0000","last_undegraded":"2024-04-25T13:41:32.583136+0000","last_fullsized":"2024-04-25T13:41:32.583136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.586323+0000","last_change":"2024-04-25T13:41:32.586323+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.586323+0000","last_undegraded":"2024-04-25T13:41:32.586323+0000","last_fullsized":"2024-04-25T13:41:32.586323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573531+0000","last_change":"2024-04-25T13:41:32.573531+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573531+0000","last_undegraded":"2024-04-25T13:41:32.573531+0000","last_fullsized":"2024-04-25T13:41:32.573531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583817+0000","last_change":"2024-04-25T13:41:32.583817+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583817+0000","last_undegraded":"2024-04-25T13:41:32.583817+0000","last_fullsized":"2024-04-25T13:41:32.583817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.571611+0000","last_change":"2024-04-25T13:41:32.571611+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.571611+0000","last_undegraded":"2024-04-25T13:41:32.571611+0000","last_fullsized":"2024-04-25T13:41:32.571611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585490+0000","last_change":"2024-04-25T13:41:32.585490+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585490+0000","last_undegraded":"2024-04-25T13:41:32.585490+0000","last_fullsized":"2024-04-25T13:41:32.585490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576011+0000","last_change":"2024-04-25T13:41:32.576011+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576011+0000","last_undegraded":"2024-04-25T13:41:32.576011+0000","last_fullsized":"2024-04-25T13:41:32.576011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585019+0000","last_change":"2024-04-25T13:41:32.585019+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585019+0000","last_undegraded":"2024-04-25T13:41:32.585019+0000","last_fullsized":"2024-04-25T13:41:32.585019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576129+0000","last_change":"2024-04-25T13:41:32.576129+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576129+0000","last_undegraded":"2024-04-25T13:41:32.576129+0000","last_fullsized":"2024-04-25T13:41:32.576129+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583916+0000","last_change":"2024-04-25T13:41:32.583916+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583916+0000","last_undegraded":"2024-04-25T13:41:32.583916+0000","last_fullsized":"2024-04-25T13:41:32.583916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.579893+0000","last_change":"2024-04-25T13:41:32.579893+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.579893+0000","last_undegraded":"2024-04-25T13:41:32.579893+0000","last_fullsized":"2024-04-25T13:41:32.579893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581018+0000","last_change":"2024-04-25T13:41:32.581018+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581018+0000","last_undegraded":"2024-04-25T13:41:32.581018+0000","last_fullsized":"2024-04-25T13:41:32.581018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.571707+0000","last_change":"2024-04-25T13:41:32.571707+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.571707+0000","last_undegraded":"2024-04-25T13:41:32.571707+0000","last_fullsized":"2024-04-25T13:41:32.571707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585982+0000","last_change":"2024-04-25T13:41:32.585982+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585982+0000","last_undegraded":"2024-04-25T13:41:32.585982+0000","last_fullsized":"2024-04-25T13:41:32.585982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572059+0000","last_change":"2024-04-25T13:41:32.572059+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572059+0000","last_undegraded":"2024-04-25T13:41:32.572059+0000","last_fullsized":"2024-04-25T13:41:32.572059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578159+0000","last_change":"2024-04-25T13:41:32.578159+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578159+0000","last_undegraded":"2024-04-25T13:41:32.578159+0000","last_fullsized":"2024-04-25T13:41:32.578159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.580079+0000","last_change":"2024-04-25T13:41:32.580079+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.580079+0000","last_undegraded":"2024-04-25T13:41:32.580079+0000","last_fullsized":"2024-04-25T13:41:32.580079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585445+0000","last_change":"2024-04-25T13:41:32.585445+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585445+0000","last_undegraded":"2024-04-25T13:41:32.585445+0000","last_fullsized":"2024-04-25T13:41:32.585445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585538+0000","last_change":"2024-04-25T13:41:32.585538+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585538+0000","last_undegraded":"2024-04-25T13:41:32.585538+0000","last_fullsized":"2024-04-25T13:41:32.585538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570881+0000","last_change":"2024-04-25T13:41:32.570881+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570881+0000","last_undegraded":"2024-04-25T13:41:32.570881+0000","last_fullsized":"2024-04-25T13:41:32.570881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.574825+0000","last_change":"2024-04-25T13:41:32.574825+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.574825+0000","last_undegraded":"2024-04-25T13:41:32.574825+0000","last_fullsized":"2024-04-25T13:41:32.574825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.589854+0000","last_change":"2024-04-25T13:41:32.589854+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.589854+0000","last_undegraded":"2024-04-25T13:41:32.589854+0000","last_fullsized":"2024-04-25T13:41:32.589854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577095+0000","last_change":"2024-04-25T13:41:32.577095+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577095+0000","last_undegraded":"2024-04-25T13:41:32.577095+0000","last_fullsized":"2024-04-25T13:41:32.577095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573074+0000","last_change":"2024-04-25T13:41:32.573074+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573074+0000","last_undegraded":"2024-04-25T13:41:32.573074+0000","last_fullsized":"2024-04-25T13:41:32.573074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583161+0000","last_change":"2024-04-25T13:41:32.583161+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583161+0000","last_undegraded":"2024-04-25T13:41:32.583161+0000","last_fullsized":"2024-04-25T13:41:32.583161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584676+0000","last_change":"2024-04-25T13:41:32.584676+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584676+0000","last_undegraded":"2024-04-25T13:41:32.584676+0000","last_fullsized":"2024-04-25T13:41:32.584676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582937+0000","last_change":"2024-04-25T13:41:32.582937+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582937+0000","last_undegraded":"2024-04-25T13:41:32.582937+0000","last_fullsized":"2024-04-25T13:41:32.582937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577565+0000","last_change":"2024-04-25T13:41:32.577565+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577565+0000","last_undegraded":"2024-04-25T13:41:32.577565+0000","last_fullsized":"2024-04-25T13:41:32.577565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.575137+0000","last_change":"2024-04-25T13:41:32.575137+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.575137+0000","last_undegraded":"2024-04-25T13:41:32.575137+0000","last_fullsized":"2024-04-25T13:41:32.575137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578736+0000","last_change":"2024-04-25T13:41:32.578736+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578736+0000","last_undegraded":"2024-04-25T13:41:32.578736+0000","last_fullsized":"2024-04-25T13:41:32.578736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582191+0000","last_change":"2024-04-25T13:41:32.582191+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582191+0000","last_undegraded":"2024-04-25T13:41:32.582191+0000","last_fullsized":"2024-04-25T13:41:32.582191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577621+0000","last_change":"2024-04-25T13:41:32.577621+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577621+0000","last_undegraded":"2024-04-25T13:41:32.577621+0000","last_fullsized":"2024-04-25T13:41:32.577621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577142+0000","last_change":"2024-04-25T13:41:32.577142+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577142+0000","last_undegraded":"2024-04-25T13:41:32.577142+0000","last_fullsized":"2024-04-25T13:41:32.577142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591696+0000","last_change":"2024-04-25T13:41:32.591289+0000","last_active":"2024-04-25T13:41:32.591696+0000","last_peered":"2024-04-25T13:41:32.591696+0000","last_clean":"2024-04-25T13:41:32.591696+0000","last_became_active":"2024-04-25T13:41:32.590480+0000","last_became_peered":"2024-04-25T13:41:32.590480+0000","last_unstale":"2024-04-25T13:41:32.591696+0000","last_undegraded":"2024-04-25T13:41:32.591696+0000","last_fullsized":"2024-04-25T13:41:32.591696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:43.229224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578440+0000","last_change":"2024-04-25T13:41:32.578440+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578440+0000","last_undegraded":"2024-04-25T13:41:32.578440+0000","last_fullsized":"2024-04-25T13:41:32.578440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590389+0000","last_change":"2024-04-25T13:41:32.590216+0000","last_active":"2024-04-25T13:41:32.590389+0000","last_peered":"2024-04-25T13:41:32.590389+0000","last_clean":"2024-04-25T13:41:32.590389+0000","last_became_active":"2024-04-25T13:41:32.589855+0000","last_became_peered":"2024-04-25T13:41:32.589855+0000","last_unstale":"2024-04-25T13:41:32.590389+0000","last_undegraded":"2024-04-25T13:41:32.590389+0000","last_fullsized":"2024-04-25T13:41:32.590389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:29.322913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573601+0000","last_change":"2024-04-25T13:41:32.573601+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573601+0000","last_undegraded":"2024-04-25T13:41:32.573601+0000","last_fullsized":"2024-04-25T13:41:32.573601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579266+0000","last_change":"2024-04-25T13:41:32.578789+0000","last_active":"2024-04-25T13:41:32.579266+0000","last_peered":"2024-04-25T13:41:32.579266+0000","last_clean":"2024-04-25T13:41:32.579266+0000","last_became_active":"2024-04-25T13:41:32.578278+0000","last_became_peered":"2024-04-25T13:41:32.578278+0000","last_unstale":"2024-04-25T13:41:32.579266+0000","last_undegraded":"2024-04-25T13:41:32.579266+0000","last_fullsized":"2024-04-25T13:41:32.579266+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:04.613101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582460+0000","last_change":"2024-04-25T13:41:32.582460+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582460+0000","last_undegraded":"2024-04-25T13:41:32.582460+0000","last_fullsized":"2024-04-25T13:41:32.582460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.577933+0000","last_change":"2024-04-25T13:41:32.577475+0000","last_active":"2024-04-25T13:41:32.577933+0000","last_peered":"2024-04-25T13:41:32.577933+0000","last_clean":"2024-04-25T13:41:32.577933+0000","last_became_active":"2024-04-25T13:41:32.576661+0000","last_became_peered":"2024-04-25T13:41:32.576661+0000","last_unstale":"2024-04-25T13:41:32.577933+0000","last_undegraded":"2024-04-25T13:41:32.577933+0000","last_fullsized":"2024-04-25T13:41:32.577933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:23.408800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583853+0000","last_change":"2024-04-25T13:41:32.583853+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583853+0000","last_undegraded":"2024-04-25T13:41:32.583853+0000","last_fullsized":"2024-04-25T13:41:32.583853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.603280+0000","last_change":"2024-04-25T13:41:32.602990+0000","last_active":"2024-04-25T13:41:32.603280+0000","last_peered":"2024-04-25T13:41:32.603280+0000","last_clean":"2024-04-25T13:41:32.603280+0000","last_became_active":"2024-04-25T13:41:32.602174+0000","last_became_peered":"2024-04-25T13:41:32.602174+0000","last_unstale":"2024-04-25T13:41:32.603280+0000","last_undegraded":"2024-04-25T13:41:32.603280+0000","last_fullsized":"2024-04-25T13:41:32.603280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:20.977995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.575940+0000","last_change":"2024-04-25T13:41:32.575940+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.575940+0000","last_undegraded":"2024-04-25T13:41:32.575940+0000","last_fullsized":"2024-04-25T13:41:32.575940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.606395+0000","last_change":"2024-04-25T13:41:32.606199+0000","last_active":"2024-04-25T13:41:32.606395+0000","last_peered":"2024-04-25T13:41:32.606395+0000","last_clean":"2024-04-25T13:41:32.606395+0000","last_became_active":"2024-04-25T13:41:32.604933+0000","last_became_peered":"2024-04-25T13:41:32.604933+0000","last_unstale":"2024-04-25T13:41:32.606395+0000","last_undegraded":"2024-04-25T13:41:32.606395+0000","last_fullsized":"2024-04-25T13:41:32.606395+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:50.753222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.602644+0000","last_change":"2024-04-25T13:41:32.602644+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.602644+0000","last_undegraded":"2024-04-25T13:41:32.602644+0000","last_fullsized":"2024-04-25T13:41:32.602644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.586454+0000","last_change":"2024-04-25T13:41:32.586286+0000","last_active":"2024-04-25T13:41:32.586454+0000","last_peered":"2024-04-25T13:41:32.586454+0000","last_clean":"2024-04-25T13:41:32.586454+0000","last_became_active":"2024-04-25T13:41:32.584859+0000","last_became_peered":"2024-04-25T13:41:32.584859+0000","last_unstale":"2024-04-25T13:41:32.586454+0000","last_undegraded":"2024-04-25T13:41:32.586454+0000","last_fullsized":"2024-04-25T13:41:32.586454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:21.284984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581729+0000","last_change":"2024-04-25T13:41:32.581729+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581729+0000","last_undegraded":"2024-04-25T13:41:32.581729+0000","last_fullsized":"2024-04-25T13:41:32.581729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.610432+0000","last_change":"2024-04-25T13:41:32.610199+0000","last_active":"2024-04-25T13:41:32.610432+0000","last_peered":"2024-04-25T13:41:32.610432+0000","last_clean":"2024-04-25T13:41:32.610432+0000","last_became_active":"2024-04-25T13:41:32.607495+0000","last_became_peered":"2024-04-25T13:41:32.607495+0000","last_unstale":"2024-04-25T13:41:32.610432+0000","last_undegraded":"2024-04-25T13:41:32.610432+0000","last_fullsized":"2024-04-25T13:41:32.610432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:34.040738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572876+0000","last_change":"2024-04-25T13:41:32.572876+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572876+0000","last_undegraded":"2024-04-25T13:41:32.572876+0000","last_fullsized":"2024-04-25T13:41:32.572876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591561+0000","last_change":"2024-04-25T13:41:32.591386+0000","last_active":"2024-04-25T13:41:32.591561+0000","last_peered":"2024-04-25T13:41:32.591561+0000","last_clean":"2024-04-25T13:41:32.591561+0000","last_became_active":"2024-04-25T13:41:32.591004+0000","last_became_peered":"2024-04-25T13:41:32.591004+0000","last_unstale":"2024-04-25T13:41:32.591561+0000","last_undegraded":"2024-04-25T13:41:32.591561+0000","last_fullsized":"2024-04-25T13:41:32.591561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:55.459670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584067+0000","last_change":"2024-04-25T13:41:32.584067+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584067+0000","last_undegraded":"2024-04-25T13:41:32.584067+0000","last_fullsized":"2024-04-25T13:41:32.584067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.577987+0000","last_change":"2024-04-25T13:41:32.577747+0000","last_active":"2024-04-25T13:41:32.577987+0000","last_peered":"2024-04-25T13:41:32.577987+0000","last_clean":"2024-04-25T13:41:32.577987+0000","last_became_active":"2024-04-25T13:41:32.577358+0000","last_became_peered":"2024-04-25T13:41:32.577358+0000","last_unstale":"2024-04-25T13:41:32.577987+0000","last_undegraded":"2024-04-25T13:41:32.577987+0000","last_fullsized":"2024-04-25T13:41:32.577987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:06.339192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584147+0000","last_change":"2024-04-25T13:41:32.584147+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584147+0000","last_undegraded":"2024-04-25T13:41:32.584147+0000","last_fullsized":"2024-04-25T13:41:32.584147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591771+0000","last_change":"2024-04-25T13:41:32.591430+0000","last_active":"2024-04-25T13:41:32.591771+0000","last_peered":"2024-04-25T13:41:32.591771+0000","last_clean":"2024-04-25T13:41:32.591771+0000","last_became_active":"2024-04-25T13:41:32.590686+0000","last_became_peered":"2024-04-25T13:41:32.590686+0000","last_unstale":"2024-04-25T13:41:32.591771+0000","last_undegraded":"2024-04-25T13:41:32.591771+0000","last_fullsized":"2024-04-25T13:41:32.591771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.287292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576542+0000","last_change":"2024-04-25T13:41:32.576542+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576542+0000","last_undegraded":"2024-04-25T13:41:32.576542+0000","last_fullsized":"2024-04-25T13:41:32.576542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.580413+0000","last_change":"2024-04-25T13:41:32.580227+0000","last_active":"2024-04-25T13:41:32.580413+0000","last_peered":"2024-04-25T13:41:32.580413+0000","last_clean":"2024-04-25T13:41:32.580413+0000","last_became_active":"2024-04-25T13:41:32.579922+0000","last_became_peered":"2024-04-25T13:41:32.579922+0000","last_unstale":"2024-04-25T13:41:32.580413+0000","last_undegraded":"2024-04-25T13:41:32.580413+0000","last_fullsized":"2024-04-25T13:41:32.580413+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:33.424560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581800+0000","last_change":"2024-04-25T13:41:32.581800+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581800+0000","last_undegraded":"2024-04-25T13:41:32.581800+0000","last_fullsized":"2024-04-25T13:41:32.581800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.609913+0000","last_change":"2024-04-25T13:41:32.609679+0000","last_active":"2024-04-25T13:41:32.609913+0000","last_peered":"2024-04-25T13:41:32.609913+0000","last_clean":"2024-04-25T13:41:32.609913+0000","last_became_active":"2024-04-25T13:41:32.606623+0000","last_became_peered":"2024-04-25T13:41:32.606623+0000","last_unstale":"2024-04-25T13:41:32.609913+0000","last_undegraded":"2024-04-25T13:41:32.609913+0000","last_fullsized":"2024-04-25T13:41:32.609913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.383684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582909+0000","last_change":"2024-04-25T13:41:32.582909+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582909+0000","last_undegraded":"2024-04-25T13:41:32.582909+0000","last_fullsized":"2024-04-25T13:41:32.582909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590846+0000","last_change":"2024-04-25T13:41:32.590694+0000","last_active":"2024-04-25T13:41:32.590846+0000","last_peered":"2024-04-25T13:41:32.590846+0000","last_clean":"2024-04-25T13:41:32.590846+0000","last_became_active":"2024-04-25T13:41:32.590377+0000","last_became_peered":"2024-04-25T13:41:32.590377+0000","last_unstale":"2024-04-25T13:41:32.590846+0000","last_undegraded":"2024-04-25T13:41:32.590846+0000","last_fullsized":"2024-04-25T13:41:32.590846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:33.114772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582801+0000","last_change":"2024-04-25T13:41:32.582801+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582801+0000","last_undegraded":"2024-04-25T13:41:32.582801+0000","last_fullsized":"2024-04-25T13:41:32.582801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590381+0000","last_change":"2024-04-25T13:41:32.590186+0000","last_active":"2024-04-25T13:41:32.590381+0000","last_peered":"2024-04-25T13:41:32.590381+0000","last_clean":"2024-04-25T13:41:32.590381+0000","last_became_active":"2024-04-25T13:41:32.589854+0000","last_became_peered":"2024-04-25T13:41:32.589854+0000","last_unstale":"2024-04-25T13:41:32.590381+0000","last_undegraded":"2024-04-25T13:41:32.590381+0000","last_fullsized":"2024-04-25T13:41:32.590381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:48.433156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.580608+0000","last_change":"2024-04-25T13:41:32.580608+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.580608+0000","last_undegraded":"2024-04-25T13:41:32.580608+0000","last_fullsized":"2024-04-25T13:41:32.580608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.578994+0000","last_change":"2024-04-25T13:41:32.578797+0000","last_active":"2024-04-25T13:41:32.578994+0000","last_peered":"2024-04-25T13:41:32.578994+0000","last_clean":"2024-04-25T13:41:32.578994+0000","last_became_active":"2024-04-25T13:41:32.578411+0000","last_became_peered":"2024-04-25T13:41:32.578411+0000","last_unstale":"2024-04-25T13:41:32.578994+0000","last_undegraded":"2024-04-25T13:41:32.578994+0000","last_fullsized":"2024-04-25T13:41:32.578994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:46.073052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.600550+0000","last_change":"2024-04-25T13:41:32.600550+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.600550+0000","last_undegraded":"2024-04-25T13:41:32.600550+0000","last_fullsized":"2024-04-25T13:41:32.600550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579352+0000","last_change":"2024-04-25T13:41:32.579069+0000","last_active":"2024-04-25T13:41:32.579352+0000","last_peered":"2024-04-25T13:41:32.579352+0000","last_clean":"2024-04-25T13:41:32.579352+0000","last_became_active":"2024-04-25T13:41:32.578548+0000","last_became_peered":"2024-04-25T13:41:32.578548+0000","last_unstale":"2024-04-25T13:41:32.579352+0000","last_undegraded":"2024-04-25T13:41:32.579352+0000","last_fullsized":"2024-04-25T13:41:32.579352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:28.487849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572005+0000","last_change":"2024-04-25T13:41:32.572005+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572005+0000","last_undegraded":"2024-04-25T13:41:32.572005+0000","last_fullsized":"2024-04-25T13:41:32.572005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.609401+0000","last_change":"2024-04-25T13:41:32.609157+0000","last_active":"2024-04-25T13:41:32.609401+0000","last_peered":"2024-04-25T13:41:32.609401+0000","last_clean":"2024-04-25T13:41:32.609401+0000","last_became_active":"2024-04-25T13:41:32.605222+0000","last_became_peered":"2024-04-25T13:41:32.605222+0000","last_unstale":"2024-04-25T13:41:32.609401+0000","last_undegraded":"2024-04-25T13:41:32.609401+0000","last_fullsized":"2024-04-25T13:41:32.609401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:02.704888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581520+0000","last_change":"2024-04-25T13:41:32.581520+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581520+0000","last_undegraded":"2024-04-25T13:41:32.581520+0000","last_fullsized":"2024-04-25T13:41:32.581520+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.596477+0000","last_change":"2024-04-25T13:41:32.595845+0000","last_active":"2024-04-25T13:41:32.596477+0000","last_peered":"2024-04-25T13:41:32.596477+0000","last_clean":"2024-04-25T13:41:32.596477+0000","last_became_active":"2024-04-25T13:41:32.594815+0000","last_became_peered":"2024-04-25T13:41:32.594815+0000","last_unstale":"2024-04-25T13:41:32.596477+0000","last_undegraded":"2024-04-25T13:41:32.596477+0000","last_fullsized":"2024-04-25T13:41:32.596477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:02.015421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570714+0000","last_change":"2024-04-25T13:41:32.570714+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570714+0000","last_undegraded":"2024-04-25T13:41:32.570714+0000","last_fullsized":"2024-04-25T13:41:32.570714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.602883+0000","last_change":"2024-04-25T13:41:32.602637+0000","last_active":"2024-04-25T13:41:32.602883+0000","last_peered":"2024-04-25T13:41:32.602883+0000","last_clean":"2024-04-25T13:41:32.602883+0000","last_became_active":"2024-04-25T13:41:32.601334+0000","last_became_peered":"2024-04-25T13:41:32.601334+0000","last_unstale":"2024-04-25T13:41:32.602883+0000","last_undegraded":"2024-04-25T13:41:32.602883+0000","last_fullsized":"2024-04-25T13:41:32.602883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:17.292465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569941+0000","last_change":"2024-04-25T13:41:32.569941+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569941+0000","last_undegraded":"2024-04-25T13:41:32.569941+0000","last_fullsized":"2024-04-25T13:41:32.569941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.581327+0000","last_change":"2024-04-25T13:41:32.581137+0000","last_active":"2024-04-25T13:41:32.581327+0000","last_peered":"2024-04-25T13:41:32.581327+0000","last_clean":"2024-04-25T13:41:32.581327+0000","last_became_active":"2024-04-25T13:41:32.580368+0000","last_became_peered":"2024-04-25T13:41:32.580368+0000","last_unstale":"2024-04-25T13:41:32.581327+0000","last_undegraded":"2024-04-25T13:41:32.581327+0000","last_fullsized":"2024-04-25T13:41:32.581327+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:25.222450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570534+0000","last_change":"2024-04-25T13:41:32.570534+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570534+0000","last_undegraded":"2024-04-25T13:41:32.570534+0000","last_fullsized":"2024-04-25T13:41:32.570534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.588429+0000","last_change":"2024-04-25T13:41:32.588053+0000","last_active":"2024-04-25T13:41:32.588429+0000","last_peered":"2024-04-25T13:41:32.588429+0000","last_clean":"2024-04-25T13:41:32.588429+0000","last_became_active":"2024-04-25T13:41:32.587211+0000","last_became_peered":"2024-04-25T13:41:32.587211+0000","last_unstale":"2024-04-25T13:41:32.588429+0000","last_undegraded":"2024-04-25T13:41:32.588429+0000","last_fullsized":"2024-04-25T13:41:32.588429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:56.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569593+0000","last_change":"2024-04-25T13:41:32.569593+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569593+0000","last_undegraded":"2024-04-25T13:41:32.569593+0000","last_fullsized":"2024-04-25T13:41:32.569593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.608840+0000","last_change":"2024-04-25T13:41:32.608569+0000","last_active":"2024-04-25T13:41:32.608840+0000","last_peered":"2024-04-25T13:41:32.608840+0000","last_clean":"2024-04-25T13:41:32.608840+0000","last_became_active":"2024-04-25T13:41:32.604150+0000","last_became_peered":"2024-04-25T13:41:32.604150+0000","last_unstale":"2024-04-25T13:41:32.608840+0000","last_undegraded":"2024-04-25T13:41:32.608840+0000","last_fullsized":"2024-04-25T13:41:32.608840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:49.689785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577294+0000","last_change":"2024-04-25T13:41:32.577294+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577294+0000","last_undegraded":"2024-04-25T13:41:32.577294+0000","last_fullsized":"2024-04-25T13:41:32.577294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.592391+0000","last_change":"2024-04-25T13:41:32.591996+0000","last_active":"2024-04-25T13:41:32.592391+0000","last_peered":"2024-04-25T13:41:32.592391+0000","last_clean":"2024-04-25T13:41:32.592391+0000","last_became_active":"2024-04-25T13:41:32.591149+0000","last_became_peered":"2024-04-25T13:41:32.591149+0000","last_unstale":"2024-04-25T13:41:32.592391+0000","last_undegraded":"2024-04-25T13:41:32.592391+0000","last_fullsized":"2024-04-25T13:41:32.592391+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:47.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569832+0000","last_change":"2024-04-25T13:41:32.569832+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569832+0000","last_undegraded":"2024-04-25T13:41:32.569832+0000","last_fullsized":"2024-04-25T13:41:32.569832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.587494+0000","last_change":"2024-04-25T13:41:32.587338+0000","last_active":"2024-04-25T13:41:32.587494+0000","last_peered":"2024-04-25T13:41:32.587494+0000","last_clean":"2024-04-25T13:41:32.587494+0000","last_became_active":"2024-04-25T13:41:32.587021+0000","last_became_peered":"2024-04-25T13:41:32.587021+0000","last_unstale":"2024-04-25T13:41:32.587494+0000","last_undegraded":"2024-04-25T13:41:32.587494+0000","last_fullsized":"2024-04-25T13:41:32.587494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:02.289278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.574871+0000","last_change":"2024-04-25T13:41:32.574871+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.574871+0000","last_undegraded":"2024-04-25T13:41:32.574871+0000","last_fullsized":"2024-04-25T13:41:32.574871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.581004+0000","last_change":"2024-04-25T13:41:32.580819+0000","last_active":"2024-04-25T13:41:32.581004+0000","last_peered":"2024-04-25T13:41:32.581004+0000","last_clean":"2024-04-25T13:41:32.581004+0000","last_became_active":"2024-04-25T13:41:32.579887+0000","last_became_peered":"2024-04-25T13:41:32.579887+0000","last_unstale":"2024-04-25T13:41:32.581004+0000","last_undegraded":"2024-04-25T13:41:32.581004+0000","last_fullsized":"2024-04-25T13:41:32.581004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:56.380154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576518+0000","last_change":"2024-04-25T13:41:32.576518+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576518+0000","last_undegraded":"2024-04-25T13:41:32.576518+0000","last_fullsized":"2024-04-25T13:41:32.576518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579600+0000","last_change":"2024-04-25T13:41:32.579441+0000","last_active":"2024-04-25T13:41:32.579600+0000","last_peered":"2024-04-25T13:41:32.579600+0000","last_clean":"2024-04-25T13:41:32.579600+0000","last_became_active":"2024-04-25T13:41:32.579097+0000","last_became_peered":"2024-04-25T13:41:32.579097+0000","last_unstale":"2024-04-25T13:41:32.579600+0000","last_undegraded":"2024-04-25T13:41:32.579600+0000","last_fullsized":"2024-04-25T13:41:32.579600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:57.842622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.579544+0000","last_change":"2024-04-25T13:41:32.579544+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.579544+0000","last_undegraded":"2024-04-25T13:41:32.579544+0000","last_fullsized":"2024-04-25T13:41:32.579544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590013+0000","last_change":"2024-04-25T13:41:32.589797+0000","last_active":"2024-04-25T13:41:32.590013+0000","last_peered":"2024-04-25T13:41:32.590013+0000","last_clean":"2024-04-25T13:41:32.590013+0000","last_became_active":"2024-04-25T13:41:32.589337+0000","last_became_peered":"2024-04-25T13:41:32.589337+0000","last_unstale":"2024-04-25T13:41:32.590013+0000","last_undegraded":"2024-04-25T13:41:32.590013+0000","last_fullsized":"2024-04-25T13:41:32.590013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.774287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577739+0000","last_change":"2024-04-25T13:41:32.577739+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577739+0000","last_undegraded":"2024-04-25T13:41:32.577739+0000","last_fullsized":"2024-04-25T13:41:32.577739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590236+0000","last_change":"2024-04-25T13:41:32.590039+0000","last_active":"2024-04-25T13:41:32.590236+0000","last_peered":"2024-04-25T13:41:32.590236+0000","last_clean":"2024-04-25T13:41:32.590236+0000","last_became_active":"2024-04-25T13:41:32.589673+0000","last_became_peered":"2024-04-25T13:41:32.589673+0000","last_unstale":"2024-04-25T13:41:32.590236+0000","last_undegraded":"2024-04-25T13:41:32.590236+0000","last_fullsized":"2024-04-25T13:41:32.590236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:36.047945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583410+0000","last_change":"2024-04-25T13:41:32.583410+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583410+0000","last_undegraded":"2024-04-25T13:41:32.583410+0000","last_fullsized":"2024-04-25T13:41:32.583410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.605796+0000","last_change":"2024-04-25T13:41:32.605521+0000","last_active":"2024-04-25T13:41:32.605796+0000","last_peered":"2024-04-25T13:41:32.605796+0000","last_clean":"2024-04-25T13:41:32.605796+0000","last_became_active":"2024-04-25T13:41:32.603724+0000","last_became_peered":"2024-04-25T13:41:32.603724+0000","last_unstale":"2024-04-25T13:41:32.605796+0000","last_undegraded":"2024-04-25T13:41:32.605796+0000","last_fullsized":"2024-04-25T13:41:32.605796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:22.787359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.596171+0000","last_change":"2024-04-25T13:41:32.596171+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.596171+0000","last_undegraded":"2024-04-25T13:41:32.596171+0000","last_fullsized":"2024-04-25T13:41:32.596171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590930+0000","last_change":"2024-04-25T13:41:32.590768+0000","last_active":"2024-04-25T13:41:32.590930+0000","last_peered":"2024-04-25T13:41:32.590930+0000","last_clean":"2024-04-25T13:41:32.590930+0000","last_became_active":"2024-04-25T13:41:32.589798+0000","last_became_peered":"2024-04-25T13:41:32.589798+0000","last_unstale":"2024-04-25T13:41:32.590930+0000","last_undegraded":"2024-04-25T13:41:32.590930+0000","last_fullsized":"2024-04-25T13:41:32.590930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:15.102750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576040+0000","last_change":"2024-04-25T13:41:32.576040+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576040+0000","last_undegraded":"2024-04-25T13:41:32.576040+0000","last_fullsized":"2024-04-25T13:41:32.576040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.597276+0000","last_change":"2024-04-25T13:41:32.596829+0000","last_active":"2024-04-25T13:41:32.597276+0000","last_peered":"2024-04-25T13:41:32.597276+0000","last_clean":"2024-04-25T13:41:32.597276+0000","last_became_active":"2024-04-25T13:41:32.596092+0000","last_became_peered":"2024-04-25T13:41:32.596092+0000","last_unstale":"2024-04-25T13:41:32.597276+0000","last_undegraded":"2024-04-25T13:41:32.597276+0000","last_fullsized":"2024-04-25T13:41:32.597276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:55.603960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":129,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.585989+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:41:32.585989+0000","last_peered":"2024-04-25T13:41:32.585989+0000","last_clean":"2024-04-25T13:41:32.585989+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:41:32.585989+0000","last_undegraded":"2024-04-25T13:41:32.585989+0000","last_fullsized":"2024-04-25T13:41:32.585989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:41:38.148 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:41:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32294d1640 1 Processor -- start 2024-04-25T13:41:38.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32294d1640 1 -- start start 2024-04-25T13:41:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32294d1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322406e5e0 0x7f322406e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32294d1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32240b0ee0 0x7f32240b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32294d1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32240cca70 con 0x7f3224076cb0 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32294d1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f322406ef30 con 0x7f32240b0ee0 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32294d1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f32240ce4b0 con 0x7f322406e5e0 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32237fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32240b0ee0 0x7f32240b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32237fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32240b0ee0 0x7f32240b12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38740/0 (socket says 172.21.15.80:38740) 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32237fe640 1 -- 172.21.15.80:0/2680198917 learned_addr learned my addr 172.21.15.80:0/2680198917 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32227fc640 1 --2- 172.21.15.80:0/2680198917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322406e5e0 0x7f322406e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32237fe640 1 -- 172.21.15.80:0/2680198917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322406e5e0 msgr2=0x7f322406e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32237fe640 1 --2- 172.21.15.80:0/2680198917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322406e5e0 0x7f322406e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32237fe640 1 -- 172.21.15.80:0/2680198917 >> v1:172.21.15.87:6789/0 conn(0x7f3224076cb0 legacy=0x7f32240d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:38.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.334+0000 7f32237fe640 1 -- 172.21.15.80:0/2680198917 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32240ce790 con 0x7f32240b0ee0 2024-04-25T13:41:38.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32237fe640 1 --2- 172.21.15.80:0/2680198917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32240b0ee0 0x7f32240b12f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f321c009e50 tx=0x7f321c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a2e4ad8a1e38bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:38.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f3221ffb640 1 -- 172.21.15.80:0/2680198917 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0a55f0 con 0x7f32240b0ee0 2024-04-25T13:41:38.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f3221ffb640 1 -- 172.21.15.80:0/2680198917 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f321c0a5790 con 0x7f32240b0ee0 2024-04-25T13:41:38.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f3221ffb640 1 -- 172.21.15.80:0/2680198917 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f321c0a5b60 con 0x7f32240b0ee0 2024-04-25T13:41:38.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32294d1640 1 -- 172.21.15.80:0/2680198917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32240b0ee0 msgr2=0x7f32240b12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32294d1640 1 --2- 172.21.15.80:0/2680198917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32240b0ee0 0x7f32240b12f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f321c009e50 tx=0x7f321c01c960 comp rx=0 tx=0).stop 2024-04-25T13:41:38.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32294d1640 1 -- 172.21.15.80:0/2680198917 shutdown_connections 2024-04-25T13:41:38.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32294d1640 1 --2- 172.21.15.80:0/2680198917 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32240b0ee0 0x7f32240b12f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32294d1640 1 --2- 172.21.15.80:0/2680198917 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f322406e5e0 0x7f322406e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32294d1640 1 -- 172.21.15.80:0/2680198917 >> 172.21.15.80:0/2680198917 conn(0x7f32240d20d0 msgr2=0x7f32240d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:38.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32294d1640 1 -- 172.21.15.80:0/2680198917 shutdown_connections 2024-04-25T13:41:38.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.338+0000 7f32294d1640 1 -- 172.21.15.80:0/2680198917 wait complete. 2024-04-25T13:41:38.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 Processor -- start 2024-04-25T13:41:38.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 -- start start 2024-04-25T13:41:38.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f322406e5e0 0x7f32240c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3224076cb0 0x7f3224152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32240b0ee0 0x7f32241525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f32240ce250 con 0x7f322406e5e0 2024-04-25T13:41:38.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f32240ce0d0 con 0x7f3224076cb0 2024-04-25T13:41:38.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f32241646c0 con 0x7f32240b0ee0 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f322406e5e0 0x7f32240c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f322406e5e0 0x7f32240c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38752/0 (socket says 172.21.15.80:38752) 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 -- 172.21.15.80:0/1916595418 learned_addr learned my addr 172.21.15.80:0/1916595418 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32227fc640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3224076cb0 0x7f3224152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32237fe640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32240b0ee0 0x7f32241525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 -- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32240b0ee0 msgr2=0x7f32241525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32240b0ee0 0x7f32241525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 -- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3224076cb0 msgr2=0x7f3224152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3224076cb0 0x7f3224152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 -- 172.21.15.80:0/1916595418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32240c6120 con 0x7f322406e5e0 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f3222ffd640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f322406e5e0 0x7f32240c5be0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f321400b3f0 tx=0x7f321400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f320b7fe640 1 -- 172.21.15.80:0/1916595418 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214015020 con 0x7f322406e5e0 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f320b7fe640 1 -- 172.21.15.80:0/1916595418 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3214011830 con 0x7f322406e5e0 2024-04-25T13:41:38.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3224153810 con 0x7f322406e5e0 2024-04-25T13:41:38.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f320b7fe640 1 -- 172.21.15.80:0/1916595418 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3214011c00 con 0x7f322406e5e0 2024-04-25T13:41:38.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3224153dd0 con 0x7f322406e5e0 2024-04-25T13:41:38.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.342+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f322406a3a0 con 0x7f322406e5e0 2024-04-25T13:41:38.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.346+0000 7f320b7fe640 1 -- 172.21.15.80:0/1916595418 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3214018020 con 0x7f322406e5e0 2024-04-25T13:41:38.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.346+0000 7f320b7fe640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f321006f400 0x7f3210071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.346+0000 7f320b7fe640 1 -- 172.21.15.80:0/1916595418 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f321408edf0 con 0x7f322406e5e0 2024-04-25T13:41:38.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.346+0000 7f32227fc640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f321006f400 0x7f3210071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.346+0000 7f32227fc640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f321006f400 0x7f3210071910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f32240b0340 tx=0x7f3218008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:38.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.350+0000 7f320b7fe640 1 -- 172.21.15.80:0/1916595418 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f321401f030 con 0x7f322406e5e0 2024-04-25T13:41:38.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.510+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32240d1a80 con 0x7f321006f400 2024-04-25T13:41:38.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.530+0000 7f320b7fe640 1 -- 172.21.15.80:0/1916595418 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242206 (secure 0 0 0) 0x7f32240d1a80 con 0x7f321006f400 2024-04-25T13:41:38.535 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:38.536 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f321006f400 msgr2=0x7f3210071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f321006f400 0x7f3210071910 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f32240b0340 tx=0x7f3218008040 comp rx=0 tx=0).stop 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f322406e5e0 msgr2=0x7f32240c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f322406e5e0 0x7f32240c5be0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f321400b3f0 tx=0x7f321400b8b0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 shutdown_connections 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f321006f400 0x7f3210071910 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32240b0ee0 0x7f32241525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3224076cb0 0x7f3224152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 --2- 172.21.15.80:0/1916595418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f322406e5e0 0x7f32240c5be0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 >> 172.21.15.80:0/1916595418 conn(0x7f32240d20d0 msgr2=0x7f32240d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 shutdown_connections 2024-04-25T13:41:38.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.534+0000 7f32294d1640 1 -- 172.21.15.80:0/1916595418 wait complete. 2024-04-25T13:41:38.555 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-25T13:41:38.117264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2176,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560323,"statfs":{"total":1600000000,"available":1597771017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001925"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583136+0000","last_change":"2024-04-25T13:41:32.583136+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583136+0000","last_undegraded":"2024-04-25T13:41:32.583136+0000","last_fullsized":"2024-04-25T13:41:32.583136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.586323+0000","last_change":"2024-04-25T13:41:32.586323+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.586323+0000","last_undegraded":"2024-04-25T13:41:32.586323+0000","last_fullsized":"2024-04-25T13:41:32.586323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573531+0000","last_change":"2024-04-25T13:41:32.573531+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573531+0000","last_undegraded":"2024-04-25T13:41:32.573531+0000","last_fullsized":"2024-04-25T13:41:32.573531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583817+0000","last_change":"2024-04-25T13:41:32.583817+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583817+0000","last_undegraded":"2024-04-25T13:41:32.583817+0000","last_fullsized":"2024-04-25T13:41:32.583817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.571611+0000","last_change":"2024-04-25T13:41:32.571611+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.571611+0000","last_undegraded":"2024-04-25T13:41:32.571611+0000","last_fullsized":"2024-04-25T13:41:32.571611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585490+0000","last_change":"2024-04-25T13:41:32.585490+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585490+0000","last_undegraded":"2024-04-25T13:41:32.585490+0000","last_fullsized":"2024-04-25T13:41:32.585490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576011+0000","last_change":"2024-04-25T13:41:32.576011+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576011+0000","last_undegraded":"2024-04-25T13:41:32.576011+0000","last_fullsized":"2024-04-25T13:41:32.576011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585019+0000","last_change":"2024-04-25T13:41:32.585019+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585019+0000","last_undegraded":"2024-04-25T13:41:32.585019+0000","last_fullsized":"2024-04-25T13:41:32.585019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576129+0000","last_change":"2024-04-25T13:41:32.576129+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576129+0000","last_undegraded":"2024-04-25T13:41:32.576129+0000","last_fullsized":"2024-04-25T13:41:32.576129+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583916+0000","last_change":"2024-04-25T13:41:32.583916+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583916+0000","last_undegraded":"2024-04-25T13:41:32.583916+0000","last_fullsized":"2024-04-25T13:41:32.583916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.579893+0000","last_change":"2024-04-25T13:41:32.579893+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.579893+0000","last_undegraded":"2024-04-25T13:41:32.579893+0000","last_fullsized":"2024-04-25T13:41:32.579893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581018+0000","last_change":"2024-04-25T13:41:32.581018+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581018+0000","last_undegraded":"2024-04-25T13:41:32.581018+0000","last_fullsized":"2024-04-25T13:41:32.581018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.571707+0000","last_change":"2024-04-25T13:41:32.571707+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.571707+0000","last_undegraded":"2024-04-25T13:41:32.571707+0000","last_fullsized":"2024-04-25T13:41:32.571707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585982+0000","last_change":"2024-04-25T13:41:32.585982+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585982+0000","last_undegraded":"2024-04-25T13:41:32.585982+0000","last_fullsized":"2024-04-25T13:41:32.585982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572059+0000","last_change":"2024-04-25T13:41:32.572059+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572059+0000","last_undegraded":"2024-04-25T13:41:32.572059+0000","last_fullsized":"2024-04-25T13:41:32.572059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578159+0000","last_change":"2024-04-25T13:41:32.578159+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578159+0000","last_undegraded":"2024-04-25T13:41:32.578159+0000","last_fullsized":"2024-04-25T13:41:32.578159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.580079+0000","last_change":"2024-04-25T13:41:32.580079+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.580079+0000","last_undegraded":"2024-04-25T13:41:32.580079+0000","last_fullsized":"2024-04-25T13:41:32.580079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585445+0000","last_change":"2024-04-25T13:41:32.585445+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585445+0000","last_undegraded":"2024-04-25T13:41:32.585445+0000","last_fullsized":"2024-04-25T13:41:32.585445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585538+0000","last_change":"2024-04-25T13:41:32.585538+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585538+0000","last_undegraded":"2024-04-25T13:41:32.585538+0000","last_fullsized":"2024-04-25T13:41:32.585538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570881+0000","last_change":"2024-04-25T13:41:32.570881+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570881+0000","last_undegraded":"2024-04-25T13:41:32.570881+0000","last_fullsized":"2024-04-25T13:41:32.570881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.574825+0000","last_change":"2024-04-25T13:41:32.574825+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.574825+0000","last_undegraded":"2024-04-25T13:41:32.574825+0000","last_fullsized":"2024-04-25T13:41:32.574825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.589854+0000","last_change":"2024-04-25T13:41:32.589854+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.589854+0000","last_undegraded":"2024-04-25T13:41:32.589854+0000","last_fullsized":"2024-04-25T13:41:32.589854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577095+0000","last_change":"2024-04-25T13:41:32.577095+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577095+0000","last_undegraded":"2024-04-25T13:41:32.577095+0000","last_fullsized":"2024-04-25T13:41:32.577095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573074+0000","last_change":"2024-04-25T13:41:32.573074+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573074+0000","last_undegraded":"2024-04-25T13:41:32.573074+0000","last_fullsized":"2024-04-25T13:41:32.573074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583161+0000","last_change":"2024-04-25T13:41:32.583161+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583161+0000","last_undegraded":"2024-04-25T13:41:32.583161+0000","last_fullsized":"2024-04-25T13:41:32.583161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584676+0000","last_change":"2024-04-25T13:41:32.584676+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584676+0000","last_undegraded":"2024-04-25T13:41:32.584676+0000","last_fullsized":"2024-04-25T13:41:32.584676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582937+0000","last_change":"2024-04-25T13:41:32.582937+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582937+0000","last_undegraded":"2024-04-25T13:41:32.582937+0000","last_fullsized":"2024-04-25T13:41:32.582937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577565+0000","last_change":"2024-04-25T13:41:32.577565+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577565+0000","last_undegraded":"2024-04-25T13:41:32.577565+0000","last_fullsized":"2024-04-25T13:41:32.577565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.575137+0000","last_change":"2024-04-25T13:41:32.575137+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.575137+0000","last_undegraded":"2024-04-25T13:41:32.575137+0000","last_fullsized":"2024-04-25T13:41:32.575137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578736+0000","last_change":"2024-04-25T13:41:32.578736+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578736+0000","last_undegraded":"2024-04-25T13:41:32.578736+0000","last_fullsized":"2024-04-25T13:41:32.578736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582191+0000","last_change":"2024-04-25T13:41:32.582191+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582191+0000","last_undegraded":"2024-04-25T13:41:32.582191+0000","last_fullsized":"2024-04-25T13:41:32.582191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577621+0000","last_change":"2024-04-25T13:41:32.577621+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577621+0000","last_undegraded":"2024-04-25T13:41:32.577621+0000","last_fullsized":"2024-04-25T13:41:32.577621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577142+0000","last_change":"2024-04-25T13:41:32.577142+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577142+0000","last_undegraded":"2024-04-25T13:41:32.577142+0000","last_fullsized":"2024-04-25T13:41:32.577142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591696+0000","last_change":"2024-04-25T13:41:32.591289+0000","last_active":"2024-04-25T13:41:32.591696+0000","last_peered":"2024-04-25T13:41:32.591696+0000","last_clean":"2024-04-25T13:41:32.591696+0000","last_became_active":"2024-04-25T13:41:32.590480+0000","last_became_peered":"2024-04-25T13:41:32.590480+0000","last_unstale":"2024-04-25T13:41:32.591696+0000","last_undegraded":"2024-04-25T13:41:32.591696+0000","last_fullsized":"2024-04-25T13:41:32.591696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:43.229224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578440+0000","last_change":"2024-04-25T13:41:32.578440+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578440+0000","last_undegraded":"2024-04-25T13:41:32.578440+0000","last_fullsized":"2024-04-25T13:41:32.578440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590389+0000","last_change":"2024-04-25T13:41:32.590216+0000","last_active":"2024-04-25T13:41:32.590389+0000","last_peered":"2024-04-25T13:41:32.590389+0000","last_clean":"2024-04-25T13:41:32.590389+0000","last_became_active":"2024-04-25T13:41:32.589855+0000","last_became_peered":"2024-04-25T13:41:32.589855+0000","last_unstale":"2024-04-25T13:41:32.590389+0000","last_undegraded":"2024-04-25T13:41:32.590389+0000","last_fullsized":"2024-04-25T13:41:32.590389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:29.322913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573601+0000","last_change":"2024-04-25T13:41:32.573601+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573601+0000","last_undegraded":"2024-04-25T13:41:32.573601+0000","last_fullsized":"2024-04-25T13:41:32.573601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579266+0000","last_change":"2024-04-25T13:41:32.578789+0000","last_active":"2024-04-25T13:41:32.579266+0000","last_peered":"2024-04-25T13:41:32.579266+0000","last_clean":"2024-04-25T13:41:32.579266+0000","last_became_active":"2024-04-25T13:41:32.578278+0000","last_became_peered":"2024-04-25T13:41:32.578278+0000","last_unstale":"2024-04-25T13:41:32.579266+0000","last_undegraded":"2024-04-25T13:41:32.579266+0000","last_fullsized":"2024-04-25T13:41:32.579266+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:04.613101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582460+0000","last_change":"2024-04-25T13:41:32.582460+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582460+0000","last_undegraded":"2024-04-25T13:41:32.582460+0000","last_fullsized":"2024-04-25T13:41:32.582460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.577933+0000","last_change":"2024-04-25T13:41:32.577475+0000","last_active":"2024-04-25T13:41:32.577933+0000","last_peered":"2024-04-25T13:41:32.577933+0000","last_clean":"2024-04-25T13:41:32.577933+0000","last_became_active":"2024-04-25T13:41:32.576661+0000","last_became_peered":"2024-04-25T13:41:32.576661+0000","last_unstale":"2024-04-25T13:41:32.577933+0000","last_undegraded":"2024-04-25T13:41:32.577933+0000","last_fullsized":"2024-04-25T13:41:32.577933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:23.408800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583853+0000","last_change":"2024-04-25T13:41:32.583853+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583853+0000","last_undegraded":"2024-04-25T13:41:32.583853+0000","last_fullsized":"2024-04-25T13:41:32.583853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.603280+0000","last_change":"2024-04-25T13:41:32.602990+0000","last_active":"2024-04-25T13:41:32.603280+0000","last_peered":"2024-04-25T13:41:32.603280+0000","last_clean":"2024-04-25T13:41:32.603280+0000","last_became_active":"2024-04-25T13:41:32.602174+0000","last_became_peered":"2024-04-25T13:41:32.602174+0000","last_unstale":"2024-04-25T13:41:32.603280+0000","last_undegraded":"2024-04-25T13:41:32.603280+0000","last_fullsized":"2024-04-25T13:41:32.603280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:20.977995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.575940+0000","last_change":"2024-04-25T13:41:32.575940+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.575940+0000","last_undegraded":"2024-04-25T13:41:32.575940+0000","last_fullsized":"2024-04-25T13:41:32.575940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.606395+0000","last_change":"2024-04-25T13:41:32.606199+0000","last_active":"2024-04-25T13:41:32.606395+0000","last_peered":"2024-04-25T13:41:32.606395+0000","last_clean":"2024-04-25T13:41:32.606395+0000","last_became_active":"2024-04-25T13:41:32.604933+0000","last_became_peered":"2024-04-25T13:41:32.604933+0000","last_unstale":"2024-04-25T13:41:32.606395+0000","last_undegraded":"2024-04-25T13:41:32.606395+0000","last_fullsized":"2024-04-25T13:41:32.606395+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:50.753222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.602644+0000","last_change":"2024-04-25T13:41:32.602644+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.602644+0000","last_undegraded":"2024-04-25T13:41:32.602644+0000","last_fullsized":"2024-04-25T13:41:32.602644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.586454+0000","last_change":"2024-04-25T13:41:32.586286+0000","last_active":"2024-04-25T13:41:32.586454+0000","last_peered":"2024-04-25T13:41:32.586454+0000","last_clean":"2024-04-25T13:41:32.586454+0000","last_became_active":"2024-04-25T13:41:32.584859+0000","last_became_peered":"2024-04-25T13:41:32.584859+0000","last_unstale":"2024-04-25T13:41:32.586454+0000","last_undegraded":"2024-04-25T13:41:32.586454+0000","last_fullsized":"2024-04-25T13:41:32.586454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:21.284984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581729+0000","last_change":"2024-04-25T13:41:32.581729+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581729+0000","last_undegraded":"2024-04-25T13:41:32.581729+0000","last_fullsized":"2024-04-25T13:41:32.581729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.610432+0000","last_change":"2024-04-25T13:41:32.610199+0000","last_active":"2024-04-25T13:41:32.610432+0000","last_peered":"2024-04-25T13:41:32.610432+0000","last_clean":"2024-04-25T13:41:32.610432+0000","last_became_active":"2024-04-25T13:41:32.607495+0000","last_became_peered":"2024-04-25T13:41:32.607495+0000","last_unstale":"2024-04-25T13:41:32.610432+0000","last_undegraded":"2024-04-25T13:41:32.610432+0000","last_fullsized":"2024-04-25T13:41:32.610432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:34.040738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572876+0000","last_change":"2024-04-25T13:41:32.572876+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572876+0000","last_undegraded":"2024-04-25T13:41:32.572876+0000","last_fullsized":"2024-04-25T13:41:32.572876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591561+0000","last_change":"2024-04-25T13:41:32.591386+0000","last_active":"2024-04-25T13:41:32.591561+0000","last_peered":"2024-04-25T13:41:32.591561+0000","last_clean":"2024-04-25T13:41:32.591561+0000","last_became_active":"2024-04-25T13:41:32.591004+0000","last_became_peered":"2024-04-25T13:41:32.591004+0000","last_unstale":"2024-04-25T13:41:32.591561+0000","last_undegraded":"2024-04-25T13:41:32.591561+0000","last_fullsized":"2024-04-25T13:41:32.591561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:55.459670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584067+0000","last_change":"2024-04-25T13:41:32.584067+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584067+0000","last_undegraded":"2024-04-25T13:41:32.584067+0000","last_fullsized":"2024-04-25T13:41:32.584067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.577987+0000","last_change":"2024-04-25T13:41:32.577747+0000","last_active":"2024-04-25T13:41:32.577987+0000","last_peered":"2024-04-25T13:41:32.577987+0000","last_clean":"2024-04-25T13:41:32.577987+0000","last_became_active":"2024-04-25T13:41:32.577358+0000","last_became_peered":"2024-04-25T13:41:32.577358+0000","last_unstale":"2024-04-25T13:41:32.577987+0000","last_undegraded":"2024-04-25T13:41:32.577987+0000","last_fullsized":"2024-04-25T13:41:32.577987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:06.339192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584147+0000","last_change":"2024-04-25T13:41:32.584147+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584147+0000","last_undegraded":"2024-04-25T13:41:32.584147+0000","last_fullsized":"2024-04-25T13:41:32.584147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591771+0000","last_change":"2024-04-25T13:41:32.591430+0000","last_active":"2024-04-25T13:41:32.591771+0000","last_peered":"2024-04-25T13:41:32.591771+0000","last_clean":"2024-04-25T13:41:32.591771+0000","last_became_active":"2024-04-25T13:41:32.590686+0000","last_became_peered":"2024-04-25T13:41:32.590686+0000","last_unstale":"2024-04-25T13:41:32.591771+0000","last_undegraded":"2024-04-25T13:41:32.591771+0000","last_fullsized":"2024-04-25T13:41:32.591771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.287292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576542+0000","last_change":"2024-04-25T13:41:32.576542+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576542+0000","last_undegraded":"2024-04-25T13:41:32.576542+0000","last_fullsized":"2024-04-25T13:41:32.576542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.580413+0000","last_change":"2024-04-25T13:41:32.580227+0000","last_active":"2024-04-25T13:41:32.580413+0000","last_peered":"2024-04-25T13:41:32.580413+0000","last_clean":"2024-04-25T13:41:32.580413+0000","last_became_active":"2024-04-25T13:41:32.579922+0000","last_became_peered":"2024-04-25T13:41:32.579922+0000","last_unstale":"2024-04-25T13:41:32.580413+0000","last_undegraded":"2024-04-25T13:41:32.580413+0000","last_fullsized":"2024-04-25T13:41:32.580413+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:33.424560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581800+0000","last_change":"2024-04-25T13:41:32.581800+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581800+0000","last_undegraded":"2024-04-25T13:41:32.581800+0000","last_fullsized":"2024-04-25T13:41:32.581800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.609913+0000","last_change":"2024-04-25T13:41:32.609679+0000","last_active":"2024-04-25T13:41:32.609913+0000","last_peered":"2024-04-25T13:41:32.609913+0000","last_clean":"2024-04-25T13:41:32.609913+0000","last_became_active":"2024-04-25T13:41:32.606623+0000","last_became_peered":"2024-04-25T13:41:32.606623+0000","last_unstale":"2024-04-25T13:41:32.609913+0000","last_undegraded":"2024-04-25T13:41:32.609913+0000","last_fullsized":"2024-04-25T13:41:32.609913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.383684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582909+0000","last_change":"2024-04-25T13:41:32.582909+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582909+0000","last_undegraded":"2024-04-25T13:41:32.582909+0000","last_fullsized":"2024-04-25T13:41:32.582909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590846+0000","last_change":"2024-04-25T13:41:32.590694+0000","last_active":"2024-04-25T13:41:32.590846+0000","last_peered":"2024-04-25T13:41:32.590846+0000","last_clean":"2024-04-25T13:41:32.590846+0000","last_became_active":"2024-04-25T13:41:32.590377+0000","last_became_peered":"2024-04-25T13:41:32.590377+0000","last_unstale":"2024-04-25T13:41:32.590846+0000","last_undegraded":"2024-04-25T13:41:32.590846+0000","last_fullsized":"2024-04-25T13:41:32.590846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:33.114772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582801+0000","last_change":"2024-04-25T13:41:32.582801+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582801+0000","last_undegraded":"2024-04-25T13:41:32.582801+0000","last_fullsized":"2024-04-25T13:41:32.582801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590381+0000","last_change":"2024-04-25T13:41:32.590186+0000","last_active":"2024-04-25T13:41:32.590381+0000","last_peered":"2024-04-25T13:41:32.590381+0000","last_clean":"2024-04-25T13:41:32.590381+0000","last_became_active":"2024-04-25T13:41:32.589854+0000","last_became_peered":"2024-04-25T13:41:32.589854+0000","last_unstale":"2024-04-25T13:41:32.590381+0000","last_undegraded":"2024-04-25T13:41:32.590381+0000","last_fullsized":"2024-04-25T13:41:32.590381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:48.433156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.580608+0000","last_change":"2024-04-25T13:41:32.580608+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.580608+0000","last_undegraded":"2024-04-25T13:41:32.580608+0000","last_fullsized":"2024-04-25T13:41:32.580608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.578994+0000","last_change":"2024-04-25T13:41:32.578797+0000","last_active":"2024-04-25T13:41:32.578994+0000","last_peered":"2024-04-25T13:41:32.578994+0000","last_clean":"2024-04-25T13:41:32.578994+0000","last_became_active":"2024-04-25T13:41:32.578411+0000","last_became_peered":"2024-04-25T13:41:32.578411+0000","last_unstale":"2024-04-25T13:41:32.578994+0000","last_undegraded":"2024-04-25T13:41:32.578994+0000","last_fullsized":"2024-04-25T13:41:32.578994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:46.073052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.600550+0000","last_change":"2024-04-25T13:41:32.600550+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.600550+0000","last_undegraded":"2024-04-25T13:41:32.600550+0000","last_fullsized":"2024-04-25T13:41:32.600550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579352+0000","last_change":"2024-04-25T13:41:32.579069+0000","last_active":"2024-04-25T13:41:32.579352+0000","last_peered":"2024-04-25T13:41:32.579352+0000","last_clean":"2024-04-25T13:41:32.579352+0000","last_became_active":"2024-04-25T13:41:32.578548+0000","last_became_peered":"2024-04-25T13:41:32.578548+0000","last_unstale":"2024-04-25T13:41:32.579352+0000","last_undegraded":"2024-04-25T13:41:32.579352+0000","last_fullsized":"2024-04-25T13:41:32.579352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:28.487849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572005+0000","last_change":"2024-04-25T13:41:32.572005+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572005+0000","last_undegraded":"2024-04-25T13:41:32.572005+0000","last_fullsized":"2024-04-25T13:41:32.572005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.609401+0000","last_change":"2024-04-25T13:41:32.609157+0000","last_active":"2024-04-25T13:41:32.609401+0000","last_peered":"2024-04-25T13:41:32.609401+0000","last_clean":"2024-04-25T13:41:32.609401+0000","last_became_active":"2024-04-25T13:41:32.605222+0000","last_became_peered":"2024-04-25T13:41:32.605222+0000","last_unstale":"2024-04-25T13:41:32.609401+0000","last_undegraded":"2024-04-25T13:41:32.609401+0000","last_fullsized":"2024-04-25T13:41:32.609401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:02.704888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581520+0000","last_change":"2024-04-25T13:41:32.581520+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581520+0000","last_undegraded":"2024-04-25T13:41:32.581520+0000","last_fullsized":"2024-04-25T13:41:32.581520+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.596477+0000","last_change":"2024-04-25T13:41:32.595845+0000","last_active":"2024-04-25T13:41:32.596477+0000","last_peered":"2024-04-25T13:41:32.596477+0000","last_clean":"2024-04-25T13:41:32.596477+0000","last_became_active":"2024-04-25T13:41:32.594815+0000","last_became_peered":"2024-04-25T13:41:32.594815+0000","last_unstale":"2024-04-25T13:41:32.596477+0000","last_undegraded":"2024-04-25T13:41:32.596477+0000","last_fullsized":"2024-04-25T13:41:32.596477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:02.015421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570714+0000","last_change":"2024-04-25T13:41:32.570714+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570714+0000","last_undegraded":"2024-04-25T13:41:32.570714+0000","last_fullsized":"2024-04-25T13:41:32.570714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.602883+0000","last_change":"2024-04-25T13:41:32.602637+0000","last_active":"2024-04-25T13:41:32.602883+0000","last_peered":"2024-04-25T13:41:32.602883+0000","last_clean":"2024-04-25T13:41:32.602883+0000","last_became_active":"2024-04-25T13:41:32.601334+0000","last_became_peered":"2024-04-25T13:41:32.601334+0000","last_unstale":"2024-04-25T13:41:32.602883+0000","last_undegraded":"2024-04-25T13:41:32.602883+0000","last_fullsized":"2024-04-25T13:41:32.602883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:17.292465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569941+0000","last_change":"2024-04-25T13:41:32.569941+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569941+0000","last_undegraded":"2024-04-25T13:41:32.569941+0000","last_fullsized":"2024-04-25T13:41:32.569941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.581327+0000","last_change":"2024-04-25T13:41:32.581137+0000","last_active":"2024-04-25T13:41:32.581327+0000","last_peered":"2024-04-25T13:41:32.581327+0000","last_clean":"2024-04-25T13:41:32.581327+0000","last_became_active":"2024-04-25T13:41:32.580368+0000","last_became_peered":"2024-04-25T13:41:32.580368+0000","last_unstale":"2024-04-25T13:41:32.581327+0000","last_undegraded":"2024-04-25T13:41:32.581327+0000","last_fullsized":"2024-04-25T13:41:32.581327+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:25.222450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570534+0000","last_change":"2024-04-25T13:41:32.570534+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570534+0000","last_undegraded":"2024-04-25T13:41:32.570534+0000","last_fullsized":"2024-04-25T13:41:32.570534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.588429+0000","last_change":"2024-04-25T13:41:32.588053+0000","last_active":"2024-04-25T13:41:32.588429+0000","last_peered":"2024-04-25T13:41:32.588429+0000","last_clean":"2024-04-25T13:41:32.588429+0000","last_became_active":"2024-04-25T13:41:32.587211+0000","last_became_peered":"2024-04-25T13:41:32.587211+0000","last_unstale":"2024-04-25T13:41:32.588429+0000","last_undegraded":"2024-04-25T13:41:32.588429+0000","last_fullsized":"2024-04-25T13:41:32.588429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:56.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569593+0000","last_change":"2024-04-25T13:41:32.569593+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569593+0000","last_undegraded":"2024-04-25T13:41:32.569593+0000","last_fullsized":"2024-04-25T13:41:32.569593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.608840+0000","last_change":"2024-04-25T13:41:32.608569+0000","last_active":"2024-04-25T13:41:32.608840+0000","last_peered":"2024-04-25T13:41:32.608840+0000","last_clean":"2024-04-25T13:41:32.608840+0000","last_became_active":"2024-04-25T13:41:32.604150+0000","last_became_peered":"2024-04-25T13:41:32.604150+0000","last_unstale":"2024-04-25T13:41:32.608840+0000","last_undegraded":"2024-04-25T13:41:32.608840+0000","last_fullsized":"2024-04-25T13:41:32.608840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:49.689785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577294+0000","last_change":"2024-04-25T13:41:32.577294+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577294+0000","last_undegraded":"2024-04-25T13:41:32.577294+0000","last_fullsized":"2024-04-25T13:41:32.577294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.592391+0000","last_change":"2024-04-25T13:41:32.591996+0000","last_active":"2024-04-25T13:41:32.592391+0000","last_peered":"2024-04-25T13:41:32.592391+0000","last_clean":"2024-04-25T13:41:32.592391+0000","last_became_active":"2024-04-25T13:41:32.591149+0000","last_became_peered":"2024-04-25T13:41:32.591149+0000","last_unstale":"2024-04-25T13:41:32.592391+0000","last_undegraded":"2024-04-25T13:41:32.592391+0000","last_fullsized":"2024-04-25T13:41:32.592391+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:47.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569832+0000","last_change":"2024-04-25T13:41:32.569832+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569832+0000","last_undegraded":"2024-04-25T13:41:32.569832+0000","last_fullsized":"2024-04-25T13:41:32.569832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.587494+0000","last_change":"2024-04-25T13:41:32.587338+0000","last_active":"2024-04-25T13:41:32.587494+0000","last_peered":"2024-04-25T13:41:32.587494+0000","last_clean":"2024-04-25T13:41:32.587494+0000","last_became_active":"2024-04-25T13:41:32.587021+0000","last_became_peered":"2024-04-25T13:41:32.587021+0000","last_unstale":"2024-04-25T13:41:32.587494+0000","last_undegraded":"2024-04-25T13:41:32.587494+0000","last_fullsized":"2024-04-25T13:41:32.587494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:02.289278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.574871+0000","last_change":"2024-04-25T13:41:32.574871+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.574871+0000","last_undegraded":"2024-04-25T13:41:32.574871+0000","last_fullsized":"2024-04-25T13:41:32.574871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.581004+0000","last_change":"2024-04-25T13:41:32.580819+0000","last_active":"2024-04-25T13:41:32.581004+0000","last_peered":"2024-04-25T13:41:32.581004+0000","last_clean":"2024-04-25T13:41:32.581004+0000","last_became_active":"2024-04-25T13:41:32.579887+0000","last_became_peered":"2024-04-25T13:41:32.579887+0000","last_unstale":"2024-04-25T13:41:32.581004+0000","last_undegraded":"2024-04-25T13:41:32.581004+0000","last_fullsized":"2024-04-25T13:41:32.581004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:56.380154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576518+0000","last_change":"2024-04-25T13:41:32.576518+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576518+0000","last_undegraded":"2024-04-25T13:41:32.576518+0000","last_fullsized":"2024-04-25T13:41:32.576518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579600+0000","last_change":"2024-04-25T13:41:32.579441+0000","last_active":"2024-04-25T13:41:32.579600+0000","last_peered":"2024-04-25T13:41:32.579600+0000","last_clean":"2024-04-25T13:41:32.579600+0000","last_became_active":"2024-04-25T13:41:32.579097+0000","last_became_peered":"2024-04-25T13:41:32.579097+0000","last_unstale":"2024-04-25T13:41:32.579600+0000","last_undegraded":"2024-04-25T13:41:32.579600+0000","last_fullsized":"2024-04-25T13:41:32.579600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:57.842622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.579544+0000","last_change":"2024-04-25T13:41:32.579544+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.579544+0000","last_undegraded":"2024-04-25T13:41:32.579544+0000","last_fullsized":"2024-04-25T13:41:32.579544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590013+0000","last_change":"2024-04-25T13:41:32.589797+0000","last_active":"2024-04-25T13:41:32.590013+0000","last_peered":"2024-04-25T13:41:32.590013+0000","last_clean":"2024-04-25T13:41:32.590013+0000","last_became_active":"2024-04-25T13:41:32.589337+0000","last_became_peered":"2024-04-25T13:41:32.589337+0000","last_unstale":"2024-04-25T13:41:32.590013+0000","last_undegraded":"2024-04-25T13:41:32.590013+0000","last_fullsized":"2024-04-25T13:41:32.590013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.774287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577739+0000","last_change":"2024-04-25T13:41:32.577739+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577739+0000","last_undegraded":"2024-04-25T13:41:32.577739+0000","last_fullsized":"2024-04-25T13:41:32.577739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590236+0000","last_change":"2024-04-25T13:41:32.590039+0000","last_active":"2024-04-25T13:41:32.590236+0000","last_peered":"2024-04-25T13:41:32.590236+0000","last_clean":"2024-04-25T13:41:32.590236+0000","last_became_active":"2024-04-25T13:41:32.589673+0000","last_became_peered":"2024-04-25T13:41:32.589673+0000","last_unstale":"2024-04-25T13:41:32.590236+0000","last_undegraded":"2024-04-25T13:41:32.590236+0000","last_fullsized":"2024-04-25T13:41:32.590236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:36.047945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583410+0000","last_change":"2024-04-25T13:41:32.583410+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583410+0000","last_undegraded":"2024-04-25T13:41:32.583410+0000","last_fullsized":"2024-04-25T13:41:32.583410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.605796+0000","last_change":"2024-04-25T13:41:32.605521+0000","last_active":"2024-04-25T13:41:32.605796+0000","last_peered":"2024-04-25T13:41:32.605796+0000","last_clean":"2024-04-25T13:41:32.605796+0000","last_became_active":"2024-04-25T13:41:32.603724+0000","last_became_peered":"2024-04-25T13:41:32.603724+0000","last_unstale":"2024-04-25T13:41:32.605796+0000","last_undegraded":"2024-04-25T13:41:32.605796+0000","last_fullsized":"2024-04-25T13:41:32.605796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:22.787359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.596171+0000","last_change":"2024-04-25T13:41:32.596171+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.596171+0000","last_undegraded":"2024-04-25T13:41:32.596171+0000","last_fullsized":"2024-04-25T13:41:32.596171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590930+0000","last_change":"2024-04-25T13:41:32.590768+0000","last_active":"2024-04-25T13:41:32.590930+0000","last_peered":"2024-04-25T13:41:32.590930+0000","last_clean":"2024-04-25T13:41:32.590930+0000","last_became_active":"2024-04-25T13:41:32.589798+0000","last_became_peered":"2024-04-25T13:41:32.589798+0000","last_unstale":"2024-04-25T13:41:32.590930+0000","last_undegraded":"2024-04-25T13:41:32.590930+0000","last_fullsized":"2024-04-25T13:41:32.590930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:15.102750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576040+0000","last_change":"2024-04-25T13:41:32.576040+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576040+0000","last_undegraded":"2024-04-25T13:41:32.576040+0000","last_fullsized":"2024-04-25T13:41:32.576040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.597276+0000","last_change":"2024-04-25T13:41:32.596829+0000","last_active":"2024-04-25T13:41:32.597276+0000","last_peered":"2024-04-25T13:41:32.597276+0000","last_clean":"2024-04-25T13:41:32.597276+0000","last_became_active":"2024-04-25T13:41:32.596092+0000","last_became_peered":"2024-04-25T13:41:32.596092+0000","last_unstale":"2024-04-25T13:41:32.597276+0000","last_undegraded":"2024-04-25T13:41:32.597276+0000","last_fullsized":"2024-04-25T13:41:32.597276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:55.603960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":129,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.585989+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:41:32.585989+0000","last_peered":"2024-04-25T13:41:32.585989+0000","last_clean":"2024-04-25T13:41:32.585989+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:41:32.585989+0000","last_undegraded":"2024-04-25T13:41:32.585989+0000","last_fullsized":"2024-04-25T13:41:32.585989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:41:38.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2024-04-25T13:41:38.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.583136+0000', 'last_change': '2024-04-25T13:41:32.583136+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.583136+0000', 'last_undegraded': '2024-04-25T13:41:32.583136+0000', 'last_fullsized': '2024-04-25T13:41:32.583136+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.586323+0000', 'last_change': '2024-04-25T13:41:32.586323+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.586323+0000', 'last_undegraded': '2024-04-25T13:41:32.586323+0000', 'last_fullsized': '2024-04-25T13:41:32.586323+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.573531+0000', 'last_change': '2024-04-25T13:41:32.573531+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.573531+0000', 'last_undegraded': '2024-04-25T13:41:32.573531+0000', 'last_fullsized': '2024-04-25T13:41:32.573531+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.583817+0000', 'last_change': '2024-04-25T13:41:32.583817+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.583817+0000', 'last_undegraded': '2024-04-25T13:41:32.583817+0000', 'last_fullsized': '2024-04-25T13:41:32.583817+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.571611+0000', 'last_change': '2024-04-25T13:41:32.571611+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.571611+0000', 'last_undegraded': '2024-04-25T13:41:32.571611+0000', 'last_fullsized': '2024-04-25T13:41:32.571611+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2024-04-25T13:41:38.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.585490+0000', 'last_change': '2024-04-25T13:41:32.585490+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.585490+0000', 'last_undegraded': '2024-04-25T13:41:32.585490+0000', 'last_fullsized': '2024-04-25T13:41:32.585490+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.576011+0000', 'last_change': '2024-04-25T13:41:32.576011+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.576011+0000', 'last_undegraded': '2024-04-25T13:41:32.576011+0000', 'last_fullsized': '2024-04-25T13:41:32.576011+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.585019+0000', 'last_change': '2024-04-25T13:41:32.585019+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.585019+0000', 'last_undegraded': '2024-04-25T13:41:32.585019+0000', 'last_fullsized': '2024-04-25T13:41:32.585019+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.576129+0000', 'last_change': '2024-04-25T13:41:32.576129+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.576129+0000', 'last_undegraded': '2024-04-25T13:41:32.576129+0000', 'last_fullsized': '2024-04-25T13:41:32.576129+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.583916+0000', 'last_change': '2024-04-25T13:41:32.583916+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.583916+0000', 'last_undegraded': '2024-04-25T13:41:32.583916+0000', 'last_fullsized': '2024-04-25T13:41:32.583916+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.579893+0000', 'last_change': '2024-04-25T13:41:32.579893+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.579893+0000', 'last_undegraded': '2024-04-25T13:41:32.579893+0000', 'last_fullsized': '2024-04-25T13:41:32.579893+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2024-04-25T13:41:38.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.581018+0000', 'last_change': '2024-04-25T13:41:32.581018+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.581018+0000', 'last_undegraded': '2024-04-25T13:41:32.581018+0000', 'last_fullsized': '2024-04-25T13:41:32.581018+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.571707+0000', 'last_change': '2024-04-25T13:41:32.571707+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.571707+0000', 'last_undegraded': '2024-04-25T13:41:32.571707+0000', 'last_fullsized': '2024-04-25T13:41:32.571707+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.585982+0000', 'last_change': '2024-04-25T13:41:32.585982+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.585982+0000', 'last_undegraded': '2024-04-25T13:41:32.585982+0000', 'last_fullsized': '2024-04-25T13:41:32.585982+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.572059+0000', 'last_change': '2024-04-25T13:41:32.572059+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.572059+0000', 'last_undegraded': '2024-04-25T13:41:32.572059+0000', 'last_fullsized': '2024-04-25T13:41:32.572059+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.578159+0000', 'last_change': '2024-04-25T13:41:32.578159+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.578159+0000', 'last_undegraded': '2024-04-25T13:41:32.578159+0000', 'last_fullsized': '2024-04-25T13:41:32.578159+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.580079+0000', 'last_change': '2024-04-25T13:41:32.580079+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.580079+0000', 'last_undegraded': '2024-04-25T13:41:32.580079+0000', 'last_fullsized': '2024-04-25T13:41:32.580079+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.585445+0000', 'last_change': '2024-04-25T13:41:32.585445+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.585445+0000', 'last_undegraded': '2024-04-25T13:41:32.585445+0000', 'last_fullsized': '2024-04-25T13:41:32.585445+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.585538+0000', 'last_change': '2024-04-25T13:41:32.585538+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.585538+0000', 'last_undegraded': '2024-04-25T13:41:32.585538+0000', 'last_fullsized': '2024-04-25T13:41:32.585538+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.570881+0000', 'last_change': '2024-04-25T13:41:32.570881+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.570881+0000', 'last_undegraded': '2024-04-25T13:41:32.570881+0000', 'last_fullsized': '2024-04-25T13:41:32.570881+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.574825+0000', 'last_change': '2024-04-25T13:41:32.574825+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.574825+0000', 'last_undegraded': '2024-04-25T13:41:32.574825+0000', 'last_fullsized': '2024-04-25T13:41:32.574825+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.589854+0000', 'last_change': '2024-04-25T13:41:32.589854+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.589854+0000', 'last_undegraded': '2024-04-25T13:41:32.589854+0000', 'last_fullsized': '2024-04-25T13:41:32.589854+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:41:38.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.577095+0000', 'last_change': '2024-04-25T13:41:32.577095+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.577095+0000', 'last_undegraded': '2024-04-25T13:41:32.577095+0000', 'last_fullsized': '2024-04-25T13:41:32.577095+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.573074+0000', 'last_change': '2024-04-25T13:41:32.573074+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.573074+0000', 'last_undegraded': '2024-04-25T13:41:32.573074+0000', 'last_fullsized': '2024-04-25T13:41:32.573074+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.583161+0000', 'last_change': '2024-04-25T13:41:32.583161+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.583161+0000', 'last_undegraded': '2024-04-25T13:41:32.583161+0000', 'last_fullsized': '2024-04-25T13:41:32.583161+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.584676+0000', 'last_change': '2024-04-25T13:41:32.584676+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.584676+0000', 'last_undegraded': '2024-04-25T13:41:32.584676+0000', 'last_fullsized': '2024-04-25T13:41:32.584676+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.582937+0000', 'last_change': '2024-04-25T13:41:32.582937+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.582937+0000', 'last_undegraded': '2024-04-25T13:41:32.582937+0000', 'last_fullsized': '2024-04-25T13:41:32.582937+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2024-04-25T13:41:38.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.577565+0000', 'last_change': '2024-04-25T13:41:32.577565+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.577565+0000', 'last_undegraded': '2024-04-25T13:41:32.577565+0000', 'last_fullsized': '2024-04-25T13:41:32.577565+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.575137+0000', 'last_change': '2024-04-25T13:41:32.575137+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.575137+0000', 'last_undegraded': '2024-04-25T13:41:32.575137+0000', 'last_fullsized': '2024-04-25T13:41:32.575137+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.578736+0000', 'last_change': '2024-04-25T13:41:32.578736+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.578736+0000', 'last_undegraded': '2024-04-25T13:41:32.578736+0000', 'last_fullsized': '2024-04-25T13:41:32.578736+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.582191+0000', 'last_change': '2024-04-25T13:41:32.582191+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.582191+0000', 'last_undegraded': '2024-04-25T13:41:32.582191+0000', 'last_fullsized': '2024-04-25T13:41:32.582191+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.577621+0000', 'last_change': '2024-04-25T13:41:32.577621+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.577621+0000', 'last_undegraded': '2024-04-25T13:41:32.577621+0000', 'last_fullsized': '2024-04-25T13:41:32.577621+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.577142+0000', 'last_change': '2024-04-25T13:41:32.577142+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.577142+0000', 'last_undegraded': '2024-04-25T13:41:32.577142+0000', 'last_fullsized': '2024-04-25T13:41:32.577142+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2024-04-25T13:41:38.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.578440+0000', 'last_change': '2024-04-25T13:41:32.578440+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.578440+0000', 'last_undegraded': '2024-04-25T13:41:32.578440+0000', 'last_fullsized': '2024-04-25T13:41:32.578440+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.573601+0000', 'last_change': '2024-04-25T13:41:32.573601+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.573601+0000', 'last_undegraded': '2024-04-25T13:41:32.573601+0000', 'last_fullsized': '2024-04-25T13:41:32.573601+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.582460+0000', 'last_change': '2024-04-25T13:41:32.582460+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.582460+0000', 'last_undegraded': '2024-04-25T13:41:32.582460+0000', 'last_fullsized': '2024-04-25T13:41:32.582460+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.583853+0000', 'last_change': '2024-04-25T13:41:32.583853+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.583853+0000', 'last_undegraded': '2024-04-25T13:41:32.583853+0000', 'last_fullsized': '2024-04-25T13:41:32.583853+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.575940+0000', 'last_change': '2024-04-25T13:41:32.575940+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.575940+0000', 'last_undegraded': '2024-04-25T13:41:32.575940+0000', 'last_fullsized': '2024-04-25T13:41:32.575940+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.602644+0000', 'last_change': '2024-04-25T13:41:32.602644+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.602644+0000', 'last_undegraded': '2024-04-25T13:41:32.602644+0000', 'last_fullsized': '2024-04-25T13:41:32.602644+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2024-04-25T13:41:38.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.581729+0000', 'last_change': '2024-04-25T13:41:32.581729+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.581729+0000', 'last_undegraded': '2024-04-25T13:41:32.581729+0000', 'last_fullsized': '2024-04-25T13:41:32.581729+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.572876+0000', 'last_change': '2024-04-25T13:41:32.572876+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.572876+0000', 'last_undegraded': '2024-04-25T13:41:32.572876+0000', 'last_fullsized': '2024-04-25T13:41:32.572876+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.584067+0000', 'last_change': '2024-04-25T13:41:32.584067+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.584067+0000', 'last_undegraded': '2024-04-25T13:41:32.584067+0000', 'last_fullsized': '2024-04-25T13:41:32.584067+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.584147+0000', 'last_change': '2024-04-25T13:41:32.584147+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.584147+0000', 'last_undegraded': '2024-04-25T13:41:32.584147+0000', 'last_fullsized': '2024-04-25T13:41:32.584147+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.576542+0000', 'last_change': '2024-04-25T13:41:32.576542+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.576542+0000', 'last_undegraded': '2024-04-25T13:41:32.576542+0000', 'last_fullsized': '2024-04-25T13:41:32.576542+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2024-04-25T13:41:38.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.581800+0000', 'last_change': '2024-04-25T13:41:32.581800+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.581800+0000', 'last_undegraded': '2024-04-25T13:41:32.581800+0000', 'last_fullsized': '2024-04-25T13:41:32.581800+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.582909+0000', 'last_change': '2024-04-25T13:41:32.582909+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.582909+0000', 'last_undegraded': '2024-04-25T13:41:32.582909+0000', 'last_fullsized': '2024-04-25T13:41:32.582909+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.582801+0000', 'last_change': '2024-04-25T13:41:32.582801+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.582801+0000', 'last_undegraded': '2024-04-25T13:41:32.582801+0000', 'last_fullsized': '2024-04-25T13:41:32.582801+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.580608+0000', 'last_change': '2024-04-25T13:41:32.580608+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.580608+0000', 'last_undegraded': '2024-04-25T13:41:32.580608+0000', 'last_fullsized': '2024-04-25T13:41:32.580608+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.600550+0000', 'last_change': '2024-04-25T13:41:32.600550+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.600550+0000', 'last_undegraded': '2024-04-25T13:41:32.600550+0000', 'last_fullsized': '2024-04-25T13:41:32.600550+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.572005+0000', 'last_change': '2024-04-25T13:41:32.572005+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.572005+0000', 'last_undegraded': '2024-04-25T13:41:32.572005+0000', 'last_fullsized': '2024-04-25T13:41:32.572005+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2024-04-25T13:41:38.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.581520+0000', 'last_change': '2024-04-25T13:41:32.581520+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.581520+0000', 'last_undegraded': '2024-04-25T13:41:32.581520+0000', 'last_fullsized': '2024-04-25T13:41:32.581520+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.570714+0000', 'last_change': '2024-04-25T13:41:32.570714+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.570714+0000', 'last_undegraded': '2024-04-25T13:41:32.570714+0000', 'last_fullsized': '2024-04-25T13:41:32.570714+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.569941+0000', 'last_change': '2024-04-25T13:41:32.569941+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.569941+0000', 'last_undegraded': '2024-04-25T13:41:32.569941+0000', 'last_fullsized': '2024-04-25T13:41:32.569941+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.570534+0000', 'last_change': '2024-04-25T13:41:32.570534+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.570534+0000', 'last_undegraded': '2024-04-25T13:41:32.570534+0000', 'last_fullsized': '2024-04-25T13:41:32.570534+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.569593+0000', 'last_change': '2024-04-25T13:41:32.569593+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.569593+0000', 'last_undegraded': '2024-04-25T13:41:32.569593+0000', 'last_fullsized': '2024-04-25T13:41:32.569593+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2024-04-25T13:41:38.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.577294+0000', 'last_change': '2024-04-25T13:41:32.577294+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.577294+0000', 'last_undegraded': '2024-04-25T13:41:32.577294+0000', 'last_fullsized': '2024-04-25T13:41:32.577294+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.569832+0000', 'last_change': '2024-04-25T13:41:32.569832+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.569832+0000', 'last_undegraded': '2024-04-25T13:41:32.569832+0000', 'last_fullsized': '2024-04-25T13:41:32.569832+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.574871+0000', 'last_change': '2024-04-25T13:41:32.574871+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.574871+0000', 'last_undegraded': '2024-04-25T13:41:32.574871+0000', 'last_fullsized': '2024-04-25T13:41:32.574871+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.576518+0000', 'last_change': '2024-04-25T13:41:32.576518+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.576518+0000', 'last_undegraded': '2024-04-25T13:41:32.576518+0000', 'last_fullsized': '2024-04-25T13:41:32.576518+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.579544+0000', 'last_change': '2024-04-25T13:41:32.579544+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.579544+0000', 'last_undegraded': '2024-04-25T13:41:32.579544+0000', 'last_fullsized': '2024-04-25T13:41:32.579544+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.577739+0000', 'last_change': '2024-04-25T13:41:32.577739+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.577739+0000', 'last_undegraded': '2024-04-25T13:41:32.577739+0000', 'last_fullsized': '2024-04-25T13:41:32.577739+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2024-04-25T13:41:38.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.583410+0000', 'last_change': '2024-04-25T13:41:32.583410+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.583410+0000', 'last_undegraded': '2024-04-25T13:41:32.583410+0000', 'last_fullsized': '2024-04-25T13:41:32.583410+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:41:38.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2024-04-25T13:41:38.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.596171+0000', 'last_change': '2024-04-25T13:41:32.596171+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.596171+0000', 'last_undegraded': '2024-04-25T13:41:32.596171+0000', 'last_fullsized': '2024-04-25T13:41:32.596171+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:41:38.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2024-04-25T13:41:38.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 31, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:41:32.576040+0000', 'last_change': '2024-04-25T13:41:32.576040+0000', 'last_active': '2024-04-25T13:41:32.548343+0000', 'last_peered': '2024-04-25T13:41:32.548343+0000', 'last_clean': '2024-04-25T13:41:32.548343+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:41:32.576040+0000', 'last_undegraded': '2024-04-25T13:41:32.576040+0000', 'last_fullsized': '2024-04-25T13:41:32.576040+0000', 'mapping_epoch': 31, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 31, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'last_clean_scrub_stamp': '2024-04-25T13:41:32.548343+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:41:38.570 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T13:41:38.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.738+0000 7f08e401c640 1 Processor -- start 2024-04-25T13:41:38.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.738+0000 7f08e401c640 1 -- start start 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f08dc161f90 0x7f08dc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f08dc15ec40 0x7f08dc15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f08dc0d0d70 con 0x7f08dc0d0510 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f08dc0d1050 con 0x7f08dc161f90 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f08dc06dca0 con 0x7f08dc15ec40 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e1db8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f08dc161f90 0x7f08dc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e1db8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f08dc161f90 0x7f08dc0cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38764/0 (socket says 172.21.15.80:38764) 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e1db8640 1 -- 172.21.15.80:0/1186225468 learned_addr learned my addr 172.21.15.80:0/1186225468 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:38.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e1db8640 1 -- 172.21.15.80:0/1186225468 >> v2:172.21.15.87:3300/0 conn(0x7f08dc15ec40 msgr2=0x7f08dc15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:38.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e25b9640 1 --2- 172.21.15.80:0/1186225468 >> v2:172.21.15.87:3300/0 conn(0x7f08dc15ec40 0x7f08dc15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e1db8640 1 --2- 172.21.15.80:0/1186225468 >> v2:172.21.15.87:3300/0 conn(0x7f08dc15ec40 0x7f08dc15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e1db8640 1 -- 172.21.15.80:0/1186225468 >> v1:172.21.15.87:6789/0 conn(0x7f08dc0d0510 legacy=0x7f08dc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:38.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e1db8640 1 -- 172.21.15.80:0/1186225468 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc06df80 con 0x7f08dc161f90 2024-04-25T13:41:38.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e25b9640 1 --2- 172.21.15.80:0/1186225468 >> v2:172.21.15.87:3300/0 conn(0x7f08dc15ec40 0x7f08dc15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:38.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e1db8640 1 --2- 172.21.15.80:0/1186225468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc161f90 0x7f08dc0cffd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0b4b80 tx=0x7f08cc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0bef7db7021970 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:38.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e0db6640 1 -- 172.21.15.80:0/1186225468 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0b0070 con 0x7f08dc161f90 2024-04-25T13:41:38.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e0db6640 1 -- 172.21.15.80:0/1186225468 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08cc0ab600 con 0x7f08dc161f90 2024-04-25T13:41:38.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e0db6640 1 -- 172.21.15.80:0/1186225468 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0ab9d0 con 0x7f08dc161f90 2024-04-25T13:41:38.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 -- 172.21.15.80:0/1186225468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc161f90 msgr2=0x7f08dc0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 --2- 172.21.15.80:0/1186225468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc161f90 0x7f08dc0cffd0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f08dc0b4b80 tx=0x7f08cc01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 -- 172.21.15.80:0/1186225468 shutdown_connections 2024-04-25T13:41:38.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 --2- 172.21.15.80:0/1186225468 >> v2:172.21.15.87:3300/0 conn(0x7f08dc15ec40 0x7f08dc15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 --2- 172.21.15.80:0/1186225468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc161f90 0x7f08dc0cffd0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 -- 172.21.15.80:0/1186225468 >> 172.21.15.80:0/1186225468 conn(0x7f08dc06d080 msgr2=0x7f08dc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:38.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 -- 172.21.15.80:0/1186225468 shutdown_connections 2024-04-25T13:41:38.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.742+0000 7f08e401c640 1 -- 172.21.15.80:0/1186225468 wait complete. 2024-04-25T13:41:38.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.746+0000 7f08e401c640 1 Processor -- start 2024-04-25T13:41:38.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.746+0000 7f08e401c640 1 -- start start 2024-04-25T13:41:38.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.746+0000 7f08e401c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08dc0d0510 0x7f08dc1f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.746+0000 7f08e401c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc15ec40 0x7f08dc13fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.746+0000 7f08e401c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08dc161f90 0x7f08dc15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.746+0000 7f08e401c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc06f630 con 0x7f08dc15ec40 2024-04-25T13:41:38.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.746+0000 7f08e401c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f08dc06f4b0 con 0x7f08dc0d0510 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.746+0000 7f08e401c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08dc0d0920 con 0x7f08dc161f90 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc15ec40 0x7f08dc13fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc15ec40 0x7f08dc13fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38780/0 (socket says 172.21.15.80:38780) 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 -- 172.21.15.80:0/2980795048 learned_addr learned my addr 172.21.15.80:0/2980795048 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e1db8640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08dc0d0510 0x7f08dc1f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e25b9640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08dc161f90 0x7f08dc15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 -- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08dc161f90 msgr2=0x7f08dc15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08dc161f90 0x7f08dc15b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 -- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08dc0d0510 msgr2=0x7f08dc1f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08dc0d0510 0x7f08dc1f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 -- 172.21.15.80:0/2980795048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08dc1fb4b0 con 0x7f08dc15ec40 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e1db8640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08dc0d0510 0x7f08dc1f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e15b7640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc15ec40 0x7f08dc13fb40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f08d000b440 tx=0x7f08d000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:38.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08da7fc640 1 -- 172.21.15.80:0/2980795048 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0015020 con 0x7f08dc15ec40 2024-04-25T13:41:38.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08da7fc640 1 -- 172.21.15.80:0/2980795048 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08d0011890 con 0x7f08dc15ec40 2024-04-25T13:41:38.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08dc1fb790 con 0x7f08dc15ec40 2024-04-25T13:41:38.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08dc15ac60 con 0x7f08dc15ec40 2024-04-25T13:41:38.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08da7fc640 1 -- 172.21.15.80:0/2980795048 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08d0011c80 con 0x7f08dc15ec40 2024-04-25T13:41:38.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08bc0052a0 con 0x7f08dc15ec40 2024-04-25T13:41:38.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08da7fc640 1 -- 172.21.15.80:0/2980795048 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f08d0018020 con 0x7f08dc15ec40 2024-04-25T13:41:38.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08da7fc640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08c806f430 0x7f08c8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:38.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08da7fc640 1 -- 172.21.15.80:0/2980795048 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f08d008f3c0 con 0x7f08dc15ec40 2024-04-25T13:41:38.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.750+0000 7f08e1db8640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08c806f430 0x7f08c8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:38.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.754+0000 7f08e1db8640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08c806f430 0x7f08c8071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0ad000 tx=0x7f08cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:38.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.758+0000 7f08da7fc640 1 -- 172.21.15.80:0/2980795048 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f08d001f020 con 0x7f08dc15ec40 2024-04-25T13:41:38.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.962+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f08bc005860 con 0x7f08dc15ec40 2024-04-25T13:41:38.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.962+0000 7f08da7fc640 1 -- 172.21.15.80:0/2980795048 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1188 (secure 0 0 0) 0x7f08d000d040 con 0x7f08dc15ec40 2024-04-25T13:41:38.966 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:38.966 INFO:teuthology.orchestra.run.smithi080.stdout:{"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":87,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":33,"num_osds":8,"num_up_osds":8,"osd_up_since":1714052417,"num_in_osds":8,"osd_in_since":1714052414,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":2228983,"bytes_avail":1597771017,"bytes_total":1600000000,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":5190}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-25T13:41:24.113088+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T13:41:38.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08c806f430 msgr2=0x7f08c8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08c806f430 0x7f08c8071940 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0ad000 tx=0x7f08cc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc15ec40 msgr2=0x7f08dc13fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc15ec40 0x7f08dc13fb40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f08d000b440 tx=0x7f08d000b900 comp rx=0 tx=0).stop 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 shutdown_connections 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08c806f430 0x7f08c8071940 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f08dc161f90 0x7f08dc15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08dc15ec40 0x7f08dc13fb40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 --2- 172.21.15.80:0/2980795048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08dc0d0510 0x7f08dc1f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 >> 172.21.15.80:0/2980795048 conn(0x7f08dc06d080 msgr2=0x7f08dc0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 shutdown_connections 2024-04-25T13:41:38.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:38.966+0000 7f08e401c640 1 -- 172.21.15.80:0/2980795048 wait complete. 2024-04-25T13:41:38.986 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T13:41:38.986 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:41:39.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.126+0000 7fd17cad1640 1 Processor -- start 2024-04-25T13:41:39.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.126+0000 7fd17cad1640 1 -- start start 2024-04-25T13:41:39.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd17cad1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd17806db00 0x7fd1780c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:39.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd17cad1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd1780c8a60 0x7fd1780c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd17cad1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1780d26f0 con 0x7fd178160b10 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd17cad1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd1780c93b0 con 0x7fd1780c8a60 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd17cad1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd17806ece0 con 0x7fd17806db00 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd175d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd1780c8a60 0x7fd1780c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd176d9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd178160b10 0x7fd178160f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:36458/0 (socket says 172.21.15.80:36458) 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd176d9d640 1 -- 172.21.15.80:0/3351955686 learned_addr learned my addr 172.21.15.80:0/3351955686 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd17659c640 1 --2- 172.21.15.80:0/3351955686 >> v2:172.21.15.87:3300/0 conn(0x7fd17806db00 0x7fd1780c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd175d9b640 1 -- 172.21.15.80:0/3351955686 >> v2:172.21.15.87:3300/0 conn(0x7fd17806db00 msgr2=0x7fd1780c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd175d9b640 1 --2- 172.21.15.80:0/3351955686 >> v2:172.21.15.87:3300/0 conn(0x7fd17806db00 0x7fd1780c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd175d9b640 1 -- 172.21.15.80:0/3351955686 >> v1:172.21.15.80:6789/0 conn(0x7fd178160b10 legacy=0x7fd178160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd175d9b640 1 -- 172.21.15.80:0/3351955686 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17806df10 con 0x7fd1780c8a60 2024-04-25T13:41:39.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.130+0000 7fd17559a640 1 -- 172.21.15.80:0/3351955686 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 811942059 0 0) 0x7fd1780d26f0 con 0x7fd178160b10 2024-04-25T13:41:39.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd175d9b640 1 --2- 172.21.15.80:0/3351955686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1780c8a60 0x7fd1780c8e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd1780d3960 tx=0x7fd16c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1be926c2606feda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:39.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17559a640 1 -- 172.21.15.80:0/3351955686 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0ab650 con 0x7fd1780c8a60 2024-04-25T13:41:39.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17559a640 1 -- 172.21.15.80:0/3351955686 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd16c0b0070 con 0x7fd1780c8a60 2024-04-25T13:41:39.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17559a640 1 -- 172.21.15.80:0/3351955686 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c0aba10 con 0x7fd1780c8a60 2024-04-25T13:41:39.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 -- 172.21.15.80:0/3351955686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1780c8a60 msgr2=0x7fd1780c8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:39.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 --2- 172.21.15.80:0/3351955686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1780c8a60 0x7fd1780c8e70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd1780d3960 tx=0x7fd16c01c450 comp rx=0 tx=0).stop 2024-04-25T13:41:39.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 -- 172.21.15.80:0/3351955686 shutdown_connections 2024-04-25T13:41:39.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 --2- 172.21.15.80:0/3351955686 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1780c8a60 0x7fd1780c8e70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 --2- 172.21.15.80:0/3351955686 >> v2:172.21.15.87:3300/0 conn(0x7fd17806db00 0x7fd1780c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 -- 172.21.15.80:0/3351955686 >> 172.21.15.80:0/3351955686 conn(0x7fd1780cf390 msgr2=0x7fd1780cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:39.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 -- 172.21.15.80:0/3351955686 shutdown_connections 2024-04-25T13:41:39.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 -- 172.21.15.80:0/3351955686 wait complete. 2024-04-25T13:41:39.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 Processor -- start 2024-04-25T13:41:39.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.134+0000 7fd17cad1640 1 -- start start 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd17806db00 0x7fd17813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1780c8a60 0x7fd1780b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd178160b10 0x7fd1780cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd1780d2410 con 0x7fd178160b10 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd178161a40 con 0x7fd17806db00 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd17816a6c0 con 0x7fd1780c8a60 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd178160b10 0x7fd1780cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd178160b10 0x7fd1780cddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38792/0 (socket says 172.21.15.80:38792) 2024-04-25T13:41:39.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 -- 172.21.15.80:0/77824307 learned_addr learned my addr 172.21.15.80:0/77824307 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17659c640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd17806db00 0x7fd17813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd175d9b640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1780c8a60 0x7fd1780b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 -- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1780c8a60 msgr2=0x7fd1780b2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1780c8a60 0x7fd1780b2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 -- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd17806db00 msgr2=0x7fd17813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd17806db00 0x7fd17813d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 -- 172.21.15.80:0/77824307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1780b2e70 con 0x7fd178160b10 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd175d9b640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1780c8a60 0x7fd1780b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd176d9d640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd178160b10 0x7fd1780cddb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd17000c7b0 tx=0x7fd17000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd166ffd640 1 -- 172.21.15.80:0/77824307 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170018070 con 0x7fd178160b10 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd166ffd640 1 -- 172.21.15.80:0/77824307 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17000a5e0 con 0x7fd178160b10 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd166ffd640 1 -- 172.21.15.80:0/77824307 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17000a9d0 con 0x7fd178160b10 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1780b33a0 con 0x7fd178160b10 2024-04-25T13:41:39.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1780b3960 con 0x7fd178160b10 2024-04-25T13:41:39.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd17806a3a0 con 0x7fd178160b10 2024-04-25T13:41:39.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.138+0000 7fd166ffd640 1 -- 172.21.15.80:0/77824307 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd17000ab70 con 0x7fd178160b10 2024-04-25T13:41:39.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.142+0000 7fd166ffd640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd15806f400 0x7fd158071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:39.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.142+0000 7fd166ffd640 1 -- 172.21.15.80:0/77824307 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd17008e770 con 0x7fd178160b10 2024-04-25T13:41:39.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.142+0000 7fd17659c640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd15806f400 0x7fd158071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:39.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.142+0000 7fd17659c640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd15806f400 0x7fd158071910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd16800b930 tx=0x7fd168008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:39.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.146+0000 7fd166ffd640 1 -- 172.21.15.80:0/77824307 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd17005b660 con 0x7fd178160b10 2024-04-25T13:41:39.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.306+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1780b30a0 con 0x7fd15806f400 2024-04-25T13:41:39.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.322+0000 7fd166ffd640 1 -- 172.21.15.80:0/77824307 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242206 (secure 0 0 0) 0x7fd1780b30a0 con 0x7fd15806f400 2024-04-25T13:41:39.329 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:39.330 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:41:39.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd15806f400 msgr2=0x7fd158071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:39.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd15806f400 0x7fd158071910 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd16800b930 tx=0x7fd168008040 comp rx=0 tx=0).stop 2024-04-25T13:41:39.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd178160b10 msgr2=0x7fd1780cddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:39.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd178160b10 0x7fd1780cddb0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd17000c7b0 tx=0x7fd17000cc70 comp rx=0 tx=0).stop 2024-04-25T13:41:39.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 shutdown_connections 2024-04-25T13:41:39.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd15806f400 0x7fd158071910 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd178160b10 0x7fd1780cddb0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd1780c8a60 0x7fd1780b2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 --2- 172.21.15.80:0/77824307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd17806db00 0x7fd17813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:39.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 >> 172.21.15.80:0/77824307 conn(0x7fd1780cf390 msgr2=0x7fd178160f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:39.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 shutdown_connections 2024-04-25T13:41:39.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:39.326+0000 7fd17cad1640 1 -- 172.21.15.80:0/77824307 wait complete. 2024-04-25T13:41:39.349 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-04-25T13:41:38.117264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2176,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560323,"statfs":{"total":1600000000,"available":1597771017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001925"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583136+0000","last_change":"2024-04-25T13:41:32.583136+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583136+0000","last_undegraded":"2024-04-25T13:41:32.583136+0000","last_fullsized":"2024-04-25T13:41:32.583136+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.586323+0000","last_change":"2024-04-25T13:41:32.586323+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.586323+0000","last_undegraded":"2024-04-25T13:41:32.586323+0000","last_fullsized":"2024-04-25T13:41:32.586323+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573531+0000","last_change":"2024-04-25T13:41:32.573531+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573531+0000","last_undegraded":"2024-04-25T13:41:32.573531+0000","last_fullsized":"2024-04-25T13:41:32.573531+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583817+0000","last_change":"2024-04-25T13:41:32.583817+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583817+0000","last_undegraded":"2024-04-25T13:41:32.583817+0000","last_fullsized":"2024-04-25T13:41:32.583817+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.571611+0000","last_change":"2024-04-25T13:41:32.571611+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.571611+0000","last_undegraded":"2024-04-25T13:41:32.571611+0000","last_fullsized":"2024-04-25T13:41:32.571611+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585490+0000","last_change":"2024-04-25T13:41:32.585490+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585490+0000","last_undegraded":"2024-04-25T13:41:32.585490+0000","last_fullsized":"2024-04-25T13:41:32.585490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576011+0000","last_change":"2024-04-25T13:41:32.576011+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576011+0000","last_undegraded":"2024-04-25T13:41:32.576011+0000","last_fullsized":"2024-04-25T13:41:32.576011+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585019+0000","last_change":"2024-04-25T13:41:32.585019+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585019+0000","last_undegraded":"2024-04-25T13:41:32.585019+0000","last_fullsized":"2024-04-25T13:41:32.585019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576129+0000","last_change":"2024-04-25T13:41:32.576129+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576129+0000","last_undegraded":"2024-04-25T13:41:32.576129+0000","last_fullsized":"2024-04-25T13:41:32.576129+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583916+0000","last_change":"2024-04-25T13:41:32.583916+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583916+0000","last_undegraded":"2024-04-25T13:41:32.583916+0000","last_fullsized":"2024-04-25T13:41:32.583916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.579893+0000","last_change":"2024-04-25T13:41:32.579893+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.579893+0000","last_undegraded":"2024-04-25T13:41:32.579893+0000","last_fullsized":"2024-04-25T13:41:32.579893+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581018+0000","last_change":"2024-04-25T13:41:32.581018+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581018+0000","last_undegraded":"2024-04-25T13:41:32.581018+0000","last_fullsized":"2024-04-25T13:41:32.581018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.571707+0000","last_change":"2024-04-25T13:41:32.571707+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.571707+0000","last_undegraded":"2024-04-25T13:41:32.571707+0000","last_fullsized":"2024-04-25T13:41:32.571707+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585982+0000","last_change":"2024-04-25T13:41:32.585982+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585982+0000","last_undegraded":"2024-04-25T13:41:32.585982+0000","last_fullsized":"2024-04-25T13:41:32.585982+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572059+0000","last_change":"2024-04-25T13:41:32.572059+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572059+0000","last_undegraded":"2024-04-25T13:41:32.572059+0000","last_fullsized":"2024-04-25T13:41:32.572059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578159+0000","last_change":"2024-04-25T13:41:32.578159+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578159+0000","last_undegraded":"2024-04-25T13:41:32.578159+0000","last_fullsized":"2024-04-25T13:41:32.578159+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.580079+0000","last_change":"2024-04-25T13:41:32.580079+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.580079+0000","last_undegraded":"2024-04-25T13:41:32.580079+0000","last_fullsized":"2024-04-25T13:41:32.580079+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585445+0000","last_change":"2024-04-25T13:41:32.585445+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585445+0000","last_undegraded":"2024-04-25T13:41:32.585445+0000","last_fullsized":"2024-04-25T13:41:32.585445+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.585538+0000","last_change":"2024-04-25T13:41:32.585538+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.585538+0000","last_undegraded":"2024-04-25T13:41:32.585538+0000","last_fullsized":"2024-04-25T13:41:32.585538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570881+0000","last_change":"2024-04-25T13:41:32.570881+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570881+0000","last_undegraded":"2024-04-25T13:41:32.570881+0000","last_fullsized":"2024-04-25T13:41:32.570881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.574825+0000","last_change":"2024-04-25T13:41:32.574825+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.574825+0000","last_undegraded":"2024-04-25T13:41:32.574825+0000","last_fullsized":"2024-04-25T13:41:32.574825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.589854+0000","last_change":"2024-04-25T13:41:32.589854+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.589854+0000","last_undegraded":"2024-04-25T13:41:32.589854+0000","last_fullsized":"2024-04-25T13:41:32.589854+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577095+0000","last_change":"2024-04-25T13:41:32.577095+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577095+0000","last_undegraded":"2024-04-25T13:41:32.577095+0000","last_fullsized":"2024-04-25T13:41:32.577095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573074+0000","last_change":"2024-04-25T13:41:32.573074+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573074+0000","last_undegraded":"2024-04-25T13:41:32.573074+0000","last_fullsized":"2024-04-25T13:41:32.573074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583161+0000","last_change":"2024-04-25T13:41:32.583161+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583161+0000","last_undegraded":"2024-04-25T13:41:32.583161+0000","last_fullsized":"2024-04-25T13:41:32.583161+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584676+0000","last_change":"2024-04-25T13:41:32.584676+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584676+0000","last_undegraded":"2024-04-25T13:41:32.584676+0000","last_fullsized":"2024-04-25T13:41:32.584676+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582937+0000","last_change":"2024-04-25T13:41:32.582937+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582937+0000","last_undegraded":"2024-04-25T13:41:32.582937+0000","last_fullsized":"2024-04-25T13:41:32.582937+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577565+0000","last_change":"2024-04-25T13:41:32.577565+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577565+0000","last_undegraded":"2024-04-25T13:41:32.577565+0000","last_fullsized":"2024-04-25T13:41:32.577565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.575137+0000","last_change":"2024-04-25T13:41:32.575137+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.575137+0000","last_undegraded":"2024-04-25T13:41:32.575137+0000","last_fullsized":"2024-04-25T13:41:32.575137+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578736+0000","last_change":"2024-04-25T13:41:32.578736+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578736+0000","last_undegraded":"2024-04-25T13:41:32.578736+0000","last_fullsized":"2024-04-25T13:41:32.578736+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582191+0000","last_change":"2024-04-25T13:41:32.582191+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582191+0000","last_undegraded":"2024-04-25T13:41:32.582191+0000","last_fullsized":"2024-04-25T13:41:32.582191+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577621+0000","last_change":"2024-04-25T13:41:32.577621+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577621+0000","last_undegraded":"2024-04-25T13:41:32.577621+0000","last_fullsized":"2024-04-25T13:41:32.577621+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577142+0000","last_change":"2024-04-25T13:41:32.577142+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577142+0000","last_undegraded":"2024-04-25T13:41:32.577142+0000","last_fullsized":"2024-04-25T13:41:32.577142+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591696+0000","last_change":"2024-04-25T13:41:32.591289+0000","last_active":"2024-04-25T13:41:32.591696+0000","last_peered":"2024-04-25T13:41:32.591696+0000","last_clean":"2024-04-25T13:41:32.591696+0000","last_became_active":"2024-04-25T13:41:32.590480+0000","last_became_peered":"2024-04-25T13:41:32.590480+0000","last_unstale":"2024-04-25T13:41:32.591696+0000","last_undegraded":"2024-04-25T13:41:32.591696+0000","last_fullsized":"2024-04-25T13:41:32.591696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:43.229224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.578440+0000","last_change":"2024-04-25T13:41:32.578440+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.578440+0000","last_undegraded":"2024-04-25T13:41:32.578440+0000","last_fullsized":"2024-04-25T13:41:32.578440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590389+0000","last_change":"2024-04-25T13:41:32.590216+0000","last_active":"2024-04-25T13:41:32.590389+0000","last_peered":"2024-04-25T13:41:32.590389+0000","last_clean":"2024-04-25T13:41:32.590389+0000","last_became_active":"2024-04-25T13:41:32.589855+0000","last_became_peered":"2024-04-25T13:41:32.589855+0000","last_unstale":"2024-04-25T13:41:32.590389+0000","last_undegraded":"2024-04-25T13:41:32.590389+0000","last_fullsized":"2024-04-25T13:41:32.590389+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:29.322913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.573601+0000","last_change":"2024-04-25T13:41:32.573601+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.573601+0000","last_undegraded":"2024-04-25T13:41:32.573601+0000","last_fullsized":"2024-04-25T13:41:32.573601+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579266+0000","last_change":"2024-04-25T13:41:32.578789+0000","last_active":"2024-04-25T13:41:32.579266+0000","last_peered":"2024-04-25T13:41:32.579266+0000","last_clean":"2024-04-25T13:41:32.579266+0000","last_became_active":"2024-04-25T13:41:32.578278+0000","last_became_peered":"2024-04-25T13:41:32.578278+0000","last_unstale":"2024-04-25T13:41:32.579266+0000","last_undegraded":"2024-04-25T13:41:32.579266+0000","last_fullsized":"2024-04-25T13:41:32.579266+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:04.613101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582460+0000","last_change":"2024-04-25T13:41:32.582460+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582460+0000","last_undegraded":"2024-04-25T13:41:32.582460+0000","last_fullsized":"2024-04-25T13:41:32.582460+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.577933+0000","last_change":"2024-04-25T13:41:32.577475+0000","last_active":"2024-04-25T13:41:32.577933+0000","last_peered":"2024-04-25T13:41:32.577933+0000","last_clean":"2024-04-25T13:41:32.577933+0000","last_became_active":"2024-04-25T13:41:32.576661+0000","last_became_peered":"2024-04-25T13:41:32.576661+0000","last_unstale":"2024-04-25T13:41:32.577933+0000","last_undegraded":"2024-04-25T13:41:32.577933+0000","last_fullsized":"2024-04-25T13:41:32.577933+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:23.408800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583853+0000","last_change":"2024-04-25T13:41:32.583853+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583853+0000","last_undegraded":"2024-04-25T13:41:32.583853+0000","last_fullsized":"2024-04-25T13:41:32.583853+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.603280+0000","last_change":"2024-04-25T13:41:32.602990+0000","last_active":"2024-04-25T13:41:32.603280+0000","last_peered":"2024-04-25T13:41:32.603280+0000","last_clean":"2024-04-25T13:41:32.603280+0000","last_became_active":"2024-04-25T13:41:32.602174+0000","last_became_peered":"2024-04-25T13:41:32.602174+0000","last_unstale":"2024-04-25T13:41:32.603280+0000","last_undegraded":"2024-04-25T13:41:32.603280+0000","last_fullsized":"2024-04-25T13:41:32.603280+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:20.977995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.575940+0000","last_change":"2024-04-25T13:41:32.575940+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.575940+0000","last_undegraded":"2024-04-25T13:41:32.575940+0000","last_fullsized":"2024-04-25T13:41:32.575940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.606395+0000","last_change":"2024-04-25T13:41:32.606199+0000","last_active":"2024-04-25T13:41:32.606395+0000","last_peered":"2024-04-25T13:41:32.606395+0000","last_clean":"2024-04-25T13:41:32.606395+0000","last_became_active":"2024-04-25T13:41:32.604933+0000","last_became_peered":"2024-04-25T13:41:32.604933+0000","last_unstale":"2024-04-25T13:41:32.606395+0000","last_undegraded":"2024-04-25T13:41:32.606395+0000","last_fullsized":"2024-04-25T13:41:32.606395+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:50.753222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.602644+0000","last_change":"2024-04-25T13:41:32.602644+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.602644+0000","last_undegraded":"2024-04-25T13:41:32.602644+0000","last_fullsized":"2024-04-25T13:41:32.602644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.586454+0000","last_change":"2024-04-25T13:41:32.586286+0000","last_active":"2024-04-25T13:41:32.586454+0000","last_peered":"2024-04-25T13:41:32.586454+0000","last_clean":"2024-04-25T13:41:32.586454+0000","last_became_active":"2024-04-25T13:41:32.584859+0000","last_became_peered":"2024-04-25T13:41:32.584859+0000","last_unstale":"2024-04-25T13:41:32.586454+0000","last_undegraded":"2024-04-25T13:41:32.586454+0000","last_fullsized":"2024-04-25T13:41:32.586454+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:21.284984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581729+0000","last_change":"2024-04-25T13:41:32.581729+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581729+0000","last_undegraded":"2024-04-25T13:41:32.581729+0000","last_fullsized":"2024-04-25T13:41:32.581729+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.610432+0000","last_change":"2024-04-25T13:41:32.610199+0000","last_active":"2024-04-25T13:41:32.610432+0000","last_peered":"2024-04-25T13:41:32.610432+0000","last_clean":"2024-04-25T13:41:32.610432+0000","last_became_active":"2024-04-25T13:41:32.607495+0000","last_became_peered":"2024-04-25T13:41:32.607495+0000","last_unstale":"2024-04-25T13:41:32.610432+0000","last_undegraded":"2024-04-25T13:41:32.610432+0000","last_fullsized":"2024-04-25T13:41:32.610432+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:34.040738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572876+0000","last_change":"2024-04-25T13:41:32.572876+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572876+0000","last_undegraded":"2024-04-25T13:41:32.572876+0000","last_fullsized":"2024-04-25T13:41:32.572876+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591561+0000","last_change":"2024-04-25T13:41:32.591386+0000","last_active":"2024-04-25T13:41:32.591561+0000","last_peered":"2024-04-25T13:41:32.591561+0000","last_clean":"2024-04-25T13:41:32.591561+0000","last_became_active":"2024-04-25T13:41:32.591004+0000","last_became_peered":"2024-04-25T13:41:32.591004+0000","last_unstale":"2024-04-25T13:41:32.591561+0000","last_undegraded":"2024-04-25T13:41:32.591561+0000","last_fullsized":"2024-04-25T13:41:32.591561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:55.459670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584067+0000","last_change":"2024-04-25T13:41:32.584067+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584067+0000","last_undegraded":"2024-04-25T13:41:32.584067+0000","last_fullsized":"2024-04-25T13:41:32.584067+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.577987+0000","last_change":"2024-04-25T13:41:32.577747+0000","last_active":"2024-04-25T13:41:32.577987+0000","last_peered":"2024-04-25T13:41:32.577987+0000","last_clean":"2024-04-25T13:41:32.577987+0000","last_became_active":"2024-04-25T13:41:32.577358+0000","last_became_peered":"2024-04-25T13:41:32.577358+0000","last_unstale":"2024-04-25T13:41:32.577987+0000","last_undegraded":"2024-04-25T13:41:32.577987+0000","last_fullsized":"2024-04-25T13:41:32.577987+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:06.339192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.584147+0000","last_change":"2024-04-25T13:41:32.584147+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.584147+0000","last_undegraded":"2024-04-25T13:41:32.584147+0000","last_fullsized":"2024-04-25T13:41:32.584147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.591771+0000","last_change":"2024-04-25T13:41:32.591430+0000","last_active":"2024-04-25T13:41:32.591771+0000","last_peered":"2024-04-25T13:41:32.591771+0000","last_clean":"2024-04-25T13:41:32.591771+0000","last_became_active":"2024-04-25T13:41:32.590686+0000","last_became_peered":"2024-04-25T13:41:32.590686+0000","last_unstale":"2024-04-25T13:41:32.591771+0000","last_undegraded":"2024-04-25T13:41:32.591771+0000","last_fullsized":"2024-04-25T13:41:32.591771+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:11.287292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576542+0000","last_change":"2024-04-25T13:41:32.576542+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576542+0000","last_undegraded":"2024-04-25T13:41:32.576542+0000","last_fullsized":"2024-04-25T13:41:32.576542+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.580413+0000","last_change":"2024-04-25T13:41:32.580227+0000","last_active":"2024-04-25T13:41:32.580413+0000","last_peered":"2024-04-25T13:41:32.580413+0000","last_clean":"2024-04-25T13:41:32.580413+0000","last_became_active":"2024-04-25T13:41:32.579922+0000","last_became_peered":"2024-04-25T13:41:32.579922+0000","last_unstale":"2024-04-25T13:41:32.580413+0000","last_undegraded":"2024-04-25T13:41:32.580413+0000","last_fullsized":"2024-04-25T13:41:32.580413+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:33.424560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581800+0000","last_change":"2024-04-25T13:41:32.581800+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581800+0000","last_undegraded":"2024-04-25T13:41:32.581800+0000","last_fullsized":"2024-04-25T13:41:32.581800+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.609913+0000","last_change":"2024-04-25T13:41:32.609679+0000","last_active":"2024-04-25T13:41:32.609913+0000","last_peered":"2024-04-25T13:41:32.609913+0000","last_clean":"2024-04-25T13:41:32.609913+0000","last_became_active":"2024-04-25T13:41:32.606623+0000","last_became_peered":"2024-04-25T13:41:32.606623+0000","last_unstale":"2024-04-25T13:41:32.609913+0000","last_undegraded":"2024-04-25T13:41:32.609913+0000","last_fullsized":"2024-04-25T13:41:32.609913+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:00.383684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582909+0000","last_change":"2024-04-25T13:41:32.582909+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582909+0000","last_undegraded":"2024-04-25T13:41:32.582909+0000","last_fullsized":"2024-04-25T13:41:32.582909+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590846+0000","last_change":"2024-04-25T13:41:32.590694+0000","last_active":"2024-04-25T13:41:32.590846+0000","last_peered":"2024-04-25T13:41:32.590846+0000","last_clean":"2024-04-25T13:41:32.590846+0000","last_became_active":"2024-04-25T13:41:32.590377+0000","last_became_peered":"2024-04-25T13:41:32.590377+0000","last_unstale":"2024-04-25T13:41:32.590846+0000","last_undegraded":"2024-04-25T13:41:32.590846+0000","last_fullsized":"2024-04-25T13:41:32.590846+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:33.114772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.582801+0000","last_change":"2024-04-25T13:41:32.582801+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.582801+0000","last_undegraded":"2024-04-25T13:41:32.582801+0000","last_fullsized":"2024-04-25T13:41:32.582801+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590381+0000","last_change":"2024-04-25T13:41:32.590186+0000","last_active":"2024-04-25T13:41:32.590381+0000","last_peered":"2024-04-25T13:41:32.590381+0000","last_clean":"2024-04-25T13:41:32.590381+0000","last_became_active":"2024-04-25T13:41:32.589854+0000","last_became_peered":"2024-04-25T13:41:32.589854+0000","last_unstale":"2024-04-25T13:41:32.590381+0000","last_undegraded":"2024-04-25T13:41:32.590381+0000","last_fullsized":"2024-04-25T13:41:32.590381+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:48.433156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.580608+0000","last_change":"2024-04-25T13:41:32.580608+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.580608+0000","last_undegraded":"2024-04-25T13:41:32.580608+0000","last_fullsized":"2024-04-25T13:41:32.580608+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.578994+0000","last_change":"2024-04-25T13:41:32.578797+0000","last_active":"2024-04-25T13:41:32.578994+0000","last_peered":"2024-04-25T13:41:32.578994+0000","last_clean":"2024-04-25T13:41:32.578994+0000","last_became_active":"2024-04-25T13:41:32.578411+0000","last_became_peered":"2024-04-25T13:41:32.578411+0000","last_unstale":"2024-04-25T13:41:32.578994+0000","last_undegraded":"2024-04-25T13:41:32.578994+0000","last_fullsized":"2024-04-25T13:41:32.578994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:05:46.073052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.600550+0000","last_change":"2024-04-25T13:41:32.600550+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.600550+0000","last_undegraded":"2024-04-25T13:41:32.600550+0000","last_fullsized":"2024-04-25T13:41:32.600550+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579352+0000","last_change":"2024-04-25T13:41:32.579069+0000","last_active":"2024-04-25T13:41:32.579352+0000","last_peered":"2024-04-25T13:41:32.579352+0000","last_clean":"2024-04-25T13:41:32.579352+0000","last_became_active":"2024-04-25T13:41:32.578548+0000","last_became_peered":"2024-04-25T13:41:32.578548+0000","last_unstale":"2024-04-25T13:41:32.579352+0000","last_undegraded":"2024-04-25T13:41:32.579352+0000","last_fullsized":"2024-04-25T13:41:32.579352+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:28.487849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.572005+0000","last_change":"2024-04-25T13:41:32.572005+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.572005+0000","last_undegraded":"2024-04-25T13:41:32.572005+0000","last_fullsized":"2024-04-25T13:41:32.572005+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.609401+0000","last_change":"2024-04-25T13:41:32.609157+0000","last_active":"2024-04-25T13:41:32.609401+0000","last_peered":"2024-04-25T13:41:32.609401+0000","last_clean":"2024-04-25T13:41:32.609401+0000","last_became_active":"2024-04-25T13:41:32.605222+0000","last_became_peered":"2024-04-25T13:41:32.605222+0000","last_unstale":"2024-04-25T13:41:32.609401+0000","last_undegraded":"2024-04-25T13:41:32.609401+0000","last_fullsized":"2024-04-25T13:41:32.609401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:02.704888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.581520+0000","last_change":"2024-04-25T13:41:32.581520+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.581520+0000","last_undegraded":"2024-04-25T13:41:32.581520+0000","last_fullsized":"2024-04-25T13:41:32.581520+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.596477+0000","last_change":"2024-04-25T13:41:32.595845+0000","last_active":"2024-04-25T13:41:32.596477+0000","last_peered":"2024-04-25T13:41:32.596477+0000","last_clean":"2024-04-25T13:41:32.596477+0000","last_became_active":"2024-04-25T13:41:32.594815+0000","last_became_peered":"2024-04-25T13:41:32.594815+0000","last_unstale":"2024-04-25T13:41:32.596477+0000","last_undegraded":"2024-04-25T13:41:32.596477+0000","last_fullsized":"2024-04-25T13:41:32.596477+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:02.015421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570714+0000","last_change":"2024-04-25T13:41:32.570714+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570714+0000","last_undegraded":"2024-04-25T13:41:32.570714+0000","last_fullsized":"2024-04-25T13:41:32.570714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.602883+0000","last_change":"2024-04-25T13:41:32.602637+0000","last_active":"2024-04-25T13:41:32.602883+0000","last_peered":"2024-04-25T13:41:32.602883+0000","last_clean":"2024-04-25T13:41:32.602883+0000","last_became_active":"2024-04-25T13:41:32.601334+0000","last_became_peered":"2024-04-25T13:41:32.601334+0000","last_unstale":"2024-04-25T13:41:32.602883+0000","last_undegraded":"2024-04-25T13:41:32.602883+0000","last_fullsized":"2024-04-25T13:41:32.602883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:25:17.292465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569941+0000","last_change":"2024-04-25T13:41:32.569941+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569941+0000","last_undegraded":"2024-04-25T13:41:32.569941+0000","last_fullsized":"2024-04-25T13:41:32.569941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.581327+0000","last_change":"2024-04-25T13:41:32.581137+0000","last_active":"2024-04-25T13:41:32.581327+0000","last_peered":"2024-04-25T13:41:32.581327+0000","last_clean":"2024-04-25T13:41:32.581327+0000","last_became_active":"2024-04-25T13:41:32.580368+0000","last_became_peered":"2024-04-25T13:41:32.580368+0000","last_unstale":"2024-04-25T13:41:32.581327+0000","last_undegraded":"2024-04-25T13:41:32.581327+0000","last_fullsized":"2024-04-25T13:41:32.581327+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:25.222450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.570534+0000","last_change":"2024-04-25T13:41:32.570534+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.570534+0000","last_undegraded":"2024-04-25T13:41:32.570534+0000","last_fullsized":"2024-04-25T13:41:32.570534+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.588429+0000","last_change":"2024-04-25T13:41:32.588053+0000","last_active":"2024-04-25T13:41:32.588429+0000","last_peered":"2024-04-25T13:41:32.588429+0000","last_clean":"2024-04-25T13:41:32.588429+0000","last_became_active":"2024-04-25T13:41:32.587211+0000","last_became_peered":"2024-04-25T13:41:32.587211+0000","last_unstale":"2024-04-25T13:41:32.588429+0000","last_undegraded":"2024-04-25T13:41:32.588429+0000","last_fullsized":"2024-04-25T13:41:32.588429+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:35:56.623180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569593+0000","last_change":"2024-04-25T13:41:32.569593+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569593+0000","last_undegraded":"2024-04-25T13:41:32.569593+0000","last_fullsized":"2024-04-25T13:41:32.569593+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.608840+0000","last_change":"2024-04-25T13:41:32.608569+0000","last_active":"2024-04-25T13:41:32.608840+0000","last_peered":"2024-04-25T13:41:32.608840+0000","last_clean":"2024-04-25T13:41:32.608840+0000","last_became_active":"2024-04-25T13:41:32.604150+0000","last_became_peered":"2024-04-25T13:41:32.604150+0000","last_unstale":"2024-04-25T13:41:32.608840+0000","last_undegraded":"2024-04-25T13:41:32.608840+0000","last_fullsized":"2024-04-25T13:41:32.608840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:49.689785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577294+0000","last_change":"2024-04-25T13:41:32.577294+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577294+0000","last_undegraded":"2024-04-25T13:41:32.577294+0000","last_fullsized":"2024-04-25T13:41:32.577294+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.592391+0000","last_change":"2024-04-25T13:41:32.591996+0000","last_active":"2024-04-25T13:41:32.592391+0000","last_peered":"2024-04-25T13:41:32.592391+0000","last_clean":"2024-04-25T13:41:32.592391+0000","last_became_active":"2024-04-25T13:41:32.591149+0000","last_became_peered":"2024-04-25T13:41:32.591149+0000","last_unstale":"2024-04-25T13:41:32.592391+0000","last_undegraded":"2024-04-25T13:41:32.592391+0000","last_fullsized":"2024-04-25T13:41:32.592391+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:47.239360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.569832+0000","last_change":"2024-04-25T13:41:32.569832+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.569832+0000","last_undegraded":"2024-04-25T13:41:32.569832+0000","last_fullsized":"2024-04-25T13:41:32.569832+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.587494+0000","last_change":"2024-04-25T13:41:32.587338+0000","last_active":"2024-04-25T13:41:32.587494+0000","last_peered":"2024-04-25T13:41:32.587494+0000","last_clean":"2024-04-25T13:41:32.587494+0000","last_became_active":"2024-04-25T13:41:32.587021+0000","last_became_peered":"2024-04-25T13:41:32.587021+0000","last_unstale":"2024-04-25T13:41:32.587494+0000","last_undegraded":"2024-04-25T13:41:32.587494+0000","last_fullsized":"2024-04-25T13:41:32.587494+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:02.289278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.574871+0000","last_change":"2024-04-25T13:41:32.574871+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.574871+0000","last_undegraded":"2024-04-25T13:41:32.574871+0000","last_fullsized":"2024-04-25T13:41:32.574871+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.581004+0000","last_change":"2024-04-25T13:41:32.580819+0000","last_active":"2024-04-25T13:41:32.581004+0000","last_peered":"2024-04-25T13:41:32.581004+0000","last_clean":"2024-04-25T13:41:32.581004+0000","last_became_active":"2024-04-25T13:41:32.579887+0000","last_became_peered":"2024-04-25T13:41:32.579887+0000","last_unstale":"2024-04-25T13:41:32.581004+0000","last_undegraded":"2024-04-25T13:41:32.581004+0000","last_fullsized":"2024-04-25T13:41:32.581004+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:56.380154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576518+0000","last_change":"2024-04-25T13:41:32.576518+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576518+0000","last_undegraded":"2024-04-25T13:41:32.576518+0000","last_fullsized":"2024-04-25T13:41:32.576518+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.579600+0000","last_change":"2024-04-25T13:41:32.579441+0000","last_active":"2024-04-25T13:41:32.579600+0000","last_peered":"2024-04-25T13:41:32.579600+0000","last_clean":"2024-04-25T13:41:32.579600+0000","last_became_active":"2024-04-25T13:41:32.579097+0000","last_became_peered":"2024-04-25T13:41:32.579097+0000","last_unstale":"2024-04-25T13:41:32.579600+0000","last_undegraded":"2024-04-25T13:41:32.579600+0000","last_fullsized":"2024-04-25T13:41:32.579600+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:57.842622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.579544+0000","last_change":"2024-04-25T13:41:32.579544+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.579544+0000","last_undegraded":"2024-04-25T13:41:32.579544+0000","last_fullsized":"2024-04-25T13:41:32.579544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590013+0000","last_change":"2024-04-25T13:41:32.589797+0000","last_active":"2024-04-25T13:41:32.590013+0000","last_peered":"2024-04-25T13:41:32.590013+0000","last_clean":"2024-04-25T13:41:32.590013+0000","last_became_active":"2024-04-25T13:41:32.589337+0000","last_became_peered":"2024-04-25T13:41:32.589337+0000","last_unstale":"2024-04-25T13:41:32.590013+0000","last_undegraded":"2024-04-25T13:41:32.590013+0000","last_fullsized":"2024-04-25T13:41:32.590013+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.774287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.577739+0000","last_change":"2024-04-25T13:41:32.577739+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.577739+0000","last_undegraded":"2024-04-25T13:41:32.577739+0000","last_fullsized":"2024-04-25T13:41:32.577739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590236+0000","last_change":"2024-04-25T13:41:32.590039+0000","last_active":"2024-04-25T13:41:32.590236+0000","last_peered":"2024-04-25T13:41:32.590236+0000","last_clean":"2024-04-25T13:41:32.590236+0000","last_became_active":"2024-04-25T13:41:32.589673+0000","last_became_peered":"2024-04-25T13:41:32.589673+0000","last_unstale":"2024-04-25T13:41:32.590236+0000","last_undegraded":"2024-04-25T13:41:32.590236+0000","last_fullsized":"2024-04-25T13:41:32.590236+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:36.047945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.583410+0000","last_change":"2024-04-25T13:41:32.583410+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.583410+0000","last_undegraded":"2024-04-25T13:41:32.583410+0000","last_fullsized":"2024-04-25T13:41:32.583410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.605796+0000","last_change":"2024-04-25T13:41:32.605521+0000","last_active":"2024-04-25T13:41:32.605796+0000","last_peered":"2024-04-25T13:41:32.605796+0000","last_clean":"2024-04-25T13:41:32.605796+0000","last_became_active":"2024-04-25T13:41:32.603724+0000","last_became_peered":"2024-04-25T13:41:32.603724+0000","last_unstale":"2024-04-25T13:41:32.605796+0000","last_undegraded":"2024-04-25T13:41:32.605796+0000","last_fullsized":"2024-04-25T13:41:32.605796+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:59:22.787359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.596171+0000","last_change":"2024-04-25T13:41:32.596171+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.596171+0000","last_undegraded":"2024-04-25T13:41:32.596171+0000","last_fullsized":"2024-04-25T13:41:32.596171+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.590930+0000","last_change":"2024-04-25T13:41:32.590768+0000","last_active":"2024-04-25T13:41:32.590930+0000","last_peered":"2024-04-25T13:41:32.590930+0000","last_clean":"2024-04-25T13:41:32.590930+0000","last_became_active":"2024-04-25T13:41:32.589798+0000","last_became_peered":"2024-04-25T13:41:32.589798+0000","last_unstale":"2024-04-25T13:41:32.590930+0000","last_undegraded":"2024-04-25T13:41:32.590930+0000","last_fullsized":"2024-04-25T13:41:32.590930+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:42:15.102750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":31,"state":"creating+peering","last_fresh":"2024-04-25T13:41:32.576040+0000","last_change":"2024-04-25T13:41:32.576040+0000","last_active":"2024-04-25T13:41:32.548343+0000","last_peered":"2024-04-25T13:41:32.548343+0000","last_clean":"2024-04-25T13:41:32.548343+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:41:32.576040+0000","last_undegraded":"2024-04-25T13:41:32.576040+0000","last_fullsized":"2024-04-25T13:41:32.576040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.597276+0000","last_change":"2024-04-25T13:41:32.596829+0000","last_active":"2024-04-25T13:41:32.597276+0000","last_peered":"2024-04-25T13:41:32.597276+0000","last_clean":"2024-04-25T13:41:32.597276+0000","last_became_active":"2024-04-25T13:41:32.596092+0000","last_became_peered":"2024-04-25T13:41:32.596092+0000","last_unstale":"2024-04-25T13:41:32.597276+0000","last_undegraded":"2024-04-25T13:41:32.597276+0000","last_fullsized":"2024-04-25T13:41:32.597276+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:55.603960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":129,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-04-25T13:41:32.585989+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:41:32.585989+0000","last_peered":"2024-04-25T13:41:32.585989+0000","last_clean":"2024-04-25T13:41:32.585989+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:41:32.585989+0000","last_undegraded":"2024-04-25T13:41:32.585989+0000","last_fullsized":"2024-04-25T13:41:32.585989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":130,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195182,"statfs":{"total":200000000,"available":199866619,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":124,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195188,"statfs":{"total":200000000,"available":199872886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":706,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194605,"statfs":{"total":200000000,"available":199276251,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:41:42.354 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:41:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03db6ed640 1 Processor -- start 2024-04-25T13:41:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03db6ed640 1 -- start start 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03db6ed640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f03d40b0ed0 0x7f03d415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03db6ed640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f03d40ce730 0x7f03d40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03db6ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d406e490 0x7f03d406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03db6ed640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f03d406ede0 con 0x7f03d40ce730 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03db6ed640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f03d415cda0 con 0x7f03d40b0ed0 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03db6ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f03d4073710 con 0x7f03d406e490 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d8c88640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f03d40ce730 0x7f03d40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d8c88640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f03d40ce730 0x7f03d40ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53140/0 (socket says 172.21.15.80:53140) 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d8c88640 1 -- 172.21.15.80:0/4257808937 learned_addr learned my addr 172.21.15.80:0/4257808937 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d9c8a640 1 --2- 172.21.15.80:0/4257808937 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d406e490 0x7f03d406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d8c88640 1 -- 172.21.15.80:0/4257808937 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d406e490 msgr2=0x7f03d406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d8c88640 1 --2- 172.21.15.80:0/4257808937 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d406e490 0x7f03d406e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d8c88640 1 -- 172.21.15.80:0/4257808937 >> v2:172.21.15.87:3300/0 conn(0x7f03d40b0ed0 msgr2=0x7f03d415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d8c88640 1 --2- 172.21.15.80:0/4257808937 >> v2:172.21.15.87:3300/0 conn(0x7f03d40b0ed0 0x7f03d415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d8c88640 1 -- 172.21.15.80:0/4257808937 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d40739f0 con 0x7f03d40ce730 2024-04-25T13:41:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.514+0000 7f03d9c8a640 1 --2- 172.21.15.80:0/4257808937 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d406e490 0x7f03d406e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03d8c88640 1 --2- 172.21.15.80:0/4257808937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d40ce730 0x7f03d40ceb40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f03d4071650 tx=0x7f03d00211c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df6a61e204e632d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03cbfff640 1 -- 172.21.15.80:0/4257808937 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0004030 con 0x7f03d40ce730 2024-04-25T13:41:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03cbfff640 1 -- 172.21.15.80:0/4257808937 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03d00041d0 con 0x7f03d40ce730 2024-04-25T13:41:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03cbfff640 1 -- 172.21.15.80:0/4257808937 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0003620 con 0x7f03d40ce730 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 -- 172.21.15.80:0/4257808937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d40ce730 msgr2=0x7f03d40ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 --2- 172.21.15.80:0/4257808937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d40ce730 0x7f03d40ceb40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f03d4071650 tx=0x7f03d00211c0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 -- 172.21.15.80:0/4257808937 shutdown_connections 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 --2- 172.21.15.80:0/4257808937 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d406e490 0x7f03d406e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 --2- 172.21.15.80:0/4257808937 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d40ce730 0x7f03d40ceb40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 --2- 172.21.15.80:0/4257808937 >> v2:172.21.15.87:3300/0 conn(0x7f03d40b0ed0 0x7f03d415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 -- 172.21.15.80:0/4257808937 >> 172.21.15.80:0/4257808937 conn(0x7f03d406d080 msgr2=0x7f03d40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 -- 172.21.15.80:0/4257808937 shutdown_connections 2024-04-25T13:41:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 -- 172.21.15.80:0/4257808937 wait complete. 2024-04-25T13:41:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 Processor -- start 2024-04-25T13:41:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.518+0000 7f03db6ed640 1 -- start start 2024-04-25T13:41:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d406e490 0x7f03d4153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d40b0ed0 0x7f03d41591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d40ce730 0x7f03d41524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f03d40b12e0 con 0x7f03d406e490 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f03d4073fb0 con 0x7f03d40b0ed0 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f03d40733a0 con 0x7f03d40ce730 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d406e490 0x7f03d4153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d406e490 0x7f03d4153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53154/0 (socket says 172.21.15.80:53154) 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 -- 172.21.15.80:0/757411258 learned_addr learned my addr 172.21.15.80:0/757411258 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d8c88640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d40b0ed0 0x7f03d41591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9c8a640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d40ce730 0x7f03d41524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 -- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d40ce730 msgr2=0x7f03d41524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d40ce730 0x7f03d41524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 -- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d40b0ed0 msgr2=0x7f03d41591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d40b0ed0 0x7f03d41591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 -- 172.21.15.80:0/757411258 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d4156850 con 0x7f03d406e490 2024-04-25T13:41:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03d9489640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d406e490 0x7f03d4153080 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f03c400b440 tx=0x7f03c400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03c9ffb640 1 -- 172.21.15.80:0/757411258 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4015020 con 0x7f03d406e490 2024-04-25T13:41:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03c9ffb640 1 -- 172.21.15.80:0/757411258 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03c40118b0 con 0x7f03d406e490 2024-04-25T13:41:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d4156b30 con 0x7f03d406e490 2024-04-25T13:41:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03c9ffb640 1 -- 172.21.15.80:0/757411258 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4011ca0 con 0x7f03d406e490 2024-04-25T13:41:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d41570f0 con 0x7f03d406e490 2024-04-25T13:41:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d406a3a0 con 0x7f03d406e490 2024-04-25T13:41:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.522+0000 7f03c9ffb640 1 -- 172.21.15.80:0/757411258 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f03c4018020 con 0x7f03d406e490 2024-04-25T13:41:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.526+0000 7f03c9ffb640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f03b806f430 0x7f03b8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.526+0000 7f03c9ffb640 1 -- 172.21.15.80:0/757411258 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f03c408f230 con 0x7f03d406e490 2024-04-25T13:41:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.526+0000 7f03d8c88640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f03b806f430 0x7f03b8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.526+0000 7f03d8c88640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f03b806f430 0x7f03b8071940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b4000 tx=0x7f03d00ac000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.530+0000 7f03c9ffb640 1 -- 172.21.15.80:0/757411258 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f03c401f020 con 0x7f03d406e490 2024-04-25T13:41:42.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.690+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03d4069c90 con 0x7f03b806f430 2024-04-25T13:41:42.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.706+0000 7f03c9ffb640 1 -- 172.21.15.80:0/757411258 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247480 (secure 0 0 0) 0x7f03d4069c90 con 0x7f03b806f430 2024-04-25T13:41:42.713 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:42.714 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f03b806f430 msgr2=0x7f03b8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f03b806f430 0x7f03b8071940 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f03d00b4000 tx=0x7f03d00ac000 comp rx=0 tx=0).stop 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d406e490 msgr2=0x7f03d4153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d406e490 0x7f03d4153080 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f03c400b440 tx=0x7f03c400b900 comp rx=0 tx=0).stop 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 shutdown_connections 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f03b806f430 0x7f03b8071940 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03d40ce730 0x7f03d41524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03d40b0ed0 0x7f03d41591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 --2- 172.21.15.80:0/757411258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03d406e490 0x7f03d4153080 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 >> 172.21.15.80:0/757411258 conn(0x7f03d406d080 msgr2=0x7f03d40d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 shutdown_connections 2024-04-25T13:41:42.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.710+0000 7f03db6ed640 1 -- 172.21.15.80:0/757411258 wait complete. 2024-04-25T13:41:42.734 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-04-25T13:41:42.119528+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2301,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560198,"statfs":{"total":1600000000,"available":1597642796,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.550023"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630179+0000","last_change":"2024-04-25T13:41:33.582848+0000","last_active":"2024-04-25T13:41:34.630179+0000","last_peered":"2024-04-25T13:41:34.630179+0000","last_clean":"2024-04-25T13:41:34.630179+0000","last_became_active":"2024-04-25T13:41:33.582479+0000","last_became_peered":"2024-04-25T13:41:33.582479+0000","last_unstale":"2024-04-25T13:41:34.630179+0000","last_undegraded":"2024-04-25T13:41:34.630179+0000","last_fullsized":"2024-04-25T13:41:34.630179+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:21.260206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.130967+0000","last_change":"2024-04-25T13:41:33.579763+0000","last_active":"2024-04-25T13:41:35.130967+0000","last_peered":"2024-04-25T13:41:35.130967+0000","last_clean":"2024-04-25T13:41:35.130967+0000","last_became_active":"2024-04-25T13:41:33.579476+0000","last_became_peered":"2024-04-25T13:41:33.579476+0000","last_unstale":"2024-04-25T13:41:35.130967+0000","last_undegraded":"2024-04-25T13:41:35.130967+0000","last_fullsized":"2024-04-25T13:41:35.130967+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:43.374566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131895+0000","last_change":"2024-04-25T13:41:33.590278+0000","last_active":"2024-04-25T13:41:35.131895+0000","last_peered":"2024-04-25T13:41:35.131895+0000","last_clean":"2024-04-25T13:41:35.131895+0000","last_became_active":"2024-04-25T13:41:33.589554+0000","last_became_peered":"2024-04-25T13:41:33.589554+0000","last_unstale":"2024-04-25T13:41:35.131895+0000","last_undegraded":"2024-04-25T13:41:35.131895+0000","last_fullsized":"2024-04-25T13:41:35.131895+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:16.617620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.629633+0000","last_change":"2024-04-25T13:41:33.586231+0000","last_active":"2024-04-25T13:41:34.629633+0000","last_peered":"2024-04-25T13:41:34.629633+0000","last_clean":"2024-04-25T13:41:34.629633+0000","last_became_active":"2024-04-25T13:41:33.585100+0000","last_became_peered":"2024-04-25T13:41:33.585100+0000","last_unstale":"2024-04-25T13:41:34.629633+0000","last_undegraded":"2024-04-25T13:41:34.629633+0000","last_fullsized":"2024-04-25T13:41:34.629633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:22.132058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.575421+0000","last_change":"2024-04-25T13:41:33.594487+0000","last_active":"2024-04-25T13:41:34.575421+0000","last_peered":"2024-04-25T13:41:34.575421+0000","last_clean":"2024-04-25T13:41:34.575421+0000","last_became_active":"2024-04-25T13:41:33.593864+0000","last_became_peered":"2024-04-25T13:41:33.593864+0000","last_unstale":"2024-04-25T13:41:34.575421+0000","last_undegraded":"2024-04-25T13:41:34.575421+0000","last_fullsized":"2024-04-25T13:41:34.575421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:46.434182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131180+0000","last_change":"2024-04-25T13:41:33.586691+0000","last_active":"2024-04-25T13:41:35.131180+0000","last_peered":"2024-04-25T13:41:35.131180+0000","last_clean":"2024-04-25T13:41:35.131180+0000","last_became_active":"2024-04-25T13:41:33.583398+0000","last_became_peered":"2024-04-25T13:41:33.583398+0000","last_unstale":"2024-04-25T13:41:35.131180+0000","last_undegraded":"2024-04-25T13:41:35.131180+0000","last_fullsized":"2024-04-25T13:41:35.131180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:39:36.724272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.133061+0000","last_change":"2024-04-25T13:41:33.588723+0000","last_active":"2024-04-25T13:41:35.133061+0000","last_peered":"2024-04-25T13:41:35.133061+0000","last_clean":"2024-04-25T13:41:35.133061+0000","last_became_active":"2024-04-25T13:41:33.588184+0000","last_became_peered":"2024-04-25T13:41:33.588184+0000","last_unstale":"2024-04-25T13:41:35.133061+0000","last_undegraded":"2024-04-25T13:41:35.133061+0000","last_fullsized":"2024-04-25T13:41:35.133061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:32.061831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132140+0000","last_change":"2024-04-25T13:41:33.592554+0000","last_active":"2024-04-25T13:41:35.132140+0000","last_peered":"2024-04-25T13:41:35.132140+0000","last_clean":"2024-04-25T13:41:35.132140+0000","last_became_active":"2024-04-25T13:41:33.592219+0000","last_became_peered":"2024-04-25T13:41:33.592219+0000","last_unstale":"2024-04-25T13:41:35.132140+0000","last_undegraded":"2024-04-25T13:41:35.132140+0000","last_fullsized":"2024-04-25T13:41:35.132140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:58.092260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050321+0000","last_change":"2024-04-25T13:41:33.581689+0000","last_active":"2024-04-25T13:41:35.050321+0000","last_peered":"2024-04-25T13:41:35.050321+0000","last_clean":"2024-04-25T13:41:35.050321+0000","last_became_active":"2024-04-25T13:41:33.581052+0000","last_became_peered":"2024-04-25T13:41:33.581052+0000","last_unstale":"2024-04-25T13:41:35.050321+0000","last_undegraded":"2024-04-25T13:41:35.050321+0000","last_fullsized":"2024-04-25T13:41:35.050321+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:27:23.500221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630284+0000","last_change":"2024-04-25T13:41:33.595058+0000","last_active":"2024-04-25T13:41:34.630284+0000","last_peered":"2024-04-25T13:41:34.630284+0000","last_clean":"2024-04-25T13:41:34.630284+0000","last_became_active":"2024-04-25T13:41:33.594290+0000","last_became_peered":"2024-04-25T13:41:33.594290+0000","last_unstale":"2024-04-25T13:41:34.630284+0000","last_undegraded":"2024-04-25T13:41:34.630284+0000","last_fullsized":"2024-04-25T13:41:34.630284+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:42.311438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132412+0000","last_change":"2024-04-25T13:41:33.586298+0000","last_active":"2024-04-25T13:41:35.132412+0000","last_peered":"2024-04-25T13:41:35.132412+0000","last_clean":"2024-04-25T13:41:35.132412+0000","last_became_active":"2024-04-25T13:41:33.586013+0000","last_became_peered":"2024-04-25T13:41:33.586013+0000","last_unstale":"2024-04-25T13:41:35.132412+0000","last_undegraded":"2024-04-25T13:41:35.132412+0000","last_fullsized":"2024-04-25T13:41:35.132412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:56.417471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.130636+0000","last_change":"2024-04-25T13:41:33.595356+0000","last_active":"2024-04-25T13:41:35.130636+0000","last_peered":"2024-04-25T13:41:35.130636+0000","last_clean":"2024-04-25T13:41:35.130636+0000","last_became_active":"2024-04-25T13:41:33.594740+0000","last_became_peered":"2024-04-25T13:41:33.594740+0000","last_unstale":"2024-04-25T13:41:35.130636+0000","last_undegraded":"2024-04-25T13:41:35.130636+0000","last_fullsized":"2024-04-25T13:41:35.130636+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:12.588387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.575061+0000","last_change":"2024-04-25T13:41:33.589463+0000","last_active":"2024-04-25T13:41:34.575061+0000","last_peered":"2024-04-25T13:41:34.575061+0000","last_clean":"2024-04-25T13:41:34.575061+0000","last_became_active":"2024-04-25T13:41:33.589198+0000","last_became_peered":"2024-04-25T13:41:33.589198+0000","last_unstale":"2024-04-25T13:41:34.575061+0000","last_undegraded":"2024-04-25T13:41:34.575061+0000","last_fullsized":"2024-04-25T13:41:34.575061+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:29.927336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131409+0000","last_change":"2024-04-25T13:41:33.598165+0000","last_active":"2024-04-25T13:41:35.131409+0000","last_peered":"2024-04-25T13:41:35.131409+0000","last_clean":"2024-04-25T13:41:35.131409+0000","last_became_active":"2024-04-25T13:41:33.595525+0000","last_became_peered":"2024-04-25T13:41:33.595525+0000","last_unstale":"2024-04-25T13:41:35.131409+0000","last_undegraded":"2024-04-25T13:41:35.131409+0000","last_fullsized":"2024-04-25T13:41:35.131409+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:57.913926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131084+0000","last_change":"2024-04-25T13:41:33.585751+0000","last_active":"2024-04-25T13:41:35.131084+0000","last_peered":"2024-04-25T13:41:35.131084+0000","last_clean":"2024-04-25T13:41:35.131084+0000","last_became_active":"2024-04-25T13:41:33.585424+0000","last_became_peered":"2024-04-25T13:41:33.585424+0000","last_unstale":"2024-04-25T13:41:35.131084+0000","last_undegraded":"2024-04-25T13:41:35.131084+0000","last_fullsized":"2024-04-25T13:41:35.131084+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:24.363161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132865+0000","last_change":"2024-04-25T13:41:33.594551+0000","last_active":"2024-04-25T13:41:35.132865+0000","last_peered":"2024-04-25T13:41:35.132865+0000","last_clean":"2024-04-25T13:41:35.132865+0000","last_became_active":"2024-04-25T13:41:33.594149+0000","last_became_peered":"2024-04-25T13:41:33.594149+0000","last_unstale":"2024-04-25T13:41:35.132865+0000","last_undegraded":"2024-04-25T13:41:35.132865+0000","last_fullsized":"2024-04-25T13:41:35.132865+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:58:34.787785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131689+0000","last_change":"2024-04-25T13:41:33.588457+0000","last_active":"2024-04-25T13:41:35.131689+0000","last_peered":"2024-04-25T13:41:35.131689+0000","last_clean":"2024-04-25T13:41:35.131689+0000","last_became_active":"2024-04-25T13:41:33.587691+0000","last_became_peered":"2024-04-25T13:41:33.587691+0000","last_unstale":"2024-04-25T13:41:35.131689+0000","last_undegraded":"2024-04-25T13:41:35.131689+0000","last_fullsized":"2024-04-25T13:41:35.131689+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:56.038009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.130680+0000","last_change":"2024-04-25T13:41:33.592270+0000","last_active":"2024-04-25T13:41:35.130680+0000","last_peered":"2024-04-25T13:41:35.130680+0000","last_clean":"2024-04-25T13:41:35.130680+0000","last_became_active":"2024-04-25T13:41:33.587629+0000","last_became_peered":"2024-04-25T13:41:33.587629+0000","last_unstale":"2024-04-25T13:41:35.130680+0000","last_undegraded":"2024-04-25T13:41:35.130680+0000","last_fullsized":"2024-04-25T13:41:35.130680+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:54.899213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.130372+0000","last_change":"2024-04-25T13:41:33.597896+0000","last_active":"2024-04-25T13:41:35.130372+0000","last_peered":"2024-04-25T13:41:35.130372+0000","last_clean":"2024-04-25T13:41:35.130372+0000","last_became_active":"2024-04-25T13:41:33.594123+0000","last_became_peered":"2024-04-25T13:41:33.594123+0000","last_unstale":"2024-04-25T13:41:35.130372+0000","last_undegraded":"2024-04-25T13:41:35.130372+0000","last_fullsized":"2024-04-25T13:41:35.130372+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:30.278036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.575825+0000","last_change":"2024-04-25T13:41:33.592007+0000","last_active":"2024-04-25T13:41:34.575825+0000","last_peered":"2024-04-25T13:41:34.575825+0000","last_clean":"2024-04-25T13:41:34.575825+0000","last_became_active":"2024-04-25T13:41:33.591637+0000","last_became_peered":"2024-04-25T13:41:33.591637+0000","last_unstale":"2024-04-25T13:41:34.575825+0000","last_undegraded":"2024-04-25T13:41:34.575825+0000","last_fullsized":"2024-04-25T13:41:34.575825+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:26.662453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.049819+0000","last_change":"2024-04-25T13:41:33.581994+0000","last_active":"2024-04-25T13:41:35.049819+0000","last_peered":"2024-04-25T13:41:35.049819+0000","last_clean":"2024-04-25T13:41:35.049819+0000","last_became_active":"2024-04-25T13:41:33.581586+0000","last_became_peered":"2024-04-25T13:41:33.581586+0000","last_unstale":"2024-04-25T13:41:35.049819+0000","last_undegraded":"2024-04-25T13:41:35.049819+0000","last_fullsized":"2024-04-25T13:41:35.049819+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:53:26.422890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.051089+0000","last_change":"2024-04-25T13:41:33.574607+0000","last_active":"2024-04-25T13:41:35.051089+0000","last_peered":"2024-04-25T13:41:35.051089+0000","last_clean":"2024-04-25T13:41:35.051089+0000","last_became_active":"2024-04-25T13:41:33.574271+0000","last_became_peered":"2024-04-25T13:41:33.574271+0000","last_unstale":"2024-04-25T13:41:35.051089+0000","last_undegraded":"2024-04-25T13:41:35.051089+0000","last_fullsized":"2024-04-25T13:41:35.051089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:38.343546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.571313+0000","last_change":"2024-04-25T13:41:33.584694+0000","last_active":"2024-04-25T13:41:34.571313+0000","last_peered":"2024-04-25T13:41:34.571313+0000","last_clean":"2024-04-25T13:41:34.571313+0000","last_became_active":"2024-04-25T13:41:33.584417+0000","last_became_peered":"2024-04-25T13:41:33.584417+0000","last_unstale":"2024-04-25T13:41:34.571313+0000","last_undegraded":"2024-04-25T13:41:34.571313+0000","last_fullsized":"2024-04-25T13:41:34.571313+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:20.520548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.130513+0000","last_change":"2024-04-25T13:41:33.593568+0000","last_active":"2024-04-25T13:41:35.130513+0000","last_peered":"2024-04-25T13:41:35.130513+0000","last_clean":"2024-04-25T13:41:35.130513+0000","last_became_active":"2024-04-25T13:41:33.593249+0000","last_became_peered":"2024-04-25T13:41:33.593249+0000","last_unstale":"2024-04-25T13:41:35.130513+0000","last_undegraded":"2024-04-25T13:41:35.130513+0000","last_fullsized":"2024-04-25T13:41:35.130513+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:25:13.032057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.629420+0000","last_change":"2024-04-25T13:41:33.589364+0000","last_active":"2024-04-25T13:41:34.629420+0000","last_peered":"2024-04-25T13:41:34.629420+0000","last_clean":"2024-04-25T13:41:34.629420+0000","last_became_active":"2024-04-25T13:41:33.588702+0000","last_became_peered":"2024-04-25T13:41:33.588702+0000","last_unstale":"2024-04-25T13:41:34.629420+0000","last_undegraded":"2024-04-25T13:41:34.629420+0000","last_fullsized":"2024-04-25T13:41:34.629420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:31.027611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131646+0000","last_change":"2024-04-25T13:41:33.580396+0000","last_active":"2024-04-25T13:41:35.131646+0000","last_peered":"2024-04-25T13:41:35.131646+0000","last_clean":"2024-04-25T13:41:35.131646+0000","last_became_active":"2024-04-25T13:41:33.579534+0000","last_became_peered":"2024-04-25T13:41:33.579534+0000","last_unstale":"2024-04-25T13:41:35.131646+0000","last_undegraded":"2024-04-25T13:41:35.131646+0000","last_fullsized":"2024-04-25T13:41:35.131646+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:24.475558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.629980+0000","last_change":"2024-04-25T13:41:33.588857+0000","last_active":"2024-04-25T13:41:34.629980+0000","last_peered":"2024-04-25T13:41:34.629980+0000","last_clean":"2024-04-25T13:41:34.629980+0000","last_became_active":"2024-04-25T13:41:33.586180+0000","last_became_peered":"2024-04-25T13:41:33.586180+0000","last_unstale":"2024-04-25T13:41:34.629980+0000","last_undegraded":"2024-04-25T13:41:34.629980+0000","last_fullsized":"2024-04-25T13:41:34.629980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:42.661623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.130739+0000","last_change":"2024-04-25T13:41:33.585306+0000","last_active":"2024-04-25T13:41:35.130739+0000","last_peered":"2024-04-25T13:41:35.130739+0000","last_clean":"2024-04-25T13:41:35.130739+0000","last_became_active":"2024-04-25T13:41:33.584398+0000","last_became_peered":"2024-04-25T13:41:33.584398+0000","last_unstale":"2024-04-25T13:41:35.130739+0000","last_undegraded":"2024-04-25T13:41:35.130739+0000","last_fullsized":"2024-04-25T13:41:35.130739+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:46.593081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050867+0000","last_change":"2024-04-25T13:41:33.598401+0000","last_active":"2024-04-25T13:41:35.050867+0000","last_peered":"2024-04-25T13:41:35.050867+0000","last_clean":"2024-04-25T13:41:35.050867+0000","last_became_active":"2024-04-25T13:41:33.598041+0000","last_became_peered":"2024-04-25T13:41:33.598041+0000","last_unstale":"2024-04-25T13:41:35.050867+0000","last_undegraded":"2024-04-25T13:41:35.050867+0000","last_fullsized":"2024-04-25T13:41:35.050867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:12.944523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132019+0000","last_change":"2024-04-25T13:41:33.587421+0000","last_active":"2024-04-25T13:41:35.132019+0000","last_peered":"2024-04-25T13:41:35.132019+0000","last_clean":"2024-04-25T13:41:35.132019+0000","last_became_active":"2024-04-25T13:41:33.587089+0000","last_became_peered":"2024-04-25T13:41:33.587089+0000","last_unstale":"2024-04-25T13:41:35.132019+0000","last_undegraded":"2024-04-25T13:41:35.132019+0000","last_fullsized":"2024-04-25T13:41:35.132019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:39:31.687482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630356+0000","last_change":"2024-04-25T13:41:33.588398+0000","last_active":"2024-04-25T13:41:34.630356+0000","last_peered":"2024-04-25T13:41:34.630356+0000","last_clean":"2024-04-25T13:41:34.630356+0000","last_became_active":"2024-04-25T13:41:33.588102+0000","last_became_peered":"2024-04-25T13:41:33.588102+0000","last_unstale":"2024-04-25T13:41:34.630356+0000","last_undegraded":"2024-04-25T13:41:34.630356+0000","last_fullsized":"2024-04-25T13:41:34.630356+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:56.379642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132165+0000","last_change":"2024-04-25T13:41:33.584528+0000","last_active":"2024-04-25T13:41:35.132165+0000","last_peered":"2024-04-25T13:41:35.132165+0000","last_clean":"2024-04-25T13:41:35.132165+0000","last_became_active":"2024-04-25T13:41:33.584226+0000","last_became_peered":"2024-04-25T13:41:33.584226+0000","last_unstale":"2024-04-25T13:41:35.132165+0000","last_undegraded":"2024-04-25T13:41:35.132165+0000","last_fullsized":"2024-04-25T13:41:35.132165+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:20.654318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.571590+0000","last_change":"2024-04-25T13:41:33.584104+0000","last_active":"2024-04-25T13:41:34.571590+0000","last_peered":"2024-04-25T13:41:34.571590+0000","last_clean":"2024-04-25T13:41:34.571590+0000","last_became_active":"2024-04-25T13:41:33.583499+0000","last_became_peered":"2024-04-25T13:41:33.583499+0000","last_unstale":"2024-04-25T13:41:34.571590+0000","last_undegraded":"2024-04-25T13:41:34.571590+0000","last_fullsized":"2024-04-25T13:41:34.571590+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:41.286192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630401+0000","last_change":"2024-04-25T13:41:32.591289+0000","last_active":"2024-04-25T13:41:34.630401+0000","last_peered":"2024-04-25T13:41:34.630401+0000","last_clean":"2024-04-25T13:41:34.630401+0000","last_became_active":"2024-04-25T13:41:32.590480+0000","last_became_peered":"2024-04-25T13:41:32.590480+0000","last_unstale":"2024-04-25T13:41:34.630401+0000","last_undegraded":"2024-04-25T13:41:34.630401+0000","last_fullsized":"2024-04-25T13:41:34.630401+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:38:15.509589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630074+0000","last_change":"2024-04-25T13:41:33.587054+0000","last_active":"2024-04-25T13:41:34.630074+0000","last_peered":"2024-04-25T13:41:34.630074+0000","last_clean":"2024-04-25T13:41:34.630074+0000","last_became_active":"2024-04-25T13:41:33.584833+0000","last_became_peered":"2024-04-25T13:41:33.584833+0000","last_unstale":"2024-04-25T13:41:34.630074+0000","last_undegraded":"2024-04-25T13:41:34.630074+0000","last_fullsized":"2024-04-25T13:41:34.630074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:48.153990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.571150+0000","last_change":"2024-04-25T13:41:32.590216+0000","last_active":"2024-04-25T13:41:34.571150+0000","last_peered":"2024-04-25T13:41:34.571150+0000","last_clean":"2024-04-25T13:41:34.571150+0000","last_became_active":"2024-04-25T13:41:32.589855+0000","last_became_peered":"2024-04-25T13:41:32.589855+0000","last_unstale":"2024-04-25T13:41:34.571150+0000","last_undegraded":"2024-04-25T13:41:34.571150+0000","last_fullsized":"2024-04-25T13:41:34.571150+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:27.119682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050335+0000","last_change":"2024-04-25T13:41:33.594134+0000","last_active":"2024-04-25T13:41:35.050335+0000","last_peered":"2024-04-25T13:41:35.050335+0000","last_clean":"2024-04-25T13:41:35.050335+0000","last_became_active":"2024-04-25T13:41:33.593813+0000","last_became_peered":"2024-04-25T13:41:33.593813+0000","last_unstale":"2024-04-25T13:41:35.050335+0000","last_undegraded":"2024-04-25T13:41:35.050335+0000","last_fullsized":"2024-04-25T13:41:35.050335+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:47.089900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131590+0000","last_change":"2024-04-25T13:41:32.578789+0000","last_active":"2024-04-25T13:41:35.131590+0000","last_peered":"2024-04-25T13:41:35.131590+0000","last_clean":"2024-04-25T13:41:35.131590+0000","last_became_active":"2024-04-25T13:41:32.578278+0000","last_became_peered":"2024-04-25T13:41:32.578278+0000","last_unstale":"2024-04-25T13:41:35.131590+0000","last_undegraded":"2024-04-25T13:41:35.131590+0000","last_fullsized":"2024-04-25T13:41:35.131590+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:00.106743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.629862+0000","last_change":"2024-04-25T13:41:33.591734+0000","last_active":"2024-04-25T13:41:34.629862+0000","last_peered":"2024-04-25T13:41:34.629862+0000","last_clean":"2024-04-25T13:41:34.629862+0000","last_became_active":"2024-04-25T13:41:33.590954+0000","last_became_peered":"2024-04-25T13:41:33.590954+0000","last_unstale":"2024-04-25T13:41:34.629862+0000","last_undegraded":"2024-04-25T13:41:34.629862+0000","last_fullsized":"2024-04-25T13:41:34.629862+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:10.000224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.576020+0000","last_change":"2024-04-25T13:41:32.577475+0000","last_active":"2024-04-25T13:41:34.576020+0000","last_peered":"2024-04-25T13:41:34.576020+0000","last_clean":"2024-04-25T13:41:34.576020+0000","last_became_active":"2024-04-25T13:41:32.576661+0000","last_became_peered":"2024-04-25T13:41:32.576661+0000","last_unstale":"2024-04-25T13:41:34.576020+0000","last_undegraded":"2024-04-25T13:41:34.576020+0000","last_fullsized":"2024-04-25T13:41:34.576020+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:05.047364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132717+0000","last_change":"2024-04-25T13:41:33.587494+0000","last_active":"2024-04-25T13:41:35.132717+0000","last_peered":"2024-04-25T13:41:35.132717+0000","last_clean":"2024-04-25T13:41:35.132717+0000","last_became_active":"2024-04-25T13:41:33.586775+0000","last_became_peered":"2024-04-25T13:41:33.586775+0000","last_unstale":"2024-04-25T13:41:35.132717+0000","last_undegraded":"2024-04-25T13:41:35.132717+0000","last_fullsized":"2024-04-25T13:41:35.132717+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:38.906133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050506+0000","last_change":"2024-04-25T13:41:32.602990+0000","last_active":"2024-04-25T13:41:35.050506+0000","last_peered":"2024-04-25T13:41:35.050506+0000","last_clean":"2024-04-25T13:41:35.050506+0000","last_became_active":"2024-04-25T13:41:32.602174+0000","last_became_peered":"2024-04-25T13:41:32.602174+0000","last_unstale":"2024-04-25T13:41:35.050506+0000","last_undegraded":"2024-04-25T13:41:35.050506+0000","last_fullsized":"2024-04-25T13:41:35.050506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:27.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131123+0000","last_change":"2024-04-25T13:41:33.583973+0000","last_active":"2024-04-25T13:41:35.131123+0000","last_peered":"2024-04-25T13:41:35.131123+0000","last_clean":"2024-04-25T13:41:35.131123+0000","last_became_active":"2024-04-25T13:41:33.583677+0000","last_became_peered":"2024-04-25T13:41:33.583677+0000","last_unstale":"2024-04-25T13:41:35.131123+0000","last_undegraded":"2024-04-25T13:41:35.131123+0000","last_fullsized":"2024-04-25T13:41:35.131123+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:05.564972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050278+0000","last_change":"2024-04-25T13:41:32.606199+0000","last_active":"2024-04-25T13:41:35.050278+0000","last_peered":"2024-04-25T13:41:35.050278+0000","last_clean":"2024-04-25T13:41:35.050278+0000","last_became_active":"2024-04-25T13:41:32.604933+0000","last_became_peered":"2024-04-25T13:41:32.604933+0000","last_unstale":"2024-04-25T13:41:35.050278+0000","last_undegraded":"2024-04-25T13:41:35.050278+0000","last_fullsized":"2024-04-25T13:41:35.050278+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:37.164479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.049951+0000","last_change":"2024-04-25T13:41:33.590787+0000","last_active":"2024-04-25T13:41:35.049951+0000","last_peered":"2024-04-25T13:41:35.049951+0000","last_clean":"2024-04-25T13:41:35.049951+0000","last_became_active":"2024-04-25T13:41:33.587750+0000","last_became_peered":"2024-04-25T13:41:33.587750+0000","last_unstale":"2024-04-25T13:41:35.049951+0000","last_undegraded":"2024-04-25T13:41:35.049951+0000","last_fullsized":"2024-04-25T13:41:35.049951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:16.608151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131671+0000","last_change":"2024-04-25T13:41:32.586286+0000","last_active":"2024-04-25T13:41:35.131671+0000","last_peered":"2024-04-25T13:41:35.131671+0000","last_clean":"2024-04-25T13:41:35.131671+0000","last_became_active":"2024-04-25T13:41:32.584859+0000","last_became_peered":"2024-04-25T13:41:32.584859+0000","last_unstale":"2024-04-25T13:41:35.131671+0000","last_undegraded":"2024-04-25T13:41:35.131671+0000","last_fullsized":"2024-04-25T13:41:35.131671+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:38.138128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630065+0000","last_change":"2024-04-25T13:41:33.586611+0000","last_active":"2024-04-25T13:41:34.630065+0000","last_peered":"2024-04-25T13:41:34.630065+0000","last_clean":"2024-04-25T13:41:34.630065+0000","last_became_active":"2024-04-25T13:41:33.585664+0000","last_became_peered":"2024-04-25T13:41:33.585664+0000","last_unstale":"2024-04-25T13:41:34.630065+0000","last_undegraded":"2024-04-25T13:41:34.630065+0000","last_fullsized":"2024-04-25T13:41:34.630065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:52.263623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.051042+0000","last_change":"2024-04-25T13:41:32.610199+0000","last_active":"2024-04-25T13:41:35.051042+0000","last_peered":"2024-04-25T13:41:35.051042+0000","last_clean":"2024-04-25T13:41:35.051042+0000","last_became_active":"2024-04-25T13:41:32.607495+0000","last_became_peered":"2024-04-25T13:41:32.607495+0000","last_unstale":"2024-04-25T13:41:35.051042+0000","last_undegraded":"2024-04-25T13:41:35.051042+0000","last_fullsized":"2024-04-25T13:41:35.051042+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:54:11.256982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132377+0000","last_change":"2024-04-25T13:41:33.595071+0000","last_active":"2024-04-25T13:41:35.132377+0000","last_peered":"2024-04-25T13:41:35.132377+0000","last_clean":"2024-04-25T13:41:35.132377+0000","last_became_active":"2024-04-25T13:41:33.594228+0000","last_became_peered":"2024-04-25T13:41:33.594228+0000","last_unstale":"2024-04-25T13:41:35.132377+0000","last_undegraded":"2024-04-25T13:41:35.132377+0000","last_fullsized":"2024-04-25T13:41:35.132377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:40:04.007024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132279+0000","last_change":"2024-04-25T13:41:32.591386+0000","last_active":"2024-04-25T13:41:35.132279+0000","last_peered":"2024-04-25T13:41:35.132279+0000","last_clean":"2024-04-25T13:41:35.132279+0000","last_became_active":"2024-04-25T13:41:32.591004+0000","last_became_peered":"2024-04-25T13:41:32.591004+0000","last_unstale":"2024-04-25T13:41:35.132279+0000","last_undegraded":"2024-04-25T13:41:35.132279+0000","last_fullsized":"2024-04-25T13:41:35.132279+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:48.601675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132055+0000","last_change":"2024-04-25T13:41:33.580035+0000","last_active":"2024-04-25T13:41:35.132055+0000","last_peered":"2024-04-25T13:41:35.132055+0000","last_clean":"2024-04-25T13:41:35.132055+0000","last_became_active":"2024-04-25T13:41:33.579068+0000","last_became_peered":"2024-04-25T13:41:33.579068+0000","last_unstale":"2024-04-25T13:41:35.132055+0000","last_undegraded":"2024-04-25T13:41:35.132055+0000","last_fullsized":"2024-04-25T13:41:35.132055+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:10:38.615139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.571099+0000","last_change":"2024-04-25T13:41:32.577747+0000","last_active":"2024-04-25T13:41:34.571099+0000","last_peered":"2024-04-25T13:41:34.571099+0000","last_clean":"2024-04-25T13:41:34.571099+0000","last_became_active":"2024-04-25T13:41:32.577358+0000","last_became_peered":"2024-04-25T13:41:32.577358+0000","last_unstale":"2024-04-25T13:41:34.571099+0000","last_undegraded":"2024-04-25T13:41:34.571099+0000","last_fullsized":"2024-04-25T13:41:34.571099+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:14.017616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132202+0000","last_change":"2024-04-25T13:41:33.597082+0000","last_active":"2024-04-25T13:41:35.132202+0000","last_peered":"2024-04-25T13:41:35.132202+0000","last_clean":"2024-04-25T13:41:35.132202+0000","last_became_active":"2024-04-25T13:41:33.593399+0000","last_became_peered":"2024-04-25T13:41:33.593399+0000","last_unstale":"2024-04-25T13:41:35.132202+0000","last_undegraded":"2024-04-25T13:41:35.132202+0000","last_fullsized":"2024-04-25T13:41:35.132202+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:14:47.665218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132199+0000","last_change":"2024-04-25T13:41:32.591430+0000","last_active":"2024-04-25T13:41:35.132199+0000","last_peered":"2024-04-25T13:41:35.132199+0000","last_clean":"2024-04-25T13:41:35.132199+0000","last_became_active":"2024-04-25T13:41:32.590686+0000","last_became_peered":"2024-04-25T13:41:32.590686+0000","last_unstale":"2024-04-25T13:41:35.132199+0000","last_undegraded":"2024-04-25T13:41:35.132199+0000","last_fullsized":"2024-04-25T13:41:35.132199+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:11.567956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131456+0000","last_change":"2024-04-25T13:41:33.590533+0000","last_active":"2024-04-25T13:41:35.131456+0000","last_peered":"2024-04-25T13:41:35.131456+0000","last_clean":"2024-04-25T13:41:35.131456+0000","last_became_active":"2024-04-25T13:41:33.589985+0000","last_became_peered":"2024-04-25T13:41:33.589985+0000","last_unstale":"2024-04-25T13:41:35.131456+0000","last_undegraded":"2024-04-25T13:41:35.131456+0000","last_fullsized":"2024-04-25T13:41:35.131456+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:49.716411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":20,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131904+0000","last_change":"2024-04-25T13:41:32.580227+0000","last_active":"2024-04-25T13:41:35.131904+0000","last_peered":"2024-04-25T13:41:35.131904+0000","last_clean":"2024-04-25T13:41:35.131904+0000","last_became_active":"2024-04-25T13:41:32.579922+0000","last_became_peered":"2024-04-25T13:41:32.579922+0000","last_unstale":"2024-04-25T13:41:35.131904+0000","last_undegraded":"2024-04-25T13:41:35.131904+0000","last_fullsized":"2024-04-25T13:41:35.131904+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:09.088585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.629535+0000","last_change":"2024-04-25T13:41:33.583212+0000","last_active":"2024-04-25T13:41:34.629535+0000","last_peered":"2024-04-25T13:41:34.629535+0000","last_clean":"2024-04-25T13:41:34.629535+0000","last_became_active":"2024-04-25T13:41:33.582951+0000","last_became_peered":"2024-04-25T13:41:33.582951+0000","last_unstale":"2024-04-25T13:41:34.629535+0000","last_undegraded":"2024-04-25T13:41:34.629535+0000","last_fullsized":"2024-04-25T13:41:34.629535+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:33.299420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.051416+0000","last_change":"2024-04-25T13:41:32.609679+0000","last_active":"2024-04-25T13:41:35.051416+0000","last_peered":"2024-04-25T13:41:35.051416+0000","last_clean":"2024-04-25T13:41:35.051416+0000","last_became_active":"2024-04-25T13:41:32.606623+0000","last_became_peered":"2024-04-25T13:41:32.606623+0000","last_unstale":"2024-04-25T13:41:35.051416+0000","last_undegraded":"2024-04-25T13:41:35.051416+0000","last_fullsized":"2024-04-25T13:41:35.051416+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:21.985685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131092+0000","last_change":"2024-04-25T13:41:33.577929+0000","last_active":"2024-04-25T13:41:35.131092+0000","last_peered":"2024-04-25T13:41:35.131092+0000","last_clean":"2024-04-25T13:41:35.131092+0000","last_became_active":"2024-04-25T13:41:33.577406+0000","last_became_peered":"2024-04-25T13:41:33.577406+0000","last_unstale":"2024-04-25T13:41:35.131092+0000","last_undegraded":"2024-04-25T13:41:35.131092+0000","last_fullsized":"2024-04-25T13:41:35.131092+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:51.396150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630005+0000","last_change":"2024-04-25T13:41:32.590694+0000","last_active":"2024-04-25T13:41:34.630005+0000","last_peered":"2024-04-25T13:41:34.630005+0000","last_clean":"2024-04-25T13:41:34.630005+0000","last_became_active":"2024-04-25T13:41:32.590377+0000","last_became_peered":"2024-04-25T13:41:32.590377+0000","last_unstale":"2024-04-25T13:41:34.630005+0000","last_undegraded":"2024-04-25T13:41:34.630005+0000","last_fullsized":"2024-04-25T13:41:34.630005+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:01.539254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132984+0000","last_change":"2024-04-25T13:41:33.588069+0000","last_active":"2024-04-25T13:41:35.132984+0000","last_peered":"2024-04-25T13:41:35.132984+0000","last_clean":"2024-04-25T13:41:35.132984+0000","last_became_active":"2024-04-25T13:41:33.587194+0000","last_became_peered":"2024-04-25T13:41:33.587194+0000","last_unstale":"2024-04-25T13:41:35.132984+0000","last_undegraded":"2024-04-25T13:41:35.132984+0000","last_fullsized":"2024-04-25T13:41:35.132984+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:56.990306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131996+0000","last_change":"2024-04-25T13:41:32.590186+0000","last_active":"2024-04-25T13:41:35.131996+0000","last_peered":"2024-04-25T13:41:35.131996+0000","last_clean":"2024-04-25T13:41:35.131996+0000","last_became_active":"2024-04-25T13:41:32.589854+0000","last_became_peered":"2024-04-25T13:41:32.589854+0000","last_unstale":"2024-04-25T13:41:35.131996+0000","last_undegraded":"2024-04-25T13:41:35.131996+0000","last_fullsized":"2024-04-25T13:41:35.131996+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:35.263381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630735+0000","last_change":"2024-04-25T13:41:33.587589+0000","last_active":"2024-04-25T13:41:34.630735+0000","last_peered":"2024-04-25T13:41:34.630735+0000","last_clean":"2024-04-25T13:41:34.630735+0000","last_became_active":"2024-04-25T13:41:33.587160+0000","last_became_peered":"2024-04-25T13:41:33.587160+0000","last_unstale":"2024-04-25T13:41:34.630735+0000","last_undegraded":"2024-04-25T13:41:34.630735+0000","last_fullsized":"2024-04-25T13:41:34.630735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:47.981243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.130924+0000","last_change":"2024-04-25T13:41:32.578797+0000","last_active":"2024-04-25T13:41:35.130924+0000","last_peered":"2024-04-25T13:41:35.130924+0000","last_clean":"2024-04-25T13:41:35.130924+0000","last_became_active":"2024-04-25T13:41:32.578411+0000","last_became_peered":"2024-04-25T13:41:32.578411+0000","last_unstale":"2024-04-25T13:41:35.130924+0000","last_undegraded":"2024-04-25T13:41:35.130924+0000","last_fullsized":"2024-04-25T13:41:35.130924+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:24.610890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050724+0000","last_change":"2024-04-25T13:41:33.591086+0000","last_active":"2024-04-25T13:41:35.050724+0000","last_peered":"2024-04-25T13:41:35.050724+0000","last_clean":"2024-04-25T13:41:35.050724+0000","last_became_active":"2024-04-25T13:41:33.590376+0000","last_became_peered":"2024-04-25T13:41:33.590376+0000","last_unstale":"2024-04-25T13:41:35.050724+0000","last_undegraded":"2024-04-25T13:41:35.050724+0000","last_fullsized":"2024-04-25T13:41:35.050724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:24.182247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630889+0000","last_change":"2024-04-25T13:41:32.579069+0000","last_active":"2024-04-25T13:41:34.630889+0000","last_peered":"2024-04-25T13:41:34.630889+0000","last_clean":"2024-04-25T13:41:34.630889+0000","last_became_active":"2024-04-25T13:41:32.578548+0000","last_became_peered":"2024-04-25T13:41:32.578548+0000","last_unstale":"2024-04-25T13:41:34.630889+0000","last_undegraded":"2024-04-25T13:41:34.630889+0000","last_fullsized":"2024-04-25T13:41:34.630889+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:48.072093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131109+0000","last_change":"2024-04-25T13:41:33.586384+0000","last_active":"2024-04-25T13:41:35.131109+0000","last_peered":"2024-04-25T13:41:35.131109+0000","last_clean":"2024-04-25T13:41:35.131109+0000","last_became_active":"2024-04-25T13:41:33.586078+0000","last_became_peered":"2024-04-25T13:41:33.586078+0000","last_unstale":"2024-04-25T13:41:35.131109+0000","last_undegraded":"2024-04-25T13:41:35.131109+0000","last_fullsized":"2024-04-25T13:41:35.131109+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:44.830386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.051811+0000","last_change":"2024-04-25T13:41:32.609157+0000","last_active":"2024-04-25T13:41:35.051811+0000","last_peered":"2024-04-25T13:41:35.051811+0000","last_clean":"2024-04-25T13:41:35.051811+0000","last_became_active":"2024-04-25T13:41:32.605222+0000","last_became_peered":"2024-04-25T13:41:32.605222+0000","last_unstale":"2024-04-25T13:41:35.051811+0000","last_undegraded":"2024-04-25T13:41:35.051811+0000","last_fullsized":"2024-04-25T13:41:35.051811+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:26.424418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.629480+0000","last_change":"2024-04-25T13:41:33.585615+0000","last_active":"2024-04-25T13:41:34.629480+0000","last_peered":"2024-04-25T13:41:34.629480+0000","last_clean":"2024-04-25T13:41:34.629480+0000","last_became_active":"2024-04-25T13:41:33.583852+0000","last_became_peered":"2024-04-25T13:41:33.583852+0000","last_unstale":"2024-04-25T13:41:34.629480+0000","last_undegraded":"2024-04-25T13:41:34.629480+0000","last_fullsized":"2024-04-25T13:41:34.629480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:01.767599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050670+0000","last_change":"2024-04-25T13:41:32.595845+0000","last_active":"2024-04-25T13:41:35.050670+0000","last_peered":"2024-04-25T13:41:35.050670+0000","last_clean":"2024-04-25T13:41:35.050670+0000","last_became_active":"2024-04-25T13:41:32.594815+0000","last_became_peered":"2024-04-25T13:41:32.594815+0000","last_unstale":"2024-04-25T13:41:35.050670+0000","last_undegraded":"2024-04-25T13:41:35.050670+0000","last_fullsized":"2024-04-25T13:41:35.050670+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:44:39.817329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.575336+0000","last_change":"2024-04-25T13:41:33.589586+0000","last_active":"2024-04-25T13:41:34.575336+0000","last_peered":"2024-04-25T13:41:34.575336+0000","last_clean":"2024-04-25T13:41:34.575336+0000","last_became_active":"2024-04-25T13:41:33.587122+0000","last_became_peered":"2024-04-25T13:41:33.587122+0000","last_unstale":"2024-04-25T13:41:34.575336+0000","last_undegraded":"2024-04-25T13:41:34.575336+0000","last_fullsized":"2024-04-25T13:41:34.575336+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:36.350188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.051430+0000","last_change":"2024-04-25T13:41:32.602637+0000","last_active":"2024-04-25T13:41:35.051430+0000","last_peered":"2024-04-25T13:41:35.051430+0000","last_clean":"2024-04-25T13:41:35.051430+0000","last_became_active":"2024-04-25T13:41:32.601334+0000","last_became_peered":"2024-04-25T13:41:32.601334+0000","last_unstale":"2024-04-25T13:41:35.051430+0000","last_undegraded":"2024-04-25T13:41:35.051430+0000","last_fullsized":"2024-04-25T13:41:35.051430+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:02.989836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.575319+0000","last_change":"2024-04-25T13:41:33.586272+0000","last_active":"2024-04-25T13:41:34.575319+0000","last_peered":"2024-04-25T13:41:34.575319+0000","last_clean":"2024-04-25T13:41:34.575319+0000","last_became_active":"2024-04-25T13:41:33.586058+0000","last_became_peered":"2024-04-25T13:41:33.586058+0000","last_unstale":"2024-04-25T13:41:34.575319+0000","last_undegraded":"2024-04-25T13:41:34.575319+0000","last_fullsized":"2024-04-25T13:41:34.575319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:26.044289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.571569+0000","last_change":"2024-04-25T13:41:32.581137+0000","last_active":"2024-04-25T13:41:34.571569+0000","last_peered":"2024-04-25T13:41:34.571569+0000","last_clean":"2024-04-25T13:41:34.571569+0000","last_became_active":"2024-04-25T13:41:32.580368+0000","last_became_peered":"2024-04-25T13:41:32.580368+0000","last_unstale":"2024-04-25T13:41:34.571569+0000","last_undegraded":"2024-04-25T13:41:34.571569+0000","last_fullsized":"2024-04-25T13:41:34.571569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:08.206684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.575499+0000","last_change":"2024-04-25T13:41:33.586732+0000","last_active":"2024-04-25T13:41:34.575499+0000","last_peered":"2024-04-25T13:41:34.575499+0000","last_clean":"2024-04-25T13:41:34.575499+0000","last_became_active":"2024-04-25T13:41:33.585464+0000","last_became_peered":"2024-04-25T13:41:33.585464+0000","last_unstale":"2024-04-25T13:41:34.575499+0000","last_undegraded":"2024-04-25T13:41:34.575499+0000","last_fullsized":"2024-04-25T13:41:34.575499+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:56.403724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.629826+0000","last_change":"2024-04-25T13:41:32.588053+0000","last_active":"2024-04-25T13:41:34.629826+0000","last_peered":"2024-04-25T13:41:34.629826+0000","last_clean":"2024-04-25T13:41:34.629826+0000","last_became_active":"2024-04-25T13:41:32.587211+0000","last_became_peered":"2024-04-25T13:41:32.587211+0000","last_unstale":"2024-04-25T13:41:34.629826+0000","last_undegraded":"2024-04-25T13:41:34.629826+0000","last_fullsized":"2024-04-25T13:41:34.629826+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:01:45.951841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.575764+0000","last_change":"2024-04-25T13:41:33.589962+0000","last_active":"2024-04-25T13:41:34.575764+0000","last_peered":"2024-04-25T13:41:34.575764+0000","last_clean":"2024-04-25T13:41:34.575764+0000","last_became_active":"2024-04-25T13:41:33.589222+0000","last_became_peered":"2024-04-25T13:41:33.589222+0000","last_unstale":"2024-04-25T13:41:34.575764+0000","last_undegraded":"2024-04-25T13:41:34.575764+0000","last_fullsized":"2024-04-25T13:41:34.575764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:55.476713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.052027+0000","last_change":"2024-04-25T13:41:32.608569+0000","last_active":"2024-04-25T13:41:35.052027+0000","last_peered":"2024-04-25T13:41:35.052027+0000","last_clean":"2024-04-25T13:41:35.052027+0000","last_became_active":"2024-04-25T13:41:32.604150+0000","last_became_peered":"2024-04-25T13:41:32.604150+0000","last_unstale":"2024-04-25T13:41:35.052027+0000","last_undegraded":"2024-04-25T13:41:35.052027+0000","last_fullsized":"2024-04-25T13:41:35.052027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:02.714853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132545+0000","last_change":"2024-04-25T13:41:33.589862+0000","last_active":"2024-04-25T13:41:35.132545+0000","last_peered":"2024-04-25T13:41:35.132545+0000","last_clean":"2024-04-25T13:41:35.132545+0000","last_became_active":"2024-04-25T13:41:33.589583+0000","last_became_peered":"2024-04-25T13:41:33.589583+0000","last_unstale":"2024-04-25T13:41:35.132545+0000","last_undegraded":"2024-04-25T13:41:35.132545+0000","last_fullsized":"2024-04-25T13:41:35.132545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:39.950812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131185+0000","last_change":"2024-04-25T13:41:32.591996+0000","last_active":"2024-04-25T13:41:35.131185+0000","last_peered":"2024-04-25T13:41:35.131185+0000","last_clean":"2024-04-25T13:41:35.131185+0000","last_became_active":"2024-04-25T13:41:32.591149+0000","last_became_peered":"2024-04-25T13:41:32.591149+0000","last_unstale":"2024-04-25T13:41:35.131185+0000","last_undegraded":"2024-04-25T13:41:35.131185+0000","last_fullsized":"2024-04-25T13:41:35.131185+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:21.969256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.575743+0000","last_change":"2024-04-25T13:41:33.585759+0000","last_active":"2024-04-25T13:41:34.575743+0000","last_peered":"2024-04-25T13:41:34.575743+0000","last_clean":"2024-04-25T13:41:34.575743+0000","last_became_active":"2024-04-25T13:41:33.585138+0000","last_became_peered":"2024-04-25T13:41:33.585138+0000","last_unstale":"2024-04-25T13:41:34.575743+0000","last_undegraded":"2024-04-25T13:41:34.575743+0000","last_fullsized":"2024-04-25T13:41:34.575743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:52.490488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.629680+0000","last_change":"2024-04-25T13:41:32.587338+0000","last_active":"2024-04-25T13:41:34.629680+0000","last_peered":"2024-04-25T13:41:34.629680+0000","last_clean":"2024-04-25T13:41:34.629680+0000","last_became_active":"2024-04-25T13:41:32.587021+0000","last_became_peered":"2024-04-25T13:41:32.587021+0000","last_unstale":"2024-04-25T13:41:34.629680+0000","last_undegraded":"2024-04-25T13:41:34.629680+0000","last_fullsized":"2024-04-25T13:41:34.629680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:12.869038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131728+0000","last_change":"2024-04-25T13:41:33.590048+0000","last_active":"2024-04-25T13:41:35.131728+0000","last_peered":"2024-04-25T13:41:35.131728+0000","last_clean":"2024-04-25T13:41:35.131728+0000","last_became_active":"2024-04-25T13:41:33.589577+0000","last_became_peered":"2024-04-25T13:41:33.589577+0000","last_unstale":"2024-04-25T13:41:35.131728+0000","last_undegraded":"2024-04-25T13:41:35.131728+0000","last_fullsized":"2024-04-25T13:41:35.131728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:32.564231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"33'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.571975+0000","last_change":"2024-04-25T13:41:32.580819+0000","last_active":"2024-04-25T13:41:34.571975+0000","last_peered":"2024-04-25T13:41:34.571975+0000","last_clean":"2024-04-25T13:41:34.571975+0000","last_became_active":"2024-04-25T13:41:32.579887+0000","last_became_peered":"2024-04-25T13:41:32.579887+0000","last_unstale":"2024-04-25T13:41:34.571975+0000","last_undegraded":"2024-04-25T13:41:34.571975+0000","last_fullsized":"2024-04-25T13:41:34.571975+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:54.349686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.570821+0000","last_change":"2024-04-25T13:41:33.589578+0000","last_active":"2024-04-25T13:41:34.570821+0000","last_peered":"2024-04-25T13:41:34.570821+0000","last_clean":"2024-04-25T13:41:34.570821+0000","last_became_active":"2024-04-25T13:41:33.589325+0000","last_became_peered":"2024-04-25T13:41:33.589325+0000","last_unstale":"2024-04-25T13:41:34.570821+0000","last_undegraded":"2024-04-25T13:41:34.570821+0000","last_fullsized":"2024-04-25T13:41:34.570821+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:13.915620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.631062+0000","last_change":"2024-04-25T13:41:32.579441+0000","last_active":"2024-04-25T13:41:34.631062+0000","last_peered":"2024-04-25T13:41:34.631062+0000","last_clean":"2024-04-25T13:41:34.631062+0000","last_became_active":"2024-04-25T13:41:32.579097+0000","last_became_peered":"2024-04-25T13:41:32.579097+0000","last_unstale":"2024-04-25T13:41:34.631062+0000","last_undegraded":"2024-04-25T13:41:34.631062+0000","last_fullsized":"2024-04-25T13:41:34.631062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:37.057071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.631187+0000","last_change":"2024-04-25T13:41:33.583773+0000","last_active":"2024-04-25T13:41:34.631187+0000","last_peered":"2024-04-25T13:41:34.631187+0000","last_clean":"2024-04-25T13:41:34.631187+0000","last_became_active":"2024-04-25T13:41:33.583328+0000","last_became_peered":"2024-04-25T13:41:33.583328+0000","last_unstale":"2024-04-25T13:41:34.631187+0000","last_undegraded":"2024-04-25T13:41:34.631187+0000","last_fullsized":"2024-04-25T13:41:34.631187+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:14.069898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.571870+0000","last_change":"2024-04-25T13:41:32.589797+0000","last_active":"2024-04-25T13:41:34.571870+0000","last_peered":"2024-04-25T13:41:34.571870+0000","last_clean":"2024-04-25T13:41:34.571870+0000","last_became_active":"2024-04-25T13:41:32.589337+0000","last_became_peered":"2024-04-25T13:41:32.589337+0000","last_unstale":"2024-04-25T13:41:34.571870+0000","last_undegraded":"2024-04-25T13:41:34.571870+0000","last_fullsized":"2024-04-25T13:41:34.571870+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:03.239151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.571391+0000","last_change":"2024-04-25T13:41:33.583037+0000","last_active":"2024-04-25T13:41:34.571391+0000","last_peered":"2024-04-25T13:41:34.571391+0000","last_clean":"2024-04-25T13:41:34.571391+0000","last_became_active":"2024-04-25T13:41:33.582735+0000","last_became_peered":"2024-04-25T13:41:33.582735+0000","last_unstale":"2024-04-25T13:41:34.571391+0000","last_undegraded":"2024-04-25T13:41:34.571391+0000","last_fullsized":"2024-04-25T13:41:34.571391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:53.353181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630131+0000","last_change":"2024-04-25T13:41:32.590039+0000","last_active":"2024-04-25T13:41:34.630131+0000","last_peered":"2024-04-25T13:41:34.630131+0000","last_clean":"2024-04-25T13:41:34.630131+0000","last_became_active":"2024-04-25T13:41:32.589673+0000","last_became_peered":"2024-04-25T13:41:32.589673+0000","last_unstale":"2024-04-25T13:41:34.630131+0000","last_undegraded":"2024-04-25T13:41:34.630131+0000","last_fullsized":"2024-04-25T13:41:34.630131+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:57.694785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.130778+0000","last_change":"2024-04-25T13:41:33.597564+0000","last_active":"2024-04-25T13:41:35.130778+0000","last_peered":"2024-04-25T13:41:35.130778+0000","last_clean":"2024-04-25T13:41:35.130778+0000","last_became_active":"2024-04-25T13:41:33.593771+0000","last_became_peered":"2024-04-25T13:41:33.593771+0000","last_unstale":"2024-04-25T13:41:35.130778+0000","last_undegraded":"2024-04-25T13:41:35.130778+0000","last_fullsized":"2024-04-25T13:41:35.130778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:27.503230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050663+0000","last_change":"2024-04-25T13:41:32.605521+0000","last_active":"2024-04-25T13:41:35.050663+0000","last_peered":"2024-04-25T13:41:35.050663+0000","last_clean":"2024-04-25T13:41:35.050663+0000","last_became_active":"2024-04-25T13:41:32.603724+0000","last_became_peered":"2024-04-25T13:41:32.603724+0000","last_unstale":"2024-04-25T13:41:35.050663+0000","last_undegraded":"2024-04-25T13:41:35.050663+0000","last_fullsized":"2024-04-25T13:41:35.050663+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:41.608673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.050021+0000","last_change":"2024-04-25T13:41:33.574746+0000","last_active":"2024-04-25T13:41:35.050021+0000","last_peered":"2024-04-25T13:41:35.050021+0000","last_clean":"2024-04-25T13:41:35.050021+0000","last_became_active":"2024-04-25T13:41:33.574387+0000","last_became_peered":"2024-04-25T13:41:33.574387+0000","last_unstale":"2024-04-25T13:41:35.050021+0000","last_undegraded":"2024-04-25T13:41:35.050021+0000","last_fullsized":"2024-04-25T13:41:35.050021+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:38.423494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.132354+0000","last_change":"2024-04-25T13:41:32.590768+0000","last_active":"2024-04-25T13:41:35.132354+0000","last_peered":"2024-04-25T13:41:35.132354+0000","last_clean":"2024-04-25T13:41:35.132354+0000","last_became_active":"2024-04-25T13:41:32.589798+0000","last_became_peered":"2024-04-25T13:41:32.589798+0000","last_unstale":"2024-04-25T13:41:35.132354+0000","last_undegraded":"2024-04-25T13:41:35.132354+0000","last_fullsized":"2024-04-25T13:41:35.132354+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:54.959598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.133001+0000","last_change":"2024-04-25T13:41:33.592675+0000","last_active":"2024-04-25T13:41:35.133001+0000","last_peered":"2024-04-25T13:41:35.133001+0000","last_clean":"2024-04-25T13:41:35.133001+0000","last_became_active":"2024-04-25T13:41:33.592441+0000","last_became_peered":"2024-04-25T13:41:33.592441+0000","last_unstale":"2024-04-25T13:41:35.133001+0000","last_undegraded":"2024-04-25T13:41:35.133001+0000","last_fullsized":"2024-04-25T13:41:35.133001+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:32.548343+0000","last_clean_scrub_stamp":"2024-04-25T13:41:32.548343+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:26.016638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:34.630003+0000","last_change":"2024-04-25T13:41:32.596829+0000","last_active":"2024-04-25T13:41:34.630003+0000","last_peered":"2024-04-25T13:41:34.630003+0000","last_clean":"2024-04-25T13:41:34.630003+0000","last_became_active":"2024-04-25T13:41:32.596092+0000","last_became_peered":"2024-04-25T13:41:32.596092+0000","last_unstale":"2024-04-25T13:41:34.630003+0000","last_undegraded":"2024-04-25T13:41:34.630003+0000","last_fullsized":"2024-04-25T13:41:34.630003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:41:31.533692+0000","last_clean_scrub_stamp":"2024-04-25T13:41:31.533692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:22.407088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":133,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T13:41:35.131644+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:41:35.131644+0000","last_peered":"2024-04-25T13:41:35.131644+0000","last_clean":"2024-04-25T13:41:35.131644+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:41:35.131644+0000","last_undegraded":"2024-04-25T13:41:35.131644+0000","last_fullsized":"2024-04-25T13:41:35.131644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199263086,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":143,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195168,"statfs":{"total":200000000,"available":199852928,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":143,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195168,"statfs":{"total":200000000,"available":199852872,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":143,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195169,"statfs":{"total":200000000,"available":199853440,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195167,"statfs":{"total":200000000,"available":199851910,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":143,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195169,"statfs":{"total":200000000,"available":199853474,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":144,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195167,"statfs":{"total":200000000,"available":199851910,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":719,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194592,"statfs":{"total":200000000,"available":199263176,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:41:42.738 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:41:42.738 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:41:42.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.930+0000 7fb1b9576640 1 Processor -- start 2024-04-25T13:41:42.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.930+0000 7fb1b9576640 1 -- start start 2024-04-25T13:41:42.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b9576640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1b4161f90 0x7fb1b40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b9576640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b40d0d70 con 0x7fb1b40d0510 2024-04-25T13:41:42.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b9576640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1b40d1050 con 0x7fb1b415ec40 2024-04-25T13:41:42.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b9576640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb1b406dca0 con 0x7fb1b4161f90 2024-04-25T13:41:42.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b2ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1b4161f90 0x7fb1b40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b27fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb1b40d0510 0x7fb1b415e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51348/0 (socket says 172.21.15.80:51348) 2024-04-25T13:41:42.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b27fc640 1 -- 172.21.15.80:0/937173749 learned_addr learned my addr 172.21.15.80:0/937173749 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:42.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b2ffd640 1 -- 172.21.15.80:0/937173749 >> v1:172.21.15.87:6789/0 conn(0x7fb1b415ec40 legacy=0x7fb1b4168cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:42.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b1ffb640 1 -- 172.21.15.80:0/937173749 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2132230940 0 0) 0x7fb1b40d0d70 con 0x7fb1b40d0510 2024-04-25T13:41:42.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b2ffd640 1 -- 172.21.15.80:0/937173749 >> v1:172.21.15.80:6789/0 conn(0x7fb1b40d0510 legacy=0x7fb1b415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.934+0000 7fb1b2ffd640 1 -- 172.21.15.80:0/937173749 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b406df80 con 0x7fb1b4161f90 2024-04-25T13:41:42.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b2ffd640 1 --2- 172.21.15.80:0/937173749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b4161f90 0x7fb1b40cffd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40097c0 tx=0x7fb1a401c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=532b22d19d4aa0f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b1ffb640 1 -- 172.21.15.80:0/937173749 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40b0070 con 0x7fb1b4161f90 2024-04-25T13:41:42.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b1ffb640 1 -- 172.21.15.80:0/937173749 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a40ab600 con 0x7fb1b4161f90 2024-04-25T13:41:42.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b1ffb640 1 -- 172.21.15.80:0/937173749 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40ab9d0 con 0x7fb1b4161f90 2024-04-25T13:41:42.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 -- 172.21.15.80:0/937173749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b4161f90 msgr2=0x7fb1b40cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 --2- 172.21.15.80:0/937173749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b4161f90 0x7fb1b40cffd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40097c0 tx=0x7fb1a401c320 comp rx=0 tx=0).stop 2024-04-25T13:41:42.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 -- 172.21.15.80:0/937173749 shutdown_connections 2024-04-25T13:41:42.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 --2- 172.21.15.80:0/937173749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b4161f90 0x7fb1b40cffd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 -- 172.21.15.80:0/937173749 >> 172.21.15.80:0/937173749 conn(0x7fb1b406d080 msgr2=0x7fb1b40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:42.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 -- 172.21.15.80:0/937173749 shutdown_connections 2024-04-25T13:41:42.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 -- 172.21.15.80:0/937173749 wait complete. 2024-04-25T13:41:42.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 Processor -- start 2024-04-25T13:41:42.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.938+0000 7fb1b9576640 1 -- start start 2024-04-25T13:41:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b40d0510 0x7fb1b4141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1b415ec40 0x7fb1b4155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1b4161f90 0x7fb1b413c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b406f630 con 0x7fb1b40d0510 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b406f4b0 con 0x7fb1b4161f90 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b40d0920 con 0x7fb1b415ec40 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b40d0510 0x7fb1b4141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b40d0510 0x7fb1b4141600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53176/0 (socket says 172.21.15.80:53176) 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 -- 172.21.15.80:0/1740034876 learned_addr learned my addr 172.21.15.80:0/1740034876 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 -- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1b415ec40 msgr2=0x7fb1b4155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b27fc640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1b415ec40 0x7fb1b4155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b37fe640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1b4161f90 0x7fb1b413c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1b415ec40 0x7fb1b4155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 -- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1b4161f90 msgr2=0x7fb1b413c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:42.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1b4161f90 0x7fb1b413c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 -- 172.21.15.80:0/1740034876 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b413fae0 con 0x7fb1b40d0510 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b37fe640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1b4161f90 0x7fb1b413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b2ffd640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b40d0510 0x7fb1b4141600 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4018f90 tx=0x7fb1a401abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb19b7fe640 1 -- 172.21.15.80:0/1740034876 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40b0050 con 0x7fb1b40d0510 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb19b7fe640 1 -- 172.21.15.80:0/1740034876 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1a40bc070 con 0x7fb1b40d0510 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b413bea0 con 0x7fb1b40d0510 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb19b7fe640 1 -- 172.21.15.80:0/1740034876 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a40b5a70 con 0x7fb1b40d0510 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b413d190 con 0x7fb1b40d0510 2024-04-25T13:41:42.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b406a3a0 con 0x7fb1b40d0510 2024-04-25T13:41:42.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb19b7fe640 1 -- 172.21.15.80:0/1740034876 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb1a40ab4c0 con 0x7fb1b40d0510 2024-04-25T13:41:42.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.942+0000 7fb19b7fe640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a006f430 0x7fb1a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:42.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.946+0000 7fb19b7fe640 1 -- 172.21.15.80:0/1740034876 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fb1a4129a00 con 0x7fb1b40d0510 2024-04-25T13:41:42.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.946+0000 7fb1b27fc640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a006f430 0x7fb1a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:42.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.946+0000 7fb1b27fc640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a006f430 0x7fb1a0071940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8000f70 tx=0x7fb1a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:42.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:42.950+0000 7fb19b7fe640 1 -- 172.21.15.80:0/1740034876 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb1a4016090 con 0x7fb1b40d0510 2024-04-25T13:41:43.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.170+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb1b40b4880 con 0x7fb1b40d0510 2024-04-25T13:41:43.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.178+0000 7fb19b7fe640 1 -- 172.21.15.80:0/1740034876 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7fb1a40f68f0 con 0x7fb1b40d0510 2024-04-25T13:41:43.183 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:41:43.183 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:41:43.183 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:41:43.183 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:41:43.183 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:41:43.183 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a006f430 msgr2=0x7fb1a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a006f430 0x7fb1a0071940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8000f70 tx=0x7fb1a8009040 comp rx=0 tx=0).stop 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b40d0510 msgr2=0x7fb1b4141600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b40d0510 0x7fb1b4141600 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4018f90 tx=0x7fb1a401abc0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 shutdown_connections 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a006f430 0x7fb1a0071940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1b4161f90 0x7fb1b413c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1b415ec40 0x7fb1b4155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 --2- 172.21.15.80:0/1740034876 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1b40d0510 0x7fb1b4141600 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 >> 172.21.15.80:0/1740034876 conn(0x7fb1b406d080 msgr2=0x7fb1b40d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 shutdown_connections 2024-04-25T13:41:43.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.182+0000 7fb1b9576640 1 -- 172.21.15.80:0/1740034876 wait complete. 2024-04-25T13:41:43.205 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:43.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.338+0000 7fc21d474640 1 Processor -- start 2024-04-25T13:41:43.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.338+0000 7fc21d474640 1 -- start start 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc21d474640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc21806f5a0 0x7fc218160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc21d474640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc21806fcf0 con 0x7fc218161070 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc21d474640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2180b59f0 con 0x7fc2180733b0 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc21d474640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc218073f10 con 0x7fc21806f5a0 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc216ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc21806f5a0 0x7fc218160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc20e7fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc218161070 0x7fc2180b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51362/0 (socket says 172.21.15.80:51362) 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc20e7fc640 1 -- 172.21.15.80:0/2636425180 learned_addr learned my addr 172.21.15.80:0/2636425180 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc216ffd640 1 -- 172.21.15.80:0/2636425180 >> v1:172.21.15.87:6789/0 conn(0x7fc2180733b0 legacy=0x7fc2180b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc216ffd640 1 -- 172.21.15.80:0/2636425180 >> v1:172.21.15.80:6789/0 conn(0x7fc218161070 legacy=0x7fc2180b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc2167fc640 1 -- 172.21.15.80:0/2636425180 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 567734607 0 0) 0x7fc21806fcf0 con 0x7fc218161070 2024-04-25T13:41:43.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc216ffd640 1 -- 172.21.15.80:0/2636425180 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2180b0c60 con 0x7fc21806f5a0 2024-04-25T13:41:43.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc216ffd640 1 --2- 172.21.15.80:0/2636425180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc21806f5a0 0x7fc218160b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc2080097c0 tx=0x7fc20801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68d1ca442726f76b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc2167fc640 1 -- 172.21.15.80:0/2636425180 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080b0070 con 0x7fc21806f5a0 2024-04-25T13:41:43.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.342+0000 7fc2167fc640 1 -- 172.21.15.80:0/2636425180 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2080ab600 con 0x7fc21806f5a0 2024-04-25T13:41:43.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc2167fc640 1 -- 172.21.15.80:0/2636425180 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2080ab9d0 con 0x7fc21806f5a0 2024-04-25T13:41:43.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- 172.21.15.80:0/2636425180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc21806f5a0 msgr2=0x7fc218160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 --2- 172.21.15.80:0/2636425180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc21806f5a0 0x7fc218160b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fc2080097c0 tx=0x7fc20801c3d0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- 172.21.15.80:0/2636425180 shutdown_connections 2024-04-25T13:41:43.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 --2- 172.21.15.80:0/2636425180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc21806f5a0 0x7fc218160b30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- 172.21.15.80:0/2636425180 >> 172.21.15.80:0/2636425180 conn(0x7fc21806d080 msgr2=0x7fc2180b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- 172.21.15.80:0/2636425180 shutdown_connections 2024-04-25T13:41:43.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- 172.21.15.80:0/2636425180 wait complete. 2024-04-25T13:41:43.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 Processor -- start 2024-04-25T13:41:43.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- start start 2024-04-25T13:41:43.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc21806f5a0 0x7fc21815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2180733b0 0x7fc21815d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc218161070 0x7fc21815dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc21816b6f0 con 0x7fc218161070 2024-04-25T13:41:43.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc218161890 con 0x7fc21806f5a0 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc21d474640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc21816b870 con 0x7fc2180733b0 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc2177fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc218161070 0x7fc21815dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.346+0000 7fc2177fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc218161070 0x7fc21815dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53190/0 (socket says 172.21.15.80:53190) 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc2177fe640 1 -- 172.21.15.80:0/2240612593 learned_addr learned my addr 172.21.15.80:0/2240612593 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc20e7fc640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2180733b0 0x7fc21815d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc216ffd640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc21806f5a0 0x7fc21815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc2177fe640 1 -- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2180733b0 msgr2=0x7fc21815d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc2177fe640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2180733b0 0x7fc21815d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc2177fe640 1 -- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc21806f5a0 msgr2=0x7fc21815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc2177fe640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc21806f5a0 0x7fc21815ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc2177fe640 1 -- 172.21.15.80:0/2240612593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc218159a30 con 0x7fc218161070 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc2177fe640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc218161070 0x7fc21815dad0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc21000be60 tx=0x7fc21000bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc20ffff640 1 -- 172.21.15.80:0/2240612593 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210019070 con 0x7fc218161070 2024-04-25T13:41:43.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc20ffff640 1 -- 172.21.15.80:0/2240612593 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc210014440 con 0x7fc218161070 2024-04-25T13:41:43.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc218159d10 con 0x7fc218161070 2024-04-25T13:41:43.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc20ffff640 1 -- 172.21.15.80:0/2240612593 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210014870 con 0x7fc218161070 2024-04-25T13:41:43.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21815a2d0 con 0x7fc218161070 2024-04-25T13:41:43.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21806a3a0 con 0x7fc218161070 2024-04-25T13:41:43.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc20ffff640 1 -- 172.21.15.80:0/2240612593 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc21001c070 con 0x7fc218161070 2024-04-25T13:41:43.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc20ffff640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1f806f400 0x7fc1f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:43.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc20ffff640 1 -- 172.21.15.80:0/2240612593 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fc21008f430 con 0x7fc218161070 2024-04-25T13:41:43.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.350+0000 7fc216ffd640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1f806f400 0x7fc1f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:43.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.354+0000 7fc216ffd640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1f806f400 0x7fc1f8071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ad000 tx=0x7fc208008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:43.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.354+0000 7fc20ffff640 1 -- 172.21.15.80:0/2240612593 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc210023080 con 0x7fc218161070 2024-04-25T13:41:43.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.542+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2180739c0 con 0x7fc218161070 2024-04-25T13:41:43.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.542+0000 7fc20ffff640 1 -- 172.21.15.80:0/2240612593 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5175 (secure 0 0 0) 0x7fc21005c490 con 0x7fc218161070 2024-04-25T13:41:43.545 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:43.546 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:41:32.974439+0000","modified":"2024-04-25T13:41:36.017164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/966995060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":966995060},{"type":"v1","addr":"172.21.15.87:6835","nonce":966995060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5190,"qdb_cluster":[5190]},"id":2}]} 2024-04-25T13:41:43.546 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 16 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1f806f400 msgr2=0x7fc1f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1f806f400 0x7fc1f8071910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc2080ad000 tx=0x7fc208008040 comp rx=0 tx=0).stop 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc218161070 msgr2=0x7fc21815dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc218161070 0x7fc21815dad0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fc21000be60 tx=0x7fc21000bf50 comp rx=0 tx=0).stop 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 shutdown_connections 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1f806f400 0x7fc1f8071910 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc218161070 0x7fc21815dad0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2180733b0 0x7fc21815d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 --2- 172.21.15.80:0/2240612593 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc21806f5a0 0x7fc21815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 >> 172.21.15.80:0/2240612593 conn(0x7fc21806d080 msgr2=0x7fc2180b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 shutdown_connections 2024-04-25T13:41:43.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:43.546+0000 7fc21d474640 1 -- 172.21.15.80:0/2240612593 wait complete. 2024-04-25T13:41:43.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:41:32.974439+0000', 'modified': '2024-04-25T13:41:36.017164+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5190}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5190': {'gid': 5190, 'name': 'b', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6835/966995060', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6834', 'nonce': 966995060}, {'type': 'v1', 'addr': '172.21.15.87:6835', 'nonce': 966995060}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5190, 'qdb_cluster': [5190]} 2024-04-25T13:41:43.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:41:43.969 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:41:44.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff52db1640 1 Processor -- start 2024-04-25T13:41:44.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff52db1640 1 -- start start 2024-04-25T13:41:44.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff52db1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7eff4c06f5a0 0x7eff4c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff52db1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c161070 0x7eff4c161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff52db1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff4c06fcf0 con 0x7eff4c0b4bd0 2024-04-25T13:41:44.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff52db1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7eff4c1619c0 con 0x7eff4c06f5a0 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff52db1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c073f10 con 0x7eff4c161070 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff50b4d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7eff4c06f5a0 0x7eff4c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff50b4d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7eff4c06f5a0 0x7eff4c160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53192/0 (socket says 172.21.15.80:53192) 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff50b4d640 1 -- 172.21.15.80:0/630110625 learned_addr learned my addr 172.21.15.80:0/630110625 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff4bfff640 1 --2- 172.21.15.80:0/630110625 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c161070 0x7eff4c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff50b4d640 1 -- 172.21.15.80:0/630110625 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c161070 msgr2=0x7eff4c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff50b4d640 1 --2- 172.21.15.80:0/630110625 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c161070 0x7eff4c161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff50b4d640 1 -- 172.21.15.80:0/630110625 >> v1:172.21.15.87:6789/0 conn(0x7eff4c0b4bd0 legacy=0x7eff4c0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.122+0000 7eff50b4d640 1 -- 172.21.15.80:0/630110625 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c0b0c60 con 0x7eff4c06f5a0 2024-04-25T13:41:44.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff50b4d640 1 --2- 172.21.15.80:0/630110625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c160b30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0098a0 tx=0x7eff3c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c2cfd1a2aaf1f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff4b7fe640 1 -- 172.21.15.80:0/630110625 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0b1070 con 0x7eff4c06f5a0 2024-04-25T13:41:44.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff4b7fe640 1 -- 172.21.15.80:0/630110625 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3c0ac600 con 0x7eff4c06f5a0 2024-04-25T13:41:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff4b7fe640 1 -- 172.21.15.80:0/630110625 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0ac9d0 con 0x7eff4c06f5a0 2024-04-25T13:41:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 -- 172.21.15.80:0/630110625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 msgr2=0x7eff4c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 --2- 172.21.15.80:0/630110625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c160b30 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0098a0 tx=0x7eff3c01c4b0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 -- 172.21.15.80:0/630110625 shutdown_connections 2024-04-25T13:41:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 --2- 172.21.15.80:0/630110625 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c161070 0x7eff4c161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 --2- 172.21.15.80:0/630110625 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c160b30 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 -- 172.21.15.80:0/630110625 >> 172.21.15.80:0/630110625 conn(0x7eff4c06d080 msgr2=0x7eff4c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 -- 172.21.15.80:0/630110625 shutdown_connections 2024-04-25T13:41:44.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 -- 172.21.15.80:0/630110625 wait complete. 2024-04-25T13:41:44.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.126+0000 7eff52db1640 1 Processor -- start 2024-04-25T13:41:44.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 -- start start 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c1529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c0b4bd0 0x7eff4c157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff4c161070 0x7eff4c151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c16a860 con 0x7eff4c06f5a0 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c16a6e0 con 0x7eff4c161070 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c16a9e0 con 0x7eff4c0b4bd0 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c1529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c1529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53206/0 (socket says 172.21.15.80:53206) 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 -- 172.21.15.80:0/463716175 learned_addr learned my addr 172.21.15.80:0/463716175 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:44.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff4bfff640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c0b4bd0 0x7eff4c157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 -- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c0b4bd0 msgr2=0x7eff4c157c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c0b4bd0 0x7eff4c157c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 -- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff4c161070 msgr2=0x7eff4c151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff5134e640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff4c161070 0x7eff4c151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff4c161070 0x7eff4c151610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 -- 172.21.15.80:0/463716175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c0d0a60 con 0x7eff4c06f5a0 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff4bfff640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c0b4bd0 0x7eff4c157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff50b4d640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c1529d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0098a0 tx=0x7eff3c01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff497fa640 1 -- 172.21.15.80:0/463716175 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0bb020 con 0x7eff4c06f5a0 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff497fa640 1 -- 172.21.15.80:0/463716175 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff3c0b7620 con 0x7eff4c06f5a0 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 -- 172.21.15.80:0/463716175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c1581d0 con 0x7eff4c06f5a0 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff497fa640 1 -- 172.21.15.80:0/463716175 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0b1040 con 0x7eff4c06f5a0 2024-04-25T13:41:44.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff52db1640 1 -- 172.21.15.80:0/463716175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c150850 con 0x7eff4c06f5a0 2024-04-25T13:41:44.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.130+0000 7eff497fa640 1 -- 172.21.15.80:0/463716175 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7eff3c0be020 con 0x7eff4c06f5a0 2024-04-25T13:41:44.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff497fa640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff3806f400 0x7eff38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff497fa640 1 -- 172.21.15.80:0/463716175 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7eff3c129230 con 0x7eff4c06f5a0 2024-04-25T13:41:44.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff4bfff640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff3806f400 0x7eff38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5a10e000 1 Processor -- start 2024-04-25T13:41:44.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5a10e000 1 -- start start 2024-04-25T13:41:44.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff4bfff640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff3806f400 0x7eff38071910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff4c1532c0 tx=0x7eff40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 0x5586804f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x5586804f5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586804f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5a10e000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x558680360160 con 0x55867ff77280 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5a10e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5586802f5e20 con 0x558680392c20 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5a10e000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5586802aea20 con 0x55868037eea0 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586804f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586804f5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53216/0 (socket says 172.21.15.80:53216) 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 -- 172.21.15.80:0/1937124126 learned_addr learned my addr 172.21.15.80:0/1937124126 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff4bfff640 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x5586804f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff50b4d640 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 0x5586804f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 -- 172.21.15.80:0/1937124126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 msgr2=0x5586804f5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x5586804f5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 -- 172.21.15.80:0/1937124126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 msgr2=0x5586804f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 0x5586804f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 -- 172.21.15.80:0/1937124126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586802b3100 con 0x55867ff77280 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff5134e640 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586804f5dc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7eff44001c60 tx=0x7eff44002120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36120da65e34e064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff327fc640 1 -- 172.21.15.80:0/1937124126 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff4400dc60 con 0x55867ff77280 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff327fc640 1 -- 172.21.15.80:0/1937124126 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4400de00 con 0x55867ff77280 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.134+0000 7eff327fc640 1 -- 172.21.15.80:0/1937124126 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44003b80 con 0x55867ff77280 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- 172.21.15.80:0/1937124126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 msgr2=0x5586804f5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586804f5dc0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7eff44001c60 tx=0x7eff44002120 comp rx=0 tx=0).stop 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- 172.21.15.80:0/1937124126 shutdown_connections 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586804f5dc0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x5586804f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 --2- 172.21.15.80:0/1937124126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 0x5586804f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- 172.21.15.80:0/1937124126 >> 172.21.15.80:0/1937124126 conn(0x55867ffb6ff0 msgr2=0x5586804a64c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- 172.21.15.80:0/1937124126 shutdown_connections 2024-04-25T13:41:44.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- 172.21.15.80:0/1937124126 wait complete. 2024-04-25T13:41:44.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- start start 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 Processor -- start 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55867ff77280 0x5586804fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55868037eea0 0x5586804fad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804b5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x558680303220 con 0x558680392c20 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558680299ea0 con 0x55868037eea0 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5586802f0b00 con 0x55867ff77280 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 10 client.0 _ll_register_callbacks cb 0x558680302dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53220/0 (socket says 172.21.15.80:53220) 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 -- 172.21.15.80:0/3917805204 learned_addr learned my addr 172.21.15.80:0/3917805204 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff50b4d640 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55867ff77280 0x5586804fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff4bfff640 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55868037eea0 0x5586804fad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 -- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55867ff77280 msgr2=0x5586804fa810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55867ff77280 0x5586804fa810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 -- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55868037eea0 msgr2=0x5586804fad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55868037eea0 0x5586804fad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 -- 172.21.15.80:0/3917805204 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558680295500 con 0x558680392c20 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff50b4d640 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55867ff77280 0x5586804fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5134e640 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804b5440 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7eff44000f90 tx=0x7eff44001260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7f2619f741a79b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff2b7fe640 1 -- 172.21.15.80:0/3917805204 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44003d90 con 0x558680392c20 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- 172.21.15.80:0/3917805204 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff4400de00 con 0x558680392c20 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff2b7fe640 1 -- 172.21.15.80:0/3917805204 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7eff44026ab0 con 0x558680392c20 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff2b7fe640 1 -- 172.21.15.80:0/3917805204 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7eff44026cb0 con 0x558680392c20 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 10 client.5409 fetch_fsmap learned FSMap version 16 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- 172.21.15.80:0/3917805204 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55868027bea0 con 0x558680392c20 2024-04-25T13:41:44.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff2b7fe640 1 -- 172.21.15.80:0/3917805204 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff4400de00 con 0x558680392c20 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff2b7fe640 1 -- 172.21.15.80:0/3917805204 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7eff44005070 con 0x558680392c20 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 10 client.5409 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 10 client.5409 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 20 client.5409 populate_metadata read hostname 'smithi080' 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] conn(0x55867ff9b7d0 0x5586804b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 4 client.5409 mds_command: new command op to 5190 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.138+0000 7eff5a10e000 1 -- 172.21.15.80:0/3917805204 --> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55868028ebe0 con 0x55867ff9b7d0 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff50b4d640 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] conn(0x55867ff9b7d0 0x5586804b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff2b7fe640 10 client.5409 ms_handle_connect on v2:172.21.15.87:6834/966995060 2024-04-25T13:41:44.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff50b4d640 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] conn(0x55867ff9b7d0 0x5586804b61b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfda072dc6b61367 server_cookie=5bc071a6ffcab3fc in_seq=0 out_seq=0 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff2b7fe640 1 -- 172.21.15.80:0/3917805204 <== mds.0 v2:172.21.15.87:6834/966995060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55868028ebe0 con 0x55867ff9b7d0 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff2b7fe640 10 client.5409 handle_command_reply: tid=0 2024-04-25T13:41:44.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 client.5409 shutdown 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 -- 172.21.15.80:0/3917805204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 msgr2=0x5586804b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804b5440 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7eff44000f90 tx=0x7eff44001260 comp rx=0 tx=0).stop 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 -- 172.21.15.80:0/3917805204 shutdown_connections 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] conn(0x55867ff9b7d0 0x5586804b61b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804b5440 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55868037eea0 0x5586804fad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff2b7fe640 0 client.5409 ms_handle_reset on v2:172.21.15.87:6834/966995060 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 --2- 172.21.15.80:0/3917805204 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55867ff77280 0x5586804fa810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 -- 172.21.15.80:0/3917805204 >> 172.21.15.80:0/3917805204 conn(0x55867ffb6ff0 msgr2=0x5586804ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 -- 172.21.15.80:0/3917805204 shutdown_connections 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 1 -- 172.21.15.80:0/3917805204 wait complete. 2024-04-25T13:41:44.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.142+0000 7eff5a10e000 20 client.5409 trim_cache size 0 max 16384 2024-04-25T13:41:44.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 Processor -- start 2024-04-25T13:41:44.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- start start 2024-04-25T13:41:44.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55867ff77280 0x558680500a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x558680500fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804fc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c15f600 con 0x558680392c20 2024-04-25T13:41:44.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c15e960 con 0x55867ff77280 2024-04-25T13:41:44.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7eff4c15efb0 con 0x55868037eea0 2024-04-25T13:41:44.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804fc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804fc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53234/0 (socket says 172.21.15.80:53234) 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 -- 172.21.15.80:0/282862359 learned_addr learned my addr 172.21.15.80:0/282862359 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 -- 172.21.15.80:0/282862359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 msgr2=0x558680500fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 --2- 172.21.15.80:0/282862359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x558680500fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff50b4d640 1 --2- 172.21.15.80:0/282862359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55867ff77280 0x558680500a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 -- 172.21.15.80:0/282862359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55867ff77280 msgr2=0x558680500a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 --2- 172.21.15.80:0/282862359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55867ff77280 0x558680500a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 -- 172.21.15.80:0/282862359 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586802aec70 con 0x558680392c20 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5134e640 1 --2- 172.21.15.80:0/282862359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804fc820 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7eff44001230 tx=0x7eff4401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9769949ccb93b33d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff31ffb640 1 -- 172.21.15.80:0/282862359 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44005070 con 0x558680392c20 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff31ffb640 1 -- 172.21.15.80:0/282862359 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff44035040 con 0x558680392c20 2024-04-25T13:41:44.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff31ffb640 1 -- 172.21.15.80:0/282862359 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff44003b90 con 0x558680392c20 2024-04-25T13:41:44.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- 172.21.15.80:0/282862359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 msgr2=0x5586804fc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 --2- 172.21.15.80:0/282862359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804fc820 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7eff44001230 tx=0x7eff4401e000 comp rx=0 tx=0).stop 2024-04-25T13:41:44.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- 172.21.15.80:0/282862359 shutdown_connections 2024-04-25T13:41:44.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 --2- 172.21.15.80:0/282862359 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558680392c20 0x5586804fc820 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 --2- 172.21.15.80:0/282862359 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x558680500fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 --2- 172.21.15.80:0/282862359 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55867ff77280 0x558680500a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- 172.21.15.80:0/282862359 >> 172.21.15.80:0/282862359 conn(0x55867ffb6ff0 msgr2=0x5586804a6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- 172.21.15.80:0/282862359 shutdown_connections 2024-04-25T13:41:44.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.162+0000 7eff5a10e000 1 -- 172.21.15.80:0/282862359 wait complete. 2024-04-25T13:41:44.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 -- start start 2024-04-25T13:41:44.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 Processor -- start 2024-04-25T13:41:44.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586805007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x558680523310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 0x558680523850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5586802f5130 con 0x55867ff77280 2024-04-25T13:41:44.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55868028b7d0 con 0x558680392c20 2024-04-25T13:41:44.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5586802fa450 con 0x55868037eea0 2024-04-25T13:41:44.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 10 client.0 _ll_register_callbacks cb 0x7eff44001b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:41:44.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586805007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586805007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53246/0 (socket says 172.21.15.80:53246) 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 -- 172.21.15.80:0/806187330 learned_addr learned my addr 172.21.15.80:0/806187330 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5134e640 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 0x558680523850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff4bfff640 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x558680523310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 -- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 msgr2=0x558680523310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x558680523310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 -- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 msgr2=0x558680523850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 0x558680523850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 -- 172.21.15.80:0/806187330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558680378180 con 0x55867ff77280 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff4bfff640 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x558680523310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff50b4d640 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586805007c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0f57b0 tx=0x7eff3c019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440e085dda578fee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff2b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:41:44.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff2b7fe640 1 -- 172.21.15.80:0/806187330 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3c0f5ab0 con 0x55867ff77280 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff2b7fe640 1 -- 172.21.15.80:0/806187330 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7eff3c0f1d40 con 0x55867ff77280 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 -- 172.21.15.80:0/806187330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff44035040 con 0x55867ff77280 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff2b7fe640 1 -- 172.21.15.80:0/806187330 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7eff3c0f9e60 con 0x55867ff77280 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 10 client.5415 fetch_fsmap learned FSMap version 16 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 -- 172.21.15.80:0/806187330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x55867ffaf0c0 con 0x55867ff77280 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff2b7fe640 1 -- 172.21.15.80:0/806187330 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff44035040 con 0x55867ff77280 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff2b7fe640 1 -- 172.21.15.80:0/806187330 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 16) ==== 3494+0+0 (secure 0 0 0) 0x7eff3c0b4030 con 0x55867ff77280 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 10 client.5415 fetch_fsmap finished waiting for FSMap version 16 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 10 client.5415 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 20 client.5415 populate_metadata read hostname 'smithi080' 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] conn(0x55867ff89830 0x558680566b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 4 client.5415 mds_command: new command op to 5190 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff5a10e000 1 -- 172.21.15.80:0/806187330 --> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] -- command(tid 0: {"prefix": "status"}) -- 0x558680303a30 con 0x55867ff89830 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.166+0000 7eff4bfff640 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] conn(0x55867ff89830 0x558680566b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff2b7fe640 10 client.5415 ms_handle_connect on v2:172.21.15.87:6834/966995060 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff4bfff640 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] conn(0x55867ff89830 0x558680566b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=446c9bfcdb389945 server_cookie=77a0e285686d8e0b in_seq=0 out_seq=0 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff2b7fe640 1 -- 172.21.15.80:0/806187330 <== mds.0 v2:172.21.15.87:6834/966995060 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x558680303a30 con 0x55867ff89830 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff2b7fe640 10 client.5415 handle_command_reply: tid=0 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 client.5415 shutdown 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 -- 172.21.15.80:0/806187330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 msgr2=0x5586805007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586805007c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0f57b0 tx=0x7eff3c019250 comp rx=0 tx=0).stop 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 -- 172.21.15.80:0/806187330 shutdown_connections 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:6834/966995060,v1:172.21.15.87:6835/966995060] conn(0x55867ff89830 0x558680566b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558680392c20 0x558680523850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55868037eea0 0x558680523310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff2b7fe640 0 client.5415 ms_handle_reset on v2:172.21.15.87:6834/966995060 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 --2- 172.21.15.80:0/806187330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55867ff77280 0x5586805007c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 -- 172.21.15.80:0/806187330 >> 172.21.15.80:0/806187330 conn(0x55867ffb6ff0 msgr2=0x5586805099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 -- 172.21.15.80:0/806187330 shutdown_connections 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 1 -- 172.21.15.80:0/806187330 wait complete. 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff5a10e000 20 client.5415 trim_cache size 0 max 16384 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5190, 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 10.605217652, 2024-04-25T13:41:44.177 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 16, 2024-04-25T13:41:44.178 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 33, 2024-04-25T13:41:44.178 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 33, 2024-04-25T13:41:44.178 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 18.490933319 2024-04-25T13:41:44.178 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 -- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff3806f400 msgr2=0x7eff38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff3806f400 0x7eff38071910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff4c1532c0 tx=0x7eff40008040 comp rx=0 tx=0).stop 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 -- 172.21.15.80:0/463716175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 msgr2=0x7eff4c1529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c1529d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7eff3c0098a0 tx=0x7eff3c01acf0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 -- 172.21.15.80:0/463716175 shutdown_connections 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff3806f400 0x7eff38071910 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff4c161070 0x7eff4c151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff4c0b4bd0 0x7eff4c157c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 --2- 172.21.15.80:0/463716175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff4c06f5a0 0x7eff4c1529d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 -- 172.21.15.80:0/463716175 >> 172.21.15.80:0/463716175 conn(0x7eff4c06d080 msgr2=0x7eff4c15f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 -- 172.21.15.80:0/463716175 shutdown_connections 2024-04-25T13:41:44.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.170+0000 7eff52db1640 1 -- 172.21.15.80:0/463716175 wait complete. 2024-04-25T13:41:44.191 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:41:44.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.322+0000 7f31d3031640 1 Processor -- start 2024-04-25T13:41:44.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.322+0000 7f31d3031640 1 -- start start 2024-04-25T13:41:44.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d3031640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f31cc06f5a0 0x7f31cc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d3031640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31cc161070 0x7f31cc161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d3031640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31cc06fcf0 con 0x7f31cc0b4bd0 2024-04-25T13:41:44.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d3031640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f31cc1619c0 con 0x7f31cc06f5a0 2024-04-25T13:41:44.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d3031640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f31cc073f10 con 0x7f31cc161070 2024-04-25T13:41:44.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d0dcd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f31cc06f5a0 0x7f31cc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d15ce640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f31cc0b4bd0 0x7f31cc0b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51378/0 (socket says 172.21.15.80:51378) 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d15ce640 1 -- 172.21.15.80:0/1587254019 learned_addr learned my addr 172.21.15.80:0/1587254019 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31c3fff640 1 --2- 172.21.15.80:0/1587254019 >> v2:172.21.15.87:3300/0 conn(0x7f31cc161070 0x7f31cc161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d0dcd640 1 -- 172.21.15.80:0/1587254019 >> v2:172.21.15.87:3300/0 conn(0x7f31cc161070 msgr2=0x7f31cc161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d0dcd640 1 --2- 172.21.15.80:0/1587254019 >> v2:172.21.15.87:3300/0 conn(0x7f31cc161070 0x7f31cc161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d0dcd640 1 -- 172.21.15.80:0/1587254019 >> v1:172.21.15.80:6789/0 conn(0x7f31cc0b4bd0 legacy=0x7f31cc0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d0dcd640 1 -- 172.21.15.80:0/1587254019 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc0b0c60 con 0x7f31cc06f5a0 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31c37fe640 1 -- 172.21.15.80:0/1587254019 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3749719297 0 0) 0x7f31cc06fcf0 con 0x7f31cc0b4bd0 2024-04-25T13:41:44.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31c3fff640 1 --2- 172.21.15.80:0/1587254019 >> v2:172.21.15.87:3300/0 conn(0x7f31cc161070 0x7f31cc161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:44.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.326+0000 7f31d0dcd640 1 --2- 172.21.15.80:0/1587254019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160b30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f31cc06ff70 tx=0x7f31c801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44c3d2561574798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31c37fe640 1 -- 172.21.15.80:0/1587254019 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80b0070 con 0x7f31cc06f5a0 2024-04-25T13:41:44.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31c37fe640 1 -- 172.21.15.80:0/1587254019 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c80ab600 con 0x7f31cc06f5a0 2024-04-25T13:41:44.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31c37fe640 1 -- 172.21.15.80:0/1587254019 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80ab9d0 con 0x7f31cc06f5a0 2024-04-25T13:41:44.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 -- 172.21.15.80:0/1587254019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 msgr2=0x7f31cc160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 --2- 172.21.15.80:0/1587254019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160b30 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f31cc06ff70 tx=0x7f31c801c3d0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 -- 172.21.15.80:0/1587254019 shutdown_connections 2024-04-25T13:41:44.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 --2- 172.21.15.80:0/1587254019 >> v2:172.21.15.87:3300/0 conn(0x7f31cc161070 0x7f31cc161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 --2- 172.21.15.80:0/1587254019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160b30 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 -- 172.21.15.80:0/1587254019 >> 172.21.15.80:0/1587254019 conn(0x7f31cc06d080 msgr2=0x7f31cc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 -- 172.21.15.80:0/1587254019 shutdown_connections 2024-04-25T13:41:44.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 -- 172.21.15.80:0/1587254019 wait complete. 2024-04-25T13:41:44.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 Processor -- start 2024-04-25T13:41:44.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 -- start start 2024-04-25T13:41:44.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.330+0000 7f31d3031640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31cc0b4bd0 0x7f31cc159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d3031640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31cc161070 0x7f31cc155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d3031640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc16a860 con 0x7f31cc06f5a0 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d3031640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f31cc16a6e0 con 0x7f31cc161070 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d3031640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f31cc16a9e0 con 0x7f31cc0b4bd0 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53258/0 (socket says 172.21.15.80:53258) 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 -- 172.21.15.80:0/2501905276 learned_addr learned my addr 172.21.15.80:0/2501905276 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31c3fff640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31cc0b4bd0 0x7f31cc159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d15ce640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31cc161070 0x7f31cc155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 -- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31cc0b4bd0 msgr2=0x7f31cc159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31cc0b4bd0 0x7f31cc159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 -- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31cc161070 msgr2=0x7f31cc155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31cc161070 0x7f31cc155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 -- 172.21.15.80:0/2501905276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31cc15a980 con 0x7f31cc06f5a0 2024-04-25T13:41:44.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d15ce640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31cc161070 0x7f31cc155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d0dcd640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31c80ae040 tx=0x7f31c801ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31c17fa640 1 -- 172.21.15.80:0/2501905276 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80b0050 con 0x7f31cc06f5a0 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31c17fa640 1 -- 172.21.15.80:0/2501905276 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31c80bc070 con 0x7f31cc06f5a0 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31c17fa640 1 -- 172.21.15.80:0/2501905276 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31c80b5870 con 0x7f31cc06f5a0 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31cc15ac60 con 0x7f31cc06f5a0 2024-04-25T13:41:44.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31cc15b220 con 0x7f31cc06f5a0 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31cc06a3a0 con 0x7f31cc06f5a0 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31c17fa640 1 -- 172.21.15.80:0/2501905276 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f31c80b5a10 con 0x7f31cc06f5a0 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31c17fa640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31b806f400 0x7f31b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31c17fa640 1 -- 172.21.15.80:0/2501905276 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f31c8129b00 con 0x7f31cc06f5a0 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.334+0000 7f31c3fff640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31b806f400 0x7f31b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:44.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.338+0000 7f31c3fff640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31b806f400 0x7f31b8071910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f31bc006fd0 tx=0x7f31bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:44.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.338+0000 7f31c17fa640 1 -- 172.21.15.80:0/2501905276 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31c8016050 con 0x7f31cc06f5a0 2024-04-25T13:41:44.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.522+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f31cc0739c0 con 0x7f31cc06f5a0 2024-04-25T13:41:44.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.522+0000 7f31c17fa640 1 -- 172.21.15.80:0/2501905276 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f31c80f6a80 con 0x7f31cc06f5a0 2024-04-25T13:41:44.526 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:41:44.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31b806f400 msgr2=0x7f31b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31b806f400 0x7f31b8071910 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f31bc006fd0 tx=0x7f31bc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:44.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 msgr2=0x7f31cc160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:44.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160250 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f31c80ae040 tx=0x7f31c801ac10 comp rx=0 tx=0).stop 2024-04-25T13:41:44.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 shutdown_connections 2024-04-25T13:41:44.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31b806f400 0x7f31b8071910 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31cc161070 0x7f31cc155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31cc0b4bd0 0x7f31cc159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 --2- 172.21.15.80:0/2501905276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31cc06f5a0 0x7f31cc160250 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:44.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 >> 172.21.15.80:0/2501905276 conn(0x7f31cc06d080 msgr2=0x7f31cc0b2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:44.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 shutdown_connections 2024-04-25T13:41:44.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44.526+0000 7f31d3031640 1 -- 172.21.15.80:0/2501905276 wait complete. 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:41:44.547 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:41:44.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:44.548 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:41:44.554 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42464sec preferred_lft 42464sec 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:41:44.555 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:41:44.556 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:44.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:44.557 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:41:44.605 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:41:44.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:44.607 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:41:44.660 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:41:44.660 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:41:44.660 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:41:44.660 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:41:44.661 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:41:44.661 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:41:44.661 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:41:44.661 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:41:44.661 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:41:44.661 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:41:44.661 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:41:44.661 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:41:44.661 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:41:44.661 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:41:44.662 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:41:44.662 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:41:44.662 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:41:44.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:41:44.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:44.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:44.786 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:41:44.791 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:41:44.792 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:41:44.792 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:41:44.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:44.792 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:41:44.846 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:41:44.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:41:44.847 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:44.891 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:41:44.892 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:41:44.892 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:44.940 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:41:44.997 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:41:45.045 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:41:45.046 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:41:45.046 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:41:45.046 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:41:45.046 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:41:45.046 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:41:45.047 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:41:45.048 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:41:45.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:45.049 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:41:45.096 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:41:45.096 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:41:45.141 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:41:45.156 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:41:45.204 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:41:45.204 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:41:45.204 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:41:45.204 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:41:45.204 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:41:45.204 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:41:45.205 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:41:45.206 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:41:45.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:45.207 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:41:45.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:41:45.246+0000 7f043ce13580 -1 init, newargv = 0x7f042c005830 newargc=15 2024-04-25T13:41:45.250 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[28248]: starting ceph client 2024-04-25T13:41:45.255 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[28248]: starting fuse 2024-04-25T13:41:46.356 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:41:46.373 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:41:46.421 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:41:46.422 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:41:46.423 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:46.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:46.424 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:41:46.472 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:41:46.473 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:41:46.473 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:41:46.474 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:41:46.475 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:41:46.475 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:46.475 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:41:46.475 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:41:46.601 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:46.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:46.612 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:46.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:46.613 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.28248.asok status 2024-04-25T13:41:46.789 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "28248", 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:41:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5430, 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 5430 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.5430 192.168.0.1:0/1283825421", 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1283825421", 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 16, 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 33, 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:41:46.792 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:41:46.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:46.807 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:41:46.807 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:46.813 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:41:46.813 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:46.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:46.813 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:46.869 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:41:47.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79f522640 1 Processor -- start 2024-04-25T13:41:47.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79f522640 1 -- start start 2024-04-25T13:41:47.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79f522640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe7980d4330 0x7fe7980d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79f522640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe798161060 con 0x7fe7980d2e80 2024-04-25T13:41:47.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79f522640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe798161340 con 0x7fe7980d59b0 2024-04-25T13:41:47.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79f522640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe7980d2680 con 0x7fe7980d4330 2024-04-25T13:41:47.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79d2be640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe7980d2e80 0x7fe7981607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51382/0 (socket says 172.21.15.80:51382) 2024-04-25T13:41:47.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79d2be640 1 -- 172.21.15.80:0/65745039 learned_addr learned my addr 172.21.15.80:0/65745039 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:47.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79dabf640 1 --2- 172.21.15.80:0/65745039 >> v2:172.21.15.80:3300/0 conn(0x7fe7980d4330 0x7fe7980d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe78ffff640 1 -- 172.21.15.80:0/65745039 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3907910136 0 0) 0x7fe798161060 con 0x7fe7980d2e80 2024-04-25T13:41:47.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe78ffff640 1 -- 172.21.15.80:0/65745039 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe784003530 con 0x7fe7980d2e80 2024-04-25T13:41:47.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79dabf640 1 -- 172.21.15.80:0/65745039 >> v1:172.21.15.87:6789/0 conn(0x7fe7980d59b0 legacy=0x7fe7980d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79dabf640 1 -- 172.21.15.80:0/65745039 >> v1:172.21.15.80:6789/0 conn(0x7fe7980d2e80 legacy=0x7fe7981607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.054+0000 7fe79dabf640 1 -- 172.21.15.80:0/65745039 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7980d2960 con 0x7fe7980d4330 2024-04-25T13:41:47.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe79dabf640 1 --2- 172.21.15.80:0/65745039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d4330 0x7fe7980d4740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe790009d90 tx=0x7fe79001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6466fb33b87ec8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe78ffff640 1 -- 172.21.15.80:0/65745039 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900b2070 con 0x7fe7980d4330 2024-04-25T13:41:47.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe78ffff640 1 -- 172.21.15.80:0/65745039 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7900a58b0 con 0x7fe7980d4330 2024-04-25T13:41:47.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe78ffff640 1 -- 172.21.15.80:0/65745039 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900a5c80 con 0x7fe7980d4330 2024-04-25T13:41:47.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe79f522640 1 -- 172.21.15.80:0/65745039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d4330 msgr2=0x7fe7980d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe79f522640 1 --2- 172.21.15.80:0/65745039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d4330 0x7fe7980d4740 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe790009d90 tx=0x7fe79001c8e0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe79f522640 1 -- 172.21.15.80:0/65745039 shutdown_connections 2024-04-25T13:41:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe79f522640 1 --2- 172.21.15.80:0/65745039 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d4330 0x7fe7980d4740 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe79f522640 1 -- 172.21.15.80:0/65745039 >> 172.21.15.80:0/65745039 conn(0x7fe7980737a0 msgr2=0x7fe79806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe79f522640 1 -- 172.21.15.80:0/65745039 shutdown_connections 2024-04-25T13:41:47.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.058+0000 7fe79f522640 1 -- 172.21.15.80:0/65745039 wait complete. 2024-04-25T13:41:47.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 Processor -- start 2024-04-25T13:41:47.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 -- start start 2024-04-25T13:41:47.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7980d2e80 0x7fe798158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7980d4330 0x7fe798156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d59b0 0x7fe7980ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe79815e0e0 con 0x7fe7980d59b0 2024-04-25T13:41:47.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe79815df60 con 0x7fe7980d2e80 2024-04-25T13:41:47.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe79816e950 con 0x7fe7980d4330 2024-04-25T13:41:47.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d59b0 0x7fe7980ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d59b0 0x7fe7980ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53270/0 (socket says 172.21.15.80:53270) 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 -- 172.21.15.80:0/3480458328 learned_addr learned my addr 172.21.15.80:0/3480458328 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79d2be640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7980d2e80 0x7fe798158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 -- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7980d4330 msgr2=0x7fe798156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79cabd640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7980d4330 0x7fe798156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7980d4330 0x7fe798156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 -- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7980d2e80 msgr2=0x7fe798158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7980d2e80 0x7fe798158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 -- 172.21.15.80:0/3480458328 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe798156e40 con 0x7fe7980d59b0 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79dabf640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d59b0 0x7fe7980ccb10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe7900b0040 tx=0x7fe790019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe78dffb640 1 -- 172.21.15.80:0/3480458328 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900b2050 con 0x7fe7980d59b0 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe78dffb640 1 -- 172.21.15.80:0/3480458328 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7900be070 con 0x7fe7980d59b0 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7981581a0 con 0x7fe7980d59b0 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe78dffb640 1 -- 172.21.15.80:0/3480458328 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7900ad850 con 0x7fe7980d59b0 2024-04-25T13:41:47.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe798158760 con 0x7fe7980d59b0 2024-04-25T13:41:47.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.062+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7980cd3f0 con 0x7fe7980d59b0 2024-04-25T13:41:47.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.066+0000 7fe78dffb640 1 -- 172.21.15.80:0/3480458328 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7900ad9f0 con 0x7fe7980d59b0 2024-04-25T13:41:47.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.066+0000 7fe78dffb640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe77c06f400 0x7fe77c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:47.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.066+0000 7fe78dffb640 1 -- 172.21.15.80:0/3480458328 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fe79012acc0 con 0x7fe7980d59b0 2024-04-25T13:41:47.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.066+0000 7fe79d2be640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe77c06f400 0x7fe77c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:47.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.066+0000 7fe79d2be640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe77c06f400 0x7fe77c071910 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe794005890 tx=0x7fe794009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:47.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.070+0000 7fe78dffb640 1 -- 172.21.15.80:0/3480458328 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe790016560 con 0x7fe7980d59b0 2024-04-25T13:41:47.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.266+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe79806a3a0 con 0x7fe7980d59b0 2024-04-25T13:41:47.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.266+0000 7fe78dffb640 1 -- 172.21.15.80:0/3480458328 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7fe7900f7c90 con 0x7fe7980d59b0 2024-04-25T13:41:47.269 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe77c06f400 msgr2=0x7fe77c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe77c06f400 0x7fe77c071910 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe794005890 tx=0x7fe794009040 comp rx=0 tx=0).stop 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d59b0 msgr2=0x7fe7980ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d59b0 0x7fe7980ccb10 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe7900b0040 tx=0x7fe790019680 comp rx=0 tx=0).stop 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 shutdown_connections 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe77c06f400 0x7fe77c071910 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7980d59b0 0x7fe7980ccb10 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7980d4330 0x7fe798156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 --2- 172.21.15.80:0/3480458328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7980d2e80 0x7fe798158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 >> 172.21.15.80:0/3480458328 conn(0x7fe7980737a0 msgr2=0x7fe7981624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:47.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 shutdown_connections 2024-04-25T13:41:47.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:47.270+0000 7fe79f522640 1 -- 172.21.15.80:0/3480458328 wait complete. 2024-04-25T13:41:47.289 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:41:47.289 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:41:47.289 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:41:47.289 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:41:47.289 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:47.289 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:41:47.289 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:41:47.289 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:41:47.290 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:41:47.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:47.290 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:41:47.297 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:41:47.297 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:41:47.297 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:41:47.297 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:47.297 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:41:47.297 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:47.297 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:41:47.297 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42462sec preferred_lft 42462sec 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:41:47.298 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:41:47.299 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:41:47.300 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:41:47.300 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:41:47.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:47.300 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:41:47.350 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:41:47.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:47.352 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:41:47.403 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:41:47.403 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:41:47.403 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:41:47.403 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:41:47.403 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:41:47.404 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:41:47.404 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:41:47.404 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:41:47.404 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:41:47.404 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:41:47.404 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:41:47.404 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:41:47.404 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:41:47.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:41:47.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:41:47.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:41:47.405 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:41:47.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:41:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:41:47.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:41:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:47.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:47.532 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:41:47.538 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:41:47.538 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:41:47.538 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:41:47.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:47.538 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:41:47.590 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:41:47.590 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:41:47.591 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:47.639 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:41:47.640 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:41:47.640 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:47.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:41:47.744 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:41:47.792 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:41:47.793 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:41:47.794 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:41:47.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:47.795 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:41:47.848 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:41:47.848 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:41:47.892 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:41:47.906 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:41:47.960 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:41:47.960 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:41:47.961 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:41:47.962 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:41:47.963 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:41:47.963 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:41:47.963 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:41:47.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:47.964 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:41:47.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:41:47.992+0000 7f1f5a9c5580 -1 init, newargv = 0x7f1f4c0015c0 newargc=15 2024-04-25T13:41:47.998 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24238]: starting ceph client 2024-04-25T13:41:48.003 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24238]: starting fuse 2024-04-25T13:41:49.103 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:41:49.120 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:41:49.170 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:41:49.170 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:41:49.170 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:41:49.170 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:41:49.170 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:41:49.170 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:41:49.170 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:41:49.171 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:41:49.172 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:41:49.173 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:49.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:49.173 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:41:49.219 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:41:49.220 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:41:49.220 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:41:49.221 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:41:49.222 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:41:49.222 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:41:49.344 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.24238.asok 2024-04-25T13:41:49.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:41:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:49.355 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.24238.asok 2024-04-25T13:41:49.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:49.355 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.24238.asok status 2024-04-25T13:41:49.538 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:41:49.538 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:41:49.538 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:41:49.538 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:41:49.538 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:41:49.538 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "24238", 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4766, 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 4766 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:41:49.539 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 3152408071 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 3152408071 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.4766 192.168.0.1:0/3152408071", 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/3152408071", 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 16, 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 33, 2024-04-25T13:41:49.540 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:41:49.541 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:41:49.541 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:41:49.541 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:41:49.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:49.554 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:41:49.554 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:49.560 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:41:49.560 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:49.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:49.561 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:49.620 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:49.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.774+0000 7f5722280640 1 Processor -- start 2024-04-25T13:41:49.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.774+0000 7f5722280640 1 -- start start 2024-04-25T13:41:49.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f5722280640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f571c0d14a0 0x7f571c0b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f5722280640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f5722280640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f571c0b7740 0x7f571c0b7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f5722280640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f571c0d18b0 con 0x7f571c0d14a0 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f5722280640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f571c0b5120 con 0x7f571c0b7740 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f5722280640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f571c0b9210 con 0x7f571c0d26d0 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f571c0d14a0 0x7f571c0b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f571c0d14a0 0x7f571c0b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53286/0 (socket says 172.21.15.80:53286) 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571bfff640 1 -- 172.21.15.80:0/257713605 learned_addr learned my addr 172.21.15.80:0/257713605 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f572081d640 1 --2- 172.21.15.80:0/257713605 >> v2:172.21.15.87:3300/0 conn(0x7f571c0b7740 0x7f571c0b7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571b7fe640 1 --2- 172.21.15.80:0/257713605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571bfff640 1 -- 172.21.15.80:0/257713605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 msgr2=0x7f571c0d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571bfff640 1 --2- 172.21.15.80:0/257713605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571bfff640 1 -- 172.21.15.80:0/257713605 >> v2:172.21.15.87:3300/0 conn(0x7f571c0b7740 msgr2=0x7f571c0b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571bfff640 1 --2- 172.21.15.80:0/257713605 >> v2:172.21.15.87:3300/0 conn(0x7f571c0b7740 0x7f571c0b7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571bfff640 1 -- 172.21.15.80:0/257713605 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c0d3020 con 0x7f571c0d14a0 2024-04-25T13:41:49.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.778+0000 7f571b7fe640 1 --2- 172.21.15.80:0/257713605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:49.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f571bfff640 1 --2- 172.21.15.80:0/257713605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c0b4860 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f571c0b93c0 tx=0x7f570c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19b05c0aac3ee83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:49.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f571affd640 1 -- 172.21.15.80:0/257713605 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c0b0070 con 0x7f571c0d14a0 2024-04-25T13:41:49.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f571affd640 1 -- 172.21.15.80:0/257713605 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f570c0ab400 con 0x7f571c0d14a0 2024-04-25T13:41:49.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f571affd640 1 -- 172.21.15.80:0/257713605 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c0ab7d0 con 0x7f571c0d14a0 2024-04-25T13:41:49.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 -- 172.21.15.80:0/257713605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 msgr2=0x7f571c0b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 --2- 172.21.15.80:0/257713605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c0b4860 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f571c0b93c0 tx=0x7f570c01c470 comp rx=0 tx=0).stop 2024-04-25T13:41:49.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 -- 172.21.15.80:0/257713605 shutdown_connections 2024-04-25T13:41:49.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 --2- 172.21.15.80:0/257713605 >> v2:172.21.15.87:3300/0 conn(0x7f571c0b7740 0x7f571c0b7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 --2- 172.21.15.80:0/257713605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0d2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 --2- 172.21.15.80:0/257713605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c0b4860 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 -- 172.21.15.80:0/257713605 >> 172.21.15.80:0/257713605 conn(0x7f571c073770 msgr2=0x7f571c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:49.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 -- 172.21.15.80:0/257713605 shutdown_connections 2024-04-25T13:41:49.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 -- 172.21.15.80:0/257713605 wait complete. 2024-04-25T13:41:49.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 Processor -- start 2024-04-25T13:41:49.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.782+0000 7f5722280640 1 -- start start 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f571c0b7740 0x7f571c14d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c15e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0ba3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f571c164d20 con 0x7f571c0d14a0 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f571c164800 con 0x7f571c0b7740 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f571c0b84a0 con 0x7f571c0d26d0 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c15e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c15e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53292/0 (socket says 172.21.15.80:53292) 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 -- 172.21.15.80:0/442357353 learned_addr learned my addr 172.21.15.80:0/442357353 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571bfff640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f571c0b7740 0x7f571c14d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f572081d640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0ba3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 -- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 msgr2=0x7f571c0ba3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0ba3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 -- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f571c0b7740 msgr2=0x7f571c14d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f571c0b7740 0x7f571c14d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 -- 172.21.15.80:0/442357353 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571c1589d0 con 0x7f571c0d14a0 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f571b7fe640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c15e200 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f571400cef0 tx=0x7f571400a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5718ff9640 1 -- 172.21.15.80:0/442357353 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571400e860 con 0x7f571c0d14a0 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5718ff9640 1 -- 172.21.15.80:0/442357353 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f571400ea00 con 0x7f571c0d14a0 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5718ff9640 1 -- 172.21.15.80:0/442357353 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571400edb0 con 0x7f571c0d14a0 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571c158cb0 con 0x7f571c0d14a0 2024-04-25T13:41:49.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f571c159270 con 0x7f571c0d14a0 2024-04-25T13:41:49.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f571c06a3a0 con 0x7f571c0d14a0 2024-04-25T13:41:49.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5718ff9640 1 -- 172.21.15.80:0/442357353 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5714016420 con 0x7f571c0d14a0 2024-04-25T13:41:49.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5718ff9640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f56fc06f400 0x7f56fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:49.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.786+0000 7f5718ff9640 1 -- 172.21.15.80:0/442357353 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f571408ff00 con 0x7f571c0d14a0 2024-04-25T13:41:49.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.790+0000 7f571bfff640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f56fc06f400 0x7f56fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:49.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.790+0000 7f571bfff640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f56fc06f400 0x7f56fc071910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f570c0ad000 tx=0x7f570c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:49.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.794+0000 7f5718ff9640 1 -- 172.21.15.80:0/442357353 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f571405cf10 con 0x7f571c0d14a0 2024-04-25T13:41:49.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.986+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f571c158720 con 0x7f571c0d14a0 2024-04-25T13:41:49.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5718ff9640 1 -- 172.21.15.80:0/442357353 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5175 (secure 0 0 0) 0x7f57140630e0 con 0x7f571c0d14a0 2024-04-25T13:41:49.991 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:49.991 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:41:32.974439+0000","modified":"2024-04-25T13:41:36.017164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/966995060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":966995060},{"type":"v1","addr":"172.21.15.87:6835","nonce":966995060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5190,"qdb_cluster":[5190]},"id":2}]} 2024-04-25T13:41:49.992 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 16 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f56fc06f400 msgr2=0x7f56fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f56fc06f400 0x7f56fc071910 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f570c0ad000 tx=0x7f570c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 msgr2=0x7f571c15e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c15e200 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f571400cef0 tx=0x7f571400a6c0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 shutdown_connections 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f56fc06f400 0x7f56fc071910 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f571c0d26d0 0x7f571c0ba3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f571c0d14a0 0x7f571c15e200 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 --2- 172.21.15.80:0/442357353 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f571c0b7740 0x7f571c14d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 >> 172.21.15.80:0/442357353 conn(0x7f571c073770 msgr2=0x7f571c14d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:49.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 shutdown_connections 2024-04-25T13:41:49.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:49.990+0000 7f5722280640 1 -- 172.21.15.80:0/442357353 wait complete. 2024-04-25T13:41:50.013 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-04-25T13:41:50.191 INFO:teuthology.orchestra.run.smithi087.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":5190,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.608493106000001,"mdsmap_epoch":16,"osdmap_epoch":33,"osdmap_epoch_barrier":33,"uptime":24.494208307000001} 2024-04-25T13:41:50.191 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 5190, "mdsmap_epoch": 16, "osdmap_epoch": 33, "osdmap_epoch_barrier": 33, "rank_uptime": 16.608493106, "state": "up:active", "uptime": 24.494208307, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:41:50.192 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2024-04-25T13:41:50.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.350+0000 7f0f60eca640 1 Processor -- start 2024-04-25T13:41:50.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.350+0000 7f0f60eca640 1 -- start start 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f60eca640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0f5c0b0ed0 0x7f0f5c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f60eca640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0ce730 0x7f0f5c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f60eca640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f5c15cda0 con 0x7f0f5c06e490 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f60eca640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0f5c073710 con 0x7f0f5c0b0ed0 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f60eca640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c0739f0 con 0x7f0f5c0ce730 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5a59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0f5c0b0ed0 0x7f0f5c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5a59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0f5c0b0ed0 0x7f0f5c15c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53308/0 (socket says 172.21.15.80:53308) 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5a59c640 1 -- 172.21.15.80:0/1793043678 learned_addr learned my addr 172.21.15.80:0/1793043678 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f59d9b640 1 --2- 172.21.15.80:0/1793043678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0ce730 0x7f0f5c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:50.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5a59c640 1 -- 172.21.15.80:0/1793043678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0ce730 msgr2=0x7f0f5c0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5a59c640 1 --2- 172.21.15.80:0/1793043678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0ce730 0x7f0f5c0ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5a59c640 1 -- 172.21.15.80:0/1793043678 >> v1:172.21.15.87:6789/0 conn(0x7f0f5c06e490 legacy=0x7f0f5c06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5a59c640 1 -- 172.21.15.80:0/1793043678 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c073cd0 con 0x7f0f5c0b0ed0 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5a59c640 1 --2- 172.21.15.80:0/1793043678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c0b0ed0 0x7f0f5c15c860 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400af60 tx=0x7f0f54020330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626933bec7e51dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5959a640 1 -- 172.21.15.80:0/1793043678 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540b5070 con 0x7f0f5c0b0ed0 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5959a640 1 -- 172.21.15.80:0/1793043678 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5400a2d0 con 0x7f0f5c0b0ed0 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.354+0000 7f0f5959a640 1 -- 172.21.15.80:0/1793043678 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54008720 con 0x7f0f5c0b0ed0 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- 172.21.15.80:0/1793043678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c0b0ed0 msgr2=0x7f0f5c15c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1793043678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c0b0ed0 0x7f0f5c15c860 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400af60 tx=0x7f0f54020330 comp rx=0 tx=0).stop 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- 172.21.15.80:0/1793043678 shutdown_connections 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1793043678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0ce730 0x7f0f5c0ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1793043678 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c0b0ed0 0x7f0f5c15c860 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- 172.21.15.80:0/1793043678 >> 172.21.15.80:0/1793043678 conn(0x7f0f5c06d080 msgr2=0x7f0f5c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:50.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- 172.21.15.80:0/1793043678 shutdown_connections 2024-04-25T13:41:50.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- 172.21.15.80:0/1793043678 wait complete. 2024-04-25T13:41:50.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 Processor -- start 2024-04-25T13:41:50.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- start start 2024-04-25T13:41:50.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c06e490 0x7f0f5c0d0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:50.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0b0ed0 0x7f0f5c0cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:50.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c0ce730 0x7f0f5c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:50.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c0733a0 con 0x7f0f5c06e490 2024-04-25T13:41:50.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c0b12e0 con 0x7f0f5c0ce730 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f60eca640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c06dc00 con 0x7f0f5c0b0ed0 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f5a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c06e490 0x7f0f5c0d0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f5a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c06e490 0x7f0f5c0d0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53324/0 (socket says 172.21.15.80:53324) 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.358+0000 7f0f5a59c640 1 -- 172.21.15.80:0/1483506780 learned_addr learned my addr 172.21.15.80:0/1483506780 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f5ad9d640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c0ce730 0x7f0f5c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f59d9b640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0b0ed0 0x7f0f5c0cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f5a59c640 1 -- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0b0ed0 msgr2=0x7f0f5c0cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f5a59c640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0b0ed0 0x7f0f5c0cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f5a59c640 1 -- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c0ce730 msgr2=0x7f0f5c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f5a59c640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c0ce730 0x7f0f5c0af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f5a59c640 1 -- 172.21.15.80:0/1483506780 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c0cf9f0 con 0x7f0f5c06e490 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f5ad9d640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c0ce730 0x7f0f5c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:50.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f5a59c640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c06e490 0x7f0f5c0d0c30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0f5401e9e0 tx=0x7f0f5401d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:50.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f52ffd640 1 -- 172.21.15.80:0/1483506780 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540c0070 con 0x7f0f5c06e490 2024-04-25T13:41:50.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f52ffd640 1 -- 172.21.15.80:0/1483506780 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f54008ce0 con 0x7f0f5c06e490 2024-04-25T13:41:50.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c0cfcd0 con 0x7f0f5c06e490 2024-04-25T13:41:50.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f52ffd640 1 -- 172.21.15.80:0/1483506780 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f540b5040 con 0x7f0f5c06e490 2024-04-25T13:41:50.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c15d1b0 con 0x7f0f5c06e490 2024-04-25T13:41:50.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c15cfa0 con 0x7f0f5c06e490 2024-04-25T13:41:50.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f52ffd640 1 -- 172.21.15.80:0/1483506780 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0f540c3070 con 0x7f0f5c06e490 2024-04-25T13:41:50.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f52ffd640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f3c06f430 0x7f0f3c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:50.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f52ffd640 1 -- 172.21.15.80:0/1483506780 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f0f5412ca20 con 0x7f0f5c06e490 2024-04-25T13:41:50.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.362+0000 7f0f59d9b640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f3c06f430 0x7f0f3c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:50.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.366+0000 7f0f59d9b640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f3c06f430 0x7f0f3c071940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c1fa470 tx=0x7f0f44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:50.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.366+0000 7f0f52ffd640 1 -- 172.21.15.80:0/1483506780 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0f54019e90 con 0x7f0f5c06e490 2024-04-25T13:41:50.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.582+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) -- 0x7f0f5c06a3a0 con 0x7f0f5c06e490 2024-04-25T13:41:50.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.582+0000 7f0f52ffd640 1 -- 172.21.15.80:0/1483506780 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) ==== 75+0+1620 (secure 0 0 0) 0x7f0f540f9a80 con 0x7f0f5c06e490 2024-04-25T13:41:50.586 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "stats": { 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "total_bytes": 1600000000, 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "total_avail_bytes": 1597641914, 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "total_used_bytes": 2358086, 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "total_used_raw_bytes": 2358086, 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "total_used_raw_ratio": 0.0014738037716597319, 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "num_osds": 8, 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "num_per_pool_osds": 0, 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "num_per_pool_omap_osds": 0 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "stats_by_class": { 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "hdd": { 2024-04-25T13:41:50.587 INFO:teuthology.orchestra.run.smithi080.stdout: "total_bytes": 1600000000, 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "total_avail_bytes": 1597641914, 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "total_used_bytes": 2358086, 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "total_used_raw_bytes": 2358086, 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "total_used_raw_ratio": 0.0014738037716597319 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "name": ".mgr", 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 1, 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "stats": { 2024-04-25T13:41:50.588 INFO:teuthology.orchestra.run.smithi080.stdout: "stored": 590368, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "objects": 2, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "kb_used": 577, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bytes_used": 590368, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "percent_used": 0.00052962568588554859, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "max_avail": 557049408 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cephfs_metadata", 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 4, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "stats": { 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "stored": 4107, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "objects": 22, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "kb_used": 5, 2024-04-25T13:41:50.589 INFO:teuthology.orchestra.run.smithi080.stdout: "bytes_used": 4107, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "percent_used": 3.6863739296677522e-06, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "max_avail": 557049408 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "name": "cephfs_data", 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "stats": { 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "stored": 0, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "objects": 0, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "kb_used": 0, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "bytes_used": 0, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "percent_used": 0, 2024-04-25T13:41:50.590 INFO:teuthology.orchestra.run.smithi080.stdout: "max_avail": 557049408 2024-04-25T13:41:50.591 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:50.591 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:50.591 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:41:50.591 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f3c06f430 msgr2=0x7f0f3c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f3c06f430 0x7f0f3c071940 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c1fa470 tx=0x7f0f44008040 comp rx=0 tx=0).stop 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c06e490 msgr2=0x7f0f5c0d0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c06e490 0x7f0f5c0d0c30 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0f5401e9e0 tx=0x7f0f5401d0d0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 shutdown_connections 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f3c06f430 0x7f0f3c071940 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f5c0ce730 0x7f0f5c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0f5c0b0ed0 0x7f0f5c0cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 --2- 172.21.15.80:0/1483506780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0f5c06e490 0x7f0f5c0d0c30 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 >> 172.21.15.80:0/1483506780 conn(0x7f0f5c06d080 msgr2=0x7f0f5c1e5570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:50.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 shutdown_connections 2024-04-25T13:41:50.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50.586+0000 7f0f60eca640 1 -- 172.21.15.80:0/1483506780 wait complete. 2024-04-25T13:41:50.607 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:41:50.607 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:41:50.607 DEBUG:teuthology.orchestra.run.smithi080:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-25T13:41:50.608 DEBUG:teuthology.orchestra.run.smithi080:> os.fchmod(fd, 0O666) 2024-04-25T13:41:50.608 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(fd) 2024-04-25T13:41:50.608 DEBUG:teuthology.orchestra.run.smithi080:> os.close(fd) 2024-04-25T13:41:50.608 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:41:50.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:41:50.680 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:41:50.681 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:41:50.789 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:50.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:50.799 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:50.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:50.799 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.28248.asok status 2024-04-25T13:41:50.971 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:41:50.971 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:41:50.971 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:41:50.971 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:41:50.971 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:41:50.971 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:41:50.971 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "28248", 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 1, 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5430, 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 5430 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:50.972 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.5430 192.168.0.1:0/1283825421", 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1283825421", 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 2, 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 16, 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 33, 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 33, 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:41:50.973 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:41:50.974 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:41:50.985 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-04-25T13:41:51.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd452bfa640 1 Processor -- start 2024-04-25T13:41:51.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd452bfa640 1 -- start start 2024-04-25T13:41:51.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd452bfa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c161f90 0x7fd44c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:51.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd452bfa640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd44c15ec40 0x7fd44c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:51.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd452bfa640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd44c0d0d70 con 0x7fd44c0d0510 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd452bfa640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd44c0d1050 con 0x7fd44c15ec40 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd452bfa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c06dca0 con 0x7fd44c161f90 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd451197640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd44c15ec40 0x7fd44c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd44bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd44c0d0510 0x7fd44c15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49636/0 (socket says 172.21.15.80:49636) 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd44bfff640 1 -- 172.21.15.80:0/737394629 learned_addr learned my addr 172.21.15.80:0/737394629 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd450996640 1 --2- 172.21.15.80:0/737394629 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c161f90 0x7fd44c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd451197640 1 -- 172.21.15.80:0/737394629 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c161f90 msgr2=0x7fd44c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd451197640 1 --2- 172.21.15.80:0/737394629 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c161f90 0x7fd44c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd451197640 1 -- 172.21.15.80:0/737394629 >> v1:172.21.15.80:6789/0 conn(0x7fd44c0d0510 legacy=0x7fd44c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd451197640 1 -- 172.21.15.80:0/737394629 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c06df80 con 0x7fd44c15ec40 2024-04-25T13:41:51.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.134+0000 7fd44b7fe640 1 -- 172.21.15.80:0/737394629 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509919620 0 0) 0x7fd44c0d0d70 con 0x7fd44c0d0510 2024-04-25T13:41:51.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd451197640 1 --2- 172.21.15.80:0/737394629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c15f050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009d90 tx=0x7fd43c01c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bdbdc828f64e373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:51.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd44b7fe640 1 -- 172.21.15.80:0/737394629 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0b2070 con 0x7fd44c15ec40 2024-04-25T13:41:51.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd44b7fe640 1 -- 172.21.15.80:0/737394629 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd43c0a5890 con 0x7fd44c15ec40 2024-04-25T13:41:51.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd44b7fe640 1 -- 172.21.15.80:0/737394629 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd43c0a5c60 con 0x7fd44c15ec40 2024-04-25T13:41:51.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd452bfa640 1 -- 172.21.15.80:0/737394629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 msgr2=0x7fd44c15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:51.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd452bfa640 1 --2- 172.21.15.80:0/737394629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c15f050 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd43c009d90 tx=0x7fd43c01c8a0 comp rx=0 tx=0).stop 2024-04-25T13:41:51.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd452bfa640 1 -- 172.21.15.80:0/737394629 shutdown_connections 2024-04-25T13:41:51.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd452bfa640 1 --2- 172.21.15.80:0/737394629 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c15f050 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:51.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd452bfa640 1 --2- 172.21.15.80:0/737394629 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c161f90 0x7fd44c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:51.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd452bfa640 1 -- 172.21.15.80:0/737394629 >> 172.21.15.80:0/737394629 conn(0x7fd44c06d080 msgr2=0x7fd44c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:51.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd452bfa640 1 -- 172.21.15.80:0/737394629 shutdown_connections 2024-04-25T13:41:51.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.138+0000 7fd452bfa640 1 -- 172.21.15.80:0/737394629 wait complete. 2024-04-25T13:41:51.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 Processor -- start 2024-04-25T13:41:51.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 -- start start 2024-04-25T13:41:51.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c0d0510 0x7fd44c0afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:51.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:51.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd44c161f90 0x7fd44c1470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:51.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c06f630 con 0x7fd44c15ec40 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd44c06f4b0 con 0x7fd44c161f90 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd44c0d0920 con 0x7fd44c0d0510 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c146b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58476/0 (socket says 172.21.15.80:58476) 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 -- 172.21.15.80:0/1544516093 learned_addr learned my addr 172.21.15.80:0/1544516093 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd451197640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd44c161f90 0x7fd44c1470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd450996640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c0d0510 0x7fd44c0afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 -- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c0d0510 msgr2=0x7fd44c0afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c0d0510 0x7fd44c0afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 -- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd44c161f90 msgr2=0x7fd44c1470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd44c161f90 0x7fd44c1470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 -- 172.21.15.80:0/1544516093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd44c145b10 con 0x7fd44c15ec40 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd450996640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c0d0510 0x7fd44c0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:51.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd44bfff640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c146b90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c410 tx=0x7fd440001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:51.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd4497fa640 1 -- 172.21.15.80:0/1544516093 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4400074e0 con 0x7fd44c15ec40 2024-04-25T13:41:51.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd4497fa640 1 -- 172.21.15.80:0/1544516093 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd440007680 con 0x7fd44c15ec40 2024-04-25T13:41:51.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd44c145df0 con 0x7fd44c15ec40 2024-04-25T13:41:51.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd4497fa640 1 -- 172.21.15.80:0/1544516093 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd440004a10 con 0x7fd44c15ec40 2024-04-25T13:41:51.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd44c1463b0 con 0x7fd44c15ec40 2024-04-25T13:41:51.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.142+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd44c06a3a0 con 0x7fd44c15ec40 2024-04-25T13:41:51.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.146+0000 7fd4497fa640 1 -- 172.21.15.80:0/1544516093 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd440011070 con 0x7fd44c15ec40 2024-04-25T13:41:51.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.146+0000 7fd4497fa640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd43806f400 0x7fd438071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:51.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.146+0000 7fd4497fa640 1 -- 172.21.15.80:0/1544516093 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7fd44008ee90 con 0x7fd44c15ec40 2024-04-25T13:41:51.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.146+0000 7fd450996640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd43806f400 0x7fd438071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:51.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.146+0000 7fd450996640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd43806f400 0x7fd438071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd44c06fa20 tx=0x7fd444008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:51.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.150+0000 7fd4497fa640 1 -- 172.21.15.80:0/1544516093 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd44005bd80 con 0x7fd44c15ec40 2024-04-25T13:41:51.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:51.318+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) -- 0x7fd44c0b6150 con 0x7fd44c15ec40 2024-04-25T13:41:52.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.222+0000 7fd4497fa640 1 -- 172.21.15.80:0/1544516093 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v34) ==== 93+0+0 (secure 0 0 0) 0x7fd44005b680 con 0x7fd44c15ec40 2024-04-25T13:41:52.226 INFO:teuthology.orchestra.run.smithi080.stderr:pauserd,pausewr is set 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd43806f400 msgr2=0x7fd438071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd43806f400 0x7fd438071910 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd44c06fa20 tx=0x7fd444008040 comp rx=0 tx=0).stop 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 msgr2=0x7fd44c146b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c146b90 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fd44000c410 tx=0x7fd440001e80 comp rx=0 tx=0).stop 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 shutdown_connections 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd43806f400 0x7fd438071910 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd44c161f90 0x7fd44c1470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd44c15ec40 0x7fd44c146b90 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 --2- 172.21.15.80:0/1544516093 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd44c0d0510 0x7fd44c0afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 >> 172.21.15.80:0/1544516093 conn(0x7fd44c06d080 msgr2=0x7fd44c1611f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:52.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 shutdown_connections 2024-04-25T13:41:52.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.226+0000 7fd452bfa640 1 -- 172.21.15.80:0/1544516093 wait complete. 2024-04-25T13:41:52.254 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-04-25T13:41:52.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f601533b640 1 Processor -- start 2024-04-25T13:41:52.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f601533b640 1 -- start start 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f601533b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f60100d14a0 0x7f60100b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f601533b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60100d26d0 0x7f60100d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f601533b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60100b9210 con 0x7f60100b7740 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f601533b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f60100d18b0 con 0x7f60100d26d0 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f601533b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f60100b5120 con 0x7f60100d14a0 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600f7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f60100b7740 0x7f60100b7b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49648/0 (socket says 172.21.15.80:49648) 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600f7fe640 1 -- 172.21.15.80:0/2119212442 learned_addr learned my addr 172.21.15.80:0/2119212442 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600e7fc640 1 --2- 172.21.15.80:0/2119212442 >> v2:172.21.15.80:3300/0 conn(0x7f60100d26d0 0x7f60100d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:52.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600effd640 1 --2- 172.21.15.80:0/2119212442 >> v2:172.21.15.87:3300/0 conn(0x7f60100d14a0 0x7f60100b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:52.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600dffb640 1 -- 172.21.15.80:0/2119212442 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4120006395 0 0) 0x7f60080014a0 con 0x7f60100b7740 2024-04-25T13:41:52.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600dffb640 1 -- 172.21.15.80:0/2119212442 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ff8003530 con 0x7f60100b7740 2024-04-25T13:41:52.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600e7fc640 1 -- 172.21.15.80:0/2119212442 >> v2:172.21.15.87:3300/0 conn(0x7f60100d14a0 msgr2=0x7f60100b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:52.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600e7fc640 1 --2- 172.21.15.80:0/2119212442 >> v2:172.21.15.87:3300/0 conn(0x7f60100d14a0 0x7f60100b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600e7fc640 1 -- 172.21.15.80:0/2119212442 >> v1:172.21.15.80:6789/0 conn(0x7f60100b7740 legacy=0x7f60100b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:52.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.406+0000 7f600e7fc640 1 -- 172.21.15.80:0/2119212442 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60100d56b0 con 0x7f60100d26d0 2024-04-25T13:41:52.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f600e7fc640 1 --2- 172.21.15.80:0/2119212442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100d26d0 0x7f60100d2ae0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f60100d6d60 tx=0x7f600401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba76e2a3bc5a3bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:52.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f600dffb640 1 -- 172.21.15.80:0/2119212442 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60040b0070 con 0x7f60100d26d0 2024-04-25T13:41:52.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f600dffb640 1 -- 172.21.15.80:0/2119212442 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60040ab600 con 0x7f60100d26d0 2024-04-25T13:41:52.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f600dffb640 1 -- 172.21.15.80:0/2119212442 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60040ab9d0 con 0x7f60100d26d0 2024-04-25T13:41:52.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f601533b640 1 -- 172.21.15.80:0/2119212442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100d26d0 msgr2=0x7f60100d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:52.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f601533b640 1 --2- 172.21.15.80:0/2119212442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100d26d0 0x7f60100d2ae0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f60100d6d60 tx=0x7f600401c3b0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f601533b640 1 -- 172.21.15.80:0/2119212442 shutdown_connections 2024-04-25T13:41:52.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f601533b640 1 --2- 172.21.15.80:0/2119212442 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100d26d0 0x7f60100d2ae0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f601533b640 1 --2- 172.21.15.80:0/2119212442 >> v2:172.21.15.87:3300/0 conn(0x7f60100d14a0 0x7f60100b4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f601533b640 1 -- 172.21.15.80:0/2119212442 >> 172.21.15.80:0/2119212442 conn(0x7f6010073770 msgr2=0x7f601006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:52.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f601533b640 1 -- 172.21.15.80:0/2119212442 shutdown_connections 2024-04-25T13:41:52.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.410+0000 7f601533b640 1 -- 172.21.15.80:0/2119212442 wait complete. 2024-04-25T13:41:52.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 Processor -- start 2024-04-25T13:41:52.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 -- start start 2024-04-25T13:41:52.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100b7740 0x7f6010156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:52.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60100d14a0 0x7f60101590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:52.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60100d26d0 0x7f6010159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:52.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6010164800 con 0x7f60100b7740 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6010164520 con 0x7f60100d26d0 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6010164d20 con 0x7f60100d14a0 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100b7740 0x7f6010156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100b7740 0x7f6010156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58488/0 (socket says 172.21.15.80:58488) 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 -- 172.21.15.80:0/3228744321 learned_addr learned my addr 172.21.15.80:0/3228744321 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 -- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60100d14a0 msgr2=0x7f60101590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60100d14a0 0x7f60101590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 -- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60100d26d0 msgr2=0x7f6010159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600f7fe640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60100d26d0 0x7f6010159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60100d26d0 0x7f6010159620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 -- 172.21.15.80:0/3228744321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6010159df0 con 0x7f60100b7740 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f600effd640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100b7740 0x7f6010156030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f600000d5f0 tx=0x7f600000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000125f0 con 0x7f60100b7740 2024-04-25T13:41:52.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6000019070 con 0x7f60100b7740 2024-04-25T13:41:52.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601015a0d0 con 0x7f60100b7740 2024-04-25T13:41:52.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60000129b0 con 0x7f60100b7740 2024-04-25T13:41:52.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601015a690 con 0x7f60100b7740 2024-04-25T13:41:52.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.414+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601006a3a0 con 0x7f60100b7740 2024-04-25T13:41:52.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.418+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6000012b50 con 0x7f60100b7740 2024-04-25T13:41:52.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.418+0000 7f5ff77fe640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ffc06f400 0x7f5ffc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:52.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.418+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f6000090140 con 0x7f60100b7740 2024-04-25T13:41:52.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.418+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=35+}) -- 0x7f5ffc076fe0 con 0x7f60100b7740 2024-04-25T13:41:52.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.418+0000 7f600e7fc640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ffc06f400 0x7f5ffc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:52.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.418+0000 7f600e7fc640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ffc06f400 0x7f5ffc071910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60040ad000 tx=0x7f6004004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:52.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.422+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f600005d110 con 0x7f60100b7740 2024-04-25T13:41:52.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:52.594+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) -- 0x7f60100d4130 con 0x7f60100b7740 2024-04-25T13:41:53.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.226+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v35) ==== 97+0+0 (secure 0 0 0) 0x7f600005ca10 con 0x7f60100b7740 2024-04-25T13:41:53.230 INFO:teuthology.orchestra.run.smithi080.stderr:pauserd,pausewr is unset 2024-04-25T13:41:53.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.226+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 <== mon.0 v2:172.21.15.80:3300/0 8 ==== osd_map(35..35 src has 1..35) ==== 288+0+0 (secure 0 0 0) 0x7f6000012e40 con 0x7f60100b7740 2024-04-25T13:41:53.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.226+0000 7f5ff77fe640 1 -- 172.21.15.80:0/3228744321 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=36}) -- 0x7f5ffc077900 con 0x7f60100b7740 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ffc06f400 msgr2=0x7f5ffc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ffc06f400 0x7f5ffc071910 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f60040ad000 tx=0x7f6004004740 comp rx=0 tx=0).stop 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100b7740 msgr2=0x7f6010156030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100b7740 0x7f6010156030 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f600000d5f0 tx=0x7f600000dab0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 shutdown_connections 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ffc06f400 0x7f5ffc071910 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60100d26d0 0x7f6010159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60100d14a0 0x7f60101590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 --2- 172.21.15.80:0/3228744321 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60100b7740 0x7f6010156030 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 >> 172.21.15.80:0/3228744321 conn(0x7f6010073770 msgr2=0x7f60100d4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:53.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 shutdown_connections 2024-04-25T13:41:53.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.234+0000 7f601533b640 1 -- 172.21.15.80:0/3228744321 wait complete. 2024-04-25T13:41:53.263 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:53.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.414+0000 7f26a95fd640 1 Processor -- start 2024-04-25T13:41:53.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.414+0000 7f26a95fd640 1 -- start start 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a95fd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f26a406ebb0 0x7f26a406efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a95fd640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a40d0af0 con 0x7f26a40d06e0 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a95fd640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26a40d0d70 con 0x7f26a4161f90 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a95fd640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f26a40d1050 con 0x7f26a406ebb0 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a27fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f26a40d06e0 0x7f26a406e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49656/0 (socket says 172.21.15.80:49656) 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a27fc640 1 -- 172.21.15.80:0/1251794489 learned_addr learned my addr 172.21.15.80:0/1251794489 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a37fe640 1 --2- 172.21.15.80:0/1251794489 >> v2:172.21.15.87:3300/0 conn(0x7f26a406ebb0 0x7f26a406efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 315678265 0 0) 0x7f2698001120 con 0x7f26a40d06e0 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f268c003530 con 0x7f26a40d06e0 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1000483631 0 0) 0x7f268c003530 con 0x7f26a40d06e0 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 >> v2:172.21.15.87:3300/0 conn(0x7f26a406ebb0 msgr2=0x7f26a406efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 --2- 172.21.15.80:0/1251794489 >> v2:172.21.15.87:3300/0 conn(0x7f26a406ebb0 0x7f26a406efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 >> v1:172.21.15.87:6789/0 conn(0x7f26a4161f90 legacy=0x7f26a40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a406dca0 con 0x7f26a40d06e0 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2698001850 con 0x7f26a40d06e0 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 >> v1:172.21.15.80:6789/0 conn(0x7f26a40d06e0 legacy=0x7f26a406e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f268c0039c0 0x7f268c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c004810 0x7f268c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268c005360 0x7f268c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26a40d1050 con 0x7f268c0039c0 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f268c00c290 con 0x7f268c004810 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f268c00c490 con 0x7f268c005360 2024-04-25T13:41:53.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a2ffd640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f268c0039c0 0x7f268c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a27fc640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c004810 0x7f268c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a37fe640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268c005360 0x7f268c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a2ffd640 1 -- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268c005360 msgr2=0x7f268c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a2ffd640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268c005360 0x7f268c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a2ffd640 1 -- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c004810 msgr2=0x7f268c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a2ffd640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c004810 0x7f268c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a37fe640 1 -- 172.21.15.80:0/1251794489 reap_dead start 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a2ffd640 1 -- 172.21.15.80:0/1251794489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268c003530 con 0x7f268c0039c0 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.418+0000 7f26a27fc640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c004810 0x7f268c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a2ffd640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f268c0039c0 0x7f268c003e30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f26a4147e20 tx=0x7f269401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7df74b2df51cc7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26940013a0 con 0x7f268c0039c0 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26940052f0 con 0x7f268c0039c0 2024-04-25T13:41:53.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a1ffb640 1 -- 172.21.15.80:0/1251794489 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2694003840 con 0x7f268c0039c0 2024-04-25T13:41:53.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a95fd640 1 -- 172.21.15.80:0/1251794489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f268c0039c0 msgr2=0x7f268c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a95fd640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f268c0039c0 0x7f268c003e30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f26a4147e20 tx=0x7f269401c390 comp rx=0 tx=0).stop 2024-04-25T13:41:53.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a95fd640 1 -- 172.21.15.80:0/1251794489 shutdown_connections 2024-04-25T13:41:53.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a95fd640 1 --2- 172.21.15.80:0/1251794489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f268c0039c0 0x7f268c003e30 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a95fd640 1 -- 172.21.15.80:0/1251794489 >> 172.21.15.80:0/1251794489 conn(0x7f26a406d080 msgr2=0x7f26a40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:53.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a95fd640 1 -- 172.21.15.80:0/1251794489 shutdown_connections 2024-04-25T13:41:53.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.422+0000 7f26a95fd640 1 -- 172.21.15.80:0/1251794489 wait complete. 2024-04-25T13:41:53.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 Processor -- start 2024-04-25T13:41:53.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 -- start start 2024-04-25T13:41:53.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c0039c0 0x7f26a4157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:53.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26a4154720 0x7f26a40af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26a4154b30 0x7f26a40afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26a4162520 con 0x7f26a4154b30 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f26a41623a0 con 0x7f268c0039c0 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26a406e8b0 con 0x7f26a4154720 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26a4154b30 0x7f26a40afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26a4154b30 0x7f26a40afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58516/0 (socket says 172.21.15.80:58516) 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 -- 172.21.15.80:0/3445288489 learned_addr learned my addr 172.21.15.80:0/3445288489 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a27fc640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26a4154720 0x7f26a40af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a2ffd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c0039c0 0x7f26a4157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 -- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26a4154720 msgr2=0x7f26a40af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26a4154720 0x7f26a40af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 -- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c0039c0 msgr2=0x7f26a4157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c0039c0 0x7f26a4157f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 -- 172.21.15.80:0/3445288489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26a4140370 con 0x7f26a4154b30 2024-04-25T13:41:53.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a37fe640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26a4154b30 0x7f26a40afe20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f26a40b1080 tx=0x7f269c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:53.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f268b7fe640 1 -- 172.21.15.80:0/3445288489 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c016620 con 0x7f26a4154b30 2024-04-25T13:41:53.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f268b7fe640 1 -- 172.21.15.80:0/3445288489 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f269c01b070 con 0x7f26a4154b30 2024-04-25T13:41:53.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26a415f900 con 0x7f26a4154b30 2024-04-25T13:41:53.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f268b7fe640 1 -- 172.21.15.80:0/3445288489 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f269c016a20 con 0x7f26a4154b30 2024-04-25T13:41:53.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26a415fec0 con 0x7f26a4154b30 2024-04-25T13:41:53.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.426+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26a406a3a0 con 0x7f26a4154b30 2024-04-25T13:41:53.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.430+0000 7f268b7fe640 1 -- 172.21.15.80:0/3445288489 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f269c016bc0 con 0x7f26a4154b30 2024-04-25T13:41:53.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.430+0000 7f268b7fe640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f269006f430 0x7f2690071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:53.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.430+0000 7f268b7fe640 1 -- 172.21.15.80:0/3445288489 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f269c08fe40 con 0x7f26a4154b30 2024-04-25T13:41:53.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.430+0000 7f26a2ffd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f269006f430 0x7f2690071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:53.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.430+0000 7f26a2ffd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f269006f430 0x7f2690071940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f26a406fa20 tx=0x7f26940011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:53.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.434+0000 7f268b7fe640 1 -- 172.21.15.80:0/3445288489 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26a4140370 con 0x7f26a4154b30 2024-04-25T13:41:53.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.598+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26a4155db0 con 0x7f26a4154b30 2024-04-25T13:41:53.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.598+0000 7f268b7fe640 1 -- 172.21.15.80:0/3445288489 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7f269c05cdc0 con 0x7f26a4154b30 2024-04-25T13:41:53.601 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:53.601 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":35,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:53.226827+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:53.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f269006f430 msgr2=0x7f2690071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f269006f430 0x7f2690071940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f26a406fa20 tx=0x7f26940011a0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26a4154b30 msgr2=0x7f26a40afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26a4154b30 0x7f26a40afe20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f26a40b1080 tx=0x7f269c00f520 comp rx=0 tx=0).stop 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 shutdown_connections 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f269006f430 0x7f2690071940 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26a4154b30 0x7f26a40afe20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26a4154720 0x7f26a40af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 --2- 172.21.15.80:0/3445288489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f268c0039c0 0x7f26a4157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 >> 172.21.15.80:0/3445288489 conn(0x7f26a406d080 msgr2=0x7f26a40b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 shutdown_connections 2024-04-25T13:41:53.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53.602+0000 7f26a95fd640 1 -- 172.21.15.80:0/3445288489 wait complete. 2024-04-25T13:41:53.627 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:41:53.627 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:41:53.627 DEBUG:teuthology.orchestra.run.smithi080:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-25T13:41:53.627 DEBUG:teuthology.orchestra.run.smithi080:> os.fchmod(fd, 0O666) 2024-04-25T13:41:53.627 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(fd) 2024-04-25T13:41:53.627 DEBUG:teuthology.orchestra.run.smithi080:> os.close(fd) 2024-04-25T13:41:53.627 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:41:53.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:53.704 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:41:53.704 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:41:53.704 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:41:53.704 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:41:53.705 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:41:53.706 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:41:53.706 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:41:53.706 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:41:53.706 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:41:53.706 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:41:53.706 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:53.706 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:41:53.706 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:41:53.822 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:53.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:53.834 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:53.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:53.834 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.28248.asok status 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "28248", 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 1, 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:41:54.013 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5430, 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 5430 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:54.014 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.5430 192.168.0.1:0/1283825421", 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1283825421", 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 2, 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 16, 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 33, 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 33, 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:41:54.015 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:41:54.028 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:54.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.174+0000 7effd72b5640 1 Processor -- start 2024-04-25T13:41:54.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.174+0000 7effd72b5640 1 -- start start 2024-04-25T13:41:54.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd72b5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd015c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:54.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd72b5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7effd00ce730 0x7effd00ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:54.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd72b5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effd015cda0 con 0x7effd006e490 2024-04-25T13:41:54.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd72b5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7effd0073710 con 0x7effd00ce730 2024-04-25T13:41:54.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd72b5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7effd00739f0 con 0x7effd00b0ed0 2024-04-25T13:41:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd5051640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd4850640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7effd00ce730 0x7effd00ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd5051640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd015c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:57322/0 (socket says 172.21.15.80:57322) 2024-04-25T13:41:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd5051640 1 -- 172.21.15.80:0/3917299580 learned_addr learned my addr 172.21.15.80:0/3917299580 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd5051640 1 -- 172.21.15.80:0/3917299580 >> v2:172.21.15.87:3300/0 conn(0x7effd00ce730 msgr2=0x7effd00ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd5051640 1 --2- 172.21.15.80:0/3917299580 >> v2:172.21.15.87:3300/0 conn(0x7effd00ce730 0x7effd00ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd5051640 1 -- 172.21.15.80:0/3917299580 >> v1:172.21.15.87:6789/0 conn(0x7effd006e490 legacy=0x7effd006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.178+0000 7effd5051640 1 -- 172.21.15.80:0/3917299580 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd0073cd0 con 0x7effd00b0ed0 2024-04-25T13:41:54.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd5051640 1 --2- 172.21.15.80:0/3917299580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd015c860 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7effcc009870 tx=0x7effcc01ed50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34d7a9e022457440 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:54.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effc7fff640 1 -- 172.21.15.80:0/3917299580 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0b5070 con 0x7effd00b0ed0 2024-04-25T13:41:54.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effc7fff640 1 -- 172.21.15.80:0/3917299580 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effcc0a8a50 con 0x7effd00b0ed0 2024-04-25T13:41:54.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effc7fff640 1 -- 172.21.15.80:0/3917299580 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0a8e20 con 0x7effd00b0ed0 2024-04-25T13:41:54.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd72b5640 1 -- 172.21.15.80:0/3917299580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 msgr2=0x7effd015c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:54.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd72b5640 1 --2- 172.21.15.80:0/3917299580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd015c860 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7effcc009870 tx=0x7effcc01ed50 comp rx=0 tx=0).stop 2024-04-25T13:41:54.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd72b5640 1 -- 172.21.15.80:0/3917299580 shutdown_connections 2024-04-25T13:41:54.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd72b5640 1 --2- 172.21.15.80:0/3917299580 >> v2:172.21.15.87:3300/0 conn(0x7effd00ce730 0x7effd00ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd72b5640 1 --2- 172.21.15.80:0/3917299580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd015c860 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd72b5640 1 -- 172.21.15.80:0/3917299580 >> 172.21.15.80:0/3917299580 conn(0x7effd006d080 msgr2=0x7effd00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:54.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd72b5640 1 -- 172.21.15.80:0/3917299580 shutdown_connections 2024-04-25T13:41:54.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.182+0000 7effd72b5640 1 -- 172.21.15.80:0/3917299580 wait complete. 2024-04-25T13:41:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 Processor -- start 2024-04-25T13:41:54.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 -- start start 2024-04-25T13:41:54.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd006e490 0x7effd01f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:54.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd0154d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:54.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd00ce730 0x7effd015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:54.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7effd00733a0 con 0x7effd00ce730 2024-04-25T13:41:54.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7effd00b12e0 con 0x7effd006e490 2024-04-25T13:41:54.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7effd006dc00 con 0x7effd00b0ed0 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd00ce730 0x7effd015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd00ce730 0x7effd015d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58520/0 (socket says 172.21.15.80:58520) 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 -- 172.21.15.80:0/2701420735 learned_addr learned my addr 172.21.15.80:0/2701420735 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd4850640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd0154d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5051640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd006e490 0x7effd01f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 -- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 msgr2=0x7effd0154d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd0154d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 -- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd006e490 msgr2=0x7effd01f5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd006e490 0x7effd01f5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 -- 172.21.15.80:0/2701420735 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd01f6670 con 0x7effd00ce730 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd4850640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd0154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5051640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd006e490 0x7effd01f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd5852640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd00ce730 0x7effd015d8d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7effc800c730 tx=0x7effc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:54.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effc5ffb640 1 -- 172.21.15.80:0/2701420735 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8003ba0 con 0x7effd00ce730 2024-04-25T13:41:54.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effc5ffb640 1 -- 172.21.15.80:0/2701420735 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effc8003d40 con 0x7effd00ce730 2024-04-25T13:41:54.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd01f6950 con 0x7effd00ce730 2024-04-25T13:41:54.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effc5ffb640 1 -- 172.21.15.80:0/2701420735 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8005310 con 0x7effd00ce730 2024-04-25T13:41:54.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd015d3b0 con 0x7effd00ce730 2024-04-25T13:41:54.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.186+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effd006a3a0 con 0x7effd00ce730 2024-04-25T13:41:54.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.190+0000 7effc5ffb640 1 -- 172.21.15.80:0/2701420735 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7effc80054b0 con 0x7effd00ce730 2024-04-25T13:41:54.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.190+0000 7effc5ffb640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7effb406f400 0x7effb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:54.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.190+0000 7effc5ffb640 1 -- 172.21.15.80:0/2701420735 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7effc808f0b0 con 0x7effd00ce730 2024-04-25T13:41:54.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.190+0000 7effd5051640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7effb406f400 0x7effb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:54.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.190+0000 7effd5051640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7effb406f400 0x7effb4071910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7effcc0b2000 tx=0x7effcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:54.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.194+0000 7effc5ffb640 1 -- 172.21.15.80:0/2701420735 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7effc805c030 con 0x7effd00ce730 2024-04-25T13:41:54.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.394+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7effd006f880 con 0x7effd00ce730 2024-04-25T13:41:54.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.394+0000 7effc5ffb640 1 -- 172.21.15.80:0/2701420735 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5175 (secure 0 0 0) 0x7effc805b930 con 0x7effd00ce730 2024-04-25T13:41:54.397 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:54.398 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:41:32.974439+0000","modified":"2024-04-25T13:41:36.017164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/966995060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":966995060},{"type":"v1","addr":"172.21.15.87:6835","nonce":966995060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5190,"qdb_cluster":[5190]},"id":2}]} 2024-04-25T13:41:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 16 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7effb406f400 msgr2=0x7effb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7effb406f400 0x7effb4071910 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7effcc0b2000 tx=0x7effcc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd00ce730 msgr2=0x7effd015d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd00ce730 0x7effd015d8d0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7effc800c730 tx=0x7effc800cbf0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 shutdown_connections 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7effb406f400 0x7effb4071910 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effd00ce730 0x7effd015d8d0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effd00b0ed0 0x7effd0154d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 --2- 172.21.15.80:0/2701420735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effd006e490 0x7effd01f5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 >> 172.21.15.80:0/2701420735 conn(0x7effd006d080 msgr2=0x7effd00cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 shutdown_connections 2024-04-25T13:41:54.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54.398+0000 7effd72b5640 1 -- 172.21.15.80:0/2701420735 wait complete. 2024-04-25T13:41:54.419 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json osdmap barrier 35 2024-04-25T13:41:54.595 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-04-25T13:41:54.596 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:41:54.596 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:41:54.596 DEBUG:teuthology.orchestra.run.smithi080:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-25T13:41:54.596 DEBUG:teuthology.orchestra.run.smithi080:> os.fchmod(fd, 0O666) 2024-04-25T13:41:54.596 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(fd) 2024-04-25T13:41:54.596 DEBUG:teuthology.orchestra.run.smithi080:> os.close(fd) 2024-04-25T13:41:54.596 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:41:54.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:41:54.680 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:41:54.681 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:41:54.682 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:54.682 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:41:54.682 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:41:54.803 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:54.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:54.815 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:54.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:54.815 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.28248.asok status 2024-04-25T13:41:54.997 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:41:54.997 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:41:54.997 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "28248", 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 2, 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5430, 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:41:54.998 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 5430 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:54.999 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.5430 192.168.0.1:0/1283825421", 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1283825421", 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 3, 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 16, 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 35, 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 35, 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:41:55.000 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:41:55.013 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:41:55.013 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:41:55.013 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:41:55.013 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:41:55.013 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:41:55.014 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:41:55.015 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:41:55.092 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:55.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:41:55.104 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.28248.asok 2024-04-25T13:41:55.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:55.105 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.28248.asok status 2024-04-25T13:41:55.283 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:41:55.283 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:41:55.283 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:41:55.283 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "28248", 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 2, 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5430, 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 5430 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:55.284 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1283825421 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.5430 192.168.0.1:0/1283825421", 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1283825421", 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 3, 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 16, 2024-04-25T13:41:55.285 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 35, 2024-04-25T13:41:55.286 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 35, 2024-04-25T13:41:55.286 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:41:55.286 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:41:55.286 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:41:55.301 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:41:55.302 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:41:55.302 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:41:55.349 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:41:55.413 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:41:55.413 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:41:55.413 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:55.413 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:55.418 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:55.418 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:55.464 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4299c3d640 1 Processor -- start 2024-04-25T13:41:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4299c3d640 1 -- start start 2024-04-25T13:41:55.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4299c3d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d2e80 0x7f42941607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:55.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4299c3d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42940d59b0 0x7f42940d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:55.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4299c3d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42940d6300 con 0x7f42940d4330 2024-04-25T13:41:55.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4299c3d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4294161060 con 0x7f42940d59b0 2024-04-25T13:41:55.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4299c3d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4294161340 con 0x7f42940d2e80 2024-04-25T13:41:55.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4292ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42940d59b0 0x7f42940d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:55.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4292ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f42940d59b0 0x7f42940d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58524/0 (socket says 172.21.15.80:58524) 2024-04-25T13:41:55.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4292ffd640 1 -- 172.21.15.80:0/3006268614 learned_addr learned my addr 172.21.15.80:0/3006268614 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:55.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4293fff640 1 --1- 172.21.15.80:0/3006268614 >> v1:172.21.15.80:6789/0 conn(0x7f42940d4330 0x7f42940d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49670/0 (socket says 172.21.15.80:49670) 2024-04-25T13:41:55.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f42937fe640 1 --2- 172.21.15.80:0/3006268614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d2e80 0x7f42941607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:55.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4292ffd640 1 -- 172.21.15.80:0/3006268614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d2e80 msgr2=0x7f42941607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:55.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4292ffd640 1 --2- 172.21.15.80:0/3006268614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d2e80 0x7f42941607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4292ffd640 1 -- 172.21.15.80:0/3006268614 >> v1:172.21.15.80:6789/0 conn(0x7f42940d4330 legacy=0x7f42940d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:55.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.666+0000 7f4292ffd640 1 -- 172.21.15.80:0/3006268614 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42940d2680 con 0x7f42940d59b0 2024-04-25T13:41:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4292ffd640 1 --2- 172.21.15.80:0/3006268614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d59b0 0x7f42940d5dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f427c009870 tx=0x7f427c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916abe772854541a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f42927fc640 1 -- 172.21.15.80:0/3006268614 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c0b1070 con 0x7f42940d59b0 2024-04-25T13:41:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f42927fc640 1 -- 172.21.15.80:0/3006268614 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f427c0a8050 con 0x7f42940d59b0 2024-04-25T13:41:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f42927fc640 1 -- 172.21.15.80:0/3006268614 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427c0ac860 con 0x7f42940d59b0 2024-04-25T13:41:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4299c3d640 1 -- 172.21.15.80:0/3006268614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d59b0 msgr2=0x7f42940d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4299c3d640 1 --2- 172.21.15.80:0/3006268614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d59b0 0x7f42940d5dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f427c009870 tx=0x7f427c01c4d0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4299c3d640 1 -- 172.21.15.80:0/3006268614 shutdown_connections 2024-04-25T13:41:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4299c3d640 1 --2- 172.21.15.80:0/3006268614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d59b0 0x7f42940d5dc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4299c3d640 1 --2- 172.21.15.80:0/3006268614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d2e80 0x7f42941607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4299c3d640 1 -- 172.21.15.80:0/3006268614 >> 172.21.15.80:0/3006268614 conn(0x7f42940737a0 msgr2=0x7f429406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:55.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4299c3d640 1 -- 172.21.15.80:0/3006268614 shutdown_connections 2024-04-25T13:41:55.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.670+0000 7f4299c3d640 1 -- 172.21.15.80:0/3006268614 wait complete. 2024-04-25T13:41:55.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 Processor -- start 2024-04-25T13:41:55.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 -- start start 2024-04-25T13:41:55.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d2e80 0x7f4294168a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42940d4330 0x7f42941fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d59b0 0x7f429415c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f42940d3410 con 0x7f42940d2e80 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f42940d3290 con 0x7f42940d4330 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f429415df60 con 0x7f42940d59b0 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d2e80 0x7f4294168a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d2e80 0x7f4294168a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58526/0 (socket says 172.21.15.80:58526) 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 -- 172.21.15.80:0/900804991 learned_addr learned my addr 172.21.15.80:0/900804991 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4292ffd640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42940d4330 0x7f42941fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4293fff640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d59b0 0x7f429415c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 -- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d59b0 msgr2=0x7f429415c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d59b0 0x7f429415c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 -- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42940d4330 msgr2=0x7f42941fdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:55.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42940d4330 0x7f42941fdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 -- 172.21.15.80:0/900804991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42941fe280 con 0x7f42940d2e80 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4292ffd640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42940d4330 0x7f42941fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4293fff640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d59b0 0x7f429415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f42937fe640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d2e80 0x7f4294168a20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f428c00d600 tx=0x7f428c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f428bfff640 1 -- 172.21.15.80:0/900804991 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c004240 con 0x7f42940d2e80 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f428bfff640 1 -- 172.21.15.80:0/900804991 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f428c0043e0 con 0x7f42940d2e80 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42941591f0 con 0x7f42940d2e80 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f428bfff640 1 -- 172.21.15.80:0/900804991 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c003840 con 0x7f42940d2e80 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42941597b0 con 0x7f42940d2e80 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.674+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429406a3a0 con 0x7f42940d2e80 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.678+0000 7f428bfff640 1 -- 172.21.15.80:0/900804991 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f428c0039e0 con 0x7f42940d2e80 2024-04-25T13:41:55.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.678+0000 7f428bfff640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f427406f400 0x7f4274071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:55.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.678+0000 7f428bfff640 1 -- 172.21.15.80:0/900804991 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f428c08f580 con 0x7f42940d2e80 2024-04-25T13:41:55.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.678+0000 7f4292ffd640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f427406f400 0x7f4274071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:55.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.678+0000 7f4292ffd640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f427406f400 0x7f4274071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f427c0ae000 tx=0x7f427c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:55.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.682+0000 7f428bfff640 1 -- 172.21.15.80:0/900804991 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f42941fe280 con 0x7f42940d2e80 2024-04-25T13:41:55.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.850+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42940d4110 con 0x7f42940d2e80 2024-04-25T13:41:55.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.850+0000 7f428bfff640 1 -- 172.21.15.80:0/900804991 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7f428c05c470 con 0x7f42940d2e80 2024-04-25T13:41:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:55.854 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":35,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:53.226827+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:55.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f427406f400 msgr2=0x7f4274071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f427406f400 0x7f4274071910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f427c0ae000 tx=0x7f427c008040 comp rx=0 tx=0).stop 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d2e80 msgr2=0x7f4294168a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d2e80 0x7f4294168a20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f428c00d600 tx=0x7f428c00dac0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 shutdown_connections 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f427406f400 0x7f4274071910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f42940d59b0 0x7f429415c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42940d4330 0x7f42941fdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 --2- 172.21.15.80:0/900804991 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f42940d2e80 0x7f4294168a20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 >> 172.21.15.80:0/900804991 conn(0x7f42940737a0 msgr2=0x7f4294162f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 shutdown_connections 2024-04-25T13:41:55.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:55.854+0000 7f4299c3d640 1 -- 172.21.15.80:0/900804991 wait complete. 2024-04-25T13:41:55.876 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:41:30.714412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:41:31.911193+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:55.876 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:56.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.026+0000 7f3a5c525640 1 Processor -- start 2024-04-25T13:41:56.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5c525640 1 -- start start 2024-04-25T13:41:56.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5c525640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3a54071e60 0x7f3a541574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5c525640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54073c40 0x7f3a54160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5c525640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3a541613b0 con 0x7f3a54160fa0 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5c525640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3a540b1350 con 0x7f3a54071e60 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5c525640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3a5406f9a0 con 0x7f3a54073c40 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5a2c1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3a54071e60 0x7f3a541574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5a2c1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3a54071e60 0x7f3a541574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58530/0 (socket says 172.21.15.80:58530) 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5a2c1640 1 -- 172.21.15.80:0/1288979143 learned_addr learned my addr 172.21.15.80:0/1288979143 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a59ac0640 1 --2- 172.21.15.80:0/1288979143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54073c40 0x7f3a54160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5a2c1640 1 -- 172.21.15.80:0/1288979143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54073c40 msgr2=0x7f3a54160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5a2c1640 1 --2- 172.21.15.80:0/1288979143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54073c40 0x7f3a54160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5a2c1640 1 -- 172.21.15.80:0/1288979143 >> v1:172.21.15.87:6789/0 conn(0x7f3a54160fa0 legacy=0x7f3a540b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.030+0000 7f3a5a2c1640 1 -- 172.21.15.80:0/1288979143 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a5406fbf0 con 0x7f3a54071e60 2024-04-25T13:41:56.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5a2c1640 1 --2- 172.21.15.80:0/1288979143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54071e60 0x7f3a541574c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000bb10 tx=0x7f3a5001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0490cf056c097a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a592bf640 1 -- 172.21.15.80:0/1288979143 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500b4070 con 0x7f3a54071e60 2024-04-25T13:41:56.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a592bf640 1 -- 172.21.15.80:0/1288979143 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a500a7af0 con 0x7f3a54071e60 2024-04-25T13:41:56.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a592bf640 1 -- 172.21.15.80:0/1288979143 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a500af440 con 0x7f3a54071e60 2024-04-25T13:41:56.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 -- 172.21.15.80:0/1288979143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54071e60 msgr2=0x7f3a541574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 --2- 172.21.15.80:0/1288979143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54071e60 0x7f3a541574c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3a5000bb10 tx=0x7f3a5001ec30 comp rx=0 tx=0).stop 2024-04-25T13:41:56.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 -- 172.21.15.80:0/1288979143 shutdown_connections 2024-04-25T13:41:56.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 --2- 172.21.15.80:0/1288979143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54073c40 0x7f3a54160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 --2- 172.21.15.80:0/1288979143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54071e60 0x7f3a541574c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 -- 172.21.15.80:0/1288979143 >> 172.21.15.80:0/1288979143 conn(0x7f3a540b4160 msgr2=0x7f3a540d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:56.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 -- 172.21.15.80:0/1288979143 shutdown_connections 2024-04-25T13:41:56.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 -- 172.21.15.80:0/1288979143 wait complete. 2024-04-25T13:41:56.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 Processor -- start 2024-04-25T13:41:56.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.034+0000 7f3a5c525640 1 -- start start 2024-04-25T13:41:56.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a54071e60 0x7f3a540b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54073c40 0x7f3a5414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54160fa0 0x7f3a54166d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54074050 con 0x7f3a54073c40 2024-04-25T13:41:56.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3a54072270 con 0x7f3a54071e60 2024-04-25T13:41:56.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3a540b56c0 con 0x7f3a54160fa0 2024-04-25T13:41:56.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54073c40 0x7f3a5414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54073c40 0x7f3a5414e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58544/0 (socket says 172.21.15.80:58544) 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 -- 172.21.15.80:0/369455959 learned_addr learned my addr 172.21.15.80:0/369455959 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5aac2640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54160fa0 0x7f3a54166d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5a2c1640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a54071e60 0x7f3a540b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 -- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54160fa0 msgr2=0x7f3a54166d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54160fa0 0x7f3a54166d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 -- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a54071e60 msgr2=0x7f3a540b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a54071e60 0x7f3a540b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 -- 172.21.15.80:0/369455959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a540b27c0 con 0x7f3a54073c40 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a59ac0640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54073c40 0x7f3a5414e4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400d620 tx=0x7f3a4400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a4affd640 1 -- 172.21.15.80:0/369455959 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400b620 con 0x7f3a54073c40 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a4affd640 1 -- 172.21.15.80:0/369455959 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a44019070 con 0x7f3a54073c40 2024-04-25T13:41:56.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a54150400 con 0x7f3a54073c40 2024-04-25T13:41:56.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a4affd640 1 -- 172.21.15.80:0/369455959 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a4400b9e0 con 0x7f3a54073c40 2024-04-25T13:41:56.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a54153650 con 0x7f3a54073c40 2024-04-25T13:41:56.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a540b2530 con 0x7f3a54073c40 2024-04-25T13:41:56.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a4affd640 1 -- 172.21.15.80:0/369455959 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3a4400bb80 con 0x7f3a54073c40 2024-04-25T13:41:56.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a4affd640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3a3806f400 0x7f3a38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.038+0000 7f3a4affd640 1 -- 172.21.15.80:0/369455959 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f3a4408f6d0 con 0x7f3a54073c40 2024-04-25T13:41:56.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.042+0000 7f3a5a2c1640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3a3806f400 0x7f3a38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.042+0000 7f3a5a2c1640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3a3806f400 0x7f3a38071910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a500b1000 tx=0x7f3a500a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.046+0000 7f3a4affd640 1 -- 172.21.15.80:0/369455959 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3a4405c650 con 0x7f3a54073c40 2024-04-25T13:41:56.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.202+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3a5406a3a0 con 0x7f3a54073c40 2024-04-25T13:41:56.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.202+0000 7f3a4affd640 1 -- 172.21.15.80:0/369455959 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f3a4405bf50 con 0x7f3a54073c40 2024-04-25T13:41:56.205 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3a3806f400 msgr2=0x7f3a38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3a3806f400 0x7f3a38071910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a500b1000 tx=0x7f3a500a9070 comp rx=0 tx=0).stop 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54073c40 msgr2=0x7f3a5414e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54073c40 0x7f3a5414e4a0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f3a4400d620 tx=0x7f3a4400dae0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 shutdown_connections 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3a3806f400 0x7f3a38071910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3a54160fa0 0x7f3a54166d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3a54073c40 0x7f3a5414e4a0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 --2- 172.21.15.80:0/369455959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a54071e60 0x7f3a540b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 >> 172.21.15.80:0/369455959 conn(0x7f3a540b4160 msgr2=0x7f3a540b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:56.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 shutdown_connections 2024-04-25T13:41:56.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.206+0000 7f3a5c525640 1 -- 172.21.15.80:0/369455959 wait complete. 2024-04-25T13:41:56.226 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:56.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9df1a48640 1 Processor -- start 2024-04-25T13:41:56.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9df1a48640 1 -- start start 2024-04-25T13:41:56.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9df1a48640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9dec071e60 0x7f9dec1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9df1a48640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9dec161170 0x7f9dec0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9df1a48640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9dec06fbf0 con 0x7f9dec073c40 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9df1a48640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9dec0b1350 con 0x7f9dec161170 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9df1a48640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9dec06f9a0 con 0x7f9dec071e60 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9dec161170 0x7f9dec0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9dec161170 0x7f9dec0b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58550/0 (socket says 172.21.15.80:58550) 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9de27fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9dec073c40 0x7f9dec160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49682/0 (socket says 172.21.15.80:49682) 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deb7fe640 1 -- 172.21.15.80:0/3951889222 learned_addr learned my addr 172.21.15.80:0/3951889222 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deaffd640 1 --2- 172.21.15.80:0/3951889222 >> v2:172.21.15.87:3300/0 conn(0x7f9dec071e60 0x7f9dec1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deb7fe640 1 -- 172.21.15.80:0/3951889222 >> v2:172.21.15.87:3300/0 conn(0x7f9dec071e60 msgr2=0x7f9dec1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deb7fe640 1 --2- 172.21.15.80:0/3951889222 >> v2:172.21.15.87:3300/0 conn(0x7f9dec071e60 0x7f9dec1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deb7fe640 1 -- 172.21.15.80:0/3951889222 >> v1:172.21.15.80:6789/0 conn(0x7f9dec073c40 legacy=0x7f9dec160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9dea7fc640 1 -- 172.21.15.80:0/3951889222 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 120053176 0 0) 0x7f9dec06fbf0 con 0x7f9dec073c40 2024-04-25T13:41:56.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deb7fe640 1 -- 172.21.15.80:0/3951889222 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0b0a20 con 0x7f9dec161170 2024-04-25T13:41:56.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.374+0000 7f9deaffd640 1 --2- 172.21.15.80:0/3951889222 >> v2:172.21.15.87:3300/0 conn(0x7f9dec071e60 0x7f9dec1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:56.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9deb7fe640 1 --2- 172.21.15.80:0/3951889222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec161170 0x7f9dec0b4ba0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9dec071630 tx=0x7f9de401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2cf1e7d4c7c1f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9dea7fc640 1 -- 172.21.15.80:0/3951889222 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40a57a0 con 0x7f9dec161170 2024-04-25T13:41:56.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9dea7fc640 1 -- 172.21.15.80:0/3951889222 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de40a5940 con 0x7f9dec161170 2024-04-25T13:41:56.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9dea7fc640 1 -- 172.21.15.80:0/3951889222 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40a5d10 con 0x7f9dec161170 2024-04-25T13:41:56.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 -- 172.21.15.80:0/3951889222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec161170 msgr2=0x7f9dec0b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 --2- 172.21.15.80:0/3951889222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec161170 0x7f9dec0b4ba0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9dec071630 tx=0x7f9de401c9f0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 -- 172.21.15.80:0/3951889222 shutdown_connections 2024-04-25T13:41:56.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 --2- 172.21.15.80:0/3951889222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec161170 0x7f9dec0b4ba0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 --2- 172.21.15.80:0/3951889222 >> v2:172.21.15.87:3300/0 conn(0x7f9dec071e60 0x7f9dec1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 -- 172.21.15.80:0/3951889222 >> 172.21.15.80:0/3951889222 conn(0x7f9dec0b4160 msgr2=0x7f9dec0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:56.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 -- 172.21.15.80:0/3951889222 shutdown_connections 2024-04-25T13:41:56.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 -- 172.21.15.80:0/3951889222 wait complete. 2024-04-25T13:41:56.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 Processor -- start 2024-04-25T13:41:56.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.378+0000 7f9df1a48640 1 -- start start 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9dec071e60 0x7f9dec154320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dec073c40 0x7f9dec13aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec150e80 0x7f9dec13b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec169bb0 con 0x7f9dec150e80 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec074050 con 0x7f9dec073c40 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9dec169d30 con 0x7f9dec071e60 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec150e80 0x7f9dec13b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec150e80 0x7f9dec13b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58566/0 (socket says 172.21.15.80:58566) 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 -- 172.21.15.80:0/2647413043 learned_addr learned my addr 172.21.15.80:0/2647413043 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:56.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9de27fc640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dec073c40 0x7f9dec13aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deaffd640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9dec071e60 0x7f9dec154320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 -- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9dec071e60 msgr2=0x7f9dec154320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9dec071e60 0x7f9dec154320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 -- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dec073c40 msgr2=0x7f9dec13aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dec073c40 0x7f9dec13aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 -- 172.21.15.80:0/2647413043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec0d1000 con 0x7f9dec150e80 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deaffd640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9dec071e60 0x7f9dec154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9deb7fe640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec150e80 0x7f9dec13b7a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9de40af040 tx=0x7f9de4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9de8ff9640 1 -- 172.21.15.80:0/2647413043 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40ba070 con 0x7f9dec150e80 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9de8ff9640 1 -- 172.21.15.80:0/2647413043 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9de40b4640 con 0x7f9dec150e80 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9de8ff9640 1 -- 172.21.15.80:0/2647413043 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de40b4a30 con 0x7f9dec150e80 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec0b1fb0 con 0x7f9dec150e80 2024-04-25T13:41:56.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec0b2570 con 0x7f9dec150e80 2024-04-25T13:41:56.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.382+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dec06a3a0 con 0x7f9dec150e80 2024-04-25T13:41:56.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.386+0000 7f9de8ff9640 1 -- 172.21.15.80:0/2647413043 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9de40b4bd0 con 0x7f9dec150e80 2024-04-25T13:41:56.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.386+0000 7f9de8ff9640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dcc06f400 0x7f9dcc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.386+0000 7f9de8ff9640 1 -- 172.21.15.80:0/2647413043 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9de412bf50 con 0x7f9dec150e80 2024-04-25T13:41:56.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.386+0000 7f9deaffd640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dcc06f400 0x7f9dcc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.386+0000 7f9deaffd640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dcc06f400 0x7f9dcc071910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc008890 tx=0x7f9ddc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.390+0000 7f9de8ff9640 1 -- 172.21.15.80:0/2647413043 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9de40165e0 con 0x7f9dec150e80 2024-04-25T13:41:56.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.550+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9dec13e460 con 0x7f9dec150e80 2024-04-25T13:41:56.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9de8ff9640 1 -- 172.21.15.80:0/2647413043 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f9de40f8f20 con 0x7f9dec150e80 2024-04-25T13:41:56.556 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:56.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dcc06f400 msgr2=0x7f9dcc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dcc06f400 0x7f9dcc071910 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc008890 tx=0x7f9ddc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:56.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec150e80 msgr2=0x7f9dec13b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec150e80 0x7f9dec13b7a0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9de40af040 tx=0x7f9de4019790 comp rx=0 tx=0).stop 2024-04-25T13:41:56.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 shutdown_connections 2024-04-25T13:41:56.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9dcc06f400 0x7f9dcc071910 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9dec150e80 0x7f9dec13b7a0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dec073c40 0x7f9dec13aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 --2- 172.21.15.80:0/2647413043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9dec071e60 0x7f9dec154320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 >> 172.21.15.80:0/2647413043 conn(0x7f9dec0b4160 msgr2=0x7f9dec167230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:56.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 shutdown_connections 2024-04-25T13:41:56.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.554+0000 7f9df1a48640 1 -- 172.21.15.80:0/2647413043 wait complete. 2024-04-25T13:41:56.577 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45c5bad640 1 Processor -- start 2024-04-25T13:41:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45c5bad640 1 -- start start 2024-04-25T13:41:56.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45c5bad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45c0070070 0x7f45c016aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45c5bad640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c006cb60 con 0x7f45c0068590 2024-04-25T13:41:56.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45c5bad640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45c006ce40 con 0x7f45c00b6190 2024-04-25T13:41:56.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45c5bad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f45c006ebd0 con 0x7f45c0070070 2024-04-25T13:41:56.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45beffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f45c0068590 0x7f45c00689a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49696/0 (socket says 172.21.15.80:49696) 2024-04-25T13:41:56.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45beffd640 1 -- 172.21.15.80:0/3368107507 learned_addr learned my addr 172.21.15.80:0/3368107507 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45bffff640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45c0070070 0x7f45c016aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3613243026 0 0) 0x7f45c006cb60 con 0x7f45c0068590 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.714+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f45a8003530 con 0x7f45c0068590 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1067851242 0 0) 0x7f45a8003530 con 0x7f45c0068590 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45c0070070 msgr2=0x7f45c016aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45c0070070 0x7f45c016aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 >> v1:172.21.15.87:6789/0 conn(0x7f45c00b6190 legacy=0x7f45c0149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c006d120 con 0x7f45c0068590 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f45b4001680 con 0x7f45c0068590 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 >> v1:172.21.15.80:6789/0 conn(0x7f45c0068590 legacy=0x7f45c00689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a80039f0 0x7f45a8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45c0070070 0x7f45a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a8004d50 0x7f45a8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f45c006ebd0 con 0x7f45a8004d50 2024-04-25T13:41:56.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f45a800bb00 con 0x7f45a80039f0 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f45a800bd00 con 0x7f45c0070070 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bffff640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a8004d50 0x7f45a8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45beffd640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45c0070070 0x7f45a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bf7fe640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a80039f0 0x7f45a8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bffff640 1 -- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45c0070070 msgr2=0x7f45a8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bffff640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45c0070070 0x7f45a8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bffff640 1 -- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a80039f0 msgr2=0x7f45a8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bffff640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45a80039f0 0x7f45a8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bffff640 1 -- 172.21.15.80:0/3368107507 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45a8003530 con 0x7f45a8004d50 2024-04-25T13:41:56.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bffff640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a8004d50 0x7f45a8005160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009f90 tx=0x7f45b801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f699de6805b113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b80b2070 con 0x7f45a8004d50 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b80a5c70 con 0x7f45a8004d50 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45be7fc640 1 -- 172.21.15.80:0/3368107507 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b8009c10 con 0x7f45a8004d50 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45c5bad640 1 -- 172.21.15.80:0/3368107507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a8004d50 msgr2=0x7f45a8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45c5bad640 1 --2- 172.21.15.80:0/3368107507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45a8004d50 0x7f45a8005160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009f90 tx=0x7f45b801cc60 comp rx=0 tx=0).stop 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45bffff640 1 -- 172.21.15.80:0/3368107507 reap_dead start 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45c5bad640 1 -- 172.21.15.80:0/3368107507 shutdown_connections 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45c5bad640 1 -- 172.21.15.80:0/3368107507 >> 172.21.15.80:0/3368107507 conn(0x7f45c00d2b20 msgr2=0x7f45c00d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45c5bad640 1 -- 172.21.15.80:0/3368107507 shutdown_connections 2024-04-25T13:41:56.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.718+0000 7f45c5bad640 1 -- 172.21.15.80:0/3368107507 wait complete. 2024-04-25T13:41:56.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 Processor -- start 2024-04-25T13:41:56.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 -- start start 2024-04-25T13:41:56.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45a8004d50 0x7f45c0142b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45c0167ac0 0x7f45c0161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45c0143300 0x7f45c0162330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f45c0168ea0 con 0x7f45c0167ac0 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00b65a0 con 0x7f45c0143300 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f45c0169020 con 0x7f45a8004d50 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45c0167ac0 0x7f45c0161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45c0167ac0 0x7f45c0161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58584/0 (socket says 172.21.15.80:58584) 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 -- 172.21.15.80:0/4081579680 learned_addr learned my addr 172.21.15.80:0/4081579680 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45bffff640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45c0143300 0x7f45c0162330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45bf7fe640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45a8004d50 0x7f45c0142b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 -- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45a8004d50 msgr2=0x7f45c0142b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45a8004d50 0x7f45c0142b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 -- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45c0143300 msgr2=0x7f45c0162330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45c0143300 0x7f45c0162330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 -- 172.21.15.80:0/4081579680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c0164f20 con 0x7f45c0167ac0 2024-04-25T13:41:56.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45beffd640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45c0167ac0 0x7f45c0161df0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e7e0 tx=0x7f45b400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45a7fff640 1 -- 172.21.15.80:0/4081579680 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b400ee90 con 0x7f45c0167ac0 2024-04-25T13:41:56.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45a7fff640 1 -- 172.21.15.80:0/4081579680 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45b4018070 con 0x7f45c0167ac0 2024-04-25T13:41:56.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c0165190 con 0x7f45c0167ac0 2024-04-25T13:41:56.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45a7fff640 1 -- 172.21.15.80:0/4081579680 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45b400caf0 con 0x7f45c0167ac0 2024-04-25T13:41:56.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.722+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c0165750 con 0x7f45c0167ac0 2024-04-25T13:41:56.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.726+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c006a3a0 con 0x7f45c0167ac0 2024-04-25T13:41:56.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.726+0000 7f45a7fff640 1 -- 172.21.15.80:0/4081579680 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f45b400cc90 con 0x7f45c0167ac0 2024-04-25T13:41:56.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.726+0000 7f45a7fff640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45ac06f5a0 0x7f45ac071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:56.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.726+0000 7f45a7fff640 1 -- 172.21.15.80:0/4081579680 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f45b408fa60 con 0x7f45c0167ac0 2024-04-25T13:41:56.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.726+0000 7f45bf7fe640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45ac06f5a0 0x7f45ac071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:56.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.726+0000 7f45bf7fe640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45ac06f5a0 0x7f45ac071ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005510 tx=0x7f45b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:56.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.730+0000 7f45a7fff640 1 -- 172.21.15.80:0/4081579680 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f45b405cb50 con 0x7f45c0167ac0 2024-04-25T13:41:56.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.878+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f45c00689a0 con 0x7f45c0167ac0 2024-04-25T13:41:56.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.878+0000 7f45a7fff640 1 -- 172.21.15.80:0/4081579680 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f45b405c450 con 0x7f45c0167ac0 2024-04-25T13:41:56.882 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45ac06f5a0 msgr2=0x7f45ac071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45ac06f5a0 0x7f45ac071ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f45b0005510 tx=0x7f45b0009040 comp rx=0 tx=0).stop 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45c0167ac0 msgr2=0x7f45c0161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45c0167ac0 0x7f45c0161df0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f45b400e7e0 tx=0x7f45b400eca0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 shutdown_connections 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45ac06f5a0 0x7f45ac071ab0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45c0143300 0x7f45c0162330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45c0167ac0 0x7f45c0161df0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 --2- 172.21.15.80:0/4081579680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45a8004d50 0x7f45c0142b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:56.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 >> 172.21.15.80:0/4081579680 conn(0x7f45c00d2b20 msgr2=0x7f45c0143d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:56.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 shutdown_connections 2024-04-25T13:41:56.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:56.882+0000 7f45c5bad640 1 -- 172.21.15.80:0/4081579680 wait complete. 2024-04-25T13:41:56.902 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:57.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c42df640 1 Processor -- start 2024-04-25T13:41:57.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c42df640 1 -- start start 2024-04-25T13:41:57.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c42df640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f77bc0d4330 0x7f77bc0d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c42df640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77bc161060 con 0x7f77bc0d59b0 2024-04-25T13:41:57.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c42df640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77bc161340 con 0x7f77bc0d2e80 2024-04-25T13:41:57.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c42df640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f77bc0d2680 con 0x7f77bc0d4330 2024-04-25T13:41:57.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c187a640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f77bc0d59b0 0x7f77bc0d5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49708/0 (socket says 172.21.15.80:49708) 2024-04-25T13:41:57.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c187a640 1 -- 172.21.15.80:0/159465732 learned_addr learned my addr 172.21.15.80:0/159465732 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c287c640 1 --2- 172.21.15.80:0/159465732 >> v2:172.21.15.80:3300/0 conn(0x7f77bc0d4330 0x7f77bc0d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c1079640 1 -- 172.21.15.80:0/159465732 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1999899011 0 0) 0x7f77bc161060 con 0x7f77bc0d59b0 2024-04-25T13:41:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c1079640 1 -- 172.21.15.80:0/159465732 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77a8003530 con 0x7f77bc0d59b0 2024-04-25T13:41:57.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c287c640 1 -- 172.21.15.80:0/159465732 >> v1:172.21.15.87:6789/0 conn(0x7f77bc0d2e80 legacy=0x7f77bc1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c287c640 1 -- 172.21.15.80:0/159465732 >> v1:172.21.15.80:6789/0 conn(0x7f77bc0d59b0 legacy=0x7f77bc0d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.034+0000 7f77c287c640 1 -- 172.21.15.80:0/159465732 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc0d2960 con 0x7f77bc0d4330 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c287c640 1 --2- 172.21.15.80:0/159465732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0d4740 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009d90 tx=0x7f77b401c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e5038319017437 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c1079640 1 -- 172.21.15.80:0/159465732 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40b2070 con 0x7f77bc0d4330 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c1079640 1 -- 172.21.15.80:0/159465732 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b40a58b0 con 0x7f77bc0d4330 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c1079640 1 -- 172.21.15.80:0/159465732 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b40a5c80 con 0x7f77bc0d4330 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c42df640 1 -- 172.21.15.80:0/159465732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 msgr2=0x7f77bc0d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c42df640 1 --2- 172.21.15.80:0/159465732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0d4740 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f77b4009d90 tx=0x7f77b401c8e0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c42df640 1 -- 172.21.15.80:0/159465732 shutdown_connections 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c42df640 1 --2- 172.21.15.80:0/159465732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0d4740 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c42df640 1 -- 172.21.15.80:0/159465732 >> 172.21.15.80:0/159465732 conn(0x7f77bc0737a0 msgr2=0x7f77bc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:57.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c42df640 1 -- 172.21.15.80:0/159465732 shutdown_connections 2024-04-25T13:41:57.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.038+0000 7f77c42df640 1 -- 172.21.15.80:0/159465732 wait complete. 2024-04-25T13:41:57.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 Processor -- start 2024-04-25T13:41:57.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 -- start start 2024-04-25T13:41:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77bc0d2e80 0x7f77bc1fc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77bc0d59b0 0x7f77bc0cde70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc15e0e0 con 0x7f77bc0d4330 2024-04-25T13:41:57.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc15df60 con 0x7f77bc0d2e80 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f77bc16e950 con 0x7f77bc0d59b0 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58592/0 (socket says 172.21.15.80:58592) 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 -- 172.21.15.80:0/4260731335 learned_addr learned my addr 172.21.15.80:0/4260731335 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c207b640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77bc0d2e80 0x7f77bc1fc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c287c640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77bc0d59b0 0x7f77bc0cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 -- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77bc0d59b0 msgr2=0x7f77bc0cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77bc0d59b0 0x7f77bc0cde70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 -- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77bc0d2e80 msgr2=0x7f77bc1fc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77bc0d2e80 0x7f77bc1fc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 -- 172.21.15.80:0/4260731335 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc1fa020 con 0x7f77bc0d4330 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c287c640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77bc0d59b0 0x7f77bc0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c187a640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0cd930 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c460 tx=0x7f77ac00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77b2ffd640 1 -- 172.21.15.80:0/4260731335 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac017070 con 0x7f77bc0d4330 2024-04-25T13:41:57.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77b2ffd640 1 -- 172.21.15.80:0/4260731335 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77ac005cf0 con 0x7f77bc0d4330 2024-04-25T13:41:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc1fa300 con 0x7f77bc0d4330 2024-04-25T13:41:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77b2ffd640 1 -- 172.21.15.80:0/4260731335 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac007290 con 0x7f77bc0d4330 2024-04-25T13:41:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc0cfaa0 con 0x7f77bc0d4330 2024-04-25T13:41:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.042+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc0c94a0 con 0x7f77bc0d4330 2024-04-25T13:41:57.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.046+0000 7f77b2ffd640 1 -- 172.21.15.80:0/4260731335 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77ac01a070 con 0x7f77bc0d4330 2024-04-25T13:41:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.046+0000 7f77b2ffd640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77a006f430 0x7f77a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.046+0000 7f77b2ffd640 1 -- 172.21.15.80:0/4260731335 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f77ac08e4f0 con 0x7f77bc0d4330 2024-04-25T13:41:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.046+0000 7f77c207b640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77a006f430 0x7f77a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.046+0000 7f77c207b640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77a006f430 0x7f77a0071940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0b8fd0 tx=0x7f77b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.050+0000 7f77b2ffd640 1 -- 172.21.15.80:0/4260731335 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77ac05b3e0 con 0x7f77bc0d4330 2024-04-25T13:41:57.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.206+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f77bc159500 con 0x7f77bc0d4330 2024-04-25T13:41:57.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.206+0000 7f77b2ffd640 1 -- 172.21.15.80:0/4260731335 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f77ac05ace0 con 0x7f77bc0d4330 2024-04-25T13:41:57.210 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:57.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77a006f430 msgr2=0x7f77a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77a006f430 0x7f77a0071940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f77bc0b8fd0 tx=0x7f77b8009040 comp rx=0 tx=0).stop 2024-04-25T13:41:57.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 msgr2=0x7f77bc0cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0cd930 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00c460 tx=0x7f77ac00c920 comp rx=0 tx=0).stop 2024-04-25T13:41:57.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 shutdown_connections 2024-04-25T13:41:57.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77a006f430 0x7f77a0071940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77bc0d59b0 0x7f77bc0cde70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77bc0d4330 0x7f77bc0cd930 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 --2- 172.21.15.80:0/4260731335 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77bc0d2e80 0x7f77bc1fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 >> 172.21.15.80:0/4260731335 conn(0x7f77bc0737a0 msgr2=0x7f77bc0bb300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:57.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 shutdown_connections 2024-04-25T13:41:57.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.210+0000 7f77c42df640 1 -- 172.21.15.80:0/4260731335 wait complete. 2024-04-25T13:41:57.229 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:41:57.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:57.229 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:41:57.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[28248]: fuse finished with error 0 and tester_r 0 2024-04-25T13:41:57.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:41:57.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:57.242 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:41:57.406 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:41:57.406 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:57.406 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:57.413 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:57.413 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:57.460 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:57.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02ae19b640 1 Processor -- start 2024-04-25T13:41:57.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02ae19b640 1 -- start start 2024-04-25T13:41:57.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02ae19b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f02a8161650 0x7f02a8072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02ae19b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a806fcf0 con 0x7f02a8161240 2024-04-25T13:41:57.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02ae19b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02a80b0c60 con 0x7f02a806f5a0 2024-04-25T13:41:57.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02ae19b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f02a8073f10 con 0x7f02a8161650 2024-04-25T13:41:57.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02a7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f02a8161650 0x7f02a8072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02a6ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f02a8161240 0x7f02a80723a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49720/0 (socket says 172.21.15.80:49720) 2024-04-25T13:41:57.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02a6ffd640 1 -- 172.21.15.80:0/1619829881 learned_addr learned my addr 172.21.15.80:0/1619829881 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:57.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02a7fff640 1 -- 172.21.15.80:0/1619829881 >> v1:172.21.15.87:6789/0 conn(0x7f02a806f5a0 legacy=0x7f02a8160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02a7fff640 1 -- 172.21.15.80:0/1619829881 >> v1:172.21.15.80:6789/0 conn(0x7f02a8161240 legacy=0x7f02a80723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02a67fc640 1 -- 172.21.15.80:0/1619829881 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27081697 0 0) 0x7f02a806fcf0 con 0x7f02a8161240 2024-04-25T13:41:57.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.602+0000 7f02a7fff640 1 -- 172.21.15.80:0/1619829881 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a80b0f40 con 0x7f02a8161650 2024-04-25T13:41:57.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02a7fff640 1 --2- 172.21.15.80:0/1619829881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a8161650 0x7f02a8072ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009d90 tx=0x7f02a001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7f2f119ddae292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02a67fc640 1 -- 172.21.15.80:0/1619829881 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a00b2070 con 0x7f02a8161650 2024-04-25T13:41:57.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02a67fc640 1 -- 172.21.15.80:0/1619829881 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02a00a5820 con 0x7f02a8161650 2024-04-25T13:41:57.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02a67fc640 1 -- 172.21.15.80:0/1619829881 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a00a5bf0 con 0x7f02a8161650 2024-04-25T13:41:57.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 -- 172.21.15.80:0/1619829881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a8161650 msgr2=0x7f02a8072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 --2- 172.21.15.80:0/1619829881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a8161650 0x7f02a8072ab0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009d90 tx=0x7f02a001c8e0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 -- 172.21.15.80:0/1619829881 shutdown_connections 2024-04-25T13:41:57.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 --2- 172.21.15.80:0/1619829881 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a8161650 0x7f02a8072ab0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 -- 172.21.15.80:0/1619829881 >> 172.21.15.80:0/1619829881 conn(0x7f02a806d080 msgr2=0x7f02a80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:57.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 -- 172.21.15.80:0/1619829881 shutdown_connections 2024-04-25T13:41:57.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 -- 172.21.15.80:0/1619829881 wait complete. 2024-04-25T13:41:57.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 Processor -- start 2024-04-25T13:41:57.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.606+0000 7f02ae19b640 1 -- start start 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a806f5a0 0x7f02a81455c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02a8161240 0x7f02a814e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02a8161650 0x7f02a814eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f02a806e820 con 0x7f02a806f5a0 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f02a806e6a0 con 0x7f02a8161240 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f02a806e9a0 con 0x7f02a8161650 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a806f5a0 0x7f02a81455c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a806f5a0 0x7f02a81455c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58618/0 (socket says 172.21.15.80:58618) 2024-04-25T13:41:57.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 -- 172.21.15.80:0/3485229999 learned_addr learned my addr 172.21.15.80:0/3485229999 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a6ffd640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02a8161240 0x7f02a814e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a7fff640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02a8161650 0x7f02a814eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 -- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02a8161650 msgr2=0x7f02a814eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02a8161650 0x7f02a814eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 -- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02a8161240 msgr2=0x7f02a814e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02a8161240 0x7f02a814e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 -- 172.21.15.80:0/3485229999 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02a814aa20 con 0x7f02a806f5a0 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a7fff640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02a8161650 0x7f02a814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02a77fe640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a806f5a0 0x7f02a81455c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f029800c3c0 tx=0x7f029800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f029ffff640 1 -- 172.21.15.80:0/3485229999 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298017070 con 0x7f02a806f5a0 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f029ffff640 1 -- 172.21.15.80:0/3485229999 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02980052d0 con 0x7f02a806f5a0 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02a814ad00 con 0x7f02a806f5a0 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f029ffff640 1 -- 172.21.15.80:0/3485229999 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0298003820 con 0x7f02a806f5a0 2024-04-25T13:41:57.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02a814b2c0 con 0x7f02a806f5a0 2024-04-25T13:41:57.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02a806a3a0 con 0x7f02a806f5a0 2024-04-25T13:41:57.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f029ffff640 1 -- 172.21.15.80:0/3485229999 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f029801a070 con 0x7f02a806f5a0 2024-04-25T13:41:57.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f029ffff640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f028c06f400 0x7f028c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.610+0000 7f029ffff640 1 -- 172.21.15.80:0/3485229999 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f029808e580 con 0x7f02a806f5a0 2024-04-25T13:41:57.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.614+0000 7f02a6ffd640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f028c06f400 0x7f028c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.614+0000 7f02a6ffd640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f028c06f400 0x7f028c071910 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0290000f70 tx=0x7f0290007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.618+0000 7f029ffff640 1 -- 172.21.15.80:0/3485229999 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0298005480 con 0x7f02a806f5a0 2024-04-25T13:41:57.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.778+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f02a80739c0 con 0x7f02a806f5a0 2024-04-25T13:41:57.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f029ffff640 1 -- 172.21.15.80:0/3485229999 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7f029805b500 con 0x7f02a806f5a0 2024-04-25T13:41:57.784 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:57.784 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":35,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:53.226827+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f028c06f400 msgr2=0x7f028c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f028c06f400 0x7f028c071910 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0290000f70 tx=0x7f0290007d30 comp rx=0 tx=0).stop 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a806f5a0 msgr2=0x7f02a81455c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a806f5a0 0x7f02a81455c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f029800c3c0 tx=0x7f029800c880 comp rx=0 tx=0).stop 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 shutdown_connections 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f028c06f400 0x7f028c071910 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02a8161650 0x7f02a814eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02a8161240 0x7f02a814e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 --2- 172.21.15.80:0/3485229999 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02a806f5a0 0x7f02a81455c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 >> 172.21.15.80:0/3485229999 conn(0x7f02a806d080 msgr2=0x7f02a8153c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:57.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 shutdown_connections 2024-04-25T13:41:57.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.782+0000 7f02ae19b640 1 -- 172.21.15.80:0/3485229999 wait complete. 2024-04-25T13:41:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:41:30.714412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:41:31.911193+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:41:57.805 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:41:57.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fded0d5c640 1 Processor -- start 2024-04-25T13:41:57.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fded0d5c640 1 -- start start 2024-04-25T13:41:57.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fded0d5c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdecc0d3ed0 0x7fdecc0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fded0d5c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdecc0b92a0 con 0x7fdecc0d2ad0 2024-04-25T13:41:57.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fded0d5c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdecc0b84f0 con 0x7fdecc0ba770 2024-04-25T13:41:57.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fded0d5c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdecc0d4c10 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fdec9d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fdecc0d2ad0 0x7fdecc0b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49730/0 (socket says 172.21.15.80:49730) 2024-04-25T13:41:57.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fdec9d9b640 1 -- 172.21.15.80:0/790600130 learned_addr learned my addr 172.21.15.80:0/790600130 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:57.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fdeca59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdecc0d3ed0 0x7fdecc0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fdec959a640 1 -- 172.21.15.80:0/790600130 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2813685164 0 0) 0x7fdeb40014a0 con 0x7fdecc0d2ad0 2024-04-25T13:41:57.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fdec959a640 1 -- 172.21.15.80:0/790600130 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdeb8003530 con 0x7fdecc0d2ad0 2024-04-25T13:41:57.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fdeca59c640 1 -- 172.21.15.80:0/790600130 >> v1:172.21.15.87:6789/0 conn(0x7fdecc0ba770 legacy=0x7fdecc0b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fdeca59c640 1 -- 172.21.15.80:0/790600130 >> v1:172.21.15.80:6789/0 conn(0x7fdecc0d2ad0 legacy=0x7fdecc0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.942+0000 7fdeca59c640 1 -- 172.21.15.80:0/790600130 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc0d4ef0 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fdeca59c640 1 --2- 172.21.15.80:0/790600130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0d2590 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0097c0 tx=0x7fdebc01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b35d208280d672 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fdec959a640 1 -- 172.21.15.80:0/790600130 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc0b0070 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fdec959a640 1 -- 172.21.15.80:0/790600130 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdebc004230 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fdec959a640 1 -- 172.21.15.80:0/790600130 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc003620 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fded0d5c640 1 -- 172.21.15.80:0/790600130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 msgr2=0x7fdecc0d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fded0d5c640 1 --2- 172.21.15.80:0/790600130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0d2590 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0097c0 tx=0x7fdebc01c350 comp rx=0 tx=0).stop 2024-04-25T13:41:57.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fded0d5c640 1 -- 172.21.15.80:0/790600130 shutdown_connections 2024-04-25T13:41:57.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fded0d5c640 1 --2- 172.21.15.80:0/790600130 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0d2590 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fded0d5c640 1 -- 172.21.15.80:0/790600130 >> 172.21.15.80:0/790600130 conn(0x7fdecc0737b0 msgr2=0x7fdecc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:57.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fded0d5c640 1 -- 172.21.15.80:0/790600130 shutdown_connections 2024-04-25T13:41:57.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.946+0000 7fded0d5c640 1 -- 172.21.15.80:0/790600130 wait complete. 2024-04-25T13:41:57.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 Processor -- start 2024-04-25T13:41:57.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 -- start start 2024-04-25T13:41:57.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdecc0ba770 0x7fdecc14b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc0d2ad0 0x7fdecc0cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc0ccae0 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc0cc960 con 0x7fdecc0d2ad0 2024-04-25T13:41:57.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdecc160620 con 0x7fdecc0ba770 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58646/0 (socket says 172.21.15.80:58646) 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 -- 172.21.15.80:0/2433562772 learned_addr learned my addr 172.21.15.80:0/2433562772 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdec9d9b640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc0d2ad0 0x7fdecc0cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdeca59c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdecc0ba770 0x7fdecc14b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 -- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdecc0ba770 msgr2=0x7fdecc14b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdecc0ba770 0x7fdecc14b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 -- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc0d2ad0 msgr2=0x7fdecc0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc0d2ad0 0x7fdecc0cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 -- 172.21.15.80:0/2433562772 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc14cf80 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdec9d9b640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc0d2ad0 0x7fdecc0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdecad9d640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0cdb90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c370 tx=0x7fdec400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdec2ffd640 1 -- 172.21.15.80:0/2433562772 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4005220 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdec2ffd640 1 -- 172.21.15.80:0/2433562772 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdec40053c0 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc1541f0 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fdec2ffd640 1 -- 172.21.15.80:0/2433562772 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec4007270 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc159d30 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.950+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdecc159f90 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.954+0000 7fdec2ffd640 1 -- 172.21.15.80:0/2433562772 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdec4019020 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.954+0000 7fdec2ffd640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdeac06f430 0x7fdeac071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:57.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.954+0000 7fdec2ffd640 1 -- 172.21.15.80:0/2433562772 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdec408e540 con 0x7fdecc0d3ed0 2024-04-25T13:41:57.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.954+0000 7fdeca59c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdeac06f430 0x7fdeac071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:57.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.954+0000 7fdeca59c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdeac06f430 0x7fdeac071940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0ad000 tx=0x7fdebc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:57.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:57.958+0000 7fdec2ffd640 1 -- 172.21.15.80:0/2433562772 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdec405b4c0 con 0x7fdecc0d3ed0 2024-04-25T13:41:58.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdecc06a3a0 con 0x7fdecc0d3ed0 2024-04-25T13:41:58.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fdec2ffd640 1 -- 172.21.15.80:0/2433562772 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7fdec405adc0 con 0x7fdecc0d3ed0 2024-04-25T13:41:58.121 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:41:58.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdeac06f430 msgr2=0x7fdeac071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdeac06f430 0x7fdeac071940 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0ad000 tx=0x7fdebc008040 comp rx=0 tx=0).stop 2024-04-25T13:41:58.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 msgr2=0x7fdecc0cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0cdb90 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fdec400c370 tx=0x7fdec400c830 comp rx=0 tx=0).stop 2024-04-25T13:41:58.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 shutdown_connections 2024-04-25T13:41:58.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdeac06f430 0x7fdeac071940 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdecc0d3ed0 0x7fdecc0cdb90 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdecc0d2ad0 0x7fdecc0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 --2- 172.21.15.80:0/2433562772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdecc0ba770 0x7fdecc14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 >> 172.21.15.80:0/2433562772 conn(0x7fdecc0737b0 msgr2=0x7fdecc0c9780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:58.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 shutdown_connections 2024-04-25T13:41:58.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.118+0000 7fded0d5c640 1 -- 172.21.15.80:0/2433562772 wait complete. 2024-04-25T13:41:58.142 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:41:58.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8287804640 1 Processor -- start 2024-04-25T13:41:58.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8287804640 1 -- start start 2024-04-25T13:41:58.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8287804640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f82800b5330 0x7f82800d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8287804640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82800d0ff0 0x7f828006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8287804640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82800b5740 con 0x7f828006f2f0 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8287804640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f828006f700 con 0x7f82800b5330 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8287804640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82801629b0 con 0x7f82800d0ff0 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8285da1640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f828006f2f0 0x7f8280161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49740/0 (socket says 172.21.15.80:49740) 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8285da1640 1 -- 172.21.15.80:0/4142311279 learned_addr learned my addr 172.21.15.80:0/4142311279 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f82855a0640 1 --2- 172.21.15.80:0/4142311279 >> v2:172.21.15.80:3300/0 conn(0x7f82800b5330 0x7f82800d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8273fff640 1 -- 172.21.15.80:0/4142311279 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1612776394 0 0) 0x7f82800b5740 con 0x7f828006f2f0 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8284d9f640 1 --2- 172.21.15.80:0/4142311279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82800d0ff0 0x7f828006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f8273fff640 1 -- 172.21.15.80:0/4142311279 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f826c003530 con 0x7f828006f2f0 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f82855a0640 1 -- 172.21.15.80:0/4142311279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82800d0ff0 msgr2=0x7f828006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f82855a0640 1 --2- 172.21.15.80:0/4142311279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82800d0ff0 0x7f828006edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f82855a0640 1 -- 172.21.15.80:0/4142311279 >> v1:172.21.15.80:6789/0 conn(0x7f828006f2f0 legacy=0x7f8280161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.294+0000 7f82855a0640 1 -- 172.21.15.80:0/4142311279 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828006dd10 con 0x7f82800b5330 2024-04-25T13:41:58.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f82855a0640 1 --2- 172.21.15.80:0/4142311279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f82800d0ab0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f82740097c0 tx=0x7f827401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6599e3b91e3fe87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8273fff640 1 -- 172.21.15.80:0/4142311279 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740b0070 con 0x7f82800b5330 2024-04-25T13:41:58.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8273fff640 1 -- 172.21.15.80:0/4142311279 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82740ab600 con 0x7f82800b5330 2024-04-25T13:41:58.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8273fff640 1 -- 172.21.15.80:0/4142311279 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82740ab9d0 con 0x7f82800b5330 2024-04-25T13:41:58.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8287804640 1 -- 172.21.15.80:0/4142311279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 msgr2=0x7f82800d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8287804640 1 --2- 172.21.15.80:0/4142311279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f82800d0ab0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f82740097c0 tx=0x7f827401c410 comp rx=0 tx=0).stop 2024-04-25T13:41:58.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8287804640 1 -- 172.21.15.80:0/4142311279 shutdown_connections 2024-04-25T13:41:58.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8287804640 1 --2- 172.21.15.80:0/4142311279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82800d0ff0 0x7f828006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8287804640 1 --2- 172.21.15.80:0/4142311279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f82800d0ab0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8287804640 1 -- 172.21.15.80:0/4142311279 >> 172.21.15.80:0/4142311279 conn(0x7f82800b4160 msgr2=0x7f82800d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:58.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8287804640 1 -- 172.21.15.80:0/4142311279 shutdown_connections 2024-04-25T13:41:58.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.298+0000 7f8287804640 1 -- 172.21.15.80:0/4142311279 wait complete. 2024-04-25T13:41:58.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 Processor -- start 2024-04-25T13:41:58.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 -- start start 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f828006f2f0 0x7f828014a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f8280157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82800d0ff0 0x7f8280160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f82800b0cd0 con 0x7f82800b5330 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f828006e890 con 0x7f82800d0ff0 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82800b0e50 con 0x7f828006f2f0 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f8280157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f8280157020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58668/0 (socket says 172.21.15.80:58668) 2024-04-25T13:41:58.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 -- 172.21.15.80:0/1146007472 learned_addr learned my addr 172.21.15.80:0/1146007472 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f82855a0640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f828006f2f0 0x7f828014a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8285da1640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82800d0ff0 0x7f8280160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 -- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f828006f2f0 msgr2=0x7f828014a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f828006f2f0 0x7f828014a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 -- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82800d0ff0 msgr2=0x7f8280160710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82800d0ff0 0x7f8280160710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 -- 172.21.15.80:0/1146007472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8280157ac0 con 0x7f82800b5330 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8285da1640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82800d0ff0 0x7f8280160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8284d9f640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f8280157020 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d620 tx=0x7f827c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8271ffb640 1 -- 172.21.15.80:0/1146007472 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c00b620 con 0x7f82800b5330 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8271ffb640 1 -- 172.21.15.80:0/1146007472 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f827c019070 con 0x7f82800b5330 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8280157da0 con 0x7f82800b5330 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8271ffb640 1 -- 172.21.15.80:0/1146007472 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827c00b9e0 con 0x7f82800b5330 2024-04-25T13:41:58.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8280135400 con 0x7f82800b5330 2024-04-25T13:41:58.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.302+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f828006a3a0 con 0x7f82800b5330 2024-04-25T13:41:58.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.306+0000 7f8271ffb640 1 -- 172.21.15.80:0/1146007472 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f827c00bb80 con 0x7f82800b5330 2024-04-25T13:41:58.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.306+0000 7f8271ffb640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f826406f400 0x7f8264071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.306+0000 7f8271ffb640 1 -- 172.21.15.80:0/1146007472 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f827c090190 con 0x7f82800b5330 2024-04-25T13:41:58.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.306+0000 7f82855a0640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f826406f400 0x7f8264071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.306+0000 7f82855a0640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f826406f400 0x7f8264071910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82740ad000 tx=0x7f8274008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.310+0000 7f8271ffb640 1 -- 172.21.15.80:0/1146007472 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f827c05d180 con 0x7f82800b5330 2024-04-25T13:41:58.400 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:41:58.402 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:41:58.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.470+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f82800cea20 con 0x7f82800b5330 2024-04-25T13:41:58.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.470+0000 7f8271ffb640 1 -- 172.21.15.80:0/1146007472 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f827c05ca80 con 0x7f82800b5330 2024-04-25T13:41:58.474 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:41:58.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f826406f400 msgr2=0x7f8264071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f826406f400 0x7f8264071910 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f82740ad000 tx=0x7f8274008040 comp rx=0 tx=0).stop 2024-04-25T13:41:58.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 msgr2=0x7f8280157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f8280157020 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f827c00d620 tx=0x7f827c00dae0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 shutdown_connections 2024-04-25T13:41:58.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f826406f400 0x7f8264071910 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82800d0ff0 0x7f8280160710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82800b5330 0x7f8280157020 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 --2- 172.21.15.80:0/1146007472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f828006f2f0 0x7f828014a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 >> 172.21.15.80:0/1146007472 conn(0x7f82800b4160 msgr2=0x7f82800b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:58.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 shutdown_connections 2024-04-25T13:41:58.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.474+0000 7f8287804640 1 -- 172.21.15.80:0/1146007472 wait complete. 2024-04-25T13:41:58.496 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:41:58.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38c4c81640 1 Processor -- start 2024-04-25T13:41:58.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38c4c81640 1 -- start start 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38c4c81640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 0x7f38c00d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38c4c81640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38c00d2ad0 0x7f38c00b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38c4c81640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38c00b92a0 con 0x7f38c00b7c10 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38c4c81640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f38c00d4c10 con 0x7f38c00d2ad0 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38c4c81640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f38c00d4ef0 con 0x7f38c00d3ed0 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bed9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f38c00b7c10 0x7f38c00b9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49744/0 (socket says 172.21.15.80:49744) 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bed9d640 1 -- 172.21.15.80:0/2545609834 learned_addr learned my addr 172.21.15.80:0/2545609834 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bdd9b640 1 --2- 172.21.15.80:0/2545609834 >> v2:172.21.15.80:3300/0 conn(0x7f38c00d2ad0 0x7f38c00b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38be59c640 1 --2- 172.21.15.80:0/2545609834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 0x7f38c00d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bd59a640 1 -- 172.21.15.80:0/2545609834 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2639339337 0 0) 0x7f38b80014a0 con 0x7f38c00b7c10 2024-04-25T13:41:58.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bd59a640 1 -- 172.21.15.80:0/2545609834 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f38a8003530 con 0x7f38c00b7c10 2024-04-25T13:41:58.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bdd9b640 1 -- 172.21.15.80:0/2545609834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 msgr2=0x7f38c00d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:58.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bdd9b640 1 --2- 172.21.15.80:0/2545609834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 0x7f38c00d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bdd9b640 1 -- 172.21.15.80:0/2545609834 >> v1:172.21.15.80:6789/0 conn(0x7f38c00b7c10 legacy=0x7f38c00b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.650+0000 7f38bdd9b640 1 -- 172.21.15.80:0/2545609834 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c00d51d0 con 0x7f38c00d2ad0 2024-04-25T13:41:58.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38bdd9b640 1 --2- 172.21.15.80:0/2545609834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00d2ad0 0x7f38c00b76d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f38b00097c0 tx=0x7f38b001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b20e3f20c5b990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38bd59a640 1 -- 172.21.15.80:0/2545609834 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00ab600 con 0x7f38c00d2ad0 2024-04-25T13:41:58.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38bd59a640 1 -- 172.21.15.80:0/2545609834 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b00b0070 con 0x7f38c00d2ad0 2024-04-25T13:41:58.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38bd59a640 1 -- 172.21.15.80:0/2545609834 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b00ab9c0 con 0x7f38c00d2ad0 2024-04-25T13:41:58.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38c4c81640 1 -- 172.21.15.80:0/2545609834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00d2ad0 msgr2=0x7f38c00b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38c4c81640 1 --2- 172.21.15.80:0/2545609834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00d2ad0 0x7f38c00b76d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f38b00097c0 tx=0x7f38b001c3e0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38c4c81640 1 -- 172.21.15.80:0/2545609834 shutdown_connections 2024-04-25T13:41:58.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38c4c81640 1 --2- 172.21.15.80:0/2545609834 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00d2ad0 0x7f38c00b76d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38c4c81640 1 --2- 172.21.15.80:0/2545609834 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 0x7f38c00d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38c4c81640 1 -- 172.21.15.80:0/2545609834 >> 172.21.15.80:0/2545609834 conn(0x7f38c00737b0 msgr2=0x7f38c006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:58.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38c4c81640 1 -- 172.21.15.80:0/2545609834 shutdown_connections 2024-04-25T13:41:58.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.654+0000 7f38c4c81640 1 -- 172.21.15.80:0/2545609834 wait complete. 2024-04-25T13:41:58.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 Processor -- start 2024-04-25T13:41:58.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 -- start start 2024-04-25T13:41:58.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00b7c10 0x7f38c0154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38c00d2ad0 0x7f38c00c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 0x7f38c014bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f38c00ccae0 con 0x7f38c00b7c10 2024-04-25T13:41:58.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f38c00cc960 con 0x7f38c00d2ad0 2024-04-25T13:41:58.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f38c0160620 con 0x7f38c00d3ed0 2024-04-25T13:41:58.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00b7c10 0x7f38c0154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00b7c10 0x7f38c0154080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58680/0 (socket says 172.21.15.80:58680) 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 -- 172.21.15.80:0/569415775 learned_addr learned my addr 172.21.15.80:0/569415775 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 -- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 msgr2=0x7f38c014bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 0x7f38c014bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 -- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38c00d2ad0 msgr2=0x7f38c00c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38c00d2ad0 0x7f38c00c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 -- 172.21.15.80:0/569415775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c01545c0 con 0x7f38c00b7c10 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38be59c640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00b7c10 0x7f38c0154080 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f38b400e8a0 tx=0x7f38b400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38aeffd640 1 -- 172.21.15.80:0/569415775 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b4017070 con 0x7f38c00b7c10 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38aeffd640 1 -- 172.21.15.80:0/569415775 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38b400c940 con 0x7f38c00b7c10 2024-04-25T13:41:58.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38aeffd640 1 -- 172.21.15.80:0/569415775 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b400ccf0 con 0x7f38c00b7c10 2024-04-25T13:41:58.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c0151ce0 con 0x7f38c00b7c10 2024-04-25T13:41:58.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.658+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c01522a0 con 0x7f38c00b7c10 2024-04-25T13:41:58.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.662+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c006a3a0 con 0x7f38c00b7c10 2024-04-25T13:41:58.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.662+0000 7f38aeffd640 1 -- 172.21.15.80:0/569415775 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f38b401a070 con 0x7f38c00b7c10 2024-04-25T13:41:58.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.662+0000 7f38aeffd640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f38a006f400 0x7f38a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.662+0000 7f38aeffd640 1 -- 172.21.15.80:0/569415775 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f38b408f210 con 0x7f38c00b7c10 2024-04-25T13:41:58.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.662+0000 7f38bdd9b640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f38a006f400 0x7f38a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.662+0000 7f38bdd9b640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f38a006f400 0x7f38a0071910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f38b00af000 tx=0x7f38b0004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.666+0000 7f38aeffd640 1 -- 172.21.15.80:0/569415775 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38b405c0f0 con 0x7f38c00b7c10 2024-04-25T13:41:58.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.813+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f38c00cc6c0 con 0x7f38c00b7c10 2024-04-25T13:41:58.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.813+0000 7f38aeffd640 1 -- 172.21.15.80:0/569415775 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7f38b405b9f0 con 0x7f38c00b7c10 2024-04-25T13:41:58.818 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:41:58.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f38a006f400 msgr2=0x7f38a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f38a006f400 0x7f38a0071910 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f38b00af000 tx=0x7f38b0004740 comp rx=0 tx=0).stop 2024-04-25T13:41:58.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00b7c10 msgr2=0x7f38c0154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00b7c10 0x7f38c0154080 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f38b400e8a0 tx=0x7f38b400eca0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 shutdown_connections 2024-04-25T13:41:58.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f38a006f400 0x7f38a0071910 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38c00d3ed0 0x7f38c014bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38c00d2ad0 0x7f38c00c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 --2- 172.21.15.80:0/569415775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38c00b7c10 0x7f38c0154080 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 >> 172.21.15.80:0/569415775 conn(0x7f38c00737b0 msgr2=0x7f38c00cf5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:58.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 shutdown_connections 2024-04-25T13:41:58.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.817+0000 7f38c4c81640 1 -- 172.21.15.80:0/569415775 wait complete. 2024-04-25T13:41:58.838 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:41:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e7bec640 1 Processor -- start 2024-04-25T13:41:58.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e7bec640 1 -- start start 2024-04-25T13:41:58.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e7bec640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e00d06e0 0x7f02e006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e7bec640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f02e006e9e0 0x7f02e006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e7bec640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02e00d0af0 con 0x7f02e0161f90 2024-04-25T13:41:58.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e7bec640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f02e00d0d70 con 0x7f02e006e9e0 2024-04-25T13:41:58.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e7bec640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f02e00d1050 con 0x7f02e00d06e0 2024-04-25T13:41:58.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e6189640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f02e006e9e0 0x7f02e006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e6189640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f02e006e9e0 0x7f02e006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58690/0 (socket says 172.21.15.80:58690) 2024-04-25T13:41:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e6189640 1 -- 172.21.15.80:0/3836391580 learned_addr learned my addr 172.21.15.80:0/3836391580 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e5187640 1 --2- 172.21.15.80:0/3836391580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e00d06e0 0x7f02e006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e6189640 1 -- 172.21.15.80:0/3836391580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e00d06e0 msgr2=0x7f02e006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e6189640 1 --2- 172.21.15.80:0/3836391580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e00d06e0 0x7f02e006e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e6189640 1 -- 172.21.15.80:0/3836391580 >> v1:172.21.15.80:6789/0 conn(0x7f02e0161f90 legacy=0x7f02e00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e4986640 1 -- 172.21.15.80:0/3836391580 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1413542390 0 0) 0x7f02d0001120 con 0x7f02e0161f90 2024-04-25T13:41:58.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.969+0000 7f02e6189640 1 -- 172.21.15.80:0/3836391580 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e006dca0 con 0x7f02e006e9e0 2024-04-25T13:41:58.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e6189640 1 --2- 172.21.15.80:0/3836391580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e006e9e0 0x7f02e006edf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009e50 tx=0x7f02d801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4473bd2a8342ba99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e4986640 1 -- 172.21.15.80:0/3836391580 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d80a5740 con 0x7f02e006e9e0 2024-04-25T13:41:58.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e4986640 1 -- 172.21.15.80:0/3836391580 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02d80a58e0 con 0x7f02e006e9e0 2024-04-25T13:41:58.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e4986640 1 -- 172.21.15.80:0/3836391580 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02d80a5cb0 con 0x7f02e006e9e0 2024-04-25T13:41:58.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e7bec640 1 -- 172.21.15.80:0/3836391580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e006e9e0 msgr2=0x7f02e006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e7bec640 1 --2- 172.21.15.80:0/3836391580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e006e9e0 0x7f02e006edf0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009e50 tx=0x7f02d801c970 comp rx=0 tx=0).stop 2024-04-25T13:41:58.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e7bec640 1 -- 172.21.15.80:0/3836391580 shutdown_connections 2024-04-25T13:41:58.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e7bec640 1 --2- 172.21.15.80:0/3836391580 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e006e9e0 0x7f02e006edf0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e7bec640 1 --2- 172.21.15.80:0/3836391580 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e00d06e0 0x7f02e006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e7bec640 1 -- 172.21.15.80:0/3836391580 >> 172.21.15.80:0/3836391580 conn(0x7f02e006d080 msgr2=0x7f02e00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:58.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e7bec640 1 -- 172.21.15.80:0/3836391580 shutdown_connections 2024-04-25T13:41:58.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.973+0000 7f02e7bec640 1 -- 172.21.15.80:0/3836391580 wait complete. 2024-04-25T13:41:58.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 Processor -- start 2024-04-25T13:41:58.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 -- start start 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02e006e9e0 0x7f02e01685d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e00d06e0 0x7f02e00ae870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e0161f90 0x7f02e0140420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f02e01447a0 con 0x7f02e00d06e0 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f02e0144620 con 0x7f02e006e9e0 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f02e016b510 con 0x7f02e0161f90 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e00d06e0 0x7f02e00ae870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e00d06e0 0x7f02e00ae870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58706/0 (socket says 172.21.15.80:58706) 2024-04-25T13:41:58.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 -- 172.21.15.80:0/1728614232 learned_addr learned my addr 172.21.15.80:0/1728614232 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 -- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e0161f90 msgr2=0x7f02e0140420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e6189640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e0161f90 0x7f02e0140420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5988640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02e006e9e0 0x7f02e01685d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e0161f90 0x7f02e0140420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 -- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02e006e9e0 msgr2=0x7f02e01685d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02e006e9e0 0x7f02e01685d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 -- 172.21.15.80:0/1728614232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e0168b10 con 0x7f02e00d06e0 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e5187640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e00d06e0 0x7f02e00ae870 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00d620 tx=0x7f02dc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02d67fc640 1 -- 172.21.15.80:0/1728614232 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00b620 con 0x7f02e00d06e0 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02d67fc640 1 -- 172.21.15.80:0/1728614232 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02dc019070 con 0x7f02e00d06e0 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e0146e30 con 0x7f02e00d06e0 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02d67fc640 1 -- 172.21.15.80:0/1728614232 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02dc00b9e0 con 0x7f02e00d06e0 2024-04-25T13:41:58.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e01473f0 con 0x7f02e00d06e0 2024-04-25T13:41:58.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.977+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e006a3a0 con 0x7f02e00d06e0 2024-04-25T13:41:58.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.981+0000 7f02d67fc640 1 -- 172.21.15.80:0/1728614232 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f02dc00bb80 con 0x7f02e00d06e0 2024-04-25T13:41:58.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.981+0000 7f02d67fc640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f02c406f400 0x7f02c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:58.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.981+0000 7f02d67fc640 1 -- 172.21.15.80:0/1728614232 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f02dc08f6d0 con 0x7f02e00d06e0 2024-04-25T13:41:58.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.981+0000 7f02e5988640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f02c406f400 0x7f02c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:58.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.981+0000 7f02e5988640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f02c406f400 0x7f02c4071910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f02d0000f70 tx=0x7f02d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:58.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:58.985+0000 7f02d67fc640 1 -- 172.21.15.80:0/1728614232 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f02e0168b10 con 0x7f02e00d06e0 2024-04-25T13:41:59.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.149+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f02e00d0220 con 0x7f02e00d06e0 2024-04-25T13:41:59.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.149+0000 7f02d67fc640 1 -- 172.21.15.80:0/1728614232 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f02dc05c650 con 0x7f02e00d06e0 2024-04-25T13:41:59.154 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:41:59.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f02c406f400 msgr2=0x7f02c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f02c406f400 0x7f02c4071910 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f02d0000f70 tx=0x7f02d0009040 comp rx=0 tx=0).stop 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e00d06e0 msgr2=0x7f02e00ae870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e00d06e0 0x7f02e00ae870 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00d620 tx=0x7f02dc00dae0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 shutdown_connections 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f02c406f400 0x7f02c4071910 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f02e0161f90 0x7f02e0140420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f02e00d06e0 0x7f02e00ae870 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 --2- 172.21.15.80:0/1728614232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02e006e9e0 0x7f02e01685d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 >> 172.21.15.80:0/1728614232 conn(0x7f02e006d080 msgr2=0x7f02e00b66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 shutdown_connections 2024-04-25T13:41:59.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.153+0000 7f02e7bec640 1 -- 172.21.15.80:0/1728614232 wait complete. 2024-04-25T13:41:59.174 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:41:59.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:59.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:41:59.187 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24238]: fuse finished with error 0 and tester_r 0 2024-04-25T13:41:59.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:41:59.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:41:59.189 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:41:59.353 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:41:59.354 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:41:59.354 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:59.360 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:41:59.360 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:41:59.360 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:41:59.360 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:41:59.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:41:59.400 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:41:59.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f94a13c3640 1 Processor -- start 2024-04-25T13:41:59.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f94a13c3640 1 -- start start 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f94a13c3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f949c06f5a0 0x7f949c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f94a13c3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f949c0733b0 0x7f949c161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f94a13c3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f949c06fcf0 con 0x7f949c161070 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f94a13c3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f949c1619c0 con 0x7f949c06f5a0 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f94a13c3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f949c073f10 con 0x7f949c0733b0 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f949affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f949c06f5a0 0x7f949c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f949affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f949c06f5a0 0x7f949c160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58722/0 (socket says 172.21.15.80:58722) 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f949affd640 1 -- 172.21.15.80:0/203782308 learned_addr learned my addr 172.21.15.80:0/203782308 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:59.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f9493fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f949c161070 0x7f949c0b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49762/0 (socket says 172.21.15.80:49762) 2024-04-25T13:41:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f949b7fe640 1 --2- 172.21.15.80:0/203782308 >> v2:172.21.15.87:3300/0 conn(0x7f949c0733b0 0x7f949c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f949affd640 1 -- 172.21.15.80:0/203782308 >> v2:172.21.15.87:3300/0 conn(0x7f949c0733b0 msgr2=0x7f949c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f949affd640 1 --2- 172.21.15.80:0/203782308 >> v2:172.21.15.87:3300/0 conn(0x7f949c0733b0 0x7f949c161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f949affd640 1 -- 172.21.15.80:0/203782308 >> v1:172.21.15.80:6789/0 conn(0x7f949c161070 legacy=0x7f949c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.553+0000 7f949affd640 1 -- 172.21.15.80:0/203782308 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949c0b0c60 con 0x7f949c06f5a0 2024-04-25T13:41:59.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f949affd640 1 --2- 172.21.15.80:0/203782308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c160b30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f949c070270 tx=0x7f948c01eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a37a77f6ecf02ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:59.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f949a7fc640 1 -- 172.21.15.80:0/203782308 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c0b4070 con 0x7f949c06f5a0 2024-04-25T13:41:59.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f949a7fc640 1 -- 172.21.15.80:0/203782308 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948c0a7a40 con 0x7f949c06f5a0 2024-04-25T13:41:59.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f949a7fc640 1 -- 172.21.15.80:0/203782308 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c0a7e10 con 0x7f949c06f5a0 2024-04-25T13:41:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 -- 172.21.15.80:0/203782308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 msgr2=0x7f949c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 --2- 172.21.15.80:0/203782308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c160b30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f949c070270 tx=0x7f948c01eb60 comp rx=0 tx=0).stop 2024-04-25T13:41:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 -- 172.21.15.80:0/203782308 shutdown_connections 2024-04-25T13:41:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 --2- 172.21.15.80:0/203782308 >> v2:172.21.15.87:3300/0 conn(0x7f949c0733b0 0x7f949c161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 --2- 172.21.15.80:0/203782308 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c160b30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 -- 172.21.15.80:0/203782308 >> 172.21.15.80:0/203782308 conn(0x7f949c06d080 msgr2=0x7f949c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:59.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 -- 172.21.15.80:0/203782308 shutdown_connections 2024-04-25T13:41:59.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 -- 172.21.15.80:0/203782308 wait complete. 2024-04-25T13:41:59.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 Processor -- start 2024-04-25T13:41:59.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.557+0000 7f94a13c3640 1 -- start start 2024-04-25T13:41:59.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c13ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f949c0733b0 0x7f949c13e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f949c161070 0x7f949c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f949c06e610 con 0x7f949c06f5a0 2024-04-25T13:41:59.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f949c06e490 con 0x7f949c0733b0 2024-04-25T13:41:59.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f949c06e790 con 0x7f949c161070 2024-04-25T13:41:59.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c13ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c13ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58726/0 (socket says 172.21.15.80:58726) 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 -- 172.21.15.80:0/1441215603 learned_addr learned my addr 172.21.15.80:0/1441215603 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f9493fff640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f949c0733b0 0x7f949c13e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 -- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f949c161070 msgr2=0x7f949c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949b7fe640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f949c161070 0x7f949c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f949c161070 0x7f949c1524b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 -- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f949c0733b0 msgr2=0x7f949c13e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f949c0733b0 0x7f949c13e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 -- 172.21.15.80:0/1441215603 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949c1603e0 con 0x7f949c06f5a0 2024-04-25T13:41:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f9493fff640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f949c0733b0 0x7f949c13e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:41:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f949affd640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c13ba30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f948c01b770 tx=0x7f948c01b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94937fe640 1 -- 172.21.15.80:0/1441215603 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c0b4040 con 0x7f949c06f5a0 2024-04-25T13:41:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94937fe640 1 -- 172.21.15.80:0/1441215603 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f948c0bf070 con 0x7f949c06f5a0 2024-04-25T13:41:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94937fe640 1 -- 172.21.15.80:0/1441215603 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c0ba680 con 0x7f949c06f5a0 2024-04-25T13:41:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949c1606c0 con 0x7f949c06f5a0 2024-04-25T13:41:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949c13bfc0 con 0x7f949c06f5a0 2024-04-25T13:41:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.561+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f949c06a3a0 con 0x7f949c06f5a0 2024-04-25T13:41:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.565+0000 7f94937fe640 1 -- 172.21.15.80:0/1441215603 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f948c0ba820 con 0x7f949c06f5a0 2024-04-25T13:41:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.565+0000 7f94937fe640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f947c06f430 0x7f947c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.565+0000 7f94937fe640 1 -- 172.21.15.80:0/1441215603 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f948c12c770 con 0x7f949c06f5a0 2024-04-25T13:41:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.565+0000 7f9493fff640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f947c06f430 0x7f947c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.565+0000 7f9493fff640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f947c06f430 0x7f947c071940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f949c13bf70 tx=0x7f9484009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:59.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.569+0000 7f94937fe640 1 -- 172.21.15.80:0/1441215603 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f948c0187e0 con 0x7f949c06f5a0 2024-04-25T13:41:59.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.749+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f949c0739c0 con 0x7f949c06f5a0 2024-04-25T13:41:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94937fe640 1 -- 172.21.15.80:0/1441215603 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5175 (secure 0 0 0) 0x7f948c0f96e0 con 0x7f949c06f5a0 2024-04-25T13:41:59.756 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:41:59.756 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:41:32.974439+0000","modified":"2024-04-25T13:41:36.017164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/966995060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":966995060},{"type":"v1","addr":"172.21.15.87:6835","nonce":966995060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5190,"qdb_cluster":[5190]},"id":2}]} 2024-04-25T13:41:59.756 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 16 2024-04-25T13:41:59.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f947c06f430 msgr2=0x7f947c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f947c06f430 0x7f947c071940 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f949c13bf70 tx=0x7f9484009040 comp rx=0 tx=0).stop 2024-04-25T13:41:59.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 msgr2=0x7f949c13ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c13ba30 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f948c01b770 tx=0x7f948c01b900 comp rx=0 tx=0).stop 2024-04-25T13:41:59.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 shutdown_connections 2024-04-25T13:41:59.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f947c06f430 0x7f947c071940 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f949c161070 0x7f949c1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f949c0733b0 0x7f949c13e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 --2- 172.21.15.80:0/1441215603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f949c06f5a0 0x7f949c13ba30 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 >> 172.21.15.80:0/1441215603 conn(0x7f949c06d080 msgr2=0x7f949c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:59.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 shutdown_connections 2024-04-25T13:41:59.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.753+0000 7f94a13c3640 1 -- 172.21.15.80:0/1441215603 wait complete. 2024-04-25T13:41:59.777 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:41:59.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.913+0000 7ff4643bb640 1 Processor -- start 2024-04-25T13:41:59.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.913+0000 7ff4643bb640 1 -- start start 2024-04-25T13:41:59.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff4643bb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff45c161f90 0x7ff45c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff4643bb640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45c0d0d70 con 0x7ff45c0d0510 2024-04-25T13:41:59.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff4643bb640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff45c0d1050 con 0x7ff45c15ec40 2024-04-25T13:41:59.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff4643bb640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff45c06dca0 con 0x7ff45c161f90 2024-04-25T13:41:59.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff462157640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff45c161f90 0x7ff45c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff461956640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff45c0d0510 0x7ff45c15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49774/0 (socket says 172.21.15.80:49774) 2024-04-25T13:41:59.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff461956640 1 -- 172.21.15.80:0/2201665006 learned_addr learned my addr 172.21.15.80:0/2201665006 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:41:59.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff462157640 1 -- 172.21.15.80:0/2201665006 >> v1:172.21.15.87:6789/0 conn(0x7ff45c15ec40 legacy=0x7ff45c168cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:41:59.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff462157640 1 -- 172.21.15.80:0/2201665006 >> v1:172.21.15.80:6789/0 conn(0x7ff45c0d0510 legacy=0x7ff45c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.917+0000 7ff462157640 1 -- 172.21.15.80:0/2201665006 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c06df80 con 0x7ff45c161f90 2024-04-25T13:41:59.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff462157640 1 --2- 172.21.15.80:0/2201665006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c161f90 0x7ff45c0cffd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff4580097c0 tx=0x7ff45801c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7308b0bd32f58bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:59.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff461155640 1 -- 172.21.15.80:0/2201665006 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580b0070 con 0x7ff45c161f90 2024-04-25T13:41:59.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff461155640 1 -- 172.21.15.80:0/2201665006 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4580ab600 con 0x7ff45c161f90 2024-04-25T13:41:59.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff461155640 1 -- 172.21.15.80:0/2201665006 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580ab9d0 con 0x7ff45c161f90 2024-04-25T13:41:59.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 -- 172.21.15.80:0/2201665006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c161f90 msgr2=0x7ff45c0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 --2- 172.21.15.80:0/2201665006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c161f90 0x7ff45c0cffd0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff4580097c0 tx=0x7ff45801c2e0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 -- 172.21.15.80:0/2201665006 shutdown_connections 2024-04-25T13:41:59.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 --2- 172.21.15.80:0/2201665006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c161f90 0x7ff45c0cffd0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 -- 172.21.15.80:0/2201665006 >> 172.21.15.80:0/2201665006 conn(0x7ff45c06d080 msgr2=0x7ff45c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:41:59.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 -- 172.21.15.80:0/2201665006 shutdown_connections 2024-04-25T13:41:59.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 -- 172.21.15.80:0/2201665006 wait complete. 2024-04-25T13:41:59.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 Processor -- start 2024-04-25T13:41:59.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.921+0000 7ff4643bb640 1 -- start start 2024-04-25T13:41:59.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c0d0510 0x7ff45c13e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c15ec40 0x7ff45c159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff45c161f90 0x7ff45c15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c06f630 con 0x7ff45c0d0510 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff45c06f4b0 con 0x7ff45c15ec40 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff45c0d0920 con 0x7ff45c161f90 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c0d0510 0x7ff45c13e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c0d0510 0x7ff45c13e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58756/0 (socket says 172.21.15.80:58756) 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 -- 172.21.15.80:0/1625781601 learned_addr learned my addr 172.21.15.80:0/1625781601 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 -- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff45c161f90 msgr2=0x7ff45c15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff45c161f90 0x7ff45c15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 -- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c15ec40 msgr2=0x7ff45c159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c15ec40 0x7ff45c159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 -- 172.21.15.80:0/1625781601 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45c15d290 con 0x7ff45c0d0510 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff462157640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c0d0510 0x7ff45c13e760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff4580ae040 tx=0x7ff45801ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff452ffd640 1 -- 172.21.15.80:0/1625781601 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580b0050 con 0x7ff45c0d0510 2024-04-25T13:41:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff452ffd640 1 -- 172.21.15.80:0/1625781601 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4580bc070 con 0x7ff45c0d0510 2024-04-25T13:41:59.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff452ffd640 1 -- 172.21.15.80:0/1625781601 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4580b5a50 con 0x7ff45c0d0510 2024-04-25T13:41:59.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45c15d570 con 0x7ff45c0d0510 2024-04-25T13:41:59.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45c15db30 con 0x7ff45c0d0510 2024-04-25T13:41:59.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff45c06a3a0 con 0x7ff45c0d0510 2024-04-25T13:41:59.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff452ffd640 1 -- 172.21.15.80:0/1625781601 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4580ab4c0 con 0x7ff45c0d0510 2024-04-25T13:41:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.925+0000 7ff452ffd640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44006f430 0x7ff440071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:41:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.929+0000 7ff452ffd640 1 -- 172.21.15.80:0/1625781601 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7ff458129b50 con 0x7ff45c0d0510 2024-04-25T13:41:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.929+0000 7ff461956640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44006f430 0x7ff440071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:41:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.929+0000 7ff461956640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44006f430 0x7ff440071940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff44c000f70 tx=0x7ff44c007a90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:41:59.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:41:59.933+0000 7ff452ffd640 1 -- 172.21.15.80:0/1625781601 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff458016050 con 0x7ff45c0d0510 2024-04-25T13:42:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.085+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff45c15cff0 con 0x7ff45c0d0510 2024-04-25T13:42:00.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.085+0000 7ff452ffd640 1 -- 172.21.15.80:0/1625781601 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7ff45c15cff0 con 0x7ff45c0d0510 2024-04-25T13:42:00.091 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:00.091 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":35,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:53.226827+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:00.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44006f430 msgr2=0x7ff440071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44006f430 0x7ff440071940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff44c000f70 tx=0x7ff44c007a90 comp rx=0 tx=0).stop 2024-04-25T13:42:00.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c0d0510 msgr2=0x7ff45c13e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c0d0510 0x7ff45c13e760 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7ff4580ae040 tx=0x7ff45801ab80 comp rx=0 tx=0).stop 2024-04-25T13:42:00.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 shutdown_connections 2024-04-25T13:42:00.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff44006f430 0x7ff440071940 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff45c161f90 0x7ff45c15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff45c15ec40 0x7ff45c159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 --2- 172.21.15.80:0/1625781601 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff45c0d0510 0x7ff45c13e760 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 >> 172.21.15.80:0/1625781601 conn(0x7ff45c06d080 msgr2=0x7ff45c15e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:00.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 shutdown_connections 2024-04-25T13:42:00.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.089+0000 7ff4643bb640 1 -- 172.21.15.80:0/1625781601 wait complete. 2024-04-25T13:42:00.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:41:30.714412+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:41:31.911193+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:42:00.111 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:42:00.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.253+0000 7f1a6fcca640 1 Processor -- start 2024-04-25T13:42:00.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.253+0000 7f1a6fcca640 1 -- start start 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.253+0000 7f1a6fcca640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a68071e60 0x7f1a681574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.253+0000 7f1a6fcca640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68161170 0x7f1a680b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.253+0000 7f1a6fcca640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a6806fbf0 con 0x7f1a68073c40 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.253+0000 7f1a6fcca640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1a680b1350 con 0x7f1a68071e60 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.253+0000 7f1a6fcca640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1a6806f9a0 con 0x7f1a68161170 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6da66640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a68071e60 0x7f1a681574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6da66640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a68071e60 0x7f1a681574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58758/0 (socket says 172.21.15.80:58758) 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6da66640 1 -- 172.21.15.80:0/3973724913 learned_addr learned my addr 172.21.15.80:0/3973724913 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6e267640 1 --2- 172.21.15.80:0/3973724913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68161170 0x7f1a680b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6da66640 1 -- 172.21.15.80:0/3973724913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68161170 msgr2=0x7f1a680b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:00.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6da66640 1 --2- 172.21.15.80:0/3973724913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68161170 0x7f1a680b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6da66640 1 -- 172.21.15.80:0/3973724913 >> v1:172.21.15.87:6789/0 conn(0x7f1a68073c40 legacy=0x7f1a68160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6da66640 1 -- 172.21.15.80:0/3973724913 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a680b0a20 con 0x7f1a68071e60 2024-04-25T13:42:00.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6da66640 1 --2- 172.21.15.80:0/3973724913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68071e60 0x7f1a681574c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009870 tx=0x7f1a6401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4bccdaf7bfb42b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6ca64640 1 -- 172.21.15.80:0/3973724913 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a640ab650 con 0x7f1a68071e60 2024-04-25T13:42:00.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6ca64640 1 -- 172.21.15.80:0/3973724913 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a640b0070 con 0x7f1a68071e60 2024-04-25T13:42:00.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6ca64640 1 -- 172.21.15.80:0/3973724913 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a640aba10 con 0x7f1a68071e60 2024-04-25T13:42:00.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6fcca640 1 -- 172.21.15.80:0/3973724913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68071e60 msgr2=0x7f1a681574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/3973724913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68071e60 0x7f1a681574c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1a64009870 tx=0x7f1a6401c560 comp rx=0 tx=0).stop 2024-04-25T13:42:00.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6fcca640 1 -- 172.21.15.80:0/3973724913 shutdown_connections 2024-04-25T13:42:00.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/3973724913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68161170 0x7f1a680b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/3973724913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68071e60 0x7f1a681574c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6fcca640 1 -- 172.21.15.80:0/3973724913 >> 172.21.15.80:0/3973724913 conn(0x7f1a680b4160 msgr2=0x7f1a680d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:00.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6fcca640 1 -- 172.21.15.80:0/3973724913 shutdown_connections 2024-04-25T13:42:00.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.257+0000 7f1a6fcca640 1 -- 172.21.15.80:0/3973724913 wait complete. 2024-04-25T13:42:00.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6fcca640 1 Processor -- start 2024-04-25T13:42:00.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6fcca640 1 -- start start 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6fcca640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a68071e60 0x7f1a680b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6fcca640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68073c40 0x7f1a68144b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6fcca640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68145470 0x7f1a68165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6fcca640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68169bb0 con 0x7f1a68073c40 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6fcca640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68074050 con 0x7f1a68071e60 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6fcca640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1a68169d30 con 0x7f1a68145470 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68073c40 0x7f1a68144b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68073c40 0x7f1a68144b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58760/0 (socket says 172.21.15.80:58760) 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 -- 172.21.15.80:0/288904226 learned_addr learned my addr 172.21.15.80:0/288904226 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6e267640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68145470 0x7f1a68165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6da66640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a68071e60 0x7f1a680b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 -- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68145470 msgr2=0x7f1a68165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68145470 0x7f1a68165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 -- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a68071e60 msgr2=0x7f1a680b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a68071e60 0x7f1a680b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 -- 172.21.15.80:0/288904226 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a680b2460 con 0x7f1a68073c40 2024-04-25T13:42:00.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.261+0000 7f1a6d265640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68073c40 0x7f1a68144b70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00c3a0 tx=0x7f1a5c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.265+0000 7f1a5a7fc640 1 -- 172.21.15.80:0/288904226 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0052d0 con 0x7f1a68073c40 2024-04-25T13:42:00.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.265+0000 7f1a5a7fc640 1 -- 172.21.15.80:0/288904226 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a5c005470 con 0x7f1a68073c40 2024-04-25T13:42:00.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.265+0000 7f1a5a7fc640 1 -- 172.21.15.80:0/288904226 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c007260 con 0x7f1a68073c40 2024-04-25T13:42:00.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.265+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a68151520 con 0x7f1a68073c40 2024-04-25T13:42:00.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.265+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a68151ae0 con 0x7f1a68073c40 2024-04-25T13:42:00.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.265+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a6806a3a0 con 0x7f1a68073c40 2024-04-25T13:42:00.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.265+0000 7f1a5a7fc640 1 -- 172.21.15.80:0/288904226 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1a5c007400 con 0x7f1a68073c40 2024-04-25T13:42:00.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.269+0000 7f1a5a7fc640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a4c06f400 0x7f1a4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.269+0000 7f1a5a7fc640 1 -- 172.21.15.80:0/288904226 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1a5c08e6c0 con 0x7f1a68073c40 2024-04-25T13:42:00.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.269+0000 7f1a6da66640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a4c06f400 0x7f1a4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.269+0000 7f1a6da66640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a4c06f400 0x7f1a4c071910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1a640af000 tx=0x7f1a640a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.273+0000 7f1a5a7fc640 1 -- 172.21.15.80:0/288904226 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a5c05b640 con 0x7f1a68073c40 2024-04-25T13:42:00.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.433+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1a680b3050 con 0x7f1a68073c40 2024-04-25T13:42:00.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.433+0000 7f1a5a7fc640 1 -- 172.21.15.80:0/288904226 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f1a5c05af40 con 0x7f1a68073c40 2024-04-25T13:42:00.436 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:42:00.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a4c06f400 msgr2=0x7f1a4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a4c06f400 0x7f1a4c071910 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1a640af000 tx=0x7f1a640a7000 comp rx=0 tx=0).stop 2024-04-25T13:42:00.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68073c40 msgr2=0x7f1a68144b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68073c40 0x7f1a68144b70 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1a5c00c3a0 tx=0x7f1a5c00c860 comp rx=0 tx=0).stop 2024-04-25T13:42:00.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 shutdown_connections 2024-04-25T13:42:00.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a4c06f400 0x7f1a4c071910 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a68145470 0x7f1a68165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a68073c40 0x7f1a68144b70 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 --2- 172.21.15.80:0/288904226 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a68071e60 0x7f1a680b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 >> 172.21.15.80:0/288904226 conn(0x7f1a680b4160 msgr2=0x7f1a6815e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:00.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 shutdown_connections 2024-04-25T13:42:00.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.437+0000 7f1a6fcca640 1 -- 172.21.15.80:0/288904226 wait complete. 2024-04-25T13:42:00.458 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:42:00.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.597+0000 7f0431c73640 1 Processor -- start 2024-04-25T13:42:00.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f0431c73640 1 -- start start 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f0431c73640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f042c06eb80 0x7f042c167660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f0431c73640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042c0737f0 con 0x7f042c073ad0 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f0431c73640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f042c1604d0 con 0x7f042c073080 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f0431c73640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f042c0b53d0 con 0x7f042c06eb80 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f042c073ad0 0x7f042c06e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49788/0 (socket says 172.21.15.80:49788) 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042affd640 1 -- 172.21.15.80:0/2241475329 learned_addr learned my addr 172.21.15.80:0/2241475329 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042bfff640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f042c06eb80 0x7f042c167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2226741217 0 0) 0x7f042c0737f0 con 0x7f042c073ad0 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0414003530 con 0x7f042c073ad0 2024-04-25T13:42:00.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 731157301 0 0) 0x7f0414003530 con 0x7f042c073ad0 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f042c06eb80 msgr2=0x7f042c167660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f042c06eb80 0x7f042c167660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 >> v1:172.21.15.87:6789/0 conn(0x7f042c073080 legacy=0x7f042c15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c1607b0 con 0x7f042c073ad0 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0420001680 con 0x7f042c073ad0 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 >> v1:172.21.15.80:6789/0 conn(0x7f042c073ad0 legacy=0x7f042c06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04140039f0 0x7f0414003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f04140048d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140050d0 0x7f041400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0b53d0 con 0x7f0414004380 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f041400c020 con 0x7f04140050d0 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f041400c220 con 0x7f04140039f0 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042affd640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f04140048d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042b7fe640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04140039f0 0x7f0414003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042bfff640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140050d0 0x7f041400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042affd640 1 -- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04140039f0 msgr2=0x7f0414003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:00.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042affd640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04140039f0 0x7f0414003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042affd640 1 -- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140050d0 msgr2=0x7f041400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:00.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042affd640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140050d0 0x7f041400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042affd640 1 -- 172.21.15.80:0/2241475329 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0414003530 con 0x7f0414004380 2024-04-25T13:42:00.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.601+0000 7f042bfff640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04140050d0 0x7f041400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:00.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f042affd640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f04140048d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f042000c1c0 tx=0x7f042001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3b1d3fe711fc07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420007d00 con 0x7f0414004380 2024-04-25T13:42:00.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04200a9020 con 0x7f0414004380 2024-04-25T13:42:00.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f042a7fc640 1 -- 172.21.15.80:0/2241475329 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0420005b00 con 0x7f0414004380 2024-04-25T13:42:00.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f0431c73640 1 -- 172.21.15.80:0/2241475329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 msgr2=0x7f04140048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f0431c73640 1 --2- 172.21.15.80:0/2241475329 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f04140048d0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f042000c1c0 tx=0x7f042001cc60 comp rx=0 tx=0).stop 2024-04-25T13:42:00.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f042bfff640 1 -- 172.21.15.80:0/2241475329 reap_dead start 2024-04-25T13:42:00.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f0431c73640 1 -- 172.21.15.80:0/2241475329 shutdown_connections 2024-04-25T13:42:00.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f0431c73640 1 -- 172.21.15.80:0/2241475329 >> 172.21.15.80:0/2241475329 conn(0x7f042c0b4160 msgr2=0x7f042c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:00.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f0431c73640 1 -- 172.21.15.80:0/2241475329 shutdown_connections 2024-04-25T13:42:00.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.605+0000 7f0431c73640 1 -- 172.21.15.80:0/2241475329 wait complete. 2024-04-25T13:42:00.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 Processor -- start 2024-04-25T13:42:00.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 -- start start 2024-04-25T13:42:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f042c152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f042c14f600 0x7f042c14a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f042c0b1be0 0x7f042c0b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f042c167660 con 0x7f0414004380 2024-04-25T13:42:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f042c06ed90 con 0x7f042c14f600 2024-04-25T13:42:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f042c1677e0 con 0x7f042c0b1be0 2024-04-25T13:42:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f042c152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f042c152620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58774/0 (socket says 172.21.15.80:58774) 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 -- 172.21.15.80:0/1824511904 learned_addr learned my addr 172.21.15.80:0/1824511904 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042affd640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f042c14f600 0x7f042c14a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 -- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f042c0b1be0 msgr2=0x7f042c0b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f042c0b1be0 0x7f042c0b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 -- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f042c14f600 msgr2=0x7f042c14a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f042c14f600 0x7f042c14a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 -- 172.21.15.80:0/1824511904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c0b2530 con 0x7f0414004380 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f042b7fe640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f042c152620 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f041c00e390 tx=0x7f041c00e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0413fff640 1 -- 172.21.15.80:0/1824511904 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c019070 con 0x7f0414004380 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0413fff640 1 -- 172.21.15.80:0/1824511904 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f041c0052e0 con 0x7f0414004380 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c14c200 con 0x7f0414004380 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0413fff640 1 -- 172.21.15.80:0/1824511904 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c003820 con 0x7f0414004380 2024-04-25T13:42:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c14c7c0 con 0x7f0414004380 2024-04-25T13:42:00.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042c06a3a0 con 0x7f0414004380 2024-04-25T13:42:00.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.609+0000 7f0413fff640 1 -- 172.21.15.80:0/1824511904 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f041c01c070 con 0x7f0414004380 2024-04-25T13:42:00.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.613+0000 7f0413fff640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f041806f400 0x7f0418071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.613+0000 7f0413fff640 1 -- 172.21.15.80:0/1824511904 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f041c08e840 con 0x7f0414004380 2024-04-25T13:42:00.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.613+0000 7f042affd640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f041806f400 0x7f0418071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.613+0000 7f042affd640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f041806f400 0x7f0418071910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f04200aa000 tx=0x7f04200b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.617+0000 7f0413fff640 1 -- 172.21.15.80:0/1824511904 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f042c0b2530 con 0x7f0414004380 2024-04-25T13:42:00.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.761+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f042c0d1d10 con 0x7f0414004380 2024-04-25T13:42:00.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0413fff640 1 -- 172.21.15.80:0/1824511904 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) ==== 104+0+11 (secure 0 0 0) 0x7f041c05b0e0 con 0x7f0414004380 2024-04-25T13:42:00.767 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:42:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f041806f400 msgr2=0x7f0418071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f041806f400 0x7f0418071910 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f04200aa000 tx=0x7f04200b0040 comp rx=0 tx=0).stop 2024-04-25T13:42:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 msgr2=0x7f042c152620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f042c152620 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f041c00e390 tx=0x7f041c00e850 comp rx=0 tx=0).stop 2024-04-25T13:42:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 shutdown_connections 2024-04-25T13:42:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f041806f400 0x7f0418071910 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f042c0b1be0 0x7f042c0b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f042c14f600 0x7f042c14a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 --2- 172.21.15.80:0/1824511904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0414004380 0x7f042c152620 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 >> 172.21.15.80:0/1824511904 conn(0x7f042c0b4160 msgr2=0x7f042c15b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 shutdown_connections 2024-04-25T13:42:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.765+0000 7f0431c73640 1 -- 172.21.15.80:0/1824511904 wait complete. 2024-04-25T13:42:00.787 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:42:00.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.929+0000 7fd606561640 1 Processor -- start 2024-04-25T13:42:00.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd606561640 1 -- start start 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd606561640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6000b5330 0x7fd6000d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd606561640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd6000d0ff0 0x7fd60006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd606561640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6000b5740 con 0x7fd60006f2f0 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd606561640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd60006f700 con 0x7fd6000d0ff0 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd606561640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6001629b0 con 0x7fd6000b5330 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd5ff7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd6000d0ff0 0x7fd60006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd604afe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd60006f2f0 0x7fd600161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:49802/0 (socket says 172.21.15.80:49802) 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd604afe640 1 -- 172.21.15.80:0/1943431685 learned_addr learned my addr 172.21.15.80:0/1943431685 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd5fffff640 1 --2- 172.21.15.80:0/1943431685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6000b5330 0x7fd6000d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd5feffd640 1 -- 172.21.15.80:0/1943431685 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 115540104 0 0) 0x7fd6000b5740 con 0x7fd60006f2f0 2024-04-25T13:42:00.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd5feffd640 1 -- 172.21.15.80:0/1943431685 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5e8003530 con 0x7fd60006f2f0 2024-04-25T13:42:00.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd5ff7fe640 1 -- 172.21.15.80:0/1943431685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6000b5330 msgr2=0x7fd6000d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:00.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd5ff7fe640 1 --2- 172.21.15.80:0/1943431685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6000b5330 0x7fd6000d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd5ff7fe640 1 -- 172.21.15.80:0/1943431685 >> v1:172.21.15.80:6789/0 conn(0x7fd60006f2f0 legacy=0x7fd600161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.933+0000 7fd5ff7fe640 1 -- 172.21.15.80:0/1943431685 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60006dd10 con 0x7fd6000d0ff0 2024-04-25T13:42:00.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd5ff7fe640 1 --2- 172.21.15.80:0/1943431685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000d0ff0 0x7fd60006edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40097c0 tx=0x7fd5f401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9357212a2d621c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd5feffd640 1 -- 172.21.15.80:0/1943431685 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40b0070 con 0x7fd6000d0ff0 2024-04-25T13:42:00.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd5feffd640 1 -- 172.21.15.80:0/1943431685 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f40ab600 con 0x7fd6000d0ff0 2024-04-25T13:42:00.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd5feffd640 1 -- 172.21.15.80:0/1943431685 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40ab9d0 con 0x7fd6000d0ff0 2024-04-25T13:42:00.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd606561640 1 -- 172.21.15.80:0/1943431685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000d0ff0 msgr2=0x7fd60006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd606561640 1 --2- 172.21.15.80:0/1943431685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000d0ff0 0x7fd60006edb0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40097c0 tx=0x7fd5f401c410 comp rx=0 tx=0).stop 2024-04-25T13:42:00.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd606561640 1 -- 172.21.15.80:0/1943431685 shutdown_connections 2024-04-25T13:42:00.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd606561640 1 --2- 172.21.15.80:0/1943431685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000d0ff0 0x7fd60006edb0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd606561640 1 --2- 172.21.15.80:0/1943431685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6000b5330 0x7fd6000d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd606561640 1 -- 172.21.15.80:0/1943431685 >> 172.21.15.80:0/1943431685 conn(0x7fd6000b4160 msgr2=0x7fd6000d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:00.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd606561640 1 -- 172.21.15.80:0/1943431685 shutdown_connections 2024-04-25T13:42:00.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.937+0000 7fd606561640 1 -- 172.21.15.80:0/1943431685 wait complete. 2024-04-25T13:42:00.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 Processor -- start 2024-04-25T13:42:00.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 -- start start 2024-04-25T13:42:00.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd60006f2f0 0x7fd60013caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000b5330 0x7fd60015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6000d0ff0 0x7fd60015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd6000b0cd0 con 0x7fd6000b5330 2024-04-25T13:42:00.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd60006e890 con 0x7fd6000d0ff0 2024-04-25T13:42:00.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6000b0e50 con 0x7fd60006f2f0 2024-04-25T13:42:00.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000b5330 0x7fd60015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000b5330 0x7fd60015b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58792/0 (socket says 172.21.15.80:58792) 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 -- 172.21.15.80:0/2141579527 learned_addr learned my addr 172.21.15.80:0/2141579527 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5fffff640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd60006f2f0 0x7fd60013caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd604afe640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6000d0ff0 0x7fd60015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 -- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd60006f2f0 msgr2=0x7fd60013caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd60006f2f0 0x7fd60013caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 -- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6000d0ff0 msgr2=0x7fd60015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6000d0ff0 0x7fd60015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 -- 172.21.15.80:0/2141579527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60013d030 con 0x7fd6000b5330 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5fffff640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd60006f2f0 0x7fd60013caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5ff7fe640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000b5330 0x7fd60015b0c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5f401aac0 tx=0x7fd5f40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5fcff9640 1 -- 172.21.15.80:0/2141579527 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40b0040 con 0x7fd6000b5330 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5fcff9640 1 -- 172.21.15.80:0/2141579527 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5f40bc070 con 0x7fd6000b5330 2024-04-25T13:42:00.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd600146750 con 0x7fd6000b5330 2024-04-25T13:42:00.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd5fcff9640 1 -- 172.21.15.80:0/2141579527 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f40b56c0 con 0x7fd6000b5330 2024-04-25T13:42:00.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.941+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd600146d10 con 0x7fd6000b5330 2024-04-25T13:42:00.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.945+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60006a3a0 con 0x7fd6000b5330 2024-04-25T13:42:00.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.945+0000 7fd5fcff9640 1 -- 172.21.15.80:0/2141579527 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd5f40b3070 con 0x7fd6000b5330 2024-04-25T13:42:00.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.945+0000 7fd5fcff9640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5ec06f400 0x7fd5ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:00.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.945+0000 7fd5fcff9640 1 -- 172.21.15.80:0/2141579527 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd5f4129dd0 con 0x7fd6000b5330 2024-04-25T13:42:00.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.945+0000 7fd5fffff640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5ec06f400 0x7fd5ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:00.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.945+0000 7fd5fffff640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5ec06f400 0x7fd5ec071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8007440 tx=0x7fd5f800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:00.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:00.949+0000 7fd5fcff9640 1 -- 172.21.15.80:0/2141579527 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd5f4016090 con 0x7fd6000b5330 2024-04-25T13:42:01.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.109+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd60006fd00 con 0x7fd6000b5330 2024-04-25T13:42:01.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.109+0000 7fd5fcff9640 1 -- 172.21.15.80:0/2141579527 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) ==== 100+0+11 (secure 0 0 0) 0x7fd5f40f6da0 con 0x7fd6000b5330 2024-04-25T13:42:01.114 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:42:01.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5ec06f400 msgr2=0x7fd5ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5ec06f400 0x7fd5ec071910 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8007440 tx=0x7fd5f800b040 comp rx=0 tx=0).stop 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000b5330 msgr2=0x7fd60015b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000b5330 0x7fd60015b0c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fd5f401aac0 tx=0x7fd5f40191b0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 shutdown_connections 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5ec06f400 0x7fd5ec071910 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6000d0ff0 0x7fd60015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6000b5330 0x7fd60015b0c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 --2- 172.21.15.80:0/2141579527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd60006f2f0 0x7fd60013caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 >> 172.21.15.80:0/2141579527 conn(0x7fd6000b4160 msgr2=0x7fd6000b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 shutdown_connections 2024-04-25T13:42:01.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.113+0000 7fd606561640 1 -- 172.21.15.80:0/2141579527 wait complete. 2024-04-25T13:42:01.135 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:01.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f72832f9640 1 Processor -- start 2024-04-25T13:42:01.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f72832f9640 1 -- start start 2024-04-25T13:42:01.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f72832f9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0b2d30 0x7f727c0d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f72832f9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f727c0d30b0 con 0x7f727c0d0350 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f72832f9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f727c0ce1c0 con 0x7f727c0ceca0 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f72832f9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f727c0d1120 con 0x7f727c0b2d30 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7280894640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f727c0d0350 0x7f727c0d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57396/0 (socket says 172.21.15.80:57396) 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7280894640 1 -- 172.21.15.80:0/4147350507 learned_addr learned my addr 172.21.15.80:0/4147350507 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7281095640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0b2d30 0x7f727c0d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352165646 0 0) 0x7f727c0d30b0 con 0x7f727c0d0350 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7268003530 con 0x7f727c0d0350 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2578625705 0 0) 0x7f7268003530 con 0x7f727c0d0350 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0b2d30 msgr2=0x7f727c0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0b2d30 0x7f727c0d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 >> v1:172.21.15.87:6789/0 conn(0x7f727c0ceca0 legacy=0x7f727c0cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c0ce4a0 con 0x7f727c0d0350 2024-04-25T13:42:01.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f726c001680 con 0x7f727c0d0350 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 >> v1:172.21.15.80:6789/0 conn(0x7f727c0d0350 legacy=0x7f727c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0b2d30 0x7f72680039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f7268004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7268004d50 0x7f7268005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f727c0d1120 con 0x7f7268003f70 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f726800bb60 con 0x7f7268004d50 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f726800bd60 con 0x7f727c0b2d30 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7280894640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f7268004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7281095640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0b2d30 0x7f72680039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.277+0000 7f7281896640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7268004d50 0x7f7268005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7280894640 1 -- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0b2d30 msgr2=0x7f72680039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7280894640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0b2d30 0x7f72680039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7280894640 1 -- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7268004d50 msgr2=0x7f7268005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7280894640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7268004d50 0x7f7268005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7280894640 1 -- 172.21.15.80:0/4147350507 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7268003530 con 0x7f7268003f70 2024-04-25T13:42:01.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7280894640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f7268004810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f727c069670 tx=0x7f726c01f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c162e8f784dac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c007d60 con 0x7f7268003f70 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726c005ae0 con 0x7f7268003f70 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7273fff640 1 -- 172.21.15.80:0/4147350507 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c007070 con 0x7f7268003f70 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f72832f9640 1 -- 172.21.15.80:0/4147350507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 msgr2=0x7f7268004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f72832f9640 1 --2- 172.21.15.80:0/4147350507 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f7268004810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f727c069670 tx=0x7f726c01f330 comp rx=0 tx=0).stop 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f7281896640 1 -- 172.21.15.80:0/4147350507 reap_dead start 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f72832f9640 1 -- 172.21.15.80:0/4147350507 shutdown_connections 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f72832f9640 1 -- 172.21.15.80:0/4147350507 >> 172.21.15.80:0/4147350507 conn(0x7f727c0d3ad0 msgr2=0x7f727c0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f72832f9640 1 -- 172.21.15.80:0/4147350507 shutdown_connections 2024-04-25T13:42:01.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.281+0000 7f72832f9640 1 -- 172.21.15.80:0/4147350507 wait complete. 2024-04-25T13:42:01.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 Processor -- start 2024-04-25T13:42:01.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 -- start start 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f727c159f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f727c0cb540 0x7f727c0ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0cd4f0 0x7f727c0b7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f727c0d3390 con 0x7f7268003f70 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f727c0d2dc0 con 0x7f727c0cb540 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f727c0d1dd0 con 0x7f727c0cd4f0 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f727c159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f727c159f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39240/0 (socket says 172.21.15.80:39240) 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 -- 172.21.15.80:0/3931298336 learned_addr learned my addr 172.21.15.80:0/3931298336 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7280894640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f727c0cb540 0x7f727c0ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281896640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0cd4f0 0x7f727c0b7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 -- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0cd4f0 msgr2=0x7f727c0b7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0cd4f0 0x7f727c0b7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 -- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f727c0cb540 msgr2=0x7f727c0ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f727c0cb540 0x7f727c0ccfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 -- 172.21.15.80:0/3931298336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c0b1ea0 con 0x7f7268003f70 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7280894640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f727c0cb540 0x7f727c0ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7281095640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f727c159f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7278011590 tx=0x7f7278011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7271ffb640 1 -- 172.21.15.80:0/3931298336 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278019070 con 0x7f7268003f70 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7271ffb640 1 -- 172.21.15.80:0/3931298336 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7278005db0 con 0x7f7268003f70 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f7271ffb640 1 -- 172.21.15.80:0/3931298336 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278006270 con 0x7f7268003f70 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c0b3be0 con 0x7f7268003f70 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c15d8a0 con 0x7f7268003f70 2024-04-25T13:42:01.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.285+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727c06a3a0 con 0x7f7268003f70 2024-04-25T13:42:01.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.289+0000 7f7271ffb640 1 -- 172.21.15.80:0/3931298336 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f727801c070 con 0x7f7268003f70 2024-04-25T13:42:01.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.289+0000 7f7271ffb640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f726006f430 0x7f7260071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.289+0000 7f7271ffb640 1 -- 172.21.15.80:0/3931298336 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f727808f6d0 con 0x7f7268003f70 2024-04-25T13:42:01.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.289+0000 7f7280894640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f726006f430 0x7f7260071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.289+0000 7f7280894640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f726006f430 0x7f7260071940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f727c069670 tx=0x7f726c0b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.293+0000 7f7271ffb640 1 -- 172.21.15.80:0/3931298336 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f727805c650 con 0x7f7268003f70 2024-04-25T13:42:01.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.485+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f727c074c30 con 0x7f7268003f70 2024-04-25T13:42:01.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.485+0000 7f7271ffb640 1 -- 172.21.15.80:0/3931298336 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5175 (secure 0 0 0) 0x7f727805bf50 con 0x7f7268003f70 2024-04-25T13:42:01.491 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:01.491 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:41:32.974439+0000","modified":"2024-04-25T13:41:36.017164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/966995060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":966995060},{"type":"v1","addr":"172.21.15.87:6835","nonce":966995060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5190,"qdb_cluster":[5190]},"id":2}]} 2024-04-25T13:42:01.491 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 16 2024-04-25T13:42:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f726006f430 msgr2=0x7f7260071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f726006f430 0x7f7260071940 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f727c069670 tx=0x7f726c0b2040 comp rx=0 tx=0).stop 2024-04-25T13:42:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 msgr2=0x7f727c159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f727c159f30 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f7278011590 tx=0x7f7278011a50 comp rx=0 tx=0).stop 2024-04-25T13:42:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 shutdown_connections 2024-04-25T13:42:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f726006f430 0x7f7260071940 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f727c0cd4f0 0x7f727c0b7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f727c0cb540 0x7f727c0ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 --2- 172.21.15.80:0/3931298336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7268003f70 0x7f727c159f30 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 >> 172.21.15.80:0/3931298336 conn(0x7f727c0d3ad0 msgr2=0x7f727c0b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 shutdown_connections 2024-04-25T13:42:01.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.489+0000 7f72832f9640 1 -- 172.21.15.80:0/3931298336 wait complete. 2024-04-25T13:42:01.511 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:01.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.641+0000 7f9b312e8640 1 Processor -- start 2024-04-25T13:42:01.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.641+0000 7f9b312e8640 1 -- start start 2024-04-25T13:42:01.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b312e8640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b2c161f90 0x7f9b2c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b312e8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9b2c0d0510 0x7f9b2c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b312e8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b2c0d0d70 con 0x7f9b2c15ea70 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b312e8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9b2c0d1050 con 0x7f9b2c0d0510 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b312e8640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9b2c06dca0 con 0x7f9b2c161f90 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2a7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9b2c0d0510 0x7f9b2c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2b7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9b2c15ea70 0x7f9b2c15ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57398/0 (socket says 172.21.15.80:57398) 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2b7fe640 1 -- 172.21.15.80:0/1503805673 learned_addr learned my addr 172.21.15.80:0/1503805673 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2affd640 1 --2- 172.21.15.80:0/1503805673 >> v2:172.21.15.87:3300/0 conn(0x7f9b2c161f90 0x7f9b2c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2a7fc640 1 -- 172.21.15.80:0/1503805673 >> v2:172.21.15.87:3300/0 conn(0x7f9b2c161f90 msgr2=0x7f9b2c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2a7fc640 1 --2- 172.21.15.80:0/1503805673 >> v2:172.21.15.87:3300/0 conn(0x7f9b2c161f90 0x7f9b2c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2a7fc640 1 -- 172.21.15.80:0/1503805673 >> v1:172.21.15.80:6789/0 conn(0x7f9b2c15ea70 legacy=0x7f9b2c15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2a7fc640 1 -- 172.21.15.80:0/1503805673 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c06df80 con 0x7f9b2c0d0510 2024-04-25T13:42:01.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b2a7fc640 1 --2- 172.21.15.80:0/1503805673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c0d0510 0x7f9b2c15e530 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0d17b0 tx=0x7f9b1401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35a289e3ad8c4239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b29ffb640 1 -- 172.21.15.80:0/1503805673 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140b0070 con 0x7f9b2c0d0510 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b29ffb640 1 -- 172.21.15.80:0/1503805673 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b140ab600 con 0x7f9b2c0d0510 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.645+0000 7f9b29ffb640 1 -- 172.21.15.80:0/1503805673 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b140ab9d0 con 0x7f9b2c0d0510 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- 172.21.15.80:0/1503805673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c0d0510 msgr2=0x7f9b2c15e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 --2- 172.21.15.80:0/1503805673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c0d0510 0x7f9b2c15e530 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c0d17b0 tx=0x7f9b1401c390 comp rx=0 tx=0).stop 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- 172.21.15.80:0/1503805673 shutdown_connections 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 --2- 172.21.15.80:0/1503805673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c0d0510 0x7f9b2c15e530 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 --2- 172.21.15.80:0/1503805673 >> v2:172.21.15.87:3300/0 conn(0x7f9b2c161f90 0x7f9b2c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- 172.21.15.80:0/1503805673 >> 172.21.15.80:0/1503805673 conn(0x7f9b2c06d080 msgr2=0x7f9b2c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- 172.21.15.80:0/1503805673 shutdown_connections 2024-04-25T13:42:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- 172.21.15.80:0/1503805673 wait complete. 2024-04-25T13:42:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 Processor -- start 2024-04-25T13:42:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- start start 2024-04-25T13:42:01.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b2c0d0510 0x7f9b2c15dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b2c15ea70 0x7f9b2c1471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c161f90 0x7f9b2c13d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c06f630 con 0x7f9b2c161f90 2024-04-25T13:42:01.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b2c06f4b0 con 0x7f9b2c15ea70 2024-04-25T13:42:01.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b312e8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b2c0d0920 con 0x7f9b2c0d0510 2024-04-25T13:42:01.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b2b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c161f90 0x7f9b2c13d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b2b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c161f90 0x7f9b2c13d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39252/0 (socket says 172.21.15.80:39252) 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.649+0000 7f9b2b7fe640 1 -- 172.21.15.80:0/3290485791 learned_addr learned my addr 172.21.15.80:0/3290485791 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2affd640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b2c0d0510 0x7f9b2c15dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2a7fc640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b2c15ea70 0x7f9b2c1471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2b7fe640 1 -- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b2c0d0510 msgr2=0x7f9b2c15dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2b7fe640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b2c0d0510 0x7f9b2c15dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2b7fe640 1 -- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b2c15ea70 msgr2=0x7f9b2c1471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2b7fe640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b2c15ea70 0x7f9b2c1471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2b7fe640 1 -- 172.21.15.80:0/3290485791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b2c146140 con 0x7f9b2c161f90 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2b7fe640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c161f90 0x7f9b2c13d0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00caa0 tx=0x7f9b1c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b237fe640 1 -- 172.21.15.80:0/3290485791 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c017070 con 0x7f9b2c161f90 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b237fe640 1 -- 172.21.15.80:0/3290485791 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b1c00ac00 con 0x7f9b2c161f90 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b2c146420 con 0x7f9b2c161f90 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b237fe640 1 -- 172.21.15.80:0/3290485791 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b1c012400 con 0x7f9b2c161f90 2024-04-25T13:42:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b2c1469e0 con 0x7f9b2c161f90 2024-04-25T13:42:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b2c06a3a0 con 0x7f9b2c161f90 2024-04-25T13:42:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b237fe640 1 -- 172.21.15.80:0/3290485791 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b1c01a070 con 0x7f9b2c161f90 2024-04-25T13:42:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b237fe640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0c06f400 0x7f9b0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b237fe640 1 -- 172.21.15.80:0/3290485791 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9b1c08eed0 con 0x7f9b2c161f90 2024-04-25T13:42:01.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2affd640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0c06f400 0x7f9b0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.653+0000 7f9b2affd640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0c06f400 0x7f9b0c071910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c06fa20 tx=0x7f9b24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.657+0000 7f9b237fe640 1 -- 172.21.15.80:0/3290485791 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b1c05b770 con 0x7f9b2c161f90 2024-04-25T13:42:01.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.805+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b2c15f050 con 0x7f9b2c161f90 2024-04-25T13:42:01.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b237fe640 1 -- 172.21.15.80:0/3290485791 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7f9b1c05fa70 con 0x7f9b2c161f90 2024-04-25T13:42:01.812 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:01.812 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":35,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:53.226827+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:01.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0c06f400 msgr2=0x7f9b0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0c06f400 0x7f9b0c071910 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c06fa20 tx=0x7f9b24008040 comp rx=0 tx=0).stop 2024-04-25T13:42:01.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c161f90 msgr2=0x7f9b2c13d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c161f90 0x7f9b2c13d0a0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9b1c00caa0 tx=0x7f9b1c00cf60 comp rx=0 tx=0).stop 2024-04-25T13:42:01.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 shutdown_connections 2024-04-25T13:42:01.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b0c06f400 0x7f9b0c071910 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b2c161f90 0x7f9b2c13d0a0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b2c15ea70 0x7f9b2c1471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 --2- 172.21.15.80:0/3290485791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b2c0d0510 0x7f9b2c15dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 >> 172.21.15.80:0/3290485791 conn(0x7f9b2c06d080 msgr2=0x7f9b2c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 shutdown_connections 2024-04-25T13:42:01.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.809+0000 7f9b312e8640 1 -- 172.21.15.80:0/3290485791 wait complete. 2024-04-25T13:42:01.832 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:42:01.832 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:01.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.957+0000 7f2197713640 1 Processor -- start 2024-04-25T13:42:01.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.957+0000 7f2197713640 1 -- start start 2024-04-25T13:42:01.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2197713640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21900b8cd0 0x7f21900b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2197713640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f21900d0b00 0x7f21900b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2197713640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f219015e8c0 con 0x7f21900cf690 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2197713640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f21900ce8d0 con 0x7f21900d0b00 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2197713640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f21900cebb0 con 0x7f21900b8cd0 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2194cae640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f21900cf690 0x7f21900d03f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57402/0 (socket says 172.21.15.80:57402) 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2194cae640 1 -- 172.21.15.80:0/962133522 learned_addr learned my addr 172.21.15.80:0/962133522 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f21954af640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21900b8cd0 0x7f21900b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2195cb0640 1 --2- 172.21.15.80:0/962133522 >> v2:172.21.15.87:3300/0 conn(0x7f21900d0b00 0x7f21900b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1045191974 0 0) 0x7f219015e8c0 con 0x7f21900cf690 2024-04-25T13:42:01.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f217c003530 con 0x7f21900cf690 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2221497132 0 0) 0x7f217c003530 con 0x7f21900cf690 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21900b8cd0 msgr2=0x7f21900b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21900b8cd0 0x7f21900b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 >> v2:172.21.15.87:3300/0 conn(0x7f21900d0b00 msgr2=0x7f21900b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 --2- 172.21.15.80:0/962133522 >> v2:172.21.15.87:3300/0 conn(0x7f21900d0b00 0x7f21900b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21900cee90 con 0x7f21900cf690 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2180001680 con 0x7f21900cf690 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 >> v1:172.21.15.80:6789/0 conn(0x7f21900cf690 legacy=0x7f21900d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21900b8cd0 0x7f217c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f217c003f40 0x7f217c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f217c004d50 0x7f217c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f21900ce8d0 con 0x7f217c004d50 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f21900cebb0 con 0x7f217c003f40 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f217c00bad0 con 0x7f21900b8cd0 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2195cb0640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f217c004d50 0x7f217c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f21954af640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21900b8cd0 0x7f217c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2195cb0640 1 -- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21900b8cd0 msgr2=0x7f217c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2195cb0640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21900b8cd0 0x7f217c0039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2195cb0640 1 -- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f217c003f40 msgr2=0x7f217c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2195cb0640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f217c003f40 0x7f217c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.961+0000 7f2195cb0640 1 -- 172.21.15.80:0/962133522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f217c003530 con 0x7f217c004d50 2024-04-25T13:42:01.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2195cb0640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f217c004d50 0x7f217c005160 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2190073d50 tx=0x7f218801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb54244134e2ccc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21880b2070 con 0x7f217c004d50 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21880a5c70 con 0x7f217c004d50 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2187fff640 1 -- 172.21.15.80:0/962133522 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2188009c10 con 0x7f217c004d50 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2197713640 1 -- 172.21.15.80:0/962133522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f217c004d50 msgr2=0x7f217c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2197713640 1 --2- 172.21.15.80:0/962133522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f217c004d50 0x7f217c005160 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2190073d50 tx=0x7f218801cc60 comp rx=0 tx=0).stop 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2195cb0640 1 -- 172.21.15.80:0/962133522 reap_dead start 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2197713640 1 -- 172.21.15.80:0/962133522 shutdown_connections 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2197713640 1 -- 172.21.15.80:0/962133522 >> 172.21.15.80:0/962133522 conn(0x7f21900d3ad0 msgr2=0x7f21900d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2197713640 1 -- 172.21.15.80:0/962133522 shutdown_connections 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2197713640 1 -- 172.21.15.80:0/962133522 wait complete. 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2197713640 1 Processor -- start 2024-04-25T13:42:01.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.965+0000 7f2197713640 1 -- start start 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f217c004d50 0x7f21901f4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f219009f470 0x7f2190151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f219015bf90 0x7f21900b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f21900d2aa0 con 0x7f219009f470 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f21900b9440 con 0x7f217c004d50 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f21900d2c20 con 0x7f219015bf90 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f219009f470 0x7f2190151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f219009f470 0x7f2190151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39280/0 (socket says 172.21.15.80:39280) 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 -- 172.21.15.80:0/3161194860 learned_addr learned my addr 172.21.15.80:0/3161194860 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:01.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f21954af640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f217c004d50 0x7f21901f4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2195cb0640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f219015bf90 0x7f21900b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 -- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f219015bf90 msgr2=0x7f21900b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f219015bf90 0x7f21900b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 -- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f217c004d50 msgr2=0x7f21901f4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f217c004d50 0x7f21901f4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 -- 172.21.15.80:0/3161194860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f219009fc50 con 0x7f219009f470 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2194cae640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f219009f470 0x7f2190151f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2180004880 tx=0x7f218000f760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2185ffb640 1 -- 172.21.15.80:0/3161194860 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2180007d00 con 0x7f219009f470 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2185ffb640 1 -- 172.21.15.80:0/3161194860 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f218000a0d0 con 0x7f219009f470 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f219009d830 con 0x7f219009f470 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2185ffb640 1 -- 172.21.15.80:0/3161194860 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f218000a500 con 0x7f219009f470 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f219009ddf0 con 0x7f219009f470 2024-04-25T13:42:01.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f219009f9a0 con 0x7f219009f470 2024-04-25T13:42:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2185ffb640 1 -- 172.21.15.80:0/3161194860 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2180014070 con 0x7f219009f470 2024-04-25T13:42:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.969+0000 7f2185ffb640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f217406f430 0x7f2174071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.973+0000 7f2185ffb640 1 -- 172.21.15.80:0/3161194860 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f21800922c0 con 0x7f219009f470 2024-04-25T13:42:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.973+0000 7f21954af640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f217406f430 0x7f2174071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.973+0000 7f21954af640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f217406f430 0x7f2174071940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f218c009a30 tx=0x7f218c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:01.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:01.973+0000 7f2185ffb640 1 -- 172.21.15.80:0/3161194860 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f218005f290 con 0x7f219009f470 2024-04-25T13:42:02.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.153+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f219006a3a0 con 0x7f219009f470 2024-04-25T13:42:02.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2185ffb640 1 -- 172.21.15.80:0/3161194860 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5175 (secure 0 0 0) 0x7f218005eb90 con 0x7f219009f470 2024-04-25T13:42:02.160 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:02.160 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:41:32.974439+0000","modified":"2024-04-25T13:41:36.017164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/966995060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":966995060},{"type":"v1","addr":"172.21.15.87:6835","nonce":966995060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5190,"qdb_cluster":[5190]},"id":2}]} 2024-04-25T13:42:02.160 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 16 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f217406f430 msgr2=0x7f2174071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f217406f430 0x7f2174071940 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f218c009a30 tx=0x7f218c008040 comp rx=0 tx=0).stop 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f219009f470 msgr2=0x7f2190151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f219009f470 0x7f2190151f40 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2180004880 tx=0x7f218000f760 comp rx=0 tx=0).stop 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 shutdown_connections 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f217406f430 0x7f2174071940 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f219015bf90 0x7f21900b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f219009f470 0x7f2190151f40 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 --2- 172.21.15.80:0/3161194860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f217c004d50 0x7f21901f4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 >> 172.21.15.80:0/3161194860 conn(0x7f21900d3ad0 msgr2=0x7f219015c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:02.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 shutdown_connections 2024-04-25T13:42:02.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.157+0000 7f2197713640 1 -- 172.21.15.80:0/3161194860 wait complete. 2024-04-25T13:42:02.179 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:02.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f93a2148640 1 Processor -- start 2024-04-25T13:42:02.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f93a2148640 1 -- start start 2024-04-25T13:42:02.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f93a2148640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f93a2148640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f939c149410 0x7f939c068ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f93a2148640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f939c06caf0 con 0x7f939c163de0 2024-04-25T13:42:02.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f93a2148640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f939c06cdd0 con 0x7f939c149410 2024-04-25T13:42:02.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f93a2148640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f939c06d0b0 con 0x7f939c0b2020 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f939affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f939c149410 0x7f939c068ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f939bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f939c163de0 0x7f939c1641f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57418/0 (socket says 172.21.15.80:57418) 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.325+0000 7f939bfff640 1 -- 172.21.15.80:0/3515813980 learned_addr learned my addr 172.21.15.80:0/3515813980 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939b7fe640 1 --2- 172.21.15.80:0/3515813980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939affd640 1 -- 172.21.15.80:0/3515813980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 msgr2=0x7f939c068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939affd640 1 --2- 172.21.15.80:0/3515813980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939affd640 1 -- 172.21.15.80:0/3515813980 >> v1:172.21.15.80:6789/0 conn(0x7f939c163de0 legacy=0x7f939c1641f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939a7fc640 1 -- 172.21.15.80:0/3515813980 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3450756253 0 0) 0x7f939c06caf0 con 0x7f939c163de0 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939affd640 1 -- 172.21.15.80:0/3515813980 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c06d390 con 0x7f939c149410 2024-04-25T13:42:02.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939b7fe640 1 --2- 172.21.15.80:0/3515813980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:02.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939affd640 1 --2- 172.21.15.80:0/3515813980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c149410 0x7f939c068ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f938c009d90 tx=0x7f938c01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc8d72080240709f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:02.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939a7fc640 1 -- 172.21.15.80:0/3515813980 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0b2070 con 0x7f939c149410 2024-04-25T13:42:02.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939a7fc640 1 -- 172.21.15.80:0/3515813980 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c0a5a50 con 0x7f939c149410 2024-04-25T13:42:02.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f939a7fc640 1 -- 172.21.15.80:0/3515813980 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c0a5e20 con 0x7f939c149410 2024-04-25T13:42:02.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f93a2148640 1 -- 172.21.15.80:0/3515813980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c149410 msgr2=0x7f939c068ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f93a2148640 1 --2- 172.21.15.80:0/3515813980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c149410 0x7f939c068ad0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f938c009d90 tx=0x7f938c01c970 comp rx=0 tx=0).stop 2024-04-25T13:42:02.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f93a2148640 1 -- 172.21.15.80:0/3515813980 shutdown_connections 2024-04-25T13:42:02.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f93a2148640 1 --2- 172.21.15.80:0/3515813980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c149410 0x7f939c068ad0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f93a2148640 1 --2- 172.21.15.80:0/3515813980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f93a2148640 1 -- 172.21.15.80:0/3515813980 >> 172.21.15.80:0/3515813980 conn(0x7f939c0d28f0 msgr2=0x7f939c0d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:02.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f93a2148640 1 -- 172.21.15.80:0/3515813980 shutdown_connections 2024-04-25T13:42:02.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.329+0000 7f93a2148640 1 -- 172.21.15.80:0/3515813980 wait complete. 2024-04-25T13:42:02.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 Processor -- start 2024-04-25T13:42:02.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 -- start start 2024-04-25T13:42:02.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c15f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f939c149410 0x7f939c13c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c163de0 0x7f939c13ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f939c16dd20 con 0x7f939c163de0 2024-04-25T13:42:02.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f939c16dba0 con 0x7f939c149410 2024-04-25T13:42:02.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f939c16dea0 con 0x7f939c0b2020 2024-04-25T13:42:02.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c163de0 0x7f939c13ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c163de0 0x7f939c13ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39296/0 (socket says 172.21.15.80:39296) 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 -- 172.21.15.80:0/1817769440 learned_addr learned my addr 172.21.15.80:0/1817769440 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939b7fe640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c15f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 -- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 msgr2=0x7f939c15f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c15f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 -- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f939c149410 msgr2=0x7f939c13c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f939c149410 0x7f939c13c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 -- 172.21.15.80:0/1817769440 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939c142d10 con 0x7f939c163de0 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f939bfff640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c163de0 0x7f939c13ce30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f939400c440 tx=0x7f939400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f9393fff640 1 -- 172.21.15.80:0/1817769440 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93940074e0 con 0x7f939c163de0 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f9393fff640 1 -- 172.21.15.80:0/1817769440 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93940047f0 con 0x7f939c163de0 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939c142ff0 con 0x7f939c163de0 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f9393fff640 1 -- 172.21.15.80:0/1817769440 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9394004be0 con 0x7f939c163de0 2024-04-25T13:42:02.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.333+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f939c161f60 con 0x7f939c163de0 2024-04-25T13:42:02.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.337+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939c06a3a0 con 0x7f939c163de0 2024-04-25T13:42:02.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.337+0000 7f9393fff640 1 -- 172.21.15.80:0/1817769440 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9394011070 con 0x7f939c163de0 2024-04-25T13:42:02.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.337+0000 7f9393fff640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f937c06f400 0x7f937c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.337+0000 7f9393fff640 1 -- 172.21.15.80:0/1817769440 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f939408ee60 con 0x7f939c163de0 2024-04-25T13:42:02.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.337+0000 7f939b7fe640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f937c06f400 0x7f937c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.337+0000 7f939b7fe640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f937c06f400 0x7f937c071910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f938400ba10 tx=0x7f9384008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:02.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.341+0000 7f9393fff640 1 -- 172.21.15.80:0/1817769440 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f939405bd50 con 0x7f939c163de0 2024-04-25T13:42:02.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.529+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f939c142aa0 con 0x7f939c163de0 2024-04-25T13:42:02.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.529+0000 7f9393fff640 1 -- 172.21.15.80:0/1817769440 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5175 (secure 0 0 0) 0x7f939405b650 con 0x7f939c163de0 2024-04-25T13:42:02.533 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:02.534 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:41:32.974439+0000","modified":"2024-04-25T13:41:36.017164+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5190},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5190":{"gid":5190,"name":"b","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6835/966995060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":966995060},{"type":"v1","addr":"172.21.15.87:6835","nonce":966995060}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5190,"qdb_cluster":[5190]},"id":2}]} 2024-04-25T13:42:02.534 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 16 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f937c06f400 msgr2=0x7f937c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f937c06f400 0x7f937c071910 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f938400ba10 tx=0x7f9384008040 comp rx=0 tx=0).stop 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c163de0 msgr2=0x7f939c13ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c163de0 0x7f939c13ce30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f939400c440 tx=0x7f939400c900 comp rx=0 tx=0).stop 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 shutdown_connections 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f937c06f400 0x7f937c071910 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f939c163de0 0x7f939c13ce30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f939c149410 0x7f939c13c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 --2- 172.21.15.80:0/1817769440 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f939c0b2020 0x7f939c15f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 >> 172.21.15.80:0/1817769440 conn(0x7f939c0d28f0 msgr2=0x7f939c0af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 shutdown_connections 2024-04-25T13:42:02.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.533+0000 7f93a2148640 1 -- 172.21.15.80:0/1817769440 wait complete. 2024-04-25T13:42:02.554 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:02.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.697+0000 7fc9ac7a5640 1 Processor -- start 2024-04-25T13:42:02.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.697+0000 7fc9ac7a5640 1 -- start start 2024-04-25T13:42:02.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9ac7a5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 0x7fc9a4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9ac7a5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc9a40733b0 0x7fc9a4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9ac7a5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9a406fcf0 con 0x7fc9a4161070 2024-04-25T13:42:02.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9ac7a5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc9a41619c0 con 0x7fc9a40733b0 2024-04-25T13:42:02.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9ac7a5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a4073f10 con 0x7fc9a406f5a0 2024-04-25T13:42:02.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9aad42640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc9a40733b0 0x7fc9a4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9a9d40640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc9a4161070 0x7fc9a40b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57430/0 (socket says 172.21.15.80:57430) 2024-04-25T13:42:02.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9a9d40640 1 -- 172.21.15.80:0/3166864129 learned_addr learned my addr 172.21.15.80:0/3166864129 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:02.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9aa541640 1 --2- 172.21.15.80:0/3166864129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 0x7fc9a4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9aad42640 1 -- 172.21.15.80:0/3166864129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 msgr2=0x7fc9a4160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:02.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9aad42640 1 --2- 172.21.15.80:0/3166864129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 0x7fc9a4160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9aad42640 1 -- 172.21.15.80:0/3166864129 >> v1:172.21.15.80:6789/0 conn(0x7fc9a4161070 legacy=0x7fc9a40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9a953f640 1 -- 172.21.15.80:0/3166864129 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3891092364 0 0) 0x7fc9a406fcf0 con 0x7fc9a4161070 2024-04-25T13:42:02.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.701+0000 7fc9aad42640 1 -- 172.21.15.80:0/3166864129 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a40b0c60 con 0x7fc9a40733b0 2024-04-25T13:42:02.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9aad42640 1 --2- 172.21.15.80:0/3166864129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a40733b0 0x7fc9a4161480 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009dc0 tx=0x7fc99c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d260cc2058a79e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:02.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9a953f640 1 -- 172.21.15.80:0/3166864129 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0b2070 con 0x7fc9a40733b0 2024-04-25T13:42:02.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9a953f640 1 -- 172.21.15.80:0/3166864129 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc99c0a57b0 con 0x7fc9a40733b0 2024-04-25T13:42:02.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9a953f640 1 -- 172.21.15.80:0/3166864129 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0a5b80 con 0x7fc9a40733b0 2024-04-25T13:42:02.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/3166864129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a40733b0 msgr2=0x7fc9a4161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/3166864129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a40733b0 0x7fc9a4161480 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc99c009dc0 tx=0x7fc99c01c8e0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/3166864129 shutdown_connections 2024-04-25T13:42:02.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/3166864129 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a40733b0 0x7fc9a4161480 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/3166864129 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 0x7fc9a4160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/3166864129 >> 172.21.15.80:0/3166864129 conn(0x7fc9a406d080 msgr2=0x7fc9a40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:02.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/3166864129 shutdown_connections 2024-04-25T13:42:02.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/3166864129 wait complete. 2024-04-25T13:42:02.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 Processor -- start 2024-04-25T13:42:02.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.705+0000 7fc9ac7a5640 1 -- start start 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 0x7fc9a415c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9a40733b0 0x7fc9a4142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a4161070 0x7fc9a41581f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4168040 con 0x7fc9a4161070 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc9a4167ec0 con 0x7fc9a40733b0 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc9a41681c0 con 0x7fc9a406f5a0 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a4161070 0x7fc9a41581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a4161070 0x7fc9a41581f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39314/0 (socket says 172.21.15.80:39314) 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 -- 172.21.15.80:0/2163173894 learned_addr learned my addr 172.21.15.80:0/2163173894 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:02.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aa541640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 0x7fc9a415c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9a9d40640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9a40733b0 0x7fc9a4142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 -- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 msgr2=0x7fc9a415c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 0x7fc9a415c6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 -- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9a40733b0 msgr2=0x7fc9a4142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9a40733b0 0x7fc9a4142270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 -- 172.21.15.80:0/2163173894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9a415b8f0 con 0x7fc9a4161070 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9a9d40640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9a40733b0 0x7fc9a4142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9aad42640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a4161070 0x7fc9a41581f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc99c01cdc0 tx=0x7fc99c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc99affd640 1 -- 172.21.15.80:0/2163173894 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0b2050 con 0x7fc9a4161070 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc99affd640 1 -- 172.21.15.80:0/2163173894 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc99c0be070 con 0x7fc9a4161070 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9a415bbd0 con 0x7fc9a4161070 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc99affd640 1 -- 172.21.15.80:0/2163173894 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0ad870 con 0x7fc9a4161070 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9a415c190 con 0x7fc9a4161070 2024-04-25T13:42:02.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9a415b680 con 0x7fc9a4161070 2024-04-25T13:42:02.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc99affd640 1 -- 172.21.15.80:0/2163173894 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc99c0ada10 con 0x7fc9a4161070 2024-04-25T13:42:02.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.709+0000 7fc99affd640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc98806f400 0x7fc988071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:02.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.713+0000 7fc99affd640 1 -- 172.21.15.80:0/2163173894 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc99c12acb0 con 0x7fc9a4161070 2024-04-25T13:42:02.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.713+0000 7fc9aa541640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc98806f400 0x7fc988071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:02.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.713+0000 7fc9aa541640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc98806f400 0x7fc988071910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40b6e70 tx=0x7fc9a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:02.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.713+0000 7fc99affd640 1 -- 172.21.15.80:0/2163173894 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc99c016560 con 0x7fc9a4161070 2024-04-25T13:42:02.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.877+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc9a406a3a0 con 0x7fc9a4161070 2024-04-25T13:42:02.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc99affd640 1 -- 172.21.15.80:0/2163173894 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16597 (secure 0 0 0) 0x7fc99c0f7d10 con 0x7fc9a4161070 2024-04-25T13:42:02.883 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:02.883 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":35,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:41:53.226827+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:41:30.714412+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T13:41:31.911193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:02.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc98806f400 msgr2=0x7fc988071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc98806f400 0x7fc988071910 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40b6e70 tx=0x7fc9a0008040 comp rx=0 tx=0).stop 2024-04-25T13:42:02.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a4161070 msgr2=0x7fc9a41581f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:02.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a4161070 0x7fc9a41581f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc99c01cdc0 tx=0x7fc99c019680 comp rx=0 tx=0).stop 2024-04-25T13:42:02.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 shutdown_connections 2024-04-25T13:42:02.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc98806f400 0x7fc988071910 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9a4161070 0x7fc9a41581f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9a40733b0 0x7fc9a4142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 --2- 172.21.15.80:0/2163173894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9a406f5a0 0x7fc9a415c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:02.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 >> 172.21.15.80:0/2163173894 conn(0x7fc9a406d080 msgr2=0x7fc9a40b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:02.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 shutdown_connections 2024-04-25T13:42:02.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:02.881+0000 7fc9ac7a5640 1 -- 172.21.15.80:0/2163173894 wait complete. 2024-04-25T13:42:02.906 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:42:03.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65697e9640 1 Processor -- start 2024-04-25T13:42:03.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65697e9640 1 -- start start 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65697e9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f65640cc720 0x7f6564073e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65697e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f65640cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65697e9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f65640ce010 0x7f65640ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65697e9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f656415ea50 con 0x7f65640ce010 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65697e9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f65640ce960 con 0x7f65640cc720 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65697e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65640cd6c0 con 0x7f65640743d0 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65637fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f65640ce010 0x7f65640ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65637fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f65640ce010 0x7f65640ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39324/0 (socket says 172.21.15.80:39324) 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65637fe640 1 -- 172.21.15.80:0/4156275377 learned_addr learned my addr 172.21.15.80:0/4156275377 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:03.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f6562ffd640 1 --2- 172.21.15.80:0/4156275377 >> v2:172.21.15.87:3300/0 conn(0x7f65640cc720 0x7f6564073e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65627fc640 1 --2- 172.21.15.80:0/4156275377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f65640cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65637fe640 1 -- 172.21.15.80:0/4156275377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 msgr2=0x7f65640cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:03.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65637fe640 1 --2- 172.21.15.80:0/4156275377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f65640cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65637fe640 1 -- 172.21.15.80:0/4156275377 >> v2:172.21.15.87:3300/0 conn(0x7f65640cc720 msgr2=0x7f6564073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:03.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65637fe640 1 --2- 172.21.15.80:0/4156275377 >> v2:172.21.15.87:3300/0 conn(0x7f65640cc720 0x7f6564073e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65637fe640 1 -- 172.21.15.80:0/4156275377 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65640729e0 con 0x7f65640ce010 2024-04-25T13:42:03.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.041+0000 7f65627fc640 1 --2- 172.21.15.80:0/4156275377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f65640cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:03.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65637fe640 1 --2- 172.21.15.80:0/4156275377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ce420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6564071e80 tx=0x7f654c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab23f6a8dfd559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:03.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f6561ffb640 1 -- 172.21.15.80:0/4156275377 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0ab620 con 0x7f65640ce010 2024-04-25T13:42:03.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f6561ffb640 1 -- 172.21.15.80:0/4156275377 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c0b0070 con 0x7f65640ce010 2024-04-25T13:42:03.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f6561ffb640 1 -- 172.21.15.80:0/4156275377 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0ab9e0 con 0x7f65640ce010 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 -- 172.21.15.80:0/4156275377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 msgr2=0x7f65640ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 --2- 172.21.15.80:0/4156275377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ce420 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6564071e80 tx=0x7f654c01c460 comp rx=0 tx=0).stop 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 -- 172.21.15.80:0/4156275377 shutdown_connections 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 --2- 172.21.15.80:0/4156275377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ce420 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 --2- 172.21.15.80:0/4156275377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f65640cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 --2- 172.21.15.80:0/4156275377 >> v2:172.21.15.87:3300/0 conn(0x7f65640cc720 0x7f6564073e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 -- 172.21.15.80:0/4156275377 >> 172.21.15.80:0/4156275377 conn(0x7f65640d20d0 msgr2=0x7f65640d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 -- 172.21.15.80:0/4156275377 shutdown_connections 2024-04-25T13:42:03.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.045+0000 7f65697e9640 1 -- 172.21.15.80:0/4156275377 wait complete. 2024-04-25T13:42:03.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 Processor -- start 2024-04-25T13:42:03.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 -- start start 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f656414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65640cc720 0x7f6564153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ceae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f65640ccb30 con 0x7f65640ce010 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f656406e710 con 0x7f65640cc720 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f65640747e0 con 0x7f65640743d0 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ceae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ceae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39330/0 (socket says 172.21.15.80:39330) 2024-04-25T13:42:03.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 -- 172.21.15.80:0/75781184 learned_addr learned my addr 172.21.15.80:0/75781184 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65627fc640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65640cc720 0x7f6564153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f6562ffd640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f656414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 -- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 msgr2=0x7f656414c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f656414c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 -- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65640cc720 msgr2=0x7f6564153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65640cc720 0x7f6564153170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 -- 172.21.15.80:0/75781184 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65641fab00 con 0x7f65640ce010 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65637fe640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ceae0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f654c01cd00 tx=0x7f654c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f655b7fe640 1 -- 172.21.15.80:0/75781184 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0abd20 con 0x7f65640ce010 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f655b7fe640 1 -- 172.21.15.80:0/75781184 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f654c0be3f0 con 0x7f65640ce010 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65641fade0 con 0x7f65640ce010 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f655b7fe640 1 -- 172.21.15.80:0/75781184 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0be7e0 con 0x7f65640ce010 2024-04-25T13:42:03.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65641fb3a0 con 0x7f65640ce010 2024-04-25T13:42:03.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.049+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f656406a3a0 con 0x7f65640ce010 2024-04-25T13:42:03.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.053+0000 7f655b7fe640 1 -- 172.21.15.80:0/75781184 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f654c0b43f0 con 0x7f65640ce010 2024-04-25T13:42:03.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.053+0000 7f655b7fe640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f654406f430 0x7f6544071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.053+0000 7f655b7fe640 1 -- 172.21.15.80:0/75781184 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f654c129080 con 0x7f65640ce010 2024-04-25T13:42:03.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.053+0000 7f6562ffd640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f654406f430 0x7f6544071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.053+0000 7f6562ffd640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f654406f430 0x7f6544071940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f655c003fd0 tx=0x7f655c0039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:03.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.057+0000 7f655b7fe640 1 -- 172.21.15.80:0/75781184 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f654c0c4060 con 0x7f65640ce010 2024-04-25T13:42:03.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.245+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f65641fa850 con 0x7f65640ce010 2024-04-25T13:42:03.248 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:42:03.245+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:42:03.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.245+0000 7f655b7fe640 1 -- 172.21.15.80:0/75781184 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) ==== 161+0+0 (secure 0 0 0) 0x7f654c016110 con 0x7f65640ce010 2024-04-25T13:42:03.251 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:42:03.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f654406f430 msgr2=0x7f6544071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f654406f430 0x7f6544071940 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f655c003fd0 tx=0x7f655c0039a0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 msgr2=0x7f65640ceae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ceae0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f654c01cd00 tx=0x7f654c01aca0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 shutdown_connections 2024-04-25T13:42:03.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f654406f430 0x7f6544071940 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f65640ce010 0x7f65640ceae0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f65640cc720 0x7f6564153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 --2- 172.21.15.80:0/75781184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f65640743d0 0x7f656414c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 >> 172.21.15.80:0/75781184 conn(0x7f65640d20d0 msgr2=0x7f65640af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:03.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 shutdown_connections 2024-04-25T13:42:03.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.249+0000 7f65697e9640 1 -- 172.21.15.80:0/75781184 wait complete. 2024-04-25T13:42:03.268 INFO:tasks.ceph.mds.b.smithi087.stderr: -3658> 2024-04-25T13:41:37.521+0000 7f00ad614640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:42:03.279 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:42:03.348 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:42:03.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.409+0000 7fd6e9b3a640 1 Processor -- start 2024-04-25T13:42:03.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.409+0000 7fd6e9b3a640 1 -- start start 2024-04-25T13:42:03.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e9b3a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd6e4073080 0x7fd6e4073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e9b3a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e40737f0 con 0x7fd6e415c710 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e9b3a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd6e41604d0 con 0x7fd6e406eb80 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e9b3a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd6e40b53d0 con 0x7fd6e4073080 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e2ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd6e415c710 0x7fd6e406e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57442/0 (socket says 172.21.15.80:57442) 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e2ffd640 1 -- 172.21.15.80:0/2988988696 learned_addr learned my addr 172.21.15.80:0/2988988696 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e37fe640 1 --2- 172.21.15.80:0/2988988696 >> v2:172.21.15.87:3300/0 conn(0x7fd6e4073080 0x7fd6e4073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 172011422 0 0) 0x7fd6e40737f0 con 0x7fd6e415c710 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd6cc003530 con 0x7fd6e415c710 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3314212567 0 0) 0x7fd6cc003530 con 0x7fd6e415c710 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 >> v2:172.21.15.87:3300/0 conn(0x7fd6e4073080 msgr2=0x7fd6e4073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 --2- 172.21.15.80:0/2988988696 >> v2:172.21.15.87:3300/0 conn(0x7fd6e4073080 0x7fd6e4073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 >> v1:172.21.15.87:6789/0 conn(0x7fd6e406eb80 legacy=0x7fd6e40ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e41607b0 con 0x7fd6e415c710 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fd6d8001680 con 0x7fd6e415c710 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 >> v1:172.21.15.80:6789/0 conn(0x7fd6e415c710 legacy=0x7fd6e406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc0039c0 0x7fd6cc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6cc004810 0x7fd6cc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6cc005260 0x7fd6cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e40b53d0 con 0x7fd6cc005260 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6cc00c1f0 con 0x7fd6cc004810 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6cc00d500 con 0x7fd6cc0039c0 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e3fff640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6cc005260 0x7fd6cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e37fe640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc0039c0 0x7fd6cc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e2ffd640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6cc004810 0x7fd6cc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e3fff640 1 -- 172.21.15.80:0/2988988696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc0039c0 msgr2=0x7fd6cc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:03.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e3fff640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc0039c0 0x7fd6cc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e3fff640 1 -- 172.21.15.80:0/2988988696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6cc004810 msgr2=0x7fd6cc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:03.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e3fff640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6cc004810 0x7fd6cc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e3fff640 1 -- 172.21.15.80:0/2988988696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6cc003530 con 0x7fd6cc005260 2024-04-25T13:42:03.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e3fff640 1 -- 172.21.15.80:0/2988988696 reap_dead start 2024-04-25T13:42:03.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.413+0000 7fd6e3fff640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6cc005260 0x7fd6cc00b9d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4160ad0 tx=0x7fd6dc01c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a25e5573350d0bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:03.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc001a90 con 0x7fd6cc005260 2024-04-25T13:42:03.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6dc003b80 con 0x7fd6cc005260 2024-04-25T13:42:03.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e27fc640 1 -- 172.21.15.80:0/2988988696 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc0050d0 con 0x7fd6cc005260 2024-04-25T13:42:03.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/2988988696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6cc005260 msgr2=0x7fd6cc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6cc005260 0x7fd6cc00b9d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4160ad0 tx=0x7fd6dc01c720 comp rx=0 tx=0).stop 2024-04-25T13:42:03.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/2988988696 shutdown_connections 2024-04-25T13:42:03.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 --2- 172.21.15.80:0/2988988696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6cc005260 0x7fd6cc00b9d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/2988988696 >> 172.21.15.80:0/2988988696 conn(0x7fd6e40b4160 msgr2=0x7fd6e40d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:03.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/2988988696 shutdown_connections 2024-04-25T13:42:03.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/2988988696 wait complete. 2024-04-25T13:42:03.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 Processor -- start 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 -- start start 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.417+0000 7fd6e9b3a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc005260 0x7fd6e415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e9b3a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e4152ec0 0x7fd6e414c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e9b3a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e41547f0 0x7fd6e4154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e9b3a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4073490 con 0x7fd6e4152ec0 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e9b3a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e4160b20 con 0x7fd6e41547f0 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e9b3a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e4073610 con 0x7fd6cc005260 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e4152ec0 0x7fd6e414c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e4152ec0 0x7fd6e414c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39350/0 (socket says 172.21.15.80:39350) 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 -- 172.21.15.80:0/835703936 learned_addr learned my addr 172.21.15.80:0/835703936 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e37fe640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc005260 0x7fd6e415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e3fff640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e41547f0 0x7fd6e4154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 -- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc005260 msgr2=0x7fd6e415b620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc005260 0x7fd6e415b620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 -- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e41547f0 msgr2=0x7fd6e4154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e41547f0 0x7fd6e4154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 -- 172.21.15.80:0/835703936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e4151e20 con 0x7fd6e4152ec0 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e2ffd640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e4152ec0 0x7fd6e414c0c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8011860 tx=0x7fd6d8011d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6cbfff640 1 -- 172.21.15.80:0/835703936 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8010040 con 0x7fd6e4152ec0 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6cbfff640 1 -- 172.21.15.80:0/835703936 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd6d801a070 con 0x7fd6e4152ec0 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e4152100 con 0x7fd6e4152ec0 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6cbfff640 1 -- 172.21.15.80:0/835703936 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800fd80 con 0x7fd6e4152ec0 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e41526c0 con 0x7fd6e4152ec0 2024-04-25T13:42:03.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e4151b90 con 0x7fd6e4152ec0 2024-04-25T13:42:03.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6cbfff640 1 -- 172.21.15.80:0/835703936 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd6d801f460 con 0x7fd6e4152ec0 2024-04-25T13:42:03.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6cbfff640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd6d006f400 0x7fd6d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:03.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6cbfff640 1 -- 172.21.15.80:0/835703936 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fd6d80926c0 con 0x7fd6e4152ec0 2024-04-25T13:42:03.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.421+0000 7fd6e37fe640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd6d006f400 0x7fd6d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:03.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.425+0000 7fd6e37fe640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd6d006f400 0x7fd6d0071910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4160290 tx=0x7fd6d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:03.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.429+0000 7fd6cbfff640 1 -- 172.21.15.80:0/835703936 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd6d805f550 con 0x7fd6e4152ec0 2024-04-25T13:42:03.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:03.613+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd6e406a3a0 con 0x7fd6e4152ec0 2024-04-25T13:42:04.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.285+0000 7fd6cbfff640 1 -- 172.21.15.80:0/835703936 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) ==== 104+0+0 (secure 0 0 0) 0x7fd6d805ee50 con 0x7fd6e4152ec0 2024-04-25T13:42:04.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.285+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd6d006f400 msgr2=0x7fd6d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:04.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.285+0000 7fd6e9b3a640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd6d006f400 0x7fd6d0071910 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd6e4160290 tx=0x7fd6d4008040 comp rx=0 tx=0).stop 2024-04-25T13:42:04.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.285+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e4152ec0 msgr2=0x7fd6e414c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:04.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.285+0000 7fd6e9b3a640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e4152ec0 0x7fd6e414c0c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8011860 tx=0x7fd6d8011d20 comp rx=0 tx=0).stop 2024-04-25T13:42:04.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.289+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 shutdown_connections 2024-04-25T13:42:04.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.289+0000 7fd6e9b3a640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd6d006f400 0x7fd6d0071910 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.289+0000 7fd6e9b3a640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e41547f0 0x7fd6e4154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.289+0000 7fd6e9b3a640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd6e4152ec0 0x7fd6e414c0c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.289+0000 7fd6e9b3a640 1 --2- 172.21.15.80:0/835703936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd6cc005260 0x7fd6e415b620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.289+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 >> 172.21.15.80:0/835703936 conn(0x7fd6e40b4160 msgr2=0x7fd6e40cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:04.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.289+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 shutdown_connections 2024-04-25T13:42:04.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.289+0000 7fd6e9b3a640 1 -- 172.21.15.80:0/835703936 wait complete. 2024-04-25T13:42:04.313 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:42:04.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8baa5e640 1 Processor -- start 2024-04-25T13:42:04.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8baa5e640 1 -- start start 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8baa5e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 0x7fd8b40d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8baa5e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b4164040 con 0x7fd8b40d10e0 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8baa5e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8b40d1620 con 0x7fd8b40b7df0 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8baa5e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b40d1900 con 0x7fd8b4164b00 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b37fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd8b40d10e0 0x7fd8b40b76e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57458/0 (socket says 172.21.15.80:57458) 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b37fe640 1 -- 172.21.15.80:0/3252633035 learned_addr learned my addr 172.21.15.80:0/3252633035 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b3fff640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 0x7fd8b40d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461524836 0 0) 0x7fd8b4164040 con 0x7fd8b40d10e0 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd89c003530 con 0x7fd8b40d10e0 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1969813320 0 0) 0x7fd89c003530 con 0x7fd8b40d10e0 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 msgr2=0x7fd8b40d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 0x7fd8b40d0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 >> v1:172.21.15.87:6789/0 conn(0x7fd8b40b7df0 legacy=0x7fd8b40b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b4164300 con 0x7fd8b40d10e0 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fd8a80016a0 con 0x7fd8b40d10e0 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 >> v1:172.21.15.80:6789/0 conn(0x7fd8b40d10e0 legacy=0x7fd8b40b76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 0x7fd89c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c003f70 0x7fd89c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c004d50 0x7fd89c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b40d1900 con 0x7fd89c003f70 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd89c00bc10 con 0x7fd89c004d50 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd89c00be10 con 0x7fd8b4164b00 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b37fe640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c003f70 0x7fd89c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b8ffb640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c004d50 0x7fd89c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b3fff640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 0x7fd89c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b37fe640 1 -- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 msgr2=0x7fd89c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:04.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b37fe640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 0x7fd89c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b37fe640 1 -- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c004d50 msgr2=0x7fd89c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:04.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b37fe640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c004d50 0x7fd89c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b37fe640 1 -- 172.21.15.80:0/3252633035 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89c003530 con 0x7fd89c003f70 2024-04-25T13:42:04.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.469+0000 7fd8b3fff640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b4164b00 0x7fd89c0039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:04.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8b37fe640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c003f70 0x7fd89c004810 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd8b408e9c0 tx=0x7fd8a801cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3c20c20c26ca7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:04.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8007d60 con 0x7fd89c003f70 2024-04-25T13:42:04.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a80a9070 con 0x7fd89c003f70 2024-04-25T13:42:04.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8b2ffd640 1 -- 172.21.15.80:0/3252633035 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8005d20 con 0x7fd89c003f70 2024-04-25T13:42:04.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8baa5e640 1 -- 172.21.15.80:0/3252633035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c003f70 msgr2=0x7fd89c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:04.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8baa5e640 1 --2- 172.21.15.80:0/3252633035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd89c003f70 0x7fd89c004810 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fd8b408e9c0 tx=0x7fd8a801cbe0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8b8ffb640 1 -- 172.21.15.80:0/3252633035 reap_dead start 2024-04-25T13:42:04.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8baa5e640 1 -- 172.21.15.80:0/3252633035 shutdown_connections 2024-04-25T13:42:04.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8baa5e640 1 -- 172.21.15.80:0/3252633035 >> 172.21.15.80:0/3252633035 conn(0x7fd8b4073620 msgr2=0x7fd8b40d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:04.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8baa5e640 1 -- 172.21.15.80:0/3252633035 shutdown_connections 2024-04-25T13:42:04.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.473+0000 7fd8baa5e640 1 -- 172.21.15.80:0/3252633035 wait complete. 2024-04-25T13:42:04.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 Processor -- start 2024-04-25T13:42:04.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 -- start start 2024-04-25T13:42:04.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c003f70 0x7fd8b4153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:04.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8b416b8c0 0x7fd8b41626a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:04.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b416bcd0 0x7fd8b4162be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:04.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b416c5f0 con 0x7fd8b416b8c0 2024-04-25T13:42:04.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd8b416c470 con 0x7fd89c003f70 2024-04-25T13:42:04.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd8b416c770 con 0x7fd8b416bcd0 2024-04-25T13:42:04.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8b416b8c0 0x7fd8b41626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:04.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8b416b8c0 0x7fd8b41626a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39378/0 (socket says 172.21.15.80:39378) 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 -- 172.21.15.80:0/2274837930 learned_addr learned my addr 172.21.15.80:0/2274837930 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b3fff640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c003f70 0x7fd8b4153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b8ffb640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b416bcd0 0x7fd8b4162be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 -- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b416bcd0 msgr2=0x7fd8b4162be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b416bcd0 0x7fd8b4162be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 -- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c003f70 msgr2=0x7fd8b4153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c003f70 0x7fd8b4153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 -- 172.21.15.80:0/2274837930 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8b40cd1a0 con 0x7fd8b416b8c0 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b3fff640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c003f70 0x7fd8b4153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b37fe640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8b416b8c0 0x7fd8b41626a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80983e0 tx=0x7fd89c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b0ff9640 1 -- 172.21.15.80:0/2274837930 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8007d60 con 0x7fd8b416b8c0 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b0ff9640 1 -- 172.21.15.80:0/2274837930 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b40ba1d0 con 0x7fd8b416b8c0 2024-04-25T13:42:04.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8b40cd480 con 0x7fd8b416b8c0 2024-04-25T13:42:04.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8b0ff9640 1 -- 172.21.15.80:0/2274837930 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8004e50 con 0x7fd8b416b8c0 2024-04-25T13:42:04.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.477+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8b40cda40 con 0x7fd8b416b8c0 2024-04-25T13:42:04.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.481+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8b406a440 con 0x7fd8b416b8c0 2024-04-25T13:42:04.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.481+0000 7fd8b0ff9640 1 -- 172.21.15.80:0/2274837930 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd8a8007180 con 0x7fd8b416b8c0 2024-04-25T13:42:04.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.481+0000 7fd8b0ff9640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8a006f5d0 0x7fd8a0071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:04.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.481+0000 7fd8b0ff9640 1 -- 172.21.15.80:0/2274837930 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7fd8a8128a60 con 0x7fd8b416b8c0 2024-04-25T13:42:04.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.481+0000 7fd8b3fff640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8a006f5d0 0x7fd8a0071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:04.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.481+0000 7fd8b3fff640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8a006f5d0 0x7fd8a0071ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009b60 tx=0x7fd8a4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:04.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.485+0000 7fd8b0ff9640 1 -- 172.21.15.80:0/2274837930 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd8a80168f0 con 0x7fd8b416b8c0 2024-04-25T13:42:04.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:04.637+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd8b40ccf30 con 0x7fd8b416b8c0 2024-04-25T13:42:05.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.297+0000 7fd8b0ff9640 1 -- 172.21.15.80:0/2274837930 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7fd8a80f5a60 con 0x7fd8b416b8c0 2024-04-25T13:42:05.300 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8a006f5d0 msgr2=0x7fd8a0071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8a006f5d0 0x7fd8a0071ae0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd8a4009b60 tx=0x7fd8a4008040 comp rx=0 tx=0).stop 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8b416b8c0 msgr2=0x7fd8b41626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8b416b8c0 0x7fd8b41626a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fd8a80983e0 tx=0x7fd89c005840 comp rx=0 tx=0).stop 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 shutdown_connections 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8a006f5d0 0x7fd8a0071ae0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8b416bcd0 0x7fd8b4162be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8b416b8c0 0x7fd8b41626a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 --2- 172.21.15.80:0/2274837930 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd89c003f70 0x7fd8b4153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 >> 172.21.15.80:0/2274837930 conn(0x7fd8b4073620 msgr2=0x7fd8b40b79b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 shutdown_connections 2024-04-25T13:42:05.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.301+0000 7fd8baa5e640 1 -- 172.21.15.80:0/2274837930 wait complete. 2024-04-25T13:42:05.330 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:42:05.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6e409640 1 Processor -- start 2024-04-25T13:42:05.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6e409640 1 -- start start 2024-04-25T13:42:05.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6e409640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6806f7b0 0x7fca680726b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:05.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6e409640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca6806fbc0 0x7fca68072bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:05.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6e409640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca6815bd20 0x7fca6815c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:05.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6e409640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fca680ccd50 con 0x7fca6815bd20 2024-04-25T13:42:05.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6e409640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fca680d1e00 con 0x7fca6806fbc0 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6e409640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fca6806e710 con 0x7fca6806f7b0 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6c9a6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca6815bd20 0x7fca6815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6c9a6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fca6815bd20 0x7fca6815c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39388/0 (socket says 172.21.15.80:39388) 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6c9a6640 1 -- 172.21.15.80:0/1265963256 learned_addr learned my addr 172.21.15.80:0/1265963256 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca67fff640 1 --2- 172.21.15.80:0/1265963256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6806f7b0 0x7fca680726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca677fe640 1 --2- 172.21.15.80:0/1265963256 >> v2:172.21.15.87:3300/0 conn(0x7fca6806fbc0 0x7fca68072bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6c9a6640 1 -- 172.21.15.80:0/1265963256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6806f7b0 msgr2=0x7fca680726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6c9a6640 1 --2- 172.21.15.80:0/1265963256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6806f7b0 0x7fca680726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6c9a6640 1 -- 172.21.15.80:0/1265963256 >> v2:172.21.15.87:3300/0 conn(0x7fca6806fbc0 msgr2=0x7fca68072bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6c9a6640 1 --2- 172.21.15.80:0/1265963256 >> v2:172.21.15.87:3300/0 conn(0x7fca6806fbc0 0x7fca68072bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca6c9a6640 1 -- 172.21.15.80:0/1265963256 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6806e9f0 con 0x7fca6815bd20 2024-04-25T13:42:05.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.485+0000 7fca677fe640 1 --2- 172.21.15.80:0/1265963256 >> v2:172.21.15.87:3300/0 conn(0x7fca6806fbc0 0x7fca68072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:05.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6c9a6640 1 --2- 172.21.15.80:0/1265963256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6815bd20 0x7fca6815c130 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fca6806dfb0 tx=0x7fca6001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d15606438aa5fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:05.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca66ffd640 1 -- 172.21.15.80:0/1265963256 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6000b2d0 con 0x7fca6815bd20 2024-04-25T13:42:05.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca66ffd640 1 -- 172.21.15.80:0/1265963256 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca6000b470 con 0x7fca6815bd20 2024-04-25T13:42:05.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca66ffd640 1 -- 172.21.15.80:0/1265963256 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca600a7a80 con 0x7fca6815bd20 2024-04-25T13:42:05.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 -- 172.21.15.80:0/1265963256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6815bd20 msgr2=0x7fca6815c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:05.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 --2- 172.21.15.80:0/1265963256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6815bd20 0x7fca6815c130 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fca6806dfb0 tx=0x7fca6001eb60 comp rx=0 tx=0).stop 2024-04-25T13:42:05.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 -- 172.21.15.80:0/1265963256 shutdown_connections 2024-04-25T13:42:05.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 --2- 172.21.15.80:0/1265963256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6815bd20 0x7fca6815c130 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 --2- 172.21.15.80:0/1265963256 >> v2:172.21.15.87:3300/0 conn(0x7fca6806fbc0 0x7fca68072bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 --2- 172.21.15.80:0/1265963256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6806f7b0 0x7fca680726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 -- 172.21.15.80:0/1265963256 >> 172.21.15.80:0/1265963256 conn(0x7fca680d20d0 msgr2=0x7fca680d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:05.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 -- 172.21.15.80:0/1265963256 shutdown_connections 2024-04-25T13:42:05.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.489+0000 7fca6e409640 1 -- 172.21.15.80:0/1265963256 wait complete. 2024-04-25T13:42:05.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6e409640 1 Processor -- start 2024-04-25T13:42:05.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6e409640 1 -- start start 2024-04-25T13:42:05.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6e409640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6806f7b0 0x7fca6814f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:05.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6e409640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6806fbc0 0x7fca680b6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:05.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6e409640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6815bd20 0x7fca680b6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:05.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6e409640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fca6815ca80 con 0x7fca6806f7b0 2024-04-25T13:42:05.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6e409640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca680cd480 con 0x7fca6806fbc0 2024-04-25T13:42:05.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6e409640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fca68160220 con 0x7fca6815bd20 2024-04-25T13:42:05.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6806f7b0 0x7fca6814f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:05.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6806f7b0 0x7fca6814f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39396/0 (socket says 172.21.15.80:39396) 2024-04-25T13:42:05.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 -- 172.21.15.80:0/2917311059 learned_addr learned my addr 172.21.15.80:0/2917311059 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:05.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca6c9a6640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6815bd20 0x7fca680b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:05.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca677fe640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6806fbc0 0x7fca680b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:05.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 -- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6815bd20 msgr2=0x7fca680b6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:05.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6815bd20 0x7fca680b6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 -- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6806fbc0 msgr2=0x7fca680b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:05.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6806fbc0 0x7fca680b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:05.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 -- 172.21.15.80:0/2917311059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca680b70b0 con 0x7fca6806f7b0 2024-04-25T13:42:05.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca677fe640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6806fbc0 0x7fca680b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:05.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca67fff640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6806f7b0 0x7fca6814f830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fca5800d830 tx=0x7fca5800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:05.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca64ff9640 1 -- 172.21.15.80:0/2917311059 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58018030 con 0x7fca6806f7b0 2024-04-25T13:42:05.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca64ff9640 1 -- 172.21.15.80:0/2917311059 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca5800be10 con 0x7fca6806f7b0 2024-04-25T13:42:05.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.493+0000 7fca64ff9640 1 -- 172.21.15.80:0/2917311059 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca580098b0 con 0x7fca6806f7b0 2024-04-25T13:42:05.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.497+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca680b7390 con 0x7fca6806f7b0 2024-04-25T13:42:05.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.497+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca680af240 con 0x7fca6806f7b0 2024-04-25T13:42:05.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.497+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6806a440 con 0x7fca6806f7b0 2024-04-25T13:42:05.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.497+0000 7fca64ff9640 1 -- 172.21.15.80:0/2917311059 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fca5801b070 con 0x7fca6806f7b0 2024-04-25T13:42:05.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.497+0000 7fca64ff9640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca5406f400 0x7fca54071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:05.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.497+0000 7fca64ff9640 1 -- 172.21.15.80:0/2917311059 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5357+0+0 (secure 0 0 0) 0x7fca5808f290 con 0x7fca6806f7b0 2024-04-25T13:42:05.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.497+0000 7fca677fe640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca5406f400 0x7fca54071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:05.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.497+0000 7fca677fe640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca5406f400 0x7fca54071910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca680b1d80 tx=0x7fca5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:05.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.501+0000 7fca64ff9640 1 -- 172.21.15.80:0/2917311059 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca5805c400 con 0x7fca6806f7b0 2024-04-25T13:42:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:05.669+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fca680ccf10 con 0x7fca6806f7b0 2024-04-25T13:42:06.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.309+0000 7fca64ff9640 1 -- 172.21.15.80:0/2917311059 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7fca5805bd00 con 0x7fca6806f7b0 2024-04-25T13:42:06.313 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:42:06.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca5406f400 msgr2=0x7fca54071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca5406f400 0x7fca54071910 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fca680b1d80 tx=0x7fca5c008040 comp rx=0 tx=0).stop 2024-04-25T13:42:06.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6806f7b0 msgr2=0x7fca6814f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6806f7b0 0x7fca6814f830 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fca5800d830 tx=0x7fca5800dcf0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 shutdown_connections 2024-04-25T13:42:06.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca5406f400 0x7fca54071910 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca6815bd20 0x7fca680b6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca6806fbc0 0x7fca680b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 --2- 172.21.15.80:0/2917311059 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca6806f7b0 0x7fca6814f830 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 >> 172.21.15.80:0/2917311059 conn(0x7fca680d20d0 msgr2=0x7fca680b3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:06.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 shutdown_connections 2024-04-25T13:42:06.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.313+0000 7fca6e409640 1 -- 172.21.15.80:0/2917311059 wait complete. 2024-04-25T13:42:06.340 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:42:06.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57aba07640 1 Processor -- start 2024-04-25T13:42:06.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57aba07640 1 -- start start 2024-04-25T13:42:06.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57aba07640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57a40cf940 0x7f57a40b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57aba07640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57a40b7250 0x7f57a40d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57aba07640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57a40b4cc0 con 0x7f57a40b6e40 2024-04-25T13:42:06.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57aba07640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f57a40b4fe0 con 0x7f57a40cf940 2024-04-25T13:42:06.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57aba07640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f57a40d45d0 con 0x7f57a40b7250 2024-04-25T13:42:06.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57a97a3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57a40cf940 0x7f57a40b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57a97a3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57a40cf940 0x7f57a40b6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39410/0 (socket says 172.21.15.80:39410) 2024-04-25T13:42:06.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57a97a3640 1 -- 172.21.15.80:0/2429625674 learned_addr learned my addr 172.21.15.80:0/2429625674 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:06.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57a9fa4640 1 --2- 172.21.15.80:0/2429625674 >> v2:172.21.15.87:3300/0 conn(0x7f57a40b7250 0x7f57a40d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57a97a3640 1 -- 172.21.15.80:0/2429625674 >> v2:172.21.15.87:3300/0 conn(0x7f57a40b7250 msgr2=0x7f57a40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:06.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57a97a3640 1 --2- 172.21.15.80:0/2429625674 >> v2:172.21.15.87:3300/0 conn(0x7f57a40b7250 0x7f57a40d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57a97a3640 1 -- 172.21.15.80:0/2429625674 >> v1:172.21.15.80:6789/0 conn(0x7f57a40b6e40 legacy=0x7f57a40d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.489+0000 7f57a97a3640 1 -- 172.21.15.80:0/2429625674 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a40cfd50 con 0x7f57a40cf940 2024-04-25T13:42:06.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57a97a3640 1 --2- 172.21.15.80:0/2429625674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40cf940 0x7f57a40b6900 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f57a40d3a50 tx=0x7f57a001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed95116174ffb085 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:06.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f579bfff640 1 -- 172.21.15.80:0/2429625674 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a00b0070 con 0x7f57a40cf940 2024-04-25T13:42:06.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f579bfff640 1 -- 172.21.15.80:0/2429625674 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a00ab600 con 0x7f57a40cf940 2024-04-25T13:42:06.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f579bfff640 1 -- 172.21.15.80:0/2429625674 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a00ab9d0 con 0x7f57a40cf940 2024-04-25T13:42:06.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57aba07640 1 -- 172.21.15.80:0/2429625674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40cf940 msgr2=0x7f57a40b6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57aba07640 1 --2- 172.21.15.80:0/2429625674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40cf940 0x7f57a40b6900 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f57a40d3a50 tx=0x7f57a001c490 comp rx=0 tx=0).stop 2024-04-25T13:42:06.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57aba07640 1 -- 172.21.15.80:0/2429625674 shutdown_connections 2024-04-25T13:42:06.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57aba07640 1 --2- 172.21.15.80:0/2429625674 >> v2:172.21.15.87:3300/0 conn(0x7f57a40b7250 0x7f57a40d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57aba07640 1 --2- 172.21.15.80:0/2429625674 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40cf940 0x7f57a40b6900 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57aba07640 1 -- 172.21.15.80:0/2429625674 >> 172.21.15.80:0/2429625674 conn(0x7f57a40d3ab0 msgr2=0x7f57a40d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:06.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57aba07640 1 -- 172.21.15.80:0/2429625674 shutdown_connections 2024-04-25T13:42:06.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.493+0000 7f57aba07640 1 -- 172.21.15.80:0/2429625674 wait complete. 2024-04-25T13:42:06.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 Processor -- start 2024-04-25T13:42:06.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 -- start start 2024-04-25T13:42:06.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40b6e40 0x7f57a4167bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a40b7250 0x7f57a4150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a40cf940 0x7f57a40b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40d10e0 con 0x7f57a40b6e40 2024-04-25T13:42:06.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57a40d0f60 con 0x7f57a40cf940 2024-04-25T13:42:06.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57a40d2460 con 0x7f57a40b7250 2024-04-25T13:42:06.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40b6e40 0x7f57a4167bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40b6e40 0x7f57a4167bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39414/0 (socket says 172.21.15.80:39414) 2024-04-25T13:42:06.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 -- 172.21.15.80:0/3564912080 learned_addr learned my addr 172.21.15.80:0/3564912080 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:06.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a9fa4640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a40cf940 0x7f57a40b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a8fa2640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a40b7250 0x7f57a4150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 -- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a40b7250 msgr2=0x7f57a4150d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a40b7250 0x7f57a4150d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 -- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a40cf940 msgr2=0x7f57a40b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a40cf940 0x7f57a40b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 -- 172.21.15.80:0/3564912080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57a4168dd0 con 0x7f57a40b6e40 2024-04-25T13:42:06.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a8fa2640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a40b7250 0x7f57a4150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:06.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57a97a3640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40b6e40 0x7f57a4167bc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f57a001a0c0 tx=0x7f57a001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:06.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f5799ffb640 1 -- 172.21.15.80:0/3564912080 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a00b0040 con 0x7f57a40b6e40 2024-04-25T13:42:06.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f5799ffb640 1 -- 172.21.15.80:0/3564912080 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a00bc070 con 0x7f57a40b6e40 2024-04-25T13:42:06.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57a4168530 con 0x7f57a40b6e40 2024-04-25T13:42:06.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f5799ffb640 1 -- 172.21.15.80:0/3564912080 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a00b6640 con 0x7f57a40b6e40 2024-04-25T13:42:06.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57a4168af0 con 0x7f57a40b6e40 2024-04-25T13:42:06.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.497+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57a406a3a0 con 0x7f57a40b6e40 2024-04-25T13:42:06.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.501+0000 7f5799ffb640 1 -- 172.21.15.80:0/3564912080 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57a00b67e0 con 0x7f57a40b6e40 2024-04-25T13:42:06.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.501+0000 7f5799ffb640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578806f430 0x7f5788071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.501+0000 7f5799ffb640 1 -- 172.21.15.80:0/3564912080 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f57a0129310 con 0x7f57a40b6e40 2024-04-25T13:42:06.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.501+0000 7f57a8fa2640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578806f430 0x7f5788071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.501+0000 7f57a8fa2640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578806f430 0x7f5788071940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f57a40b8f10 tx=0x7f5794009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:06.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.505+0000 7f5799ffb640 1 -- 172.21.15.80:0/3564912080 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57a0016110 con 0x7f57a40b6e40 2024-04-25T13:42:06.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.661+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f57a4071ed0 con 0x7f57a40b6e40 2024-04-25T13:42:06.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.665+0000 7f5799ffb640 1 -- 172.21.15.80:0/3564912080 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) ==== 104+0+0 (secure 0 0 0) 0x7f57a00f6500 con 0x7f57a40b6e40 2024-04-25T13:42:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578806f430 msgr2=0x7f5788071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578806f430 0x7f5788071940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f57a40b8f10 tx=0x7f5794009040 comp rx=0 tx=0).stop 2024-04-25T13:42:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40b6e40 msgr2=0x7f57a4167bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40b6e40 0x7f57a4167bc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f57a001a0c0 tx=0x7f57a001acd0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 shutdown_connections 2024-04-25T13:42:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f578806f430 0x7f5788071940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57a40cf940 0x7f57a40b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57a40b7250 0x7f57a4150d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 --2- 172.21.15.80:0/3564912080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57a40b6e40 0x7f57a4167bc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 >> 172.21.15.80:0/3564912080 conn(0x7f57a40d3ab0 msgr2=0x7f57a40b55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:06.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 shutdown_connections 2024-04-25T13:42:06.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.669+0000 7f57aba07640 1 -- 172.21.15.80:0/3564912080 wait complete. 2024-04-25T13:42:06.691 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2024-04-25T13:42:06.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c67e6640 1 Processor -- start 2024-04-25T13:42:06.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c67e6640 1 -- start start 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c67e6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f31c015e960 0x7f31c00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c67e6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f31c0073f00 0x7f31c0074310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c67e6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31c015eeb0 con 0x7f31c0157ce0 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c67e6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f31c006e9f0 con 0x7f31c015e960 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c67e6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f31c015f190 con 0x7f31c0073f00 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31bffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f31c015e960 0x7f31c00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c4d83640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f31c0157ce0 0x7f31c01580f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57484/0 (socket says 172.21.15.80:57484) 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31c4d83640 1 -- 172.21.15.80:0/2512382180 learned_addr learned my addr 172.21.15.80:0/2512382180 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31bf7fe640 1 --2- 172.21.15.80:0/2512382180 >> v2:172.21.15.87:3300/0 conn(0x7f31c0073f00 0x7f31c0074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31bffff640 1 -- 172.21.15.80:0/2512382180 >> v2:172.21.15.87:3300/0 conn(0x7f31c0073f00 msgr2=0x7f31c0074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31bffff640 1 --2- 172.21.15.80:0/2512382180 >> v2:172.21.15.87:3300/0 conn(0x7f31c0073f00 0x7f31c0074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31bffff640 1 -- 172.21.15.80:0/2512382180 >> v1:172.21.15.80:6789/0 conn(0x7f31c0157ce0 legacy=0x7f31c01580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31beffd640 1 -- 172.21.15.80:0/2512382180 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2815923787 0 0) 0x7f31c015eeb0 con 0x7f31c0157ce0 2024-04-25T13:42:06.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31bffff640 1 -- 172.21.15.80:0/2512382180 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c006e680 con 0x7f31c015e960 2024-04-25T13:42:06.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.821+0000 7f31bf7fe640 1 --2- 172.21.15.80:0/2512382180 >> v2:172.21.15.87:3300/0 conn(0x7f31c0073f00 0x7f31c0074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:06.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31bffff640 1 --2- 172.21.15.80:0/2512382180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c015e960 0x7f31c00ccee0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f31c0051160 tx=0x7f31b001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce1ce48a5d8d1a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:06.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31beffd640 1 -- 172.21.15.80:0/2512382180 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00b2070 con 0x7f31c015e960 2024-04-25T13:42:06.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31beffd640 1 -- 172.21.15.80:0/2512382180 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b00a59b0 con 0x7f31c015e960 2024-04-25T13:42:06.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31beffd640 1 -- 172.21.15.80:0/2512382180 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00a5d80 con 0x7f31c015e960 2024-04-25T13:42:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31c67e6640 1 -- 172.21.15.80:0/2512382180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c015e960 msgr2=0x7f31c00ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2512382180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c015e960 0x7f31c00ccee0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f31c0051160 tx=0x7f31b001c8d0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31c67e6640 1 -- 172.21.15.80:0/2512382180 shutdown_connections 2024-04-25T13:42:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2512382180 >> v2:172.21.15.87:3300/0 conn(0x7f31c0073f00 0x7f31c0074310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2512382180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c015e960 0x7f31c00ccee0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31c67e6640 1 -- 172.21.15.80:0/2512382180 >> 172.21.15.80:0/2512382180 conn(0x7f31c00d20d0 msgr2=0x7f31c00d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31c67e6640 1 -- 172.21.15.80:0/2512382180 shutdown_connections 2024-04-25T13:42:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.825+0000 7f31c67e6640 1 -- 172.21.15.80:0/2512382180 wait complete. 2024-04-25T13:42:06.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 Processor -- start 2024-04-25T13:42:06.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 -- start start 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c0073f00 0x7f31c0157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0157ce0 0x7f31c00af460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31c015e960 0x7f31c009d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f31c0158800 con 0x7f31c0073f00 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f31c00cd630 con 0x7f31c0157ce0 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f31c0158980 con 0x7f31c015e960 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c0073f00 0x7f31c0157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c0073f00 0x7f31c0157390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39442/0 (socket says 172.21.15.80:39442) 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 -- 172.21.15.80:0/2185033387 learned_addr learned my addr 172.21.15.80:0/2185033387 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c4d83640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31c015e960 0x7f31c009d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bf7fe640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0157ce0 0x7f31c00af460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 -- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31c015e960 msgr2=0x7f31c009d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31c015e960 0x7f31c009d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 -- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0157ce0 msgr2=0x7f31c00af460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0157ce0 0x7f31c00af460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 -- 172.21.15.80:0/2185033387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31c01562f0 con 0x7f31c0073f00 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c4d83640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31c015e960 0x7f31c009d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bffff640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c0073f00 0x7f31c0157390 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f31b001a560 tx=0x7f31b00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bcff9640 1 -- 172.21.15.80:0/2185033387 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00b2040 con 0x7f31c0073f00 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bcff9640 1 -- 172.21.15.80:0/2185033387 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31b00bf070 con 0x7f31c0073f00 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31c01565d0 con 0x7f31c0073f00 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31bcff9640 1 -- 172.21.15.80:0/2185033387 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31b00ada60 con 0x7f31c0073f00 2024-04-25T13:42:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.829+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31c0156b90 con 0x7f31c0073f00 2024-04-25T13:42:06.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.833+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31c006a3a0 con 0x7f31c0073f00 2024-04-25T13:42:06.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.833+0000 7f31bcff9640 1 -- 172.21.15.80:0/2185033387 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f31b00adc00 con 0x7f31c0073f00 2024-04-25T13:42:06.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.833+0000 7f31bcff9640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31a006f400 0x7f31a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:06.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.833+0000 7f31bcff9640 1 -- 172.21.15.80:0/2185033387 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f31b012a250 con 0x7f31c0073f00 2024-04-25T13:42:06.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.833+0000 7f31bf7fe640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31a006f400 0x7f31a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:06.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.833+0000 7f31bf7fe640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31a006f400 0x7f31a0071910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f31a8006fd0 tx=0x7f31a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:06.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:06.837+0000 7f31bcff9640 1 -- 172.21.15.80:0/2185033387 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31b0016520 con 0x7f31c0073f00 2024-04-25T13:42:07.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.037+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) -- 0x7f31c0150500 con 0x7f31c0073f00 2024-04-25T13:42:07.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.313+0000 7f31bcff9640 1 -- 172.21.15.80:0/2185033387 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v108) ==== 130+0+0 (secure 0 0 0) 0x7f31b00f74d0 con 0x7f31c0073f00 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31a006f400 msgr2=0x7f31a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31a006f400 0x7f31a0071910 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f31a8006fd0 tx=0x7f31a8008040 comp rx=0 tx=0).stop 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c0073f00 msgr2=0x7f31c0157390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c0073f00 0x7f31c0157390 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f31b001a560 tx=0x7f31b00196d0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 shutdown_connections 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31a006f400 0x7f31a0071910 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f31c015e960 0x7f31c009d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31c0157ce0 0x7f31c00af460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 --2- 172.21.15.80:0/2185033387 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31c0073f00 0x7f31c0157390 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 >> 172.21.15.80:0/2185033387 conn(0x7f31c00d20d0 msgr2=0x7f31c00d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 shutdown_connections 2024-04-25T13:42:07.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.317+0000 7f31c67e6640 1 -- 172.21.15.80:0/2185033387 wait complete. 2024-04-25T13:42:07.344 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-04-25T13:42:07.345 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2024-04-25T13:42:07.345 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:07.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34d100f640 1 Processor -- start 2024-04-25T13:42:07.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34d100f640 1 -- start start 2024-04-25T13:42:07.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34d100f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 0x7f34cc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34d100f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34cc06fcf0 con 0x7f34cc0733b0 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34d100f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f34cc0b59f0 con 0x7f34cc161070 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34d100f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc073f10 con 0x7f34cc06f5a0 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cb59e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f34cc0733b0 0x7f34cc0b52e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57486/0 (socket says 172.21.15.80:57486) 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cb59e640 1 -- 172.21.15.80:0/3081668972 learned_addr learned my addr 172.21.15.80:0/3081668972 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cad9d640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 0x7f34cc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4294864786 0 0) 0x7f34cc06fcf0 con 0x7f34cc0733b0 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f34b8003530 con 0x7f34cc0733b0 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2382287354 0 0) 0x7f34b8003530 con 0x7f34cc0733b0 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 msgr2=0x7f34cc160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 0x7f34cc160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 >> v1:172.21.15.87:6789/0 conn(0x7f34cc161070 legacy=0x7f34cc0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc0b0c60 con 0x7f34cc0733b0 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f34c4001a00 con 0x7f34cc0733b0 2024-04-25T13:42:07.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 >> v1:172.21.15.80:6789/0 conn(0x7f34cc0733b0 legacy=0x7f34cc0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34b80039f0 0x7f34b8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b8004810 0x7f34b8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 0x7f34b80052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc073f10 con 0x7f34b80039f0 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f34b800bb90 con 0x7f34b8004810 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34b800bd90 con 0x7f34cc06f5a0 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cad9d640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34b80039f0 0x7f34b8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cb59e640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 0x7f34b80052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cad9d640 1 -- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 msgr2=0x7f34b80052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34c3fff640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b8004810 0x7f34b8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cad9d640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 0x7f34b80052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cad9d640 1 -- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b8004810 msgr2=0x7f34b8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cad9d640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34b8004810 0x7f34b8004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cad9d640 1 -- 172.21.15.80:0/3081668972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b8003530 con 0x7f34b80039f0 2024-04-25T13:42:07.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.489+0000 7f34cb59e640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34cc06f5a0 0x7f34b80052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34cad9d640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34b80039f0 0x7f34b8003e60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009ab0 tx=0x7f34bc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76a698538c53e8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0096b0 con 0x7f34b80039f0 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34bc0a9450 con 0x7f34b80039f0 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34ca59c640 1 -- 172.21.15.80:0/3081668972 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0a9800 con 0x7f34b80039f0 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34d100f640 1 -- 172.21.15.80:0/3081668972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34b80039f0 msgr2=0x7f34b8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34d100f640 1 --2- 172.21.15.80:0/3081668972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34b80039f0 0x7f34b8003e60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f34bc009ab0 tx=0x7f34bc01ca70 comp rx=0 tx=0).stop 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34cb59e640 1 -- 172.21.15.80:0/3081668972 reap_dead start 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34d100f640 1 -- 172.21.15.80:0/3081668972 shutdown_connections 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34d100f640 1 -- 172.21.15.80:0/3081668972 >> 172.21.15.80:0/3081668972 conn(0x7f34cc06d080 msgr2=0x7f34cc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:07.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34d100f640 1 -- 172.21.15.80:0/3081668972 shutdown_connections 2024-04-25T13:42:07.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.493+0000 7f34d100f640 1 -- 172.21.15.80:0/3081668972 wait complete. 2024-04-25T13:42:07.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 Processor -- start 2024-04-25T13:42:07.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 -- start start 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34b80039f0 0x7f34cc159be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34cc148e00 0x7f34cc16a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34cc16a650 0x7f34cc15f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc06eca0 con 0x7f34cc148e00 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f34cc161890 con 0x7f34cc16a650 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34cc06ee20 con 0x7f34b80039f0 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34cc148e00 0x7f34cc16a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34cc148e00 0x7f34cc16a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39468/0 (socket says 172.21.15.80:39468) 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 -- 172.21.15.80:0/3047779585 learned_addr learned my addr 172.21.15.80:0/3047779585 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:07.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34cad9d640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34b80039f0 0x7f34cc159be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34cb59e640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34cc16a650 0x7f34cc15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 -- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34b80039f0 msgr2=0x7f34cc159be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34b80039f0 0x7f34cc159be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 -- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34cc16a650 msgr2=0x7f34cc15f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34cc16a650 0x7f34cc15f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 -- 172.21.15.80:0/3047779585 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34cc16aa60 con 0x7f34cc148e00 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34cb59e640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34cc16a650 0x7f34cc15f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c3fff640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34cc148e00 0x7f34cc16a110 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f34cc071650 tx=0x7f34b400d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c37fe640 1 -- 172.21.15.80:0/3047779585 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4018070 con 0x7f34cc148e00 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c37fe640 1 -- 172.21.15.80:0/3047779585 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b4003620 con 0x7f34cc148e00 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34cc156780 con 0x7f34cc148e00 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34c37fe640 1 -- 172.21.15.80:0/3047779585 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b40039f0 con 0x7f34cc148e00 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34cc156d00 con 0x7f34cc148e00 2024-04-25T13:42:07.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.497+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34cc06a3a0 con 0x7f34cc148e00 2024-04-25T13:42:07.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.501+0000 7f34c37fe640 1 -- 172.21.15.80:0/3047779585 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f34b401b070 con 0x7f34cc148e00 2024-04-25T13:42:07.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.501+0000 7f34c37fe640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34ac06f400 0x7f34ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.501+0000 7f34c37fe640 1 -- 172.21.15.80:0/3047779585 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f34b408e600 con 0x7f34cc148e00 2024-04-25T13:42:07.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.501+0000 7f34cad9d640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34ac06f400 0x7f34ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.501+0000 7f34cad9d640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34ac06f400 0x7f34ac071910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0ae000 tx=0x7f34bc0b3040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:07.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.505+0000 7f34c37fe640 1 -- 172.21.15.80:0/3047779585 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34cc16aa60 con 0x7f34cc148e00 2024-04-25T13:42:07.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.661+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f34cc069cb0 con 0x7f34cc148e00 2024-04-25T13:42:07.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34c37fe640 1 -- 172.21.15.80:0/3047779585 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+12946 (secure 0 0 0) 0x7f34b405b2c0 con 0x7f34cc148e00 2024-04-25T13:42:07.668 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:07.669 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":38,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:06.301127+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6835/966995060":"2024-04-26T13:42:03.249649+0000","172.21.15.87:6834/966995060":"2024-04-26T13:42:03.249649+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34ac06f400 msgr2=0x7f34ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34ac06f400 0x7f34ac071910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0ae000 tx=0x7f34bc0b3040 comp rx=0 tx=0).stop 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34cc148e00 msgr2=0x7f34cc16a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34cc148e00 0x7f34cc16a110 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f34cc071650 tx=0x7f34b400d900 comp rx=0 tx=0).stop 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 shutdown_connections 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34ac06f400 0x7f34ac071910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34cc16a650 0x7f34cc15f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34cc148e00 0x7f34cc16a110 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 --2- 172.21.15.80:0/3047779585 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34b80039f0 0x7f34cc159be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 >> 172.21.15.80:0/3047779585 conn(0x7f34cc06d080 msgr2=0x7f34cc0cf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 shutdown_connections 2024-04-25T13:42:07.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.665+0000 7f34d100f640 1 -- 172.21.15.80:0/3047779585 wait complete. 2024-04-25T13:42:07.689 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:42:07.690 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:42:07.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff653da3640 1 Processor -- start 2024-04-25T13:42:07.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff653da3640 1 -- start start 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff653da3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06edb0 0x7ff64c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff653da3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64c0b5740 con 0x7ff64c161b60 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff653da3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff64c06dd10 con 0x7ff64c0b5330 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff653da3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff64c06f700 con 0x7ff64c06edb0 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff652340640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff64c161b60 0x7ff64c161f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57498/0 (socket says 172.21.15.80:57498) 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff652340640 1 -- 172.21.15.80:0/4289820744 learned_addr learned my addr 172.21.15.80:0/4289820744 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff65133e640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06edb0 0x7ff64c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 894051101 0 0) 0x7ff64c0b5740 con 0x7ff64c161b60 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff638003530 con 0x7ff64c161b60 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 556263001 0 0) 0x7ff638003530 con 0x7ff64c161b60 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06edb0 msgr2=0x7ff64c06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06edb0 0x7ff64c06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 >> v1:172.21.15.87:6789/0 conn(0x7ff64c0b5330 legacy=0x7ff64c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c06dff0 con 0x7ff64c161b60 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff644001a00 con 0x7ff64c161b60 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 >> v1:172.21.15.80:6789/0 conn(0x7ff64c161b60 legacy=0x7ff64c161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6380039f0 0x7ff638003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06edb0 0x7ff638004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff638004d50 0x7ff638005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c06f700 con 0x7ff638004d50 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff63800bb90 con 0x7ff6380039f0 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff63800bd90 con 0x7ff64c06edb0 2024-04-25T13:42:07.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff652340640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff638004d50 0x7ff638005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff651b3f640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6380039f0 0x7ff638003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff65133e640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06edb0 0x7ff638004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff652340640 1 -- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06edb0 msgr2=0x7ff638004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:07.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff652340640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c06edb0 0x7ff638004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff652340640 1 -- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6380039f0 msgr2=0x7ff638003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:07.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff652340640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6380039f0 0x7ff638003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff652340640 1 -- 172.21.15.80:0/4289820744 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff638003530 con 0x7ff638004d50 2024-04-25T13:42:07.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.825+0000 7ff651b3f640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6380039f0 0x7ff638003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff652340640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff638004d50 0x7ff638005160 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff64400a830 tx=0x7ff64401fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c309c88862984b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff64400a480 con 0x7ff638004d50 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6440aa6f0 con 0x7ff638004d50 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff650b3d640 1 -- 172.21.15.80:0/4289820744 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6440aab20 con 0x7ff638004d50 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff653da3640 1 -- 172.21.15.80:0/4289820744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff638004d50 msgr2=0x7ff638005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff653da3640 1 --2- 172.21.15.80:0/4289820744 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff638004d50 0x7ff638005160 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff64400a830 tx=0x7ff64401fac0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff652340640 1 -- 172.21.15.80:0/4289820744 reap_dead start 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff653da3640 1 -- 172.21.15.80:0/4289820744 shutdown_connections 2024-04-25T13:42:07.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff653da3640 1 -- 172.21.15.80:0/4289820744 >> 172.21.15.80:0/4289820744 conn(0x7ff64c0b4160 msgr2=0x7ff64c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:07.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff653da3640 1 -- 172.21.15.80:0/4289820744 shutdown_connections 2024-04-25T13:42:07.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff653da3640 1 -- 172.21.15.80:0/4289820744 wait complete. 2024-04-25T13:42:07.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff653da3640 1 Processor -- start 2024-04-25T13:42:07.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.829+0000 7ff653da3640 1 -- start start 2024-04-25T13:42:07.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64c139260 0x7ff64c1579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c15a6c0 0x7ff64c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c15aad0 0x7ff64c0b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c06e5b0 con 0x7ff64c139260 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff64c1629b0 con 0x7ff64c15a6c0 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff64c06e890 con 0x7ff64c15aad0 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64c139260 0x7ff64c1579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64c139260 0x7ff64c1579a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39500/0 (socket says 172.21.15.80:39500) 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 -- 172.21.15.80:0/2525429373 learned_addr learned my addr 172.21.15.80:0/2525429373 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff65133e640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c15a6c0 0x7ff64c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 -- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c15aad0 msgr2=0x7ff64c0b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff652340640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c15aad0 0x7ff64c0b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c15aad0 0x7ff64c0b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 -- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c15a6c0 msgr2=0x7ff64c0b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c15a6c0 0x7ff64c0b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 -- 172.21.15.80:0/2525429373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff64c15b010 con 0x7ff64c139260 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff65133e640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c15a6c0 0x7ff64c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff651b3f640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64c139260 0x7ff64c1579a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff64800d770 tx=0x7ff64800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff6427fc640 1 -- 172.21.15.80:0/2525429373 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648005070 con 0x7ff64c139260 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff6427fc640 1 -- 172.21.15.80:0/2525429373 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff648005230 con 0x7ff64c139260 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff64c15b2f0 con 0x7ff64c139260 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff6427fc640 1 -- 172.21.15.80:0/2525429373 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff648007030 con 0x7ff64c139260 2024-04-25T13:42:07.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff64c15b8b0 con 0x7ff64c139260 2024-04-25T13:42:07.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.833+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff64c06a3a0 con 0x7ff64c139260 2024-04-25T13:42:07.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.837+0000 7ff6427fc640 1 -- 172.21.15.80:0/2525429373 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff64801a020 con 0x7ff64c139260 2024-04-25T13:42:07.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.837+0000 7ff6427fc640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff63006f400 0x7ff630071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:07.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.837+0000 7ff6427fc640 1 -- 172.21.15.80:0/2525429373 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7ff64808f140 con 0x7ff64c139260 2024-04-25T13:42:07.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.837+0000 7ff65133e640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff63006f400 0x7ff630071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:07.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.837+0000 7ff65133e640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff63006f400 0x7ff630071910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff64c1613a0 tx=0x7ff63c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:07.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.841+0000 7ff6427fc640 1 -- 172.21.15.80:0/2525429373 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff64805c4a0 con 0x7ff64c139260 2024-04-25T13:42:08.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.997+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff64c06fd00 con 0x7ff64c139260 2024-04-25T13:42:08.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.997+0000 7ff6427fc640 1 -- 172.21.15.80:0/2525429373 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7ff64805bda0 con 0x7ff64c139260 2024-04-25T13:42:08.001 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:42:08.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:07.997+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff63006f400 msgr2=0x7ff630071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:08.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff63006f400 0x7ff630071910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff64c1613a0 tx=0x7ff63c008040 comp rx=0 tx=0).stop 2024-04-25T13:42:08.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64c139260 msgr2=0x7ff64c1579a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:08.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64c139260 0x7ff64c1579a0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff64800d770 tx=0x7ff64800dc30 comp rx=0 tx=0).stop 2024-04-25T13:42:08.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 shutdown_connections 2024-04-25T13:42:08.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff63006f400 0x7ff630071910 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff64c15aad0 0x7ff64c0b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff64c15a6c0 0x7ff64c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 --2- 172.21.15.80:0/2525429373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff64c139260 0x7ff64c1579a0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 >> 172.21.15.80:0/2525429373 conn(0x7ff64c0b4160 msgr2=0x7ff64c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:08.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 shutdown_connections 2024-04-25T13:42:08.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.001+0000 7ff653da3640 1 -- 172.21.15.80:0/2525429373 wait complete. 2024-04-25T13:42:08.022 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-04-25T13:42:08.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.153+0000 7f59d1c3f640 1 Processor -- start 2024-04-25T13:42:08.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59d1c3f640 1 -- start start 2024-04-25T13:42:08.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59d1c3f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f59cc06eba0 0x7f59cc0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:08.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59d1c3f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59cc073710 con 0x7f59cc0b0ed0 2024-04-25T13:42:08.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59d1c3f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59cc0739f0 con 0x7f59cc0ce730 2024-04-25T13:42:08.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59d1c3f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f59cc073cd0 con 0x7f59cc06eba0 2024-04-25T13:42:08.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59cb7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f59cc0b0ed0 0x7f59cc15c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57502/0 (socket says 172.21.15.80:57502) 2024-04-25T13:42:08.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59cb7fe640 1 -- 172.21.15.80:0/1256364019 learned_addr learned my addr 172.21.15.80:0/1256364019 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:08.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59cbfff640 1 --2- 172.21.15.80:0/1256364019 >> v2:172.21.15.87:3300/0 conn(0x7f59cc06eba0 0x7f59cc0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:08.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4133748669 0 0) 0x7f59cc073710 con 0x7f59cc0b0ed0 2024-04-25T13:42:08.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59b4004640 con 0x7f59cc0b0ed0 2024-04-25T13:42:08.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1259890623 0 0) 0x7f59b4004640 con 0x7f59cc0b0ed0 2024-04-25T13:42:08.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 >> v2:172.21.15.87:3300/0 conn(0x7f59cc06eba0 msgr2=0x7f59cc0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:08.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 --2- 172.21.15.80:0/1256364019 >> v2:172.21.15.87:3300/0 conn(0x7f59cc06eba0 0x7f59cc0ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 >> v1:172.21.15.87:6789/0 conn(0x7f59cc0ce730 legacy=0x7f59cc06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:08.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc073fb0 con 0x7f59cc0b0ed0 2024-04-25T13:42:08.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f59bc001680 con 0x7f59cc0b0ed0 2024-04-25T13:42:08.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 >> v1:172.21.15.80:6789/0 conn(0x7f59cc0b0ed0 legacy=0x7f59cc15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:08.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59b4004ad0 0x7f59b4004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:08.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59b4005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:08.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59b40062e0 0x7f59b400cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:08.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc073cd0 con 0x7f59b4005920 2024-04-25T13:42:08.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f59b400d270 con 0x7f59b40062e0 2024-04-25T13:42:08.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f59b400d470 con 0x7f59b4004ad0 2024-04-25T13:42:08.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59caffd640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59b4005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:08.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59cb7fe640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59b4004ad0 0x7f59b4004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:08.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59cbfff640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59b40062e0 0x7f59b400cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:08.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59caffd640 1 -- 172.21.15.80:0/1256364019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59b4004ad0 msgr2=0x7f59b4004f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:08.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59caffd640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59b4004ad0 0x7f59b4004f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59caffd640 1 -- 172.21.15.80:0/1256364019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59b40062e0 msgr2=0x7f59b400cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:08.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59caffd640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59b40062e0 0x7f59b400cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59caffd640 1 -- 172.21.15.80:0/1256364019 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b4004640 con 0x7f59b4005920 2024-04-25T13:42:08.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.157+0000 7f59cbfff640 1 -- 172.21.15.80:0/1256364019 reap_dead start 2024-04-25T13:42:08.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59caffd640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59b4005d30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f59c0001780 tx=0x7f59c001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4967093cf30a82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:08.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c00013a0 con 0x7f59b4005920 2024-04-25T13:42:08.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59c00052f0 con 0x7f59b4005920 2024-04-25T13:42:08.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59ca7fc640 1 -- 172.21.15.80:0/1256364019 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0003840 con 0x7f59b4005920 2024-04-25T13:42:08.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1256364019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 msgr2=0x7f59b4005d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:08.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59d1c3f640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59b4005d30 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f59c0001780 tx=0x7f59c001c400 comp rx=0 tx=0).stop 2024-04-25T13:42:08.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1256364019 shutdown_connections 2024-04-25T13:42:08.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59d1c3f640 1 --2- 172.21.15.80:0/1256364019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59b4005d30 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1256364019 >> 172.21.15.80:0/1256364019 conn(0x7f59cc06d080 msgr2=0x7f59cc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:08.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1256364019 shutdown_connections 2024-04-25T13:42:08.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.161+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1256364019 wait complete. 2024-04-25T13:42:08.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 Processor -- start 2024-04-25T13:42:08.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 -- start start 2024-04-25T13:42:08.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59cc1f3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:08.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59cc15afc0 0x7f59cc15b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:08.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59cc15ea30 0x7f59cc15ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:08.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc0ceb40 con 0x7f59b4005920 2024-04-25T13:42:08.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f59cc0733a0 con 0x7f59cc15ea30 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f59cc0cecc0 con 0x7f59cc15afc0 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59cc1f3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59cc1f3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39526/0 (socket says 172.21.15.80:39526) 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 -- 172.21.15.80:0/1797679866 learned_addr learned my addr 172.21.15.80:0/1797679866 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59caffd640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59cc15afc0 0x7f59cc15b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cbfff640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59cc15ea30 0x7f59cc15ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 -- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59cc15afc0 msgr2=0x7f59cc15b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59cc15afc0 0x7f59cc15b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 -- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59cc15ea30 msgr2=0x7f59cc15ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:08.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59cc15ea30 0x7f59cc15ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 -- 172.21.15.80:0/1797679866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59cc15f380 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59cb7fe640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59cc1f3f70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f59bc001210 tx=0x7f59bc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59b3fff640 1 -- 172.21.15.80:0/1797679866 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00ee60 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59b3fff640 1 -- 172.21.15.80:0/1797679866 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59bc018070 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59cc15bcb0 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59b3fff640 1 -- 172.21.15.80:0/1797679866 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00cae0 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.165+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59cc141d60 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.169+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59cc06a3a0 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.169+0000 7f59b3fff640 1 -- 172.21.15.80:0/1797679866 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f59bc00cc80 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.169+0000 7f59b3fff640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59b806f430 0x7f59b8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.169+0000 7f59b3fff640 1 -- 172.21.15.80:0/1797679866 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f59bc08eb90 con 0x7f59b4005920 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.169+0000 7f59caffd640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59b806f430 0x7f59b8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:08.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.169+0000 7f59caffd640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59b806f430 0x7f59b8071940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f59c0004000 tx=0x7f59c00011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:08.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.173+0000 7f59b3fff640 1 -- 172.21.15.80:0/1797679866 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59cc15f380 con 0x7f59b4005920 2024-04-25T13:42:08.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:08.385+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) -- 0x7f59cc06f880 con 0x7f59b4005920 2024-04-25T13:42:09.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.325+0000 7f59b3fff640 1 -- 172.21.15.80:0/1797679866 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v110) ==== 145+0+0 (secure 0 0 0) 0x7f59bc05bce0 con 0x7f59b4005920 2024-04-25T13:42:09.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.325+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59b806f430 msgr2=0x7f59b8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.325+0000 7f59d1c3f640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59b806f430 0x7f59b8071940 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f59c0004000 tx=0x7f59c00011a0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.325+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 msgr2=0x7f59cc1f3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.325+0000 7f59d1c3f640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59cc1f3f70 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f59bc001210 tx=0x7f59bc00ec10 comp rx=0 tx=0).stop 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.325+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 shutdown_connections 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.329+0000 7f59d1c3f640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59b806f430 0x7f59b8071940 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.329+0000 7f59d1c3f640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59cc15ea30 0x7f59cc15ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.329+0000 7f59d1c3f640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59cc15afc0 0x7f59cc15b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.329+0000 7f59d1c3f640 1 --2- 172.21.15.80:0/1797679866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59b4005920 0x7f59cc1f3f70 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.329+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 >> 172.21.15.80:0/1797679866 conn(0x7f59cc06d080 msgr2=0x7f59cc0b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:09.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.329+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 shutdown_connections 2024-04-25T13:42:09.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.329+0000 7f59d1c3f640 1 -- 172.21.15.80:0/1797679866 wait complete. 2024-04-25T13:42:09.356 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-25T13:42:09.522 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_objectstore":"memstore"} 2024-04-25T13:42:09.522 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-25T13:42:09.522 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:42:09.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe350793640 1 Processor -- start 2024-04-25T13:42:09.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe350793640 1 -- start start 2024-04-25T13:42:09.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe350793640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe348162270 0x7fe34806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:09.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe350793640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3480b5740 con 0x7fe3480b5330 2024-04-25T13:42:09.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe350793640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe34806dd10 con 0x7fe34806edb0 2024-04-25T13:42:09.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe350793640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe34806f700 con 0x7fe348162270 2024-04-25T13:42:09.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34e52f640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe3480b5330 0x7fe3480d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57512/0 (socket says 172.21.15.80:57512) 2024-04-25T13:42:09.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34e52f640 1 -- 172.21.15.80:0/2200745489 learned_addr learned my addr 172.21.15.80:0/2200745489 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:09.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 817929214 0 0) 0x7fe3480b5740 con 0x7fe3480b5330 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34ed30640 1 --2- 172.21.15.80:0/2200745489 >> v2:172.21.15.87:3300/0 conn(0x7fe348162270 0x7fe34806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe334003530 con 0x7fe3480b5330 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3464605857 0 0) 0x7fe334003530 con 0x7fe3480b5330 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 >> v2:172.21.15.87:3300/0 conn(0x7fe348162270 msgr2=0x7fe34806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 --2- 172.21.15.80:0/2200745489 >> v2:172.21.15.87:3300/0 conn(0x7fe348162270 0x7fe34806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 >> v1:172.21.15.87:6789/0 conn(0x7fe34806edb0 legacy=0x7fe348161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34806dff0 con 0x7fe3480b5330 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fe338001680 con 0x7fe3480b5330 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 >> v1:172.21.15.80:6789/0 conn(0x7fe3480b5330 legacy=0x7fe3480d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe334003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe334004810 0x7fe334004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe334005260 0x7fe33400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe34806f700 con 0x7fe3340039c0 2024-04-25T13:42:09.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe33400c1f0 con 0x7fe334004810 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe33400c3f0 con 0x7fe334005260 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34e52f640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe334003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34dd2e640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe334004810 0x7fe334004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34e52f640 1 -- 172.21.15.80:0/2200745489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe334005260 msgr2=0x7fe33400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34e52f640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe334005260 0x7fe33400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34e52f640 1 -- 172.21.15.80:0/2200745489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe334004810 msgr2=0x7fe334004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34e52f640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe334004810 0x7fe334004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34e52f640 1 -- 172.21.15.80:0/2200745489 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe334003530 con 0x7fe3340039c0 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34ed30640 1 -- 172.21.15.80:0/2200745489 reap_dead start 2024-04-25T13:42:09.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.669+0000 7fe34dd2e640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe334004810 0x7fe334004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:09.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe34e52f640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe334003e30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c240 tx=0x7fe33801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf36b388e3da66d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:09.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338007d20 con 0x7fe3340039c0 2024-04-25T13:42:09.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3380a9070 con 0x7fe3340039c0 2024-04-25T13:42:09.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe34d52d640 1 -- 172.21.15.80:0/2200745489 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338005da0 con 0x7fe3340039c0 2024-04-25T13:42:09.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe350793640 1 -- 172.21.15.80:0/2200745489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 msgr2=0x7fe334003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe350793640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe334003e30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c240 tx=0x7fe33801cc40 comp rx=0 tx=0).stop 2024-04-25T13:42:09.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe350793640 1 -- 172.21.15.80:0/2200745489 shutdown_connections 2024-04-25T13:42:09.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe350793640 1 --2- 172.21.15.80:0/2200745489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe334003e30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe350793640 1 -- 172.21.15.80:0/2200745489 >> 172.21.15.80:0/2200745489 conn(0x7fe3480b4160 msgr2=0x7fe3480d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:09.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe350793640 1 -- 172.21.15.80:0/2200745489 shutdown_connections 2024-04-25T13:42:09.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.673+0000 7fe350793640 1 -- 172.21.15.80:0/2200745489 wait complete. 2024-04-25T13:42:09.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 Processor -- start 2024-04-25T13:42:09.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 -- start start 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe34815e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3481ec780 0x7fe3481eb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3481eb760 0x7fe3481ebb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe3480b0e50 con 0x7fe3340039c0 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe3480b0cd0 con 0x7fe3481ec780 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe348161b60 con 0x7fe3481eb760 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe34815e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe34815e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39542/0 (socket says 172.21.15.80:39542) 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 -- 172.21.15.80:0/2138622494 learned_addr learned my addr 172.21.15.80:0/2138622494 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:09.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34ed30640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3481eb760 0x7fe3481ebb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34dd2e640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3481ec780 0x7fe3481eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 -- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3481eb760 msgr2=0x7fe3481ebb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3481eb760 0x7fe3481ebb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 -- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3481ec780 msgr2=0x7fe3481eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3481ec780 0x7fe3481eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 -- 172.21.15.80:0/2138622494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34815e840 con 0x7fe3340039c0 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe34e52f640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe34815e010 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe33801ca70 tx=0x7fe334005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe33effd640 1 -- 172.21.15.80:0/2138622494 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe338007180 con 0x7fe3340039c0 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe33effd640 1 -- 172.21.15.80:0/2138622494 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe338007320 con 0x7fe3340039c0 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34815eb20 con 0x7fe3340039c0 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe33effd640 1 -- 172.21.15.80:0/2138622494 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3380076d0 con 0x7fe3340039c0 2024-04-25T13:42:09.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3481f8d90 con 0x7fe3340039c0 2024-04-25T13:42:09.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.677+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34815f040 con 0x7fe3340039c0 2024-04-25T13:42:09.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.681+0000 7fe33effd640 1 -- 172.21.15.80:0/2138622494 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe3380047f0 con 0x7fe3340039c0 2024-04-25T13:42:09.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.681+0000 7fe33effd640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe32c06f400 0x7fe32c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:09.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.681+0000 7fe33effd640 1 -- 172.21.15.80:0/2138622494 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fe3381286d0 con 0x7fe3340039c0 2024-04-25T13:42:09.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.681+0000 7fe34dd2e640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe32c06f400 0x7fe32c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:09.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.681+0000 7fe34dd2e640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe32c06f400 0x7fe32c071910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3481ecec0 tx=0x7fe344009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:09.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.685+0000 7fe33effd640 1 -- 172.21.15.80:0/2138622494 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe338016950 con 0x7fe3340039c0 2024-04-25T13:42:09.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.833+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fe34806a3a0 con 0x7fe3340039c0 2024-04-25T13:42:09.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe33effd640 1 -- 172.21.15.80:0/2138622494 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) ==== 122+0+0 (secure 0 0 0) 0x7fe3380f59e0 con 0x7fe3340039c0 2024-04-25T13:42:09.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe32c06f400 msgr2=0x7fe32c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe32c06f400 0x7fe32c071910 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3481ecec0 tx=0x7fe344009040 comp rx=0 tx=0).stop 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 msgr2=0x7fe34815e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe34815e010 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe33801ca70 tx=0x7fe334005840 comp rx=0 tx=0).stop 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 shutdown_connections 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe32c06f400 0x7fe32c071910 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3481eb760 0x7fe3481ebb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3481ec780 0x7fe3481eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 --2- 172.21.15.80:0/2138622494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3340039c0 0x7fe34815e010 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 >> 172.21.15.80:0/2138622494 conn(0x7fe3480b4160 msgr2=0x7fe348161ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 shutdown_connections 2024-04-25T13:42:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:09.837+0000 7fe350793640 1 -- 172.21.15.80:0/2138622494 wait complete. 2024-04-25T13:42:09.860 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:42:09.860 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:42:09.896 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:42:09.944 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:42:09.945 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:09.945 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:09.950 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:42:09.950 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:42:09.950 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:09.956 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:42:09.957 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:09.957 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:09.995 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:42:09.996 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:42:09.996 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:10.003 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:42:10.003 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:42:10.003 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:10.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e51e1a640 1 Processor -- start 2024-04-25T13:42:10.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e51e1a640 1 -- start start 2024-04-25T13:42:10.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e51e1a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9e4c0cc930 0x7f9e4c160260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e51e1a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c160bb0 0x7f9e4c15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e51e1a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e4c0ccd40 con 0x7f9e4c1607a0 2024-04-25T13:42:10.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e51e1a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9e4c0d12d0 con 0x7f9e4c0cc930 2024-04-25T13:42:10.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e51e1a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c0d15b0 con 0x7f9e4c160bb0 2024-04-25T13:42:10.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9e4c1607a0 0x7f9e4c06e950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:57518/0 (socket says 172.21.15.80:57518) 2024-04-25T13:42:10.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9e4c0cc930 0x7f9e4c160260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4affd640 1 -- 172.21.15.80:0/1448896641 learned_addr learned my addr 172.21.15.80:0/1448896641 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:10.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4bfff640 1 --2- 172.21.15.80:0/1448896641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c160bb0 0x7f9e4c15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4b7fe640 1 -- 172.21.15.80:0/1448896641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c160bb0 msgr2=0x7f9e4c15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:10.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4b7fe640 1 --2- 172.21.15.80:0/1448896641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c160bb0 0x7f9e4c15cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4b7fe640 1 -- 172.21.15.80:0/1448896641 >> v1:172.21.15.80:6789/0 conn(0x7f9e4c1607a0 legacy=0x7f9e4c06e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4b7fe640 1 -- 172.21.15.80:0/1448896641 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c0d1890 con 0x7f9e4c0cc930 2024-04-25T13:42:10.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.185+0000 7f9e4a7fc640 1 -- 172.21.15.80:0/1448896641 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3396427339 0 0) 0x7f9e4c0ccd40 con 0x7f9e4c1607a0 2024-04-25T13:42:10.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e4b7fe640 1 --2- 172.21.15.80:0/1448896641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c0cc930 0x7f9e4c160260 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9e440097c0 tx=0x7f9e4401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f6f2de21ad8d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:10.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e4a7fc640 1 -- 172.21.15.80:0/1448896641 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e440b0070 con 0x7f9e4c0cc930 2024-04-25T13:42:10.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e4a7fc640 1 -- 172.21.15.80:0/1448896641 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e440ab600 con 0x7f9e4c0cc930 2024-04-25T13:42:10.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e4a7fc640 1 -- 172.21.15.80:0/1448896641 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e440ab9d0 con 0x7f9e4c0cc930 2024-04-25T13:42:10.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e51e1a640 1 -- 172.21.15.80:0/1448896641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c0cc930 msgr2=0x7f9e4c160260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/1448896641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c0cc930 0x7f9e4c160260 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f9e440097c0 tx=0x7f9e4401c410 comp rx=0 tx=0).stop 2024-04-25T13:42:10.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e51e1a640 1 -- 172.21.15.80:0/1448896641 shutdown_connections 2024-04-25T13:42:10.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/1448896641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c160bb0 0x7f9e4c15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/1448896641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c0cc930 0x7f9e4c160260 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e51e1a640 1 -- 172.21.15.80:0/1448896641 >> 172.21.15.80:0/1448896641 conn(0x7f9e4c0d20d0 msgr2=0x7f9e4c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:10.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e51e1a640 1 -- 172.21.15.80:0/1448896641 shutdown_connections 2024-04-25T13:42:10.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.189+0000 7f9e51e1a640 1 -- 172.21.15.80:0/1448896641 wait complete. 2024-04-25T13:42:10.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 Processor -- start 2024-04-25T13:42:10.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 -- start start 2024-04-25T13:42:10.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e4c0cc930 0x7f9e4c1ff130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c1607a0 0x7f9e4c0b6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c160bb0 0x7f9e4c0b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c06f870 con 0x7f9e4c160bb0 2024-04-25T13:42:10.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9e4c15d680 con 0x7f9e4c0cc930 2024-04-25T13:42:10.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9e4c06f9f0 con 0x7f9e4c1607a0 2024-04-25T13:42:10.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c160bb0 0x7f9e4c0b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c160bb0 0x7f9e4c0b2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39564/0 (socket says 172.21.15.80:39564) 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 -- 172.21.15.80:0/2063450768 learned_addr learned my addr 172.21.15.80:0/2063450768 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4affd640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c1607a0 0x7f9e4c0b6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4b7fe640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e4c0cc930 0x7f9e4c1ff130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 -- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c1607a0 msgr2=0x7f9e4c0b6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c1607a0 0x7f9e4c0b6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 -- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e4c0cc930 msgr2=0x7f9e4c1ff130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e4c0cc930 0x7f9e4c1ff130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 -- 172.21.15.80:0/2063450768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e4c1ff670 con 0x7f9e4c160bb0 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e4bfff640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c160bb0 0x7f9e4c0b2ba0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00d500 tx=0x7f9e3c00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e43fff640 1 -- 172.21.15.80:0/2063450768 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0125f0 con 0x7f9e4c160bb0 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e43fff640 1 -- 172.21.15.80:0/2063450768 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e3c019070 con 0x7f9e4c160bb0 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e4c0b6ed0 con 0x7f9e4c160bb0 2024-04-25T13:42:10.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e43fff640 1 -- 172.21.15.80:0/2063450768 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0129d0 con 0x7f9e4c160bb0 2024-04-25T13:42:10.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.193+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e4c0b7490 con 0x7f9e4c160bb0 2024-04-25T13:42:10.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.197+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e4c06a3a0 con 0x7f9e4c160bb0 2024-04-25T13:42:10.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.197+0000 7f9e43fff640 1 -- 172.21.15.80:0/2063450768 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e3c012b70 con 0x7f9e4c160bb0 2024-04-25T13:42:10.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.197+0000 7f9e43fff640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2c06f400 0x7f9e2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.197+0000 7f9e43fff640 1 -- 172.21.15.80:0/2063450768 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f9e3c08f070 con 0x7f9e4c160bb0 2024-04-25T13:42:10.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.197+0000 7f9e4b7fe640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2c06f400 0x7f9e2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.197+0000 7f9e4b7fe640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2c06f400 0x7f9e2c071910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e440ad000 tx=0x7f9e440a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:10.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.201+0000 7f9e43fff640 1 -- 172.21.15.80:0/2063450768 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9e3c023020 con 0x7f9e4c160bb0 2024-04-25T13:42:10.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.393+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9e4c0b6c60 con 0x7f9e4c160bb0 2024-04-25T13:42:10.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e43fff640 1 -- 172.21.15.80:0/2063450768 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+3866 (secure 0 0 0) 0x7f9e3c05c260 con 0x7f9e4c160bb0 2024-04-25T13:42:10.400 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:10.400 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1057323506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1057323506},{"type":"v1","addr":"172.21.15.87:6837","nonce":1057323506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5104,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/4226119081","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":4226119081},{"type":"v1","addr":"172.21.15.87:6835","nonce":4226119081}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":18},{"gid":5172,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3564491675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3564491675},{"type":"v1","addr":"172.21.15.80:6833","nonce":3564491675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5175,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/934847458","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":934847458},{"type":"v1","addr":"172.21.15.80:6835","nonce":934847458}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T13:42:10.400 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 18 2024-04-25T13:42:10.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2c06f400 msgr2=0x7f9e2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2c06f400 0x7f9e2c071910 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9e440ad000 tx=0x7f9e440a6000 comp rx=0 tx=0).stop 2024-04-25T13:42:10.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c160bb0 msgr2=0x7f9e4c0b2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c160bb0 0x7f9e4c0b2ba0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00d500 tx=0x7f9e3c00d9c0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 shutdown_connections 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2c06f400 0x7f9e2c071910 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e4c160bb0 0x7f9e4c0b2ba0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e4c1607a0 0x7f9e4c0b6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 --2- 172.21.15.80:0/2063450768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e4c0cc930 0x7f9e4c1ff130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 >> 172.21.15.80:0/2063450768 conn(0x7f9e4c0d20d0 msgr2=0x7f9e4c0aeec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 shutdown_connections 2024-04-25T13:42:10.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.397+0000 7f9e51e1a640 1 -- 172.21.15.80:0/2063450768 wait complete. 2024-04-25T13:42:10.419 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:42:10.420 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:42:10.420 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:42:10.420 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:42:10.420 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:42:10.420 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:42:10.420 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:42:10.420 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:42:10.421 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:42:10.421 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:42:10.421 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:42:10.421 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:42:10.421 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:42:10.421 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:42:10.421 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:42:10.422 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:42:10.432 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:42:10.432 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:42:10.433 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:42:10.434 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:42:10.439 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:42:10.440 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:42:10.444 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:42:10.444 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:42:10.477 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:42:10.477 DEBUG:teuthology.parallel:result is None 2024-04-25T13:42:10.478 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:42:10.478 DEBUG:teuthology.parallel:result is None 2024-04-25T13:42:10.487 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:42:10.488 DEBUG:teuthology.parallel:result is None 2024-04-25T13:42:10.489 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:42:10.489 DEBUG:teuthology.parallel:result is None 2024-04-25T13:42:10.489 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:42:10.561 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:42:10.566 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:42:10.585 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:42:10.586 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:42:10.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.625+0000 7fd8219bf640 1 Processor -- start 2024-04-25T13:42:10.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.625+0000 7fd8219bf640 1 -- start start 2024-04-25T13:42:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd8219bf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d26d0 0x7fd81c0d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd8219bf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd81c0d46a0 0x7fd81c0d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd8219bf640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd81c0b9210 con 0x7fd81c0d14a0 2024-04-25T13:42:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd8219bf640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd81c0d18b0 con 0x7fd81c0d46a0 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd8219bf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd81c0b5120 con 0x7fd81c0d26d0 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81a7fc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d26d0 0x7fd81c0d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81b7fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd81c0d46a0 0x7fd81c0d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81a7fc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d26d0 0x7fd81c0d2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:54604/0 (socket says 172.21.15.80:54604) 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81a7fc640 1 -- 172.21.15.80:0/836573332 learned_addr learned my addr 172.21.15.80:0/836573332 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81affd640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fd81c0d14a0 0x7fd81c0b4860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:45676/0 (socket says 172.21.15.80:45676) 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81a7fc640 1 -- 172.21.15.80:0/836573332 >> v2:172.21.15.87:3300/0 conn(0x7fd81c0d46a0 msgr2=0x7fd81c0d4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81a7fc640 1 --2- 172.21.15.80:0/836573332 >> v2:172.21.15.87:3300/0 conn(0x7fd81c0d46a0 0x7fd81c0d4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81a7fc640 1 -- 172.21.15.80:0/836573332 >> v1:172.21.15.87:6789/0 conn(0x7fd81c0d14a0 legacy=0x7fd81c0b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81a7fc640 1 -- 172.21.15.80:0/836573332 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c0d3020 con 0x7fd81c0d26d0 2024-04-25T13:42:10.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81b7fe640 1 --2- 172.21.15.80:0/836573332 >> v2:172.21.15.87:3300/0 conn(0x7fd81c0d46a0 0x7fd81c0d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:10.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd81a7fc640 1 --2- 172.21.15.80:0/836573332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d26d0 0x7fd81c0d2ae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd810006fd0 tx=0x7fd81001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=533ffb694e2704fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd819ffb640 1 -- 172.21.15.80:0/836573332 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8100b0070 con 0x7fd81c0d26d0 2024-04-25T13:42:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd819ffb640 1 -- 172.21.15.80:0/836573332 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8100ab600 con 0x7fd81c0d26d0 2024-04-25T13:42:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.629+0000 7fd819ffb640 1 -- 172.21.15.80:0/836573332 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8100ab9d0 con 0x7fd81c0d26d0 2024-04-25T13:42:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- 172.21.15.80:0/836573332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d26d0 msgr2=0x7fd81c0d2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 --2- 172.21.15.80:0/836573332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d26d0 0x7fd81c0d2ae0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fd810006fd0 tx=0x7fd81001c4b0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- 172.21.15.80:0/836573332 shutdown_connections 2024-04-25T13:42:10.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 --2- 172.21.15.80:0/836573332 >> v2:172.21.15.87:3300/0 conn(0x7fd81c0d46a0 0x7fd81c0d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 --2- 172.21.15.80:0/836573332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d26d0 0x7fd81c0d2ae0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- 172.21.15.80:0/836573332 >> 172.21.15.80:0/836573332 conn(0x7fd81c073770 msgr2=0x7fd81c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:10.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- 172.21.15.80:0/836573332 shutdown_connections 2024-04-25T13:42:10.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- 172.21.15.80:0/836573332 wait complete. 2024-04-25T13:42:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 Processor -- start 2024-04-25T13:42:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- start start 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd81c0d14a0 0x7fd81c155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd81c0d26d0 0x7fd81c161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d46a0 0x7fd81c14ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c164d20 con 0x7fd81c0d14a0 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd81c164800 con 0x7fd81c0d26d0 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd8219bf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd81c16d1d0 con 0x7fd81c0d46a0 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd81affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd81c0d14a0 0x7fd81c155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd81affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd81c0d14a0 0x7fd81c155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:39584/0 (socket says 172.21.15.80:39584) 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.633+0000 7fd81affd640 1 -- 172.21.15.80:0/2572857289 learned_addr learned my addr 172.21.15.80:0/2572857289 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81b7fe640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d46a0 0x7fd81c14ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81a7fc640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd81c0d26d0 0x7fd81c161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81affd640 1 -- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d46a0 msgr2=0x7fd81c14ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81affd640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d46a0 0x7fd81c14ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81affd640 1 -- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd81c0d26d0 msgr2=0x7fd81c161850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81affd640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd81c0d26d0 0x7fd81c161850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81affd640 1 -- 172.21.15.80:0/2572857289 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd81c161d90 con 0x7fd81c0d14a0 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81a7fc640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd81c0d26d0 0x7fd81c161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81affd640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd81c0d14a0 0x7fd81c155610 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00c3a0 tx=0x7fd80c00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd803fff640 1 -- 172.21.15.80:0/2572857289 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c0074e0 con 0x7fd81c0d14a0 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd803fff640 1 -- 172.21.15.80:0/2572857289 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd80c007680 con 0x7fd81c0d14a0 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd81c160a90 con 0x7fd81c0d14a0 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd803fff640 1 -- 172.21.15.80:0/2572857289 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd80c004a30 con 0x7fd81c0d14a0 2024-04-25T13:42:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd81c161050 con 0x7fd81c0d14a0 2024-04-25T13:42:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd81c06a3a0 con 0x7fd81c0d14a0 2024-04-25T13:42:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd803fff640 1 -- 172.21.15.80:0/2572857289 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd80c01a020 con 0x7fd81c0d14a0 2024-04-25T13:42:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd803fff640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd80806f430 0x7fd808071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd803fff640 1 -- 172.21.15.80:0/2572857289 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fd80c08e690 con 0x7fd81c0d14a0 2024-04-25T13:42:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81a7fc640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd80806f430 0x7fd808071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:10.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.637+0000 7fd81a7fc640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd80806f430 0x7fd808071940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd810006fd0 tx=0x7fd8100a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.641+0000 7fd803fff640 1 -- 172.21.15.80:0/2572857289 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd81c161d90 con 0x7fd81c0d14a0 2024-04-25T13:42:10.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:10.805+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fd81c0d4130 con 0x7fd81c0d14a0 2024-04-25T13:42:11.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd803fff640 1 -- 172.21.15.80:0/2572857289 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7fd81c0b9210 con 0x7fd81c0d14a0 2024-04-25T13:42:11.866 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:42:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd80806f430 msgr2=0x7fd808071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd80806f430 0x7fd808071940 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fd810006fd0 tx=0x7fd8100a6000 comp rx=0 tx=0).stop 2024-04-25T13:42:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd81c0d14a0 msgr2=0x7fd81c155610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd81c0d14a0 0x7fd81c155610 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd80c00c3a0 tx=0x7fd80c00c7a0 comp rx=0 tx=0).stop 2024-04-25T13:42:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 shutdown_connections 2024-04-25T13:42:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd80806f430 0x7fd808071940 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:11.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd81c0d46a0 0x7fd81c14ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd81c0d26d0 0x7fd81c161850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.861+0000 7fd8219bf640 1 --2- 172.21.15.80:0/2572857289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd81c0d14a0 0x7fd81c155610 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.865+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 >> 172.21.15.80:0/2572857289 conn(0x7fd81c073770 msgr2=0x7fd81c0b93f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.865+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 shutdown_connections 2024-04-25T13:42:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:11.865+0000 7fd8219bf640 1 -- 172.21.15.80:0/2572857289 wait complete. 2024-04-25T13:42:11.892 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:42:12.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c3146640 1 Processor -- start 2024-04-25T13:42:12.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c3146640 1 -- start start 2024-04-25T13:42:12.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c3146640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc9bc0b4ec0 0x7fc9bc0b8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c3146640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc9bc06d870 0x7fc9bc0d2d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c3146640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9bc0b7890 con 0x7fc9bc0b8e50 2024-04-25T13:42:12.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c3146640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc9bc0b8480 con 0x7fc9bc06d870 2024-04-25T13:42:12.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c3146640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc9bc0b7b50 con 0x7fc9bc0b4ec0 2024-04-25T13:42:12.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c16e3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc9bc06d870 0x7fc9bc0d2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9bbfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc9bc0b8e50 0x7fc9bc06d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:46954/0 (socket says 172.21.15.80:46954) 2024-04-25T13:42:12.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9bbfff640 1 -- 172.21.15.80:0/2629165310 learned_addr learned my addr 172.21.15.80:0/2629165310 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:12.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.041+0000 7fc9c0ee2640 1 --2- 172.21.15.80:0/2629165310 >> v2:172.21.15.87:3300/0 conn(0x7fc9bc0b4ec0 0x7fc9bc0b8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c16e3640 1 -- 172.21.15.80:0/2629165310 >> v2:172.21.15.87:3300/0 conn(0x7fc9bc0b4ec0 msgr2=0x7fc9bc0b8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:12.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c16e3640 1 --2- 172.21.15.80:0/2629165310 >> v2:172.21.15.87:3300/0 conn(0x7fc9bc0b4ec0 0x7fc9bc0b8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c16e3640 1 -- 172.21.15.80:0/2629165310 >> v1:172.21.15.80:6789/0 conn(0x7fc9bc0b8e50 legacy=0x7fc9bc06d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c16e3640 1 -- 172.21.15.80:0/2629165310 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9bc0b7e30 con 0x7fc9bc06d870 2024-04-25T13:42:12.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9bb7fe640 1 -- 172.21.15.80:0/2629165310 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3804539229 0 0) 0x7fc9bc0b7890 con 0x7fc9bc0b8e50 2024-04-25T13:42:12.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c16e3640 1 --2- 172.21.15.80:0/2629165310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc0d2d00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0b8600 tx=0x7fc9b401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792d73fd4980046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9bb7fe640 1 -- 172.21.15.80:0/2629165310 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40b2070 con 0x7fc9bc06d870 2024-04-25T13:42:12.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9bb7fe640 1 -- 172.21.15.80:0/2629165310 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b40a5760 con 0x7fc9bc06d870 2024-04-25T13:42:12.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9bb7fe640 1 -- 172.21.15.80:0/2629165310 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b40a5b30 con 0x7fc9bc06d870 2024-04-25T13:42:12.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c3146640 1 -- 172.21.15.80:0/2629165310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 msgr2=0x7fc9bc0d2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c3146640 1 --2- 172.21.15.80:0/2629165310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc0d2d00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0b8600 tx=0x7fc9b401c8a0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c3146640 1 -- 172.21.15.80:0/2629165310 shutdown_connections 2024-04-25T13:42:12.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c3146640 1 --2- 172.21.15.80:0/2629165310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc0d2d00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c3146640 1 --2- 172.21.15.80:0/2629165310 >> v2:172.21.15.87:3300/0 conn(0x7fc9bc0b4ec0 0x7fc9bc0b8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c3146640 1 -- 172.21.15.80:0/2629165310 >> 172.21.15.80:0/2629165310 conn(0x7fc9bc0d3ab0 msgr2=0x7fc9bc0d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:12.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c3146640 1 -- 172.21.15.80:0/2629165310 shutdown_connections 2024-04-25T13:42:12.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.045+0000 7fc9c3146640 1 -- 172.21.15.80:0/2629165310 wait complete. 2024-04-25T13:42:12.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 Processor -- start 2024-04-25T13:42:12.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 -- start start 2024-04-25T13:42:12.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9bc0b4ec0 0x7fc9bc1540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9bc0b8e50 0x7fc9bc0c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc16d340 con 0x7fc9bc06d870 2024-04-25T13:42:12.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc16d1c0 con 0x7fc9bc0b8e50 2024-04-25T13:42:12.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc9bc16d4c0 con 0x7fc9bc0b4ec0 2024-04-25T13:42:12.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52924/0 (socket says 172.21.15.80:52924) 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 -- 172.21.15.80:0/1311041331 learned_addr learned my addr 172.21.15.80:0/1311041331 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c16e3640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9bc0b8e50 0x7fc9bc0c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9bbfff640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9bc0b4ec0 0x7fc9bc1540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 -- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9bc0b4ec0 msgr2=0x7fc9bc1540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9bc0b4ec0 0x7fc9bc1540d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 -- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9bc0b8e50 msgr2=0x7fc9bc0c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9bc0b8e50 0x7fc9bc0c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 -- 172.21.15.80:0/1311041331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9bc15f910 con 0x7fc9bc06d870 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c16e3640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9bc0b8e50 0x7fc9bc0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c0ee2640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc153b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00d620 tx=0x7fc9ac00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9b97fa640 1 -- 172.21.15.80:0/1311041331 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00b620 con 0x7fc9bc06d870 2024-04-25T13:42:12.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9b97fa640 1 -- 172.21.15.80:0/1311041331 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9ac019070 con 0x7fc9bc06d870 2024-04-25T13:42:12.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9bc154610 con 0x7fc9bc06d870 2024-04-25T13:42:12.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9b97fa640 1 -- 172.21.15.80:0/1311041331 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ac00b9e0 con 0x7fc9bc06d870 2024-04-25T13:42:12.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.049+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9bc154bf0 con 0x7fc9bc06d870 2024-04-25T13:42:12.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.053+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9bc06a3a0 con 0x7fc9bc06d870 2024-04-25T13:42:12.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.053+0000 7fc9b97fa640 1 -- 172.21.15.80:0/1311041331 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc9ac00bb80 con 0x7fc9bc06d870 2024-04-25T13:42:12.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.053+0000 7fc9b97fa640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc99c06f430 0x7fc99c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.053+0000 7fc9b97fa640 1 -- 172.21.15.80:0/1311041331 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fc9ac08f430 con 0x7fc9bc06d870 2024-04-25T13:42:12.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.053+0000 7fc9bbfff640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc99c06f430 0x7fc99c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.053+0000 7fc9bbfff640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc99c06f430 0x7fc99c071940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0d2e00 tx=0x7fc9a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.057+0000 7fc9b97fa640 1 -- 172.21.15.80:0/1311041331 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc9ac05c5d0 con 0x7fc9bc06d870 2024-04-25T13:42:12.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc9bc0711e0 con 0x7fc9bc06d870 2024-04-25T13:42:12.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9b97fa640 1 -- 172.21.15.80:0/1311041331 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) ==== 80+0+5304 (secure 0 0 0) 0x7fc9ac05bed0 con 0x7fc9bc06d870 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.273 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:42:12.274 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:42:12.275 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.276 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:42:12.277 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.278 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.280 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:42:12.281 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.282 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:42:12.283 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.284 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:42:12.285 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:42:12.286 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.286 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:12.286 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:12.286 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc99c06f430 msgr2=0x7fc99c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc99c06f430 0x7fc99c071940 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc0d2e00 tx=0x7fc9a4009040 comp rx=0 tx=0).stop 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 msgr2=0x7fc9bc153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc153b90 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac00d620 tx=0x7fc9ac00dae0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 shutdown_connections 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc99c06f430 0x7fc99c071940 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9bc0b8e50 0x7fc9bc0c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc9bc0b4ec0 0x7fc9bc1540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 --2- 172.21.15.80:0/1311041331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc9bc06d870 0x7fc9bc153b90 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 >> 172.21.15.80:0/1311041331 conn(0x7fc9bc0d3ab0 msgr2=0x7fc9bc0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 shutdown_connections 2024-04-25T13:42:12.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.269+0000 7fc9c3146640 1 -- 172.21.15.80:0/1311041331 wait complete. 2024-04-25T13:42:12.292 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:12.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad29a07640 1 Processor -- start 2024-04-25T13:42:12.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad29a07640 1 -- start start 2024-04-25T13:42:12.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad29a07640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad240b8660 0x7fad24168f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad29a07640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad240cfb20 con 0x7fad240b9240 2024-04-25T13:42:12.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad29a07640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad240b6b70 con 0x7fad240d04e0 2024-04-25T13:42:12.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad29a07640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad240b6920 con 0x7fad240b8660 2024-04-25T13:42:12.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad22ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fad240b9240 0x7fad240cf1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:46956/0 (socket says 172.21.15.80:46956) 2024-04-25T13:42:12.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad22ffd640 1 -- 172.21.15.80:0/2138236148 learned_addr learned my addr 172.21.15.80:0/2138236148 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad237fe640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad240b8660 0x7fad24168f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3651032478 0 0) 0x7fad240cfb20 con 0x7fad240b9240 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.437+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad10003530 con 0x7fad240b9240 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1321168444 0 0) 0x7fad10003530 con 0x7fad240b9240 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad240b8660 msgr2=0x7fad24168f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad240b8660 0x7fad24168f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 >> v1:172.21.15.87:6789/0 conn(0x7fad240d04e0 legacy=0x7fad240d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad240b6e50 con 0x7fad240b9240 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fad1c001800 con 0x7fad240b9240 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 >> v1:172.21.15.80:6789/0 conn(0x7fad240b9240 legacy=0x7fad240cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad10003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad10004810 0x7fad10004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad240b8660 0x7fad100052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad240b6920 con 0x7fad100039f0 2024-04-25T13:42:12.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad1000bb90 con 0x7fad10004810 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad1000bd90 con 0x7fad240b8660 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad22ffd640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad10003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad237fe640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad240b8660 0x7fad100052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad227fc640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad10004810 0x7fad10004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad22ffd640 1 -- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad240b8660 msgr2=0x7fad100052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad22ffd640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad240b8660 0x7fad100052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad22ffd640 1 -- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad10004810 msgr2=0x7fad10004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad22ffd640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad10004810 0x7fad10004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad22ffd640 1 -- 172.21.15.80:0/2138236148 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad10003530 con 0x7fad100039f0 2024-04-25T13:42:12.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad22ffd640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad10003e60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fad24076e80 tx=0x7fad1c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ec0747c5f65c3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c007d30 con 0x7fad100039f0 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1c0a9070 con 0x7fad100039f0 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad21ffb640 1 -- 172.21.15.80:0/2138236148 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c007470 con 0x7fad100039f0 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad29a07640 1 -- 172.21.15.80:0/2138236148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 msgr2=0x7fad10003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad29a07640 1 --2- 172.21.15.80:0/2138236148 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad10003e60 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fad24076e80 tx=0x7fad1c01cc60 comp rx=0 tx=0).stop 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad237fe640 1 -- 172.21.15.80:0/2138236148 reap_dead start 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad29a07640 1 -- 172.21.15.80:0/2138236148 shutdown_connections 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad29a07640 1 -- 172.21.15.80:0/2138236148 >> 172.21.15.80:0/2138236148 conn(0x7fad240d3ad0 msgr2=0x7fad240d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad29a07640 1 -- 172.21.15.80:0/2138236148 shutdown_connections 2024-04-25T13:42:12.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.441+0000 7fad29a07640 1 -- 172.21.15.80:0/2138236148 wait complete. 2024-04-25T13:42:12.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 Processor -- start 2024-04-25T13:42:12.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 -- start start 2024-04-25T13:42:12.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad240c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad240cb510 0x7fad240cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad241660a0 0x7fad241664b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad24169b00 con 0x7fad100039f0 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad24169980 con 0x7fad240cb510 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad24169c80 con 0x7fad241660a0 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad240c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad240c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52940/0 (socket says 172.21.15.80:52940) 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 -- 172.21.15.80:0/1102281126 learned_addr learned my addr 172.21.15.80:0/1102281126 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad227fc640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad240cb510 0x7fad240cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad237fe640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad241660a0 0x7fad241664b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 -- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad241660a0 msgr2=0x7fad241664b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad241660a0 0x7fad241664b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 -- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad240cb510 msgr2=0x7fad240cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad240cb510 0x7fad240cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 -- 172.21.15.80:0/1102281126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad241669f0 con 0x7fad100039f0 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad237fe640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad241660a0 0x7fad241664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad22ffd640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad240c9740 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fad1c01c6d0 tx=0x7fad10005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad1bfff640 1 -- 172.21.15.80:0/1102281126 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0a5850 con 0x7fad100039f0 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad1bfff640 1 -- 172.21.15.80:0/1102281126 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad1c0a59f0 con 0x7fad100039f0 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad240c9c80 con 0x7fad100039f0 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad1bfff640 1 -- 172.21.15.80:0/1102281126 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1c0a5da0 con 0x7fad100039f0 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.445+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad241f98f0 con 0x7fad100039f0 2024-04-25T13:42:12.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.449+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad2406a3a0 con 0x7fad100039f0 2024-04-25T13:42:12.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.449+0000 7fad1bfff640 1 -- 172.21.15.80:0/1102281126 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad1c007090 con 0x7fad100039f0 2024-04-25T13:42:12.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.449+0000 7fad1bfff640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad0406f400 0x7fad04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.449+0000 7fad1bfff640 1 -- 172.21.15.80:0/1102281126 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fad1c1289a0 con 0x7fad100039f0 2024-04-25T13:42:12.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.449+0000 7fad227fc640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad0406f400 0x7fad04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.449+0000 7fad227fc640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad0406f400 0x7fad04071910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad2406c7c0 tx=0x7fad0c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.453+0000 7fad1bfff640 1 -- 172.21.15.80:0/1102281126 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad241669f0 con 0x7fad100039f0 2024-04-25T13:42:12.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.601+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad24076610 con 0x7fad100039f0 2024-04-25T13:42:12.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad1bfff640 1 -- 172.21.15.80:0/1102281126 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+12819 (secure 0 0 0) 0x7fad1c016970 con 0x7fad100039f0 2024-04-25T13:42:12.608 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:12.608 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":43,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:11.862675+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:12.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad0406f400 msgr2=0x7fad04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad0406f400 0x7fad04071910 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fad2406c7c0 tx=0x7fad0c009040 comp rx=0 tx=0).stop 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 msgr2=0x7fad240c9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad240c9740 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fad1c01c6d0 tx=0x7fad10005840 comp rx=0 tx=0).stop 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 shutdown_connections 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad0406f400 0x7fad04071910 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad241660a0 0x7fad241664b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad240cb510 0x7fad240cb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 --2- 172.21.15.80:0/1102281126 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad100039f0 0x7fad240c9740 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 >> 172.21.15.80:0/1102281126 conn(0x7fad240d3ad0 msgr2=0x7fad241588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 shutdown_connections 2024-04-25T13:42:12.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.605+0000 7fad29a07640 1 -- 172.21.15.80:0/1102281126 wait complete. 2024-04-25T13:42:12.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:42:12.628 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:42:12.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.773+0000 7f0aea935640 1 Processor -- start 2024-04-25T13:42:12.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.773+0000 7f0aea935640 1 -- start start 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d3ed0 0x7f0ae40d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ae40d2ad0 0x7f0ae40b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ae40b92a0 con 0x7f0ae40b7c10 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0ae40d4c10 con 0x7f0ae40d2ad0 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae40d4ef0 con 0x7f0ae40d3ed0 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ae40d2ad0 0x7f0ae40b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ae40d2ad0 0x7f0ae40b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52942/0 (socket says 172.21.15.80:52942) 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae37fe640 1 -- 172.21.15.80:0/2310488517 learned_addr learned my addr 172.21.15.80:0/2310488517 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae8ed2640 1 --1- 172.21.15.80:0/2310488517 >> v1:172.21.15.80:6789/0 conn(0x7f0ae40b7c10 0x7f0ae40b9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:46966/0 (socket says 172.21.15.80:46966) 2024-04-25T13:42:12.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae3fff640 1 --2- 172.21.15.80:0/2310488517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d3ed0 0x7f0ae40d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae37fe640 1 -- 172.21.15.80:0/2310488517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d3ed0 msgr2=0x7f0ae40d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae37fe640 1 --2- 172.21.15.80:0/2310488517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d3ed0 0x7f0ae40d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae37fe640 1 -- 172.21.15.80:0/2310488517 >> v1:172.21.15.80:6789/0 conn(0x7f0ae40b7c10 legacy=0x7f0ae40b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae37fe640 1 -- 172.21.15.80:0/2310488517 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae40d51d0 con 0x7f0ae40d2ad0 2024-04-25T13:42:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae37fe640 1 --2- 172.21.15.80:0/2310488517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40d2ad0 0x7f0ae40b76d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009870 tx=0x7f0ad801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4107daefd20a13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae2ffd640 1 -- 172.21.15.80:0/2310488517 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80aa620 con 0x7f0ae40d2ad0 2024-04-25T13:42:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae2ffd640 1 -- 172.21.15.80:0/2310488517 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad80af070 con 0x7f0ae40d2ad0 2024-04-25T13:42:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0ae2ffd640 1 -- 172.21.15.80:0/2310488517 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80aa9e0 con 0x7f0ae40d2ad0 2024-04-25T13:42:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 -- 172.21.15.80:0/2310488517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40d2ad0 msgr2=0x7f0ae40b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 --2- 172.21.15.80:0/2310488517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40d2ad0 0x7f0ae40b76d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0ad8009870 tx=0x7f0ad801c4a0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 -- 172.21.15.80:0/2310488517 shutdown_connections 2024-04-25T13:42:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 --2- 172.21.15.80:0/2310488517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40d2ad0 0x7f0ae40b76d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 --2- 172.21.15.80:0/2310488517 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d3ed0 0x7f0ae40d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 -- 172.21.15.80:0/2310488517 >> 172.21.15.80:0/2310488517 conn(0x7f0ae40737b0 msgr2=0x7f0ae406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 -- 172.21.15.80:0/2310488517 shutdown_connections 2024-04-25T13:42:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.777+0000 7f0aea935640 1 -- 172.21.15.80:0/2310488517 wait complete. 2024-04-25T13:42:12.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0aea935640 1 Processor -- start 2024-04-25T13:42:12.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0aea935640 1 -- start start 2024-04-25T13:42:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0aea935640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40b7c10 0x7f0ae414d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0aea935640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d2ad0 0x7f0ae40c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0aea935640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ae40d3ed0 0x7f0ae414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0aea935640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40ccae0 con 0x7f0ae40b7c10 2024-04-25T13:42:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0aea935640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae40cc960 con 0x7f0ae40d3ed0 2024-04-25T13:42:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0aea935640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0ae4160620 con 0x7f0ae40d2ad0 2024-04-25T13:42:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40b7c10 0x7f0ae414d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40b7c10 0x7f0ae414d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52946/0 (socket says 172.21.15.80:52946) 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 -- 172.21.15.80:0/2156183024 learned_addr learned my addr 172.21.15.80:0/2156183024 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 -- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d2ad0 msgr2=0x7f0ae40c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d2ad0 0x7f0ae40c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 -- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ae40d3ed0 msgr2=0x7f0ae414e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae8ed2640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ae40d3ed0 0x7f0ae414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ae40d3ed0 0x7f0ae414e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 -- 172.21.15.80:0/2156183024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae414c630 con 0x7f0ae40b7c10 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae3fff640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40b7c10 0x7f0ae414d6d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400d620 tx=0x7f0ad400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae0ff9640 1 -- 172.21.15.80:0/2156183024 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400b620 con 0x7f0ae40b7c10 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.781+0000 7f0ae0ff9640 1 -- 172.21.15.80:0/2156183024 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad4019070 con 0x7f0ae40b7c10 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae414c910 con 0x7f0ae40b7c10 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0ae0ff9640 1 -- 172.21.15.80:0/2156183024 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400b9e0 con 0x7f0ae40b7c10 2024-04-25T13:42:12.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae414ced0 con 0x7f0ae40b7c10 2024-04-25T13:42:12.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ae406a3a0 con 0x7f0ae40b7c10 2024-04-25T13:42:12.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0ae0ff9640 1 -- 172.21.15.80:0/2156183024 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0ad400bb80 con 0x7f0ae40b7c10 2024-04-25T13:42:12.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0ae0ff9640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ad006f400 0x7f0ad0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:12.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0ae0ff9640 1 -- 172.21.15.80:0/2156183024 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f0ad408ff10 con 0x7f0ae40b7c10 2024-04-25T13:42:12.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0ae37fe640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ad006f400 0x7f0ad0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:12.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.785+0000 7f0ae37fe640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ad006f400 0x7f0ad0071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80ae000 tx=0x7f0ad8004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:12.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.789+0000 7f0ae0ff9640 1 -- 172.21.15.80:0/2156183024 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ad405d130 con 0x7f0ae40b7c10 2024-04-25T13:42:12.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.953+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ae406cae0 con 0x7f0ae40b7c10 2024-04-25T13:42:12.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.953+0000 7f0ae0ff9640 1 -- 172.21.15.80:0/2156183024 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7f0ad405ca30 con 0x7f0ae40b7c10 2024-04-25T13:42:12.957 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ad006f400 msgr2=0x7f0ad0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ad006f400 0x7f0ad0071910 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80ae000 tx=0x7f0ad8004740 comp rx=0 tx=0).stop 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40b7c10 msgr2=0x7f0ae414d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40b7c10 0x7f0ae414d6d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400d620 tx=0x7f0ad400dae0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 shutdown_connections 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ad006f400 0x7f0ad0071910 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ae40d3ed0 0x7f0ae414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ae40d2ad0 0x7f0ae40c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 --2- 172.21.15.80:0/2156183024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ae40b7c10 0x7f0ae414d6d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 >> 172.21.15.80:0/2156183024 conn(0x7f0ae40737b0 msgr2=0x7f0ae40c9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 shutdown_connections 2024-04-25T13:42:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:12.957+0000 7f0aea935640 1 -- 172.21.15.80:0/2156183024 wait complete. 2024-04-25T13:42:12.979 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:42:13.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe021d41640 1 Processor -- start 2024-04-25T13:42:13.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe021d41640 1 -- start start 2024-04-25T13:42:13.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe021d41640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe01c0d0c90 0x7fe01c06fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe021d41640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe01c160be0 0x7fe01c160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe021d41640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 0x7fe01c15da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe021d41640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c070dc0 con 0x7fe01c0d0c90 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe021d41640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe01c161530 con 0x7fe01c160be0 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe021d41640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c070240 con 0x7fe01c15d5f0 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe01c0d0c90 0x7fe01c06fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe01c0d0c90 0x7fe01c06fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52950/0 (socket says 172.21.15.80:52950) 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01b7fe640 1 -- 172.21.15.80:0/3140796004 learned_addr learned my addr 172.21.15.80:0/3140796004 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01affd640 1 --2- 172.21.15.80:0/3140796004 >> v2:172.21.15.87:3300/0 conn(0x7fe01c160be0 0x7fe01c160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01bfff640 1 --2- 172.21.15.80:0/3140796004 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 0x7fe01c15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01b7fe640 1 -- 172.21.15.80:0/3140796004 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 msgr2=0x7fe01c15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01b7fe640 1 --2- 172.21.15.80:0/3140796004 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 0x7fe01c15da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01b7fe640 1 -- 172.21.15.80:0/3140796004 >> v2:172.21.15.87:3300/0 conn(0x7fe01c160be0 msgr2=0x7fe01c160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01b7fe640 1 --2- 172.21.15.80:0/3140796004 >> v2:172.21.15.87:3300/0 conn(0x7fe01c160be0 0x7fe01c160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.117+0000 7fe01b7fe640 1 -- 172.21.15.80:0/3140796004 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c070520 con 0x7fe01c0d0c90 2024-04-25T13:42:13.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe01b7fe640 1 --2- 172.21.15.80:0/3140796004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c0d0c90 0x7fe01c06fa10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0d00d0 tx=0x7fe01401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5b8898f35e863d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:13.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe01a7fc640 1 -- 172.21.15.80:0/3140796004 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140a7880 con 0x7fe01c0d0c90 2024-04-25T13:42:13.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe01a7fc640 1 -- 172.21.15.80:0/3140796004 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0140a7a20 con 0x7fe01c0d0c90 2024-04-25T13:42:13.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe01a7fc640 1 -- 172.21.15.80:0/3140796004 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140a7df0 con 0x7fe01c0d0c90 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 -- 172.21.15.80:0/3140796004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c0d0c90 msgr2=0x7fe01c06fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 --2- 172.21.15.80:0/3140796004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c0d0c90 0x7fe01c06fa10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe01c0d00d0 tx=0x7fe01401ec00 comp rx=0 tx=0).stop 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 -- 172.21.15.80:0/3140796004 shutdown_connections 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 --2- 172.21.15.80:0/3140796004 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 0x7fe01c15da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 --2- 172.21.15.80:0/3140796004 >> v2:172.21.15.87:3300/0 conn(0x7fe01c160be0 0x7fe01c160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 --2- 172.21.15.80:0/3140796004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c0d0c90 0x7fe01c06fa10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 -- 172.21.15.80:0/3140796004 >> 172.21.15.80:0/3140796004 conn(0x7fe01c0cf450 msgr2=0x7fe01c0cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 -- 172.21.15.80:0/3140796004 shutdown_connections 2024-04-25T13:42:13.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.121+0000 7fe021d41640 1 -- 172.21.15.80:0/3140796004 wait complete. 2024-04-25T13:42:13.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 Processor -- start 2024-04-25T13:42:13.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 -- start start 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe01c0d0c90 0x7fe01c0b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 0x7fe01c155960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c160be0 0x7fe01c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c15e4d0 con 0x7fe01c160be0 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe01c15e350 con 0x7fe01c0d0c90 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe01c068da0 con 0x7fe01c15d5f0 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c160be0 0x7fe01c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c160be0 0x7fe01c14ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52952/0 (socket says 172.21.15.80:52952) 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 -- 172.21.15.80:0/2613945104 learned_addr learned my addr 172.21.15.80:0/2613945104 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:13.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01b7fe640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe01c0d0c90 0x7fe01c0b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01affd640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 0x7fe01c155960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 -- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 msgr2=0x7fe01c155960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 0x7fe01c155960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 -- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe01c0d0c90 msgr2=0x7fe01c0b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe01c0d0c90 0x7fe01c0b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 -- 172.21.15.80:0/2613945104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe01c0b8710 con 0x7fe01c160be0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe01bfff640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c160be0 0x7fe01c14ca20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00d5d0 tx=0x7fe00c00da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe003fff640 1 -- 172.21.15.80:0/2613945104 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c018020 con 0x7fe01c160be0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe003fff640 1 -- 172.21.15.80:0/2613945104 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe00c014620 con 0x7fe01c160be0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe01c14d450 con 0x7fe01c160be0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe003fff640 1 -- 172.21.15.80:0/2613945104 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0149f0 con 0x7fe01c160be0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe01c0b70d0 con 0x7fe01c160be0 2024-04-25T13:42:13.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.125+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe01c14d970 con 0x7fe01c160be0 2024-04-25T13:42:13.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.129+0000 7fe003fff640 1 -- 172.21.15.80:0/2613945104 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe00c01b020 con 0x7fe01c160be0 2024-04-25T13:42:13.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.129+0000 7fe003fff640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe00806f400 0x7fe008071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.129+0000 7fe003fff640 1 -- 172.21.15.80:0/2613945104 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fe00c08f410 con 0x7fe01c160be0 2024-04-25T13:42:13.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.129+0000 7fe01b7fe640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe00806f400 0x7fe008071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.129+0000 7fe01b7fe640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe00806f400 0x7fe008071910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0140b0000 tx=0x7fe0140a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:13.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.133+0000 7fe003fff640 1 -- 172.21.15.80:0/2613945104 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe00c022020 con 0x7fe01c160be0 2024-04-25T13:42:13.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.289+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe01c06a3a0 con 0x7fe01c160be0 2024-04-25T13:42:13.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.289+0000 7fe003fff640 1 -- 172.21.15.80:0/2613945104 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) ==== 84+0+65 (secure 0 0 0) 0x7fe00c05c650 con 0x7fe01c160be0 2024-04-25T13:42:13.293 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:13.293 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:42:13.293 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:13.293 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:42:13.293 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:42:13.294 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:13.294 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:42:13.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe00806f400 msgr2=0x7fe008071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe00806f400 0x7fe008071910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe0140b0000 tx=0x7fe0140a9070 comp rx=0 tx=0).stop 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c160be0 msgr2=0x7fe01c14ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c160be0 0x7fe01c14ca20 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00d5d0 tx=0x7fe00c00da90 comp rx=0 tx=0).stop 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 shutdown_connections 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe00806f400 0x7fe008071910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe01c160be0 0x7fe01c14ca20 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe01c15d5f0 0x7fe01c155960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 --2- 172.21.15.80:0/2613945104 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe01c0d0c90 0x7fe01c0b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 >> 172.21.15.80:0/2613945104 conn(0x7fe01c0cf450 msgr2=0x7fe01c0b3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 shutdown_connections 2024-04-25T13:42:13.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.293+0000 7fe021d41640 1 -- 172.21.15.80:0/2613945104 wait complete. 2024-04-25T13:42:13.313 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:42:13.313 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:42:13.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.461+0000 7f288ace4640 1 Processor -- start 2024-04-25T13:42:13.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.461+0000 7f288ace4640 1 -- start start 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f288ace4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f28840d0ed0 0x7f28840d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f288ace4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d4820 0x7f28840d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f288ace4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28840d1820 con 0x7f28840d24a0 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f288ace4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2884164040 con 0x7f28840d0ed0 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f288ace4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2884164bc0 con 0x7f28840d4820 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2888a80640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f28840d0ed0 0x7f28840d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2883fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f28840d24a0 0x7f28840d28b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:46980/0 (socket says 172.21.15.80:46980) 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2883fff640 1 -- 172.21.15.80:0/3445533559 learned_addr learned my addr 172.21.15.80:0/3445533559 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2889281640 1 --2- 172.21.15.80:0/3445533559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d4820 0x7f28840d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2888a80640 1 -- 172.21.15.80:0/3445533559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d4820 msgr2=0x7f28840d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2888a80640 1 --2- 172.21.15.80:0/3445533559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d4820 0x7f28840d4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2888a80640 1 -- 172.21.15.80:0/3445533559 >> v1:172.21.15.80:6789/0 conn(0x7f28840d24a0 legacy=0x7f28840d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2888a80640 1 -- 172.21.15.80:0/3445533559 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28840d2fc0 con 0x7f28840d0ed0 2024-04-25T13:42:13.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.465+0000 7f2889281640 1 --2- 172.21.15.80:0/3445533559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d4820 0x7f28840d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:13.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f2888a80640 1 --2- 172.21.15.80:0/3445533559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d0ed0 0x7f28840d12e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f28740097c0 tx=0x7f287401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f47ddfab161840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:13.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f28837fe640 1 -- 172.21.15.80:0/3445533559 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28740a7070 con 0x7f28840d0ed0 2024-04-25T13:42:13.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f28837fe640 1 -- 172.21.15.80:0/3445533559 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28740b0070 con 0x7f28840d0ed0 2024-04-25T13:42:13.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f28837fe640 1 -- 172.21.15.80:0/3445533559 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28740ab890 con 0x7f28840d0ed0 2024-04-25T13:42:13.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 -- 172.21.15.80:0/3445533559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d0ed0 msgr2=0x7f28840d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 --2- 172.21.15.80:0/3445533559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d0ed0 0x7f28840d12e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f28740097c0 tx=0x7f287401c470 comp rx=0 tx=0).stop 2024-04-25T13:42:13.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 -- 172.21.15.80:0/3445533559 shutdown_connections 2024-04-25T13:42:13.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 --2- 172.21.15.80:0/3445533559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d4820 0x7f28840d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 --2- 172.21.15.80:0/3445533559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d0ed0 0x7f28840d12e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 -- 172.21.15.80:0/3445533559 >> 172.21.15.80:0/3445533559 conn(0x7f288406c5f0 msgr2=0x7f288406ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:13.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 -- 172.21.15.80:0/3445533559 shutdown_connections 2024-04-25T13:42:13.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 -- 172.21.15.80:0/3445533559 wait complete. 2024-04-25T13:42:13.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 Processor -- start 2024-04-25T13:42:13.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.469+0000 7f288ace4640 1 -- start start 2024-04-25T13:42:13.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28840d0ed0 0x7f288414bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d24a0 0x7f2884159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d4820 0x7f288415a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f288416d3e0 con 0x7f28840d4820 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2884157820 con 0x7f28840d0ed0 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f288416d560 con 0x7f28840d24a0 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d4820 0x7f288415a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d4820 0x7f288415a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52966/0 (socket says 172.21.15.80:52966) 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 -- 172.21.15.80:0/2012518497 learned_addr learned my addr 172.21.15.80:0/2012518497 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2888a80640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28840d0ed0 0x7f288414bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2883fff640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d24a0 0x7f2884159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 -- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d24a0 msgr2=0x7f2884159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d24a0 0x7f2884159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 -- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28840d0ed0 msgr2=0x7f288414bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:13.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28840d0ed0 0x7f288414bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:13.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 -- 172.21.15.80:0/2012518497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f288415b1b0 con 0x7f28840d4820 2024-04-25T13:42:13.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2883fff640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d24a0 0x7f2884159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:13.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f2889281640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d4820 0x7f288415a0a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f287c00cd80 tx=0x7f287c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:13.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f28817fa640 1 -- 172.21.15.80:0/2012518497 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c003ba0 con 0x7f28840d4820 2024-04-25T13:42:13.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f28817fa640 1 -- 172.21.15.80:0/2012518497 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f287c003d60 con 0x7f28840d4820 2024-04-25T13:42:13.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f28817fa640 1 -- 172.21.15.80:0/2012518497 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287c004260 con 0x7f28840d4820 2024-04-25T13:42:13.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f288415b490 con 0x7f28840d4820 2024-04-25T13:42:13.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28840c73e0 con 0x7f28840d4820 2024-04-25T13:42:13.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.473+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28840c7640 con 0x7f28840d4820 2024-04-25T13:42:13.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.477+0000 7f28817fa640 1 -- 172.21.15.80:0/2012518497 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f287c004400 con 0x7f28840d4820 2024-04-25T13:42:13.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.477+0000 7f28817fa640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f287006f400 0x7f2870071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:13.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.477+0000 7f28817fa640 1 -- 172.21.15.80:0/2012518497 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f287c08f990 con 0x7f28840d4820 2024-04-25T13:42:13.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.477+0000 7f2888a80640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f287006f400 0x7f2870071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:13.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.477+0000 7f2888a80640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f287006f400 0x7f2870071910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28740af000 tx=0x7f28740a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:13.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.481+0000 7f28817fa640 1 -- 172.21.15.80:0/2012518497 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f287c014030 con 0x7f28840d4820 2024-04-25T13:42:13.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:13.633+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f288406a440 con 0x7f28840d4820 2024-04-25T13:42:14.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.357+0000 7f28817fa640 1 -- 172.21.15.80:0/2012518497 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) ==== 138+0+0 (secure 0 0 0) 0x7f287c05cbc0 con 0x7f28840d4820 2024-04-25T13:42:14.360 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:42:14.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f287006f400 msgr2=0x7f2870071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f287006f400 0x7f2870071910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28740af000 tx=0x7f28740a7000 comp rx=0 tx=0).stop 2024-04-25T13:42:14.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d4820 msgr2=0x7f288415a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d4820 0x7f288415a0a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f287c00cd80 tx=0x7f287c00a650 comp rx=0 tx=0).stop 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 shutdown_connections 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f287006f400 0x7f2870071910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28840d4820 0x7f288415a0a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28840d24a0 0x7f2884159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 --2- 172.21.15.80:0/2012518497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28840d0ed0 0x7f288414bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 >> 172.21.15.80:0/2012518497 conn(0x7f288406c5f0 msgr2=0x7f2884161d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 shutdown_connections 2024-04-25T13:42:14.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.361+0000 7f288ace4640 1 -- 172.21.15.80:0/2012518497 wait complete. 2024-04-25T13:42:14.392 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:42:14.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.541+0000 7f7b4f9ee640 1 Processor -- start 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4f9ee640 1 -- start start 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4f9ee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b480b8660 0x7f7b48168f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4f9ee640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b480cfb20 con 0x7f7b480d04e0 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4f9ee640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b480b6b70 con 0x7f7b480b9240 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4f9ee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b480b6920 con 0x7f7b480b8660 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4cf89640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7b480d04e0 0x7f7b480d08f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:46994/0 (socket says 172.21.15.80:46994) 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4cf89640 1 -- 172.21.15.80:0/378316963 learned_addr learned my addr 172.21.15.80:0/378316963 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4df8b640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b480b8660 0x7f7b48168f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 273208568 0 0) 0x7f7b480cfb20 con 0x7f7b480d04e0 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7b34003530 con 0x7f7b480d04e0 2024-04-25T13:42:14.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2397736318 0 0) 0x7f7b34003530 con 0x7f7b480d04e0 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b480b8660 msgr2=0x7f7b48168f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b480b8660 0x7f7b48168f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 >> v1:172.21.15.87:6789/0 conn(0x7f7b480b9240 legacy=0x7f7b480cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b480b6e50 con 0x7f7b480d04e0 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7b38001680 con 0x7f7b480d04e0 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 >> v1:172.21.15.80:6789/0 conn(0x7f7b480d04e0 legacy=0x7f7b480d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b480b8660 0x7f7b340039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b34004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b34004d50 0x7f7b34005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7b480b6920 con 0x7f7b34003f70 2024-04-25T13:42:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b3400bc10 con 0x7f7b34004d50 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b3400be10 con 0x7f7b480b8660 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4cf89640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b34004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4df8b640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b34004d50 0x7f7b34005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4d78a640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b480b8660 0x7f7b340039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4cf89640 1 -- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b480b8660 msgr2=0x7f7b340039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4cf89640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b480b8660 0x7f7b340039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4cf89640 1 -- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b34004d50 msgr2=0x7f7b34005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4cf89640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b34004d50 0x7f7b34005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.545+0000 7f7b4cf89640 1 -- 172.21.15.80:0/378316963 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b34003530 con 0x7f7b34003f70 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b4cf89640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b34004810 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800c210 tx=0x7f7b3801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbfc2cdda1e6bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38007d40 con 0x7f7b34003f70 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b380a9070 con 0x7f7b34003f70 2024-04-25T13:42:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b3ffff640 1 -- 172.21.15.80:0/378316963 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38005d80 con 0x7f7b34003f70 2024-04-25T13:42:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/378316963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 msgr2=0x7f7b34004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b4f9ee640 1 --2- 172.21.15.80:0/378316963 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b34004810 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f7b3800c210 tx=0x7f7b3801cca0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b4df8b640 1 -- 172.21.15.80:0/378316963 reap_dead start 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/378316963 shutdown_connections 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/378316963 >> 172.21.15.80:0/378316963 conn(0x7f7b480d3ad0 msgr2=0x7f7b480d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/378316963 shutdown_connections 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.549+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/378316963 wait complete. 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 Processor -- start 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 -- start start 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b481608e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b481561b0 0x7f7b48166070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b481665b0 0x7f7b480cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48169b00 con 0x7f7b34003f70 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48169980 con 0x7f7b481561b0 2024-04-25T13:42:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7b48169c80 con 0x7f7b481665b0 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b481608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b481608e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52976/0 (socket says 172.21.15.80:52976) 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 -- 172.21.15.80:0/2450250830 learned_addr learned my addr 172.21.15.80:0/2450250830 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4cf89640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b481561b0 0x7f7b48166070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4df8b640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b481665b0 0x7f7b480cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 -- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b481665b0 msgr2=0x7f7b480cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b481665b0 0x7f7b480cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 -- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b481561b0 msgr2=0x7f7b48166070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b481561b0 0x7f7b48166070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 -- 172.21.15.80:0/2450250830 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b480cf6f0 con 0x7f7b34003f70 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4cf89640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b481561b0 0x7f7b48166070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4d78a640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b481608e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400e490 tx=0x7f7b4400e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b3dffb640 1 -- 172.21.15.80:0/2450250830 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b440052e0 con 0x7f7b34003f70 2024-04-25T13:42:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b3dffb640 1 -- 172.21.15.80:0/2450250830 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b44018070 con 0x7f7b34003f70 2024-04-25T13:42:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b481669c0 con 0x7f7b34003f70 2024-04-25T13:42:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b3dffb640 1 -- 172.21.15.80:0/2450250830 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44003650 con 0x7f7b34003f70 2024-04-25T13:42:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b481fdf60 con 0x7f7b34003f70 2024-04-25T13:42:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.553+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b4806a3a0 con 0x7f7b34003f70 2024-04-25T13:42:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.557+0000 7f7b3dffb640 1 -- 172.21.15.80:0/2450250830 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7b440037f0 con 0x7f7b34003f70 2024-04-25T13:42:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.557+0000 7f7b3dffb640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b2c06f400 0x7f7b2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.557+0000 7f7b3dffb640 1 -- 172.21.15.80:0/2450250830 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5227+0+0 (secure 0 0 0) 0x7f7b4408ecc0 con 0x7f7b34003f70 2024-04-25T13:42:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.557+0000 7f7b4cf89640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b2c06f400 0x7f7b2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.557+0000 7f7b4cf89640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b2c06f400 0x7f7b2c071910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b380ab000 tx=0x7f7b380a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.561+0000 7f7b3dffb640 1 -- 172.21.15.80:0/2450250830 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7b481669c0 con 0x7f7b34003f70 2024-04-25T13:42:14.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:14.709+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7b48074c30 con 0x7f7b34003f70 2024-04-25T13:42:15.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.369+0000 7f7b3dffb640 1 -- 172.21.15.80:0/2450250830 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) ==== 170+0+0 (secure 0 0 0) 0x7f7b4405bf70 con 0x7f7b34003f70 2024-04-25T13:42:15.375 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:42:15.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.373+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b2c06f400 msgr2=0x7f7b2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:15.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.373+0000 7f7b4f9ee640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b2c06f400 0x7f7b2c071910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7b380ab000 tx=0x7f7b380a7070 comp rx=0 tx=0).stop 2024-04-25T13:42:15.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.373+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 msgr2=0x7f7b481608e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:15.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.373+0000 7f7b4f9ee640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b481608e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400e490 tx=0x7f7b4400e950 comp rx=0 tx=0).stop 2024-04-25T13:42:15.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.377+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 shutdown_connections 2024-04-25T13:42:15.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.377+0000 7f7b4f9ee640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7b2c06f400 0x7f7b2c071910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.377+0000 7f7b4f9ee640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7b481665b0 0x7f7b480cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.377+0000 7f7b4f9ee640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b481561b0 0x7f7b48166070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.377+0000 7f7b4f9ee640 1 --2- 172.21.15.80:0/2450250830 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7b34003f70 0x7f7b481608e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.377+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 >> 172.21.15.80:0/2450250830 conn(0x7f7b480d3ad0 msgr2=0x7f7b480d08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:15.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.377+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 shutdown_connections 2024-04-25T13:42:15.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.377+0000 7f7b4f9ee640 1 -- 172.21.15.80:0/2450250830 wait complete. 2024-04-25T13:42:15.415 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:42:15.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c553e3640 1 Processor -- start 2024-04-25T13:42:15.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c553e3640 1 -- start start 2024-04-25T13:42:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c553e3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3c50070de0 0x7f3c500711f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c553e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50068590 0x7f3c500689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c553e3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c5006f500 0x7f3c5006f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c553e3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3c5015f530 con 0x7f3c5006f500 2024-04-25T13:42:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c553e3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3c50068ee0 con 0x7f3c50070de0 2024-04-25T13:42:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c553e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5015e9b0 con 0x7f3c50068590 2024-04-25T13:42:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c5006f500 0x7f3c5006f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3c5006f500 0x7f3c5006f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52990/0 (socket says 172.21.15.80:52990) 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4f7fe640 1 -- 172.21.15.80:0/295422456 learned_addr learned my addr 172.21.15.80:0/295422456 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4effd640 1 --2- 172.21.15.80:0/295422456 >> v2:172.21.15.87:3300/0 conn(0x7f3c50070de0 0x7f3c500711f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4e7fc640 1 --2- 172.21.15.80:0/295422456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50068590 0x7f3c500689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4f7fe640 1 -- 172.21.15.80:0/295422456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50068590 msgr2=0x7f3c500689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4f7fe640 1 --2- 172.21.15.80:0/295422456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50068590 0x7f3c500689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4f7fe640 1 -- 172.21.15.80:0/295422456 >> v2:172.21.15.87:3300/0 conn(0x7f3c50070de0 msgr2=0x7f3c500711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4f7fe640 1 --2- 172.21.15.80:0/295422456 >> v2:172.21.15.87:3300/0 conn(0x7f3c50070de0 0x7f3c500711f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4f7fe640 1 -- 172.21.15.80:0/295422456 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5015ec90 con 0x7f3c5006f500 2024-04-25T13:42:15.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.565+0000 7f3c4effd640 1 --2- 172.21.15.80:0/295422456 >> v2:172.21.15.87:3300/0 conn(0x7f3c50070de0 0x7f3c500711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:15.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c4f7fe640 1 --2- 172.21.15.80:0/295422456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5006f500 0x7f3c5006f910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3c500727d0 tx=0x7f3c4801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908764db8fc1d9be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:15.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c4dffb640 1 -- 172.21.15.80:0/295422456 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c480aa610 con 0x7f3c5006f500 2024-04-25T13:42:15.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c4dffb640 1 -- 172.21.15.80:0/295422456 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c480af070 con 0x7f3c5006f500 2024-04-25T13:42:15.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c4dffb640 1 -- 172.21.15.80:0/295422456 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c480aa9d0 con 0x7f3c5006f500 2024-04-25T13:42:15.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 -- 172.21.15.80:0/295422456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5006f500 msgr2=0x7f3c5006f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:15.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 --2- 172.21.15.80:0/295422456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5006f500 0x7f3c5006f910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3c500727d0 tx=0x7f3c4801c4f0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 -- 172.21.15.80:0/295422456 shutdown_connections 2024-04-25T13:42:15.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 --2- 172.21.15.80:0/295422456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c5006f500 0x7f3c5006f910 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 --2- 172.21.15.80:0/295422456 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50068590 0x7f3c500689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 --2- 172.21.15.80:0/295422456 >> v2:172.21.15.87:3300/0 conn(0x7f3c50070de0 0x7f3c500711f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 -- 172.21.15.80:0/295422456 >> 172.21.15.80:0/295422456 conn(0x7f3c500737c0 msgr2=0x7f3c50073c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:15.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 -- 172.21.15.80:0/295422456 shutdown_connections 2024-04-25T13:42:15.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.569+0000 7f3c553e3640 1 -- 172.21.15.80:0/295422456 wait complete. 2024-04-25T13:42:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 Processor -- start 2024-04-25T13:42:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 -- start start 2024-04-25T13:42:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c50068590 0x7f3c501497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5006f500 0x7f3c50151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50070de0 0x7f3c50151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3c50161910 con 0x7f3c50068590 2024-04-25T13:42:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3c50161790 con 0x7f3c5006f500 2024-04-25T13:42:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3c5015e560 con 0x7f3c50070de0 2024-04-25T13:42:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c50068590 0x7f3c501497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c50068590 0x7f3c501497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52992/0 (socket says 172.21.15.80:52992) 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 -- 172.21.15.80:0/534895502 learned_addr learned my addr 172.21.15.80:0/534895502 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4e7fc640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5006f500 0x7f3c50151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4f7fe640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50070de0 0x7f3c50151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 -- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50070de0 msgr2=0x7f3c50151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50070de0 0x7f3c50151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 -- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5006f500 msgr2=0x7f3c50151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5006f500 0x7f3c50151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 -- 172.21.15.80:0/534895502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c500763e0 con 0x7f3c50068590 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4e7fc640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5006f500 0x7f3c50151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4f7fe640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50070de0 0x7f3c50151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c4effd640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c50068590 0x7f3c501497c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3c40007ef0 tx=0x7f3c4000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c477fe640 1 -- 172.21.15.80:0/534895502 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c40014620 con 0x7f3c50068590 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c477fe640 1 -- 172.21.15.80:0/534895502 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c40019070 con 0x7f3c50068590 2024-04-25T13:42:15.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5014fb90 con 0x7f3c50068590 2024-04-25T13:42:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c477fe640 1 -- 172.21.15.80:0/534895502 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c400149e0 con 0x7f3c50068590 2024-04-25T13:42:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.573+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c50151fe0 con 0x7f3c50068590 2024-04-25T13:42:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.577+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c5006a3a0 con 0x7f3c50068590 2024-04-25T13:42:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.577+0000 7f3c477fe640 1 -- 172.21.15.80:0/534895502 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3c40014b80 con 0x7f3c50068590 2024-04-25T13:42:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.577+0000 7f3c477fe640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3c3006f400 0x7f3c30071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.577+0000 7f3c477fe640 1 -- 172.21.15.80:0/534895502 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5609+0+0 (secure 0 0 0) 0x7f3c400904d0 con 0x7f3c50068590 2024-04-25T13:42:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.577+0000 7f3c4e7fc640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3c3006f400 0x7f3c30071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:15.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.581+0000 7f3c4e7fc640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3c3006f400 0x7f3c30071910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3c50160e60 tx=0x7f3c38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:15.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.581+0000 7f3c477fe640 1 -- 172.21.15.80:0/534895502 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3c4005d500 con 0x7f3c50068590 2024-04-25T13:42:15.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:15.777+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f3c500705b0 con 0x7f3c50068590 2024-04-25T13:42:16.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.417+0000 7f3c477fe640 1 -- 172.21.15.80:0/534895502 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) ==== 172+0+0 (secure 0 0 0) 0x7f3c4005ce00 con 0x7f3c50068590 2024-04-25T13:42:16.420 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T13:42:16.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3c3006f400 msgr2=0x7f3c30071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3c3006f400 0x7f3c30071910 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3c50160e60 tx=0x7f3c38008040 comp rx=0 tx=0).stop 2024-04-25T13:42:16.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c50068590 msgr2=0x7f3c501497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c50068590 0x7f3c501497c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3c40007ef0 tx=0x7f3c4000e4b0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 shutdown_connections 2024-04-25T13:42:16.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3c3006f400 0x7f3c30071910 secure :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f3c50160e60 tx=0x7f3c38008040 comp rx=0 tx=0).stop 2024-04-25T13:42:16.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3c50070de0 0x7f3c50151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3c5006f500 0x7f3c50151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 --2- 172.21.15.80:0/534895502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3c50068590 0x7f3c501497c0 secure :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3c40007ef0 tx=0x7f3c4000e4b0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 >> 172.21.15.80:0/534895502 conn(0x7f3c500737c0 msgr2=0x7f3c5015c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:16.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 shutdown_connections 2024-04-25T13:42:16.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.421+0000 7f3c553e3640 1 -- 172.21.15.80:0/534895502 wait complete. 2024-04-25T13:42:16.457 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:16.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f26659a4640 1 Processor -- start 2024-04-25T13:42:16.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f26659a4640 1 -- start start 2024-04-25T13:42:16.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f26659a4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2660161f90 0x7f26600cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f26659a4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f266015ec40 0x7f266015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f26659a4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26600d0d70 con 0x7f26600d0510 2024-04-25T13:42:16.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f26659a4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f26600d1050 con 0x7f266015ec40 2024-04-25T13:42:16.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f26659a4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f266006dca0 con 0x7f2660161f90 2024-04-25T13:42:16.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f265f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f266015ec40 0x7f266015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f265f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f266015ec40 0x7f266015f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53002/0 (socket says 172.21.15.80:53002) 2024-04-25T13:42:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f265f7fe640 1 -- 172.21.15.80:0/681211594 learned_addr learned my addr 172.21.15.80:0/681211594 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f265effd640 1 --2- 172.21.15.80:0/681211594 >> v2:172.21.15.87:3300/0 conn(0x7f2660161f90 0x7f26600cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f265f7fe640 1 -- 172.21.15.80:0/681211594 >> v2:172.21.15.87:3300/0 conn(0x7f2660161f90 msgr2=0x7f26600cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f265f7fe640 1 --2- 172.21.15.80:0/681211594 >> v2:172.21.15.87:3300/0 conn(0x7f2660161f90 0x7f26600cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f265f7fe640 1 -- 172.21.15.80:0/681211594 >> v1:172.21.15.87:6789/0 conn(0x7f26600d0510 legacy=0x7f266015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.601+0000 7f265f7fe640 1 -- 172.21.15.80:0/681211594 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f266006df80 con 0x7f266015ec40 2024-04-25T13:42:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f265f7fe640 1 --2- 172.21.15.80:0/681211594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266015f050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f26600d17b0 tx=0x7f265801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99b4875d4f7662bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f265dffb640 1 -- 172.21.15.80:0/681211594 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580a56b0 con 0x7f266015ec40 2024-04-25T13:42:16.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f265dffb640 1 -- 172.21.15.80:0/681211594 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26580a5850 con 0x7f266015ec40 2024-04-25T13:42:16.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f265dffb640 1 -- 172.21.15.80:0/681211594 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26580a5c20 con 0x7f266015ec40 2024-04-25T13:42:16.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f26659a4640 1 -- 172.21.15.80:0/681211594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 msgr2=0x7f266015f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f26659a4640 1 --2- 172.21.15.80:0/681211594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266015f050 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f26600d17b0 tx=0x7f265801c8c0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f26659a4640 1 -- 172.21.15.80:0/681211594 shutdown_connections 2024-04-25T13:42:16.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f26659a4640 1 --2- 172.21.15.80:0/681211594 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266015f050 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f26659a4640 1 --2- 172.21.15.80:0/681211594 >> v2:172.21.15.87:3300/0 conn(0x7f2660161f90 0x7f26600cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f26659a4640 1 -- 172.21.15.80:0/681211594 >> 172.21.15.80:0/681211594 conn(0x7f266006d080 msgr2=0x7f26600b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:16.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f26659a4640 1 -- 172.21.15.80:0/681211594 shutdown_connections 2024-04-25T13:42:16.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.605+0000 7f26659a4640 1 -- 172.21.15.80:0/681211594 wait complete. 2024-04-25T13:42:16.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 Processor -- start 2024-04-25T13:42:16.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 -- start start 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26600d0510 0x7f266013dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266013aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2660161f90 0x7f26600b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f266006f630 con 0x7f266015ec40 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f266006f4b0 con 0x7f26600d0510 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26600d0920 con 0x7f2660161f90 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266013aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266013aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53016/0 (socket says 172.21.15.80:53016) 2024-04-25T13:42:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 -- 172.21.15.80:0/3127061590 learned_addr learned my addr 172.21.15.80:0/3127061590 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265effd640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26600d0510 0x7f266013dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265f7fe640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2660161f90 0x7f26600b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 -- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2660161f90 msgr2=0x7f26600b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2660161f90 0x7f26600b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 -- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26600d0510 msgr2=0x7f266013dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26600d0510 0x7f266013dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 -- 172.21.15.80:0/3127061590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26601618c0 con 0x7f266015ec40 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265f7fe640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2660161f90 0x7f26600b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f265e7fc640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266013aa50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f265000b440 tx=0x7f265000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f2647fff640 1 -- 172.21.15.80:0/3127061590 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650015020 con 0x7f266015ec40 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f2647fff640 1 -- 172.21.15.80:0/3127061590 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2650011890 con 0x7f266015ec40 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f2647fff640 1 -- 172.21.15.80:0/3127061590 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650011c80 con 0x7f266015ec40 2024-04-25T13:42:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2660140ca0 con 0x7f266015ec40 2024-04-25T13:42:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.609+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2660141260 con 0x7f266015ec40 2024-04-25T13:42:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.613+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f266006a3a0 con 0x7f266015ec40 2024-04-25T13:42:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.613+0000 7f2647fff640 1 -- 172.21.15.80:0/3127061590 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2650018020 con 0x7f266015ec40 2024-04-25T13:42:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.613+0000 7f2647fff640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f264c06f400 0x7f264c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.613+0000 7f2647fff640 1 -- 172.21.15.80:0/3127061590 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f265008f080 con 0x7f266015ec40 2024-04-25T13:42:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.613+0000 7f265effd640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f264c06f400 0x7f264c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.613+0000 7f265effd640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f264c06f400 0x7f264c071910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f266006fa20 tx=0x7f2654008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:16.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.617+0000 7f2647fff640 1 -- 172.21.15.80:0/3127061590 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f265001f030 con 0x7f266015ec40 2024-04-25T13:42:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.781+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26600b17d0 con 0x7f266015ec40 2024-04-25T13:42:16.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.781+0000 7f2647fff640 1 -- 172.21.15.80:0/3127061590 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16554 (secure 0 0 0) 0x7f2650060e20 con 0x7f266015ec40 2024-04-25T13:42:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:16.787 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":46,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:15.784348+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:42:13.650442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T13:42:14.725209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f264c06f400 msgr2=0x7f264c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f264c06f400 0x7f264c071910 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f266006fa20 tx=0x7f2654008040 comp rx=0 tx=0).stop 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 msgr2=0x7f266013aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266013aa50 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f265000b440 tx=0x7f265000b900 comp rx=0 tx=0).stop 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 shutdown_connections 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f264c06f400 0x7f264c071910 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2660161f90 0x7f26600b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f266015ec40 0x7f266013aa50 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 --2- 172.21.15.80:0/3127061590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26600d0510 0x7f266013dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 >> 172.21.15.80:0/3127061590 conn(0x7f266006d080 msgr2=0x7f26600d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:16.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 shutdown_connections 2024-04-25T13:42:16.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.785+0000 7f26659a4640 1 -- 172.21.15.80:0/3127061590 wait complete. 2024-04-25T13:42:16.809 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.949+0000 7fd4a56d4640 1 Processor -- start 2024-04-25T13:42:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.949+0000 7fd4a56d4640 1 -- start start 2024-04-25T13:42:16.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd4a56d4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4a00d2e80 0x7fd4a01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd4a56d4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d59b0 0x7fd4a00d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd4a56d4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4a00d6300 con 0x7fd4a00d4330 2024-04-25T13:42:16.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd4a56d4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd4a0161060 con 0x7fd4a00d2e80 2024-04-25T13:42:16.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd4a56d4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a0161340 con 0x7fd4a00d59b0 2024-04-25T13:42:16.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd49e7fc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d59b0 0x7fd4a00d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd49effd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4a00d2e80 0x7fd4a01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd49e7fc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d59b0 0x7fd4a00d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:47076/0 (socket says 172.21.15.80:47076) 2024-04-25T13:42:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd49e7fc640 1 -- 172.21.15.80:0/396752477 learned_addr learned my addr 172.21.15.80:0/396752477 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd49e7fc640 1 -- 172.21.15.80:0/396752477 >> v2:172.21.15.87:3300/0 conn(0x7fd4a00d2e80 msgr2=0x7fd4a01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd49e7fc640 1 --2- 172.21.15.80:0/396752477 >> v2:172.21.15.87:3300/0 conn(0x7fd4a00d2e80 0x7fd4a01607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd49e7fc640 1 -- 172.21.15.80:0/396752477 >> v1:172.21.15.87:6789/0 conn(0x7fd4a00d4330 legacy=0x7fd4a00d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.953+0000 7fd49e7fc640 1 -- 172.21.15.80:0/396752477 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a00d2680 con 0x7fd4a00d59b0 2024-04-25T13:42:16.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd49e7fc640 1 --2- 172.21.15.80:0/396752477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d59b0 0x7fd4a00d5dc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd494009870 tx=0x7fd49401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5110e46c13c6a4f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:16.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd49dffb640 1 -- 172.21.15.80:0/396752477 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4940a7070 con 0x7fd4a00d59b0 2024-04-25T13:42:16.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd49dffb640 1 -- 172.21.15.80:0/396752477 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4940b0070 con 0x7fd4a00d59b0 2024-04-25T13:42:16.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd49dffb640 1 -- 172.21.15.80:0/396752477 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4940ab890 con 0x7fd4a00d59b0 2024-04-25T13:42:16.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd4a56d4640 1 -- 172.21.15.80:0/396752477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d59b0 msgr2=0x7fd4a00d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/396752477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d59b0 0x7fd4a00d5dc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd494009870 tx=0x7fd49401c560 comp rx=0 tx=0).stop 2024-04-25T13:42:16.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd4a56d4640 1 -- 172.21.15.80:0/396752477 shutdown_connections 2024-04-25T13:42:16.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/396752477 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d59b0 0x7fd4a00d5dc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/396752477 >> v2:172.21.15.87:3300/0 conn(0x7fd4a00d2e80 0x7fd4a01607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd4a56d4640 1 -- 172.21.15.80:0/396752477 >> 172.21.15.80:0/396752477 conn(0x7fd4a00737a0 msgr2=0x7fd4a006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:16.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd4a56d4640 1 -- 172.21.15.80:0/396752477 shutdown_connections 2024-04-25T13:42:16.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.957+0000 7fd4a56d4640 1 -- 172.21.15.80:0/396752477 wait complete. 2024-04-25T13:42:16.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 Processor -- start 2024-04-25T13:42:16.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 -- start start 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d2e80 0x7fd4a01554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4a00d4330 0x7fd4a015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4a00d59b0 0x7fd4a015c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00d3410 con 0x7fd4a00d4330 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4a00d3290 con 0x7fd4a00d59b0 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4a015df60 con 0x7fd4a00d2e80 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4a00d4330 0x7fd4a015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4a00d4330 0x7fd4a015bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53030/0 (socket says 172.21.15.80:53030) 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 -- 172.21.15.80:0/1463790520 learned_addr learned my addr 172.21.15.80:0/1463790520 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49f7fe640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4a00d59b0 0x7fd4a015c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49effd640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d2e80 0x7fd4a01554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 -- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d2e80 msgr2=0x7fd4a01554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d2e80 0x7fd4a01554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 -- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4a00d59b0 msgr2=0x7fd4a015c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4a00d59b0 0x7fd4a015c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 -- 172.21.15.80:0/1463790520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4a00ba410 con 0x7fd4a00d4330 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49f7fe640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4a00d59b0 0x7fd4a015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd49e7fc640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4a00d4330 0x7fd4a015bf70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd49401ce00 tx=0x7fd49401ada0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4877fe640 1 -- 172.21.15.80:0/1463790520 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4940be5f0 con 0x7fd4a00d4330 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4877fe640 1 -- 172.21.15.80:0/1463790520 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4940be7b0 con 0x7fd4a00d4330 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4a00ba6f0 con 0x7fd4a00d4330 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4877fe640 1 -- 172.21.15.80:0/1463790520 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4940beba0 con 0x7fd4a00d4330 2024-04-25T13:42:16.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4a00bacb0 con 0x7fd4a00d4330 2024-04-25T13:42:16.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.961+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a015fd90 con 0x7fd4a00d4330 2024-04-25T13:42:16.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.965+0000 7fd4877fe640 1 -- 172.21.15.80:0/1463790520 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4940b43f0 con 0x7fd4a00d4330 2024-04-25T13:42:16.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.965+0000 7fd4877fe640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd48c06f400 0x7fd48c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:16.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.965+0000 7fd4877fe640 1 -- 172.21.15.80:0/1463790520 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7fd494129f60 con 0x7fd4a00d4330 2024-04-25T13:42:16.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.965+0000 7fd49effd640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd48c06f400 0x7fd48c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:16.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.965+0000 7fd49effd640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd48c06f400 0x7fd48c071910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00b8fd0 tx=0x7fd490008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:16.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:16.969+0000 7fd4877fe640 1 -- 172.21.15.80:0/1463790520 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4940c4060 con 0x7fd4a00d4330 2024-04-25T13:42:17.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.129+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd4a006a3a0 con 0x7fd4a00d4330 2024-04-25T13:42:17.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.133+0000 7fd4877fe640 1 -- 172.21.15.80:0/1463790520 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16554 (secure 0 0 0) 0x7fd494016150 con 0x7fd4a00d4330 2024-04-25T13:42:17.137 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:17.137 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":46,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:15.784348+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:42:13.650442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T13:42:14.725209+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:17.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.133+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd48c06f400 msgr2=0x7fd48c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:17.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.133+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd48c06f400 0x7fd48c071910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd4a00b8fd0 tx=0x7fd490008040 comp rx=0 tx=0).stop 2024-04-25T13:42:17.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.133+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4a00d4330 msgr2=0x7fd4a015bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.133+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4a00d4330 0x7fd4a015bf70 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fd49401ce00 tx=0x7fd49401ada0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.137+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 shutdown_connections 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.137+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd48c06f400 0x7fd48c071910 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.137+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4a00d59b0 0x7fd4a015c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.137+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4a00d4330 0x7fd4a015bf70 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.137+0000 7fd4a56d4640 1 --2- 172.21.15.80:0/1463790520 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4a00d2e80 0x7fd4a01554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.137+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 >> 172.21.15.80:0/1463790520 conn(0x7fd4a00737a0 msgr2=0x7fd4a0162250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.137+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 shutdown_connections 2024-04-25T13:42:17.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.137+0000 7fd4a56d4640 1 -- 172.21.15.80:0/1463790520 wait complete. 2024-04-25T13:42:17.159 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:42:17.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff271e80640 1 Processor -- start 2024-04-25T13:42:17.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff271e80640 1 -- start start 2024-04-25T13:42:17.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff271e80640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff26c0b5330 0x7ff26c0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:17.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff271e80640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff26c0d0ff0 0x7ff26c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:17.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff271e80640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff26c0b5740 con 0x7ff26c06f2f0 2024-04-25T13:42:17.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff271e80640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff26c06f700 con 0x7ff26c0d0ff0 2024-04-25T13:42:17.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff271e80640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff26c1629b0 con 0x7ff26c0b5330 2024-04-25T13:42:17.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff26affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff26c0d0ff0 0x7ff26c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:17.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff26affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff26c0d0ff0 0x7ff26c06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53040/0 (socket says 172.21.15.80:53040) 2024-04-25T13:42:17.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.301+0000 7ff26affd640 1 -- 172.21.15.80:0/2373916381 learned_addr learned my addr 172.21.15.80:0/2373916381 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:17.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26b7fe640 1 --2- 172.21.15.80:0/2373916381 >> v2:172.21.15.87:3300/0 conn(0x7ff26c0b5330 0x7ff26c0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:17.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26affd640 1 -- 172.21.15.80:0/2373916381 >> v2:172.21.15.87:3300/0 conn(0x7ff26c0b5330 msgr2=0x7ff26c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:17.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26affd640 1 --2- 172.21.15.80:0/2373916381 >> v2:172.21.15.87:3300/0 conn(0x7ff26c0b5330 0x7ff26c0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26affd640 1 -- 172.21.15.80:0/2373916381 >> v1:172.21.15.87:6789/0 conn(0x7ff26c06f2f0 legacy=0x7ff26c161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:17.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26affd640 1 -- 172.21.15.80:0/2373916381 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c06dd10 con 0x7ff26c0d0ff0 2024-04-25T13:42:17.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26affd640 1 --2- 172.21.15.80:0/2373916381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0d0ff0 0x7ff26c06edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0701f0 tx=0x7ff26001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd6e8ba993a6b19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:17.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26a7fc640 1 -- 172.21.15.80:0/2373916381 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600b0070 con 0x7ff26c0d0ff0 2024-04-25T13:42:17.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26a7fc640 1 -- 172.21.15.80:0/2373916381 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2600ab600 con 0x7ff26c0d0ff0 2024-04-25T13:42:17.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff26a7fc640 1 -- 172.21.15.80:0/2373916381 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600ab9d0 con 0x7ff26c0d0ff0 2024-04-25T13:42:17.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff271e80640 1 -- 172.21.15.80:0/2373916381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0d0ff0 msgr2=0x7ff26c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:17.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff271e80640 1 --2- 172.21.15.80:0/2373916381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0d0ff0 0x7ff26c06edb0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff26c0701f0 tx=0x7ff26001c450 comp rx=0 tx=0).stop 2024-04-25T13:42:17.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff271e80640 1 -- 172.21.15.80:0/2373916381 shutdown_connections 2024-04-25T13:42:17.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff271e80640 1 --2- 172.21.15.80:0/2373916381 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0d0ff0 0x7ff26c06edb0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff271e80640 1 --2- 172.21.15.80:0/2373916381 >> v2:172.21.15.87:3300/0 conn(0x7ff26c0b5330 0x7ff26c0d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff271e80640 1 -- 172.21.15.80:0/2373916381 >> 172.21.15.80:0/2373916381 conn(0x7ff26c0b4160 msgr2=0x7ff26c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:17.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff271e80640 1 -- 172.21.15.80:0/2373916381 shutdown_connections 2024-04-25T13:42:17.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.305+0000 7ff271e80640 1 -- 172.21.15.80:0/2373916381 wait complete. 2024-04-25T13:42:17.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 Processor -- start 2024-04-25T13:42:17.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 -- start start 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff26c06f2f0 0x7ff26c141f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0b5330 0x7ff26c158f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff26c0d0ff0 0x7ff26c160ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c0b0cd0 con 0x7ff26c0b5330 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c06e890 con 0x7ff26c06f2f0 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff26c0b0e50 con 0x7ff26c0d0ff0 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0b5330 0x7ff26c158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0b5330 0x7ff26c158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53042/0 (socket says 172.21.15.80:53042) 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 -- 172.21.15.80:0/3951127462 learned_addr learned my addr 172.21.15.80:0/3951127462 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26b7fe640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff26c06f2f0 0x7ff26c141f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:17.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26bfff640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff26c0d0ff0 0x7ff26c160ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 -- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff26c0d0ff0 msgr2=0x7ff26c160ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff26c0d0ff0 0x7ff26c160ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 -- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff26c06f2f0 msgr2=0x7ff26c141f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff26c06f2f0 0x7ff26c141f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 -- 172.21.15.80:0/3951127462 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c159840 con 0x7ff26c0b5330 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26b7fe640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff26c06f2f0 0x7ff26c141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff26affd640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0b5330 0x7ff26c158f40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff26001a080 tx=0x7ff2600191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff253fff640 1 -- 172.21.15.80:0/3951127462 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600ba070 con 0x7ff26c0b5330 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff253fff640 1 -- 172.21.15.80:0/3951127462 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2600b4400 con 0x7ff26c0b5330 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c159b20 con 0x7ff26c0b5330 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff253fff640 1 -- 172.21.15.80:0/3951127462 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600b0040 con 0x7ff26c0b5330 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c15a0e0 con 0x7ff26c0b5330 2024-04-25T13:42:17.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.309+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff26c15a4c0 con 0x7ff26c0b5330 2024-04-25T13:42:17.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.313+0000 7ff253fff640 1 -- 172.21.15.80:0/3951127462 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff2600b4810 con 0x7ff26c0b5330 2024-04-25T13:42:17.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.313+0000 7ff253fff640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff25806f400 0x7ff258071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:17.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.313+0000 7ff253fff640 1 -- 172.21.15.80:0/3951127462 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7ff2601284c0 con 0x7ff26c0b5330 2024-04-25T13:42:17.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.313+0000 7ff26b7fe640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff25806f400 0x7ff258071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:17.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.313+0000 7ff26b7fe640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff25806f400 0x7ff258071910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0097a0 tx=0x7ff25c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:17.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.317+0000 7ff253fff640 1 -- 172.21.15.80:0/3951127462 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff2600eee70 con 0x7ff26c0b5330 2024-04-25T13:42:17.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:17.505+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff26c06a3a0 con 0x7ff26c0b5330 2024-04-25T13:42:18.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.429+0000 7ff253fff640 1 -- 172.21.15.80:0/3951127462 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) ==== 118+0+0 (secure 0 0 0) 0x7ff26c15a4c0 con 0x7ff26c0b5330 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff25806f400 msgr2=0x7ff258071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff25806f400 0x7ff258071910 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff25c0097a0 tx=0x7ff25c008040 comp rx=0 tx=0).stop 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0b5330 msgr2=0x7ff26c158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0b5330 0x7ff26c158f40 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff26001a080 tx=0x7ff2600191f0 comp rx=0 tx=0).stop 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 shutdown_connections 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff25806f400 0x7ff258071910 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff26c0d0ff0 0x7ff26c160ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff26c0b5330 0x7ff26c158f40 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 --2- 172.21.15.80:0/3951127462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff26c06f2f0 0x7ff26c141f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:18.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 >> 172.21.15.80:0/3951127462 conn(0x7ff26c0b4160 msgr2=0x7ff26c0b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:18.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 shutdown_connections 2024-04-25T13:42:18.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.433+0000 7ff271e80640 1 -- 172.21.15.80:0/3951127462 wait complete. 2024-04-25T13:42:18.462 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:42:18.462 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:42:18.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cf2051640 1 Processor -- start 2024-04-25T13:42:18.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cf2051640 1 -- start start 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cf2051640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1cec06eb80 0x7f1cec167660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cf2051640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cec0737f0 con 0x7f1cec073080 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cf2051640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1cec1604d0 con 0x7f1cec073ad0 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cf2051640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1cec0b53d0 con 0x7f1cec06eb80 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cebfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1cec06eb80 0x7f1cec167660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1ceb7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1cec073080 0x7f1cec15c710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:46996/0 (socket says 172.21.15.80:46996) 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1ceb7fe640 1 -- 172.21.15.80:0/2765307993 learned_addr learned my addr 172.21.15.80:0/2765307993 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cebfff640 1 -- 172.21.15.80:0/2765307993 >> v1:172.21.15.87:6789/0 conn(0x7f1cec073ad0 legacy=0x7f1cec06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:18.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cebfff640 1 -- 172.21.15.80:0/2765307993 >> v1:172.21.15.80:6789/0 conn(0x7f1cec073080 legacy=0x7f1cec15c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:18.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cebfff640 1 -- 172.21.15.80:0/2765307993 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec1607b0 con 0x7f1cec06eb80 2024-04-25T13:42:18.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.617+0000 7f1cea7fc640 1 -- 172.21.15.80:0/2765307993 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1764622028 0 0) 0x7f1cec0737f0 con 0x7f1cec073080 2024-04-25T13:42:18.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cebfff640 1 --2- 172.21.15.80:0/2765307993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec06eb80 0x7f1cec167660 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40097c0 tx=0x7f1ce401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f25090f9457d5f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:18.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cea7fc640 1 -- 172.21.15.80:0/2765307993 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40b0070 con 0x7f1cec06eb80 2024-04-25T13:42:18.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cea7fc640 1 -- 172.21.15.80:0/2765307993 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce40ab600 con 0x7f1cec06eb80 2024-04-25T13:42:18.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cea7fc640 1 -- 172.21.15.80:0/2765307993 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40ab9d0 con 0x7f1cec06eb80 2024-04-25T13:42:18.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cf2051640 1 -- 172.21.15.80:0/2765307993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec06eb80 msgr2=0x7f1cec167660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:18.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cf2051640 1 --2- 172.21.15.80:0/2765307993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec06eb80 0x7f1cec167660 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40097c0 tx=0x7f1ce401c410 comp rx=0 tx=0).stop 2024-04-25T13:42:18.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cf2051640 1 -- 172.21.15.80:0/2765307993 shutdown_connections 2024-04-25T13:42:18.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cf2051640 1 --2- 172.21.15.80:0/2765307993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec06eb80 0x7f1cec167660 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:18.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cf2051640 1 -- 172.21.15.80:0/2765307993 >> 172.21.15.80:0/2765307993 conn(0x7f1cec0b4160 msgr2=0x7f1cec0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:18.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cf2051640 1 -- 172.21.15.80:0/2765307993 shutdown_connections 2024-04-25T13:42:18.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.621+0000 7f1cf2051640 1 -- 172.21.15.80:0/2765307993 wait complete. 2024-04-25T13:42:18.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 Processor -- start 2024-04-25T13:42:18.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 -- start start 2024-04-25T13:42:18.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cec073080 0x7f1cec14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cec073ad0 0x7f1cec152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec152680 0x7f1cec158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec169f40 con 0x7f1cec152680 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1cec169dc0 con 0x7f1cec073080 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1cec16a0c0 con 0x7f1cec073ad0 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec152680 0x7f1cec158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec152680 0x7f1cec158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53052/0 (socket says 172.21.15.80:53052) 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 -- 172.21.15.80:0/2499586303 learned_addr learned my addr 172.21.15.80:0/2499586303 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1ceb7fe640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cec073080 0x7f1cec14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1ceaffd640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cec073ad0 0x7f1cec152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 -- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cec073ad0 msgr2=0x7f1cec152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cec073ad0 0x7f1cec152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 -- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cec073080 msgr2=0x7f1cec14e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cec073080 0x7f1cec14e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:18.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 -- 172.21.15.80:0/2499586303 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cec15c9f0 con 0x7f1cec152680 2024-04-25T13:42:18.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cebfff640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec152680 0x7f1cec158340 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4019020 tx=0x7f1ce401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:18.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1ce3fff640 1 -- 172.21.15.80:0/2499586303 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40b3610 con 0x7f1cec152680 2024-04-25T13:42:18.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1ce3fff640 1 -- 172.21.15.80:0/2499586303 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ce40b37b0 con 0x7f1cec152680 2024-04-25T13:42:18.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1ce3fff640 1 -- 172.21.15.80:0/2499586303 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ce40b3b80 con 0x7f1cec152680 2024-04-25T13:42:18.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cec15ccd0 con 0x7f1cec152680 2024-04-25T13:42:18.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.625+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cec15d290 con 0x7f1cec152680 2024-04-25T13:42:18.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.629+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cec15c760 con 0x7f1cec152680 2024-04-25T13:42:18.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.629+0000 7f1ce3fff640 1 -- 172.21.15.80:0/2499586303 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1ce40c73f0 con 0x7f1cec152680 2024-04-25T13:42:18.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.629+0000 7f1ce3fff640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1ccc06f400 0x7f1ccc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:18.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.629+0000 7f1ce3fff640 1 -- 172.21.15.80:0/2499586303 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f1ce412bb50 con 0x7f1cec152680 2024-04-25T13:42:18.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.629+0000 7f1ceb7fe640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1ccc06f400 0x7f1ccc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:18.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.629+0000 7f1ceb7fe640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1ccc06f400 0x7f1ccc071910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005fd0 tx=0x7f1cdc005c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:18.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.633+0000 7f1ce3fff640 1 -- 172.21.15.80:0/2499586303 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ce40a4e20 con 0x7f1cec152680 2024-04-25T13:42:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:18.833+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f1cec06a3a0 con 0x7f1cec152680 2024-04-25T13:42:19.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.429+0000 7f1ce3fff640 1 -- 172.21.15.80:0/2499586303 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) ==== 122+0+0 (secure 0 0 0) 0x7f1ce4016090 con 0x7f1cec152680 2024-04-25T13:42:19.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1ccc06f400 msgr2=0x7f1ccc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1ccc06f400 0x7f1ccc071910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc005fd0 tx=0x7f1cdc005c20 comp rx=0 tx=0).stop 2024-04-25T13:42:19.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec152680 msgr2=0x7f1cec158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec152680 0x7f1cec158340 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1ce4019020 tx=0x7f1ce401ac50 comp rx=0 tx=0).stop 2024-04-25T13:42:19.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 shutdown_connections 2024-04-25T13:42:19.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1ccc06f400 0x7f1ccc071910 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1cec152680 0x7f1cec158340 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1cec073ad0 0x7f1cec152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 --2- 172.21.15.80:0/2499586303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1cec073080 0x7f1cec14e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 >> 172.21.15.80:0/2499586303 conn(0x7f1cec0b4160 msgr2=0x7f1cec159230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 shutdown_connections 2024-04-25T13:42:19.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.433+0000 7f1cf2051640 1 -- 172.21.15.80:0/2499586303 wait complete. 2024-04-25T13:42:19.463 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:42:19.463 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:19.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4198a3640 1 Processor -- start 2024-04-25T13:42:19.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4198a3640 1 -- start start 2024-04-25T13:42:19.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4198a3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4140d2310 0x7fd4140d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4198a3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd4140689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4198a3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd414162ae0 con 0x7fd41406c5f0 2024-04-25T13:42:19.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4198a3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd414068ee0 con 0x7fd4140d2310 2024-04-25T13:42:19.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4198a3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4140c77d0 con 0x7fd414068590 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4127fc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd4140689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd412ffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4140d2310 0x7fd4140d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd412ffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4140d2310 0x7fd4140d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:41376/0 (socket says 172.21.15.80:41376) 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd412ffd640 1 -- 172.21.15.80:0/2044661786 learned_addr learned my addr 172.21.15.80:0/2044661786 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.613+0000 7fd4137fe640 1 --1- 172.21.15.80:0/2044661786 >> v1:172.21.15.87:6789/0 conn(0x7fd41406c5f0 0x7fd41406ca00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:46760/0 (socket says 172.21.15.80:46760) 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd412ffd640 1 -- 172.21.15.80:0/2044661786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 msgr2=0x7fd4140689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd412ffd640 1 --2- 172.21.15.80:0/2044661786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd4140689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd412ffd640 1 -- 172.21.15.80:0/2044661786 >> v1:172.21.15.87:6789/0 conn(0x7fd41406c5f0 legacy=0x7fd41406ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd412ffd640 1 -- 172.21.15.80:0/2044661786 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4140d0cf0 con 0x7fd4140d2310 2024-04-25T13:42:19.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4127fc640 1 --2- 172.21.15.80:0/2044661786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd4140689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:19.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd412ffd640 1 --2- 172.21.15.80:0/2044661786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4140d2310 0x7fd4140d04e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd404009870 tx=0x7fd40401c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e3fd86eca36b3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd411ffb640 1 -- 172.21.15.80:0/2044661786 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040b0070 con 0x7fd4140d2310 2024-04-25T13:42:19.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd411ffb640 1 -- 172.21.15.80:0/2044661786 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4040ab600 con 0x7fd4140d2310 2024-04-25T13:42:19.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd411ffb640 1 -- 172.21.15.80:0/2044661786 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4040ab9d0 con 0x7fd4140d2310 2024-04-25T13:42:19.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4198a3640 1 -- 172.21.15.80:0/2044661786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4140d2310 msgr2=0x7fd4140d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2044661786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4140d2310 0x7fd4140d04e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd404009870 tx=0x7fd40401c4d0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4198a3640 1 -- 172.21.15.80:0/2044661786 shutdown_connections 2024-04-25T13:42:19.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2044661786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd4140689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2044661786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4140d2310 0x7fd4140d04e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4198a3640 1 -- 172.21.15.80:0/2044661786 >> 172.21.15.80:0/2044661786 conn(0x7fd4140cf390 msgr2=0x7fd4140cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4198a3640 1 -- 172.21.15.80:0/2044661786 shutdown_connections 2024-04-25T13:42:19.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.617+0000 7fd4198a3640 1 -- 172.21.15.80:0/2044661786 wait complete. 2024-04-25T13:42:19.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4198a3640 1 Processor -- start 2024-04-25T13:42:19.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4198a3640 1 -- start start 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4198a3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd414151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4198a3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd41406c5f0 0x7fd41413e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4198a3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4140d2310 0x7fd41415fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4198a3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd41416b730 con 0x7fd4140d2310 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4198a3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4141622e0 con 0x7fd41406c5f0 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4198a3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd41416b8b0 con 0x7fd414068590 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4140d2310 0x7fd41415fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4140d2310 0x7fd41415fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53066/0 (socket says 172.21.15.80:53066) 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 -- 172.21.15.80:0/2794230187 learned_addr learned my addr 172.21.15.80:0/2794230187 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:19.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd412ffd640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd414151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4127fc640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd41406c5f0 0x7fd41413e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 -- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 msgr2=0x7fd414151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd414151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 -- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd41406c5f0 msgr2=0x7fd41413e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd41406c5f0 0x7fd41413e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 -- 172.21.15.80:0/2794230187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4140cddb0 con 0x7fd4140d2310 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd4137fe640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4140d2310 0x7fd41415fd40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c6c0 tx=0x7fd40c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.621+0000 7fd3fbfff640 1 -- 172.21.15.80:0/2794230187 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c018070 con 0x7fd4140d2310 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd3fbfff640 1 -- 172.21.15.80:0/2794230187 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd40c004d70 con 0x7fd4140d2310 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd3fbfff640 1 -- 172.21.15.80:0/2794230187 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40c00a840 con 0x7fd4140d2310 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41415e770 con 0x7fd4140d2310 2024-04-25T13:42:19.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41415ed30 con 0x7fd4140d2310 2024-04-25T13:42:19.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd41406a3a0 con 0x7fd4140d2310 2024-04-25T13:42:19.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd3fbfff640 1 -- 172.21.15.80:0/2794230187 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd40c00a9e0 con 0x7fd4140d2310 2024-04-25T13:42:19.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd3fbfff640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd40006f430 0x7fd400071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd3fbfff640 1 -- 172.21.15.80:0/2794230187 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fd40c08e7c0 con 0x7fd4140d2310 2024-04-25T13:42:19.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd412ffd640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd40006f430 0x7fd400071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.625+0000 7fd412ffd640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd40006f430 0x7fd400071940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd4040ad000 tx=0x7fd4040a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.629+0000 7fd3fbfff640 1 -- 172.21.15.80:0/2794230187 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd40c05b6b0 con 0x7fd4140d2310 2024-04-25T13:42:19.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.825+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd41415e4e0 con 0x7fd4140d2310 2024-04-25T13:42:19.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.825+0000 7fd3fbfff640 1 -- 172.21.15.80:0/2794230187 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7fd40c05afb0 con 0x7fd4140d2310 2024-04-25T13:42:19.831 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:19.831 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:42:19.831 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:42:19.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd40006f430 msgr2=0x7fd400071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd40006f430 0x7fd400071940 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd4040ad000 tx=0x7fd4040a6000 comp rx=0 tx=0).stop 2024-04-25T13:42:19.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4140d2310 msgr2=0x7fd41415fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4140d2310 0x7fd41415fd40 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd40c00c6c0 tx=0x7fd40c00cb80 comp rx=0 tx=0).stop 2024-04-25T13:42:19.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 shutdown_connections 2024-04-25T13:42:19.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd40006f430 0x7fd400071940 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4140d2310 0x7fd41415fd40 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd41406c5f0 0x7fd41413e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 --2- 172.21.15.80:0/2794230187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd414068590 0x7fd414151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 >> 172.21.15.80:0/2794230187 conn(0x7fd4140cf390 msgr2=0x7fd4140b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 shutdown_connections 2024-04-25T13:42:19.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.829+0000 7fd4198a3640 1 -- 172.21.15.80:0/2794230187 wait complete. 2024-04-25T13:42:19.852 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effbaff1640 1 Processor -- start 2024-04-25T13:42:19.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effbaff1640 1 -- start start 2024-04-25T13:42:19.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effbaff1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7effb40d4330 0x7effb40d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:19.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effbaff1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb4161060 con 0x7effb40d59b0 2024-04-25T13:42:19.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effbaff1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7effb4161340 con 0x7effb40d2e80 2024-04-25T13:42:19.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effbaff1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7effb40d2680 con 0x7effb40d4330 2024-04-25T13:42:19.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effb958e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7effb40d4330 0x7effb40d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:19.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effb3fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7effb40d59b0 0x7effb40d5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:46998/0 (socket says 172.21.15.80:46998) 2024-04-25T13:42:19.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effb3fff640 1 -- 172.21.15.80:0/73357214 learned_addr learned my addr 172.21.15.80:0/73357214 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:19.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effb958e640 1 -- 172.21.15.80:0/73357214 >> v1:172.21.15.87:6789/0 conn(0x7effb40d2e80 legacy=0x7effb41607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effb958e640 1 -- 172.21.15.80:0/73357214 >> v1:172.21.15.80:6789/0 conn(0x7effb40d59b0 legacy=0x7effb40d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effb37fe640 1 -- 172.21.15.80:0/73357214 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2840978606 0 0) 0x7effb4161060 con 0x7effb40d59b0 2024-04-25T13:42:19.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.989+0000 7effb958e640 1 -- 172.21.15.80:0/73357214 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb40d2960 con 0x7effb40d4330 2024-04-25T13:42:19.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effb958e640 1 --2- 172.21.15.80:0/73357214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d4330 0x7effb40d4740 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7effac009d90 tx=0x7effac01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf8e1cebb809f65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:19.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effb37fe640 1 -- 172.21.15.80:0/73357214 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0b2070 con 0x7effb40d4330 2024-04-25T13:42:19.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effb37fe640 1 -- 172.21.15.80:0/73357214 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac0a58b0 con 0x7effb40d4330 2024-04-25T13:42:19.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effb37fe640 1 -- 172.21.15.80:0/73357214 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0a5c80 con 0x7effb40d4330 2024-04-25T13:42:19.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 -- 172.21.15.80:0/73357214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d4330 msgr2=0x7effb40d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:19.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 --2- 172.21.15.80:0/73357214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d4330 0x7effb40d4740 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7effac009d90 tx=0x7effac01c8e0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 -- 172.21.15.80:0/73357214 shutdown_connections 2024-04-25T13:42:19.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 --2- 172.21.15.80:0/73357214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d4330 0x7effb40d4740 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:19.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 -- 172.21.15.80:0/73357214 >> 172.21.15.80:0/73357214 conn(0x7effb40737a0 msgr2=0x7effb406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:19.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 -- 172.21.15.80:0/73357214 shutdown_connections 2024-04-25T13:42:19.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 -- 172.21.15.80:0/73357214 wait complete. 2024-04-25T13:42:19.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 Processor -- start 2024-04-25T13:42:19.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.993+0000 7effbaff1640 1 -- start start 2024-04-25T13:42:20.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effb40d2e80 0x7effb4156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effb40d4330 0x7effb40cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d59b0 0x7effb40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7effb415df60 con 0x7effb40d59b0 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7effb40d34a0 con 0x7effb40d4330 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7effb415e0e0 con 0x7effb40d2e80 2024-04-25T13:42:20.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d59b0 0x7effb40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d59b0 0x7effb40ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53098/0 (socket says 172.21.15.80:53098) 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 -- 172.21.15.80:0/1115300157 learned_addr learned my addr 172.21.15.80:0/1115300157 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb8d8d640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effb40d2e80 0x7effb4156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb3fff640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effb40d4330 0x7effb40cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 -- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effb40d2e80 msgr2=0x7effb4156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effb40d2e80 0x7effb4156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 -- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effb40d4330 msgr2=0x7effb40cc9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effb40d4330 0x7effb40cc9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 -- 172.21.15.80:0/1115300157 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb40cbd80 con 0x7effb40d59b0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb958e640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d59b0 0x7effb40ccef0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7effac0b0040 tx=0x7effac019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb17fa640 1 -- 172.21.15.80:0/1115300157 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac0b2050 con 0x7effb40d59b0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb17fa640 1 -- 172.21.15.80:0/1115300157 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7effac0b3040 con 0x7effb40d59b0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb40cefb0 con 0x7effb40d59b0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effb17fa640 1 -- 172.21.15.80:0/1115300157 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effac003dc0 con 0x7effb40d59b0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb40cf570 con 0x7effb40d59b0 2024-04-25T13:42:20.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:19.997+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effb40cf740 con 0x7effb40d59b0 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.001+0000 7effb17fa640 1 -- 172.21.15.80:0/1115300157 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7effac0ad710 con 0x7effb40d59b0 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.001+0000 7effb17fa640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff9806f400 0x7eff98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.001+0000 7effb17fa640 1 -- 172.21.15.80:0/1115300157 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7effac129fe0 con 0x7effb40d59b0 2024-04-25T13:42:20.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.001+0000 7effb8d8d640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff9806f400 0x7eff98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.001+0000 7effb8d8d640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff9806f400 0x7eff98071910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7effa8000f70 tx=0x7effa8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.005+0000 7effb17fa640 1 -- 172.21.15.80:0/1115300157 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7effb40cf740 con 0x7effb40d59b0 2024-04-25T13:42:20.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.153+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7effb406a3a0 con 0x7effb40d59b0 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.153+0000 7effb17fa640 1 -- 172.21.15.80:0/1115300157 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16541 (secure 0 0 0) 0x7effac0c8070 con 0x7effb40d59b0 2024-04-25T13:42:20.157 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:20.158 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":47,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:17.378064+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:42:13.650442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T13:42:14.725209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff9806f400 msgr2=0x7eff98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff9806f400 0x7eff98071910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7effa8000f70 tx=0x7effa8009040 comp rx=0 tx=0).stop 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d59b0 msgr2=0x7effb40ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d59b0 0x7effb40ccef0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7effac0b0040 tx=0x7effac019680 comp rx=0 tx=0).stop 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 shutdown_connections 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff9806f400 0x7eff98071910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7effb40d59b0 0x7effb40ccef0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7effb40d4330 0x7effb40cc9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 --2- 172.21.15.80:0/1115300157 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7effb40d2e80 0x7effb4156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 >> 172.21.15.80:0/1115300157 conn(0x7effb40737a0 msgr2=0x7effb40d5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 shutdown_connections 2024-04-25T13:42:20.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.157+0000 7effbaff1640 1 -- 172.21.15.80:0/1115300157 wait complete. 2024-04-25T13:42:20.178 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:42:20.178 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:42:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf3c7f640 1 Processor -- start 2024-04-25T13:42:20.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf3c7f640 1 -- start start 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf3c7f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4cec0d11c0 0x7f4cec072e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf3c7f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4cec1605b0 0x7f4cec06f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf3c7f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec06fc30 0x7f4cec0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf3c7f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4cec0cc720 con 0x7f4cec1605b0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf3c7f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4cec0cd210 con 0x7f4cec0d11c0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf3c7f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4cec070040 con 0x7f4cec06fc30 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf121a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4cec1605b0 0x7f4cec06f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf121a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4cec1605b0 0x7f4cec06f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53112/0 (socket says 172.21.15.80:53112) 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf121a640 1 -- 172.21.15.80:0/2136596338 learned_addr learned my addr 172.21.15.80:0/2136596338 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf221c640 1 --2- 172.21.15.80:0/2136596338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec06fc30 0x7f4cec0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf1a1b640 1 --2- 172.21.15.80:0/2136596338 >> v2:172.21.15.87:3300/0 conn(0x7f4cec0d11c0 0x7f4cec072e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf121a640 1 -- 172.21.15.80:0/2136596338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec06fc30 msgr2=0x7f4cec0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf121a640 1 --2- 172.21.15.80:0/2136596338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec06fc30 0x7f4cec0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf121a640 1 -- 172.21.15.80:0/2136596338 >> v2:172.21.15.87:3300/0 conn(0x7f4cec0d11c0 msgr2=0x7f4cec072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf121a640 1 --2- 172.21.15.80:0/2136596338 >> v2:172.21.15.87:3300/0 conn(0x7f4cec0d11c0 0x7f4cec072e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf121a640 1 -- 172.21.15.80:0/2136596338 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec073390 con 0x7f4cec1605b0 2024-04-25T13:42:20.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.309+0000 7f4cf1a1b640 1 --2- 172.21.15.80:0/2136596338 >> v2:172.21.15.87:3300/0 conn(0x7f4cec0d11c0 0x7f4cec072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf121a640 1 --2- 172.21.15.80:0/2136596338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec1605b0 0x7f4cec06f6f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4cec072160 tx=0x7f4ce001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3232da828a483f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf0a19640 1 -- 172.21.15.80:0/2136596338 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce00b0070 con 0x7f4cec1605b0 2024-04-25T13:42:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf0a19640 1 -- 172.21.15.80:0/2136596338 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce0004230 con 0x7f4cec1605b0 2024-04-25T13:42:20.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf0a19640 1 -- 172.21.15.80:0/2136596338 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0003620 con 0x7f4cec1605b0 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2136596338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec1605b0 msgr2=0x7f4cec06f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2136596338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec1605b0 0x7f4cec06f6f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4cec072160 tx=0x7f4ce001c490 comp rx=0 tx=0).stop 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2136596338 shutdown_connections 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2136596338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec06fc30 0x7f4cec0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2136596338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec1605b0 0x7f4cec06f6f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2136596338 >> v2:172.21.15.87:3300/0 conn(0x7f4cec0d11c0 0x7f4cec072e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2136596338 >> 172.21.15.80:0/2136596338 conn(0x7f4cec0d20d0 msgr2=0x7f4cec0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2136596338 shutdown_connections 2024-04-25T13:42:20.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.313+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2136596338 wait complete. 2024-04-25T13:42:20.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf3c7f640 1 Processor -- start 2024-04-25T13:42:20.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf3c7f640 1 -- start start 2024-04-25T13:42:20.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf3c7f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec06fc30 0x7f4cec0b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf3c7f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec0d11c0 0x7f4cec06e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf3c7f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cec06ecf0 0x7f4cec0b7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf3c7f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec0cfdd0 con 0x7f4cec06fc30 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf3c7f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4cec0ccf30 con 0x7f4cec06ecf0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf3c7f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4cec1609c0 con 0x7f4cec0d11c0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec06fc30 0x7f4cec0b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec06fc30 0x7f4cec0b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53114/0 (socket says 172.21.15.80:53114) 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 -- 172.21.15.80:0/2380275498 learned_addr learned my addr 172.21.15.80:0/2380275498 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf121a640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec0d11c0 0x7f4cec06e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf221c640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cec06ecf0 0x7f4cec0b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 -- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec0d11c0 msgr2=0x7f4cec06e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec0d11c0 0x7f4cec06e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 -- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cec06ecf0 msgr2=0x7f4cec0b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cec06ecf0 0x7f4cec0b7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 -- 172.21.15.80:0/2380275498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cec159f60 con 0x7f4cec06fc30 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.317+0000 7f4cf1a1b640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec06fc30 0x7f4cec0b27d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400cde0 tx=0x7f4ce400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cea7fc640 1 -- 172.21.15.80:0/2380275498 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400f680 con 0x7f4cec06fc30 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cea7fc640 1 -- 172.21.15.80:0/2380275498 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ce400f820 con 0x7f4cec06fc30 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cea7fc640 1 -- 172.21.15.80:0/2380275498 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ce400fc10 con 0x7f4cec06fc30 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cec159c80 con 0x7f4cec06fc30 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cec166990 con 0x7f4cec06fc30 2024-04-25T13:42:20.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cec06a3a0 con 0x7f4cec06fc30 2024-04-25T13:42:20.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cea7fc640 1 -- 172.21.15.80:0/2380275498 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4ce4018400 con 0x7f4cec06fc30 2024-04-25T13:42:20.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cea7fc640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4cd806f400 0x7f4cd8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cea7fc640 1 -- 172.21.15.80:0/2380275498 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f4ce4090440 con 0x7f4cec06fc30 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cf121a640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4cd806f400 0x7f4cd8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.321+0000 7f4cf121a640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4cd806f400 0x7f4cd8071910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00ad000 tx=0x7f4ce0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.325+0000 7f4cea7fc640 1 -- 172.21.15.80:0/2380275498 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4cec159f60 con 0x7f4cec06fc30 2024-04-25T13:42:20.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.469+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4cec070500 con 0x7f4cd806f400 2024-04-25T13:42:20.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.489+0000 7f4cea7fc640 1 -- 172.21.15.80:0/2380275498 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247478 (secure 0 0 0) 0x7f4cec070500 con 0x7f4cd806f400 2024-04-25T13:42:20.496 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:20.498 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4cd806f400 msgr2=0x7f4cd8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4cd806f400 0x7f4cd8071910 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4ce00ad000 tx=0x7f4ce0009040 comp rx=0 tx=0).stop 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec06fc30 msgr2=0x7f4cec0b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec06fc30 0x7f4cec0b27d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4ce400cde0 tx=0x7f4ce400a650 comp rx=0 tx=0).stop 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 shutdown_connections 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4cd806f400 0x7f4cd8071910 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4cec06ecf0 0x7f4cec0b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4cec0d11c0 0x7f4cec06e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 --2- 172.21.15.80:0/2380275498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4cec06fc30 0x7f4cec0b27d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 >> 172.21.15.80:0/2380275498 conn(0x7f4cec0d20d0 msgr2=0x7f4cec0af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 shutdown_connections 2024-04-25T13:42:20.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.493+0000 7f4cf3c7f640 1 -- 172.21.15.80:0/2380275498 wait complete. 2024-04-25T13:42:20.515 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T13:42:20.132929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2923,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559576,"statfs":{"total":1600000000,"available":1597006496,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.769632"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384687+0000","last_change":"2024-04-25T13:42:16.425298+0000","last_active":"2024-04-25T13:42:17.384687+0000","last_peered":"2024-04-25T13:42:17.384687+0000","last_clean":"2024-04-25T13:42:17.384687+0000","last_became_active":"2024-04-25T13:42:16.423503+0000","last_became_peered":"2024-04-25T13:42:16.423503+0000","last_unstale":"2024-04-25T13:42:17.384687+0000","last_undegraded":"2024-04-25T13:42:17.384687+0000","last_fullsized":"2024-04-25T13:42:17.384687+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:53.043550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460165+0000","last_change":"2024-04-25T13:42:16.416452+0000","last_active":"2024-04-25T13:42:17.460165+0000","last_peered":"2024-04-25T13:42:17.460165+0000","last_clean":"2024-04-25T13:42:17.460165+0000","last_became_active":"2024-04-25T13:42:16.414344+0000","last_became_peered":"2024-04-25T13:42:16.414344+0000","last_unstale":"2024-04-25T13:42:17.460165+0000","last_undegraded":"2024-04-25T13:42:17.460165+0000","last_fullsized":"2024-04-25T13:42:17.460165+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:39.032537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430936+0000","last_change":"2024-04-25T13:42:16.416442+0000","last_active":"2024-04-25T13:42:17.430936+0000","last_peered":"2024-04-25T13:42:17.430936+0000","last_clean":"2024-04-25T13:42:17.430936+0000","last_became_active":"2024-04-25T13:42:16.411947+0000","last_became_peered":"2024-04-25T13:42:16.411947+0000","last_unstale":"2024-04-25T13:42:17.430936+0000","last_undegraded":"2024-04-25T13:42:17.430936+0000","last_fullsized":"2024-04-25T13:42:17.430936+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:18.168600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461770+0000","last_change":"2024-04-25T13:42:16.423547+0000","last_active":"2024-04-25T13:42:17.461770+0000","last_peered":"2024-04-25T13:42:17.461770+0000","last_clean":"2024-04-25T13:42:17.461770+0000","last_became_active":"2024-04-25T13:42:16.412632+0000","last_became_peered":"2024-04-25T13:42:16.412632+0000","last_unstale":"2024-04-25T13:42:17.461770+0000","last_undegraded":"2024-04-25T13:42:17.461770+0000","last_fullsized":"2024-04-25T13:42:17.461770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.188328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461294+0000","last_change":"2024-04-25T13:42:16.414903+0000","last_active":"2024-04-25T13:42:17.461294+0000","last_peered":"2024-04-25T13:42:17.461294+0000","last_clean":"2024-04-25T13:42:17.461294+0000","last_became_active":"2024-04-25T13:42:16.414161+0000","last_became_peered":"2024-04-25T13:42:16.414161+0000","last_unstale":"2024-04-25T13:42:17.461294+0000","last_undegraded":"2024-04-25T13:42:17.461294+0000","last_fullsized":"2024-04-25T13:42:17.461294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:53.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383102+0000","last_change":"2024-04-25T13:42:16.429599+0000","last_active":"2024-04-25T13:42:17.383102+0000","last_peered":"2024-04-25T13:42:17.383102+0000","last_clean":"2024-04-25T13:42:17.383102+0000","last_became_active":"2024-04-25T13:42:16.428798+0000","last_became_peered":"2024-04-25T13:42:16.428798+0000","last_unstale":"2024-04-25T13:42:17.383102+0000","last_undegraded":"2024-04-25T13:42:17.383102+0000","last_fullsized":"2024-04-25T13:42:17.383102+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:51.933268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384030+0000","last_change":"2024-04-25T13:42:16.408282+0000","last_active":"2024-04-25T13:42:17.384030+0000","last_peered":"2024-04-25T13:42:17.384030+0000","last_clean":"2024-04-25T13:42:17.384030+0000","last_became_active":"2024-04-25T13:42:16.407524+0000","last_became_peered":"2024-04-25T13:42:16.407524+0000","last_unstale":"2024-04-25T13:42:17.384030+0000","last_undegraded":"2024-04-25T13:42:17.384030+0000","last_fullsized":"2024-04-25T13:42:17.384030+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:57.446397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461188+0000","last_change":"2024-04-25T13:42:16.426549+0000","last_active":"2024-04-25T13:42:17.461188+0000","last_peered":"2024-04-25T13:42:17.461188+0000","last_clean":"2024-04-25T13:42:17.461188+0000","last_became_active":"2024-04-25T13:42:16.426269+0000","last_became_peered":"2024-04-25T13:42:16.426269+0000","last_unstale":"2024-04-25T13:42:17.461188+0000","last_undegraded":"2024-04-25T13:42:17.461188+0000","last_fullsized":"2024-04-25T13:42:17.461188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:49.864859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462328+0000","last_change":"2024-04-25T13:42:16.434114+0000","last_active":"2024-04-25T13:42:17.462328+0000","last_peered":"2024-04-25T13:42:17.462328+0000","last_clean":"2024-04-25T13:42:17.462328+0000","last_became_active":"2024-04-25T13:42:16.427148+0000","last_became_peered":"2024-04-25T13:42:16.427148+0000","last_unstale":"2024-04-25T13:42:17.462328+0000","last_undegraded":"2024-04-25T13:42:17.462328+0000","last_fullsized":"2024-04-25T13:42:17.462328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:46.743085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460184+0000","last_change":"2024-04-25T13:42:16.430434+0000","last_active":"2024-04-25T13:42:17.460184+0000","last_peered":"2024-04-25T13:42:17.460184+0000","last_clean":"2024-04-25T13:42:17.460184+0000","last_became_active":"2024-04-25T13:42:16.420975+0000","last_became_peered":"2024-04-25T13:42:16.420975+0000","last_unstale":"2024-04-25T13:42:17.460184+0000","last_undegraded":"2024-04-25T13:42:17.460184+0000","last_fullsized":"2024-04-25T13:42:17.460184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:58.365482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459964+0000","last_change":"2024-04-25T13:42:16.419331+0000","last_active":"2024-04-25T13:42:17.459964+0000","last_peered":"2024-04-25T13:42:17.459964+0000","last_clean":"2024-04-25T13:42:17.459964+0000","last_became_active":"2024-04-25T13:42:16.417868+0000","last_became_peered":"2024-04-25T13:42:16.417868+0000","last_unstale":"2024-04-25T13:42:17.459964+0000","last_undegraded":"2024-04-25T13:42:17.459964+0000","last_fullsized":"2024-04-25T13:42:17.459964+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:58.820317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460683+0000","last_change":"2024-04-25T13:42:16.415786+0000","last_active":"2024-04-25T13:42:17.460683+0000","last_peered":"2024-04-25T13:42:17.460683+0000","last_clean":"2024-04-25T13:42:17.460683+0000","last_became_active":"2024-04-25T13:42:16.413178+0000","last_became_peered":"2024-04-25T13:42:16.413178+0000","last_unstale":"2024-04-25T13:42:17.460683+0000","last_undegraded":"2024-04-25T13:42:17.460683+0000","last_fullsized":"2024-04-25T13:42:17.460683+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:51.094841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461000+0000","last_change":"2024-04-25T13:42:16.408151+0000","last_active":"2024-04-25T13:42:17.461000+0000","last_peered":"2024-04-25T13:42:17.461000+0000","last_clean":"2024-04-25T13:42:17.461000+0000","last_became_active":"2024-04-25T13:42:16.407434+0000","last_became_peered":"2024-04-25T13:42:16.407434+0000","last_unstale":"2024-04-25T13:42:17.461000+0000","last_undegraded":"2024-04-25T13:42:17.461000+0000","last_fullsized":"2024-04-25T13:42:17.461000+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:39.327118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.463266+0000","last_change":"2024-04-25T13:42:16.436951+0000","last_active":"2024-04-25T13:42:17.463266+0000","last_peered":"2024-04-25T13:42:17.463266+0000","last_clean":"2024-04-25T13:42:17.463266+0000","last_became_active":"2024-04-25T13:42:16.433595+0000","last_became_peered":"2024-04-25T13:42:16.433595+0000","last_unstale":"2024-04-25T13:42:17.463266+0000","last_undegraded":"2024-04-25T13:42:17.463266+0000","last_fullsized":"2024-04-25T13:42:17.463266+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:37.573930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383243+0000","last_change":"2024-04-25T13:42:16.400608+0000","last_active":"2024-04-25T13:42:17.383243+0000","last_peered":"2024-04-25T13:42:17.383243+0000","last_clean":"2024-04-25T13:42:17.383243+0000","last_became_active":"2024-04-25T13:42:16.399982+0000","last_became_peered":"2024-04-25T13:42:16.399982+0000","last_unstale":"2024-04-25T13:42:17.383243+0000","last_undegraded":"2024-04-25T13:42:17.383243+0000","last_fullsized":"2024-04-25T13:42:17.383243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:28.443058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461652+0000","last_change":"2024-04-25T13:42:16.419438+0000","last_active":"2024-04-25T13:42:17.461652+0000","last_peered":"2024-04-25T13:42:17.461652+0000","last_clean":"2024-04-25T13:42:17.461652+0000","last_became_active":"2024-04-25T13:42:16.417710+0000","last_became_peered":"2024-04-25T13:42:16.417710+0000","last_unstale":"2024-04-25T13:42:17.461652+0000","last_undegraded":"2024-04-25T13:42:17.461652+0000","last_fullsized":"2024-04-25T13:42:17.461652+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:16.068579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430638+0000","last_change":"2024-04-25T13:42:16.404837+0000","last_active":"2024-04-25T13:42:17.430638+0000","last_peered":"2024-04-25T13:42:17.430638+0000","last_clean":"2024-04-25T13:42:17.430638+0000","last_became_active":"2024-04-25T13:42:16.403007+0000","last_became_peered":"2024-04-25T13:42:16.403007+0000","last_unstale":"2024-04-25T13:42:17.430638+0000","last_undegraded":"2024-04-25T13:42:17.430638+0000","last_fullsized":"2024-04-25T13:42:17.430638+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:32.234466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431664+0000","last_change":"2024-04-25T13:42:16.417270+0000","last_active":"2024-04-25T13:42:17.431664+0000","last_peered":"2024-04-25T13:42:17.431664+0000","last_clean":"2024-04-25T13:42:17.431664+0000","last_became_active":"2024-04-25T13:42:16.415514+0000","last_became_peered":"2024-04-25T13:42:16.415514+0000","last_unstale":"2024-04-25T13:42:17.431664+0000","last_undegraded":"2024-04-25T13:42:17.431664+0000","last_fullsized":"2024-04-25T13:42:17.431664+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:14.076352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462082+0000","last_change":"2024-04-25T13:42:16.419608+0000","last_active":"2024-04-25T13:42:17.462082+0000","last_peered":"2024-04-25T13:42:17.462082+0000","last_clean":"2024-04-25T13:42:17.462082+0000","last_became_active":"2024-04-25T13:42:16.418110+0000","last_became_peered":"2024-04-25T13:42:16.418110+0000","last_unstale":"2024-04-25T13:42:17.462082+0000","last_undegraded":"2024-04-25T13:42:17.462082+0000","last_fullsized":"2024-04-25T13:42:17.462082+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:02.829267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460218+0000","last_change":"2024-04-25T13:42:16.418587+0000","last_active":"2024-04-25T13:42:17.460218+0000","last_peered":"2024-04-25T13:42:17.460218+0000","last_clean":"2024-04-25T13:42:17.460218+0000","last_became_active":"2024-04-25T13:42:16.417861+0000","last_became_peered":"2024-04-25T13:42:16.417861+0000","last_unstale":"2024-04-25T13:42:17.460218+0000","last_undegraded":"2024-04-25T13:42:17.460218+0000","last_fullsized":"2024-04-25T13:42:17.460218+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:20.548128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383306+0000","last_change":"2024-04-25T13:42:16.427105+0000","last_active":"2024-04-25T13:42:17.383306+0000","last_peered":"2024-04-25T13:42:17.383306+0000","last_clean":"2024-04-25T13:42:17.383306+0000","last_became_active":"2024-04-25T13:42:16.424151+0000","last_became_peered":"2024-04-25T13:42:16.424151+0000","last_unstale":"2024-04-25T13:42:17.383306+0000","last_undegraded":"2024-04-25T13:42:17.383306+0000","last_fullsized":"2024-04-25T13:42:17.383306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:05.099427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459889+0000","last_change":"2024-04-25T13:42:16.432416+0000","last_active":"2024-04-25T13:42:17.459889+0000","last_peered":"2024-04-25T13:42:17.459889+0000","last_clean":"2024-04-25T13:42:17.459889+0000","last_became_active":"2024-04-25T13:42:16.421186+0000","last_became_peered":"2024-04-25T13:42:16.421186+0000","last_unstale":"2024-04-25T13:42:17.459889+0000","last_undegraded":"2024-04-25T13:42:17.459889+0000","last_fullsized":"2024-04-25T13:42:17.459889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:25.545826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462875+0000","last_change":"2024-04-25T13:42:16.416439+0000","last_active":"2024-04-25T13:42:17.462875+0000","last_peered":"2024-04-25T13:42:17.462875+0000","last_clean":"2024-04-25T13:42:17.462875+0000","last_became_active":"2024-04-25T13:42:16.408062+0000","last_became_peered":"2024-04-25T13:42:16.408062+0000","last_unstale":"2024-04-25T13:42:17.462875+0000","last_undegraded":"2024-04-25T13:42:17.462875+0000","last_fullsized":"2024-04-25T13:42:17.462875+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.454315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460919+0000","last_change":"2024-04-25T13:42:16.414813+0000","last_active":"2024-04-25T13:42:17.460919+0000","last_peered":"2024-04-25T13:42:17.460919+0000","last_clean":"2024-04-25T13:42:17.460919+0000","last_became_active":"2024-04-25T13:42:16.410809+0000","last_became_peered":"2024-04-25T13:42:16.410809+0000","last_unstale":"2024-04-25T13:42:17.460919+0000","last_undegraded":"2024-04-25T13:42:17.460919+0000","last_fullsized":"2024-04-25T13:42:17.460919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:19.955127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461868+0000","last_change":"2024-04-25T13:42:16.417740+0000","last_active":"2024-04-25T13:42:17.461868+0000","last_peered":"2024-04-25T13:42:17.461868+0000","last_clean":"2024-04-25T13:42:17.461868+0000","last_became_active":"2024-04-25T13:42:16.416888+0000","last_became_peered":"2024-04-25T13:42:16.416888+0000","last_unstale":"2024-04-25T13:42:17.461868+0000","last_undegraded":"2024-04-25T13:42:17.461868+0000","last_fullsized":"2024-04-25T13:42:17.461868+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:16.250839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460814+0000","last_change":"2024-04-25T13:42:16.428901+0000","last_active":"2024-04-25T13:42:17.460814+0000","last_peered":"2024-04-25T13:42:17.460814+0000","last_clean":"2024-04-25T13:42:17.460814+0000","last_became_active":"2024-04-25T13:42:16.428423+0000","last_became_peered":"2024-04-25T13:42:16.428423+0000","last_unstale":"2024-04-25T13:42:17.460814+0000","last_undegraded":"2024-04-25T13:42:17.460814+0000","last_fullsized":"2024-04-25T13:42:17.460814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:44.403509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430958+0000","last_change":"2024-04-25T13:42:16.403733+0000","last_active":"2024-04-25T13:42:17.430958+0000","last_peered":"2024-04-25T13:42:17.430958+0000","last_clean":"2024-04-25T13:42:17.430958+0000","last_became_active":"2024-04-25T13:42:16.401364+0000","last_became_peered":"2024-04-25T13:42:16.401364+0000","last_unstale":"2024-04-25T13:42:17.430958+0000","last_undegraded":"2024-04-25T13:42:17.430958+0000","last_fullsized":"2024-04-25T13:42:17.430958+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.181499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460534+0000","last_change":"2024-04-25T13:42:16.408097+0000","last_active":"2024-04-25T13:42:17.460534+0000","last_peered":"2024-04-25T13:42:17.460534+0000","last_clean":"2024-04-25T13:42:17.460534+0000","last_became_active":"2024-04-25T13:42:16.407350+0000","last_became_peered":"2024-04-25T13:42:16.407350+0000","last_unstale":"2024-04-25T13:42:17.460534+0000","last_undegraded":"2024-04-25T13:42:17.460534+0000","last_fullsized":"2024-04-25T13:42:17.460534+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:42.539110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461399+0000","last_change":"2024-04-25T13:42:16.419040+0000","last_active":"2024-04-25T13:42:17.461399+0000","last_peered":"2024-04-25T13:42:17.461399+0000","last_clean":"2024-04-25T13:42:17.461399+0000","last_became_active":"2024-04-25T13:42:16.416376+0000","last_became_peered":"2024-04-25T13:42:16.416376+0000","last_unstale":"2024-04-25T13:42:17.461399+0000","last_undegraded":"2024-04-25T13:42:17.461399+0000","last_fullsized":"2024-04-25T13:42:17.461399+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:34.729285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431159+0000","last_change":"2024-04-25T13:42:16.403446+0000","last_active":"2024-04-25T13:42:17.431159+0000","last_peered":"2024-04-25T13:42:17.431159+0000","last_clean":"2024-04-25T13:42:17.431159+0000","last_became_active":"2024-04-25T13:42:16.400150+0000","last_became_peered":"2024-04-25T13:42:16.400150+0000","last_unstale":"2024-04-25T13:42:17.431159+0000","last_undegraded":"2024-04-25T13:42:17.431159+0000","last_fullsized":"2024-04-25T13:42:17.431159+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:33.293784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383482+0000","last_change":"2024-04-25T13:42:16.425221+0000","last_active":"2024-04-25T13:42:17.383482+0000","last_peered":"2024-04-25T13:42:17.383482+0000","last_clean":"2024-04-25T13:42:17.383482+0000","last_became_active":"2024-04-25T13:42:16.419396+0000","last_became_peered":"2024-04-25T13:42:16.419396+0000","last_unstale":"2024-04-25T13:42:17.383482+0000","last_undegraded":"2024-04-25T13:42:17.383482+0000","last_fullsized":"2024-04-25T13:42:17.383482+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:48.202636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460376+0000","last_change":"2024-04-25T13:42:16.410076+0000","last_active":"2024-04-25T13:42:17.460376+0000","last_peered":"2024-04-25T13:42:17.460376+0000","last_clean":"2024-04-25T13:42:17.460376+0000","last_became_active":"2024-04-25T13:42:16.409390+0000","last_became_peered":"2024-04-25T13:42:16.409390+0000","last_unstale":"2024-04-25T13:42:17.460376+0000","last_undegraded":"2024-04-25T13:42:17.460376+0000","last_fullsized":"2024-04-25T13:42:17.460376+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:03.966036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.458979+0000","last_change":"2024-04-25T13:42:16.421671+0000","last_active":"2024-04-25T13:42:17.458979+0000","last_peered":"2024-04-25T13:42:17.458979+0000","last_clean":"2024-04-25T13:42:17.458979+0000","last_became_active":"2024-04-25T13:42:16.420405+0000","last_became_peered":"2024-04-25T13:42:16.420405+0000","last_unstale":"2024-04-25T13:42:17.458979+0000","last_undegraded":"2024-04-25T13:42:17.458979+0000","last_fullsized":"2024-04-25T13:42:17.458979+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:10.569450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383430+0000","last_change":"2024-04-25T13:42:15.411341+0000","last_active":"2024-04-25T13:42:17.383430+0000","last_peered":"2024-04-25T13:42:17.383430+0000","last_clean":"2024-04-25T13:42:17.383430+0000","last_became_active":"2024-04-25T13:42:15.411109+0000","last_became_peered":"2024-04-25T13:42:15.411109+0000","last_unstale":"2024-04-25T13:42:17.383430+0000","last_undegraded":"2024-04-25T13:42:17.383430+0000","last_fullsized":"2024-04-25T13:42:17.383430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:17.066430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462483+0000","last_change":"2024-04-25T13:42:16.424669+0000","last_active":"2024-04-25T13:42:17.462483+0000","last_peered":"2024-04-25T13:42:17.462483+0000","last_clean":"2024-04-25T13:42:17.462483+0000","last_became_active":"2024-04-25T13:42:16.423653+0000","last_became_peered":"2024-04-25T13:42:16.423653+0000","last_unstale":"2024-04-25T13:42:17.462483+0000","last_undegraded":"2024-04-25T13:42:17.462483+0000","last_fullsized":"2024-04-25T13:42:17.462483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:28.054773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430521+0000","last_change":"2024-04-25T13:42:15.413835+0000","last_active":"2024-04-25T13:42:17.430521+0000","last_peered":"2024-04-25T13:42:17.430521+0000","last_clean":"2024-04-25T13:42:17.430521+0000","last_became_active":"2024-04-25T13:42:15.412893+0000","last_became_peered":"2024-04-25T13:42:15.412893+0000","last_unstale":"2024-04-25T13:42:17.430521+0000","last_undegraded":"2024-04-25T13:42:17.430521+0000","last_fullsized":"2024-04-25T13:42:17.430521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:35.794581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460733+0000","last_change":"2024-04-25T13:42:16.421262+0000","last_active":"2024-04-25T13:42:17.460733+0000","last_peered":"2024-04-25T13:42:17.460733+0000","last_clean":"2024-04-25T13:42:17.460733+0000","last_became_active":"2024-04-25T13:42:16.420320+0000","last_became_peered":"2024-04-25T13:42:16.420320+0000","last_unstale":"2024-04-25T13:42:17.460733+0000","last_undegraded":"2024-04-25T13:42:17.460733+0000","last_fullsized":"2024-04-25T13:42:17.460733+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.872433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460618+0000","last_change":"2024-04-25T13:42:15.399964+0000","last_active":"2024-04-25T13:42:17.460618+0000","last_peered":"2024-04-25T13:42:17.460618+0000","last_clean":"2024-04-25T13:42:17.460618+0000","last_became_active":"2024-04-25T13:42:15.399066+0000","last_became_peered":"2024-04-25T13:42:15.399066+0000","last_unstale":"2024-04-25T13:42:17.460618+0000","last_undegraded":"2024-04-25T13:42:17.460618+0000","last_fullsized":"2024-04-25T13:42:17.460618+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:08.429549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460106+0000","last_change":"2024-04-25T13:42:16.416411+0000","last_active":"2024-04-25T13:42:17.460106+0000","last_peered":"2024-04-25T13:42:17.460106+0000","last_clean":"2024-04-25T13:42:17.460106+0000","last_became_active":"2024-04-25T13:42:16.407485+0000","last_became_peered":"2024-04-25T13:42:16.407485+0000","last_unstale":"2024-04-25T13:42:17.460106+0000","last_undegraded":"2024-04-25T13:42:17.460106+0000","last_fullsized":"2024-04-25T13:42:17.460106+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.364772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.385122+0000","last_change":"2024-04-25T13:42:15.411790+0000","last_active":"2024-04-25T13:42:17.385122+0000","last_peered":"2024-04-25T13:42:17.385122+0000","last_clean":"2024-04-25T13:42:17.385122+0000","last_became_active":"2024-04-25T13:42:15.408452+0000","last_became_peered":"2024-04-25T13:42:15.408452+0000","last_unstale":"2024-04-25T13:42:17.385122+0000","last_undegraded":"2024-04-25T13:42:17.385122+0000","last_fullsized":"2024-04-25T13:42:17.385122+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:30.813268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430612+0000","last_change":"2024-04-25T13:42:16.425462+0000","last_active":"2024-04-25T13:42:17.430612+0000","last_peered":"2024-04-25T13:42:17.430612+0000","last_clean":"2024-04-25T13:42:17.430612+0000","last_became_active":"2024-04-25T13:42:16.424644+0000","last_became_peered":"2024-04-25T13:42:16.424644+0000","last_unstale":"2024-04-25T13:42:17.430612+0000","last_undegraded":"2024-04-25T13:42:17.430612+0000","last_fullsized":"2024-04-25T13:42:17.430612+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:35.478074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383650+0000","last_change":"2024-04-25T13:42:15.407795+0000","last_active":"2024-04-25T13:42:17.383650+0000","last_peered":"2024-04-25T13:42:17.383650+0000","last_clean":"2024-04-25T13:42:17.383650+0000","last_became_active":"2024-04-25T13:42:15.407081+0000","last_became_peered":"2024-04-25T13:42:15.407081+0000","last_unstale":"2024-04-25T13:42:17.383650+0000","last_undegraded":"2024-04-25T13:42:17.383650+0000","last_fullsized":"2024-04-25T13:42:17.383650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:14.043951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461081+0000","last_change":"2024-04-25T13:42:16.429073+0000","last_active":"2024-04-25T13:42:17.461081+0000","last_peered":"2024-04-25T13:42:17.461081+0000","last_clean":"2024-04-25T13:42:17.461081+0000","last_became_active":"2024-04-25T13:42:16.418988+0000","last_became_peered":"2024-04-25T13:42:16.418988+0000","last_unstale":"2024-04-25T13:42:17.461081+0000","last_undegraded":"2024-04-25T13:42:17.461081+0000","last_fullsized":"2024-04-25T13:42:17.461081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:10.874165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431881+0000","last_change":"2024-04-25T13:42:15.417766+0000","last_active":"2024-04-25T13:42:17.431881+0000","last_peered":"2024-04-25T13:42:17.431881+0000","last_clean":"2024-04-25T13:42:17.431881+0000","last_became_active":"2024-04-25T13:42:15.414864+0000","last_became_peered":"2024-04-25T13:42:15.414864+0000","last_unstale":"2024-04-25T13:42:17.431881+0000","last_undegraded":"2024-04-25T13:42:17.431881+0000","last_fullsized":"2024-04-25T13:42:17.431881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:26.104533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460669+0000","last_change":"2024-04-25T13:42:16.430904+0000","last_active":"2024-04-25T13:42:17.460669+0000","last_peered":"2024-04-25T13:42:17.460669+0000","last_clean":"2024-04-25T13:42:17.460669+0000","last_became_active":"2024-04-25T13:42:16.423825+0000","last_became_peered":"2024-04-25T13:42:16.423825+0000","last_unstale":"2024-04-25T13:42:17.460669+0000","last_undegraded":"2024-04-25T13:42:17.460669+0000","last_fullsized":"2024-04-25T13:42:17.460669+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.901944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459943+0000","last_change":"2024-04-25T13:42:15.404953+0000","last_active":"2024-04-25T13:42:17.459943+0000","last_peered":"2024-04-25T13:42:17.459943+0000","last_clean":"2024-04-25T13:42:17.459943+0000","last_became_active":"2024-04-25T13:42:15.404172+0000","last_became_peered":"2024-04-25T13:42:15.404172+0000","last_unstale":"2024-04-25T13:42:17.459943+0000","last_undegraded":"2024-04-25T13:42:17.459943+0000","last_fullsized":"2024-04-25T13:42:17.459943+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:20.602777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461714+0000","last_change":"2024-04-25T13:42:16.408091+0000","last_active":"2024-04-25T13:42:17.461714+0000","last_peered":"2024-04-25T13:42:17.461714+0000","last_clean":"2024-04-25T13:42:17.461714+0000","last_became_active":"2024-04-25T13:42:16.401009+0000","last_became_peered":"2024-04-25T13:42:16.401009+0000","last_unstale":"2024-04-25T13:42:17.461714+0000","last_undegraded":"2024-04-25T13:42:17.461714+0000","last_fullsized":"2024-04-25T13:42:17.461714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:35.388008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.463093+0000","last_change":"2024-04-25T13:42:15.392736+0000","last_active":"2024-04-25T13:42:17.463093+0000","last_peered":"2024-04-25T13:42:17.463093+0000","last_clean":"2024-04-25T13:42:17.463093+0000","last_became_active":"2024-04-25T13:42:15.392434+0000","last_became_peered":"2024-04-25T13:42:15.392434+0000","last_unstale":"2024-04-25T13:42:17.463093+0000","last_undegraded":"2024-04-25T13:42:17.463093+0000","last_fullsized":"2024-04-25T13:42:17.463093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:57.244911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431256+0000","last_change":"2024-04-25T13:42:16.413741+0000","last_active":"2024-04-25T13:42:17.431256+0000","last_peered":"2024-04-25T13:42:17.431256+0000","last_clean":"2024-04-25T13:42:17.431256+0000","last_became_active":"2024-04-25T13:42:16.412905+0000","last_became_peered":"2024-04-25T13:42:16.412905+0000","last_unstale":"2024-04-25T13:42:17.431256+0000","last_undegraded":"2024-04-25T13:42:17.431256+0000","last_fullsized":"2024-04-25T13:42:17.431256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:26.341048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430838+0000","last_change":"2024-04-25T13:42:15.413276+0000","last_active":"2024-04-25T13:42:17.430838+0000","last_peered":"2024-04-25T13:42:17.430838+0000","last_clean":"2024-04-25T13:42:17.430838+0000","last_became_active":"2024-04-25T13:42:15.412451+0000","last_became_peered":"2024-04-25T13:42:15.412451+0000","last_unstale":"2024-04-25T13:42:17.430838+0000","last_undegraded":"2024-04-25T13:42:17.430838+0000","last_fullsized":"2024-04-25T13:42:17.430838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:32.415100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384953+0000","last_change":"2024-04-25T13:42:16.424634+0000","last_active":"2024-04-25T13:42:17.384953+0000","last_peered":"2024-04-25T13:42:17.384953+0000","last_clean":"2024-04-25T13:42:17.384953+0000","last_became_active":"2024-04-25T13:42:16.422596+0000","last_became_peered":"2024-04-25T13:42:16.422596+0000","last_unstale":"2024-04-25T13:42:17.384953+0000","last_undegraded":"2024-04-25T13:42:17.384953+0000","last_fullsized":"2024-04-25T13:42:17.384953+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:32.640227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461292+0000","last_change":"2024-04-25T13:42:15.407541+0000","last_active":"2024-04-25T13:42:17.461292+0000","last_peered":"2024-04-25T13:42:17.461292+0000","last_clean":"2024-04-25T13:42:17.461292+0000","last_became_active":"2024-04-25T13:42:15.407323+0000","last_became_peered":"2024-04-25T13:42:15.407323+0000","last_unstale":"2024-04-25T13:42:17.461292+0000","last_undegraded":"2024-04-25T13:42:17.461292+0000","last_fullsized":"2024-04-25T13:42:17.461292+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:26.948924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383655+0000","last_change":"2024-04-25T13:42:16.430684+0000","last_active":"2024-04-25T13:42:17.383655+0000","last_peered":"2024-04-25T13:42:17.383655+0000","last_clean":"2024-04-25T13:42:17.383655+0000","last_became_active":"2024-04-25T13:42:16.429463+0000","last_became_peered":"2024-04-25T13:42:16.429463+0000","last_unstale":"2024-04-25T13:42:17.383655+0000","last_undegraded":"2024-04-25T13:42:17.383655+0000","last_fullsized":"2024-04-25T13:42:17.383655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:25.261455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384852+0000","last_change":"2024-04-25T13:42:15.413865+0000","last_active":"2024-04-25T13:42:17.384852+0000","last_peered":"2024-04-25T13:42:17.384852+0000","last_clean":"2024-04-25T13:42:17.384852+0000","last_became_active":"2024-04-25T13:42:15.413149+0000","last_became_peered":"2024-04-25T13:42:15.413149+0000","last_unstale":"2024-04-25T13:42:17.384852+0000","last_undegraded":"2024-04-25T13:42:17.384852+0000","last_fullsized":"2024-04-25T13:42:17.384852+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:48.061192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384332+0000","last_change":"2024-04-25T13:42:16.411824+0000","last_active":"2024-04-25T13:42:17.384332+0000","last_peered":"2024-04-25T13:42:17.384332+0000","last_clean":"2024-04-25T13:42:17.384332+0000","last_became_active":"2024-04-25T13:42:16.406980+0000","last_became_peered":"2024-04-25T13:42:16.406980+0000","last_unstale":"2024-04-25T13:42:17.384332+0000","last_undegraded":"2024-04-25T13:42:17.384332+0000","last_fullsized":"2024-04-25T13:42:17.384332+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:08.936662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460256+0000","last_change":"2024-04-25T13:42:15.414113+0000","last_active":"2024-04-25T13:42:17.460256+0000","last_peered":"2024-04-25T13:42:17.460256+0000","last_clean":"2024-04-25T13:42:17.460256+0000","last_became_active":"2024-04-25T13:42:15.413880+0000","last_became_peered":"2024-04-25T13:42:15.413880+0000","last_unstale":"2024-04-25T13:42:17.460256+0000","last_undegraded":"2024-04-25T13:42:17.460256+0000","last_fullsized":"2024-04-25T13:42:17.460256+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:47.734749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462113+0000","last_change":"2024-04-25T13:42:16.418851+0000","last_active":"2024-04-25T13:42:17.462113+0000","last_peered":"2024-04-25T13:42:17.462113+0000","last_clean":"2024-04-25T13:42:17.462113+0000","last_became_active":"2024-04-25T13:42:16.416677+0000","last_became_peered":"2024-04-25T13:42:16.416677+0000","last_unstale":"2024-04-25T13:42:17.462113+0000","last_undegraded":"2024-04-25T13:42:17.462113+0000","last_fullsized":"2024-04-25T13:42:17.462113+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:56.135744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383597+0000","last_change":"2024-04-25T13:42:15.389941+0000","last_active":"2024-04-25T13:42:17.383597+0000","last_peered":"2024-04-25T13:42:17.383597+0000","last_clean":"2024-04-25T13:42:17.383597+0000","last_became_active":"2024-04-25T13:42:15.389636+0000","last_became_peered":"2024-04-25T13:42:15.389636+0000","last_unstale":"2024-04-25T13:42:17.383597+0000","last_undegraded":"2024-04-25T13:42:17.383597+0000","last_fullsized":"2024-04-25T13:42:17.383597+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:12.021337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461108+0000","last_change":"2024-04-25T13:42:16.429523+0000","last_active":"2024-04-25T13:42:17.461108+0000","last_peered":"2024-04-25T13:42:17.461108+0000","last_clean":"2024-04-25T13:42:17.461108+0000","last_became_active":"2024-04-25T13:42:16.419757+0000","last_became_peered":"2024-04-25T13:42:16.419757+0000","last_unstale":"2024-04-25T13:42:17.461108+0000","last_undegraded":"2024-04-25T13:42:17.461108+0000","last_fullsized":"2024-04-25T13:42:17.461108+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:35.924977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.385263+0000","last_change":"2024-04-25T13:42:15.414744+0000","last_active":"2024-04-25T13:42:17.385263+0000","last_peered":"2024-04-25T13:42:17.385263+0000","last_clean":"2024-04-25T13:42:17.385263+0000","last_became_active":"2024-04-25T13:42:15.409519+0000","last_became_peered":"2024-04-25T13:42:15.409519+0000","last_unstale":"2024-04-25T13:42:17.385263+0000","last_undegraded":"2024-04-25T13:42:17.385263+0000","last_fullsized":"2024-04-25T13:42:17.385263+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:13.605108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383067+0000","last_change":"2024-04-25T13:42:16.429336+0000","last_active":"2024-04-25T13:42:17.383067+0000","last_peered":"2024-04-25T13:42:17.383067+0000","last_clean":"2024-04-25T13:42:17.383067+0000","last_became_active":"2024-04-25T13:42:16.428703+0000","last_became_peered":"2024-04-25T13:42:16.428703+0000","last_unstale":"2024-04-25T13:42:17.383067+0000","last_undegraded":"2024-04-25T13:42:17.383067+0000","last_fullsized":"2024-04-25T13:42:17.383067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:39.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383932+0000","last_change":"2024-04-25T13:42:15.385603+0000","last_active":"2024-04-25T13:42:17.383932+0000","last_peered":"2024-04-25T13:42:17.383932+0000","last_clean":"2024-04-25T13:42:17.383932+0000","last_became_active":"2024-04-25T13:42:15.384733+0000","last_became_peered":"2024-04-25T13:42:15.384733+0000","last_unstale":"2024-04-25T13:42:17.383932+0000","last_undegraded":"2024-04-25T13:42:17.383932+0000","last_fullsized":"2024-04-25T13:42:17.383932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:16.197245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384521+0000","last_change":"2024-04-25T13:42:16.401537+0000","last_active":"2024-04-25T13:42:17.384521+0000","last_peered":"2024-04-25T13:42:17.384521+0000","last_clean":"2024-04-25T13:42:17.384521+0000","last_became_active":"2024-04-25T13:42:16.400818+0000","last_became_peered":"2024-04-25T13:42:16.400818+0000","last_unstale":"2024-04-25T13:42:17.384521+0000","last_undegraded":"2024-04-25T13:42:17.384521+0000","last_fullsized":"2024-04-25T13:42:17.384521+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:04.157809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462140+0000","last_change":"2024-04-25T13:42:15.391073+0000","last_active":"2024-04-25T13:42:17.462140+0000","last_peered":"2024-04-25T13:42:17.462140+0000","last_clean":"2024-04-25T13:42:17.462140+0000","last_became_active":"2024-04-25T13:42:15.390745+0000","last_became_peered":"2024-04-25T13:42:15.390745+0000","last_unstale":"2024-04-25T13:42:17.462140+0000","last_undegraded":"2024-04-25T13:42:17.462140+0000","last_fullsized":"2024-04-25T13:42:17.462140+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.905103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460399+0000","last_change":"2024-04-25T13:42:16.403900+0000","last_active":"2024-04-25T13:42:17.460399+0000","last_peered":"2024-04-25T13:42:17.460399+0000","last_clean":"2024-04-25T13:42:17.460399+0000","last_became_active":"2024-04-25T13:42:16.403329+0000","last_became_peered":"2024-04-25T13:42:16.403329+0000","last_unstale":"2024-04-25T13:42:17.460399+0000","last_undegraded":"2024-04-25T13:42:17.460399+0000","last_fullsized":"2024-04-25T13:42:17.460399+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:16.398749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460524+0000","last_change":"2024-04-25T13:42:15.385603+0000","last_active":"2024-04-25T13:42:17.460524+0000","last_peered":"2024-04-25T13:42:17.460524+0000","last_clean":"2024-04-25T13:42:17.460524+0000","last_became_active":"2024-04-25T13:42:15.385172+0000","last_became_peered":"2024-04-25T13:42:15.385172+0000","last_unstale":"2024-04-25T13:42:17.460524+0000","last_undegraded":"2024-04-25T13:42:17.460524+0000","last_fullsized":"2024-04-25T13:42:17.460524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:31.805247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462685+0000","last_change":"2024-04-25T13:42:16.434718+0000","last_active":"2024-04-25T13:42:17.462685+0000","last_peered":"2024-04-25T13:42:17.462685+0000","last_clean":"2024-04-25T13:42:17.462685+0000","last_became_active":"2024-04-25T13:42:16.428260+0000","last_became_peered":"2024-04-25T13:42:16.428260+0000","last_unstale":"2024-04-25T13:42:17.462685+0000","last_undegraded":"2024-04-25T13:42:17.462685+0000","last_fullsized":"2024-04-25T13:42:17.462685+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:17.272795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431374+0000","last_change":"2024-04-25T13:42:15.413526+0000","last_active":"2024-04-25T13:42:17.431374+0000","last_peered":"2024-04-25T13:42:17.431374+0000","last_clean":"2024-04-25T13:42:17.431374+0000","last_became_active":"2024-04-25T13:42:15.412705+0000","last_became_peered":"2024-04-25T13:42:15.412705+0000","last_unstale":"2024-04-25T13:42:17.431374+0000","last_undegraded":"2024-04-25T13:42:17.431374+0000","last_fullsized":"2024-04-25T13:42:17.431374+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:40.499839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462101+0000","last_change":"2024-04-25T13:42:16.417972+0000","last_active":"2024-04-25T13:42:17.462101+0000","last_peered":"2024-04-25T13:42:17.462101+0000","last_clean":"2024-04-25T13:42:17.462101+0000","last_became_active":"2024-04-25T13:42:16.416376+0000","last_became_peered":"2024-04-25T13:42:16.416376+0000","last_unstale":"2024-04-25T13:42:17.462101+0000","last_undegraded":"2024-04-25T13:42:17.462101+0000","last_fullsized":"2024-04-25T13:42:17.462101+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:12.325792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461011+0000","last_change":"2024-04-25T13:42:15.397902+0000","last_active":"2024-04-25T13:42:17.461011+0000","last_peered":"2024-04-25T13:42:17.461011+0000","last_clean":"2024-04-25T13:42:17.461011+0000","last_became_active":"2024-04-25T13:42:15.396626+0000","last_became_peered":"2024-04-25T13:42:15.396626+0000","last_unstale":"2024-04-25T13:42:17.461011+0000","last_undegraded":"2024-04-25T13:42:17.461011+0000","last_fullsized":"2024-04-25T13:42:17.461011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:06.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462427+0000","last_change":"2024-04-25T13:42:16.407378+0000","last_active":"2024-04-25T13:42:17.462427+0000","last_peered":"2024-04-25T13:42:17.462427+0000","last_clean":"2024-04-25T13:42:17.462427+0000","last_became_active":"2024-04-25T13:42:16.406685+0000","last_became_peered":"2024-04-25T13:42:16.406685+0000","last_unstale":"2024-04-25T13:42:17.462427+0000","last_undegraded":"2024-04-25T13:42:17.462427+0000","last_fullsized":"2024-04-25T13:42:17.462427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:43.797398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384186+0000","last_change":"2024-04-25T13:42:15.408070+0000","last_active":"2024-04-25T13:42:17.384186+0000","last_peered":"2024-04-25T13:42:17.384186+0000","last_clean":"2024-04-25T13:42:17.384186+0000","last_became_active":"2024-04-25T13:42:15.407495+0000","last_became_peered":"2024-04-25T13:42:15.407495+0000","last_unstale":"2024-04-25T13:42:17.384186+0000","last_undegraded":"2024-04-25T13:42:17.384186+0000","last_fullsized":"2024-04-25T13:42:17.384186+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:01.122799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383216+0000","last_change":"2024-04-25T13:42:16.430133+0000","last_active":"2024-04-25T13:42:17.383216+0000","last_peered":"2024-04-25T13:42:17.383216+0000","last_clean":"2024-04-25T13:42:17.383216+0000","last_became_active":"2024-04-25T13:42:16.419312+0000","last_became_peered":"2024-04-25T13:42:16.419312+0000","last_unstale":"2024-04-25T13:42:17.383216+0000","last_undegraded":"2024-04-25T13:42:17.383216+0000","last_fullsized":"2024-04-25T13:42:17.383216+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:06.396676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383954+0000","last_change":"2024-04-25T13:42:15.405691+0000","last_active":"2024-04-25T13:42:17.383954+0000","last_peered":"2024-04-25T13:42:17.383954+0000","last_clean":"2024-04-25T13:42:17.383954+0000","last_became_active":"2024-04-25T13:42:15.405438+0000","last_became_peered":"2024-04-25T13:42:15.405438+0000","last_unstale":"2024-04-25T13:42:17.383954+0000","last_undegraded":"2024-04-25T13:42:17.383954+0000","last_fullsized":"2024-04-25T13:42:17.383954+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:24.931013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459154+0000","last_change":"2024-04-25T13:42:16.387370+0000","last_active":"2024-04-25T13:42:17.459154+0000","last_peered":"2024-04-25T13:42:17.459154+0000","last_clean":"2024-04-25T13:42:17.459154+0000","last_became_active":"2024-04-25T13:42:16.386859+0000","last_became_peered":"2024-04-25T13:42:16.386859+0000","last_unstale":"2024-04-25T13:42:17.459154+0000","last_undegraded":"2024-04-25T13:42:17.459154+0000","last_fullsized":"2024-04-25T13:42:17.459154+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:16.111737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460236+0000","last_change":"2024-04-25T13:42:15.410324+0000","last_active":"2024-04-25T13:42:17.460236+0000","last_peered":"2024-04-25T13:42:17.460236+0000","last_clean":"2024-04-25T13:42:17.460236+0000","last_became_active":"2024-04-25T13:42:15.409511+0000","last_became_peered":"2024-04-25T13:42:15.409511+0000","last_unstale":"2024-04-25T13:42:17.460236+0000","last_undegraded":"2024-04-25T13:42:17.460236+0000","last_fullsized":"2024-04-25T13:42:17.460236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:01.086491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460915+0000","last_change":"2024-04-25T13:42:16.425006+0000","last_active":"2024-04-25T13:42:17.460915+0000","last_peered":"2024-04-25T13:42:17.460915+0000","last_clean":"2024-04-25T13:42:17.460915+0000","last_became_active":"2024-04-25T13:42:16.424668+0000","last_became_peered":"2024-04-25T13:42:16.424668+0000","last_unstale":"2024-04-25T13:42:17.460915+0000","last_undegraded":"2024-04-25T13:42:17.460915+0000","last_fullsized":"2024-04-25T13:42:17.460915+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:23.908161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.463295+0000","last_change":"2024-04-25T13:42:15.405678+0000","last_active":"2024-04-25T13:42:17.463295+0000","last_peered":"2024-04-25T13:42:17.463295+0000","last_clean":"2024-04-25T13:42:17.463295+0000","last_became_active":"2024-04-25T13:42:15.405438+0000","last_became_peered":"2024-04-25T13:42:15.405438+0000","last_unstale":"2024-04-25T13:42:17.463295+0000","last_undegraded":"2024-04-25T13:42:17.463295+0000","last_fullsized":"2024-04-25T13:42:17.463295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.389047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461662+0000","last_change":"2024-04-25T13:42:16.431862+0000","last_active":"2024-04-25T13:42:17.461662+0000","last_peered":"2024-04-25T13:42:17.461662+0000","last_clean":"2024-04-25T13:42:17.461662+0000","last_became_active":"2024-04-25T13:42:16.429155+0000","last_became_peered":"2024-04-25T13:42:16.429155+0000","last_unstale":"2024-04-25T13:42:17.461662+0000","last_undegraded":"2024-04-25T13:42:17.461662+0000","last_fullsized":"2024-04-25T13:42:17.461662+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:48.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.385529+0000","last_change":"2024-04-25T13:42:15.415424+0000","last_active":"2024-04-25T13:42:17.385529+0000","last_peered":"2024-04-25T13:42:17.385529+0000","last_clean":"2024-04-25T13:42:17.385529+0000","last_became_active":"2024-04-25T13:42:15.411095+0000","last_became_peered":"2024-04-25T13:42:15.411095+0000","last_unstale":"2024-04-25T13:42:17.385529+0000","last_undegraded":"2024-04-25T13:42:17.385529+0000","last_fullsized":"2024-04-25T13:42:17.385529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:56.921837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462520+0000","last_change":"2024-04-25T13:42:16.410045+0000","last_active":"2024-04-25T13:42:17.462520+0000","last_peered":"2024-04-25T13:42:17.462520+0000","last_clean":"2024-04-25T13:42:17.462520+0000","last_became_active":"2024-04-25T13:42:16.409320+0000","last_became_peered":"2024-04-25T13:42:16.409320+0000","last_unstale":"2024-04-25T13:42:17.462520+0000","last_undegraded":"2024-04-25T13:42:17.462520+0000","last_fullsized":"2024-04-25T13:42:17.462520+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:42.499763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.429531+0000","last_change":"2024-04-25T13:42:15.416407+0000","last_active":"2024-04-25T13:42:17.429531+0000","last_peered":"2024-04-25T13:42:17.429531+0000","last_clean":"2024-04-25T13:42:17.429531+0000","last_became_active":"2024-04-25T13:42:15.416220+0000","last_became_peered":"2024-04-25T13:42:15.416220+0000","last_unstale":"2024-04-25T13:42:17.429531+0000","last_undegraded":"2024-04-25T13:42:17.429531+0000","last_fullsized":"2024-04-25T13:42:17.429531+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:55.196144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462279+0000","last_change":"2024-04-25T13:42:16.431379+0000","last_active":"2024-04-25T13:42:17.462279+0000","last_peered":"2024-04-25T13:42:17.462279+0000","last_clean":"2024-04-25T13:42:17.462279+0000","last_became_active":"2024-04-25T13:42:16.425076+0000","last_became_peered":"2024-04-25T13:42:16.425076+0000","last_unstale":"2024-04-25T13:42:17.462279+0000","last_undegraded":"2024-04-25T13:42:17.462279+0000","last_fullsized":"2024-04-25T13:42:17.462279+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:54.679365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384575+0000","last_change":"2024-04-25T13:42:15.413673+0000","last_active":"2024-04-25T13:42:17.384575+0000","last_peered":"2024-04-25T13:42:17.384575+0000","last_clean":"2024-04-25T13:42:17.384575+0000","last_became_active":"2024-04-25T13:42:15.412944+0000","last_became_peered":"2024-04-25T13:42:15.412944+0000","last_unstale":"2024-04-25T13:42:17.384575+0000","last_undegraded":"2024-04-25T13:42:17.384575+0000","last_fullsized":"2024-04-25T13:42:17.384575+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:31.674149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384243+0000","last_change":"2024-04-25T13:42:16.410338+0000","last_active":"2024-04-25T13:42:17.384243+0000","last_peered":"2024-04-25T13:42:17.384243+0000","last_clean":"2024-04-25T13:42:17.384243+0000","last_became_active":"2024-04-25T13:42:16.409511+0000","last_became_peered":"2024-04-25T13:42:16.409511+0000","last_unstale":"2024-04-25T13:42:17.384243+0000","last_undegraded":"2024-04-25T13:42:17.384243+0000","last_fullsized":"2024-04-25T13:42:17.384243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:15.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460892+0000","last_change":"2024-04-25T13:42:15.400120+0000","last_active":"2024-04-25T13:42:17.460892+0000","last_peered":"2024-04-25T13:42:17.460892+0000","last_clean":"2024-04-25T13:42:17.460892+0000","last_became_active":"2024-04-25T13:42:15.399445+0000","last_became_peered":"2024-04-25T13:42:15.399445+0000","last_unstale":"2024-04-25T13:42:17.460892+0000","last_undegraded":"2024-04-25T13:42:17.460892+0000","last_fullsized":"2024-04-25T13:42:17.460892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:07.489364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383310+0000","last_change":"2024-04-25T13:42:16.411710+0000","last_active":"2024-04-25T13:42:17.383310+0000","last_peered":"2024-04-25T13:42:17.383310+0000","last_clean":"2024-04-25T13:42:17.383310+0000","last_became_active":"2024-04-25T13:42:16.410819+0000","last_became_peered":"2024-04-25T13:42:16.410819+0000","last_unstale":"2024-04-25T13:42:17.383310+0000","last_undegraded":"2024-04-25T13:42:17.383310+0000","last_fullsized":"2024-04-25T13:42:17.383310+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:44.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.385397+0000","last_change":"2024-04-25T13:42:15.417068+0000","last_active":"2024-04-25T13:42:17.385397+0000","last_peered":"2024-04-25T13:42:17.385397+0000","last_clean":"2024-04-25T13:42:17.385397+0000","last_became_active":"2024-04-25T13:42:15.416290+0000","last_became_peered":"2024-04-25T13:42:15.416290+0000","last_unstale":"2024-04-25T13:42:17.385397+0000","last_undegraded":"2024-04-25T13:42:17.385397+0000","last_fullsized":"2024-04-25T13:42:17.385397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:21.007577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383666+0000","last_change":"2024-04-25T13:42:16.427646+0000","last_active":"2024-04-25T13:42:17.383666+0000","last_peered":"2024-04-25T13:42:17.383666+0000","last_clean":"2024-04-25T13:42:17.383666+0000","last_became_active":"2024-04-25T13:42:16.426004+0000","last_became_peered":"2024-04-25T13:42:16.426004+0000","last_unstale":"2024-04-25T13:42:17.383666+0000","last_undegraded":"2024-04-25T13:42:17.383666+0000","last_fullsized":"2024-04-25T13:42:17.383666+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:30.863151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431448+0000","last_change":"2024-04-25T13:42:15.395662+0000","last_active":"2024-04-25T13:42:17.431448+0000","last_peered":"2024-04-25T13:42:17.431448+0000","last_clean":"2024-04-25T13:42:17.431448+0000","last_became_active":"2024-04-25T13:42:15.394697+0000","last_became_peered":"2024-04-25T13:42:15.394697+0000","last_unstale":"2024-04-25T13:42:17.431448+0000","last_undegraded":"2024-04-25T13:42:17.431448+0000","last_fullsized":"2024-04-25T13:42:17.431448+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:07.448537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459650+0000","last_change":"2024-04-25T13:42:16.388333+0000","last_active":"2024-04-25T13:42:17.459650+0000","last_peered":"2024-04-25T13:42:17.459650+0000","last_clean":"2024-04-25T13:42:17.459650+0000","last_became_active":"2024-04-25T13:42:16.387684+0000","last_became_peered":"2024-04-25T13:42:16.387684+0000","last_unstale":"2024-04-25T13:42:17.459650+0000","last_undegraded":"2024-04-25T13:42:17.459650+0000","last_fullsized":"2024-04-25T13:42:17.459650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:01.926554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459283+0000","last_change":"2024-04-25T13:42:15.411890+0000","last_active":"2024-04-25T13:42:17.459283+0000","last_peered":"2024-04-25T13:42:17.459283+0000","last_clean":"2024-04-25T13:42:17.459283+0000","last_became_active":"2024-04-25T13:42:15.411682+0000","last_became_peered":"2024-04-25T13:42:15.411682+0000","last_unstale":"2024-04-25T13:42:17.459283+0000","last_undegraded":"2024-04-25T13:42:17.459283+0000","last_fullsized":"2024-04-25T13:42:17.459283+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:21.495394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431251+0000","last_change":"2024-04-25T13:42:16.409658+0000","last_active":"2024-04-25T13:42:17.431251+0000","last_peered":"2024-04-25T13:42:17.431251+0000","last_clean":"2024-04-25T13:42:17.431251+0000","last_became_active":"2024-04-25T13:42:16.408520+0000","last_became_peered":"2024-04-25T13:42:16.408520+0000","last_unstale":"2024-04-25T13:42:17.431251+0000","last_undegraded":"2024-04-25T13:42:17.431251+0000","last_fullsized":"2024-04-25T13:42:17.431251+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:51.209293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462898+0000","last_change":"2024-04-25T13:42:15.390830+0000","last_active":"2024-04-25T13:42:17.462898+0000","last_peered":"2024-04-25T13:42:17.462898+0000","last_clean":"2024-04-25T13:42:17.462898+0000","last_became_active":"2024-04-25T13:42:15.390392+0000","last_became_peered":"2024-04-25T13:42:15.390392+0000","last_unstale":"2024-04-25T13:42:17.462898+0000","last_undegraded":"2024-04-25T13:42:17.462898+0000","last_fullsized":"2024-04-25T13:42:17.462898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.764917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461797+0000","last_change":"2024-04-25T13:42:16.412510+0000","last_active":"2024-04-25T13:42:17.461797+0000","last_peered":"2024-04-25T13:42:17.461797+0000","last_clean":"2024-04-25T13:42:17.461797+0000","last_became_active":"2024-04-25T13:42:16.411801+0000","last_became_peered":"2024-04-25T13:42:16.411801+0000","last_unstale":"2024-04-25T13:42:17.461797+0000","last_undegraded":"2024-04-25T13:42:17.461797+0000","last_fullsized":"2024-04-25T13:42:17.461797+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:04.674217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461255+0000","last_change":"2024-04-25T13:42:15.396434+0000","last_active":"2024-04-25T13:42:17.461255+0000","last_peered":"2024-04-25T13:42:17.461255+0000","last_clean":"2024-04-25T13:42:17.461255+0000","last_became_active":"2024-04-25T13:42:15.395632+0000","last_became_peered":"2024-04-25T13:42:15.395632+0000","last_unstale":"2024-04-25T13:42:17.461255+0000","last_undegraded":"2024-04-25T13:42:17.461255+0000","last_fullsized":"2024-04-25T13:42:17.461255+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:08.577527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384734+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:42:17.384734+0000","last_peered":"2024-04-25T13:42:17.384734+0000","last_clean":"2024-04-25T13:42:17.384734+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:42:17.384734+0000","last_undegraded":"2024-04-25T13:42:17.384734+0000","last_fullsized":"2024-04-25T13:42:17.384734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":798,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194514,"statfs":{"total":200000000,"available":199182761,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":222,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199772547,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":221,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199773149,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":223,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195089,"statfs":{"total":200000000,"available":199771607,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":222,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199772581,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":223,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195089,"statfs":{"total":200000000,"available":199771585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":221,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199773149,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":791,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194520,"statfs":{"total":200000000,"available":199189117,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:42:20.520 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:42:20.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.697+0000 7f4e437f5640 1 Processor -- start 2024-04-25T13:42:20.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.697+0000 7f4e437f5640 1 -- start start 2024-04-25T13:42:20.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e437f5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c161f90 0x7f4e3c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e437f5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e3c15ec40 0x7f4e3c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e437f5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4e3c0d0d70 con 0x7f4e3c0d0510 2024-04-25T13:42:20.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e437f5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4e3c0d1050 con 0x7f4e3c15ec40 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e437f5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c06dca0 con 0x7f4e3c161f90 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e40d90640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4e3c0d0510 0x7f4e3c15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:47002/0 (socket says 172.21.15.80:47002) 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e40d90640 1 -- 172.21.15.80:0/4274971225 learned_addr learned my addr 172.21.15.80:0/4274971225 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41d92640 1 --2- 172.21.15.80:0/4274971225 >> v2:172.21.15.87:3300/0 conn(0x7f4e3c15ec40 0x7f4e3c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1070788797 0 0) 0x7f4e3c0d0d70 con 0x7f4e3c0d0510 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41591640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c161f90 0x7f4e3c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4e28003530 con 0x7f4e3c0d0510 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2472662629 0 0) 0x7f4e28003530 con 0x7f4e3c0d0510 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c161f90 msgr2=0x7f4e3c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c161f90 0x7f4e3c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 >> v2:172.21.15.87:3300/0 conn(0x7f4e3c15ec40 msgr2=0x7f4e3c15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 --2- 172.21.15.80:0/4274971225 >> v2:172.21.15.87:3300/0 conn(0x7f4e3c15ec40 0x7f4e3c15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c06df80 con 0x7f4e3c0d0510 2024-04-25T13:42:20.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4e2c001680 con 0x7f4e3c0d0510 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 >> v1:172.21.15.80:6789/0 conn(0x7f4e3c0d0510 legacy=0x7f4e3c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c161f90 0x7f4e280039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e28003f40 0x7f4e28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e28005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c0d1050 con 0x7f4e28004d50 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c06dca0 con 0x7f4e28003f40 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2800bad0 con 0x7f4e3c161f90 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41d92640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e28005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41591640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c161f90 0x7f4e280039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41d92640 1 -- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c161f90 msgr2=0x7f4e280039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41d92640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c161f90 0x7f4e280039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41d92640 1 -- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e28003f40 msgr2=0x7f4e28004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:20.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41d92640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e28003f40 0x7f4e28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.701+0000 7f4e41d92640 1 -- 172.21.15.80:0/4274971225 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e28003530 con 0x7f4e28004d50 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e41d92640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e28005160 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009fc0 tx=0x7f4e3401ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc0621f8fd2c11a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e340b2070 con 0x7f4e28004d50 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e340a5cd0 con 0x7f4e28004d50 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e33fff640 1 -- 172.21.15.80:0/4274971225 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34009c10 con 0x7f4e28004d50 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e437f5640 1 -- 172.21.15.80:0/4274971225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 msgr2=0x7f4e28005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e437f5640 1 --2- 172.21.15.80:0/4274971225 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e28005160 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009fc0 tx=0x7f4e3401ccc0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e41d92640 1 -- 172.21.15.80:0/4274971225 reap_dead start 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e437f5640 1 -- 172.21.15.80:0/4274971225 shutdown_connections 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e437f5640 1 -- 172.21.15.80:0/4274971225 >> 172.21.15.80:0/4274971225 conn(0x7f4e3c06d080 msgr2=0x7f4e3c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e437f5640 1 -- 172.21.15.80:0/4274971225 shutdown_connections 2024-04-25T13:42:20.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e437f5640 1 -- 172.21.15.80:0/4274971225 wait complete. 2024-04-25T13:42:20.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e437f5640 1 Processor -- start 2024-04-25T13:42:20.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.705+0000 7f4e437f5640 1 -- start start 2024-04-25T13:42:20.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e3c0738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3c13a580 0x7f4e3c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c0afdf0 0x7f4e3c071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c162520 con 0x7f4e28004d50 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1623a0 con 0x7f4e3c13a580 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e3c144620 con 0x7f4e3c0afdf0 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e3c0738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e3c0738f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53142/0 (socket says 172.21.15.80:53142) 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 -- 172.21.15.80:0/1232236688 learned_addr learned my addr 172.21.15.80:0/1232236688 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e40d90640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3c13a580 0x7f4e3c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 -- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c0afdf0 msgr2=0x7f4e3c071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c0afdf0 0x7f4e3c071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 -- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3c13a580 msgr2=0x7f4e3c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3c13a580 0x7f4e3c0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 -- 172.21.15.80:0/1232236688 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c0b0200 con 0x7f4e28004d50 2024-04-25T13:42:20.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e41591640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e3c0738f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4e38011590 tx=0x7f4e38011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e31ffb640 1 -- 172.21.15.80:0/1232236688 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38019070 con 0x7f4e28004d50 2024-04-25T13:42:20.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e31ffb640 1 -- 172.21.15.80:0/1232236688 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e38003620 con 0x7f4e28004d50 2024-04-25T13:42:20.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c151580 con 0x7f4e28004d50 2024-04-25T13:42:20.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e31ffb640 1 -- 172.21.15.80:0/1232236688 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38003a10 con 0x7f4e28004d50 2024-04-25T13:42:20.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c151b40 con 0x7f4e28004d50 2024-04-25T13:42:20.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e3c06a3a0 con 0x7f4e28004d50 2024-04-25T13:42:20.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.709+0000 7f4e31ffb640 1 -- 172.21.15.80:0/1232236688 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4e3801c070 con 0x7f4e28004d50 2024-04-25T13:42:20.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.713+0000 7f4e31ffb640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e2006f430 0x7f4e20071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:20.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.713+0000 7f4e31ffb640 1 -- 172.21.15.80:0/1232236688 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f4e3808f9b0 con 0x7f4e28004d50 2024-04-25T13:42:20.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.713+0000 7f4e40d90640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e2006f430 0x7f4e20071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:20.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.713+0000 7f4e40d90640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e2006f430 0x7f4e20071940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c001a20 tx=0x7f4e2c00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:20.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.717+0000 7f4e31ffb640 1 -- 172.21.15.80:0/1232236688 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4e3c0b0200 con 0x7f4e28004d50 2024-04-25T13:42:20.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.869+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4e3c0b3880 con 0x7f4e2006f430 2024-04-25T13:42:20.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e31ffb640 1 -- 172.21.15.80:0/1232236688 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247478 (secure 0 0 0) 0x7f4e3c0b3880 con 0x7f4e2006f430 2024-04-25T13:42:20.891 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:20.893 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:42:20.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e2006f430 msgr2=0x7f4e20071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e2006f430 0x7f4e20071940 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c001a20 tx=0x7f4e2c00c040 comp rx=0 tx=0).stop 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 msgr2=0x7f4e3c0738f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e3c0738f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4e38011590 tx=0x7f4e38011a50 comp rx=0 tx=0).stop 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 shutdown_connections 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e2006f430 0x7f4e20071940 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e3c0afdf0 0x7f4e3c071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e3c13a580 0x7f4e3c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 --2- 172.21.15.80:0/1232236688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e28004d50 0x7f4e3c0738f0 secure :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4e38011590 tx=0x7f4e38011a50 comp rx=0 tx=0).stop 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 >> 172.21.15.80:0/1232236688 conn(0x7f4e3c06d080 msgr2=0x7f4e3c15fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 shutdown_connections 2024-04-25T13:42:20.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:20.889+0000 7f4e437f5640 1 -- 172.21.15.80:0/1232236688 wait complete. 2024-04-25T13:42:20.911 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-25T13:42:20.132929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2923,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559576,"statfs":{"total":1600000000,"available":1597006496,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.769632"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384687+0000","last_change":"2024-04-25T13:42:16.425298+0000","last_active":"2024-04-25T13:42:17.384687+0000","last_peered":"2024-04-25T13:42:17.384687+0000","last_clean":"2024-04-25T13:42:17.384687+0000","last_became_active":"2024-04-25T13:42:16.423503+0000","last_became_peered":"2024-04-25T13:42:16.423503+0000","last_unstale":"2024-04-25T13:42:17.384687+0000","last_undegraded":"2024-04-25T13:42:17.384687+0000","last_fullsized":"2024-04-25T13:42:17.384687+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:53.043550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460165+0000","last_change":"2024-04-25T13:42:16.416452+0000","last_active":"2024-04-25T13:42:17.460165+0000","last_peered":"2024-04-25T13:42:17.460165+0000","last_clean":"2024-04-25T13:42:17.460165+0000","last_became_active":"2024-04-25T13:42:16.414344+0000","last_became_peered":"2024-04-25T13:42:16.414344+0000","last_unstale":"2024-04-25T13:42:17.460165+0000","last_undegraded":"2024-04-25T13:42:17.460165+0000","last_fullsized":"2024-04-25T13:42:17.460165+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:39.032537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430936+0000","last_change":"2024-04-25T13:42:16.416442+0000","last_active":"2024-04-25T13:42:17.430936+0000","last_peered":"2024-04-25T13:42:17.430936+0000","last_clean":"2024-04-25T13:42:17.430936+0000","last_became_active":"2024-04-25T13:42:16.411947+0000","last_became_peered":"2024-04-25T13:42:16.411947+0000","last_unstale":"2024-04-25T13:42:17.430936+0000","last_undegraded":"2024-04-25T13:42:17.430936+0000","last_fullsized":"2024-04-25T13:42:17.430936+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:18.168600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461770+0000","last_change":"2024-04-25T13:42:16.423547+0000","last_active":"2024-04-25T13:42:17.461770+0000","last_peered":"2024-04-25T13:42:17.461770+0000","last_clean":"2024-04-25T13:42:17.461770+0000","last_became_active":"2024-04-25T13:42:16.412632+0000","last_became_peered":"2024-04-25T13:42:16.412632+0000","last_unstale":"2024-04-25T13:42:17.461770+0000","last_undegraded":"2024-04-25T13:42:17.461770+0000","last_fullsized":"2024-04-25T13:42:17.461770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:58.188328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461294+0000","last_change":"2024-04-25T13:42:16.414903+0000","last_active":"2024-04-25T13:42:17.461294+0000","last_peered":"2024-04-25T13:42:17.461294+0000","last_clean":"2024-04-25T13:42:17.461294+0000","last_became_active":"2024-04-25T13:42:16.414161+0000","last_became_peered":"2024-04-25T13:42:16.414161+0000","last_unstale":"2024-04-25T13:42:17.461294+0000","last_undegraded":"2024-04-25T13:42:17.461294+0000","last_fullsized":"2024-04-25T13:42:17.461294+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:53.892421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383102+0000","last_change":"2024-04-25T13:42:16.429599+0000","last_active":"2024-04-25T13:42:17.383102+0000","last_peered":"2024-04-25T13:42:17.383102+0000","last_clean":"2024-04-25T13:42:17.383102+0000","last_became_active":"2024-04-25T13:42:16.428798+0000","last_became_peered":"2024-04-25T13:42:16.428798+0000","last_unstale":"2024-04-25T13:42:17.383102+0000","last_undegraded":"2024-04-25T13:42:17.383102+0000","last_fullsized":"2024-04-25T13:42:17.383102+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:51.933268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384030+0000","last_change":"2024-04-25T13:42:16.408282+0000","last_active":"2024-04-25T13:42:17.384030+0000","last_peered":"2024-04-25T13:42:17.384030+0000","last_clean":"2024-04-25T13:42:17.384030+0000","last_became_active":"2024-04-25T13:42:16.407524+0000","last_became_peered":"2024-04-25T13:42:16.407524+0000","last_unstale":"2024-04-25T13:42:17.384030+0000","last_undegraded":"2024-04-25T13:42:17.384030+0000","last_fullsized":"2024-04-25T13:42:17.384030+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:10:57.446397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461188+0000","last_change":"2024-04-25T13:42:16.426549+0000","last_active":"2024-04-25T13:42:17.461188+0000","last_peered":"2024-04-25T13:42:17.461188+0000","last_clean":"2024-04-25T13:42:17.461188+0000","last_became_active":"2024-04-25T13:42:16.426269+0000","last_became_peered":"2024-04-25T13:42:16.426269+0000","last_unstale":"2024-04-25T13:42:17.461188+0000","last_undegraded":"2024-04-25T13:42:17.461188+0000","last_fullsized":"2024-04-25T13:42:17.461188+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:49.864859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462328+0000","last_change":"2024-04-25T13:42:16.434114+0000","last_active":"2024-04-25T13:42:17.462328+0000","last_peered":"2024-04-25T13:42:17.462328+0000","last_clean":"2024-04-25T13:42:17.462328+0000","last_became_active":"2024-04-25T13:42:16.427148+0000","last_became_peered":"2024-04-25T13:42:16.427148+0000","last_unstale":"2024-04-25T13:42:17.462328+0000","last_undegraded":"2024-04-25T13:42:17.462328+0000","last_fullsized":"2024-04-25T13:42:17.462328+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:46.743085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460184+0000","last_change":"2024-04-25T13:42:16.430434+0000","last_active":"2024-04-25T13:42:17.460184+0000","last_peered":"2024-04-25T13:42:17.460184+0000","last_clean":"2024-04-25T13:42:17.460184+0000","last_became_active":"2024-04-25T13:42:16.420975+0000","last_became_peered":"2024-04-25T13:42:16.420975+0000","last_unstale":"2024-04-25T13:42:17.460184+0000","last_undegraded":"2024-04-25T13:42:17.460184+0000","last_fullsized":"2024-04-25T13:42:17.460184+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:58.365482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459964+0000","last_change":"2024-04-25T13:42:16.419331+0000","last_active":"2024-04-25T13:42:17.459964+0000","last_peered":"2024-04-25T13:42:17.459964+0000","last_clean":"2024-04-25T13:42:17.459964+0000","last_became_active":"2024-04-25T13:42:16.417868+0000","last_became_peered":"2024-04-25T13:42:16.417868+0000","last_unstale":"2024-04-25T13:42:17.459964+0000","last_undegraded":"2024-04-25T13:42:17.459964+0000","last_fullsized":"2024-04-25T13:42:17.459964+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:58.820317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460683+0000","last_change":"2024-04-25T13:42:16.415786+0000","last_active":"2024-04-25T13:42:17.460683+0000","last_peered":"2024-04-25T13:42:17.460683+0000","last_clean":"2024-04-25T13:42:17.460683+0000","last_became_active":"2024-04-25T13:42:16.413178+0000","last_became_peered":"2024-04-25T13:42:16.413178+0000","last_unstale":"2024-04-25T13:42:17.460683+0000","last_undegraded":"2024-04-25T13:42:17.460683+0000","last_fullsized":"2024-04-25T13:42:17.460683+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:51.094841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461000+0000","last_change":"2024-04-25T13:42:16.408151+0000","last_active":"2024-04-25T13:42:17.461000+0000","last_peered":"2024-04-25T13:42:17.461000+0000","last_clean":"2024-04-25T13:42:17.461000+0000","last_became_active":"2024-04-25T13:42:16.407434+0000","last_became_peered":"2024-04-25T13:42:16.407434+0000","last_unstale":"2024-04-25T13:42:17.461000+0000","last_undegraded":"2024-04-25T13:42:17.461000+0000","last_fullsized":"2024-04-25T13:42:17.461000+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:39.327118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.463266+0000","last_change":"2024-04-25T13:42:16.436951+0000","last_active":"2024-04-25T13:42:17.463266+0000","last_peered":"2024-04-25T13:42:17.463266+0000","last_clean":"2024-04-25T13:42:17.463266+0000","last_became_active":"2024-04-25T13:42:16.433595+0000","last_became_peered":"2024-04-25T13:42:16.433595+0000","last_unstale":"2024-04-25T13:42:17.463266+0000","last_undegraded":"2024-04-25T13:42:17.463266+0000","last_fullsized":"2024-04-25T13:42:17.463266+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:37.573930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383243+0000","last_change":"2024-04-25T13:42:16.400608+0000","last_active":"2024-04-25T13:42:17.383243+0000","last_peered":"2024-04-25T13:42:17.383243+0000","last_clean":"2024-04-25T13:42:17.383243+0000","last_became_active":"2024-04-25T13:42:16.399982+0000","last_became_peered":"2024-04-25T13:42:16.399982+0000","last_unstale":"2024-04-25T13:42:17.383243+0000","last_undegraded":"2024-04-25T13:42:17.383243+0000","last_fullsized":"2024-04-25T13:42:17.383243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:44:28.443058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461652+0000","last_change":"2024-04-25T13:42:16.419438+0000","last_active":"2024-04-25T13:42:17.461652+0000","last_peered":"2024-04-25T13:42:17.461652+0000","last_clean":"2024-04-25T13:42:17.461652+0000","last_became_active":"2024-04-25T13:42:16.417710+0000","last_became_peered":"2024-04-25T13:42:16.417710+0000","last_unstale":"2024-04-25T13:42:17.461652+0000","last_undegraded":"2024-04-25T13:42:17.461652+0000","last_fullsized":"2024-04-25T13:42:17.461652+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:16.068579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430638+0000","last_change":"2024-04-25T13:42:16.404837+0000","last_active":"2024-04-25T13:42:17.430638+0000","last_peered":"2024-04-25T13:42:17.430638+0000","last_clean":"2024-04-25T13:42:17.430638+0000","last_became_active":"2024-04-25T13:42:16.403007+0000","last_became_peered":"2024-04-25T13:42:16.403007+0000","last_unstale":"2024-04-25T13:42:17.430638+0000","last_undegraded":"2024-04-25T13:42:17.430638+0000","last_fullsized":"2024-04-25T13:42:17.430638+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:32.234466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431664+0000","last_change":"2024-04-25T13:42:16.417270+0000","last_active":"2024-04-25T13:42:17.431664+0000","last_peered":"2024-04-25T13:42:17.431664+0000","last_clean":"2024-04-25T13:42:17.431664+0000","last_became_active":"2024-04-25T13:42:16.415514+0000","last_became_peered":"2024-04-25T13:42:16.415514+0000","last_unstale":"2024-04-25T13:42:17.431664+0000","last_undegraded":"2024-04-25T13:42:17.431664+0000","last_fullsized":"2024-04-25T13:42:17.431664+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:14.076352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462082+0000","last_change":"2024-04-25T13:42:16.419608+0000","last_active":"2024-04-25T13:42:17.462082+0000","last_peered":"2024-04-25T13:42:17.462082+0000","last_clean":"2024-04-25T13:42:17.462082+0000","last_became_active":"2024-04-25T13:42:16.418110+0000","last_became_peered":"2024-04-25T13:42:16.418110+0000","last_unstale":"2024-04-25T13:42:17.462082+0000","last_undegraded":"2024-04-25T13:42:17.462082+0000","last_fullsized":"2024-04-25T13:42:17.462082+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:02.829267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460218+0000","last_change":"2024-04-25T13:42:16.418587+0000","last_active":"2024-04-25T13:42:17.460218+0000","last_peered":"2024-04-25T13:42:17.460218+0000","last_clean":"2024-04-25T13:42:17.460218+0000","last_became_active":"2024-04-25T13:42:16.417861+0000","last_became_peered":"2024-04-25T13:42:16.417861+0000","last_unstale":"2024-04-25T13:42:17.460218+0000","last_undegraded":"2024-04-25T13:42:17.460218+0000","last_fullsized":"2024-04-25T13:42:17.460218+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:20.548128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383306+0000","last_change":"2024-04-25T13:42:16.427105+0000","last_active":"2024-04-25T13:42:17.383306+0000","last_peered":"2024-04-25T13:42:17.383306+0000","last_clean":"2024-04-25T13:42:17.383306+0000","last_became_active":"2024-04-25T13:42:16.424151+0000","last_became_peered":"2024-04-25T13:42:16.424151+0000","last_unstale":"2024-04-25T13:42:17.383306+0000","last_undegraded":"2024-04-25T13:42:17.383306+0000","last_fullsized":"2024-04-25T13:42:17.383306+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:05.099427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459889+0000","last_change":"2024-04-25T13:42:16.432416+0000","last_active":"2024-04-25T13:42:17.459889+0000","last_peered":"2024-04-25T13:42:17.459889+0000","last_clean":"2024-04-25T13:42:17.459889+0000","last_became_active":"2024-04-25T13:42:16.421186+0000","last_became_peered":"2024-04-25T13:42:16.421186+0000","last_unstale":"2024-04-25T13:42:17.459889+0000","last_undegraded":"2024-04-25T13:42:17.459889+0000","last_fullsized":"2024-04-25T13:42:17.459889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:25.545826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462875+0000","last_change":"2024-04-25T13:42:16.416439+0000","last_active":"2024-04-25T13:42:17.462875+0000","last_peered":"2024-04-25T13:42:17.462875+0000","last_clean":"2024-04-25T13:42:17.462875+0000","last_became_active":"2024-04-25T13:42:16.408062+0000","last_became_peered":"2024-04-25T13:42:16.408062+0000","last_unstale":"2024-04-25T13:42:17.462875+0000","last_undegraded":"2024-04-25T13:42:17.462875+0000","last_fullsized":"2024-04-25T13:42:17.462875+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:37.454315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460919+0000","last_change":"2024-04-25T13:42:16.414813+0000","last_active":"2024-04-25T13:42:17.460919+0000","last_peered":"2024-04-25T13:42:17.460919+0000","last_clean":"2024-04-25T13:42:17.460919+0000","last_became_active":"2024-04-25T13:42:16.410809+0000","last_became_peered":"2024-04-25T13:42:16.410809+0000","last_unstale":"2024-04-25T13:42:17.460919+0000","last_undegraded":"2024-04-25T13:42:17.460919+0000","last_fullsized":"2024-04-25T13:42:17.460919+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:19.955127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461868+0000","last_change":"2024-04-25T13:42:16.417740+0000","last_active":"2024-04-25T13:42:17.461868+0000","last_peered":"2024-04-25T13:42:17.461868+0000","last_clean":"2024-04-25T13:42:17.461868+0000","last_became_active":"2024-04-25T13:42:16.416888+0000","last_became_peered":"2024-04-25T13:42:16.416888+0000","last_unstale":"2024-04-25T13:42:17.461868+0000","last_undegraded":"2024-04-25T13:42:17.461868+0000","last_fullsized":"2024-04-25T13:42:17.461868+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:16.250839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460814+0000","last_change":"2024-04-25T13:42:16.428901+0000","last_active":"2024-04-25T13:42:17.460814+0000","last_peered":"2024-04-25T13:42:17.460814+0000","last_clean":"2024-04-25T13:42:17.460814+0000","last_became_active":"2024-04-25T13:42:16.428423+0000","last_became_peered":"2024-04-25T13:42:16.428423+0000","last_unstale":"2024-04-25T13:42:17.460814+0000","last_undegraded":"2024-04-25T13:42:17.460814+0000","last_fullsized":"2024-04-25T13:42:17.460814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:00:44.403509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430958+0000","last_change":"2024-04-25T13:42:16.403733+0000","last_active":"2024-04-25T13:42:17.430958+0000","last_peered":"2024-04-25T13:42:17.430958+0000","last_clean":"2024-04-25T13:42:17.430958+0000","last_became_active":"2024-04-25T13:42:16.401364+0000","last_became_peered":"2024-04-25T13:42:16.401364+0000","last_unstale":"2024-04-25T13:42:17.430958+0000","last_undegraded":"2024-04-25T13:42:17.430958+0000","last_fullsized":"2024-04-25T13:42:17.430958+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.181499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460534+0000","last_change":"2024-04-25T13:42:16.408097+0000","last_active":"2024-04-25T13:42:17.460534+0000","last_peered":"2024-04-25T13:42:17.460534+0000","last_clean":"2024-04-25T13:42:17.460534+0000","last_became_active":"2024-04-25T13:42:16.407350+0000","last_became_peered":"2024-04-25T13:42:16.407350+0000","last_unstale":"2024-04-25T13:42:17.460534+0000","last_undegraded":"2024-04-25T13:42:17.460534+0000","last_fullsized":"2024-04-25T13:42:17.460534+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:42.539110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461399+0000","last_change":"2024-04-25T13:42:16.419040+0000","last_active":"2024-04-25T13:42:17.461399+0000","last_peered":"2024-04-25T13:42:17.461399+0000","last_clean":"2024-04-25T13:42:17.461399+0000","last_became_active":"2024-04-25T13:42:16.416376+0000","last_became_peered":"2024-04-25T13:42:16.416376+0000","last_unstale":"2024-04-25T13:42:17.461399+0000","last_undegraded":"2024-04-25T13:42:17.461399+0000","last_fullsized":"2024-04-25T13:42:17.461399+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:34.729285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431159+0000","last_change":"2024-04-25T13:42:16.403446+0000","last_active":"2024-04-25T13:42:17.431159+0000","last_peered":"2024-04-25T13:42:17.431159+0000","last_clean":"2024-04-25T13:42:17.431159+0000","last_became_active":"2024-04-25T13:42:16.400150+0000","last_became_peered":"2024-04-25T13:42:16.400150+0000","last_unstale":"2024-04-25T13:42:17.431159+0000","last_undegraded":"2024-04-25T13:42:17.431159+0000","last_fullsized":"2024-04-25T13:42:17.431159+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:33.293784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383482+0000","last_change":"2024-04-25T13:42:16.425221+0000","last_active":"2024-04-25T13:42:17.383482+0000","last_peered":"2024-04-25T13:42:17.383482+0000","last_clean":"2024-04-25T13:42:17.383482+0000","last_became_active":"2024-04-25T13:42:16.419396+0000","last_became_peered":"2024-04-25T13:42:16.419396+0000","last_unstale":"2024-04-25T13:42:17.383482+0000","last_undegraded":"2024-04-25T13:42:17.383482+0000","last_fullsized":"2024-04-25T13:42:17.383482+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:48.202636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460376+0000","last_change":"2024-04-25T13:42:16.410076+0000","last_active":"2024-04-25T13:42:17.460376+0000","last_peered":"2024-04-25T13:42:17.460376+0000","last_clean":"2024-04-25T13:42:17.460376+0000","last_became_active":"2024-04-25T13:42:16.409390+0000","last_became_peered":"2024-04-25T13:42:16.409390+0000","last_unstale":"2024-04-25T13:42:17.460376+0000","last_undegraded":"2024-04-25T13:42:17.460376+0000","last_fullsized":"2024-04-25T13:42:17.460376+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:03.966036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.458979+0000","last_change":"2024-04-25T13:42:16.421671+0000","last_active":"2024-04-25T13:42:17.458979+0000","last_peered":"2024-04-25T13:42:17.458979+0000","last_clean":"2024-04-25T13:42:17.458979+0000","last_became_active":"2024-04-25T13:42:16.420405+0000","last_became_peered":"2024-04-25T13:42:16.420405+0000","last_unstale":"2024-04-25T13:42:17.458979+0000","last_undegraded":"2024-04-25T13:42:17.458979+0000","last_fullsized":"2024-04-25T13:42:17.458979+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:10.569450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383430+0000","last_change":"2024-04-25T13:42:15.411341+0000","last_active":"2024-04-25T13:42:17.383430+0000","last_peered":"2024-04-25T13:42:17.383430+0000","last_clean":"2024-04-25T13:42:17.383430+0000","last_became_active":"2024-04-25T13:42:15.411109+0000","last_became_peered":"2024-04-25T13:42:15.411109+0000","last_unstale":"2024-04-25T13:42:17.383430+0000","last_undegraded":"2024-04-25T13:42:17.383430+0000","last_fullsized":"2024-04-25T13:42:17.383430+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:17.066430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462483+0000","last_change":"2024-04-25T13:42:16.424669+0000","last_active":"2024-04-25T13:42:17.462483+0000","last_peered":"2024-04-25T13:42:17.462483+0000","last_clean":"2024-04-25T13:42:17.462483+0000","last_became_active":"2024-04-25T13:42:16.423653+0000","last_became_peered":"2024-04-25T13:42:16.423653+0000","last_unstale":"2024-04-25T13:42:17.462483+0000","last_undegraded":"2024-04-25T13:42:17.462483+0000","last_fullsized":"2024-04-25T13:42:17.462483+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:28.054773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430521+0000","last_change":"2024-04-25T13:42:15.413835+0000","last_active":"2024-04-25T13:42:17.430521+0000","last_peered":"2024-04-25T13:42:17.430521+0000","last_clean":"2024-04-25T13:42:17.430521+0000","last_became_active":"2024-04-25T13:42:15.412893+0000","last_became_peered":"2024-04-25T13:42:15.412893+0000","last_unstale":"2024-04-25T13:42:17.430521+0000","last_undegraded":"2024-04-25T13:42:17.430521+0000","last_fullsized":"2024-04-25T13:42:17.430521+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:35.794581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460733+0000","last_change":"2024-04-25T13:42:16.421262+0000","last_active":"2024-04-25T13:42:17.460733+0000","last_peered":"2024-04-25T13:42:17.460733+0000","last_clean":"2024-04-25T13:42:17.460733+0000","last_became_active":"2024-04-25T13:42:16.420320+0000","last_became_peered":"2024-04-25T13:42:16.420320+0000","last_unstale":"2024-04-25T13:42:17.460733+0000","last_undegraded":"2024-04-25T13:42:17.460733+0000","last_fullsized":"2024-04-25T13:42:17.460733+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.872433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460618+0000","last_change":"2024-04-25T13:42:15.399964+0000","last_active":"2024-04-25T13:42:17.460618+0000","last_peered":"2024-04-25T13:42:17.460618+0000","last_clean":"2024-04-25T13:42:17.460618+0000","last_became_active":"2024-04-25T13:42:15.399066+0000","last_became_peered":"2024-04-25T13:42:15.399066+0000","last_unstale":"2024-04-25T13:42:17.460618+0000","last_undegraded":"2024-04-25T13:42:17.460618+0000","last_fullsized":"2024-04-25T13:42:17.460618+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:08.429549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460106+0000","last_change":"2024-04-25T13:42:16.416411+0000","last_active":"2024-04-25T13:42:17.460106+0000","last_peered":"2024-04-25T13:42:17.460106+0000","last_clean":"2024-04-25T13:42:17.460106+0000","last_became_active":"2024-04-25T13:42:16.407485+0000","last_became_peered":"2024-04-25T13:42:16.407485+0000","last_unstale":"2024-04-25T13:42:17.460106+0000","last_undegraded":"2024-04-25T13:42:17.460106+0000","last_fullsized":"2024-04-25T13:42:17.460106+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:19.364772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.385122+0000","last_change":"2024-04-25T13:42:15.411790+0000","last_active":"2024-04-25T13:42:17.385122+0000","last_peered":"2024-04-25T13:42:17.385122+0000","last_clean":"2024-04-25T13:42:17.385122+0000","last_became_active":"2024-04-25T13:42:15.408452+0000","last_became_peered":"2024-04-25T13:42:15.408452+0000","last_unstale":"2024-04-25T13:42:17.385122+0000","last_undegraded":"2024-04-25T13:42:17.385122+0000","last_fullsized":"2024-04-25T13:42:17.385122+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:30.813268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430612+0000","last_change":"2024-04-25T13:42:16.425462+0000","last_active":"2024-04-25T13:42:17.430612+0000","last_peered":"2024-04-25T13:42:17.430612+0000","last_clean":"2024-04-25T13:42:17.430612+0000","last_became_active":"2024-04-25T13:42:16.424644+0000","last_became_peered":"2024-04-25T13:42:16.424644+0000","last_unstale":"2024-04-25T13:42:17.430612+0000","last_undegraded":"2024-04-25T13:42:17.430612+0000","last_fullsized":"2024-04-25T13:42:17.430612+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:35.478074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383650+0000","last_change":"2024-04-25T13:42:15.407795+0000","last_active":"2024-04-25T13:42:17.383650+0000","last_peered":"2024-04-25T13:42:17.383650+0000","last_clean":"2024-04-25T13:42:17.383650+0000","last_became_active":"2024-04-25T13:42:15.407081+0000","last_became_peered":"2024-04-25T13:42:15.407081+0000","last_unstale":"2024-04-25T13:42:17.383650+0000","last_undegraded":"2024-04-25T13:42:17.383650+0000","last_fullsized":"2024-04-25T13:42:17.383650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:14.043951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461081+0000","last_change":"2024-04-25T13:42:16.429073+0000","last_active":"2024-04-25T13:42:17.461081+0000","last_peered":"2024-04-25T13:42:17.461081+0000","last_clean":"2024-04-25T13:42:17.461081+0000","last_became_active":"2024-04-25T13:42:16.418988+0000","last_became_peered":"2024-04-25T13:42:16.418988+0000","last_unstale":"2024-04-25T13:42:17.461081+0000","last_undegraded":"2024-04-25T13:42:17.461081+0000","last_fullsized":"2024-04-25T13:42:17.461081+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:10.874165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431881+0000","last_change":"2024-04-25T13:42:15.417766+0000","last_active":"2024-04-25T13:42:17.431881+0000","last_peered":"2024-04-25T13:42:17.431881+0000","last_clean":"2024-04-25T13:42:17.431881+0000","last_became_active":"2024-04-25T13:42:15.414864+0000","last_became_peered":"2024-04-25T13:42:15.414864+0000","last_unstale":"2024-04-25T13:42:17.431881+0000","last_undegraded":"2024-04-25T13:42:17.431881+0000","last_fullsized":"2024-04-25T13:42:17.431881+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:26.104533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460669+0000","last_change":"2024-04-25T13:42:16.430904+0000","last_active":"2024-04-25T13:42:17.460669+0000","last_peered":"2024-04-25T13:42:17.460669+0000","last_clean":"2024-04-25T13:42:17.460669+0000","last_became_active":"2024-04-25T13:42:16.423825+0000","last_became_peered":"2024-04-25T13:42:16.423825+0000","last_unstale":"2024-04-25T13:42:17.460669+0000","last_undegraded":"2024-04-25T13:42:17.460669+0000","last_fullsized":"2024-04-25T13:42:17.460669+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:59.901944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459943+0000","last_change":"2024-04-25T13:42:15.404953+0000","last_active":"2024-04-25T13:42:17.459943+0000","last_peered":"2024-04-25T13:42:17.459943+0000","last_clean":"2024-04-25T13:42:17.459943+0000","last_became_active":"2024-04-25T13:42:15.404172+0000","last_became_peered":"2024-04-25T13:42:15.404172+0000","last_unstale":"2024-04-25T13:42:17.459943+0000","last_undegraded":"2024-04-25T13:42:17.459943+0000","last_fullsized":"2024-04-25T13:42:17.459943+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:20.602777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461714+0000","last_change":"2024-04-25T13:42:16.408091+0000","last_active":"2024-04-25T13:42:17.461714+0000","last_peered":"2024-04-25T13:42:17.461714+0000","last_clean":"2024-04-25T13:42:17.461714+0000","last_became_active":"2024-04-25T13:42:16.401009+0000","last_became_peered":"2024-04-25T13:42:16.401009+0000","last_unstale":"2024-04-25T13:42:17.461714+0000","last_undegraded":"2024-04-25T13:42:17.461714+0000","last_fullsized":"2024-04-25T13:42:17.461714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:35.388008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.463093+0000","last_change":"2024-04-25T13:42:15.392736+0000","last_active":"2024-04-25T13:42:17.463093+0000","last_peered":"2024-04-25T13:42:17.463093+0000","last_clean":"2024-04-25T13:42:17.463093+0000","last_became_active":"2024-04-25T13:42:15.392434+0000","last_became_peered":"2024-04-25T13:42:15.392434+0000","last_unstale":"2024-04-25T13:42:17.463093+0000","last_undegraded":"2024-04-25T13:42:17.463093+0000","last_fullsized":"2024-04-25T13:42:17.463093+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:57.244911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431256+0000","last_change":"2024-04-25T13:42:16.413741+0000","last_active":"2024-04-25T13:42:17.431256+0000","last_peered":"2024-04-25T13:42:17.431256+0000","last_clean":"2024-04-25T13:42:17.431256+0000","last_became_active":"2024-04-25T13:42:16.412905+0000","last_became_peered":"2024-04-25T13:42:16.412905+0000","last_unstale":"2024-04-25T13:42:17.431256+0000","last_undegraded":"2024-04-25T13:42:17.431256+0000","last_fullsized":"2024-04-25T13:42:17.431256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:26.341048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.430838+0000","last_change":"2024-04-25T13:42:15.413276+0000","last_active":"2024-04-25T13:42:17.430838+0000","last_peered":"2024-04-25T13:42:17.430838+0000","last_clean":"2024-04-25T13:42:17.430838+0000","last_became_active":"2024-04-25T13:42:15.412451+0000","last_became_peered":"2024-04-25T13:42:15.412451+0000","last_unstale":"2024-04-25T13:42:17.430838+0000","last_undegraded":"2024-04-25T13:42:17.430838+0000","last_fullsized":"2024-04-25T13:42:17.430838+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:32.415100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384953+0000","last_change":"2024-04-25T13:42:16.424634+0000","last_active":"2024-04-25T13:42:17.384953+0000","last_peered":"2024-04-25T13:42:17.384953+0000","last_clean":"2024-04-25T13:42:17.384953+0000","last_became_active":"2024-04-25T13:42:16.422596+0000","last_became_peered":"2024-04-25T13:42:16.422596+0000","last_unstale":"2024-04-25T13:42:17.384953+0000","last_undegraded":"2024-04-25T13:42:17.384953+0000","last_fullsized":"2024-04-25T13:42:17.384953+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:32.640227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461292+0000","last_change":"2024-04-25T13:42:15.407541+0000","last_active":"2024-04-25T13:42:17.461292+0000","last_peered":"2024-04-25T13:42:17.461292+0000","last_clean":"2024-04-25T13:42:17.461292+0000","last_became_active":"2024-04-25T13:42:15.407323+0000","last_became_peered":"2024-04-25T13:42:15.407323+0000","last_unstale":"2024-04-25T13:42:17.461292+0000","last_undegraded":"2024-04-25T13:42:17.461292+0000","last_fullsized":"2024-04-25T13:42:17.461292+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:26.948924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383655+0000","last_change":"2024-04-25T13:42:16.430684+0000","last_active":"2024-04-25T13:42:17.383655+0000","last_peered":"2024-04-25T13:42:17.383655+0000","last_clean":"2024-04-25T13:42:17.383655+0000","last_became_active":"2024-04-25T13:42:16.429463+0000","last_became_peered":"2024-04-25T13:42:16.429463+0000","last_unstale":"2024-04-25T13:42:17.383655+0000","last_undegraded":"2024-04-25T13:42:17.383655+0000","last_fullsized":"2024-04-25T13:42:17.383655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:25.261455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384852+0000","last_change":"2024-04-25T13:42:15.413865+0000","last_active":"2024-04-25T13:42:17.384852+0000","last_peered":"2024-04-25T13:42:17.384852+0000","last_clean":"2024-04-25T13:42:17.384852+0000","last_became_active":"2024-04-25T13:42:15.413149+0000","last_became_peered":"2024-04-25T13:42:15.413149+0000","last_unstale":"2024-04-25T13:42:17.384852+0000","last_undegraded":"2024-04-25T13:42:17.384852+0000","last_fullsized":"2024-04-25T13:42:17.384852+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:48.061192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384332+0000","last_change":"2024-04-25T13:42:16.411824+0000","last_active":"2024-04-25T13:42:17.384332+0000","last_peered":"2024-04-25T13:42:17.384332+0000","last_clean":"2024-04-25T13:42:17.384332+0000","last_became_active":"2024-04-25T13:42:16.406980+0000","last_became_peered":"2024-04-25T13:42:16.406980+0000","last_unstale":"2024-04-25T13:42:17.384332+0000","last_undegraded":"2024-04-25T13:42:17.384332+0000","last_fullsized":"2024-04-25T13:42:17.384332+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:08.936662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460256+0000","last_change":"2024-04-25T13:42:15.414113+0000","last_active":"2024-04-25T13:42:17.460256+0000","last_peered":"2024-04-25T13:42:17.460256+0000","last_clean":"2024-04-25T13:42:17.460256+0000","last_became_active":"2024-04-25T13:42:15.413880+0000","last_became_peered":"2024-04-25T13:42:15.413880+0000","last_unstale":"2024-04-25T13:42:17.460256+0000","last_undegraded":"2024-04-25T13:42:17.460256+0000","last_fullsized":"2024-04-25T13:42:17.460256+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:47.734749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462113+0000","last_change":"2024-04-25T13:42:16.418851+0000","last_active":"2024-04-25T13:42:17.462113+0000","last_peered":"2024-04-25T13:42:17.462113+0000","last_clean":"2024-04-25T13:42:17.462113+0000","last_became_active":"2024-04-25T13:42:16.416677+0000","last_became_peered":"2024-04-25T13:42:16.416677+0000","last_unstale":"2024-04-25T13:42:17.462113+0000","last_undegraded":"2024-04-25T13:42:17.462113+0000","last_fullsized":"2024-04-25T13:42:17.462113+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:56.135744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383597+0000","last_change":"2024-04-25T13:42:15.389941+0000","last_active":"2024-04-25T13:42:17.383597+0000","last_peered":"2024-04-25T13:42:17.383597+0000","last_clean":"2024-04-25T13:42:17.383597+0000","last_became_active":"2024-04-25T13:42:15.389636+0000","last_became_peered":"2024-04-25T13:42:15.389636+0000","last_unstale":"2024-04-25T13:42:17.383597+0000","last_undegraded":"2024-04-25T13:42:17.383597+0000","last_fullsized":"2024-04-25T13:42:17.383597+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:12.021337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461108+0000","last_change":"2024-04-25T13:42:16.429523+0000","last_active":"2024-04-25T13:42:17.461108+0000","last_peered":"2024-04-25T13:42:17.461108+0000","last_clean":"2024-04-25T13:42:17.461108+0000","last_became_active":"2024-04-25T13:42:16.419757+0000","last_became_peered":"2024-04-25T13:42:16.419757+0000","last_unstale":"2024-04-25T13:42:17.461108+0000","last_undegraded":"2024-04-25T13:42:17.461108+0000","last_fullsized":"2024-04-25T13:42:17.461108+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:35.924977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.385263+0000","last_change":"2024-04-25T13:42:15.414744+0000","last_active":"2024-04-25T13:42:17.385263+0000","last_peered":"2024-04-25T13:42:17.385263+0000","last_clean":"2024-04-25T13:42:17.385263+0000","last_became_active":"2024-04-25T13:42:15.409519+0000","last_became_peered":"2024-04-25T13:42:15.409519+0000","last_unstale":"2024-04-25T13:42:17.385263+0000","last_undegraded":"2024-04-25T13:42:17.385263+0000","last_fullsized":"2024-04-25T13:42:17.385263+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:13.605108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383067+0000","last_change":"2024-04-25T13:42:16.429336+0000","last_active":"2024-04-25T13:42:17.383067+0000","last_peered":"2024-04-25T13:42:17.383067+0000","last_clean":"2024-04-25T13:42:17.383067+0000","last_became_active":"2024-04-25T13:42:16.428703+0000","last_became_peered":"2024-04-25T13:42:16.428703+0000","last_unstale":"2024-04-25T13:42:17.383067+0000","last_undegraded":"2024-04-25T13:42:17.383067+0000","last_fullsized":"2024-04-25T13:42:17.383067+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:39.066081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383932+0000","last_change":"2024-04-25T13:42:15.385603+0000","last_active":"2024-04-25T13:42:17.383932+0000","last_peered":"2024-04-25T13:42:17.383932+0000","last_clean":"2024-04-25T13:42:17.383932+0000","last_became_active":"2024-04-25T13:42:15.384733+0000","last_became_peered":"2024-04-25T13:42:15.384733+0000","last_unstale":"2024-04-25T13:42:17.383932+0000","last_undegraded":"2024-04-25T13:42:17.383932+0000","last_fullsized":"2024-04-25T13:42:17.383932+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:16.197245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384521+0000","last_change":"2024-04-25T13:42:16.401537+0000","last_active":"2024-04-25T13:42:17.384521+0000","last_peered":"2024-04-25T13:42:17.384521+0000","last_clean":"2024-04-25T13:42:17.384521+0000","last_became_active":"2024-04-25T13:42:16.400818+0000","last_became_peered":"2024-04-25T13:42:16.400818+0000","last_unstale":"2024-04-25T13:42:17.384521+0000","last_undegraded":"2024-04-25T13:42:17.384521+0000","last_fullsized":"2024-04-25T13:42:17.384521+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:04.157809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":22,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462140+0000","last_change":"2024-04-25T13:42:15.391073+0000","last_active":"2024-04-25T13:42:17.462140+0000","last_peered":"2024-04-25T13:42:17.462140+0000","last_clean":"2024-04-25T13:42:17.462140+0000","last_became_active":"2024-04-25T13:42:15.390745+0000","last_became_peered":"2024-04-25T13:42:15.390745+0000","last_unstale":"2024-04-25T13:42:17.462140+0000","last_undegraded":"2024-04-25T13:42:17.462140+0000","last_fullsized":"2024-04-25T13:42:17.462140+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:02.905103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460399+0000","last_change":"2024-04-25T13:42:16.403900+0000","last_active":"2024-04-25T13:42:17.460399+0000","last_peered":"2024-04-25T13:42:17.460399+0000","last_clean":"2024-04-25T13:42:17.460399+0000","last_became_active":"2024-04-25T13:42:16.403329+0000","last_became_peered":"2024-04-25T13:42:16.403329+0000","last_unstale":"2024-04-25T13:42:17.460399+0000","last_undegraded":"2024-04-25T13:42:17.460399+0000","last_fullsized":"2024-04-25T13:42:17.460399+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:16.398749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460524+0000","last_change":"2024-04-25T13:42:15.385603+0000","last_active":"2024-04-25T13:42:17.460524+0000","last_peered":"2024-04-25T13:42:17.460524+0000","last_clean":"2024-04-25T13:42:17.460524+0000","last_became_active":"2024-04-25T13:42:15.385172+0000","last_became_peered":"2024-04-25T13:42:15.385172+0000","last_unstale":"2024-04-25T13:42:17.460524+0000","last_undegraded":"2024-04-25T13:42:17.460524+0000","last_fullsized":"2024-04-25T13:42:17.460524+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:31.805247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462685+0000","last_change":"2024-04-25T13:42:16.434718+0000","last_active":"2024-04-25T13:42:17.462685+0000","last_peered":"2024-04-25T13:42:17.462685+0000","last_clean":"2024-04-25T13:42:17.462685+0000","last_became_active":"2024-04-25T13:42:16.428260+0000","last_became_peered":"2024-04-25T13:42:16.428260+0000","last_unstale":"2024-04-25T13:42:17.462685+0000","last_undegraded":"2024-04-25T13:42:17.462685+0000","last_fullsized":"2024-04-25T13:42:17.462685+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:17.272795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431374+0000","last_change":"2024-04-25T13:42:15.413526+0000","last_active":"2024-04-25T13:42:17.431374+0000","last_peered":"2024-04-25T13:42:17.431374+0000","last_clean":"2024-04-25T13:42:17.431374+0000","last_became_active":"2024-04-25T13:42:15.412705+0000","last_became_peered":"2024-04-25T13:42:15.412705+0000","last_unstale":"2024-04-25T13:42:17.431374+0000","last_undegraded":"2024-04-25T13:42:17.431374+0000","last_fullsized":"2024-04-25T13:42:17.431374+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:40.499839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462101+0000","last_change":"2024-04-25T13:42:16.417972+0000","last_active":"2024-04-25T13:42:17.462101+0000","last_peered":"2024-04-25T13:42:17.462101+0000","last_clean":"2024-04-25T13:42:17.462101+0000","last_became_active":"2024-04-25T13:42:16.416376+0000","last_became_peered":"2024-04-25T13:42:16.416376+0000","last_unstale":"2024-04-25T13:42:17.462101+0000","last_undegraded":"2024-04-25T13:42:17.462101+0000","last_fullsized":"2024-04-25T13:42:17.462101+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:12.325792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461011+0000","last_change":"2024-04-25T13:42:15.397902+0000","last_active":"2024-04-25T13:42:17.461011+0000","last_peered":"2024-04-25T13:42:17.461011+0000","last_clean":"2024-04-25T13:42:17.461011+0000","last_became_active":"2024-04-25T13:42:15.396626+0000","last_became_peered":"2024-04-25T13:42:15.396626+0000","last_unstale":"2024-04-25T13:42:17.461011+0000","last_undegraded":"2024-04-25T13:42:17.461011+0000","last_fullsized":"2024-04-25T13:42:17.461011+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:06.890881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462427+0000","last_change":"2024-04-25T13:42:16.407378+0000","last_active":"2024-04-25T13:42:17.462427+0000","last_peered":"2024-04-25T13:42:17.462427+0000","last_clean":"2024-04-25T13:42:17.462427+0000","last_became_active":"2024-04-25T13:42:16.406685+0000","last_became_peered":"2024-04-25T13:42:16.406685+0000","last_unstale":"2024-04-25T13:42:17.462427+0000","last_undegraded":"2024-04-25T13:42:17.462427+0000","last_fullsized":"2024-04-25T13:42:17.462427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:43.797398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384186+0000","last_change":"2024-04-25T13:42:15.408070+0000","last_active":"2024-04-25T13:42:17.384186+0000","last_peered":"2024-04-25T13:42:17.384186+0000","last_clean":"2024-04-25T13:42:17.384186+0000","last_became_active":"2024-04-25T13:42:15.407495+0000","last_became_peered":"2024-04-25T13:42:15.407495+0000","last_unstale":"2024-04-25T13:42:17.384186+0000","last_undegraded":"2024-04-25T13:42:17.384186+0000","last_fullsized":"2024-04-25T13:42:17.384186+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:01.122799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383216+0000","last_change":"2024-04-25T13:42:16.430133+0000","last_active":"2024-04-25T13:42:17.383216+0000","last_peered":"2024-04-25T13:42:17.383216+0000","last_clean":"2024-04-25T13:42:17.383216+0000","last_became_active":"2024-04-25T13:42:16.419312+0000","last_became_peered":"2024-04-25T13:42:16.419312+0000","last_unstale":"2024-04-25T13:42:17.383216+0000","last_undegraded":"2024-04-25T13:42:17.383216+0000","last_fullsized":"2024-04-25T13:42:17.383216+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:06.396676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383954+0000","last_change":"2024-04-25T13:42:15.405691+0000","last_active":"2024-04-25T13:42:17.383954+0000","last_peered":"2024-04-25T13:42:17.383954+0000","last_clean":"2024-04-25T13:42:17.383954+0000","last_became_active":"2024-04-25T13:42:15.405438+0000","last_became_peered":"2024-04-25T13:42:15.405438+0000","last_unstale":"2024-04-25T13:42:17.383954+0000","last_undegraded":"2024-04-25T13:42:17.383954+0000","last_fullsized":"2024-04-25T13:42:17.383954+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:24.931013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459154+0000","last_change":"2024-04-25T13:42:16.387370+0000","last_active":"2024-04-25T13:42:17.459154+0000","last_peered":"2024-04-25T13:42:17.459154+0000","last_clean":"2024-04-25T13:42:17.459154+0000","last_became_active":"2024-04-25T13:42:16.386859+0000","last_became_peered":"2024-04-25T13:42:16.386859+0000","last_unstale":"2024-04-25T13:42:17.459154+0000","last_undegraded":"2024-04-25T13:42:17.459154+0000","last_fullsized":"2024-04-25T13:42:17.459154+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:16.111737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460236+0000","last_change":"2024-04-25T13:42:15.410324+0000","last_active":"2024-04-25T13:42:17.460236+0000","last_peered":"2024-04-25T13:42:17.460236+0000","last_clean":"2024-04-25T13:42:17.460236+0000","last_became_active":"2024-04-25T13:42:15.409511+0000","last_became_peered":"2024-04-25T13:42:15.409511+0000","last_unstale":"2024-04-25T13:42:17.460236+0000","last_undegraded":"2024-04-25T13:42:17.460236+0000","last_fullsized":"2024-04-25T13:42:17.460236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:01.086491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460915+0000","last_change":"2024-04-25T13:42:16.425006+0000","last_active":"2024-04-25T13:42:17.460915+0000","last_peered":"2024-04-25T13:42:17.460915+0000","last_clean":"2024-04-25T13:42:17.460915+0000","last_became_active":"2024-04-25T13:42:16.424668+0000","last_became_peered":"2024-04-25T13:42:16.424668+0000","last_unstale":"2024-04-25T13:42:17.460915+0000","last_undegraded":"2024-04-25T13:42:17.460915+0000","last_fullsized":"2024-04-25T13:42:17.460915+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:23.908161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.463295+0000","last_change":"2024-04-25T13:42:15.405678+0000","last_active":"2024-04-25T13:42:17.463295+0000","last_peered":"2024-04-25T13:42:17.463295+0000","last_clean":"2024-04-25T13:42:17.463295+0000","last_became_active":"2024-04-25T13:42:15.405438+0000","last_became_peered":"2024-04-25T13:42:15.405438+0000","last_unstale":"2024-04-25T13:42:17.463295+0000","last_undegraded":"2024-04-25T13:42:17.463295+0000","last_fullsized":"2024-04-25T13:42:17.463295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:45.389047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461662+0000","last_change":"2024-04-25T13:42:16.431862+0000","last_active":"2024-04-25T13:42:17.461662+0000","last_peered":"2024-04-25T13:42:17.461662+0000","last_clean":"2024-04-25T13:42:17.461662+0000","last_became_active":"2024-04-25T13:42:16.429155+0000","last_became_peered":"2024-04-25T13:42:16.429155+0000","last_unstale":"2024-04-25T13:42:17.461662+0000","last_undegraded":"2024-04-25T13:42:17.461662+0000","last_fullsized":"2024-04-25T13:42:17.461662+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:48.457812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.385529+0000","last_change":"2024-04-25T13:42:15.415424+0000","last_active":"2024-04-25T13:42:17.385529+0000","last_peered":"2024-04-25T13:42:17.385529+0000","last_clean":"2024-04-25T13:42:17.385529+0000","last_became_active":"2024-04-25T13:42:15.411095+0000","last_became_peered":"2024-04-25T13:42:15.411095+0000","last_unstale":"2024-04-25T13:42:17.385529+0000","last_undegraded":"2024-04-25T13:42:17.385529+0000","last_fullsized":"2024-04-25T13:42:17.385529+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:56.921837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462520+0000","last_change":"2024-04-25T13:42:16.410045+0000","last_active":"2024-04-25T13:42:17.462520+0000","last_peered":"2024-04-25T13:42:17.462520+0000","last_clean":"2024-04-25T13:42:17.462520+0000","last_became_active":"2024-04-25T13:42:16.409320+0000","last_became_peered":"2024-04-25T13:42:16.409320+0000","last_unstale":"2024-04-25T13:42:17.462520+0000","last_undegraded":"2024-04-25T13:42:17.462520+0000","last_fullsized":"2024-04-25T13:42:17.462520+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:42.499763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.429531+0000","last_change":"2024-04-25T13:42:15.416407+0000","last_active":"2024-04-25T13:42:17.429531+0000","last_peered":"2024-04-25T13:42:17.429531+0000","last_clean":"2024-04-25T13:42:17.429531+0000","last_became_active":"2024-04-25T13:42:15.416220+0000","last_became_peered":"2024-04-25T13:42:15.416220+0000","last_unstale":"2024-04-25T13:42:17.429531+0000","last_undegraded":"2024-04-25T13:42:17.429531+0000","last_fullsized":"2024-04-25T13:42:17.429531+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:55.196144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462279+0000","last_change":"2024-04-25T13:42:16.431379+0000","last_active":"2024-04-25T13:42:17.462279+0000","last_peered":"2024-04-25T13:42:17.462279+0000","last_clean":"2024-04-25T13:42:17.462279+0000","last_became_active":"2024-04-25T13:42:16.425076+0000","last_became_peered":"2024-04-25T13:42:16.425076+0000","last_unstale":"2024-04-25T13:42:17.462279+0000","last_undegraded":"2024-04-25T13:42:17.462279+0000","last_fullsized":"2024-04-25T13:42:17.462279+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:54.679365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384575+0000","last_change":"2024-04-25T13:42:15.413673+0000","last_active":"2024-04-25T13:42:17.384575+0000","last_peered":"2024-04-25T13:42:17.384575+0000","last_clean":"2024-04-25T13:42:17.384575+0000","last_became_active":"2024-04-25T13:42:15.412944+0000","last_became_peered":"2024-04-25T13:42:15.412944+0000","last_unstale":"2024-04-25T13:42:17.384575+0000","last_undegraded":"2024-04-25T13:42:17.384575+0000","last_fullsized":"2024-04-25T13:42:17.384575+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:31.674149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384243+0000","last_change":"2024-04-25T13:42:16.410338+0000","last_active":"2024-04-25T13:42:17.384243+0000","last_peered":"2024-04-25T13:42:17.384243+0000","last_clean":"2024-04-25T13:42:17.384243+0000","last_became_active":"2024-04-25T13:42:16.409511+0000","last_became_peered":"2024-04-25T13:42:16.409511+0000","last_unstale":"2024-04-25T13:42:17.384243+0000","last_undegraded":"2024-04-25T13:42:17.384243+0000","last_fullsized":"2024-04-25T13:42:17.384243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:15.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.460892+0000","last_change":"2024-04-25T13:42:15.400120+0000","last_active":"2024-04-25T13:42:17.460892+0000","last_peered":"2024-04-25T13:42:17.460892+0000","last_clean":"2024-04-25T13:42:17.460892+0000","last_became_active":"2024-04-25T13:42:15.399445+0000","last_became_peered":"2024-04-25T13:42:15.399445+0000","last_unstale":"2024-04-25T13:42:17.460892+0000","last_undegraded":"2024-04-25T13:42:17.460892+0000","last_fullsized":"2024-04-25T13:42:17.460892+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:07.489364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383310+0000","last_change":"2024-04-25T13:42:16.411710+0000","last_active":"2024-04-25T13:42:17.383310+0000","last_peered":"2024-04-25T13:42:17.383310+0000","last_clean":"2024-04-25T13:42:17.383310+0000","last_became_active":"2024-04-25T13:42:16.410819+0000","last_became_peered":"2024-04-25T13:42:16.410819+0000","last_unstale":"2024-04-25T13:42:17.383310+0000","last_undegraded":"2024-04-25T13:42:17.383310+0000","last_fullsized":"2024-04-25T13:42:17.383310+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:44.088925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.385397+0000","last_change":"2024-04-25T13:42:15.417068+0000","last_active":"2024-04-25T13:42:17.385397+0000","last_peered":"2024-04-25T13:42:17.385397+0000","last_clean":"2024-04-25T13:42:17.385397+0000","last_became_active":"2024-04-25T13:42:15.416290+0000","last_became_peered":"2024-04-25T13:42:15.416290+0000","last_unstale":"2024-04-25T13:42:17.385397+0000","last_undegraded":"2024-04-25T13:42:17.385397+0000","last_fullsized":"2024-04-25T13:42:17.385397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:21.007577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.383666+0000","last_change":"2024-04-25T13:42:16.427646+0000","last_active":"2024-04-25T13:42:17.383666+0000","last_peered":"2024-04-25T13:42:17.383666+0000","last_clean":"2024-04-25T13:42:17.383666+0000","last_became_active":"2024-04-25T13:42:16.426004+0000","last_became_peered":"2024-04-25T13:42:16.426004+0000","last_unstale":"2024-04-25T13:42:17.383666+0000","last_undegraded":"2024-04-25T13:42:17.383666+0000","last_fullsized":"2024-04-25T13:42:17.383666+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:04:30.863151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431448+0000","last_change":"2024-04-25T13:42:15.395662+0000","last_active":"2024-04-25T13:42:17.431448+0000","last_peered":"2024-04-25T13:42:17.431448+0000","last_clean":"2024-04-25T13:42:17.431448+0000","last_became_active":"2024-04-25T13:42:15.394697+0000","last_became_peered":"2024-04-25T13:42:15.394697+0000","last_unstale":"2024-04-25T13:42:17.431448+0000","last_undegraded":"2024-04-25T13:42:17.431448+0000","last_fullsized":"2024-04-25T13:42:17.431448+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:07.448537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459650+0000","last_change":"2024-04-25T13:42:16.388333+0000","last_active":"2024-04-25T13:42:17.459650+0000","last_peered":"2024-04-25T13:42:17.459650+0000","last_clean":"2024-04-25T13:42:17.459650+0000","last_became_active":"2024-04-25T13:42:16.387684+0000","last_became_peered":"2024-04-25T13:42:16.387684+0000","last_unstale":"2024-04-25T13:42:17.459650+0000","last_undegraded":"2024-04-25T13:42:17.459650+0000","last_fullsized":"2024-04-25T13:42:17.459650+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:01.926554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.459283+0000","last_change":"2024-04-25T13:42:15.411890+0000","last_active":"2024-04-25T13:42:17.459283+0000","last_peered":"2024-04-25T13:42:17.459283+0000","last_clean":"2024-04-25T13:42:17.459283+0000","last_became_active":"2024-04-25T13:42:15.411682+0000","last_became_peered":"2024-04-25T13:42:15.411682+0000","last_unstale":"2024-04-25T13:42:17.459283+0000","last_undegraded":"2024-04-25T13:42:17.459283+0000","last_fullsized":"2024-04-25T13:42:17.459283+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:21.495394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.431251+0000","last_change":"2024-04-25T13:42:16.409658+0000","last_active":"2024-04-25T13:42:17.431251+0000","last_peered":"2024-04-25T13:42:17.431251+0000","last_clean":"2024-04-25T13:42:17.431251+0000","last_became_active":"2024-04-25T13:42:16.408520+0000","last_became_peered":"2024-04-25T13:42:16.408520+0000","last_unstale":"2024-04-25T13:42:17.431251+0000","last_undegraded":"2024-04-25T13:42:17.431251+0000","last_fullsized":"2024-04-25T13:42:17.431251+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:51.209293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.462898+0000","last_change":"2024-04-25T13:42:15.390830+0000","last_active":"2024-04-25T13:42:17.462898+0000","last_peered":"2024-04-25T13:42:17.462898+0000","last_clean":"2024-04-25T13:42:17.462898+0000","last_became_active":"2024-04-25T13:42:15.390392+0000","last_became_peered":"2024-04-25T13:42:15.390392+0000","last_unstale":"2024-04-25T13:42:17.462898+0000","last_undegraded":"2024-04-25T13:42:17.462898+0000","last_fullsized":"2024-04-25T13:42:17.462898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:49.764917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461797+0000","last_change":"2024-04-25T13:42:16.412510+0000","last_active":"2024-04-25T13:42:17.461797+0000","last_peered":"2024-04-25T13:42:17.461797+0000","last_clean":"2024-04-25T13:42:17.461797+0000","last_became_active":"2024-04-25T13:42:16.411801+0000","last_became_peered":"2024-04-25T13:42:16.411801+0000","last_unstale":"2024-04-25T13:42:17.461797+0000","last_undegraded":"2024-04-25T13:42:17.461797+0000","last_fullsized":"2024-04-25T13:42:17.461797+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:15.361419+0000","last_clean_scrub_stamp":"2024-04-25T13:42:15.361419+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:04.674217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.461255+0000","last_change":"2024-04-25T13:42:15.396434+0000","last_active":"2024-04-25T13:42:17.461255+0000","last_peered":"2024-04-25T13:42:17.461255+0000","last_clean":"2024-04-25T13:42:17.461255+0000","last_became_active":"2024-04-25T13:42:15.395632+0000","last_became_peered":"2024-04-25T13:42:15.395632+0000","last_unstale":"2024-04-25T13:42:17.461255+0000","last_undegraded":"2024-04-25T13:42:17.461255+0000","last_fullsized":"2024-04-25T13:42:17.461255+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:42:14.350346+0000","last_clean_scrub_stamp":"2024-04-25T13:42:14.350346+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:52:08.577527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-04-25T13:42:17.384734+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:42:17.384734+0000","last_peered":"2024-04-25T13:42:17.384734+0000","last_clean":"2024-04-25T13:42:17.384734+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:42:17.384734+0000","last_undegraded":"2024-04-25T13:42:17.384734+0000","last_fullsized":"2024-04-25T13:42:17.384734+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:21.321904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":798,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194514,"statfs":{"total":200000000,"available":199182761,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":222,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199772547,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":221,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199773149,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":223,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195089,"statfs":{"total":200000000,"available":199771607,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":222,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199772581,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":223,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195089,"statfs":{"total":200000000,"available":199771585,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":221,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195090,"statfs":{"total":200000000,"available":199773149,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":791,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194520,"statfs":{"total":200000000,"available":199189117,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:42:20.915 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:42:20.916 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:42:21.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f314971e640 1 Processor -- start 2024-04-25T13:42:21.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f314971e640 1 -- start start 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f314971e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3144161630 0x7f314406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f314971e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f314406cf90 con 0x7f31440cfb60 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f314971e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f314406d270 con 0x7f314415d630 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f314971e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3144143920 con 0x7f3144161630 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f3142ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3144161630 0x7f314406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f31427fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f31440cfb60 0x7f31440cff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:40434/0 (socket says 172.21.15.80:40434) 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f31427fc640 1 -- 172.21.15.80:0/1850724563 learned_addr learned my addr 172.21.15.80:0/1850724563 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f3141ffb640 1 -- 172.21.15.80:0/1850724563 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3414428962 0 0) 0x7f314406cf90 con 0x7f31440cfb60 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f3141ffb640 1 -- 172.21.15.80:0/1850724563 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f312c003530 con 0x7f31440cfb60 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f3142ffd640 1 -- 172.21.15.80:0/1850724563 >> v1:172.21.15.87:6789/0 conn(0x7f314415d630 legacy=0x7f314415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f3142ffd640 1 -- 172.21.15.80:0/1850724563 >> v1:172.21.15.80:6789/0 conn(0x7f31440cfb60 legacy=0x7f31440cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.105+0000 7f3142ffd640 1 -- 172.21.15.80:0/1850724563 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3144160e30 con 0x7f3144161630 2024-04-25T13:42:21.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f3142ffd640 1 --2- 172.21.15.80:0/1850724563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3144161630 0x7f314406c5f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f313c009d90 tx=0x7f313c01f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1b29c04837ace63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f3141ffb640 1 -- 172.21.15.80:0/1850724563 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c0b4070 con 0x7f3144161630 2024-04-25T13:42:21.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f3141ffb640 1 -- 172.21.15.80:0/1850724563 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c0af600 con 0x7f3144161630 2024-04-25T13:42:21.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f3141ffb640 1 -- 172.21.15.80:0/1850724563 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c0af9b0 con 0x7f3144161630 2024-04-25T13:42:21.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f314971e640 1 -- 172.21.15.80:0/1850724563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3144161630 msgr2=0x7f314406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f314971e640 1 --2- 172.21.15.80:0/1850724563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3144161630 0x7f314406c5f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f313c009d90 tx=0x7f313c01f120 comp rx=0 tx=0).stop 2024-04-25T13:42:21.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f314971e640 1 -- 172.21.15.80:0/1850724563 shutdown_connections 2024-04-25T13:42:21.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f314971e640 1 --2- 172.21.15.80:0/1850724563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3144161630 0x7f314406c5f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f314971e640 1 -- 172.21.15.80:0/1850724563 >> 172.21.15.80:0/1850724563 conn(0x7f31440d33a0 msgr2=0x7f31440cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f314971e640 1 -- 172.21.15.80:0/1850724563 shutdown_connections 2024-04-25T13:42:21.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.109+0000 7f314971e640 1 -- 172.21.15.80:0/1850724563 wait complete. 2024-04-25T13:42:21.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 Processor -- start 2024-04-25T13:42:21.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 -- start start 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31440cfb60 0x7f31441527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f314415d630 0x7f314415fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3144161630 0x7f31441601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3144161a40 con 0x7f31440cfb60 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f314415e150 con 0x7f3144161630 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f31441f6350 con 0x7f314415d630 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31440cfb60 0x7f31441527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31440cfb60 0x7f31441527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34898/0 (socket says 172.21.15.80:34898) 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 -- 172.21.15.80:0/1234415819 learned_addr learned my addr 172.21.15.80:0/1234415819 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f31437fe640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3144161630 0x7f31441601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f31427fc640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f314415d630 0x7f314415fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 -- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f314415d630 msgr2=0x7f314415fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f314415d630 0x7f314415fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 -- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3144161630 msgr2=0x7f31441601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3144161630 0x7f31441601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 -- 172.21.15.80:0/1234415819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3144156d10 con 0x7f31440cfb60 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f31427fc640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f314415d630 0x7f314415fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f3142ffd640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31440cfb60 0x7f31441527f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f313c01cd50 tx=0x7f313c01bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f312b7fe640 1 -- 172.21.15.80:0/1234415819 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c0b4070 con 0x7f31440cfb60 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f312b7fe640 1 -- 172.21.15.80:0/1234415819 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f313c0b5040 con 0x7f31440cfb60 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31441e8c70 con 0x7f31440cfb60 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f312b7fe640 1 -- 172.21.15.80:0/1234415819 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313c0af9c0 con 0x7f31440cfb60 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.113+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31441606e0 con 0x7f31440cfb60 2024-04-25T13:42:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.117+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314406a3a0 con 0x7f31440cfb60 2024-04-25T13:42:21.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.117+0000 7f312b7fe640 1 -- 172.21.15.80:0/1234415819 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f313c0c3020 con 0x7f31440cfb60 2024-04-25T13:42:21.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.117+0000 7f312b7fe640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f313006f430 0x7f3130071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.117+0000 7f312b7fe640 1 -- 172.21.15.80:0/1234415819 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f313c12d270 con 0x7f31440cfb60 2024-04-25T13:42:21.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.117+0000 7f31427fc640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f313006f430 0x7f3130071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.117+0000 7f31427fc640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f313006f430 0x7f3130071940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31441e8480 tx=0x7f3138009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.121+0000 7f312b7fe640 1 -- 172.21.15.80:0/1234415819 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f313c0ca020 con 0x7f31440cfb60 2024-04-25T13:42:21.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.325+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f31440d0c40 con 0x7f31440cfb60 2024-04-25T13:42:21.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.329+0000 7f312b7fe640 1 -- 172.21.15.80:0/1234415819 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) ==== 190+0+147 (secure 0 0 0) 0x7f313c018c80 con 0x7f31440cfb60 2024-04-25T13:42:21.334 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:42:21.334 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:42:21.334 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:42:21.334 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:42:21.335 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:42:21.335 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:42:21.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f313006f430 msgr2=0x7f3130071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f313006f430 0x7f3130071940 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f31441e8480 tx=0x7f3138009040 comp rx=0 tx=0).stop 2024-04-25T13:42:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31440cfb60 msgr2=0x7f31441527f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31440cfb60 0x7f31441527f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f313c01cd50 tx=0x7f313c01bec0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 shutdown_connections 2024-04-25T13:42:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f313006f430 0x7f3130071940 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3144161630 0x7f31441601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f314415d630 0x7f314415fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 --2- 172.21.15.80:0/1234415819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31440cfb60 0x7f31441527f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 >> 172.21.15.80:0/1234415819 conn(0x7f31440d33a0 msgr2=0x7f314415da40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 shutdown_connections 2024-04-25T13:42:21.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.333+0000 7f314971e640 1 -- 172.21.15.80:0/1234415819 wait complete. 2024-04-25T13:42:21.347 INFO:tasks.ceph.mds.c.smithi080.stderr:2024-04-25T13:42:21.341+0000 7fd67622d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:42:21.354 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:21.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828f1c1640 1 Processor -- start 2024-04-25T13:42:21.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828f1c1640 1 -- start start 2024-04-25T13:42:21.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828f1c1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f82880d04e0 0x7f82880d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828f1c1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82880d0e30 con 0x7f82880b7650 2024-04-25T13:42:21.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828f1c1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82880cfb20 con 0x7f82880b9240 2024-04-25T13:42:21.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828f1c1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f82880b6920 con 0x7f82880d04e0 2024-04-25T13:42:21.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828d75e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f82880b7650 0x7f82880b7a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:40444/0 (socket says 172.21.15.80:40444) 2024-04-25T13:42:21.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828d75e640 1 -- 172.21.15.80:0/3168566496 learned_addr learned my addr 172.21.15.80:0/3168566496 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:21.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f8287fff640 1 --2- 172.21.15.80:0/3168566496 >> v2:172.21.15.87:3300/0 conn(0x7f82880d04e0 0x7f82880d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1346064641 0 0) 0x7f82800014a0 con 0x7f82880b7650 2024-04-25T13:42:21.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8270003530 con 0x7f82880b7650 2024-04-25T13:42:21.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1394228231 0 0) 0x7f8270003530 con 0x7f82880b7650 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 >> v2:172.21.15.87:3300/0 conn(0x7f82880d04e0 msgr2=0x7f82880d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 --2- 172.21.15.80:0/3168566496 >> v2:172.21.15.87:3300/0 conn(0x7f82880d04e0 0x7f82880d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 >> v1:172.21.15.87:6789/0 conn(0x7f82880b9240 legacy=0x7f82880cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82880b6b70 con 0x7f82880b7650 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f8280001bd0 con 0x7f82880b7650 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 >> v1:172.21.15.80:6789/0 conn(0x7f82880b7650 legacy=0x7f82880b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82700039c0 0x7f8270003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8270004810 0x7f8270004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8270005260 0x7f827000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f82880b6920 con 0x7f8270005260 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f827000c1f0 con 0x7f82700039c0 2024-04-25T13:42:21.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f827000c3f0 con 0x7f8270004810 2024-04-25T13:42:21.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.501+0000 7f828d75e640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8270005260 0x7f827000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828d75e640 1 -- 172.21.15.80:0/3168566496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8270004810 msgr2=0x7f8270004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828d75e640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8270004810 0x7f8270004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828d75e640 1 -- 172.21.15.80:0/3168566496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82700039c0 msgr2=0x7f8270003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:21.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828d75e640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82700039c0 0x7f8270003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828d75e640 1 -- 172.21.15.80:0/3168566496 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8270003530 con 0x7f8270005260 2024-04-25T13:42:21.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828d75e640 1 -- 172.21.15.80:0/3168566496 reap_dead start 2024-04-25T13:42:21.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828d75e640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8270005260 0x7f827000b9d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f82880b97d0 tx=0x7f828001faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334018e59d455ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828000a670 con 0x7f8270005260 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82800aa6f0 con 0x7f8270005260 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f82877fe640 1 -- 172.21.15.80:0/3168566496 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82800aaae0 con 0x7f8270005260 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828f1c1640 1 -- 172.21.15.80:0/3168566496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8270005260 msgr2=0x7f827000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828f1c1640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8270005260 0x7f827000b9d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f82880b97d0 tx=0x7f828001faa0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828f1c1640 1 -- 172.21.15.80:0/3168566496 shutdown_connections 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828f1c1640 1 --2- 172.21.15.80:0/3168566496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8270005260 0x7f827000b9d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828f1c1640 1 -- 172.21.15.80:0/3168566496 >> 172.21.15.80:0/3168566496 conn(0x7f82880d3ad0 msgr2=0x7f82880d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828f1c1640 1 -- 172.21.15.80:0/3168566496 shutdown_connections 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.505+0000 7f828f1c1640 1 -- 172.21.15.80:0/3168566496 wait complete. 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 Processor -- start 2024-04-25T13:42:21.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 -- start start 2024-04-25T13:42:21.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270005260 0x7f828815cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82881f4e10 0x7f8288160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288161080 0x7f828815a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f82880d08f0 con 0x7f8288161080 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82880b7130 con 0x7f8270005260 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82880d0a70 con 0x7f82881f4e10 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288161080 0x7f828815a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288161080 0x7f828815a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34918/0 (socket says 172.21.15.80:34918) 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 -- 172.21.15.80:0/1133856232 learned_addr learned my addr 172.21.15.80:0/1133856232 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828cf5d640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270005260 0x7f828815cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f8287fff640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82881f4e10 0x7f8288160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 -- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82881f4e10 msgr2=0x7f8288160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82881f4e10 0x7f8288160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 -- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270005260 msgr2=0x7f828815cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270005260 0x7f828815cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 -- 172.21.15.80:0/1133856232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82880cc580 con 0x7f8288161080 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f8287fff640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82881f4e10 0x7f8288160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828cf5d640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270005260 0x7f828815cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828d75e640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288161080 0x7f828815a470 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8280020340 tx=0x7f8270005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f82857fa640 1 -- 172.21.15.80:0/1133856232 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82800bd070 con 0x7f8288161080 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f82857fa640 1 -- 172.21.15.80:0/1133856232 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8280001df0 con 0x7f8288161080 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f82857fa640 1 -- 172.21.15.80:0/1133856232 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82800014a0 con 0x7f8288161080 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82880cca80 con 0x7f8288161080 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.509+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82880cd040 con 0x7f8288161080 2024-04-25T13:42:21.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.513+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82880cc7f0 con 0x7f8288161080 2024-04-25T13:42:21.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.513+0000 7f82857fa640 1 -- 172.21.15.80:0/1133856232 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f82800c0070 con 0x7f8288161080 2024-04-25T13:42:21.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.513+0000 7f82857fa640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f827406f430 0x7f8274071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:21.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.513+0000 7f82857fa640 1 -- 172.21.15.80:0/1133856232 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f828012bd20 con 0x7f8288161080 2024-04-25T13:42:21.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.513+0000 7f828cf5d640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f827406f430 0x7f8274071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:21.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.513+0000 7f828cf5d640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f827406f430 0x7f8274071940 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8278000fc0 tx=0x7f8278009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:21.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.517+0000 7f82857fa640 1 -- 172.21.15.80:0/1133856232 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82880d0e30 con 0x7f8288161080 2024-04-25T13:42:21.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.697+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f828806a3a0 con 0x7f8288161080 2024-04-25T13:42:21.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f82857fa640 1 -- 172.21.15.80:0/1133856232 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f82800f8ca0 con 0x7f8288161080 2024-04-25T13:42:21.703 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:21.703 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:42:21.703 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:42:21.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f827406f430 msgr2=0x7f8274071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f827406f430 0x7f8274071940 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f8278000fc0 tx=0x7f8278009040 comp rx=0 tx=0).stop 2024-04-25T13:42:21.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288161080 msgr2=0x7f828815a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:21.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288161080 0x7f828815a470 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8280020340 tx=0x7f8270005840 comp rx=0 tx=0).stop 2024-04-25T13:42:21.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 shutdown_connections 2024-04-25T13:42:21.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f827406f430 0x7f8274071940 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8288161080 0x7f828815a470 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82881f4e10 0x7f8288160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 --2- 172.21.15.80:0/1133856232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8270005260 0x7f828815cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:21.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 >> 172.21.15.80:0/1133856232 conn(0x7f82880d3ad0 msgr2=0x7f8288071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:21.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 shutdown_connections 2024-04-25T13:42:21.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:21.701+0000 7f828f1c1640 1 -- 172.21.15.80:0/1133856232 wait complete. 2024-04-25T13:42:22.125 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:42:15.784264+0000', 'modified': '2024-04-25T13:42:18.840205+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5730}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5730': {'gid': 5730, 'name': 'c', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6835/1692562493', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 1692562493}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 1692562493}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5730, 'qdb_cluster': [5730]} 2024-04-25T13:42:22.125 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:42:22.126 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T13:42:22.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa268cf4640 1 Processor -- start 2024-04-25T13:42:22.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa268cf4640 1 -- start start 2024-04-25T13:42:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa268cf4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 0x7fa264160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa268cf4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa2640733b0 0x7fa264161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa268cf4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa26406fcf0 con 0x7fa264161070 2024-04-25T13:42:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa268cf4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa2641619c0 con 0x7fa2640733b0 2024-04-25T13:42:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa268cf4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa264073f10 con 0x7fa26406f5a0 2024-04-25T13:42:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa262d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa2640733b0 0x7fa264161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa262d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa2640733b0 0x7fa264161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34930/0 (socket says 172.21.15.80:34930) 2024-04-25T13:42:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa262d9d640 1 -- 172.21.15.80:0/4196662221 learned_addr learned my addr 172.21.15.80:0/4196662221 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:22.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa26259c640 1 --2- 172.21.15.80:0/4196662221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 0x7fa264160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa262d9d640 1 -- 172.21.15.80:0/4196662221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 msgr2=0x7fa264160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa262d9d640 1 --2- 172.21.15.80:0/4196662221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 0x7fa264160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.277+0000 7fa262d9d640 1 -- 172.21.15.80:0/4196662221 >> v1:172.21.15.87:6789/0 conn(0x7fa264161070 legacy=0x7fa2640b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa262d9d640 1 -- 172.21.15.80:0/4196662221 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2640b0c60 con 0x7fa2640733b0 2024-04-25T13:42:22.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa262d9d640 1 --2- 172.21.15.80:0/4196662221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa264161480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009e80 tx=0x7fa25c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb4defeae67c385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa26159a640 1 -- 172.21.15.80:0/4196662221 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0a5670 con 0x7fa2640733b0 2024-04-25T13:42:22.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa26159a640 1 -- 172.21.15.80:0/4196662221 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25c0a5810 con 0x7fa2640733b0 2024-04-25T13:42:22.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa26159a640 1 -- 172.21.15.80:0/4196662221 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0a5be0 con 0x7fa2640733b0 2024-04-25T13:42:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa268cf4640 1 -- 172.21.15.80:0/4196662221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 msgr2=0x7fa264161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa268cf4640 1 --2- 172.21.15.80:0/4196662221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa264161480 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fa25c009e80 tx=0x7fa25c01c9a0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa268cf4640 1 -- 172.21.15.80:0/4196662221 shutdown_connections 2024-04-25T13:42:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa268cf4640 1 --2- 172.21.15.80:0/4196662221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa264161480 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa268cf4640 1 --2- 172.21.15.80:0/4196662221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 0x7fa264160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa268cf4640 1 -- 172.21.15.80:0/4196662221 >> 172.21.15.80:0/4196662221 conn(0x7fa26406d080 msgr2=0x7fa2640b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa268cf4640 1 -- 172.21.15.80:0/4196662221 shutdown_connections 2024-04-25T13:42:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.281+0000 7fa268cf4640 1 -- 172.21.15.80:0/4196662221 wait complete. 2024-04-25T13:42:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 Processor -- start 2024-04-25T13:42:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 -- start start 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 0x7fa26414c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa26415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa264161070 0x7fa2640af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa26416a050 con 0x7fa2640733b0 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa264169ed0 con 0x7fa264161070 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa26416a1d0 con 0x7fa26406f5a0 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa26415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa26415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34932/0 (socket says 172.21.15.80:34932) 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 -- 172.21.15.80:0/2905956591 learned_addr learned my addr 172.21.15.80:0/2905956591 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa262d9d640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa264161070 0x7fa2640af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa26259c640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 0x7fa26414c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 -- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 msgr2=0x7fa26414c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 0x7fa26414c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 -- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa264161070 msgr2=0x7fa2640af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa264161070 0x7fa2640af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 -- 172.21.15.80:0/2905956591 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa26415daf0 con 0x7fa2640733b0 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa261d9b640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa26415d5b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c310 tx=0x7fa25800c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa252ffd640 1 -- 172.21.15.80:0/2905956591 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258017070 con 0x7fa2640733b0 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa252ffd640 1 -- 172.21.15.80:0/2905956591 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa2580052d0 con 0x7fa2640733b0 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 -- 172.21.15.80:0/2905956591 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa26415c150 con 0x7fa2640733b0 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa252ffd640 1 -- 172.21.15.80:0/2905956591 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258003840 con 0x7fa2640733b0 2024-04-25T13:42:22.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.285+0000 7fa268cf4640 1 -- 172.21.15.80:0/2905956591 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa26415ad10 con 0x7fa2640733b0 2024-04-25T13:42:22.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.289+0000 7fa252ffd640 1 -- 172.21.15.80:0/2905956591 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa25801a070 con 0x7fa2640733b0 2024-04-25T13:42:22.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.289+0000 7fa252ffd640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa24406f430 0x7fa244071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.289+0000 7fa252ffd640 1 -- 172.21.15.80:0/2905956591 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fa25808e480 con 0x7fa2640733b0 2024-04-25T13:42:22.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.289+0000 7fa26259c640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa24406f430 0x7fa244071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.289+0000 7fa270051000 1 Processor -- start 2024-04-25T13:42:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.289+0000 7fa270051000 1 -- start start 2024-04-25T13:42:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.289+0000 7fa26259c640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa24406f430 0x7fa244071940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa2640b6e70 tx=0x7fa254008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582680b1c20 0x55826820fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x558268210420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558267c96280 0x558268210960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55826807f160 con 0x558267c96280 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558268014e20 con 0x5582680b1c20 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x558267fcda20 con 0x55826809dea0 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558267c96280 0x558268210960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558267c96280 0x558268210960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34936/0 (socket says 172.21.15.80:34936) 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 -- 172.21.15.80:0/651694966 learned_addr learned my addr 172.21.15.80:0/651694966 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:22.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa261d9b640 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x558268210420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa26259c640 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582680b1c20 0x55826820fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 -- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 msgr2=0x558268210420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x558268210420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 -- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582680b1c20 msgr2=0x55826820fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582680b1c20 0x55826820fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 -- 172.21.15.80:0/651694966 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558267fd2100 con 0x558267c96280 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa26259c640 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582680b1c20 0x55826820fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa262d9d640 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558267c96280 0x558268210960 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa25c01a5d0 tx=0x7fa25c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d51c13b2face67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa243fff640 1 -- 172.21.15.80:0/651694966 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0bd070 con 0x558267c96280 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa243fff640 1 -- 172.21.15.80:0/651694966 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25c0a5670 con 0x558267c96280 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa243fff640 1 -- 172.21.15.80:0/651694966 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0a5a20 con 0x558267c96280 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- 172.21.15.80:0/651694966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558267c96280 msgr2=0x558268210960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558267c96280 0x558268210960 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa25c01a5d0 tx=0x7fa25c004ac0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- 172.21.15.80:0/651694966 shutdown_connections 2024-04-25T13:42:22.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x558267c96280 0x558268210960 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x558268210420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 --2- 172.21.15.80:0/651694966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582680b1c20 0x55826820fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- 172.21.15.80:0/651694966 >> 172.21.15.80:0/651694966 conn(0x558267cd5ff0 msgr2=0x5582681c54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:22.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- 172.21.15.80:0/651694966 shutdown_connections 2024-04-25T13:42:22.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- 172.21.15.80:0/651694966 wait complete. 2024-04-25T13:42:22.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 -- start start 2024-04-25T13:42:22.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.293+0000 7fa270051000 1 Processor -- start 2024-04-25T13:42:22.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x558268217070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x5582682175b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x5582681d20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x558268022220 con 0x5582680b1c20 2024-04-25T13:42:22.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558267fb8ea0 con 0x558267c96280 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55826800fb00 con 0x55826809dea0 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 10 client.0 _ll_register_callbacks cb 0x558268021dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x5582681d20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x5582681d20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34944/0 (socket says 172.21.15.80:34944) 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 -- 172.21.15.80:0/432198270 learned_addr learned my addr 172.21.15.80:0/432198270 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa26259c640 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x558268217070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa261d9b640 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x5582682175b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 -- 172.21.15.80:0/432198270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 msgr2=0x5582682175b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x5582682175b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 -- 172.21.15.80:0/432198270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 msgr2=0x558268217070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x558268217070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 -- 172.21.15.80:0/432198270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558267fb4500 con 0x5582680b1c20 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa262d9d640 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x5582681d20c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0b9040 tx=0x7fa25c01b1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5a53cc0901b6ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:42:22.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 1 -- 172.21.15.80:0/432198270 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0c6060 con 0x5582680b1c20 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- 172.21.15.80:0/432198270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa25c0a5670 con 0x5582680b1c20 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 1 -- 172.21.15.80:0/432198270 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fa25c003ca0 con 0x5582680b1c20 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 1 -- 172.21.15.80:0/432198270 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa25c0ba070 con 0x5582680b1c20 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 10 client.5868 fetch_fsmap learned FSMap version 24 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- 172.21.15.80:0/432198270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x558267f9aea0 con 0x5582680b1c20 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 1 -- 172.21.15.80:0/432198270 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25c0a5670 con 0x5582680b1c20 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 1 -- 172.21.15.80:0/432198270 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7fa25c0a5820 con 0x5582680b1c20 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 10 client.5868 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 10 client.5868 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 20 client.5868 populate_metadata read hostname 'smithi080' 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] conn(0x558267cba7d0 0x5582681d2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 4 client.5868 mds_command: new command op to 5730 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- 172.21.15.80:0/432198270 --> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558267fadbe0 con 0x558267cba7d0 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa26259c640 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] conn(0x558267cba7d0 0x5582681d2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa26259c640 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] conn(0x558267cba7d0 0x5582681d2e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a57c763e8681b2b server_cookie=6d99fff6fc811b11 in_seq=0 out_seq=0 2024-04-25T13:42:22.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 10 client.5868 ms_handle_connect on v2:172.21.15.80:6834/1692562493 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 1 -- 172.21.15.80:0/432198270 <== mds.0 v2:172.21.15.80:6834/1692562493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558267fadbe0 con 0x558267cba7d0 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 10 client.5868 handle_command_reply: tid=0 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 client.5868 shutdown 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- 172.21.15.80:0/432198270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 msgr2=0x5582681d20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x5582681d20c0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0b9040 tx=0x7fa25c01b1e0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- 172.21.15.80:0/432198270 shutdown_connections 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] conn(0x558267cba7d0 0x5582681d2e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x5582681d20c0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x5582682175b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 --2- 172.21.15.80:0/432198270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x558268217070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- 172.21.15.80:0/432198270 >> 172.21.15.80:0/432198270 conn(0x558267cd5ff0 msgr2=0x5582681c5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa2417fa640 0 client.5868 ms_handle_reset on v2:172.21.15.80:6834/1692562493 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- 172.21.15.80:0/432198270 shutdown_connections 2024-04-25T13:42:22.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 1 -- 172.21.15.80:0/432198270 wait complete. 2024-04-25T13:42:22.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.297+0000 7fa270051000 20 client.5868 trim_cache size 0 max 16384 2024-04-25T13:42:22.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 Processor -- start 2024-04-25T13:42:22.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- start start 2024-04-25T13:42:22.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x558268216440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x558268227210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x558268227750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa26415fe80 con 0x5582680b1c20 2024-04-25T13:42:22.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa26415f1e0 con 0x558267c96280 2024-04-25T13:42:22.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa26415f830 con 0x55826809dea0 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x558268227750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x558268227750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34960/0 (socket says 172.21.15.80:34960) 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 -- 172.21.15.80:0/1489645499 learned_addr learned my addr 172.21.15.80:0/1489645499 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 -- 172.21.15.80:0/1489645499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 msgr2=0x558268227210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 --2- 172.21.15.80:0/1489645499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x558268227210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 -- 172.21.15.80:0/1489645499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 msgr2=0x558268216440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa26259c640 1 --2- 172.21.15.80:0/1489645499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x558268216440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 --2- 172.21.15.80:0/1489645499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x558268216440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 -- 172.21.15.80:0/1489645499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582681c9840 con 0x5582680b1c20 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa262d9d640 1 --2- 172.21.15.80:0/1489645499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x558268227750 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa25c01c9e0 tx=0x7fa25c0ba000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50c7bf14908612e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa2437fe640 1 -- 172.21.15.80:0/1489645499 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0bd040 con 0x5582680b1c20 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa2437fe640 1 -- 172.21.15.80:0/1489645499 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25c0a5670 con 0x5582680b1c20 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa2437fe640 1 -- 172.21.15.80:0/1489645499 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c0039a0 con 0x5582680b1c20 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- 172.21.15.80:0/1489645499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 msgr2=0x558268227750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 --2- 172.21.15.80:0/1489645499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x558268227750 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa25c01c9e0 tx=0x7fa25c0ba000 comp rx=0 tx=0).stop 2024-04-25T13:42:22.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- 172.21.15.80:0/1489645499 shutdown_connections 2024-04-25T13:42:22.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 --2- 172.21.15.80:0/1489645499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5582680b1c20 0x558268227750 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 --2- 172.21.15.80:0/1489645499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55826809dea0 0x558268227210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 --2- 172.21.15.80:0/1489645499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x558268216440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- 172.21.15.80:0/1489645499 >> 172.21.15.80:0/1489645499 conn(0x558267cd5ff0 msgr2=0x5582681cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:22.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- 172.21.15.80:0/1489645499 shutdown_connections 2024-04-25T13:42:22.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.317+0000 7fa270051000 1 -- 172.21.15.80:0/1489645499 wait complete. 2024-04-25T13:42:22.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- start start 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 Processor -- start 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x5582682827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55826809dea0 0x558268282ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5582680b1c20 0x55826821b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x558268014130 con 0x55826809dea0 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x558267faa7d0 con 0x558267c96280 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x558268019450 con 0x5582680b1c20 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 10 client.0 _ll_register_callbacks cb 0x7fa25c009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55826809dea0 0x558268282ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55826809dea0 0x558268282ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34976/0 (socket says 172.21.15.80:34976) 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 -- 172.21.15.80:0/3591071173 learned_addr learned my addr 172.21.15.80:0/3591071173 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 -- 172.21.15.80:0/3591071173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5582680b1c20 msgr2=0x55826821b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5582680b1c20 0x55826821b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa26259c640 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x5582682827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 -- 172.21.15.80:0/3591071173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 msgr2=0x5582682827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x5582682827a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 -- 172.21.15.80:0/3591071173 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582681d2430 con 0x55826809dea0 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa261d9b640 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55826809dea0 0x558268282ce0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa258056d00 tx=0x7fa25805ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1792132e900ab0c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 1 -- 172.21.15.80:0/3591071173 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa258074020 con 0x55826809dea0 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 1 -- 172.21.15.80:0/3591071173 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fa25805a0e0 con 0x55826809dea0 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- 172.21.15.80:0/3591071173 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa25c0a5670 con 0x55826809dea0 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 1 -- 172.21.15.80:0/3591071173 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa25808e2c0 con 0x55826809dea0 2024-04-25T13:42:22.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 10 client.5874 fetch_fsmap learned FSMap version 24 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- 172.21.15.80:0/3591071173 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x558267fcdc70 con 0x55826809dea0 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 1 -- 172.21.15.80:0/3591071173 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa25c0a5670 con 0x55826809dea0 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 1 -- 172.21.15.80:0/3591071173 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 24) ==== 3494+0+0 (secure 0 0 0) 0x7fa2580565f0 con 0x55826809dea0 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 10 client.5874 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 10 client.5874 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 20 client.5874 populate_metadata read hostname 'smithi080' 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] conn(0x558267ca8830 0x55826821ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 4 client.5874 mds_command: new command op to 5730 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- 172.21.15.80:0/3591071173 --> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] -- command(tid 0: {"prefix": "status"}) -- 0x558268022a30 con 0x558267ca8830 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa26259c640 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] conn(0x558267ca8830 0x55826821ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 10 client.5874 ms_handle_connect on v2:172.21.15.80:6834/1692562493 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa26259c640 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] conn(0x558267ca8830 0x55826821ba90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f07b72eaaf9cb22 server_cookie=fdb0e29e591c73ad in_seq=0 out_seq=0 2024-04-25T13:42:22.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 1 -- 172.21.15.80:0/3591071173 <== mds.0 v2:172.21.15.80:6834/1692562493 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x558268022a30 con 0x558267ca8830 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 10 client.5874 handle_command_reply: tid=0 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 client.5874 shutdown 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- 172.21.15.80:0/3591071173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55826809dea0 msgr2=0x558268282ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55826809dea0 0x558268282ce0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fa258056d00 tx=0x7fa25805ab60 comp rx=0 tx=0).stop 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- 172.21.15.80:0/3591071173 shutdown_connections 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5582680b1c20 0x55826821b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.80:6834/1692562493,v1:172.21.15.80:6835/1692562493] conn(0x558267ca8830 0x55826821ba90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55826809dea0 0x558268282ce0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 --2- 172.21.15.80:0/3591071173 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x558267c96280 0x5582682827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa2417fa640 0 client.5874 ms_handle_reset on v2:172.21.15.80:6834/1692562493 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- 172.21.15.80:0/3591071173 >> 172.21.15.80:0/3591071173 conn(0x558267cd5ff0 msgr2=0x558268226200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- 172.21.15.80:0/3591071173 shutdown_connections 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 1 -- 172.21.15.80:0/3591071173 wait complete. 2024-04-25T13:42:22.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.321+0000 7fa270051000 20 client.5874 trim_cache size 0 max 16384 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5730, 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 5.9056030430000002, 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 24, 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 47, 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 47, 2024-04-25T13:42:22.333 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 11.758932719000001 2024-04-25T13:42:22.334 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:22.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 -- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa24406f430 msgr2=0x7fa244071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa24406f430 0x7fa244071940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa2640b6e70 tx=0x7fa254008040 comp rx=0 tx=0).stop 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 -- 172.21.15.80:0/2905956591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 msgr2=0x7fa26415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa26415d5b0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa25800c310 tx=0x7fa25800c7d0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 -- 172.21.15.80:0/2905956591 shutdown_connections 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa24406f430 0x7fa244071940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa264161070 0x7fa2640af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa2640733b0 0x7fa26415d5b0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 --2- 172.21.15.80:0/2905956591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa26406f5a0 0x7fa26414c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 -- 172.21.15.80:0/2905956591 >> 172.21.15.80:0/2905956591 conn(0x7fa26406d080 msgr2=0x7fa2640d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 -- 172.21.15.80:0/2905956591 shutdown_connections 2024-04-25T13:42:22.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.325+0000 7fa268cf4640 1 -- 172.21.15.80:0/2905956591 wait complete. 2024-04-25T13:42:22.344 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:42:22.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b26d15640 1 Processor -- start 2024-04-25T13:42:22.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b26d15640 1 -- start start 2024-04-25T13:42:22.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b26d15640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b200d06e0 0x7f1b2006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b26d15640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1b2006e9e0 0x7f1b2006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b26d15640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b200d0af0 con 0x7f1b20161f90 2024-04-25T13:42:22.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b26d15640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1b200d0d70 con 0x7f1b2006e9e0 2024-04-25T13:42:22.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b26d15640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1b200d1050 con 0x7f1b200d06e0 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b200d06e0 0x7f1b2006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b252b2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1b2006e9e0 0x7f1b2006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b24ab1640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f1b20161f90 0x7f1b200cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:35906/0 (socket says 172.21.15.80:35906) 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b1ffff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b200d06e0 0x7f1b2006e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:39562/0 (socket says 172.21.15.80:39562) 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b252b2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1b2006e9e0 0x7f1b2006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:54398/0 (socket says 172.21.15.80:54398) 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.473+0000 7f1b24ab1640 1 -- 172.21.15.80:0/2318367636 learned_addr learned my addr 172.21.15.80:0/2318367636 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b1ffff640 1 -- 172.21.15.80:0/2318367636 >> v2:172.21.15.87:3300/0 conn(0x7f1b2006e9e0 msgr2=0x7f1b2006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b1ffff640 1 --2- 172.21.15.80:0/2318367636 >> v2:172.21.15.87:3300/0 conn(0x7f1b2006e9e0 0x7f1b2006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b1ffff640 1 -- 172.21.15.80:0/2318367636 >> v1:172.21.15.87:6789/0 conn(0x7f1b20161f90 legacy=0x7f1b200cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b1ffff640 1 -- 172.21.15.80:0/2318367636 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2006dca0 con 0x7f1b200d06e0 2024-04-25T13:42:22.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b252b2640 1 --2- 172.21.15.80:0/2318367636 >> v2:172.21.15.87:3300/0 conn(0x7f1b2006e9e0 0x7f1b2006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:22.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b1ffff640 1 --2- 172.21.15.80:0/2318367636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b200d06e0 0x7f1b2006e4a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b080097c0 tx=0x7f1b0801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=883d86be5dbe8436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b1f7fe640 1 -- 172.21.15.80:0/2318367636 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b080b0070 con 0x7f1b200d06e0 2024-04-25T13:42:22.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b1f7fe640 1 -- 172.21.15.80:0/2318367636 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b080ab600 con 0x7f1b200d06e0 2024-04-25T13:42:22.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b1f7fe640 1 -- 172.21.15.80:0/2318367636 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b080ab9d0 con 0x7f1b200d06e0 2024-04-25T13:42:22.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b26d15640 1 -- 172.21.15.80:0/2318367636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b200d06e0 msgr2=0x7f1b2006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b26d15640 1 --2- 172.21.15.80:0/2318367636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b200d06e0 0x7f1b2006e4a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f1b080097c0 tx=0x7f1b0801c3e0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b26d15640 1 -- 172.21.15.80:0/2318367636 shutdown_connections 2024-04-25T13:42:22.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b26d15640 1 --2- 172.21.15.80:0/2318367636 >> v2:172.21.15.87:3300/0 conn(0x7f1b2006e9e0 0x7f1b2006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b26d15640 1 --2- 172.21.15.80:0/2318367636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b200d06e0 0x7f1b2006e4a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b26d15640 1 -- 172.21.15.80:0/2318367636 >> 172.21.15.80:0/2318367636 conn(0x7f1b2006d080 msgr2=0x7f1b200b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:22.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b26d15640 1 -- 172.21.15.80:0/2318367636 shutdown_connections 2024-04-25T13:42:22.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.477+0000 7f1b26d15640 1 -- 172.21.15.80:0/2318367636 wait complete. 2024-04-25T13:42:22.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b26d15640 1 Processor -- start 2024-04-25T13:42:22.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b26d15640 1 -- start start 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b26d15640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2006e9e0 0x7f1b20146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b26d15640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b200d06e0 0x7f1b200abae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b26d15640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b20161f90 0x7f1b200ac020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b26d15640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1b201447a0 con 0x7f1b20161f90 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b26d15640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1b20144620 con 0x7f1b200d06e0 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b26d15640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1b2016b510 con 0x7f1b2006e9e0 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b252b2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b20161f90 0x7f1b200ac020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b252b2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b20161f90 0x7f1b200ac020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34988/0 (socket says 172.21.15.80:34988) 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.481+0000 7f1b252b2640 1 -- 172.21.15.80:0/595025325 learned_addr learned my addr 172.21.15.80:0/595025325 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b24ab1640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2006e9e0 0x7f1b20146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b1ffff640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b200d06e0 0x7f1b200abae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b252b2640 1 -- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2006e9e0 msgr2=0x7f1b20146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b252b2640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2006e9e0 0x7f1b20146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b252b2640 1 -- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b200d06e0 msgr2=0x7f1b200abae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b252b2640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b200d06e0 0x7f1b200abae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b252b2640 1 -- 172.21.15.80:0/595025325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b20145eb0 con 0x7f1b20161f90 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b252b2640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b20161f90 0x7f1b200ac020 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000efc0 tx=0x7f1b1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b1d7fa640 1 -- 172.21.15.80:0/595025325 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10018030 con 0x7f1b20161f90 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b1d7fa640 1 -- 172.21.15.80:0/595025325 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b1000ccd0 con 0x7f1b20161f90 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b20146190 con 0x7f1b20161f90 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b1d7fa640 1 -- 172.21.15.80:0/595025325 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b10003bc0 con 0x7f1b20161f90 2024-04-25T13:42:22.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b20146750 con 0x7f1b20161f90 2024-04-25T13:42:22.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2006a3a0 con 0x7f1b20161f90 2024-04-25T13:42:22.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b1d7fa640 1 -- 172.21.15.80:0/595025325 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1b1001b070 con 0x7f1b20161f90 2024-04-25T13:42:22.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b1d7fa640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1b0006f430 0x7f1b00071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:22.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b1d7fa640 1 -- 172.21.15.80:0/595025325 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f1b100901c0 con 0x7f1b20161f90 2024-04-25T13:42:22.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b24ab1640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1b0006f430 0x7f1b00071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:22.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.485+0000 7f1b24ab1640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1b0006f430 0x7f1b00071940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b18000f70 tx=0x7f1b18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:22.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.489+0000 7f1b1d7fa640 1 -- 172.21.15.80:0/595025325 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1b10010040 con 0x7f1b20161f90 2024-04-25T13:42:22.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.673+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1b201683e0 con 0x7f1b20161f90 2024-04-25T13:42:22.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.673+0000 7f1b1d7fa640 1 -- 172.21.15.80:0/595025325 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f1b1005d190 con 0x7f1b20161f90 2024-04-25T13:42:22.678 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:42:22.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1b0006f430 msgr2=0x7f1b00071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1b0006f430 0x7f1b00071940 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f1b18000f70 tx=0x7f1b18009040 comp rx=0 tx=0).stop 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b20161f90 msgr2=0x7f1b200ac020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b20161f90 0x7f1b200ac020 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1b1000efc0 tx=0x7f1b1000c490 comp rx=0 tx=0).stop 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 shutdown_connections 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1b0006f430 0x7f1b00071940 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1b20161f90 0x7f1b200ac020 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1b200d06e0 0x7f1b200abae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 --2- 172.21.15.80:0/595025325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1b2006e9e0 0x7f1b20146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 >> 172.21.15.80:0/595025325 conn(0x7f1b2006d080 msgr2=0x7f1b200cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 shutdown_connections 2024-04-25T13:42:22.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22.677+0000 7f1b26d15640 1 -- 172.21.15.80:0/595025325 wait complete. 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:42:22.698 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:42:22.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:22.699 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:22.705 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42426sec preferred_lft 42426sec 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:22.706 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:22.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:22.708 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:42:22.756 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:42:22.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:22.758 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:42:22.807 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:42:22.807 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:42:22.807 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:42:22.808 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:42:22.808 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:42:22.808 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:42:22.808 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:42:22.808 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:42:22.808 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:42:22.808 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:42:22.808 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:42:22.808 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:42:22.808 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:42:22.808 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:42:22.809 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:42:22.809 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:42:22.809 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:42:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:42:22.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:42:22.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:22.929 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:42:22.975 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:42:22.976 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:42:22.976 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:42:22.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:22.976 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:42:23.031 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:42:23.031 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:42:23.031 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:42:23.084 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:42:23.084 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:42:23.084 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:42:23.132 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:42:23.189 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:42:23.237 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:42:23.237 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:42:23.237 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:42:23.238 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:42:23.239 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:42:23.240 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:42:23.240 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:42:23.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:23.240 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:42:23.283 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:42:23.284 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:42:23.333 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:42:23.347 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:42:23.400 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:42:23.400 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:42:23.400 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:42:23.400 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:42:23.400 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:42:23.400 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:42:23.400 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:42:23.401 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:42:23.402 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:42:23.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:23.403 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:42:23.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:42:23.437+0000 7fc37a58a580 -1 init, newargv = 0x56422a78d320 newargc=15 2024-04-25T13:42:23.440 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[29981]: starting ceph client 2024-04-25T13:42:23.445 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[29981]: starting fuse 2024-04-25T13:42:24.546 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:42:24.562 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:42:24.609 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:42:24.610 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:42:24.611 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:42:24.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:24.612 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:42:24.660 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:42:24.660 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:42:24.660 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:42:24.660 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:42:24.660 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:42:24.660 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:42:24.660 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:42:24.660 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:42:24.661 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:42:24.662 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:42:24.662 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:42:24.662 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:42:24.662 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:42:24.662 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:42:24.785 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.29981.asok 2024-04-25T13:42:24.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:42:24.797 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.29981.asok 2024-04-25T13:42:24.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:24.797 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.29981.asok status 2024-04-25T13:42:24.976 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "29981", 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:42:24.977 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 5883, 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 5883 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1678523986 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:42:24.978 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1678523986 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.5883 192.168.0.1:0/1678523986", 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1678523986", 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 24, 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 47, 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:42:24.979 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:24.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:24.992 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:42:24.992 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:42:24.998 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:42:24.999 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:42:24.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:24.999 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:42:25.056 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:42:25.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff78ab1d640 1 Processor -- start 2024-04-25T13:42:25.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff78ab1d640 1 -- start start 2024-04-25T13:42:25.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff78ab1d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff78406cce0 0x7ff7840b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:25.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff78ab1d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff7840b4fc0 0x7ff78416a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:25.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff78ab1d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7840b53d0 con 0x7ff7840d3350 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff78ab1d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff7840b9680 con 0x7ff7840b4fc0 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff78ab1d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff7840d2ec0 con 0x7ff78406cce0 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7890ba640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff7840b4fc0 0x7ff78416a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7890ba640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff7840b4fc0 0x7ff78416a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:34992/0 (socket says 172.21.15.80:34992) 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7888b9640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff7840d3350 0x7ff7840d0520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:40452/0 (socket says 172.21.15.80:40452) 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7890ba640 1 -- 172.21.15.80:0/2613136380 learned_addr learned my addr 172.21.15.80:0/2613136380 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff783fff640 1 --2- 172.21.15.80:0/2613136380 >> v2:172.21.15.87:3300/0 conn(0x7ff78406cce0 0x7ff7840b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7890ba640 1 -- 172.21.15.80:0/2613136380 >> v2:172.21.15.87:3300/0 conn(0x7ff78406cce0 msgr2=0x7ff7840b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7890ba640 1 --2- 172.21.15.80:0/2613136380 >> v2:172.21.15.87:3300/0 conn(0x7ff78406cce0 0x7ff7840b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7890ba640 1 -- 172.21.15.80:0/2613136380 >> v1:172.21.15.80:6789/0 conn(0x7ff7840d3350 legacy=0x7ff7840d0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7837fe640 1 -- 172.21.15.80:0/2613136380 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 340649305 0 0) 0x7ff7840b53d0 con 0x7ff7840d3350 2024-04-25T13:42:25.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.249+0000 7ff7890ba640 1 -- 172.21.15.80:0/2613136380 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7840d22b0 con 0x7ff7840b4fc0 2024-04-25T13:42:25.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff7890ba640 1 --2- 172.21.15.80:0/2613136380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840b4fc0 0x7ff78416a4c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff7841640f0 tx=0x7ff77c01c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db33767432814c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:25.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff7837fe640 1 -- 172.21.15.80:0/2613136380 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0a5880 con 0x7ff7840b4fc0 2024-04-25T13:42:25.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff7837fe640 1 -- 172.21.15.80:0/2613136380 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77c0a5a20 con 0x7ff7840b4fc0 2024-04-25T13:42:25.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff7837fe640 1 -- 172.21.15.80:0/2613136380 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0a5df0 con 0x7ff7840b4fc0 2024-04-25T13:42:25.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2613136380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840b4fc0 msgr2=0x7ff78416a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:25.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2613136380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840b4fc0 0x7ff78416a4c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff7841640f0 tx=0x7ff77c01c9c0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2613136380 shutdown_connections 2024-04-25T13:42:25.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2613136380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840b4fc0 0x7ff78416a4c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2613136380 >> v2:172.21.15.87:3300/0 conn(0x7ff78406cce0 0x7ff7840b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2613136380 >> 172.21.15.80:0/2613136380 conn(0x7ff7840d3ad0 msgr2=0x7ff7840d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:25.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2613136380 shutdown_connections 2024-04-25T13:42:25.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.253+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2613136380 wait complete. 2024-04-25T13:42:25.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 Processor -- start 2024-04-25T13:42:25.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 -- start start 2024-04-25T13:42:25.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78406cce0 0x7ff7841582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:25.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7840b4fc0 0x7ff784159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:25.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840d3350 0x7ff78415a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:25.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff7840d1ed0 con 0x7ff7840d3350 2024-04-25T13:42:25.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff7840d0c30 con 0x7ff78406cce0 2024-04-25T13:42:25.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff7840d2050 con 0x7ff7840b4fc0 2024-04-25T13:42:25.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840d3350 0x7ff78415a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:25.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840d3350 0x7ff78415a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35006/0 (socket says 172.21.15.80:35006) 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 -- 172.21.15.80:0/2324966345 learned_addr learned my addr 172.21.15.80:0/2324966345 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7888b9640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78406cce0 0x7ff7841582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff783fff640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7840b4fc0 0x7ff784159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 -- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7840b4fc0 msgr2=0x7ff784159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7840b4fc0 0x7ff784159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 -- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78406cce0 msgr2=0x7ff7841582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78406cce0 0x7ff7841582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 -- 172.21.15.80:0/2324966345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7841545c0 con 0x7ff7840d3350 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff783fff640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7840b4fc0 0x7ff784159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7890ba640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840d3350 0x7ff78415a200 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0195d0 tx=0x7ff77c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7817fa640 1 -- 172.21.15.80:0/2324966345 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0b8670 con 0x7ff7840d3350 2024-04-25T13:42:25.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7817fa640 1 -- 172.21.15.80:0/2324966345 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77c0be070 con 0x7ff7840d3350 2024-04-25T13:42:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7841548a0 con 0x7ff7840d3350 2024-04-25T13:42:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff7817fa640 1 -- 172.21.15.80:0/2324966345 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77c0b8a90 con 0x7ff7840d3350 2024-04-25T13:42:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.257+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff784154e60 con 0x7ff7840d3350 2024-04-25T13:42:25.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.261+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff784154320 con 0x7ff7840d3350 2024-04-25T13:42:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.261+0000 7ff7817fa640 1 -- 172.21.15.80:0/2324966345 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff77c0b8c30 con 0x7ff7840d3350 2024-04-25T13:42:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.261+0000 7ff7817fa640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff77006f400 0x7ff770071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.261+0000 7ff7817fa640 1 -- 172.21.15.80:0/2324966345 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7ff77c12a530 con 0x7ff7840d3350 2024-04-25T13:42:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.261+0000 7ff7888b9640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff77006f400 0x7ff770071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:25.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.261+0000 7ff7888b9640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff77006f400 0x7ff770071910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff778000f70 tx=0x7ff778007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:25.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.265+0000 7ff7817fa640 1 -- 172.21.15.80:0/2324966345 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff784154320 con 0x7ff7840d3350 2024-04-25T13:42:25.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.453+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff78406a3a0 con 0x7ff7840d3350 2024-04-25T13:42:25.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff7817fa640 1 -- 172.21.15.80:0/2324966345 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7ff77c0165e0 con 0x7ff7840d3350 2024-04-25T13:42:25.459 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:42:25.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff77006f400 msgr2=0x7ff770071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:25.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff77006f400 0x7ff770071910 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff778000f70 tx=0x7ff778007c10 comp rx=0 tx=0).stop 2024-04-25T13:42:25.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840d3350 msgr2=0x7ff78415a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840d3350 0x7ff78415a200 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff77c0195d0 tx=0x7ff77c019760 comp rx=0 tx=0).stop 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 shutdown_connections 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff77006f400 0x7ff770071910 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff7840d3350 0x7ff78415a200 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff7840b4fc0 0x7ff784159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 --2- 172.21.15.80:0/2324966345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff78406cce0 0x7ff7841582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 >> 172.21.15.80:0/2324966345 conn(0x7ff7840d3ad0 msgr2=0x7ff7840b7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 shutdown_connections 2024-04-25T13:42:25.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:25.457+0000 7ff78ab1d640 1 -- 172.21.15.80:0/2324966345 wait complete. 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:42:25.480 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:42:25.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:25.481 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:42:25.488 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:42:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:42:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:42:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:25.488 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:42:25.489 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42423sec preferred_lft 42423sec 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:25.490 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:42:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:42:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:42:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:25.491 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:42:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:42:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:42:25.491 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:42:25.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:25.491 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:42:25.542 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:42:25.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:25.544 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:42:25.595 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:42:25.595 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:42:25.596 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:42:25.596 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:42:25.596 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:42:25.597 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:42:25.597 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:42:25.597 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:42:25.597 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:42:25.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:42:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:42:25.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:42:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:42:25.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:25.720 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:42:25.725 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:42:25.726 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:42:25.726 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:42:25.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:25.726 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:42:25.778 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:42:25.778 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:42:25.778 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:42:25.827 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:42:25.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:42:25.827 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:42:25.875 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:42:25.932 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:42:25.982 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:42:25.982 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:42:25.982 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:42:25.982 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:42:25.982 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:42:25.982 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:42:25.983 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:42:25.983 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:42:25.983 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:42:25.983 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:42:25.983 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:42:25.983 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:42:25.983 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:42:25.984 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:42:25.984 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:42:25.984 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:42:25.984 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:42:25.984 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:42:25.984 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:42:25.984 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:42:25.985 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:42:25.985 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:42:25.985 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:42:25.985 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:42:25.985 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:42:25.985 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:42:25.986 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:42:25.986 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:42:25.986 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:42:25.986 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:42:25.986 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:42:25.986 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:42:25.987 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:42:25.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:25.988 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:42:26.036 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:42:26.036 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:42:26.080 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:42:26.095 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:42:26.148 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:42:26.148 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:42:26.148 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:42:26.148 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:42:26.148 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:42:26.148 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:42:26.148 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:42:26.148 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:42:26.149 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:42:26.149 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:42:26.149 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:42:26.149 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:42:26.149 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:42:26.149 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:42:26.150 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:42:26.150 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:42:26.150 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:42:26.150 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:42:26.150 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:42:26.150 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:42:26.151 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:42:26.151 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:42:26.151 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:42:26.151 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:42:26.151 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:42:26.152 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:42:26.152 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:42:26.152 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:42:26.152 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:42:26.152 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:42:26.152 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:42:26.152 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:42:26.153 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:42:26.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:26.153 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:42:26.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:42:26.180+0000 7fb5d54e7580 -1 init, newargv = 0x7fb5c0001090 newargc=15 2024-04-25T13:42:26.185 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24433]: starting ceph client 2024-04-25T13:42:26.191 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24433]: starting fuse 2024-04-25T13:42:27.290 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:42:27.306 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:42:27.357 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:42:27.358 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:42:27.359 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:42:27.360 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:42:27.360 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:42:27.360 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:42:27.360 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:42:27.360 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:42:27.360 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:42:27.360 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:42:27.361 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:42:27.361 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:42:27.361 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:42:27.361 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:42:27.361 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:42:27.361 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:42:27.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:27.362 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:42:27.403 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:42:27.403 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:42:27.404 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:42:27.405 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:42:27.532 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.24433.asok 2024-04-25T13:42:27.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:42:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:42:27.544 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.24433.asok 2024-04-25T13:42:27.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:27.544 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.24433.asok status 2024-04-25T13:42:27.727 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:42:27.727 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "24433", 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 4982, 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:42:27.728 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 4982 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2274547422 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2274547422 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:42:27.729 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.4982 192.168.0.1:0/2274547422", 2024-04-25T13:42:27.730 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/2274547422", 2024-04-25T13:42:27.730 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:42:27.730 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 24, 2024-04-25T13:42:27.730 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 47, 2024-04-25T13:42:27.730 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:42:27.730 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:42:27.730 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:42:27.730 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:42:27.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:27.743 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:42:27.743 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:42:27.750 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:42:27.750 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:42:27.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:27.750 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:42:27.806 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:27.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff936541640 1 Processor -- start 2024-04-25T13:42:27.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff936541640 1 -- start start 2024-04-25T13:42:27.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff936541640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff930161f90 0x7ff9300cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff936541640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 0x7ff93015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff936541640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9300d0d70 con 0x7ff93015ea70 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff936541640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff9300d1050 con 0x7ff930161f90 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff936541640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff93006dca0 con 0x7ff9300d0510 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff934ade640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff93015ea70 0x7ff93015ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:40460/0 (socket says 172.21.15.80:40460) 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff934ade640 1 -- 172.21.15.80:0/1443443054 learned_addr learned my addr 172.21.15.80:0/1443443054 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff92ffff640 1 --2- 172.21.15.80:0/1443443054 >> v2:172.21.15.87:3300/0 conn(0x7ff930161f90 0x7ff9300cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff92f7fe640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 0x7ff93015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162146920 0 0) 0x7ff9300d0d70 con 0x7ff93015ea70 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.961+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff91c003530 con 0x7ff93015ea70 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2706129699 0 0) 0x7ff91c003530 con 0x7ff93015ea70 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 msgr2=0x7ff93015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 0x7ff93015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 >> v2:172.21.15.87:3300/0 conn(0x7ff930161f90 msgr2=0x7ff9300cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:27.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 --2- 172.21.15.80:0/1443443054 >> v2:172.21.15.87:3300/0 conn(0x7ff930161f90 0x7ff9300cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93006df80 con 0x7ff93015ea70 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff928001a00 con 0x7ff93015ea70 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 >> v1:172.21.15.80:6789/0 conn(0x7ff93015ea70 legacy=0x7ff93015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff91c0039c0 0x7ff91c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c004810 0x7ff91c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 0x7ff91c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9300d1050 con 0x7ff91c0039c0 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff93006dca0 con 0x7ff91c004810 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff91c00ba80 con 0x7ff9300d0510 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92ffff640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff91c0039c0 0x7ff91c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff934ade640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 0x7ff91c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92f7fe640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c004810 0x7ff91c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92ffff640 1 -- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 msgr2=0x7ff91c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92ffff640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 0x7ff91c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92ffff640 1 -- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c004810 msgr2=0x7ff91c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92ffff640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c004810 0x7ff91c004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92ffff640 1 -- 172.21.15.80:0/1443443054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91c003530 con 0x7ff91c0039c0 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff934ade640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9300d0510 0x7ff91c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92ffff640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff91c0039c0 0x7ff91c003e30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff930147e20 tx=0x7ff91801c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19eaf499b915dec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9180096b0 con 0x7ff91c0039c0 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9180a64e0 con 0x7ff91c0039c0 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff92effd640 1 -- 172.21.15.80:0/1443443054 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9180a6890 con 0x7ff91c0039c0 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff936541640 1 -- 172.21.15.80:0/1443443054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff91c0039c0 msgr2=0x7ff91c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff936541640 1 --2- 172.21.15.80:0/1443443054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff91c0039c0 0x7ff91c003e30 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff930147e20 tx=0x7ff91801c9f0 comp rx=0 tx=0).stop 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff934ade640 1 -- 172.21.15.80:0/1443443054 reap_dead start 2024-04-25T13:42:27.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff936541640 1 -- 172.21.15.80:0/1443443054 shutdown_connections 2024-04-25T13:42:27.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff936541640 1 -- 172.21.15.80:0/1443443054 >> 172.21.15.80:0/1443443054 conn(0x7ff93006d080 msgr2=0x7ff9300b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:27.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.965+0000 7ff936541640 1 -- 172.21.15.80:0/1443443054 shutdown_connections 2024-04-25T13:42:27.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 -- 172.21.15.80:0/1443443054 wait complete. 2024-04-25T13:42:27.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 Processor -- start 2024-04-25T13:42:27.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 -- start start 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c0039c0 0x7ff93015d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff930143180 0x7ff9300b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93015d810 0x7ff9300b5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff93006f4b0 con 0x7ff930143180 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff93006f630 con 0x7ff91c0039c0 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff936541640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9300d0920 con 0x7ff93015d810 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff930143180 0x7ff9300b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff930143180 0x7ff9300b4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35030/0 (socket says 172.21.15.80:35030) 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 -- 172.21.15.80:0/2956316407 learned_addr learned my addr 172.21.15.80:0/2956316407 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92ffff640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c0039c0 0x7ff93015d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff934ade640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93015d810 0x7ff9300b5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 -- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93015d810 msgr2=0x7ff9300b5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93015d810 0x7ff9300b5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:27.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 -- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c0039c0 msgr2=0x7ff93015d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c0039c0 0x7ff93015d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 -- 172.21.15.80:0/2956316407 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93013fa00 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92ffff640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c0039c0 0x7ff93015d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.969+0000 7ff92f7fe640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff930143180 0x7ff9300b4de0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff9200108f0 tx=0x7ff920010db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff92cff9640 1 -- 172.21.15.80:0/2956316407 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92000f040 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff92cff9640 1 -- 172.21.15.80:0/2956316407 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92000eb60 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93013fce0 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff92cff9640 1 -- 172.21.15.80:0/2956316407 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92001c400 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9300b5860 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff930141070 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff92cff9640 1 -- 172.21.15.80:0/2956316407 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff92001c5a0 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff92cff9640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91006f430 0x7ff910071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff92cff9640 1 -- 172.21.15.80:0/2956316407 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7ff92008f980 con 0x7ff930143180 2024-04-25T13:42:27.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff92ffff640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91006f430 0x7ff910071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:27.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.973+0000 7ff92ffff640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91006f430 0x7ff910071940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff930147e20 tx=0x7ff9180046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:27.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:27.977+0000 7ff92cff9640 1 -- 172.21.15.80:0/2956316407 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff920018030 con 0x7ff930143180 2024-04-25T13:42:28.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.177+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9300b6780 con 0x7ff930143180 2024-04-25T13:42:28.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.177+0000 7ff92cff9640 1 -- 172.21.15.80:0/2956316407 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7ff92005c870 con 0x7ff930143180 2024-04-25T13:42:28.181 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:28.181 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:42:28.181 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:42:28.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91006f430 msgr2=0x7ff910071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91006f430 0x7ff910071940 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7ff930147e20 tx=0x7ff9180046d0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff930143180 msgr2=0x7ff9300b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff930143180 0x7ff9300b4de0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff9200108f0 tx=0x7ff920010db0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 shutdown_connections 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91006f430 0x7ff910071940 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93015d810 0x7ff9300b5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff930143180 0x7ff9300b4de0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 --2- 172.21.15.80:0/2956316407 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff91c0039c0 0x7ff93015d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 >> 172.21.15.80:0/2956316407 conn(0x7ff93006d080 msgr2=0x7ff93015ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 shutdown_connections 2024-04-25T13:42:28.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:28.181+0000 7ff936541640 1 -- 172.21.15.80:0/2956316407 wait complete. 2024-04-25T13:42:28.202 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-04-25T13:42:28.355 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":5730,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.918662984999999,"mdsmap_epoch":24,"osdmap_epoch":47,"osdmap_epoch_barrier":47,"uptime":17.772008197000002} 2024-04-25T13:42:28.356 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 5730, "mdsmap_epoch": 24, "osdmap_epoch": 47, "osdmap_epoch_barrier": 47, "rank_uptime": 11.918662985, "state": "up:active", "uptime": 17.772008197, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:42:28.356 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-25T13:42:28.521 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_mon_report_interval":"5"} 2024-04-25T13:42:28.522 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-25T13:42:28.522 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-04-25T13:42:28.522 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:28.522 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:28.526 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:42:28.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:28.529 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-04-25T13:42:28.572 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:42:31.172 INFO:teuthology.orchestra.run.smithi080.stderr:265+0 records in 2024-04-25T13:42:31.173 INFO:teuthology.orchestra.run.smithi080.stderr:265+0 records out 2024-04-25T13:42:31.173 INFO:teuthology.orchestra.run.smithi080.stderr:277872640 bytes (278 MB, 265 MiB) copied, 2.58925 s, 107 MB/s 2024-04-25T13:42:31.174 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:31.174 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:31.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:31.180 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-04-25T13:42:33.842 INFO:teuthology.orchestra.run.smithi080.stderr:292+0 records in 2024-04-25T13:42:33.842 INFO:teuthology.orchestra.run.smithi080.stderr:292+0 records out 2024-04-25T13:42:33.843 INFO:teuthology.orchestra.run.smithi080.stderr:306184192 bytes (306 MB, 292 MiB) copied, 2.61107 s, 117 MB/s 2024-04-25T13:42:33.844 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-04-25T13:42:33.844 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:42:33.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893ea21640 1 Processor -- start 2024-04-25T13:42:33.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893ea21640 1 -- start start 2024-04-25T13:42:33.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893ea21640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8938161f90 0x7f89380cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893ea21640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893ea21640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89380d0d70 con 0x7f893815ea70 2024-04-25T13:42:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893ea21640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f89380d1050 con 0x7f8938161f90 2024-04-25T13:42:33.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893ea21640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f893806dca0 con 0x7f89380d0510 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f89377fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f8937fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8938161f90 0x7f89380cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893cfbe640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f893815ea70 0x7f893815ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:33608/0 (socket says 172.21.15.80:33608) 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f893cfbe640 1 -- 172.21.15.80:0/2840713452 learned_addr learned my addr 172.21.15.80:0/2840713452 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f8937fff640 1 -- 172.21.15.80:0/2840713452 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 msgr2=0x7f893815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f8937fff640 1 --2- 172.21.15.80:0/2840713452 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f8937fff640 1 -- 172.21.15.80:0/2840713452 >> v1:172.21.15.87:6789/0 conn(0x7f893815ea70 legacy=0x7f893815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f8937fff640 1 -- 172.21.15.80:0/2840713452 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893806df80 con 0x7f8938161f90 2024-04-25T13:42:33.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.993+0000 7f89377fe640 1 --2- 172.21.15.80:0/2840713452 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:42:34.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f8937fff640 1 --2- 172.21.15.80:0/2840713452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8938161f90 0x7f89380cffd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f89280097c0 tx=0x7f892801c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6317c50bceb585b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:34.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f8936ffd640 1 -- 172.21.15.80:0/2840713452 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89280b0070 con 0x7f8938161f90 2024-04-25T13:42:34.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f8936ffd640 1 -- 172.21.15.80:0/2840713452 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89280ab600 con 0x7f8938161f90 2024-04-25T13:42:34.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f8936ffd640 1 -- 172.21.15.80:0/2840713452 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89280ab9d0 con 0x7f8938161f90 2024-04-25T13:42:34.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f893ea21640 1 -- 172.21.15.80:0/2840713452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8938161f90 msgr2=0x7f89380cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:34.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f893ea21640 1 --2- 172.21.15.80:0/2840713452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8938161f90 0x7f89380cffd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f89280097c0 tx=0x7f892801c3b0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f893ea21640 1 -- 172.21.15.80:0/2840713452 shutdown_connections 2024-04-25T13:42:34.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f893ea21640 1 --2- 172.21.15.80:0/2840713452 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f893ea21640 1 --2- 172.21.15.80:0/2840713452 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8938161f90 0x7f89380cffd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f893ea21640 1 -- 172.21.15.80:0/2840713452 >> 172.21.15.80:0/2840713452 conn(0x7f893806d080 msgr2=0x7f89380b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:34.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f893ea21640 1 -- 172.21.15.80:0/2840713452 shutdown_connections 2024-04-25T13:42:34.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:33.997+0000 7f893ea21640 1 -- 172.21.15.80:0/2840713452 wait complete. 2024-04-25T13:42:34.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 Processor -- start 2024-04-25T13:42:34.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 -- start start 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893813fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815ea70 0x7f893815f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8938161f90 0x7f893815fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f893806f4b0 con 0x7f8938161f90 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f893806f630 con 0x7f893815ea70 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89380d0920 con 0x7f89380d0510 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8938161f90 0x7f893815fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8938161f90 0x7f893815fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54058/0 (socket says 172.21.15.80:54058) 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 -- 172.21.15.80:0/4022210684 learned_addr learned my addr 172.21.15.80:0/4022210684 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f8937fff640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893813fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:34.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f89377fe640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815ea70 0x7f893815f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:34.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 -- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 msgr2=0x7f893813fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:34.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893813fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 -- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815ea70 msgr2=0x7f893815f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:34.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815ea70 0x7f893815f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 -- 172.21.15.80:0/4022210684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893815ba30 con 0x7f8938161f90 2024-04-25T13:42:34.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f8937fff640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893813fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:34.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893cfbe640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8938161f90 0x7f893815fc60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f893000cad0 tx=0x7f893000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:34.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f8934ff9640 1 -- 172.21.15.80:0/4022210684 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8930017070 con 0x7f8938161f90 2024-04-25T13:42:34.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f8934ff9640 1 -- 172.21.15.80:0/4022210684 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f893000ab70 con 0x7f8938161f90 2024-04-25T13:42:34.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8938159650 con 0x7f8938161f90 2024-04-25T13:42:34.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f8934ff9640 1 -- 172.21.15.80:0/4022210684 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8930012400 con 0x7f8938161f90 2024-04-25T13:42:34.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.001+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8938159c10 con 0x7f8938161f90 2024-04-25T13:42:34.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.005+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893815b7a0 con 0x7f8938161f90 2024-04-25T13:42:34.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.005+0000 7f8934ff9640 1 -- 172.21.15.80:0/4022210684 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f893001a070 con 0x7f8938161f90 2024-04-25T13:42:34.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.005+0000 7f8934ff9640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892406f400 0x7f8924071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:34.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.005+0000 7f8934ff9640 1 -- 172.21.15.80:0/4022210684 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f893008f5d0 con 0x7f8938161f90 2024-04-25T13:42:34.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.005+0000 7f8937fff640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892406f400 0x7f8924071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:34.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.005+0000 7f8937fff640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892406f400 0x7f8924071910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f89280ad000 tx=0x7f89280a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:34.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.009+0000 7f8934ff9640 1 -- 172.21.15.80:0/4022210684 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8930021070 con 0x7f8938161f90 2024-04-25T13:42:34.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.169+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f893806a3a0 con 0x7f8938161f90 2024-04-25T13:42:34.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f8934ff9640 1 -- 172.21.15.80:0/4022210684 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v47) ==== 81+0+34030 (secure 0 0 0) 0x7f893005c5a0 con 0x7f8938161f90 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 47, 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:42:17.378064+0000", 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:42:34.177 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 7, 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:42:34.178 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:34.179 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:34.180 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "16", 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:34.181 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:34.182 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:34.183 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:34.184 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 6, 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:13.650442+0000", 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:34.185 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:42:34.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "46", 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:34.187 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:34.188 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:34.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 7, 2024-04-25T13:42:34.191 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:14.725209+0000", 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:34.192 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:34.193 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "47", 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:34.194 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:34.195 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:42:34.196 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:34.197 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:34.198 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.199 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.200 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.201 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.202 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:34.203 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:42:34.204 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:42:34.205 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:34.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:42:34.207 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:34.208 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:34.209 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:42:34.210 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:34.213 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.213 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.213 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.215 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.216 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:34.217 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:42:34.218 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:34.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:42:34.226 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:34.227 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:42:34.232 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:42:34.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:42:34.235 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:42:34.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:42:34.237 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.238 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:42:34.240 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:42:34.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:42:34.242 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:34.243 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:34.245 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:34.246 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:34.247 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:34.248 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:42:34.249 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:42:34.250 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:34.251 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:34.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892406f400 msgr2=0x7f8924071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:34.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892406f400 0x7f8924071910 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f89280ad000 tx=0x7f89280a6000 comp rx=0 tx=0).stop 2024-04-25T13:42:34.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8938161f90 msgr2=0x7f893815fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:34.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8938161f90 0x7f893815fc60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f893000cad0 tx=0x7f893000cf90 comp rx=0 tx=0).stop 2024-04-25T13:42:34.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 shutdown_connections 2024-04-25T13:42:34.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892406f400 0x7f8924071910 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8938161f90 0x7f893815fc60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893815ea70 0x7f893815f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 --2- 172.21.15.80:0/4022210684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89380d0510 0x7f893813fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:34.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.173+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 >> 172.21.15.80:0/4022210684 conn(0x7f893806d080 msgr2=0x7f89380b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:34.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.177+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 shutdown_connections 2024-04-25T13:42:34.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:34.177+0000 7f893ea21640 1 -- 172.21.15.80:0/4022210684 wait complete. 2024-04-25T13:42:34.254 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:42:35.503 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:42:35.497+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-04-25T13:42:39.255 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:42:39.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.413+0000 7f136677c640 1 Processor -- start 2024-04-25T13:42:39.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.413+0000 7f136677c640 1 -- start start 2024-04-25T13:42:39.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f136677c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13600b4950 0x7f13601603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f136677c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f13600d2b10 0x7f13600d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f136677c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1360164270 con 0x7f13600b93e0 2024-04-25T13:42:39.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f136677c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1360164e80 con 0x7f13600d2b10 2024-04-25T13:42:39.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f136677c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1360164550 con 0x7f13600b4950 2024-04-25T13:42:39.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f1364d19640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f13600d2b10 0x7f13600d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f135f7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f13600b93e0 0x7f13600d2400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52206/0 (socket says 172.21.15.80:52206) 2024-04-25T13:42:39.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f135f7fe640 1 -- 172.21.15.80:0/2760058903 learned_addr learned my addr 172.21.15.80:0/2760058903 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:39.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f135ffff640 1 --2- 172.21.15.80:0/2760058903 >> v2:172.21.15.87:3300/0 conn(0x7f13600b4950 0x7f13601603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f1364d19640 1 -- 172.21.15.80:0/2760058903 >> v2:172.21.15.87:3300/0 conn(0x7f13600b4950 msgr2=0x7f13601603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:39.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f1364d19640 1 --2- 172.21.15.80:0/2760058903 >> v2:172.21.15.87:3300/0 conn(0x7f13600b4950 0x7f13601603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f1364d19640 1 -- 172.21.15.80:0/2760058903 >> v1:172.21.15.80:6789/0 conn(0x7f13600b93e0 legacy=0x7f13600d2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f1364d19640 1 -- 172.21.15.80:0/2760058903 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1360164830 con 0x7f13600d2b10 2024-04-25T13:42:39.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.417+0000 7f135effd640 1 -- 172.21.15.80:0/2760058903 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4020518323 0 0) 0x7f1360164270 con 0x7f13600b93e0 2024-04-25T13:42:39.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f1364d19640 1 --2- 172.21.15.80:0/2760058903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f13600d4cd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f136006dcc0 tx=0x7f135801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5025b097f4442cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:39.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f135effd640 1 -- 172.21.15.80:0/2760058903 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580b2070 con 0x7f13600d2b10 2024-04-25T13:42:39.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f135effd640 1 -- 172.21.15.80:0/2760058903 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13580a5900 con 0x7f13600d2b10 2024-04-25T13:42:39.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f135effd640 1 -- 172.21.15.80:0/2760058903 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580a5cd0 con 0x7f13600d2b10 2024-04-25T13:42:39.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 -- 172.21.15.80:0/2760058903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 msgr2=0x7f13600d4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 --2- 172.21.15.80:0/2760058903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f13600d4cd0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f136006dcc0 tx=0x7f135801c930 comp rx=0 tx=0).stop 2024-04-25T13:42:39.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 -- 172.21.15.80:0/2760058903 shutdown_connections 2024-04-25T13:42:39.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 --2- 172.21.15.80:0/2760058903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f13600d4cd0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 --2- 172.21.15.80:0/2760058903 >> v2:172.21.15.87:3300/0 conn(0x7f13600b4950 0x7f13601603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 -- 172.21.15.80:0/2760058903 >> 172.21.15.80:0/2760058903 conn(0x7f136006c5f0 msgr2=0x7f136006ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:39.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 -- 172.21.15.80:0/2760058903 shutdown_connections 2024-04-25T13:42:39.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 -- 172.21.15.80:0/2760058903 wait complete. 2024-04-25T13:42:39.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 Processor -- start 2024-04-25T13:42:39.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.421+0000 7f136677c640 1 -- start start 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13600b4950 0x7f136015b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13600b93e0 0x7f136015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f136015a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f136016d420 con 0x7f13600d2b10 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f136016d2a0 con 0x7f13600b4950 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f136016d5a0 con 0x7f13600b93e0 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f136015a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f136015a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54070/0 (socket says 172.21.15.80:54070) 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 -- 172.21.15.80:0/3556584415 learned_addr learned my addr 172.21.15.80:0/3556584415 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:39.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f135ffff640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13600b4950 0x7f136015b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f135f7fe640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13600b93e0 0x7f136015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 -- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13600b93e0 msgr2=0x7f136015b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13600b93e0 0x7f136015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 -- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13600b4950 msgr2=0x7f136015b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13600b4950 0x7f136015b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 -- 172.21.15.80:0/3556584415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136015e9b0 con 0x7f13600d2b10 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f1364d19640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f136015a310 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f135801a560 tx=0x7f13580196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f135cff9640 1 -- 172.21.15.80:0/3556584415 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580b2050 con 0x7f13600d2b10 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f135cff9640 1 -- 172.21.15.80:0/3556584415 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13580be070 con 0x7f13600d2b10 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136015ec90 con 0x7f13600d2b10 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f135cff9640 1 -- 172.21.15.80:0/3556584415 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580ada70 con 0x7f13600d2b10 2024-04-25T13:42:39.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1360160d00 con 0x7f13600d2b10 2024-04-25T13:42:39.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136015e720 con 0x7f13600d2b10 2024-04-25T13:42:39.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.425+0000 7f135cff9640 1 -- 172.21.15.80:0/3556584415 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13580adc10 con 0x7f13600d2b10 2024-04-25T13:42:39.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.429+0000 7f135cff9640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134c06f400 0x7f134c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.429+0000 7f135cff9640 1 -- 172.21.15.80:0/3556584415 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f135812a3f0 con 0x7f13600d2b10 2024-04-25T13:42:39.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.429+0000 7f135cff9640 1 -- 172.21.15.80:0/3556584415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7f134c076fe0 con 0x7f13600d2b10 2024-04-25T13:42:39.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.429+0000 7f135ffff640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134c06f400 0x7f134c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.429+0000 7f135ffff640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134c06f400 0x7f134c071910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f135400b930 tx=0x7f1354008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:39.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.433+0000 7f135cff9640 1 -- 172.21.15.80:0/3556584415 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1358016520 con 0x7f13600d2b10 2024-04-25T13:42:39.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.597+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f13600d42c0 con 0x7f13600d2b10 2024-04-25T13:42:39.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f135cff9640 1 -- 172.21.15.80:0/3556584415 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) ==== 81+0+34173 (secure 0 0 0) 0x7f13580f72e0 con 0x7f13600d2b10 2024-04-25T13:42:39.604 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:39.604 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:39.604 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 50, 2024-04-25T13:42:39.604 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:42:39.604 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:42:39.604 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:42:36.507236+0000", 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:42:39.605 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 7, 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:39.606 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:42:39.607 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:39.608 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:39.609 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:39.610 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:39.611 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 6, 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:13.650442+0000", 2024-04-25T13:42:39.612 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:42:39.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:39.614 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:39.615 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:39.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.617 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 7, 2024-04-25T13:42:39.618 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:14.725209+0000", 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:39.619 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:39.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:39.621 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:39.622 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:42:39.623 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.624 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:39.625 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.626 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.627 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.628 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:42:39.629 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:39.630 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:39.631 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:39.636 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:39.637 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:42:39.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.640 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:39.641 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:39.642 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.644 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.645 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.646 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull", 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:39.647 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.650 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:42:39.652 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:39.653 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:42:39.656 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.658 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.661 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:39.663 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:39.664 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:39.665 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:39.666 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.667 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.668 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:42:39.669 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:39.670 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:39.671 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:39.672 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:39.673 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:39.674 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:42:39.675 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:42:39.676 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:42:39.677 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:42:39.677 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:42:39.677 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:42:39.677 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:42:39.677 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:42:39.677 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:39.677 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:39.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134c06f400 msgr2=0x7f134c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134c06f400 0x7f134c071910 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f135400b930 tx=0x7f1354008040 comp rx=0 tx=0).stop 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 msgr2=0x7f136015a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f136015a310 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f135801a560 tx=0x7f13580196d0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 shutdown_connections 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134c06f400 0x7f134c071910 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13600d2b10 0x7f136015a310 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13600b93e0 0x7f136015b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 --2- 172.21.15.80:0/3556584415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13600b4950 0x7f136015b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 >> 172.21.15.80:0/3556584415 conn(0x7f136006c5f0 msgr2=0x7f13600b5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 shutdown_connections 2024-04-25T13:42:39.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.601+0000 7f136677c640 1 -- 172.21.15.80:0/3556584415 wait complete. 2024-04-25T13:42:39.680 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2024-04-25T13:42:39.680 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:39.680 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:39.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:39.686 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-04-25T13:42:39.821 INFO:teuthology.orchestra.run.smithi080.stderr:dd: error writing 'large_file_b': No space left on device 2024-04-25T13:42:39.821 INFO:teuthology.orchestra.run.smithi080.stderr:dd: closing output file 'large_file_b': No space left on device 2024-04-25T13:42:39.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:42:39.822 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:42:39.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f3034b86640 1 Processor -- start 2024-04-25T13:42:39.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f3034b86640 1 -- start start 2024-04-25T13:42:39.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f3034b86640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30300d4330 0x7f30300d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f3034b86640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3030161060 con 0x7f30300d59b0 2024-04-25T13:42:39.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f3034b86640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3030161340 con 0x7f30300d2e80 2024-04-25T13:42:39.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f3034b86640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f30300d2680 con 0x7f30300d4330 2024-04-25T13:42:39.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302dd9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f30300d59b0 0x7f30300d5dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52210/0 (socket says 172.21.15.80:52210) 2024-04-25T13:42:39.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302dd9b640 1 -- 172.21.15.80:0/4006456756 learned_addr learned my addr 172.21.15.80:0/4006456756 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:39.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302ed9d640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30300d4330 0x7f30300d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3679320456 0 0) 0x7f3030161060 con 0x7f30300d59b0 2024-04-25T13:42:39.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3018003530 con 0x7f30300d59b0 2024-04-25T13:42:39.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2047625515 0 0) 0x7f3018003530 con 0x7f30300d59b0 2024-04-25T13:42:39.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30300d4330 msgr2=0x7f30300d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:39.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30300d4330 0x7f30300d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 >> v1:172.21.15.87:6789/0 conn(0x7f30300d2e80 legacy=0x7f30301607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30300d2960 con 0x7f30300d59b0 2024-04-25T13:42:39.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f3020001680 con 0x7f30300d59b0 2024-04-25T13:42:39.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 >> v1:172.21.15.80:6789/0 conn(0x7f30300d59b0 legacy=0x7f30300d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30180039f0 0x7f3018003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30300d4330 0x7f3018004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f3018005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f30300d2680 con 0x7f3018004d50 2024-04-25T13:42:39.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f301800bb90 con 0x7f30180039f0 2024-04-25T13:42:39.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f301800bd90 con 0x7f30300d4330 2024-04-25T13:42:39.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302ed9d640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f3018005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302dd9b640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30300d4330 0x7f3018004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302e59c640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30180039f0 0x7f3018003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302ed9d640 1 -- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30300d4330 msgr2=0x7f3018004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:39.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302ed9d640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30300d4330 0x7f3018004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302ed9d640 1 -- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30180039f0 msgr2=0x7f3018003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:39.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302ed9d640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30180039f0 0x7f3018003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302ed9d640 1 -- 172.21.15.80:0/4006456756 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3018003530 con 0x7f3018004d50 2024-04-25T13:42:39.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.973+0000 7f302e59c640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30180039f0 0x7f3018003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:39.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f302ed9d640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f3018005160 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3028009f90 tx=0x7f302801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1242307c25f1cc39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:39.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30280b2070 con 0x7f3018004d50 2024-04-25T13:42:39.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f30280a5b60 con 0x7f3018004d50 2024-04-25T13:42:39.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f302d59a640 1 -- 172.21.15.80:0/4006456756 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3028009c10 con 0x7f3018004d50 2024-04-25T13:42:39.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f3034b86640 1 -- 172.21.15.80:0/4006456756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 msgr2=0x7f3018005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f3034b86640 1 --2- 172.21.15.80:0/4006456756 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f3018005160 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3028009f90 tx=0x7f302801cc60 comp rx=0 tx=0).stop 2024-04-25T13:42:39.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f302ed9d640 1 -- 172.21.15.80:0/4006456756 reap_dead start 2024-04-25T13:42:39.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f3034b86640 1 -- 172.21.15.80:0/4006456756 shutdown_connections 2024-04-25T13:42:39.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f3034b86640 1 -- 172.21.15.80:0/4006456756 >> 172.21.15.80:0/4006456756 conn(0x7f30300737a0 msgr2=0x7f303006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:39.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f3034b86640 1 -- 172.21.15.80:0/4006456756 shutdown_connections 2024-04-25T13:42:39.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.977+0000 7f3034b86640 1 -- 172.21.15.80:0/4006456756 wait complete. 2024-04-25T13:42:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 Processor -- start 2024-04-25T13:42:39.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 -- start start 2024-04-25T13:42:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f30300cbb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3030159be0 0x7f3030158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3030161830 0x7f3030161c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f30300d3410 con 0x7f3018004d50 2024-04-25T13:42:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f30300d3290 con 0x7f3030161830 2024-04-25T13:42:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f303015df60 con 0x7f3030159be0 2024-04-25T13:42:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f30300cbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f30300cbb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54076/0 (socket says 172.21.15.80:54076) 2024-04-25T13:42:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 -- 172.21.15.80:0/1526928869 learned_addr learned my addr 172.21.15.80:0/1526928869 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302dd9b640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3030159be0 0x7f3030158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302ed9d640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3030161830 0x7f3030161c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 -- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3030159be0 msgr2=0x7f3030158680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3030159be0 0x7f3030158680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 -- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3030161830 msgr2=0x7f3030161c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3030161830 0x7f3030161c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 -- 172.21.15.80:0/1526928869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3030162180 con 0x7f3018004d50 2024-04-25T13:42:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302ed9d640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3030161830 0x7f3030161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f302e59c640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f30300cbb50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f302400e4e0 tx=0x7f302400e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f301effd640 1 -- 172.21.15.80:0/1526928869 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30240052e0 con 0x7f3018004d50 2024-04-25T13:42:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f301effd640 1 -- 172.21.15.80:0/1526928869 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3024018070 con 0x7f3018004d50 2024-04-25T13:42:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f301effd640 1 -- 172.21.15.80:0/1526928869 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024005480 con 0x7f3018004d50 2024-04-25T13:42:39.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f303015a3c0 con 0x7f3018004d50 2024-04-25T13:42:39.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.981+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f303015a980 con 0x7f3018004d50 2024-04-25T13:42:39.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.985+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f303006a3a0 con 0x7f3018004d50 2024-04-25T13:42:39.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.985+0000 7f301effd640 1 -- 172.21.15.80:0/1526928869 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3024003800 con 0x7f3018004d50 2024-04-25T13:42:39.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.985+0000 7f301effd640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f301006f430 0x7f3010071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:39.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.985+0000 7f301effd640 1 -- 172.21.15.80:0/1526928869 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f302408f020 con 0x7f3018004d50 2024-04-25T13:42:39.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.985+0000 7f301effd640 1 -- 172.21.15.80:0/1526928869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7f3010076fb0 con 0x7f3018004d50 2024-04-25T13:42:39.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.985+0000 7f302dd9b640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f301006f430 0x7f3010071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:39.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.985+0000 7f302dd9b640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f301006f430 0x7f3010071940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f303015b410 tx=0x7f3020009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:39.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:39.989+0000 7f301effd640 1 -- 172.21.15.80:0/1526928869 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3030162180 con 0x7f3018004d50 2024-04-25T13:42:40.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.149+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f303015ab50 con 0x7f3018004d50 2024-04-25T13:42:40.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.149+0000 7f301effd640 1 -- 172.21.15.80:0/1526928869 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) ==== 81+0+34173 (secure 0 0 0) 0x7f302405be70 con 0x7f3018004d50 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 50, 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:42:36.507236+0000", 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:42:40.154 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 7, 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:42:40.155 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:40.156 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:40.157 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:40.158 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:40.159 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:40.160 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:42:40.161 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 6, 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:13.650442+0000", 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:40.162 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:40.163 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:40.164 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:40.165 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:40.166 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:40.167 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 7, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:14.725209+0000", 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:40.168 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:40.169 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:40.170 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:40.171 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:40.172 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.173 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:42:40.174 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.175 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.176 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:40.177 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.178 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:42:40.179 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:40.180 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:42:40.181 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:42:40.182 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:42:40.183 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.184 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.185 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.186 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.187 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.188 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.189 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:42:40.190 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:40.191 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:42:40.192 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:42:40.193 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:42:40.194 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.195 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull", 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.196 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:40.197 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.198 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.199 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.200 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.201 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:40.202 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.203 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:42:40.204 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:42:40.205 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:42:40.207 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:40.208 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.209 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.211 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:40.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:42:40.213 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:40.214 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.215 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.216 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:40.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:42:40.218 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:40.219 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:42:40.220 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:40.221 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:40.222 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.223 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:40.224 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:40.225 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:40.227 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f301006f430 msgr2=0x7f3010071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f301006f430 0x7f3010071940 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f303015b410 tx=0x7f3020009040 comp rx=0 tx=0).stop 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 msgr2=0x7f30300cbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f30300cbb50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f302400e4e0 tx=0x7f302400e9a0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 shutdown_connections 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f301006f430 0x7f3010071940 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3030161830 0x7f3030161c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3030159be0 0x7f3030158680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 --2- 172.21.15.80:0/1526928869 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3018004d50 0x7f30300cbb50 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 >> 172.21.15.80:0/1526928869 conn(0x7f30300737a0 msgr2=0x7f30301593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:40.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 shutdown_connections 2024-04-25T13:42:40.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.153+0000 7f3034b86640 1 -- 172.21.15.80:0/1526928869 wait complete. 2024-04-25T13:42:40.230 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:40.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.373+0000 7fc8642ae640 1 Processor -- start 2024-04-25T13:42:40.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.373+0000 7fc8642ae640 1 -- start start 2024-04-25T13:42:40.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc8642ae640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc85c0d06e0 0x7fc85c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:40.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc8642ae640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc85c0d0af0 con 0x7fc85c06e9e0 2024-04-25T13:42:40.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc8642ae640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc85c0d0d70 con 0x7fc85c161f90 2024-04-25T13:42:40.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc8642ae640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc85c0d1050 con 0x7fc85c0d06e0 2024-04-25T13:42:40.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc861849640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc85c0d06e0 0x7fc85c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:40.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc86284b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc85c06e9e0 0x7fc85c06edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52220/0 (socket says 172.21.15.80:52220) 2024-04-25T13:42:40.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc86284b640 1 -- 172.21.15.80:0/388375452 learned_addr learned my addr 172.21.15.80:0/388375452 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:40.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc861849640 1 -- 172.21.15.80:0/388375452 >> v1:172.21.15.87:6789/0 conn(0x7fc85c161f90 legacy=0x7fc85c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc861849640 1 -- 172.21.15.80:0/388375452 >> v1:172.21.15.80:6789/0 conn(0x7fc85c06e9e0 legacy=0x7fc85c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc861048640 1 -- 172.21.15.80:0/388375452 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3274881276 0 0) 0x7fc8540014a0 con 0x7fc85c06e9e0 2024-04-25T13:42:40.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.377+0000 7fc861849640 1 -- 172.21.15.80:0/388375452 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c06dca0 con 0x7fc85c0d06e0 2024-04-25T13:42:40.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc861849640 1 --2- 172.21.15.80:0/388375452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c0d06e0 0x7fc85c06e4a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0097c0 tx=0x7fc84c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbfe824c14668b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:40.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc861048640 1 -- 172.21.15.80:0/388375452 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0b1070 con 0x7fc85c0d06e0 2024-04-25T13:42:40.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc861048640 1 -- 172.21.15.80:0/388375452 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc84c0ac640 con 0x7fc85c0d06e0 2024-04-25T13:42:40.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc861048640 1 -- 172.21.15.80:0/388375452 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0aca10 con 0x7fc85c0d06e0 2024-04-25T13:42:40.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc8642ae640 1 -- 172.21.15.80:0/388375452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c0d06e0 msgr2=0x7fc85c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc8642ae640 1 --2- 172.21.15.80:0/388375452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c0d06e0 0x7fc85c06e4a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0097c0 tx=0x7fc84c01c350 comp rx=0 tx=0).stop 2024-04-25T13:42:40.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc8642ae640 1 -- 172.21.15.80:0/388375452 shutdown_connections 2024-04-25T13:42:40.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc8642ae640 1 --2- 172.21.15.80:0/388375452 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c0d06e0 0x7fc85c06e4a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc8642ae640 1 -- 172.21.15.80:0/388375452 >> 172.21.15.80:0/388375452 conn(0x7fc85c06d080 msgr2=0x7fc85c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:40.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc8642ae640 1 -- 172.21.15.80:0/388375452 shutdown_connections 2024-04-25T13:42:40.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.381+0000 7fc8642ae640 1 -- 172.21.15.80:0/388375452 wait complete. 2024-04-25T13:42:40.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 Processor -- start 2024-04-25T13:42:40.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 -- start start 2024-04-25T13:42:40.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c06e9e0 0x7fc85c161090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:40.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0d06e0 0x7fc85c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:40.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c161f90 0x7fc85c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:40.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c06f680 con 0x7fc85c06e9e0 2024-04-25T13:42:40.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc85c06f500 con 0x7fc85c161f90 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc85c144620 con 0x7fc85c0d06e0 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c06e9e0 0x7fc85c161090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c06e9e0 0x7fc85c161090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54100/0 (socket says 172.21.15.80:54100) 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 -- 172.21.15.80:0/3689876141 learned_addr learned my addr 172.21.15.80:0/3689876141 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86284b640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c161f90 0x7fc85c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc861849640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0d06e0 0x7fc85c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 -- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0d06e0 msgr2=0x7fc85c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0d06e0 0x7fc85c0af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 -- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c161f90 msgr2=0x7fc85c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c161f90 0x7fc85c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 -- 172.21.15.80:0/3689876141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc85c072020 con 0x7fc85c06e9e0 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc86204a640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c06e9e0 0x7fc85c161090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c3c0 tx=0x7fc85800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc852ffd640 1 -- 172.21.15.80:0/3689876141 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858017070 con 0x7fc85c06e9e0 2024-04-25T13:42:40.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc852ffd640 1 -- 172.21.15.80:0/3689876141 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8580052d0 con 0x7fc85c06e9e0 2024-04-25T13:42:40.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc852ffd640 1 -- 172.21.15.80:0/3689876141 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc858003840 con 0x7fc85c06e9e0 2024-04-25T13:42:40.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc85c072300 con 0x7fc85c06e9e0 2024-04-25T13:42:40.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.385+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc85c0728c0 con 0x7fc85c06e9e0 2024-04-25T13:42:40.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.389+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc85c06a3a0 con 0x7fc85c06e9e0 2024-04-25T13:42:40.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.389+0000 7fc852ffd640 1 -- 172.21.15.80:0/3689876141 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc85801a070 con 0x7fc85c06e9e0 2024-04-25T13:42:40.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.389+0000 7fc852ffd640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc84006f400 0x7fc840071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:40.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.389+0000 7fc852ffd640 1 -- 172.21.15.80:0/3689876141 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7fc85808e8f0 con 0x7fc85c06e9e0 2024-04-25T13:42:40.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.389+0000 7fc852ffd640 1 -- 172.21.15.80:0/3689876141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7fc840076f80 con 0x7fc85c06e9e0 2024-04-25T13:42:40.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.389+0000 7fc861849640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc84006f400 0x7fc840071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:40.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.389+0000 7fc861849640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc84006f400 0x7fc840071910 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0ae000 tx=0x7fc84c0a6070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:40.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.393+0000 7fc852ffd640 1 -- 172.21.15.80:0/3689876141 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc858021020 con 0x7fc85c06e9e0 2024-04-25T13:42:40.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.577+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc85c0ce620 con 0x7fc85c06e9e0 2024-04-25T13:42:40.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc852ffd640 1 -- 172.21.15.80:0/3689876141 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7fc858005480 con 0x7fc85c06e9e0 2024-04-25T13:42:40.584 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:40.584 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:42:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc84006f400 msgr2=0x7fc840071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc84006f400 0x7fc840071910 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc84c0ae000 tx=0x7fc84c0a6070 comp rx=0 tx=0).stop 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c06e9e0 msgr2=0x7fc85c161090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c06e9e0 0x7fc85c161090 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fc85800c3c0 tx=0x7fc85800c880 comp rx=0 tx=0).stop 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 shutdown_connections 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc84006f400 0x7fc840071910 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc85c161f90 0x7fc85c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc85c0d06e0 0x7fc85c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 --2- 172.21.15.80:0/3689876141 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc85c06e9e0 0x7fc85c161090 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 >> 172.21.15.80:0/3689876141 conn(0x7fc85c06d080 msgr2=0x7fc85c0b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 shutdown_connections 2024-04-25T13:42:40.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:40.581+0000 7fc8642ae640 1 -- 172.21.15.80:0/3689876141 wait complete. 2024-04-25T13:42:40.604 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-04-25T13:42:40.768 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":5730,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":24.331229908000001,"mdsmap_epoch":24,"osdmap_epoch":50,"osdmap_epoch_barrier":47,"uptime":30.184568465000002} 2024-04-25T13:42:40.769 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 5730, "mdsmap_epoch": 24, "osdmap_epoch": 50, "osdmap_epoch_barrier": 47, "rank_uptime": 24.331229908, "state": "up:active", "uptime": 30.184568465, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:42:40.769 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:42:40.769 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:40.769 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:40.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:40.775 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-04-25T13:42:40.825 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:42:40.821+0000 7fc35bfff640 -1 client.5883 handle_client_reply no pending request on tid 14 safe is:1 2024-04-25T13:42:40.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:42:40.826 INFO:teuthology.orchestra.run.smithi080.stderr:dd: failed to open 'small_file_1': Permission denied 2024-04-25T13:42:40.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:40.826 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-04-25T13:42:40.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:40.900 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-04-25T13:42:40.957 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:42:41.152 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:42:41.149+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-04-25T13:42:41.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.149+0000 7f649ea61640 1 Processor -- start 2024-04-25T13:42:41.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.149+0000 7f649ea61640 1 -- start start 2024-04-25T13:42:41.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f649ea61640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 0x7f6498070920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f649ea61640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f64980ccf60 0x7f6498068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f649ea61640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64980cd370 con 0x7f6498068ad0 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f649ea61640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f64980c7a60 con 0x7f64980ccf60 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f649ea61640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f649806d580 con 0x7f6498160400 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f648f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f64980ccf60 0x7f6498068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f648f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f64980ccf60 0x7f6498068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35330/0 (socket says 172.21.15.80:35330) 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f648f7fe640 1 -- 172.21.15.80:0/1315971395 learned_addr learned my addr 172.21.15.80:0/1315971395 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f6497fff640 1 --2- 172.21.15.80:0/1315971395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 0x7f6498070920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f648f7fe640 1 -- 172.21.15.80:0/1315971395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 msgr2=0x7f6498070920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f648f7fe640 1 --2- 172.21.15.80:0/1315971395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 0x7f6498070920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f648f7fe640 1 -- 172.21.15.80:0/1315971395 >> v1:172.21.15.87:6789/0 conn(0x7f6498068ad0 legacy=0x7f6498164100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f648f7fe640 1 -- 172.21.15.80:0/1315971395 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6498068ee0 con 0x7f64980ccf60 2024-04-25T13:42:41.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f648f7fe640 1 --2- 172.21.15.80:0/1315971395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64980ccf60 0x7f6498068590 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f64800098a0 tx=0x7f648001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=234c787a55c4795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f64977fe640 1 -- 172.21.15.80:0/1315971395 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64800b0070 con 0x7f64980ccf60 2024-04-25T13:42:41.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f64977fe640 1 -- 172.21.15.80:0/1315971395 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64800ab600 con 0x7f64980ccf60 2024-04-25T13:42:41.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.153+0000 7f64977fe640 1 -- 172.21.15.80:0/1315971395 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64800ab9d0 con 0x7f64980ccf60 2024-04-25T13:42:41.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 -- 172.21.15.80:0/1315971395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64980ccf60 msgr2=0x7f6498068590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 --2- 172.21.15.80:0/1315971395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64980ccf60 0x7f6498068590 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f64800098a0 tx=0x7f648001c4b0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 -- 172.21.15.80:0/1315971395 shutdown_connections 2024-04-25T13:42:41.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 --2- 172.21.15.80:0/1315971395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f64980ccf60 0x7f6498068590 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 --2- 172.21.15.80:0/1315971395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 0x7f6498070920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 -- 172.21.15.80:0/1315971395 >> 172.21.15.80:0/1315971395 conn(0x7f64980cf390 msgr2=0x7f64980cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:41.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 -- 172.21.15.80:0/1315971395 shutdown_connections 2024-04-25T13:42:41.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 -- 172.21.15.80:0/1315971395 wait complete. 2024-04-25T13:42:41.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 Processor -- start 2024-04-25T13:42:41.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.157+0000 7f649ea61640 1 -- start start 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6498068ad0 0x7f6498155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64980ccf60 0x7f6498157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 0x7f6498158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6498164c20 con 0x7f6498068ad0 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f649806f310 con 0x7f64980ccf60 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6498164da0 con 0x7f6498160400 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6498068ad0 0x7f6498155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6498068ad0 0x7f6498155000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35332/0 (socket says 172.21.15.80:35332) 2024-04-25T13:42:41.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 -- 172.21.15.80:0/253348187 learned_addr learned my addr 172.21.15.80:0/253348187 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 -- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 msgr2=0x7f6498158390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 0x7f6498158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 -- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64980ccf60 msgr2=0x7f6498157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64980ccf60 0x7f6498157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 -- 172.21.15.80:0/253348187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6498155bd0 con 0x7f6498068ad0 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f6497fff640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6498068ad0 0x7f6498155000 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f648800d600 tx=0x7f648800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f64957fa640 1 -- 172.21.15.80:0/253348187 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488004240 con 0x7f6498068ad0 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f64957fa640 1 -- 172.21.15.80:0/253348187 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64880043e0 con 0x7f6498068ad0 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6498155eb0 con 0x7f6498068ad0 2024-04-25T13:42:41.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f64957fa640 1 -- 172.21.15.80:0/253348187 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488003840 con 0x7f6498068ad0 2024-04-25T13:42:41.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6498156470 con 0x7f6498068ad0 2024-04-25T13:42:41.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649806a3a0 con 0x7f6498068ad0 2024-04-25T13:42:41.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f64957fa640 1 -- 172.21.15.80:0/253348187 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f64880039e0 con 0x7f6498068ad0 2024-04-25T13:42:41.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.161+0000 7f64957fa640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f647806f430 0x7f6478071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:41.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.165+0000 7f64957fa640 1 -- 172.21.15.80:0/253348187 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5709+0+0 (secure 0 0 0) 0x7f648808f520 con 0x7f6498068ad0 2024-04-25T13:42:41.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.165+0000 7f64957fa640 1 -- 172.21.15.80:0/253348187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=51}) -- 0x7f6478077010 con 0x7f6498068ad0 2024-04-25T13:42:41.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.165+0000 7f648f7fe640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f647806f430 0x7f6478071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:41.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.165+0000 7f648f7fe640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f647806f430 0x7f6478071940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f64800ad000 tx=0x7f6480004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:41.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.169+0000 7f64957fa640 1 -- 172.21.15.80:0/253348187 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f648805c4a0 con 0x7f6498068ad0 2024-04-25T13:42:41.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.321+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f649806db90 con 0x7f6498068ad0 2024-04-25T13:42:41.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.325+0000 7f64957fa640 1 -- 172.21.15.80:0/253348187 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v50) ==== 81+0+34173 (secure 0 0 0) 0x7f648805bda0 con 0x7f6498068ad0 2024-04-25T13:42:41.330 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:41.330 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:41.330 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 50, 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:42:36.507236+0000", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:42:41.331 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 7, 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:42:41.332 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:41.333 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.334 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:41.335 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:41.336 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:41.337 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.338 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 6, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:13.650442+0000", 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:41.339 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:41.340 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:41.341 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:41.342 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:41.343 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.344 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 7, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:14.725209+0000", 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:41.345 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:41.346 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:41.347 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:41.348 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.349 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:41.350 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:41.351 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.353 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.354 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.355 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:41.356 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:41.357 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:41.358 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.359 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.360 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.361 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.362 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.363 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:42:41.364 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:41.365 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:41.366 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:41.367 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.368 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.369 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:41.370 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:42:41.371 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull", 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:41.372 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.373 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.374 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:42:41.375 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:41.376 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:41.377 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:42:41.378 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.379 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.380 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.381 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.382 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.383 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.384 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:42:41.385 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:42:41.386 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:41.387 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:41.388 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.389 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.390 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.391 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.393 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:41.394 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:41.395 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:41.396 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:41.397 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.398 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:41.399 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.400 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:42:41.401 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:42:41.402 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:42:41.403 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:41.403 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:41.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f647806f430 msgr2=0x7f6478071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f647806f430 0x7f6478071940 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f64800ad000 tx=0x7f6480004740 comp rx=0 tx=0).stop 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6498068ad0 msgr2=0x7f6498155000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6498068ad0 0x7f6498155000 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f648800d600 tx=0x7f648800dac0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 shutdown_connections 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f647806f430 0x7f6478071940 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6498160400 0x7f6498158390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64980ccf60 0x7f6498157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 --2- 172.21.15.80:0/253348187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6498068ad0 0x7f6498155000 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 >> 172.21.15.80:0/253348187 conn(0x7f64980cf390 msgr2=0x7f6498159650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 shutdown_connections 2024-04-25T13:42:41.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:41.329+0000 7f649ea61640 1 -- 172.21.15.80:0/253348187 wait complete. 2024-04-25T13:42:41.406 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:42:46.154 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:42:46.153+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-04-25T13:42:46.407 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:42:46.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a104ba640 1 Processor -- start 2024-04-25T13:42:46.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a104ba640 1 -- start start 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a104ba640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8a080d1f10 0x7f8a08169830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a104ba640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a080b4da0 con 0x7f8a080d1b00 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a104ba640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a080b5320 con 0x7f8a080d1280 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a104ba640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8a080b5040 con 0x7f8a080d1f10 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a0e256640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8a080d1b00 0x7f8a080cf530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:46650/0 (socket says 172.21.15.80:46650) 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a0e256640 1 -- 172.21.15.80:0/2518800513 learned_addr learned my addr 172.21.15.80:0/2518800513 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a0ea57640 1 --2- 172.21.15.80:0/2518800513 >> v2:172.21.15.80:3300/0 conn(0x7f8a080d1f10 0x7f8a08169830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a0d254640 1 -- 172.21.15.80:0/2518800513 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3704418057 0 0) 0x7f8a080b4da0 con 0x7f8a080d1b00 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a0d254640 1 -- 172.21.15.80:0/2518800513 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89f4003530 con 0x7f8a080d1b00 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a0ea57640 1 -- 172.21.15.80:0/2518800513 >> v1:172.21.15.87:6789/0 conn(0x7f8a080d1280 legacy=0x7f8a080b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a0ea57640 1 -- 172.21.15.80:0/2518800513 >> v1:172.21.15.80:6789/0 conn(0x7f8a080d1b00 legacy=0x7f8a080cf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:46.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.569+0000 7f8a0ea57640 1 -- 172.21.15.80:0/2518800513 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a080d2500 con 0x7f8a080d1f10 2024-04-25T13:42:46.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a0ea57640 1 --2- 172.21.15.80:0/2518800513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1f10 0x7f8a08169830 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009d90 tx=0x7f8a0001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc16673c557bb01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:46.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a0d254640 1 -- 172.21.15.80:0/2518800513 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000b2070 con 0x7f8a080d1f10 2024-04-25T13:42:46.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a0d254640 1 -- 172.21.15.80:0/2518800513 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a000a58b0 con 0x7f8a080d1f10 2024-04-25T13:42:46.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a0d254640 1 -- 172.21.15.80:0/2518800513 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a000a5c80 con 0x7f8a080d1f10 2024-04-25T13:42:46.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a104ba640 1 -- 172.21.15.80:0/2518800513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1f10 msgr2=0x7f8a08169830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:46.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a104ba640 1 --2- 172.21.15.80:0/2518800513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1f10 0x7f8a08169830 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009d90 tx=0x7f8a0001c8e0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a104ba640 1 -- 172.21.15.80:0/2518800513 shutdown_connections 2024-04-25T13:42:46.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a104ba640 1 --2- 172.21.15.80:0/2518800513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1f10 0x7f8a08169830 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a104ba640 1 -- 172.21.15.80:0/2518800513 >> 172.21.15.80:0/2518800513 conn(0x7f8a080d3b50 msgr2=0x7f8a080d3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:46.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a104ba640 1 -- 172.21.15.80:0/2518800513 shutdown_connections 2024-04-25T13:42:46.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.573+0000 7f8a104ba640 1 -- 172.21.15.80:0/2518800513 wait complete. 2024-04-25T13:42:46.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 Processor -- start 2024-04-25T13:42:46.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 -- start start 2024-04-25T13:42:46.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1280 0x7f8a08166b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:46.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a080d1b00 0x7f8a080b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:46.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a080d1f10 0x7f8a080b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8a080b7230 con 0x7f8a080d1280 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8a080b70b0 con 0x7f8a080d1b00 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0816c590 con 0x7f8a080d1f10 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1280 0x7f8a08166b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1280 0x7f8a08166b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35362/0 (socket says 172.21.15.80:35362) 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 -- 172.21.15.80:0/3383773346 learned_addr learned my addr 172.21.15.80:0/3383773346 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0da55640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a080d1b00 0x7f8a080b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0ea57640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a080d1f10 0x7f8a080b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 -- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a080d1f10 msgr2=0x7f8a080b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a080d1f10 0x7f8a080b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 -- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a080d1b00 msgr2=0x7f8a080b3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a080d1b00 0x7f8a080b3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 -- 172.21.15.80:0/3383773346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a080cc4b0 con 0x7f8a080d1280 2024-04-25T13:42:46.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a0e256640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1280 0x7f8a08166b20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c430 tx=0x7f8a0400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:46.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f89feffd640 1 -- 172.21.15.80:0/3383773346 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04017070 con 0x7f8a080d1280 2024-04-25T13:42:46.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f89feffd640 1 -- 172.21.15.80:0/3383773346 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04005d10 con 0x7f8a080d1280 2024-04-25T13:42:46.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a080cc790 con 0x7f8a080d1280 2024-04-25T13:42:46.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f89feffd640 1 -- 172.21.15.80:0/3383773346 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04007310 con 0x7f8a080d1280 2024-04-25T13:42:46.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a080ccd50 con 0x7f8a080d1280 2024-04-25T13:42:46.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.577+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0806a3a0 con 0x7f8a080d1280 2024-04-25T13:42:46.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.581+0000 7f89feffd640 1 -- 172.21.15.80:0/3383773346 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8a0401a070 con 0x7f8a080d1280 2024-04-25T13:42:46.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.581+0000 7f89feffd640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89ec06f400 0x7f89ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:46.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.581+0000 7f89feffd640 1 -- 172.21.15.80:0/3383773346 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5709+0+0 (secure 0 0 0) 0x7f8a0408e8e0 con 0x7f8a080d1280 2024-04-25T13:42:46.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.581+0000 7f89feffd640 1 -- 172.21.15.80:0/3383773346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=52}) -- 0x7f89ec076fe0 con 0x7f8a080d1280 2024-04-25T13:42:46.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.581+0000 7f8a0da55640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89ec06f400 0x7f89ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:46.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.581+0000 7f8a0da55640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89ec06f400 0x7f89ec071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8a080b8a30 tx=0x7f89f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:46.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.585+0000 7f89feffd640 1 -- 172.21.15.80:0/3383773346 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8a0405b940 con 0x7f8a080d1280 2024-04-25T13:42:46.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.745+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f8a080d4680 con 0x7f8a080d1280 2024-04-25T13:42:46.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.745+0000 7f89feffd640 1 -- 172.21.15.80:0/3383773346 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v51) ==== 81+0+34093 (secure 0 0 0) 0x7f8a0405b240 con 0x7f8a080d1280 2024-04-25T13:42:46.750 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:46.750 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:46.750 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 51, 2024-04-25T13:42:46.750 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:42:46.750 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:42:46.750 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:42:45.197560+0000", 2024-04-25T13:42:46.750 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:42:46.751 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 7, 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:46.752 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:42:46.753 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:46.754 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:46.755 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:46.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:42:46.757 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:46.758 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 6, 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:13.650442+0000", 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:46.759 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:46.760 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:46.761 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:46.762 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:46.763 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:42:46.764 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:46.765 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.766 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.766 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.766 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 7, 2024-04-25T13:42:46.766 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:42:46.766 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:14.725209+0000", 2024-04-25T13:42:46.766 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:42:46.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "49", 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:46.768 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:46.769 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:46.770 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.771 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.772 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:46.773 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.774 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.775 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:42:46.776 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:42:46.777 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.778 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.779 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.780 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.781 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.782 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:42:46.783 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:46.784 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.785 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.786 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:46.787 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.788 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:46.789 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:46.790 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.791 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.792 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.793 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.793 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:42:46.793 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:46.793 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.793 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.793 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:46.794 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.794 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.794 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.794 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:42:46.794 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:46.794 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.795 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.795 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.795 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:42:46.795 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:46.795 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.795 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.796 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:46.796 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.796 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.796 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.796 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:42:46.796 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:46.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.797 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.797 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.797 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:42:46.797 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:46.797 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.798 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.798 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.798 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:42:46.798 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:42:46.798 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:42:46.798 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:42:46.799 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:46.799 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:46.799 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:46.799 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.799 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.799 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.799 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:46.800 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:46.800 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:46.800 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:46.800 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:46.800 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:46.800 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:46.801 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:46.801 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:46.801 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:46.801 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:46.801 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:46.801 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:46.801 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.802 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.802 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.802 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:42:46.802 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:46.802 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.802 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.803 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.803 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:42:46.803 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:46.803 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.803 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.803 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.804 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:46.804 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.804 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.804 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.804 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:42:46.804 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:46.805 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.805 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.805 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.805 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:42:46.805 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:46.805 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.805 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.806 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.806 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:46.806 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.806 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.806 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.806 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:42:46.807 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:46.807 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.807 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.807 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.807 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:42:46.807 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:46.807 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.808 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.808 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.808 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:46.808 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.808 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.808 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.808 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:42:46.809 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:46.809 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.809 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.809 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.809 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:42:46.809 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:46.809 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.810 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.810 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.810 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:42:46.810 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:42:46.810 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:42:46.810 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:42:46.810 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:46.811 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:46.811 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:46.811 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.811 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.811 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.811 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:46.811 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:46.811 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:46.812 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:46.812 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:46.812 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:46.812 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:46.812 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:46.812 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:46.812 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:46.812 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:46.813 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:46.813 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:46.813 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.813 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.813 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.813 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:42:46.813 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:46.814 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.814 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.814 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.814 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:42:46.814 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:46.814 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.814 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.815 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.815 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:46.815 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.815 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.815 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.815 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:42:46.815 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:46.816 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.816 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.816 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.816 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:42:46.816 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:46.816 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.816 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.817 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.817 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:46.817 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.817 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.817 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.817 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:42:46.817 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:46.818 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.818 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.818 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.818 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:42:46.818 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:46.818 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.818 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:42:46.819 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:42:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.820 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.821 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.821 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:42:46.821 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:42:46.821 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:42:46.821 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:42:46.821 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:46.821 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:46.822 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:46.822 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.822 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.822 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.822 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:46.822 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:46.822 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:46.823 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:46.823 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:46.823 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:46.823 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:46.823 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:46.824 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:46.824 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:46.824 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:46.824 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:46.824 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:46.824 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.824 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.825 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.825 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:42:46.825 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:46.825 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.825 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.825 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.825 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:42:46.826 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:46.826 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.826 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.826 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.826 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:46.826 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.826 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.826 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.827 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:42:46.827 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:46.827 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.827 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.827 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.827 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:42:46.827 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:46.827 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.828 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.828 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.828 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:46.828 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.828 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.828 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.828 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:42:46.828 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:46.829 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.829 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.829 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.829 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:42:46.829 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:46.829 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.829 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.830 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.830 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:46.830 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.830 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.830 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.830 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:42:46.831 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:46.831 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.831 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.831 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.831 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:42:46.831 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:46.831 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.831 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.832 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.832 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:42:46.832 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:42:46.832 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:42:46.832 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:42:46.832 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:46.832 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:46.833 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.834 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.835 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:42:46.836 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:46.837 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:42:46.838 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:46.839 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.840 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:46.841 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:46.842 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:46.843 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:42:46.844 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:42:46.845 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:42:46.846 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:42:46.846 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:42:46.846 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:42:46.846 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:42:46.846 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:46.846 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89ec06f400 msgr2=0x7f89ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89ec06f400 0x7f89ec071910 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8a080b8a30 tx=0x7f89f8009040 comp rx=0 tx=0).stop 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1280 msgr2=0x7f8a08166b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1280 0x7f8a08166b20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f8a0400c430 tx=0x7f8a0400c8f0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 shutdown_connections 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89ec06f400 0x7f89ec071910 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8a080d1f10 0x7f8a080b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8a080d1b00 0x7f8a080b3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 --2- 172.21.15.80:0/3383773346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8a080d1280 0x7f8a08166b20 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 >> 172.21.15.80:0/3383773346 conn(0x7f8a080d3b50 msgr2=0x7f8a080d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 shutdown_connections 2024-04-25T13:42:46.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:46.749+0000 7f8a104ba640 1 -- 172.21.15.80:0/3383773346 wait complete. 2024-04-25T13:42:46.849 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:42:51.851 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:42:52.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.005+0000 7f234f871640 1 Processor -- start 2024-04-25T13:42:52.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.005+0000 7f234f871640 1 -- start start 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234f871640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f234806f5a0 0x7f2348160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234f871640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 0x7f2348161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234f871640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f23480b4bd0 0x7f23480b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234f871640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f23481619c0 con 0x7f234806f5a0 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234f871640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2348073f10 con 0x7f23480b4bd0 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234f871640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f234806fcf0 con 0x7f2348161070 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234d60d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f234806f5a0 0x7f2348160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234d60d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f234806f5a0 0x7f2348160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42932/0 (socket says 172.21.15.80:42932) 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234d60d640 1 -- 172.21.15.80:0/3567218827 learned_addr learned my addr 172.21.15.80:0/3567218827 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:52.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234ce0c640 1 --2- 172.21.15.80:0/3567218827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 0x7f2348161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234de0e640 1 --2- 172.21.15.80:0/3567218827 >> v2:172.21.15.87:3300/0 conn(0x7f23480b4bd0 0x7f23480b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234d60d640 1 -- 172.21.15.80:0/3567218827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 msgr2=0x7f2348161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234d60d640 1 --2- 172.21.15.80:0/3567218827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 0x7f2348161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234d60d640 1 -- 172.21.15.80:0/3567218827 >> v2:172.21.15.87:3300/0 conn(0x7f23480b4bd0 msgr2=0x7f23480b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234d60d640 1 --2- 172.21.15.80:0/3567218827 >> v2:172.21.15.87:3300/0 conn(0x7f23480b4bd0 0x7f23480b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234d60d640 1 -- 172.21.15.80:0/3567218827 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23480b0c60 con 0x7f234806f5a0 2024-04-25T13:42:52.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.009+0000 7f234de0e640 1 --2- 172.21.15.80:0/3567218827 >> v2:172.21.15.87:3300/0 conn(0x7f23480b4bd0 0x7f23480b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:52.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234d60d640 1 --2- 172.21.15.80:0/3567218827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f234806f5a0 0x7f2348160b30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f23480ce160 tx=0x7f234401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24dc7127f5a0d6ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f233ffff640 1 -- 172.21.15.80:0/3567218827 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440b0070 con 0x7f234806f5a0 2024-04-25T13:42:52.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f233ffff640 1 -- 172.21.15.80:0/3567218827 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23440ab600 con 0x7f234806f5a0 2024-04-25T13:42:52.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f233ffff640 1 -- 172.21.15.80:0/3567218827 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23440ab9d0 con 0x7f234806f5a0 2024-04-25T13:42:52.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 -- 172.21.15.80:0/3567218827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f234806f5a0 msgr2=0x7f2348160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 --2- 172.21.15.80:0/3567218827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f234806f5a0 0x7f2348160b30 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f23480ce160 tx=0x7f234401c490 comp rx=0 tx=0).stop 2024-04-25T13:42:52.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 -- 172.21.15.80:0/3567218827 shutdown_connections 2024-04-25T13:42:52.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 --2- 172.21.15.80:0/3567218827 >> v2:172.21.15.87:3300/0 conn(0x7f23480b4bd0 0x7f23480b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 --2- 172.21.15.80:0/3567218827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 0x7f2348161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 --2- 172.21.15.80:0/3567218827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f234806f5a0 0x7f2348160b30 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 -- 172.21.15.80:0/3567218827 >> 172.21.15.80:0/3567218827 conn(0x7f234806d080 msgr2=0x7f23480b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 -- 172.21.15.80:0/3567218827 shutdown_connections 2024-04-25T13:42:52.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.013+0000 7f234f871640 1 -- 172.21.15.80:0/3567218827 wait complete. 2024-04-25T13:42:52.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 Processor -- start 2024-04-25T13:42:52.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 -- start start 2024-04-25T13:42:52.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234806f5a0 0x7f2348149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23480b4bd0 0x7f23481559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 0x7f2348151da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2348072bb0 con 0x7f23480b4bd0 2024-04-25T13:42:52.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f23480b5930 con 0x7f234806f5a0 2024-04-25T13:42:52.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2348072d30 con 0x7f2348161070 2024-04-25T13:42:52.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23480b4bd0 0x7f23481559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23480b4bd0 0x7f23481559b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42934/0 (socket says 172.21.15.80:42934) 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 -- 172.21.15.80:0/3873770488 learned_addr learned my addr 172.21.15.80:0/3873770488 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234de0e640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 0x7f2348151da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234d60d640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234806f5a0 0x7f2348149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 -- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 msgr2=0x7f2348151da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 0x7f2348151da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 -- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234806f5a0 msgr2=0x7f2348149c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234806f5a0 0x7f2348149c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 -- 172.21.15.80:0/3873770488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2348156910 con 0x7f23480b4bd0 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234d60d640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234806f5a0 0x7f2348149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234ce0c640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23480b4bd0 0x7f23481559b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f233800d5f0 tx=0x7f233800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f233dffb640 1 -- 172.21.15.80:0/3873770488 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380125f0 con 0x7f23480b4bd0 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f233dffb640 1 -- 172.21.15.80:0/3873770488 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2338019070 con 0x7f23480b4bd0 2024-04-25T13:42:52.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2348156630 con 0x7f23480b4bd0 2024-04-25T13:42:52.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f233dffb640 1 -- 172.21.15.80:0/3873770488 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23380129b0 con 0x7f23480b4bd0 2024-04-25T13:42:52.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.017+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23480cfae0 con 0x7f23480b4bd0 2024-04-25T13:42:52.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.021+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23481563a0 con 0x7f23480b4bd0 2024-04-25T13:42:52.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.021+0000 7f233dffb640 1 -- 172.21.15.80:0/3873770488 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2338012b50 con 0x7f23480b4bd0 2024-04-25T13:42:52.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.021+0000 7f233dffb640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f232c06f430 0x7f232c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.021+0000 7f233dffb640 1 -- 172.21.15.80:0/3873770488 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f233808f5e0 con 0x7f23480b4bd0 2024-04-25T13:42:52.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.021+0000 7f234d60d640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f232c06f430 0x7f232c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.021+0000 7f234d60d640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f232c06f430 0x7f232c071940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2348071e30 tx=0x7f2344008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.025+0000 7f233dffb640 1 -- 172.21.15.80:0/3873770488 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f233805c4d0 con 0x7f23480b4bd0 2024-04-25T13:42:52.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.193+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f234806a3a0 con 0x7f23480b4bd0 2024-04-25T13:42:52.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.193+0000 7f233dffb640 1 -- 172.21.15.80:0/3873770488 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) ==== 81+0+34030 (secure 0 0 0) 0x7f233805bdd0 con 0x7f23480b4bd0 2024-04-25T13:42:52.197 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:52.197 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:52.197 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 52, 2024-04-25T13:42:52.197 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:42:52.197 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:42:52.197 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:42:49.227779+0000", 2024-04-25T13:42:52.197 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:42:52.198 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 7, 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:42:52.199 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:42:52.200 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "52", 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:52.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:52.202 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:52.203 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:52.204 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.205 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 6, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:13.650442+0000", 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:52.206 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:52.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "52", 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:52.208 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.209 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:42:52.210 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:52.211 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 7, 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:14.725209+0000", 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:52.212 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:52.213 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "52", 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:52.214 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:52.215 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:52.216 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:52.217 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.218 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.219 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.222 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:42:52.223 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.224 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.226 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.227 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.229 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:42:52.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.232 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.235 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.237 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.238 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:42:52.239 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.240 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.241 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.242 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.243 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:42:52.244 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.245 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.246 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:42:52.248 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.249 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.250 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:42:52.251 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.252 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.253 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.254 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.255 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.256 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.257 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.258 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.259 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:42:52.260 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.261 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.262 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:52.263 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.264 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.265 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.266 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:42:52.267 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:42:52.268 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:42:52.269 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:42:52.269 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:42:52.269 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:42:52.269 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:42:52.269 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:42:52.269 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.269 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:52.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f232c06f430 msgr2=0x7f232c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f232c06f430 0x7f232c071940 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2348071e30 tx=0x7f2344008040 comp rx=0 tx=0).stop 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23480b4bd0 msgr2=0x7f23481559b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23480b4bd0 0x7f23481559b0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f233800d5f0 tx=0x7f233800dab0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 shutdown_connections 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f232c06f430 0x7f232c071940 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2348161070 0x7f2348151da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23480b4bd0 0x7f23481559b0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 --2- 172.21.15.80:0/3873770488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f234806f5a0 0x7f2348149c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 >> 172.21.15.80:0/3873770488 conn(0x7f234806d080 msgr2=0x7f23480b1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 shutdown_connections 2024-04-25T13:42:52.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.197+0000 7f234f871640 1 -- 172.21.15.80:0/3873770488 wait complete. 2024-04-25T13:42:52.272 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-25T13:42:52.272 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:42:52.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf755e9640 1 Processor -- start 2024-04-25T13:42:52.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf755e9640 1 -- start start 2024-04-25T13:42:52.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf755e9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcf7015eae0 0x7fcf700d1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf755e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf700d17d0 0x7fcf70157d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf755e9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf7015eef0 con 0x7fcf701582c0 2024-04-25T13:42:52.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf755e9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcf7006ec70 con 0x7fcf7015eae0 2024-04-25T13:42:52.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf755e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7015f1b0 con 0x7fcf700d17d0 2024-04-25T13:42:52.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf6effd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcf7015eae0 0x7fcf700d1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf6effd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcf7015eae0 0x7fcf700d1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42938/0 (socket says 172.21.15.80:42938) 2024-04-25T13:42:52.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf6effd640 1 -- 172.21.15.80:0/636562787 learned_addr learned my addr 172.21.15.80:0/636562787 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:52.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf6e7fc640 1 --2- 172.21.15.80:0/636562787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf700d17d0 0x7fcf70157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf6effd640 1 -- 172.21.15.80:0/636562787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf700d17d0 msgr2=0x7fcf70157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf6effd640 1 --2- 172.21.15.80:0/636562787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf700d17d0 0x7fcf70157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf6effd640 1 -- 172.21.15.80:0/636562787 >> v1:172.21.15.87:6789/0 conn(0x7fcf701582c0 legacy=0x7fcf700d0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.421+0000 7fcf6effd640 1 -- 172.21.15.80:0/636562787 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7015f490 con 0x7fcf7015eae0 2024-04-25T13:42:52.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf6effd640 1 --2- 172.21.15.80:0/636562787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf7015eae0 0x7fcf700d1290 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009870 tx=0x7fcf5801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fbbef5ad7f7a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf6dffb640 1 -- 172.21.15.80:0/636562787 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf580b0070 con 0x7fcf7015eae0 2024-04-25T13:42:52.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf6dffb640 1 -- 172.21.15.80:0/636562787 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf580ab600 con 0x7fcf7015eae0 2024-04-25T13:42:52.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf6dffb640 1 -- 172.21.15.80:0/636562787 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf580ab9d0 con 0x7fcf7015eae0 2024-04-25T13:42:52.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf755e9640 1 -- 172.21.15.80:0/636562787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf7015eae0 msgr2=0x7fcf700d1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf755e9640 1 --2- 172.21.15.80:0/636562787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf7015eae0 0x7fcf700d1290 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fcf58009870 tx=0x7fcf5801c4d0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf755e9640 1 -- 172.21.15.80:0/636562787 shutdown_connections 2024-04-25T13:42:52.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf755e9640 1 --2- 172.21.15.80:0/636562787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf700d17d0 0x7fcf70157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf755e9640 1 --2- 172.21.15.80:0/636562787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf7015eae0 0x7fcf700d1290 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf755e9640 1 -- 172.21.15.80:0/636562787 >> 172.21.15.80:0/636562787 conn(0x7fcf700d2170 msgr2=0x7fcf700d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf755e9640 1 -- 172.21.15.80:0/636562787 shutdown_connections 2024-04-25T13:42:52.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.425+0000 7fcf755e9640 1 -- 172.21.15.80:0/636562787 wait complete. 2024-04-25T13:42:52.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 Processor -- start 2024-04-25T13:42:52.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 -- start start 2024-04-25T13:42:52.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf700d17d0 0x7fcf700b7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf701582c0 0x7fcf701557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eae0 0x7fcf7014f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70076e00 con 0x7fcf701582c0 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf70076c80 con 0x7fcf700d17d0 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf700b1910 con 0x7fcf7015eae0 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf701582c0 0x7fcf701557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf701582c0 0x7fcf701557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42950/0 (socket says 172.21.15.80:42950) 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 -- 172.21.15.80:0/2555567385 learned_addr learned my addr 172.21.15.80:0/2555567385 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6effd640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf700d17d0 0x7fcf700b7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6f7fe640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eae0 0x7fcf7014f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 -- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eae0 msgr2=0x7fcf7014f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eae0 0x7fcf7014f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 -- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf700d17d0 msgr2=0x7fcf700b7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf700d17d0 0x7fcf700b7460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 -- 172.21.15.80:0/2555567385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7014e270 con 0x7fcf701582c0 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6effd640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf700d17d0 0x7fcf700b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf6e7fc640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf701582c0 0x7fcf701557f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d650 tx=0x7fcf6000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf677fe640 1 -- 172.21.15.80:0/2555567385 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000b660 con 0x7fcf701582c0 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf677fe640 1 -- 172.21.15.80:0/2555567385 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf60019070 con 0x7fcf701582c0 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf701563d0 con 0x7fcf701582c0 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf677fe640 1 -- 172.21.15.80:0/2555567385 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000ba20 con 0x7fcf701582c0 2024-04-25T13:42:52.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf70156990 con 0x7fcf701582c0 2024-04-25T13:42:52.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.429+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf7006a3a0 con 0x7fcf701582c0 2024-04-25T13:42:52.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.433+0000 7fcf677fe640 1 -- 172.21.15.80:0/2555567385 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcf6000bbc0 con 0x7fcf701582c0 2024-04-25T13:42:52.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.433+0000 7fcf677fe640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf5006f430 0x7fcf50071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.433+0000 7fcf677fe640 1 -- 172.21.15.80:0/2555567385 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fcf600900a0 con 0x7fcf701582c0 2024-04-25T13:42:52.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.433+0000 7fcf6effd640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf5006f430 0x7fcf50071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.433+0000 7fcf6effd640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf5006f430 0x7fcf50071940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcf580ad000 tx=0x7fcf580a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.437+0000 7fcf677fe640 1 -- 172.21.15.80:0/2555567385 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf6005d070 con 0x7fcf701582c0 2024-04-25T13:42:52.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.601+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fcf700b79a0 con 0x7fcf701582c0 2024-04-25T13:42:52.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf677fe640 1 -- 172.21.15.80:0/2555567385 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v52) ==== 81+0+34030 (secure 0 0 0) 0x7fcf6005c970 con 0x7fcf701582c0 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 52, 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:42:49.227779+0000", 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:42:52.608 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 7, 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:42:52.609 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:52.610 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:52.611 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "52", 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:52.612 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:52.613 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:42:52.614 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:52.615 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 6, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:13.650442+0000", 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:52.616 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.617 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "52", 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:52.618 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:52.619 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:52.620 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:52.621 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 7, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:42:14.725209+0000", 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:42:52.622 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:42:52.623 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "52", 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:42:52.624 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:42:52.625 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:42:52.626 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:42:52.627 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.628 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:42:52.629 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:42:52.630 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.631 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.634 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.635 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.636 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:42:52.638 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.639 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.641 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.642 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.644 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:42:52.647 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:42:52.648 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.649 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.650 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.651 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.655 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.656 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.658 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:42:52.660 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.661 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.662 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.663 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.664 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:42:52.666 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 45, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.667 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.668 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.669 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.670 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.671 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.672 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.673 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.674 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.675 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:42:52.676 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:42:52.677 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:42:52.678 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:42:52.679 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:42:52.679 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:42:52.679 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:42:52.679 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf5006f430 msgr2=0x7fcf50071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf5006f430 0x7fcf50071940 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fcf580ad000 tx=0x7fcf580a6000 comp rx=0 tx=0).stop 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf701582c0 msgr2=0x7fcf701557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf701582c0 0x7fcf701557f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d650 tx=0x7fcf6000db10 comp rx=0 tx=0).stop 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 shutdown_connections 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf5006f430 0x7fcf50071940 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf7015eae0 0x7fcf7014f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf701582c0 0x7fcf701557f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 --2- 172.21.15.80:0/2555567385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf700d17d0 0x7fcf700b7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 >> 172.21.15.80:0/2555567385 conn(0x7fcf700d2170 msgr2=0x7fcf700af9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 shutdown_connections 2024-04-25T13:42:52.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.605+0000 7fcf755e9640 1 -- 172.21.15.80:0/2555567385 wait complete. 2024-04-25T13:42:52.682 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:52.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0ccd1a0640 1 Processor -- start 2024-04-25T13:42:52.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0ccd1a0640 1 -- start start 2024-04-25T13:42:52.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0ccd1a0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc8161f90 0x7f0cc80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0ccd1a0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0cc80d0510 0x7f0cc815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0ccd1a0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0cc815ea70 0x7f0cc815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0ccd1a0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80d0d70 con 0x7f0cc80d0510 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0ccd1a0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0cc80d1050 con 0x7f0cc815ea70 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0ccd1a0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc806dca0 con 0x7f0cc8161f90 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cbe59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0cc80d0510 0x7f0cc815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cc759e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0cc815ea70 0x7f0cc815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cc6d9d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc8161f90 0x7f0cc80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cbe59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0cc80d0510 0x7f0cc815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42960/0 (socket says 172.21.15.80:42960) 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cbe59c640 1 -- 172.21.15.80:0/4081693179 learned_addr learned my addr 172.21.15.80:0/4081693179 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cbe59c640 1 -- 172.21.15.80:0/4081693179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc8161f90 msgr2=0x7f0cc80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cbe59c640 1 --2- 172.21.15.80:0/4081693179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc8161f90 0x7f0cc80cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cbe59c640 1 -- 172.21.15.80:0/4081693179 >> v2:172.21.15.87:3300/0 conn(0x7f0cc815ea70 msgr2=0x7f0cc815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:52.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cbe59c640 1 --2- 172.21.15.80:0/4081693179 >> v2:172.21.15.87:3300/0 conn(0x7f0cc815ea70 0x7f0cc815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.829+0000 7f0cbe59c640 1 -- 172.21.15.80:0/4081693179 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc806df80 con 0x7f0cc80d0510 2024-04-25T13:42:52.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0cbe59c640 1 --2- 172.21.15.80:0/4081693179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80d0510 0x7f0cc815e530 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80d1260 tx=0x7f0cb001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b296c65ac718abaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0cc659c640 1 -- 172.21.15.80:0/4081693179 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00b1070 con 0x7f0cc80d0510 2024-04-25T13:42:52.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0cc659c640 1 -- 172.21.15.80:0/4081693179 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb00a8050 con 0x7f0cc80d0510 2024-04-25T13:42:52.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0cc659c640 1 -- 172.21.15.80:0/4081693179 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00ac860 con 0x7f0cc80d0510 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/4081693179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80d0510 msgr2=0x7f0cc815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/4081693179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80d0510 0x7f0cc815e530 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f0cc80d1260 tx=0x7f0cb001c460 comp rx=0 tx=0).stop 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/4081693179 shutdown_connections 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/4081693179 >> v2:172.21.15.87:3300/0 conn(0x7f0cc815ea70 0x7f0cc815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/4081693179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc80d0510 0x7f0cc815e530 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/4081693179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc8161f90 0x7f0cc80cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/4081693179 >> 172.21.15.80:0/4081693179 conn(0x7f0cc806d080 msgr2=0x7f0cc80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/4081693179 shutdown_connections 2024-04-25T13:42:52.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.833+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/4081693179 wait complete. 2024-04-25T13:42:52.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 Processor -- start 2024-04-25T13:42:52.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 -- start start 2024-04-25T13:42:52.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc80d0510 0x7f0cc815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc815ea70 0x7f0cc815d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cc8161f90 0x7f0cc8165240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc81447a0 con 0x7f0cc815ea70 2024-04-25T13:42:52.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc8144620 con 0x7f0cc8161f90 2024-04-25T13:42:52.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc80d0920 con 0x7f0cc80d0510 2024-04-25T13:42:52.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc815ea70 0x7f0cc815d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc815ea70 0x7f0cc815d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42962/0 (socket says 172.21.15.80:42962) 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 -- 172.21.15.80:0/1137907083 learned_addr learned my addr 172.21.15.80:0/1137907083 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cc759e640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cc8161f90 0x7f0cc8165240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cc6d9d640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc80d0510 0x7f0cc815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 -- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc80d0510 msgr2=0x7f0cc815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc80d0510 0x7f0cc815dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 -- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cc8161f90 msgr2=0x7f0cc8165240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cc8161f90 0x7f0cc8165240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 -- 172.21.15.80:0/1137907083 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc81678e0 con 0x7f0cc815ea70 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cc759e640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cc8161f90 0x7f0cc8165240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbe59c640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc815ea70 0x7f0cc815d1f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0cb001cd00 tx=0x7f0cb001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbffff640 1 -- 172.21.15.80:0/1137907083 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00b1040 con 0x7f0cc815ea70 2024-04-25T13:42:52.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbffff640 1 -- 172.21.15.80:0/1137907083 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cb00bc070 con 0x7f0cc815ea70 2024-04-25T13:42:52.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc8167bc0 con 0x7f0cc815ea70 2024-04-25T13:42:52.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0cbffff640 1 -- 172.21.15.80:0/1137907083 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb00b7450 con 0x7f0cc815ea70 2024-04-25T13:42:52.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.837+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc8163270 con 0x7f0cc815ea70 2024-04-25T13:42:52.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.841+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc806a3a0 con 0x7f0cc815ea70 2024-04-25T13:42:52.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.841+0000 7f0cbffff640 1 -- 172.21.15.80:0/1137907083 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0cb00b75f0 con 0x7f0cc815ea70 2024-04-25T13:42:52.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.841+0000 7f0cbffff640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ca806f400 0x7f0ca8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:52.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.841+0000 7f0cbffff640 1 -- 172.21.15.80:0/1137907083 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f0cb0129540 con 0x7f0cc815ea70 2024-04-25T13:42:52.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.841+0000 7f0cc6d9d640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ca806f400 0x7f0ca8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:52.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.841+0000 7f0cc6d9d640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ca806f400 0x7f0ca8071910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0cc806fa20 tx=0x7f0cb8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:52.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:52.845+0000 7f0cbffff640 1 -- 172.21.15.80:0/1137907083 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0cb0016110 con 0x7f0cc815ea70 2024-04-25T13:42:53.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cc8167d90 con 0x7f0cc815ea70 2024-04-25T13:42:53.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0cbffff640 1 -- 172.21.15.80:0/1137907083 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f0cb00f64c0 con 0x7f0cc815ea70 2024-04-25T13:42:53.036 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:53.037 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:42:53.037 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:42:53.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ca806f400 msgr2=0x7f0ca8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ca806f400 0x7f0ca8071910 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0cc806fa20 tx=0x7f0cb8008040 comp rx=0 tx=0).stop 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc815ea70 msgr2=0x7f0cc815d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc815ea70 0x7f0cc815d1f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0cb001cd00 tx=0x7f0cb001aca0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 shutdown_connections 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ca806f400 0x7f0ca8071910 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cc8161f90 0x7f0cc8165240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0cc815ea70 0x7f0cc815d1f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 --2- 172.21.15.80:0/1137907083 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0cc80d0510 0x7f0cc815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 >> 172.21.15.80:0/1137907083 conn(0x7f0cc806d080 msgr2=0x7f0cc815fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.033+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 shutdown_connections 2024-04-25T13:42:53.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:53.037+0000 7f0ccd1a0640 1 -- 172.21.15.80:0/1137907083 wait complete. 2024-04-25T13:42:53.057 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2024-04-25T13:42:53.224 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":5730,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":36.786242508000001,"mdsmap_epoch":24,"osdmap_epoch":52,"osdmap_epoch_barrier":50,"uptime":42.639587593999998} 2024-04-25T13:42:53.225 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 5730, "mdsmap_epoch": 24, "osdmap_epoch": 52, "osdmap_epoch_barrier": 50, "rank_uptime": 36.786242508, "state": "up:active", "uptime": 42.639587594, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:42:53.225 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:42:53.225 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:53.225 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:53.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:53.231 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-04-25T13:42:53.637 INFO:teuthology.orchestra.run.smithi080.stderr:50+0 records in 2024-04-25T13:42:53.637 INFO:teuthology.orchestra.run.smithi080.stderr:50+0 records out 2024-04-25T13:42:53.637 INFO:teuthology.orchestra.run.smithi080.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.354835 s, 148 MB/s 2024-04-25T13:42:53.639 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:42:53.639 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:42:53.701 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:42:53.744 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:42:53.744 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:42:53.745 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:53.745 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:53.751 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:53.751 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:53.804 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:54.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf9853c640 1 Processor -- start 2024-04-25T13:42:54.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf9853c640 1 -- start start 2024-04-25T13:42:54.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf9853c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbf9006db00 0x7fbf900c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf9853c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbf900c8a60 0x7fbf900c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf9853c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf90160b10 0x7fbf90160f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf9853c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbf900c93b0 con 0x7fbf9006db00 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf9853c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbf9006ece0 con 0x7fbf900c8a60 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf9853c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbf900d26f0 con 0x7fbf90160b10 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf962d8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbf9006db00 0x7fbf900c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf962d8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbf9006db00 0x7fbf900c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42964/0 (socket says 172.21.15.80:42964) 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf962d8640 1 -- 172.21.15.80:0/879076345 learned_addr learned my addr 172.21.15.80:0/879076345 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf95ad7640 1 --2- 172.21.15.80:0/879076345 >> v2:172.21.15.87:3300/0 conn(0x7fbf900c8a60 0x7fbf900c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf962d8640 1 -- 172.21.15.80:0/879076345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf90160b10 msgr2=0x7fbf90160f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf962d8640 1 --2- 172.21.15.80:0/879076345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf90160b10 0x7fbf90160f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.001+0000 7fbf962d8640 1 -- 172.21.15.80:0/879076345 >> v2:172.21.15.87:3300/0 conn(0x7fbf900c8a60 msgr2=0x7fbf900c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf962d8640 1 --2- 172.21.15.80:0/879076345 >> v2:172.21.15.87:3300/0 conn(0x7fbf900c8a60 0x7fbf900c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf962d8640 1 -- 172.21.15.80:0/879076345 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf9006df10 con 0x7fbf9006db00 2024-04-25T13:42:54.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf95ad7640 1 --2- 172.21.15.80:0/879076345 >> v2:172.21.15.87:3300/0 conn(0x7fbf900c8a60 0x7fbf900c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:54.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf962d8640 1 --2- 172.21.15.80:0/879076345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf9006db00 0x7fbf900c8520 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbf90072600 tx=0x7fbf8c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2abb9ef65ef655 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf952d6640 1 -- 172.21.15.80:0/879076345 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0aa610 con 0x7fbf9006db00 2024-04-25T13:42:54.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf952d6640 1 -- 172.21.15.80:0/879076345 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8c0af070 con 0x7fbf9006db00 2024-04-25T13:42:54.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf952d6640 1 -- 172.21.15.80:0/879076345 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf8c0aa9d0 con 0x7fbf9006db00 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 -- 172.21.15.80:0/879076345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf9006db00 msgr2=0x7fbf900c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 --2- 172.21.15.80:0/879076345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf9006db00 0x7fbf900c8520 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbf90072600 tx=0x7fbf8c01c4f0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 -- 172.21.15.80:0/879076345 shutdown_connections 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 --2- 172.21.15.80:0/879076345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf90160b10 0x7fbf90160f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 --2- 172.21.15.80:0/879076345 >> v2:172.21.15.87:3300/0 conn(0x7fbf900c8a60 0x7fbf900c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 --2- 172.21.15.80:0/879076345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf9006db00 0x7fbf900c8520 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 -- 172.21.15.80:0/879076345 >> 172.21.15.80:0/879076345 conn(0x7fbf900cf390 msgr2=0x7fbf900cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 -- 172.21.15.80:0/879076345 shutdown_connections 2024-04-25T13:42:54.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.005+0000 7fbf9853c640 1 -- 172.21.15.80:0/879076345 wait complete. 2024-04-25T13:42:54.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 Processor -- start 2024-04-25T13:42:54.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 -- start start 2024-04-25T13:42:54.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf9006db00 0x7fbf900ca230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf900c8a60 0x7fbf901f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf90160b10 0x7fbf90068590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbf90161870 con 0x7fbf90160b10 2024-04-25T13:42:54.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbf900d2410 con 0x7fbf900c8a60 2024-04-25T13:42:54.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbf901619f0 con 0x7fbf9006db00 2024-04-25T13:42:54.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf90160b10 0x7fbf90068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf90160b10 0x7fbf90068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42978/0 (socket says 172.21.15.80:42978) 2024-04-25T13:42:54.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 -- 172.21.15.80:0/305268813 learned_addr learned my addr 172.21.15.80:0/305268813 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:54.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf95ad7640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf900c8a60 0x7fbf901f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf962d8640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf9006db00 0x7fbf900ca230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 -- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf9006db00 msgr2=0x7fbf900ca230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf9006db00 0x7fbf900ca230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 -- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf900c8a60 msgr2=0x7fbf901f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf900c8a60 0x7fbf901f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 -- 172.21.15.80:0/305268813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf900cb400 con 0x7fbf90160b10 2024-04-25T13:42:54.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf96ad9640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf90160b10 0x7fbf90068590 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800ba10 tx=0x7fbf8800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf86ffd640 1 -- 172.21.15.80:0/305268813 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88019070 con 0x7fbf90160b10 2024-04-25T13:42:54.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf86ffd640 1 -- 172.21.15.80:0/305268813 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8800cb60 con 0x7fbf90160b10 2024-04-25T13:42:54.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf900cb6e0 con 0x7fbf90160b10 2024-04-25T13:42:54.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf86ffd640 1 -- 172.21.15.80:0/305268813 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88014440 con 0x7fbf90160b10 2024-04-25T13:42:54.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.009+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf900cbca0 con 0x7fbf90160b10 2024-04-25T13:42:54.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.013+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf900cb170 con 0x7fbf90160b10 2024-04-25T13:42:54.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.013+0000 7fbf86ffd640 1 -- 172.21.15.80:0/305268813 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbf8801c070 con 0x7fbf90160b10 2024-04-25T13:42:54.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.013+0000 7fbf86ffd640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbf7406f400 0x7fbf74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.013+0000 7fbf86ffd640 1 -- 172.21.15.80:0/305268813 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fbf8808ef10 con 0x7fbf90160b10 2024-04-25T13:42:54.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.013+0000 7fbf962d8640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbf7406f400 0x7fbf74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.013+0000 7fbf962d8640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbf7406f400 0x7fbf74071910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0ae000 tx=0x7fbf8c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.017+0000 7fbf86ffd640 1 -- 172.21.15.80:0/305268813 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbf8805be90 con 0x7fbf90160b10 2024-04-25T13:42:54.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.177+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf9006a3a0 con 0x7fbf90160b10 2024-04-25T13:42:54.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.181+0000 7fbf86ffd640 1 -- 172.21.15.80:0/305268813 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16541 (secure 0 0 0) 0x7fbf8805b790 con 0x7fbf90160b10 2024-04-25T13:42:54.185 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:54.185 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":52,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:49.227779+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:42:13.650442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T13:42:14.725209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.181+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbf7406f400 msgr2=0x7fbf74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.181+0000 7fbf9853c640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbf7406f400 0x7fbf74071910 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c0ae000 tx=0x7fbf8c0a7000 comp rx=0 tx=0).stop 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.181+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf90160b10 msgr2=0x7fbf90068590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.181+0000 7fbf9853c640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf90160b10 0x7fbf90068590 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800ba10 tx=0x7fbf8800bed0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.185+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 shutdown_connections 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.185+0000 7fbf9853c640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbf7406f400 0x7fbf74071910 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.185+0000 7fbf9853c640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbf90160b10 0x7fbf90068590 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.185+0000 7fbf9853c640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbf900c8a60 0x7fbf901f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.185+0000 7fbf9853c640 1 --2- 172.21.15.80:0/305268813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbf9006db00 0x7fbf900ca230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.185+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 >> 172.21.15.80:0/305268813 conn(0x7fbf900cf390 msgr2=0x7fbf900b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.185+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 shutdown_connections 2024-04-25T13:42:54.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.185+0000 7fbf9853c640 1 -- 172.21.15.80:0/305268813 wait complete. 2024-04-25T13:42:54.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:42:13.650442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:42:14.725209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:42:54.207 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:42:54.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0fa5e2b640 1 Processor -- start 2024-04-25T13:42:54.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0fa5e2b640 1 -- start start 2024-04-25T13:42:54.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0fa5e2b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0fa00d3ed0 0x7f0fa00d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0fa5e2b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0fa00d2ad0 0x7f0fa00b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0fa5e2b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00b7c10 0x7f0fa00b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0fa5e2b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa00b8560 con 0x7f0fa00d3ed0 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0fa5e2b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0fa00b92a0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0fa5e2b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa00d4c10 con 0x7f0fa00b7c10 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0fa00d3ed0 0x7f0fa00d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0fa00d3ed0 0x7f0fa00d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42994/0 (socket says 172.21.15.80:42994) 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9f7fe640 1 -- 172.21.15.80:0/2750932368 learned_addr learned my addr 172.21.15.80:0/2750932368 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9effd640 1 --2- 172.21.15.80:0/2750932368 >> v2:172.21.15.87:3300/0 conn(0x7f0fa00d2ad0 0x7f0fa00b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9ffff640 1 --2- 172.21.15.80:0/2750932368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00b7c10 0x7f0fa00b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9f7fe640 1 -- 172.21.15.80:0/2750932368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00b7c10 msgr2=0x7f0fa00b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9f7fe640 1 --2- 172.21.15.80:0/2750932368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00b7c10 0x7f0fa00b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9f7fe640 1 -- 172.21.15.80:0/2750932368 >> v2:172.21.15.87:3300/0 conn(0x7f0fa00d2ad0 msgr2=0x7f0fa00b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9f7fe640 1 --2- 172.21.15.80:0/2750932368 >> v2:172.21.15.87:3300/0 conn(0x7f0fa00d2ad0 0x7f0fa00b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.357+0000 7f0f9f7fe640 1 -- 172.21.15.80:0/2750932368 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa00d4ef0 con 0x7f0fa00d3ed0 2024-04-25T13:42:54.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0f9f7fe640 1 --2- 172.21.15.80:0/2750932368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d3ed0 0x7f0fa00d2590 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00b8dc0 tx=0x7f0f9001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283da1d9aba3c1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0f9e7fc640 1 -- 172.21.15.80:0/2750932368 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900b2070 con 0x7f0fa00d3ed0 2024-04-25T13:42:54.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0f9e7fc640 1 -- 172.21.15.80:0/2750932368 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f900ad640 con 0x7f0fa00d3ed0 2024-04-25T13:42:54.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0f9e7fc640 1 -- 172.21.15.80:0/2750932368 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f900ada10 con 0x7f0fa00d3ed0 2024-04-25T13:42:54.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/2750932368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d3ed0 msgr2=0x7f0fa00d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/2750932368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d3ed0 0x7f0fa00d2590 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0fa00b8dc0 tx=0x7f0f9001c3d0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/2750932368 shutdown_connections 2024-04-25T13:42:54.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/2750932368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00b7c10 0x7f0fa00b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/2750932368 >> v2:172.21.15.87:3300/0 conn(0x7f0fa00d2ad0 0x7f0fa00b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/2750932368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d3ed0 0x7f0fa00d2590 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/2750932368 >> 172.21.15.80:0/2750932368 conn(0x7f0fa00737b0 msgr2=0x7f0fa006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/2750932368 shutdown_connections 2024-04-25T13:42:54.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/2750932368 wait complete. 2024-04-25T13:42:54.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 Processor -- start 2024-04-25T13:42:54.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.361+0000 7f0fa5e2b640 1 -- start start 2024-04-25T13:42:54.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fa00b7c10 0x7f0fa0160d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d2ad0 0x7f0fa015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00d3ed0 0x7f0fa0158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00ccae0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa00cc960 con 0x7f0fa00b7c10 2024-04-25T13:42:54.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0fa0160620 con 0x7f0fa00d3ed0 2024-04-25T13:42:54.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d2ad0 0x7f0fa015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d2ad0 0x7f0fa015a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42996/0 (socket says 172.21.15.80:42996) 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 -- 172.21.15.80:0/3706028030 learned_addr learned my addr 172.21.15.80:0/3706028030 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9f7fe640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fa00b7c10 0x7f0fa0160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 -- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00d3ed0 msgr2=0x7f0fa0158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00d3ed0 0x7f0fa0158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 -- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fa00b7c10 msgr2=0x7f0fa0160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fa00b7c10 0x7f0fa0160d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 -- 172.21.15.80:0/3706028030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa0161710 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9f7fe640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fa00b7c10 0x7f0fa0160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f9effd640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d2ad0 0x7f0fa015a020 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400d5f0 tx=0x7f0f9400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f87fff640 1 -- 172.21.15.80:0/3706028030 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f940125f0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f87fff640 1 -- 172.21.15.80:0/3706028030 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f94019070 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa01619f0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0f87fff640 1 -- 172.21.15.80:0/3706028030 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f940129b0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa0161fb0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.365+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa0161480 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.369+0000 7f0f87fff640 1 -- 172.21.15.80:0/3706028030 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0f94012b50 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.369+0000 7f0f87fff640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f8c06f430 0x7f0f8c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.369+0000 7f0f87fff640 1 -- 172.21.15.80:0/3706028030 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f0f9408f660 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.369+0000 7f0f9f7fe640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f8c06f430 0x7f0f8c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.369+0000 7f0f9f7fe640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f8c06f430 0x7f0f8c071940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0f900af000 tx=0x7f0f900a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.373+0000 7f0f87fff640 1 -- 172.21.15.80:0/3706028030 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0f9405c5e0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.533+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0fa006a3a0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0f87fff640 1 -- 172.21.15.80:0/3706028030 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f0f9405bee0 con 0x7f0fa00d2ad0 2024-04-25T13:42:54.539 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:42:54.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f8c06f430 msgr2=0x7f0f8c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f8c06f430 0x7f0f8c071940 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0f900af000 tx=0x7f0f900a7070 comp rx=0 tx=0).stop 2024-04-25T13:42:54.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d2ad0 msgr2=0x7f0fa015a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d2ad0 0x7f0fa015a020 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0f9400d5f0 tx=0x7f0f9400dab0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 shutdown_connections 2024-04-25T13:42:54.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0f8c06f430 0x7f0f8c071940 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fa00d3ed0 0x7f0fa0158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fa00d2ad0 0x7f0fa015a020 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 --2- 172.21.15.80:0/3706028030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fa00b7c10 0x7f0fa0160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 >> 172.21.15.80:0/3706028030 conn(0x7f0fa00737b0 msgr2=0x7f0fa015df60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 shutdown_connections 2024-04-25T13:42:54.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.537+0000 7f0fa5e2b640 1 -- 172.21.15.80:0/3706028030 wait complete. 2024-04-25T13:42:54.561 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:42:54.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f522ae9a640 1 Processor -- start 2024-04-25T13:42:54.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f522ae9a640 1 -- start start 2024-04-25T13:42:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f522ae9a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f52240d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f522ae9a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5224161b60 0x7f5224161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f522ae9a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52241624b0 con 0x7f52240d0ff0 2024-04-25T13:42:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f522ae9a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f52241629b0 con 0x7f5224161b60 2024-04-25T13:42:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f522ae9a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52240b5740 con 0x7f52240b5330 2024-04-25T13:42:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f5229437640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5224161b60 0x7f5224161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f5223fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f52240d0ff0 0x7f522406edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52350/0 (socket says 172.21.15.80:52350) 2024-04-25T13:42:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f5223fff640 1 -- 172.21.15.80:0/3367324680 learned_addr learned my addr 172.21.15.80:0/3367324680 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f5228c36640 1 --2- 172.21.15.80:0/3367324680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f52240d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f5229437640 1 -- 172.21.15.80:0/3367324680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 msgr2=0x7f52240d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f5229437640 1 --2- 172.21.15.80:0/3367324680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f52240d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f5229437640 1 -- 172.21.15.80:0/3367324680 >> v1:172.21.15.80:6789/0 conn(0x7f52240d0ff0 legacy=0x7f522406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f5229437640 1 -- 172.21.15.80:0/3367324680 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f522406dd10 con 0x7f5224161b60 2024-04-25T13:42:54.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.721+0000 7f52237fe640 1 -- 172.21.15.80:0/3367324680 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2219778762 0 0) 0x7f5218001120 con 0x7f52240d0ff0 2024-04-25T13:42:54.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f5229437640 1 --2- 172.21.15.80:0/3367324680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f5224161f70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f521c009d90 tx=0x7f521c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278d8cdaddff3287 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f52237fe640 1 -- 172.21.15.80:0/3367324680 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c0b2070 con 0x7f5224161b60 2024-04-25T13:42:54.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f52237fe640 1 -- 172.21.15.80:0/3367324680 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c0a5890 con 0x7f5224161b60 2024-04-25T13:42:54.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f52237fe640 1 -- 172.21.15.80:0/3367324680 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c0a5c60 con 0x7f5224161b60 2024-04-25T13:42:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f522ae9a640 1 -- 172.21.15.80:0/3367324680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 msgr2=0x7f5224161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f522ae9a640 1 --2- 172.21.15.80:0/3367324680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f5224161f70 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f521c009d90 tx=0x7f521c01c8e0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f522ae9a640 1 -- 172.21.15.80:0/3367324680 shutdown_connections 2024-04-25T13:42:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f522ae9a640 1 --2- 172.21.15.80:0/3367324680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f5224161f70 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f522ae9a640 1 --2- 172.21.15.80:0/3367324680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f52240d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f522ae9a640 1 -- 172.21.15.80:0/3367324680 >> 172.21.15.80:0/3367324680 conn(0x7f52240b4160 msgr2=0x7f52240d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f522ae9a640 1 -- 172.21.15.80:0/3367324680 shutdown_connections 2024-04-25T13:42:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.725+0000 7f522ae9a640 1 -- 172.21.15.80:0/3367324680 wait complete. 2024-04-25T13:42:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 Processor -- start 2024-04-25T13:42:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 -- start start 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f522415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52240d0ff0 0x7f522413f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f522415c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f52240b0cd0 con 0x7f5224161b60 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f522406e890 con 0x7f52240d0ff0 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52240b0e50 con 0x7f52240b5330 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f522415c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f522415c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43012/0 (socket says 172.21.15.80:43012) 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 -- 172.21.15.80:0/452332950 learned_addr learned my addr 172.21.15.80:0/452332950 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5223fff640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52240d0ff0 0x7f522413f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5228c36640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f522415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 -- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 msgr2=0x7f522415e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f522415e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 -- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52240d0ff0 msgr2=0x7f522413f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52240d0ff0 0x7f522413f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 -- 172.21.15.80:0/452332950 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5224156b20 con 0x7f5224161b60 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5228c36640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f522415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f5229437640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f522415c980 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f521c01af90 tx=0x7f521c019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f52217fa640 1 -- 172.21.15.80:0/452332950 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c0b2050 con 0x7f5224161b60 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f52217fa640 1 -- 172.21.15.80:0/452332950 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f521c0be070 con 0x7f5224161b60 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5224156e00 con 0x7f5224161b60 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f52217fa640 1 -- 172.21.15.80:0/452332950 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f521c0ada50 con 0x7f5224161b60 2024-04-25T13:42:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.729+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52241573c0 con 0x7f5224161b60 2024-04-25T13:42:54.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.733+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5224156890 con 0x7f5224161b60 2024-04-25T13:42:54.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.733+0000 7f52217fa640 1 -- 172.21.15.80:0/452332950 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f521c0a5710 con 0x7f5224161b60 2024-04-25T13:42:54.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.733+0000 7f52217fa640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f521006f400 0x7f5210071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:54.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.733+0000 7f52217fa640 1 -- 172.21.15.80:0/452332950 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f521c12aeb0 con 0x7f5224161b60 2024-04-25T13:42:54.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.733+0000 7f5228c36640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f521006f400 0x7f5210071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:54.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.733+0000 7f5228c36640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f521006f400 0x7f5210071910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f52140097a0 tx=0x7f5214008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:54.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.737+0000 7f52217fa640 1 -- 172.21.15.80:0/452332950 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f521c016560 con 0x7f5224161b60 2024-04-25T13:42:54.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.897+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f522406a3a0 con 0x7f5224161b60 2024-04-25T13:42:54.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.897+0000 7f52217fa640 1 -- 172.21.15.80:0/452332950 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) ==== 104+0+11 (secure 0 0 0) 0x7f521c0f7f10 con 0x7f5224161b60 2024-04-25T13:42:54.900 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f521006f400 msgr2=0x7f5210071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f521006f400 0x7f5210071910 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f52140097a0 tx=0x7f5214008040 comp rx=0 tx=0).stop 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 msgr2=0x7f522415c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f522415c980 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f521c01af90 tx=0x7f521c019680 comp rx=0 tx=0).stop 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 shutdown_connections 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f521006f400 0x7f5210071910 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5224161b60 0x7f522415c980 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52240d0ff0 0x7f522413f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 --2- 172.21.15.80:0/452332950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52240b5330 0x7f522415e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:54.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 >> 172.21.15.80:0/452332950 conn(0x7f52240b4160 msgr2=0x7f52240d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:54.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 shutdown_connections 2024-04-25T13:42:54.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:54.901+0000 7f522ae9a640 1 -- 172.21.15.80:0/452332950 wait complete. 2024-04-25T13:42:54.922 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:42:55.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ed1e08640 1 Processor -- start 2024-04-25T13:42:55.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ed1e08640 1 -- start start 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ed1e08640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ecc0b5330 0x7f5ecc0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ed1e08640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc0d0ff0 0x7f5ecc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ed1e08640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5ecc06f2f0 0x7f5ecc161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ed1e08640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5ecc06f700 con 0x7f5ecc0b5330 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ed1e08640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5ecc1629b0 con 0x7f5ecc06f2f0 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ed1e08640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b5740 con 0x7f5ecc0d0ff0 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ecb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ecc0b5330 0x7f5ecc0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ecb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ecc0b5330 0x7f5ecc0d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43022/0 (socket says 172.21.15.80:43022) 2024-04-25T13:42:55.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.069+0000 7f5ecb7fe640 1 -- 172.21.15.80:0/3196091662 learned_addr learned my addr 172.21.15.80:0/3196091662 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:55.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ecbfff640 1 --2- 172.21.15.80:0/3196091662 >> v2:172.21.15.87:3300/0 conn(0x7f5ecc06f2f0 0x7f5ecc161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ecaffd640 1 --2- 172.21.15.80:0/3196091662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc0d0ff0 0x7f5ecc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ecb7fe640 1 -- 172.21.15.80:0/3196091662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc0d0ff0 msgr2=0x7f5ecc06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:55.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ecb7fe640 1 --2- 172.21.15.80:0/3196091662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc0d0ff0 0x7f5ecc06edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ecb7fe640 1 -- 172.21.15.80:0/3196091662 >> v2:172.21.15.87:3300/0 conn(0x7f5ecc06f2f0 msgr2=0x7f5ecc161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:55.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ecb7fe640 1 --2- 172.21.15.80:0/3196091662 >> v2:172.21.15.87:3300/0 conn(0x7f5ecc06f2f0 0x7f5ecc161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ecb7fe640 1 -- 172.21.15.80:0/3196091662 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc06dd10 con 0x7f5ecc0b5330 2024-04-25T13:42:55.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ecb7fe640 1 --2- 172.21.15.80:0/3196091662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc0d0ab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0b1460 tx=0x7f5ebc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac2284b96c0f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:55.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5eca7fc640 1 -- 172.21.15.80:0/3196091662 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0b0070 con 0x7f5ecc0b5330 2024-04-25T13:42:55.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5eca7fc640 1 -- 172.21.15.80:0/3196091662 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ebc0ab600 con 0x7f5ecc0b5330 2024-04-25T13:42:55.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5eca7fc640 1 -- 172.21.15.80:0/3196091662 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0ab9d0 con 0x7f5ecc0b5330 2024-04-25T13:42:55.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3196091662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 msgr2=0x7f5ecc0d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3196091662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc0d0ab0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc0b1460 tx=0x7f5ebc01c490 comp rx=0 tx=0).stop 2024-04-25T13:42:55.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3196091662 shutdown_connections 2024-04-25T13:42:55.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3196091662 >> v2:172.21.15.87:3300/0 conn(0x7f5ecc06f2f0 0x7f5ecc161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3196091662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc0d0ff0 0x7f5ecc06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3196091662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc0d0ab0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.073+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3196091662 >> 172.21.15.80:0/3196091662 conn(0x7f5ecc0b4160 msgr2=0x7f5ecc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:55.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3196091662 shutdown_connections 2024-04-25T13:42:55.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3196091662 wait complete. 2024-04-25T13:42:55.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 Processor -- start 2024-04-25T13:42:55.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 -- start start 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc06f2f0 0x7f5ecc138d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc13b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ecc0d0ff0 0x7f5ecc13ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b0cd0 con 0x7f5ecc0b5330 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc06e890 con 0x7f5ecc0d0ff0 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ed1e08640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5ecc0b0e50 con 0x7f5ecc06f2f0 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc13b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc13b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43034/0 (socket says 172.21.15.80:43034) 2024-04-25T13:42:55.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 -- 172.21.15.80:0/3787201809 learned_addr learned my addr 172.21.15.80:0/3787201809 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecb7fe640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc06f2f0 0x7f5ecc138d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecbfff640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ecc0d0ff0 0x7f5ecc13ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 -- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc06f2f0 msgr2=0x7f5ecc138d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc06f2f0 0x7f5ecc138d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 -- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ecc0d0ff0 msgr2=0x7f5ecc13ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ecc0d0ff0 0x7f5ecc13ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 -- 172.21.15.80:0/3787201809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc0b2ed0 con 0x7f5ecc0b5330 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.077+0000 7f5ecaffd640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc13b550 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000d620 tx=0x7f5ec000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5eb3fff640 1 -- 172.21.15.80:0/3787201809 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000b620 con 0x7f5ecc0b5330 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5eb3fff640 1 -- 172.21.15.80:0/3787201809 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ec0019070 con 0x7f5ecc0b5330 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc156fb0 con 0x7f5ecc0b5330 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5eb3fff640 1 -- 172.21.15.80:0/3787201809 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec000b9e0 con 0x7f5ecc0b5330 2024-04-25T13:42:55.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc157570 con 0x7f5ecc0b5330 2024-04-25T13:42:55.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ecc06a3a0 con 0x7f5ecc0b5330 2024-04-25T13:42:55.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5eb3fff640 1 -- 172.21.15.80:0/3787201809 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ec000bb80 con 0x7f5ecc0b5330 2024-04-25T13:42:55.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5eb3fff640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5eb806f400 0x7f5eb8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5eb3fff640 1 -- 172.21.15.80:0/3787201809 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f5ec008f6d0 con 0x7f5ecc0b5330 2024-04-25T13:42:55.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5ecb7fe640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5eb806f400 0x7f5eb8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.081+0000 7f5ecb7fe640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5eb806f400 0x7f5eb8071910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0ad000 tx=0x7f5ebc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:55.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.085+0000 7f5eb3fff640 1 -- 172.21.15.80:0/3787201809 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ec005c650 con 0x7f5ecc0b5330 2024-04-25T13:42:55.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.241+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5ecc06fd00 con 0x7f5ecc0b5330 2024-04-25T13:42:55.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.241+0000 7f5eb3fff640 1 -- 172.21.15.80:0/3787201809 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) ==== 100+0+11 (secure 0 0 0) 0x7f5ec005bf50 con 0x7f5ecc0b5330 2024-04-25T13:42:55.245 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5eb806f400 msgr2=0x7f5eb8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5eb806f400 0x7f5eb8071910 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc0ad000 tx=0x7f5ebc008040 comp rx=0 tx=0).stop 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 msgr2=0x7f5ecc13b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc13b550 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000d620 tx=0x7f5ec000dae0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 shutdown_connections 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5eb806f400 0x7f5eb8071910 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ecc0d0ff0 0x7f5ecc13ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ecc0b5330 0x7f5ecc13b550 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 --2- 172.21.15.80:0/3787201809 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ecc06f2f0 0x7f5ecc138d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 >> 172.21.15.80:0/3787201809 conn(0x7f5ecc0b4160 msgr2=0x7f5ecc0b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:55.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 shutdown_connections 2024-04-25T13:42:55.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.245+0000 7f5ed1e08640 1 -- 172.21.15.80:0/3787201809 wait complete. 2024-04-25T13:42:55.265 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:42:55.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.409+0000 7efe8f9a0640 1 Processor -- start 2024-04-25T13:42:55.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.409+0000 7efe8f9a0640 1 -- start start 2024-04-25T13:42:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8f9a0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe880b2020 0x7efe88068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8f9a0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe88164c00 con 0x7efe88149410 2024-04-25T13:42:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8f9a0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe8806caf0 con 0x7efe880702b0 2024-04-25T13:42:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8f9a0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efe8806cdd0 con 0x7efe880b2020 2024-04-25T13:42:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8cf3b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7efe88149410 0x7efe88163de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52352/0 (socket says 172.21.15.80:52352) 2024-04-25T13:42:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8cf3b640 1 -- 172.21.15.80:0/859658996 learned_addr learned my addr 172.21.15.80:0/859658996 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8d73c640 1 --2- 172.21.15.80:0/859658996 >> v2:172.21.15.87:3300/0 conn(0x7efe880b2020 0x7efe88068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2505501748 0 0) 0x7efe88164c00 con 0x7efe88149410 2024-04-25T13:42:55.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efe74003530 con 0x7efe88149410 2024-04-25T13:42:55.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 254427898 0 0) 0x7efe74003530 con 0x7efe88149410 2024-04-25T13:42:55.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 >> v2:172.21.15.87:3300/0 conn(0x7efe880b2020 msgr2=0x7efe88068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:55.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 --2- 172.21.15.80:0/859658996 >> v2:172.21.15.87:3300/0 conn(0x7efe880b2020 0x7efe88068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 >> v1:172.21.15.87:6789/0 conn(0x7efe880702b0 legacy=0x7efe881644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8806d0b0 con 0x7efe88149410 2024-04-25T13:42:55.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7efe78001680 con 0x7efe88149410 2024-04-25T13:42:55.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 >> v1:172.21.15.80:6789/0 conn(0x7efe88149410 legacy=0x7efe88163de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe740039c0 0x7efe74003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe74004810 0x7efe74004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe74005280 0x7efe7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efe8806cdd0 con 0x7efe740039c0 2024-04-25T13:42:55.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe7400c210 con 0x7efe74005280 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe7400c410 con 0x7efe74004810 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8d73c640 1 --2- 172.21.15.80:0/859658996 >> v2:172.21.15.87:3300/0 conn(0x7efe880b2020 0x7efe88068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8d73c640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe740039c0 0x7efe74003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8cf3b640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe74004810 0x7efe74004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8df3d640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe74005280 0x7efe7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8d73c640 1 -- 172.21.15.80:0/859658996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe74004810 msgr2=0x7efe74004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8d73c640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe74004810 0x7efe74004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8d73c640 1 -- 172.21.15.80:0/859658996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe74005280 msgr2=0x7efe7400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8d73c640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe74005280 0x7efe7400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8d73c640 1 -- 172.21.15.80:0/859658996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe74003530 con 0x7efe740039c0 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.413+0000 7efe8df3d640 1 -- 172.21.15.80:0/859658996 reap_dead start 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8d73c640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe740039c0 0x7efe74003e30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efe840098b0 tx=0x7efe8401c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ae45591ddd4f09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8400f040 con 0x7efe740039c0 2024-04-25T13:42:55.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe840a6040 con 0x7efe740039c0 2024-04-25T13:42:55.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe7ffff640 1 -- 172.21.15.80:0/859658996 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe84004240 con 0x7efe740039c0 2024-04-25T13:42:55.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 -- 172.21.15.80:0/859658996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe740039c0 msgr2=0x7efe74003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe740039c0 0x7efe74003e30 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7efe840098b0 tx=0x7efe8401c580 comp rx=0 tx=0).stop 2024-04-25T13:42:55.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 -- 172.21.15.80:0/859658996 shutdown_connections 2024-04-25T13:42:55.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 --2- 172.21.15.80:0/859658996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe740039c0 0x7efe74003e30 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 -- 172.21.15.80:0/859658996 >> 172.21.15.80:0/859658996 conn(0x7efe880d28f0 msgr2=0x7efe880d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:55.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 -- 172.21.15.80:0/859658996 shutdown_connections 2024-04-25T13:42:55.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 -- 172.21.15.80:0/859658996 wait complete. 2024-04-25T13:42:55.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 Processor -- start 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.417+0000 7efe8f9a0640 1 -- start start 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe740039c0 0x7efe8815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe881fcfe0 0x7efe881fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880b08c0 0x7efe880b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efe881641b0 con 0x7efe881fcfe0 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe88164030 con 0x7efe740039c0 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe88164330 con 0x7efe880b08c0 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe881fcfe0 0x7efe881fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe881fcfe0 0x7efe881fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43054/0 (socket says 172.21.15.80:43054) 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 -- 172.21.15.80:0/727525019 learned_addr learned my addr 172.21.15.80:0/727525019 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8d73c640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe740039c0 0x7efe8815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8df3d640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880b08c0 0x7efe880b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 -- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880b08c0 msgr2=0x7efe880b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880b08c0 0x7efe880b0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 -- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe740039c0 msgr2=0x7efe8815e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe740039c0 0x7efe8815e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 -- 172.21.15.80:0/727525019 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe880b1210 con 0x7efe881fcfe0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8d73c640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe740039c0 0x7efe8815e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8cf3b640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe881fcfe0 0x7efe881fd3f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efe7800d830 tx=0x7efe7800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe7dffb640 1 -- 172.21.15.80:0/727525019 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78019070 con 0x7efe881fcfe0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe7dffb640 1 -- 172.21.15.80:0/727525019 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe780074e0 con 0x7efe881fcfe0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe881f9a90 con 0x7efe881fcfe0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe7dffb640 1 -- 172.21.15.80:0/727525019 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe780049f0 con 0x7efe881fcfe0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe881fa050 con 0x7efe881fcfe0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8806a3a0 con 0x7efe881fcfe0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe7dffb640 1 -- 172.21.15.80:0/727525019 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe78004b90 con 0x7efe881fcfe0 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe7dffb640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6c06f400 0x7efe6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:55.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe7dffb640 1 -- 172.21.15.80:0/727525019 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7efe7808f2c0 con 0x7efe881fcfe0 2024-04-25T13:42:55.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.421+0000 7efe8d73c640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6c06f400 0x7efe6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:55.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.425+0000 7efe8d73c640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6c06f400 0x7efe6c071910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe840ad000 tx=0x7efe840046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:55.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.425+0000 7efe7dffb640 1 -- 172.21.15.80:0/727525019 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe78022020 con 0x7efe881fcfe0 2024-04-25T13:42:55.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.601+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efe881491d0 con 0x7efe881fcfe0 2024-04-25T13:42:55.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.601+0000 7efe7dffb640 1 -- 172.21.15.80:0/727525019 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7efe880b1210 con 0x7efe881fcfe0 2024-04-25T13:42:55.604 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.601+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6c06f400 msgr2=0x7efe6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.601+0000 7efe8f9a0640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6c06f400 0x7efe6c071910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7efe840ad000 tx=0x7efe840046d0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.601+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe881fcfe0 msgr2=0x7efe881fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.601+0000 7efe8f9a0640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe881fcfe0 0x7efe881fd3f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7efe7800d830 tx=0x7efe7800dcf0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.601+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 shutdown_connections 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.605+0000 7efe8f9a0640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6c06f400 0x7efe6c071910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.605+0000 7efe8f9a0640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880b08c0 0x7efe880b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.605+0000 7efe8f9a0640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe881fcfe0 0x7efe881fd3f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.605+0000 7efe8f9a0640 1 --2- 172.21.15.80:0/727525019 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe740039c0 0x7efe8815e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.605+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 >> 172.21.15.80:0/727525019 conn(0x7efe880d28f0 msgr2=0x7efe880b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:55.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.605+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 shutdown_connections 2024-04-25T13:42:55.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:55.605+0000 7efe8f9a0640 1 -- 172.21.15.80:0/727525019 wait complete. 2024-04-25T13:42:55.626 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:42:55.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:55.626 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:42:55.639 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[29981]: fuse finished with error 0 and tester_r 0 2024-04-25T13:42:55.643 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:42:55.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:55.643 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:42:55.808 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:42:55.808 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:42:55.808 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:55.815 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:42:55.815 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:55.863 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:56.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.005+0000 7f2928917640 1 Processor -- start 2024-04-25T13:42:56.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.005+0000 7f2928917640 1 -- start start 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2928917640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2924161f90 0x7f29240cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2928917640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f29240d0510 0x7f292415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2928917640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29240d0d70 con 0x7f292415ea70 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2928917640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f29240d1050 con 0x7f29240d0510 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2928917640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f292406dca0 con 0x7f2924161f90 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2921d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f29240d0510 0x7f292415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2922d9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f292415ea70 0x7f292415ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52356/0 (socket says 172.21.15.80:52356) 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2921d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f29240d0510 0x7f292415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43062/0 (socket says 172.21.15.80:43062) 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2922d9d640 1 -- 172.21.15.80:0/155837918 learned_addr learned my addr 172.21.15.80:0/155837918 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f292259c640 1 --2- 172.21.15.80:0/155837918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2924161f90 0x7f29240cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2921d9b640 1 -- 172.21.15.80:0/155837918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2924161f90 msgr2=0x7f29240cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:56.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2921d9b640 1 --2- 172.21.15.80:0/155837918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2924161f90 0x7f29240cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2921d9b640 1 -- 172.21.15.80:0/155837918 >> v1:172.21.15.80:6789/0 conn(0x7f292415ea70 legacy=0x7f292415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f2921d9b640 1 -- 172.21.15.80:0/155837918 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292406df80 con 0x7f29240d0510 2024-04-25T13:42:56.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f292159a640 1 -- 172.21.15.80:0/155837918 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3786056082 0 0) 0x7f29240d0d70 con 0x7f292415ea70 2024-04-25T13:42:56.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.009+0000 7f292259c640 1 --2- 172.21.15.80:0/155837918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2924161f90 0x7f29240cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:56.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2921d9b640 1 --2- 172.21.15.80:0/155837918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f292415e530 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f29180097c0 tx=0x7f291801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ace6ce181638ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f292159a640 1 -- 172.21.15.80:0/155837918 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29180b0070 con 0x7f29240d0510 2024-04-25T13:42:56.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f292159a640 1 -- 172.21.15.80:0/155837918 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29180ab600 con 0x7f29240d0510 2024-04-25T13:42:56.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f292159a640 1 -- 172.21.15.80:0/155837918 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29180ab9d0 con 0x7f29240d0510 2024-04-25T13:42:56.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 -- 172.21.15.80:0/155837918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 msgr2=0x7f292415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 --2- 172.21.15.80:0/155837918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f292415e530 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f29180097c0 tx=0x7f291801c3b0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 -- 172.21.15.80:0/155837918 shutdown_connections 2024-04-25T13:42:56.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 --2- 172.21.15.80:0/155837918 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f292415e530 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 --2- 172.21.15.80:0/155837918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2924161f90 0x7f29240cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 -- 172.21.15.80:0/155837918 >> 172.21.15.80:0/155837918 conn(0x7f292406d080 msgr2=0x7f29240b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:56.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 -- 172.21.15.80:0/155837918 shutdown_connections 2024-04-25T13:42:56.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 -- 172.21.15.80:0/155837918 wait complete. 2024-04-25T13:42:56.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 Processor -- start 2024-04-25T13:42:56.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.013+0000 7f2928917640 1 -- start start 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f29240b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f292415ea70 0x7f2924168660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2924161f90 0x7f2924144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f292406f630 con 0x7f29240d0510 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f292406f4b0 con 0x7f2924161f90 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f29240d0920 con 0x7f292415ea70 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f29240b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f29240b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43076/0 (socket says 172.21.15.80:43076) 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 -- 172.21.15.80:0/4238654279 learned_addr learned my addr 172.21.15.80:0/4238654279 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2921d9b640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f292415ea70 0x7f2924168660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 -- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f292415ea70 msgr2=0x7f2924168660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f292415ea70 0x7f2924168660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 -- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2924161f90 msgr2=0x7f2924144830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2922d9d640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2924161f90 0x7f2924144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2924161f90 0x7f2924144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 -- 172.21.15.80:0/4238654279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29240af7a0 con 0x7f29240d0510 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f292259c640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f29240b0840 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f291400c820 tx=0x7f291400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2912ffd640 1 -- 172.21.15.80:0/4238654279 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914015070 con 0x7f29240d0510 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2912ffd640 1 -- 172.21.15.80:0/4238654279 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2914007a30 con 0x7f29240d0510 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29240afa80 con 0x7f29240d0510 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2912ffd640 1 -- 172.21.15.80:0/4238654279 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2914007e20 con 0x7f29240d0510 2024-04-25T13:42:56.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29240b0040 con 0x7f29240d0510 2024-04-25T13:42:56.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f292406a3a0 con 0x7f29240d0510 2024-04-25T13:42:56.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2912ffd640 1 -- 172.21.15.80:0/4238654279 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2914018070 con 0x7f29240d0510 2024-04-25T13:42:56.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2912ffd640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f290406f430 0x7f2904071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2912ffd640 1 -- 172.21.15.80:0/4238654279 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f291408f500 con 0x7f29240d0510 2024-04-25T13:42:56.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.017+0000 7f2921d9b640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f290406f430 0x7f2904071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.021+0000 7f2921d9b640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f290406f430 0x7f2904071940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29180ad000 tx=0x7f2918008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.021+0000 7f2912ffd640 1 -- 172.21.15.80:0/4238654279 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f291405c4d0 con 0x7f29240d0510 2024-04-25T13:42:56.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.169+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2924141830 con 0x7f29240d0510 2024-04-25T13:42:56.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.173+0000 7f2912ffd640 1 -- 172.21.15.80:0/4238654279 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16541 (secure 0 0 0) 0x7f291405bdd0 con 0x7f29240d0510 2024-04-25T13:42:56.177 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:56.177 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":52,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:49.227779+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:42:13.650442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T13:42:14.725209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:56.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f290406f430 msgr2=0x7f2904071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f290406f430 0x7f2904071940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f29180ad000 tx=0x7f2918008040 comp rx=0 tx=0).stop 2024-04-25T13:42:56.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 msgr2=0x7f29240b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f29240b0840 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f291400c820 tx=0x7f291400cce0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 shutdown_connections 2024-04-25T13:42:56.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f290406f430 0x7f2904071940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2924161f90 0x7f2924144830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f292415ea70 0x7f2924168660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 --2- 172.21.15.80:0/4238654279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29240d0510 0x7f29240b0840 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 >> 172.21.15.80:0/4238654279 conn(0x7f292406d080 msgr2=0x7f29240b62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:56.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 shutdown_connections 2024-04-25T13:42:56.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.177+0000 7f2928917640 1 -- 172.21.15.80:0/4238654279 wait complete. 2024-04-25T13:42:56.198 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:42:13.650442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:42:14.725209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:42:56.198 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:42:56.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f01a65c9640 1 Processor -- start 2024-04-25T13:42:56.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f01a65c9640 1 -- start start 2024-04-25T13:42:56.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f01a65c9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f01a006edb0 0x7f01a006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f01a65c9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a00b5740 con 0x7f01a00b5330 2024-04-25T13:42:56.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f01a65c9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01a006dd10 con 0x7f01a0161b60 2024-04-25T13:42:56.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f01a65c9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f01a006f700 con 0x7f01a006edb0 2024-04-25T13:42:56.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f019ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f01a00b5330 0x7f01a00d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52366/0 (socket says 172.21.15.80:52366) 2024-04-25T13:42:56.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f019ffff640 1 -- 172.21.15.80:0/1511691587 learned_addr learned my addr 172.21.15.80:0/1511691587 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:56.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f019f7fe640 1 --2- 172.21.15.80:0/1511691587 >> v2:172.21.15.80:3300/0 conn(0x7f01a006edb0 0x7f01a006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f019effd640 1 -- 172.21.15.80:0/1511691587 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2062629496 0 0) 0x7f01a00b5740 con 0x7f01a00b5330 2024-04-25T13:42:56.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f019effd640 1 -- 172.21.15.80:0/1511691587 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0188003530 con 0x7f01a00b5330 2024-04-25T13:42:56.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f019f7fe640 1 -- 172.21.15.80:0/1511691587 >> v1:172.21.15.87:6789/0 conn(0x7f01a0161b60 legacy=0x7f01a0161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f019f7fe640 1 -- 172.21.15.80:0/1511691587 >> v1:172.21.15.80:6789/0 conn(0x7f01a00b5330 legacy=0x7f01a00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.349+0000 7f019f7fe640 1 -- 172.21.15.80:0/1511691587 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a006dff0 con 0x7f01a006edb0 2024-04-25T13:42:56.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f019f7fe640 1 --2- 172.21.15.80:0/1511691587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a006edb0 0x7f01a006f1c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f01940097c0 tx=0x7f019401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7da5d8d4331a46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f019effd640 1 -- 172.21.15.80:0/1511691587 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940b0070 con 0x7f01a006edb0 2024-04-25T13:42:56.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f019effd640 1 -- 172.21.15.80:0/1511691587 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01940ab600 con 0x7f01a006edb0 2024-04-25T13:42:56.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f019effd640 1 -- 172.21.15.80:0/1511691587 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940ab9d0 con 0x7f01a006edb0 2024-04-25T13:42:56.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f01a65c9640 1 -- 172.21.15.80:0/1511691587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a006edb0 msgr2=0x7f01a006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f01a65c9640 1 --2- 172.21.15.80:0/1511691587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a006edb0 0x7f01a006f1c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f01940097c0 tx=0x7f019401c410 comp rx=0 tx=0).stop 2024-04-25T13:42:56.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f01a65c9640 1 -- 172.21.15.80:0/1511691587 shutdown_connections 2024-04-25T13:42:56.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f01a65c9640 1 --2- 172.21.15.80:0/1511691587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a006edb0 0x7f01a006f1c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f01a65c9640 1 -- 172.21.15.80:0/1511691587 >> 172.21.15.80:0/1511691587 conn(0x7f01a00b4160 msgr2=0x7f01a00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:56.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f01a65c9640 1 -- 172.21.15.80:0/1511691587 shutdown_connections 2024-04-25T13:42:56.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.353+0000 7f01a65c9640 1 -- 172.21.15.80:0/1511691587 wait complete. 2024-04-25T13:42:56.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 Processor -- start 2024-04-25T13:42:56.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 -- start start 2024-04-25T13:42:56.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01a006edb0 0x7f01a01531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a00b5330 0x7f01a0146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01a0161b60 0x7f01a01470d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00b0e50 con 0x7f01a00b5330 2024-04-25T13:42:56.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00b0cd0 con 0x7f01a0161b60 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f01a00d11c0 con 0x7f01a006edb0 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a00b5330 0x7f01a0146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a00b5330 0x7f01a0146b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43094/0 (socket says 172.21.15.80:43094) 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 -- 172.21.15.80:0/4078180512 learned_addr learned my addr 172.21.15.80:0/4078180512 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019ffff640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01a006edb0 0x7f01a01531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a4b66640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01a0161b60 0x7f01a01470d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 -- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01a006edb0 msgr2=0x7f01a01531c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01a006edb0 0x7f01a01531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 -- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01a0161b60 msgr2=0x7f01a01470d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01a0161b60 0x7f01a01470d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 -- 172.21.15.80:0/4078180512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a015b2f0 con 0x7f01a00b5330 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019f7fe640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a00b5330 0x7f01a0146b90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0194019020 tx=0x7f01940191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019cff9640 1 -- 172.21.15.80:0/4078180512 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940ba070 con 0x7f01a00b5330 2024-04-25T13:42:56.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019cff9640 1 -- 172.21.15.80:0/4078180512 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01940b43f0 con 0x7f01a00b5330 2024-04-25T13:42:56.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a015b7f0 con 0x7f01a00b5330 2024-04-25T13:42:56.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f019cff9640 1 -- 172.21.15.80:0/4078180512 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01940b0040 con 0x7f01a00b5330 2024-04-25T13:42:56.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a014e020 con 0x7f01a00b5330 2024-04-25T13:42:56.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.357+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a015b540 con 0x7f01a00b5330 2024-04-25T13:42:56.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.361+0000 7f019cff9640 1 -- 172.21.15.80:0/4078180512 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01940bd070 con 0x7f01a00b5330 2024-04-25T13:42:56.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.361+0000 7f019cff9640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f018c06f430 0x7f018c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.361+0000 7f019cff9640 1 -- 172.21.15.80:0/4078180512 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f0194128330 con 0x7f01a00b5330 2024-04-25T13:42:56.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.361+0000 7f019ffff640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f018c06f430 0x7f018c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.361+0000 7f019ffff640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f018c06f430 0x7f018c071940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f01900011f0 tx=0x7f0190009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.365+0000 7f019cff9640 1 -- 172.21.15.80:0/4078180512 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0194016090 con 0x7f01a00b5330 2024-04-25T13:42:56.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f01a006a3a0 con 0x7f01a00b5330 2024-04-25T13:42:56.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f019cff9640 1 -- 172.21.15.80:0/4078180512 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f01940f5320 con 0x7f01a00b5330 2024-04-25T13:42:56.536 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:42:56.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f018c06f430 msgr2=0x7f018c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f018c06f430 0x7f018c071940 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f01900011f0 tx=0x7f0190009040 comp rx=0 tx=0).stop 2024-04-25T13:42:56.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a00b5330 msgr2=0x7f01a0146b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a00b5330 0x7f01a0146b90 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f0194019020 tx=0x7f01940191b0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 shutdown_connections 2024-04-25T13:42:56.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f018c06f430 0x7f018c071940 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01a0161b60 0x7f01a01470d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01a00b5330 0x7f01a0146b90 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 --2- 172.21.15.80:0/4078180512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01a006edb0 0x7f01a01531c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 >> 172.21.15.80:0/4078180512 conn(0x7f01a00b4160 msgr2=0x7f01a015f6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:56.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 shutdown_connections 2024-04-25T13:42:56.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.533+0000 7f01a65c9640 1 -- 172.21.15.80:0/4078180512 wait complete. 2024-04-25T13:42:56.557 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:42:56.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.701+0000 7f482e459640 1 Processor -- start 2024-04-25T13:42:56.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.701+0000 7f482e459640 1 -- start start 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f482e459640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f48281574c0 0x7f4828073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f482e459640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f482806fbf0 con 0x7f4828161170 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f482e459640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f48280b0a20 con 0x7f4828071e60 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f482e459640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f48280b1350 con 0x7f48281574c0 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f481f7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4828161170 0x7f48280b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52378/0 (socket says 172.21.15.80:52378) 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f481f7fe640 1 -- 172.21.15.80:0/3401755548 learned_addr learned my addr 172.21.15.80:0/3401755548 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f482c9f6640 1 --2- 172.21.15.80:0/3401755548 >> v2:172.21.15.80:3300/0 conn(0x7f48281574c0 0x7f4828073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f48277fe640 1 -- 172.21.15.80:0/3401755548 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924566533 0 0) 0x7f482806fbf0 con 0x7f4828161170 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f48277fe640 1 -- 172.21.15.80:0/3401755548 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4814003530 con 0x7f4828161170 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f482c9f6640 1 -- 172.21.15.80:0/3401755548 >> v1:172.21.15.87:6789/0 conn(0x7f4828071e60 legacy=0x7f4828160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f482c9f6640 1 -- 172.21.15.80:0/3401755548 >> v1:172.21.15.80:6789/0 conn(0x7f4828161170 legacy=0x7f48280b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.705+0000 7f482c9f6640 1 -- 172.21.15.80:0/3401755548 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48280b0d00 con 0x7f48281574c0 2024-04-25T13:42:56.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482c9f6640 1 --2- 172.21.15.80:0/3401755548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f4828073c40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f48280b14d0 tx=0x7f482001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc720f2776bcfa1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f48277fe640 1 -- 172.21.15.80:0/3401755548 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48200b2070 con 0x7f48281574c0 2024-04-25T13:42:56.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f48277fe640 1 -- 172.21.15.80:0/3401755548 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48200a5710 con 0x7f48281574c0 2024-04-25T13:42:56.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f48277fe640 1 -- 172.21.15.80:0/3401755548 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48200a5ae0 con 0x7f48281574c0 2024-04-25T13:42:56.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 -- 172.21.15.80:0/3401755548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 msgr2=0x7f4828073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 --2- 172.21.15.80:0/3401755548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f4828073c40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f48280b14d0 tx=0x7f482001c950 comp rx=0 tx=0).stop 2024-04-25T13:42:56.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 -- 172.21.15.80:0/3401755548 shutdown_connections 2024-04-25T13:42:56.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 --2- 172.21.15.80:0/3401755548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f4828073c40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 -- 172.21.15.80:0/3401755548 >> 172.21.15.80:0/3401755548 conn(0x7f48280b4160 msgr2=0x7f48280d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:56.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 -- 172.21.15.80:0/3401755548 shutdown_connections 2024-04-25T13:42:56.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 -- 172.21.15.80:0/3401755548 wait complete. 2024-04-25T13:42:56.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 Processor -- start 2024-04-25T13:42:56.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.709+0000 7f482e459640 1 -- start start 2024-04-25T13:42:56.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4828071e60 0x7f4828159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f48281558d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48281562b0 0x7f4828075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f48280b56c0 con 0x7f48281574c0 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f482806ec80 con 0x7f4828071e60 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f482815a320 con 0x7f48281562b0 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f48281558d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f48281558d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43106/0 (socket says 172.21.15.80:43106) 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 -- 172.21.15.80:0/1258521127 learned_addr learned my addr 172.21.15.80:0/1258521127 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f4827fff640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4828071e60 0x7f4828159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482c9f6640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48281562b0 0x7f4828075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 -- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48281562b0 msgr2=0x7f4828075b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48281562b0 0x7f4828075b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 -- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4828071e60 msgr2=0x7f4828159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4828071e60 0x7f4828159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 -- 172.21.15.80:0/1258521127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f482815cb60 con 0x7f48281574c0 2024-04-25T13:42:56.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f481f7fe640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f48281558d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f481000c420 tx=0x7f481000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f48257fa640 1 -- 172.21.15.80:0/1258521127 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4810017070 con 0x7f48281574c0 2024-04-25T13:42:56.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f48257fa640 1 -- 172.21.15.80:0/1258521127 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4810005cf0 con 0x7f48281574c0 2024-04-25T13:42:56.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4828159e50 con 0x7f48281574c0 2024-04-25T13:42:56.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f48257fa640 1 -- 172.21.15.80:0/1258521127 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48100072d0 con 0x7f48281574c0 2024-04-25T13:42:56.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48280b1ed0 con 0x7f48281574c0 2024-04-25T13:42:56.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f482806a3a0 con 0x7f48281574c0 2024-04-25T13:42:56.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f48257fa640 1 -- 172.21.15.80:0/1258521127 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f481001a070 con 0x7f48281574c0 2024-04-25T13:42:56.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.713+0000 7f48257fa640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f480806f430 0x7f4808071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:56.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.717+0000 7f48257fa640 1 -- 172.21.15.80:0/1258521127 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f481008e540 con 0x7f48281574c0 2024-04-25T13:42:56.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.717+0000 7f4827fff640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f480806f430 0x7f4808071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:56.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.717+0000 7f4827fff640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f480806f430 0x7f4808071940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f48180015b0 tx=0x7f4818009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:56.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.721+0000 7f48257fa640 1 -- 172.21.15.80:0/1258521127 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f481005ade0 con 0x7f48281574c0 2024-04-25T13:42:56.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.877+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f48280b3150 con 0x7f48281574c0 2024-04-25T13:42:56.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f48257fa640 1 -- 172.21.15.80:0/1258521127 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) ==== 104+0+11 (secure 0 0 0) 0x7f481005f0e0 con 0x7f48281574c0 2024-04-25T13:42:56.884 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:42:56.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f480806f430 msgr2=0x7f4808071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f480806f430 0x7f4808071940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f48180015b0 tx=0x7f4818009040 comp rx=0 tx=0).stop 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 msgr2=0x7f48281558d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f48281558d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f481000c420 tx=0x7f481000c8e0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 shutdown_connections 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f480806f430 0x7f4808071940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f48281562b0 0x7f4828075b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f48281574c0 0x7f48281558d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 --2- 172.21.15.80:0/1258521127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4828071e60 0x7f4828159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 >> 172.21.15.80:0/1258521127 conn(0x7f48280b4160 msgr2=0x7f482815dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 shutdown_connections 2024-04-25T13:42:56.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:56.881+0000 7f482e459640 1 -- 172.21.15.80:0/1258521127 wait complete. 2024-04-25T13:42:56.906 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:42:57.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa9281b2640 1 Processor -- start 2024-04-25T13:42:57.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa9281b2640 1 -- start start 2024-04-25T13:42:57.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa9281b2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa920071e60 0x7fa9201574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa9281b2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa920073c40 0x7fa920160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa9281b2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9201613b0 con 0x7fa920160fa0 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa9281b2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa9200b1350 con 0x7fa920071e60 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa9281b2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa92006f9a0 con 0x7fa920073c40 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa925f4e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa920071e60 0x7fa9201574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa925f4e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa920071e60 0x7fa9201574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43118/0 (socket says 172.21.15.80:43118) 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa925f4e640 1 -- 172.21.15.80:0/414125478 learned_addr learned my addr 172.21.15.80:0/414125478 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa92574d640 1 --2- 172.21.15.80:0/414125478 >> v2:172.21.15.87:3300/0 conn(0x7fa920073c40 0x7fa920160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa925f4e640 1 -- 172.21.15.80:0/414125478 >> v2:172.21.15.87:3300/0 conn(0x7fa920073c40 msgr2=0x7fa920160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa925f4e640 1 --2- 172.21.15.80:0/414125478 >> v2:172.21.15.87:3300/0 conn(0x7fa920073c40 0x7fa920160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa925f4e640 1 -- 172.21.15.80:0/414125478 >> v1:172.21.15.87:6789/0 conn(0x7fa920160fa0 legacy=0x7fa9200b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.053+0000 7fa925f4e640 1 -- 172.21.15.80:0/414125478 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa92006fbf0 con 0x7fa920071e60 2024-04-25T13:42:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa925f4e640 1 --2- 172.21.15.80:0/414125478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920071e60 0x7fa9201574c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa9200cecc0 tx=0x7fa91001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464fbe4721703b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa924f4c640 1 -- 172.21.15.80:0/414125478 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9100ab610 con 0x7fa920071e60 2024-04-25T13:42:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa924f4c640 1 -- 172.21.15.80:0/414125478 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9100b0070 con 0x7fa920071e60 2024-04-25T13:42:57.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa924f4c640 1 -- 172.21.15.80:0/414125478 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9100ab9d0 con 0x7fa920071e60 2024-04-25T13:42:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa9281b2640 1 -- 172.21.15.80:0/414125478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920071e60 msgr2=0x7fa9201574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa9281b2640 1 --2- 172.21.15.80:0/414125478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920071e60 0x7fa9201574c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa9200cecc0 tx=0x7fa91001c520 comp rx=0 tx=0).stop 2024-04-25T13:42:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa9281b2640 1 -- 172.21.15.80:0/414125478 shutdown_connections 2024-04-25T13:42:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa9281b2640 1 --2- 172.21.15.80:0/414125478 >> v2:172.21.15.87:3300/0 conn(0x7fa920073c40 0x7fa920160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa9281b2640 1 --2- 172.21.15.80:0/414125478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920071e60 0x7fa9201574c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa9281b2640 1 -- 172.21.15.80:0/414125478 >> 172.21.15.80:0/414125478 conn(0x7fa9200b4160 msgr2=0x7fa9200d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa9281b2640 1 -- 172.21.15.80:0/414125478 shutdown_connections 2024-04-25T13:42:57.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.057+0000 7fa9281b2640 1 -- 172.21.15.80:0/414125478 wait complete. 2024-04-25T13:42:57.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 Processor -- start 2024-04-25T13:42:57.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 -- start start 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa920071e60 0x7fa9200b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920073c40 0x7fa92014abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa920160fa0 0x7fa9201556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa920074050 con 0x7fa920073c40 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa920072270 con 0x7fa920160fa0 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa9200b56c0 con 0x7fa920071e60 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920073c40 0x7fa92014abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920073c40 0x7fa92014abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43124/0 (socket says 172.21.15.80:43124) 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 -- 172.21.15.80:0/2278799970 learned_addr learned my addr 172.21.15.80:0/2278799970 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:57.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92674f640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa920160fa0 0x7fa9201556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa925f4e640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa920071e60 0x7fa9200b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 -- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa920071e60 msgr2=0x7fa9200b1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa920071e60 0x7fa9200b1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 -- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa920160fa0 msgr2=0x7fa9201556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa920160fa0 0x7fa9201556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 -- 172.21.15.80:0/2278799970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9201f7cb0 con 0x7fa920073c40 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92674f640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa920160fa0 0x7fa9201556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa92574d640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920073c40 0x7fa92014abf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00d620 tx=0x7fa91c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9167fc640 1 -- 172.21.15.80:0/2278799970 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0125f0 con 0x7fa920073c40 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9167fc640 1 -- 172.21.15.80:0/2278799970 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa91c019070 con 0x7fa920073c40 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9167fc640 1 -- 172.21.15.80:0/2278799970 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c0129b0 con 0x7fa920073c40 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa920152310 con 0x7fa920073c40 2024-04-25T13:42:57.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9201528d0 con 0x7fa920073c40 2024-04-25T13:42:57.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.061+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa92006a3a0 con 0x7fa920073c40 2024-04-25T13:42:57.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.065+0000 7fa9167fc640 1 -- 172.21.15.80:0/2278799970 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa91c012b50 con 0x7fa920073c40 2024-04-25T13:42:57.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.065+0000 7fa9167fc640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa90406f400 0x7fa904071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.065+0000 7fa9167fc640 1 -- 172.21.15.80:0/2278799970 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fa91c0900a0 con 0x7fa920073c40 2024-04-25T13:42:57.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.065+0000 7fa925f4e640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa90406f400 0x7fa904071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.065+0000 7fa925f4e640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa90406f400 0x7fa904071910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa9100af000 tx=0x7fa9100a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:57.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.069+0000 7fa9167fc640 1 -- 172.21.15.80:0/2278799970 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa91c05d100 con 0x7fa920073c40 2024-04-25T13:42:57.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa9200cddb0 con 0x7fa920073c40 2024-04-25T13:42:57.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9167fc640 1 -- 172.21.15.80:0/2278799970 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) ==== 100+0+11 (secure 0 0 0) 0x7fa91c05ca00 con 0x7fa920073c40 2024-04-25T13:42:57.232 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:42:57.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa90406f400 msgr2=0x7fa904071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa90406f400 0x7fa904071910 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa9100af000 tx=0x7fa9100a7000 comp rx=0 tx=0).stop 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920073c40 msgr2=0x7fa92014abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920073c40 0x7fa92014abf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa91c00d620 tx=0x7fa91c00dae0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 shutdown_connections 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa90406f400 0x7fa904071910 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa920160fa0 0x7fa9201556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa920073c40 0x7fa92014abf0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 --2- 172.21.15.80:0/2278799970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa920071e60 0x7fa9200b1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 >> 172.21.15.80:0/2278799970 conn(0x7fa9200b4160 msgr2=0x7fa92015d6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.229+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 shutdown_connections 2024-04-25T13:42:57.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.233+0000 7fa9281b2640 1 -- 172.21.15.80:0/2278799970 wait complete. 2024-04-25T13:42:57.255 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:42:57.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.401+0000 7f396ccdf640 1 Processor -- start 2024-04-25T13:42:57.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.401+0000 7f396ccdf640 1 -- start start 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396ccdf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f39680d59b0 0x7f39680d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396ccdf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f39680d4330 0x7f39680d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396ccdf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39680d6300 con 0x7f39680d2e80 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396ccdf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3968161060 con 0x7f39680d4330 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396ccdf640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3968161340 con 0x7f39680d59b0 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396659c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f39680d2e80 0x7f39681607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52392/0 (socket says 172.21.15.80:52392) 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f3966d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f39680d4330 0x7f39680d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396659c640 1 -- 172.21.15.80:0/675369839 learned_addr learned my addr 172.21.15.80:0/675369839 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f3965d9b640 1 --2- 172.21.15.80:0/675369839 >> v2:172.21.15.87:3300/0 conn(0x7f39680d59b0 0x7f39680d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396559a640 1 -- 172.21.15.80:0/675369839 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3412559801 0 0) 0x7f3958001120 con 0x7f39680d2e80 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396559a640 1 -- 172.21.15.80:0/675369839 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3950003530 con 0x7f39680d2e80 2024-04-25T13:42:57.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f3966d9d640 1 -- 172.21.15.80:0/675369839 >> v2:172.21.15.87:3300/0 conn(0x7f39680d59b0 msgr2=0x7f39680d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:57.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f3966d9d640 1 --2- 172.21.15.80:0/675369839 >> v2:172.21.15.87:3300/0 conn(0x7f39680d59b0 0x7f39680d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f3966d9d640 1 -- 172.21.15.80:0/675369839 >> v1:172.21.15.80:6789/0 conn(0x7f39680d2e80 legacy=0x7f39681607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f3966d9d640 1 -- 172.21.15.80:0/675369839 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39680d2680 con 0x7f39680d4330 2024-04-25T13:42:57.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f3966d9d640 1 --2- 172.21.15.80:0/675369839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f39680d4740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f39680c9670 tx=0x7f396001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a849b950df26be3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:57.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.405+0000 7f396559a640 1 -- 172.21.15.80:0/675369839 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39600b2070 con 0x7f39680d4330 2024-04-25T13:42:57.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396559a640 1 -- 172.21.15.80:0/675369839 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39600a5870 con 0x7f39680d4330 2024-04-25T13:42:57.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396559a640 1 -- 172.21.15.80:0/675369839 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39600a5c20 con 0x7f39680d4330 2024-04-25T13:42:57.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 -- 172.21.15.80:0/675369839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 msgr2=0x7f39680d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 --2- 172.21.15.80:0/675369839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f39680d4740 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f39680c9670 tx=0x7f396001c8a0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 -- 172.21.15.80:0/675369839 shutdown_connections 2024-04-25T13:42:57.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 --2- 172.21.15.80:0/675369839 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f39680d4740 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 --2- 172.21.15.80:0/675369839 >> v2:172.21.15.87:3300/0 conn(0x7f39680d59b0 0x7f39680d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 -- 172.21.15.80:0/675369839 >> 172.21.15.80:0/675369839 conn(0x7f39680737a0 msgr2=0x7f396806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:57.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 -- 172.21.15.80:0/675369839 shutdown_connections 2024-04-25T13:42:57.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 -- 172.21.15.80:0/675369839 wait complete. 2024-04-25T13:42:57.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 Processor -- start 2024-04-25T13:42:57.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.409+0000 7f396ccdf640 1 -- start start 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39680d2e80 0x7f39681664a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f3968154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39680d59b0 0x7f39680ba270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39680d3410 con 0x7f39680d4330 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39680d3290 con 0x7f39680d2e80 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f396815df60 con 0x7f39680d59b0 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f3968154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f3968154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43136/0 (socket says 172.21.15.80:43136) 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 -- 172.21.15.80:0/91906711 learned_addr learned my addr 172.21.15.80:0/91906711 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396659c640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39680d2e80 0x7f39681664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3966d9d640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39680d59b0 0x7f39680ba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 -- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39680d59b0 msgr2=0x7f39680ba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39680d59b0 0x7f39680ba270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 -- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39680d2e80 msgr2=0x7f39681664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39680d2e80 0x7f39681664a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 -- 172.21.15.80:0/91906711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3968167f20 con 0x7f39680d4330 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3965d9b640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f3968154eb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f395c00e870 tx=0x7f395c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3956ffd640 1 -- 172.21.15.80:0/91906711 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c017070 con 0x7f39680d4330 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3956ffd640 1 -- 172.21.15.80:0/91906711 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c00c940 con 0x7f39680d4330 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3956ffd640 1 -- 172.21.15.80:0/91906711 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c00cd30 con 0x7f39680d4330 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3968166e80 con 0x7f39680d4330 2024-04-25T13:42:57.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3968167440 con 0x7f39680d4330 2024-04-25T13:42:57.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3968166bf0 con 0x7f39680d4330 2024-04-25T13:42:57.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3956ffd640 1 -- 172.21.15.80:0/91906711 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f395c01a070 con 0x7f39680d4330 2024-04-25T13:42:57.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3956ffd640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394806f430 0x7f3948071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:57.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f3956ffd640 1 -- 172.21.15.80:0/91906711 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f395c08f0d0 con 0x7f39680d4330 2024-04-25T13:42:57.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.413+0000 7f396659c640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394806f430 0x7f3948071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:57.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.417+0000 7f396659c640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394806f430 0x7f3948071940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3958007fd0 tx=0x7f3958009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:57.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.417+0000 7f3956ffd640 1 -- 172.21.15.80:0/91906711 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f395c05c050 con 0x7f39680d4330 2024-04-25T13:42:57.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.589+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f396806a3a0 con 0x7f39680d4330 2024-04-25T13:42:57.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.589+0000 7f3956ffd640 1 -- 172.21.15.80:0/91906711 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v52) ==== 80+0+0 (secure 0 0 0) 0x7f395c05b950 con 0x7f39680d4330 2024-04-25T13:42:57.592 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.589+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394806f430 msgr2=0x7f3948071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.589+0000 7f396ccdf640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394806f430 0x7f3948071940 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3958007fd0 tx=0x7f3958009040 comp rx=0 tx=0).stop 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.589+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 msgr2=0x7f3968154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.589+0000 7f396ccdf640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f3968154eb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f395c00e870 tx=0x7f395c00ec70 comp rx=0 tx=0).stop 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.589+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 shutdown_connections 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.589+0000 7f396ccdf640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394806f430 0x7f3948071940 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.593+0000 7f396ccdf640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39680d59b0 0x7f39680ba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.593+0000 7f396ccdf640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39680d4330 0x7f3968154eb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.593+0000 7f396ccdf640 1 --2- 172.21.15.80:0/91906711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39680d2e80 0x7f39681664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.593+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 >> 172.21.15.80:0/91906711 conn(0x7f39680737a0 msgr2=0x7f39680757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.593+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 shutdown_connections 2024-04-25T13:42:57.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.593+0000 7f396ccdf640 1 -- 172.21.15.80:0/91906711 wait complete. 2024-04-25T13:42:57.613 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:42:57.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:57.614 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:42:57.626 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24433]: fuse finished with error 0 and tester_r 0 2024-04-25T13:42:57.628 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:42:57.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:42:57.628 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:42:57.797 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:42:57.797 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:42:57.797 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:57.803 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:42:57.804 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:42:57.804 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:42:57.804 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:42:57.844 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:42:57.844 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:57.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f45147b7640 1 Processor -- start 2024-04-25T13:42:57.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f45147b7640 1 -- start start 2024-04-25T13:42:58.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f45147b7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c161f90 0x7f450c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f45147b7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f450c0d0d70 con 0x7f450c15ec40 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f45147b7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f450c0d1050 con 0x7f450c0d0510 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f45147b7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f450c06dca0 con 0x7f450c161f90 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512d54640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f450c15ec40 0x7f450c168cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52396/0 (socket says 172.21.15.80:52396) 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512d54640 1 -- 172.21.15.80:0/434446860 learned_addr learned my addr 172.21.15.80:0/434446860 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512553640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c161f90 0x7f450c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3249368412 0 0) 0x7f450c0d0d70 con 0x7f450c15ec40 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f44f8003530 con 0x7f450c15ec40 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1572412547 0 0) 0x7f44f8003530 con 0x7f450c15ec40 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c161f90 msgr2=0x7f450c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c161f90 0x7f450c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 >> v1:172.21.15.87:6789/0 conn(0x7f450c0d0510 legacy=0x7f450c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c06df80 con 0x7f450c15ec40 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4504001a00 con 0x7f450c15ec40 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 >> v1:172.21.15.80:6789/0 conn(0x7f450c15ec40 legacy=0x7f450c168cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c161f90 0x7f44f80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8003f70 0x7f44f8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8004d50 0x7f44f8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f450c06dca0 con 0x7f44f8004d50 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f44f800bc10 con 0x7f44f8003f70 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f44f800be10 con 0x7f450c161f90 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512d54640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8004d50 0x7f44f8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512553640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c161f90 0x7f44f80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4511d52640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8003f70 0x7f44f8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512d54640 1 -- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c161f90 msgr2=0x7f44f80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:58.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512d54640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c161f90 0x7f44f80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512d54640 1 -- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8003f70 msgr2=0x7f44f8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:58.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512d54640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44f8003f70 0x7f44f8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:57.997+0000 7f4512d54640 1 -- 172.21.15.80:0/434446860 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f8003530 con 0x7f44f8004d50 2024-04-25T13:42:58.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f4512d54640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8004d50 0x7f44f8005160 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f450400a840 tx=0x7f450401fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133a6d0874af3355 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450400a460 con 0x7f44f8004d50 2024-04-25T13:42:58.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45040aa710 con 0x7f44f8004d50 2024-04-25T13:42:58.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f4511551640 1 -- 172.21.15.80:0/434446860 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45040aab00 con 0x7f44f8004d50 2024-04-25T13:42:58.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f45147b7640 1 -- 172.21.15.80:0/434446860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8004d50 msgr2=0x7f44f8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f45147b7640 1 --2- 172.21.15.80:0/434446860 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44f8004d50 0x7f44f8005160 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f450400a840 tx=0x7f450401fac0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f4512d54640 1 -- 172.21.15.80:0/434446860 reap_dead start 2024-04-25T13:42:58.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f45147b7640 1 -- 172.21.15.80:0/434446860 shutdown_connections 2024-04-25T13:42:58.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f45147b7640 1 -- 172.21.15.80:0/434446860 >> 172.21.15.80:0/434446860 conn(0x7f450c06d080 msgr2=0x7f450c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:58.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f45147b7640 1 -- 172.21.15.80:0/434446860 shutdown_connections 2024-04-25T13:42:58.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.001+0000 7f45147b7640 1 -- 172.21.15.80:0/434446860 wait complete. 2024-04-25T13:42:58.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 Processor -- start 2024-04-25T13:42:58.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 -- start start 2024-04-25T13:42:58.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c15da10 0x7f450c1476f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f450c1458f0 0x7f450c145d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f450c0ad2c0 0x7f450c0ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f450c162520 con 0x7f450c1458f0 2024-04-25T13:42:58.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f450c1623a0 con 0x7f450c0ad2c0 2024-04-25T13:42:58.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f450c144620 con 0x7f450c15da10 2024-04-25T13:42:58.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f450c1458f0 0x7f450c145d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f450c1458f0 0x7f450c145d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43156/0 (socket says 172.21.15.80:43156) 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 -- 172.21.15.80:0/200335666 learned_addr learned my addr 172.21.15.80:0/200335666 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4512553640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c15da10 0x7f450c1476f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4512d54640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f450c0ad2c0 0x7f450c0ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 -- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c15da10 msgr2=0x7f450c1476f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c15da10 0x7f450c1476f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 -- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f450c0ad2c0 msgr2=0x7f450c0ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f450c0ad2c0 0x7f450c0ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 -- 172.21.15.80:0/200335666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450c146650 con 0x7f450c1458f0 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4512553640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c15da10 0x7f450c1476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:42:58.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4511d52640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f450c1458f0 0x7f450c145d00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00e3f0 tx=0x7f44fc00e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4502ffd640 1 -- 172.21.15.80:0/200335666 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc019070 con 0x7f450c1458f0 2024-04-25T13:42:58.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4502ffd640 1 -- 172.21.15.80:0/200335666 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44fc0052e0 con 0x7f450c1458f0 2024-04-25T13:42:58.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450c146930 con 0x7f450c1458f0 2024-04-25T13:42:58.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f4502ffd640 1 -- 172.21.15.80:0/200335666 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44fc003840 con 0x7f450c1458f0 2024-04-25T13:42:58.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450c146ef0 con 0x7f450c1458f0 2024-04-25T13:42:58.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.005+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450c147330 con 0x7f450c1458f0 2024-04-25T13:42:58.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.009+0000 7f4502ffd640 1 -- 172.21.15.80:0/200335666 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f44fc01c070 con 0x7f450c1458f0 2024-04-25T13:42:58.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.009+0000 7f4502ffd640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f006f400 0x7f44f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.009+0000 7f4502ffd640 1 -- 172.21.15.80:0/200335666 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f44fc08eec0 con 0x7f450c1458f0 2024-04-25T13:42:58.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.009+0000 7f4512553640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f006f400 0x7f44f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.009+0000 7f4512553640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f006f400 0x7f44f0071910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f450c15ec10 tx=0x7f4508008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.013+0000 7f4502ffd640 1 -- 172.21.15.80:0/200335666 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44fc05b960 con 0x7f450c1458f0 2024-04-25T13:42:58.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.209+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f450c06a3a0 con 0x7f450c1458f0 2024-04-25T13:42:58.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.209+0000 7f4502ffd640 1 -- 172.21.15.80:0/200335666 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f44fc05fc60 con 0x7f450c1458f0 2024-04-25T13:42:58.213 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:58.213 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:42:58.213 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.209+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f006f400 msgr2=0x7f44f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.209+0000 7f45147b7640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f006f400 0x7f44f0071910 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f450c15ec10 tx=0x7f4508008040 comp rx=0 tx=0).stop 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.209+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f450c1458f0 msgr2=0x7f450c145d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f450c1458f0 0x7f450c145d00 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f44fc00e3f0 tx=0x7f44fc00e8b0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 shutdown_connections 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f006f400 0x7f44f0071910 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f450c0ad2c0 0x7f450c0ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f450c1458f0 0x7f450c145d00 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 --2- 172.21.15.80:0/200335666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f450c15da10 0x7f450c1476f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 >> 172.21.15.80:0/200335666 conn(0x7f450c06d080 msgr2=0x7f450c071e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 shutdown_connections 2024-04-25T13:42:58.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.213+0000 7f45147b7640 1 -- 172.21.15.80:0/200335666 wait complete. 2024-04-25T13:42:58.235 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:42:58.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.381+0000 7f9bf6049640 1 Processor -- start 2024-04-25T13:42:58.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.381+0000 7f9bf6049640 1 -- start start 2024-04-25T13:42:58.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9bf6049640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9bf00d06e0 0x7f9bf006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9bf6049640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf00d0af0 con 0x7f9bf0161f90 2024-04-25T13:42:58.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9bf6049640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9bf00d0d70 con 0x7f9bf006e9e0 2024-04-25T13:42:58.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9bf6049640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9bf00d1050 con 0x7f9bf00d06e0 2024-04-25T13:42:58.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9bef7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9bf0161f90 0x7f9bf00cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52412/0 (socket says 172.21.15.80:52412) 2024-04-25T13:42:58.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9bef7fe640 1 -- 172.21.15.80:0/962839806 learned_addr learned my addr 172.21.15.80:0/962839806 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:58.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9beeffd640 1 --2- 172.21.15.80:0/962839806 >> v2:172.21.15.80:3300/0 conn(0x7f9bf00d06e0 0x7f9bf006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9bee7fc640 1 -- 172.21.15.80:0/962839806 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3067568506 0 0) 0x7f9be0001120 con 0x7f9bf0161f90 2024-04-25T13:42:58.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9bee7fc640 1 -- 172.21.15.80:0/962839806 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9bd8003530 con 0x7f9bf0161f90 2024-04-25T13:42:58.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9beeffd640 1 -- 172.21.15.80:0/962839806 >> v1:172.21.15.87:6789/0 conn(0x7f9bf006e9e0 legacy=0x7f9bf006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:58.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9beeffd640 1 -- 172.21.15.80:0/962839806 >> v1:172.21.15.80:6789/0 conn(0x7f9bf0161f90 legacy=0x7f9bf00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.385+0000 7f9beeffd640 1 -- 172.21.15.80:0/962839806 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf006dca0 con 0x7f9bf00d06e0 2024-04-25T13:42:58.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9beeffd640 1 --2- 172.21.15.80:0/962839806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf00d06e0 0x7f9bf006e4a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097c0 tx=0x7f9be401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66e59beebd2bd03b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bee7fc640 1 -- 172.21.15.80:0/962839806 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be40b0070 con 0x7f9bf00d06e0 2024-04-25T13:42:58.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bee7fc640 1 -- 172.21.15.80:0/962839806 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be40ab600 con 0x7f9bf00d06e0 2024-04-25T13:42:58.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bee7fc640 1 -- 172.21.15.80:0/962839806 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be40ab9d0 con 0x7f9bf00d06e0 2024-04-25T13:42:58.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 -- 172.21.15.80:0/962839806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf00d06e0 msgr2=0x7f9bf006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 --2- 172.21.15.80:0/962839806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf00d06e0 0x7f9bf006e4a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9be40097c0 tx=0x7f9be401c310 comp rx=0 tx=0).stop 2024-04-25T13:42:58.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 -- 172.21.15.80:0/962839806 shutdown_connections 2024-04-25T13:42:58.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 --2- 172.21.15.80:0/962839806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf00d06e0 0x7f9bf006e4a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 -- 172.21.15.80:0/962839806 >> 172.21.15.80:0/962839806 conn(0x7f9bf006d080 msgr2=0x7f9bf00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:58.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 -- 172.21.15.80:0/962839806 shutdown_connections 2024-04-25T13:42:58.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 -- 172.21.15.80:0/962839806 wait complete. 2024-04-25T13:42:58.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 Processor -- start 2024-04-25T13:42:58.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.389+0000 7f9bf6049640 1 -- start start 2024-04-25T13:42:58.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf006e9e0 0x7f9bf01412c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bf00d06e0 0x7f9bf0157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bf0161f90 0x7f9bf01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf006f680 con 0x7f9bf006e9e0 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9bf006f500 con 0x7f9bf0161f90 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9bf0144620 con 0x7f9bf00d06e0 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf006e9e0 0x7f9bf01412c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf006e9e0 0x7f9bf01412c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43176/0 (socket says 172.21.15.80:43176) 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 -- 172.21.15.80:0/170140765 learned_addr learned my addr 172.21.15.80:0/170140765 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9beeffd640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bf00d06e0 0x7f9bf0157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9beffff640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bf0161f90 0x7f9bf01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 -- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bf00d06e0 msgr2=0x7f9bf0157f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bf00d06e0 0x7f9bf0157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 -- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bf0161f90 msgr2=0x7f9bf01608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bf0161f90 0x7f9bf01608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 -- 172.21.15.80:0/170140765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bf015bd50 con 0x7f9bf006e9e0 2024-04-25T13:42:58.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bef7fe640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf006e9e0 0x7f9bf01412c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c3f0 tx=0x7f9be0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bd7fff640 1 -- 172.21.15.80:0/170140765 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be0017070 con 0x7f9bf006e9e0 2024-04-25T13:42:58.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bd7fff640 1 -- 172.21.15.80:0/170140765 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9be0005cf0 con 0x7f9bf006e9e0 2024-04-25T13:42:58.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bf015c030 con 0x7f9bf006e9e0 2024-04-25T13:42:58.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bd7fff640 1 -- 172.21.15.80:0/170140765 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9be00072f0 con 0x7f9bf006e9e0 2024-04-25T13:42:58.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bf015c5f0 con 0x7f9bf006e9e0 2024-04-25T13:42:58.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bf006a3a0 con 0x7f9bf006e9e0 2024-04-25T13:42:58.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bd7fff640 1 -- 172.21.15.80:0/170140765 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9be001a070 con 0x7f9bf006e9e0 2024-04-25T13:42:58.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bd7fff640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9bdc06f430 0x7f9bdc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9bd7fff640 1 -- 172.21.15.80:0/170140765 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f9be008e700 con 0x7f9bf006e9e0 2024-04-25T13:42:58.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.393+0000 7f9beeffd640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9bdc06f430 0x7f9bdc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.397+0000 7f9beeffd640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9bdc06f430 0x7f9bdc071940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9be40ad000 tx=0x7f9be4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.401+0000 7f9bd7fff640 1 -- 172.21.15.80:0/170140765 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9bf00d0af0 con 0x7f9bf006e9e0 2024-04-25T13:42:58.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.561+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9bf00b0700 con 0x7f9bf006e9e0 2024-04-25T13:42:58.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.565+0000 7f9bd7fff640 1 -- 172.21.15.80:0/170140765 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16541 (secure 0 0 0) 0x7f9be005b570 con 0x7f9bf006e9e0 2024-04-25T13:42:58.569 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:58.569 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":52,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:49.227779+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:42:13.650442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T13:42:14.725209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:42:58.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9bdc06f430 msgr2=0x7f9bdc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9bdc06f430 0x7f9bdc071940 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9be40ad000 tx=0x7f9be4008040 comp rx=0 tx=0).stop 2024-04-25T13:42:58.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf006e9e0 msgr2=0x7f9bf01412c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf006e9e0 0x7f9bf01412c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9be000c3f0 tx=0x7f9be0007f90 comp rx=0 tx=0).stop 2024-04-25T13:42:58.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 shutdown_connections 2024-04-25T13:42:58.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9bdc06f430 0x7f9bdc071940 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9bf0161f90 0x7f9bf01608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9bf00d06e0 0x7f9bf0157f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 --2- 172.21.15.80:0/170140765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9bf006e9e0 0x7f9bf01412c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 >> 172.21.15.80:0/170140765 conn(0x7f9bf006d080 msgr2=0x7f9bf00b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:58.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 shutdown_connections 2024-04-25T13:42:58.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.569+0000 7f9bf6049640 1 -- 172.21.15.80:0/170140765 wait complete. 2024-04-25T13:42:58.594 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:42:13.650442+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:42:14.725209+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:42:58.594 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:42:58.617 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:42:58.640 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:42:58.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91c496a640 1 Processor -- start 2024-04-25T13:42:58.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91c496a640 1 -- start start 2024-04-25T13:42:58.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91c496a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f91c00d3ed0 0x7f91c00d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91c496a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f91c00d2ad0 0x7f91c00b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91c496a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c00b92a0 con 0x7f91c00b7c10 2024-04-25T13:42:58.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91c496a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f91c00d4c30 con 0x7f91c00d3ed0 2024-04-25T13:42:58.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91c496a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f91c00d4ef0 con 0x7f91c00d2ad0 2024-04-25T13:42:58.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91be59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f91c00d3ed0 0x7f91c00d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91bed9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f91c00b7c10 0x7f91c00b9760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52416/0 (socket says 172.21.15.80:52416) 2024-04-25T13:42:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91bed9d640 1 -- 172.21.15.80:0/3809299624 learned_addr learned my addr 172.21.15.80:0/3809299624 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91bdd9b640 1 --2- 172.21.15.80:0/3809299624 >> v2:172.21.15.87:3300/0 conn(0x7f91c00d2ad0 0x7f91c00b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91bd59a640 1 -- 172.21.15.80:0/3809299624 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 748619896 0 0) 0x7f91b80014a0 con 0x7f91c00b7c10 2024-04-25T13:42:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91bd59a640 1 -- 172.21.15.80:0/3809299624 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91a8003530 con 0x7f91c00b7c10 2024-04-25T13:42:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91be59c640 1 -- 172.21.15.80:0/3809299624 >> v2:172.21.15.87:3300/0 conn(0x7f91c00d2ad0 msgr2=0x7f91c00b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91be59c640 1 --2- 172.21.15.80:0/3809299624 >> v2:172.21.15.87:3300/0 conn(0x7f91c00d2ad0 0x7f91c00b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91be59c640 1 -- 172.21.15.80:0/3809299624 >> v1:172.21.15.80:6789/0 conn(0x7f91c00b7c10 legacy=0x7f91c00b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.733+0000 7f91be59c640 1 -- 172.21.15.80:0/3809299624 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c00d51d0 con 0x7f91c00d3ed0 2024-04-25T13:42:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91be59c640 1 --2- 172.21.15.80:0/3809299624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c00d2590 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f91c00b8dc0 tx=0x7f91b001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3274fb9072c0f25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91bd59a640 1 -- 172.21.15.80:0/3809299624 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b00b0070 con 0x7f91c00d3ed0 2024-04-25T13:42:58.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91bd59a640 1 -- 172.21.15.80:0/3809299624 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b0004230 con 0x7f91c00d3ed0 2024-04-25T13:42:58.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91bd59a640 1 -- 172.21.15.80:0/3809299624 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0003620 con 0x7f91c00d3ed0 2024-04-25T13:42:58.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 -- 172.21.15.80:0/3809299624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 msgr2=0x7f91c00d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 --2- 172.21.15.80:0/3809299624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c00d2590 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f91c00b8dc0 tx=0x7f91b001c330 comp rx=0 tx=0).stop 2024-04-25T13:42:58.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 -- 172.21.15.80:0/3809299624 shutdown_connections 2024-04-25T13:42:58.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 --2- 172.21.15.80:0/3809299624 >> v2:172.21.15.87:3300/0 conn(0x7f91c00d2ad0 0x7f91c00b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 --2- 172.21.15.80:0/3809299624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c00d2590 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 -- 172.21.15.80:0/3809299624 >> 172.21.15.80:0/3809299624 conn(0x7f91c00737b0 msgr2=0x7f91c006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:58.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 -- 172.21.15.80:0/3809299624 shutdown_connections 2024-04-25T13:42:58.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 -- 172.21.15.80:0/3809299624 wait complete. 2024-04-25T13:42:58.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 Processor -- start 2024-04-25T13:42:58.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.737+0000 7f91c496a640 1 -- start start 2024-04-25T13:42:58.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c00b7c10 0x7f91c0150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c00d2ad0 0x7f91c0152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c01585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00ccae0 con 0x7f91c00d3ed0 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00cc960 con 0x7f91c00d2ad0 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91c0160620 con 0x7f91c00b7c10 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c01585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c01585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43192/0 (socket says 172.21.15.80:43192) 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 -- 172.21.15.80:0/3522418166 learned_addr learned my addr 172.21.15.80:0/3522418166 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bdd9b640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c00d2ad0 0x7f91c0152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91be59c640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c00b7c10 0x7f91c0150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 -- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c00b7c10 msgr2=0x7f91c0150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c00b7c10 0x7f91c0150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 -- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c00d2ad0 msgr2=0x7f91c0152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c00d2ad0 0x7f91c0152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 -- 172.21.15.80:0/3522418166 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c0152430 con 0x7f91c00d3ed0 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bdd9b640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c00d2ad0 0x7f91c0152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91be59c640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c00b7c10 0x7f91c0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91bed9d640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c01585f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f91b800cb40 tx=0x7f91b800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91aeffd640 1 -- 172.21.15.80:0/3522418166 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8017070 con 0x7f91c00d3ed0 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91aeffd640 1 -- 172.21.15.80:0/3522418166 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b800ac40 con 0x7f91c00d3ed0 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c0151ed0 con 0x7f91c00d3ed0 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91aeffd640 1 -- 172.21.15.80:0/3522418166 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8012400 con 0x7f91c00d3ed0 2024-04-25T13:42:58.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c0151bf0 con 0x7f91c00d3ed0 2024-04-25T13:42:58.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c006a3a0 con 0x7f91c00d3ed0 2024-04-25T13:42:58.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91aeffd640 1 -- 172.21.15.80:0/3522418166 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f91b801a070 con 0x7f91c00d3ed0 2024-04-25T13:42:58.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91aeffd640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91a006f430 0x7f91a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:58.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.741+0000 7f91aeffd640 1 -- 172.21.15.80:0/3522418166 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f91b808efe0 con 0x7f91c00d3ed0 2024-04-25T13:42:58.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.745+0000 7f91be59c640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91a006f430 0x7f91a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:58.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.745+0000 7f91be59c640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91a006f430 0x7f91a0071940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91b00ad000 tx=0x7f91b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:58.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.749+0000 7f91aeffd640 1 -- 172.21.15.80:0/3522418166 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91c00b92a0 con 0x7f91c00d3ed0 2024-04-25T13:42:58.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.893+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f91c015c860 con 0x7f91c00d3ed0 2024-04-25T13:42:58.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91aeffd640 1 -- 172.21.15.80:0/3522418166 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f91b805bf60 con 0x7f91c00d3ed0 2024-04-25T13:42:58.899 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:42:58.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91a006f430 msgr2=0x7f91a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91a006f430 0x7f91a0071940 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91b00ad000 tx=0x7f91b0008040 comp rx=0 tx=0).stop 2024-04-25T13:42:58.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 msgr2=0x7f91c01585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:58.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c01585f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f91b800cb40 tx=0x7f91b800a060 comp rx=0 tx=0).stop 2024-04-25T13:42:58.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 shutdown_connections 2024-04-25T13:42:58.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91a006f430 0x7f91a0071940 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c00d3ed0 0x7f91c01585f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c00d2ad0 0x7f91c0152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 --2- 172.21.15.80:0/3522418166 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c00b7c10 0x7f91c0150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:58.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 >> 172.21.15.80:0/3522418166 conn(0x7f91c00737b0 msgr2=0x7f91c00cccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:58.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 shutdown_connections 2024-04-25T13:42:58.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:58.897+0000 7f91c496a640 1 -- 172.21.15.80:0/3522418166 wait complete. 2024-04-25T13:42:58.919 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:42:59.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.053+0000 7f66588e1640 1 Processor -- start 2024-04-25T13:42:59.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.053+0000 7f66588e1640 1 -- start start 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f66588e1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654071e60 0x7f66541574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f66588e1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6654073c40 0x7f6654160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f66588e1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66541613b0 con 0x7f6654160fa0 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f66588e1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f66540b1350 con 0x7f6654073c40 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f66588e1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f665406f9a0 con 0x7f6654071e60 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f6651d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6654073c40 0x7f6654160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f6651d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6654073c40 0x7f6654160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43206/0 (socket says 172.21.15.80:43206) 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f6651d9b640 1 -- 172.21.15.80:0/2988778259 learned_addr learned my addr 172.21.15.80:0/2988778259 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f665259c640 1 --2- 172.21.15.80:0/2988778259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654071e60 0x7f66541574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f6651d9b640 1 -- 172.21.15.80:0/2988778259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654071e60 msgr2=0x7f66541574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f6651d9b640 1 --2- 172.21.15.80:0/2988778259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654071e60 0x7f66541574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f6651d9b640 1 -- 172.21.15.80:0/2988778259 >> v1:172.21.15.87:6789/0 conn(0x7f6654160fa0 legacy=0x7f66540b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:59.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f6651d9b640 1 -- 172.21.15.80:0/2988778259 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665406fbf0 con 0x7f6654073c40 2024-04-25T13:42:59.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f6651d9b640 1 --2- 172.21.15.80:0/2988778259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654073c40 0x7f6654160a60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6648009870 tx=0x7f664801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4b5ee4f35a5693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f665159a640 1 -- 172.21.15.80:0/2988778259 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66480b0070 con 0x7f6654073c40 2024-04-25T13:42:59.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f665159a640 1 -- 172.21.15.80:0/2988778259 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66480ab600 con 0x7f6654073c40 2024-04-25T13:42:59.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.057+0000 7f665159a640 1 -- 172.21.15.80:0/2988778259 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66480ab9d0 con 0x7f6654073c40 2024-04-25T13:42:59.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- 172.21.15.80:0/2988778259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654073c40 msgr2=0x7f6654160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 --2- 172.21.15.80:0/2988778259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654073c40 0x7f6654160a60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f6648009870 tx=0x7f664801c340 comp rx=0 tx=0).stop 2024-04-25T13:42:59.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- 172.21.15.80:0/2988778259 shutdown_connections 2024-04-25T13:42:59.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 --2- 172.21.15.80:0/2988778259 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654073c40 0x7f6654160a60 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 --2- 172.21.15.80:0/2988778259 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654071e60 0x7f66541574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- 172.21.15.80:0/2988778259 >> 172.21.15.80:0/2988778259 conn(0x7f66540b4160 msgr2=0x7f66540d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:59.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- 172.21.15.80:0/2988778259 shutdown_connections 2024-04-25T13:42:59.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- 172.21.15.80:0/2988778259 wait complete. 2024-04-25T13:42:59.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 Processor -- start 2024-04-25T13:42:59.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- start start 2024-04-25T13:42:59.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654071e60 0x7f665414a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654073c40 0x7f6654149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6654160fa0 0x7f665414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6654074050 con 0x7f6654071e60 2024-04-25T13:42:59.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6654072270 con 0x7f6654160fa0 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f66588e1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66540b56c0 con 0x7f6654073c40 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f665259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654071e60 0x7f665414a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f665259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654071e60 0x7f665414a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43218/0 (socket says 172.21.15.80:43218) 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.061+0000 7f665259c640 1 -- 172.21.15.80:0/1729051290 learned_addr learned my addr 172.21.15.80:0/1729051290 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6651d9b640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654073c40 0x7f6654149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6652d9d640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6654160fa0 0x7f665414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f665259c640 1 -- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654073c40 msgr2=0x7f6654149e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f665259c640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654073c40 0x7f6654149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f665259c640 1 -- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6654160fa0 msgr2=0x7f665414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f665259c640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6654160fa0 0x7f665414e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f665259c640 1 -- 172.21.15.80:0/1729051290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f665414ae00 con 0x7f6654071e60 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6652d9d640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6654160fa0 0x7f665414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:59.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f665259c640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654071e60 0x7f665414a5d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f664400c7f0 tx=0x7f664400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6642ffd640 1 -- 172.21.15.80:0/1729051290 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644015070 con 0x7f6654071e60 2024-04-25T13:42:59.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6642ffd640 1 -- 172.21.15.80:0/1729051290 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6644007820 con 0x7f6654071e60 2024-04-25T13:42:59.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f665414b0e0 con 0x7f6654071e60 2024-04-25T13:42:59.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6642ffd640 1 -- 172.21.15.80:0/1729051290 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6644007c10 con 0x7f6654071e60 2024-04-25T13:42:59.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f665414b6a0 con 0x7f6654071e60 2024-04-25T13:42:59.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f665406a3a0 con 0x7f6654071e60 2024-04-25T13:42:59.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6642ffd640 1 -- 172.21.15.80:0/1729051290 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6644018070 con 0x7f6654071e60 2024-04-25T13:42:59.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6642ffd640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f663406f430 0x7f6634071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6642ffd640 1 -- 172.21.15.80:0/1729051290 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f664408f3b0 con 0x7f6654071e60 2024-04-25T13:42:59.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6651d9b640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f663406f430 0x7f6634071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.065+0000 7f6651d9b640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f663406f430 0x7f6634071940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f66480ad000 tx=0x7f6648008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.069+0000 7f6642ffd640 1 -- 172.21.15.80:0/1729051290 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f664405c2a0 con 0x7f6654071e60 2024-04-25T13:42:59.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.217+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6654069cb0 con 0x7f6654071e60 2024-04-25T13:42:59.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f6642ffd640 1 -- 172.21.15.80:0/1729051290 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v52) ==== 104+0+11 (secure 0 0 0) 0x7f664405bba0 con 0x7f6654071e60 2024-04-25T13:42:59.223 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:42:59.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f663406f430 msgr2=0x7f6634071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f663406f430 0x7f6634071940 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f66480ad000 tx=0x7f6648008040 comp rx=0 tx=0).stop 2024-04-25T13:42:59.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654071e60 msgr2=0x7f665414a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654071e60 0x7f665414a5d0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f664400c7f0 tx=0x7f664400ccb0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 shutdown_connections 2024-04-25T13:42:59.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f663406f430 0x7f6634071940 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6654160fa0 0x7f665414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6654073c40 0x7f6654149e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 --2- 172.21.15.80:0/1729051290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6654071e60 0x7f665414a5d0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 >> 172.21.15.80:0/1729051290 conn(0x7f66540b4160 msgr2=0x7f665415e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:59.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 shutdown_connections 2024-04-25T13:42:59.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.221+0000 7f66588e1640 1 -- 172.21.15.80:0/1729051290 wait complete. 2024-04-25T13:42:59.243 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:42:59.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9d185640 1 Processor -- start 2024-04-25T13:42:59.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9d185640 1 -- start start 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9d185640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a980d22e0 0x7f1a98144580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9d185640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a980d0270 0x7f1a98161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9d185640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a98068c90 con 0x7f1a9815efd0 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9d185640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1a98068f70 con 0x7f1a980d22e0 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9d185640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1a98069250 con 0x7f1a980d0270 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a96d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a980d22e0 0x7f1a98144580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9659c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1a9815efd0 0x7f1a980cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52426/0 (socket says 172.21.15.80:52426) 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9659c640 1 -- 172.21.15.80:0/4022367975 learned_addr learned my addr 172.21.15.80:0/4022367975 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a9759e640 1 --2- 172.21.15.80:0/4022367975 >> v2:172.21.15.87:3300/0 conn(0x7f1a980d0270 0x7f1a98161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a96d9d640 1 -- 172.21.15.80:0/4022367975 >> v2:172.21.15.87:3300/0 conn(0x7f1a980d0270 msgr2=0x7f1a98161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:59.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a96d9d640 1 --2- 172.21.15.80:0/4022367975 >> v2:172.21.15.87:3300/0 conn(0x7f1a980d0270 0x7f1a98161dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a96d9d640 1 -- 172.21.15.80:0/4022367975 >> v1:172.21.15.80:6789/0 conn(0x7f1a9815efd0 legacy=0x7f1a980cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a96d9d640 1 -- 172.21.15.80:0/4022367975 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9815eab0 con 0x7f1a980d22e0 2024-04-25T13:42:59.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.381+0000 7f1a95d9b640 1 -- 172.21.15.80:0/4022367975 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1958951509 0 0) 0x7f1a98068c90 con 0x7f1a9815efd0 2024-04-25T13:42:59.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a96d9d640 1 --2- 172.21.15.80:0/4022367975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a980d22e0 0x7f1a98144580 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1a98162e20 tx=0x7f1a8801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1626e031cbf6bcd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a95d9b640 1 -- 172.21.15.80:0/4022367975 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880b0070 con 0x7f1a980d22e0 2024-04-25T13:42:59.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a95d9b640 1 -- 172.21.15.80:0/4022367975 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a880ab600 con 0x7f1a980d22e0 2024-04-25T13:42:59.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a95d9b640 1 -- 172.21.15.80:0/4022367975 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a880ab9d0 con 0x7f1a980d22e0 2024-04-25T13:42:59.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 -- 172.21.15.80:0/4022367975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a980d22e0 msgr2=0x7f1a98144580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 --2- 172.21.15.80:0/4022367975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a980d22e0 0x7f1a98144580 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1a98162e20 tx=0x7f1a8801c3f0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 -- 172.21.15.80:0/4022367975 shutdown_connections 2024-04-25T13:42:59.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 --2- 172.21.15.80:0/4022367975 >> v2:172.21.15.87:3300/0 conn(0x7f1a980d0270 0x7f1a98161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 --2- 172.21.15.80:0/4022367975 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a980d22e0 0x7f1a98144580 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 -- 172.21.15.80:0/4022367975 >> 172.21.15.80:0/4022367975 conn(0x7f1a980d3370 msgr2=0x7f1a980cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:59.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 -- 172.21.15.80:0/4022367975 shutdown_connections 2024-04-25T13:42:59.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 -- 172.21.15.80:0/4022367975 wait complete. 2024-04-25T13:42:59.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 Processor -- start 2024-04-25T13:42:59.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.385+0000 7f1a9d185640 1 -- start start 2024-04-25T13:42:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a980d22e0 0x7f1a9815ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a9815efd0 0x7f1a9813e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9813ce10 0x7f1a9814d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1a9816b740 con 0x7f1a9815efd0 2024-04-25T13:42:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1a98162710 con 0x7f1a9813ce10 2024-04-25T13:42:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1a9816b8c0 con 0x7f1a980d22e0 2024-04-25T13:42:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a9815efd0 0x7f1a9813e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a9815efd0 0x7f1a9813e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43248/0 (socket says 172.21.15.80:43248) 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 -- 172.21.15.80:0/1389793871 learned_addr learned my addr 172.21.15.80:0/1389793871 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a96d9d640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a980d22e0 0x7f1a9815ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9759e640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9813ce10 0x7f1a9814d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 -- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a980d22e0 msgr2=0x7f1a9815ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a980d22e0 0x7f1a9815ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 -- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9813ce10 msgr2=0x7f1a9814d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9813ce10 0x7f1a9814d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 -- 172.21.15.80:0/1389793871 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a9813d560 con 0x7f1a9815efd0 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9659c640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a9815efd0 0x7f1a9813e600 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00c440 tx=0x7f1a8c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a7f7fe640 1 -- 172.21.15.80:0/1389793871 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c018070 con 0x7f1a9815efd0 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a7f7fe640 1 -- 172.21.15.80:0/1389793871 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a8c007500 con 0x7f1a9815efd0 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a9813d840 con 0x7f1a9815efd0 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a7f7fe640 1 -- 172.21.15.80:0/1389793871 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a8c004a30 con 0x7f1a9815efd0 2024-04-25T13:42:59.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a9813de00 con 0x7f1a9815efd0 2024-04-25T13:42:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a9806a3a0 con 0x7f1a9815efd0 2024-04-25T13:42:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a7f7fe640 1 -- 172.21.15.80:0/1389793871 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1a8c01b070 con 0x7f1a9815efd0 2024-04-25T13:42:59.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.389+0000 7f1a7f7fe640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a8406f400 0x7f1a84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.393+0000 7f1a7f7fe640 1 -- 172.21.15.80:0/1389793871 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f1a8c08e4e0 con 0x7f1a9815efd0 2024-04-25T13:42:59.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.393+0000 7f1a96d9d640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a8406f400 0x7f1a84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.393+0000 7f1a96d9d640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a8406f400 0x7f1a84071910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a880ad000 tx=0x7f1a88008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.397+0000 7f1a7f7fe640 1 -- 172.21.15.80:0/1389793871 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a8c05b460 con 0x7f1a9815efd0 2024-04-25T13:42:59.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.545+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1a9813d2f0 con 0x7f1a9815efd0 2024-04-25T13:42:59.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.545+0000 7f1a7f7fe640 1 -- 172.21.15.80:0/1389793871 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v52) ==== 100+0+11 (secure 0 0 0) 0x7f1a8c05ad60 con 0x7f1a9815efd0 2024-04-25T13:42:59.549 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:42:59.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a8406f400 msgr2=0x7f1a84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a8406f400 0x7f1a84071910 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f1a880ad000 tx=0x7f1a88008040 comp rx=0 tx=0).stop 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a9815efd0 msgr2=0x7f1a9813e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a9815efd0 0x7f1a9813e600 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c00c440 tx=0x7f1a8c00c900 comp rx=0 tx=0).stop 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 shutdown_connections 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1a8406f400 0x7f1a84071910 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a9813ce10 0x7f1a9814d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a9815efd0 0x7f1a9813e600 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 --2- 172.21.15.80:0/1389793871 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a980d22e0 0x7f1a9815ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 >> 172.21.15.80:0/1389793871 conn(0x7f1a980d3370 msgr2=0x7f1a980cf3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 shutdown_connections 2024-04-25T13:42:59.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.549+0000 7f1a9d185640 1 -- 172.21.15.80:0/1389793871 wait complete. 2024-04-25T13:42:59.571 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:42:59.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a5453b640 1 Processor -- start 2024-04-25T13:42:59.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a5453b640 1 -- start start 2024-04-25T13:42:59.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a5453b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4c1639e0 0x7f9a4c0d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a5453b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4c0d4e00 con 0x7f9a4c0b9770 2024-04-25T13:42:59.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a5453b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a4c0d50e0 con 0x7f9a4c0b7760 2024-04-25T13:42:59.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a5453b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4c0d1a70 con 0x7f9a4c1639e0 2024-04-25T13:42:59.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a52ad8640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9a4c0b9770 0x7f9a4c0b9b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52442/0 (socket says 172.21.15.80:52442) 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a52ad8640 1 -- 172.21.15.80:0/1512009725 learned_addr learned my addr 172.21.15.80:0/1512009725 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a522d7640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4c1639e0 0x7f9a4c0d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 660823721 0 0) 0x7f9a4c0d4e00 con 0x7f9a4c0b9770 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.709+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9a38003530 con 0x7f9a4c0b9770 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1467351628 0 0) 0x7f9a38003530 con 0x7f9a4c0b9770 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4c1639e0 msgr2=0x7f9a4c0d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4c1639e0 0x7f9a4c0d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 >> v1:172.21.15.87:6789/0 conn(0x7f9a4c0b7760 legacy=0x7f9a4c0b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c0d53c0 con 0x7f9a4c0b9770 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f9a44001a00 con 0x7f9a4c0b9770 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 >> v1:172.21.15.80:6789/0 conn(0x7f9a4c0b9770 legacy=0x7f9a4c0b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a380039f0 0x7f9a38003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a38004810 0x7f9a38004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4c1639e0 0x7f9a38005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c0d1a70 con 0x7f9a38004810 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a3800bb00 con 0x7f9a380039f0 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a3800bd00 con 0x7f9a4c1639e0 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a51ad6640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a38004810 0x7f9a38004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a522d7640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a380039f0 0x7f9a38003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a51ad6640 1 -- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4c1639e0 msgr2=0x7f9a38005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a51ad6640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a4c1639e0 0x7f9a38005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a51ad6640 1 -- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a380039f0 msgr2=0x7f9a38003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a51ad6640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a380039f0 0x7f9a38003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a51ad6640 1 -- 172.21.15.80:0/1512009725 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a38003530 con 0x7f9a38004810 2024-04-25T13:42:59.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a522d7640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a380039f0 0x7f9a38003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:59.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a51ad6640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a38004810 0x7f9a38004c20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0d3730 tx=0x7f9a3c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c132f59d6c6dad0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c0013a0 con 0x7f9a38004810 2024-04-25T13:42:59.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a3c0052f0 con 0x7f9a38004810 2024-04-25T13:42:59.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a512d5640 1 -- 172.21.15.80:0/1512009725 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c003840 con 0x7f9a38004810 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a5453b640 1 -- 172.21.15.80:0/1512009725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a38004810 msgr2=0x7f9a38004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a5453b640 1 --2- 172.21.15.80:0/1512009725 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a38004810 0x7f9a38004c20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0d3730 tx=0x7f9a3c01c3c0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a52ad8640 1 -- 172.21.15.80:0/1512009725 reap_dead start 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a5453b640 1 -- 172.21.15.80:0/1512009725 shutdown_connections 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a5453b640 1 -- 172.21.15.80:0/1512009725 >> 172.21.15.80:0/1512009725 conn(0x7f9a4c06c9f0 msgr2=0x7f9a4c06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a5453b640 1 -- 172.21.15.80:0/1512009725 shutdown_connections 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.713+0000 7f9a5453b640 1 -- 172.21.15.80:0/1512009725 wait complete. 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 Processor -- start 2024-04-25T13:42:59.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 -- start start 2024-04-25T13:42:59.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a38004810 0x7f9a4c0ccd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a4c0d0070 0x7f9a4c1f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a4c15d5f0 0x7f9a4c15da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c163df0 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9a4c0d5a10 con 0x7f9a4c0d0070 2024-04-25T13:42:59.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4c0d4ad0 con 0x7f9a38004810 2024-04-25T13:42:59.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a4c15d5f0 0x7f9a4c15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a4c15d5f0 0x7f9a4c15da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43266/0 (socket says 172.21.15.80:43266) 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 -- 172.21.15.80:0/3575653185 learned_addr learned my addr 172.21.15.80:0/3575653185 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a522d7640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a38004810 0x7f9a4c0ccd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a51ad6640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a4c0d0070 0x7f9a4c1f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 -- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a38004810 msgr2=0x7f9a4c0ccd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a38004810 0x7f9a4c0ccd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 -- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a4c0d0070 msgr2=0x7f9a4c1f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a4c0d0070 0x7f9a4c1f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 -- 172.21.15.80:0/3575653185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a4c15df40 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a51ad6640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a4c0d0070 0x7f9a4c1f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a52ad8640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a4c15d5f0 0x7f9a4c15da00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9a44001340 tx=0x7f9a4400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a42ffd640 1 -- 172.21.15.80:0/3575653185 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4401b070 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a42ffd640 1 -- 172.21.15.80:0/3575653185 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9a4400da20 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a4c0cac50 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a42ffd640 1 -- 172.21.15.80:0/3575653185 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a4400de10 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.717+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a4c0cd7d0 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.721+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a4c06a3a0 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.721+0000 7f9a42ffd640 1 -- 172.21.15.80:0/3575653185 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9a4401e070 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.721+0000 7f9a42ffd640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9a3006f400 0x7f9a30071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:42:59.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.721+0000 7f9a42ffd640 1 -- 172.21.15.80:0/3575653185 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f9a440918f0 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.721+0000 7f9a522d7640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9a3006f400 0x7f9a30071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:42:59.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.721+0000 7f9a522d7640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9a3006f400 0x7f9a30071910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009ac0 tx=0x7f9a48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:42:59.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.725+0000 7f9a42ffd640 1 -- 172.21.15.80:0/3575653185 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9a4c15df40 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.925+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a4c0b7210 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.925+0000 7f9a42ffd640 1 -- 172.21.15.80:0/3575653185 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f9a4405e850 con 0x7f9a4c15d5f0 2024-04-25T13:42:59.929 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:42:59.929 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:42:59.930 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:42:59.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9a3006f400 msgr2=0x7f9a30071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9a3006f400 0x7f9a30071910 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009ac0 tx=0x7f9a48008040 comp rx=0 tx=0).stop 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a4c15d5f0 msgr2=0x7f9a4c15da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a4c15d5f0 0x7f9a4c15da00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f9a44001340 tx=0x7f9a4400fc60 comp rx=0 tx=0).stop 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 shutdown_connections 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9a3006f400 0x7f9a30071910 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9a4c15d5f0 0x7f9a4c15da00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a4c0d0070 0x7f9a4c1f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 --2- 172.21.15.80:0/3575653185 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9a38004810 0x7f9a4c0ccd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 >> 172.21.15.80:0/3575653185 conn(0x7f9a4c06c9f0 msgr2=0x7f9a4c0bb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 shutdown_connections 2024-04-25T13:42:59.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:42:59.929+0000 7f9a5453b640 1 -- 172.21.15.80:0/3575653185 wait complete. 2024-04-25T13:42:59.951 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:43:00.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47fa7d1640 1 Processor -- start 2024-04-25T13:43:00.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47fa7d1640 1 -- start start 2024-04-25T13:43:00.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47fa7d1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f415bb90 0x7f47f40c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47fa7d1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47f415e9a0 0x7f47f415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47fa7d1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47f40c8020 con 0x7f47f4068590 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47fa7d1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f47f40d0970 con 0x7f47f415e9a0 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47fa7d1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f47f40b1c90 con 0x7f47f415bb90 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f8d6e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47f415e9a0 0x7f47f415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f8d6e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47f415e9a0 0x7f47f415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43278/0 (socket says 172.21.15.80:43278) 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f8d6e640 1 -- 172.21.15.80:0/2997734655 learned_addr learned my addr 172.21.15.80:0/2997734655 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f3fff640 1 --2- 172.21.15.80:0/2997734655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f415bb90 0x7f47f40c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f8d6e640 1 -- 172.21.15.80:0/2997734655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f415bb90 msgr2=0x7f47f40c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f8d6e640 1 --2- 172.21.15.80:0/2997734655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f415bb90 0x7f47f40c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f8d6e640 1 -- 172.21.15.80:0/2997734655 >> v1:172.21.15.87:6789/0 conn(0x7f47f4068590 legacy=0x7f47f40689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f8d6e640 1 -- 172.21.15.80:0/2997734655 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f40b1f70 con 0x7f47f415e9a0 2024-04-25T13:43:00.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.093+0000 7f47f3fff640 1 --2- 172.21.15.80:0/2997734655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f415bb90 0x7f47f40c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:00.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47f8d6e640 1 --2- 172.21.15.80:0/2997734655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f415edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b91dea23d1b87f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47f2ffd640 1 -- 172.21.15.80:0/2997734655 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0a5710 con 0x7f47f415e9a0 2024-04-25T13:43:00.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47f2ffd640 1 -- 172.21.15.80:0/2997734655 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec0a58b0 con 0x7f47f415e9a0 2024-04-25T13:43:00.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47f2ffd640 1 -- 172.21.15.80:0/2997734655 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0a5c80 con 0x7f47f415e9a0 2024-04-25T13:43:00.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47fa7d1640 1 -- 172.21.15.80:0/2997734655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 msgr2=0x7f47f415edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/2997734655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f415edb0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f47ec009e50 tx=0x7f47ec01c9a0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47fa7d1640 1 -- 172.21.15.80:0/2997734655 shutdown_connections 2024-04-25T13:43:00.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/2997734655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f415edb0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/2997734655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f415bb90 0x7f47f40c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47fa7d1640 1 -- 172.21.15.80:0/2997734655 >> 172.21.15.80:0/2997734655 conn(0x7f47f40cf390 msgr2=0x7f47f40cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:00.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47fa7d1640 1 -- 172.21.15.80:0/2997734655 shutdown_connections 2024-04-25T13:43:00.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.097+0000 7f47fa7d1640 1 -- 172.21.15.80:0/2997734655 wait complete. 2024-04-25T13:43:00.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 Processor -- start 2024-04-25T13:43:00.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 -- start start 2024-04-25T13:43:00.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f4068590 0x7f47f41fbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47f415bb90 0x7f47f4156c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f414d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f47f40d04e0 con 0x7f47f415e9a0 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f47f415bfa0 con 0x7f47f415bb90 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f47f40d0660 con 0x7f47f4068590 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f414d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f414d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43284/0 (socket says 172.21.15.80:43284) 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 -- 172.21.15.80:0/671861401 learned_addr learned my addr 172.21.15.80:0/671861401 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f37fe640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47f415bb90 0x7f47f4156c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f3fff640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f4068590 0x7f47f41fbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 -- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f4068590 msgr2=0x7f47f41fbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f4068590 0x7f47f41fbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 -- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47f415bb90 msgr2=0x7f47f4156c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47f415bb90 0x7f47f4156c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 -- 172.21.15.80:0/671861401 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f4157d40 con 0x7f47f415e9a0 2024-04-25T13:43:00.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f8d6e640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f414d490 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f47ec01a5d0 tx=0x7f47ec019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f0ff9640 1 -- 172.21.15.80:0/671861401 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0bc020 con 0x7f47f415e9a0 2024-04-25T13:43:00.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f0ff9640 1 -- 172.21.15.80:0/671861401 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ec0b8670 con 0x7f47f415e9a0 2024-04-25T13:43:00.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f41574a0 con 0x7f47f415e9a0 2024-04-25T13:43:00.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47f0ff9640 1 -- 172.21.15.80:0/671861401 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ec0b8a20 con 0x7f47f415e9a0 2024-04-25T13:43:00.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f4157a60 con 0x7f47f415e9a0 2024-04-25T13:43:00.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.101+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f406a3a0 con 0x7f47f415e9a0 2024-04-25T13:43:00.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.105+0000 7f47f0ff9640 1 -- 172.21.15.80:0/671861401 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f47ec0bf020 con 0x7f47f415e9a0 2024-04-25T13:43:00.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.105+0000 7f47f0ff9640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47e006f400 0x7f47e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.105+0000 7f47f0ff9640 1 -- 172.21.15.80:0/671861401 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f47ec12a210 con 0x7f47f415e9a0 2024-04-25T13:43:00.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.105+0000 7f47f3fff640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47e006f400 0x7f47e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.105+0000 7f47f3fff640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47e006f400 0x7f47e0071910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47f40b1920 tx=0x7f47e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.109+0000 7f47f0ff9640 1 -- 172.21.15.80:0/671861401 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f47ec0c6030 con 0x7f47f415e9a0 2024-04-25T13:43:00.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.257+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47f4156a50 con 0x7f47f415e9a0 2024-04-25T13:43:00.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47f0ff9640 1 -- 172.21.15.80:0/671861401 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16541 (secure 0 0 0) 0x7f47ec016620 con 0x7f47f415e9a0 2024-04-25T13:43:00.264 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:00.264 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":52,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:49.227779+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:42:13.650442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T13:42:14.725209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:43:00.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47e006f400 msgr2=0x7f47e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47e006f400 0x7f47e0071910 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f47f40b1920 tx=0x7f47e4008040 comp rx=0 tx=0).stop 2024-04-25T13:43:00.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 msgr2=0x7f47f414d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f414d490 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f47ec01a5d0 tx=0x7f47ec019740 comp rx=0 tx=0).stop 2024-04-25T13:43:00.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 shutdown_connections 2024-04-25T13:43:00.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47e006f400 0x7f47e0071910 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47f415e9a0 0x7f47f414d490 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47f415bb90 0x7f47f4156c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 --2- 172.21.15.80:0/671861401 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47f4068590 0x7f47f41fbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 >> 172.21.15.80:0/671861401 conn(0x7f47f40cf390 msgr2=0x7f47f4158080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 shutdown_connections 2024-04-25T13:43:00.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.261+0000 7f47fa7d1640 1 -- 172.21.15.80:0/671861401 wait complete. 2024-04-25T13:43:00.285 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:43:00.285 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:43:00.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1f2d6a640 1 Processor -- start 2024-04-25T13:43:00.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1f2d6a640 1 -- start start 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1f2d6a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec161f90 0x7fa1ec0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1f2d6a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa1ec0d0510 0x7fa1ec15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1f2d6a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa1ec15ea70 0x7fa1ec15ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1f2d6a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ec0d0d70 con 0x7fa1ec0d0510 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1f2d6a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa1ec0d1050 con 0x7fa1ec15ea70 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1f2d6a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec06dca0 con 0x7fa1ec161f90 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1ebfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa1ec0d0510 0x7fa1ec15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1ebfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa1ec0d0510 0x7fa1ec15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43292/0 (socket says 172.21.15.80:43292) 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.413+0000 7fa1ebfff640 1 -- 172.21.15.80:0/1405573056 learned_addr learned my addr 172.21.15.80:0/1405573056 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:00.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f1307640 1 --2- 172.21.15.80:0/1405573056 >> v2:172.21.15.87:3300/0 conn(0x7fa1ec15ea70 0x7fa1ec15ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f0b06640 1 --2- 172.21.15.80:0/1405573056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec161f90 0x7fa1ec0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1ebfff640 1 -- 172.21.15.80:0/1405573056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec161f90 msgr2=0x7fa1ec0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:00.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1ebfff640 1 --2- 172.21.15.80:0/1405573056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec161f90 0x7fa1ec0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1ebfff640 1 -- 172.21.15.80:0/1405573056 >> v2:172.21.15.87:3300/0 conn(0x7fa1ec15ea70 msgr2=0x7fa1ec15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:00.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1ebfff640 1 --2- 172.21.15.80:0/1405573056 >> v2:172.21.15.87:3300/0 conn(0x7fa1ec15ea70 0x7fa1ec15ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1ebfff640 1 -- 172.21.15.80:0/1405573056 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec06df80 con 0x7fa1ec0d0510 2024-04-25T13:43:00.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1ebfff640 1 --2- 172.21.15.80:0/1405573056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec0d0510 0x7fa1ec15e530 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0d1260 tx=0x7fa1e001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fea3c25367d0624 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1eb7fe640 1 -- 172.21.15.80:0/1405573056 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00b1070 con 0x7fa1ec0d0510 2024-04-25T13:43:00.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1eb7fe640 1 -- 172.21.15.80:0/1405573056 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e00a8050 con 0x7fa1ec0d0510 2024-04-25T13:43:00.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1eb7fe640 1 -- 172.21.15.80:0/1405573056 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00ac860 con 0x7fa1ec0d0510 2024-04-25T13:43:00.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/1405573056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec0d0510 msgr2=0x7fa1ec15e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/1405573056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec0d0510 0x7fa1ec15e530 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec0d1260 tx=0x7fa1e001c460 comp rx=0 tx=0).stop 2024-04-25T13:43:00.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/1405573056 shutdown_connections 2024-04-25T13:43:00.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/1405573056 >> v2:172.21.15.87:3300/0 conn(0x7fa1ec15ea70 0x7fa1ec15ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/1405573056 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec0d0510 0x7fa1ec15e530 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/1405573056 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec161f90 0x7fa1ec0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/1405573056 >> 172.21.15.80:0/1405573056 conn(0x7fa1ec06d080 msgr2=0x7fa1ec0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:00.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/1405573056 shutdown_connections 2024-04-25T13:43:00.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.417+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/1405573056 wait complete. 2024-04-25T13:43:00.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 Processor -- start 2024-04-25T13:43:00.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 -- start start 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec0d0510 0x7fa1ec15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec15ea70 0x7fa1ec147160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ec161f90 0x7fa1ec1476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec1447a0 con 0x7fa1ec15ea70 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1ec144620 con 0x7fa1ec161f90 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa1ec0d0920 con 0x7fa1ec0d0510 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec15ea70 0x7fa1ec147160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec15ea70 0x7fa1ec147160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43298/0 (socket says 172.21.15.80:43298) 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 -- 172.21.15.80:0/101124108 learned_addr learned my addr 172.21.15.80:0/101124108 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:00.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f0b06640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec0d0510 0x7fa1ec15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f1307640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ec161f90 0x7fa1ec1476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 -- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec0d0510 msgr2=0x7fa1ec15d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec0d0510 0x7fa1ec15d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 -- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ec161f90 msgr2=0x7fa1ec1476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ec161f90 0x7fa1ec1476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 -- 172.21.15.80:0/101124108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1ec142240 con 0x7fa1ec15ea70 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f1307640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ec161f90 0x7fa1ec1476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1ebfff640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec15ea70 0x7fa1ec147160 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa1e001cd00 tx=0x7fa1e001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1e97fa640 1 -- 172.21.15.80:0/101124108 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00b1040 con 0x7fa1ec15ea70 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1e97fa640 1 -- 172.21.15.80:0/101124108 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1e00bc070 con 0x7fa1ec15ea70 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1ec142520 con 0x7fa1ec15ea70 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1e97fa640 1 -- 172.21.15.80:0/101124108 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00b7650 con 0x7fa1ec15ea70 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.421+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1ec142ae0 con 0x7fa1ec15ea70 2024-04-25T13:43:00.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.425+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1ec06a3a0 con 0x7fa1ec15ea70 2024-04-25T13:43:00.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.425+0000 7fa1e97fa640 1 -- 172.21.15.80:0/101124108 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa1e00b77f0 con 0x7fa1ec15ea70 2024-04-25T13:43:00.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.425+0000 7fa1e97fa640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1d806f400 0x7fa1d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.425+0000 7fa1e97fa640 1 -- 172.21.15.80:0/101124108 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fa1e01296e0 con 0x7fa1ec15ea70 2024-04-25T13:43:00.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.425+0000 7fa1f0b06640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1d806f400 0x7fa1d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.425+0000 7fa1f0b06640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1d806f400 0x7fa1d8071910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec06fa20 tx=0x7fa1dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.429+0000 7fa1e97fa640 1 -- 172.21.15.80:0/101124108 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1e0016110 con 0x7fa1ec15ea70 2024-04-25T13:43:00.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.609+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1ec0b37f0 con 0x7fa1ec15ea70 2024-04-25T13:43:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1e97fa640 1 -- 172.21.15.80:0/101124108 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7fa1e00f65d0 con 0x7fa1ec15ea70 2024-04-25T13:43:00.615 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:00.615 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:43:00.615 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:43:00.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1d806f400 msgr2=0x7fa1d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1d806f400 0x7fa1d8071910 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec06fa20 tx=0x7fa1dc008040 comp rx=0 tx=0).stop 2024-04-25T13:43:00.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec15ea70 msgr2=0x7fa1ec147160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec15ea70 0x7fa1ec147160 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa1e001cd00 tx=0x7fa1e001aca0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 shutdown_connections 2024-04-25T13:43:00.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1d806f400 0x7fa1d8071910 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1ec161f90 0x7fa1ec1476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1ec15ea70 0x7fa1ec147160 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 --2- 172.21.15.80:0/101124108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1ec0d0510 0x7fa1ec15d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 >> 172.21.15.80:0/101124108 conn(0x7fa1ec06d080 msgr2=0x7fa1ec141390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:00.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 shutdown_connections 2024-04-25T13:43:00.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.613+0000 7fa1f2d6a640 1 -- 172.21.15.80:0/101124108 wait complete. 2024-04-25T13:43:00.635 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:43:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f181c482640 1 Processor -- start 2024-04-25T13:43:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f181c482640 1 -- start start 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f181c482640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1814161f90 0x7f18140cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f181c482640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18140d0510 0x7f181415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f181c482640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18140d0d70 con 0x7f181415ea70 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f181c482640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f18140d1050 con 0x7f18140d0510 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f181c482640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f181406dca0 con 0x7f1814161f90 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f1819a1d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18140d0510 0x7f181415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f1819a1d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f18140d0510 0x7f181415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43300/0 (socket says 172.21.15.80:43300) 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f1819a1d640 1 -- 172.21.15.80:0/3239401094 learned_addr learned my addr 172.21.15.80:0/3239401094 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f181a21e640 1 --2- 172.21.15.80:0/3239401094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1814161f90 0x7f18140cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f1819a1d640 1 -- 172.21.15.80:0/3239401094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1814161f90 msgr2=0x7f18140cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:00.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f1819a1d640 1 --2- 172.21.15.80:0/3239401094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1814161f90 0x7f18140cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f1819a1d640 1 -- 172.21.15.80:0/3239401094 >> v1:172.21.15.87:6789/0 conn(0x7f181415ea70 legacy=0x7f181415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:00.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.769+0000 7f1819a1d640 1 -- 172.21.15.80:0/3239401094 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181406df80 con 0x7f18140d0510 2024-04-25T13:43:00.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f1819a1d640 1 --2- 172.21.15.80:0/3239401094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f181415e530 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1804009870 tx=0x7f180401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbb97b02adb99e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181921c640 1 -- 172.21.15.80:0/3239401094 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18040b0070 con 0x7f18140d0510 2024-04-25T13:43:00.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181921c640 1 -- 172.21.15.80:0/3239401094 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18040ab600 con 0x7f18140d0510 2024-04-25T13:43:00.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181921c640 1 -- 172.21.15.80:0/3239401094 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18040ab9d0 con 0x7f18140d0510 2024-04-25T13:43:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 -- 172.21.15.80:0/3239401094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 msgr2=0x7f181415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 --2- 172.21.15.80:0/3239401094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f181415e530 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f1804009870 tx=0x7f180401c430 comp rx=0 tx=0).stop 2024-04-25T13:43:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 -- 172.21.15.80:0/3239401094 shutdown_connections 2024-04-25T13:43:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 --2- 172.21.15.80:0/3239401094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f181415e530 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 --2- 172.21.15.80:0/3239401094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1814161f90 0x7f18140cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 -- 172.21.15.80:0/3239401094 >> 172.21.15.80:0/3239401094 conn(0x7f181406d080 msgr2=0x7f18140b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 -- 172.21.15.80:0/3239401094 shutdown_connections 2024-04-25T13:43:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 -- 172.21.15.80:0/3239401094 wait complete. 2024-04-25T13:43:00.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 Processor -- start 2024-04-25T13:43:00.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.773+0000 7f181c482640 1 -- start start 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f18140ad6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f181415ea70 0x7f18140ac970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1814161f90 0x7f18140ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f181406f630 con 0x7f18140d0510 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f181406f4b0 con 0x7f1814161f90 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18140d0920 con 0x7f181415ea70 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f18140ad6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f18140ad6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43310/0 (socket says 172.21.15.80:43310) 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 -- 172.21.15.80:0/1846619237 learned_addr learned my addr 172.21.15.80:0/1846619237 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181aa1f640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1814161f90 0x7f18140ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 -- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f181415ea70 msgr2=0x7f18140ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f181415ea70 0x7f18140ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 -- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1814161f90 msgr2=0x7f18140ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1814161f90 0x7f18140ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 -- 172.21.15.80:0/1846619237 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18140af840 con 0x7f18140d0510 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181a21e640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f18140ad6b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f181000c7f0 tx=0x7f181000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f180affd640 1 -- 172.21.15.80:0/1846619237 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810015070 con 0x7f18140d0510 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f180affd640 1 -- 172.21.15.80:0/1846619237 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1810004220 con 0x7f18140d0510 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18140adbf0 con 0x7f18140d0510 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f180affd640 1 -- 172.21.15.80:0/1846619237 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1810003620 con 0x7f18140d0510 2024-04-25T13:43:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18140afae0 con 0x7f18140d0510 2024-04-25T13:43:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f181406a3a0 con 0x7f18140d0510 2024-04-25T13:43:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f180affd640 1 -- 172.21.15.80:0/1846619237 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f18100039d0 con 0x7f18140d0510 2024-04-25T13:43:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f180affd640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f17f806f400 0x7f17f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f180affd640 1 -- 172.21.15.80:0/1846619237 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f181008f290 con 0x7f18140d0510 2024-04-25T13:43:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.777+0000 7f1819a1d640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f17f806f400 0x7f17f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:00.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.781+0000 7f1819a1d640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f17f806f400 0x7f17f8071910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18040ad000 tx=0x7f1804004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:00.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.785+0000 7f180affd640 1 -- 172.21.15.80:0/1846619237 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f181005c180 con 0x7f18140d0510 2024-04-25T13:43:00.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.961+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f18140aceb0 con 0x7f18140d0510 2024-04-25T13:43:00.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f180affd640 1 -- 172.21.15.80:0/1846619237 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5181 (secure 0 0 0) 0x7f1810064030 con 0x7f18140d0510 2024-04-25T13:43:00.967 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:00.967 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:42:15.784264+0000","modified":"2024-04-25T13:42:18.840205+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5730},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5730":{"gid":5730,"name":"c","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1692562493","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1692562493},{"type":"v1","addr":"172.21.15.80:6835","nonce":1692562493}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5730,"qdb_cluster":[5730]},"id":3}]} 2024-04-25T13:43:00.968 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 24 2024-04-25T13:43:00.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f17f806f400 msgr2=0x7f17f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f17f806f400 0x7f17f8071910 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f18040ad000 tx=0x7f1804004740 comp rx=0 tx=0).stop 2024-04-25T13:43:00.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 msgr2=0x7f18140ad6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:00.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f18140ad6b0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f181000c7f0 tx=0x7f181000ccb0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 shutdown_connections 2024-04-25T13:43:00.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f17f806f400 0x7f17f8071910 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1814161f90 0x7f18140ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f181415ea70 0x7f18140ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 --2- 172.21.15.80:0/1846619237 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18140d0510 0x7f18140ad6b0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:00.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 >> 172.21.15.80:0/1846619237 conn(0x7f181406d080 msgr2=0x7f18141412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:00.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 shutdown_connections 2024-04-25T13:43:00.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:00.965+0000 7f181c482640 1 -- 172.21.15.80:0/1846619237 wait complete. 2024-04-25T13:43:00.988 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:43:01.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e7ecf640 1 Processor -- start 2024-04-25T13:43:01.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e7ecf640 1 -- start start 2024-04-25T13:43:01.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e7ecf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f00e006f5a0 0x7f00e0160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e7ecf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0161070 0x7f00e0161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e7ecf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00e006fcf0 con 0x7f00e00b4bd0 2024-04-25T13:43:01.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e7ecf640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f00e01619c0 con 0x7f00e006f5a0 2024-04-25T13:43:01.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e7ecf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f00e0073f10 con 0x7f00e0161070 2024-04-25T13:43:01.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e646c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f00e00b4bd0 0x7f00e00b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42410/0 (socket says 172.21.15.80:42410) 2024-04-25T13:43:01.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e646c640 1 -- 172.21.15.80:0/2339471654 learned_addr learned my addr 172.21.15.80:0/2339471654 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:01.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e546a640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0161070 0x7f00e0161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e5c6b640 1 --2- 172.21.15.80:0/2339471654 >> v2:172.21.15.87:3300/0 conn(0x7f00e006f5a0 0x7f00e0160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1435442412 0 0) 0x7f00e006fcf0 con 0x7f00e00b4bd0 2024-04-25T13:43:01.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f00cc005740 con 0x7f00e00b4bd0 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 618452487 0 0) 0x7f00cc005740 con 0x7f00e00b4bd0 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0161070 msgr2=0x7f00e0161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0161070 0x7f00e0161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 >> v2:172.21.15.87:3300/0 conn(0x7f00e006f5a0 msgr2=0x7f00e0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 --2- 172.21.15.80:0/2339471654 >> v2:172.21.15.87:3300/0 conn(0x7f00e006f5a0 0x7f00e0160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e00b0c60 con 0x7f00e00b4bd0 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f00dc001a00 con 0x7f00e00b4bd0 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 >> v1:172.21.15.80:6789/0 conn(0x7f00e00b4bd0 legacy=0x7f00e00b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0161070 0x7f00cc005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00cc006180 0x7f00cc006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00cc006f60 0x7f00cc007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f00e01619c0 con 0x7f00cc006f60 2024-04-25T13:43:01.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f00e0073f10 con 0x7f00cc006180 2024-04-25T13:43:01.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.117+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f00cc00dd10 con 0x7f00e0161070 2024-04-25T13:43:01.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e646c640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00cc006f60 0x7f00cc007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e5c6b640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0161070 0x7f00cc005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e646c640 1 -- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0161070 msgr2=0x7f00cc005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e646c640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00e0161070 0x7f00cc005c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e646c640 1 -- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00cc006180 msgr2=0x7f00cc006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e646c640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00cc006180 0x7f00cc006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e646c640 1 -- 172.21.15.80:0/2339471654 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00cc005740 con 0x7f00cc006f60 2024-04-25T13:43:01.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e646c640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00cc006f60 0x7f00cc007370 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f00e0070290 tx=0x7f00dc01fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db37c8e4ea14996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc00a4c0 con 0x7f00cc006f60 2024-04-25T13:43:01.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc0aa6e0 con 0x7f00cc006f60 2024-04-25T13:43:01.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e4c69640 1 -- 172.21.15.80:0/2339471654 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0aab10 con 0x7f00cc006f60 2024-04-25T13:43:01.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e7ecf640 1 -- 172.21.15.80:0/2339471654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00cc006f60 msgr2=0x7f00cc007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e7ecf640 1 --2- 172.21.15.80:0/2339471654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00cc006f60 0x7f00cc007370 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f00e0070290 tx=0x7f00dc01fb20 comp rx=0 tx=0).stop 2024-04-25T13:43:01.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e646c640 1 -- 172.21.15.80:0/2339471654 reap_dead start 2024-04-25T13:43:01.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e7ecf640 1 -- 172.21.15.80:0/2339471654 shutdown_connections 2024-04-25T13:43:01.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e7ecf640 1 -- 172.21.15.80:0/2339471654 >> 172.21.15.80:0/2339471654 conn(0x7f00e006d080 msgr2=0x7f00e00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:01.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e7ecf640 1 -- 172.21.15.80:0/2339471654 shutdown_connections 2024-04-25T13:43:01.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.121+0000 7f00e7ecf640 1 -- 172.21.15.80:0/2339471654 wait complete. 2024-04-25T13:43:01.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 Processor -- start 2024-04-25T13:43:01.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 -- start start 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00cc006f60 0x7f00e014f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e015df90 0x7f00e00b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e015e3a0 0x7f00e00b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f00e006f9b0 con 0x7f00e015df90 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f00e00b1220 con 0x7f00e015e3a0 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f00e00733b0 con 0x7f00cc006f60 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e015df90 0x7f00e00b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e015df90 0x7f00e00b1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50152/0 (socket says 172.21.15.80:50152) 2024-04-25T13:43:01.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 -- 172.21.15.80:0/1186994673 learned_addr learned my addr 172.21.15.80:0/1186994673 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e5c6b640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00cc006f60 0x7f00e014f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e646c640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e015e3a0 0x7f00e00b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 -- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00cc006f60 msgr2=0x7f00e014f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00cc006f60 0x7f00e014f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 -- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e015e3a0 msgr2=0x7f00e00b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e015e3a0 0x7f00e00b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 -- 172.21.15.80:0/1186994673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e014faf0 con 0x7f00e015df90 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e646c640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e015e3a0 0x7f00e00b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e546a640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e015df90 0x7f00e00b1670 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f00d000e7f0 tx=0x7f00d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00d67fc640 1 -- 172.21.15.80:0/1186994673 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0017070 con 0x7f00e015df90 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00d67fc640 1 -- 172.21.15.80:0/1186994673 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d000c940 con 0x7f00e015df90 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e01f2500 con 0x7f00e015df90 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00d67fc640 1 -- 172.21.15.80:0/1186994673 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d000eea0 con 0x7f00e015df90 2024-04-25T13:43:01.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.125+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e01f2ac0 con 0x7f00e015df90 2024-04-25T13:43:01.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.129+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e006a3a0 con 0x7f00e015df90 2024-04-25T13:43:01.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.129+0000 7f00d67fc640 1 -- 172.21.15.80:0/1186994673 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f00d001a070 con 0x7f00e015df90 2024-04-25T13:43:01.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.129+0000 7f00d67fc640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f00c406f400 0x7f00c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.129+0000 7f00d67fc640 1 -- 172.21.15.80:0/1186994673 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7f00d008f170 con 0x7f00e015df90 2024-04-25T13:43:01.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.129+0000 7f00e5c6b640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f00c406f400 0x7f00c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.129+0000 7f00e5c6b640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f00c406f400 0x7f00c4071910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f00e00b6e70 tx=0x7f00d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.133+0000 7f00d67fc640 1 -- 172.21.15.80:0/1186994673 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00d005c060 con 0x7f00e015df90 2024-04-25T13:43:01.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00e00739c0 con 0x7f00e015df90 2024-04-25T13:43:01.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00d67fc640 1 -- 172.21.15.80:0/1186994673 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+16541 (secure 0 0 0) 0x7f00d005b960 con 0x7f00e015df90 2024-04-25T13:43:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:01.288 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":52,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:42:49.227779+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:42:13.650442+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T13:42:14.725209+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:43:01.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f00c406f400 msgr2=0x7f00c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f00c406f400 0x7f00c4071910 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f00e00b6e70 tx=0x7f00d8008040 comp rx=0 tx=0).stop 2024-04-25T13:43:01.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e015df90 msgr2=0x7f00e00b1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e015df90 0x7f00e00b1670 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f00d000e7f0 tx=0x7f00d000ecb0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 shutdown_connections 2024-04-25T13:43:01.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f00c406f400 0x7f00c4071910 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00e015e3a0 0x7f00e00b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f00e015df90 0x7f00e00b1670 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 --2- 172.21.15.80:0/1186994673 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f00cc006f60 0x7f00e014f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 >> 172.21.15.80:0/1186994673 conn(0x7f00e006d080 msgr2=0x7f00e00b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 shutdown_connections 2024-04-25T13:43:01.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.285+0000 7f00e7ecf640 1 -- 172.21.15.80:0/1186994673 wait complete. 2024-04-25T13:43:01.309 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:43:01.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa47321f640 1 Processor -- start 2024-04-25T13:43:01.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa47321f640 1 -- start start 2024-04-25T13:43:01.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa47321f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa46c1614b0 0x7fa46c06dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa47321f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa46c0cfb60 0x7fa46c0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa47321f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa46c0d04b0 con 0x7fa46c163eb0 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa47321f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa46c0b2100 con 0x7fa46c1614b0 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa47321f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa46c0b23e0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa470fbb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa46c1614b0 0x7fa46c06dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa470fbb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa46c1614b0 0x7fa46c06dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50164/0 (socket says 172.21.15.80:50164) 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa470fbb640 1 -- 172.21.15.80:0/1538002770 learned_addr learned my addr 172.21.15.80:0/1538002770 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa46bfff640 1 --2- 172.21.15.80:0/1538002770 >> v2:172.21.15.87:3300/0 conn(0x7fa46c0cfb60 0x7fa46c0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa470fbb640 1 -- 172.21.15.80:0/1538002770 >> v2:172.21.15.87:3300/0 conn(0x7fa46c0cfb60 msgr2=0x7fa46c0cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa470fbb640 1 --2- 172.21.15.80:0/1538002770 >> v2:172.21.15.87:3300/0 conn(0x7fa46c0cfb60 0x7fa46c0cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa470fbb640 1 -- 172.21.15.80:0/1538002770 >> v1:172.21.15.87:6789/0 conn(0x7fa46c163eb0 legacy=0x7fa46c1642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.457+0000 7fa470fbb640 1 -- 172.21.15.80:0/1538002770 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c1609d0 con 0x7fa46c1614b0 2024-04-25T13:43:01.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa470fbb640 1 --2- 172.21.15.80:0/1538002770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c1614b0 0x7fa46c06dae0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0d0720 tx=0x7fa46401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f5293a02f6efea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa46b7fe640 1 -- 172.21.15.80:0/1538002770 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4640b0070 con 0x7fa46c1614b0 2024-04-25T13:43:01.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa46b7fe640 1 -- 172.21.15.80:0/1538002770 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4640ab600 con 0x7fa46c1614b0 2024-04-25T13:43:01.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa46b7fe640 1 -- 172.21.15.80:0/1538002770 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4640ab9d0 con 0x7fa46c1614b0 2024-04-25T13:43:01.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa47321f640 1 -- 172.21.15.80:0/1538002770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c1614b0 msgr2=0x7fa46c06dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa47321f640 1 --2- 172.21.15.80:0/1538002770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c1614b0 0x7fa46c06dae0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fa46c0d0720 tx=0x7fa46401c300 comp rx=0 tx=0).stop 2024-04-25T13:43:01.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa47321f640 1 -- 172.21.15.80:0/1538002770 shutdown_connections 2024-04-25T13:43:01.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa47321f640 1 --2- 172.21.15.80:0/1538002770 >> v2:172.21.15.87:3300/0 conn(0x7fa46c0cfb60 0x7fa46c0cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa47321f640 1 --2- 172.21.15.80:0/1538002770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c1614b0 0x7fa46c06dae0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa47321f640 1 -- 172.21.15.80:0/1538002770 >> 172.21.15.80:0/1538002770 conn(0x7fa46c0d33a0 msgr2=0x7fa46c0cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:01.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa47321f640 1 -- 172.21.15.80:0/1538002770 shutdown_connections 2024-04-25T13:43:01.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.461+0000 7fa47321f640 1 -- 172.21.15.80:0/1538002770 wait complete. 2024-04-25T13:43:01.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 Processor -- start 2024-04-25T13:43:01.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 -- start start 2024-04-25T13:43:01.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c0cfb60 0x7fa46c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa46c1614b0 0x7fa46c153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa46c163eb0 0x7fa46c162ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c06d3b0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c06d230 con 0x7fa46c1614b0 2024-04-25T13:43:01.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa46c16d810 con 0x7fa46c163eb0 2024-04-25T13:43:01.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c0cfb60 0x7fa46c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c0cfb60 0x7fa46c06f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50176/0 (socket says 172.21.15.80:50176) 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 -- 172.21.15.80:0/14281544 learned_addr learned my addr 172.21.15.80:0/14281544 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa46bfff640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa46c1614b0 0x7fa46c153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa4717bc640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa46c163eb0 0x7fa46c162ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 -- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa46c163eb0 msgr2=0x7fa46c162ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa46c163eb0 0x7fa46c162ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 -- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa46c1614b0 msgr2=0x7fa46c153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa46c1614b0 0x7fa46c153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 -- 172.21.15.80:0/14281544 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c1489d0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa4717bc640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa46c163eb0 0x7fa46c162ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa46bfff640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa46c1614b0 0x7fa46c153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa470fbb640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c0cfb60 0x7fa46c06f1b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa4640ae040 tx=0x7fa46401ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa4697fa640 1 -- 172.21.15.80:0/14281544 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4640b0040 con 0x7fa46c0cfb60 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa4697fa640 1 -- 172.21.15.80:0/14281544 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4640bc070 con 0x7fa46c0cfb60 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa4697fa640 1 -- 172.21.15.80:0/14281544 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4640b6880 con 0x7fa46c0cfb60 2024-04-25T13:43:01.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46c148cb0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.465+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa46c149270 con 0x7fa46c0cfb60 2024-04-25T13:43:01.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.469+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa46c06a3a0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.469+0000 7fa4697fa640 1 -- 172.21.15.80:0/14281544 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa4640ab480 con 0x7fa46c0cfb60 2024-04-25T13:43:01.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.469+0000 7fa4697fa640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45806f400 0x7fa458071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.469+0000 7fa4697fa640 1 -- 172.21.15.80:0/14281544 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 5709+0+0 (secure 0 0 0) 0x7fa4641295e0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.469+0000 7fa46bfff640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45806f400 0x7fa458071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.469+0000 7fa46bfff640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45806f400 0x7fa458071910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa46c152b40 tx=0x7fa460008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.473+0000 7fa4697fa640 1 -- 172.21.15.80:0/14281544 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa46c0d04b0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.657+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa46c1606a0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.663 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:43:01.661+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:43:01.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.661+0000 7fa4697fa640 1 -- 172.21.15.80:0/14281544 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7fa4640f65f0 con 0x7fa46c0cfb60 2024-04-25T13:43:01.666 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:43:01.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.665+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45806f400 msgr2=0x7fa458071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.665+0000 7fa47321f640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45806f400 0x7fa458071910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa46c152b40 tx=0x7fa460008040 comp rx=0 tx=0).stop 2024-04-25T13:43:01.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.665+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c0cfb60 msgr2=0x7fa46c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.665+0000 7fa47321f640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c0cfb60 0x7fa46c06f1b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa4640ae040 tx=0x7fa46401ab40 comp rx=0 tx=0).stop 2024-04-25T13:43:01.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.669+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 shutdown_connections 2024-04-25T13:43:01.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.669+0000 7fa47321f640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa45806f400 0x7fa458071910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.669+0000 7fa47321f640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa46c163eb0 0x7fa46c162ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.669+0000 7fa47321f640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa46c1614b0 0x7fa46c153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.669+0000 7fa47321f640 1 --2- 172.21.15.80:0/14281544 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa46c0cfb60 0x7fa46c06f1b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.669+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 >> 172.21.15.80:0/14281544 conn(0x7fa46c0d33a0 msgr2=0x7fa46c0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:01.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.669+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 shutdown_connections 2024-04-25T13:43:01.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.669+0000 7fa47321f640 1 -- 172.21.15.80:0/14281544 wait complete. 2024-04-25T13:43:01.691 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:43:01.757 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:43:01.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.821+0000 7fb762ee0640 1 Processor -- start 2024-04-25T13:43:01.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.821+0000 7fb762ee0640 1 -- start start 2024-04-25T13:43:01.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.821+0000 7fb762ee0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb75c1574c0 0x7fb75c073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.821+0000 7fb762ee0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb75c06fbf0 con 0x7fb75c161170 2024-04-25T13:43:01.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.821+0000 7fb762ee0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb75c0b0a20 con 0x7fb75c071e60 2024-04-25T13:43:01.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.821+0000 7fb762ee0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb75c0b1350 con 0x7fb75c1574c0 2024-04-25T13:43:01.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb753fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb75c161170 0x7fb75c0b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42432/0 (socket says 172.21.15.80:42432) 2024-04-25T13:43:01.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb753fff640 1 -- 172.21.15.80:0/2637955400 learned_addr learned my addr 172.21.15.80:0/2637955400 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:01.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb76147d640 1 --2- 172.21.15.80:0/2637955400 >> v2:172.21.15.87:3300/0 conn(0x7fb75c1574c0 0x7fb75c073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 673932862 0 0) 0x7fb75c06fbf0 con 0x7fb75c161170 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb744003530 con 0x7fb75c161170 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2955929525 0 0) 0x7fb744003530 con 0x7fb75c161170 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 >> v2:172.21.15.87:3300/0 conn(0x7fb75c1574c0 msgr2=0x7fb75c073c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 --2- 172.21.15.80:0/2637955400 >> v2:172.21.15.87:3300/0 conn(0x7fb75c1574c0 0x7fb75c073c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 >> v1:172.21.15.87:6789/0 conn(0x7fb75c071e60 legacy=0x7fb75c160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c0b0d00 con 0x7fb75c161170 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb74c001680 con 0x7fb75c161170 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 >> v1:172.21.15.80:6789/0 conn(0x7fb75c161170 legacy=0x7fb75c0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7440039c0 0x7fb744003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb744004810 0x7fb744004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb744005280 0x7fb74400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c0b1350 con 0x7fb7440039c0 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb74400c1a0 con 0x7fb744005280 2024-04-25T13:43:01.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb74400c3a0 con 0x7fb744004810 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb760c7c640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7440039c0 0x7fb744003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb753fff640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb744004810 0x7fb744004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb76147d640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb744005280 0x7fb74400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb760c7c640 1 -- 172.21.15.80:0/2637955400 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb744004810 msgr2=0x7fb744004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb760c7c640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb744004810 0x7fb744004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb760c7c640 1 -- 172.21.15.80:0/2637955400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb744005280 msgr2=0x7fb74400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb760c7c640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb744005280 0x7fb74400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb760c7c640 1 -- 172.21.15.80:0/2637955400 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb744003530 con 0x7fb7440039c0 2024-04-25T13:43:01.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb76147d640 1 -- 172.21.15.80:0/2637955400 reap_dead start 2024-04-25T13:43:01.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb760c7c640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7440039c0 0x7fb744003e50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb758001780 tx=0x7fb75801c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5432613f9e54f7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7580013a0 con 0x7fb7440039c0 2024-04-25T13:43:01.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7580052f0 con 0x7fb7440039c0 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.825+0000 7fb7537fe640 1 -- 172.21.15.80:0/2637955400 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb758003840 con 0x7fb7440039c0 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- 172.21.15.80:0/2637955400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7440039c0 msgr2=0x7fb744003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7440039c0 0x7fb744003e50 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fb758001780 tx=0x7fb75801c400 comp rx=0 tx=0).stop 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- 172.21.15.80:0/2637955400 shutdown_connections 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 --2- 172.21.15.80:0/2637955400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7440039c0 0x7fb744003e50 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- 172.21.15.80:0/2637955400 >> 172.21.15.80:0/2637955400 conn(0x7fb75c0b4160 msgr2=0x7fb75c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- 172.21.15.80:0/2637955400 shutdown_connections 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- 172.21.15.80:0/2637955400 wait complete. 2024-04-25T13:43:01.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 Processor -- start 2024-04-25T13:43:01.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- start start 2024-04-25T13:43:01.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7440039c0 0x7fb75c151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb75c1532e0 0x7fb75c0b2980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb75c1406a0 0x7fb75c0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c0b4ba0 con 0x7fb75c1406a0 2024-04-25T13:43:01.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb75c074050 con 0x7fb75c1532e0 2024-04-25T13:43:01.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb762ee0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb75c0b4d20 con 0x7fb7440039c0 2024-04-25T13:43:01.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb75c1406a0 0x7fb75c0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb75c1406a0 0x7fb75c0b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50198/0 (socket says 172.21.15.80:50198) 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 -- 172.21.15.80:0/3000194716 learned_addr learned my addr 172.21.15.80:0/3000194716 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb760c7c640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7440039c0 0x7fb75c151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb753fff640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb75c1532e0 0x7fb75c0b2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 -- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7440039c0 msgr2=0x7fb75c151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7440039c0 0x7fb75c151780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 -- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb75c1532e0 msgr2=0x7fb75c0b2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb75c1532e0 0x7fb75c0b2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 -- 172.21.15.80:0/3000194716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb75c06f3b0 con 0x7fb75c1406a0 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb753fff640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb75c1532e0 0x7fb75c0b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.829+0000 7fb76147d640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb75c1406a0 0x7fb75c0b2ec0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb75c070120 tx=0x7fb754011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb7517fa640 1 -- 172.21.15.80:0/3000194716 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754011d40 con 0x7fb75c1406a0 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb7517fa640 1 -- 172.21.15.80:0/3000194716 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb754017470 con 0x7fb75c1406a0 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb75c153a50 con 0x7fb75c1406a0 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb7517fa640 1 -- 172.21.15.80:0/3000194716 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb754017860 con 0x7fb75c1406a0 2024-04-25T13:43:01.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb75c154010 con 0x7fb75c1406a0 2024-04-25T13:43:01.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb75c06a3a0 con 0x7fb75c1406a0 2024-04-25T13:43:01.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb7517fa640 1 -- 172.21.15.80:0/3000194716 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb75401e020 con 0x7fb75c1406a0 2024-04-25T13:43:01.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb7517fa640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb74806f400 0x7fb748071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:01.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb7517fa640 1 -- 172.21.15.80:0/3000194716 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5795+0+0 (secure 0 0 0) 0x7fb7540914a0 con 0x7fb75c1406a0 2024-04-25T13:43:01.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb760c7c640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb74806f400 0x7fb748071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:01.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.833+0000 7fb760c7c640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb74806f400 0x7fb748071910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb758001750 tx=0x7fb7580011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:01.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:01.837+0000 7fb7517fa640 1 -- 172.21.15.80:0/3000194716 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb754025020 con 0x7fb75c1406a0 2024-04-25T13:43:02.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.017+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb75c0cdc80 con 0x7fb75c1406a0 2024-04-25T13:43:02.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.665+0000 7fb7517fa640 1 -- 172.21.15.80:0/3000194716 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7fb75405e450 con 0x7fb75c1406a0 2024-04-25T13:43:02.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb74806f400 msgr2=0x7fb748071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:02.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb74806f400 0x7fb748071910 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb758001750 tx=0x7fb7580011a0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb75c1406a0 msgr2=0x7fb75c0b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:02.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb75c1406a0 0x7fb75c0b2ec0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fb75c070120 tx=0x7fb754011520 comp rx=0 tx=0).stop 2024-04-25T13:43:02.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 shutdown_connections 2024-04-25T13:43:02.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb74806f400 0x7fb748071910 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb75c1406a0 0x7fb75c0b2ec0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb75c1532e0 0x7fb75c0b2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 --2- 172.21.15.80:0/3000194716 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7440039c0 0x7fb75c151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 >> 172.21.15.80:0/3000194716 conn(0x7fb75c0b4160 msgr2=0x7fb75c0b50b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:02.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 shutdown_connections 2024-04-25T13:43:02.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.669+0000 7fb762ee0640 1 -- 172.21.15.80:0/3000194716 wait complete. 2024-04-25T13:43:02.697 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:43:02.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb906a6640 1 Processor -- start 2024-04-25T13:43:02.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb906a6640 1 -- start start 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb906a6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffb880d0f50 0x7ffb88161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb906a6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb880b0e10 con 0x7ffb880b48c0 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb906a6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb880b10f0 con 0x7ffb8815d520 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb906a6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ffb880b0bc0 con 0x7ffb880d0f50 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8e442640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ffb880b48c0 0x7ffb880d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42438/0 (socket says 172.21.15.80:42438) 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8e442640 1 -- 172.21.15.80:0/456807911 learned_addr learned my addr 172.21.15.80:0/456807911 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8dc41640 1 --2- 172.21.15.80:0/456807911 >> v2:172.21.15.87:3300/0 conn(0x7ffb880d0f50 0x7ffb88161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1639058613 0 0) 0x7ffb880b0e10 con 0x7ffb880b48c0 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffb74003530 con 0x7ffb880b48c0 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 601555070 0 0) 0x7ffb74003530 con 0x7ffb880b48c0 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 >> v2:172.21.15.87:3300/0 conn(0x7ffb880d0f50 msgr2=0x7ffb88161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:02.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 --2- 172.21.15.80:0/456807911 >> v2:172.21.15.87:3300/0 conn(0x7ffb880d0f50 0x7ffb88161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 >> v1:172.21.15.87:6789/0 conn(0x7ffb8815d520 legacy=0x7ffb8815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8806f710 con 0x7ffb880b48c0 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ffb80001a00 con 0x7ffb880b48c0 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 >> v1:172.21.15.80:6789/0 conn(0x7ffb880b48c0 legacy=0x7ffb880d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb740039c0 0x7ffb74003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb74004810 0x7ffb74004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb74005260 0x7ffb7400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ffb880b0bc0 con 0x7ffb740039c0 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffb7400c1f0 con 0x7ffb74004810 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ffb7400c3f0 con 0x7ffb74005260 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.845+0000 7ffb8e442640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb740039c0 0x7ffb74003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8e442640 1 -- 172.21.15.80:0/456807911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb74005260 msgr2=0x7ffb7400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8e442640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb74005260 0x7ffb7400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8e442640 1 -- 172.21.15.80:0/456807911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb74004810 msgr2=0x7ffb74004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:02.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8e442640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb74004810 0x7ffb74004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8e442640 1 -- 172.21.15.80:0/456807911 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb74003530 con 0x7ffb740039c0 2024-04-25T13:43:02.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8ec43640 1 -- 172.21.15.80:0/456807911 reap_dead start 2024-04-25T13:43:02.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8e442640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb740039c0 0x7ffb74003e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000a810 tx=0x7ffb8001dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b54996cb114ba2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:02.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8000a460 con 0x7ffb740039c0 2024-04-25T13:43:02.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb800072f0 con 0x7ffb740039c0 2024-04-25T13:43:02.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb8d440640 1 -- 172.21.15.80:0/456807911 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb80007720 con 0x7ffb740039c0 2024-04-25T13:43:02.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb906a6640 1 -- 172.21.15.80:0/456807911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb740039c0 msgr2=0x7ffb74003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:02.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb906a6640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb740039c0 0x7ffb74003e30 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000a810 tx=0x7ffb8001dda0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb906a6640 1 -- 172.21.15.80:0/456807911 shutdown_connections 2024-04-25T13:43:02.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb906a6640 1 --2- 172.21.15.80:0/456807911 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb740039c0 0x7ffb74003e30 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb906a6640 1 -- 172.21.15.80:0/456807911 >> 172.21.15.80:0/456807911 conn(0x7ffb880b35c0 msgr2=0x7ffb880b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:02.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb906a6640 1 -- 172.21.15.80:0/456807911 shutdown_connections 2024-04-25T13:43:02.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.849+0000 7ffb906a6640 1 -- 172.21.15.80:0/456807911 wait complete. 2024-04-25T13:43:02.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 Processor -- start 2024-04-25T13:43:02.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 -- start start 2024-04-25T13:43:02.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb740039c0 0x7ffb881fa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:02.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb88156e50 0x7ffb88157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:02.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb88159450 0x7ffb88159860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:02.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8815dab0 con 0x7ffb88156e50 2024-04-25T13:43:02.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8815d930 con 0x7ffb740039c0 2024-04-25T13:43:02.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ffb8815dc30 con 0x7ffb88159450 2024-04-25T13:43:02.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb88156e50 0x7ffb88157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb88156e50 0x7ffb88157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50230/0 (socket says 172.21.15.80:50230) 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 -- 172.21.15.80:0/2819115065 learned_addr learned my addr 172.21.15.80:0/2819115065 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8ec43640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb88159450 0x7ffb88159860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8e442640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb740039c0 0x7ffb881fa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 -- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb88159450 msgr2=0x7ffb88159860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb88159450 0x7ffb88159860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 -- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb740039c0 msgr2=0x7ffb881fa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb740039c0 0x7ffb881fa620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 -- 172.21.15.80:0/2819115065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb88159da0 con 0x7ffb88156e50 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8ec43640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb88159450 0x7ffb88159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb8dc41640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb88156e50 0x7ffb88157260 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e770 tx=0x7ffb7800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb7effd640 1 -- 172.21.15.80:0/2819115065 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb78017070 con 0x7ffb88156e50 2024-04-25T13:43:02.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb7effd640 1 -- 172.21.15.80:0/2819115065 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb7800ee80 con 0x7ffb88156e50 2024-04-25T13:43:02.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb88157c10 con 0x7ffb88156e50 2024-04-25T13:43:02.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb7effd640 1 -- 172.21.15.80:0/2819115065 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7800ca80 con 0x7ffb88156e50 2024-04-25T13:43:02.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.853+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb881581d0 con 0x7ffb88156e50 2024-04-25T13:43:02.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.857+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb88157980 con 0x7ffb88156e50 2024-04-25T13:43:02.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.857+0000 7ffb7effd640 1 -- 172.21.15.80:0/2819115065 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffb7801a070 con 0x7ffb88156e50 2024-04-25T13:43:02.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.857+0000 7ffb7effd640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb6c06f400 0x7ffb6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:02.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.857+0000 7ffb7effd640 1 -- 172.21.15.80:0/2819115065 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5795+0+0 (secure 0 0 0) 0x7ffb7808eff0 con 0x7ffb88156e50 2024-04-25T13:43:02.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.857+0000 7ffb8e442640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb6c06f400 0x7ffb6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:02.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.857+0000 7ffb8e442640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb6c06f400 0x7ffb6c071910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffb800aa000 tx=0x7ffb8806f8e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:02.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:02.861+0000 7ffb7effd640 1 -- 172.21.15.80:0/2819115065 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb88159da0 con 0x7ffb88156e50 2024-04-25T13:43:03.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.009+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ffb8806a440 con 0x7ffb88156e50 2024-04-25T13:43:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.677+0000 7ffb7effd640 1 -- 172.21.15.80:0/2819115065 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v54) ==== 181+0+0 (secure 0 0 0) 0x7ffb7805bf10 con 0x7ffb88156e50 2024-04-25T13:43:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:43:03.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb6c06f400 msgr2=0x7ffb6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:03.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb6c06f400 0x7ffb6c071910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ffb800aa000 tx=0x7ffb8806f8e0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb88156e50 msgr2=0x7ffb88157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:03.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb88156e50 0x7ffb88157260 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffb7800e770 tx=0x7ffb7800ec30 comp rx=0 tx=0).stop 2024-04-25T13:43:03.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 shutdown_connections 2024-04-25T13:43:03.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb6c06f400 0x7ffb6c071910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb88159450 0x7ffb88159860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb88156e50 0x7ffb88157260 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 --2- 172.21.15.80:0/2819115065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb740039c0 0x7ffb881fa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 >> 172.21.15.80:0/2819115065 conn(0x7ffb880b35c0 msgr2=0x7ffb880cebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 shutdown_connections 2024-04-25T13:43:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.681+0000 7ffb906a6640 1 -- 172.21.15.80:0/2819115065 wait complete. 2024-04-25T13:43:03.713 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:43:03.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa03586a640 1 Processor -- start 2024-04-25T13:43:03.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa03586a640 1 -- start start 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa03586a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa0300d0f50 0x7fa030161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa03586a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa03015d520 0x7fa03015d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa03586a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0300b0e10 con 0x7fa0300b48c0 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa03586a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa0300b0bc0 con 0x7fa0300d0f50 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa03586a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa0300b10f0 con 0x7fa03015d520 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa027fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa0300d0f50 0x7fa030161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa027fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa0300d0f50 0x7fa030161600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50240/0 (socket says 172.21.15.80:50240) 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa02effd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa0300b48c0 0x7fa0300d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42446/0 (socket says 172.21.15.80:42446) 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.861+0000 7fa027fff640 1 -- 172.21.15.80:0/2676991866 learned_addr learned my addr 172.21.15.80:0/2676991866 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa02f7fe640 1 --2- 172.21.15.80:0/2676991866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa03015d520 0x7fa03015d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa027fff640 1 -- 172.21.15.80:0/2676991866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa03015d520 msgr2=0x7fa03015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:03.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa027fff640 1 --2- 172.21.15.80:0/2676991866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa03015d520 0x7fa03015d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa027fff640 1 -- 172.21.15.80:0/2676991866 >> v1:172.21.15.80:6789/0 conn(0x7fa0300b48c0 legacy=0x7fa0300d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:03.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa027fff640 1 -- 172.21.15.80:0/2676991866 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa03006f710 con 0x7fa0300d0f50 2024-04-25T13:43:03.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa02e7fc640 1 -- 172.21.15.80:0/2676991866 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 17789787 0 0) 0x7fa0300b0e10 con 0x7fa0300b48c0 2024-04-25T13:43:03.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa02f7fe640 1 --2- 172.21.15.80:0/2676991866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa03015d520 0x7fa03015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:03.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa027fff640 1 --2- 172.21.15.80:0/2676991866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300d0f50 0x7fa030161600 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa018009870 tx=0x7fa01801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9874a911c5dc1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:03.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa02e7fc640 1 -- 172.21.15.80:0/2676991866 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180b0070 con 0x7fa0300d0f50 2024-04-25T13:43:03.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa02e7fc640 1 -- 172.21.15.80:0/2676991866 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0180ab600 con 0x7fa0300d0f50 2024-04-25T13:43:03.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa02e7fc640 1 -- 172.21.15.80:0/2676991866 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0180ab9d0 con 0x7fa0300d0f50 2024-04-25T13:43:03.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa03586a640 1 -- 172.21.15.80:0/2676991866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300d0f50 msgr2=0x7fa030161600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:03.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa03586a640 1 --2- 172.21.15.80:0/2676991866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300d0f50 0x7fa030161600 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fa018009870 tx=0x7fa01801c490 comp rx=0 tx=0).stop 2024-04-25T13:43:03.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa03586a640 1 -- 172.21.15.80:0/2676991866 shutdown_connections 2024-04-25T13:43:03.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa03586a640 1 --2- 172.21.15.80:0/2676991866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa03015d520 0x7fa03015d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa03586a640 1 --2- 172.21.15.80:0/2676991866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300d0f50 0x7fa030161600 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa03586a640 1 -- 172.21.15.80:0/2676991866 >> 172.21.15.80:0/2676991866 conn(0x7fa0300b35c0 msgr2=0x7fa0300b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:03.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa03586a640 1 -- 172.21.15.80:0/2676991866 shutdown_connections 2024-04-25T13:43:03.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.865+0000 7fa03586a640 1 -- 172.21.15.80:0/2676991866 wait complete. 2024-04-25T13:43:03.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 Processor -- start 2024-04-25T13:43:03.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 -- start start 2024-04-25T13:43:03.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0300b48c0 0x7fa0301fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:03.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa03015d520 0x7fa0300afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300affc0 0x7fa03015c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa03016a080 con 0x7fa0300affc0 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa030169f00 con 0x7fa03015d520 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa03016a200 con 0x7fa0300b48c0 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300affc0 0x7fa03015c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300affc0 0x7fa03015c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50256/0 (socket says 172.21.15.80:50256) 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 -- 172.21.15.80:0/189694506 learned_addr learned my addr 172.21.15.80:0/189694506 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02effd640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0300b48c0 0x7fa0301fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa027fff640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa03015d520 0x7fa0300afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 -- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0300b48c0 msgr2=0x7fa0301fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0300b48c0 0x7fa0301fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 -- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa03015d520 msgr2=0x7fa0300afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:03.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa03015d520 0x7fa0300afa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 -- 172.21.15.80:0/189694506 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa030158590 con 0x7fa0300affc0 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa027fff640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa03015d520 0x7fa0300afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02f7fe640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300affc0 0x7fa03015c4a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa02800cd50 tx=0x7fa02800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02cff9640 1 -- 172.21.15.80:0/189694506 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02800f680 con 0x7fa0300affc0 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02cff9640 1 -- 172.21.15.80:0/189694506 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa02800f840 con 0x7fa0300affc0 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa030158870 con 0x7fa0300affc0 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa02cff9640 1 -- 172.21.15.80:0/189694506 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02800fc30 con 0x7fa0300affc0 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.869+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa03015c1e0 con 0x7fa0300affc0 2024-04-25T13:43:03.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.873+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa03006a440 con 0x7fa0300affc0 2024-04-25T13:43:03.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.873+0000 7fa02cff9640 1 -- 172.21.15.80:0/189694506 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa02800e3e0 con 0x7fa0300affc0 2024-04-25T13:43:03.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.873+0000 7fa02cff9640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01006f400 0x7fa010071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:03.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.873+0000 7fa02cff9640 1 -- 172.21.15.80:0/189694506 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5357+0+0 (secure 0 0 0) 0x7fa028090040 con 0x7fa0300affc0 2024-04-25T13:43:03.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.873+0000 7fa02effd640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01006f400 0x7fa010071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:03.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.873+0000 7fa02effd640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01006f400 0x7fa010071910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa020000f70 tx=0x7fa020007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:03.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:03.877+0000 7fa02cff9640 1 -- 172.21.15.80:0/189694506 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa028014030 con 0x7fa0300affc0 2024-04-25T13:43:04.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.025+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa030158320 con 0x7fa0300affc0 2024-04-25T13:43:04.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.685+0000 7fa02cff9640 1 -- 172.21.15.80:0/189694506 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v55) ==== 169+0+0 (secure 0 0 0) 0x7fa02805cf70 con 0x7fa0300affc0 2024-04-25T13:43:04.688 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:43:04.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.689+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01006f400 msgr2=0x7fa010071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:04.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.689+0000 7fa03586a640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01006f400 0x7fa010071910 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa020000f70 tx=0x7fa020007c10 comp rx=0 tx=0).stop 2024-04-25T13:43:04.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.689+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300affc0 msgr2=0x7fa03015c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:04.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.689+0000 7fa03586a640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300affc0 0x7fa03015c4a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fa02800cd50 tx=0x7fa02800a650 comp rx=0 tx=0).stop 2024-04-25T13:43:04.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.697+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 shutdown_connections 2024-04-25T13:43:04.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.697+0000 7fa03586a640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa01006f400 0x7fa010071910 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.697+0000 7fa03586a640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa0300affc0 0x7fa03015c4a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.697+0000 7fa03586a640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa03015d520 0x7fa0300afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.697+0000 7fa03586a640 1 --2- 172.21.15.80:0/189694506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa0300b48c0 0x7fa0301fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.697+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 >> 172.21.15.80:0/189694506 conn(0x7fa0300b35c0 msgr2=0x7fa03006ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:04.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.697+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 shutdown_connections 2024-04-25T13:43:04.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.697+0000 7fa03586a640 1 -- 172.21.15.80:0/189694506 wait complete. 2024-04-25T13:43:04.728 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:43:04.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.881+0000 7f1337532640 1 Processor -- start 2024-04-25T13:43:04.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.881+0000 7f1337532640 1 -- start start 2024-04-25T13:43:04.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1337532640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 0x7f1330162750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1337532640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1330162c90 0x7f13301630a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1337532640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13300b4cc0 con 0x7f13300cf940 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1337532640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f13301635e0 con 0x7f1330162c90 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1337532640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13300b4fe0 con 0x7f13300b7010 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f13352ce640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f13300cf940 0x7f13300b6900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42462/0 (socket says 172.21.15.80:42462) 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f13352ce640 1 -- 172.21.15.80:0/596980183 learned_addr learned my addr 172.21.15.80:0/596980183 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1335acf640 1 --2- 172.21.15.80:0/596980183 >> v2:172.21.15.87:3300/0 conn(0x7f1330162c90 0x7f13301630a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1334acd640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 0x7f1330162750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1243187796 0 0) 0x7f13300b4cc0 con 0x7f13300cf940 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f131c003530 con 0x7f13300cf940 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 582754524 0 0) 0x7f131c003530 con 0x7f13300cf940 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 msgr2=0x7f1330162750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 0x7f1330162750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 >> v2:172.21.15.87:3300/0 conn(0x7f1330162c90 msgr2=0x7f13301630a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 --2- 172.21.15.80:0/596980183 >> v2:172.21.15.87:3300/0 conn(0x7f1330162c90 0x7f13301630a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13300cfd50 con 0x7f13300cf940 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f132c001680 con 0x7f13300cf940 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 >> v1:172.21.15.80:6789/0 conn(0x7f13300cf940 legacy=0x7f13300b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 0x7f131c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f131c003f60 0x7f131c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c004d50 0x7f131c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13301635e0 con 0x7f131c004d50 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13300b4fe0 con 0x7f131c003f60 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f131c00bb00 con 0x7f13300b7010 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1335acf640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c004d50 0x7f131c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f13352ce640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 0x7f131c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1334acd640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f131c003f60 0x7f131c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1335acf640 1 -- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 msgr2=0x7f131c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:04.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1335acf640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 0x7f131c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1335acf640 1 -- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f131c003f60 msgr2=0x7f131c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:04.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1335acf640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f131c003f60 0x7f131c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f1335acf640 1 -- 172.21.15.80:0/596980183 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f131c003530 con 0x7f131c004d50 2024-04-25T13:43:04.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.885+0000 7f13352ce640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13300b7010 0x7f131c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1335acf640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c004d50 0x7f131c005160 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1328009f90 tx=0x7f132801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7edebf88a8b053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280b2070 con 0x7f131c004d50 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13280a5c90 con 0x7f131c004d50 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1327fff640 1 -- 172.21.15.80:0/596980183 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1328009c10 con 0x7f131c004d50 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1337532640 1 -- 172.21.15.80:0/596980183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c004d50 msgr2=0x7f131c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1337532640 1 --2- 172.21.15.80:0/596980183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f131c004d50 0x7f131c005160 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f1328009f90 tx=0x7f132801cc80 comp rx=0 tx=0).stop 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1335acf640 1 -- 172.21.15.80:0/596980183 reap_dead start 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1337532640 1 -- 172.21.15.80:0/596980183 shutdown_connections 2024-04-25T13:43:04.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1337532640 1 -- 172.21.15.80:0/596980183 >> 172.21.15.80:0/596980183 conn(0x7f13300d3ab0 msgr2=0x7f13300d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:04.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1337532640 1 -- 172.21.15.80:0/596980183 shutdown_connections 2024-04-25T13:43:04.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1337532640 1 -- 172.21.15.80:0/596980183 wait complete. 2024-04-25T13:43:04.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1337532640 1 Processor -- start 2024-04-25T13:43:04.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.889+0000 7f1337532640 1 -- start start 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f131c004d50 0x7f13301f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300b2510 0x7f133015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300b2920 0x7f133015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13300cf1d0 con 0x7f13300b2920 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13300d45d0 con 0x7f13300b2510 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13300d4820 con 0x7f131c004d50 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300b2920 0x7f133015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300b2920 0x7f133015b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50278/0 (socket says 172.21.15.80:50278) 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 -- 172.21.15.80:0/4162390400 learned_addr learned my addr 172.21.15.80:0/4162390400 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1334acd640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300b2510 0x7f133015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f13352ce640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f131c004d50 0x7f13301f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 -- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f131c004d50 msgr2=0x7f13301f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:04.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f131c004d50 0x7f13301f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 -- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300b2510 msgr2=0x7f133015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300b2510 0x7f133015b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 -- 172.21.15.80:0/4162390400 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1330159750 con 0x7f13300b2920 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1335acf640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300b2920 0x7f133015b5b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f132801c6f0 tx=0x7f1328009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1325ffb640 1 -- 172.21.15.80:0/4162390400 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280bd070 con 0x7f13300b2920 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1325ffb640 1 -- 172.21.15.80:0/4162390400 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13280a5a90 con 0x7f13300b2920 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13300b3080 con 0x7f13300b2920 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1325ffb640 1 -- 172.21.15.80:0/4162390400 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13280b2040 con 0x7f13300b2920 2024-04-25T13:43:04.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133015d4f0 con 0x7f13300b2920 2024-04-25T13:43:04.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f133015d2e0 con 0x7f13300b2920 2024-04-25T13:43:04.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.893+0000 7f1325ffb640 1 -- 172.21.15.80:0/4162390400 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13280c0070 con 0x7f13300b2920 2024-04-25T13:43:04.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.897+0000 7f1325ffb640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:04.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.897+0000 7f1325ffb640 1 -- 172.21.15.80:0/4162390400 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f1328128e40 con 0x7f13300b2920 2024-04-25T13:43:04.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.897+0000 7f13352ce640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:04.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.897+0000 7f13352ce640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f132c000c10 tx=0x7f132c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:04.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:04.901+0000 7f1325ffb640 1 -- 172.21.15.80:0/4162390400 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1328016990 con 0x7f13300b2920 2024-04-25T13:43:05.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.061+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f13300b2e10 con 0x7f13300b2920 2024-04-25T13:43:05.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.065+0000 7f1325ffb640 1 -- 172.21.15.80:0/4162390400 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) ==== 104+0+0 (secure 0 0 0) 0x7f13280f59e0 con 0x7f13300b2920 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 msgr2=0x7f1314071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f132c000c10 tx=0x7f132c009040 comp rx=0 tx=0).stop 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300b2920 msgr2=0x7f133015b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300b2920 0x7f133015b5b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f132801c6f0 tx=0x7f1328009750 comp rx=0 tx=0).stop 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 shutdown_connections 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f131406f400 0x7f1314071910 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13300b2920 0x7f133015b5b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13300b2510 0x7f133015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 --2- 172.21.15.80:0/4162390400 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f131c004d50 0x7f13301f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 >> 172.21.15.80:0/4162390400 conn(0x7f13300d3ab0 msgr2=0x7f13301623e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 shutdown_connections 2024-04-25T13:43:05.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.069+0000 7f1337532640 1 -- 172.21.15.80:0/4162390400 wait complete. 2024-04-25T13:43:05.093 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2024-04-25T13:43:05.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabce3ac640 1 Processor -- start 2024-04-25T13:43:05.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabce3ac640 1 -- start start 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabce3ac640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fabc806f5a0 0x7fabc8160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabce3ac640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fabc8161070 0x7fabc8161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabce3ac640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fabc806fcf0 con 0x7fabc80b4bd0 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabce3ac640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fabc81619c0 con 0x7fabc806f5a0 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabce3ac640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fabc8073f10 con 0x7fabc8161070 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabc7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fabc806f5a0 0x7fabc8160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabcc949640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fabc80b4bd0 0x7fabc80b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42466/0 (socket says 172.21.15.80:42466) 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabcc949640 1 -- 172.21.15.80:0/2204447331 learned_addr learned my addr 172.21.15.80:0/2204447331 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabc77fe640 1 --2- 172.21.15.80:0/2204447331 >> v2:172.21.15.87:3300/0 conn(0x7fabc8161070 0x7fabc8161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabc7fff640 1 -- 172.21.15.80:0/2204447331 >> v2:172.21.15.87:3300/0 conn(0x7fabc8161070 msgr2=0x7fabc8161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabc7fff640 1 --2- 172.21.15.80:0/2204447331 >> v2:172.21.15.87:3300/0 conn(0x7fabc8161070 0x7fabc8161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabc7fff640 1 -- 172.21.15.80:0/2204447331 >> v1:172.21.15.80:6789/0 conn(0x7fabc80b4bd0 legacy=0x7fabc80b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabc6ffd640 1 -- 172.21.15.80:0/2204447331 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1730118849 0 0) 0x7fabc806fcf0 con 0x7fabc80b4bd0 2024-04-25T13:43:05.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.241+0000 7fabc7fff640 1 -- 172.21.15.80:0/2204447331 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc80b0c60 con 0x7fabc806f5a0 2024-04-25T13:43:05.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabc7fff640 1 --2- 172.21.15.80:0/2204447331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8160b30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fabc806ff70 tx=0x7fabb801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0b13dc46610570 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:05.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabc6ffd640 1 -- 172.21.15.80:0/2204447331 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80b2070 con 0x7fabc806f5a0 2024-04-25T13:43:05.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabc6ffd640 1 -- 172.21.15.80:0/2204447331 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb80a5850 con 0x7fabc806f5a0 2024-04-25T13:43:05.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabc6ffd640 1 -- 172.21.15.80:0/2204447331 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80a5c20 con 0x7fabc806f5a0 2024-04-25T13:43:05.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 -- 172.21.15.80:0/2204447331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 msgr2=0x7fabc8160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 --2- 172.21.15.80:0/2204447331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8160b30 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fabc806ff70 tx=0x7fabb801c8a0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 -- 172.21.15.80:0/2204447331 shutdown_connections 2024-04-25T13:43:05.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 --2- 172.21.15.80:0/2204447331 >> v2:172.21.15.87:3300/0 conn(0x7fabc8161070 0x7fabc8161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 --2- 172.21.15.80:0/2204447331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8160b30 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 -- 172.21.15.80:0/2204447331 >> 172.21.15.80:0/2204447331 conn(0x7fabc806d080 msgr2=0x7fabc80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:05.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 -- 172.21.15.80:0/2204447331 shutdown_connections 2024-04-25T13:43:05.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 -- 172.21.15.80:0/2204447331 wait complete. 2024-04-25T13:43:05.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 Processor -- start 2024-04-25T13:43:05.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.245+0000 7fabce3ac640 1 -- start start 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabc80b4bd0 0x7fabc8140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabc8161070 0x7fabc815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fabc816a860 con 0x7fabc806f5a0 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fabc816a6e0 con 0x7fabc80b4bd0 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fabc816a9e0 con 0x7fabc8161070 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8155860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50296/0 (socket says 172.21.15.80:50296) 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 -- 172.21.15.80:0/1053507193 learned_addr learned my addr 172.21.15.80:0/1053507193 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc77fe640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabc80b4bd0 0x7fabc8140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabcc949640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabc8161070 0x7fabc815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 -- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabc8161070 msgr2=0x7fabc815a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabc8161070 0x7fabc815a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 -- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabc80b4bd0 msgr2=0x7fabc8140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabc80b4bd0 0x7fabc8140b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 -- 172.21.15.80:0/1053507193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc81560b0 con 0x7fabc806f5a0 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc77fe640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabc80b4bd0 0x7fabc8140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc7fff640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8155860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009d90 tx=0x7fabb8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc4ff9640 1 -- 172.21.15.80:0/1053507193 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80b2040 con 0x7fabc806f5a0 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc4ff9640 1 -- 172.21.15.80:0/1053507193 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fabb80bf070 con 0x7fabc806f5a0 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc8156390 con 0x7fabc806f5a0 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc4ff9640 1 -- 172.21.15.80:0/1053507193 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb80b8870 con 0x7fabc806f5a0 2024-04-25T13:43:05.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc8156950 con 0x7fabc806f5a0 2024-04-25T13:43:05.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc806a3a0 con 0x7fabc806f5a0 2024-04-25T13:43:05.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.249+0000 7fabc4ff9640 1 -- 172.21.15.80:0/1053507193 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fabb80a56d0 con 0x7fabc806f5a0 2024-04-25T13:43:05.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.253+0000 7fabc4ff9640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fabb406f400 0x7fabb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.253+0000 7fabc4ff9640 1 -- 172.21.15.80:0/1053507193 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fabb812a090 con 0x7fabc806f5a0 2024-04-25T13:43:05.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.253+0000 7fabc77fe640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fabb406f400 0x7fabb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.253+0000 7fabc77fe640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fabb406f400 0x7fabb4071910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabc8075f60 tx=0x7fabbc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:05.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.257+0000 7fabc4ff9640 1 -- 172.21.15.80:0/1053507193 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fabb8016520 con 0x7fabc806f5a0 2024-04-25T13:43:05.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.473+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) -- 0x7fabc80739c0 con 0x7fabc806f5a0 2024-04-25T13:43:05.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.685+0000 7fabc4ff9640 1 -- 172.21.15.80:0/1053507193 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v159) ==== 142+0+0 (secure 0 0 0) 0x7fabb80f7280 con 0x7fabc806f5a0 2024-04-25T13:43:05.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fabb406f400 msgr2=0x7fabb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fabb406f400 0x7fabb4071910 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fabc8075f60 tx=0x7fabbc008040 comp rx=0 tx=0).stop 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 msgr2=0x7fabc8155860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8155860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fabb8009d90 tx=0x7fabb8019640 comp rx=0 tx=0).stop 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 shutdown_connections 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fabb406f400 0x7fabb4071910 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fabc8161070 0x7fabc815a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fabc80b4bd0 0x7fabc8140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 --2- 172.21.15.80:0/1053507193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fabc806f5a0 0x7fabc8155860 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.689+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 >> 172.21.15.80:0/1053507193 conn(0x7fabc806d080 msgr2=0x7fabc80d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.693+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 shutdown_connections 2024-04-25T13:43:05.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.693+0000 7fabce3ac640 1 -- 172.21.15.80:0/1053507193 wait complete. 2024-04-25T13:43:05.719 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-04-25T13:43:05.719 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2024-04-25T13:43:05.719 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:43:05.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f639e8d4640 1 Processor -- start 2024-04-25T13:43:05.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f639e8d4640 1 -- start start 2024-04-25T13:43:05.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f639e8d4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f63980b0ed0 0x7f639815c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f639e8d4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f639815cda0 con 0x7f639806eba0 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f639e8d4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6398073710 con 0x7f63980ce730 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f639e8d4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f63980739f0 con 0x7f63980b0ed0 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f6397fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f63980b0ed0 0x7f639815c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f639ce71640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f639806eba0 0x7f63981608f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42468/0 (socket says 172.21.15.80:42468) 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f639ce71640 1 -- 172.21.15.80:0/2354202976 learned_addr learned my addr 172.21.15.80:0/2354202976 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f6397fff640 1 -- 172.21.15.80:0/2354202976 >> v1:172.21.15.87:6789/0 conn(0x7f63980ce730 legacy=0x7f639806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f6397fff640 1 -- 172.21.15.80:0/2354202976 >> v1:172.21.15.80:6789/0 conn(0x7f639806eba0 legacy=0x7f63981608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.869+0000 7f6397fff640 1 -- 172.21.15.80:0/2354202976 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6398073cd0 con 0x7f63980b0ed0 2024-04-25T13:43:05.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f6397fff640 1 --2- 172.21.15.80:0/2354202976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f639815c860 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6398071650 tx=0x7f639001fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b81d09f8148193bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:05.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f6396ffd640 1 -- 172.21.15.80:0/2354202976 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900b6070 con 0x7f63980b0ed0 2024-04-25T13:43:05.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f6396ffd640 1 -- 172.21.15.80:0/2354202976 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900a9a40 con 0x7f63980b0ed0 2024-04-25T13:43:05.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f6396ffd640 1 -- 172.21.15.80:0/2354202976 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900a9e10 con 0x7f63980b0ed0 2024-04-25T13:43:05.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f639e8d4640 1 -- 172.21.15.80:0/2354202976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 msgr2=0x7f639815c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f639e8d4640 1 --2- 172.21.15.80:0/2354202976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f639815c860 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f6398071650 tx=0x7f639001fd40 comp rx=0 tx=0).stop 2024-04-25T13:43:05.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f639e8d4640 1 -- 172.21.15.80:0/2354202976 shutdown_connections 2024-04-25T13:43:05.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f639e8d4640 1 --2- 172.21.15.80:0/2354202976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f639815c860 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f639e8d4640 1 -- 172.21.15.80:0/2354202976 >> 172.21.15.80:0/2354202976 conn(0x7f639806d080 msgr2=0x7f63980b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:05.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f639e8d4640 1 -- 172.21.15.80:0/2354202976 shutdown_connections 2024-04-25T13:43:05.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.873+0000 7f639e8d4640 1 -- 172.21.15.80:0/2354202976 wait complete. 2024-04-25T13:43:05.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 Processor -- start 2024-04-25T13:43:05.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 -- start start 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f63981f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63980ce730 0x7f6398159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f639815a300 0x7f63980af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f63980733a0 con 0x7f63980b0ed0 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f63980b12e0 con 0x7f639815a300 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63980ceb40 con 0x7f63980ce730 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f63981f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f63981f5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50306/0 (socket says 172.21.15.80:50306) 2024-04-25T13:43:05.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 -- 172.21.15.80:0/1940785815 learned_addr learned my addr 172.21.15.80:0/1940785815 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639ce71640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f639815a300 0x7f63980af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 -- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63980ce730 msgr2=0x7f6398159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f63977fe640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63980ce730 0x7f6398159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63980ce730 0x7f6398159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 -- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f639815a300 msgr2=0x7f63980af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f639815a300 0x7f63980af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 -- 172.21.15.80:0/1940785815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63980cef10 con 0x7f63980b0ed0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639ce71640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f639815a300 0x7f63980af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6397fff640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f63981f5fe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f639001c950 tx=0x7f639001cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6394ff9640 1 -- 172.21.15.80:0/1940785815 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900b6040 con 0x7f63980b0ed0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6394ff9640 1 -- 172.21.15.80:0/1940785815 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63900c0070 con 0x7f63980b0ed0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f639815ab40 con 0x7f63980b0ed0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f6394ff9640 1 -- 172.21.15.80:0/1940785815 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63900ba410 con 0x7f63980b0ed0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63980cf4b0 con 0x7f63980b0ed0 2024-04-25T13:43:05.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.877+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f639806a3a0 con 0x7f63980b0ed0 2024-04-25T13:43:05.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.881+0000 7f6394ff9640 1 -- 172.21.15.80:0/1940785815 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f63900ba5b0 con 0x7f63980b0ed0 2024-04-25T13:43:05.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.881+0000 7f6394ff9640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f638406f400 0x7f6384071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:05.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.881+0000 7f6394ff9640 1 -- 172.21.15.80:0/1940785815 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f639012cb30 con 0x7f63980b0ed0 2024-04-25T13:43:05.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.881+0000 7f63977fe640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f638406f400 0x7f6384071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.881+0000 7f63977fe640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f638406f400 0x7f6384071910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f63980b3b20 tx=0x7f638c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:05.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:05.885+0000 7f6394ff9640 1 -- 172.21.15.80:0/1940785815 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f63900198a0 con 0x7f63980b0ed0 2024-04-25T13:43:06.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.049+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f639806f880 con 0x7f63980b0ed0 2024-04-25T13:43:06.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.049+0000 7f6394ff9640 1 -- 172.21.15.80:0/1940785815 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) ==== 74+0+12948 (secure 0 0 0) 0x7f63900f9d20 con 0x7f63980b0ed0 2024-04-25T13:43:06.053 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:06.053 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":55,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:43:04.681216+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/1692562493":"2024-04-26T13:43:01.664917+0000","172.21.15.80:6834/1692562493":"2024-04-26T13:43:01.664917+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:43:06.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f638406f400 msgr2=0x7f6384071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f638406f400 0x7f6384071910 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f63980b3b20 tx=0x7f638c008040 comp rx=0 tx=0).stop 2024-04-25T13:43:06.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 msgr2=0x7f63981f5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f63981f5fe0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f639001c950 tx=0x7f639001cae0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 shutdown_connections 2024-04-25T13:43:06.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f638406f400 0x7f6384071910 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f639815a300 0x7f63980af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63980ce730 0x7f6398159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 --2- 172.21.15.80:0/1940785815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63980b0ed0 0x7f63981f5fe0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 >> 172.21.15.80:0/1940785815 conn(0x7f639806d080 msgr2=0x7f63980b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:06.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 shutdown_connections 2024-04-25T13:43:06.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.053+0000 7f639e8d4640 1 -- 172.21.15.80:0/1940785815 wait complete. 2024-04-25T13:43:06.075 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:43:06.076 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:43:06.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4fd93b640 1 Processor -- start 2024-04-25T13:43:06.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4fd93b640 1 -- start start 2024-04-25T13:43:06.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4fd93b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff4f8072700 0x7ff4f80d0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4fd93b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff4f80d1650 0x7ff4f8164630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4fd93b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4f80b6e90 con 0x7ff4f80d1240 2024-04-25T13:43:06.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4fd93b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f80cffd0 con 0x7ff4f80d1650 2024-04-25T13:43:06.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4fd93b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff4f80d0870 con 0x7ff4f8072700 2024-04-25T13:43:06.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4f77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff4f80d1650 0x7ff4f8164630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4f77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff4f80d1650 0x7ff4f8164630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50310/0 (socket says 172.21.15.80:50310) 2024-04-25T13:43:06.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4f77fe640 1 -- 172.21.15.80:0/2013688708 learned_addr learned my addr 172.21.15.80:0/2013688708 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:06.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4f6ffd640 1 --2- 172.21.15.80:0/2013688708 >> v2:172.21.15.87:3300/0 conn(0x7ff4f8072700 0x7ff4f80d0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4f77fe640 1 -- 172.21.15.80:0/2013688708 >> v2:172.21.15.87:3300/0 conn(0x7ff4f8072700 msgr2=0x7ff4f80d0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:06.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4f77fe640 1 --2- 172.21.15.80:0/2013688708 >> v2:172.21.15.87:3300/0 conn(0x7ff4f8072700 0x7ff4f80d0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4f77fe640 1 -- 172.21.15.80:0/2013688708 >> v1:172.21.15.87:6789/0 conn(0x7ff4f80d1240 legacy=0x7ff4f8163f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.225+0000 7ff4f77fe640 1 -- 172.21.15.80:0/2013688708 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f80d1a60 con 0x7ff4f80d1650 2024-04-25T13:43:06.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4f77fe640 1 --2- 172.21.15.80:0/2013688708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f80d1650 0x7ff4f8164630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80b4d80 tx=0x7ff4f001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=623fd4691026ce60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:06.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4f5ffb640 1 -- 172.21.15.80:0/2013688708 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00b2070 con 0x7ff4f80d1650 2024-04-25T13:43:06.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4f5ffb640 1 -- 172.21.15.80:0/2013688708 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4f00ad680 con 0x7ff4f80d1650 2024-04-25T13:43:06.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4f5ffb640 1 -- 172.21.15.80:0/2013688708 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00ada50 con 0x7ff4f80d1650 2024-04-25T13:43:06.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4fd93b640 1 -- 172.21.15.80:0/2013688708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f80d1650 msgr2=0x7ff4f8164630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/2013688708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f80d1650 0x7ff4f8164630 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7ff4f80b4d80 tx=0x7ff4f001c490 comp rx=0 tx=0).stop 2024-04-25T13:43:06.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4fd93b640 1 -- 172.21.15.80:0/2013688708 shutdown_connections 2024-04-25T13:43:06.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/2013688708 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f80d1650 0x7ff4f8164630 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/2013688708 >> v2:172.21.15.87:3300/0 conn(0x7ff4f8072700 0x7ff4f80d0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4fd93b640 1 -- 172.21.15.80:0/2013688708 >> 172.21.15.80:0/2013688708 conn(0x7ff4f80b3cc0 msgr2=0x7ff4f80b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:06.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4fd93b640 1 -- 172.21.15.80:0/2013688708 shutdown_connections 2024-04-25T13:43:06.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.229+0000 7ff4fd93b640 1 -- 172.21.15.80:0/2013688708 wait complete. 2024-04-25T13:43:06.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 Processor -- start 2024-04-25T13:43:06.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 -- start start 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f8072700 0x7ff4f815f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f80d1240 0x7ff4f8073310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4f80d1650 0x7ff4f8152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f80d0500 con 0x7ff4f8072700 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f80d0220 con 0x7ff4f80d1240 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f8164b70 con 0x7ff4f80d1650 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f8072700 0x7ff4f815f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f8072700 0x7ff4f815f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50322/0 (socket says 172.21.15.80:50322) 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 -- 172.21.15.80:0/1449036821 learned_addr learned my addr 172.21.15.80:0/1449036821 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f67fc640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f80d1240 0x7ff4f8073310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 -- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4f80d1650 msgr2=0x7ff4f8152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f77fe640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4f80d1650 0x7ff4f8152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4f80d1650 0x7ff4f8152300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 -- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f80d1240 msgr2=0x7ff4f8073310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f80d1240 0x7ff4f8073310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 -- 172.21.15.80:0/1449036821 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f815d7e0 con 0x7ff4f8072700 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f67fc640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f80d1240 0x7ff4f8073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4f6ffd640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f8072700 0x7ff4f815f260 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009cd0 tx=0x7ff4e800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4effff640 1 -- 172.21.15.80:0/1449036821 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8018020 con 0x7ff4f8072700 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4effff640 1 -- 172.21.15.80:0/1449036821 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e8014480 con 0x7ff4f8072700 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f815dac0 con 0x7ff4f8072700 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4effff640 1 -- 172.21.15.80:0/1449036821 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8014850 con 0x7ff4f8072700 2024-04-25T13:43:06.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f815e080 con 0x7ff4f8072700 2024-04-25T13:43:06.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.233+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f806a3a0 con 0x7ff4f8072700 2024-04-25T13:43:06.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.237+0000 7ff4effff640 1 -- 172.21.15.80:0/1449036821 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff4e801b020 con 0x7ff4f8072700 2024-04-25T13:43:06.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.237+0000 7ff4effff640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff4d806f430 0x7ff4d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.237+0000 7ff4effff640 1 -- 172.21.15.80:0/1449036821 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7ff4e808fcf0 con 0x7ff4f8072700 2024-04-25T13:43:06.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.237+0000 7ff4f67fc640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff4d806f430 0x7ff4d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.237+0000 7ff4f67fc640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff4d806f430 0x7ff4d8071940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8158ee0 tx=0x7ff4e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:06.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.241+0000 7ff4effff640 1 -- 172.21.15.80:0/1449036821 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff4e8094070 con 0x7ff4f8072700 2024-04-25T13:43:06.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.389+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff4f8076b60 con 0x7ff4f8072700 2024-04-25T13:43:06.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4effff640 1 -- 172.21.15.80:0/1449036821 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v55) ==== 93+0+10 (secure 0 0 0) 0x7ff4e8022030 con 0x7ff4f8072700 2024-04-25T13:43:06.395 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff4d806f430 msgr2=0x7ff4d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff4d806f430 0x7ff4d8071940 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8158ee0 tx=0x7ff4e0009040 comp rx=0 tx=0).stop 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f8072700 msgr2=0x7ff4f815f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f8072700 0x7ff4f815f260 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8009cd0 tx=0x7ff4e800c490 comp rx=0 tx=0).stop 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 shutdown_connections 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff4d806f430 0x7ff4d8071940 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff4f80d1650 0x7ff4f8152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4f80d1240 0x7ff4f8073310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 --2- 172.21.15.80:0/1449036821 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff4f8072700 0x7ff4f815f260 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 >> 172.21.15.80:0/1449036821 conn(0x7ff4f80b3cc0 msgr2=0x7ff4f80b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 shutdown_connections 2024-04-25T13:43:06.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.393+0000 7ff4fd93b640 1 -- 172.21.15.80:0/1449036821 wait complete. 2024-04-25T13:43:06.415 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-04-25T13:43:06.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.545+0000 7f58664ff640 1 Processor -- start 2024-04-25T13:43:06.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.545+0000 7f58664ff640 1 -- start start 2024-04-25T13:43:06.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f58664ff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5860161240 0x7f58600723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f58664ff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5860161650 0x7f58600728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f58664ff640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f586006fcf0 con 0x7f586006f5a0 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f58664ff640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5860073f10 con 0x7f5860161650 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f58664ff640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f58600b0c60 con 0x7f5860161240 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f5864a9c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5860161650 0x7f58600728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f5864a9c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5860161650 0x7f58600728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50338/0 (socket says 172.21.15.80:50338) 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f5864a9c640 1 -- 172.21.15.80:0/3281274698 learned_addr learned my addr 172.21.15.80:0/3281274698 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f585f7fe640 1 --2- 172.21.15.80:0/3281274698 >> v2:172.21.15.87:3300/0 conn(0x7f5860161240 0x7f58600723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f5864a9c640 1 -- 172.21.15.80:0/3281274698 >> v2:172.21.15.87:3300/0 conn(0x7f5860161240 msgr2=0x7f58600723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f5864a9c640 1 --2- 172.21.15.80:0/3281274698 >> v2:172.21.15.87:3300/0 conn(0x7f5860161240 0x7f58600723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f5864a9c640 1 -- 172.21.15.80:0/3281274698 >> v1:172.21.15.87:6789/0 conn(0x7f586006f5a0 legacy=0x7f5860160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:06.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.549+0000 7f5864a9c640 1 -- 172.21.15.80:0/3281274698 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58600b0f40 con 0x7f5860161650 2024-04-25T13:43:06.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f5864a9c640 1 --2- 172.21.15.80:0/3281274698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5860161650 0x7f58600728e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f586006ff70 tx=0x7f585801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a1eabd71ef8957 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:06.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f585effd640 1 -- 172.21.15.80:0/3281274698 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580a5660 con 0x7f5860161650 2024-04-25T13:43:06.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f585effd640 1 -- 172.21.15.80:0/3281274698 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58580a5800 con 0x7f5860161650 2024-04-25T13:43:06.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f585effd640 1 -- 172.21.15.80:0/3281274698 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58580a5bd0 con 0x7f5860161650 2024-04-25T13:43:06.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 -- 172.21.15.80:0/3281274698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5860161650 msgr2=0x7f58600728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 --2- 172.21.15.80:0/3281274698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5860161650 0x7f58600728e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f586006ff70 tx=0x7f585801c920 comp rx=0 tx=0).stop 2024-04-25T13:43:06.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 -- 172.21.15.80:0/3281274698 shutdown_connections 2024-04-25T13:43:06.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 --2- 172.21.15.80:0/3281274698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5860161650 0x7f58600728e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 --2- 172.21.15.80:0/3281274698 >> v2:172.21.15.87:3300/0 conn(0x7f5860161240 0x7f58600723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 -- 172.21.15.80:0/3281274698 >> 172.21.15.80:0/3281274698 conn(0x7f586006d080 msgr2=0x7f58600b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:06.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 -- 172.21.15.80:0/3281274698 shutdown_connections 2024-04-25T13:43:06.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 -- 172.21.15.80:0/3281274698 wait complete. 2024-04-25T13:43:06.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 Processor -- start 2024-04-25T13:43:06.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.553+0000 7f58664ff640 1 -- start start 2024-04-25T13:43:06.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f586006f5a0 0x7f586014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5860161240 0x7f58600cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5860161650 0x7f58600d0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f586006e820 con 0x7f586006f5a0 2024-04-25T13:43:06.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f586006e6a0 con 0x7f5860161240 2024-04-25T13:43:06.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f586006e9a0 con 0x7f5860161650 2024-04-25T13:43:06.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f586006f5a0 0x7f586014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f586006f5a0 0x7f586014ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50342/0 (socket says 172.21.15.80:50342) 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 -- 172.21.15.80:0/393999366 learned_addr learned my addr 172.21.15.80:0/393999366 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585f7fe640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5860161240 0x7f58600cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f5864a9c640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5860161650 0x7f58600d0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 -- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5860161650 msgr2=0x7f58600d0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5860161650 0x7f58600d0370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 -- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5860161240 msgr2=0x7f58600cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5860161240 0x7f58600cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 -- 172.21.15.80:0/393999366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586014cab0 con 0x7f586006f5a0 2024-04-25T13:43:06.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585f7fe640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5860161240 0x7f58600cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:06.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585ffff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f586006f5a0 0x7f586014ea10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f585000b440 tx=0x7f585000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:06.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585cff9640 1 -- 172.21.15.80:0/393999366 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850015020 con 0x7f586006f5a0 2024-04-25T13:43:06.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585cff9640 1 -- 172.21.15.80:0/393999366 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5850011890 con 0x7f586006f5a0 2024-04-25T13:43:06.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586014cd90 con 0x7f586006f5a0 2024-04-25T13:43:06.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f585cff9640 1 -- 172.21.15.80:0/393999366 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850011c80 con 0x7f586006f5a0 2024-04-25T13:43:06.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586014d350 con 0x7f586006f5a0 2024-04-25T13:43:06.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.557+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f586006a3a0 con 0x7f586006f5a0 2024-04-25T13:43:06.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.561+0000 7f585cff9640 1 -- 172.21.15.80:0/393999366 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5850018020 con 0x7f586006f5a0 2024-04-25T13:43:06.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.561+0000 7f585cff9640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f584c06f430 0x7f584c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:06.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.561+0000 7f585cff9640 1 -- 172.21.15.80:0/393999366 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f585008eea0 con 0x7f586006f5a0 2024-04-25T13:43:06.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.561+0000 7f585f7fe640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f584c06f430 0x7f584c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:06.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.561+0000 7f585f7fe640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f584c06f430 0x7f584c071940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f58600b3b30 tx=0x7f5854008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:06.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.565+0000 7f585cff9640 1 -- 172.21.15.80:0/393999366 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f585001f030 con 0x7f586006f5a0 2024-04-25T13:43:06.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:06.769+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) -- 0x7f58600739c0 con 0x7f586006f5a0 2024-04-25T13:43:07.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f585cff9640 1 -- 172.21.15.80:0/393999366 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v161) ==== 137+0+0 (secure 0 0 0) 0x7f585000d040 con 0x7f586006f5a0 2024-04-25T13:43:07.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f584c06f430 msgr2=0x7f584c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f584c06f430 0x7f584c071940 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f58600b3b30 tx=0x7f5854008040 comp rx=0 tx=0).stop 2024-04-25T13:43:07.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f586006f5a0 msgr2=0x7f586014ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:07.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f586006f5a0 0x7f586014ea10 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f585000b440 tx=0x7f585000b900 comp rx=0 tx=0).stop 2024-04-25T13:43:07.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 shutdown_connections 2024-04-25T13:43:07.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f584c06f430 0x7f584c071940 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5860161650 0x7f58600d0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5860161240 0x7f58600cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 --2- 172.21.15.80:0/393999366 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f586006f5a0 0x7f586014ea10 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:07.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 >> 172.21.15.80:0/393999366 conn(0x7f586006d080 msgr2=0x7f58600d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:07.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 shutdown_connections 2024-04-25T13:43:07.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:07.701+0000 7f58664ff640 1 -- 172.21.15.80:0/393999366 wait complete. 2024-04-25T13:43:07.728 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-25T13:43:07.891 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_objectstore":"memstore"} 2024-04-25T13:43:07.891 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-25T13:43:07.891 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:43:08.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.041+0000 7f1298087640 1 Processor -- start 2024-04-25T13:43:08.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1298087640 1 -- start start 2024-04-25T13:43:08.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1298087640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12900d3ed0 0x7f12900d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1298087640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f12900ba770 0x7f12900b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1298087640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12900b92a0 con 0x7f12900d2ad0 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1298087640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f12900d4c10 con 0x7f12900ba770 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1298087640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f12900d4ef0 con 0x7f12900d3ed0 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295622640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f12900d2ad0 0x7f12900b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42470/0 (socket says 172.21.15.80:42470) 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295622640 1 -- 172.21.15.80:0/3213291184 learned_addr learned my addr 172.21.15.80:0/3213291184 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1296624640 1 --2- 172.21.15.80:0/3213291184 >> v2:172.21.15.87:3300/0 conn(0x7f12900ba770 0x7f12900b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295e23640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12900d3ed0 0x7f12900d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 896656559 0 0) 0x7f1280001120 con 0x7f12900d2ad0 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f127c003530 con 0x7f12900d2ad0 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4247089159 0 0) 0x7f127c003530 con 0x7f12900d2ad0 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12900d3ed0 msgr2=0x7f12900d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12900d3ed0 0x7f12900d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 >> v2:172.21.15.87:3300/0 conn(0x7f12900ba770 msgr2=0x7f12900b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 --2- 172.21.15.80:0/3213291184 >> v2:172.21.15.87:3300/0 conn(0x7f12900ba770 0x7f12900b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12900d51d0 con 0x7f12900d2ad0 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1280001850 con 0x7f12900d2ad0 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 >> v1:172.21.15.80:6789/0 conn(0x7f12900d2ad0 legacy=0x7f12900b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f127c0039c0 0x7f127c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c004810 0x7f127c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12900d3ed0 0x7f127c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f12900d4c10 con 0x7f127c0039c0 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f12900d4ef0 con 0x7f127c004810 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f127c00ba80 con 0x7f12900d3ed0 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295e23640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f127c0039c0 0x7f127c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295622640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c004810 0x7f127c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1296624640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12900d3ed0 0x7f127c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295e23640 1 -- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12900d3ed0 msgr2=0x7f127c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295e23640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12900d3ed0 0x7f127c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295e23640 1 -- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c004810 msgr2=0x7f127c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:08.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295e23640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c004810 0x7f127c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295e23640 1 -- 172.21.15.80:0/3213291184 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c003530 con 0x7f127c0039c0 2024-04-25T13:43:08.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.045+0000 7f1295622640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c004810 0x7f127c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1295e23640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f127c0039c0 0x7f127c003e30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f128c009ae0 tx=0x7f128c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9847cb5341cee862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c0b2070 con 0x7f127c0039c0 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f128c0a5570 con 0x7f127c0039c0 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1294e21640 1 -- 172.21.15.80:0/3213291184 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c0096b0 con 0x7f127c0039c0 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1298087640 1 -- 172.21.15.80:0/3213291184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f127c0039c0 msgr2=0x7f127c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1298087640 1 --2- 172.21.15.80:0/3213291184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f127c0039c0 0x7f127c003e30 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f128c009ae0 tx=0x7f128c01ca90 comp rx=0 tx=0).stop 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1296624640 1 -- 172.21.15.80:0/3213291184 reap_dead start 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1298087640 1 -- 172.21.15.80:0/3213291184 shutdown_connections 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1298087640 1 -- 172.21.15.80:0/3213291184 >> 172.21.15.80:0/3213291184 conn(0x7f12900737b0 msgr2=0x7f129006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:08.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1298087640 1 -- 172.21.15.80:0/3213291184 shutdown_connections 2024-04-25T13:43:08.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.049+0000 7f1298087640 1 -- 172.21.15.80:0/3213291184 wait complete. 2024-04-25T13:43:08.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 Processor -- start 2024-04-25T13:43:08.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 -- start start 2024-04-25T13:43:08.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c0039c0 0x7f129016a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12900cd710 0x7f1290158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1290159110 0x7f129014c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f12900b8320 con 0x7f12900cd710 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f12900b84a0 con 0x7f127c0039c0 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1290160620 con 0x7f1290159110 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12900cd710 0x7f1290158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12900cd710 0x7f1290158bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50348/0 (socket says 172.21.15.80:50348) 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 -- 172.21.15.80:0/126057224 learned_addr learned my addr 172.21.15.80:0/126057224 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295e23640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c0039c0 0x7f129016a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1296624640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1290159110 0x7f129014c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 -- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1290159110 msgr2=0x7f129014c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1290159110 0x7f129014c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 -- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c0039c0 msgr2=0x7f129016a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c0039c0 0x7f129016a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 -- 172.21.15.80:0/126057224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1290159520 con 0x7f12900cd710 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1296624640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1290159110 0x7f129014c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:08.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1295622640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12900cd710 0x7f1290158bd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1280001a40 tx=0x7f128000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:08.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f12867fc640 1 -- 172.21.15.80:0/126057224 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128000c8d0 con 0x7f12900cd710 2024-04-25T13:43:08.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f12867fc640 1 -- 172.21.15.80:0/126057224 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1280018070 con 0x7f12900cd710 2024-04-25T13:43:08.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f12867fc640 1 -- 172.21.15.80:0/126057224 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128000cc70 con 0x7f12900cd710 2024-04-25T13:43:08.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12901f9ef0 con 0x7f12900cd710 2024-04-25T13:43:08.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12901fa4b0 con 0x7f12900cd710 2024-04-25T13:43:08.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.053+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129006a3a0 con 0x7f12900cd710 2024-04-25T13:43:08.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.057+0000 7f12867fc640 1 -- 172.21.15.80:0/126057224 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f128001d460 con 0x7f12900cd710 2024-04-25T13:43:08.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.057+0000 7f12867fc640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f127406f430 0x7f1274071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.057+0000 7f12867fc640 1 -- 172.21.15.80:0/126057224 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f128008ee30 con 0x7f12900cd710 2024-04-25T13:43:08.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.057+0000 7f1295e23640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f127406f430 0x7f1274071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.057+0000 7f1295e23640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f127406f430 0x7f1274071940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f128c009ae0 tx=0x7f128c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:08.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.061+0000 7f12867fc640 1 -- 172.21.15.80:0/126057224 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1290159520 con 0x7f12900cd710 2024-04-25T13:43:08.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.229+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1290156f50 con 0x7f12900cd710 2024-04-25T13:43:08.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f12867fc640 1 -- 172.21.15.80:0/126057224 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) ==== 122+0+0 (secure 0 0 0) 0x7f12900b92a0 con 0x7f12900cd710 2024-04-25T13:43:08.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f127406f430 msgr2=0x7f1274071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f127406f430 0x7f1274071940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f128c009ae0 tx=0x7f128c0046d0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12900cd710 msgr2=0x7f1290158bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12900cd710 0x7f1290158bd0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f1280001a40 tx=0x7f128000ec80 comp rx=0 tx=0).stop 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 shutdown_connections 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f127406f430 0x7f1274071940 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1290159110 0x7f129014c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12900cd710 0x7f1290158bd0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 --2- 172.21.15.80:0/126057224 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f127c0039c0 0x7f129016a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.233+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 >> 172.21.15.80:0/126057224 conn(0x7f12900737b0 msgr2=0x7f129015a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.237+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 shutdown_connections 2024-04-25T13:43:08.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.237+0000 7f1298087640 1 -- 172.21.15.80:0/126057224 wait complete. 2024-04-25T13:43:08.258 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:43:08.259 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:43:08.308 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:43:08.356 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:43:08.356 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:43:08.356 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:43:08.361 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:43:08.361 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:43:08.361 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:43:08.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:43:08.367 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:43:08.367 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:43:08.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:43:08.407 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:43:08.407 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:43:08.415 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:43:08.415 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:43:08.415 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:43:08.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.593+0000 7f21d40be640 1 Processor -- start 2024-04-25T13:43:08.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.593+0000 7f21d40be640 1 -- start start 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d40be640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f21cc161f90 0x7f21cc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d40be640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f21cc15ec40 0x7f21cc15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d40be640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21cc0d0d70 con 0x7f21cc0d0510 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d40be640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc0d1050 con 0x7f21cc15ec40 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d40be640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f21cc06dca0 con 0x7f21cc161f90 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d265b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f21cc15ec40 0x7f21cc15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d1659640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f21cc0d0510 0x7f21cc15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42472/0 (socket says 172.21.15.80:42472) 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d1659640 1 -- 172.21.15.80:0/336359835 learned_addr learned my addr 172.21.15.80:0/336359835 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d1e5a640 1 --2- 172.21.15.80:0/336359835 >> v2:172.21.15.87:3300/0 conn(0x7f21cc161f90 0x7f21cc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d265b640 1 -- 172.21.15.80:0/336359835 >> v2:172.21.15.87:3300/0 conn(0x7f21cc161f90 msgr2=0x7f21cc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d265b640 1 --2- 172.21.15.80:0/336359835 >> v2:172.21.15.87:3300/0 conn(0x7f21cc161f90 0x7f21cc0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d265b640 1 -- 172.21.15.80:0/336359835 >> v1:172.21.15.80:6789/0 conn(0x7f21cc0d0510 legacy=0x7f21cc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.597+0000 7f21d265b640 1 -- 172.21.15.80:0/336359835 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc06df80 con 0x7f21cc15ec40 2024-04-25T13:43:08.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d265b640 1 --2- 172.21.15.80:0/336359835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc15f050 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0d17b0 tx=0x7f21c401c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeea750e89d3ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:08.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d0e58640 1 -- 172.21.15.80:0/336359835 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c40b2070 con 0x7f21cc15ec40 2024-04-25T13:43:08.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d0e58640 1 -- 172.21.15.80:0/336359835 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c40a5830 con 0x7f21cc15ec40 2024-04-25T13:43:08.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d0e58640 1 -- 172.21.15.80:0/336359835 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c40a5c00 con 0x7f21cc15ec40 2024-04-25T13:43:08.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 -- 172.21.15.80:0/336359835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 msgr2=0x7f21cc15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 --2- 172.21.15.80:0/336359835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc15f050 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0d17b0 tx=0x7f21c401c840 comp rx=0 tx=0).stop 2024-04-25T13:43:08.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 -- 172.21.15.80:0/336359835 shutdown_connections 2024-04-25T13:43:08.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 --2- 172.21.15.80:0/336359835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc15f050 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 --2- 172.21.15.80:0/336359835 >> v2:172.21.15.87:3300/0 conn(0x7f21cc161f90 0x7f21cc0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 -- 172.21.15.80:0/336359835 >> 172.21.15.80:0/336359835 conn(0x7f21cc06d080 msgr2=0x7f21cc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:08.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 -- 172.21.15.80:0/336359835 shutdown_connections 2024-04-25T13:43:08.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 -- 172.21.15.80:0/336359835 wait complete. 2024-04-25T13:43:08.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 Processor -- start 2024-04-25T13:43:08.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.601+0000 7f21d40be640 1 -- start start 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21cc0d0510 0x7f21cc142650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc142b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21cc161f90 0x7f21cc163640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc06f4b0 con 0x7f21cc15ec40 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f21cc06f630 con 0x7f21cc0d0510 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f21cc0d0920 con 0x7f21cc161f90 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc142b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc142b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50360/0 (socket says 172.21.15.80:50360) 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 -- 172.21.15.80:0/2736877573 learned_addr learned my addr 172.21.15.80:0/2736877573 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 -- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21cc161f90 msgr2=0x7f21cc163640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1e5a640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21cc0d0510 0x7f21cc142650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d265b640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21cc161f90 0x7f21cc163640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21cc161f90 0x7f21cc163640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 -- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21cc0d0510 msgr2=0x7f21cc142650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21cc0d0510 0x7f21cc142650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 -- 172.21.15.80:0/2736877573 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21cc167090 con 0x7f21cc15ec40 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d1659640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc142b90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c370 tx=0x7f21c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21ca7fc640 1 -- 172.21.15.80:0/2736877573 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c0017070 con 0x7f21cc15ec40 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21ca7fc640 1 -- 172.21.15.80:0/2736877573 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21c0005cf0 con 0x7f21cc15ec40 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21cc167370 con 0x7f21cc15ec40 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21ca7fc640 1 -- 172.21.15.80:0/2736877573 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21c00072f0 con 0x7f21cc15ec40 2024-04-25T13:43:08.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21cc1646c0 con 0x7f21cc15ec40 2024-04-25T13:43:08.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21cc06a3a0 con 0x7f21cc15ec40 2024-04-25T13:43:08.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.605+0000 7f21ca7fc640 1 -- 172.21.15.80:0/2736877573 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f21c001a070 con 0x7f21cc15ec40 2024-04-25T13:43:08.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.609+0000 7f21ca7fc640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f21b806f400 0x7f21b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:08.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.609+0000 7f21ca7fc640 1 -- 172.21.15.80:0/2736877573 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7f21c008e6b0 con 0x7f21cc15ec40 2024-04-25T13:43:08.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.609+0000 7f21d1e5a640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f21b806f400 0x7f21b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:08.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.609+0000 7f21d1e5a640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f21b806f400 0x7f21b8071910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f21cc06fa20 tx=0x7f21bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:08.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.609+0000 7f21ca7fc640 1 -- 172.21.15.80:0/2736877573 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f21c0021020 con 0x7f21cc15ec40 2024-04-25T13:43:08.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.797+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21cc069cb0 con 0x7f21cc15ec40 2024-04-25T13:43:08.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.797+0000 7f21ca7fc640 1 -- 172.21.15.80:0/2736877573 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3866 (secure 0 0 0) 0x7f21c005ba10 con 0x7f21cc15ec40 2024-04-25T13:43:08.802 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:08.802 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3317328766","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3317328766},{"type":"v1","addr":"172.21.15.87:6837","nonce":3317328766}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5182,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1586731006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1586731006},{"type":"v1","addr":"172.21.15.87:6835","nonce":1586731006}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":5727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/3229537153","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3229537153},{"type":"v1","addr":"172.21.15.80:6833","nonce":3229537153}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":6117,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/834032246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":834032246},{"type":"v1","addr":"172.21.15.80:6835","nonce":834032246}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-04-25T13:43:08.802 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 26 2024-04-25T13:43:08.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f21b806f400 msgr2=0x7f21b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f21b806f400 0x7f21b8071910 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f21cc06fa20 tx=0x7f21bc008040 comp rx=0 tx=0).stop 2024-04-25T13:43:08.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 msgr2=0x7f21cc142b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc142b90 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f21c000c370 tx=0x7f21c000c830 comp rx=0 tx=0).stop 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 shutdown_connections 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f21b806f400 0x7f21b8071910 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21cc161f90 0x7f21cc163640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f21cc15ec40 0x7f21cc142b90 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 --2- 172.21.15.80:0/2736877573 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f21cc0d0510 0x7f21cc142650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 >> 172.21.15.80:0/2736877573 conn(0x7f21cc06d080 msgr2=0x7f21cc0d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 shutdown_connections 2024-04-25T13:43:08.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:08.801+0000 7f21d40be640 1 -- 172.21.15.80:0/2736877573 wait complete. 2024-04-25T13:43:08.823 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:43:08.823 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:43:08.824 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:43:08.824 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:43:08.824 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:43:08.824 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:43:08.824 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:43:08.824 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:43:08.824 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:43:08.825 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:43:08.825 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:43:08.825 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:43:08.825 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:43:08.825 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:43:08.825 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:43:08.825 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:43:08.837 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:43:08.838 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:43:08.838 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:43:08.839 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:43:08.849 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:43:08.849 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:43:08.852 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:43:08.852 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:43:08.880 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:43:08.880 DEBUG:teuthology.parallel:result is None 2024-04-25T13:43:08.880 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:43:08.881 DEBUG:teuthology.parallel:result is None 2024-04-25T13:43:08.895 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:43:08.896 DEBUG:teuthology.parallel:result is None 2024-04-25T13:43:08.897 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:43:08.897 DEBUG:teuthology.parallel:result is None 2024-04-25T13:43:08.898 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:43:08.966 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:43:08.966 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:43:08.998 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:43:09.000 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:43:09.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.033+0000 7fbe3ffd7640 1 Processor -- start 2024-04-25T13:43:09.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.033+0000 7fbe3ffd7640 1 -- start start 2024-04-25T13:43:09.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3ffd7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe38160430 0x7fbe380d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3ffd7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbe3806d9f0 0x7fbe3806de00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3ffd7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe380d2cc0 con 0x7fbe380ccdc0 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3ffd7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbe380d38d0 con 0x7fbe3806d9f0 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3ffd7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe380d3310 con 0x7fbe38160430 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3e574640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fbe380ccdc0 0x7fbe380cd1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42482/0 (socket says 172.21.15.80:42482) 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3e574640 1 -- 172.21.15.80:0/1899820364 learned_addr learned my addr 172.21.15.80:0/1899820364 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3d572640 1 --2- 172.21.15.80:0/1899820364 >> v2:172.21.15.87:3300/0 conn(0x7fbe3806d9f0 0x7fbe3806de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3dd73640 1 --2- 172.21.15.80:0/1899820364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe38160430 0x7fbe380d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3cd71640 1 -- 172.21.15.80:0/1899820364 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1078854685 0 0) 0x7fbe380d2cc0 con 0x7fbe380ccdc0 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3cd71640 1 -- 172.21.15.80:0/1899820364 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbe24003530 con 0x7fbe380ccdc0 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3d572640 1 -- 172.21.15.80:0/1899820364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe38160430 msgr2=0x7fbe380d0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3d572640 1 --2- 172.21.15.80:0/1899820364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe38160430 0x7fbe380d0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3d572640 1 -- 172.21.15.80:0/1899820364 >> v1:172.21.15.80:6789/0 conn(0x7fbe380ccdc0 legacy=0x7fbe380cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.037+0000 7fbe3d572640 1 -- 172.21.15.80:0/1899820364 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe380d35f0 con 0x7fbe3806d9f0 2024-04-25T13:43:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3d572640 1 --2- 172.21.15.80:0/1899820364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 0x7fbe3806de00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe280097c0 tx=0x7fbe2801c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc802bee938bd53a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:09.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3cd71640 1 -- 172.21.15.80:0/1899820364 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280b0070 con 0x7fbe3806d9f0 2024-04-25T13:43:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3cd71640 1 -- 172.21.15.80:0/1899820364 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe280ab600 con 0x7fbe3806d9f0 2024-04-25T13:43:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3cd71640 1 -- 172.21.15.80:0/1899820364 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe280ab9d0 con 0x7fbe3806d9f0 2024-04-25T13:43:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1899820364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 msgr2=0x7fbe3806de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1899820364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 0x7fbe3806de00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbe280097c0 tx=0x7fbe2801c4a0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1899820364 shutdown_connections 2024-04-25T13:43:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1899820364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 0x7fbe3806de00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1899820364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe38160430 0x7fbe380d0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1899820364 >> 172.21.15.80:0/1899820364 conn(0x7fbe380b5740 msgr2=0x7fbe380b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1899820364 shutdown_connections 2024-04-25T13:43:09.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1899820364 wait complete. 2024-04-25T13:43:09.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 Processor -- start 2024-04-25T13:43:09.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.041+0000 7fbe3ffd7640 1 -- start start 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 0x7fbe380b6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe380ccdc0 0x7fbe380b6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe38160430 0x7fbe380b71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3806e340 con 0x7fbe38160430 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe38160840 con 0x7fbe3806d9f0 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe3816d870 con 0x7fbe380ccdc0 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe38160430 0x7fbe380b71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe38160430 0x7fbe380b71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50380/0 (socket says 172.21.15.80:50380) 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 -- 172.21.15.80:0/1973162685 learned_addr learned my addr 172.21.15.80:0/1973162685 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3dd73640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 0x7fbe380b6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3d572640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe380ccdc0 0x7fbe380b6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 -- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe380ccdc0 msgr2=0x7fbe380b6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe380ccdc0 0x7fbe380b6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 -- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 msgr2=0x7fbe380b6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 0x7fbe380b6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 -- 172.21.15.80:0/1973162685 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe380b31d0 con 0x7fbe38160430 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3e574640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe38160430 0x7fbe380b71b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000cad0 tx=0x7fbe3000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe2e7fc640 1 -- 172.21.15.80:0/1973162685 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30017070 con 0x7fbe38160430 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe2e7fc640 1 -- 172.21.15.80:0/1973162685 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe3000abd0 con 0x7fbe38160430 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe380b1cd0 con 0x7fbe38160430 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe2e7fc640 1 -- 172.21.15.80:0/1973162685 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe30012400 con 0x7fbe38160430 2024-04-25T13:43:09.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe380b1f90 con 0x7fbe38160430 2024-04-25T13:43:09.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe3806a3a0 con 0x7fbe38160430 2024-04-25T13:43:09.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe2e7fc640 1 -- 172.21.15.80:0/1973162685 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbe3001a070 con 0x7fbe38160430 2024-04-25T13:43:09.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe2e7fc640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe1c06f430 0x7fbe1c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe2e7fc640 1 -- 172.21.15.80:0/1973162685 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 4943+0+0 (secure 0 0 0) 0x7fbe3008ec10 con 0x7fbe38160430 2024-04-25T13:43:09.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.045+0000 7fbe3dd73640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe1c06f430 0x7fbe1c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.049+0000 7fbe3dd73640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe1c06f430 0x7fbe1c071940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009ac0 tx=0x7fbe34009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:09.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.053+0000 7fbe2e7fc640 1 -- 172.21.15.80:0/1973162685 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe3005b7b0 con 0x7fbe38160430 2024-04-25T13:43:09.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.217+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fbe380b33a0 con 0x7fbe38160430 2024-04-25T13:43:09.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.725+0000 7fbe2e7fc640 1 -- 172.21.15.80:0/1973162685 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) ==== 97+0+0 (secure 0 0 0) 0x7fbe3005fab0 con 0x7fbe38160430 2024-04-25T13:43:09.730 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:43:09.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe1c06f430 msgr2=0x7fbe1c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe1c06f430 0x7fbe1c071940 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009ac0 tx=0x7fbe34009190 comp rx=0 tx=0).stop 2024-04-25T13:43:09.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe38160430 msgr2=0x7fbe380b71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe38160430 0x7fbe380b71b0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fbe3000cad0 tx=0x7fbe3000cf90 comp rx=0 tx=0).stop 2024-04-25T13:43:09.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 shutdown_connections 2024-04-25T13:43:09.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe1c06f430 0x7fbe1c071940 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe38160430 0x7fbe380b71b0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe380ccdc0 0x7fbe380b6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 --2- 172.21.15.80:0/1973162685 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe3806d9f0 0x7fbe380b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 >> 172.21.15.80:0/1973162685 conn(0x7fbe380b5740 msgr2=0x7fbe381580c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:09.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 shutdown_connections 2024-04-25T13:43:09.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.729+0000 7fbe3ffd7640 1 -- 172.21.15.80:0/1973162685 wait complete. 2024-04-25T13:43:09.760 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:43:09.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f69a4f4e640 1 Processor -- start 2024-04-25T13:43:09.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f69a4f4e640 1 -- start start 2024-04-25T13:43:09.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f69a4f4e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f69a00726b0 0x7f69a00d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f69a4f4e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a006fd00 0x7f69a00d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f69a4f4e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a00ce0d0 con 0x7f69a0072ac0 2024-04-25T13:43:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f69a4f4e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f69a00d18e0 con 0x7f69a00726b0 2024-04-25T13:43:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f69a4f4e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f69a00ce930 con 0x7f69a006fd00 2024-04-25T13:43:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f699ed9d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a006fd00 0x7f69a00d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f699e59c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f69a00726b0 0x7f69a00d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f699e59c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f69a00726b0 0x7f69a00d0e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:54664/0 (socket says 172.21.15.80:54664) 2024-04-25T13:43:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.897+0000 7f699e59c640 1 -- 172.21.15.80:0/2983861754 learned_addr learned my addr 172.21.15.80:0/2983861754 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:09.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f699e59c640 1 -- 172.21.15.80:0/2983861754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a006fd00 msgr2=0x7f69a00d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:09.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f699e59c640 1 --2- 172.21.15.80:0/2983861754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a006fd00 0x7f69a00d13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f699e59c640 1 -- 172.21.15.80:0/2983861754 >> v1:172.21.15.87:6789/0 conn(0x7f69a0072ac0 legacy=0x7f69a006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f699e59c640 1 -- 172.21.15.80:0/2983861754 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a00cc930 con 0x7f69a00726b0 2024-04-25T13:43:09.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f699e59c640 1 --2- 172.21.15.80:0/2983861754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a00726b0 0x7f69a00d0e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6988009870 tx=0x7f698801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51abd2fc88a38cdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:09.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f699d59a640 1 -- 172.21.15.80:0/2983861754 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880b2070 con 0x7f69a00726b0 2024-04-25T13:43:09.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f699d59a640 1 -- 172.21.15.80:0/2983861754 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69880a9070 con 0x7f69a00726b0 2024-04-25T13:43:09.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f699d59a640 1 -- 172.21.15.80:0/2983861754 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880ad880 con 0x7f69a00726b0 2024-04-25T13:43:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f69a4f4e640 1 -- 172.21.15.80:0/2983861754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a00726b0 msgr2=0x7f69a00d0e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/2983861754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a00726b0 0x7f69a00d0e60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6988009870 tx=0x7f698801c560 comp rx=0 tx=0).stop 2024-04-25T13:43:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f69a4f4e640 1 -- 172.21.15.80:0/2983861754 shutdown_connections 2024-04-25T13:43:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/2983861754 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a006fd00 0x7f69a00d13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/2983861754 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a00726b0 0x7f69a00d0e60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f69a4f4e640 1 -- 172.21.15.80:0/2983861754 >> 172.21.15.80:0/2983861754 conn(0x7f69a00d20d0 msgr2=0x7f69a00d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f69a4f4e640 1 -- 172.21.15.80:0/2983861754 shutdown_connections 2024-04-25T13:43:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.901+0000 7f69a4f4e640 1 -- 172.21.15.80:0/2983861754 wait complete. 2024-04-25T13:43:09.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 Processor -- start 2024-04-25T13:43:09.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 -- start start 2024-04-25T13:43:09.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a006fd00 0x7f69a015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a00726b0 0x7f69a0151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a0072ac0 0x7f69a01563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f69a015e2c0 con 0x7f69a006fd00 2024-04-25T13:43:09.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f69a015e140 con 0x7f69a0072ac0 2024-04-25T13:43:09.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f69a0164640 con 0x7f69a00726b0 2024-04-25T13:43:09.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a006fd00 0x7f69a015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a006fd00 0x7f69a015c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50388/0 (socket says 172.21.15.80:50388) 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 -- 172.21.15.80:0/4234331808 learned_addr learned my addr 172.21.15.80:0/4234331808 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699dd9b640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a00726b0 0x7f69a0151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699ed9d640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a0072ac0 0x7f69a01563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 -- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a00726b0 msgr2=0x7f69a0151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a00726b0 0x7f69a0151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 -- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a0072ac0 msgr2=0x7f69a01563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a0072ac0 0x7f69a01563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 -- 172.21.15.80:0/4234331808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a015c910 con 0x7f69a006fd00 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f699e59c640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a006fd00 0x7f69a015c0c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f69880b0040 tx=0x7f6988019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f6996ffd640 1 -- 172.21.15.80:0/4234331808 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880bb070 con 0x7f69a006fd00 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f6996ffd640 1 -- 172.21.15.80:0/4234331808 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69880add80 con 0x7f69a006fd00 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a015cbf0 con 0x7f69a006fd00 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f6996ffd640 1 -- 172.21.15.80:0/4234331808 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69880b2040 con 0x7f69a006fd00 2024-04-25T13:43:09.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.905+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a015d1b0 con 0x7f69a006fd00 2024-04-25T13:43:09.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.909+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a006a3a0 con 0x7f69a006fd00 2024-04-25T13:43:09.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.909+0000 7f6996ffd640 1 -- 172.21.15.80:0/4234331808 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f69880be070 con 0x7f69a006fd00 2024-04-25T13:43:09.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.909+0000 7f6996ffd640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f698006f400 0x7f6980071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:09.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.909+0000 7f6996ffd640 1 -- 172.21.15.80:0/4234331808 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f6988127ed0 con 0x7f69a006fd00 2024-04-25T13:43:09.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.909+0000 7f699dd9b640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f698006f400 0x7f6980071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:09.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.909+0000 7f699dd9b640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f698006f400 0x7f6980071910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f69a01533f0 tx=0x7f6990009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:09.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:09.913+0000 7f6996ffd640 1 -- 172.21.15.80:0/4234331808 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6988016150 con 0x7f69a006fd00 2024-04-25T13:43:10.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.113+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f69a00cc500 con 0x7f69a006fd00 2024-04-25T13:43:10.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.113+0000 7f6996ffd640 1 -- 172.21.15.80:0/4234331808 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5304 (secure 0 0 0) 0x7f69880f4ac0 con 0x7f69a006fd00 2024-04-25T13:43:10.117 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:10.117 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:43:10.118 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:43:10.119 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.120 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:43:10.121 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.122 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:43:10.123 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:43:10.124 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.125 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:43:10.126 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.127 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.128 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:10.129 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:43:10.130 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:43:10.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f698006f400 msgr2=0x7f6980071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f698006f400 0x7f6980071910 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f69a01533f0 tx=0x7f6990009040 comp rx=0 tx=0).stop 2024-04-25T13:43:10.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a006fd00 msgr2=0x7f69a015c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a006fd00 0x7f69a015c0c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f69880b0040 tx=0x7f6988019300 comp rx=0 tx=0).stop 2024-04-25T13:43:10.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 shutdown_connections 2024-04-25T13:43:10.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f698006f400 0x7f6980071910 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f69a0072ac0 0x7f69a01563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69a00726b0 0x7f69a0151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 --2- 172.21.15.80:0/4234331808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69a006fd00 0x7f69a015c0c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 >> 172.21.15.80:0/4234331808 conn(0x7f69a00d20d0 msgr2=0x7f69a00d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:10.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 shutdown_connections 2024-04-25T13:43:10.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.117+0000 7f69a4f4e640 1 -- 172.21.15.80:0/4234331808 wait complete. 2024-04-25T13:43:10.138 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:43:10.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe389d84640 1 Processor -- start 2024-04-25T13:43:10.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe389d84640 1 -- start start 2024-04-25T13:43:10.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe389d84640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe38406f5a0 0x7fe384160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe389d84640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe384161070 0x7fe384161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe389d84640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe38406fcf0 con 0x7fe3840b4bd0 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe389d84640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe3841619c0 con 0x7fe384161070 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe389d84640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe384073f10 con 0x7fe38406f5a0 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe3837fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe38406f5a0 0x7fe384160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe37affd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe384161070 0x7fe384161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe3837fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe38406f5a0 0x7fe384160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:38862/0 (socket says 172.21.15.80:38862) 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.281+0000 7fe3837fe640 1 -- 172.21.15.80:0/4091593653 learned_addr learned my addr 172.21.15.80:0/4091593653 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe37affd640 1 -- 172.21.15.80:0/4091593653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe38406f5a0 msgr2=0x7fe384160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe37affd640 1 --2- 172.21.15.80:0/4091593653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe38406f5a0 0x7fe384160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe37affd640 1 -- 172.21.15.80:0/4091593653 >> v1:172.21.15.87:6789/0 conn(0x7fe3840b4bd0 legacy=0x7fe3840b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe37affd640 1 -- 172.21.15.80:0/4091593653 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3840b0c60 con 0x7fe384161070 2024-04-25T13:43:10.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe37affd640 1 --2- 172.21.15.80:0/4091593653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 0x7fe384161480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0097f0 tx=0x7fe36c01c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36404f72173f07e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe382ffd640 1 -- 172.21.15.80:0/4091593653 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0b0070 con 0x7fe384161070 2024-04-25T13:43:10.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe382ffd640 1 -- 172.21.15.80:0/4091593653 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe36c0ab600 con 0x7fe384161070 2024-04-25T13:43:10.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe382ffd640 1 -- 172.21.15.80:0/4091593653 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c0ab9d0 con 0x7fe384161070 2024-04-25T13:43:10.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe389d84640 1 -- 172.21.15.80:0/4091593653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 msgr2=0x7fe384161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe389d84640 1 --2- 172.21.15.80:0/4091593653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 0x7fe384161480 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0097f0 tx=0x7fe36c01c410 comp rx=0 tx=0).stop 2024-04-25T13:43:10.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe389d84640 1 -- 172.21.15.80:0/4091593653 shutdown_connections 2024-04-25T13:43:10.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe389d84640 1 --2- 172.21.15.80:0/4091593653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 0x7fe384161480 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe389d84640 1 --2- 172.21.15.80:0/4091593653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe38406f5a0 0x7fe384160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe389d84640 1 -- 172.21.15.80:0/4091593653 >> 172.21.15.80:0/4091593653 conn(0x7fe38406d080 msgr2=0x7fe3840b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:10.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe389d84640 1 -- 172.21.15.80:0/4091593653 shutdown_connections 2024-04-25T13:43:10.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.285+0000 7fe389d84640 1 -- 172.21.15.80:0/4091593653 wait complete. 2024-04-25T13:43:10.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe389d84640 1 Processor -- start 2024-04-25T13:43:10.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe389d84640 1 -- start start 2024-04-25T13:43:10.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe389d84640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe38406f5a0 0x7fe38415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe389d84640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3840b4bd0 0x7fe384152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe389d84640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 0x7fe3840b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe389d84640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe38406ee20 con 0x7fe38406f5a0 2024-04-25T13:43:10.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe389d84640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe38406eca0 con 0x7fe384161070 2024-04-25T13:43:10.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe389d84640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe38406efa0 con 0x7fe3840b4bd0 2024-04-25T13:43:10.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe3837fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe38406f5a0 0x7fe38415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe3837fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe38406f5a0 0x7fe38415f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50392/0 (socket says 172.21.15.80:50392) 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe3837fe640 1 -- 172.21.15.80:0/3052380444 learned_addr learned my addr 172.21.15.80:0/3052380444 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe37affd640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3840b4bd0 0x7fe384152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe383fff640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 0x7fe3840b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe3837fe640 1 -- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3840b4bd0 msgr2=0x7fe384152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe3837fe640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3840b4bd0 0x7fe384152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe3837fe640 1 -- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 msgr2=0x7fe3840b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe3837fe640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 0x7fe3840b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe3837fe640 1 -- 172.21.15.80:0/3052380444 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe384156b60 con 0x7fe38406f5a0 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.289+0000 7fe383fff640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 0x7fe3840b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe3837fe640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe38406f5a0 0x7fe38415f6b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe374010e20 tx=0x7fe37400e730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe380ff9640 1 -- 172.21.15.80:0/3052380444 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37401e070 con 0x7fe38406f5a0 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe380ff9640 1 -- 172.21.15.80:0/3052380444 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe374011660 con 0x7fe38406f5a0 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe384156e40 con 0x7fe38406f5a0 2024-04-25T13:43:10.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe380ff9640 1 -- 172.21.15.80:0/3052380444 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe374011a30 con 0x7fe38406f5a0 2024-04-25T13:43:10.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe384157400 con 0x7fe38406f5a0 2024-04-25T13:43:10.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe38406a3a0 con 0x7fe38406f5a0 2024-04-25T13:43:10.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe380ff9640 1 -- 172.21.15.80:0/3052380444 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe374021070 con 0x7fe38406f5a0 2024-04-25T13:43:10.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe380ff9640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe36406f400 0x7fe364071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe380ff9640 1 -- 172.21.15.80:0/3052380444 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fe3740919f0 con 0x7fe38406f5a0 2024-04-25T13:43:10.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe37affd640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe36406f400 0x7fe364071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.293+0000 7fe37affd640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe36406f400 0x7fe364071910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0ad000 tx=0x7fe36c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.297+0000 7fe380ff9640 1 -- 172.21.15.80:0/3052380444 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe37405e5e0 con 0x7fe38406f5a0 2024-04-25T13:43:10.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3840739c0 con 0x7fe38406f5a0 2024-04-25T13:43:10.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe380ff9640 1 -- 172.21.15.80:0/3052380444 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+12819 (secure 0 0 0) 0x7fe3740628e0 con 0x7fe38406f5a0 2024-04-25T13:43:10.463 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:10.463 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:43:09.742756+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:43:10.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe36406f400 msgr2=0x7fe364071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe36406f400 0x7fe364071910 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe36c0ad000 tx=0x7fe36c0a6000 comp rx=0 tx=0).stop 2024-04-25T13:43:10.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe38406f5a0 msgr2=0x7fe38415f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe38406f5a0 0x7fe38415f6b0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe374010e20 tx=0x7fe37400e730 comp rx=0 tx=0).stop 2024-04-25T13:43:10.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 shutdown_connections 2024-04-25T13:43:10.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe36406f400 0x7fe364071910 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe384161070 0x7fe3840b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3840b4bd0 0x7fe384152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 --2- 172.21.15.80:0/3052380444 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe38406f5a0 0x7fe38415f6b0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 >> 172.21.15.80:0/3052380444 conn(0x7fe38406d080 msgr2=0x7fe384155470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:10.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.457+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 shutdown_connections 2024-04-25T13:43:10.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.461+0000 7fe389d84640 1 -- 172.21.15.80:0/3052380444 wait complete. 2024-04-25T13:43:10.481 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '52', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:43:10.481 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:43:10.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.629+0000 7fbc0caf9640 1 Processor -- start 2024-04-25T13:43:10.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.629+0000 7fbc0caf9640 1 -- start start 2024-04-25T13:43:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0caf9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080b5330 0x7fbc080d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0caf9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbc08161b60 0x7fbc08161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0caf9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc081624b0 con 0x7fbc080d0ff0 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0caf9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbc081629b0 con 0x7fbc08161b60 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0caf9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc080b5740 con 0x7fbc080b5330 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc05d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fbc080d0ff0 0x7fbc0806edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42498/0 (socket says 172.21.15.80:42498) 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc05d9b640 1 -- 172.21.15.80:0/2921969377 learned_addr learned my addr 172.21.15.80:0/2921969377 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0659c640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080b5330 0x7fbc080d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc06d9d640 1 --2- 172.21.15.80:0/2921969377 >> v2:172.21.15.87:3300/0 conn(0x7fbc08161b60 0x7fbc08161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 367895430 0 0) 0x7fbbfc001120 con 0x7fbc080d0ff0 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbf0003530 con 0x7fbc080d0ff0 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3186140640 0 0) 0x7fbbf0003530 con 0x7fbc080d0ff0 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080b5330 msgr2=0x7fbc080d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080b5330 0x7fbc080d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 >> v2:172.21.15.87:3300/0 conn(0x7fbc08161b60 msgr2=0x7fbc08161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 --2- 172.21.15.80:0/2921969377 >> v2:172.21.15.87:3300/0 conn(0x7fbc08161b60 0x7fbc08161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0806dd10 con 0x7fbc080d0ff0 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fbbfc001850 con 0x7fbc080d0ff0 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 >> v1:172.21.15.80:6789/0 conn(0x7fbc080d0ff0 legacy=0x7fbc0806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf00039c0 0x7fbbf0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080b5330 0x7fbbf0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbf0004d50 0x7fbbf0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc081629b0 con 0x7fbbf0004d50 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080b5740 con 0x7fbbf00039c0 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf000ba50 con 0x7fbc080b5330 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc06d9d640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbf0004d50 0x7fbbf0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0659c640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf00039c0 0x7fbbf0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc05d9b640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080b5330 0x7fbbf0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc06d9d640 1 -- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080b5330 msgr2=0x7fbbf0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc06d9d640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080b5330 0x7fbbf0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc06d9d640 1 -- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf00039c0 msgr2=0x7fbbf0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc06d9d640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf00039c0 0x7fbbf0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc06d9d640 1 -- 172.21.15.80:0/2921969377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf0003530 con 0x7fbbf0004d50 2024-04-25T13:43:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.633+0000 7fbc0659c640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbf00039c0 0x7fbbf0003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc06d9d640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbf0004d50 0x7fbbf0005160 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009f90 tx=0x7fbc0001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb528bc54380634 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc000b2070 con 0x7fbbf0004d50 2024-04-25T13:43:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc000a5c30 con 0x7fbbf0004d50 2024-04-25T13:43:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0559a640 1 -- 172.21.15.80:0/2921969377 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00009c10 con 0x7fbbf0004d50 2024-04-25T13:43:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2921969377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbf0004d50 msgr2=0x7fbbf0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0caf9640 1 --2- 172.21.15.80:0/2921969377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbbf0004d50 0x7fbbf0005160 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fbc00009f90 tx=0x7fbc0001cc20 comp rx=0 tx=0).stop 2024-04-25T13:43:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc06d9d640 1 -- 172.21.15.80:0/2921969377 reap_dead start 2024-04-25T13:43:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2921969377 shutdown_connections 2024-04-25T13:43:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2921969377 >> 172.21.15.80:0/2921969377 conn(0x7fbc080b4160 msgr2=0x7fbc080d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2921969377 shutdown_connections 2024-04-25T13:43:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.637+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2921969377 wait complete. 2024-04-25T13:43:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 Processor -- start 2024-04-25T13:43:10.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 -- start start 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080c7240 0x7fbc08145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc080cbf50 0x7fbc080cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080cc360 0x7fbc080cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0806e5b0 con 0x7fbc080cc360 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0806e2d0 con 0x7fbc080cbf50 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbc0806e890 con 0x7fbc080c7240 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080cc360 0x7fbc080cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080cc360 0x7fbc080cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50408/0 (socket says 172.21.15.80:50408) 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 -- 172.21.15.80:0/2090856658 learned_addr learned my addr 172.21.15.80:0/2090856658 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:10.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0659c640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080c7240 0x7fbc08145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc05d9b640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc080cbf50 0x7fbc080cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 -- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080c7240 msgr2=0x7fbc08145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080c7240 0x7fbc08145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 -- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc080cbf50 msgr2=0x7fbc080cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc080cbf50 0x7fbc080cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 -- 172.21.15.80:0/2090856658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc080c7820 con 0x7fbc080cc360 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc06d9d640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080cc360 0x7fbc080cdf80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b0040 tx=0x7fbc00009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbbf6ffd640 1 -- 172.21.15.80:0/2090856658 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc000bd070 con 0x7fbc080cc360 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbbf6ffd640 1 -- 172.21.15.80:0/2090856658 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc000a5a30 con 0x7fbc080cc360 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc080c7b00 con 0x7fbc080cc360 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbbf6ffd640 1 -- 172.21.15.80:0/2090856658 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc000b2040 con 0x7fbc080cc360 2024-04-25T13:43:10.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc080c80c0 con 0x7fbc080cc360 2024-04-25T13:43:10.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0806a3a0 con 0x7fbc080cc360 2024-04-25T13:43:10.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.641+0000 7fbbf6ffd640 1 -- 172.21.15.80:0/2090856658 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbc000c0070 con 0x7fbc080cc360 2024-04-25T13:43:10.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.645+0000 7fbbf6ffd640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbe806f430 0x7fbbe8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.645+0000 7fbbf6ffd640 1 -- 172.21.15.80:0/2090856658 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fbc001289b0 con 0x7fbc080cc360 2024-04-25T13:43:10.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.645+0000 7fbc0659c640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbe806f430 0x7fbbe8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.645+0000 7fbc0659c640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbe806f430 0x7fbbe8071940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbc080b1140 tx=0x7fbbf8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.649+0000 7fbbf6ffd640 1 -- 172.21.15.80:0/2090856658 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbc00016990 con 0x7fbc080cc360 2024-04-25T13:43:10.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.801+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbc0806fd00 con 0x7fbc080cc360 2024-04-25T13:43:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.801+0000 7fbbf6ffd640 1 -- 172.21.15.80:0/2090856658 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) ==== 93+0+10 (secure 0 0 0) 0x7fbc000f5630 con 0x7fbc080cc360 2024-04-25T13:43:10.805 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbe806f430 msgr2=0x7fbbe8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbe806f430 0x7fbbe8071940 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbc080b1140 tx=0x7fbbf8008040 comp rx=0 tx=0).stop 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080cc360 msgr2=0x7fbc080cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080cc360 0x7fbc080cdf80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fbc000b0040 tx=0x7fbc00009750 comp rx=0 tx=0).stop 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 shutdown_connections 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbbe806f430 0x7fbbe8071940 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbc080cc360 0x7fbc080cdf80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbc080cbf50 0x7fbc080cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 --2- 172.21.15.80:0/2090856658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbc080c7240 0x7fbc08145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 >> 172.21.15.80:0/2090856658 conn(0x7fbc080b4160 msgr2=0x7fbc080b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 shutdown_connections 2024-04-25T13:43:10.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.805+0000 7fbc0caf9640 1 -- 172.21.15.80:0/2090856658 wait complete. 2024-04-25T13:43:10.826 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:43:10.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc258df7640 1 Processor -- start 2024-04-25T13:43:10.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc258df7640 1 -- start start 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc258df7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc2540b0ed0 0x7fc25415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc258df7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc25406eba0 0x7fc2540ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc258df7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc25415cda0 con 0x7fc2540ce730 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc258df7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc254073710 con 0x7fc2540b0ed0 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc258df7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc2540739f0 con 0x7fc25406eba0 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc25259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc2540b0ed0 0x7fc25415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc24bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc2540ce730 0x7fc25406e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42514/0 (socket says 172.21.15.80:42514) 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc24bfff640 1 -- 172.21.15.80:0/2779687704 learned_addr learned my addr 172.21.15.80:0/2779687704 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc252d9d640 1 --2- 172.21.15.80:0/2779687704 >> v2:172.21.15.87:3300/0 conn(0x7fc25406eba0 0x7fc2540ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc25259c640 1 -- 172.21.15.80:0/2779687704 >> v2:172.21.15.87:3300/0 conn(0x7fc25406eba0 msgr2=0x7fc2540ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc25259c640 1 --2- 172.21.15.80:0/2779687704 >> v2:172.21.15.87:3300/0 conn(0x7fc25406eba0 0x7fc2540ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc25259c640 1 -- 172.21.15.80:0/2779687704 >> v1:172.21.15.80:6789/0 conn(0x7fc2540ce730 legacy=0x7fc25406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc25259c640 1 -- 172.21.15.80:0/2779687704 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc254073cd0 con 0x7fc2540b0ed0 2024-04-25T13:43:10.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.969+0000 7fc252d9d640 1 --2- 172.21.15.80:0/2779687704 >> v2:172.21.15.87:3300/0 conn(0x7fc25406eba0 0x7fc2540ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc25259c640 1 --2- 172.21.15.80:0/2779687704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc25415c860 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc2540b4950 tx=0x7fc244020e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39d18cc7719d86c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc251d9b640 1 -- 172.21.15.80:0/2779687704 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440b7070 con 0x7fc2540b0ed0 2024-04-25T13:43:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc251d9b640 1 -- 172.21.15.80:0/2779687704 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2440aab80 con 0x7fc2540b0ed0 2024-04-25T13:43:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc251d9b640 1 -- 172.21.15.80:0/2779687704 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440b2440 con 0x7fc2540b0ed0 2024-04-25T13:43:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 -- 172.21.15.80:0/2779687704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 msgr2=0x7fc25415c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 --2- 172.21.15.80:0/2779687704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc25415c860 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fc2540b4950 tx=0x7fc244020e80 comp rx=0 tx=0).stop 2024-04-25T13:43:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 -- 172.21.15.80:0/2779687704 shutdown_connections 2024-04-25T13:43:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 --2- 172.21.15.80:0/2779687704 >> v2:172.21.15.87:3300/0 conn(0x7fc25406eba0 0x7fc2540ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 --2- 172.21.15.80:0/2779687704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc25415c860 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 -- 172.21.15.80:0/2779687704 >> 172.21.15.80:0/2779687704 conn(0x7fc25406d080 msgr2=0x7fc2540b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 -- 172.21.15.80:0/2779687704 shutdown_connections 2024-04-25T13:43:10.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 -- 172.21.15.80:0/2779687704 wait complete. 2024-04-25T13:43:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 Processor -- start 2024-04-25T13:43:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.973+0000 7fc258df7640 1 -- start start 2024-04-25T13:43:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc254152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2540ce730 0x7fc25415f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2540b2ef0 0x7fc2540b3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc2540733a0 con 0x7fc2540b0ed0 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc2540b12e0 con 0x7fc2540b2ef0 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc25406dc00 con 0x7fc2540ce730 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc254152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc254152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50432/0 (socket says 172.21.15.80:50432) 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 -- 172.21.15.80:0/1821951587 learned_addr learned my addr 172.21.15.80:0/1821951587 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc252d9d640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2540b2ef0 0x7fc2540b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 -- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2540ce730 msgr2=0x7fc25415f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc24bfff640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2540ce730 0x7fc25415f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2540ce730 0x7fc25415f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 -- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2540b2ef0 msgr2=0x7fc2540b3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:10.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2540b2ef0 0x7fc2540b3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 -- 172.21.15.80:0/1821951587 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2540b3840 con 0x7fc2540b0ed0 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc252d9d640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2540b2ef0 0x7fc2540b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc25259c640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc254152500 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc2440b5040 tx=0x7fc24401dc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc24affd640 1 -- 172.21.15.80:0/1821951587 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440b7040 con 0x7fc2540b0ed0 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc24affd640 1 -- 172.21.15.80:0/1821951587 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2440c0070 con 0x7fc2540b0ed0 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25415e8b0 con 0x7fc2540b0ed0 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc24affd640 1 -- 172.21.15.80:0/1821951587 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2440bb890 con 0x7fc2540b0ed0 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2541f5e60 con 0x7fc2540b0ed0 2024-04-25T13:43:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc25406a3a0 con 0x7fc2540b0ed0 2024-04-25T13:43:10.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc24affd640 1 -- 172.21.15.80:0/1821951587 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc2440aaa40 con 0x7fc2540b0ed0 2024-04-25T13:43:10.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.977+0000 7fc24affd640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23406f400 0x7fc234071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:10.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.981+0000 7fc24affd640 1 -- 172.21.15.80:0/1821951587 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7fc24412ebd0 con 0x7fc2540b0ed0 2024-04-25T13:43:10.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.981+0000 7fc24bfff640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23406f400 0x7fc234071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:10.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.981+0000 7fc24bfff640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23406f400 0x7fc234071910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc254074660 tx=0x7fc23c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:10.985+0000 7fc24affd640 1 -- 172.21.15.80:0/1821951587 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2440ca020 con 0x7fc2540b0ed0 2024-04-25T13:43:11.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.145+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc25406f880 con 0x7fc2540b0ed0 2024-04-25T13:43:11.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.145+0000 7fc24affd640 1 -- 172.21.15.80:0/1821951587 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v59) ==== 84+0+65 (secure 0 0 0) 0x7fc2540b3840 con 0x7fc2540b0ed0 2024-04-25T13:43:11.148 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:11.148 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:43:11.149 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:43:11.149 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:43:11.149 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:43:11.149 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:11.149 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23406f400 msgr2=0x7fc234071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23406f400 0x7fc234071910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc254074660 tx=0x7fc23c009040 comp rx=0 tx=0).stop 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 msgr2=0x7fc254152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc254152500 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fc2440b5040 tx=0x7fc24401dc20 comp rx=0 tx=0).stop 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 shutdown_connections 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23406f400 0x7fc234071910 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc2540b2ef0 0x7fc2540b3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2540ce730 0x7fc25415f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 --2- 172.21.15.80:0/1821951587 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2540b0ed0 0x7fc254152500 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 >> 172.21.15.80:0/1821951587 conn(0x7fc25406d080 msgr2=0x7fc2541511f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:11.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 shutdown_connections 2024-04-25T13:43:11.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.149+0000 7fc258df7640 1 -- 172.21.15.80:0/1821951587 wait complete. 2024-04-25T13:43:11.170 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:43:11.170 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:43:11.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe43a4640 1 Processor -- start 2024-04-25T13:43:11.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe43a4640 1 -- start start 2024-04-25T13:43:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe43a4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0b48c0 0x7f6fdc161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe43a4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6fdc0d0840 0x7f6fdc0d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe43a4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6fdc0ce220 0x7f6fdc0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe43a4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6fdc0b0bc0 con 0x7f6fdc0ce220 2024-04-25T13:43:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe43a4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6fdc0d1190 con 0x7f6fdc0d0840 2024-04-25T13:43:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe43a4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc0b0e10 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe2941640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6fdc0ce220 0x7f6fdc0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe2941640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6fdc0ce220 0x7f6fdc0ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50184/0 (socket says 172.21.15.80:50184) 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe2941640 1 -- 172.21.15.80:0/2023894250 learned_addr learned my addr 172.21.15.80:0/2023894250 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe2140640 1 --2- 172.21.15.80:0/2023894250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0b48c0 0x7f6fdc161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.321+0000 7f6fe193f640 1 --2- 172.21.15.80:0/2023894250 >> v2:172.21.15.87:3300/0 conn(0x7f6fdc0d0840 0x7f6fdc0d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe2941640 1 -- 172.21.15.80:0/2023894250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0b48c0 msgr2=0x7f6fdc161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe2941640 1 --2- 172.21.15.80:0/2023894250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0b48c0 0x7f6fdc161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe2941640 1 -- 172.21.15.80:0/2023894250 >> v2:172.21.15.87:3300/0 conn(0x7f6fdc0d0840 msgr2=0x7f6fdc0d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe2941640 1 --2- 172.21.15.80:0/2023894250 >> v2:172.21.15.87:3300/0 conn(0x7f6fdc0d0840 0x7f6fdc0d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe2941640 1 -- 172.21.15.80:0/2023894250 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc0b10f0 con 0x7f6fdc0ce220 2024-04-25T13:43:11.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe2941640 1 --2- 172.21.15.80:0/2023894250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0ce220 0x7f6fdc0ce630 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc071aa0 tx=0x7f6fcc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbd660a77f7074c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:11.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe113e640 1 -- 172.21.15.80:0/2023894250 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0a55e0 con 0x7f6fdc0ce220 2024-04-25T13:43:11.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe113e640 1 -- 172.21.15.80:0/2023894250 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fcc0a5780 con 0x7f6fdc0ce220 2024-04-25T13:43:11.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe113e640 1 -- 172.21.15.80:0/2023894250 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0a5b50 con 0x7f6fdc0ce220 2024-04-25T13:43:11.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2023894250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0ce220 msgr2=0x7f6fdc0ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2023894250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0ce220 0x7f6fdc0ce630 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc071aa0 tx=0x7f6fcc01c960 comp rx=0 tx=0).stop 2024-04-25T13:43:11.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2023894250 shutdown_connections 2024-04-25T13:43:11.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2023894250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0ce220 0x7f6fdc0ce630 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2023894250 >> v2:172.21.15.87:3300/0 conn(0x7f6fdc0d0840 0x7f6fdc0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2023894250 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0b48c0 0x7f6fdc161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2023894250 >> 172.21.15.80:0/2023894250 conn(0x7f6fdc0b35c0 msgr2=0x7f6fdc0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:11.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2023894250 shutdown_connections 2024-04-25T13:43:11.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.325+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2023894250 wait complete. 2024-04-25T13:43:11.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 Processor -- start 2024-04-25T13:43:11.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 -- start start 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0b48c0 0x7f6fdc153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0ce220 0x7f6fdc158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fdc0d0840 0x7f6fdc147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc06fc30 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc15dd30 con 0x7f6fdc0d0840 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc1f8a10 con 0x7f6fdc0ce220 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0b48c0 0x7f6fdc153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0b48c0 0x7f6fdc153dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50192/0 (socket says 172.21.15.80:50192) 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 -- 172.21.15.80:0/2896342310 learned_addr learned my addr 172.21.15.80:0/2896342310 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2941640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fdc0d0840 0x7f6fdc147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe193f640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0ce220 0x7f6fdc158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 -- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0ce220 msgr2=0x7f6fdc158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0ce220 0x7f6fdc158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 -- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fdc0d0840 msgr2=0x7f6fdc147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fdc0d0840 0x7f6fdc147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 -- 172.21.15.80:0/2896342310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc147ca0 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe2140640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0b48c0 0x7f6fdc153dc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400d5d0 tx=0x7f6fd400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2896342310 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4004240 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2896342310 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fd40043e0 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc157980 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2896342310 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4003840 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.329+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc157f40 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.333+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc06a440 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.333+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2896342310 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6fd40039e0 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.333+0000 7f6fdaffd640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6fc806f400 0x7f6fc8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.333+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2896342310 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 4857+0+0 (secure 0 0 0) 0x7f6fd408f220 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.333+0000 7f6fe193f640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6fc806f400 0x7f6fc8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.333+0000 7f6fe193f640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6fc806f400 0x7f6fc8071910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc161740 tx=0x7f6fd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:11.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.337+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2896342310 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6fd405c4f0 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.501+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f6fdc0b54a0 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.749+0000 7f6fdaffd640 1 -- 172.21.15.80:0/2896342310 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v60) ==== 138+0+0 (secure 0 0 0) 0x7f6fd405bdf0 con 0x7f6fdc0b48c0 2024-04-25T13:43:11.752 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:43:11.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6fc806f400 msgr2=0x7f6fc8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6fc806f400 0x7f6fc8071910 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc161740 tx=0x7f6fd0008040 comp rx=0 tx=0).stop 2024-04-25T13:43:11.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0b48c0 msgr2=0x7f6fdc153dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0b48c0 0x7f6fdc153dc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400d5d0 tx=0x7f6fd400da90 comp rx=0 tx=0).stop 2024-04-25T13:43:11.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 shutdown_connections 2024-04-25T13:43:11.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6fc806f400 0x7f6fc8071910 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6fdc0d0840 0x7f6fdc147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6fdc0ce220 0x7f6fdc158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 --2- 172.21.15.80:0/2896342310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6fdc0b48c0 0x7f6fdc153dc0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 >> 172.21.15.80:0/2896342310 conn(0x7f6fdc0b35c0 msgr2=0x7f6fdc06ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:11.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 shutdown_connections 2024-04-25T13:43:11.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.753+0000 7f6fe43a4640 1 -- 172.21.15.80:0/2896342310 wait complete. 2024-04-25T13:43:11.788 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:43:11.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.949+0000 7f9fb15e2640 1 Processor -- start 2024-04-25T13:43:11.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.949+0000 7f9fb15e2640 1 -- start start 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.949+0000 7f9fb15e2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9fac161f90 0x7f9fac0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.949+0000 7f9fb15e2640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fac0d0d70 con 0x7f9fac15ec40 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.949+0000 7f9fb15e2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9fac0d1050 con 0x7f9fac0d0510 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.949+0000 7f9fb15e2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9fac06dca0 con 0x7f9fac161f90 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.949+0000 7f9fab7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9fac15ec40 0x7f9fac168cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35856/0 (socket says 172.21.15.80:35856) 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fab7fe640 1 -- 172.21.15.80:0/67405199 learned_addr learned my addr 172.21.15.80:0/67405199 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9faaffd640 1 --2- 172.21.15.80:0/67405199 >> v2:172.21.15.80:3300/0 conn(0x7f9fac161f90 0x7f9fac0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fa9ffb640 1 -- 172.21.15.80:0/67405199 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3811623150 0 0) 0x7f9fac0d0d70 con 0x7f9fac15ec40 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fa9ffb640 1 -- 172.21.15.80:0/67405199 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9f94003530 con 0x7f9fac15ec40 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9faaffd640 1 -- 172.21.15.80:0/67405199 >> v1:172.21.15.87:6789/0 conn(0x7f9fac0d0510 legacy=0x7f9fac15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9faaffd640 1 -- 172.21.15.80:0/67405199 >> v1:172.21.15.80:6789/0 conn(0x7f9fac15ec40 legacy=0x7f9fac168cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9faaffd640 1 -- 172.21.15.80:0/67405199 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac06df80 con 0x7f9fac161f90 2024-04-25T13:43:11.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9faaffd640 1 --2- 172.21.15.80:0/67405199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0cffd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0097c0 tx=0x7f9f9c01c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4432b8cd2b3b9ecf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:11.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fa9ffb640 1 -- 172.21.15.80:0/67405199 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0b0070 con 0x7f9fac161f90 2024-04-25T13:43:11.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fa9ffb640 1 -- 172.21.15.80:0/67405199 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f9c0ab600 con 0x7f9fac161f90 2024-04-25T13:43:11.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fa9ffb640 1 -- 172.21.15.80:0/67405199 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c0ab9d0 con 0x7f9fac161f90 2024-04-25T13:43:11.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fb15e2640 1 -- 172.21.15.80:0/67405199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 msgr2=0x7f9fac0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fb15e2640 1 --2- 172.21.15.80:0/67405199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0cffd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0097c0 tx=0x7f9f9c01c320 comp rx=0 tx=0).stop 2024-04-25T13:43:11.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fb15e2640 1 -- 172.21.15.80:0/67405199 shutdown_connections 2024-04-25T13:43:11.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fb15e2640 1 --2- 172.21.15.80:0/67405199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0cffd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fb15e2640 1 -- 172.21.15.80:0/67405199 >> 172.21.15.80:0/67405199 conn(0x7f9fac06d080 msgr2=0x7f9fac0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:11.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.953+0000 7f9fb15e2640 1 -- 172.21.15.80:0/67405199 shutdown_connections 2024-04-25T13:43:11.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 -- 172.21.15.80:0/67405199 wait complete. 2024-04-25T13:43:11.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 Processor -- start 2024-04-25T13:43:11.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 -- start start 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fac0d0510 0x7f9fac162e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fac15ec40 0x7f9fac15d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0ae920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac06f630 con 0x7f9fac161f90 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9fac06f4b0 con 0x7f9fac0d0510 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fb15e2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9fac0d0920 con 0x7f9fac15ec40 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fab7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0ae920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fab7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0ae920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50212/0 (socket says 172.21.15.80:50212) 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fab7fe640 1 -- 172.21.15.80:0/1273271065 learned_addr learned my addr 172.21.15.80:0/1273271065 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9faaffd640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fac0d0510 0x7f9fac162e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9faa7fc640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fac15ec40 0x7f9fac15d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fab7fe640 1 -- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fac15ec40 msgr2=0x7f9fac15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fab7fe640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fac15ec40 0x7f9fac15d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fab7fe640 1 -- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fac0d0510 msgr2=0x7f9fac162e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fab7fe640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fac0d0510 0x7f9fac162e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9fab7fe640 1 -- 172.21.15.80:0/1273271065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fac163350 con 0x7f9fac161f90 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.957+0000 7f9faaffd640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fac0d0510 0x7f9fac162e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9fab7fe640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0ae920 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400caa0 tx=0x7f9fa400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9f937fe640 1 -- 172.21.15.80:0/1273271065 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4017070 con 0x7f9fac161f90 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9f937fe640 1 -- 172.21.15.80:0/1273271065 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fa400ac00 con 0x7f9fac161f90 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9f937fe640 1 -- 172.21.15.80:0/1273271065 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4012400 con 0x7f9fac161f90 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fac0af770 con 0x7f9fac161f90 2024-04-25T13:43:11.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fac0afd30 con 0x7f9fac161f90 2024-04-25T13:43:11.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fac1635d0 con 0x7f9fac161f90 2024-04-25T13:43:11.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9f937fe640 1 -- 172.21.15.80:0/1273271065 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9fa401a070 con 0x7f9fac161f90 2024-04-25T13:43:11.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9f937fe640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:11.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9f937fe640 1 -- 172.21.15.80:0/1273271065 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5227+0+0 (secure 0 0 0) 0x7f9fa408f190 con 0x7f9fac161f90 2024-04-25T13:43:11.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.961+0000 7f9faaffd640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:11.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.965+0000 7f9faaffd640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f9806f400 0x7f9f98071910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ad000 tx=0x7f9f9c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:11.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:11.965+0000 7f9f937fe640 1 -- 172.21.15.80:0/1273271065 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9fa405bca0 con 0x7f9fac161f90 2024-04-25T13:43:12.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.129+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9fac06a3a0 con 0x7f9fac161f90 2024-04-25T13:43:12.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.761+0000 7f9f937fe640 1 -- 172.21.15.80:0/1273271065 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v61) ==== 170+0+0 (secure 0 0 0) 0x7f9fa405ffa0 con 0x7f9fac161f90 2024-04-25T13:43:12.765 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:43:12.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f9806f400 msgr2=0x7f9f98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:12.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f9806f400 0x7f9f98071910 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c0ad000 tx=0x7f9f9c0a6000 comp rx=0 tx=0).stop 2024-04-25T13:43:12.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 msgr2=0x7f9fac0ae920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:12.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0ae920 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9fa400caa0 tx=0x7f9fa400cf60 comp rx=0 tx=0).stop 2024-04-25T13:43:12.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 shutdown_connections 2024-04-25T13:43:12.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f9806f400 0x7f9f98071910 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9fac161f90 0x7f9fac0ae920 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9fac15ec40 0x7f9fac15d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 --2- 172.21.15.80:0/1273271065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9fac0d0510 0x7f9fac162e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 >> 172.21.15.80:0/1273271065 conn(0x7f9fac06d080 msgr2=0x7f9fac0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:12.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 shutdown_connections 2024-04-25T13:43:12.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.765+0000 7f9fb15e2640 1 -- 172.21.15.80:0/1273271065 wait complete. 2024-04-25T13:43:12.806 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:43:12.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.953+0000 7f6339d5c640 1 Processor -- start 2024-04-25T13:43:12.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.953+0000 7f6339d5c640 1 -- start start 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6339d5c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63340d2ca0 0x7f63340b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6339d5c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f63340b9ca0 0x7f63340ba0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6339d5c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63340b92a0 con 0x7f63340d3ed0 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6339d5c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f63340d4c10 con 0x7f63340b9ca0 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6339d5c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63340d4ef0 con 0x7f63340d2ca0 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f63337fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f63340d3ed0 0x7f63340d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35870/0 (socket says 172.21.15.80:35870) 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f63337fe640 1 -- 172.21.15.80:0/2764523857 learned_addr learned my addr 172.21.15.80:0/2764523857 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6333fff640 1 --2- 172.21.15.80:0/2764523857 >> v2:172.21.15.87:3300/0 conn(0x7f63340b9ca0 0x7f63340ba0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f632affd640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63340d2ca0 0x7f63340b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 374372676 0 0) 0x7f6324001120 con 0x7f63340d3ed0 2024-04-25T13:43:12.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6320003530 con 0x7f63340d3ed0 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3093651918 0 0) 0x7f6320003530 con 0x7f63340d3ed0 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63340d2ca0 msgr2=0x7f63340b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63340d2ca0 0x7f63340b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 >> v2:172.21.15.87:3300/0 conn(0x7f63340b9ca0 msgr2=0x7f63340ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 --2- 172.21.15.80:0/2764523857 >> v2:172.21.15.87:3300/0 conn(0x7f63340b9ca0 0x7f63340ba0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63340d51d0 con 0x7f63340d3ed0 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f6324001850 con 0x7f63340d3ed0 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 >> v1:172.21.15.80:6789/0 conn(0x7f63340d3ed0 legacy=0x7f63340d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63200039c0 0x7f6320003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6320004810 0x7f6320004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63340d2ca0 0x7f6320005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f63340d4c10 con 0x7f63200039c0 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f63340d4ef0 con 0x7f6320004810 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f632000bab0 con 0x7f63340d2ca0 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f63337fe640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63200039c0 0x7f6320003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f632affd640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6320004810 0x7f6320004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f63337fe640 1 -- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63340d2ca0 msgr2=0x7f6320005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f63337fe640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63340d2ca0 0x7f6320005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f63337fe640 1 -- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6320004810 msgr2=0x7f6320004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f63337fe640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6320004810 0x7f6320004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f63337fe640 1 -- 172.21.15.80:0/2764523857 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6320003530 con 0x7f63200039c0 2024-04-25T13:43:12.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.957+0000 7f632affd640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6320004810 0x7f6320004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:12.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f63337fe640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63200039c0 0x7f6320003e30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f632400c490 tx=0x7f632401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f163cadfd8c34c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632400c0e0 con 0x7f63200039c0 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63240a9020 con 0x7f63200039c0 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6332ffd640 1 -- 172.21.15.80:0/2764523857 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324005b20 con 0x7f63200039c0 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6339d5c640 1 -- 172.21.15.80:0/2764523857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63200039c0 msgr2=0x7f6320003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6339d5c640 1 --2- 172.21.15.80:0/2764523857 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63200039c0 0x7f6320003e30 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f632400c490 tx=0x7f632401cca0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6333fff640 1 -- 172.21.15.80:0/2764523857 reap_dead start 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6339d5c640 1 -- 172.21.15.80:0/2764523857 shutdown_connections 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6339d5c640 1 -- 172.21.15.80:0/2764523857 >> 172.21.15.80:0/2764523857 conn(0x7f63340737b0 msgr2=0x7f633406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6339d5c640 1 -- 172.21.15.80:0/2764523857 shutdown_connections 2024-04-25T13:43:12.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6339d5c640 1 -- 172.21.15.80:0/2764523857 wait complete. 2024-04-25T13:43:12.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6339d5c640 1 Processor -- start 2024-04-25T13:43:12.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.961+0000 7f6339d5c640 1 -- start start 2024-04-25T13:43:12.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63200039c0 0x7f6334162ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63341f9750 0x7f6334169ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63341f9b60 0x7f6334169ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6334160620 con 0x7f63341f9b60 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f63341607a0 con 0x7f63341f9750 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f633416b280 con 0x7f63200039c0 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63341f9b60 0x7f6334169ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63341f9b60 0x7f6334169ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50232/0 (socket says 172.21.15.80:50232) 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 -- 172.21.15.80:0/2995962410 learned_addr learned my addr 172.21.15.80:0/2995962410 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f63337fe640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63200039c0 0x7f6334162ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f632affd640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63341f9750 0x7f6334169ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 -- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63200039c0 msgr2=0x7f6334162ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63200039c0 0x7f6334162ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 -- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63341f9750 msgr2=0x7f6334169ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63341f9750 0x7f6334169ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 -- 172.21.15.80:0/2995962410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63340cfc40 con 0x7f63341f9b60 2024-04-25T13:43:12.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f63337fe640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63200039c0 0x7f6334162ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:12.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f632affd640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63341f9750 0x7f6334169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:12.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6333fff640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63341f9b60 0x7f6334169ff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f63340d4310 tx=0x7f632c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:12.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6330ff9640 1 -- 172.21.15.80:0/2995962410 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c00bb10 con 0x7f63341f9b60 2024-04-25T13:43:12.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6330ff9640 1 -- 172.21.15.80:0/2995962410 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f632c01a070 con 0x7f63341f9b60 2024-04-25T13:43:12.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63340cf3a0 con 0x7f63341f9b60 2024-04-25T13:43:12.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6330ff9640 1 -- 172.21.15.80:0/2995962410 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f632c014400 con 0x7f63341f9b60 2024-04-25T13:43:12.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63340cf960 con 0x7f63341f9b60 2024-04-25T13:43:12.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f633406a3a0 con 0x7f63341f9b60 2024-04-25T13:43:12.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.965+0000 7f6330ff9640 1 -- 172.21.15.80:0/2995962410 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f632c013070 con 0x7f63341f9b60 2024-04-25T13:43:12.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.969+0000 7f6330ff9640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f631406f400 0x7f6314071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:12.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.969+0000 7f6330ff9640 1 -- 172.21.15.80:0/2995962410 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5609+0+0 (secure 0 0 0) 0x7f632c08fc60 con 0x7f63341f9b60 2024-04-25T13:43:12.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.969+0000 7f63337fe640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f631406f400 0x7f6314071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:12.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.969+0000 7f63337fe640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f631406f400 0x7f6314071910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f63240aa000 tx=0x7f6324000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:12.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:12.973+0000 7f6330ff9640 1 -- 172.21.15.80:0/2995962410 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f632c05cc90 con 0x7f63341f9b60 2024-04-25T13:43:13.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.173+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f63340cf130 con 0x7f63341f9b60 2024-04-25T13:43:13.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.765+0000 7f6330ff9640 1 -- 172.21.15.80:0/2995962410 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) ==== 172+0+0 (secure 0 0 0) 0x7f632c05c590 con 0x7f63341f9b60 2024-04-25T13:43:13.768 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f631406f400 msgr2=0x7f6314071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f631406f400 0x7f6314071910 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f63240aa000 tx=0x7f6324000ff0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63341f9b60 msgr2=0x7f6334169ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63341f9b60 0x7f6334169ff0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f63340d4310 tx=0x7f632c00b480 comp rx=0 tx=0).stop 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 shutdown_connections 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f631406f400 0x7f6314071910 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63341f9b60 0x7f6334169ff0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63341f9750 0x7f6334169ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 --2- 172.21.15.80:0/2995962410 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63200039c0 0x7f6334162ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 >> 172.21.15.80:0/2995962410 conn(0x7f63340737b0 msgr2=0x7f633415dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 shutdown_connections 2024-04-25T13:43:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.769+0000 7f6339d5c640 1 -- 172.21.15.80:0/2995962410 wait complete. 2024-04-25T13:43:13.814 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:43:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.953+0000 7f8803a2e640 1 Processor -- start 2024-04-25T13:43:13.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.953+0000 7f8803a2e640 1 -- start start 2024-04-25T13:43:13.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8803a2e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f87fc06f5a0 0x7f87fc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8803a2e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87fc0733b0 0x7f87fc161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8803a2e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87fc06fcf0 con 0x7f87fc161070 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8803a2e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f87fc1619c0 con 0x7f87fc06f5a0 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8803a2e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87fc073f10 con 0x7f87fc0733b0 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8800fc9640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f87fc161070 0x7f87fc0b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35876/0 (socket says 172.21.15.80:35876) 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8800fc9640 1 -- 172.21.15.80:0/820288648 learned_addr learned my addr 172.21.15.80:0/820288648 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8801fcb640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87fc0733b0 0x7f87fc161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f88017ca640 1 --2- 172.21.15.80:0/820288648 >> v2:172.21.15.87:3300/0 conn(0x7f87fc06f5a0 0x7f87fc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2215043753 0 0) 0x7f87fc06fcf0 con 0x7f87fc161070 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87e8003530 con 0x7f87fc161070 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3058859402 0 0) 0x7f87e8003530 con 0x7f87fc161070 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87fc0733b0 msgr2=0x7f87fc161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87fc0733b0 0x7f87fc161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 >> v2:172.21.15.87:3300/0 conn(0x7f87fc06f5a0 msgr2=0x7f87fc160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:13.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 --2- 172.21.15.80:0/820288648 >> v2:172.21.15.87:3300/0 conn(0x7f87fc06f5a0 0x7f87fc160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc0b0c60 con 0x7f87fc161070 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f87f8001680 con 0x7f87fc161070 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8801fcb640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87fc0733b0 0x7f87fc161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 >> v1:172.21.15.80:6789/0 conn(0x7f87fc161070 legacy=0x7f87fc0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e80039f0 0x7f87e8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e8004810 0x7f87e8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87e8005160 0x7f87e800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc1619c0 con 0x7f87e8005160 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc073f10 con 0x7f87e8004810 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87e800c070 con 0x7f87e80039f0 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8801fcb640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87e8005160 0x7f87e800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8800fc9640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e8004810 0x7f87e8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f88017ca640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e80039f0 0x7f87e8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8801fcb640 1 -- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e80039f0 msgr2=0x7f87e8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:13.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8801fcb640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e80039f0 0x7f87e8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8801fcb640 1 -- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e8004810 msgr2=0x7f87e8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8801fcb640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87e8004810 0x7f87e8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.957+0000 7f8801fcb640 1 -- 172.21.15.80:0/820288648 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e8003530 con 0x7f87e8005160 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f8801fcb640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87e8005160 0x7f87e800b9d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f87fc073990 tx=0x7f87f401eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7598dfd1b8bae993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4009d60 con 0x7f87e8005160 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f40a7e10 con 0x7f87e8005160 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f87f3fff640 1 -- 172.21.15.80:0/820288648 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4003ba0 con 0x7f87e8005160 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f8803a2e640 1 -- 172.21.15.80:0/820288648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87e8005160 msgr2=0x7f87e800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f8803a2e640 1 --2- 172.21.15.80:0/820288648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87e8005160 0x7f87e800b9d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f87fc073990 tx=0x7f87f401eed0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f8801fcb640 1 -- 172.21.15.80:0/820288648 reap_dead start 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f8803a2e640 1 -- 172.21.15.80:0/820288648 shutdown_connections 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f8803a2e640 1 -- 172.21.15.80:0/820288648 >> 172.21.15.80:0/820288648 conn(0x7f87fc06d080 msgr2=0x7f87fc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:13.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f8803a2e640 1 -- 172.21.15.80:0/820288648 shutdown_connections 2024-04-25T13:43:13.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.961+0000 7f8803a2e640 1 -- 172.21.15.80:0/820288648 wait complete. 2024-04-25T13:43:13.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 Processor -- start 2024-04-25T13:43:13.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 -- start start 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8005160 0x7f87fc1502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87fc153f50 0x7f87fc158110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87fc150e60 0x7f87fc158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc06f9b0 con 0x7f87fc153f50 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87fc0b1220 con 0x7f87fc150e60 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87fc161480 con 0x7f87e8005160 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87fc153f50 0x7f87fc158110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87fc153f50 0x7f87fc158110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50260/0 (socket says 172.21.15.80:50260) 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 -- 172.21.15.80:0/3979952190 learned_addr learned my addr 172.21.15.80:0/3979952190 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:13.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f88017ca640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8005160 0x7f87fc1502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8801fcb640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87fc150e60 0x7f87fc158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 -- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8005160 msgr2=0x7f87fc1502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8005160 0x7f87fc1502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 -- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87fc150e60 msgr2=0x7f87fc158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87fc150e60 0x7f87fc158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 -- 172.21.15.80:0/3979952190 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87fc1575a0 con 0x7f87fc153f50 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8800fc9640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87fc153f50 0x7f87fc158110 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f87f80017f0 tx=0x7f87f8011de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f87f1ffb640 1 -- 172.21.15.80:0/3979952190 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8010040 con 0x7f87fc153f50 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f87f1ffb640 1 -- 172.21.15.80:0/3979952190 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f801a070 con 0x7f87fc153f50 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87fc151ac0 con 0x7f87fc153f50 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f87f1ffb640 1 -- 172.21.15.80:0/3979952190 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f800fd80 con 0x7f87fc153f50 2024-04-25T13:43:13.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87fc1570c0 con 0x7f87fc153f50 2024-04-25T13:43:13.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87fc1572f0 con 0x7f87fc153f50 2024-04-25T13:43:13.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.965+0000 7f87f1ffb640 1 -- 172.21.15.80:0/3979952190 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f87f801f460 con 0x7f87fc153f50 2024-04-25T13:43:13.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.969+0000 7f87f1ffb640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87e006f430 0x7f87e0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:13.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.969+0000 7f87f1ffb640 1 -- 172.21.15.80:0/3979952190 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f87f8092460 con 0x7f87fc153f50 2024-04-25T13:43:13.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.969+0000 7f88017ca640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87e006f430 0x7f87e0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:13.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.969+0000 7f88017ca640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87e006f430 0x7f87e0071940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0b6e70 tx=0x7f87ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:13.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:13.973+0000 7f87f1ffb640 1 -- 172.21.15.80:0/3979952190 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87f805f470 con 0x7f87fc153f50 2024-04-25T13:43:14.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.133+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87fc06a3a0 con 0x7f87fc153f50 2024-04-25T13:43:14.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.133+0000 7f87f1ffb640 1 -- 172.21.15.80:0/3979952190 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16606 (secure 0 0 0) 0x7f87f805ed70 con 0x7f87fc153f50 2024-04-25T13:43:14.138 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:14.138 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":62,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:43:13.179374+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:43:11.515229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T13:43:12.143130+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:43:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87e006f430 msgr2=0x7f87e0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87e006f430 0x7f87e0071940 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0b6e70 tx=0x7f87ec008040 comp rx=0 tx=0).stop 2024-04-25T13:43:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87fc153f50 msgr2=0x7f87fc158110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87fc153f50 0x7f87fc158110 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f87f80017f0 tx=0x7f87f8011de0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 shutdown_connections 2024-04-25T13:43:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87e006f430 0x7f87e0071940 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87fc150e60 0x7f87fc158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87fc153f50 0x7f87fc158110 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 --2- 172.21.15.80:0/3979952190 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87e8005160 0x7f87fc1502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 >> 172.21.15.80:0/3979952190 conn(0x7f87fc06d080 msgr2=0x7f87fc15c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 shutdown_connections 2024-04-25T13:43:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.137+0000 7f8803a2e640 1 -- 172.21.15.80:0/3979952190 wait complete. 2024-04-25T13:43:14.160 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:43:14.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.297+0000 7f0b05bde640 1 Processor -- start 2024-04-25T13:43:14.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.297+0000 7f0b05bde640 1 -- start start 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0b05bde640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0b0006e9d0 0x7f0b0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0b05bde640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b000cd5b0 con 0x7f0b0006f320 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0b05bde640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b000d1290 con 0x7f0b000727d0 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0b05bde640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0b000d1850 con 0x7f0b0006e9d0 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0af7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0b0006e9d0 0x7f0b0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0af7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0b0006e9d0 0x7f0b0006ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50270/0 (socket says 172.21.15.80:50270) 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0af7fff640 1 -- 172.21.15.80:0/2486020818 learned_addr learned my addr 172.21.15.80:0/2486020818 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0af7fff640 1 -- 172.21.15.80:0/2486020818 >> v1:172.21.15.87:6789/0 conn(0x7f0b000727d0 legacy=0x7f0b00072be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0af7fff640 1 -- 172.21.15.80:0/2486020818 >> v1:172.21.15.80:6789/0 conn(0x7f0b0006f320 legacy=0x7f0b0006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0afeffd640 1 -- 172.21.15.80:0/2486020818 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1873702931 0 0) 0x7f0b000cd5b0 con 0x7f0b0006f320 2024-04-25T13:43:14.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.301+0000 7f0af7fff640 1 -- 172.21.15.80:0/2486020818 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b000d1570 con 0x7f0b0006e9d0 2024-04-25T13:43:14.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0af7fff640 1 --2- 172.21.15.80:0/2486020818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0006ede0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800aa10 tx=0x7f0ae801d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962f28fd8b9d742 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:14.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0afeffd640 1 -- 172.21.15.80:0/2486020818 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae800b040 con 0x7f0b0006e9d0 2024-04-25T13:43:14.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0afeffd640 1 -- 172.21.15.80:0/2486020818 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ae80a7700 con 0x7f0b0006e9d0 2024-04-25T13:43:14.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0afeffd640 1 -- 172.21.15.80:0/2486020818 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ae80a7ad0 con 0x7f0b0006e9d0 2024-04-25T13:43:14.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 -- 172.21.15.80:0/2486020818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 msgr2=0x7f0b0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 --2- 172.21.15.80:0/2486020818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0006ede0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800aa10 tx=0x7f0ae801d8a0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 -- 172.21.15.80:0/2486020818 shutdown_connections 2024-04-25T13:43:14.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 --2- 172.21.15.80:0/2486020818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0006ede0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 -- 172.21.15.80:0/2486020818 >> 172.21.15.80:0/2486020818 conn(0x7f0b000d2170 msgr2=0x7f0b000d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:14.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 -- 172.21.15.80:0/2486020818 shutdown_connections 2024-04-25T13:43:14.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 -- 172.21.15.80:0/2486020818 wait complete. 2024-04-25T13:43:14.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 Processor -- start 2024-04-25T13:43:14.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.305+0000 7f0b05bde640 1 -- start start 2024-04-25T13:43:14.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0015efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b0006f320 0x7f0b0014e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b000727d0 0x7f0b001536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0b000741e0 con 0x7f0b0006e9d0 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0015cfd0 con 0x7f0b000727d0 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0b00074360 con 0x7f0b0006f320 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0015efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0015efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50272/0 (socket says 172.21.15.80:50272) 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 -- 172.21.15.80:0/3145001315 learned_addr learned my addr 172.21.15.80:0/3145001315 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0af7fff640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b0006f320 0x7f0b0014e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0afffff640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b000727d0 0x7f0b001536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 -- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b0006f320 msgr2=0x7f0b0014e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b0006f320 0x7f0b0014e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 -- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b000727d0 msgr2=0x7f0b001536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b000727d0 0x7f0b001536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 -- 172.21.15.80:0/3145001315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b00150e50 con 0x7f0b0006e9d0 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0afffff640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b000727d0 0x7f0b001536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0aff7fe640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0015efe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0af000c3c0 tx=0x7f0af000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0afcff9640 1 -- 172.21.15.80:0/3145001315 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0017070 con 0x7f0b0006e9d0 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0afcff9640 1 -- 172.21.15.80:0/3145001315 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0af00052d0 con 0x7f0b0006e9d0 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0afcff9640 1 -- 172.21.15.80:0/3145001315 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0003820 con 0x7f0b0006e9d0 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b00151130 con 0x7f0b0006e9d0 2024-04-25T13:43:14.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b001562b0 con 0x7f0b0006e9d0 2024-04-25T13:43:14.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0006a3a0 con 0x7f0b0006e9d0 2024-04-25T13:43:14.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.309+0000 7f0afcff9640 1 -- 172.21.15.80:0/3145001315 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0af001a070 con 0x7f0b0006e9d0 2024-04-25T13:43:14.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.313+0000 7f0afcff9640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ae006f430 0x7f0ae0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.313+0000 7f0afcff9640 1 -- 172.21.15.80:0/3145001315 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f0af008e5b0 con 0x7f0b0006e9d0 2024-04-25T13:43:14.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.313+0000 7f0af7fff640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ae006f430 0x7f0ae0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.313+0000 7f0af7fff640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ae006f430 0x7f0ae0071940 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80b0000 tx=0x7f0ae80a8000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:14.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.317+0000 7f0afcff9640 1 -- 172.21.15.80:0/3145001315 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0af0005480 con 0x7f0b0006e9d0 2024-04-25T13:43:14.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.469+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b00076500 con 0x7f0b0006e9d0 2024-04-25T13:43:14.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0afcff9640 1 -- 172.21.15.80:0/3145001315 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16606 (secure 0 0 0) 0x7f0af005b4a0 con 0x7f0b0006e9d0 2024-04-25T13:43:14.476 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:14.476 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":62,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:43:13.179374+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:43:11.515229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T13:43:12.143130+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:43:14.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ae006f430 msgr2=0x7f0ae0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ae006f430 0x7f0ae0071940 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0ae80b0000 tx=0x7f0ae80a8000 comp rx=0 tx=0).stop 2024-04-25T13:43:14.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 msgr2=0x7f0b0015efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0015efe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f0af000c3c0 tx=0x7f0af000c880 comp rx=0 tx=0).stop 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 shutdown_connections 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0ae006f430 0x7f0ae0071940 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b000727d0 0x7f0b001536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0b0006f320 0x7f0b0014e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 --2- 172.21.15.80:0/3145001315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0b0006e9d0 0x7f0b0015efe0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 >> 172.21.15.80:0/3145001315 conn(0x7f0b000d2170 msgr2=0x7f0b0006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 shutdown_connections 2024-04-25T13:43:14.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.473+0000 7f0b05bde640 1 -- 172.21.15.80:0/3145001315 wait complete. 2024-04-25T13:43:14.497 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:43:14.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f648208b640 1 Processor -- start 2024-04-25T13:43:14.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f648208b640 1 -- start start 2024-04-25T13:43:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f648208b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f647c06edb0 0x7f647c06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f648208b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f647c161b60 0x7f647c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f648208b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f647c1624b0 con 0x7f647c0b5330 2024-04-25T13:43:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f648208b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f647c06f700 con 0x7f647c06edb0 2024-04-25T13:43:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f648208b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f647c1629b0 con 0x7f647c161b60 2024-04-25T13:43:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f647c06edb0 0x7f647c06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647b7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f647c0b5330 0x7f647c0d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35902/0 (socket says 172.21.15.80:35902) 2024-04-25T13:43:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f647c06edb0 0x7f647c06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50288/0 (socket says 172.21.15.80:50288) 2024-04-25T13:43:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647b7fe640 1 -- 172.21.15.80:0/3805166557 learned_addr learned my addr 172.21.15.80:0/3805166557 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647bfff640 1 --2- 172.21.15.80:0/3805166557 >> v2:172.21.15.87:3300/0 conn(0x7f647c161b60 0x7f647c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647affd640 1 -- 172.21.15.80:0/3805166557 >> v2:172.21.15.87:3300/0 conn(0x7f647c161b60 msgr2=0x7f647c161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647affd640 1 --2- 172.21.15.80:0/3805166557 >> v2:172.21.15.87:3300/0 conn(0x7f647c161b60 0x7f647c161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647affd640 1 -- 172.21.15.80:0/3805166557 >> v1:172.21.15.80:6789/0 conn(0x7f647c0b5330 legacy=0x7f647c0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647affd640 1 -- 172.21.15.80:0/3805166557 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c0b5740 con 0x7f647c06edb0 2024-04-25T13:43:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.633+0000 7f647a7fc640 1 -- 172.21.15.80:0/3805166557 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 683703058 0 0) 0x7f646c0014a0 con 0x7f647c0b5330 2024-04-25T13:43:14.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f647affd640 1 --2- 172.21.15.80:0/3805166557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c06edb0 0x7f647c06f1c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f647c147cb0 tx=0x7f647001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e682b22f02f94d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:14.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f647a7fc640 1 -- 172.21.15.80:0/3805166557 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700b1070 con 0x7f647c06edb0 2024-04-25T13:43:14.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f647a7fc640 1 -- 172.21.15.80:0/3805166557 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64700ac620 con 0x7f647c06edb0 2024-04-25T13:43:14.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f647a7fc640 1 -- 172.21.15.80:0/3805166557 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700ac9f0 con 0x7f647c06edb0 2024-04-25T13:43:14.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 -- 172.21.15.80:0/3805166557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c06edb0 msgr2=0x7f647c06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 --2- 172.21.15.80:0/3805166557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c06edb0 0x7f647c06f1c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f647c147cb0 tx=0x7f647001c3c0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 -- 172.21.15.80:0/3805166557 shutdown_connections 2024-04-25T13:43:14.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 --2- 172.21.15.80:0/3805166557 >> v2:172.21.15.87:3300/0 conn(0x7f647c161b60 0x7f647c161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 --2- 172.21.15.80:0/3805166557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c06edb0 0x7f647c06f1c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 -- 172.21.15.80:0/3805166557 >> 172.21.15.80:0/3805166557 conn(0x7f647c0b4160 msgr2=0x7f647c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:14.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 -- 172.21.15.80:0/3805166557 shutdown_connections 2024-04-25T13:43:14.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 -- 172.21.15.80:0/3805166557 wait complete. 2024-04-25T13:43:14.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 Processor -- start 2024-04-25T13:43:14.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.637+0000 7f648208b640 1 -- start start 2024-04-25T13:43:14.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647c06edb0 0x7f647c15a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c0b5330 0x7f647c14dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f647c161b60 0x7f647c150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f647c06e890 con 0x7f647c0b5330 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f647c06e5b0 con 0x7f647c06edb0 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f647c0b0cd0 con 0x7f647c161b60 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c0b5330 0x7f647c14dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c0b5330 0x7f647c14dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50290/0 (socket says 172.21.15.80:50290) 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 -- 172.21.15.80:0/3713591421 learned_addr learned my addr 172.21.15.80:0/3713591421 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647b7fe640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647c06edb0 0x7f647c15a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647bfff640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f647c161b60 0x7f647c150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 -- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f647c161b60 msgr2=0x7f647c150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f647c161b60 0x7f647c150300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 -- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647c06edb0 msgr2=0x7f647c15a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647c06edb0 0x7f647c15a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 -- 172.21.15.80:0/3713591421 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c15aa60 con 0x7f647c0b5330 2024-04-25T13:43:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f647affd640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c0b5330 0x7f647c14dc80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f64700af040 tx=0x7f6470019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f6463fff640 1 -- 172.21.15.80:0/3713591421 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700b1040 con 0x7f647c0b5330 2024-04-25T13:43:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f6463fff640 1 -- 172.21.15.80:0/3713591421 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64700bb070 con 0x7f647c0b5330 2024-04-25T13:43:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c15ad40 con 0x7f647c0b5330 2024-04-25T13:43:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f6463fff640 1 -- 172.21.15.80:0/3713591421 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700b5870 con 0x7f647c0b5330 2024-04-25T13:43:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c15b300 con 0x7f647c0b5330 2024-04-25T13:43:14.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.641+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f647c15a7d0 con 0x7f647c0b5330 2024-04-25T13:43:14.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.645+0000 7f6463fff640 1 -- 172.21.15.80:0/3713591421 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f64700ac4e0 con 0x7f647c0b5330 2024-04-25T13:43:14.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.645+0000 7f6463fff640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:14.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.645+0000 7f6463fff640 1 -- 172.21.15.80:0/3713591421 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5709+0+0 (secure 0 0 0) 0x7f6470129d10 con 0x7f647c0b5330 2024-04-25T13:43:14.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.645+0000 7f647b7fe640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.645+0000 7f647b7fe640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f646c0012f0 tx=0x7f646c005ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:14.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.649+0000 7f6463fff640 1 -- 172.21.15.80:0/3713591421 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6470016070 con 0x7f647c0b5330 2024-04-25T13:43:14.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:14.833+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f647c0d17f0 con 0x7f647c0b5330 2024-04-25T13:43:15.744 INFO:tasks.ceph.mds.a.smithi080.stderr:2024-04-25T13:43:15.745+0000 7fd529f3a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:43:15.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.825+0000 7f6463fff640 1 -- 172.21.15.80:0/3713591421 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7f64700f6be0 con 0x7f647c0b5330 2024-04-25T13:43:15.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 msgr2=0x7f6468071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:15.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f646c0012f0 tx=0x7f646c005ec0 comp rx=0 tx=0).stop 2024-04-25T13:43:15.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c0b5330 msgr2=0x7f647c14dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:15.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c0b5330 0x7f647c14dc80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f64700af040 tx=0x7f6470019160 comp rx=0 tx=0).stop 2024-04-25T13:43:15.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 shutdown_connections 2024-04-25T13:43:15.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:15.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f647c161b60 0x7f647c150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:15.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f647c0b5330 0x7f647c14dc80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:15.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 --2- 172.21.15.80:0/3713591421 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647c06edb0 0x7f647c15a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:15.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 >> 172.21.15.80:0/3713591421 conn(0x7f647c0b4160 msgr2=0x7f647c160d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:15.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 shutdown_connections 2024-04-25T13:43:15.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:15.829+0000 7f648208b640 1 -- 172.21.15.80:0/3713591421 wait complete. 2024-04-25T13:43:15.857 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:43:15.857 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:43:16.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da739a640 1 Processor -- start 2024-04-25T13:43:16.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da739a640 1 -- start start 2024-04-25T13:43:16.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da739a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6da0160290 0x7f6da00b1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:16.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da739a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da0068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:16.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da739a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6da0068ad0 0x7f6da006f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:16.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da739a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6da01606a0 con 0x7f6da0160290 2024-04-25T13:43:16.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da739a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6da00b0eb0 con 0x7f6da0068ad0 2024-04-25T13:43:16.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da739a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6da0068ee0 con 0x7f6da00b1880 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5136640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6da0160290 0x7f6da00b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5136640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6da0160290 0x7f6da00b1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50294/0 (socket says 172.21.15.80:50294) 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5136640 1 -- 172.21.15.80:0/3499147389 learned_addr learned my addr 172.21.15.80:0/3499147389 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5937640 1 --2- 172.21.15.80:0/3499147389 >> v2:172.21.15.87:3300/0 conn(0x7f6da0068ad0 0x7f6da006f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da4935640 1 --2- 172.21.15.80:0/3499147389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da0068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5136640 1 -- 172.21.15.80:0/3499147389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 msgr2=0x7f6da0068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5136640 1 --2- 172.21.15.80:0/3499147389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da0068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5136640 1 -- 172.21.15.80:0/3499147389 >> v2:172.21.15.87:3300/0 conn(0x7f6da0068ad0 msgr2=0x7f6da006f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5136640 1 --2- 172.21.15.80:0/3499147389 >> v2:172.21.15.87:3300/0 conn(0x7f6da0068ad0 0x7f6da006f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5136640 1 -- 172.21.15.80:0/3499147389 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da00b6910 con 0x7f6da0160290 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da5937640 1 --2- 172.21.15.80:0/3499147389 >> v2:172.21.15.87:3300/0 conn(0x7f6da0068ad0 0x7f6da006f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:16.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.009+0000 7f6da4935640 1 --2- 172.21.15.80:0/3499147389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da0068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:16.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da5136640 1 --2- 172.21.15.80:0/3499147389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0160290 0x7f6da00b1340 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6da00afd00 tx=0x7f6d9c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ac97e4c19fd618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:16.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6d97fff640 1 -- 172.21.15.80:0/3499147389 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0aa430 con 0x7f6da0160290 2024-04-25T13:43:16.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6d97fff640 1 -- 172.21.15.80:0/3499147389 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d9c0af070 con 0x7f6da0160290 2024-04-25T13:43:16.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6d97fff640 1 -- 172.21.15.80:0/3499147389 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0aa7f0 con 0x7f6da0160290 2024-04-25T13:43:16.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 -- 172.21.15.80:0/3499147389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0160290 msgr2=0x7f6da00b1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:16.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 --2- 172.21.15.80:0/3499147389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0160290 0x7f6da00b1340 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6da00afd00 tx=0x7f6d9c01c4b0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 -- 172.21.15.80:0/3499147389 shutdown_connections 2024-04-25T13:43:16.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 --2- 172.21.15.80:0/3499147389 >> v2:172.21.15.87:3300/0 conn(0x7f6da0068ad0 0x7f6da006f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 --2- 172.21.15.80:0/3499147389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da0068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 --2- 172.21.15.80:0/3499147389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0160290 0x7f6da00b1340 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 -- 172.21.15.80:0/3499147389 >> 172.21.15.80:0/3499147389 conn(0x7f6da00736a0 msgr2=0x7f6da0073ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:16.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 -- 172.21.15.80:0/3499147389 shutdown_connections 2024-04-25T13:43:16.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.013+0000 7f6da739a640 1 -- 172.21.15.80:0/3499147389 wait complete. 2024-04-25T13:43:16.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 Processor -- start 2024-04-25T13:43:16.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 -- start start 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0068ad0 0x7f6da015e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da015cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da0160290 0x7f6da01507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6da00b6480 con 0x7f6da0068ad0 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6da00b0550 con 0x7f6da0160290 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6da00b6600 con 0x7f6da00b1880 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0068ad0 0x7f6da015e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0068ad0 0x7f6da015e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50308/0 (socket says 172.21.15.80:50308) 2024-04-25T13:43:16.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 -- 172.21.15.80:0/2016861742 learned_addr learned my addr 172.21.15.80:0/2016861742 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da4935640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da015cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 -- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 msgr2=0x7f6da015cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da015cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 -- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da0160290 msgr2=0x7f6da01507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5937640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da0160290 0x7f6da01507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da0160290 0x7f6da01507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 -- 172.21.15.80:0/2016861742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6da015d1e0 con 0x7f6da0068ad0 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da5136640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0068ad0 0x7f6da015e700 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c01cd50 tx=0x7f6d9c01acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6d95ffb640 1 -- 172.21.15.80:0/2016861742 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0b3800 con 0x7f6da0068ad0 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6d95ffb640 1 -- 172.21.15.80:0/2016861742 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d9c0b39c0 con 0x7f6da0068ad0 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6da015d660 con 0x7f6da0068ad0 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6d95ffb640 1 -- 172.21.15.80:0/2016861742 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c0b3db0 con 0x7f6da0068ad0 2024-04-25T13:43:16.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.017+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6da015dc20 con 0x7f6da0068ad0 2024-04-25T13:43:16.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.021+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6da006a3a0 con 0x7f6da0068ad0 2024-04-25T13:43:16.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.021+0000 7f6d95ffb640 1 -- 172.21.15.80:0/2016861742 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6d9c0bc390 con 0x7f6da0068ad0 2024-04-25T13:43:16.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.021+0000 7f6d95ffb640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d8406f430 0x7f6d84071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:16.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.021+0000 7f6d95ffb640 1 -- 172.21.15.80:0/2016861742 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f6d9c129460 con 0x7f6da0068ad0 2024-04-25T13:43:16.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.021+0000 7f6da4935640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d8406f430 0x7f6d84071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:16.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.021+0000 7f6da4935640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d8406f430 0x7f6d84071940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d90006fd0 tx=0x7f6d90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:16.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.025+0000 7f6d95ffb640 1 -- 172.21.15.80:0/2016861742 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d9c0c3060 con 0x7f6da0068ad0 2024-04-25T13:43:16.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.225+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f6da0070210 con 0x7f6da0068ad0 2024-04-25T13:43:16.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.837+0000 7f6d95ffb640 1 -- 172.21.15.80:0/2016861742 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) ==== 122+0+0 (secure 0 0 0) 0x7f6d9c0160d0 con 0x7f6da0068ad0 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d8406f430 msgr2=0x7f6d84071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d8406f430 0x7f6d84071940 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d90006fd0 tx=0x7f6d90008040 comp rx=0 tx=0).stop 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0068ad0 msgr2=0x7f6da015e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0068ad0 0x7f6da015e700 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c01cd50 tx=0x7f6d9c01acf0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 shutdown_connections 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d8406f430 0x7f6d84071940 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6da0160290 0x7f6da01507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6da00b1880 0x7f6da015cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 --2- 172.21.15.80:0/2016861742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6da0068ad0 0x7f6da015e700 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 >> 172.21.15.80:0/2016861742 conn(0x7f6da00736a0 msgr2=0x7f6da0156490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 shutdown_connections 2024-04-25T13:43:16.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:16.841+0000 7f6da739a640 1 -- 172.21.15.80:0/2016861742 wait complete. 2024-04-25T13:43:16.867 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:43:16.867 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:43:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7993e1a640 1 Processor -- start 2024-04-25T13:43:17.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7993e1a640 1 -- start start 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7993e1a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f798c0b0ed0 0x7f798c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7993e1a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c0ce730 0x7f798c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7993e1a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f798c06e490 0x7f798c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7993e1a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f798c06ede0 con 0x7f798c06e490 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7993e1a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f798c15cda0 con 0x7f798c0b0ed0 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7993e1a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f798c073710 con 0x7f798c0ce730 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79923b7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f798c06e490 0x7f798c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79923b7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f798c06e490 0x7f798c06e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50318/0 (socket says 172.21.15.80:50318) 2024-04-25T13:43:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79923b7640 1 -- 172.21.15.80:0/1918847772 learned_addr learned my addr 172.21.15.80:0/1918847772 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f7991bb6640 1 --2- 172.21.15.80:0/1918847772 >> v2:172.21.15.87:3300/0 conn(0x7f798c0b0ed0 0x7f798c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79913b5640 1 --2- 172.21.15.80:0/1918847772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c0ce730 0x7f798c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79923b7640 1 -- 172.21.15.80:0/1918847772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c0ce730 msgr2=0x7f798c0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79923b7640 1 --2- 172.21.15.80:0/1918847772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c0ce730 0x7f798c0ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79923b7640 1 -- 172.21.15.80:0/1918847772 >> v2:172.21.15.87:3300/0 conn(0x7f798c0b0ed0 msgr2=0x7f798c15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79923b7640 1 --2- 172.21.15.80:0/1918847772 >> v2:172.21.15.87:3300/0 conn(0x7f798c0b0ed0 0x7f798c15c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.021+0000 7f79923b7640 1 -- 172.21.15.80:0/1918847772 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c0739f0 con 0x7f798c06e490 2024-04-25T13:43:17.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f79923b7640 1 --2- 172.21.15.80:0/1918847772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c06e490 0x7f798c06e8a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f798c071650 tx=0x7f798401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947351a9ec23454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7990bb4640 1 -- 172.21.15.80:0/1918847772 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79840b4070 con 0x7f798c06e490 2024-04-25T13:43:17.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7990bb4640 1 -- 172.21.15.80:0/1918847772 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79840af400 con 0x7f798c06e490 2024-04-25T13:43:17.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7990bb4640 1 -- 172.21.15.80:0/1918847772 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79840af7d0 con 0x7f798c06e490 2024-04-25T13:43:17.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 -- 172.21.15.80:0/1918847772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c06e490 msgr2=0x7f798c06e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1918847772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c06e490 0x7f798c06e8a0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f798c071650 tx=0x7f798401f1e0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 -- 172.21.15.80:0/1918847772 shutdown_connections 2024-04-25T13:43:17.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1918847772 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c06e490 0x7f798c06e8a0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1918847772 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c0ce730 0x7f798c0ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1918847772 >> v2:172.21.15.87:3300/0 conn(0x7f798c0b0ed0 0x7f798c15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 -- 172.21.15.80:0/1918847772 >> 172.21.15.80:0/1918847772 conn(0x7f798c06d080 msgr2=0x7f798c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:17.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 -- 172.21.15.80:0/1918847772 shutdown_connections 2024-04-25T13:43:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 -- 172.21.15.80:0/1918847772 wait complete. 2024-04-25T13:43:17.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 Processor -- start 2024-04-25T13:43:17.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.025+0000 7f7993e1a640 1 -- start start 2024-04-25T13:43:17.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c06e490 0x7f798c13ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c0b0ed0 0x7f798c1440a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f798c0ce730 0x7f798c0b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f798c0b12e0 con 0x7f798c0b0ed0 2024-04-25T13:43:17.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f798c073fb0 con 0x7f798c0ce730 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f798c0733a0 con 0x7f798c06e490 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c0b0ed0 0x7f798c1440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c0b0ed0 0x7f798c1440a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50328/0 (socket says 172.21.15.80:50328) 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 -- 172.21.15.80:0/1942687257 learned_addr learned my addr 172.21.15.80:0/1942687257 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7991bb6640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c06e490 0x7f798c13ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79923b7640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f798c0ce730 0x7f798c0b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 -- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c06e490 msgr2=0x7f798c13ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c06e490 0x7f798c13ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 -- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f798c0ce730 msgr2=0x7f798c0b17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f798c0ce730 0x7f798c0b17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 -- 172.21.15.80:0/1942687257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798c15ea40 con 0x7f798c0b0ed0 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79913b5640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c0b0ed0 0x7f798c1440a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f797c00d620 tx=0x7f797c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79827fc640 1 -- 172.21.15.80:0/1942687257 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c00b620 con 0x7f798c0b0ed0 2024-04-25T13:43:17.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79827fc640 1 -- 172.21.15.80:0/1942687257 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f797c019070 con 0x7f798c0b0ed0 2024-04-25T13:43:17.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798c15ed20 con 0x7f798c0b0ed0 2024-04-25T13:43:17.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79827fc640 1 -- 172.21.15.80:0/1942687257 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c00b9e0 con 0x7f798c0b0ed0 2024-04-25T13:43:17.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798c15f2e0 con 0x7f798c0b0ed0 2024-04-25T13:43:17.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f798c06a3a0 con 0x7f798c0b0ed0 2024-04-25T13:43:17.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.029+0000 7f79827fc640 1 -- 172.21.15.80:0/1942687257 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f797c00bb80 con 0x7f798c0b0ed0 2024-04-25T13:43:17.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.033+0000 7f79827fc640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f797006f400 0x7f7970071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.033+0000 7f79827fc640 1 -- 172.21.15.80:0/1942687257 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f797c0900c0 con 0x7f798c0b0ed0 2024-04-25T13:43:17.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.033+0000 7f7991bb6640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f797006f400 0x7f7970071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.033+0000 7f7991bb6640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f797006f400 0x7f7970071910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f798800b990 tx=0x7f7988008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.037+0000 7f79827fc640 1 -- 172.21.15.80:0/1942687257 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f797c05d1b0 con 0x7f798c0b0ed0 2024-04-25T13:43:17.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.225+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f798c13f850 con 0x7f798c0b0ed0 2024-04-25T13:43:17.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f79827fc640 1 -- 172.21.15.80:0/1942687257 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7f797c05cab0 con 0x7f798c0b0ed0 2024-04-25T13:43:17.231 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:17.231 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1185745761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1185745761},{"type":"v1","addr":"172.21.15.80:6835","nonce":1185745761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2524166918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2524166918},{"type":"v1","addr":"172.21.15.87:6835","nonce":2524166918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3637133191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3637133191},{"type":"v1","addr":"172.21.15.87:6837","nonce":3637133191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:43:13.179327+0000","modified":"2024-04-25T13:43:16.230820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6834/1437895704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1437895704},{"type":"v1","addr":"172.21.15.80:6834","nonce":1437895704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6195,"qdb_cluster":[6195]},"id":4}]} 2024-04-25T13:43:17.231 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 33 2024-04-25T13:43:17.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f797006f400 msgr2=0x7f7970071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f797006f400 0x7f7970071910 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f798800b990 tx=0x7f7988008040 comp rx=0 tx=0).stop 2024-04-25T13:43:17.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c0b0ed0 msgr2=0x7f798c1440a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c0b0ed0 0x7f798c1440a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f797c00d620 tx=0x7f797c00dae0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 shutdown_connections 2024-04-25T13:43:17.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f797006f400 0x7f7970071910 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f798c0ce730 0x7f798c0b17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f798c0b0ed0 0x7f798c1440a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 --2- 172.21.15.80:0/1942687257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f798c06e490 0x7f798c13ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 >> 172.21.15.80:0/1942687257 conn(0x7f798c06d080 msgr2=0x7f798c152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:17.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 shutdown_connections 2024-04-25T13:43:17.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.229+0000 7f7993e1a640 1 -- 172.21.15.80:0/1942687257 wait complete. 2024-04-25T13:43:17.252 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:43:17.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1310e57640 1 Processor -- start 2024-04-25T13:43:17.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1310e57640 1 -- start start 2024-04-25T13:43:17.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1310e57640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f130c161f90 0x7f130c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1310e57640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f130c0d0510 0x7f130c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1310e57640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f130c0d0d70 con 0x7f130c15ea70 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1310e57640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f130c0d1050 con 0x7f130c0d0510 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1310e57640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f130c06dca0 con 0x7f130c161f90 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1309d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f130c0d0510 0x7f130c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1309d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f130c0d0510 0x7f130c15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50340/0 (socket says 172.21.15.80:50340) 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1309d9b640 1 -- 172.21.15.80:0/479718276 learned_addr learned my addr 172.21.15.80:0/479718276 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f130a59c640 1 --2- 172.21.15.80:0/479718276 >> v2:172.21.15.87:3300/0 conn(0x7f130c161f90 0x7f130c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1309d9b640 1 -- 172.21.15.80:0/479718276 >> v2:172.21.15.87:3300/0 conn(0x7f130c161f90 msgr2=0x7f130c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1309d9b640 1 --2- 172.21.15.80:0/479718276 >> v2:172.21.15.87:3300/0 conn(0x7f130c161f90 0x7f130c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1309d9b640 1 -- 172.21.15.80:0/479718276 >> v1:172.21.15.87:6789/0 conn(0x7f130c15ea70 legacy=0x7f130c15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:17.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.393+0000 7f1309d9b640 1 -- 172.21.15.80:0/479718276 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c06df80 con 0x7f130c0d0510 2024-04-25T13:43:17.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1309d9b640 1 --2- 172.21.15.80:0/479718276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c0d0510 0x7f130c15e530 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f130c0d17b0 tx=0x7f130001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b97c44c0358dc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f130959a640 1 -- 172.21.15.80:0/479718276 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13000b0070 con 0x7f130c0d0510 2024-04-25T13:43:17.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f130959a640 1 -- 172.21.15.80:0/479718276 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13000ab600 con 0x7f130c0d0510 2024-04-25T13:43:17.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f130959a640 1 -- 172.21.15.80:0/479718276 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13000ab9d0 con 0x7f130c0d0510 2024-04-25T13:43:17.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1310e57640 1 -- 172.21.15.80:0/479718276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c0d0510 msgr2=0x7f130c15e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1310e57640 1 --2- 172.21.15.80:0/479718276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c0d0510 0x7f130c15e530 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f130c0d17b0 tx=0x7f130001c3f0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1310e57640 1 -- 172.21.15.80:0/479718276 shutdown_connections 2024-04-25T13:43:17.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1310e57640 1 --2- 172.21.15.80:0/479718276 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c0d0510 0x7f130c15e530 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1310e57640 1 --2- 172.21.15.80:0/479718276 >> v2:172.21.15.87:3300/0 conn(0x7f130c161f90 0x7f130c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1310e57640 1 -- 172.21.15.80:0/479718276 >> 172.21.15.80:0/479718276 conn(0x7f130c06d080 msgr2=0x7f130c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:17.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1310e57640 1 -- 172.21.15.80:0/479718276 shutdown_connections 2024-04-25T13:43:17.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.397+0000 7f1310e57640 1 -- 172.21.15.80:0/479718276 wait complete. 2024-04-25T13:43:17.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 Processor -- start 2024-04-25T13:43:17.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 -- start start 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f130c0d0510 0x7f130c146b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c15ea70 0x7f130c1405e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f130c161f90 0x7f130c071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f130c06f630 con 0x7f130c15ea70 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f130c06f4b0 con 0x7f130c0d0510 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f130c0d0920 con 0x7f130c161f90 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c15ea70 0x7f130c1405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c15ea70 0x7f130c1405e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50356/0 (socket says 172.21.15.80:50356) 2024-04-25T13:43:17.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 -- 172.21.15.80:0/2026736343 learned_addr learned my addr 172.21.15.80:0/2026736343 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f130a59c640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f130c0d0510 0x7f130c146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 -- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f130c161f90 msgr2=0x7f130c071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f130ad9d640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f130c161f90 0x7f130c071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f130c161f90 0x7f130c071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 -- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f130c0d0510 msgr2=0x7f130c146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f130c0d0510 0x7f130c146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 -- 172.21.15.80:0/2026736343 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f130c145ad0 con 0x7f130c15ea70 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f130a59c640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f130c0d0510 0x7f130c146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1309d9b640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c15ea70 0x7f130c1405e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f13000ae040 tx=0x7f130001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f12faffd640 1 -- 172.21.15.80:0/2026736343 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13000ba020 con 0x7f130c15ea70 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f12faffd640 1 -- 172.21.15.80:0/2026736343 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13000b6420 con 0x7f130c15ea70 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f130c145db0 con 0x7f130c15ea70 2024-04-25T13:43:17.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f12faffd640 1 -- 172.21.15.80:0/2026736343 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13000b0040 con 0x7f130c15ea70 2024-04-25T13:43:17.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f130c146370 con 0x7f130c15ea70 2024-04-25T13:43:17.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.401+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f130c06a3a0 con 0x7f130c15ea70 2024-04-25T13:43:17.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.405+0000 7f12faffd640 1 -- 172.21.15.80:0/2026736343 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13000bd020 con 0x7f130c15ea70 2024-04-25T13:43:17.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.405+0000 7f12faffd640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f12ec06f430 0x7f12ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.405+0000 7f12faffd640 1 -- 172.21.15.80:0/2026736343 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f13001292e0 con 0x7f130c15ea70 2024-04-25T13:43:17.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.405+0000 7f130a59c640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f12ec06f430 0x7f12ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.405+0000 7f130a59c640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f12ec06f430 0x7f12ec071940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f130c06fa20 tx=0x7f12fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.409+0000 7f12faffd640 1 -- 172.21.15.80:0/2026736343 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13000c4030 con 0x7f130c15ea70 2024-04-25T13:43:17.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.565+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f130c161740 con 0x7f130c15ea70 2024-04-25T13:43:17.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f12faffd640 1 -- 172.21.15.80:0/2026736343 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) ==== 74+0+16593 (secure 0 0 0) 0x7f13000160d0 con 0x7f130c15ea70 2024-04-25T13:43:17.571 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:17.572 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":63,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:43:14.769412+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"52","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:43:11.515229+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T13:43:12.143130+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f12ec06f430 msgr2=0x7f12ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f12ec06f430 0x7f12ec071940 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f130c06fa20 tx=0x7f12fc008040 comp rx=0 tx=0).stop 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c15ea70 msgr2=0x7f130c1405e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c15ea70 0x7f130c1405e0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f13000ae040 tx=0x7f130001ac90 comp rx=0 tx=0).stop 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 shutdown_connections 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f12ec06f430 0x7f12ec071940 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f130c161f90 0x7f130c071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f130c15ea70 0x7f130c1405e0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 --2- 172.21.15.80:0/2026736343 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f130c0d0510 0x7f130c146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 >> 172.21.15.80:0/2026736343 conn(0x7f130c06d080 msgr2=0x7f130c0b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:17.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 shutdown_connections 2024-04-25T13:43:17.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.569+0000 7f1310e57640 1 -- 172.21.15.80:0/2026736343 wait complete. 2024-04-25T13:43:17.593 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:43:17.593 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:43:17.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.721+0000 7f3690e13640 1 Processor -- start 2024-04-25T13:43:17.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.721+0000 7f3690e13640 1 -- start start 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f3690e13640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f368c06f5a0 0x7f368c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f3690e13640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c0733b0 0x7f368c161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f3690e13640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f368c06fcf0 con 0x7f368c161070 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f3690e13640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f368c1619c0 con 0x7f368c06f5a0 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f3690e13640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f368c073f10 con 0x7f368c0733b0 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f3689d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f368c161070 0x7f368c0b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35906/0 (socket says 172.21.15.80:35906) 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f3689d9b640 1 -- 172.21.15.80:0/142020020 learned_addr learned my addr 172.21.15.80:0/142020020 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368ad9d640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c0733b0 0x7f368c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368a59c640 1 --2- 172.21.15.80:0/142020020 >> v2:172.21.15.87:3300/0 conn(0x7f368c06f5a0 0x7f368c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3589099353 0 0) 0x7f368c06fcf0 con 0x7f368c161070 2024-04-25T13:43:17.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3674003530 con 0x7f368c161070 2024-04-25T13:43:17.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1237243111 0 0) 0x7f3674003530 con 0x7f368c161070 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c0733b0 msgr2=0x7f368c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c0733b0 0x7f368c161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 >> v2:172.21.15.87:3300/0 conn(0x7f368c06f5a0 msgr2=0x7f368c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 --2- 172.21.15.80:0/142020020 >> v2:172.21.15.87:3300/0 conn(0x7f368c06f5a0 0x7f368c160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c0b0c60 con 0x7f368c161070 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f3680001680 con 0x7f368c161070 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 >> v1:172.21.15.80:6789/0 conn(0x7f368c161070 legacy=0x7f368c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c0733b0 0x7f36740039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3674003f70 0x7f3674004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3674004d50 0x7f3674005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f368c1619c0 con 0x7f3674004d50 2024-04-25T13:43:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f368c073f10 con 0x7f3674003f70 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f367400bb00 con 0x7f368c0733b0 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368ad9d640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3674004d50 0x7f3674005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f3689d9b640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3674003f70 0x7f3674004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368a59c640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c0733b0 0x7f36740039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368ad9d640 1 -- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c0733b0 msgr2=0x7f36740039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368ad9d640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c0733b0 0x7f36740039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368ad9d640 1 -- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3674003f70 msgr2=0x7f3674004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368ad9d640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3674003f70 0x7f3674004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.725+0000 7f368ad9d640 1 -- 172.21.15.80:0/142020020 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3674003530 con 0x7f3674004d50 2024-04-25T13:43:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f368ad9d640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3674004d50 0x7f3674005160 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f368c070290 tx=0x7f368401cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9093ec0afc8e3c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36840b2070 con 0x7f3674004d50 2024-04-25T13:43:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36840a5b00 con 0x7f3674004d50 2024-04-25T13:43:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f368959a640 1 -- 172.21.15.80:0/142020020 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684009c10 con 0x7f3674004d50 2024-04-25T13:43:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f3690e13640 1 -- 172.21.15.80:0/142020020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3674004d50 msgr2=0x7f3674005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f3690e13640 1 --2- 172.21.15.80:0/142020020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3674004d50 0x7f3674005160 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f368c070290 tx=0x7f368401cc00 comp rx=0 tx=0).stop 2024-04-25T13:43:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f368ad9d640 1 -- 172.21.15.80:0/142020020 reap_dead start 2024-04-25T13:43:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f3690e13640 1 -- 172.21.15.80:0/142020020 shutdown_connections 2024-04-25T13:43:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f3690e13640 1 -- 172.21.15.80:0/142020020 >> 172.21.15.80:0/142020020 conn(0x7f368c06d080 msgr2=0x7f368c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f3690e13640 1 -- 172.21.15.80:0/142020020 shutdown_connections 2024-04-25T13:43:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.729+0000 7f3690e13640 1 -- 172.21.15.80:0/142020020 wait complete. 2024-04-25T13:43:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 Processor -- start 2024-04-25T13:43:17.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 -- start start 2024-04-25T13:43:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c143db0 0x7f368c152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f368c146e80 0x7f368c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c0afdf0 0x7f368c15afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f368c06f9b0 con 0x7f368c0afdf0 2024-04-25T13:43:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f368c0b1220 con 0x7f368c146e80 2024-04-25T13:43:17.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f368c161480 con 0x7f368c143db0 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c0afdf0 0x7f368c15afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c0afdf0 0x7f368c15afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50364/0 (socket says 172.21.15.80:50364) 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 -- 172.21.15.80:0/889470214 learned_addr learned my addr 172.21.15.80:0/889470214 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3689d9b640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f368c146e80 0x7f368c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 -- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c143db0 msgr2=0x7f368c152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368a59c640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c143db0 0x7f368c152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c143db0 0x7f368c152c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 -- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f368c146e80 msgr2=0x7f368c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f368c146e80 0x7f368c0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 -- 172.21.15.80:0/889470214 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c0b0200 con 0x7f368c0afdf0 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f368ad9d640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c0afdf0 0x7f368c15afe0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f368401d4a0 tx=0x7f3684009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f367affd640 1 -- 172.21.15.80:0/889470214 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36840b2040 con 0x7f368c0afdf0 2024-04-25T13:43:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f367affd640 1 -- 172.21.15.80:0/889470214 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36840bc070 con 0x7f368c0afdf0 2024-04-25T13:43:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c1542c0 con 0x7f368c0afdf0 2024-04-25T13:43:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f367affd640 1 -- 172.21.15.80:0/889470214 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3684009c10 con 0x7f368c0afdf0 2024-04-25T13:43:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c153500 con 0x7f368c0afdf0 2024-04-25T13:43:17.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.733+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c1512a0 con 0x7f368c0afdf0 2024-04-25T13:43:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.737+0000 7f367affd640 1 -- 172.21.15.80:0/889470214 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f36840c1460 con 0x7f368c0afdf0 2024-04-25T13:43:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.737+0000 7f367affd640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f366c06f3a0 0x7f366c0718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.737+0000 7f367affd640 1 -- 172.21.15.80:0/889470214 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f3684129ca0 con 0x7f368c0afdf0 2024-04-25T13:43:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.737+0000 7f368a59c640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f366c06f3a0 0x7f366c0718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:17.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.737+0000 7f368a59c640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f366c06f3a0 0x7f366c0718b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f368c0b6e70 tx=0x7f367c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:17.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.741+0000 7f367affd640 1 -- 172.21.15.80:0/889470214 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f368c0b0200 con 0x7f368c0afdf0 2024-04-25T13:43:17.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.893+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f368c06a3a0 con 0x7f366c06f3a0 2024-04-25T13:43:17.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.909+0000 7f367affd640 1 -- 172.21.15.80:0/889470214 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+242205 (secure 0 0 0) 0x7f368c06a3a0 con 0x7f366c06f3a0 2024-04-25T13:43:17.915 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:17.917 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:43:17.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f366c06f3a0 msgr2=0x7f366c0718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f366c06f3a0 0x7f366c0718b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f368c0b6e70 tx=0x7f367c008040 comp rx=0 tx=0).stop 2024-04-25T13:43:17.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c0afdf0 msgr2=0x7f368c15afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:17.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c0afdf0 0x7f368c15afe0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f368401d4a0 tx=0x7f3684009750 comp rx=0 tx=0).stop 2024-04-25T13:43:17.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 shutdown_connections 2024-04-25T13:43:17.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f366c06f3a0 0x7f366c0718b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f368c0afdf0 0x7f368c15afe0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f368c146e80 0x7f368c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 --2- 172.21.15.80:0/889470214 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f368c143db0 0x7f368c152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:17.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 >> 172.21.15.80:0/889470214 conn(0x7f368c06d080 msgr2=0x7f368c0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:17.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 shutdown_connections 2024-04-25T13:43:17.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:17.913+0000 7f3690e13640 1 -- 172.21.15.80:0/889470214 wait complete. 2024-04-25T13:43:17.934 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-04-25T13:43:16.153842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":29,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3538,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558961,"statfs":{"total":1600000000,"available":1596376554,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.398250"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.772662+0000","last_change":"2024-04-25T13:43:12.772662+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.772662+0000","last_undegraded":"2024-04-25T13:43:12.772662+0000","last_fullsized":"2024-04-25T13:43:12.772662+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.784228+0000","last_change":"2024-04-25T13:43:12.784228+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.784228+0000","last_undegraded":"2024-04-25T13:43:12.784228+0000","last_fullsized":"2024-04-25T13:43:12.784228+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.789892+0000","last_change":"2024-04-25T13:43:12.789892+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.789892+0000","last_undegraded":"2024-04-25T13:43:12.789892+0000","last_fullsized":"2024-04-25T13:43:12.789892+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.776087+0000","last_change":"2024-04-25T13:43:12.776087+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.776087+0000","last_undegraded":"2024-04-25T13:43:12.776087+0000","last_fullsized":"2024-04-25T13:43:12.776087+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.772994+0000","last_change":"2024-04-25T13:43:12.772994+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.772994+0000","last_undegraded":"2024-04-25T13:43:12.772994+0000","last_fullsized":"2024-04-25T13:43:12.772994+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.780051+0000","last_change":"2024-04-25T13:43:12.780051+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.780051+0000","last_undegraded":"2024-04-25T13:43:12.780051+0000","last_fullsized":"2024-04-25T13:43:12.780051+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.789181+0000","last_change":"2024-04-25T13:43:12.789181+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.789181+0000","last_undegraded":"2024-04-25T13:43:12.789181+0000","last_fullsized":"2024-04-25T13:43:12.789181+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.780989+0000","last_change":"2024-04-25T13:43:12.780989+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.780989+0000","last_undegraded":"2024-04-25T13:43:12.780989+0000","last_fullsized":"2024-04-25T13:43:12.780989+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.782549+0000","last_change":"2024-04-25T13:43:12.782549+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.782549+0000","last_undegraded":"2024-04-25T13:43:12.782549+0000","last_fullsized":"2024-04-25T13:43:12.782549+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.780395+0000","last_change":"2024-04-25T13:43:12.780395+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.780395+0000","last_undegraded":"2024-04-25T13:43:12.780395+0000","last_fullsized":"2024-04-25T13:43:12.780395+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.778776+0000","last_change":"2024-04-25T13:43:12.778776+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.778776+0000","last_undegraded":"2024-04-25T13:43:12.778776+0000","last_fullsized":"2024-04-25T13:43:12.778776+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.783556+0000","last_change":"2024-04-25T13:43:12.783556+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.783556+0000","last_undegraded":"2024-04-25T13:43:12.783556+0000","last_fullsized":"2024-04-25T13:43:12.783556+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.782181+0000","last_change":"2024-04-25T13:43:12.782181+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.782181+0000","last_undegraded":"2024-04-25T13:43:12.782181+0000","last_fullsized":"2024-04-25T13:43:12.782181+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.781503+0000","last_change":"2024-04-25T13:43:12.781503+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.781503+0000","last_undegraded":"2024-04-25T13:43:12.781503+0000","last_fullsized":"2024-04-25T13:43:12.781503+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.781595+0000","last_change":"2024-04-25T13:43:12.781595+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.781595+0000","last_undegraded":"2024-04-25T13:43:12.781595+0000","last_fullsized":"2024-04-25T13:43:12.781595+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779084+0000","last_change":"2024-04-25T13:43:12.779084+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779084+0000","last_undegraded":"2024-04-25T13:43:12.779084+0000","last_fullsized":"2024-04-25T13:43:12.779084+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.783723+0000","last_change":"2024-04-25T13:43:12.783723+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.783723+0000","last_undegraded":"2024-04-25T13:43:12.783723+0000","last_fullsized":"2024-04-25T13:43:12.783723+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.787443+0000","last_change":"2024-04-25T13:43:12.787443+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.787443+0000","last_undegraded":"2024-04-25T13:43:12.787443+0000","last_fullsized":"2024-04-25T13:43:12.787443+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.772678+0000","last_change":"2024-04-25T13:43:12.772678+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.772678+0000","last_undegraded":"2024-04-25T13:43:12.772678+0000","last_fullsized":"2024-04-25T13:43:12.772678+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.780390+0000","last_change":"2024-04-25T13:43:12.780390+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.780390+0000","last_undegraded":"2024-04-25T13:43:12.780390+0000","last_fullsized":"2024-04-25T13:43:12.780390+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.795669+0000","last_change":"2024-04-25T13:43:12.795669+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.795669+0000","last_undegraded":"2024-04-25T13:43:12.795669+0000","last_fullsized":"2024-04-25T13:43:12.795669+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.782422+0000","last_change":"2024-04-25T13:43:12.782422+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.782422+0000","last_undegraded":"2024-04-25T13:43:12.782422+0000","last_fullsized":"2024-04-25T13:43:12.782422+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.786582+0000","last_change":"2024-04-25T13:43:12.786582+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.786582+0000","last_undegraded":"2024-04-25T13:43:12.786582+0000","last_fullsized":"2024-04-25T13:43:12.786582+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.781114+0000","last_change":"2024-04-25T13:43:12.781114+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.781114+0000","last_undegraded":"2024-04-25T13:43:12.781114+0000","last_fullsized":"2024-04-25T13:43:12.781114+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.772045+0000","last_change":"2024-04-25T13:43:12.772045+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.772045+0000","last_undegraded":"2024-04-25T13:43:12.772045+0000","last_fullsized":"2024-04-25T13:43:12.772045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.777378+0000","last_change":"2024-04-25T13:43:12.777378+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.777378+0000","last_undegraded":"2024-04-25T13:43:12.777378+0000","last_fullsized":"2024-04-25T13:43:12.777378+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.781676+0000","last_change":"2024-04-25T13:43:12.781676+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.781676+0000","last_undegraded":"2024-04-25T13:43:12.781676+0000","last_fullsized":"2024-04-25T13:43:12.781676+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.772045+0000","last_change":"2024-04-25T13:43:12.772045+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.772045+0000","last_undegraded":"2024-04-25T13:43:12.772045+0000","last_fullsized":"2024-04-25T13:43:12.772045+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.782983+0000","last_change":"2024-04-25T13:43:12.782983+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.782983+0000","last_undegraded":"2024-04-25T13:43:12.782983+0000","last_fullsized":"2024-04-25T13:43:12.782983+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.783581+0000","last_change":"2024-04-25T13:43:12.783581+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.783581+0000","last_undegraded":"2024-04-25T13:43:12.783581+0000","last_fullsized":"2024-04-25T13:43:12.783581+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.772008+0000","last_change":"2024-04-25T13:43:12.772008+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.772008+0000","last_undegraded":"2024-04-25T13:43:12.772008+0000","last_fullsized":"2024-04-25T13:43:12.772008+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.781236+0000","last_change":"2024-04-25T13:43:12.781236+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.781236+0000","last_undegraded":"2024-04-25T13:43:12.781236+0000","last_fullsized":"2024-04-25T13:43:12.781236+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.778349+0000","last_change":"2024-04-25T13:43:12.778349+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.778349+0000","last_undegraded":"2024-04-25T13:43:12.778349+0000","last_fullsized":"2024-04-25T13:43:12.778349+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.788970+0000","last_change":"2024-04-25T13:43:12.788812+0000","last_active":"2024-04-25T13:43:12.788970+0000","last_peered":"2024-04-25T13:43:12.788970+0000","last_clean":"2024-04-25T13:43:12.788970+0000","last_became_active":"2024-04-25T13:43:12.788387+0000","last_became_peered":"2024-04-25T13:43:12.788387+0000","last_unstale":"2024-04-25T13:43:12.788970+0000","last_undegraded":"2024-04-25T13:43:12.788970+0000","last_fullsized":"2024-04-25T13:43:12.788970+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:34.342483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.778942+0000","last_change":"2024-04-25T13:43:12.778942+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.778942+0000","last_undegraded":"2024-04-25T13:43:12.778942+0000","last_fullsized":"2024-04-25T13:43:12.778942+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.786378+0000","last_change":"2024-04-25T13:43:12.786208+0000","last_active":"2024-04-25T13:43:12.786378+0000","last_peered":"2024-04-25T13:43:12.786378+0000","last_clean":"2024-04-25T13:43:12.786378+0000","last_became_active":"2024-04-25T13:43:12.785877+0000","last_became_peered":"2024-04-25T13:43:12.785877+0000","last_unstale":"2024-04-25T13:43:12.786378+0000","last_undegraded":"2024-04-25T13:43:12.786378+0000","last_fullsized":"2024-04-25T13:43:12.786378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:24.645012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779569+0000","last_change":"2024-04-25T13:43:12.779569+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779569+0000","last_undegraded":"2024-04-25T13:43:12.779569+0000","last_fullsized":"2024-04-25T13:43:12.779569+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.788390+0000","last_change":"2024-04-25T13:43:12.788227+0000","last_active":"2024-04-25T13:43:12.788390+0000","last_peered":"2024-04-25T13:43:12.788390+0000","last_clean":"2024-04-25T13:43:12.788390+0000","last_became_active":"2024-04-25T13:43:12.787727+0000","last_became_peered":"2024-04-25T13:43:12.787727+0000","last_unstale":"2024-04-25T13:43:12.788390+0000","last_undegraded":"2024-04-25T13:43:12.788390+0000","last_fullsized":"2024-04-25T13:43:12.788390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:16.080696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.778686+0000","last_change":"2024-04-25T13:43:12.778686+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.778686+0000","last_undegraded":"2024-04-25T13:43:12.778686+0000","last_fullsized":"2024-04-25T13:43:12.778686+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.805966+0000","last_change":"2024-04-25T13:43:12.805761+0000","last_active":"2024-04-25T13:43:12.805966+0000","last_peered":"2024-04-25T13:43:12.805966+0000","last_clean":"2024-04-25T13:43:12.805966+0000","last_became_active":"2024-04-25T13:43:12.804624+0000","last_became_peered":"2024-04-25T13:43:12.804624+0000","last_unstale":"2024-04-25T13:43:12.805966+0000","last_undegraded":"2024-04-25T13:43:12.805966+0000","last_fullsized":"2024-04-25T13:43:12.805966+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:53.019024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779579+0000","last_change":"2024-04-25T13:43:12.779579+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779579+0000","last_undegraded":"2024-04-25T13:43:12.779579+0000","last_fullsized":"2024-04-25T13:43:12.779579+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.801249+0000","last_change":"2024-04-25T13:43:12.800849+0000","last_active":"2024-04-25T13:43:12.801249+0000","last_peered":"2024-04-25T13:43:12.801249+0000","last_clean":"2024-04-25T13:43:12.801249+0000","last_became_active":"2024-04-25T13:43:12.794945+0000","last_became_peered":"2024-04-25T13:43:12.794945+0000","last_unstale":"2024-04-25T13:43:12.801249+0000","last_undegraded":"2024-04-25T13:43:12.801249+0000","last_fullsized":"2024-04-25T13:43:12.801249+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:50.861519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.780167+0000","last_change":"2024-04-25T13:43:12.780167+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.780167+0000","last_undegraded":"2024-04-25T13:43:12.780167+0000","last_fullsized":"2024-04-25T13:43:12.780167+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.796266+0000","last_change":"2024-04-25T13:43:12.796111+0000","last_active":"2024-04-25T13:43:12.796266+0000","last_peered":"2024-04-25T13:43:12.796266+0000","last_clean":"2024-04-25T13:43:12.796266+0000","last_became_active":"2024-04-25T13:43:12.795866+0000","last_became_peered":"2024-04-25T13:43:12.795866+0000","last_unstale":"2024-04-25T13:43:12.796266+0000","last_undegraded":"2024-04-25T13:43:12.796266+0000","last_fullsized":"2024-04-25T13:43:12.796266+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:21:07.402947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779597+0000","last_change":"2024-04-25T13:43:12.779597+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779597+0000","last_undegraded":"2024-04-25T13:43:12.779597+0000","last_fullsized":"2024-04-25T13:43:12.779597+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.799959+0000","last_change":"2024-04-25T13:43:12.799619+0000","last_active":"2024-04-25T13:43:12.799959+0000","last_peered":"2024-04-25T13:43:12.799959+0000","last_clean":"2024-04-25T13:43:12.799959+0000","last_became_active":"2024-04-25T13:43:12.798541+0000","last_became_peered":"2024-04-25T13:43:12.798541+0000","last_unstale":"2024-04-25T13:43:12.799959+0000","last_undegraded":"2024-04-25T13:43:12.799959+0000","last_fullsized":"2024-04-25T13:43:12.799959+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:00.430238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.782473+0000","last_change":"2024-04-25T13:43:12.782473+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.782473+0000","last_undegraded":"2024-04-25T13:43:12.782473+0000","last_fullsized":"2024-04-25T13:43:12.782473+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.793278+0000","last_change":"2024-04-25T13:43:12.792810+0000","last_active":"2024-04-25T13:43:12.793278+0000","last_peered":"2024-04-25T13:43:12.793278+0000","last_clean":"2024-04-25T13:43:12.793278+0000","last_became_active":"2024-04-25T13:43:12.791521+0000","last_became_peered":"2024-04-25T13:43:12.791521+0000","last_unstale":"2024-04-25T13:43:12.793278+0000","last_undegraded":"2024-04-25T13:43:12.793278+0000","last_fullsized":"2024-04-25T13:43:12.793278+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:44.833047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.788119+0000","last_change":"2024-04-25T13:43:12.788119+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.788119+0000","last_undegraded":"2024-04-25T13:43:12.788119+0000","last_fullsized":"2024-04-25T13:43:12.788119+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.805252+0000","last_change":"2024-04-25T13:43:12.804964+0000","last_active":"2024-04-25T13:43:12.805252+0000","last_peered":"2024-04-25T13:43:12.805252+0000","last_clean":"2024-04-25T13:43:12.805252+0000","last_became_active":"2024-04-25T13:43:12.802321+0000","last_became_peered":"2024-04-25T13:43:12.802321+0000","last_unstale":"2024-04-25T13:43:12.805252+0000","last_undegraded":"2024-04-25T13:43:12.805252+0000","last_fullsized":"2024-04-25T13:43:12.805252+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:10.769242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.793816+0000","last_change":"2024-04-25T13:43:12.793816+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.793816+0000","last_undegraded":"2024-04-25T13:43:12.793816+0000","last_fullsized":"2024-04-25T13:43:12.793816+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.778990+0000","last_change":"2024-04-25T13:43:12.775532+0000","last_active":"2024-04-25T13:43:12.778990+0000","last_peered":"2024-04-25T13:43:12.778990+0000","last_clean":"2024-04-25T13:43:12.778990+0000","last_became_active":"2024-04-25T13:43:12.775272+0000","last_became_peered":"2024-04-25T13:43:12.775272+0000","last_unstale":"2024-04-25T13:43:12.778990+0000","last_undegraded":"2024-04-25T13:43:12.778990+0000","last_fullsized":"2024-04-25T13:43:12.778990+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:42.145149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779797+0000","last_change":"2024-04-25T13:43:12.779797+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779797+0000","last_undegraded":"2024-04-25T13:43:12.779797+0000","last_fullsized":"2024-04-25T13:43:12.779797+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.804231+0000","last_change":"2024-04-25T13:43:12.803808+0000","last_active":"2024-04-25T13:43:12.804231+0000","last_peered":"2024-04-25T13:43:12.804231+0000","last_clean":"2024-04-25T13:43:12.804231+0000","last_became_active":"2024-04-25T13:43:12.802099+0000","last_became_peered":"2024-04-25T13:43:12.802099+0000","last_unstale":"2024-04-25T13:43:12.804231+0000","last_undegraded":"2024-04-25T13:43:12.804231+0000","last_fullsized":"2024-04-25T13:43:12.804231+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:33:29.795587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.785704+0000","last_change":"2024-04-25T13:43:12.785704+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.785704+0000","last_undegraded":"2024-04-25T13:43:12.785704+0000","last_fullsized":"2024-04-25T13:43:12.785704+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.801320+0000","last_change":"2024-04-25T13:43:12.800894+0000","last_active":"2024-04-25T13:43:12.801320+0000","last_peered":"2024-04-25T13:43:12.801320+0000","last_clean":"2024-04-25T13:43:12.801320+0000","last_became_active":"2024-04-25T13:43:12.796733+0000","last_became_peered":"2024-04-25T13:43:12.796733+0000","last_unstale":"2024-04-25T13:43:12.801320+0000","last_undegraded":"2024-04-25T13:43:12.801320+0000","last_fullsized":"2024-04-25T13:43:12.801320+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:58.449537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.776379+0000","last_change":"2024-04-25T13:43:12.776379+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.776379+0000","last_undegraded":"2024-04-25T13:43:12.776379+0000","last_fullsized":"2024-04-25T13:43:12.776379+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.773208+0000","last_change":"2024-04-25T13:43:12.773012+0000","last_active":"2024-04-25T13:43:12.773208+0000","last_peered":"2024-04-25T13:43:12.773208+0000","last_clean":"2024-04-25T13:43:12.773208+0000","last_became_active":"2024-04-25T13:43:12.772654+0000","last_became_peered":"2024-04-25T13:43:12.772654+0000","last_unstale":"2024-04-25T13:43:12.773208+0000","last_undegraded":"2024-04-25T13:43:12.773208+0000","last_fullsized":"2024-04-25T13:43:12.773208+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:37.609463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.782770+0000","last_change":"2024-04-25T13:43:12.782770+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.782770+0000","last_undegraded":"2024-04-25T13:43:12.782770+0000","last_fullsized":"2024-04-25T13:43:12.782770+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.804820+0000","last_change":"2024-04-25T13:43:12.804471+0000","last_active":"2024-04-25T13:43:12.804820+0000","last_peered":"2024-04-25T13:43:12.804820+0000","last_clean":"2024-04-25T13:43:12.804820+0000","last_became_active":"2024-04-25T13:43:12.803661+0000","last_became_peered":"2024-04-25T13:43:12.803661+0000","last_unstale":"2024-04-25T13:43:12.804820+0000","last_undegraded":"2024-04-25T13:43:12.804820+0000","last_fullsized":"2024-04-25T13:43:12.804820+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:28.377613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.780458+0000","last_change":"2024-04-25T13:43:12.780458+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.780458+0000","last_undegraded":"2024-04-25T13:43:12.780458+0000","last_fullsized":"2024-04-25T13:43:12.780458+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.801800+0000","last_change":"2024-04-25T13:43:12.800897+0000","last_active":"2024-04-25T13:43:12.801800+0000","last_peered":"2024-04-25T13:43:12.801800+0000","last_clean":"2024-04-25T13:43:12.801800+0000","last_became_active":"2024-04-25T13:43:12.798720+0000","last_became_peered":"2024-04-25T13:43:12.798720+0000","last_unstale":"2024-04-25T13:43:12.801800+0000","last_undegraded":"2024-04-25T13:43:12.801800+0000","last_fullsized":"2024-04-25T13:43:12.801800+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:44.542137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779978+0000","last_change":"2024-04-25T13:43:12.779978+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779978+0000","last_undegraded":"2024-04-25T13:43:12.779978+0000","last_fullsized":"2024-04-25T13:43:12.779978+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.803534+0000","last_change":"2024-04-25T13:43:12.803174+0000","last_active":"2024-04-25T13:43:12.803534+0000","last_peered":"2024-04-25T13:43:12.803534+0000","last_clean":"2024-04-25T13:43:12.803534+0000","last_became_active":"2024-04-25T13:43:12.799480+0000","last_became_peered":"2024-04-25T13:43:12.799480+0000","last_unstale":"2024-04-25T13:43:12.803534+0000","last_undegraded":"2024-04-25T13:43:12.803534+0000","last_fullsized":"2024-04-25T13:43:12.803534+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:12.290850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.781738+0000","last_change":"2024-04-25T13:43:12.781738+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.781738+0000","last_undegraded":"2024-04-25T13:43:12.781738+0000","last_fullsized":"2024-04-25T13:43:12.781738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.779055+0000","last_change":"2024-04-25T13:43:12.776468+0000","last_active":"2024-04-25T13:43:12.779055+0000","last_peered":"2024-04-25T13:43:12.779055+0000","last_clean":"2024-04-25T13:43:12.779055+0000","last_became_active":"2024-04-25T13:43:12.776176+0000","last_became_peered":"2024-04-25T13:43:12.776176+0000","last_unstale":"2024-04-25T13:43:12.779055+0000","last_undegraded":"2024-04-25T13:43:12.779055+0000","last_fullsized":"2024-04-25T13:43:12.779055+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:34:41.261120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.775404+0000","last_change":"2024-04-25T13:43:12.775404+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.775404+0000","last_undegraded":"2024-04-25T13:43:12.775404+0000","last_fullsized":"2024-04-25T13:43:12.775404+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.792185+0000","last_change":"2024-04-25T13:43:12.791524+0000","last_active":"2024-04-25T13:43:12.792185+0000","last_peered":"2024-04-25T13:43:12.792185+0000","last_clean":"2024-04-25T13:43:12.792185+0000","last_became_active":"2024-04-25T13:43:12.790749+0000","last_became_peered":"2024-04-25T13:43:12.790749+0000","last_unstale":"2024-04-25T13:43:12.792185+0000","last_undegraded":"2024-04-25T13:43:12.792185+0000","last_fullsized":"2024-04-25T13:43:12.792185+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:39.327419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779517+0000","last_change":"2024-04-25T13:43:12.779517+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779517+0000","last_undegraded":"2024-04-25T13:43:12.779517+0000","last_fullsized":"2024-04-25T13:43:12.779517+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.805643+0000","last_change":"2024-04-25T13:43:12.805392+0000","last_active":"2024-04-25T13:43:12.805643+0000","last_peered":"2024-04-25T13:43:12.805643+0000","last_clean":"2024-04-25T13:43:12.805643+0000","last_became_active":"2024-04-25T13:43:12.803123+0000","last_became_peered":"2024-04-25T13:43:12.803123+0000","last_unstale":"2024-04-25T13:43:12.805643+0000","last_undegraded":"2024-04-25T13:43:12.805643+0000","last_fullsized":"2024-04-25T13:43:12.805643+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:43:28.563493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.784196+0000","last_change":"2024-04-25T13:43:12.784196+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.784196+0000","last_undegraded":"2024-04-25T13:43:12.784196+0000","last_fullsized":"2024-04-25T13:43:12.784196+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.789069+0000","last_change":"2024-04-25T13:43:12.788962+0000","last_active":"2024-04-25T13:43:12.789069+0000","last_peered":"2024-04-25T13:43:12.789069+0000","last_clean":"2024-04-25T13:43:12.789069+0000","last_became_active":"2024-04-25T13:43:12.788672+0000","last_became_peered":"2024-04-25T13:43:12.788672+0000","last_unstale":"2024-04-25T13:43:12.789069+0000","last_undegraded":"2024-04-25T13:43:12.789069+0000","last_fullsized":"2024-04-25T13:43:12.789069+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:48.496738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.774449+0000","last_change":"2024-04-25T13:43:12.774449+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.774449+0000","last_undegraded":"2024-04-25T13:43:12.774449+0000","last_fullsized":"2024-04-25T13:43:12.774449+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.791917+0000","last_change":"2024-04-25T13:43:12.791759+0000","last_active":"2024-04-25T13:43:12.791917+0000","last_peered":"2024-04-25T13:43:12.791917+0000","last_clean":"2024-04-25T13:43:12.791917+0000","last_became_active":"2024-04-25T13:43:12.791438+0000","last_became_peered":"2024-04-25T13:43:12.791438+0000","last_unstale":"2024-04-25T13:43:12.791917+0000","last_undegraded":"2024-04-25T13:43:12.791917+0000","last_fullsized":"2024-04-25T13:43:12.791917+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:35.842366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.783216+0000","last_change":"2024-04-25T13:43:12.783216+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.783216+0000","last_undegraded":"2024-04-25T13:43:12.783216+0000","last_fullsized":"2024-04-25T13:43:12.783216+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.782751+0000","last_change":"2024-04-25T13:43:12.782117+0000","last_active":"2024-04-25T13:43:12.782751+0000","last_peered":"2024-04-25T13:43:12.782751+0000","last_clean":"2024-04-25T13:43:12.782751+0000","last_became_active":"2024-04-25T13:43:12.781837+0000","last_became_peered":"2024-04-25T13:43:12.781837+0000","last_unstale":"2024-04-25T13:43:12.782751+0000","last_undegraded":"2024-04-25T13:43:12.782751+0000","last_fullsized":"2024-04-25T13:43:12.782751+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:47.965896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.780079+0000","last_change":"2024-04-25T13:43:12.780079+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.780079+0000","last_undegraded":"2024-04-25T13:43:12.780079+0000","last_fullsized":"2024-04-25T13:43:12.780079+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.782746+0000","last_change":"2024-04-25T13:43:12.782326+0000","last_active":"2024-04-25T13:43:12.782746+0000","last_peered":"2024-04-25T13:43:12.782746+0000","last_clean":"2024-04-25T13:43:12.782746+0000","last_became_active":"2024-04-25T13:43:12.781030+0000","last_became_peered":"2024-04-25T13:43:12.781030+0000","last_unstale":"2024-04-25T13:43:12.782746+0000","last_undegraded":"2024-04-25T13:43:12.782746+0000","last_fullsized":"2024-04-25T13:43:12.782746+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:18.149323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.773963+0000","last_change":"2024-04-25T13:43:12.773963+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.773963+0000","last_undegraded":"2024-04-25T13:43:12.773963+0000","last_fullsized":"2024-04-25T13:43:12.773963+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.782718+0000","last_change":"2024-04-25T13:43:12.781235+0000","last_active":"2024-04-25T13:43:12.782718+0000","last_peered":"2024-04-25T13:43:12.782718+0000","last_clean":"2024-04-25T13:43:12.782718+0000","last_became_active":"2024-04-25T13:43:12.780825+0000","last_became_peered":"2024-04-25T13:43:12.780825+0000","last_unstale":"2024-04-25T13:43:12.782718+0000","last_undegraded":"2024-04-25T13:43:12.782718+0000","last_fullsized":"2024-04-25T13:43:12.782718+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:08.345830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.780555+0000","last_change":"2024-04-25T13:43:12.780555+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.780555+0000","last_undegraded":"2024-04-25T13:43:12.780555+0000","last_fullsized":"2024-04-25T13:43:12.780555+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.794359+0000","last_change":"2024-04-25T13:43:12.794225+0000","last_active":"2024-04-25T13:43:12.794359+0000","last_peered":"2024-04-25T13:43:12.794359+0000","last_clean":"2024-04-25T13:43:12.794359+0000","last_became_active":"2024-04-25T13:43:12.793919+0000","last_became_peered":"2024-04-25T13:43:12.793919+0000","last_unstale":"2024-04-25T13:43:12.794359+0000","last_undegraded":"2024-04-25T13:43:12.794359+0000","last_fullsized":"2024-04-25T13:43:12.794359+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:56.996620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779905+0000","last_change":"2024-04-25T13:43:12.779905+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779905+0000","last_undegraded":"2024-04-25T13:43:12.779905+0000","last_fullsized":"2024-04-25T13:43:12.779905+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.799942+0000","last_change":"2024-04-25T13:43:12.799608+0000","last_active":"2024-04-25T13:43:12.799942+0000","last_peered":"2024-04-25T13:43:12.799942+0000","last_clean":"2024-04-25T13:43:12.799942+0000","last_became_active":"2024-04-25T13:43:12.798852+0000","last_became_peered":"2024-04-25T13:43:12.798852+0000","last_unstale":"2024-04-25T13:43:12.799942+0000","last_undegraded":"2024-04-25T13:43:12.799942+0000","last_fullsized":"2024-04-25T13:43:12.799942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:05.856558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.777875+0000","last_change":"2024-04-25T13:43:12.777875+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.777875+0000","last_undegraded":"2024-04-25T13:43:12.777875+0000","last_fullsized":"2024-04-25T13:43:12.777875+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.796134+0000","last_change":"2024-04-25T13:43:12.795584+0000","last_active":"2024-04-25T13:43:12.796134+0000","last_peered":"2024-04-25T13:43:12.796134+0000","last_clean":"2024-04-25T13:43:12.796134+0000","last_became_active":"2024-04-25T13:43:12.794735+0000","last_became_peered":"2024-04-25T13:43:12.794735+0000","last_unstale":"2024-04-25T13:43:12.796134+0000","last_undegraded":"2024-04-25T13:43:12.796134+0000","last_fullsized":"2024-04-25T13:43:12.796134+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:30.546939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.777358+0000","last_change":"2024-04-25T13:43:12.777358+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.777358+0000","last_undegraded":"2024-04-25T13:43:12.777358+0000","last_fullsized":"2024-04-25T13:43:12.777358+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.782682+0000","last_change":"2024-04-25T13:43:12.781249+0000","last_active":"2024-04-25T13:43:12.782682+0000","last_peered":"2024-04-25T13:43:12.782682+0000","last_clean":"2024-04-25T13:43:12.782682+0000","last_became_active":"2024-04-25T13:43:12.780946+0000","last_became_peered":"2024-04-25T13:43:12.780946+0000","last_unstale":"2024-04-25T13:43:12.782682+0000","last_undegraded":"2024-04-25T13:43:12.782682+0000","last_fullsized":"2024-04-25T13:43:12.782682+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:50.175893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.794738+0000","last_change":"2024-04-25T13:43:12.794738+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.794738+0000","last_undegraded":"2024-04-25T13:43:12.794738+0000","last_fullsized":"2024-04-25T13:43:12.794738+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.797406+0000","last_change":"2024-04-25T13:43:12.796960+0000","last_active":"2024-04-25T13:43:12.797406+0000","last_peered":"2024-04-25T13:43:12.797406+0000","last_clean":"2024-04-25T13:43:12.797406+0000","last_became_active":"2024-04-25T13:43:12.793772+0000","last_became_peered":"2024-04-25T13:43:12.793772+0000","last_unstale":"2024-04-25T13:43:12.797406+0000","last_undegraded":"2024-04-25T13:43:12.797406+0000","last_fullsized":"2024-04-25T13:43:12.797406+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:36.522626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.779338+0000","last_change":"2024-04-25T13:43:12.779338+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.779338+0000","last_undegraded":"2024-04-25T13:43:12.779338+0000","last_fullsized":"2024-04-25T13:43:12.779338+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.791079+0000","last_change":"2024-04-25T13:43:12.790794+0000","last_active":"2024-04-25T13:43:12.791079+0000","last_peered":"2024-04-25T13:43:12.791079+0000","last_clean":"2024-04-25T13:43:12.791079+0000","last_became_active":"2024-04-25T13:43:12.790423+0000","last_became_peered":"2024-04-25T13:43:12.790423+0000","last_unstale":"2024-04-25T13:43:12.791079+0000","last_undegraded":"2024-04-25T13:43:12.791079+0000","last_fullsized":"2024-04-25T13:43:12.791079+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:40.636893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.777205+0000","last_change":"2024-04-25T13:43:12.777205+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.777205+0000","last_undegraded":"2024-04-25T13:43:12.777205+0000","last_fullsized":"2024-04-25T13:43:12.777205+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.784686+0000","last_change":"2024-04-25T13:43:12.784489+0000","last_active":"2024-04-25T13:43:12.784686+0000","last_peered":"2024-04-25T13:43:12.784686+0000","last_clean":"2024-04-25T13:43:12.784686+0000","last_became_active":"2024-04-25T13:43:12.783452+0000","last_became_peered":"2024-04-25T13:43:12.783452+0000","last_unstale":"2024-04-25T13:43:12.784686+0000","last_undegraded":"2024-04-25T13:43:12.784686+0000","last_fullsized":"2024-04-25T13:43:12.784686+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:16.461433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":1,"reported_epoch":61,"state":"creating+peering","last_fresh":"2024-04-25T13:43:12.777199+0000","last_change":"2024-04-25T13:43:12.777199+0000","last_active":"2024-04-25T13:43:12.754362+0000","last_peered":"2024-04-25T13:43:12.754362+0000","last_clean":"2024-04-25T13:43:12.754362+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:43:12.777199+0000","last_undegraded":"2024-04-25T13:43:12.777199+0000","last_fullsized":"2024-04-25T13:43:12.777199+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.790637+0000","last_change":"2024-04-25T13:43:12.789704+0000","last_active":"2024-04-25T13:43:12.790637+0000","last_peered":"2024-04-25T13:43:12.790637+0000","last_clean":"2024-04-25T13:43:12.790637+0000","last_became_active":"2024-04-25T13:43:12.788972+0000","last_became_peered":"2024-04-25T13:43:12.788972+0000","last_unstale":"2024-04-25T13:43:12.790637+0000","last_undegraded":"2024-04-25T13:43:12.790637+0000","last_fullsized":"2024-04-25T13:43:12.790637+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:41.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":188,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-25T13:43:12.781567+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:43:12.781567+0000","last_peered":"2024-04-25T13:43:12.781567+0000","last_clean":"2024-04-25T13:43:12.781567+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:43:12.781567+0000","last_undegraded":"2024-04-25T13:43:12.781567+0000","last_fullsized":"2024-04-25T13:43:12.781567+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:28.137733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":879,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194433,"statfs":{"total":200000000,"available":199099593,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":296,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195015,"statfs":{"total":200000000,"available":199696228,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":302,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195009,"statfs":{"total":200000000,"available":199689961,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":296,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195015,"statfs":{"total":200000000,"available":199696228,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":296,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195015,"statfs":{"total":200000000,"available":199696228,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574885,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":296,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195015,"statfs":{"total":200000000,"available":199696228,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574885,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":296,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195015,"statfs":{"total":200000000,"available":199696228,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574885,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":873,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194439,"statfs":{"total":200000000,"available":199105860,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:43:17.940 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:43:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.121+0000 7fb02692f640 1 Processor -- start 2024-04-25T13:43:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.121+0000 7fb02692f640 1 -- start start 2024-04-25T13:43:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb02692f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb020161f90 0x7fb0200cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb02692f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0200d0510 0x7fb02015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb02692f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0200d0d70 con 0x7fb02015ea70 2024-04-25T13:43:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb02692f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb0200d1050 con 0x7fb0200d0510 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb02692f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb02006dca0 con 0x7fb020161f90 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb01f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0200d0510 0x7fb02015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb024ecc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb02015ea70 0x7fb02015ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35922/0 (socket says 172.21.15.80:35922) 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb024ecc640 1 -- 172.21.15.80:0/3074368058 learned_addr learned my addr 172.21.15.80:0/3074368058 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb01ffff640 1 --2- 172.21.15.80:0/3074368058 >> v2:172.21.15.87:3300/0 conn(0x7fb020161f90 0x7fb0200cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb01effd640 1 -- 172.21.15.80:0/3074368058 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3123401948 0 0) 0x7fb0200d0d70 con 0x7fb02015ea70 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb01f7fe640 1 -- 172.21.15.80:0/3074368058 >> v2:172.21.15.87:3300/0 conn(0x7fb020161f90 msgr2=0x7fb0200cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb01f7fe640 1 --2- 172.21.15.80:0/3074368058 >> v2:172.21.15.87:3300/0 conn(0x7fb020161f90 0x7fb0200cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb01f7fe640 1 -- 172.21.15.80:0/3074368058 >> v1:172.21.15.80:6789/0 conn(0x7fb02015ea70 legacy=0x7fb02015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.125+0000 7fb01f7fe640 1 -- 172.21.15.80:0/3074368058 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02006df80 con 0x7fb0200d0510 2024-04-25T13:43:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb01f7fe640 1 --2- 172.21.15.80:0/3074368058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0200d0510 0x7fb02015e530 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb0200d17b0 tx=0x7fb01401c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2e74fd2b4d0fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb01effd640 1 -- 172.21.15.80:0/3074368058 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b0070 con 0x7fb0200d0510 2024-04-25T13:43:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb01effd640 1 -- 172.21.15.80:0/3074368058 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0140ab600 con 0x7fb0200d0510 2024-04-25T13:43:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb01effd640 1 -- 172.21.15.80:0/3074368058 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140ab9d0 con 0x7fb0200d0510 2024-04-25T13:43:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 -- 172.21.15.80:0/3074368058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0200d0510 msgr2=0x7fb02015e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 --2- 172.21.15.80:0/3074368058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0200d0510 0x7fb02015e530 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb0200d17b0 tx=0x7fb01401c390 comp rx=0 tx=0).stop 2024-04-25T13:43:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 -- 172.21.15.80:0/3074368058 shutdown_connections 2024-04-25T13:43:18.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 --2- 172.21.15.80:0/3074368058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0200d0510 0x7fb02015e530 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 --2- 172.21.15.80:0/3074368058 >> v2:172.21.15.87:3300/0 conn(0x7fb020161f90 0x7fb0200cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 -- 172.21.15.80:0/3074368058 >> 172.21.15.80:0/3074368058 conn(0x7fb02006d080 msgr2=0x7fb0200b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 -- 172.21.15.80:0/3074368058 shutdown_connections 2024-04-25T13:43:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 -- 172.21.15.80:0/3074368058 wait complete. 2024-04-25T13:43:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 Processor -- start 2024-04-25T13:43:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.129+0000 7fb02692f640 1 -- start start 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200d0510 0x7fb02013dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02015ea70 0x7fb02015cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb020161f90 0x7fb0200af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb02006f630 con 0x7fb02015ea70 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb02006f4b0 con 0x7fb0200d0510 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb0200d0920 con 0x7fb020161f90 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02015ea70 0x7fb02015cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02015ea70 0x7fb02015cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50382/0 (socket says 172.21.15.80:50382) 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 -- 172.21.15.80:0/3173071732 learned_addr learned my addr 172.21.15.80:0/3173071732 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01ffff640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200d0510 0x7fb02013dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb024ecc640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb020161f90 0x7fb0200af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 -- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb020161f90 msgr2=0x7fb0200af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb020161f90 0x7fb0200af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 -- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200d0510 msgr2=0x7fb02013dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200d0510 0x7fb02013dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 -- 172.21.15.80:0/3173071732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb020073420 con 0x7fb02015ea70 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01f7fe640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02015ea70 0x7fb02015cc00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb01401a020 tx=0x7fb01401ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01cff9640 1 -- 172.21.15.80:0/3173071732 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0140b0050 con 0x7fb02015ea70 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01cff9640 1 -- 172.21.15.80:0/3173071732 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0140b1040 con 0x7fb02015ea70 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01cff9640 1 -- 172.21.15.80:0/3173071732 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014006e20 con 0x7fb02015ea70 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb020073700 con 0x7fb02015ea70 2024-04-25T13:43:18.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb020073cc0 con 0x7fb02015ea70 2024-04-25T13:43:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02006a3a0 con 0x7fb02015ea70 2024-04-25T13:43:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01cff9640 1 -- 172.21.15.80:0/3173071732 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb0140be060 con 0x7fb02015ea70 2024-04-25T13:43:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01cff9640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01cff9640 1 -- 172.21.15.80:0/3173071732 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fb014128c80 con 0x7fb02015ea70 2024-04-25T13:43:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.133+0000 7fb01ffff640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.137+0000 7fb01ffff640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb02006fa20 tx=0x7fb010008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.141+0000 7fb01cff9640 1 -- 172.21.15.80:0/3173071732 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb014016070 con 0x7fb02015ea70 2024-04-25T13:43:18.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.281+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0200769e0 con 0x7fb00c06f400 2024-04-25T13:43:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.301+0000 7fb01cff9640 1 -- 172.21.15.80:0/3173071732 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247481 (secure 0 0 0) 0x7fb0200769e0 con 0x7fb00c06f400 2024-04-25T13:43:18.306 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:18.307 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 msgr2=0x7fb00c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb02006fa20 tx=0x7fb010008040 comp rx=0 tx=0).stop 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02015ea70 msgr2=0x7fb02015cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02015ea70 0x7fb02015cc00 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb01401a020 tx=0x7fb01401ac30 comp rx=0 tx=0).stop 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 shutdown_connections 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb020161f90 0x7fb0200af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02015ea70 0x7fb02015cc00 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 --2- 172.21.15.80:0/3173071732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0200d0510 0x7fb02013dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 >> 172.21.15.80:0/3173071732 conn(0x7fb02006d080 msgr2=0x7fb02015ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 shutdown_connections 2024-04-25T13:43:18.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.305+0000 7fb02692f640 1 -- 172.21.15.80:0/3173071732 wait complete. 2024-04-25T13:43:18.325 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-04-25T13:43:18.154563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3682,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558817,"statfs":{"total":1600000000,"available":1596229532,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002024"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.677933+0000","last_change":"2024-04-25T13:43:13.786736+0000","last_active":"2024-04-25T13:43:15.677933+0000","last_peered":"2024-04-25T13:43:15.677933+0000","last_clean":"2024-04-25T13:43:15.677933+0000","last_became_active":"2024-04-25T13:43:13.786001+0000","last_became_peered":"2024-04-25T13:43:13.786001+0000","last_unstale":"2024-04-25T13:43:15.677933+0000","last_undegraded":"2024-04-25T13:43:15.677933+0000","last_fullsized":"2024-04-25T13:43:15.677933+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:22.765524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774117+0000","last_change":"2024-04-25T13:43:13.780668+0000","last_active":"2024-04-25T13:43:14.774117+0000","last_peered":"2024-04-25T13:43:14.774117+0000","last_clean":"2024-04-25T13:43:14.774117+0000","last_became_active":"2024-04-25T13:43:13.780313+0000","last_became_peered":"2024-04-25T13:43:13.780313+0000","last_unstale":"2024-04-25T13:43:14.774117+0000","last_undegraded":"2024-04-25T13:43:14.774117+0000","last_fullsized":"2024-04-25T13:43:14.774117+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:17.701403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679154+0000","last_change":"2024-04-25T13:43:13.821966+0000","last_active":"2024-04-25T13:43:15.679154+0000","last_peered":"2024-04-25T13:43:15.679154+0000","last_clean":"2024-04-25T13:43:15.679154+0000","last_became_active":"2024-04-25T13:43:13.814824+0000","last_became_peered":"2024-04-25T13:43:13.814824+0000","last_unstale":"2024-04-25T13:43:15.679154+0000","last_undegraded":"2024-04-25T13:43:15.679154+0000","last_fullsized":"2024-04-25T13:43:15.679154+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:48.650383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678314+0000","last_change":"2024-04-25T13:43:13.801027+0000","last_active":"2024-04-25T13:43:15.678314+0000","last_peered":"2024-04-25T13:43:15.678314+0000","last_clean":"2024-04-25T13:43:15.678314+0000","last_became_active":"2024-04-25T13:43:13.799743+0000","last_became_peered":"2024-04-25T13:43:13.799743+0000","last_unstale":"2024-04-25T13:43:15.678314+0000","last_undegraded":"2024-04-25T13:43:15.678314+0000","last_fullsized":"2024-04-25T13:43:15.678314+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:48.361237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678945+0000","last_change":"2024-04-25T13:43:13.800148+0000","last_active":"2024-04-25T13:43:15.678945+0000","last_peered":"2024-04-25T13:43:15.678945+0000","last_clean":"2024-04-25T13:43:15.678945+0000","last_became_active":"2024-04-25T13:43:13.799960+0000","last_became_peered":"2024-04-25T13:43:13.799960+0000","last_unstale":"2024-04-25T13:43:15.678945+0000","last_undegraded":"2024-04-25T13:43:15.678945+0000","last_fullsized":"2024-04-25T13:43:15.678945+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:56.014378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.083655+0000","last_change":"2024-04-25T13:43:13.787116+0000","last_active":"2024-04-25T13:43:15.083655+0000","last_peered":"2024-04-25T13:43:15.083655+0000","last_clean":"2024-04-25T13:43:15.083655+0000","last_became_active":"2024-04-25T13:43:13.786198+0000","last_became_peered":"2024-04-25T13:43:13.786198+0000","last_unstale":"2024-04-25T13:43:15.083655+0000","last_undegraded":"2024-04-25T13:43:15.083655+0000","last_fullsized":"2024-04-25T13:43:15.083655+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:20.294076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.677995+0000","last_change":"2024-04-25T13:43:13.799940+0000","last_active":"2024-04-25T13:43:15.677995+0000","last_peered":"2024-04-25T13:43:15.677995+0000","last_clean":"2024-04-25T13:43:15.677995+0000","last_became_active":"2024-04-25T13:43:13.790919+0000","last_became_peered":"2024-04-25T13:43:13.790919+0000","last_unstale":"2024-04-25T13:43:15.677995+0000","last_undegraded":"2024-04-25T13:43:15.677995+0000","last_fullsized":"2024-04-25T13:43:15.677995+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:28.997488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774172+0000","last_change":"2024-04-25T13:43:13.790588+0000","last_active":"2024-04-25T13:43:14.774172+0000","last_peered":"2024-04-25T13:43:14.774172+0000","last_clean":"2024-04-25T13:43:14.774172+0000","last_became_active":"2024-04-25T13:43:13.790304+0000","last_became_peered":"2024-04-25T13:43:13.790304+0000","last_unstale":"2024-04-25T13:43:14.774172+0000","last_undegraded":"2024-04-25T13:43:14.774172+0000","last_fullsized":"2024-04-25T13:43:14.774172+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:22.718543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680323+0000","last_change":"2024-04-25T13:43:13.803766+0000","last_active":"2024-04-25T13:43:15.680323+0000","last_peered":"2024-04-25T13:43:15.680323+0000","last_clean":"2024-04-25T13:43:15.680323+0000","last_became_active":"2024-04-25T13:43:13.801732+0000","last_became_peered":"2024-04-25T13:43:13.801732+0000","last_unstale":"2024-04-25T13:43:15.680323+0000","last_undegraded":"2024-04-25T13:43:15.680323+0000","last_fullsized":"2024-04-25T13:43:15.680323+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:40.739031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679480+0000","last_change":"2024-04-25T13:43:13.793084+0000","last_active":"2024-04-25T13:43:15.679480+0000","last_peered":"2024-04-25T13:43:15.679480+0000","last_clean":"2024-04-25T13:43:15.679480+0000","last_became_active":"2024-04-25T13:43:13.792785+0000","last_became_peered":"2024-04-25T13:43:13.792785+0000","last_unstale":"2024-04-25T13:43:15.679480+0000","last_undegraded":"2024-04-25T13:43:15.679480+0000","last_fullsized":"2024-04-25T13:43:15.679480+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:19:50.683664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678507+0000","last_change":"2024-04-25T13:43:13.779192+0000","last_active":"2024-04-25T13:43:15.678507+0000","last_peered":"2024-04-25T13:43:15.678507+0000","last_clean":"2024-04-25T13:43:15.678507+0000","last_became_active":"2024-04-25T13:43:13.778940+0000","last_became_peered":"2024-04-25T13:43:13.778940+0000","last_unstale":"2024-04-25T13:43:15.678507+0000","last_undegraded":"2024-04-25T13:43:15.678507+0000","last_fullsized":"2024-04-25T13:43:15.678507+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:48.286655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678316+0000","last_change":"2024-04-25T13:43:13.793822+0000","last_active":"2024-04-25T13:43:15.678316+0000","last_peered":"2024-04-25T13:43:15.678316+0000","last_clean":"2024-04-25T13:43:15.678316+0000","last_became_active":"2024-04-25T13:43:13.792498+0000","last_became_peered":"2024-04-25T13:43:13.792498+0000","last_unstale":"2024-04-25T13:43:15.678316+0000","last_undegraded":"2024-04-25T13:43:15.678316+0000","last_fullsized":"2024-04-25T13:43:15.678316+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:04.134986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775956+0000","last_change":"2024-04-25T13:43:13.806077+0000","last_active":"2024-04-25T13:43:14.775956+0000","last_peered":"2024-04-25T13:43:14.775956+0000","last_clean":"2024-04-25T13:43:14.775956+0000","last_became_active":"2024-04-25T13:43:13.802122+0000","last_became_peered":"2024-04-25T13:43:13.802122+0000","last_unstale":"2024-04-25T13:43:14.775956+0000","last_undegraded":"2024-04-25T13:43:14.775956+0000","last_fullsized":"2024-04-25T13:43:14.775956+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:26.365331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680878+0000","last_change":"2024-04-25T13:43:13.804659+0000","last_active":"2024-04-25T13:43:15.680878+0000","last_peered":"2024-04-25T13:43:15.680878+0000","last_clean":"2024-04-25T13:43:15.680878+0000","last_became_active":"2024-04-25T13:43:13.802820+0000","last_became_peered":"2024-04-25T13:43:13.802820+0000","last_unstale":"2024-04-25T13:43:15.680878+0000","last_undegraded":"2024-04-25T13:43:15.680878+0000","last_fullsized":"2024-04-25T13:43:15.680878+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:34.263507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679677+0000","last_change":"2024-04-25T13:43:13.790858+0000","last_active":"2024-04-25T13:43:15.679677+0000","last_peered":"2024-04-25T13:43:15.679677+0000","last_clean":"2024-04-25T13:43:15.679677+0000","last_became_active":"2024-04-25T13:43:13.789152+0000","last_became_peered":"2024-04-25T13:43:13.789152+0000","last_unstale":"2024-04-25T13:43:15.679677+0000","last_undegraded":"2024-04-25T13:43:15.679677+0000","last_fullsized":"2024-04-25T13:43:15.679677+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:07.884282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084748+0000","last_change":"2024-04-25T13:43:13.795771+0000","last_active":"2024-04-25T13:43:15.084748+0000","last_peered":"2024-04-25T13:43:15.084748+0000","last_clean":"2024-04-25T13:43:15.084748+0000","last_became_active":"2024-04-25T13:43:13.795577+0000","last_became_peered":"2024-04-25T13:43:13.795577+0000","last_unstale":"2024-04-25T13:43:15.084748+0000","last_undegraded":"2024-04-25T13:43:15.084748+0000","last_fullsized":"2024-04-25T13:43:15.084748+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:21.516546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774298+0000","last_change":"2024-04-25T13:43:13.795270+0000","last_active":"2024-04-25T13:43:14.774298+0000","last_peered":"2024-04-25T13:43:14.774298+0000","last_clean":"2024-04-25T13:43:14.774298+0000","last_became_active":"2024-04-25T13:43:13.794966+0000","last_became_peered":"2024-04-25T13:43:13.794966+0000","last_unstale":"2024-04-25T13:43:14.774298+0000","last_undegraded":"2024-04-25T13:43:14.774298+0000","last_fullsized":"2024-04-25T13:43:14.774298+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:03.510743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679616+0000","last_change":"2024-04-25T13:43:13.818113+0000","last_active":"2024-04-25T13:43:15.679616+0000","last_peered":"2024-04-25T13:43:15.679616+0000","last_clean":"2024-04-25T13:43:15.679616+0000","last_became_active":"2024-04-25T13:43:13.810792+0000","last_became_peered":"2024-04-25T13:43:13.810792+0000","last_unstale":"2024-04-25T13:43:15.679616+0000","last_undegraded":"2024-04-25T13:43:15.679616+0000","last_fullsized":"2024-04-25T13:43:15.679616+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:00.485210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.677868+0000","last_change":"2024-04-25T13:43:13.786221+0000","last_active":"2024-04-25T13:43:15.677868+0000","last_peered":"2024-04-25T13:43:15.677868+0000","last_clean":"2024-04-25T13:43:15.677868+0000","last_became_active":"2024-04-25T13:43:13.785944+0000","last_became_peered":"2024-04-25T13:43:13.785944+0000","last_unstale":"2024-04-25T13:43:15.677868+0000","last_undegraded":"2024-04-25T13:43:15.677868+0000","last_fullsized":"2024-04-25T13:43:15.677868+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:04:57.072707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774808+0000","last_change":"2024-04-25T13:43:13.788256+0000","last_active":"2024-04-25T13:43:14.774808+0000","last_peered":"2024-04-25T13:43:14.774808+0000","last_clean":"2024-04-25T13:43:14.774808+0000","last_became_active":"2024-04-25T13:43:13.786000+0000","last_became_peered":"2024-04-25T13:43:13.786000+0000","last_unstale":"2024-04-25T13:43:14.774808+0000","last_undegraded":"2024-04-25T13:43:14.774808+0000","last_fullsized":"2024-04-25T13:43:14.774808+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:34.083545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680698+0000","last_change":"2024-04-25T13:43:13.820683+0000","last_active":"2024-04-25T13:43:15.680698+0000","last_peered":"2024-04-25T13:43:15.680698+0000","last_clean":"2024-04-25T13:43:15.680698+0000","last_became_active":"2024-04-25T13:43:13.817765+0000","last_became_peered":"2024-04-25T13:43:13.817765+0000","last_unstale":"2024-04-25T13:43:15.680698+0000","last_undegraded":"2024-04-25T13:43:15.680698+0000","last_fullsized":"2024-04-25T13:43:15.680698+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:54.910745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774485+0000","last_change":"2024-04-25T13:43:13.790111+0000","last_active":"2024-04-25T13:43:14.774485+0000","last_peered":"2024-04-25T13:43:14.774485+0000","last_clean":"2024-04-25T13:43:14.774485+0000","last_became_active":"2024-04-25T13:43:13.788836+0000","last_became_peered":"2024-04-25T13:43:13.788836+0000","last_unstale":"2024-04-25T13:43:14.774485+0000","last_undegraded":"2024-04-25T13:43:14.774485+0000","last_fullsized":"2024-04-25T13:43:14.774485+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:59.404044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680003+0000","last_change":"2024-04-25T13:43:13.821032+0000","last_active":"2024-04-25T13:43:15.680003+0000","last_peered":"2024-04-25T13:43:15.680003+0000","last_clean":"2024-04-25T13:43:15.680003+0000","last_became_active":"2024-04-25T13:43:13.812767+0000","last_became_peered":"2024-04-25T13:43:13.812767+0000","last_unstale":"2024-04-25T13:43:15.680003+0000","last_undegraded":"2024-04-25T13:43:15.680003+0000","last_fullsized":"2024-04-25T13:43:15.680003+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:33.784637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775365+0000","last_change":"2024-04-25T13:43:13.800390+0000","last_active":"2024-04-25T13:43:14.775365+0000","last_peered":"2024-04-25T13:43:14.775365+0000","last_clean":"2024-04-25T13:43:14.775365+0000","last_became_active":"2024-04-25T13:43:13.799617+0000","last_became_peered":"2024-04-25T13:43:13.799617+0000","last_unstale":"2024-04-25T13:43:14.775365+0000","last_undegraded":"2024-04-25T13:43:14.775365+0000","last_fullsized":"2024-04-25T13:43:14.775365+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:40:59.371407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679235+0000","last_change":"2024-04-25T13:43:13.785549+0000","last_active":"2024-04-25T13:43:15.679235+0000","last_peered":"2024-04-25T13:43:15.679235+0000","last_clean":"2024-04-25T13:43:15.679235+0000","last_became_active":"2024-04-25T13:43:13.785270+0000","last_became_peered":"2024-04-25T13:43:13.785270+0000","last_unstale":"2024-04-25T13:43:15.679235+0000","last_undegraded":"2024-04-25T13:43:15.679235+0000","last_fullsized":"2024-04-25T13:43:15.679235+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:14:28.446989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.682203+0000","last_change":"2024-04-25T13:43:13.816089+0000","last_active":"2024-04-25T13:43:15.682203+0000","last_peered":"2024-04-25T13:43:15.682203+0000","last_clean":"2024-04-25T13:43:15.682203+0000","last_became_active":"2024-04-25T13:43:13.814636+0000","last_became_peered":"2024-04-25T13:43:13.814636+0000","last_unstale":"2024-04-25T13:43:15.682203+0000","last_undegraded":"2024-04-25T13:43:15.682203+0000","last_fullsized":"2024-04-25T13:43:15.682203+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:41.578494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.083749+0000","last_change":"2024-04-25T13:43:13.795880+0000","last_active":"2024-04-25T13:43:15.083749+0000","last_peered":"2024-04-25T13:43:15.083749+0000","last_clean":"2024-04-25T13:43:15.083749+0000","last_became_active":"2024-04-25T13:43:13.795627+0000","last_became_peered":"2024-04-25T13:43:13.795627+0000","last_unstale":"2024-04-25T13:43:15.083749+0000","last_undegraded":"2024-04-25T13:43:15.083749+0000","last_fullsized":"2024-04-25T13:43:15.083749+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:17.776705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678241+0000","last_change":"2024-04-25T13:43:13.791722+0000","last_active":"2024-04-25T13:43:15.678241+0000","last_peered":"2024-04-25T13:43:15.678241+0000","last_clean":"2024-04-25T13:43:15.678241+0000","last_became_active":"2024-04-25T13:43:13.791383+0000","last_became_peered":"2024-04-25T13:43:13.791383+0000","last_unstale":"2024-04-25T13:43:15.678241+0000","last_undegraded":"2024-04-25T13:43:15.678241+0000","last_fullsized":"2024-04-25T13:43:15.678241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:05.498402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774282+0000","last_change":"2024-04-25T13:43:13.798434+0000","last_active":"2024-04-25T13:43:14.774282+0000","last_peered":"2024-04-25T13:43:14.774282+0000","last_clean":"2024-04-25T13:43:14.774282+0000","last_became_active":"2024-04-25T13:43:13.798106+0000","last_became_peered":"2024-04-25T13:43:13.798106+0000","last_unstale":"2024-04-25T13:43:14.774282+0000","last_undegraded":"2024-04-25T13:43:14.774282+0000","last_fullsized":"2024-04-25T13:43:14.774282+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:46.127076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680906+0000","last_change":"2024-04-25T13:43:13.794314+0000","last_active":"2024-04-25T13:43:15.680906+0000","last_peered":"2024-04-25T13:43:15.680906+0000","last_clean":"2024-04-25T13:43:15.680906+0000","last_became_active":"2024-04-25T13:43:13.793229+0000","last_became_peered":"2024-04-25T13:43:13.793229+0000","last_unstale":"2024-04-25T13:43:15.680906+0000","last_undegraded":"2024-04-25T13:43:15.680906+0000","last_fullsized":"2024-04-25T13:43:15.680906+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:27.155741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678332+0000","last_change":"2024-04-25T13:43:13.788926+0000","last_active":"2024-04-25T13:43:15.678332+0000","last_peered":"2024-04-25T13:43:15.678332+0000","last_clean":"2024-04-25T13:43:15.678332+0000","last_became_active":"2024-04-25T13:43:13.788684+0000","last_became_peered":"2024-04-25T13:43:13.788684+0000","last_unstale":"2024-04-25T13:43:15.678332+0000","last_undegraded":"2024-04-25T13:43:15.678332+0000","last_fullsized":"2024-04-25T13:43:15.678332+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:23.543536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084285+0000","last_change":"2024-04-25T13:43:13.784571+0000","last_active":"2024-04-25T13:43:15.084285+0000","last_peered":"2024-04-25T13:43:15.084285+0000","last_clean":"2024-04-25T13:43:15.084285+0000","last_became_active":"2024-04-25T13:43:13.784138+0000","last_became_peered":"2024-04-25T13:43:13.784138+0000","last_unstale":"2024-04-25T13:43:15.084285+0000","last_undegraded":"2024-04-25T13:43:15.084285+0000","last_fullsized":"2024-04-25T13:43:15.084285+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:32:58.207099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.083935+0000","last_change":"2024-04-25T13:43:13.792326+0000","last_active":"2024-04-25T13:43:15.083935+0000","last_peered":"2024-04-25T13:43:15.083935+0000","last_clean":"2024-04-25T13:43:15.083935+0000","last_became_active":"2024-04-25T13:43:13.792023+0000","last_became_peered":"2024-04-25T13:43:13.792023+0000","last_unstale":"2024-04-25T13:43:15.083935+0000","last_undegraded":"2024-04-25T13:43:15.083935+0000","last_fullsized":"2024-04-25T13:43:15.083935+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:24:30.930079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"62'4","reported_seq":18,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084232+0000","last_change":"2024-04-25T13:43:12.788812+0000","last_active":"2024-04-25T13:43:15.084232+0000","last_peered":"2024-04-25T13:43:15.084232+0000","last_clean":"2024-04-25T13:43:15.084232+0000","last_became_active":"2024-04-25T13:43:12.788387+0000","last_became_peered":"2024-04-25T13:43:12.788387+0000","last_unstale":"2024-04-25T13:43:15.084232+0000","last_undegraded":"2024-04-25T13:43:15.084232+0000","last_fullsized":"2024-04-25T13:43:15.084232+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:08.071317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.677875+0000","last_change":"2024-04-25T13:43:13.810119+0000","last_active":"2024-04-25T13:43:15.677875+0000","last_peered":"2024-04-25T13:43:15.677875+0000","last_clean":"2024-04-25T13:43:15.677875+0000","last_became_active":"2024-04-25T13:43:13.809917+0000","last_became_peered":"2024-04-25T13:43:13.809917+0000","last_unstale":"2024-04-25T13:43:15.677875+0000","last_undegraded":"2024-04-25T13:43:15.677875+0000","last_fullsized":"2024-04-25T13:43:15.677875+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:09.965677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775042+0000","last_change":"2024-04-25T13:43:12.786208+0000","last_active":"2024-04-25T13:43:14.775042+0000","last_peered":"2024-04-25T13:43:14.775042+0000","last_clean":"2024-04-25T13:43:14.775042+0000","last_became_active":"2024-04-25T13:43:12.785877+0000","last_became_peered":"2024-04-25T13:43:12.785877+0000","last_unstale":"2024-04-25T13:43:14.775042+0000","last_undegraded":"2024-04-25T13:43:14.775042+0000","last_fullsized":"2024-04-25T13:43:14.775042+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:47.367925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084138+0000","last_change":"2024-04-25T13:43:13.787314+0000","last_active":"2024-04-25T13:43:15.084138+0000","last_peered":"2024-04-25T13:43:15.084138+0000","last_clean":"2024-04-25T13:43:15.084138+0000","last_became_active":"2024-04-25T13:43:13.786751+0000","last_became_peered":"2024-04-25T13:43:13.786751+0000","last_unstale":"2024-04-25T13:43:15.084138+0000","last_undegraded":"2024-04-25T13:43:15.084138+0000","last_fullsized":"2024-04-25T13:43:15.084138+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:20.899919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084571+0000","last_change":"2024-04-25T13:43:12.788227+0000","last_active":"2024-04-25T13:43:15.084571+0000","last_peered":"2024-04-25T13:43:15.084571+0000","last_clean":"2024-04-25T13:43:15.084571+0000","last_became_active":"2024-04-25T13:43:12.787727+0000","last_became_peered":"2024-04-25T13:43:12.787727+0000","last_unstale":"2024-04-25T13:43:15.084571+0000","last_undegraded":"2024-04-25T13:43:15.084571+0000","last_fullsized":"2024-04-25T13:43:15.084571+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.546601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084276+0000","last_change":"2024-04-25T13:43:13.800754+0000","last_active":"2024-04-25T13:43:15.084276+0000","last_peered":"2024-04-25T13:43:15.084276+0000","last_clean":"2024-04-25T13:43:15.084276+0000","last_became_active":"2024-04-25T13:43:13.800461+0000","last_became_peered":"2024-04-25T13:43:13.800461+0000","last_unstale":"2024-04-25T13:43:15.084276+0000","last_undegraded":"2024-04-25T13:43:15.084276+0000","last_fullsized":"2024-04-25T13:43:15.084276+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:39:58.923099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.683032+0000","last_change":"2024-04-25T13:43:12.805761+0000","last_active":"2024-04-25T13:43:15.683032+0000","last_peered":"2024-04-25T13:43:15.683032+0000","last_clean":"2024-04-25T13:43:15.683032+0000","last_became_active":"2024-04-25T13:43:12.804624+0000","last_became_peered":"2024-04-25T13:43:12.804624+0000","last_unstale":"2024-04-25T13:43:15.683032+0000","last_undegraded":"2024-04-25T13:43:15.683032+0000","last_fullsized":"2024-04-25T13:43:15.683032+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:25.209671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774854+0000","last_change":"2024-04-25T13:43:13.786609+0000","last_active":"2024-04-25T13:43:14.774854+0000","last_peered":"2024-04-25T13:43:14.774854+0000","last_clean":"2024-04-25T13:43:14.774854+0000","last_became_active":"2024-04-25T13:43:13.786177+0000","last_became_peered":"2024-04-25T13:43:13.786177+0000","last_unstale":"2024-04-25T13:43:14.774854+0000","last_undegraded":"2024-04-25T13:43:14.774854+0000","last_fullsized":"2024-04-25T13:43:14.774854+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:29.419182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679187+0000","last_change":"2024-04-25T13:43:12.800849+0000","last_active":"2024-04-25T13:43:15.679187+0000","last_peered":"2024-04-25T13:43:15.679187+0000","last_clean":"2024-04-25T13:43:15.679187+0000","last_became_active":"2024-04-25T13:43:12.794945+0000","last_became_peered":"2024-04-25T13:43:12.794945+0000","last_unstale":"2024-04-25T13:43:15.679187+0000","last_undegraded":"2024-04-25T13:43:15.679187+0000","last_fullsized":"2024-04-25T13:43:15.679187+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:34.955241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774670+0000","last_change":"2024-04-25T13:43:13.790347+0000","last_active":"2024-04-25T13:43:14.774670+0000","last_peered":"2024-04-25T13:43:14.774670+0000","last_clean":"2024-04-25T13:43:14.774670+0000","last_became_active":"2024-04-25T13:43:13.789317+0000","last_became_peered":"2024-04-25T13:43:13.789317+0000","last_unstale":"2024-04-25T13:43:14.774670+0000","last_undegraded":"2024-04-25T13:43:14.774670+0000","last_fullsized":"2024-04-25T13:43:14.774670+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:29.371169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084545+0000","last_change":"2024-04-25T13:43:12.796111+0000","last_active":"2024-04-25T13:43:15.084545+0000","last_peered":"2024-04-25T13:43:15.084545+0000","last_clean":"2024-04-25T13:43:15.084545+0000","last_became_active":"2024-04-25T13:43:12.795866+0000","last_became_peered":"2024-04-25T13:43:12.795866+0000","last_unstale":"2024-04-25T13:43:15.084545+0000","last_undegraded":"2024-04-25T13:43:15.084545+0000","last_fullsized":"2024-04-25T13:43:15.084545+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:26:42.227084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775762+0000","last_change":"2024-04-25T13:43:13.813527+0000","last_active":"2024-04-25T13:43:14.775762+0000","last_peered":"2024-04-25T13:43:14.775762+0000","last_clean":"2024-04-25T13:43:14.775762+0000","last_became_active":"2024-04-25T13:43:13.812741+0000","last_became_peered":"2024-04-25T13:43:13.812741+0000","last_unstale":"2024-04-25T13:43:14.775762+0000","last_undegraded":"2024-04-25T13:43:14.775762+0000","last_fullsized":"2024-04-25T13:43:14.775762+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:16.581606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680482+0000","last_change":"2024-04-25T13:43:12.799619+0000","last_active":"2024-04-25T13:43:15.680482+0000","last_peered":"2024-04-25T13:43:15.680482+0000","last_clean":"2024-04-25T13:43:15.680482+0000","last_became_active":"2024-04-25T13:43:12.798541+0000","last_became_peered":"2024-04-25T13:43:12.798541+0000","last_unstale":"2024-04-25T13:43:15.680482+0000","last_undegraded":"2024-04-25T13:43:15.680482+0000","last_fullsized":"2024-04-25T13:43:15.680482+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:13.129296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774756+0000","last_change":"2024-04-25T13:43:13.795153+0000","last_active":"2024-04-25T13:43:14.774756+0000","last_peered":"2024-04-25T13:43:14.774756+0000","last_clean":"2024-04-25T13:43:14.774756+0000","last_became_active":"2024-04-25T13:43:13.794862+0000","last_became_peered":"2024-04-25T13:43:13.794862+0000","last_unstale":"2024-04-25T13:43:14.774756+0000","last_undegraded":"2024-04-25T13:43:14.774756+0000","last_fullsized":"2024-04-25T13:43:14.774756+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:16.523208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680162+0000","last_change":"2024-04-25T13:43:12.792810+0000","last_active":"2024-04-25T13:43:15.680162+0000","last_peered":"2024-04-25T13:43:15.680162+0000","last_clean":"2024-04-25T13:43:15.680162+0000","last_became_active":"2024-04-25T13:43:12.791521+0000","last_became_peered":"2024-04-25T13:43:12.791521+0000","last_unstale":"2024-04-25T13:43:15.680162+0000","last_undegraded":"2024-04-25T13:43:15.680162+0000","last_fullsized":"2024-04-25T13:43:15.680162+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:00.561477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678841+0000","last_change":"2024-04-25T13:43:13.792055+0000","last_active":"2024-04-25T13:43:15.678841+0000","last_peered":"2024-04-25T13:43:15.678841+0000","last_clean":"2024-04-25T13:43:15.678841+0000","last_became_active":"2024-04-25T13:43:13.789437+0000","last_became_peered":"2024-04-25T13:43:13.789437+0000","last_unstale":"2024-04-25T13:43:15.678841+0000","last_undegraded":"2024-04-25T13:43:15.678841+0000","last_fullsized":"2024-04-25T13:43:15.678841+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:04.921605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.682239+0000","last_change":"2024-04-25T13:43:12.804964+0000","last_active":"2024-04-25T13:43:15.682239+0000","last_peered":"2024-04-25T13:43:15.682239+0000","last_clean":"2024-04-25T13:43:15.682239+0000","last_became_active":"2024-04-25T13:43:12.802321+0000","last_became_peered":"2024-04-25T13:43:12.802321+0000","last_unstale":"2024-04-25T13:43:15.682239+0000","last_undegraded":"2024-04-25T13:43:15.682239+0000","last_fullsized":"2024-04-25T13:43:15.682239+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:00.347511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.681656+0000","last_change":"2024-04-25T13:43:13.806775+0000","last_active":"2024-04-25T13:43:15.681656+0000","last_peered":"2024-04-25T13:43:15.681656+0000","last_clean":"2024-04-25T13:43:15.681656+0000","last_became_active":"2024-04-25T13:43:13.802901+0000","last_became_peered":"2024-04-25T13:43:13.802901+0000","last_unstale":"2024-04-25T13:43:15.681656+0000","last_undegraded":"2024-04-25T13:43:15.681656+0000","last_fullsized":"2024-04-25T13:43:15.681656+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:22:16.007919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775815+0000","last_change":"2024-04-25T13:43:12.775532+0000","last_active":"2024-04-25T13:43:14.775815+0000","last_peered":"2024-04-25T13:43:14.775815+0000","last_clean":"2024-04-25T13:43:14.775815+0000","last_became_active":"2024-04-25T13:43:12.775272+0000","last_became_peered":"2024-04-25T13:43:12.775272+0000","last_unstale":"2024-04-25T13:43:14.775815+0000","last_undegraded":"2024-04-25T13:43:14.775815+0000","last_fullsized":"2024-04-25T13:43:14.775815+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:43.723262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678285+0000","last_change":"2024-04-25T13:43:13.804271+0000","last_active":"2024-04-25T13:43:15.678285+0000","last_peered":"2024-04-25T13:43:15.678285+0000","last_clean":"2024-04-25T13:43:15.678285+0000","last_became_active":"2024-04-25T13:43:13.803976+0000","last_became_peered":"2024-04-25T13:43:13.803976+0000","last_unstale":"2024-04-25T13:43:15.678285+0000","last_undegraded":"2024-04-25T13:43:15.678285+0000","last_fullsized":"2024-04-25T13:43:15.678285+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:16.407801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"62'3","reported_seq":17,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680773+0000","last_change":"2024-04-25T13:43:12.803808+0000","last_active":"2024-04-25T13:43:15.680773+0000","last_peered":"2024-04-25T13:43:15.680773+0000","last_clean":"2024-04-25T13:43:15.680773+0000","last_became_active":"2024-04-25T13:43:12.802099+0000","last_became_peered":"2024-04-25T13:43:12.802099+0000","last_unstale":"2024-04-25T13:43:15.680773+0000","last_undegraded":"2024-04-25T13:43:15.680773+0000","last_fullsized":"2024-04-25T13:43:15.680773+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:19.778713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680466+0000","last_change":"2024-04-25T13:43:13.822602+0000","last_active":"2024-04-25T13:43:15.680466+0000","last_peered":"2024-04-25T13:43:15.680466+0000","last_clean":"2024-04-25T13:43:15.680466+0000","last_became_active":"2024-04-25T13:43:13.816678+0000","last_became_peered":"2024-04-25T13:43:13.816678+0000","last_unstale":"2024-04-25T13:43:15.680466+0000","last_undegraded":"2024-04-25T13:43:15.680466+0000","last_fullsized":"2024-04-25T13:43:15.680466+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:42.935118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"62'6","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.681424+0000","last_change":"2024-04-25T13:43:12.800894+0000","last_active":"2024-04-25T13:43:15.681424+0000","last_peered":"2024-04-25T13:43:15.681424+0000","last_clean":"2024-04-25T13:43:15.681424+0000","last_became_active":"2024-04-25T13:43:12.796733+0000","last_became_peered":"2024-04-25T13:43:12.796733+0000","last_unstale":"2024-04-25T13:43:15.681424+0000","last_undegraded":"2024-04-25T13:43:15.681424+0000","last_fullsized":"2024-04-25T13:43:15.681424+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:40.922587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.682708+0000","last_change":"2024-04-25T13:43:13.810781+0000","last_active":"2024-04-25T13:43:15.682708+0000","last_peered":"2024-04-25T13:43:15.682708+0000","last_clean":"2024-04-25T13:43:15.682708+0000","last_became_active":"2024-04-25T13:43:13.779279+0000","last_became_peered":"2024-04-25T13:43:13.779279+0000","last_unstale":"2024-04-25T13:43:15.682708+0000","last_undegraded":"2024-04-25T13:43:15.682708+0000","last_fullsized":"2024-04-25T13:43:15.682708+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:56:24.217967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775066+0000","last_change":"2024-04-25T13:43:12.773012+0000","last_active":"2024-04-25T13:43:14.775066+0000","last_peered":"2024-04-25T13:43:14.775066+0000","last_clean":"2024-04-25T13:43:14.775066+0000","last_became_active":"2024-04-25T13:43:12.772654+0000","last_became_peered":"2024-04-25T13:43:12.772654+0000","last_unstale":"2024-04-25T13:43:14.775066+0000","last_undegraded":"2024-04-25T13:43:14.775066+0000","last_fullsized":"2024-04-25T13:43:14.775066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:28.560929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.681270+0000","last_change":"2024-04-25T13:43:13.810282+0000","last_active":"2024-04-25T13:43:15.681270+0000","last_peered":"2024-04-25T13:43:15.681270+0000","last_clean":"2024-04-25T13:43:15.681270+0000","last_became_active":"2024-04-25T13:43:13.808892+0000","last_became_peered":"2024-04-25T13:43:13.808892+0000","last_unstale":"2024-04-25T13:43:15.681270+0000","last_undegraded":"2024-04-25T13:43:15.681270+0000","last_fullsized":"2024-04-25T13:43:15.681270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:30.135921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775315+0000","last_change":"2024-04-25T13:43:12.804471+0000","last_active":"2024-04-25T13:43:14.775315+0000","last_peered":"2024-04-25T13:43:14.775315+0000","last_clean":"2024-04-25T13:43:14.775315+0000","last_became_active":"2024-04-25T13:43:12.803661+0000","last_became_peered":"2024-04-25T13:43:12.803661+0000","last_unstale":"2024-04-25T13:43:14.775315+0000","last_undegraded":"2024-04-25T13:43:14.775315+0000","last_fullsized":"2024-04-25T13:43:14.775315+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:46.077074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678260+0000","last_change":"2024-04-25T13:43:13.820265+0000","last_active":"2024-04-25T13:43:15.678260+0000","last_peered":"2024-04-25T13:43:15.678260+0000","last_clean":"2024-04-25T13:43:15.678260+0000","last_became_active":"2024-04-25T13:43:13.818689+0000","last_became_peered":"2024-04-25T13:43:13.818689+0000","last_unstale":"2024-04-25T13:43:15.678260+0000","last_undegraded":"2024-04-25T13:43:15.678260+0000","last_fullsized":"2024-04-25T13:43:15.678260+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:29.245790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679097+0000","last_change":"2024-04-25T13:43:12.800897+0000","last_active":"2024-04-25T13:43:15.679097+0000","last_peered":"2024-04-25T13:43:15.679097+0000","last_clean":"2024-04-25T13:43:15.679097+0000","last_became_active":"2024-04-25T13:43:12.798720+0000","last_became_peered":"2024-04-25T13:43:12.798720+0000","last_unstale":"2024-04-25T13:43:15.679097+0000","last_undegraded":"2024-04-25T13:43:15.679097+0000","last_fullsized":"2024-04-25T13:43:15.679097+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:40:49.485933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679612+0000","last_change":"2024-04-25T13:43:13.796545+0000","last_active":"2024-04-25T13:43:15.679612+0000","last_peered":"2024-04-25T13:43:15.679612+0000","last_clean":"2024-04-25T13:43:15.679612+0000","last_became_active":"2024-04-25T13:43:13.795808+0000","last_became_peered":"2024-04-25T13:43:13.795808+0000","last_unstale":"2024-04-25T13:43:15.679612+0000","last_undegraded":"2024-04-25T13:43:15.679612+0000","last_fullsized":"2024-04-25T13:43:15.679612+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:03.252395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"62'8","reported_seq":22,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.681125+0000","last_change":"2024-04-25T13:43:12.803174+0000","last_active":"2024-04-25T13:43:15.681125+0000","last_peered":"2024-04-25T13:43:15.681125+0000","last_clean":"2024-04-25T13:43:15.681125+0000","last_became_active":"2024-04-25T13:43:12.799480+0000","last_became_peered":"2024-04-25T13:43:12.799480+0000","last_unstale":"2024-04-25T13:43:15.681125+0000","last_undegraded":"2024-04-25T13:43:15.681125+0000","last_fullsized":"2024-04-25T13:43:15.681125+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:15.378920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.681500+0000","last_change":"2024-04-25T13:43:13.811057+0000","last_active":"2024-04-25T13:43:15.681500+0000","last_peered":"2024-04-25T13:43:15.681500+0000","last_clean":"2024-04-25T13:43:15.681500+0000","last_became_active":"2024-04-25T13:43:13.809730+0000","last_became_peered":"2024-04-25T13:43:13.809730+0000","last_unstale":"2024-04-25T13:43:15.681500+0000","last_undegraded":"2024-04-25T13:43:15.681500+0000","last_fullsized":"2024-04-25T13:43:15.681500+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:07.839933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775365+0000","last_change":"2024-04-25T13:43:12.776468+0000","last_active":"2024-04-25T13:43:14.775365+0000","last_peered":"2024-04-25T13:43:14.775365+0000","last_clean":"2024-04-25T13:43:14.775365+0000","last_became_active":"2024-04-25T13:43:12.776176+0000","last_became_peered":"2024-04-25T13:43:12.776176+0000","last_unstale":"2024-04-25T13:43:14.775365+0000","last_undegraded":"2024-04-25T13:43:14.775365+0000","last_fullsized":"2024-04-25T13:43:14.775365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:34.025195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679933+0000","last_change":"2024-04-25T13:43:13.780045+0000","last_active":"2024-04-25T13:43:15.679933+0000","last_peered":"2024-04-25T13:43:15.679933+0000","last_clean":"2024-04-25T13:43:15.679933+0000","last_became_active":"2024-04-25T13:43:13.777753+0000","last_became_peered":"2024-04-25T13:43:13.777753+0000","last_unstale":"2024-04-25T13:43:15.679933+0000","last_undegraded":"2024-04-25T13:43:15.679933+0000","last_fullsized":"2024-04-25T13:43:15.679933+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:57.474348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084885+0000","last_change":"2024-04-25T13:43:12.791524+0000","last_active":"2024-04-25T13:43:15.084885+0000","last_peered":"2024-04-25T13:43:15.084885+0000","last_clean":"2024-04-25T13:43:15.084885+0000","last_became_active":"2024-04-25T13:43:12.790749+0000","last_became_peered":"2024-04-25T13:43:12.790749+0000","last_unstale":"2024-04-25T13:43:15.084885+0000","last_undegraded":"2024-04-25T13:43:15.084885+0000","last_fullsized":"2024-04-25T13:43:15.084885+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:01.647027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.774832+0000","last_change":"2024-04-25T13:43:13.791447+0000","last_active":"2024-04-25T13:43:14.774832+0000","last_peered":"2024-04-25T13:43:14.774832+0000","last_clean":"2024-04-25T13:43:14.774832+0000","last_became_active":"2024-04-25T13:43:13.791181+0000","last_became_peered":"2024-04-25T13:43:13.791181+0000","last_unstale":"2024-04-25T13:43:14.774832+0000","last_undegraded":"2024-04-25T13:43:14.774832+0000","last_fullsized":"2024-04-25T13:43:14.774832+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:28.916106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"62'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.683389+0000","last_change":"2024-04-25T13:43:12.805392+0000","last_active":"2024-04-25T13:43:15.683389+0000","last_peered":"2024-04-25T13:43:15.683389+0000","last_clean":"2024-04-25T13:43:15.683389+0000","last_became_active":"2024-04-25T13:43:12.803123+0000","last_became_peered":"2024-04-25T13:43:12.803123+0000","last_unstale":"2024-04-25T13:43:15.683389+0000","last_undegraded":"2024-04-25T13:43:15.683389+0000","last_fullsized":"2024-04-25T13:43:15.683389+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:02:50.148719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.681889+0000","last_change":"2024-04-25T13:43:13.808558+0000","last_active":"2024-04-25T13:43:15.681889+0000","last_peered":"2024-04-25T13:43:15.681889+0000","last_clean":"2024-04-25T13:43:15.681889+0000","last_became_active":"2024-04-25T13:43:13.805818+0000","last_became_peered":"2024-04-25T13:43:13.805818+0000","last_unstale":"2024-04-25T13:43:15.681889+0000","last_undegraded":"2024-04-25T13:43:15.681889+0000","last_fullsized":"2024-04-25T13:43:15.681889+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:17.259969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084600+0000","last_change":"2024-04-25T13:43:12.788962+0000","last_active":"2024-04-25T13:43:15.084600+0000","last_peered":"2024-04-25T13:43:15.084600+0000","last_clean":"2024-04-25T13:43:15.084600+0000","last_became_active":"2024-04-25T13:43:12.788672+0000","last_became_peered":"2024-04-25T13:43:12.788672+0000","last_unstale":"2024-04-25T13:43:15.084600+0000","last_undegraded":"2024-04-25T13:43:15.084600+0000","last_fullsized":"2024-04-25T13:43:15.084600+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:31:02.875950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.680323+0000","last_change":"2024-04-25T13:43:13.798821+0000","last_active":"2024-04-25T13:43:15.680323+0000","last_peered":"2024-04-25T13:43:15.680323+0000","last_clean":"2024-04-25T13:43:15.680323+0000","last_became_active":"2024-04-25T13:43:13.789229+0000","last_became_peered":"2024-04-25T13:43:13.789229+0000","last_unstale":"2024-04-25T13:43:15.680323+0000","last_undegraded":"2024-04-25T13:43:15.680323+0000","last_fullsized":"2024-04-25T13:43:15.680323+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:43.567261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.776394+0000","last_change":"2024-04-25T13:43:12.791759+0000","last_active":"2024-04-25T13:43:14.776394+0000","last_peered":"2024-04-25T13:43:14.776394+0000","last_clean":"2024-04-25T13:43:14.776394+0000","last_became_active":"2024-04-25T13:43:12.791438+0000","last_became_peered":"2024-04-25T13:43:12.791438+0000","last_unstale":"2024-04-25T13:43:14.776394+0000","last_undegraded":"2024-04-25T13:43:14.776394+0000","last_fullsized":"2024-04-25T13:43:14.776394+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:50.470195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.083939+0000","last_change":"2024-04-25T13:43:13.797274+0000","last_active":"2024-04-25T13:43:15.083939+0000","last_peered":"2024-04-25T13:43:15.083939+0000","last_clean":"2024-04-25T13:43:15.083939+0000","last_became_active":"2024-04-25T13:43:13.796989+0000","last_became_peered":"2024-04-25T13:43:13.796989+0000","last_unstale":"2024-04-25T13:43:15.083939+0000","last_undegraded":"2024-04-25T13:43:15.083939+0000","last_fullsized":"2024-04-25T13:43:15.083939+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:19.411526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678576+0000","last_change":"2024-04-25T13:43:12.782117+0000","last_active":"2024-04-25T13:43:15.678576+0000","last_peered":"2024-04-25T13:43:15.678576+0000","last_clean":"2024-04-25T13:43:15.678576+0000","last_became_active":"2024-04-25T13:43:12.781837+0000","last_became_peered":"2024-04-25T13:43:12.781837+0000","last_unstale":"2024-04-25T13:43:15.678576+0000","last_undegraded":"2024-04-25T13:43:15.678576+0000","last_fullsized":"2024-04-25T13:43:15.678576+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:41:23.023716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.083747+0000","last_change":"2024-04-25T13:43:13.796644+0000","last_active":"2024-04-25T13:43:15.083747+0000","last_peered":"2024-04-25T13:43:15.083747+0000","last_clean":"2024-04-25T13:43:15.083747+0000","last_became_active":"2024-04-25T13:43:13.796106+0000","last_became_peered":"2024-04-25T13:43:13.796106+0000","last_unstale":"2024-04-25T13:43:15.083747+0000","last_undegraded":"2024-04-25T13:43:15.083747+0000","last_fullsized":"2024-04-25T13:43:15.083747+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:13.872580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.683738+0000","last_change":"2024-04-25T13:43:12.782326+0000","last_active":"2024-04-25T13:43:15.683738+0000","last_peered":"2024-04-25T13:43:15.683738+0000","last_clean":"2024-04-25T13:43:15.683738+0000","last_became_active":"2024-04-25T13:43:12.781030+0000","last_became_peered":"2024-04-25T13:43:12.781030+0000","last_unstale":"2024-04-25T13:43:15.683738+0000","last_undegraded":"2024-04-25T13:43:15.683738+0000","last_fullsized":"2024-04-25T13:43:15.683738+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:21.071437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679420+0000","last_change":"2024-04-25T13:43:13.788334+0000","last_active":"2024-04-25T13:43:15.679420+0000","last_peered":"2024-04-25T13:43:15.679420+0000","last_clean":"2024-04-25T13:43:15.679420+0000","last_became_active":"2024-04-25T13:43:13.787599+0000","last_became_peered":"2024-04-25T13:43:13.787599+0000","last_unstale":"2024-04-25T13:43:15.679420+0000","last_undegraded":"2024-04-25T13:43:15.679420+0000","last_fullsized":"2024-04-25T13:43:15.679420+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:05:32.363864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"62'2","reported_seq":16,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679046+0000","last_change":"2024-04-25T13:43:12.781235+0000","last_active":"2024-04-25T13:43:15.679046+0000","last_peered":"2024-04-25T13:43:15.679046+0000","last_clean":"2024-04-25T13:43:15.679046+0000","last_became_active":"2024-04-25T13:43:12.780825+0000","last_became_peered":"2024-04-25T13:43:12.780825+0000","last_unstale":"2024-04-25T13:43:15.679046+0000","last_undegraded":"2024-04-25T13:43:15.679046+0000","last_fullsized":"2024-04-25T13:43:15.679046+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:59.698572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678803+0000","last_change":"2024-04-25T13:43:13.793235+0000","last_active":"2024-04-25T13:43:15.678803+0000","last_peered":"2024-04-25T13:43:15.678803+0000","last_clean":"2024-04-25T13:43:15.678803+0000","last_became_active":"2024-04-25T13:43:13.791472+0000","last_became_peered":"2024-04-25T13:43:13.791472+0000","last_unstale":"2024-04-25T13:43:15.678803+0000","last_undegraded":"2024-04-25T13:43:15.678803+0000","last_fullsized":"2024-04-25T13:43:15.678803+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:18.040836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"63'1","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084975+0000","last_change":"2024-04-25T13:43:12.794225+0000","last_active":"2024-04-25T13:43:15.084975+0000","last_peered":"2024-04-25T13:43:15.084975+0000","last_clean":"2024-04-25T13:43:15.084975+0000","last_became_active":"2024-04-25T13:43:12.793919+0000","last_became_peered":"2024-04-25T13:43:12.793919+0000","last_unstale":"2024-04-25T13:43:15.084975+0000","last_undegraded":"2024-04-25T13:43:15.084975+0000","last_fullsized":"2024-04-25T13:43:15.084975+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:20.285230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679822+0000","last_change":"2024-04-25T13:43:13.791296+0000","last_active":"2024-04-25T13:43:15.679822+0000","last_peered":"2024-04-25T13:43:15.679822+0000","last_clean":"2024-04-25T13:43:15.679822+0000","last_became_active":"2024-04-25T13:43:13.790987+0000","last_became_peered":"2024-04-25T13:43:13.790987+0000","last_unstale":"2024-04-25T13:43:15.679822+0000","last_undegraded":"2024-04-25T13:43:15.679822+0000","last_fullsized":"2024-04-25T13:43:15.679822+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:03.945265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775550+0000","last_change":"2024-04-25T13:43:12.799608+0000","last_active":"2024-04-25T13:43:14.775550+0000","last_peered":"2024-04-25T13:43:14.775550+0000","last_clean":"2024-04-25T13:43:14.775550+0000","last_became_active":"2024-04-25T13:43:12.798852+0000","last_became_peered":"2024-04-25T13:43:12.798852+0000","last_unstale":"2024-04-25T13:43:14.775550+0000","last_undegraded":"2024-04-25T13:43:14.775550+0000","last_fullsized":"2024-04-25T13:43:14.775550+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:58.121010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.084425+0000","last_change":"2024-04-25T13:43:13.802020+0000","last_active":"2024-04-25T13:43:15.084425+0000","last_peered":"2024-04-25T13:43:15.084425+0000","last_clean":"2024-04-25T13:43:15.084425+0000","last_became_active":"2024-04-25T13:43:13.801619+0000","last_became_peered":"2024-04-25T13:43:13.801619+0000","last_unstale":"2024-04-25T13:43:15.084425+0000","last_undegraded":"2024-04-25T13:43:15.084425+0000","last_fullsized":"2024-04-25T13:43:15.084425+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:25.231558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.681202+0000","last_change":"2024-04-25T13:43:12.795584+0000","last_active":"2024-04-25T13:43:15.681202+0000","last_peered":"2024-04-25T13:43:15.681202+0000","last_clean":"2024-04-25T13:43:15.681202+0000","last_became_active":"2024-04-25T13:43:12.794735+0000","last_became_peered":"2024-04-25T13:43:12.794735+0000","last_unstale":"2024-04-25T13:43:15.681202+0000","last_undegraded":"2024-04-25T13:43:15.681202+0000","last_fullsized":"2024-04-25T13:43:15.681202+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:44.260213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775093+0000","last_change":"2024-04-25T13:43:13.782394+0000","last_active":"2024-04-25T13:43:14.775093+0000","last_peered":"2024-04-25T13:43:14.775093+0000","last_clean":"2024-04-25T13:43:14.775093+0000","last_became_active":"2024-04-25T13:43:13.781998+0000","last_became_peered":"2024-04-25T13:43:13.781998+0000","last_unstale":"2024-04-25T13:43:14.775093+0000","last_undegraded":"2024-04-25T13:43:14.775093+0000","last_fullsized":"2024-04-25T13:43:14.775093+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:53.009415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678793+0000","last_change":"2024-04-25T13:43:12.781249+0000","last_active":"2024-04-25T13:43:15.678793+0000","last_peered":"2024-04-25T13:43:15.678793+0000","last_clean":"2024-04-25T13:43:15.678793+0000","last_became_active":"2024-04-25T13:43:12.780946+0000","last_became_peered":"2024-04-25T13:43:12.780946+0000","last_unstale":"2024-04-25T13:43:15.678793+0000","last_undegraded":"2024-04-25T13:43:15.678793+0000","last_fullsized":"2024-04-25T13:43:15.678793+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:51.192678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.681209+0000","last_change":"2024-04-25T13:43:13.807819+0000","last_active":"2024-04-25T13:43:15.681209+0000","last_peered":"2024-04-25T13:43:15.681209+0000","last_clean":"2024-04-25T13:43:15.681209+0000","last_became_active":"2024-04-25T13:43:13.803878+0000","last_became_peered":"2024-04-25T13:43:13.803878+0000","last_unstale":"2024-04-25T13:43:15.681209+0000","last_undegraded":"2024-04-25T13:43:15.681209+0000","last_fullsized":"2024-04-25T13:43:15.681209+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:32.996455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.679553+0000","last_change":"2024-04-25T13:43:12.796960+0000","last_active":"2024-04-25T13:43:15.679553+0000","last_peered":"2024-04-25T13:43:15.679553+0000","last_clean":"2024-04-25T13:43:15.679553+0000","last_became_active":"2024-04-25T13:43:12.793772+0000","last_became_peered":"2024-04-25T13:43:12.793772+0000","last_unstale":"2024-04-25T13:43:15.679553+0000","last_undegraded":"2024-04-25T13:43:15.679553+0000","last_fullsized":"2024-04-25T13:43:15.679553+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:29.095952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.677602+0000","last_change":"2024-04-25T13:43:13.790383+0000","last_active":"2024-04-25T13:43:15.677602+0000","last_peered":"2024-04-25T13:43:15.677602+0000","last_clean":"2024-04-25T13:43:15.677602+0000","last_became_active":"2024-04-25T13:43:13.790010+0000","last_became_peered":"2024-04-25T13:43:13.790010+0000","last_unstale":"2024-04-25T13:43:15.677602+0000","last_undegraded":"2024-04-25T13:43:15.677602+0000","last_fullsized":"2024-04-25T13:43:15.677602+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:45.831546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.678141+0000","last_change":"2024-04-25T13:43:12.790794+0000","last_active":"2024-04-25T13:43:15.678141+0000","last_peered":"2024-04-25T13:43:15.678141+0000","last_clean":"2024-04-25T13:43:15.678141+0000","last_became_active":"2024-04-25T13:43:12.790423+0000","last_became_peered":"2024-04-25T13:43:12.790423+0000","last_unstale":"2024-04-25T13:43:15.678141+0000","last_undegraded":"2024-04-25T13:43:15.678141+0000","last_fullsized":"2024-04-25T13:43:15.678141+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:56:29.913341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.776143+0000","last_change":"2024-04-25T13:43:13.795253+0000","last_active":"2024-04-25T13:43:14.776143+0000","last_peered":"2024-04-25T13:43:14.776143+0000","last_clean":"2024-04-25T13:43:14.776143+0000","last_became_active":"2024-04-25T13:43:13.794989+0000","last_became_peered":"2024-04-25T13:43:13.794989+0000","last_unstale":"2024-04-25T13:43:14.776143+0000","last_undegraded":"2024-04-25T13:43:14.776143+0000","last_fullsized":"2024-04-25T13:43:14.776143+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:05.038050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775520+0000","last_change":"2024-04-25T13:43:12.784489+0000","last_active":"2024-04-25T13:43:14.775520+0000","last_peered":"2024-04-25T13:43:14.775520+0000","last_clean":"2024-04-25T13:43:14.775520+0000","last_became_active":"2024-04-25T13:43:12.783452+0000","last_became_peered":"2024-04-25T13:43:12.783452+0000","last_unstale":"2024-04-25T13:43:14.775520+0000","last_undegraded":"2024-04-25T13:43:14.775520+0000","last_fullsized":"2024-04-25T13:43:14.775520+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:13:41.934668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.775209+0000","last_change":"2024-04-25T13:43:13.783518+0000","last_active":"2024-04-25T13:43:14.775209+0000","last_peered":"2024-04-25T13:43:14.775209+0000","last_clean":"2024-04-25T13:43:14.775209+0000","last_became_active":"2024-04-25T13:43:13.783183+0000","last_became_peered":"2024-04-25T13:43:13.783183+0000","last_unstale":"2024-04-25T13:43:14.775209+0000","last_undegraded":"2024-04-25T13:43:14.775209+0000","last_fullsized":"2024-04-25T13:43:14.775209+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:12.754362+0000","last_clean_scrub_stamp":"2024-04-25T13:43:12.754362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:49:40.188966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:14.776036+0000","last_change":"2024-04-25T13:43:12.789704+0000","last_active":"2024-04-25T13:43:14.776036+0000","last_peered":"2024-04-25T13:43:14.776036+0000","last_clean":"2024-04-25T13:43:14.776036+0000","last_became_active":"2024-04-25T13:43:12.788972+0000","last_became_peered":"2024-04-25T13:43:12.788972+0000","last_unstale":"2024-04-25T13:43:14.776036+0000","last_undegraded":"2024-04-25T13:43:14.776036+0000","last_fullsized":"2024-04-25T13:43:14.776036+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:43:11.749037+0000","last_clean_scrub_stamp":"2024-04-25T13:43:11.749037+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:00.244874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":192,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-04-25T13:43:15.677524+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:43:15.677524+0000","last_peered":"2024-04-25T13:43:15.677524+0000","last_clean":"2024-04-25T13:43:15.677524+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:43:15.677524+0000","last_undegraded":"2024-04-25T13:43:15.677524+0000","last_fullsized":"2024-04-25T13:43:15.677524+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:28.137733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":892,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194420,"statfs":{"total":200000000,"available":199086358,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194996,"statfs":{"total":200000000,"available":199676886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":316,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194996,"statfs":{"total":200000000,"available":199676234,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574886,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":317,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194995,"statfs":{"total":200000000,"available":199675274,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574886,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194996,"statfs":{"total":200000000,"available":199676830,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194996,"statfs":{"total":200000000,"available":199676830,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574886,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194996,"statfs":{"total":200000000,"available":199676796,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":894,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194418,"statfs":{"total":200000000,"available":199084324,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:43:18.329 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:43:18.329 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:43:18.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.513+0000 7f77a0bb9640 1 Processor -- start 2024-04-25T13:43:18.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.513+0000 7f77a0bb9640 1 -- start start 2024-04-25T13:43:18.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.513+0000 7f77a0bb9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f779c0d06e0 0x7f779c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.513+0000 7f77a0bb9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f779c06e9e0 0x7f779c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.513+0000 7f77a0bb9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f779c0d0af0 con 0x7f779c161f90 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.513+0000 7f77a0bb9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f779c0d0d70 con 0x7f779c0d06e0 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.513+0000 7f77a0bb9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f779c0d1050 con 0x7f779c06e9e0 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.513+0000 7f7799d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f779c0d06e0 0x7f779c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f779a59c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f779c161f90 0x7f779c0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35928/0 (socket says 172.21.15.80:35928) 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f7799d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f779c0d06e0 0x7f779c06e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50394/0 (socket says 172.21.15.80:50394) 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f779a59c640 1 -- 172.21.15.80:0/1353048704 learned_addr learned my addr 172.21.15.80:0/1353048704 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f779ad9d640 1 --2- 172.21.15.80:0/1353048704 >> v2:172.21.15.87:3300/0 conn(0x7f779c06e9e0 0x7f779c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f7799d9b640 1 -- 172.21.15.80:0/1353048704 >> v2:172.21.15.87:3300/0 conn(0x7f779c06e9e0 msgr2=0x7f779c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f7799d9b640 1 --2- 172.21.15.80:0/1353048704 >> v2:172.21.15.87:3300/0 conn(0x7f779c06e9e0 0x7f779c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f7799d9b640 1 -- 172.21.15.80:0/1353048704 >> v1:172.21.15.80:6789/0 conn(0x7f779c161f90 legacy=0x7f779c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f779959a640 1 -- 172.21.15.80:0/1353048704 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 324299764 0 0) 0x7f778c001120 con 0x7f779c161f90 2024-04-25T13:43:18.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f7799d9b640 1 -- 172.21.15.80:0/1353048704 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c06dca0 con 0x7f779c0d06e0 2024-04-25T13:43:18.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f7799d9b640 1 --2- 172.21.15.80:0/1353048704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c0d06e0 0x7f779c06e4a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f779c0b4b80 tx=0x7f779001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94372784912c9d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f779959a640 1 -- 172.21.15.80:0/1353048704 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77900b0070 con 0x7f779c0d06e0 2024-04-25T13:43:18.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f779959a640 1 -- 172.21.15.80:0/1353048704 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77900ab600 con 0x7f779c0d06e0 2024-04-25T13:43:18.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f779959a640 1 -- 172.21.15.80:0/1353048704 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77900ab9d0 con 0x7f779c0d06e0 2024-04-25T13:43:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f77a0bb9640 1 -- 172.21.15.80:0/1353048704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c0d06e0 msgr2=0x7f779c06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/1353048704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c0d06e0 0x7f779c06e4a0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f779c0b4b80 tx=0x7f779001c330 comp rx=0 tx=0).stop 2024-04-25T13:43:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f77a0bb9640 1 -- 172.21.15.80:0/1353048704 shutdown_connections 2024-04-25T13:43:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/1353048704 >> v2:172.21.15.87:3300/0 conn(0x7f779c06e9e0 0x7f779c06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/1353048704 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c0d06e0 0x7f779c06e4a0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f77a0bb9640 1 -- 172.21.15.80:0/1353048704 >> 172.21.15.80:0/1353048704 conn(0x7f779c06d080 msgr2=0x7f779c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f77a0bb9640 1 -- 172.21.15.80:0/1353048704 shutdown_connections 2024-04-25T13:43:18.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.517+0000 7f77a0bb9640 1 -- 172.21.15.80:0/1353048704 wait complete. 2024-04-25T13:43:18.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 Processor -- start 2024-04-25T13:43:18.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 -- start start 2024-04-25T13:43:18.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c06e9e0 0x7f779c13ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f779c0d06e0 0x7f779c1fba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f779c161f90 0x7f779c1637c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f779c1447a0 con 0x7f779c06e9e0 2024-04-25T13:43:18.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f779c144620 con 0x7f779c161f90 2024-04-25T13:43:18.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f779c16b510 con 0x7f779c0d06e0 2024-04-25T13:43:18.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c06e9e0 0x7f779c13ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c06e9e0 0x7f779c13ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50396/0 (socket says 172.21.15.80:50396) 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 -- 172.21.15.80:0/2525197733 learned_addr learned my addr 172.21.15.80:0/2525197733 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f7799d9b640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f779c0d06e0 0x7f779c1fba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779ad9d640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f779c161f90 0x7f779c1637c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 -- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f779c0d06e0 msgr2=0x7f779c1fba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f779c0d06e0 0x7f779c1fba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 -- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f779c161f90 msgr2=0x7f779c1637c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f779c161f90 0x7f779c1637c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 -- 172.21.15.80:0/2525197733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c162ed0 con 0x7f779c06e9e0 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f779a59c640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c06e9e0 0x7f779c13ffa0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c3e0 tx=0x7f778c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f778affd640 1 -- 172.21.15.80:0/2525197733 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c0074e0 con 0x7f779c06e9e0 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f778affd640 1 -- 172.21.15.80:0/2525197733 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f778c0047f0 con 0x7f779c06e9e0 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c1631b0 con 0x7f779c06e9e0 2024-04-25T13:43:18.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f778affd640 1 -- 172.21.15.80:0/2525197733 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c004be0 con 0x7f779c06e9e0 2024-04-25T13:43:18.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c164e70 con 0x7f779c06e9e0 2024-04-25T13:43:18.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.521+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f779c06a3a0 con 0x7f779c06e9e0 2024-04-25T13:43:18.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.525+0000 7f778affd640 1 -- 172.21.15.80:0/2525197733 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f778c011070 con 0x7f779c06e9e0 2024-04-25T13:43:18.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.525+0000 7f778affd640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f777c06f400 0x7f777c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.525+0000 7f778affd640 1 -- 172.21.15.80:0/2525197733 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f778c08ed60 con 0x7f779c06e9e0 2024-04-25T13:43:18.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.525+0000 7f7799d9b640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f777c06f400 0x7f777c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.525+0000 7f7799d9b640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f777c06f400 0x7f777c071910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77900ad000 tx=0x7f7790008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.529+0000 7f778affd640 1 -- 172.21.15.80:0/2525197733 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f779c0d0af0 con 0x7f779c06e9e0 2024-04-25T13:43:18.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.737+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f779c0cf890 con 0x7f779c06e9e0 2024-04-25T13:43:18.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.741+0000 7f778affd640 1 -- 172.21.15.80:0/2525197733 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f778c05bd70 con 0x7f779c06e9e0 2024-04-25T13:43:18.744 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:43:18.744 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:43:18.744 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:43:18.744 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:43:18.745 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:43:18.745 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:43:18.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f777c06f400 msgr2=0x7f777c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f777c06f400 0x7f777c071910 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77900ad000 tx=0x7f7790008040 comp rx=0 tx=0).stop 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c06e9e0 msgr2=0x7f779c13ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c06e9e0 0x7f779c13ffa0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c3e0 tx=0x7f778c00c8a0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 shutdown_connections 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f777c06f400 0x7f777c071910 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f779c161f90 0x7f779c1637c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f779c0d06e0 0x7f779c1fba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 --2- 172.21.15.80:0/2525197733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f779c06e9e0 0x7f779c13ffa0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 >> 172.21.15.80:0/2525197733 conn(0x7f779c06d080 msgr2=0x7f779c0b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 shutdown_connections 2024-04-25T13:43:18.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.745+0000 7f77a0bb9640 1 -- 172.21.15.80:0/2525197733 wait complete. 2024-04-25T13:43:18.766 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:43:18.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd2710e5640 1 Processor -- start 2024-04-25T13:43:18.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd2710e5640 1 -- start start 2024-04-25T13:43:18.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd2710e5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0cf5a0 0x7fd26c0cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd2710e5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd26c0d23d0 0x7fd26c0d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd2710e5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd26c0cfef0 con 0x7fd26c0d04b0 2024-04-25T13:43:18.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd2710e5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd26c0b4680 con 0x7fd26c0d23d0 2024-04-25T13:43:18.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd2710e5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd26c0d1f40 con 0x7fd26c0cf5a0 2024-04-25T13:43:18.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd26b59e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd26c0d23d0 0x7fd26c0d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd26a59c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd26c0d04b0 0x7fd26c0d08c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35936/0 (socket says 172.21.15.80:35936) 2024-04-25T13:43:18.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd26a59c640 1 -- 172.21.15.80:0/1555731419 learned_addr learned my addr 172.21.15.80:0/1555731419 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:18.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd26ad9d640 1 --2- 172.21.15.80:0/1555731419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0cf5a0 0x7fd26c0cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd26b59e640 1 -- 172.21.15.80:0/1555731419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0cf5a0 msgr2=0x7fd26c0cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:18.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd26b59e640 1 --2- 172.21.15.80:0/1555731419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0cf5a0 0x7fd26c0cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd26b59e640 1 -- 172.21.15.80:0/1555731419 >> v1:172.21.15.80:6789/0 conn(0x7fd26c0d04b0 legacy=0x7fd26c0d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd26b59e640 1 -- 172.21.15.80:0/1555731419 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c0d1c10 con 0x7fd26c0d23d0 2024-04-25T13:43:18.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.909+0000 7fd269d9b640 1 -- 172.21.15.80:0/1555731419 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3965988387 0 0) 0x7fd260001120 con 0x7fd26c0d04b0 2024-04-25T13:43:18.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd26b59e640 1 --2- 172.21.15.80:0/1555731419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c0d27e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0097c0 tx=0x7fd25c01c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664cbdccd75033ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd269d9b640 1 -- 172.21.15.80:0/1555731419 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0b0070 con 0x7fd26c0d23d0 2024-04-25T13:43:18.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd269d9b640 1 -- 172.21.15.80:0/1555731419 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25c0ab600 con 0x7fd26c0d23d0 2024-04-25T13:43:18.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd269d9b640 1 -- 172.21.15.80:0/1555731419 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0ab9d0 con 0x7fd26c0d23d0 2024-04-25T13:43:18.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd2710e5640 1 -- 172.21.15.80:0/1555731419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 msgr2=0x7fd26c0d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1555731419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c0d27e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0097c0 tx=0x7fd25c01c4c0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd2710e5640 1 -- 172.21.15.80:0/1555731419 shutdown_connections 2024-04-25T13:43:18.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1555731419 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c0d27e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1555731419 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0cf5a0 0x7fd26c0cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd2710e5640 1 -- 172.21.15.80:0/1555731419 >> 172.21.15.80:0/1555731419 conn(0x7fd26c0d3a60 msgr2=0x7fd26c0d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:18.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd2710e5640 1 -- 172.21.15.80:0/1555731419 shutdown_connections 2024-04-25T13:43:18.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.913+0000 7fd2710e5640 1 -- 172.21.15.80:0/1555731419 wait complete. 2024-04-25T13:43:18.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 Processor -- start 2024-04-25T13:43:18.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 -- start start 2024-04-25T13:43:18.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd26c0cf5a0 0x7fd26c14ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0d04b0 0x7fd26c160f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c164190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c0b4280 con 0x7fd26c0d23d0 2024-04-25T13:43:18.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c0d0fd0 con 0x7fd26c0cf5a0 2024-04-25T13:43:18.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd26c0b4400 con 0x7fd26c0d04b0 2024-04-25T13:43:18.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c164190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c164190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50424/0 (socket says 172.21.15.80:50424) 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 -- 172.21.15.80:0/1221487210 learned_addr learned my addr 172.21.15.80:0/1221487210 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26a59c640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0d04b0 0x7fd26c160f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26ad9d640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd26c0cf5a0 0x7fd26c14ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 -- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0d04b0 msgr2=0x7fd26c160f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0d04b0 0x7fd26c160f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 -- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd26c0cf5a0 msgr2=0x7fd26c14ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd26c0cf5a0 0x7fd26c14ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 -- 172.21.15.80:0/1221487210 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c075840 con 0x7fd26c0d23d0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26a59c640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0d04b0 0x7fd26c160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26ad9d640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd26c0cf5a0 0x7fd26c14ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd26b59e640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c164190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0ae040 tx=0x7fd25c019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2537fe640 1 -- 172.21.15.80:0/1221487210 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0b0040 con 0x7fd26c0d23d0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2537fe640 1 -- 172.21.15.80:0/1221487210 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd25c0ba070 con 0x7fd26c0d23d0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26c15fa90 con 0x7fd26c0d23d0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2537fe640 1 -- 172.21.15.80:0/1221487210 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0b4830 con 0x7fd26c0d23d0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26c160050 con 0x7fd26c0d23d0 2024-04-25T13:43:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.917+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26c06a3a0 con 0x7fd26c0d23d0 2024-04-25T13:43:18.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.921+0000 7fd2537fe640 1 -- 172.21.15.80:0/1221487210 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd25c0ab4c0 con 0x7fd26c0d23d0 2024-04-25T13:43:18.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.921+0000 7fd2537fe640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd25806f400 0x7fd258071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:18.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.921+0000 7fd2537fe640 1 -- 172.21.15.80:0/1221487210 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fd25c129110 con 0x7fd26c0d23d0 2024-04-25T13:43:18.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.921+0000 7fd26ad9d640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd25806f400 0x7fd258071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:18.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.921+0000 7fd26ad9d640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd25806f400 0x7fd258071910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd26c06fb10 tx=0x7fd26400b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:18.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:18.925+0000 7fd2537fe640 1 -- 172.21.15.80:0/1221487210 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd25c0160b0 con 0x7fd26c0d23d0 2024-04-25T13:43:19.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.121+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd26c0d4830 con 0x7fd26c0d23d0 2024-04-25T13:43:19.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.121+0000 7fd2537fe640 1 -- 172.21.15.80:0/1221487210 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7fd25c0f6010 con 0x7fd26c0d23d0 2024-04-25T13:43:19.124 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:19.124 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1185745761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1185745761},{"type":"v1","addr":"172.21.15.80:6835","nonce":1185745761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2524166918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2524166918},{"type":"v1","addr":"172.21.15.87:6835","nonce":2524166918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3637133191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3637133191},{"type":"v1","addr":"172.21.15.87:6837","nonce":3637133191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:43:13.179327+0000","modified":"2024-04-25T13:43:16.230820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6834/1437895704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1437895704},{"type":"v1","addr":"172.21.15.80:6834","nonce":1437895704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6195,"qdb_cluster":[6195]},"id":4}]} 2024-04-25T13:43:19.125 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 33 2024-04-25T13:43:19.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd25806f400 msgr2=0x7fd258071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd25806f400 0x7fd258071910 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd26c06fb10 tx=0x7fd26400b040 comp rx=0 tx=0).stop 2024-04-25T13:43:19.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 msgr2=0x7fd26c164190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c164190 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0ae040 tx=0x7fd25c019260 comp rx=0 tx=0).stop 2024-04-25T13:43:19.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 shutdown_connections 2024-04-25T13:43:19.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd25806f400 0x7fd258071910 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd26c0d23d0 0x7fd26c164190 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd26c0d04b0 0x7fd26c160f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 --2- 172.21.15.80:0/1221487210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd26c0cf5a0 0x7fd26c14ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 >> 172.21.15.80:0/1221487210 conn(0x7fd26c0d3a60 msgr2=0x7fd26c0710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:19.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 shutdown_connections 2024-04-25T13:43:19.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.125+0000 7fd2710e5640 1 -- 172.21.15.80:0/1221487210 wait complete. 2024-04-25T13:43:19.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:43:13.179327+0000', 'modified': '2024-04-25T13:43:16.230820+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6195}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6195': {'gid': 6195, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6834/1437895704', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6832', 'nonce': 1437895704}, {'type': 'v1', 'addr': '172.21.15.80:6834', 'nonce': 1437895704}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6195, 'qdb_cluster': [6195]} 2024-04-25T13:43:19.549 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:43:19.549 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T13:43:19.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.701+0000 7f112004f640 1 Processor -- start 2024-04-25T13:43:19.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.701+0000 7f112004f640 1 -- start start 2024-04-25T13:43:19.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f112004f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f111806d000 0x7f1118068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f112004f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f11181622c0 0x7f1118168d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f112004f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11180d2490 con 0x7f1118068ad0 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f112004f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f11180d2000 con 0x7f11181622c0 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f112004f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1118068ee0 con 0x7f111806d000 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111e5ec640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f11181622c0 0x7f1118168d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111d5ea640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1118068ad0 0x7f1118161bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:35938/0 (socket says 172.21.15.80:35938) 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111d5ea640 1 -- 172.21.15.80:0/84236822 learned_addr learned my addr 172.21.15.80:0/84236822 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111ddeb640 1 --2- 172.21.15.80:0/84236822 >> v2:172.21.15.87:3300/0 conn(0x7f111806d000 0x7f1118068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111e5ec640 1 -- 172.21.15.80:0/84236822 >> v2:172.21.15.87:3300/0 conn(0x7f111806d000 msgr2=0x7f1118068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111e5ec640 1 --2- 172.21.15.80:0/84236822 >> v2:172.21.15.87:3300/0 conn(0x7f111806d000 0x7f1118068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111e5ec640 1 -- 172.21.15.80:0/84236822 >> v1:172.21.15.80:6789/0 conn(0x7f1118068ad0 legacy=0x7f1118161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111e5ec640 1 -- 172.21.15.80:0/84236822 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11180d13f0 con 0x7f11181622c0 2024-04-25T13:43:19.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111cde9640 1 -- 172.21.15.80:0/84236822 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2258406203 0 0) 0x7f1108001120 con 0x7f1118068ad0 2024-04-25T13:43:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111e5ec640 1 --2- 172.21.15.80:0/84236822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11181622c0 0x7f1118168d30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1118071dd0 tx=0x7f111001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a778ed784a7bc5a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111cde9640 1 -- 172.21.15.80:0/84236822 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100b2070 con 0x7f11181622c0 2024-04-25T13:43:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111cde9640 1 -- 172.21.15.80:0/84236822 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11100a59c0 con 0x7f11181622c0 2024-04-25T13:43:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.705+0000 7f111cde9640 1 -- 172.21.15.80:0/84236822 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11100a5d90 con 0x7f11181622c0 2024-04-25T13:43:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- 172.21.15.80:0/84236822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11181622c0 msgr2=0x7f1118168d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 --2- 172.21.15.80:0/84236822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11181622c0 0x7f1118168d30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1118071dd0 tx=0x7f111001c900 comp rx=0 tx=0).stop 2024-04-25T13:43:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- 172.21.15.80:0/84236822 shutdown_connections 2024-04-25T13:43:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 --2- 172.21.15.80:0/84236822 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f11181622c0 0x7f1118168d30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 --2- 172.21.15.80:0/84236822 >> v2:172.21.15.87:3300/0 conn(0x7f111806d000 0x7f1118068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- 172.21.15.80:0/84236822 >> 172.21.15.80:0/84236822 conn(0x7f11180d28f0 msgr2=0x7f11180d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- 172.21.15.80:0/84236822 shutdown_connections 2024-04-25T13:43:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- 172.21.15.80:0/84236822 wait complete. 2024-04-25T13:43:19.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 Processor -- start 2024-04-25T13:43:19.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- start start 2024-04-25T13:43:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1118068ad0 0x7f111815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f111806d000 0x7f111815ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11181622c0 0x7f11181604f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f111806d410 con 0x7f1118068ad0 2024-04-25T13:43:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f11180d10a0 con 0x7f11181622c0 2024-04-25T13:43:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f112004f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f111806fed0 con 0x7f111806d000 2024-04-25T13:43:19.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f111ddeb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1118068ad0 0x7f111815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f111ddeb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1118068ad0 0x7f111815a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50428/0 (socket says 172.21.15.80:50428) 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.709+0000 7f111ddeb640 1 -- 172.21.15.80:0/3192404777 learned_addr learned my addr 172.21.15.80:0/3192404777 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111d5ea640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f111806d000 0x7f111815ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111e5ec640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11181622c0 0x7f11181604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111ddeb640 1 -- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f111806d000 msgr2=0x7f111815ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111ddeb640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f111806d000 0x7f111815ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111ddeb640 1 -- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11181622c0 msgr2=0x7f11181604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111ddeb640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11181622c0 0x7f11181604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111ddeb640 1 -- 172.21.15.80:0/3192404777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f111815b1a0 con 0x7f1118068ad0 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111d5ea640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f111806d000 0x7f111815ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111ddeb640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1118068ad0 0x7f111815a830 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f111400d5d0 tx=0x7f111400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f110e7fc640 1 -- 172.21.15.80:0/3192404777 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114004240 con 0x7f1118068ad0 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f110e7fc640 1 -- 172.21.15.80:0/3192404777 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f11140043e0 con 0x7f1118068ad0 2024-04-25T13:43:19.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f112004f640 1 -- 172.21.15.80:0/3192404777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1118157c20 con 0x7f1118068ad0 2024-04-25T13:43:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f110e7fc640 1 -- 172.21.15.80:0/3192404777 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1114003840 con 0x7f1118068ad0 2024-04-25T13:43:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f112004f640 1 -- 172.21.15.80:0/3192404777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1118159780 con 0x7f1118068ad0 2024-04-25T13:43:19.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f110e7fc640 1 -- 172.21.15.80:0/3192404777 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f11140039e0 con 0x7f1118068ad0 2024-04-25T13:43:19.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f110e7fc640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f10fc06f400 0x7f10fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f110e7fc640 1 -- 172.21.15.80:0/3192404777 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f111408f570 con 0x7f1118068ad0 2024-04-25T13:43:19.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f111d5ea640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f10fc06f400 0x7f10fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f11273ac000 1 Processor -- start 2024-04-25T13:43:19.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.713+0000 7f11273ac000 1 -- start start 2024-04-25T13:43:19.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f10fc06f400 0x7f10fc071910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1108000f70 tx=0x7f1108009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47be420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 0x556ec47be960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x556ec462d160 con 0x556ec464bea0 2024-04-25T13:43:19.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556ec45c2e20 con 0x556ec465fc20 2024-04-25T13:43:19.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x556ec457ba20 con 0x556ec4244280 2024-04-25T13:43:19.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47be420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47be420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50438/0 (socket says 172.21.15.80:50438) 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 -- 172.21.15.80:0/605351347 learned_addr learned my addr 172.21.15.80:0/605351347 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111e5ec640 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 0x556ec47be960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111ddeb640 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 -- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 msgr2=0x556ec47be960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 0x556ec47be960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 -- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 msgr2=0x556ec47bdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47bdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 -- 172.21.15.80:0/605351347 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec4580100 con 0x556ec464bea0 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111ddeb640 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f111d5ea640 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47be420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1108007360 tx=0x7f1108010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d04a87ce598cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f10fb7fe640 1 -- 172.21.15.80:0/605351347 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f110800dbb0 con 0x556ec464bea0 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f10fb7fe640 1 -- 172.21.15.80:0/605351347 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110800dd50 con 0x556ec464bea0 2024-04-25T13:43:19.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f10fb7fe640 1 -- 172.21.15.80:0/605351347 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108020650 con 0x556ec464bea0 2024-04-25T13:43:19.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- 172.21.15.80:0/605351347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 msgr2=0x556ec47be420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47be420 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f1108007360 tx=0x7f1108010000 comp rx=0 tx=0).stop 2024-04-25T13:43:19.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- 172.21.15.80:0/605351347 shutdown_connections 2024-04-25T13:43:19.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 0x556ec47be960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47be420 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 --2- 172.21.15.80:0/605351347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- 172.21.15.80:0/605351347 >> 172.21.15.80:0/605351347 conn(0x556ec4283ff0 msgr2=0x556ec47734c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- 172.21.15.80:0/605351347 shutdown_connections 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- 172.21.15.80:0/605351347 wait complete. 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 -- start start 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.717+0000 7f11273ac000 1 Processor -- start 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec47c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 0x556ec47c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 0x556ec47800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x556ec45d0220 con 0x556ec4244280 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556ec4566ea0 con 0x556ec464bea0 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x556ec45bdb00 con 0x556ec465fc20 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 10 client.0 _ll_register_callbacks cb 0x556ec45cfdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec47c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec47c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50446/0 (socket says 172.21.15.80:50446) 2024-04-25T13:43:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 -- 172.21.15.80:0/2338326175 learned_addr learned my addr 172.21.15.80:0/2338326175 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111d5ea640 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 0x556ec47c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111e5ec640 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 0x556ec47800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 -- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 msgr2=0x556ec47800c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 0x556ec47800c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 -- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 msgr2=0x556ec47c53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 0x556ec47c53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 -- 172.21.15.80:0/2338326175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec4562500 con 0x556ec4244280 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111e5ec640 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 0x556ec47800c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111ddeb640 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec47c4e60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1114004660 tx=0x7f111405c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a397ab27e672ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 1 -- 172.21.15.80:0/2338326175 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111408b060 con 0x556ec4244280 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- 172.21.15.80:0/2338326175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110800dd50 con 0x556ec4244280 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 1 -- 172.21.15.80:0/2338326175 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f111401c360 con 0x556ec4244280 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 1 -- 172.21.15.80:0/2338326175 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f1114064bc0 con 0x556ec4244280 2024-04-25T13:43:19.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 10 client.6348 fetch_fsmap learned FSMap version 33 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- 172.21.15.80:0/2338326175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x556ec4548ea0 con 0x556ec4244280 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 1 -- 172.21.15.80:0/2338326175 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110800dd50 con 0x556ec4244280 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 1 -- 172.21.15.80:0/2338326175 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f1114013070 con 0x556ec4244280 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 10 client.6348 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 10 client.6348 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 20 client.6348 populate_metadata read hostname 'smithi080' 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] conn(0x556ec42687d0 0x556ec4780e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 4 client.6348 mds_command: new command op to 6195 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- 172.21.15.80:0/2338326175 --> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ec455bbe0 con 0x556ec42687d0 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111d5ea640 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] conn(0x556ec42687d0 0x556ec4780e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 10 client.6348 ms_handle_connect on v2:172.21.15.80:6832/1437895704 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f111d5ea640 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] conn(0x556ec42687d0 0x556ec4780e00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8a4dcbefcb9993 server_cookie=a3f3ab8d153aefe in_seq=0 out_seq=0 2024-04-25T13:43:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 1 -- 172.21.15.80:0/2338326175 <== mds.0 v2:172.21.15.80:6832/1437895704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ec455bbe0 con 0x556ec42687d0 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 10 client.6348 handle_command_reply: tid=0 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 client.6348 shutdown 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- 172.21.15.80:0/2338326175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 msgr2=0x556ec47c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec47c4e60 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f1114004660 tx=0x7f111405c520 comp rx=0 tx=0).stop 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- 172.21.15.80:0/2338326175 shutdown_connections 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] conn(0x556ec42687d0 0x556ec4780e00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 0x556ec47800c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 0x556ec47c53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 --2- 172.21.15.80:0/2338326175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec47c4e60 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- 172.21.15.80:0/2338326175 >> 172.21.15.80:0/2338326175 conn(0x556ec4283ff0 msgr2=0x556ec4773c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f10f8ff9640 0 client.6348 ms_handle_reset on v2:172.21.15.80:6832/1437895704 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- 172.21.15.80:0/2338326175 shutdown_connections 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 1 -- 172.21.15.80:0/2338326175 wait complete. 2024-04-25T13:43:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.721+0000 7f11273ac000 20 client.6348 trim_cache size 0 max 16384 2024-04-25T13:43:19.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f11273ac000 1 Processor -- start 2024-04-25T13:43:19.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f11273ac000 1 -- start start 2024-04-25T13:43:19.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f11273ac000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 0x556ec47ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f11273ac000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47d3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f11273ac000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f11273ac000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1118157480 con 0x556ec464bea0 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f11273ac000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f11181567e0 con 0x556ec465fc20 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f11273ac000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1118156e30 con 0x556ec4244280 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f111d5ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47d3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f111d5ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47d3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50452/0 (socket says 172.21.15.80:50452) 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.741+0000 7f111d5ea640 1 -- 172.21.15.80:0/3368794967 learned_addr learned my addr 172.21.15.80:0/3368794967 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111e5ec640 1 --2- 172.21.15.80:0/3368794967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111d5ea640 1 -- 172.21.15.80:0/3368794967 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 msgr2=0x556ec47ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111d5ea640 1 --2- 172.21.15.80:0/3368794967 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 0x556ec47ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111d5ea640 1 -- 172.21.15.80:0/3368794967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 msgr2=0x556ec47d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111d5ea640 1 --2- 172.21.15.80:0/3368794967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111d5ea640 1 -- 172.21.15.80:0/3368794967 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ec4780430 con 0x556ec464bea0 2024-04-25T13:43:19.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111d5ea640 1 --2- 172.21.15.80:0/3368794967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47d3ff0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1108013000 tx=0x7f1108007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0ae3a5eba41111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f10faffd640 1 -- 172.21.15.80:0/3368794967 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108034070 con 0x556ec464bea0 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f10faffd640 1 -- 172.21.15.80:0/3368794967 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110800ddf0 con 0x556ec464bea0 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f10faffd640 1 -- 172.21.15.80:0/3368794967 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1108020670 con 0x556ec464bea0 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- 172.21.15.80:0/3368794967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 msgr2=0x556ec47d3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 --2- 172.21.15.80:0/3368794967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47d3ff0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f1108013000 tx=0x7f1108007000 comp rx=0 tx=0).stop 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- 172.21.15.80:0/3368794967 shutdown_connections 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 --2- 172.21.15.80:0/3368794967 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec465fc20 0x556ec47d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 --2- 172.21.15.80:0/3368794967 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec464bea0 0x556ec47d3ff0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 --2- 172.21.15.80:0/3368794967 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec4244280 0x556ec47ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- 172.21.15.80:0/3368794967 >> 172.21.15.80:0/3368794967 conn(0x556ec4283ff0 msgr2=0x556ec477b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- 172.21.15.80:0/3368794967 shutdown_connections 2024-04-25T13:43:19.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- 172.21.15.80:0/3368794967 wait complete. 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- start start 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 Processor -- start 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec4834790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 0x556ec4834cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 0x556ec47c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x556ec45c2130 con 0x556ec4244280 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x556ec45587d0 con 0x556ec464bea0 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x556ec45c7450 con 0x556ec465fc20 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f11273ac000 10 client.0 _ll_register_callbacks cb 0x7f1114057c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111ddeb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec4834790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111ddeb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec4834790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50456/0 (socket says 172.21.15.80:50456) 2024-04-25T13:43:19.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111ddeb640 1 -- 172.21.15.80:0/246877936 learned_addr learned my addr 172.21.15.80:0/246877936 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111ddeb640 1 -- 172.21.15.80:0/246877936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 msgr2=0x556ec47c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111ddeb640 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 0x556ec47c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111ddeb640 1 -- 172.21.15.80:0/246877936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 msgr2=0x556ec4834cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111ddeb640 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 0x556ec4834cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.745+0000 7f111ddeb640 1 -- 172.21.15.80:0/246877936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ec457bc70 con 0x556ec4244280 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f111ddeb640 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec4834790 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1114004780 tx=0x7f1114000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=804493e7ab848688 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 1 -- 172.21.15.80:0/246877936 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f111408b070 con 0x556ec4244280 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 1 -- 172.21.15.80:0/246877936 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f1114007300 con 0x556ec4244280 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 -- 172.21.15.80:0/246877936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f110800ddf0 con 0x556ec4244280 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 1 -- 172.21.15.80:0/246877936 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f111407cb80 con 0x556ec4244280 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 10 client.6354 fetch_fsmap learned FSMap version 33 2024-04-25T13:43:19.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 -- 172.21.15.80:0/246877936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) -- 0x556ec4645180 con 0x556ec4244280 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 1 -- 172.21.15.80:0/246877936 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110800ddf0 con 0x556ec4244280 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 1 -- 172.21.15.80:0/246877936 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 33) ==== 3494+0+0 (secure 0 0 0) 0x7f111407c050 con 0x556ec4244280 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 10 client.6354 fetch_fsmap finished waiting for FSMap version 33 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 10 client.6354 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 20 client.6354 populate_metadata read hostname 'smithi080' 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] conn(0x556ec4256830 0x556ec47c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 4 client.6354 mds_command: new command op to 6195 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 -- 172.21.15.80:0/246877936 --> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] -- command(tid 0: {"prefix": "status"}) -- 0x556ec45d0a30 con 0x556ec4256830 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f111d5ea640 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] conn(0x556ec4256830 0x556ec47c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 10 client.6354 ms_handle_connect on v2:172.21.15.80:6832/1437895704 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f111d5ea640 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] conn(0x556ec4256830 0x556ec47c92b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=489216bebbdf5f9a server_cookie=8d0a33137d181a5a in_seq=0 out_seq=0 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 1 -- 172.21.15.80:0/246877936 <== mds.0 v2:172.21.15.80:6832/1437895704 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x556ec45d0a30 con 0x556ec4256830 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 10 client.6354 handle_command_reply: tid=0 2024-04-25T13:43:19.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 client.6354 shutdown 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 -- 172.21.15.80:0/246877936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 msgr2=0x556ec4834790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec4834790 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f1114004780 tx=0x7f1114000bd0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 -- 172.21.15.80:0/246877936 shutdown_connections 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.80:6832/1437895704,v1:172.21.15.80:6834/1437895704] conn(0x556ec4256830 0x556ec47c92b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x556ec465fc20 0x556ec47c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f10f8ff9640 0 client.6354 ms_handle_reset on v2:172.21.15.80:6832/1437895704 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556ec464bea0 0x556ec4834cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 --2- 172.21.15.80:0/246877936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x556ec4244280 0x556ec4834790 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 -- 172.21.15.80:0/246877936 >> 172.21.15.80:0/246877936 conn(0x556ec4283ff0 msgr2=0x556ec477b7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 -- 172.21.15.80:0/246877936 shutdown_connections 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 1 -- 172.21.15.80:0/246877936 wait complete. 2024-04-25T13:43:19.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f11273ac000 20 client.6354 trim_cache size 0 max 16384 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 6195, 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 5.9832699790000001, 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 33, 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 63, 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 63, 2024-04-25T13:43:19.758 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 10.784624708000001 2024-04-25T13:43:19.759 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:43:19.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 -- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f10fc06f400 msgr2=0x7f10fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f10fc06f400 0x7f10fc071910 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1108000f70 tx=0x7f1108009040 comp rx=0 tx=0).stop 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 -- 172.21.15.80:0/3192404777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1118068ad0 msgr2=0x7f111815a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1118068ad0 0x7f111815a830 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f111400d5d0 tx=0x7f111400da90 comp rx=0 tx=0).stop 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 -- 172.21.15.80:0/3192404777 shutdown_connections 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f10fc06f400 0x7f10fc071910 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f11181622c0 0x7f11181604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f111806d000 0x7f111815ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 --2- 172.21.15.80:0/3192404777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1118068ad0 0x7f111815a830 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 -- 172.21.15.80:0/3192404777 >> 172.21.15.80:0/3192404777 conn(0x7f11180d28f0 msgr2=0x7f1118153320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 -- 172.21.15.80:0/3192404777 shutdown_connections 2024-04-25T13:43:19.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.749+0000 7f112004f640 1 -- 172.21.15.80:0/3192404777 wait complete. 2024-04-25T13:43:19.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:43:19.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.909+0000 7f0013dad640 1 Processor -- start 2024-04-25T13:43:19.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.909+0000 7f0013dad640 1 -- start start 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0013dad640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f000c06f5a0 0x7f000c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0013dad640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f000c0733b0 0x7f000c161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0013dad640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f000c06fcf0 con 0x7f000c161070 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0013dad640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f000c1619c0 con 0x7f000c06f5a0 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0013dad640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f000c073f10 con 0x7f000c0733b0 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0011b49640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f000c06f5a0 0x7f000c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0011b49640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f000c06f5a0 0x7f000c160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50466/0 (socket says 172.21.15.80:50466) 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0011b49640 1 -- 172.21.15.80:0/3040307420 learned_addr learned my addr 172.21.15.80:0/3040307420 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:19.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f001234a640 1 --2- 172.21.15.80:0/3040307420 >> v2:172.21.15.87:3300/0 conn(0x7f000c0733b0 0x7f000c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0011b49640 1 -- 172.21.15.80:0/3040307420 >> v2:172.21.15.87:3300/0 conn(0x7f000c0733b0 msgr2=0x7f000c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0011b49640 1 --2- 172.21.15.80:0/3040307420 >> v2:172.21.15.87:3300/0 conn(0x7f000c0733b0 0x7f000c161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0011b49640 1 -- 172.21.15.80:0/3040307420 >> v1:172.21.15.87:6789/0 conn(0x7f000c161070 legacy=0x7f000c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:19.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.913+0000 7f0011b49640 1 -- 172.21.15.80:0/3040307420 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c0b0c60 con 0x7f000c06f5a0 2024-04-25T13:43:19.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0011b49640 1 --2- 172.21.15.80:0/3040307420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c160b30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f000c070270 tx=0x7f000801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552906df1c617769 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0010b47640 1 -- 172.21.15.80:0/3040307420 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00080b0070 con 0x7f000c06f5a0 2024-04-25T13:43:19.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0010b47640 1 -- 172.21.15.80:0/3040307420 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00080ab600 con 0x7f000c06f5a0 2024-04-25T13:43:19.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0010b47640 1 -- 172.21.15.80:0/3040307420 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00080ab9d0 con 0x7f000c06f5a0 2024-04-25T13:43:19.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 -- 172.21.15.80:0/3040307420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 msgr2=0x7f000c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 --2- 172.21.15.80:0/3040307420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c160b30 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f000c070270 tx=0x7f000801c450 comp rx=0 tx=0).stop 2024-04-25T13:43:19.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 -- 172.21.15.80:0/3040307420 shutdown_connections 2024-04-25T13:43:19.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 --2- 172.21.15.80:0/3040307420 >> v2:172.21.15.87:3300/0 conn(0x7f000c0733b0 0x7f000c161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 --2- 172.21.15.80:0/3040307420 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c160b30 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 -- 172.21.15.80:0/3040307420 >> 172.21.15.80:0/3040307420 conn(0x7f000c06d080 msgr2=0x7f000c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:19.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 -- 172.21.15.80:0/3040307420 shutdown_connections 2024-04-25T13:43:19.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 -- 172.21.15.80:0/3040307420 wait complete. 2024-04-25T13:43:19.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 Processor -- start 2024-04-25T13:43:19.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.917+0000 7f0013dad640 1 -- start start 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c1fb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f000c0733b0 0x7f000c13df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000c161070 0x7f000c06e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f000c16a050 con 0x7f000c06f5a0 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f000c169ed0 con 0x7f000c161070 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f000c16a1d0 con 0x7f000c0733b0 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c1fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c1fb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50472/0 (socket says 172.21.15.80:50472) 2024-04-25T13:43:19.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 -- 172.21.15.80:0/702163542 learned_addr learned my addr 172.21.15.80:0/702163542 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f001234a640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000c161070 0x7f000c06e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 -- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f000c0733b0 msgr2=0x7f000c13df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011348640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f000c0733b0 0x7f000c13df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f000c0733b0 0x7f000c13df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 -- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000c161070 msgr2=0x7f000c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000c161070 0x7f000c06e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 -- 172.21.15.80:0/702163542 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f000c15d590 con 0x7f000c06f5a0 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f001234a640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000c161070 0x7f000c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0011b49640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c1fb890 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f00080ae040 tx=0x7f00080191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f00027fc640 1 -- 172.21.15.80:0/702163542 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00080b0040 con 0x7f000c06f5a0 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f00027fc640 1 -- 172.21.15.80:0/702163542 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00080b9070 con 0x7f000c06f5a0 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f00027fc640 1 -- 172.21.15.80:0/702163542 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00080b4660 con 0x7f000c06f5a0 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f000c1601c0 con 0x7f000c06f5a0 2024-04-25T13:43:19.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f000c14e6e0 con 0x7f000c06f5a0 2024-04-25T13:43:19.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.921+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f000c14e1f0 con 0x7f000c06f5a0 2024-04-25T13:43:19.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.925+0000 7f00027fc640 1 -- 172.21.15.80:0/702163542 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f00080b4800 con 0x7f000c06f5a0 2024-04-25T13:43:19.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.925+0000 7f00027fc640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efff006f400 0x7efff0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:19.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.925+0000 7f00027fc640 1 -- 172.21.15.80:0/702163542 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f00081299e0 con 0x7f000c06f5a0 2024-04-25T13:43:19.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.925+0000 7f0011348640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efff006f400 0x7efff0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:19.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.925+0000 7f0011348640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efff006f400 0x7efff0071910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f000c0b3980 tx=0x7efffc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:19.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:19.929+0000 7f00027fc640 1 -- 172.21.15.80:0/702163542 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f00080160d0 con 0x7f000c06f5a0 2024-04-25T13:43:20.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.109+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f000c06a3a0 con 0x7f000c06f5a0 2024-04-25T13:43:20.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.109+0000 7f00027fc640 1 -- 172.21.15.80:0/702163542 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f00080f69b0 con 0x7f000c06f5a0 2024-04-25T13:43:20.114 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:43:20.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efff006f400 msgr2=0x7efff0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efff006f400 0x7efff0071910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f000c0b3980 tx=0x7efffc008040 comp rx=0 tx=0).stop 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 msgr2=0x7f000c1fb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c1fb890 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f00080ae040 tx=0x7f00080191f0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 shutdown_connections 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efff006f400 0x7efff0071910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f000c161070 0x7f000c06e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f000c0733b0 0x7f000c13df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 --2- 172.21.15.80:0/702163542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f000c06f5a0 0x7f000c1fb890 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 >> 172.21.15.80:0/702163542 conn(0x7f000c06d080 msgr2=0x7f000c0cf3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 shutdown_connections 2024-04-25T13:43:20.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20.113+0000 7f0013dad640 1 -- 172.21.15.80:0/702163542 wait complete. 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:43:20.134 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:43:20.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:20.135 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:43:20.141 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42368sec preferred_lft 42368sec 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:43:20.142 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:43:20.143 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:20.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:20.144 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:43:20.192 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:43:20.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:20.194 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:43:20.242 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:43:20.242 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:43:20.242 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:43:20.243 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:43:20.243 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:43:20.243 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:43:20.243 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:43:20.243 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:43:20.243 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:43:20.243 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:43:20.243 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:43:20.243 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:43:20.243 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:43:20.244 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:43:20.244 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:43:20.244 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:43:20.244 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:43:20.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:43:20.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:43:20.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:20.361 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:43:20.407 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:43:20.407 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:43:20.407 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:43:20.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:20.407 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:43:20.463 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:43:20.463 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:43:20.463 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:43:20.515 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:43:20.515 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:43:20.515 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:43:20.567 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:43:20.624 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:43:20.672 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:43:20.673 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:43:20.674 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:43:20.674 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:43:20.674 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:43:20.674 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:43:20.674 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:43:20.674 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:43:20.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:20.675 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:43:20.723 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:43:20.723 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:43:20.768 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:43:20.783 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:43:20.832 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:43:20.833 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:43:20.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:20.834 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:43:20.880 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:43:20.877+0000 7ffb0bcf8580 -1 init, newargv = 0x7ffaf001b4e0 newargc=15 2024-04-25T13:43:20.881 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[31848]: starting ceph client 2024-04-25T13:43:20.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[31848]: starting fuse 2024-04-25T13:43:21.980 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:43:21.996 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:43:22.043 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:43:22.044 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:43:22.045 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:43:22.046 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:43:22.046 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:43:22.046 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:43:22.046 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:43:22.046 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:43:22.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:22.047 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:43:22.095 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:43:22.095 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:43:22.095 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:43:22.095 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:43:22.095 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:43:22.096 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:43:22.097 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:43:22.097 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:43:22.097 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:43:22.097 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:43:22.097 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:43:22.097 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:22.097 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:43:22.097 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:43:22.221 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.31848.asok 2024-04-25T13:43:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:43:22.233 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.31848.asok 2024-04-25T13:43:22.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:22.233 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.31848.asok status 2024-04-25T13:43:22.410 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:43:22.410 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:43:22.410 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:43:22.410 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:43:22.410 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:43:22.410 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:43:22.410 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "31848", 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 6369, 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 6369 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:43:22.411 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2092158374 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2092158374 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.6369 192.168.0.1:0/2092158374", 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/2092158374", 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 33, 2024-04-25T13:43:22.412 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 63, 2024-04-25T13:43:22.413 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:43:22.413 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:43:22.413 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:43:22.413 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:43:22.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:22.427 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:43:22.427 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:43:22.433 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:43:22.433 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:43:22.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:22.433 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:43:22.487 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:43:22.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b4ae3f640 1 Processor -- start 2024-04-25T13:43:22.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b4ae3f640 1 -- start start 2024-04-25T13:43:22.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b4ae3f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8b440b9240 0x7f8b440cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:22.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b4ae3f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b440cfb20 con 0x7f8b440d04e0 2024-04-25T13:43:22.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b4ae3f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b440b6b70 con 0x7f8b440b7650 2024-04-25T13:43:22.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b4ae3f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8b440b6920 con 0x7f8b440b9240 2024-04-25T13:43:22.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b43fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8b440d04e0 0x7f8b440d08f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:56650/0 (socket says 172.21.15.80:56650) 2024-04-25T13:43:22.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b43fff640 1 -- 172.21.15.80:0/3309744707 learned_addr learned my addr 172.21.15.80:0/3309744707 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:22.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b48bdb640 1 --2- 172.21.15.80:0/3309744707 >> v2:172.21.15.87:3300/0 conn(0x7f8b440b9240 0x7f8b440cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:22.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730499504 0 0) 0x7f8b440cfb20 con 0x7f8b440d04e0 2024-04-25T13:43:22.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b24003530 con 0x7f8b440d04e0 2024-04-25T13:43:22.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3901507394 0 0) 0x7f8b24003530 con 0x7f8b440d04e0 2024-04-25T13:43:22.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 >> v2:172.21.15.87:3300/0 conn(0x7f8b440b9240 msgr2=0x7f8b440cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:22.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 --2- 172.21.15.80:0/3309744707 >> v2:172.21.15.87:3300/0 conn(0x7f8b440b9240 0x7f8b440cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 >> v1:172.21.15.87:6789/0 conn(0x7f8b440b7650 legacy=0x7f8b440b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b440b6e50 con 0x7f8b440d04e0 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f8b2c001680 con 0x7f8b440d04e0 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 >> v1:172.21.15.80:6789/0 conn(0x7f8b440d04e0 legacy=0x7f8b440d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b240039c0 0x7f8b24003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b24004810 0x7f8b24004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b24005260 0x7f8b2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8b440b6920 con 0x7f8b24004810 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2400c1f0 con 0x7f8b240039c0 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8b2400c3f0 con 0x7f8b24005260 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b43fff640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b24004810 0x7f8b24004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b493dc640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b24005260 0x7f8b2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b48bdb640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b240039c0 0x7f8b24003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b43fff640 1 -- 172.21.15.80:0/3309744707 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b24005260 msgr2=0x7f8b2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:22.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b43fff640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b24005260 0x7f8b2400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b43fff640 1 -- 172.21.15.80:0/3309744707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b240039c0 msgr2=0x7f8b24003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:22.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b43fff640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b240039c0 0x7f8b24003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b43fff640 1 -- 172.21.15.80:0/3309744707 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b24003530 con 0x7f8b24004810 2024-04-25T13:43:22.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.681+0000 7f8b493dc640 1 -- 172.21.15.80:0/3309744707 reap_dead start 2024-04-25T13:43:22.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b43fff640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b24004810 0x7f8b24004c20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c210 tx=0x7f8b2c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a05f8e2b4178df8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:22.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c007d40 con 0x7f8b24004810 2024-04-25T13:43:22.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2c0a9070 con 0x7f8b24004810 2024-04-25T13:43:22.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b437fe640 1 -- 172.21.15.80:0/3309744707 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c005d80 con 0x7f8b24004810 2024-04-25T13:43:22.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/3309744707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b24004810 msgr2=0x7f8b24004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:22.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b4ae3f640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b24004810 0x7f8b24004c20 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c00c210 tx=0x7f8b2c01cc70 comp rx=0 tx=0).stop 2024-04-25T13:43:22.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/3309744707 shutdown_connections 2024-04-25T13:43:22.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b4ae3f640 1 --2- 172.21.15.80:0/3309744707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b24004810 0x7f8b24004c20 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/3309744707 >> 172.21.15.80:0/3309744707 conn(0x7f8b440d3ad0 msgr2=0x7f8b440d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:22.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/3309744707 shutdown_connections 2024-04-25T13:43:22.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.685+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/3309744707 wait complete. 2024-04-25T13:43:22.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 Processor -- start 2024-04-25T13:43:22.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 -- start start 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b24004810 0x7f8b441fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b441f7c10 0x7f8b440b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b441fcba0 0x7f8b440ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8b440d1df0 con 0x7f8b441f7c10 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8b440d1c70 con 0x7f8b24004810 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8b440d1f70 con 0x7f8b441fcba0 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b441f7c10 0x7f8b440b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b441f7c10 0x7f8b440b4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58436/0 (socket says 172.21.15.80:58436) 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 -- 172.21.15.80:0/868404380 learned_addr learned my addr 172.21.15.80:0/868404380 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b48bdb640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b24004810 0x7f8b441fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b493dc640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b441fcba0 0x7f8b440ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:22.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 -- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b441fcba0 msgr2=0x7f8b440ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b441fcba0 0x7f8b440ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 -- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b24004810 msgr2=0x7f8b441fb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b24004810 0x7f8b441fb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 -- 172.21.15.80:0/868404380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b441fbb00 con 0x7f8b441f7c10 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b43fff640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b441f7c10 0x7f8b440b4150 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c01caa0 tx=0x7f8b24005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b417fa640 1 -- 172.21.15.80:0/868404380 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c007d40 con 0x7f8b441f7c10 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b417fa640 1 -- 172.21.15.80:0/868404380 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b2c000ee0 con 0x7f8b441f7c10 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b441fbde0 con 0x7f8b441f7c10 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b417fa640 1 -- 172.21.15.80:0/868404380 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b2c001640 con 0x7f8b441f7c10 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b441fc3a0 con 0x7f8b441f7c10 2024-04-25T13:43:22.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.689+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b441fb870 con 0x7f8b441f7c10 2024-04-25T13:43:22.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.693+0000 7f8b417fa640 1 -- 172.21.15.80:0/868404380 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8b2c007180 con 0x7f8b441f7c10 2024-04-25T13:43:22.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.693+0000 7f8b417fa640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b2806f430 0x7f8b28071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:22.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.693+0000 7f8b417fa640 1 -- 172.21.15.80:0/868404380 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7f8b2c128990 con 0x7f8b441f7c10 2024-04-25T13:43:22.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.693+0000 7f8b48bdb640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b2806f430 0x7f8b28071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:22.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.693+0000 7f8b48bdb640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b2806f430 0x7f8b28071940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8b44076e80 tx=0x7f8b3c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:22.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.697+0000 7f8b417fa640 1 -- 172.21.15.80:0/868404380 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8b2c0169b0 con 0x7f8b441f7c10 2024-04-25T13:43:22.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.893+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8b4406a3a0 con 0x7f8b441f7c10 2024-04-25T13:43:22.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.893+0000 7f8b417fa640 1 -- 172.21.15.80:0/868404380 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) ==== 53+0+73 (secure 0 0 0) 0x7f8b2c0f5880 con 0x7f8b441f7c10 2024-04-25T13:43:22.896 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:43:22.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b2806f430 msgr2=0x7f8b28071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b2806f430 0x7f8b28071940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f8b44076e80 tx=0x7f8b3c00b040 comp rx=0 tx=0).stop 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b441f7c10 msgr2=0x7f8b440b4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b441f7c10 0x7f8b440b4150 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f8b2c01caa0 tx=0x7f8b24005840 comp rx=0 tx=0).stop 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 shutdown_connections 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b2806f430 0x7f8b28071940 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8b441fcba0 0x7f8b440ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8b441f7c10 0x7f8b440b4150 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 --2- 172.21.15.80:0/868404380 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b24004810 0x7f8b441fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 >> 172.21.15.80:0/868404380 conn(0x7f8b440d3ad0 msgr2=0x7f8b440700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 shutdown_connections 2024-04-25T13:43:22.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:22.897+0000 7f8b4ae3f640 1 -- 172.21.15.80:0/868404380 wait complete. 2024-04-25T13:43:22.917 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:43:22.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:43:22.918 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:43:22.918 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:43:22.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:43:22.918 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:43:22.918 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:43:22.918 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:43:22.918 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:43:22.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:22.918 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:43:22.925 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:43:22.926 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42366sec preferred_lft 42366sec 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:22.927 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:43:22.928 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:43:22.929 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:22.929 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:43:22.929 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:43:22.929 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:43:22.929 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:43:22.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:22.929 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:43:22.982 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:43:22.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:22.983 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:43:23.036 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:43:23.036 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:43:23.036 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:43:23.036 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:43:23.036 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:43:23.036 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:43:23.036 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:43:23.037 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:43:23.037 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:43:23.037 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:43:23.037 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:43:23.037 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:43:23.037 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:43:23.037 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:43:23.037 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:43:23.037 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:43:23.037 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:43:23.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:43:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:43:23.154 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:43:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:43:23.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:23.161 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:43:23.167 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:43:23.167 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:43:23.167 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:43:23.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:23.168 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:43:23.222 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:43:23.222 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:43:23.222 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:43:23.272 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:43:23.272 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:43:23.272 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:43:23.319 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:43:23.376 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:43:23.423 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:43:23.424 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:43:23.425 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:43:23.426 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:43:23.426 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:43:23.426 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:43:23.426 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:43:23.426 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:43:23.426 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:43:23.426 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:43:23.426 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:43:23.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:23.427 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:43:23.475 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:43:23.475 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:43:23.520 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:43:23.534 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:43:23.583 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:43:23.584 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:43:23.585 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:43:23.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:23.589 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:43:23.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:43:23.627+0000 7f9113d0b580 -1 init, newargv = 0x7f9100005830 newargc=15 2024-04-25T13:43:23.627 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24588]: starting ceph client 2024-04-25T13:43:23.633 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24588]: starting fuse 2024-04-25T13:43:24.732 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:43:24.748 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:43:24.796 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:43:24.796 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:43:24.796 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:43:24.796 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:43:24.796 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:43:24.796 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:43:24.797 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:43:24.798 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:43:24.799 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:43:24.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:24.800 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:43:24.847 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:43:24.847 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:43:24.848 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:43:24.849 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:43:24.850 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:43:24.850 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:43:24.850 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:43:24.850 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:43:24.969 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.24588.asok 2024-04-25T13:43:24.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:43:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:43:24.980 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.24588.asok 2024-04-25T13:43:24.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:24.981 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.24588.asok status 2024-04-25T13:43:25.164 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "24588", 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:43:25.165 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5587, 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5587 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 880463066 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:43:25.166 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 880463066 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5587 192.168.0.1:0/880463066", 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/880463066", 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 33, 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 63, 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:43:25.167 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:43:25.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:25.181 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:43:25.181 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:43:25.187 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:43:25.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:43:25.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:43:25.187 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:43:25.244 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:43:25.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc34bf47640 1 Processor -- start 2024-04-25T13:43:25.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc34bf47640 1 -- start start 2024-04-25T13:43:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc34bf47640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3440d4330 0x7fc3440d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc34bf47640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc344161060 con 0x7fc3440d2e80 2024-04-25T13:43:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc34bf47640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc344161340 con 0x7fc3440d59b0 2024-04-25T13:43:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc34bf47640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc3440d2680 con 0x7fc3440d4330 2024-04-25T13:43:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc349ce3640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc3440d2e80 0x7fc3441607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:56658/0 (socket says 172.21.15.80:56658) 2024-04-25T13:43:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc349ce3640 1 -- 172.21.15.80:0/2283840682 learned_addr learned my addr 172.21.15.80:0/2283840682 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:43:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc34a4e4640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3440d4330 0x7fc3440d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1619210547 0 0) 0x7fc344161060 con 0x7fc3440d2e80 2024-04-25T13:43:25.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.401+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc330003530 con 0x7fc3440d2e80 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1926566751 0 0) 0x7fc330003530 con 0x7fc3440d2e80 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3440d4330 msgr2=0x7fc3440d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3440d4330 0x7fc3440d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 >> v1:172.21.15.87:6789/0 conn(0x7fc3440d59b0 legacy=0x7fc3440d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3440d2960 con 0x7fc3440d2e80 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc340001680 con 0x7fc3440d2e80 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 >> v1:172.21.15.80:6789/0 conn(0x7fc3440d2e80 legacy=0x7fc3441607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3300039f0 0x7fc330003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3440d4330 0x7fc330004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc330004d50 0x7fc3300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc3440d2680 con 0x7fc3300039f0 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc33000bc00 con 0x7fc330004d50 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc33000be00 con 0x7fc3440d4330 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc349ce3640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3300039f0 0x7fc330003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc349ce3640 1 -- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3440d4330 msgr2=0x7fc330004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc3494e2640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3440d4330 0x7fc330004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc34a4e4640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc330004d50 0x7fc3300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc349ce3640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3440d4330 0x7fc330004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc349ce3640 1 -- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc330004d50 msgr2=0x7fc3300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc349ce3640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc330004d50 0x7fc3300051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc349ce3640 1 -- 172.21.15.80:0/2283840682 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc330003530 con 0x7fc3300039f0 2024-04-25T13:43:25.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc349ce3640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3300039f0 0x7fc330003e60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c160 tx=0x7fc34001cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9185cf8e09e609dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340007c70 con 0x7fc3300039f0 2024-04-25T13:43:25.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3400a9070 con 0x7fc3300039f0 2024-04-25T13:43:25.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc348ce1640 1 -- 172.21.15.80:0/2283840682 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340005d80 con 0x7fc3300039f0 2024-04-25T13:43:25.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc34bf47640 1 -- 172.21.15.80:0/2283840682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3300039f0 msgr2=0x7fc330003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc34bf47640 1 --2- 172.21.15.80:0/2283840682 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3300039f0 0x7fc330003e60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fc34000c160 tx=0x7fc34001cb40 comp rx=0 tx=0).stop 2024-04-25T13:43:25.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc34a4e4640 1 -- 172.21.15.80:0/2283840682 reap_dead start 2024-04-25T13:43:25.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc34bf47640 1 -- 172.21.15.80:0/2283840682 shutdown_connections 2024-04-25T13:43:25.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc34bf47640 1 -- 172.21.15.80:0/2283840682 >> 172.21.15.80:0/2283840682 conn(0x7fc3440737a0 msgr2=0x7fc34406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc34bf47640 1 -- 172.21.15.80:0/2283840682 shutdown_connections 2024-04-25T13:43:25.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.405+0000 7fc34bf47640 1 -- 172.21.15.80:0/2283840682 wait complete. 2024-04-25T13:43:25.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 Processor -- start 2024-04-25T13:43:25.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 -- start start 2024-04-25T13:43:25.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3300039f0 0x7fc3440b9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3440b92f0 0x7fc344154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc34414c900 0x7fc3440ce6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc34416c130 con 0x7fc3440b92f0 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc34415e170 con 0x7fc3300039f0 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc34416c2b0 con 0x7fc34414c900 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3440b92f0 0x7fc344154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3440b92f0 0x7fc344154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58444/0 (socket says 172.21.15.80:58444) 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 -- 172.21.15.80:0/1784779012 learned_addr learned my addr 172.21.15.80:0/1784779012 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34a4e4640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc34414c900 0x7fc3440ce6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc349ce3640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3300039f0 0x7fc3440b9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 -- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc34414c900 msgr2=0x7fc3440ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc34414c900 0x7fc3440ce6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 -- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3300039f0 msgr2=0x7fc3440b9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3300039f0 0x7fc3440b9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 -- 172.21.15.80:0/1784779012 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3441f9d90 con 0x7fc3440b92f0 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc349ce3640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3300039f0 0x7fc3440b9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc3494e2640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3440b92f0 0x7fc344154a40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc33400d4a0 tx=0x7fc33400d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc33a7fc640 1 -- 172.21.15.80:0/1784779012 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334018070 con 0x7fc3440b92f0 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc33a7fc640 1 -- 172.21.15.80:0/1784779012 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc334003620 con 0x7fc3440b92f0 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3440cd920 con 0x7fc3440b92f0 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc33a7fc640 1 -- 172.21.15.80:0/1784779012 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc334003a30 con 0x7fc3440b92f0 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.409+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3440cdee0 con 0x7fc3440b92f0 2024-04-25T13:43:25.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.413+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34406a3a0 con 0x7fc3440b92f0 2024-04-25T13:43:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.413+0000 7fc33a7fc640 1 -- 172.21.15.80:0/1784779012 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc33401b070 con 0x7fc3440b92f0 2024-04-25T13:43:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.413+0000 7fc33a7fc640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc32806f400 0x7fc328071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:43:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.413+0000 7fc33a7fc640 1 -- 172.21.15.80:0/1784779012 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5709+0+0 (secure 0 0 0) 0x7fc33408e900 con 0x7fc3440b92f0 2024-04-25T13:43:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.413+0000 7fc349ce3640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc32806f400 0x7fc328071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:43:25.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.413+0000 7fc349ce3640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc32806f400 0x7fc328071910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc3440b8fd0 tx=0x7fc3400a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:43:25.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.417+0000 7fc33a7fc640 1 -- 172.21.15.80:0/1784779012 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc33405b230 con 0x7fc3440b92f0 2024-04-25T13:43:25.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.601+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3440b97e0 con 0x7fc3440b92f0 2024-04-25T13:43:25.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.601+0000 7fc33a7fc640 1 -- 172.21.15.80:0/1784779012 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7fc33405f530 con 0x7fc3440b92f0 2024-04-25T13:43:25.606 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:43:25.606 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1185745761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1185745761},{"type":"v1","addr":"172.21.15.80:6835","nonce":1185745761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2524166918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2524166918},{"type":"v1","addr":"172.21.15.87:6835","nonce":2524166918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3637133191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3637133191},{"type":"v1","addr":"172.21.15.87:6837","nonce":3637133191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:43:13.179327+0000","modified":"2024-04-25T13:43:16.230820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6834/1437895704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1437895704},{"type":"v1","addr":"172.21.15.80:6834","nonce":1437895704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6195,"qdb_cluster":[6195]},"id":4}]} 2024-04-25T13:43:25.606 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 33 2024-04-25T13:43:25.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc32806f400 msgr2=0x7fc328071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc32806f400 0x7fc328071910 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc3440b8fd0 tx=0x7fc3400a7070 comp rx=0 tx=0).stop 2024-04-25T13:43:25.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3440b92f0 msgr2=0x7fc344154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:43:25.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3440b92f0 0x7fc344154a40 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc33400d4a0 tx=0x7fc33400d960 comp rx=0 tx=0).stop 2024-04-25T13:43:25.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 shutdown_connections 2024-04-25T13:43:25.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc32806f400 0x7fc328071910 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc34414c900 0x7fc3440ce6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3440b92f0 0x7fc344154a40 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 --2- 172.21.15.80:0/1784779012 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3300039f0 0x7fc3440b9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:43:25.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 >> 172.21.15.80:0/1784779012 conn(0x7fc3440737a0 msgr2=0x7fc3440d5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:43:25.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 shutdown_connections 2024-04-25T13:43:25.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:43:25.605+0000 7fc34bf47640 1 -- 172.21.15.80:0/1784779012 wait complete. 2024-04-25T13:43:25.625 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:43:25.792 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":6195,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.006867958000001,"mdsmap_epoch":33,"osdmap_epoch":63,"osdmap_epoch_barrier":63,"uptime":16.808235825000001} 2024-04-25T13:43:25.792 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 6195, "mdsmap_epoch": 33, "osdmap_epoch": 63, "osdmap_epoch_barrier": 63, "rank_uptime": 12.006867958, "state": "up:active", "uptime": 16.808235825, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:43:25.793 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-25T13:43:25.949 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_mon_report_interval":"5"} 2024-04-25T13:43:25.950 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-25T13:43:25.950 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-04-25T13:43:26.097 INFO:teuthology.orchestra.run.smithi080.stdout:{"mon_tick_interval":"5"} 2024-04-25T13:43:26.097 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-04-25T13:43:26.098 INFO:tasks.cephfs.test_full:pool capacity 557049408, 531MB should be enough to fill it 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> import time 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> import datetime 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> # Write some buffered data through before going full, all should be well 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> print("writing some data through which we expect to succeed") 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> bytes = 0 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-04-25T13:43:26.098 DEBUG:teuthology.orchestra.run.smithi080:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(f) 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> # buffered IO, the error should always only appear from fsync and not 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> # from write 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> full = False 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> for n in range(0, int(531 * 0.9)): 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> print("done writing {0} bytes".format(bytes)) 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.099 DEBUG:teuthology.orchestra.run.smithi080:> # OK, now we should sneak in under the full condition 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> # due to the time it takes the OSDs to report to the 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> # mons, and get a successful fsync on our full-making data 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(f) 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> # buffered write, add more dirty data to the buffer 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> print("starting buffered write") 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> try: 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> for n in range(0, int(531 * 0.2)): 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> time.sleep(15.0) 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> except OSError: 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> pass; 2024-04-25T13:43:26.100 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> # Wait long enough for a background flush that should fail 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> time.sleep(30) 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> if True: 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> # ...and check that the failed background flush is reflected in fclose 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> try: 2024-04-25T13:43:26.101 DEBUG:teuthology.orchestra.run.smithi080:> os.close(f) 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> except OSError: 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> print("close() returned an error as expected") 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> else: 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("close() failed to raise error") 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> else: 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> # The kernel cephfs client does not raise errors on fclose 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> os.close(f) 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-04-25T13:43:26.102 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:43:28.750 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:43:28.753 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:43:29.597 INFO:teuthology.orchestra.run.smithi080.stdout:writing some data through which we expect to succeed 2024-04-25T13:43:29.597 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-04-25T13:43:29.597 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-04-25T13:43:29.597 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-04-25T13:43:29.597 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-04-25T13:43:29.597 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-04-25T13:43:29.597 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-04-25T13:43:29.598 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 30932992 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 31981568 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 33030144 bytes via buffered write, may repeat 2024-04-25T13:43:29.599 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 34078720 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 35127296 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 36175872 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 37224448 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 38273024 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 39321600 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 40370176 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 41418752 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 42467328 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 43515904 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 44564480 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 45613056 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 46661632 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 47710208 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 48758784 bytes via buffered write, may repeat 2024-04-25T13:43:29.600 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 49807360 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 50855936 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 51904512 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 52953088 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 54001664 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 55050240 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 56098816 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 57147392 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 58195968 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 59244544 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 60293120 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 61341696 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 62390272 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 63438848 bytes via buffered write, may repeat 2024-04-25T13:43:29.601 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 64487424 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 65536000 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 66584576 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 67633152 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 68681728 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 69730304 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 70778880 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 71827456 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 72876032 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 73924608 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 74973184 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 76021760 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 77070336 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 78118912 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 79167488 bytes via buffered write, may repeat 2024-04-25T13:43:29.602 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 80216064 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 81264640 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 82313216 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 83361792 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 84410368 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 85458944 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 86507520 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 87556096 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 88604672 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 89653248 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 90701824 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 91750400 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 92798976 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 93847552 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 94896128 bytes via buffered write, may repeat 2024-04-25T13:43:29.603 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 95944704 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 96993280 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 98041856 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 99090432 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 100139008 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 101187584 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 102236160 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 103284736 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 104333312 bytes via buffered write, may repeat 2024-04-25T13:43:29.604 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 105381888 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 106430464 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 107479040 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 108527616 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 109576192 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 110624768 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 111673344 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 112721920 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 113770496 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 114819072 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 115867648 bytes via buffered write, may repeat 2024-04-25T13:43:29.605 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 116916224 bytes via buffered write, may repeat 2024-04-25T13:43:29.606 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 117964800 bytes via buffered write, may repeat 2024-04-25T13:43:29.606 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 119013376 bytes via buffered write, may repeat 2024-04-25T13:43:29.606 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 120061952 bytes via buffered write, may repeat 2024-04-25T13:43:29.606 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 121110528 bytes via buffered write, may repeat 2024-04-25T13:43:29.606 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 122159104 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 123207680 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 124256256 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 125304832 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 126353408 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 127401984 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 128450560 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 129499136 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 130547712 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 131596288 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 132644864 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 133693440 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 134742016 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 135790592 bytes via buffered write, may repeat 2024-04-25T13:43:29.607 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 136839168 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 137887744 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 138936320 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 139984896 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 141033472 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 142082048 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 143130624 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 144179200 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 145227776 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 146276352 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 147324928 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 148373504 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 149422080 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 150470656 bytes via buffered write, may repeat 2024-04-25T13:43:29.608 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 151519232 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 152567808 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 153616384 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 154664960 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 155713536 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 156762112 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 157810688 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 158859264 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 159907840 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 160956416 bytes via buffered write, may repeat 2024-04-25T13:43:29.609 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 162004992 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 163053568 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 164102144 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 165150720 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 166199296 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 167247872 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 168296448 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 169345024 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 170393600 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 171442176 bytes via buffered write, may repeat 2024-04-25T13:43:29.934 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 172490752 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 173539328 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 174587904 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 175636480 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 176685056 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 177733632 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 178782208 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 179830784 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 180879360 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 181927936 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 182976512 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 184025088 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 185073664 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 186122240 bytes via buffered write, may repeat 2024-04-25T13:43:29.935 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 187170816 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 188219392 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 189267968 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 190316544 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 191365120 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 192413696 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 193462272 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 194510848 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 195559424 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 196608000 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 197656576 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 198705152 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 199753728 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 200802304 bytes via buffered write, may repeat 2024-04-25T13:43:29.936 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 201850880 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 202899456 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 203948032 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 204996608 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 206045184 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 207093760 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 208142336 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 209190912 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 210239488 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 211288064 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 212336640 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 213385216 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 214433792 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 215482368 bytes via buffered write, may repeat 2024-04-25T13:43:29.937 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 216530944 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 217579520 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 218628096 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 219676672 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 220725248 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 221773824 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 222822400 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 223870976 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 224919552 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 225968128 bytes via buffered write, may repeat 2024-04-25T13:43:29.938 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 227016704 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 228065280 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 229113856 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 230162432 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 231211008 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 232259584 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 233308160 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 234356736 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 235405312 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 236453888 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 237502464 bytes via buffered write, may repeat 2024-04-25T13:43:29.939 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 238551040 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 239599616 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 240648192 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 241696768 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 242745344 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 243793920 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 244842496 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 245891072 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 246939648 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 247988224 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 249036800 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 250085376 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 251133952 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 252182528 bytes via buffered write, may repeat 2024-04-25T13:43:29.940 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 253231104 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 254279680 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 255328256 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 256376832 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 257425408 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 258473984 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 259522560 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 260571136 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 261619712 bytes via buffered write, may repeat 2024-04-25T13:43:29.941 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 262668288 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 263716864 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 264765440 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 265814016 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 266862592 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 267911168 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 268959744 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 270008320 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 271056896 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 272105472 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 273154048 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 274202624 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 275251200 bytes via buffered write, may repeat 2024-04-25T13:43:29.942 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 276299776 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 277348352 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 278396928 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 279445504 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 280494080 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 281542656 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 282591232 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 283639808 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 284688384 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 285736960 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 286785536 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 287834112 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 288882688 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 289931264 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 290979840 bytes via buffered write, may repeat 2024-04-25T13:43:29.943 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 292028416 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 293076992 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 294125568 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 295174144 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 296222720 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 297271296 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 298319872 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 299368448 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 300417024 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 301465600 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 302514176 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 303562752 bytes via buffered write, may repeat 2024-04-25T13:43:29.944 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 304611328 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 305659904 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 306708480 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 307757056 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 308805632 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 309854208 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 310902784 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 311951360 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 312999936 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 314048512 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 315097088 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 316145664 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 317194240 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 318242816 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 319291392 bytes via buffered write, may repeat 2024-04-25T13:43:29.945 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 320339968 bytes via buffered write, may repeat 2024-04-25T13:43:29.946 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 321388544 bytes via buffered write, may repeat 2024-04-25T13:43:29.946 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 322437120 bytes via buffered write, may repeat 2024-04-25T13:43:29.946 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 323485696 bytes via buffered write, may repeat 2024-04-25T13:43:30.225 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 324534272 bytes via buffered write, may repeat 2024-04-25T13:43:30.225 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 325582848 bytes via buffered write, may repeat 2024-04-25T13:43:30.225 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 326631424 bytes via buffered write, may repeat 2024-04-25T13:43:30.225 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 327680000 bytes via buffered write, may repeat 2024-04-25T13:43:30.225 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 328728576 bytes via buffered write, may repeat 2024-04-25T13:43:30.225 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 329777152 bytes via buffered write, may repeat 2024-04-25T13:43:30.225 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 330825728 bytes via buffered write, may repeat 2024-04-25T13:43:30.225 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 331874304 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 332922880 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 333971456 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 335020032 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 336068608 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 337117184 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 338165760 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 339214336 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 340262912 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 341311488 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 342360064 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 343408640 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 344457216 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 345505792 bytes via buffered write, may repeat 2024-04-25T13:43:30.226 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 346554368 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 347602944 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 348651520 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 349700096 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 350748672 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 351797248 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 352845824 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 353894400 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 354942976 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 355991552 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 357040128 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 358088704 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 359137280 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 360185856 bytes via buffered write, may repeat 2024-04-25T13:43:30.227 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 361234432 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 362283008 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 363331584 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 364380160 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 365428736 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 366477312 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 367525888 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 368574464 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 369623040 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 370671616 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 371720192 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 372768768 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 373817344 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 374865920 bytes via buffered write, may repeat 2024-04-25T13:43:30.228 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 375914496 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 376963072 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 378011648 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 379060224 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 380108800 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 381157376 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 382205952 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 383254528 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 384303104 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 385351680 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 386400256 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 387448832 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 388497408 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 389545984 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 390594560 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 391643136 bytes via buffered write, may repeat 2024-04-25T13:43:30.229 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 392691712 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 393740288 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 394788864 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 395837440 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 396886016 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 397934592 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 398983168 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 400031744 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 401080320 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 402128896 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 403177472 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 404226048 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 405274624 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 406323200 bytes via buffered write, may repeat 2024-04-25T13:43:30.230 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 407371776 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 408420352 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 409468928 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 410517504 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 411566080 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 412614656 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 413663232 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 414711808 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 415760384 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 416808960 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 417857536 bytes via buffered write, may repeat 2024-04-25T13:43:30.231 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 418906112 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 419954688 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 421003264 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 422051840 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 423100416 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 424148992 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 425197568 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 426246144 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 427294720 bytes via buffered write, may repeat 2024-04-25T13:43:30.232 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 428343296 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 429391872 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 430440448 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 431489024 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 432537600 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 433586176 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 434634752 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 435683328 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 436731904 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 437780480 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 438829056 bytes via buffered write, may repeat 2024-04-25T13:43:30.233 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 439877632 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 440926208 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 441974784 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 443023360 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 444071936 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 445120512 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 446169088 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 447217664 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 448266240 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 449314816 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 450363392 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 451411968 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 452460544 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 453509120 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 454557696 bytes via buffered write, may repeat 2024-04-25T13:43:30.234 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 455606272 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 456654848 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 457703424 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 458752000 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 459800576 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 460849152 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 461897728 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 462946304 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 463994880 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 465043456 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 466092032 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 467140608 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 468189184 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 469237760 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 470286336 bytes via buffered write, may repeat 2024-04-25T13:43:30.235 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 471334912 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 472383488 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 473432064 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 474480640 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 475529216 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 476577792 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 477626368 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 478674944 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 479723520 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 480772096 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 481820672 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 482869248 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 483917824 bytes via buffered write, may repeat 2024-04-25T13:43:30.236 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 484966400 bytes via buffered write, may repeat 2024-04-25T13:43:33.240 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:43:33.240+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check failed: 2 full osd(s) (OSD_FULL) 2024-04-25T13:43:41.164 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:43:41.160+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-04-25T13:43:58.803 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:43:58.831 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:44:15.390 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 486014976 bytes via buffered write, may repeat 2024-04-25T13:44:15.390 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 487063552 bytes via buffered write, may repeat 2024-04-25T13:44:15.390 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 488112128 bytes via buffered write, may repeat 2024-04-25T13:44:15.390 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 489160704 bytes via buffered write, may repeat 2024-04-25T13:44:15.390 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 490209280 bytes via buffered write, may repeat 2024-04-25T13:44:15.390 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 491257856 bytes via buffered write, may repeat 2024-04-25T13:44:15.390 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 492306432 bytes via buffered write, may repeat 2024-04-25T13:44:15.390 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 493355008 bytes via buffered write, may repeat 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 494403584 bytes via buffered write, may repeat 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 495452160 bytes via buffered write, may repeat 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 496500736 bytes via buffered write, may repeat 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 497549312 bytes via buffered write, may repeat 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 498597888 bytes via buffered write, may repeat 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 499646464 bytes via buffered write, may repeat 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 500695040 bytes via buffered write, may repeat 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:done writing 500695040 bytes 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:successfully fsync'ed prior to getting full state reported 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:starting buffered write 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-04-25T13:44:15.391 INFO:teuthology.orchestra.run.smithi080.stdout:close() returned an error as expected 2024-04-25T13:44:15.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:44:15.410 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:44:15.410 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:44:15.467 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:44:15.524 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:44:15.524 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:44:15.524 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:44:15.525 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:15.531 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:44:15.532 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:15.579 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573c65e640 1 Processor -- start 2024-04-25T13:44:15.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573c65e640 1 -- start start 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573c65e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573408e230 0x7f57340b9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573c65e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57340b9970 0x7f57340d0db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573c65e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57340d12f0 0x7f57340d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573c65e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f57340d1700 con 0x7f57340b9970 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573c65e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5734160680 con 0x7f57340d12f0 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573c65e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57340d3c40 con 0x7f573408e230 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f5739bf9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57340b9970 0x7f57340d0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f5739bf9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57340b9970 0x7f57340d0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43050/0 (socket says 172.21.15.80:43050) 2024-04-25T13:44:15.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f5739bf9640 1 -- 172.21.15.80:0/3387215731 learned_addr learned my addr 172.21.15.80:0/3387215731 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:15.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573a3fa640 1 --2- 172.21.15.80:0/3387215731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573408e230 0x7f57340b9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:15.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f573abfb640 1 --2- 172.21.15.80:0/3387215731 >> v2:172.21.15.87:3300/0 conn(0x7f57340d12f0 0x7f57340d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:15.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f5739bf9640 1 -- 172.21.15.80:0/3387215731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573408e230 msgr2=0x7f57340b9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:15.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f5739bf9640 1 --2- 172.21.15.80:0/3387215731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573408e230 0x7f57340b9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f5739bf9640 1 -- 172.21.15.80:0/3387215731 >> v2:172.21.15.87:3300/0 conn(0x7f57340d12f0 msgr2=0x7f57340d4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:15.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f5739bf9640 1 --2- 172.21.15.80:0/3387215731 >> v2:172.21.15.87:3300/0 conn(0x7f57340d12f0 0x7f57340d4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.780+0000 7f5739bf9640 1 -- 172.21.15.80:0/3387215731 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57340d3f20 con 0x7f57340b9970 2024-04-25T13:44:15.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f5739bf9640 1 --2- 172.21.15.80:0/3387215731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340d0db0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f57340cc9a0 tx=0x7f573001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca859da8ea506e14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:15.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f57393f8640 1 -- 172.21.15.80:0/3387215731 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300b0070 con 0x7f57340b9970 2024-04-25T13:44:15.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f57393f8640 1 -- 172.21.15.80:0/3387215731 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57300ab600 con 0x7f57340b9970 2024-04-25T13:44:15.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f57393f8640 1 -- 172.21.15.80:0/3387215731 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300ab9d0 con 0x7f57340b9970 2024-04-25T13:44:15.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 -- 172.21.15.80:0/3387215731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 msgr2=0x7f57340d0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:15.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 --2- 172.21.15.80:0/3387215731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340d0db0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f57340cc9a0 tx=0x7f573001c490 comp rx=0 tx=0).stop 2024-04-25T13:44:15.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 -- 172.21.15.80:0/3387215731 shutdown_connections 2024-04-25T13:44:15.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 --2- 172.21.15.80:0/3387215731 >> v2:172.21.15.87:3300/0 conn(0x7f57340d12f0 0x7f57340d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 --2- 172.21.15.80:0/3387215731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340d0db0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 --2- 172.21.15.80:0/3387215731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573408e230 0x7f57340b9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 -- 172.21.15.80:0/3387215731 >> 172.21.15.80:0/3387215731 conn(0x7f57340b78d0 msgr2=0x7f57340b7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:15.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 -- 172.21.15.80:0/3387215731 shutdown_connections 2024-04-25T13:44:15.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.784+0000 7f573c65e640 1 -- 172.21.15.80:0/3387215731 wait complete. 2024-04-25T13:44:15.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 Processor -- start 2024-04-25T13:44:15.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 -- start start 2024-04-25T13:44:15.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f573408e230 0x7f57340ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:15.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340ccdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:15.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57340d12f0 0x7f57341f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:15.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57340d38d0 con 0x7f57340b9970 2024-04-25T13:44:15.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57340d5630 con 0x7f573408e230 2024-04-25T13:44:15.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f573416c660 con 0x7f57340d12f0 2024-04-25T13:44:15.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340ccdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:15.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340ccdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43062/0 (socket says 172.21.15.80:43062) 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 -- 172.21.15.80:0/1024570342 learned_addr learned my addr 172.21.15.80:0/1024570342 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573a3fa640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f573408e230 0x7f57340ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573abfb640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57340d12f0 0x7f57341f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 -- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57340d12f0 msgr2=0x7f57341f9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57340d12f0 0x7f57341f9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 -- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f573408e230 msgr2=0x7f57340ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f573408e230 0x7f57340ca9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 -- 172.21.15.80:0/1024570342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57340cd2f0 con 0x7f57340b9970 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573a3fa640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f573408e230 0x7f57340ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f5739bf9640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340ccdb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f573001cd30 tx=0x7f5730019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f572affd640 1 -- 172.21.15.80:0/1024570342 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300b9070 con 0x7f57340b9970 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f572affd640 1 -- 172.21.15.80:0/1024570342 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57300b4420 con 0x7f57340b9970 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57341527f0 con 0x7f57340b9970 2024-04-25T13:44:15.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f572affd640 1 -- 172.21.15.80:0/1024570342 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57300b0040 con 0x7f57340b9970 2024-04-25T13:44:15.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.788+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5734152db0 con 0x7f57340b9970 2024-04-25T13:44:15.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.792+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57340cafa0 con 0x7f57340b9970 2024-04-25T13:44:15.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.792+0000 7f572affd640 1 -- 172.21.15.80:0/1024570342 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57300bc070 con 0x7f57340b9970 2024-04-25T13:44:15.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.792+0000 7f572affd640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571806f430 0x7f5718071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:15.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.792+0000 7f572affd640 1 -- 172.21.15.80:0/1024570342 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f5730128380 con 0x7f57340b9970 2024-04-25T13:44:15.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.792+0000 7f572affd640 1 -- 172.21.15.80:0/1024570342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f5718076fb0 con 0x7f57340b9970 2024-04-25T13:44:15.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.792+0000 7f573a3fa640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571806f430 0x7f5718071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:15.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.792+0000 7f573a3fa640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571806f430 0x7f5718071940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f572c003fd0 tx=0x7f572c0039e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:15.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.796+0000 7f572affd640 1 -- 172.21.15.80:0/1024570342 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5730016110 con 0x7f57340b9970 2024-04-25T13:44:15.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.960+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f573406a3a0 con 0x7f57340b9970 2024-04-25T13:44:15.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f572affd640 1 -- 172.21.15.80:0/1024570342 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16651 (secure 0 0 0) 0x7f57300f5390 con 0x7f57340b9970 2024-04-25T13:44:15.962 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:15.962 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":66,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:43:36.269783+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:43:11.515229+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T13:43:12.143130+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571806f430 msgr2=0x7f5718071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571806f430 0x7f5718071940 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f572c003fd0 tx=0x7f572c0039e0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 msgr2=0x7f57340ccdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340ccdb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f573001cd30 tx=0x7f5730019230 comp rx=0 tx=0).stop 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 shutdown_connections 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571806f430 0x7f5718071940 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57340d12f0 0x7f57341f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57340b9970 0x7f57340ccdb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 --2- 172.21.15.80:0/1024570342 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f573408e230 0x7f57340ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:15.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 >> 172.21.15.80:0/1024570342 conn(0x7f57340b78d0 msgr2=0x7f57340bb050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:15.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 shutdown_connections 2024-04-25T13:44:15.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:15.964+0000 7f573c65e640 1 -- 172.21.15.80:0/1024570342 wait complete. 2024-04-25T13:44:15.984 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:43:11.515229+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:43:12.143130+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:44:15.984 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:44:16.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6c5f82640 1 Processor -- start 2024-04-25T13:44:16.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6c5f82640 1 -- start start 2024-04-25T13:44:16.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6c5f82640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc6c00b9e70 0x7fc6c00ba280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6c5f82640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c00b92a0 con 0x7fc6c00d3ed0 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6c5f82640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6c00d4c10 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6c5f82640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc6c00d4ef0 con 0x7fc6c00b9e70 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6bffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc6c00b9e70 0x7fc6c00ba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6bf7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc6c00d3ed0 0x7fc6c00d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34498/0 (socket says 172.21.15.80:34498) 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6bf7fe640 1 -- 172.21.15.80:0/2245240258 learned_addr learned my addr 172.21.15.80:0/2245240258 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6bffff640 1 -- 172.21.15.80:0/2245240258 >> v1:172.21.15.87:6789/0 conn(0x7fc6c00d2ca0 legacy=0x7fc6c00b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6bffff640 1 -- 172.21.15.80:0/2245240258 >> v1:172.21.15.80:6789/0 conn(0x7fc6c00d3ed0 legacy=0x7fc6c00d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6bffff640 1 -- 172.21.15.80:0/2245240258 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c00d51d0 con 0x7fc6c00b9e70 2024-04-25T13:44:16.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.136+0000 7fc6be7fc640 1 -- 172.21.15.80:0/2245240258 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2016238871 0 0) 0x7fc6b8001120 con 0x7fc6c00d3ed0 2024-04-25T13:44:16.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6bffff640 1 --2- 172.21.15.80:0/2245240258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00b9e70 0x7fc6c00ba280 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00097c0 tx=0x7fc6b001c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f187915e32b2b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6be7fc640 1 -- 172.21.15.80:0/2245240258 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00b0070 con 0x7fc6c00b9e70 2024-04-25T13:44:16.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6be7fc640 1 -- 172.21.15.80:0/2245240258 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b00ab600 con 0x7fc6c00b9e70 2024-04-25T13:44:16.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6be7fc640 1 -- 172.21.15.80:0/2245240258 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00ab9d0 con 0x7fc6c00b9e70 2024-04-25T13:44:16.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6c5f82640 1 -- 172.21.15.80:0/2245240258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00b9e70 msgr2=0x7fc6c00ba280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6c5f82640 1 --2- 172.21.15.80:0/2245240258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00b9e70 0x7fc6c00ba280 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc6b00097c0 tx=0x7fc6b001c310 comp rx=0 tx=0).stop 2024-04-25T13:44:16.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6c5f82640 1 -- 172.21.15.80:0/2245240258 shutdown_connections 2024-04-25T13:44:16.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6c5f82640 1 --2- 172.21.15.80:0/2245240258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00b9e70 0x7fc6c00ba280 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6c5f82640 1 -- 172.21.15.80:0/2245240258 >> 172.21.15.80:0/2245240258 conn(0x7fc6c00737b0 msgr2=0x7fc6c006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:16.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6c5f82640 1 -- 172.21.15.80:0/2245240258 shutdown_connections 2024-04-25T13:44:16.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.140+0000 7fc6c5f82640 1 -- 172.21.15.80:0/2245240258 wait complete. 2024-04-25T13:44:16.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 Processor -- start 2024-04-25T13:44:16.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 -- start start 2024-04-25T13:44:16.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c00b9e70 0x7fc6c0150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d2ca0 0x7fc6c00c88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c00d3ed0 0x7fc6c0156d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c016e6b0 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc6c016e830 con 0x7fc6c00b9e70 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c016e9b0 con 0x7fc6c00d3ed0 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d2ca0 0x7fc6c00c88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d2ca0 0x7fc6c00c88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43074/0 (socket says 172.21.15.80:43074) 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 -- 172.21.15.80:0/38048253 learned_addr learned my addr 172.21.15.80:0/38048253 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6bffff640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c00d3ed0 0x7fc6c0156d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 -- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c00d3ed0 msgr2=0x7fc6c0156d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c00d3ed0 0x7fc6c0156d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 -- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c00b9e70 msgr2=0x7fc6c0150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6bf7fe640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c00b9e70 0x7fc6c0150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c00b9e70 0x7fc6c0150730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 -- 172.21.15.80:0/38048253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c014f340 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6beffd640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d2ca0 0x7fc6c00c88e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400ba80 tx=0x7fc6b400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6a7fff640 1 -- 172.21.15.80:0/38048253 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400cc00 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6a7fff640 1 -- 172.21.15.80:0/38048253 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6b400cda0 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6c014f620 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6a7fff640 1 -- 172.21.15.80:0/38048253 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4003bc0 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6c014fbe0 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.144+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6c006a3a0 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.148+0000 7fc6a7fff640 1 -- 172.21.15.80:0/38048253 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc6b4006d10 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.148+0000 7fc6a7fff640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6ac06f400 0x7fc6ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.148+0000 7fc6a7fff640 1 -- 172.21.15.80:0/38048253 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fc6b408f600 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.148+0000 7fc6a7fff640 1 -- 172.21.15.80:0/38048253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7fc6ac076fe0 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.148+0000 7fc6bf7fe640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6ac06f400 0x7fc6ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.148+0000 7fc6bf7fe640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6ac06f400 0x7fc6ac071910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000f70 tx=0x7fc6b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.152+0000 7fc6a7fff640 1 -- 172.21.15.80:0/38048253 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6b405c660 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.324+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc6c0150dc0 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.324+0000 7fc6a7fff640 1 -- 172.21.15.80:0/38048253 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7fc6b405bf60 con 0x7fc6c00d2ca0 2024-04-25T13:44:16.325 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:44:16.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6ac06f400 msgr2=0x7fc6ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6ac06f400 0x7fc6ac071910 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc6b8000f70 tx=0x7fc6b8009040 comp rx=0 tx=0).stop 2024-04-25T13:44:16.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d2ca0 msgr2=0x7fc6c00c88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d2ca0 0x7fc6c00c88e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc6b400ba80 tx=0x7fc6b400bf40 comp rx=0 tx=0).stop 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 shutdown_connections 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6ac06f400 0x7fc6ac071910 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c00d3ed0 0x7fc6c0156d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c00d2ca0 0x7fc6c00c88e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 --2- 172.21.15.80:0/38048253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c00b9e70 0x7fc6c0150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 >> 172.21.15.80:0/38048253 conn(0x7fc6c00737b0 msgr2=0x7fc6c00ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 shutdown_connections 2024-04-25T13:44:16.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.328+0000 7fc6c5f82640 1 -- 172.21.15.80:0/38048253 wait complete. 2024-04-25T13:44:16.347 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:44:16.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be3984640 1 Processor -- start 2024-04-25T13:44:16.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be3984640 1 -- start start 2024-04-25T13:44:16.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be3984640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc161170 0x7f8bdc1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be3984640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8bdc073c40 0x7f8bdc0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be3984640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8bdc06fbf0 con 0x7f8bdc071e60 2024-04-25T13:44:16.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be3984640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8bdc0b1350 con 0x7f8bdc073c40 2024-04-25T13:44:16.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be3984640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc06f9a0 con 0x7f8bdc161170 2024-04-25T13:44:16.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be0f1f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc161170 0x7f8bdc1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be0f1f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc161170 0x7f8bdc1574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:55826/0 (socket says 172.21.15.80:55826) 2024-04-25T13:44:16.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be1f21640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8bdc073c40 0x7f8bdc0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be0f1f640 1 -- 172.21.15.80:0/3520434619 learned_addr learned my addr 172.21.15.80:0/3520434619 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:16.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be1f21640 1 -- 172.21.15.80:0/3520434619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc161170 msgr2=0x7f8bdc1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:16.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be1f21640 1 --2- 172.21.15.80:0/3520434619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc161170 0x7f8bdc1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be1f21640 1 -- 172.21.15.80:0/3520434619 >> v1:172.21.15.87:6789/0 conn(0x7f8bdc071e60 legacy=0x7f8bdc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be1f21640 1 -- 172.21.15.80:0/3520434619 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc0b0a20 con 0x7f8bdc073c40 2024-04-25T13:44:16.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.500+0000 7f8be0f1f640 1 --2- 172.21.15.80:0/3520434619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc161170 0x7f8bdc1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:16.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be1f21640 1 --2- 172.21.15.80:0/3520434619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc073c40 0x7f8bdc0b4ba0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0097c0 tx=0x7f8bcc01c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10ef4835571e92fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8bdbfff640 1 -- 172.21.15.80:0/3520434619 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0b2070 con 0x7f8bdc073c40 2024-04-25T13:44:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8bdbfff640 1 -- 172.21.15.80:0/3520434619 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0ad640 con 0x7f8bdc073c40 2024-04-25T13:44:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8bdbfff640 1 -- 172.21.15.80:0/3520434619 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0ada10 con 0x7f8bdc073c40 2024-04-25T13:44:16.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be3984640 1 -- 172.21.15.80:0/3520434619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc073c40 msgr2=0x7f8bdc0b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be3984640 1 --2- 172.21.15.80:0/3520434619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc073c40 0x7f8bdc0b4ba0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0097c0 tx=0x7f8bcc01c4a0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be3984640 1 -- 172.21.15.80:0/3520434619 shutdown_connections 2024-04-25T13:44:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be3984640 1 --2- 172.21.15.80:0/3520434619 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc073c40 0x7f8bdc0b4ba0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be3984640 1 --2- 172.21.15.80:0/3520434619 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc161170 0x7f8bdc1574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be3984640 1 -- 172.21.15.80:0/3520434619 >> 172.21.15.80:0/3520434619 conn(0x7f8bdc0b4160 msgr2=0x7f8bdc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be3984640 1 -- 172.21.15.80:0/3520434619 shutdown_connections 2024-04-25T13:44:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.504+0000 7f8be3984640 1 -- 172.21.15.80:0/3520434619 wait complete. 2024-04-25T13:44:16.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 Processor -- start 2024-04-25T13:44:16.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 -- start start 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc071e60 0x7f8bdc155880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc073c40 0x7f8bdc153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bdc156830 0x7f8bdc0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc06e470 con 0x7f8bdc156830 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8bdc074050 con 0x7f8bdc071e60 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8bdc06e5f0 con 0x7f8bdc073c40 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bdc156830 0x7f8bdc0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bdc156830 0x7f8bdc0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43078/0 (socket says 172.21.15.80:43078) 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 -- 172.21.15.80:0/3245280441 learned_addr learned my addr 172.21.15.80:0/3245280441 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be0f1f640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc073c40 0x7f8bdc153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1720640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc071e60 0x7f8bdc155880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 -- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc073c40 msgr2=0x7f8bdc153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc073c40 0x7f8bdc153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 -- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc071e60 msgr2=0x7f8bdc155880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc071e60 0x7f8bdc155880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 -- 172.21.15.80:0/3245280441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bdc0737c0 con 0x7f8bdc156830 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be1f21640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bdc156830 0x7f8bdc0af8b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0b0040 tx=0x7f8bcc019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8bd9ffb640 1 -- 172.21.15.80:0/3245280441 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0b2040 con 0x7f8bdc156830 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8bd9ffb640 1 -- 172.21.15.80:0/3245280441 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bcc0bc070 con 0x7f8bdc156830 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bdc158540 con 0x7f8bdc156830 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8bd9ffb640 1 -- 172.21.15.80:0/3245280441 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc006e20 con 0x7f8bdc156830 2024-04-25T13:44:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.508+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bdc158b00 con 0x7f8bdc156830 2024-04-25T13:44:16.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.512+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bdc06a3a0 con 0x7f8bdc156830 2024-04-25T13:44:16.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.512+0000 7f8bd9ffb640 1 -- 172.21.15.80:0/3245280441 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8bcc0b6650 con 0x7f8bdc156830 2024-04-25T13:44:16.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.512+0000 7f8bd9ffb640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bc806f400 0x7f8bc8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.512+0000 7f8bd9ffb640 1 -- 172.21.15.80:0/3245280441 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f8bcc129860 con 0x7f8bdc156830 2024-04-25T13:44:16.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.512+0000 7f8bd9ffb640 1 -- 172.21.15.80:0/3245280441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f8bc8076f80 con 0x7f8bdc156830 2024-04-25T13:44:16.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.512+0000 7f8be1720640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bc806f400 0x7f8bc8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.512+0000 7f8be1720640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bc806f400 0x7f8bc8071910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40012f0 tx=0x7f8bd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.516+0000 7f8bd9ffb640 1 -- 172.21.15.80:0/3245280441 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8bcc0c6020 con 0x7f8bdc156830 2024-04-25T13:44:16.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8bdc160990 con 0x7f8bdc156830 2024-04-25T13:44:16.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8bd9ffb640 1 -- 172.21.15.80:0/3245280441 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7f8bcc016090 con 0x7f8bdc156830 2024-04-25T13:44:16.675 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:44:16.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bc806f400 msgr2=0x7f8bc8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bc806f400 0x7f8bc8071910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40012f0 tx=0x7f8bd4009040 comp rx=0 tx=0).stop 2024-04-25T13:44:16.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bdc156830 msgr2=0x7f8bdc0af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bdc156830 0x7f8bdc0af8b0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc0b0040 tx=0x7f8bcc019240 comp rx=0 tx=0).stop 2024-04-25T13:44:16.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 shutdown_connections 2024-04-25T13:44:16.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bc806f400 0x7f8bc8071910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8bdc156830 0x7f8bdc0af8b0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8bdc073c40 0x7f8bdc153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 --2- 172.21.15.80:0/3245280441 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc071e60 0x7f8bdc155880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 >> 172.21.15.80:0/3245280441 conn(0x7f8bdc0b4160 msgr2=0x7f8bdc0d0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:16.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 shutdown_connections 2024-04-25T13:44:16.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.676+0000 7f8be3984640 1 -- 172.21.15.80:0/3245280441 wait complete. 2024-04-25T13:44:16.696 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:44:16.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bc637a640 1 Processor -- start 2024-04-25T13:44:16.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bc637a640 1 -- start start 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bc637a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc0068ca0 0x7f5bc0144250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bc637a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc00b7640 con 0x7f5bc00b5a00 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bc637a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bc00b5e10 con 0x7f5bc015e740 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bc637a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc0144790 con 0x7f5bc0068ca0 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bbffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5bc00b5a00 0x7f5bc0068590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34510/0 (socket says 172.21.15.80:34510) 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bbffff640 1 -- 172.21.15.80:0/84503345 learned_addr learned my addr 172.21.15.80:0/84503345 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bbf7fe640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc0068ca0 0x7f5bc0144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3458828518 0 0) 0x7f5bb0001120 con 0x7f5bc00b5a00 2024-04-25T13:44:16.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.852+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ba8003530 con 0x7f5bc00b5a00 2024-04-25T13:44:16.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1310797942 0 0) 0x7f5ba8003530 con 0x7f5bc00b5a00 2024-04-25T13:44:16.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc0068ca0 msgr2=0x7f5bc0144250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:16.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc0068ca0 0x7f5bc0144250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 >> v1:172.21.15.87:6789/0 conn(0x7f5bc015e740 legacy=0x7f5bc015eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc0161dd0 con 0x7f5bc00b5a00 2024-04-25T13:44:16.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f5bb0001850 con 0x7f5bc00b5a00 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 >> v1:172.21.15.80:6789/0 conn(0x7f5bc00b5a00 legacy=0x7f5bc0068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5ba8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba8004810 0x7f5ba8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ba8005280 0x7f5ba800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc0144790 con 0x7f5ba80039f0 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5ba800bfc0 con 0x7f5ba8004810 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5ba800c190 con 0x7f5ba8005280 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbffff640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5ba8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbf7fe640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba8004810 0x7f5ba8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbffff640 1 -- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ba8005280 msgr2=0x7f5ba800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bc4917640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ba8005280 0x7f5ba800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbffff640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ba8005280 0x7f5ba800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbffff640 1 -- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba8004810 msgr2=0x7f5ba8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbffff640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba8004810 0x7f5ba8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbffff640 1 -- 172.21.15.80:0/84503345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ba8003530 con 0x7f5ba80039f0 2024-04-25T13:44:16.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbf7fe640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ba8004810 0x7f5ba8004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbffff640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5ba8003e60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c4c0 tx=0x7f5bb001cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b5bd455c65589d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000c0e0 con 0x7f5ba80039f0 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb00a9070 con 0x7f5ba80039f0 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bbeffd640 1 -- 172.21.15.80:0/84503345 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00a5db0 con 0x7f5ba80039f0 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bc637a640 1 -- 172.21.15.80:0/84503345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 msgr2=0x7f5ba8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bc637a640 1 --2- 172.21.15.80:0/84503345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5ba8003e60 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c4c0 tx=0x7f5bb001cc70 comp rx=0 tx=0).stop 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bc4917640 1 -- 172.21.15.80:0/84503345 reap_dead start 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bc637a640 1 -- 172.21.15.80:0/84503345 shutdown_connections 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bc637a640 1 -- 172.21.15.80:0/84503345 >> 172.21.15.80:0/84503345 conn(0x7f5bc00cfa50 msgr2=0x7f5bc00cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bc637a640 1 -- 172.21.15.80:0/84503345 shutdown_connections 2024-04-25T13:44:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.856+0000 7f5bc637a640 1 -- 172.21.15.80:0/84503345 wait complete. 2024-04-25T13:44:16.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 Processor -- start 2024-04-25T13:44:16.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 -- start start 2024-04-25T13:44:16.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5bc013f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc0140260 0x7f5bc006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc006f9f0 0x7f5bc00d0870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc00b6090 con 0x7f5ba80039f0 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc0162b90 con 0x7f5bc0140260 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc00b6210 con 0x7f5bc006f9f0 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5bc013f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5bc013f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43096/0 (socket says 172.21.15.80:43096) 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 -- 172.21.15.80:0/1485309470 learned_addr learned my addr 172.21.15.80:0/1485309470 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbf7fe640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc0140260 0x7f5bc006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc4917640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc006f9f0 0x7f5bc00d0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 -- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc006f9f0 msgr2=0x7f5bc00d0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc006f9f0 0x7f5bc00d0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 -- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc0140260 msgr2=0x7f5bc006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc0140260 0x7f5bc006f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:16.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 -- 172.21.15.80:0/1485309470 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc01677d0 con 0x7f5ba80039f0 2024-04-25T13:44:16.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc4917640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc006f9f0 0x7f5bc00d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:16.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbffff640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5bc013f9a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c4c0 tx=0x7f5ba8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbcff9640 1 -- 172.21.15.80:0/1485309470 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00a5950 con 0x7f5ba80039f0 2024-04-25T13:44:16.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbcff9640 1 -- 172.21.15.80:0/1485309470 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bb00a5af0 con 0x7f5ba80039f0 2024-04-25T13:44:16.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc0166fd0 con 0x7f5ba80039f0 2024-04-25T13:44:16.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bbcff9640 1 -- 172.21.15.80:0/1485309470 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb00049f0 con 0x7f5ba80039f0 2024-04-25T13:44:16.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.860+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc0070180 con 0x7f5ba80039f0 2024-04-25T13:44:16.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.864+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bc01674f0 con 0x7f5ba80039f0 2024-04-25T13:44:16.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.864+0000 7f5bbcff9640 1 -- 172.21.15.80:0/1485309470 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5bb0004b90 con 0x7f5ba80039f0 2024-04-25T13:44:16.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.864+0000 7f5bbcff9640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5bac06f400 0x7f5bac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:16.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.864+0000 7f5bbcff9640 1 -- 172.21.15.80:0/1485309470 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f5bb0128fe0 con 0x7f5ba80039f0 2024-04-25T13:44:16.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.864+0000 7f5bbcff9640 1 -- 172.21.15.80:0/1485309470 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f5bac076f80 con 0x7f5ba80039f0 2024-04-25T13:44:16.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.864+0000 7f5bbf7fe640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5bac06f400 0x7f5bac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:16.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.864+0000 7f5bbf7fe640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5bac06f400 0x7f5bac071910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40090e0 tx=0x7f5bb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:16.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:16.868+0000 7f5bbcff9640 1 -- 172.21.15.80:0/1485309470 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5bc01674f0 con 0x7f5ba80039f0 2024-04-25T13:44:17.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.024+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5bc006a3a0 con 0x7f5ba80039f0 2024-04-25T13:44:17.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bbcff9640 1 -- 172.21.15.80:0/1485309470 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7f5bb00bf060 con 0x7f5ba80039f0 2024-04-25T13:44:17.026 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5bac06f400 msgr2=0x7f5bac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5bac06f400 0x7f5bac071910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40090e0 tx=0x7f5bb4008040 comp rx=0 tx=0).stop 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 msgr2=0x7f5bc013f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5bc013f9a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000c4c0 tx=0x7f5ba8005840 comp rx=0 tx=0).stop 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 shutdown_connections 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5bac06f400 0x7f5bac071910 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5bc006f9f0 0x7f5bc00d0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5bc0140260 0x7f5bc006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 --2- 172.21.15.80:0/1485309470 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ba80039f0 0x7f5bc013f9a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 >> 172.21.15.80:0/1485309470 conn(0x7f5bc00cfa50 msgr2=0x7f5bc015cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 shutdown_connections 2024-04-25T13:44:17.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.028+0000 7f5bc637a640 1 -- 172.21.15.80:0/1485309470 wait complete. 2024-04-25T13:44:17.048 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:44:17.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14e6f6640 1 Processor -- start 2024-04-25T13:44:17.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14e6f6640 1 -- start start 2024-04-25T13:44:17.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14e6f6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd148161f90 0x7fd1480cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14e6f6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd1480d0510 0x7fd14815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14e6f6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1480d0d70 con 0x7fd14815ea70 2024-04-25T13:44:17.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14e6f6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd1480d1050 con 0x7fd1480d0510 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14e6f6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd14806dca0 con 0x7fd148161f90 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14cc93640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd14815ea70 0x7fd14815ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34514/0 (socket says 172.21.15.80:34514) 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd14cc93640 1 -- 172.21.15.80:0/732284356 learned_addr learned my addr 172.21.15.80:0/732284356 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd147fff640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd148161f90 0x7fd1480cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd1477fe640 1 --2- 172.21.15.80:0/732284356 >> v2:172.21.15.87:3300/0 conn(0x7fd1480d0510 0x7fd14815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.192+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2450838639 0 0) 0x7fd1480d0d70 con 0x7fd14815ea70 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd134003530 con 0x7fd14815ea70 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1832139707 0 0) 0x7fd134003530 con 0x7fd14815ea70 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd148161f90 msgr2=0x7fd1480cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd148161f90 0x7fd1480cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 >> v2:172.21.15.87:3300/0 conn(0x7fd1480d0510 msgr2=0x7fd14815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 --2- 172.21.15.80:0/732284356 >> v2:172.21.15.87:3300/0 conn(0x7fd1480d0510 0x7fd14815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14806df80 con 0x7fd14815ea70 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fd138001a00 con 0x7fd14815ea70 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 >> v1:172.21.15.80:6789/0 conn(0x7fd14815ea70 legacy=0x7fd14815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1340039c0 0x7fd134003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd134004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd148161f90 0x7fd134005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd1480d1050 con 0x7fd134004810 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd14806dca0 con 0x7fd1340039c0 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd13400ba80 con 0x7fd148161f90 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd1477fe640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd134004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd14cc93640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd148161f90 0x7fd134005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd147fff640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1340039c0 0x7fd134003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd1477fe640 1 -- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd148161f90 msgr2=0x7fd134005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd1477fe640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd148161f90 0x7fd134005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd1477fe640 1 -- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1340039c0 msgr2=0x7fd134003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:17.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd1477fe640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1340039c0 0x7fd134003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd1477fe640 1 -- 172.21.15.80:0/732284356 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd134003530 con 0x7fd134004810 2024-04-25T13:44:17.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd147fff640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1340039c0 0x7fd134003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:17.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd1477fe640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd134004c20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd148074290 tx=0x7fd13001c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b04144ec59e64a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:17.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300096b0 con 0x7fd134004810 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1300a6480 con 0x7fd134004810 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.196+0000 7fd146ffd640 1 -- 172.21.15.80:0/732284356 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1300a6870 con 0x7fd134004810 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- 172.21.15.80:0/732284356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 msgr2=0x7fd134004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 --2- 172.21.15.80:0/732284356 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd134004c20 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fd148074290 tx=0x7fd13001c990 comp rx=0 tx=0).stop 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14cc93640 1 -- 172.21.15.80:0/732284356 reap_dead start 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- 172.21.15.80:0/732284356 shutdown_connections 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- 172.21.15.80:0/732284356 >> 172.21.15.80:0/732284356 conn(0x7fd14806d080 msgr2=0x7fd1480b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- 172.21.15.80:0/732284356 shutdown_connections 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- 172.21.15.80:0/732284356 wait complete. 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 Processor -- start 2024-04-25T13:44:17.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- start start 2024-04-25T13:44:17.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd14815d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd14815fff0 0x7fd148160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14813f000 0x7fd14813f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd14806f4b0 con 0x7fd134004810 2024-04-25T13:44:17.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd14806f630 con 0x7fd14813f000 2024-04-25T13:44:17.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14e6f6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd1480d0920 con 0x7fd14815fff0 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd147fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd14815d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd147fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd14815d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43110/0 (socket says 172.21.15.80:43110) 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd147fff640 1 -- 172.21.15.80:0/1905952140 learned_addr learned my addr 172.21.15.80:0/1905952140 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd1477fe640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd14815fff0 0x7fd148160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.200+0000 7fd14cc93640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14813f000 0x7fd14813f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd147fff640 1 -- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd14815fff0 msgr2=0x7fd148160400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd147fff640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd14815fff0 0x7fd148160400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd147fff640 1 -- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14813f000 msgr2=0x7fd14813f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd147fff640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14813f000 0x7fd14813f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd147fff640 1 -- 172.21.15.80:0/1905952140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd14813fd80 con 0x7fd134004810 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd1477fe640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd14815fff0 0x7fd148160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd147fff640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd14815d880 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd140006c10 tx=0x7fd1400119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd144ff9640 1 -- 172.21.15.80:0/1905952140 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140015870 con 0x7fd134004810 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd144ff9640 1 -- 172.21.15.80:0/1905952140 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd140015a10 con 0x7fd134004810 2024-04-25T13:44:17.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd148140060 con 0x7fd134004810 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd144ff9640 1 -- 172.21.15.80:0/1905952140 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd140015de0 con 0x7fd134004810 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1480b11e0 con 0x7fd134004810 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd14806a3a0 con 0x7fd134004810 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd144ff9640 1 -- 172.21.15.80:0/1905952140 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd14001d400 con 0x7fd134004810 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd144ff9640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12806f400 0x7fd128071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd144ff9640 1 -- 172.21.15.80:0/1905952140 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7fd14008fbe0 con 0x7fd134004810 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd144ff9640 1 -- 172.21.15.80:0/1905952140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7fd128076f80 con 0x7fd134004810 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd1477fe640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12806f400 0x7fd128071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.204+0000 7fd1477fe640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12806f400 0x7fd128071910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd148074290 tx=0x7fd1300b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:17.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.208+0000 7fd144ff9640 1 -- 172.21.15.80:0/1905952140 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd140019030 con 0x7fd134004810 2024-04-25T13:44:17.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.372+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd1480b5720 con 0x7fd134004810 2024-04-25T13:44:17.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.372+0000 7fd144ff9640 1 -- 172.21.15.80:0/1905952140 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) ==== 80+0+0 (secure 0 0 0) 0x7fd14005cc40 con 0x7fd134004810 2024-04-25T13:44:17.372 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:44:17.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12806f400 msgr2=0x7fd128071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12806f400 0x7fd128071910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd148074290 tx=0x7fd1300b2040 comp rx=0 tx=0).stop 2024-04-25T13:44:17.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 msgr2=0x7fd14815d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd14815d880 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fd140006c10 tx=0x7fd1400119f0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 shutdown_connections 2024-04-25T13:44:17.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12806f400 0x7fd128071910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd14813f000 0x7fd14813f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd14815fff0 0x7fd148160400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 --2- 172.21.15.80:0/1905952140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd134004810 0x7fd14815d880 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 >> 172.21.15.80:0/1905952140 conn(0x7fd14806d080 msgr2=0x7fd1480b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:17.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 shutdown_connections 2024-04-25T13:44:17.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.376+0000 7fd14e6f6640 1 -- 172.21.15.80:0/1905952140 wait complete. 2024-04-25T13:44:17.393 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:44:17.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:17.393 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:44:17.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[31848]: fuse finished with error 0 and tester_r 0 2024-04-25T13:44:17.404 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:44:17.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:17.404 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:44:17.514 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:44:17.515 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:44:17.515 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:17.521 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:44:17.521 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:17.571 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:17.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.720+0000 7f377c9ef640 1 Processor -- start 2024-04-25T13:44:17.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.720+0000 7f377c9ef640 1 -- start start 2024-04-25T13:44:17.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f377c9ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37781644a0 0x7f37780b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f377c9ef640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f37780b5450 0x7f37780b5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f377c9ef640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f377806ea30 con 0x7f37780731e0 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f377c9ef640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f377815acb0 con 0x7f37780b5450 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f377c9ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f377806ed10 con 0x7f37781644a0 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f377659c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f37780731e0 0x7f3778163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34526/0 (socket says 172.21.15.80:34526) 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f377659c640 1 -- 172.21.15.80:0/1712096498 learned_addr learned my addr 172.21.15.80:0/1712096498 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f3776d9d640 1 --2- 172.21.15.80:0/1712096498 >> v2:172.21.15.80:3300/0 conn(0x7f37780b5450 0x7f37780b5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f3775d9b640 1 --2- 172.21.15.80:0/1712096498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37781644a0 0x7f37780b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f3776d9d640 1 -- 172.21.15.80:0/1712096498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37781644a0 msgr2=0x7f37780b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f3776d9d640 1 --2- 172.21.15.80:0/1712096498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37781644a0 0x7f37780b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f3776d9d640 1 -- 172.21.15.80:0/1712096498 >> v1:172.21.15.80:6789/0 conn(0x7f37780731e0 legacy=0x7f3778163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f377559a640 1 -- 172.21.15.80:0/1712096498 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 393071835 0 0) 0x7f377806ea30 con 0x7f37780731e0 2024-04-25T13:44:17.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.724+0000 7f3776d9d640 1 -- 172.21.15.80:0/1712096498 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377806eff0 con 0x7f37780b5450 2024-04-25T13:44:17.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f3776d9d640 1 --2- 172.21.15.80:0/1712096498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37780b5450 0x7f37780b5860 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3770009d90 tx=0x7f377001c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d95b53c8906b5c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:17.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377559a640 1 -- 172.21.15.80:0/1712096498 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37700b2070 con 0x7f37780b5450 2024-04-25T13:44:17.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377559a640 1 -- 172.21.15.80:0/1712096498 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37700a5940 con 0x7f37780b5450 2024-04-25T13:44:17.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377559a640 1 -- 172.21.15.80:0/1712096498 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37700a5d10 con 0x7f37780b5450 2024-04-25T13:44:17.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 -- 172.21.15.80:0/1712096498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37780b5450 msgr2=0x7f37780b5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1712096498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37780b5450 0x7f37780b5860 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f3770009d90 tx=0x7f377001c970 comp rx=0 tx=0).stop 2024-04-25T13:44:17.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 -- 172.21.15.80:0/1712096498 shutdown_connections 2024-04-25T13:44:17.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1712096498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f37780b5450 0x7f37780b5860 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1712096498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37781644a0 0x7f37780b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 -- 172.21.15.80:0/1712096498 >> 172.21.15.80:0/1712096498 conn(0x7f377806d080 msgr2=0x7f37780b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:17.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 -- 172.21.15.80:0/1712096498 shutdown_connections 2024-04-25T13:44:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 -- 172.21.15.80:0/1712096498 wait complete. 2024-04-25T13:44:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 Processor -- start 2024-04-25T13:44:17.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.728+0000 7f377c9ef640 1 -- start start 2024-04-25T13:44:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37780731e0 0x7f377815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37780b5450 0x7f377814f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377815a720 0x7f377814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f377806e7d0 con 0x7f377815a720 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f377806e650 con 0x7f37780b5450 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f377816d940 con 0x7f37780731e0 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377815a720 0x7f377814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377815a720 0x7f377814e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43128/0 (socket says 172.21.15.80:43128) 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 -- 172.21.15.80:0/1488504334 learned_addr learned my addr 172.21.15.80:0/1488504334 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377659c640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37780731e0 0x7f377815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3775d9b640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37780b5450 0x7f377814f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 -- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37780731e0 msgr2=0x7f377815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37780731e0 0x7f377815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 -- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37780b5450 msgr2=0x7f377814f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37780b5450 0x7f377814f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 -- 172.21.15.80:0/1488504334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3778159050 con 0x7f377815a720 2024-04-25T13:44:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3775d9b640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37780b5450 0x7f377814f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3776d9d640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377815a720 0x7f377814e590 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3770009d90 tx=0x7f3770019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3766ffd640 1 -- 172.21.15.80:0/1488504334 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37700b2040 con 0x7f377815a720 2024-04-25T13:44:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3766ffd640 1 -- 172.21.15.80:0/1488504334 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37700b8470 con 0x7f377815a720 2024-04-25T13:44:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3778159330 con 0x7f377815a720 2024-04-25T13:44:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3766ffd640 1 -- 172.21.15.80:0/1488504334 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37700b8840 con 0x7f377815a720 2024-04-25T13:44:17.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f377815c760 con 0x7f377815a720 2024-04-25T13:44:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f377806a3a0 con 0x7f377815a720 2024-04-25T13:44:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3766ffd640 1 -- 172.21.15.80:0/1488504334 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f37700c1020 con 0x7f377815a720 2024-04-25T13:44:17.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.732+0000 7f3766ffd640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f375806f400 0x7f3758071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:17.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.736+0000 7f3766ffd640 1 -- 172.21.15.80:0/1488504334 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f3770129ae0 con 0x7f377815a720 2024-04-25T13:44:17.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.736+0000 7f3766ffd640 1 -- 172.21.15.80:0/1488504334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f3758076fe0 con 0x7f377815a720 2024-04-25T13:44:17.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.736+0000 7f377659c640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f375806f400 0x7f3758071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:17.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.736+0000 7f377659c640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f375806f400 0x7f3758071910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3768000f70 tx=0x7f3768007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:17.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.740+0000 7f3766ffd640 1 -- 172.21.15.80:0/1488504334 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f37700c8020 con 0x7f377815a720 2024-04-25T13:44:17.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.956+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37780b5b30 con 0x7f377815a720 2024-04-25T13:44:17.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.956+0000 7f3766ffd640 1 -- 172.21.15.80:0/1488504334 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) ==== 74+0+16651 (secure 0 0 0) 0x7f3770016560 con 0x7f377815a720 2024-04-25T13:44:17.957 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:17.957 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":66,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:43:36.269783+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:43:11.515229+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T13:43:12.143130+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f375806f400 msgr2=0x7f3758071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f375806f400 0x7f3758071910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f3768000f70 tx=0x7f3768007c10 comp rx=0 tx=0).stop 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377815a720 msgr2=0x7f377814e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377815a720 0x7f377814e590 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f3770009d90 tx=0x7f3770019710 comp rx=0 tx=0).stop 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 shutdown_connections 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f375806f400 0x7f3758071910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f377815a720 0x7f377814e590 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f37780b5450 0x7f377814f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 --2- 172.21.15.80:0/1488504334 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f37780731e0 0x7f377815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:17.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 >> 172.21.15.80:0/1488504334 conn(0x7f377806d080 msgr2=0x7f3778161940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:17.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 shutdown_connections 2024-04-25T13:44:17.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:17.960+0000 7f377c9ef640 1 -- 172.21.15.80:0/1488504334 wait complete. 2024-04-25T13:44:17.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:43:11.515229+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:43:12.143130+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:44:17.978 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:44:18.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.120+0000 7f3179798640 1 Processor -- start 2024-04-25T13:44:18.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.120+0000 7f3179798640 1 -- start start 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3179798640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317406edb0 0x7f317406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3179798640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3174161b60 0x7f3174161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3179798640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31741624b0 con 0x7f31740b5330 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3179798640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f317406f700 con 0x7f3174161b60 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3179798640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f31741629b0 con 0x7f317406edb0 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3172ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f31740b5330 0x7f31740d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34536/0 (socket says 172.21.15.80:34536) 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3172ffd640 1 -- 172.21.15.80:0/4226891099 learned_addr learned my addr 172.21.15.80:0/4226891099 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31727fc640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317406edb0 0x7f317406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31737fe640 1 --2- 172.21.15.80:0/4226891099 >> v2:172.21.15.87:3300/0 conn(0x7f3174161b60 0x7f3174161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1090270571 0 0) 0x7f3164001120 con 0x7f31740b5330 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3160003530 con 0x7f31740b5330 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4001251007 0 0) 0x7f3160003530 con 0x7f31740b5330 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317406edb0 msgr2=0x7f317406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317406edb0 0x7f317406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 >> v2:172.21.15.87:3300/0 conn(0x7f3174161b60 msgr2=0x7f3174161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 --2- 172.21.15.80:0/4226891099 >> v2:172.21.15.87:3300/0 conn(0x7f3174161b60 0x7f3174161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31740b5740 con 0x7f31740b5330 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f3164001850 con 0x7f31740b5330 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 >> v1:172.21.15.80:6789/0 conn(0x7f31740b5330 legacy=0x7f31740d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317406edb0 0x7f31600039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3160003f40 0x7f3160004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3160004d50 0x7f3160005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f317406f700 con 0x7f3160004d50 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f31741629b0 con 0x7f3160003f40 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f316000bad0 con 0x7f317406edb0 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31737fe640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3160004d50 0x7f3160005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f3172ffd640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317406edb0 0x7f31600039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31727fc640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3160003f40 0x7f3160004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31737fe640 1 -- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317406edb0 msgr2=0x7f31600039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31737fe640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317406edb0 0x7f31600039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31737fe640 1 -- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3160003f40 msgr2=0x7f3160004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31737fe640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3160003f40 0x7f3160004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.124+0000 7f31737fe640 1 -- 172.21.15.80:0/4226891099 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3160003530 con 0x7f3160004d50 2024-04-25T13:44:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f31737fe640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3160004d50 0x7f3160005160 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f316c009f90 tx=0x7f316c01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c06f8da8b5e19d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0b2070 con 0x7f3160004d50 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c0a5ca0 con 0x7f3160004d50 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3171ffb640 1 -- 172.21.15.80:0/4226891099 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c009c10 con 0x7f3160004d50 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3179798640 1 -- 172.21.15.80:0/4226891099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3160004d50 msgr2=0x7f3160005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3179798640 1 --2- 172.21.15.80:0/4226891099 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3160004d50 0x7f3160005160 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f316c009f90 tx=0x7f316c01cc30 comp rx=0 tx=0).stop 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f31737fe640 1 -- 172.21.15.80:0/4226891099 reap_dead start 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3179798640 1 -- 172.21.15.80:0/4226891099 shutdown_connections 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3179798640 1 -- 172.21.15.80:0/4226891099 >> 172.21.15.80:0/4226891099 conn(0x7f31740b4160 msgr2=0x7f31740d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3179798640 1 -- 172.21.15.80:0/4226891099 shutdown_connections 2024-04-25T13:44:18.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.128+0000 7f3179798640 1 -- 172.21.15.80:0/4226891099 wait complete. 2024-04-25T13:44:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 Processor -- start 2024-04-25T13:44:18.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 -- start start 2024-04-25T13:44:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415e510 0x7f3174139b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317415bf80 0x7f31740ae500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31740aea40 0x7f31740cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f317406e2d0 con 0x7f31740aea40 2024-04-25T13:44:18.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f317406dff0 con 0x7f317415bf80 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f317406e5b0 con 0x7f317415e510 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31740aea40 0x7f31740cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31740aea40 0x7f31740cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43140/0 (socket says 172.21.15.80:43140) 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 -- 172.21.15.80:0/846495049 learned_addr learned my addr 172.21.15.80:0/846495049 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3172ffd640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415e510 0x7f3174139b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31727fc640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317415bf80 0x7f31740ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 -- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415e510 msgr2=0x7f3174139b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415e510 0x7f3174139b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 -- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317415bf80 msgr2=0x7f31740ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317415bf80 0x7f31740ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 -- 172.21.15.80:0/846495049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31740aee50 con 0x7f31740aea40 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31727fc640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317415bf80 0x7f31740ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f31737fe640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31740aea40 0x7f31740cd8a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f316c0b0040 tx=0x7f316c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f316b7fe640 1 -- 172.21.15.80:0/846495049 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c0b2040 con 0x7f31740aea40 2024-04-25T13:44:18.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f316b7fe640 1 -- 172.21.15.80:0/846495049 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f316c0a5e50 con 0x7f31740aea40 2024-04-25T13:44:18.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f317413a1e0 con 0x7f31740aea40 2024-04-25T13:44:18.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f316b7fe640 1 -- 172.21.15.80:0/846495049 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c009c10 con 0x7f31740aea40 2024-04-25T13:44:18.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f317413a7a0 con 0x7f31740aea40 2024-04-25T13:44:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.132+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317406a3a0 con 0x7f31740aea40 2024-04-25T13:44:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.136+0000 7f316b7fe640 1 -- 172.21.15.80:0/846495049 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f316c0bf060 con 0x7f31740aea40 2024-04-25T13:44:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.136+0000 7f316b7fe640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f315406f400 0x7f3154071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.136+0000 7f316b7fe640 1 -- 172.21.15.80:0/846495049 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f316c129240 con 0x7f31740aea40 2024-04-25T13:44:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.136+0000 7f316b7fe640 1 -- 172.21.15.80:0/846495049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f3154076fe0 con 0x7f31740aea40 2024-04-25T13:44:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.136+0000 7f3172ffd640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f315406f400 0x7f3154071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.136+0000 7f3172ffd640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f315406f400 0x7f3154071910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f31741f7630 tx=0x7f3164001220 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.140+0000 7f316b7fe640 1 -- 172.21.15.80:0/846495049 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31740aee50 con 0x7f31740aea40 2024-04-25T13:44:18.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.300+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f317406fd00 con 0x7f31740aea40 2024-04-25T13:44:18.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.300+0000 7f316b7fe640 1 -- 172.21.15.80:0/846495049 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) ==== 93+0+10 (secure 0 0 0) 0x7f316c016990 con 0x7f31740aea40 2024-04-25T13:44:18.301 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f315406f400 msgr2=0x7f3154071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f315406f400 0x7f3154071910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f31741f7630 tx=0x7f3164001220 comp rx=0 tx=0).stop 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31740aea40 msgr2=0x7f31740cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31740aea40 0x7f31740cd8a0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f316c0b0040 tx=0x7f316c009750 comp rx=0 tx=0).stop 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 shutdown_connections 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f315406f400 0x7f3154071910 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f31740aea40 0x7f31740cd8a0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f317415bf80 0x7f31740ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 --2- 172.21.15.80:0/846495049 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f317415e510 0x7f3174139b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 >> 172.21.15.80:0/846495049 conn(0x7f31740b4160 msgr2=0x7f31740b67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:18.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 shutdown_connections 2024-04-25T13:44:18.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.304+0000 7f3179798640 1 -- 172.21.15.80:0/846495049 wait complete. 2024-04-25T13:44:18.323 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:44:18.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed6642640 1 Processor -- start 2024-04-25T13:44:18.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed6642640 1 -- start start 2024-04-25T13:44:18.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed6642640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0073080 0x7f2ed0073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed6642640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ed015c710 0x7f2ed006e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed6642640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ed006edc0 con 0x7f2ed006e9b0 2024-04-25T13:44:18.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed6642640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2ed00b53d0 con 0x7f2ed015c710 2024-04-25T13:44:18.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed6642640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed00cf640 con 0x7f2ed0073080 2024-04-25T13:44:18.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ecf7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ed015c710 0x7f2ed006e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed4bdf640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2ed006e9b0 0x7f2ed0071e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34544/0 (socket says 172.21.15.80:34544) 2024-04-25T13:44:18.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ed4bdf640 1 -- 172.21.15.80:0/3517770402 learned_addr learned my addr 172.21.15.80:0/3517770402 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:18.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ecffff640 1 --2- 172.21.15.80:0/3517770402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0073080 0x7f2ed0073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ecf7fe640 1 -- 172.21.15.80:0/3517770402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0073080 msgr2=0x7f2ed0073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:18.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ecf7fe640 1 --2- 172.21.15.80:0/3517770402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0073080 0x7f2ed0073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ecf7fe640 1 -- 172.21.15.80:0/3517770402 >> v1:172.21.15.80:6789/0 conn(0x7f2ed006e9b0 legacy=0x7f2ed0071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.484+0000 7f2ecf7fe640 1 -- 172.21.15.80:0/3517770402 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed00737f0 con 0x7f2ed015c710 2024-04-25T13:44:18.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ecf7fe640 1 --2- 172.21.15.80:0/3517770402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed015c710 0x7f2ed006e470 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40097c0 tx=0x7f2ec401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cd08e2ead0b9635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2eceffd640 1 -- 172.21.15.80:0/3517770402 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40b0070 con 0x7f2ed015c710 2024-04-25T13:44:18.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2eceffd640 1 -- 172.21.15.80:0/3517770402 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec40ab600 con 0x7f2ed015c710 2024-04-25T13:44:18.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2eceffd640 1 -- 172.21.15.80:0/3517770402 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40ab9d0 con 0x7f2ed015c710 2024-04-25T13:44:18.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 -- 172.21.15.80:0/3517770402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed015c710 msgr2=0x7f2ed006e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 --2- 172.21.15.80:0/3517770402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed015c710 0x7f2ed006e470 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40097c0 tx=0x7f2ec401c410 comp rx=0 tx=0).stop 2024-04-25T13:44:18.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 -- 172.21.15.80:0/3517770402 shutdown_connections 2024-04-25T13:44:18.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 --2- 172.21.15.80:0/3517770402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed015c710 0x7f2ed006e470 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 --2- 172.21.15.80:0/3517770402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0073080 0x7f2ed0073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 -- 172.21.15.80:0/3517770402 >> 172.21.15.80:0/3517770402 conn(0x7f2ed00b4160 msgr2=0x7f2ed00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:18.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 -- 172.21.15.80:0/3517770402 shutdown_connections 2024-04-25T13:44:18.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 -- 172.21.15.80:0/3517770402 wait complete. 2024-04-25T13:44:18.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 Processor -- start 2024-04-25T13:44:18.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.488+0000 7f2ed6642640 1 -- start start 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006e9b0 0x7f2ed0150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0073080 0x7f2ed0155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed015c710 0x7f2ed0156130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed015cb20 con 0x7f2ed0073080 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed0072570 con 0x7f2ed006e9b0 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed014de10 con 0x7f2ed015c710 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0073080 0x7f2ed0155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0073080 0x7f2ed0155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43152/0 (socket says 172.21.15.80:43152) 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 -- 172.21.15.80:0/1454451436 learned_addr learned my addr 172.21.15.80:0/1454451436 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed4bdf640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed015c710 0x7f2ed0156130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 -- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed015c710 msgr2=0x7f2ed0156130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed015c710 0x7f2ed0156130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 -- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006e9b0 msgr2=0x7f2ed0150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006e9b0 0x7f2ed0150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 -- 172.21.15.80:0/1454451436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed015be10 con 0x7f2ed0073080 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ecf7fe640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0073080 0x7f2ed0155bf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2ec401a040 tx=0x7f2ec40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2eccff9640 1 -- 172.21.15.80:0/1454451436 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40b0040 con 0x7f2ed0073080 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2eccff9640 1 -- 172.21.15.80:0/1454451436 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ec40ab400 con 0x7f2ed0073080 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ed0150dd0 con 0x7f2ed0073080 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2eccff9640 1 -- 172.21.15.80:0/1454451436 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec40b4650 con 0x7f2ed0073080 2024-04-25T13:44:18.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ed014e340 con 0x7f2ed0073080 2024-04-25T13:44:18.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.492+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ed006a3a0 con 0x7f2ed0073080 2024-04-25T13:44:18.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.496+0000 7f2eccff9640 1 -- 172.21.15.80:0/1454451436 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2ec40bc020 con 0x7f2ed0073080 2024-04-25T13:44:18.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.496+0000 7f2eccff9640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ebc06f480 0x7f2ebc071990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.496+0000 7f2eccff9640 1 -- 172.21.15.80:0/1454451436 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f2ec41294e0 con 0x7f2ed0073080 2024-04-25T13:44:18.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.496+0000 7f2eccff9640 1 -- 172.21.15.80:0/1454451436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f2ebc077000 con 0x7f2ed0073080 2024-04-25T13:44:18.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.496+0000 7f2ecffff640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ebc06f480 0x7f2ebc071990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.496+0000 7f2ecffff640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ebc06f480 0x7f2ebc071990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0160290 tx=0x7f2ec0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.500+0000 7f2eccff9640 1 -- 172.21.15.80:0/1454451436 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ec40a4e20 con 0x7f2ed0073080 2024-04-25T13:44:18.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.656+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2ed006f840 con 0x7f2ed0073080 2024-04-25T13:44:18.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.656+0000 7f2eccff9640 1 -- 172.21.15.80:0/1454451436 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) ==== 104+0+11 (secure 0 0 0) 0x7f2ec40f9020 con 0x7f2ed0073080 2024-04-25T13:44:18.657 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:44:18.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ebc06f480 msgr2=0x7f2ebc071990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ebc06f480 0x7f2ebc071990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0160290 tx=0x7f2ec0008040 comp rx=0 tx=0).stop 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0073080 msgr2=0x7f2ed0155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0073080 0x7f2ed0155bf0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f2ec401a040 tx=0x7f2ec40191b0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 shutdown_connections 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ebc06f480 0x7f2ebc071990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed015c710 0x7f2ed0156130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0073080 0x7f2ed0155bf0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 --2- 172.21.15.80:0/1454451436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed006e9b0 0x7f2ed0150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 >> 172.21.15.80:0/1454451436 conn(0x7f2ed00b4160 msgr2=0x7f2ed00d0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 shutdown_connections 2024-04-25T13:44:18.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.660+0000 7f2ed6642640 1 -- 172.21.15.80:0/1454451436 wait complete. 2024-04-25T13:44:18.679 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:44:18.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.828+0000 7f4bd82ef640 1 Processor -- start 2024-04-25T13:44:18.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.828+0000 7f4bd82ef640 1 -- start start 2024-04-25T13:44:18.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.828+0000 7f4bd82ef640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4bd006edb0 0x7f4bd006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.828+0000 7f4bd82ef640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4bd0161b60 0x7f4bd0161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.828+0000 7f4bd82ef640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd01624b0 con 0x7f4bd00b5330 2024-04-25T13:44:18.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.828+0000 7f4bd82ef640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd006f700 con 0x7f4bd006edb0 2024-04-25T13:44:18.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.828+0000 7f4bd82ef640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4bd01629b0 con 0x7f4bd0161b60 2024-04-25T13:44:18.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd588a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4bd006edb0 0x7f4bd006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd588a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4bd006edb0 0x7f4bd006f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43160/0 (socket says 172.21.15.80:43160) 2024-04-25T13:44:18.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd588a640 1 -- 172.21.15.80:0/4178684946 learned_addr learned my addr 172.21.15.80:0/4178684946 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:18.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd688c640 1 --2- 172.21.15.80:0/4178684946 >> v2:172.21.15.87:3300/0 conn(0x7f4bd0161b60 0x7f4bd0161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd588a640 1 -- 172.21.15.80:0/4178684946 >> v2:172.21.15.87:3300/0 conn(0x7f4bd0161b60 msgr2=0x7f4bd0161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:18.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd588a640 1 --2- 172.21.15.80:0/4178684946 >> v2:172.21.15.87:3300/0 conn(0x7f4bd0161b60 0x7f4bd0161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd588a640 1 -- 172.21.15.80:0/4178684946 >> v1:172.21.15.87:6789/0 conn(0x7f4bd00b5330 legacy=0x7f4bd00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:18.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd588a640 1 -- 172.21.15.80:0/4178684946 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd00b5740 con 0x7f4bd006edb0 2024-04-25T13:44:18.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd688c640 1 --2- 172.21.15.80:0/4178684946 >> v2:172.21.15.87:3300/0 conn(0x7f4bd0161b60 0x7f4bd0161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:18.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd588a640 1 --2- 172.21.15.80:0/4178684946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd006edb0 0x7f4bd006f1c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0147cb0 tx=0x7f4bc001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca85162ea4d8eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd5089640 1 -- 172.21.15.80:0/4178684946 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00b0070 con 0x7f4bd006edb0 2024-04-25T13:44:18.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd5089640 1 -- 172.21.15.80:0/4178684946 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc00ab600 con 0x7f4bd006edb0 2024-04-25T13:44:18.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd5089640 1 -- 172.21.15.80:0/4178684946 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc00ab9d0 con 0x7f4bd006edb0 2024-04-25T13:44:18.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4178684946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd006edb0 msgr2=0x7f4bd006f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4178684946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd006edb0 0x7f4bd006f1c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0147cb0 tx=0x7f4bc001c240 comp rx=0 tx=0).stop 2024-04-25T13:44:18.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4178684946 shutdown_connections 2024-04-25T13:44:18.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4178684946 >> v2:172.21.15.87:3300/0 conn(0x7f4bd0161b60 0x7f4bd0161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4178684946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd006edb0 0x7f4bd006f1c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4178684946 >> 172.21.15.80:0/4178684946 conn(0x7f4bd00b4160 msgr2=0x7f4bd00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:18.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4178684946 shutdown_connections 2024-04-25T13:44:18.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.832+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4178684946 wait complete. 2024-04-25T13:44:18.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd82ef640 1 Processor -- start 2024-04-25T13:44:18.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd82ef640 1 -- start start 2024-04-25T13:44:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd82ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bd006edb0 0x7f4bd01fd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd82ef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bd00b5330 0x7f4bd014e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd82ef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd0161b60 0x7f4bd00af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd82ef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd006e890 con 0x7f4bd0161b60 2024-04-25T13:44:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd82ef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd006e5b0 con 0x7f4bd00b5330 2024-04-25T13:44:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd82ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd00b0cd0 con 0x7f4bd006edb0 2024-04-25T13:44:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd0161b60 0x7f4bd00af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd0161b60 0x7f4bd00af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43162/0 (socket says 172.21.15.80:43162) 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 -- 172.21.15.80:0/4190693510 learned_addr learned my addr 172.21.15.80:0/4190693510 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd588a640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bd00b5330 0x7f4bd014e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd608b640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bd006edb0 0x7f4bd01fd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 -- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bd006edb0 msgr2=0x7f4bd01fd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bd006edb0 0x7f4bd01fd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 -- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bd00b5330 msgr2=0x7f4bd014e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bd00b5330 0x7f4bd014e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 -- 172.21.15.80:0/4190693510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd014f710 con 0x7f4bd0161b60 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd608b640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bd006edb0 0x7f4bd01fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.836+0000 7f4bd688c640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd0161b60 0x7f4bd00af8b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800cd50 tx=0x7f4bc800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bc6ffd640 1 -- 172.21.15.80:0/4190693510 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800f680 con 0x7f4bd0161b60 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bc6ffd640 1 -- 172.21.15.80:0/4190693510 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc800f840 con 0x7f4bd0161b60 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd0072330 con 0x7f4bd0161b60 2024-04-25T13:44:18.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bc6ffd640 1 -- 172.21.15.80:0/4190693510 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc800fc30 con 0x7f4bd0161b60 2024-04-25T13:44:18.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd00728f0 con 0x7f4bd0161b60 2024-04-25T13:44:18.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bd014f460 con 0x7f4bd0161b60 2024-04-25T13:44:18.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bc6ffd640 1 -- 172.21.15.80:0/4190693510 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4bc80213f0 con 0x7f4bd0161b60 2024-04-25T13:44:18.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bc6ffd640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb406f400 0x7f4bb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:18.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bc6ffd640 1 -- 172.21.15.80:0/4190693510 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f4bc8090340 con 0x7f4bd0161b60 2024-04-25T13:44:18.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bc6ffd640 1 -- 172.21.15.80:0/4190693510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f4bb4076fe0 con 0x7f4bd0161b60 2024-04-25T13:44:18.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bd608b640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb406f400 0x7f4bb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:18.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.840+0000 7f4bd608b640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb406f400 0x7f4bb4071910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0098b0 tx=0x7f4bcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:18.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:18.844+0000 7f4bc6ffd640 1 -- 172.21.15.80:0/4190693510 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4bc8014030 con 0x7f4bd0161b60 2024-04-25T13:44:19.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.004+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4bd006a3a0 con 0x7f4bd0161b60 2024-04-25T13:44:19.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.004+0000 7f4bc6ffd640 1 -- 172.21.15.80:0/4190693510 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v66) ==== 100+0+11 (secure 0 0 0) 0x7f4bc805d2a0 con 0x7f4bd0161b60 2024-04-25T13:44:19.005 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:44:19.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb406f400 msgr2=0x7f4bb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb406f400 0x7f4bb4071910 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4bcc0098b0 tx=0x7f4bcc008040 comp rx=0 tx=0).stop 2024-04-25T13:44:19.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd0161b60 msgr2=0x7f4bd00af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd0161b60 0x7f4bd00af8b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800cd50 tx=0x7f4bc800a650 comp rx=0 tx=0).stop 2024-04-25T13:44:19.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 shutdown_connections 2024-04-25T13:44:19.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb406f400 0x7f4bb4071910 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bd0161b60 0x7f4bd00af8b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bd00b5330 0x7f4bd014e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 --2- 172.21.15.80:0/4190693510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bd006edb0 0x7f4bd01fd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 >> 172.21.15.80:0/4190693510 conn(0x7f4bd00b4160 msgr2=0x7f4bd0157d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:19.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 shutdown_connections 2024-04-25T13:44:19.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.008+0000 7f4bd82ef640 1 -- 172.21.15.80:0/4190693510 wait complete. 2024-04-25T13:44:19.028 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:44:19.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.176+0000 7f71f8985640 1 Processor -- start 2024-04-25T13:44:19.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.176+0000 7f71f8985640 1 -- start start 2024-04-25T13:44:19.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.176+0000 7f71f8985640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71f40d06e0 0x7f71f406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.176+0000 7f71f8985640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f406e9e0 0x7f71f406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.176+0000 7f71f8985640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f40d0af0 con 0x7f71f4161f90 2024-04-25T13:44:19.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.176+0000 7f71f8985640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f71f40d0d70 con 0x7f71f40d06e0 2024-04-25T13:44:19.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.176+0000 7f71f8985640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71f40d1050 con 0x7f71f406e9e0 2024-04-25T13:44:19.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f1d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f71f40d06e0 0x7f71f406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f259c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f71f4161f90 0x7f71f40cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34558/0 (socket says 172.21.15.80:34558) 2024-04-25T13:44:19.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f259c640 1 -- 172.21.15.80:0/272583621 learned_addr learned my addr 172.21.15.80:0/272583621 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:19.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f1d9b640 1 -- 172.21.15.80:0/272583621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f406e9e0 msgr2=0x7f71f406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:19.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f2d9d640 1 --2- 172.21.15.80:0/272583621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f406e9e0 0x7f71f406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f159a640 1 -- 172.21.15.80:0/272583621 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 683476695 0 0) 0x7f71e4001120 con 0x7f71f4161f90 2024-04-25T13:44:19.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f1d9b640 1 --2- 172.21.15.80:0/272583621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f406e9e0 0x7f71f406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f1d9b640 1 -- 172.21.15.80:0/272583621 >> v1:172.21.15.80:6789/0 conn(0x7f71f4161f90 legacy=0x7f71f40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f1d9b640 1 -- 172.21.15.80:0/272583621 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f406dca0 con 0x7f71f40d06e0 2024-04-25T13:44:19.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f2d9d640 1 --2- 172.21.15.80:0/272583621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f406e9e0 0x7f71f406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:19.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f1d9b640 1 --2- 172.21.15.80:0/272583621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f40d06e0 0x7f71f406e4a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f71e80097c0 tx=0x7f71e801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15e0b1e04e2e782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:19.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f159a640 1 -- 172.21.15.80:0/272583621 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80b0070 con 0x7f71f40d06e0 2024-04-25T13:44:19.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f159a640 1 -- 172.21.15.80:0/272583621 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e80ab600 con 0x7f71f40d06e0 2024-04-25T13:44:19.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f159a640 1 -- 172.21.15.80:0/272583621 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80ab9d0 con 0x7f71f40d06e0 2024-04-25T13:44:19.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f8985640 1 -- 172.21.15.80:0/272583621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f40d06e0 msgr2=0x7f71f406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f8985640 1 --2- 172.21.15.80:0/272583621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f40d06e0 0x7f71f406e4a0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f71e80097c0 tx=0x7f71e801c3e0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f8985640 1 -- 172.21.15.80:0/272583621 shutdown_connections 2024-04-25T13:44:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f8985640 1 --2- 172.21.15.80:0/272583621 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f406e9e0 0x7f71f406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f8985640 1 --2- 172.21.15.80:0/272583621 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f40d06e0 0x7f71f406e4a0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f8985640 1 -- 172.21.15.80:0/272583621 >> 172.21.15.80:0/272583621 conn(0x7f71f406d080 msgr2=0x7f71f40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f8985640 1 -- 172.21.15.80:0/272583621 shutdown_connections 2024-04-25T13:44:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.180+0000 7f71f8985640 1 -- 172.21.15.80:0/272583621 wait complete. 2024-04-25T13:44:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 Processor -- start 2024-04-25T13:44:19.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 -- start start 2024-04-25T13:44:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71f406e9e0 0x7f71f4165310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f40d06e0 0x7f71f4167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f4161f90 0x7f71f41675b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41447a0 con 0x7f71f4161f90 2024-04-25T13:44:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4144620 con 0x7f71f406e9e0 2024-04-25T13:44:19.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f71f416b510 con 0x7f71f40d06e0 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f4161f90 0x7f71f41675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f4161f90 0x7f71f41675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43182/0 (socket says 172.21.15.80:43182) 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 -- 172.21.15.80:0/557450643 learned_addr learned my addr 172.21.15.80:0/557450643 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f1d9b640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f40d06e0 0x7f71f4167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f259c640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71f406e9e0 0x7f71f4165310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 -- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f40d06e0 msgr2=0x7f71f4167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f40d06e0 0x7f71f4167070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 -- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71f406e9e0 msgr2=0x7f71f4165310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71f406e9e0 0x7f71f4165310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 -- 172.21.15.80:0/557450643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f4145500 con 0x7f71f4161f90 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f1d9b640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f40d06e0 0x7f71f4167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f2d9d640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f4161f90 0x7f71f41675b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00cd80 tx=0x7f71ec00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:19.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71e2ffd640 1 -- 172.21.15.80:0/557450643 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00f680 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71e2ffd640 1 -- 172.21.15.80:0/557450643 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71ec00f840 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f4147980 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71e2ffd640 1 -- 172.21.15.80:0/557450643 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00fc30 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.184+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f4145a80 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.188+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f406a3a0 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.188+0000 7f71e2ffd640 1 -- 172.21.15.80:0/557450643 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f71ec0213f0 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.188+0000 7f71e2ffd640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f71d406f400 0x7f71d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.188+0000 7f71e2ffd640 1 -- 172.21.15.80:0/557450643 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5709+0+0 (secure 0 0 0) 0x7f71ec090070 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.188+0000 7f71e2ffd640 1 -- 172.21.15.80:0/557450643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=67}) -- 0x7f71d4076fe0 con 0x7f71f4161f90 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.188+0000 7f71f259c640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f71d406f400 0x7f71d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.188+0000 7f71f259c640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f71d406f400 0x7f71d4071910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f71e4000f70 tx=0x7f71e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:19.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.192+0000 7f71e2ffd640 1 -- 172.21.15.80:0/557450643 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f71ec014030 con 0x7f71f4161f90 2024-04-25T13:44:19.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.364+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f71f406e5d0 con 0x7f71f4161f90 2024-04-25T13:44:19.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.364+0000 7f71e2ffd640 1 -- 172.21.15.80:0/557450643 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v66) ==== 80+0+0 (secure 0 0 0) 0x7f71ec05cf50 con 0x7f71f4161f90 2024-04-25T13:44:19.365 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:44:19.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f71d406f400 msgr2=0x7f71d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f71d406f400 0x7f71d4071910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f71e4000f70 tx=0x7f71e4009040 comp rx=0 tx=0).stop 2024-04-25T13:44:19.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f4161f90 msgr2=0x7f71f41675b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f4161f90 0x7f71f41675b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f71ec00cd80 tx=0x7f71ec00a650 comp rx=0 tx=0).stop 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 shutdown_connections 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f71d406f400 0x7f71d4071910 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f71f4161f90 0x7f71f41675b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f71f40d06e0 0x7f71f4167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 --2- 172.21.15.80:0/557450643 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71f406e9e0 0x7f71f4165310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 >> 172.21.15.80:0/557450643 conn(0x7f71f406d080 msgr2=0x7f71f40d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 shutdown_connections 2024-04-25T13:44:19.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.368+0000 7f71f8985640 1 -- 172.21.15.80:0/557450643 wait complete. 2024-04-25T13:44:19.387 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:44:19.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:19.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:44:19.395 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:44:19.396+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-04-25T13:44:19.403 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24588]: fuse finished with error 0 and tester_r 0 2024-04-25T13:44:19.408 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:44:19.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:19.408 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:44:19.467 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:44:19.467 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:44:19.467 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:19.473 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:44:19.473 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:44:19.473 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:44:19.473 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:19.515 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:44:19.515 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:19.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85edf52640 1 Processor -- start 2024-04-25T13:44:19.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85edf52640 1 -- start start 2024-04-25T13:44:19.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85edf52640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f85e8161f90 0x7f85e80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85edf52640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f85e80d0510 0x7f85e815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85edf52640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e815ea70 0x7f85e815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85edf52640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f85e80d0d70 con 0x7f85e8161f90 2024-04-25T13:44:19.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85edf52640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f85e80d1050 con 0x7f85e80d0510 2024-04-25T13:44:19.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85edf52640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f85e806dca0 con 0x7f85e815ea70 2024-04-25T13:44:19.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f85e8161f90 0x7f85e80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f85e8161f90 0x7f85e80cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43184/0 (socket says 172.21.15.80:43184) 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e77fe640 1 -- 172.21.15.80:0/2049627691 learned_addr learned my addr 172.21.15.80:0/2049627691 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e7fff640 1 --2- 172.21.15.80:0/2049627691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e815ea70 0x7f85e815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e6ffd640 1 --2- 172.21.15.80:0/2049627691 >> v2:172.21.15.87:3300/0 conn(0x7f85e80d0510 0x7f85e815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e77fe640 1 -- 172.21.15.80:0/2049627691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e815ea70 msgr2=0x7f85e815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e77fe640 1 --2- 172.21.15.80:0/2049627691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e815ea70 0x7f85e815ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e77fe640 1 -- 172.21.15.80:0/2049627691 >> v2:172.21.15.87:3300/0 conn(0x7f85e80d0510 msgr2=0x7f85e815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e77fe640 1 --2- 172.21.15.80:0/2049627691 >> v2:172.21.15.87:3300/0 conn(0x7f85e80d0510 0x7f85e815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.672+0000 7f85e77fe640 1 -- 172.21.15.80:0/2049627691 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e806df80 con 0x7f85e8161f90 2024-04-25T13:44:19.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.676+0000 7f85e77fe640 1 --2- 172.21.15.80:0/2049627691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e8161f90 0x7f85e80cffd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f85e8074290 tx=0x7f85e001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b38b71ac418d0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:19.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.676+0000 7f85e67fc640 1 -- 172.21.15.80:0/2049627691 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00a5770 con 0x7f85e8161f90 2024-04-25T13:44:19.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.676+0000 7f85e67fc640 1 -- 172.21.15.80:0/2049627691 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e00a5910 con 0x7f85e8161f90 2024-04-25T13:44:19.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.676+0000 7f85e67fc640 1 -- 172.21.15.80:0/2049627691 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00a5ce0 con 0x7f85e8161f90 2024-04-25T13:44:19.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- 172.21.15.80:0/2049627691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e8161f90 msgr2=0x7f85e80cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 --2- 172.21.15.80:0/2049627691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e8161f90 0x7f85e80cffd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f85e8074290 tx=0x7f85e001c930 comp rx=0 tx=0).stop 2024-04-25T13:44:19.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- 172.21.15.80:0/2049627691 shutdown_connections 2024-04-25T13:44:19.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 --2- 172.21.15.80:0/2049627691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e815ea70 0x7f85e815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 --2- 172.21.15.80:0/2049627691 >> v2:172.21.15.87:3300/0 conn(0x7f85e80d0510 0x7f85e815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 --2- 172.21.15.80:0/2049627691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e8161f90 0x7f85e80cffd0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- 172.21.15.80:0/2049627691 >> 172.21.15.80:0/2049627691 conn(0x7f85e806d080 msgr2=0x7f85e80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:19.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- 172.21.15.80:0/2049627691 shutdown_connections 2024-04-25T13:44:19.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- 172.21.15.80:0/2049627691 wait complete. 2024-04-25T13:44:19.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 Processor -- start 2024-04-25T13:44:19.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- start start 2024-04-25T13:44:19.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80d0510 0x7f85e815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85e815ea70 0x7f85e8142e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e8161f90 0x7f85e8159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f85e81447a0 con 0x7f85e80d0510 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f85e8144620 con 0x7f85e815ea70 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.680+0000 7f85edf52640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f85e80d0920 con 0x7f85e8161f90 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80d0510 0x7f85e815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80d0510 0x7f85e815da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43190/0 (socket says 172.21.15.80:43190) 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 -- 172.21.15.80:0/3373913802 learned_addr learned my addr 172.21.15.80:0/3373913802 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e6ffd640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85e815ea70 0x7f85e8142e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 -- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e8161f90 msgr2=0x7f85e8159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e8161f90 0x7f85e8159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 -- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85e815ea70 msgr2=0x7f85e8142e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85e815ea70 0x7f85e8142e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 -- 172.21.15.80:0/3373913802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85e81407b0 con 0x7f85e80d0510 2024-04-25T13:44:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e77fe640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80d0510 0x7f85e815da60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f85e001d1d0 tx=0x7f85e00196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85cffff640 1 -- 172.21.15.80:0/3373913802 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00bc020 con 0x7f85e80d0510 2024-04-25T13:44:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85cffff640 1 -- 172.21.15.80:0/3373913802 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85e00b8880 con 0x7f85e80d0510 2024-04-25T13:44:19.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85e8140a90 con 0x7f85e80d0510 2024-04-25T13:44:19.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85cffff640 1 -- 172.21.15.80:0/3373913802 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00b8c50 con 0x7f85e80d0510 2024-04-25T13:44:19.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85e8159fd0 con 0x7f85e80d0510 2024-04-25T13:44:19.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85e806a3a0 con 0x7f85e80d0510 2024-04-25T13:44:19.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85cffff640 1 -- 172.21.15.80:0/3373913802 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f85e00bf020 con 0x7f85e80d0510 2024-04-25T13:44:19.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85cffff640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f85d406f400 0x7f85d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:19.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85cffff640 1 -- 172.21.15.80:0/3373913802 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f85e012a4c0 con 0x7f85e80d0510 2024-04-25T13:44:19.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85cffff640 1 -- 172.21.15.80:0/3373913802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=68}) -- 0x7f85d4076f80 con 0x7f85e80d0510 2024-04-25T13:44:19.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.684+0000 7f85e6ffd640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f85d406f400 0x7f85d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:19.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.688+0000 7f85e6ffd640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f85d406f400 0x7f85d4071910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f85e81617f0 tx=0x7f85dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:19.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.688+0000 7f85cffff640 1 -- 172.21.15.80:0/3373913802 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85e00c6030 con 0x7f85e80d0510 2024-04-25T13:44:19.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.884+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f85e806fdb0 con 0x7f85e80d0510 2024-04-25T13:44:19.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.884+0000 7f85cffff640 1 -- 172.21.15.80:0/3373913802 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7f85e00165e0 con 0x7f85e80d0510 2024-04-25T13:44:19.886 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:19.886 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1185745761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1185745761},{"type":"v1","addr":"172.21.15.80:6835","nonce":1185745761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2524166918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2524166918},{"type":"v1","addr":"172.21.15.87:6835","nonce":2524166918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3637133191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3637133191},{"type":"v1","addr":"172.21.15.87:6837","nonce":3637133191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:43:13.179327+0000","modified":"2024-04-25T13:43:16.230820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6834/1437895704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1437895704},{"type":"v1","addr":"172.21.15.80:6834","nonce":1437895704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6195,"qdb_cluster":[6195]},"id":4}]} 2024-04-25T13:44:19.886 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 33 2024-04-25T13:44:19.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f85d406f400 msgr2=0x7f85d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f85d406f400 0x7f85d4071910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f85e81617f0 tx=0x7f85dc008040 comp rx=0 tx=0).stop 2024-04-25T13:44:19.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80d0510 msgr2=0x7f85e815da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:19.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80d0510 0x7f85e815da60 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f85e001d1d0 tx=0x7f85e00196d0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 shutdown_connections 2024-04-25T13:44:19.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f85d406f400 0x7f85d4071910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f85e8161f90 0x7f85e8159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85e815ea70 0x7f85e8142e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 --2- 172.21.15.80:0/3373913802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85e80d0510 0x7f85e815da60 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:19.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 >> 172.21.15.80:0/3373913802 conn(0x7f85e806d080 msgr2=0x7f85e8141320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:19.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 shutdown_connections 2024-04-25T13:44:19.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:19.888+0000 7f85edf52640 1 -- 172.21.15.80:0/3373913802 wait complete. 2024-04-25T13:44:19.908 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:20.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.064+0000 7f7d060bc640 1 Processor -- start 2024-04-25T13:44:20.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.064+0000 7f7d060bc640 1 -- start start 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7d060bc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7d00161f90 0x7f7d000cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7d060bc640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d000d0d70 con 0x7f7d0015ec40 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7d060bc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d000d1050 con 0x7f7d000d0510 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7d060bc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7d0006dca0 con 0x7f7d00161f90 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7cff7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7d00161f90 0x7f7d000cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7cff7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7d00161f90 0x7f7d000cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43206/0 (socket says 172.21.15.80:43206) 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7cff7fe640 1 -- 172.21.15.80:0/2555777694 learned_addr learned my addr 172.21.15.80:0/2555777694 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7cff7fe640 1 -- 172.21.15.80:0/2555777694 >> v1:172.21.15.87:6789/0 conn(0x7f7d000d0510 legacy=0x7f7d0015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7cfeffd640 1 -- 172.21.15.80:0/2555777694 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 123423525 0 0) 0x7f7d000d0d70 con 0x7f7d0015ec40 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7cff7fe640 1 -- 172.21.15.80:0/2555777694 >> v1:172.21.15.80:6789/0 conn(0x7f7d0015ec40 legacy=0x7f7d00168cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.068+0000 7f7cff7fe640 1 -- 172.21.15.80:0/2555777694 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0006df80 con 0x7f7d00161f90 2024-04-25T13:44:20.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7cff7fe640 1 --2- 172.21.15.80:0/2555777694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d00161f90 0x7f7d000cffd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009870 tx=0x7f7cf001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515eae5bf898d4a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7cfeffd640 1 -- 172.21.15.80:0/2555777694 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00b0070 con 0x7f7d00161f90 2024-04-25T13:44:20.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7cfeffd640 1 -- 172.21.15.80:0/2555777694 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cf00ab600 con 0x7f7d00161f90 2024-04-25T13:44:20.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7cfeffd640 1 -- 172.21.15.80:0/2555777694 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00ab9d0 con 0x7f7d00161f90 2024-04-25T13:44:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 -- 172.21.15.80:0/2555777694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d00161f90 msgr2=0x7f7d000cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 --2- 172.21.15.80:0/2555777694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d00161f90 0x7f7d000cffd0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009870 tx=0x7f7cf001c3e0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 -- 172.21.15.80:0/2555777694 shutdown_connections 2024-04-25T13:44:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 --2- 172.21.15.80:0/2555777694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d00161f90 0x7f7d000cffd0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 -- 172.21.15.80:0/2555777694 >> 172.21.15.80:0/2555777694 conn(0x7f7d0006d080 msgr2=0x7f7d000b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 -- 172.21.15.80:0/2555777694 shutdown_connections 2024-04-25T13:44:20.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 -- 172.21.15.80:0/2555777694 wait complete. 2024-04-25T13:44:20.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 Processor -- start 2024-04-25T13:44:20.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.072+0000 7f7d060bc640 1 -- start start 2024-04-25T13:44:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d000d0510 0x7f7d000b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d0015ec40 0x7f7d0015dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d00161f90 0x7f7d0013e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0006f630 con 0x7f7d0015ec40 2024-04-25T13:44:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7d0006f4b0 con 0x7f7d000d0510 2024-04-25T13:44:20.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7d000d0920 con 0x7f7d00161f90 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d0015ec40 0x7f7d0015dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d0015ec40 0x7f7d0015dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43222/0 (socket says 172.21.15.80:43222) 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 -- 172.21.15.80:0/3054877952 learned_addr learned my addr 172.21.15.80:0/3054877952 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cff7fe640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d000d0510 0x7f7d000b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cfffff640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d00161f90 0x7f7d0013e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 -- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d00161f90 msgr2=0x7f7d0013e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d00161f90 0x7f7d0013e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 -- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d000d0510 msgr2=0x7f7d000b0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d000d0510 0x7f7d000b0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 -- 172.21.15.80:0/3054877952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d0015cc50 con 0x7f7d0015ec40 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cfffff640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d00161f90 0x7f7d0013e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cf6ffd640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d0015ec40 0x7f7d0015dcf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800c3c0 tx=0x7f7ce800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cfcff9640 1 -- 172.21.15.80:0/3054877952 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8017070 con 0x7f7d0015ec40 2024-04-25T13:44:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cfcff9640 1 -- 172.21.15.80:0/3054877952 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce80052d0 con 0x7f7d0015ec40 2024-04-25T13:44:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7cfcff9640 1 -- 172.21.15.80:0/3054877952 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8003840 con 0x7f7d0015ec40 2024-04-25T13:44:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d0015cf30 con 0x7f7d0015ec40 2024-04-25T13:44:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d0015d4f0 con 0x7f7d0015ec40 2024-04-25T13:44:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.076+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d0006a3a0 con 0x7f7d0015ec40 2024-04-25T13:44:20.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.080+0000 7f7cfcff9640 1 -- 172.21.15.80:0/3054877952 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7ce801a070 con 0x7f7d0015ec40 2024-04-25T13:44:20.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.080+0000 7f7cfcff9640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ce006f400 0x7f7ce0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.080+0000 7f7cfcff9640 1 -- 172.21.15.80:0/3054877952 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f7ce808e5f0 con 0x7f7d0015ec40 2024-04-25T13:44:20.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.080+0000 7f7cfcff9640 1 -- 172.21.15.80:0/3054877952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=68}) -- 0x7f7ce0076f80 con 0x7f7d0015ec40 2024-04-25T13:44:20.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.080+0000 7f7cff7fe640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ce006f400 0x7f7ce0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.080+0000 7f7cff7fe640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ce006f400 0x7f7ce0071910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00ad000 tx=0x7f7cf00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.084+0000 7f7cfcff9640 1 -- 172.21.15.80:0/3054877952 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ce8005480 con 0x7f7d0015ec40 2024-04-25T13:44:20.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.244+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d000d2390 con 0x7f7d0015ec40 2024-04-25T13:44:20.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7cfcff9640 1 -- 172.21.15.80:0/3054877952 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) ==== 74+0+16644 (secure 0 0 0) 0x7f7ce805b4e0 con 0x7f7d0015ec40 2024-04-25T13:44:20.246 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:20.246 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:44:19.398677+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:43:11.515229+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T13:43:12.143130+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ce006f400 msgr2=0x7f7ce0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ce006f400 0x7f7ce0071910 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00ad000 tx=0x7f7cf00a6000 comp rx=0 tx=0).stop 2024-04-25T13:44:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d0015ec40 msgr2=0x7f7d0015dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d0015ec40 0x7f7d0015dcf0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800c3c0 tx=0x7f7ce800c880 comp rx=0 tx=0).stop 2024-04-25T13:44:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 shutdown_connections 2024-04-25T13:44:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ce006f400 0x7f7ce0071910 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7d00161f90 0x7f7d0013e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7d0015ec40 0x7f7d0015dcf0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 --2- 172.21.15.80:0/3054877952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d000d0510 0x7f7d000b0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 >> 172.21.15.80:0/3054877952 conn(0x7f7d0006d080 msgr2=0x7f7d000b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:20.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 shutdown_connections 2024-04-25T13:44:20.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.248+0000 7f7d060bc640 1 -- 172.21.15.80:0/3054877952 wait complete. 2024-04-25T13:44:20.268 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:43:11.515229+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:43:12.143130+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:44:20.269 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:44:20.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b2ccd7640 1 Processor -- start 2024-04-25T13:44:20.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b2ccd7640 1 -- start start 2024-04-25T13:44:20.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b2ccd7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b2806fb90 0x7f2b28160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b2ccd7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2b28161160 0x7f2b280d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b2ccd7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b28161570 con 0x7f2b280d2170 2024-04-25T13:44:20.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b2ccd7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2b28070ea0 con 0x7f2b28161160 2024-04-25T13:44:20.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b2ccd7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2b28070290 con 0x7f2b2806fb90 2024-04-25T13:44:20.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b1ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2b28161160 0x7f2b280d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b1ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2b28161160 0x7f2b280d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43226/0 (socket says 172.21.15.80:43226) 2024-04-25T13:44:20.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b1ffff640 1 -- 172.21.15.80:0/2259517803 learned_addr learned my addr 172.21.15.80:0/2259517803 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:20.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b2659c640 1 --2- 172.21.15.80:0/2259517803 >> v2:172.21.15.87:3300/0 conn(0x7f2b2806fb90 0x7f2b28160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b1ffff640 1 -- 172.21.15.80:0/2259517803 >> v2:172.21.15.87:3300/0 conn(0x7f2b2806fb90 msgr2=0x7f2b28160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:20.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b1ffff640 1 --2- 172.21.15.80:0/2259517803 >> v2:172.21.15.87:3300/0 conn(0x7f2b2806fb90 0x7f2b28160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b1ffff640 1 -- 172.21.15.80:0/2259517803 >> v1:172.21.15.87:6789/0 conn(0x7f2b280d2170 legacy=0x7f2b280b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:20.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.424+0000 7f2b1ffff640 1 -- 172.21.15.80:0/2259517803 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b28070570 con 0x7f2b28161160 2024-04-25T13:44:20.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b1ffff640 1 --2- 172.21.15.80:0/2259517803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b28161160 0x7f2b280d1c30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2b280b7960 tx=0x7f2b1001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd4e43cb65541e68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b25d9b640 1 -- 172.21.15.80:0/2259517803 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100b0070 con 0x7f2b28161160 2024-04-25T13:44:20.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b25d9b640 1 -- 172.21.15.80:0/2259517803 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b100ab600 con 0x7f2b28161160 2024-04-25T13:44:20.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b25d9b640 1 -- 172.21.15.80:0/2259517803 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100ab9d0 con 0x7f2b28161160 2024-04-25T13:44:20.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/2259517803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b28161160 msgr2=0x7f2b280d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/2259517803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b28161160 0x7f2b280d1c30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f2b280b7960 tx=0x7f2b1001c240 comp rx=0 tx=0).stop 2024-04-25T13:44:20.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/2259517803 shutdown_connections 2024-04-25T13:44:20.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/2259517803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b28161160 0x7f2b280d1c30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/2259517803 >> v2:172.21.15.87:3300/0 conn(0x7f2b2806fb90 0x7f2b28160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/2259517803 >> 172.21.15.80:0/2259517803 conn(0x7f2b28073790 msgr2=0x7f2b28073ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:20.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/2259517803 shutdown_connections 2024-04-25T13:44:20.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.428+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/2259517803 wait complete. 2024-04-25T13:44:20.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 Processor -- start 2024-04-25T13:44:20.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 -- start start 2024-04-25T13:44:20.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b2806fb90 0x7f2b28149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b280d2170 0x7f2b28148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b28161160 0x7f2b281465d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b2816a140 con 0x7f2b280d2170 2024-04-25T13:44:20.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b28169fc0 con 0x7f2b28161160 2024-04-25T13:44:20.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b2816a2c0 con 0x7f2b2806fb90 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b280d2170 0x7f2b28148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b280d2170 0x7f2b28148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43234/0 (socket says 172.21.15.80:43234) 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 -- 172.21.15.80:0/4214982345 learned_addr learned my addr 172.21.15.80:0/4214982345 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2659c640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b2806fb90 0x7f2b28149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b26d9d640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b28161160 0x7f2b281465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 -- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b2806fb90 msgr2=0x7f2b28149b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b2806fb90 0x7f2b28149b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 -- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b28161160 msgr2=0x7f2b281465d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b28161160 0x7f2b281465d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 -- 172.21.15.80:0/4214982345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b28146b10 con 0x7f2b280d2170 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1ffff640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b280d2170 0x7f2b28148ae0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2b100ae040 tx=0x7f2b10018fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1effd640 1 -- 172.21.15.80:0/4214982345 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100ba070 con 0x7f2b280d2170 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1effd640 1 -- 172.21.15.80:0/4214982345 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b100b4640 con 0x7f2b280d2170 2024-04-25T13:44:20.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b1effd640 1 -- 172.21.15.80:0/4214982345 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b100b0040 con 0x7f2b280d2170 2024-04-25T13:44:20.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b2814a3e0 con 0x7f2b280d2170 2024-04-25T13:44:20.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.432+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b281482e0 con 0x7f2b280d2170 2024-04-25T13:44:20.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.436+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b2806a3a0 con 0x7f2b280d2170 2024-04-25T13:44:20.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.436+0000 7f2b1effd640 1 -- 172.21.15.80:0/4214982345 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2b100bd070 con 0x7f2b280d2170 2024-04-25T13:44:20.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.436+0000 7f2b1effd640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.436+0000 7f2b1effd640 1 -- 172.21.15.80:0/4214982345 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f2b10128350 con 0x7f2b280d2170 2024-04-25T13:44:20.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.436+0000 7f2b1effd640 1 -- 172.21.15.80:0/4214982345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=68}) -- 0x7f2b08076f80 con 0x7f2b280d2170 2024-04-25T13:44:20.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.436+0000 7f2b2659c640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.436+0000 7f2b2659c640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2b180079c0 tx=0x7f2b18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.440+0000 7f2b1effd640 1 -- 172.21.15.80:0/4214982345 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b28161570 con 0x7f2b280d2170 2024-04-25T13:44:20.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2b2806f420 con 0x7f2b280d2170 2024-04-25T13:44:20.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b1effd640 1 -- 172.21.15.80:0/4214982345 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) ==== 93+0+10 (secure 0 0 0) 0x7f2b100f4bf0 con 0x7f2b280d2170 2024-04-25T13:44:20.603 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:44:20.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 msgr2=0x7f2b08071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2b180079c0 tx=0x7f2b18008040 comp rx=0 tx=0).stop 2024-04-25T13:44:20.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b280d2170 msgr2=0x7f2b28148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b280d2170 0x7f2b28148ae0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f2b100ae040 tx=0x7f2b10018fe0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 shutdown_connections 2024-04-25T13:44:20.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b28161160 0x7f2b281465d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b280d2170 0x7f2b28148ae0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 --2- 172.21.15.80:0/4214982345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b2806fb90 0x7f2b28149b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 >> 172.21.15.80:0/4214982345 conn(0x7f2b28073790 msgr2=0x7f2b2815f810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 shutdown_connections 2024-04-25T13:44:20.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.604+0000 7f2b2ccd7640 1 -- 172.21.15.80:0/4214982345 wait complete. 2024-04-25T13:44:20.624 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:44:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.760+0000 7f1ac1622640 1 Processor -- start 2024-04-25T13:44:20.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.760+0000 7f1ac1622640 1 -- start start 2024-04-25T13:44:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 0x7f1abc0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1abc0d2ad0 0x7f1abc0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1abc0b7c10 0x7f1abc0b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1abc0b8560 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1abc0b92a0 con 0x7f1abc0b7c10 2024-04-25T13:44:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc0d4c10 con 0x7f1abc0d3ed0 2024-04-25T13:44:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1abc0d2ad0 0x7f1abc0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1abc0d2ad0 0x7f1abc0b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43240/0 (socket says 172.21.15.80:43240) 2024-04-25T13:44:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 -- 172.21.15.80:0/4196643899 learned_addr learned my addr 172.21.15.80:0/4196643899 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:20.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1abb7fe640 1 --2- 172.21.15.80:0/4196643899 >> v2:172.21.15.87:3300/0 conn(0x7f1abc0b7c10 0x7f1abc0b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1abaffd640 1 --2- 172.21.15.80:0/4196643899 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 0x7f1abc0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 -- 172.21.15.80:0/4196643899 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 msgr2=0x7f1abc0d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:20.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 --2- 172.21.15.80:0/4196643899 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 0x7f1abc0d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 -- 172.21.15.80:0/4196643899 >> v2:172.21.15.87:3300/0 conn(0x7f1abc0b7c10 msgr2=0x7f1abc0b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:20.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 --2- 172.21.15.80:0/4196643899 >> v2:172.21.15.87:3300/0 conn(0x7f1abc0b7c10 0x7f1abc0b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 -- 172.21.15.80:0/4196643899 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc0d4ef0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1abb7fe640 1 --2- 172.21.15.80:0/4196643899 >> v2:172.21.15.87:3300/0 conn(0x7f1abc0b7c10 0x7f1abc0b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:20.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1aba7fc640 1 --2- 172.21.15.80:0/4196643899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc0b76d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0b8dc0 tx=0x7f1aac01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50e4c85074a47258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ab9ffb640 1 -- 172.21.15.80:0/4196643899 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0ab620 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ab9ffb640 1 -- 172.21.15.80:0/4196643899 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac0b0070 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ab9ffb640 1 -- 172.21.15.80:0/4196643899 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0ab9e0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 -- 172.21.15.80:0/4196643899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 msgr2=0x7f1abc0b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 --2- 172.21.15.80:0/4196643899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc0b76d0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f1abc0b8dc0 tx=0x7f1aac01c460 comp rx=0 tx=0).stop 2024-04-25T13:44:20.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 -- 172.21.15.80:0/4196643899 shutdown_connections 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 --2- 172.21.15.80:0/4196643899 >> v2:172.21.15.87:3300/0 conn(0x7f1abc0b7c10 0x7f1abc0b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 --2- 172.21.15.80:0/4196643899 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc0b76d0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 --2- 172.21.15.80:0/4196643899 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 0x7f1abc0d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 -- 172.21.15.80:0/4196643899 >> 172.21.15.80:0/4196643899 conn(0x7f1abc0737b0 msgr2=0x7f1abc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 -- 172.21.15.80:0/4196643899 shutdown_connections 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.764+0000 7f1ac1622640 1 -- 172.21.15.80:0/4196643899 wait complete. 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1ac1622640 1 Processor -- start 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1ac1622640 1 -- start start 2024-04-25T13:44:20.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1ac1622640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc0b7c10 0x7f1abc1500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1ac1622640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1ac1622640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 0x7f1abc1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1ac1622640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc0d2f90 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1ac1622640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc160620 con 0x7f1abc0b7c10 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1ac1622640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc1607a0 con 0x7f1abc0d3ed0 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43244/0 (socket says 172.21.15.80:43244) 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 -- 172.21.15.80:0/900427423 learned_addr learned my addr 172.21.15.80:0/900427423 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:20.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1abb7fe640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 0x7f1abc1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 -- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 msgr2=0x7f1abc1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 0x7f1abc1530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 -- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc0b7c10 msgr2=0x7f1abc1500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1abaffd640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc0b7c10 0x7f1abc1500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc0b7c10 0x7f1abc1500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 -- 172.21.15.80:0/900427423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc1505f0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aba7fc640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc157c70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1aac01ab10 tx=0x7f1aac01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aa37fe640 1 -- 172.21.15.80:0/900427423 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0abd80 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aa37fe640 1 -- 172.21.15.80:0/900427423 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1aac0be3f0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.768+0000 7f1aa37fe640 1 -- 172.21.15.80:0/900427423 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0be7e0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc0cdb60 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc0ce120 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc06a3a0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1aa37fe640 1 -- 172.21.15.80:0/900427423 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1aac0b43f0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1aa37fe640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1aa806f430 0x7f1aa8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:20.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1aa37fe640 1 -- 172.21.15.80:0/900427423 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f1aac128de0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1aa37fe640 1 -- 172.21.15.80:0/900427423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=68}) -- 0x7f1aa8076fb0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1abaffd640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1aa806f430 0x7f1aa8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:20.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.772+0000 7f1abaffd640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1aa806f430 0x7f1aa8071940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000b7c0 tx=0x7f1ab0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:20.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.776+0000 7f1aa37fe640 1 -- 172.21.15.80:0/900427423 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1aac016110 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.936+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1abc0761f0 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.936+0000 7f1aa37fe640 1 -- 172.21.15.80:0/900427423 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v67) ==== 104+0+11 (secure 0 0 0) 0x7f1aac0f5680 con 0x7f1abc0d2ad0 2024-04-25T13:44:20.937 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:44:20.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1aa806f430 msgr2=0x7f1aa8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1aa806f430 0x7f1aa8071940 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1ab000b7c0 tx=0x7f1ab0008040 comp rx=0 tx=0).stop 2024-04-25T13:44:20.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 msgr2=0x7f1abc157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:20.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc157c70 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f1aac01ab10 tx=0x7f1aac01aca0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 shutdown_connections 2024-04-25T13:44:20.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1aa806f430 0x7f1aa8071940 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1abc0d3ed0 0x7f1abc1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1abc0d2ad0 0x7f1abc157c70 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 --2- 172.21.15.80:0/900427423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1abc0b7c10 0x7f1abc1500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:20.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 >> 172.21.15.80:0/900427423 conn(0x7f1abc0737b0 msgr2=0x7f1abc157340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:20.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 shutdown_connections 2024-04-25T13:44:20.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:20.940+0000 7f1ac1622640 1 -- 172.21.15.80:0/900427423 wait complete. 2024-04-25T13:44:20.959 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:44:21.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809f4e7640 1 Processor -- start 2024-04-25T13:44:21.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809f4e7640 1 -- start start 2024-04-25T13:44:21.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809f4e7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8098071e60 0x7f80981574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809f4e7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098161170 0x7f80980b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809f4e7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f809806fbf0 con 0x7f8098073c40 2024-04-25T13:44:21.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809f4e7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f80980b1350 con 0x7f8098071e60 2024-04-25T13:44:21.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809f4e7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f809806f9a0 con 0x7f8098161170 2024-04-25T13:44:21.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809da84640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098161170 0x7f80980b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809d283640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8098071e60 0x7f80981574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809ca82640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f8098073c40 0x7f8098160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:36440/0 (socket says 172.21.15.80:36440) 2024-04-25T13:44:21.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809ca82640 1 -- 172.21.15.80:0/1686645848 learned_addr learned my addr 172.21.15.80:0/1686645848 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:21.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809d283640 1 --2- 172.21.15.80:0/1686645848 >> v2:172.21.15.87:3300/0 conn(0x7f8098071e60 0x7f80981574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:52040/0 (socket says 172.21.15.80:52040) 2024-04-25T13:44:21.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809d283640 1 -- 172.21.15.80:0/1686645848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098161170 msgr2=0x7f80980b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:21.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809d283640 1 --2- 172.21.15.80:0/1686645848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098161170 0x7f80980b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809d283640 1 -- 172.21.15.80:0/1686645848 >> v1:172.21.15.87:6789/0 conn(0x7f8098073c40 legacy=0x7f8098160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809d283640 1 -- 172.21.15.80:0/1686645848 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80980b0a20 con 0x7f8098071e60 2024-04-25T13:44:21.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.116+0000 7f809da84640 1 --2- 172.21.15.80:0/1686645848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098161170 0x7f80980b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809d283640 1 --2- 172.21.15.80:0/1686645848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 0x7f80981574c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f80940097c0 tx=0x7f809401c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a61cfc5dd987831b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f808ffff640 1 -- 172.21.15.80:0/1686645848 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940b2070 con 0x7f8098071e60 2024-04-25T13:44:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f808ffff640 1 -- 172.21.15.80:0/1686645848 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80940ad640 con 0x7f8098071e60 2024-04-25T13:44:21.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f808ffff640 1 -- 172.21.15.80:0/1686645848 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80940ada10 con 0x7f8098071e60 2024-04-25T13:44:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809f4e7640 1 -- 172.21.15.80:0/1686645848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 msgr2=0x7f80981574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809f4e7640 1 --2- 172.21.15.80:0/1686645848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 0x7f80981574c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f80940097c0 tx=0x7f809401c4a0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809f4e7640 1 -- 172.21.15.80:0/1686645848 shutdown_connections 2024-04-25T13:44:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809f4e7640 1 --2- 172.21.15.80:0/1686645848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098161170 0x7f80980b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809f4e7640 1 --2- 172.21.15.80:0/1686645848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 0x7f80981574c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809f4e7640 1 -- 172.21.15.80:0/1686645848 >> 172.21.15.80:0/1686645848 conn(0x7f80980b4160 msgr2=0x7f80980d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809f4e7640 1 -- 172.21.15.80:0/1686645848 shutdown_connections 2024-04-25T13:44:21.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.120+0000 7f809f4e7640 1 -- 172.21.15.80:0/1686645848 wait complete. 2024-04-25T13:44:21.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 Processor -- start 2024-04-25T13:44:21.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 -- start start 2024-04-25T13:44:21.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 0x7f8098166150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8098073c40 0x7f80981f8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098166690 0x7f80981f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f809806e470 con 0x7f8098073c40 2024-04-25T13:44:21.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8098074050 con 0x7f8098071e60 2024-04-25T13:44:21.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f809806e5f0 con 0x7f8098166690 2024-04-25T13:44:21.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8098073c40 0x7f80981f8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8098073c40 0x7f80981f8c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60476/0 (socket says 172.21.15.80:60476) 2024-04-25T13:44:21.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 -- 172.21.15.80:0/300946368 learned_addr learned my addr 172.21.15.80:0/300946368 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:21.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809da84640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098166690 0x7f80981f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809d283640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 0x7f8098166150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 -- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098166690 msgr2=0x7f80981f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098166690 0x7f80981f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 -- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 msgr2=0x7f8098166150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 0x7f8098166150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 -- 172.21.15.80:0/300946368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80981fd380 con 0x7f8098073c40 2024-04-25T13:44:21.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809d283640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 0x7f8098166150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:21.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809ca82640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8098073c40 0x7f80981f8c20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f808800c370 tx=0x7f808800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f808dffb640 1 -- 172.21.15.80:0/300946368 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088017070 con 0x7f8098073c40 2024-04-25T13:44:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f808dffb640 1 -- 172.21.15.80:0/300946368 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8088005cf0 con 0x7f8098073c40 2024-04-25T13:44:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80981fd660 con 0x7f8098073c40 2024-04-25T13:44:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f808dffb640 1 -- 172.21.15.80:0/300946368 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80880072f0 con 0x7f8098073c40 2024-04-25T13:44:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.124+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80981fdc20 con 0x7f8098073c40 2024-04-25T13:44:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.128+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f809806a3a0 con 0x7f8098073c40 2024-04-25T13:44:21.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.128+0000 7f808dffb640 1 -- 172.21.15.80:0/300946368 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f808801a070 con 0x7f8098073c40 2024-04-25T13:44:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.128+0000 7f808dffb640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f807c06f400 0x7f807c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.128+0000 7f808dffb640 1 -- 172.21.15.80:0/300946368 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5709+0+0 (secure 0 0 0) 0x7f808808e6c0 con 0x7f8098073c40 2024-04-25T13:44:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.128+0000 7f808dffb640 1 -- 172.21.15.80:0/300946368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=68}) -- 0x7f807c076f80 con 0x7f8098073c40 2024-04-25T13:44:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.128+0000 7f809d283640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f807c06f400 0x7f807c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.128+0000 7f809d283640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f807c06f400 0x7f807c071910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f80940af000 tx=0x7f80940a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.132+0000 7f808dffb640 1 -- 172.21.15.80:0/300946368 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f808805b520 con 0x7f8098073c40 2024-04-25T13:44:21.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.280+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8098167560 con 0x7f8098073c40 2024-04-25T13:44:21.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.280+0000 7f808dffb640 1 -- 172.21.15.80:0/300946368 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) ==== 100+0+11 (secure 0 0 0) 0x7f808805ae20 con 0x7f8098073c40 2024-04-25T13:44:21.280 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:44:21.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f807c06f400 msgr2=0x7f807c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f807c06f400 0x7f807c071910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f80940af000 tx=0x7f80940a7070 comp rx=0 tx=0).stop 2024-04-25T13:44:21.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8098073c40 msgr2=0x7f80981f8c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8098073c40 0x7f80981f8c20 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f808800c370 tx=0x7f808800c830 comp rx=0 tx=0).stop 2024-04-25T13:44:21.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 shutdown_connections 2024-04-25T13:44:21.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f807c06f400 0x7f807c071910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8098166690 0x7f80981f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8098073c40 0x7f80981f8c20 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 --2- 172.21.15.80:0/300946368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8098071e60 0x7f8098166150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 >> 172.21.15.80:0/300946368 conn(0x7f80980b4160 msgr2=0x7f80980725d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:21.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 shutdown_connections 2024-04-25T13:44:21.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.284+0000 7f809f4e7640 1 -- 172.21.15.80:0/300946368 wait complete. 2024-04-25T13:44:21.300 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:21.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.432+0000 7f178ec86640 1 Processor -- start 2024-04-25T13:44:21.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.432+0000 7f178ec86640 1 -- start start 2024-04-25T13:44:21.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f178ec86640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1788068ca0 0x7f178806eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f178ec86640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17880d2490 con 0x7f178806f400 2024-04-25T13:44:21.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f178ec86640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17880d13f0 con 0x7f178806d000 2024-04-25T13:44:21.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f178ec86640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f17880d2000 con 0x7f1788068ca0 2024-04-25T13:44:21.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f177ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1788068ca0 0x7f178806eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f177ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1788068ca0 0x7f178806eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60480/0 (socket says 172.21.15.80:60480) 2024-04-25T13:44:21.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f177ffff640 1 -- 172.21.15.80:0/3549079328 learned_addr learned my addr 172.21.15.80:0/3549079328 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:21.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f177ffff640 1 -- 172.21.15.80:0/3549079328 >> v1:172.21.15.87:6789/0 conn(0x7f178806d000 legacy=0x7f1788068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:21.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f177ffff640 1 -- 172.21.15.80:0/3549079328 >> v1:172.21.15.80:6789/0 conn(0x7f178806f400 legacy=0x7f178806f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:21.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f177ffff640 1 -- 172.21.15.80:0/3549079328 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17880d16d0 con 0x7f1788068ca0 2024-04-25T13:44:21.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.436+0000 7f177ffff640 1 --2- 172.21.15.80:0/3549079328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1788068ca0 0x7f178806eec0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1778009870 tx=0x7f177801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65ace9eadfee5cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f177f7fe640 1 -- 172.21.15.80:0/3549079328 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17780aa650 con 0x7f1788068ca0 2024-04-25T13:44:21.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f177f7fe640 1 -- 172.21.15.80:0/3549079328 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17780af070 con 0x7f1788068ca0 2024-04-25T13:44:21.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f177f7fe640 1 -- 172.21.15.80:0/3549079328 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17780aaa10 con 0x7f1788068ca0 2024-04-25T13:44:21.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f178ec86640 1 -- 172.21.15.80:0/3549079328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1788068ca0 msgr2=0x7f178806eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f178ec86640 1 --2- 172.21.15.80:0/3549079328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1788068ca0 0x7f178806eec0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1778009870 tx=0x7f177801c4f0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f178ec86640 1 -- 172.21.15.80:0/3549079328 shutdown_connections 2024-04-25T13:44:21.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f178ec86640 1 --2- 172.21.15.80:0/3549079328 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1788068ca0 0x7f178806eec0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f178ec86640 1 -- 172.21.15.80:0/3549079328 >> 172.21.15.80:0/3549079328 conn(0x7f17880d28f0 msgr2=0x7f17880d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:21.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f178ec86640 1 -- 172.21.15.80:0/3549079328 shutdown_connections 2024-04-25T13:44:21.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.440+0000 7f178ec86640 1 -- 172.21.15.80:0/3549079328 wait complete. 2024-04-25T13:44:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 Processor -- start 2024-04-25T13:44:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 -- start start 2024-04-25T13:44:21.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178806d000 0x7f178814b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178806f400 0x7f178814d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17880b5960 0x7f17880b5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f178806ff20 con 0x7f178806d000 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f178806d410 con 0x7f178806f400 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17880b1b10 con 0x7f17880b5960 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178806d000 0x7f178814b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178806d000 0x7f178814b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60482/0 (socket says 172.21.15.80:60482) 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 -- 172.21.15.80:0/3330097527 learned_addr learned my addr 172.21.15.80:0/3330097527 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f177ffff640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178806f400 0x7f178814d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178d223640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17880b5960 0x7f17880b5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 -- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17880b5960 msgr2=0x7f17880b5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17880b5960 0x7f17880b5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 -- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178806f400 msgr2=0x7f178814d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178806f400 0x7f178814d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 -- 172.21.15.80:0/3330097527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f178814c4d0 con 0x7f178806d000 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f177ffff640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178806f400 0x7f178814d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178d223640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17880b5960 0x7f17880b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ca22640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178806d000 0x7f178814b9e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f178400b410 tx=0x7f178400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f177d7fa640 1 -- 172.21.15.80:0/3330097527 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784015030 con 0x7f178806d000 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f177d7fa640 1 -- 172.21.15.80:0/3330097527 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1784004020 con 0x7f178806d000 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17880b62b0 con 0x7f178806d000 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f177d7fa640 1 -- 172.21.15.80:0/3330097527 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1784004410 con 0x7f178806d000 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f178815fb10 con 0x7f178806d000 2024-04-25T13:44:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.444+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f178806a3a0 con 0x7f178806d000 2024-04-25T13:44:21.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.448+0000 7f177d7fa640 1 -- 172.21.15.80:0/3330097527 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1784018070 con 0x7f178806d000 2024-04-25T13:44:21.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.448+0000 7f177d7fa640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177406f400 0x7f1774071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.448+0000 7f177d7fa640 1 -- 172.21.15.80:0/3330097527 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f178408ed30 con 0x7f178806d000 2024-04-25T13:44:21.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.448+0000 7f177d7fa640 1 -- 172.21.15.80:0/3330097527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=69}) -- 0x7f1774076fe0 con 0x7f178806d000 2024-04-25T13:44:21.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.448+0000 7f177ffff640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177406f400 0x7f1774071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.448+0000 7f177ffff640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177406f400 0x7f1774071910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f17780ae000 tx=0x7f1778008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.452+0000 7f177d7fa640 1 -- 172.21.15.80:0/3330097527 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f17880b62b0 con 0x7f178806d000 2024-04-25T13:44:21.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.648+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f178814bf20 con 0x7f178806d000 2024-04-25T13:44:21.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.648+0000 7f177d7fa640 1 -- 172.21.15.80:0/3330097527 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7f178814c4d0 con 0x7f178806d000 2024-04-25T13:44:21.649 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:21.649 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1185745761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1185745761},{"type":"v1","addr":"172.21.15.80:6835","nonce":1185745761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2524166918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2524166918},{"type":"v1","addr":"172.21.15.87:6835","nonce":2524166918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3637133191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3637133191},{"type":"v1","addr":"172.21.15.87:6837","nonce":3637133191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:43:13.179327+0000","modified":"2024-04-25T13:43:16.230820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6834/1437895704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1437895704},{"type":"v1","addr":"172.21.15.80:6834","nonce":1437895704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6195,"qdb_cluster":[6195]},"id":4}]} 2024-04-25T13:44:21.650 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 33 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177406f400 msgr2=0x7f1774071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177406f400 0x7f1774071910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f17780ae000 tx=0x7f1778008040 comp rx=0 tx=0).stop 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178806d000 msgr2=0x7f178814b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178806d000 0x7f178814b9e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f178400b410 tx=0x7f178400b8d0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 shutdown_connections 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f177406f400 0x7f1774071910 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f17880b5960 0x7f17880b5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f178806f400 0x7f178814d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 --2- 172.21.15.80:0/3330097527 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f178806d000 0x7f178814b9e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 >> 172.21.15.80:0/3330097527 conn(0x7f17880d28f0 msgr2=0x7f17880b2760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:21.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 shutdown_connections 2024-04-25T13:44:21.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.652+0000 7f178ec86640 1 -- 172.21.15.80:0/3330097527 wait complete. 2024-04-25T13:44:21.671 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:21.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.812+0000 7fec6fc81640 1 Processor -- start 2024-04-25T13:44:21.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.812+0000 7fec6fc81640 1 -- start start 2024-04-25T13:44:21.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6fc81640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec68160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6fc81640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec68161070 0x7fec68161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6fc81640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec6806fcf0 con 0x7fec680b4bd0 2024-04-25T13:44:21.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6fc81640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fec681619c0 con 0x7fec68161070 2024-04-25T13:44:21.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6fc81640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec68073f10 con 0x7fec6806f5a0 2024-04-25T13:44:21.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6da1d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec68160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6d21c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec68161070 0x7fec68161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6da1d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec68160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:55290/0 (socket says 172.21.15.80:55290) 2024-04-25T13:44:21.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6da1d640 1 -- 172.21.15.80:0/1587872191 learned_addr learned my addr 172.21.15.80:0/1587872191 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:21.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6d21c640 1 -- 172.21.15.80:0/1587872191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 msgr2=0x7fec68160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:21.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6d21c640 1 --2- 172.21.15.80:0/1587872191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec68160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6d21c640 1 -- 172.21.15.80:0/1587872191 >> v1:172.21.15.87:6789/0 conn(0x7fec680b4bd0 legacy=0x7fec680b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6d21c640 1 -- 172.21.15.80:0/1587872191 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec680b0c60 con 0x7fec68161070 2024-04-25T13:44:21.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.816+0000 7fec6da1d640 1 --2- 172.21.15.80:0/1587872191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec68160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:21.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6d21c640 1 --2- 172.21.15.80:0/1587872191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec68161070 0x7fec68161480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec580097f0 tx=0x7fec5801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27db8fdf60175fa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6ca1b640 1 -- 172.21.15.80:0/1587872191 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec580b0070 con 0x7fec68161070 2024-04-25T13:44:21.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6ca1b640 1 -- 172.21.15.80:0/1587872191 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec580ab600 con 0x7fec68161070 2024-04-25T13:44:21.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6ca1b640 1 -- 172.21.15.80:0/1587872191 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec580ab9d0 con 0x7fec68161070 2024-04-25T13:44:21.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 -- 172.21.15.80:0/1587872191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec68161070 msgr2=0x7fec68161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 --2- 172.21.15.80:0/1587872191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec68161070 0x7fec68161480 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fec580097f0 tx=0x7fec5801c410 comp rx=0 tx=0).stop 2024-04-25T13:44:21.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 -- 172.21.15.80:0/1587872191 shutdown_connections 2024-04-25T13:44:21.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 --2- 172.21.15.80:0/1587872191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec68161070 0x7fec68161480 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 --2- 172.21.15.80:0/1587872191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec68160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 -- 172.21.15.80:0/1587872191 >> 172.21.15.80:0/1587872191 conn(0x7fec6806d080 msgr2=0x7fec680b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:21.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 -- 172.21.15.80:0/1587872191 shutdown_connections 2024-04-25T13:44:21.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 -- 172.21.15.80:0/1587872191 wait complete. 2024-04-25T13:44:21.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 Processor -- start 2024-04-25T13:44:21.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.820+0000 7fec6fc81640 1 -- start start 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec6814d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec680b4bd0 0x7fec6815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec68161070 0x7fec6815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fec6806e610 con 0x7fec68161070 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec6806e490 con 0x7fec680b4bd0 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec6806e790 con 0x7fec6806f5a0 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec68161070 0x7fec6815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec68161070 0x7fec6815daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60488/0 (socket says 172.21.15.80:60488) 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 -- 172.21.15.80:0/2043237089 learned_addr learned my addr 172.21.15.80:0/2043237089 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6da1d640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec6814d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6d21c640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec680b4bd0 0x7fec6815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 -- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 msgr2=0x7fec6814d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec6814d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 -- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec680b4bd0 msgr2=0x7fec6815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec680b4bd0 0x7fec6815d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 -- 172.21.15.80:0/2043237089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec68160130 con 0x7fec68161070 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6d21c640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec680b4bd0 0x7fec6815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6e21e640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec68161070 0x7fec6815daf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c760 tx=0x7fec6000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec5e7fc640 1 -- 172.21.15.80:0/2043237089 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec60003b80 con 0x7fec68161070 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec5e7fc640 1 -- 172.21.15.80:0/2043237089 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec60003d20 con 0x7fec68161070 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec68160410 con 0x7fec68161070 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec5e7fc640 1 -- 172.21.15.80:0/2043237089 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec6000aa90 con 0x7fec68161070 2024-04-25T13:44:21.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec6815e850 con 0x7fec68161070 2024-04-25T13:44:21.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6814ed10 con 0x7fec68161070 2024-04-25T13:44:21.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.824+0000 7fec5e7fc640 1 -- 172.21.15.80:0/2043237089 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fec6000ac30 con 0x7fec68161070 2024-04-25T13:44:21.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.828+0000 7fec5e7fc640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4c06f430 0x7fec4c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:21.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.828+0000 7fec5e7fc640 1 -- 172.21.15.80:0/2043237089 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fec6008f040 con 0x7fec68161070 2024-04-25T13:44:21.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.828+0000 7fec5e7fc640 1 -- 172.21.15.80:0/2043237089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=69}) -- 0x7fec4c076fb0 con 0x7fec68161070 2024-04-25T13:44:21.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.828+0000 7fec6da1d640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4c06f430 0x7fec4c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:21.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.828+0000 7fec6da1d640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4c06f430 0x7fec4c071940 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fec680b6e70 tx=0x7fec64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:21.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.832+0000 7fec5e7fc640 1 -- 172.21.15.80:0/2043237089 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fec6005c050 con 0x7fec68161070 2024-04-25T13:44:21.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.992+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec6806a3a0 con 0x7fec68161070 2024-04-25T13:44:21.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.992+0000 7fec5e7fc640 1 -- 172.21.15.80:0/2043237089 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16622 (secure 0 0 0) 0x7fec60064040 con 0x7fec68161070 2024-04-25T13:44:21.993 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:21.994 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":68,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:44:21.426079+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:43:11.515229+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T13:43:12.143130+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","full","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:21.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4c06f430 msgr2=0x7fec4c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4c06f430 0x7fec4c071940 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fec680b6e70 tx=0x7fec64008040 comp rx=0 tx=0).stop 2024-04-25T13:44:21.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec68161070 msgr2=0x7fec6815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:21.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec68161070 0x7fec6815daf0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fec6000c760 tx=0x7fec6000cc20 comp rx=0 tx=0).stop 2024-04-25T13:44:21.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 shutdown_connections 2024-04-25T13:44:21.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec4c06f430 0x7fec4c071940 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec68161070 0x7fec6815daf0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec680b4bd0 0x7fec6815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 --2- 172.21.15.80:0/2043237089 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec6806f5a0 0x7fec6814d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:21.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 >> 172.21.15.80:0/2043237089 conn(0x7fec6806d080 msgr2=0x7fec680d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:21.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 shutdown_connections 2024-04-25T13:44:21.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:21.996+0000 7fec6fc81640 1 -- 172.21.15.80:0/2043237089 wait complete. 2024-04-25T13:44:22.015 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:44:22.016 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:22.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.156+0000 7fbe46586640 1 Processor -- start 2024-04-25T13:44:22.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.156+0000 7fbe46586640 1 -- start start 2024-04-25T13:44:22.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe46586640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe40161f90 0x7fbe400cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe46586640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe400d0510 0x7fbe4015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe46586640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe400d0d70 con 0x7fbe4015ea70 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe46586640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbe400d1050 con 0x7fbe40161f90 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe46586640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe4006dca0 con 0x7fbe400d0510 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe3ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe40161f90 0x7fbe400cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe3ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe40161f90 0x7fbe400cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60500/0 (socket says 172.21.15.80:60500) 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe3ffff640 1 -- 172.21.15.80:0/108077534 learned_addr learned my addr 172.21.15.80:0/108077534 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe44b23640 1 --1- 172.21.15.80:0/108077534 >> v1:172.21.15.80:6789/0 conn(0x7fbe4015ea70 0x7fbe4015ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55846/0 (socket says 172.21.15.80:55846) 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe377fe640 1 --2- 172.21.15.80:0/108077534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe400d0510 0x7fbe4015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe3ffff640 1 -- 172.21.15.80:0/108077534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe400d0510 msgr2=0x7fbe4015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe3ffff640 1 --2- 172.21.15.80:0/108077534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe400d0510 0x7fbe4015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe3ffff640 1 -- 172.21.15.80:0/108077534 >> v1:172.21.15.80:6789/0 conn(0x7fbe4015ea70 legacy=0x7fbe4015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe3f7fe640 1 -- 172.21.15.80:0/108077534 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 969747759 0 0) 0x7fbe400d0d70 con 0x7fbe4015ea70 2024-04-25T13:44:22.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.160+0000 7fbe3ffff640 1 -- 172.21.15.80:0/108077534 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4006df80 con 0x7fbe40161f90 2024-04-25T13:44:22.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe3ffff640 1 --2- 172.21.15.80:0/108077534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe40161f90 0x7fbe400cffd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009870 tx=0x7fbe3001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bffea068ccc864 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe3f7fe640 1 -- 172.21.15.80:0/108077534 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300b1070 con 0x7fbe40161f90 2024-04-25T13:44:22.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe3f7fe640 1 -- 172.21.15.80:0/108077534 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe300ac600 con 0x7fbe40161f90 2024-04-25T13:44:22.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe3f7fe640 1 -- 172.21.15.80:0/108077534 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300ac9d0 con 0x7fbe40161f90 2024-04-25T13:44:22.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 -- 172.21.15.80:0/108077534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe40161f90 msgr2=0x7fbe400cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 --2- 172.21.15.80:0/108077534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe40161f90 0x7fbe400cffd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009870 tx=0x7fbe3001c470 comp rx=0 tx=0).stop 2024-04-25T13:44:22.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 -- 172.21.15.80:0/108077534 shutdown_connections 2024-04-25T13:44:22.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 --2- 172.21.15.80:0/108077534 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe400d0510 0x7fbe4015e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 --2- 172.21.15.80:0/108077534 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe40161f90 0x7fbe400cffd0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 -- 172.21.15.80:0/108077534 >> 172.21.15.80:0/108077534 conn(0x7fbe4006d080 msgr2=0x7fbe400b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:22.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 -- 172.21.15.80:0/108077534 shutdown_connections 2024-04-25T13:44:22.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 -- 172.21.15.80:0/108077534 wait complete. 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 Processor -- start 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.164+0000 7fbe46586640 1 -- start start 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe400d0510 0x7fbe40141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe4015ea70 0x7fbe401638e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe40161f90 0x7fbe40163e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4006f4b0 con 0x7fbe400d0510 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4006f630 con 0x7fbe4015ea70 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe400d0920 con 0x7fbe40161f90 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe400d0510 0x7fbe40141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe400d0510 0x7fbe40141240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60504/0 (socket says 172.21.15.80:60504) 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 -- 172.21.15.80:0/2982648660 learned_addr learned my addr 172.21.15.80:0/2982648660 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe44b23640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe40161f90 0x7fbe40163e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe377fe640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe4015ea70 0x7fbe401638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 -- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe40161f90 msgr2=0x7fbe40163e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe40161f90 0x7fbe40163e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 -- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe4015ea70 msgr2=0x7fbe401638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe4015ea70 0x7fbe401638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 -- 172.21.15.80:0/2982648660 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe40160130 con 0x7fbe400d0510 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe377fe640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe4015ea70 0x7fbe401638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3ffff640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe400d0510 0x7fbe40141240 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbe3001a100 tx=0x7fbe3001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3d7fa640 1 -- 172.21.15.80:0/2982648660 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300b1040 con 0x7fbe400d0510 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3d7fa640 1 -- 172.21.15.80:0/2982648660 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe30004020 con 0x7fbe400d0510 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3d7fa640 1 -- 172.21.15.80:0/2982648660 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe300043d0 con 0x7fbe400d0510 2024-04-25T13:44:22.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe40160630 con 0x7fbe400d0510 2024-04-25T13:44:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe40160bf0 con 0x7fbe400d0510 2024-04-25T13:44:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe4006a3a0 con 0x7fbe400d0510 2024-04-25T13:44:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3d7fa640 1 -- 172.21.15.80:0/2982648660 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbe300b4070 con 0x7fbe400d0510 2024-04-25T13:44:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3d7fa640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe2006f400 0x7fbe20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3d7fa640 1 -- 172.21.15.80:0/2982648660 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fbe30129620 con 0x7fbe400d0510 2024-04-25T13:44:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe3d7fa640 1 -- 172.21.15.80:0/2982648660 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=69}) -- 0x7fbe20076fe0 con 0x7fbe400d0510 2024-04-25T13:44:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.168+0000 7fbe377fe640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe2006f400 0x7fbe20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.172+0000 7fbe377fe640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe2006f400 0x7fbe20071910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe400b4af0 tx=0x7fbe28008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.172+0000 7fbe3d7fa640 1 -- 172.21.15.80:0/2982648660 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe300c4030 con 0x7fbe400d0510 2024-04-25T13:44:22.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.360+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe40168810 con 0x7fbe400d0510 2024-04-25T13:44:22.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.360+0000 7fbe3d7fa640 1 -- 172.21.15.80:0/2982648660 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7fbe300bb030 con 0x7fbe400d0510 2024-04-25T13:44:22.360 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:22.361 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1185745761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1185745761},{"type":"v1","addr":"172.21.15.80:6835","nonce":1185745761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2524166918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2524166918},{"type":"v1","addr":"172.21.15.87:6835","nonce":2524166918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3637133191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3637133191},{"type":"v1","addr":"172.21.15.87:6837","nonce":3637133191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:43:13.179327+0000","modified":"2024-04-25T13:43:16.230820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6834/1437895704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1437895704},{"type":"v1","addr":"172.21.15.80:6834","nonce":1437895704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6195,"qdb_cluster":[6195]},"id":4}]} 2024-04-25T13:44:22.361 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 33 2024-04-25T13:44:22.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe2006f400 msgr2=0x7fbe20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe2006f400 0x7fbe20071910 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe400b4af0 tx=0x7fbe28008040 comp rx=0 tx=0).stop 2024-04-25T13:44:22.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe400d0510 msgr2=0x7fbe40141240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe400d0510 0x7fbe40141240 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fbe3001a100 tx=0x7fbe3001ad10 comp rx=0 tx=0).stop 2024-04-25T13:44:22.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 shutdown_connections 2024-04-25T13:44:22.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe2006f400 0x7fbe20071910 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe40161f90 0x7fbe40163e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe4015ea70 0x7fbe401638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 --2- 172.21.15.80:0/2982648660 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe400d0510 0x7fbe40141240 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 >> 172.21.15.80:0/2982648660 conn(0x7fbe4006d080 msgr2=0x7fbe400b58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:22.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 shutdown_connections 2024-04-25T13:44:22.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.364+0000 7fbe46586640 1 -- 172.21.15.80:0/2982648660 wait complete. 2024-04-25T13:44:22.381 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:22.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f50274aa640 1 Processor -- start 2024-04-25T13:44:22.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f50274aa640 1 -- start start 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f50274aa640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f50200b0ed0 0x7f502015c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f50274aa640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f50200ce730 0x7f50200ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f50274aa640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f502015cda0 con 0x7f502006e490 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f50274aa640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5020073710 con 0x7f50200b0ed0 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f50274aa640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f50200739f0 con 0x7f50200ce730 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f5025246640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f50200b0ed0 0x7f502015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f5025246640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f50200b0ed0 0x7f502015c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60516/0 (socket says 172.21.15.80:60516) 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f5025246640 1 -- 172.21.15.80:0/814974396 learned_addr learned my addr 172.21.15.80:0/814974396 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f5025246640 1 -- 172.21.15.80:0/814974396 >> v2:172.21.15.87:3300/0 conn(0x7f50200ce730 msgr2=0x7f50200ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f5025246640 1 --2- 172.21.15.80:0/814974396 >> v2:172.21.15.87:3300/0 conn(0x7f50200ce730 0x7f50200ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f5025246640 1 -- 172.21.15.80:0/814974396 >> v1:172.21.15.87:6789/0 conn(0x7f502006e490 legacy=0x7f502006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:22.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.524+0000 7f5025246640 1 -- 172.21.15.80:0/814974396 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5020073cd0 con 0x7f50200b0ed0 2024-04-25T13:44:22.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f5025246640 1 --2- 172.21.15.80:0/814974396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50200b0ed0 0x7f502015c860 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5020071650 tx=0x7f501c01ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23017812deded40d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f5017fff640 1 -- 172.21.15.80:0/814974396 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0b5070 con 0x7f50200b0ed0 2024-04-25T13:44:22.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f5017fff640 1 -- 172.21.15.80:0/814974396 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f501c0a89d0 con 0x7f50200b0ed0 2024-04-25T13:44:22.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f5017fff640 1 -- 172.21.15.80:0/814974396 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0a8da0 con 0x7f50200b0ed0 2024-04-25T13:44:22.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f50274aa640 1 -- 172.21.15.80:0/814974396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50200b0ed0 msgr2=0x7f502015c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f50274aa640 1 --2- 172.21.15.80:0/814974396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50200b0ed0 0x7f502015c860 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5020071650 tx=0x7f501c01ecd0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f50274aa640 1 -- 172.21.15.80:0/814974396 shutdown_connections 2024-04-25T13:44:22.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f50274aa640 1 --2- 172.21.15.80:0/814974396 >> v2:172.21.15.87:3300/0 conn(0x7f50200ce730 0x7f50200ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f50274aa640 1 --2- 172.21.15.80:0/814974396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f50200b0ed0 0x7f502015c860 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f50274aa640 1 -- 172.21.15.80:0/814974396 >> 172.21.15.80:0/814974396 conn(0x7f502006d080 msgr2=0x7f50200b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:22.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f50274aa640 1 -- 172.21.15.80:0/814974396 shutdown_connections 2024-04-25T13:44:22.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.528+0000 7f50274aa640 1 -- 172.21.15.80:0/814974396 wait complete. 2024-04-25T13:44:22.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 Processor -- start 2024-04-25T13:44:22.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 -- start start 2024-04-25T13:44:22.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f502006e490 0x7f50200b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50200b0ed0 0x7f50200b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50200ce730 0x7f502013d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f50200733a0 con 0x7f502006e490 2024-04-25T13:44:22.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f50200b12e0 con 0x7f50200b0ed0 2024-04-25T13:44:22.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f502006dc00 con 0x7f50200ce730 2024-04-25T13:44:22.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f502006e490 0x7f50200b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f502006e490 0x7f50200b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60530/0 (socket says 172.21.15.80:60530) 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 -- 172.21.15.80:0/1629132765 learned_addr learned my addr 172.21.15.80:0/1629132765 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025a47640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50200ce730 0x7f502013d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5024a45640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50200b0ed0 0x7f50200b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 -- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50200ce730 msgr2=0x7f502013d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50200ce730 0x7f502013d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 -- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50200b0ed0 msgr2=0x7f50200b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50200b0ed0 0x7f50200b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 -- 172.21.15.80:0/1629132765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502015cbd0 con 0x7f502006e490 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5024a45640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50200b0ed0 0x7f50200b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5025246640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f502006e490 0x7f50200b2030 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f501c01d380 tx=0x7f501c01ba70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5015ffb640 1 -- 172.21.15.80:0/1629132765 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0b5040 con 0x7f502006e490 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5015ffb640 1 -- 172.21.15.80:0/1629132765 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f501c0be070 con 0x7f502006e490 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50201518d0 con 0x7f502006e490 2024-04-25T13:44:22.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f5015ffb640 1 -- 172.21.15.80:0/1629132765 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501c0b0b10 con 0x7f502006e490 2024-04-25T13:44:22.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5020151e90 con 0x7f502006e490 2024-04-25T13:44:22.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.532+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f502006a3a0 con 0x7f502006e490 2024-04-25T13:44:22.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.536+0000 7f5015ffb640 1 -- 172.21.15.80:0/1629132765 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f501c0b0cb0 con 0x7f502006e490 2024-04-25T13:44:22.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.536+0000 7f5015ffb640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f500406f400 0x7f5004071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.536+0000 7f5015ffb640 1 -- 172.21.15.80:0/1629132765 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f501c12c9c0 con 0x7f502006e490 2024-04-25T13:44:22.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.536+0000 7f5015ffb640 1 -- 172.21.15.80:0/1629132765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=70}) -- 0x7f5004076f80 con 0x7f502006e490 2024-04-25T13:44:22.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.536+0000 7f5024a45640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f500406f400 0x7f5004071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.536+0000 7f5024a45640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f500406f400 0x7f5004071910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f50200b2bf0 tx=0x7f5010008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.540+0000 7f5015ffb640 1 -- 172.21.15.80:0/1629132765 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f501c12ee20 con 0x7f502006e490 2024-04-25T13:44:22.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.720+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f502006f880 con 0x7f502006e490 2024-04-25T13:44:22.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.720+0000 7f5015ffb640 1 -- 172.21.15.80:0/1629132765 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+5181 (secure 0 0 0) 0x7f501c018830 con 0x7f502006e490 2024-04-25T13:44:22.721 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:22.721 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1185745761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1185745761},{"type":"v1","addr":"172.21.15.80:6835","nonce":1185745761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2524166918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2524166918},{"type":"v1","addr":"172.21.15.87:6835","nonce":2524166918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3637133191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3637133191},{"type":"v1","addr":"172.21.15.87:6837","nonce":3637133191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:43:13.179327+0000","modified":"2024-04-25T13:43:16.230820+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6195},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6195":{"gid":6195,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6834/1437895704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1437895704},{"type":"v1","addr":"172.21.15.80:6834","nonce":1437895704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6195,"qdb_cluster":[6195]},"id":4}]} 2024-04-25T13:44:22.722 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 33 2024-04-25T13:44:22.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f500406f400 msgr2=0x7f5004071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f500406f400 0x7f5004071910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f50200b2bf0 tx=0x7f5010008040 comp rx=0 tx=0).stop 2024-04-25T13:44:22.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f502006e490 msgr2=0x7f50200b2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f502006e490 0x7f50200b2030 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f501c01d380 tx=0x7f501c01ba70 comp rx=0 tx=0).stop 2024-04-25T13:44:22.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 shutdown_connections 2024-04-25T13:44:22.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f500406f400 0x7f5004071910 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f50200ce730 0x7f502013d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50200b0ed0 0x7f50200b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 --2- 172.21.15.80:0/1629132765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f502006e490 0x7f50200b2030 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 >> 172.21.15.80:0/1629132765 conn(0x7f502006d080 msgr2=0x7f50200d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:22.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 shutdown_connections 2024-04-25T13:44:22.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.724+0000 7f50274aa640 1 -- 172.21.15.80:0/1629132765 wait complete. 2024-04-25T13:44:22.741 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:22.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.876+0000 7f19f52a9640 1 Processor -- start 2024-04-25T13:44:22.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.876+0000 7f19f52a9640 1 -- start start 2024-04-25T13:44:22.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19f52a9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f006f5a0 0x7f19f0160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19f52a9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19f00733b0 0x7f19f0161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19f52a9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f19f006fcf0 con 0x7f19f0161070 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19f52a9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f19f01619c0 con 0x7f19f00733b0 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19f52a9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f19f0073f10 con 0x7f19f006f5a0 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19e67fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f19f0161070 0x7f19f00b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55850/0 (socket says 172.21.15.80:55850) 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19e67fc640 1 -- 172.21.15.80:0/1784903409 learned_addr learned my addr 172.21.15.80:0/1784903409 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19eeffd640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f006f5a0 0x7f19f0160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ef7fe640 1 --2- 172.21.15.80:0/1784903409 >> v2:172.21.15.87:3300/0 conn(0x7f19f00733b0 0x7f19f0161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1106834161 0 0) 0x7f19f006fcf0 con 0x7f19f0161070 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f19dc003530 con 0x7f19f0161070 2024-04-25T13:44:22.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 967703223 0 0) 0x7f19dc003530 con 0x7f19f0161070 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f006f5a0 msgr2=0x7f19f0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f006f5a0 0x7f19f0160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 >> v2:172.21.15.87:3300/0 conn(0x7f19f00733b0 msgr2=0x7f19f0161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 --2- 172.21.15.80:0/1784903409 >> v2:172.21.15.87:3300/0 conn(0x7f19f00733b0 0x7f19f0161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f00b0c60 con 0x7f19f0161070 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19eeffd640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f006f5a0 0x7f19f0160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f19d8001680 con 0x7f19f0161070 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 >> v1:172.21.15.80:6789/0 conn(0x7f19f0161070 legacy=0x7f19f00b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19dc0039f0 0x7f19dc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc004810 0x7f19dc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f006f5a0 0x7f19dc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19eeffd640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19dc0039f0 0x7f19dc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f19f01619c0 con 0x7f19dc0039f0 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0073f10 con 0x7f19dc004810 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f19dc00b9d0 con 0x7f19f006f5a0 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19eeffd640 1 -- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f006f5a0 msgr2=0x7f19dc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19eeffd640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f006f5a0 0x7f19dc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19eeffd640 1 -- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc004810 msgr2=0x7f19dc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19eeffd640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc004810 0x7f19dc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.880+0000 7f19eeffd640 1 -- 172.21.15.80:0/1784903409 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19dc003530 con 0x7f19dc0039f0 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19eeffd640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19dc0039f0 0x7f19dc003e20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009c80 tx=0x7f19e001cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aed62ad421fb91d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e0009880 con 0x7f19dc0039f0 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e0004020 con 0x7f19dc0039f0 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19ee7fc640 1 -- 172.21.15.80:0/1784903409 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e00043f0 con 0x7f19dc0039f0 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19f52a9640 1 -- 172.21.15.80:0/1784903409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19dc0039f0 msgr2=0x7f19dc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19f52a9640 1 --2- 172.21.15.80:0/1784903409 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19dc0039f0 0x7f19dc003e20 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009c80 tx=0x7f19e001cc10 comp rx=0 tx=0).stop 2024-04-25T13:44:22.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19ef7fe640 1 -- 172.21.15.80:0/1784903409 reap_dead start 2024-04-25T13:44:22.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19f52a9640 1 -- 172.21.15.80:0/1784903409 shutdown_connections 2024-04-25T13:44:22.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19f52a9640 1 -- 172.21.15.80:0/1784903409 >> 172.21.15.80:0/1784903409 conn(0x7f19f006d080 msgr2=0x7f19f00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:22.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19f52a9640 1 -- 172.21.15.80:0/1784903409 shutdown_connections 2024-04-25T13:44:22.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19f52a9640 1 -- 172.21.15.80:0/1784903409 wait complete. 2024-04-25T13:44:22.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19f52a9640 1 Processor -- start 2024-04-25T13:44:22.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.884+0000 7f19f52a9640 1 -- start start 2024-04-25T13:44:22.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc0039f0 0x7f19f0155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f01f7cc0 0x7f19f015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f015bbd0 0x7f19f006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0168040 con 0x7f19f015bbd0 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f19f0167ec0 con 0x7f19dc0039f0 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f19f01681c0 con 0x7f19f01f7cc0 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f015bbd0 0x7f19f006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f015bbd0 0x7f19f006e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60548/0 (socket says 172.21.15.80:60548) 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 -- 172.21.15.80:0/333334386 learned_addr learned my addr 172.21.15.80:0/333334386 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19e67fc640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f01f7cc0 0x7f19f015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19eeffd640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc0039f0 0x7f19f0155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 -- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f01f7cc0 msgr2=0x7f19f015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f01f7cc0 0x7f19f015b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 -- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc0039f0 msgr2=0x7f19f0155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc0039f0 0x7f19f0155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:22.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 -- 172.21.15.80:0/333334386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f0164080 con 0x7f19f015bbd0 2024-04-25T13:44:22.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19e67fc640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f01f7cc0 0x7f19f015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:22.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19eeffd640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc0039f0 0x7f19f0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:22.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19ef7fe640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f015bbd0 0x7f19f006e490 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f19f00701f0 tx=0x7f19e800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19e7fff640 1 -- 172.21.15.80:0/333334386 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e800bb10 con 0x7f19f015bbd0 2024-04-25T13:44:22.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19e7fff640 1 -- 172.21.15.80:0/333334386 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e801a070 con 0x7f19f015bbd0 2024-04-25T13:44:22.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f0164360 con 0x7f19f015bbd0 2024-04-25T13:44:22.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19e7fff640 1 -- 172.21.15.80:0/333334386 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e8014400 con 0x7f19f015bbd0 2024-04-25T13:44:22.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f0164920 con 0x7f19f015bbd0 2024-04-25T13:44:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f006a3a0 con 0x7f19f015bbd0 2024-04-25T13:44:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.888+0000 7f19e7fff640 1 -- 172.21.15.80:0/333334386 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f19e8013070 con 0x7f19f015bbd0 2024-04-25T13:44:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.892+0000 7f19e7fff640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d006f400 0x7f19d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.892+0000 7f19e7fff640 1 -- 172.21.15.80:0/333334386 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f19e808f420 con 0x7f19f015bbd0 2024-04-25T13:44:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.892+0000 7f19e7fff640 1 -- 172.21.15.80:0/333334386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=70}) -- 0x7f19d0076fe0 con 0x7f19f015bbd0 2024-04-25T13:44:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.892+0000 7f19eeffd640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d006f400 0x7f19d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.892+0000 7f19eeffd640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d006f400 0x7f19d0071910 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f19e00ae000 tx=0x7f19e00046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:22.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:22.896+0000 7f19e7fff640 1 -- 172.21.15.80:0/333334386 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f19e8024020 con 0x7f19f015bbd0 2024-04-25T13:44:23.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.056+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19f00b66e0 con 0x7f19f015bbd0 2024-04-25T13:44:23.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19e7fff640 1 -- 172.21.15.80:0/333334386 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16615 (secure 0 0 0) 0x7f19e805c3a0 con 0x7f19f015bbd0 2024-04-25T13:44:23.058 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:23.058 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":69,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:44:22.440606+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:43:11.515229+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T13:43:12.143130+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:23.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d006f400 msgr2=0x7f19d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d006f400 0x7f19d0071910 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f19e00ae000 tx=0x7f19e00046d0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f015bbd0 msgr2=0x7f19f006e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f015bbd0 0x7f19f006e490 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f19f00701f0 tx=0x7f19e800b480 comp rx=0 tx=0).stop 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 shutdown_connections 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d006f400 0x7f19d0071910 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f015bbd0 0x7f19f006e490 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f01f7cc0 0x7f19f015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 --2- 172.21.15.80:0/333334386 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19dc0039f0 0x7f19f0155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 >> 172.21.15.80:0/333334386 conn(0x7f19f006d080 msgr2=0x7f19f00cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 shutdown_connections 2024-04-25T13:44:23.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.060+0000 7f19f52a9640 1 -- 172.21.15.80:0/333334386 wait complete. 2024-04-25T13:44:23.080 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:44:23.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.232+0000 7efd2fd9d640 1 Processor -- start 2024-04-25T13:44:23.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.232+0000 7efd2fd9d640 1 -- start start 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2fd9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efd280b7e70 0x7efd280d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2fd9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd280b9770 0x7efd280b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2fd9d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd280d4e00 con 0x7efd281639e0 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2fd9d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efd280d1a70 con 0x7efd280b9770 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2fd9d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efd280d5a10 con 0x7efd280b7e70 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2e33a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd280b9770 0x7efd280b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2e33a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd280b9770 0x7efd280b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60552/0 (socket says 172.21.15.80:60552) 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2e33a640 1 -- 172.21.15.80:0/2148439405 learned_addr learned my addr 172.21.15.80:0/2148439405 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2d338640 1 --2- 172.21.15.80:0/2148439405 >> v2:172.21.15.87:3300/0 conn(0x7efd280b7e70 0x7efd280d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2e33a640 1 -- 172.21.15.80:0/2148439405 >> v2:172.21.15.87:3300/0 conn(0x7efd280b7e70 msgr2=0x7efd280d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:23.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2e33a640 1 --2- 172.21.15.80:0/2148439405 >> v2:172.21.15.87:3300/0 conn(0x7efd280b7e70 0x7efd280d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2e33a640 1 -- 172.21.15.80:0/2148439405 >> v1:172.21.15.87:6789/0 conn(0x7efd281639e0 legacy=0x7efd280b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:23.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2e33a640 1 -- 172.21.15.80:0/2148439405 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd280d50e0 con 0x7efd280b9770 2024-04-25T13:44:23.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2e33a640 1 --2- 172.21.15.80:0/2148439405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280b9770 0x7efd280b9b80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efd280d2d20 tx=0x7efd2001c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221b78006a595648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:23.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2cb37640 1 -- 172.21.15.80:0/2148439405 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd200a5670 con 0x7efd280b9770 2024-04-25T13:44:23.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.236+0000 7efd2cb37640 1 -- 172.21.15.80:0/2148439405 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd200a5810 con 0x7efd280b9770 2024-04-25T13:44:23.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2cb37640 1 -- 172.21.15.80:0/2148439405 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd200a5be0 con 0x7efd280b9770 2024-04-25T13:44:23.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 -- 172.21.15.80:0/2148439405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280b9770 msgr2=0x7efd280b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/2148439405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280b9770 0x7efd280b9b80 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7efd280d2d20 tx=0x7efd2001c9b0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 -- 172.21.15.80:0/2148439405 shutdown_connections 2024-04-25T13:44:23.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/2148439405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280b9770 0x7efd280b9b80 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/2148439405 >> v2:172.21.15.87:3300/0 conn(0x7efd280b7e70 0x7efd280d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 -- 172.21.15.80:0/2148439405 >> 172.21.15.80:0/2148439405 conn(0x7efd2806c9f0 msgr2=0x7efd2806ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:23.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 -- 172.21.15.80:0/2148439405 shutdown_connections 2024-04-25T13:44:23.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 -- 172.21.15.80:0/2148439405 wait complete. 2024-04-25T13:44:23.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 Processor -- start 2024-04-25T13:44:23.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.240+0000 7efd2fd9d640 1 -- start start 2024-04-25T13:44:23.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd280b9770 0x7efd28153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd281639e0 0x7efd280cc300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280d5bc0 0x7efd280d5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efd2816d1f0 con 0x7efd280d5bc0 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efd2816d070 con 0x7efd280b9770 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efd2816d370 con 0x7efd281639e0 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280d5bc0 0x7efd280d5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280d5bc0 0x7efd280d5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60558/0 (socket says 172.21.15.80:60558) 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 -- 172.21.15.80:0/1164565835 learned_addr learned my addr 172.21.15.80:0/1164565835 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2db39640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd280b9770 0x7efd28153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2d338640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd281639e0 0x7efd280cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 -- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd281639e0 msgr2=0x7efd280cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd281639e0 0x7efd280cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 -- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd280b9770 msgr2=0x7efd28153b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd280b9770 0x7efd28153b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 -- 172.21.15.80:0/1164565835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd280d6510 con 0x7efd280d5bc0 2024-04-25T13:44:23.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2e33a640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280d5bc0 0x7efd280d5fd0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efd200af040 tx=0x7efd20019750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:23.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd200b8670 con 0x7efd280d5bc0 2024-04-25T13:44:23.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd200be070 con 0x7efd280d5bc0 2024-04-25T13:44:23.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd28154620 con 0x7efd280d5bc0 2024-04-25T13:44:23.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd200b8a50 con 0x7efd280d5bc0 2024-04-25T13:44:23.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd28154be0 con 0x7efd280d5bc0 2024-04-25T13:44:23.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd28154370 con 0x7efd280d5bc0 2024-04-25T13:44:23.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efd200b8bf0 con 0x7efd280d5bc0 2024-04-25T13:44:23.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd1e7fc640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd0c06f400 0x7efd0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7efd2012a440 con 0x7efd280d5bc0 2024-04-25T13:44:23.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=70}) -- 0x7efd0c076f80 con 0x7efd280d5bc0 2024-04-25T13:44:23.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.244+0000 7efd2db39640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd0c06f400 0x7efd0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.248+0000 7efd2db39640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd0c06f400 0x7efd0c071910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efd2400b810 tx=0x7efd24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:23.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.252+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efd280d6510 con 0x7efd280d5bc0 2024-04-25T13:44:23.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.444+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efd280b8830 con 0x7efd280d5bc0 2024-04-25T13:44:23.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.460+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 7 ==== osd_map(70..70 src has 1..70) ==== 1468+0+0 (secure 0 0 0) 0x7efd2015ce70 con 0x7efd280d5bc0 2024-04-25T13:44:23.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.460+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=71}) -- 0x7efd0c0780f0 con 0x7efd280d5bc0 2024-04-25T13:44:23.462 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:44:23.460+0000 7fa2247d7640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:44:23.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.472+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v34) ==== 161+0+0 (secure 0 0 0) 0x7efd200165a0 con 0x7efd280d5bc0 2024-04-25T13:44:23.473 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:44:23.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.476+0000 7efd1e7fc640 1 -- 172.21.15.80:0/1164565835 <== mon.0 v2:172.21.15.80:3300/0 9 ==== osd_map(71..71 src has 1..71) ==== 374+0+0 (secure 0 0 0) 0x7efd20128a80 con 0x7efd280d5bc0 2024-04-25T13:44:23.481 INFO:tasks.ceph.mds.a.smithi080.stderr: -7941> 2024-04-25T13:43:15.745+0000 7fd529f3a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd0c06f400 msgr2=0x7efd0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd0c06f400 0x7efd0c071910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7efd2400b810 tx=0x7efd24008040 comp rx=0 tx=0).stop 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280d5bc0 msgr2=0x7efd280d5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280d5bc0 0x7efd280d5fd0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7efd200af040 tx=0x7efd20019750 comp rx=0 tx=0).stop 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 shutdown_connections 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd0c06f400 0x7efd0c071910 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd280d5bc0 0x7efd280d5fd0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd281639e0 0x7efd280cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 --2- 172.21.15.80:0/1164565835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd280b9770 0x7efd28153b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 >> 172.21.15.80:0/1164565835 conn(0x7efd2806c9f0 msgr2=0x7efd28162c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:23.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 shutdown_connections 2024-04-25T13:44:23.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.484+0000 7efd2fd9d640 1 -- 172.21.15.80:0/1164565835 wait complete. 2024-04-25T13:44:23.505 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:44:23.542 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:44:23.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04d57ec640 1 Processor -- start 2024-04-25T13:44:23.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04d57ec640 1 -- start start 2024-04-25T13:44:23.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04d57ec640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04d00b9190 0x7f04d00d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04d57ec640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00d0b80 0x7f04d00d1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04d57ec640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04d00ce180 con 0x7f04d00d20d0 2024-04-25T13:44:23.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04d57ec640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f04d00b3230 con 0x7f04d00b9190 2024-04-25T13:44:23.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04d57ec640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04d006c8a0 con 0x7f04d00d0b80 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04ceffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04d00b9190 0x7f04d00d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04cf7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f04d00d20d0 0x7f04d00d24e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55864/0 (socket says 172.21.15.80:55864) 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04cf7fe640 1 -- 172.21.15.80:0/3738520519 learned_addr learned my addr 172.21.15.80:0/3738520519 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04ce7fc640 1 --2- 172.21.15.80:0/3738520519 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00d0b80 0x7f04d00d1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04ceffd640 1 -- 172.21.15.80:0/3738520519 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00d0b80 msgr2=0x7f04d00d1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04ceffd640 1 --2- 172.21.15.80:0/3738520519 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00d0b80 0x7f04d00d1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04ceffd640 1 -- 172.21.15.80:0/3738520519 >> v1:172.21.15.80:6789/0 conn(0x7f04d00d20d0 legacy=0x7f04d00d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04ceffd640 1 -- 172.21.15.80:0/3738520519 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d00ce420 con 0x7f04d00b9190 2024-04-25T13:44:23.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.652+0000 7f04cdffb640 1 -- 172.21.15.80:0/3738520519 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2812540985 0 0) 0x7f04d00ce180 con 0x7f04d00d20d0 2024-04-25T13:44:23.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04ceffd640 1 --2- 172.21.15.80:0/3738520519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d00d0640 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f04c00097c0 tx=0x7f04c001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c481c679b616e5ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:23.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04cdffb640 1 -- 172.21.15.80:0/3738520519 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00b0070 con 0x7f04d00b9190 2024-04-25T13:44:23.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04cdffb640 1 -- 172.21.15.80:0/3738520519 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c00ab600 con 0x7f04d00b9190 2024-04-25T13:44:23.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04cdffb640 1 -- 172.21.15.80:0/3738520519 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00ab9d0 con 0x7f04d00b9190 2024-04-25T13:44:23.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04d57ec640 1 -- 172.21.15.80:0/3738520519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 msgr2=0x7f04d00d0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04d57ec640 1 --2- 172.21.15.80:0/3738520519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d00d0640 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f04c00097c0 tx=0x7f04c001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04d57ec640 1 -- 172.21.15.80:0/3738520519 shutdown_connections 2024-04-25T13:44:23.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04d57ec640 1 --2- 172.21.15.80:0/3738520519 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00d0b80 0x7f04d00d1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04d57ec640 1 --2- 172.21.15.80:0/3738520519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d00d0640 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04d57ec640 1 -- 172.21.15.80:0/3738520519 >> 172.21.15.80:0/3738520519 conn(0x7f04d00d3a90 msgr2=0x7f04d00d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:23.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04d57ec640 1 -- 172.21.15.80:0/3738520519 shutdown_connections 2024-04-25T13:44:23.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.656+0000 7f04d57ec640 1 -- 172.21.15.80:0/3738520519 wait complete. 2024-04-25T13:44:23.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 Processor -- start 2024-04-25T13:44:23.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 -- start start 2024-04-25T13:44:23.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d0162610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04d00d20d0 0x7f04d00b87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00a1710 0x7f04d00b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f04d016d970 con 0x7f04d00b9190 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04d016d7f0 con 0x7f04d00d20d0 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04d016daf0 con 0x7f04d00a1710 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d0162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d0162610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60594/0 (socket says 172.21.15.80:60594) 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 -- 172.21.15.80:0/433904847 learned_addr learned my addr 172.21.15.80:0/433904847 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 -- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00a1710 msgr2=0x7f04d00b68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ce7fc640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04d00d20d0 0x7f04d00b87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04cf7fe640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00a1710 0x7f04d00b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00a1710 0x7f04d00b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 -- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04d00d20d0 msgr2=0x7f04d00b87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:23.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04d00d20d0 0x7f04d00b87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:23.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 -- 172.21.15.80:0/433904847 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d0163550 con 0x7f04d00b9190 2024-04-25T13:44:23.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04ceffd640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d0162610 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f04c00097c0 tx=0x7f04c001ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:23.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04b77fe640 1 -- 172.21.15.80:0/433904847 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00bd070 con 0x7f04d00b9190 2024-04-25T13:44:23.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04b77fe640 1 -- 172.21.15.80:0/433904847 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c0004030 con 0x7f04d00b9190 2024-04-25T13:44:23.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d0162cf0 con 0x7f04d00b9190 2024-04-25T13:44:23.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04b77fe640 1 -- 172.21.15.80:0/433904847 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c00b0040 con 0x7f04d00b9190 2024-04-25T13:44:23.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d0163270 con 0x7f04d00b9190 2024-04-25T13:44:23.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.660+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d006a3a0 con 0x7f04d00b9190 2024-04-25T13:44:23.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.664+0000 7f04b77fe640 1 -- 172.21.15.80:0/433904847 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04c00c0070 con 0x7f04d00b9190 2024-04-25T13:44:23.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.664+0000 7f04b77fe640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04bc06f400 0x7f04bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:23.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.664+0000 7f04b77fe640 1 -- 172.21.15.80:0/433904847 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f04c0128df0 con 0x7f04d00b9190 2024-04-25T13:44:23.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.664+0000 7f04ce7fc640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04bc06f400 0x7f04bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:23.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.664+0000 7f04ce7fc640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04bc06f400 0x7f04bc071910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04d01fd140 tx=0x7f04c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:23.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.668+0000 7f04b77fe640 1 -- 172.21.15.80:0/433904847 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04d0163550 con 0x7f04d00b9190 2024-04-25T13:44:23.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:23.864+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f04d006cde0 con 0x7f04d00b9190 2024-04-25T13:44:24.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.476+0000 7f04b77fe640 1 -- 172.21.15.80:0/433904847 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v35) ==== 104+0+0 (secure 0 0 0) 0x7f04c0016090 con 0x7f04d00b9190 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04bc06f400 msgr2=0x7f04bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04bc06f400 0x7f04bc071910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f04d01fd140 tx=0x7f04c4008040 comp rx=0 tx=0).stop 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 msgr2=0x7f04d0162610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d0162610 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f04c00097c0 tx=0x7f04c001ace0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 shutdown_connections 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04bc06f400 0x7f04bc071910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04d00a1710 0x7f04d00b68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04d00d20d0 0x7f04d00b87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 --2- 172.21.15.80:0/433904847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04d00b9190 0x7f04d0162610 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 >> 172.21.15.80:0/433904847 conn(0x7f04d00d3a90 msgr2=0x7f04d00b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:24.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 shutdown_connections 2024-04-25T13:44:24.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.480+0000 7f04d57ec640 1 -- 172.21.15.80:0/433904847 wait complete. 2024-04-25T13:44:24.503 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:44:24.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f99227cc640 1 Processor -- start 2024-04-25T13:44:24.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f99227cc640 1 -- start start 2024-04-25T13:44:24.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f99227cc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f991c0b48c0 0x7f991c161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:24.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f99227cc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0d0840 0x7f991c0d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:24.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f99227cc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f991c0ce220 0x7f991c0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:24.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f99227cc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f991c0b0bc0 con 0x7f991c0b48c0 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f99227cc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f991c0d1190 con 0x7f991c0ce220 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f99227cc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f991c0b0e10 con 0x7f991c0d0840 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f991bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f991c0b48c0 0x7f991c161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f991bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f991c0b48c0 0x7f991c161600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60608/0 (socket says 172.21.15.80:60608) 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.708+0000 7f991bfff640 1 -- 172.21.15.80:0/1743160883 learned_addr learned my addr 172.21.15.80:0/1743160883 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991b7fe640 1 --2- 172.21.15.80:0/1743160883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0d0840 0x7f991c0d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f9920d69640 1 --2- 172.21.15.80:0/1743160883 >> v2:172.21.15.87:3300/0 conn(0x7f991c0ce220 0x7f991c0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991bfff640 1 -- 172.21.15.80:0/1743160883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0d0840 msgr2=0x7f991c0d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991bfff640 1 --2- 172.21.15.80:0/1743160883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0d0840 0x7f991c0d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991bfff640 1 -- 172.21.15.80:0/1743160883 >> v2:172.21.15.87:3300/0 conn(0x7f991c0ce220 msgr2=0x7f991c0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991bfff640 1 --2- 172.21.15.80:0/1743160883 >> v2:172.21.15.87:3300/0 conn(0x7f991c0ce220 0x7f991c0ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991bfff640 1 -- 172.21.15.80:0/1743160883 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c0b10f0 con 0x7f991c0b48c0 2024-04-25T13:44:24.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991bfff640 1 --2- 172.21.15.80:0/1743160883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0b48c0 0x7f991c161600 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f991c0b6d10 tx=0x7f990c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d2736291e0d5477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:24.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991affd640 1 -- 172.21.15.80:0/1743160883 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c0b0070 con 0x7f991c0b48c0 2024-04-25T13:44:24.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991affd640 1 -- 172.21.15.80:0/1743160883 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f990c0ab600 con 0x7f991c0b48c0 2024-04-25T13:44:24.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f991affd640 1 -- 172.21.15.80:0/1743160883 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f990c0ab9d0 con 0x7f991c0b48c0 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 -- 172.21.15.80:0/1743160883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0b48c0 msgr2=0x7f991c161600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 --2- 172.21.15.80:0/1743160883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0b48c0 0x7f991c161600 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f991c0b6d10 tx=0x7f990c01c490 comp rx=0 tx=0).stop 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 -- 172.21.15.80:0/1743160883 shutdown_connections 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 --2- 172.21.15.80:0/1743160883 >> v2:172.21.15.87:3300/0 conn(0x7f991c0ce220 0x7f991c0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 --2- 172.21.15.80:0/1743160883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0d0840 0x7f991c0d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 --2- 172.21.15.80:0/1743160883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0b48c0 0x7f991c161600 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 -- 172.21.15.80:0/1743160883 >> 172.21.15.80:0/1743160883 conn(0x7f991c0b35c0 msgr2=0x7f991c0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 -- 172.21.15.80:0/1743160883 shutdown_connections 2024-04-25T13:44:24.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.712+0000 7f99227cc640 1 -- 172.21.15.80:0/1743160883 wait complete. 2024-04-25T13:44:24.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 Processor -- start 2024-04-25T13:44:24.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 -- start start 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0b48c0 0x7f991c0cdab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991c0ce220 0x7f991c1482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0d0840 0x7f991c06de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f991c06fc30 con 0x7f991c0d0840 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f991c0cff40 con 0x7f991c0ce220 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f991c158470 con 0x7f991c0b48c0 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0d0840 0x7f991c06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0d0840 0x7f991c06de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60618/0 (socket says 172.21.15.80:60618) 2024-04-25T13:44:24.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 -- 172.21.15.80:0/2942434112 learned_addr learned my addr 172.21.15.80:0/2942434112 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f991bfff640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0b48c0 0x7f991c0cdab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f991b7fe640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991c0ce220 0x7f991c1482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 -- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0b48c0 msgr2=0x7f991c0cdab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0b48c0 0x7f991c0cdab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 -- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991c0ce220 msgr2=0x7f991c1482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991c0ce220 0x7f991c1482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 -- 172.21.15.80:0/2942434112 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991c149c90 con 0x7f991c0d0840 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9920d69640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0d0840 0x7f991c06de00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9914007d80 tx=0x7f991400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9918ff9640 1 -- 172.21.15.80:0/2942434112 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914014420 con 0x7f991c0d0840 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9918ff9640 1 -- 172.21.15.80:0/2942434112 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9914019070 con 0x7f991c0d0840 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f991c14d1f0 con 0x7f991c0d0840 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f9918ff9640 1 -- 172.21.15.80:0/2942434112 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914014800 con 0x7f991c0d0840 2024-04-25T13:44:24.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.716+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f991c14d7b0 con 0x7f991c0d0840 2024-04-25T13:44:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.720+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991c06a440 con 0x7f991c0d0840 2024-04-25T13:44:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.720+0000 7f9918ff9640 1 -- 172.21.15.80:0/2942434112 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f99140149a0 con 0x7f991c0d0840 2024-04-25T13:44:24.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.720+0000 7f9918ff9640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f990806f430 0x7f9908071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:24.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.720+0000 7f9918ff9640 1 -- 172.21.15.80:0/2942434112 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f991408fa70 con 0x7f991c0d0840 2024-04-25T13:44:24.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.720+0000 7f991bfff640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f990806f430 0x7f9908071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:24.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.720+0000 7f991bfff640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f990806f430 0x7f9908071940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f990c0ad000 tx=0x7f990c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:24.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.724+0000 7f9918ff9640 1 -- 172.21.15.80:0/2942434112 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f991c149c90 con 0x7f991c0d0840 2024-04-25T13:44:24.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:24.884+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f991c0ce050 con 0x7f991c0d0840 2024-04-25T13:44:25.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.488+0000 7f9918ff9640 1 -- 172.21.15.80:0/2942434112 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) ==== 181+0+0 (secure 0 0 0) 0x7f991405c340 con 0x7f991c0d0840 2024-04-25T13:44:25.488 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:44:25.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.492+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f990806f430 msgr2=0x7f9908071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:25.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.492+0000 7f99227cc640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f990806f430 0x7f9908071940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f990c0ad000 tx=0x7f990c0a6000 comp rx=0 tx=0).stop 2024-04-25T13:44:25.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.492+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0d0840 msgr2=0x7f991c06de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:25.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.492+0000 7f99227cc640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0d0840 0x7f991c06de00 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f9914007d80 tx=0x7f991400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.496+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 shutdown_connections 2024-04-25T13:44:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.496+0000 7f99227cc640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f990806f430 0x7f9908071940 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.496+0000 7f99227cc640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f991c0d0840 0x7f991c06de00 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.496+0000 7f99227cc640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991c0ce220 0x7f991c1482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.496+0000 7f99227cc640 1 --2- 172.21.15.80:0/2942434112 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f991c0b48c0 0x7f991c0cdab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.496+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 >> 172.21.15.80:0/2942434112 conn(0x7f991c0b35c0 msgr2=0x7f991c0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.496+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 shutdown_connections 2024-04-25T13:44:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.496+0000 7f99227cc640 1 -- 172.21.15.80:0/2942434112 wait complete. 2024-04-25T13:44:25.516 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:44:25.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f453506e640 1 Processor -- start 2024-04-25T13:44:25.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f453506e640 1 -- start start 2024-04-25T13:44:25.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f453506e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45300ce2a0 0x7f45300ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f453506e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4530161d10 0x7f4530162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f453506e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f45300b1320 con 0x7f45300b0f10 2024-04-25T13:44:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f453506e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f45300b5820 con 0x7f45300ce2a0 2024-04-25T13:44:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f453506e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f45300d13f0 con 0x7f4530161d10 2024-04-25T13:44:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f452e59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45300ce2a0 0x7f45300ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:25.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f452ed9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f45300b0f10 0x7f45300d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55878/0 (socket says 172.21.15.80:55878) 2024-04-25T13:44:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f452ed9d640 1 -- 172.21.15.80:0/3365861185 learned_addr learned my addr 172.21.15.80:0/3365861185 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.672+0000 7f452e59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45300ce2a0 0x7f45300ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60632/0 (socket says 172.21.15.80:60632) 2024-04-25T13:44:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452f59e640 1 --2- 172.21.15.80:0/3365861185 >> v2:172.21.15.87:3300/0 conn(0x7f4530161d10 0x7f4530162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452e59c640 1 -- 172.21.15.80:0/3365861185 >> v2:172.21.15.87:3300/0 conn(0x7f4530161d10 msgr2=0x7f4530162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452e59c640 1 --2- 172.21.15.80:0/3365861185 >> v2:172.21.15.87:3300/0 conn(0x7f4530161d10 0x7f4530162120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452e59c640 1 -- 172.21.15.80:0/3365861185 >> v1:172.21.15.80:6789/0 conn(0x7f45300b0f10 legacy=0x7f45300d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:25.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452e59c640 1 -- 172.21.15.80:0/3365861185 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45300cebf0 con 0x7f45300ce2a0 2024-04-25T13:44:25.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452dd9b640 1 -- 172.21.15.80:0/3365861185 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3199173887 0 0) 0x7f4520001120 con 0x7f45300b0f10 2024-04-25T13:44:25.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452e59c640 1 --2- 172.21.15.80:0/3365861185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300ce2a0 0x7f45300ce6b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f45240097c0 tx=0x7f452401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693366c8b676039c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:25.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452dd9b640 1 -- 172.21.15.80:0/3365861185 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45240b0070 con 0x7f45300ce2a0 2024-04-25T13:44:25.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452dd9b640 1 -- 172.21.15.80:0/3365861185 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4524004230 con 0x7f45300ce2a0 2024-04-25T13:44:25.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f452dd9b640 1 -- 172.21.15.80:0/3365861185 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4524003620 con 0x7f45300ce2a0 2024-04-25T13:44:25.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f453506e640 1 -- 172.21.15.80:0/3365861185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300ce2a0 msgr2=0x7f45300ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:25.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f453506e640 1 --2- 172.21.15.80:0/3365861185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300ce2a0 0x7f45300ce6b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f45240097c0 tx=0x7f452401c460 comp rx=0 tx=0).stop 2024-04-25T13:44:25.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f453506e640 1 -- 172.21.15.80:0/3365861185 shutdown_connections 2024-04-25T13:44:25.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f453506e640 1 --2- 172.21.15.80:0/3365861185 >> v2:172.21.15.87:3300/0 conn(0x7f4530161d10 0x7f4530162120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f453506e640 1 --2- 172.21.15.80:0/3365861185 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300ce2a0 0x7f45300ce6b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f453506e640 1 -- 172.21.15.80:0/3365861185 >> 172.21.15.80:0/3365861185 conn(0x7f45300b35c0 msgr2=0x7f45300b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:25.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f453506e640 1 -- 172.21.15.80:0/3365861185 shutdown_connections 2024-04-25T13:44:25.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.676+0000 7f453506e640 1 -- 172.21.15.80:0/3365861185 wait complete. 2024-04-25T13:44:25.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 Processor -- start 2024-04-25T13:44:25.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 -- start start 2024-04-25T13:44:25.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300b0f10 0x7f4530156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:25.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45300ce2a0 0x7f453015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:25.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4530161d10 0x7f4530155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:25.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f453015e720 con 0x7f45300b0f10 2024-04-25T13:44:25.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f45300b5300 con 0x7f45300ce2a0 2024-04-25T13:44:25.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4530162a70 con 0x7f4530161d10 2024-04-25T13:44:25.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300b0f10 0x7f4530156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:25.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300b0f10 0x7f4530156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60646/0 (socket says 172.21.15.80:60646) 2024-04-25T13:44:25.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 -- 172.21.15.80:0/1812525152 learned_addr learned my addr 172.21.15.80:0/1812525152 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:25.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452e59c640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45300ce2a0 0x7f453015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:25.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452f59e640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4530161d10 0x7f4530155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:25.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 -- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4530161d10 msgr2=0x7f4530155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:25.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4530161d10 0x7f4530155320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 -- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45300ce2a0 msgr2=0x7f453015b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:25.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45300ce2a0 0x7f453015b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:25.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 -- 172.21.15.80:0/1812525152 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45301599a0 con 0x7f45300b0f10 2024-04-25T13:44:25.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452e59c640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45300ce2a0 0x7f453015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:25.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f452ed9d640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300b0f10 0x7f4530156d20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f452000c3e0 tx=0x7f452000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:25.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f45177fe640 1 -- 172.21.15.80:0/1812525152 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45200074e0 con 0x7f45300b0f10 2024-04-25T13:44:25.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f45177fe640 1 -- 172.21.15.80:0/1812525152 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45200047f0 con 0x7f45300b0f10 2024-04-25T13:44:25.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4530154b20 con 0x7f45300b0f10 2024-04-25T13:44:25.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f45177fe640 1 -- 172.21.15.80:0/1812525152 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4520004be0 con 0x7f45300b0f10 2024-04-25T13:44:25.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.680+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45301566f0 con 0x7f45300b0f10 2024-04-25T13:44:25.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.684+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f453006a440 con 0x7f45300b0f10 2024-04-25T13:44:25.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.684+0000 7f45177fe640 1 -- 172.21.15.80:0/1812525152 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4520011070 con 0x7f45300b0f10 2024-04-25T13:44:25.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.684+0000 7f45177fe640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f451c06f450 0x7f451c071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:25.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.684+0000 7f45177fe640 1 -- 172.21.15.80:0/1812525152 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5357+0+0 (secure 0 0 0) 0x7f452008ecb0 con 0x7f45300b0f10 2024-04-25T13:44:25.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.684+0000 7f452e59c640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f451c06f450 0x7f451c071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:25.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.688+0000 7f45177fe640 1 -- 172.21.15.80:0/1812525152 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f45300b1320 con 0x7f45300b0f10 2024-04-25T13:44:25.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.688+0000 7f452e59c640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f451c06f450 0x7f451c071960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f45240ad000 tx=0x7f4524008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:25.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:25.840+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f45300d0f90 con 0x7f45300b0f10 2024-04-25T13:44:26.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.500+0000 7f45177fe640 1 -- 172.21.15.80:0/1812525152 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) ==== 169+0+0 (secure 0 0 0) 0x7f452005bde0 con 0x7f45300b0f10 2024-04-25T13:44:26.498 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:44:26.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f451c06f450 msgr2=0x7f451c071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:26.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f451c06f450 0x7f451c071960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f45240ad000 tx=0x7f4524008040 comp rx=0 tx=0).stop 2024-04-25T13:44:26.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300b0f10 msgr2=0x7f4530156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:26.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300b0f10 0x7f4530156d20 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f452000c3e0 tx=0x7f452000c8a0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 shutdown_connections 2024-04-25T13:44:26.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f451c06f450 0x7f451c071960 secure :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f45240ad000 tx=0x7f4524008040 comp rx=0 tx=0).stop 2024-04-25T13:44:26.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4530161d10 0x7f4530155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f45300ce2a0 0x7f453015b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 --2- 172.21.15.80:0/1812525152 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45300b0f10 0x7f4530156d20 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 >> 172.21.15.80:0/1812525152 conn(0x7f45300b35c0 msgr2=0x7f4530152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:26.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 shutdown_connections 2024-04-25T13:44:26.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.504+0000 7f453506e640 1 -- 172.21.15.80:0/1812525152 wait complete. 2024-04-25T13:44:26.526 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:44:26.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.680+0000 7fb1d6709640 1 Processor -- start 2024-04-25T13:44:26.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.680+0000 7fb1d6709640 1 -- start start 2024-04-25T13:44:26.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d6709640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d006edb0 0x7fb1d006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:26.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d6709640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1d0161b60 0x7fb1d0161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:26.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d6709640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1d01624b0 con 0x7fb1d00b5330 2024-04-25T13:44:26.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d6709640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb1d006f700 con 0x7fb1d0161b60 2024-04-25T13:44:26.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d6709640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d01629b0 con 0x7fb1d006edb0 2024-04-25T13:44:26.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d4ca6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1d0161b60 0x7fb1d0161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d4ca6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb1d0161b60 0x7fb1d0161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60652/0 (socket says 172.21.15.80:60652) 2024-04-25T13:44:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d4ca6640 1 -- 172.21.15.80:0/3838349891 learned_addr learned my addr 172.21.15.80:0/3838349891 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1cf7fe640 1 --2- 172.21.15.80:0/3838349891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d006edb0 0x7fb1d006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d4ca6640 1 -- 172.21.15.80:0/3838349891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d006edb0 msgr2=0x7fb1d006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d4ca6640 1 --2- 172.21.15.80:0/3838349891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d006edb0 0x7fb1d006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d4ca6640 1 -- 172.21.15.80:0/3838349891 >> v1:172.21.15.87:6789/0 conn(0x7fb1d00b5330 legacy=0x7fb1d00d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:26.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d4ca6640 1 -- 172.21.15.80:0/3838349891 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d00b5740 con 0x7fb1d0161b60 2024-04-25T13:44:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.684+0000 7fb1d4ca6640 1 --2- 172.21.15.80:0/3838349891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d0161b60 0x7fb1d0161f70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009e50 tx=0x7fb1c801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e1b059e87bbe7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1ceffd640 1 -- 172.21.15.80:0/3838349891 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80a5830 con 0x7fb1d0161b60 2024-04-25T13:44:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1ceffd640 1 -- 172.21.15.80:0/3838349891 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c80a59d0 con 0x7fb1d0161b60 2024-04-25T13:44:26.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1ceffd640 1 -- 172.21.15.80:0/3838349891 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80a5da0 con 0x7fb1d0161b60 2024-04-25T13:44:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 -- 172.21.15.80:0/3838349891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d0161b60 msgr2=0x7fb1d0161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:26.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 --2- 172.21.15.80:0/3838349891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d0161b60 0x7fb1d0161f70 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009e50 tx=0x7fb1c801c970 comp rx=0 tx=0).stop 2024-04-25T13:44:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 -- 172.21.15.80:0/3838349891 shutdown_connections 2024-04-25T13:44:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 --2- 172.21.15.80:0/3838349891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d0161b60 0x7fb1d0161f70 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 --2- 172.21.15.80:0/3838349891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d006edb0 0x7fb1d006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 -- 172.21.15.80:0/3838349891 >> 172.21.15.80:0/3838349891 conn(0x7fb1d00b4160 msgr2=0x7fb1d00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 -- 172.21.15.80:0/3838349891 shutdown_connections 2024-04-25T13:44:26.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 -- 172.21.15.80:0/3838349891 wait complete. 2024-04-25T13:44:26.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 Processor -- start 2024-04-25T13:44:26.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.688+0000 7fb1d6709640 1 -- start start 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d006edb0 0x7fb1d013b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d00b5330 0x7fb1d0140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1d0161b60 0x7fb1d0140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d006e890 con 0x7fb1d006edb0 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d006e5b0 con 0x7fb1d0161b60 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1d00b0cd0 con 0x7fb1d00b5330 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d006edb0 0x7fb1d013b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d006edb0 0x7fb1d013b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60656/0 (socket says 172.21.15.80:60656) 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 -- 172.21.15.80:0/2188418570 learned_addr learned my addr 172.21.15.80:0/2188418570 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:26.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d4ca6640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1d0161b60 0x7fb1d0140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cf7fe640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d00b5330 0x7fb1d0140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 -- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d00b5330 msgr2=0x7fb1d0140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d00b5330 0x7fb1d0140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 -- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1d0161b60 msgr2=0x7fb1d0140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1d0161b60 0x7fb1d0140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 -- 172.21.15.80:0/2188418570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d013b7b0 con 0x7fb1d006edb0 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1cffff640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d006edb0 0x7fb1d013b270 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000c3a0 tx=0x7fb1c000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1ccff9640 1 -- 172.21.15.80:0/2188418570 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00052d0 con 0x7fb1d006edb0 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1ccff9640 1 -- 172.21.15.80:0/2188418570 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0005470 con 0x7fb1d006edb0 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1ccff9640 1 -- 172.21.15.80:0/2188418570 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0007260 con 0x7fb1d006edb0 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d0159410 con 0x7fb1d006edb0 2024-04-25T13:44:26.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d01599d0 con 0x7fb1d006edb0 2024-04-25T13:44:26.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.692+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d006a3a0 con 0x7fb1d006edb0 2024-04-25T13:44:26.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.696+0000 7fb1ccff9640 1 -- 172.21.15.80:0/2188418570 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb1c0007400 con 0x7fb1d006edb0 2024-04-25T13:44:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.696+0000 7fb1ccff9640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1bc06f430 0x7fb1bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:26.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.696+0000 7fb1ccff9640 1 -- 172.21.15.80:0/2188418570 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fb1c008e400 con 0x7fb1d006edb0 2024-04-25T13:44:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.700+0000 7fb1ccff9640 1 -- 172.21.15.80:0/2188418570 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb1d01624b0 con 0x7fb1d006edb0 2024-04-25T13:44:26.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.700+0000 7fb1cf7fe640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1bc06f430 0x7fb1bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:26.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.700+0000 7fb1cf7fe640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1bc06f430 0x7fb1bc071940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0140e70 tx=0x7fb1c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:26.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.848+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fb1d00b4600 con 0x7fb1d006edb0 2024-04-25T13:44:26.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1ccff9640 1 -- 172.21.15.80:0/2188418570 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) ==== 104+0+0 (secure 0 0 0) 0x7fb1c005b680 con 0x7fb1d006edb0 2024-04-25T13:44:26.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1bc06f430 msgr2=0x7fb1bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:26.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1bc06f430 0x7fb1bc071940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb1d0140e70 tx=0x7fb1c4008040 comp rx=0 tx=0).stop 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d006edb0 msgr2=0x7fb1d013b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d006edb0 0x7fb1d013b270 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000c3a0 tx=0x7fb1c000c860 comp rx=0 tx=0).stop 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 shutdown_connections 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1bc06f430 0x7fb1bc071940 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1d0161b60 0x7fb1d0140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1d00b5330 0x7fb1d0140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 --2- 172.21.15.80:0/2188418570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1d006edb0 0x7fb1d013b270 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 >> 172.21.15.80:0/2188418570 conn(0x7fb1d00b4160 msgr2=0x7fb1d006cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.856+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 shutdown_connections 2024-04-25T13:44:26.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:26.860+0000 7fb1d6709640 1 -- 172.21.15.80:0/2188418570 wait complete. 2024-04-25T13:44:26.876 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2024-04-25T13:44:27.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.020+0000 7f2253a05640 1 Processor -- start 2024-04-25T13:44:27.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.020+0000 7f2253a05640 1 -- start start 2024-04-25T13:44:27.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.020+0000 7f2253a05640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f224c0d06e0 0x7f224c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.020+0000 7f2253a05640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c06e9e0 0x7f224c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.020+0000 7f2253a05640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f224c0d0af0 con 0x7f224c161f90 2024-04-25T13:44:27.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.020+0000 7f2253a05640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f224c0d0d70 con 0x7f224c0d06e0 2024-04-25T13:44:27.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.020+0000 7f2253a05640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f224c0d1050 con 0x7f224c06e9e0 2024-04-25T13:44:27.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2250fa0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f224c0d06e0 0x7f224c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2251fa2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c06e9e0 0x7f224c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2251fa2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c06e9e0 0x7f224c06edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:55474/0 (socket says 172.21.15.80:55474) 2024-04-25T13:44:27.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2250fa0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f224c0d06e0 0x7f224c06e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:52232/0 (socket says 172.21.15.80:52232) 2024-04-25T13:44:27.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2251fa2640 1 -- 172.21.15.80:0/1755573331 learned_addr learned my addr 172.21.15.80:0/1755573331 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:27.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f22517a1640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f224c161f90 0x7f224c0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:36500/0 (socket says 172.21.15.80:36500) 2024-04-25T13:44:27.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2251fa2640 1 -- 172.21.15.80:0/1755573331 >> v2:172.21.15.87:3300/0 conn(0x7f224c0d06e0 msgr2=0x7f224c06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2251fa2640 1 --2- 172.21.15.80:0/1755573331 >> v2:172.21.15.87:3300/0 conn(0x7f224c0d06e0 0x7f224c06e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2251fa2640 1 -- 172.21.15.80:0/1755573331 >> v1:172.21.15.87:6789/0 conn(0x7f224c161f90 legacy=0x7f224c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2251fa2640 1 -- 172.21.15.80:0/1755573331 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c06dca0 con 0x7f224c06e9e0 2024-04-25T13:44:27.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2250fa0640 1 --2- 172.21.15.80:0/1755573331 >> v2:172.21.15.87:3300/0 conn(0x7f224c0d06e0 0x7f224c06e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:44:27.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2251fa2640 1 --2- 172.21.15.80:0/1755573331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c06e9e0 0x7f224c06edf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2244009e50 tx=0x7f224401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ed7bdddd085dd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2243fff640 1 -- 172.21.15.80:0/1755573331 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440a5740 con 0x7f224c06e9e0 2024-04-25T13:44:27.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2243fff640 1 -- 172.21.15.80:0/1755573331 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22440a58e0 con 0x7f224c06e9e0 2024-04-25T13:44:27.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.024+0000 7f2243fff640 1 -- 172.21.15.80:0/1755573331 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440a5cb0 con 0x7f224c06e9e0 2024-04-25T13:44:27.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- 172.21.15.80:0/1755573331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c06e9e0 msgr2=0x7f224c06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 --2- 172.21.15.80:0/1755573331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c06e9e0 0x7f224c06edf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2244009e50 tx=0x7f224401c970 comp rx=0 tx=0).stop 2024-04-25T13:44:27.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- 172.21.15.80:0/1755573331 shutdown_connections 2024-04-25T13:44:27.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 --2- 172.21.15.80:0/1755573331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c06e9e0 0x7f224c06edf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 --2- 172.21.15.80:0/1755573331 >> v2:172.21.15.87:3300/0 conn(0x7f224c0d06e0 0x7f224c06e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- 172.21.15.80:0/1755573331 >> 172.21.15.80:0/1755573331 conn(0x7f224c06d080 msgr2=0x7f224c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- 172.21.15.80:0/1755573331 shutdown_connections 2024-04-25T13:44:27.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- 172.21.15.80:0/1755573331 wait complete. 2024-04-25T13:44:27.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 Processor -- start 2024-04-25T13:44:27.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- start start 2024-04-25T13:44:27.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f224c06e9e0 0x7f224c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c0d06e0 0x7f224c1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f224c161f90 0x7f224c1675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f224c1447a0 con 0x7f224c161f90 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f224c144620 con 0x7f224c06e9e0 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2253a05640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f224c16b510 con 0x7f224c0d06e0 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2251fa2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f224c161f90 0x7f224c1675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2251fa2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f224c161f90 0x7f224c1675f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60660/0 (socket says 172.21.15.80:60660) 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.028+0000 7f2251fa2640 1 -- 172.21.15.80:0/370673755 learned_addr learned my addr 172.21.15.80:0/370673755 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2250fa0640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c0d06e0 0x7f224c1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f22517a1640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f224c06e9e0 0x7f224c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2251fa2640 1 -- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c0d06e0 msgr2=0x7f224c1670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2251fa2640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c0d06e0 0x7f224c1670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2251fa2640 1 -- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f224c06e9e0 msgr2=0x7f224c15dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2251fa2640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f224c06e9e0 0x7f224c15dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2251fa2640 1 -- 172.21.15.80:0/370673755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f224c146580 con 0x7f224c161f90 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2251fa2640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f224c161f90 0x7f224c1675f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2244009e50 tx=0x7f2244019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2241ffb640 1 -- 172.21.15.80:0/370673755 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440b8470 con 0x7f224c161f90 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2241ffb640 1 -- 172.21.15.80:0/370673755 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22440b8610 con 0x7f224c161f90 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f224c146860 con 0x7f224c161f90 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2241ffb640 1 -- 172.21.15.80:0/370673755 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22440b8a00 con 0x7f224c161f90 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f224c146e20 con 0x7f224c161f90 2024-04-25T13:44:27.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f224c06a3a0 con 0x7f224c161f90 2024-04-25T13:44:27.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2241ffb640 1 -- 172.21.15.80:0/370673755 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22440c0020 con 0x7f224c161f90 2024-04-25T13:44:27.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2241ffb640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f223006f430 0x7f2230071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f2241ffb640 1 -- 172.21.15.80:0/370673755 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7f224412a3b0 con 0x7f224c161f90 2024-04-25T13:44:27.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.032+0000 7f22517a1640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f223006f430 0x7f2230071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.036+0000 7f22517a1640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f223006f430 0x7f2230071940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2248000f70 tx=0x7f2248007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.036+0000 7f2241ffb640 1 -- 172.21.15.80:0/370673755 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22440c7020 con 0x7f224c161f90 2024-04-25T13:44:27.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.260+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) -- 0x7f224c0cda60 con 0x7f224c161f90 2024-04-25T13:44:27.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.500+0000 7f2241ffb640 1 -- 172.21.15.80:0/370673755 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v215) ==== 134+0+0 (secure 0 0 0) 0x7f2244016620 con 0x7f224c161f90 2024-04-25T13:44:27.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f223006f430 msgr2=0x7f2230071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f223006f430 0x7f2230071940 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2248000f70 tx=0x7f2248007e90 comp rx=0 tx=0).stop 2024-04-25T13:44:27.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f224c161f90 msgr2=0x7f224c1675f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f224c161f90 0x7f224c1675f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2244009e50 tx=0x7f2244019710 comp rx=0 tx=0).stop 2024-04-25T13:44:27.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 shutdown_connections 2024-04-25T13:44:27.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f223006f430 0x7f2230071940 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f224c161f90 0x7f224c1675f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f224c0d06e0 0x7f224c1670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 --2- 172.21.15.80:0/370673755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f224c06e9e0 0x7f224c15dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 >> 172.21.15.80:0/370673755 conn(0x7f224c06d080 msgr2=0x7f224c0cff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 shutdown_connections 2024-04-25T13:44:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.504+0000 7f2253a05640 1 -- 172.21.15.80:0/370673755 wait complete. 2024-04-25T13:44:27.528 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-04-25T13:44:27.529 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2024-04-25T13:44:27.529 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2024-04-25T13:44:27.529 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:27.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.680+0000 7fb30bee6640 1 Processor -- start 2024-04-25T13:44:27.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30bee6640 1 -- start start 2024-04-25T13:44:27.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30bee6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3040d59b0 0x7fb3040d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30bee6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3040d4330 0x7fb3040d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30bee6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3040d6300 con 0x7fb3040d2e80 2024-04-25T13:44:27.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30bee6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb304161060 con 0x7fb3040d4330 2024-04-25T13:44:27.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30bee6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb304161340 con 0x7fb3040d59b0 2024-04-25T13:44:27.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb309c82640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb3040d2e80 0x7fb3041607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55892/0 (socket says 172.21.15.80:55892) 2024-04-25T13:44:27.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb309c82640 1 -- 172.21.15.80:0/992155730 learned_addr learned my addr 172.21.15.80:0/992155730 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:27.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb309481640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3040d59b0 0x7fb3040d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30a483640 1 --2- 172.21.15.80:0/992155730 >> v2:172.21.15.87:3300/0 conn(0x7fb3040d4330 0x7fb3040d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 23812923 0 0) 0x7fb300001120 con 0x7fb3040d2e80 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb2f0003530 con 0x7fb3040d2e80 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173562686 0 0) 0x7fb2f0003530 con 0x7fb3040d2e80 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3040d59b0 msgr2=0x7fb3040d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3040d59b0 0x7fb3040d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 >> v2:172.21.15.87:3300/0 conn(0x7fb3040d4330 msgr2=0x7fb3040d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 --2- 172.21.15.80:0/992155730 >> v2:172.21.15.87:3300/0 conn(0x7fb3040d4330 0x7fb3040d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3040d2680 con 0x7fb3040d2e80 2024-04-25T13:44:27.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb300001850 con 0x7fb3040d2e80 2024-04-25T13:44:27.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 >> v1:172.21.15.80:6789/0 conn(0x7fb3040d2e80 legacy=0x7fb3041607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f00039c0 0x7fb2f0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3040d59b0 0x7fb2f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2f0004d50 0x7fb2f0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb304161060 con 0x7fb2f0004d50 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb304161340 con 0x7fb2f00039c0 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f000b9d0 con 0x7fb3040d59b0 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30a483640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2f0004d50 0x7fb2f0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb309481640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3040d59b0 0x7fb2f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb309c82640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f00039c0 0x7fb2f0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30a483640 1 -- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3040d59b0 msgr2=0x7fb2f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30a483640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3040d59b0 0x7fb2f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30a483640 1 -- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f00039c0 msgr2=0x7fb2f0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30a483640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f00039c0 0x7fb2f0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb30a483640 1 -- 172.21.15.80:0/992155730 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f0003530 con 0x7fb2f0004d50 2024-04-25T13:44:27.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.684+0000 7fb309c82640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2f00039c0 0x7fb2f0003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:27.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb30a483640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2f0004d50 0x7fb2f0005160 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009fc0 tx=0x7fb2fc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fbf195f3755693 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc0b2070 con 0x7fb2f0004d50 2024-04-25T13:44:27.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2fc0a5c90 con 0x7fb2f0004d50 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb308c80640 1 -- 172.21.15.80:0/992155730 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc009c10 con 0x7fb2f0004d50 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb30bee6640 1 -- 172.21.15.80:0/992155730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2f0004d50 msgr2=0x7fb2f0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb30bee6640 1 --2- 172.21.15.80:0/992155730 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2f0004d50 0x7fb2f0005160 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009fc0 tx=0x7fb2fc01cc80 comp rx=0 tx=0).stop 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb30a483640 1 -- 172.21.15.80:0/992155730 reap_dead start 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb30bee6640 1 -- 172.21.15.80:0/992155730 shutdown_connections 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb30bee6640 1 -- 172.21.15.80:0/992155730 >> 172.21.15.80:0/992155730 conn(0x7fb3040737a0 msgr2=0x7fb30406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb30bee6640 1 -- 172.21.15.80:0/992155730 shutdown_connections 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.688+0000 7fb30bee6640 1 -- 172.21.15.80:0/992155730 wait complete. 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 Processor -- start 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 -- start start 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3041f6c30 0x7fb30415b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb30415bb50 0x7fb3040c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3040c9c80 0x7fb30414bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb3040d4c80 con 0x7fb3041f6c30 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb304161620 con 0x7fb3040c9c80 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3040d4e00 con 0x7fb30415bb50 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3041f6c30 0x7fb30415b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3041f6c30 0x7fb30415b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60690/0 (socket says 172.21.15.80:60690) 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 -- 172.21.15.80:0/3747490262 learned_addr learned my addr 172.21.15.80:0/3747490262 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309481640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb30415bb50 0x7fb3040c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30a483640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3040c9c80 0x7fb30414bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 -- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb30415bb50 msgr2=0x7fb3040c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb30415bb50 0x7fb3040c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 -- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3040c9c80 msgr2=0x7fb30414bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3040c9c80 0x7fb30414bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 -- 172.21.15.80:0/3747490262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30415bf60 con 0x7fb3041f6c30 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309481640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb30415bb50 0x7fb3040c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:27.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb309c82640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3041f6c30 0x7fb30415b610 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb30000dae0 tx=0x7fb300001880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb2fa7fc640 1 -- 172.21.15.80:0/3747490262 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300019070 con 0x7fb3041f6c30 2024-04-25T13:44:27.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb2fa7fc640 1 -- 172.21.15.80:0/3747490262 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb300004cc0 con 0x7fb3041f6c30 2024-04-25T13:44:27.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3040ca400 con 0x7fb3041f6c30 2024-04-25T13:44:27.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb2fa7fc640 1 -- 172.21.15.80:0/3747490262 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3000072f0 con 0x7fb3041f6c30 2024-04-25T13:44:27.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb304149a30 con 0x7fb3041f6c30 2024-04-25T13:44:27.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.692+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30406a3a0 con 0x7fb3041f6c30 2024-04-25T13:44:27.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.696+0000 7fb2fa7fc640 1 -- 172.21.15.80:0/3747490262 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb30001c070 con 0x7fb3041f6c30 2024-04-25T13:44:27.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.696+0000 7fb2fa7fc640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb2e806f400 0x7fb2e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:27.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.696+0000 7fb2fa7fc640 1 -- 172.21.15.80:0/3747490262 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fb30008eb20 con 0x7fb3041f6c30 2024-04-25T13:44:27.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.696+0000 7fb309481640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb2e806f400 0x7fb2e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:27.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.696+0000 7fb309481640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb2e806f400 0x7fb2e8071910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009a20 tx=0x7fb2f4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:27.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.700+0000 7fb2fa7fc640 1 -- 172.21.15.80:0/3747490262 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb30415bf60 con 0x7fb3041f6c30 2024-04-25T13:44:27.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.872+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3040d4a60 con 0x7fb3041f6c30 2024-04-25T13:44:27.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.872+0000 7fb2fa7fc640 1 -- 172.21.15.80:0/3747490262 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+12948 (secure 0 0 0) 0x7fb3040d6300 con 0x7fb3041f6c30 2024-04-25T13:44:27.871 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:27.871 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":73,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:44:26.490936+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6834/1437895704":"2024-04-26T13:44:23.462767+0000","172.21.15.80:6832/1437895704":"2024-04-26T13:44:23.462767+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:27.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb2e806f400 msgr2=0x7fb2e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb2e806f400 0x7fb2e8071910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4009a20 tx=0x7fb2f4008040 comp rx=0 tx=0).stop 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3041f6c30 msgr2=0x7fb30415b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3041f6c30 0x7fb30415b610 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fb30000dae0 tx=0x7fb300001880 comp rx=0 tx=0).stop 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 shutdown_connections 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb2e806f400 0x7fb2e8071910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3040c9c80 0x7fb30414bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb30415bb50 0x7fb3040c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 --2- 172.21.15.80:0/3747490262 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3041f6c30 0x7fb30415b610 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 >> 172.21.15.80:0/3747490262 conn(0x7fb3040737a0 msgr2=0x7fb30415f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 shutdown_connections 2024-04-25T13:44:27.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:27.876+0000 7fb30bee6640 1 -- 172.21.15.80:0/3747490262 wait complete. 2024-04-25T13:44:27.893 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:44:27.894 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:44:28.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc37b109640 1 Processor -- start 2024-04-25T13:44:28.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc37b109640 1 -- start start 2024-04-25T13:44:28.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc37b109640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc374071e60 0x7fc3741574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc37b109640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc374161170 0x7fc3740b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc37b109640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc37406fbf0 con 0x7fc374073c40 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc37b109640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc3740b1350 con 0x7fc374071e60 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc37b109640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc37406f9a0 con 0x7fc374161170 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc378ea5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc374071e60 0x7fc3741574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc373fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc374073c40 0x7fc374160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55904/0 (socket says 172.21.15.80:55904) 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.052+0000 7fc373fff640 1 -- 172.21.15.80:0/2574680156 learned_addr learned my addr 172.21.15.80:0/2574680156 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc3796a6640 1 --2- 172.21.15.80:0/2574680156 >> v2:172.21.15.87:3300/0 conn(0x7fc374161170 0x7fc3740b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc378ea5640 1 -- 172.21.15.80:0/2574680156 >> v2:172.21.15.87:3300/0 conn(0x7fc374161170 msgr2=0x7fc3740b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc378ea5640 1 --2- 172.21.15.80:0/2574680156 >> v2:172.21.15.87:3300/0 conn(0x7fc374161170 0x7fc3740b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc378ea5640 1 -- 172.21.15.80:0/2574680156 >> v1:172.21.15.80:6789/0 conn(0x7fc374073c40 legacy=0x7fc374160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc378ea5640 1 -- 172.21.15.80:0/2574680156 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3740b0a20 con 0x7fc374071e60 2024-04-25T13:44:28.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc378ea5640 1 --2- 172.21.15.80:0/2574680156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374071e60 0x7fc3741574c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc374161a50 tx=0x7fc36401c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43d3fe9b16ca1d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:28.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc3737fe640 1 -- 172.21.15.80:0/2574680156 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3640b0070 con 0x7fc374071e60 2024-04-25T13:44:28.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc3737fe640 1 -- 172.21.15.80:0/2574680156 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3640ab600 con 0x7fc374071e60 2024-04-25T13:44:28.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc3737fe640 1 -- 172.21.15.80:0/2574680156 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3640ab9d0 con 0x7fc374071e60 2024-04-25T13:44:28.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc37b109640 1 -- 172.21.15.80:0/2574680156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374071e60 msgr2=0x7fc3741574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc37b109640 1 --2- 172.21.15.80:0/2574680156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374071e60 0x7fc3741574c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc374161a50 tx=0x7fc36401c310 comp rx=0 tx=0).stop 2024-04-25T13:44:28.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc37b109640 1 -- 172.21.15.80:0/2574680156 shutdown_connections 2024-04-25T13:44:28.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc37b109640 1 --2- 172.21.15.80:0/2574680156 >> v2:172.21.15.87:3300/0 conn(0x7fc374161170 0x7fc3740b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc37b109640 1 --2- 172.21.15.80:0/2574680156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374071e60 0x7fc3741574c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc37b109640 1 -- 172.21.15.80:0/2574680156 >> 172.21.15.80:0/2574680156 conn(0x7fc3740b4160 msgr2=0x7fc3740d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:28.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc37b109640 1 -- 172.21.15.80:0/2574680156 shutdown_connections 2024-04-25T13:44:28.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.056+0000 7fc37b109640 1 -- 172.21.15.80:0/2574680156 wait complete. 2024-04-25T13:44:28.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 Processor -- start 2024-04-25T13:44:28.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 -- start start 2024-04-25T13:44:28.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc374071e60 0x7fc3741e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374073c40 0x7fc3741f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3741426c0 0x7fc374142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc374072270 con 0x7fc374073c40 2024-04-25T13:44:28.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc3740b0fe0 con 0x7fc3741426c0 2024-04-25T13:44:28.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc374074050 con 0x7fc374071e60 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374073c40 0x7fc3741f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374073c40 0x7fc3741f4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60702/0 (socket says 172.21.15.80:60702) 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 -- 172.21.15.80:0/2673754374 learned_addr learned my addr 172.21.15.80:0/2673754374 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc378ea5640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc374071e60 0x7fc3741e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc3796a6640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3741426c0 0x7fc374142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 -- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc374071e60 msgr2=0x7fc3741e7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc374071e60 0x7fc3741e7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 -- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3741426c0 msgr2=0x7fc374142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3741426c0 0x7fc374142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 -- 172.21.15.80:0/2673754374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3740ac480 con 0x7fc374073c40 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc373fff640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374073c40 0x7fc3741f4b90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc368009e50 tx=0x7fc36800c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc3717fa640 1 -- 172.21.15.80:0/2673754374 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368019070 con 0x7fc374073c40 2024-04-25T13:44:28.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc3717fa640 1 -- 172.21.15.80:0/2673754374 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3680078f0 con 0x7fc374073c40 2024-04-25T13:44:28.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3741f50d0 con 0x7fc374073c40 2024-04-25T13:44:28.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc3717fa640 1 -- 172.21.15.80:0/2673754374 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc368007ce0 con 0x7fc374073c40 2024-04-25T13:44:28.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.060+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3741e7020 con 0x7fc374073c40 2024-04-25T13:44:28.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.064+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc37406a3a0 con 0x7fc374073c40 2024-04-25T13:44:28.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.064+0000 7fc3717fa640 1 -- 172.21.15.80:0/2673754374 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc36801c070 con 0x7fc374073c40 2024-04-25T13:44:28.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.064+0000 7fc3717fa640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc36006f430 0x7fc360071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.064+0000 7fc3717fa640 1 -- 172.21.15.80:0/2673754374 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fc368090b20 con 0x7fc374073c40 2024-04-25T13:44:28.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.064+0000 7fc378ea5640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc36006f430 0x7fc360071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.064+0000 7fc378ea5640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc36006f430 0x7fc360071940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc3640ad000 tx=0x7fc364008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:28.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.068+0000 7fc3717fa640 1 -- 172.21.15.80:0/2673754374 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc3741f50d0 con 0x7fc374073c40 2024-04-25T13:44:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc37415e0c0 con 0x7fc374073c40 2024-04-25T13:44:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc3717fa640 1 -- 172.21.15.80:0/2673754374 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fc3740ac480 con 0x7fc374073c40 2024-04-25T13:44:28.230 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:44:28.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc36006f430 msgr2=0x7fc360071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc36006f430 0x7fc360071940 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc3640ad000 tx=0x7fc364008040 comp rx=0 tx=0).stop 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374073c40 msgr2=0x7fc3741f4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374073c40 0x7fc3741f4b90 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc368009e50 tx=0x7fc36800c430 comp rx=0 tx=0).stop 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 shutdown_connections 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc36006f430 0x7fc360071940 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3741426c0 0x7fc374142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc374073c40 0x7fc3741f4b90 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 --2- 172.21.15.80:0/2673754374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc374071e60 0x7fc3741e7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 >> 172.21.15.80:0/2673754374 conn(0x7fc3740b4160 msgr2=0x7fc37406e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 shutdown_connections 2024-04-25T13:44:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.232+0000 7fc37b109640 1 -- 172.21.15.80:0/2673754374 wait complete. 2024-04-25T13:44:28.252 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-04-25T13:44:28.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f451d95e640 1 Processor -- start 2024-04-25T13:44:28.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f451d95e640 1 -- start start 2024-04-25T13:44:28.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f451d95e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4518161f90 0x7f45180cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f451d95e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45180d0510 0x7f451815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f451d95e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 0x7f451815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f451d95e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f45180d0d70 con 0x7f45180d0510 2024-04-25T13:44:28.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f451d95e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f45180d1050 con 0x7f4518161f90 2024-04-25T13:44:28.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f451d95e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f451806dca0 con 0x7f451815ea70 2024-04-25T13:44:28.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f450e7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45180d0510 0x7f451815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f450e7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45180d0510 0x7f451815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60716/0 (socket says 172.21.15.80:60716) 2024-04-25T13:44:28.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.396+0000 7f450e7fc640 1 -- 172.21.15.80:0/2111594216 learned_addr learned my addr 172.21.15.80:0/2111594216 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:28.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f45177fe640 1 --2- 172.21.15.80:0/2111594216 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 0x7f451815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f4516ffd640 1 --2- 172.21.15.80:0/2111594216 >> v2:172.21.15.87:3300/0 conn(0x7f4518161f90 0x7f45180cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f450e7fc640 1 -- 172.21.15.80:0/2111594216 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 msgr2=0x7f451815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:28.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f450e7fc640 1 --2- 172.21.15.80:0/2111594216 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 0x7f451815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f450e7fc640 1 -- 172.21.15.80:0/2111594216 >> v2:172.21.15.87:3300/0 conn(0x7f4518161f90 msgr2=0x7f45180cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:28.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f450e7fc640 1 --2- 172.21.15.80:0/2111594216 >> v2:172.21.15.87:3300/0 conn(0x7f4518161f90 0x7f45180cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f450e7fc640 1 -- 172.21.15.80:0/2111594216 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451806df80 con 0x7f45180d0510 2024-04-25T13:44:28.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f4516ffd640 1 --2- 172.21.15.80:0/2111594216 >> v2:172.21.15.87:3300/0 conn(0x7f4518161f90 0x7f45180cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:28.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f450e7fc640 1 --2- 172.21.15.80:0/2111594216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f451815e530 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f45180d17b0 tx=0x7f450001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45315c04425cf96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:28.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f45167fc640 1 -- 172.21.15.80:0/2111594216 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45000b1070 con 0x7f45180d0510 2024-04-25T13:44:28.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f45167fc640 1 -- 172.21.15.80:0/2111594216 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45000a8050 con 0x7f45180d0510 2024-04-25T13:44:28.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f45167fc640 1 -- 172.21.15.80:0/2111594216 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45000ac860 con 0x7f45180d0510 2024-04-25T13:44:28.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 -- 172.21.15.80:0/2111594216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 msgr2=0x7f451815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 --2- 172.21.15.80:0/2111594216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f451815e530 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f45180d17b0 tx=0x7f450001c460 comp rx=0 tx=0).stop 2024-04-25T13:44:28.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 -- 172.21.15.80:0/2111594216 shutdown_connections 2024-04-25T13:44:28.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 --2- 172.21.15.80:0/2111594216 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 0x7f451815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 --2- 172.21.15.80:0/2111594216 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f451815e530 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 --2- 172.21.15.80:0/2111594216 >> v2:172.21.15.87:3300/0 conn(0x7f4518161f90 0x7f45180cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 -- 172.21.15.80:0/2111594216 >> 172.21.15.80:0/2111594216 conn(0x7f451806d080 msgr2=0x7f45180b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:28.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 -- 172.21.15.80:0/2111594216 shutdown_connections 2024-04-25T13:44:28.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.400+0000 7f451d95e640 1 -- 172.21.15.80:0/2111594216 wait complete. 2024-04-25T13:44:28.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 Processor -- start 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 -- start start 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f45181630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 0x7f4518144af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4518161f90 0x7f451813eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f45181447a0 con 0x7f45180d0510 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4518144620 con 0x7f4518161f90 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f45180d0920 con 0x7f451815ea70 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f45181630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f45181630c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60722/0 (socket says 172.21.15.80:60722) 2024-04-25T13:44:28.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 -- 172.21.15.80:0/3562745431 learned_addr learned my addr 172.21.15.80:0/3562745431 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f450e7fc640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 0x7f4518144af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f45177fe640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4518161f90 0x7f451813eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 -- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 msgr2=0x7f4518144af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 0x7f4518144af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 -- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4518161f90 msgr2=0x7f451813eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4518161f90 0x7f451813eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 -- 172.21.15.80:0/3562745431 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451813dc00 con 0x7f45180d0510 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4516ffd640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f45181630c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f450800c820 tx=0x7f450800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4514ff9640 1 -- 172.21.15.80:0/3562745431 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508015070 con 0x7f45180d0510 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4514ff9640 1 -- 172.21.15.80:0/3562745431 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4508007a30 con 0x7f45180d0510 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451813dee0 con 0x7f45180d0510 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f4514ff9640 1 -- 172.21.15.80:0/3562745431 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4508007e20 con 0x7f45180d0510 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.404+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f451813e4a0 con 0x7f45180d0510 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.408+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f451806a3a0 con 0x7f45180d0510 2024-04-25T13:44:28.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.408+0000 7f4514ff9640 1 -- 172.21.15.80:0/3562745431 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4508018070 con 0x7f45180d0510 2024-04-25T13:44:28.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.408+0000 7f4514ff9640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f806f400 0x7f44f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:28.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.408+0000 7f4514ff9640 1 -- 172.21.15.80:0/3562745431 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7f450808eeb0 con 0x7f45180d0510 2024-04-25T13:44:28.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.408+0000 7f450e7fc640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f806f400 0x7f44f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:28.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.408+0000 7f450e7fc640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f806f400 0x7f44f8071910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f45000ae000 tx=0x7f4500008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:28.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.412+0000 7f4514ff9640 1 -- 172.21.15.80:0/3562745431 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f450805c0a0 con 0x7f45180d0510 2024-04-25T13:44:28.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:28.632+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) -- 0x7f451815f9e0 con 0x7f45180d0510 2024-04-25T13:44:28.924 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:44:28.927 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:44:29.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.512+0000 7f4514ff9640 1 -- 172.21.15.80:0/3562745431 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v217) ==== 136+0+0 (secure 0 0 0) 0x7f450805b9a0 con 0x7f45180d0510 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f806f400 msgr2=0x7f44f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f806f400 0x7f44f8071910 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f45000ae000 tx=0x7f4500008040 comp rx=0 tx=0).stop 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 msgr2=0x7f45181630c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f45181630c0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f450800c820 tx=0x7f450800cce0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 shutdown_connections 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44f806f400 0x7f44f8071910 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4518161f90 0x7f451813eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f451815ea70 0x7f4518144af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 --2- 172.21.15.80:0/3562745431 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45180d0510 0x7f45181630c0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 >> 172.21.15.80:0/3562745431 conn(0x7f451806d080 msgr2=0x7f4518141180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 shutdown_connections 2024-04-25T13:44:29.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.516+0000 7f451d95e640 1 -- 172.21.15.80:0/3562745431 wait complete. 2024-04-25T13:44:29.541 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-25T13:44:29.706 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_objectstore":"memstore"} 2024-04-25T13:44:29.707 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-25T13:44:29.707 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:44:29.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa334bef640 1 Processor -- start 2024-04-25T13:44:29.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa334bef640 1 -- start start 2024-04-25T13:44:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa334bef640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa330071e60 0x7fa3301574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa334bef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330073c40 0x7fa330160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa334bef640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa330160fa0 0x7fa3300b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa334bef640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa3300b1350 con 0x7fa330160fa0 2024-04-25T13:44:29.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa334bef640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa33006f9a0 con 0x7fa330071e60 2024-04-25T13:44:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa334bef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa3301613b0 con 0x7fa330073c40 2024-04-25T13:44:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa330160fa0 0x7fa3300b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa330160fa0 0x7fa3300b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60728/0 (socket says 172.21.15.80:60728) 2024-04-25T13:44:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32ed9d640 1 -- 172.21.15.80:0/174599226 learned_addr learned my addr 172.21.15.80:0/174599226 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32dd9b640 1 --2- 172.21.15.80:0/174599226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330073c40 0x7fa330160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:29.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32e59c640 1 --2- 172.21.15.80:0/174599226 >> v2:172.21.15.87:3300/0 conn(0x7fa330071e60 0x7fa3301574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32ed9d640 1 -- 172.21.15.80:0/174599226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330073c40 msgr2=0x7fa330160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32ed9d640 1 --2- 172.21.15.80:0/174599226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330073c40 0x7fa330160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32ed9d640 1 -- 172.21.15.80:0/174599226 >> v2:172.21.15.87:3300/0 conn(0x7fa330071e60 msgr2=0x7fa3301574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32ed9d640 1 --2- 172.21.15.80:0/174599226 >> v2:172.21.15.87:3300/0 conn(0x7fa330071e60 0x7fa3301574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.864+0000 7fa32ed9d640 1 -- 172.21.15.80:0/174599226 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa33006fbf0 con 0x7fa330160fa0 2024-04-25T13:44:29.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa32ed9d640 1 --2- 172.21.15.80:0/174599226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330160fa0 0x7fa3300b4ba0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa330071630 tx=0x7fa32001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7c1bdb9a21a83fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:29.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa32d59a640 1 -- 172.21.15.80:0/174599226 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200a5760 con 0x7fa330160fa0 2024-04-25T13:44:29.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa32d59a640 1 -- 172.21.15.80:0/174599226 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3200a5900 con 0x7fa330160fa0 2024-04-25T13:44:29.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa32d59a640 1 -- 172.21.15.80:0/174599226 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200a5cd0 con 0x7fa330160fa0 2024-04-25T13:44:29.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 -- 172.21.15.80:0/174599226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330160fa0 msgr2=0x7fa3300b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:29.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 --2- 172.21.15.80:0/174599226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330160fa0 0x7fa3300b4ba0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fa330071630 tx=0x7fa32001c9f0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 -- 172.21.15.80:0/174599226 shutdown_connections 2024-04-25T13:44:29.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 --2- 172.21.15.80:0/174599226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330160fa0 0x7fa3300b4ba0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 --2- 172.21.15.80:0/174599226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330073c40 0x7fa330160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 --2- 172.21.15.80:0/174599226 >> v2:172.21.15.87:3300/0 conn(0x7fa330071e60 0x7fa3301574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 -- 172.21.15.80:0/174599226 >> 172.21.15.80:0/174599226 conn(0x7fa3300b4160 msgr2=0x7fa3300d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:29.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 -- 172.21.15.80:0/174599226 shutdown_connections 2024-04-25T13:44:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.868+0000 7fa334bef640 1 -- 172.21.15.80:0/174599226 wait complete. 2024-04-25T13:44:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 Processor -- start 2024-04-25T13:44:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 -- start start 2024-04-25T13:44:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330071e60 0x7fa3301445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa330073c40 0x7fa3301585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330160fa0 0x7fa3300af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa330074050 con 0x7fa330071e60 2024-04-25T13:44:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa330072270 con 0x7fa330073c40 2024-04-25T13:44:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa33015d470 con 0x7fa330160fa0 2024-04-25T13:44:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330071e60 0x7fa3301445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330071e60 0x7fa3301445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60740/0 (socket says 172.21.15.80:60740) 2024-04-25T13:44:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 -- 172.21.15.80:0/1896446014 learned_addr learned my addr 172.21.15.80:0/1896446014 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32dd9b640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa330073c40 0x7fa3301585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32ed9d640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330160fa0 0x7fa3300af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 -- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330160fa0 msgr2=0x7fa3300af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330160fa0 0x7fa3300af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 -- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa330073c40 msgr2=0x7fa3301585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa330073c40 0x7fa3301585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:29.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 -- 172.21.15.80:0/1896446014 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa330154f00 con 0x7fa330071e60 2024-04-25T13:44:29.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32ed9d640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330160fa0 0x7fa3300af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:29.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32dd9b640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa330073c40 0x7fa3301585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:29.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa32e59c640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330071e60 0x7fa3301445b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa32800d800 tx=0x7fa32800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:29.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa31effd640 1 -- 172.21.15.80:0/1896446014 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328018030 con 0x7fa330071e60 2024-04-25T13:44:29.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa31effd640 1 -- 172.21.15.80:0/1896446014 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3280096a0 con 0x7fa330071e60 2024-04-25T13:44:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa31effd640 1 -- 172.21.15.80:0/1896446014 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa328009870 con 0x7fa330071e60 2024-04-25T13:44:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3301551e0 con 0x7fa330071e60 2024-04-25T13:44:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3301557a0 con 0x7fa330071e60 2024-04-25T13:44:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.872+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa33006a3a0 con 0x7fa330071e60 2024-04-25T13:44:29.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.876+0000 7fa31effd640 1 -- 172.21.15.80:0/1896446014 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa32801b070 con 0x7fa330071e60 2024-04-25T13:44:29.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.876+0000 7fa31effd640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa31006f430 0x7fa310071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:29.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.876+0000 7fa31effd640 1 -- 172.21.15.80:0/1896446014 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7fa32808ec60 con 0x7fa330071e60 2024-04-25T13:44:29.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.876+0000 7fa32dd9b640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa31006f430 0x7fa310071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:29.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.876+0000 7fa32dd9b640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa31006f430 0x7fa310071940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa33006dc90 tx=0x7fa324008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:29.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:29.880+0000 7fa31effd640 1 -- 172.21.15.80:0/1896446014 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa32805bfc0 con 0x7fa330071e60 2024-04-25T13:44:30.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.036+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa330158b10 con 0x7fa330071e60 2024-04-25T13:44:30.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.040+0000 7fa31effd640 1 -- 172.21.15.80:0/1896446014 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fa32805b8c0 con 0x7fa330071e60 2024-04-25T13:44:30.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa31006f430 msgr2=0x7fa310071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa31006f430 0x7fa310071940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fa33006dc90 tx=0x7fa324008040 comp rx=0 tx=0).stop 2024-04-25T13:44:30.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330071e60 msgr2=0x7fa3301445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330071e60 0x7fa3301445b0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa32800d800 tx=0x7fa32800dcc0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 shutdown_connections 2024-04-25T13:44:30.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa31006f430 0x7fa310071940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa330160fa0 0x7fa3300af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa330073c40 0x7fa3301585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 --2- 172.21.15.80:0/1896446014 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa330071e60 0x7fa3301445b0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 >> 172.21.15.80:0/1896446014 conn(0x7fa3300b4160 msgr2=0x7fa3300cdab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 shutdown_connections 2024-04-25T13:44:30.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.044+0000 7fa334bef640 1 -- 172.21.15.80:0/1896446014 wait complete. 2024-04-25T13:44:30.063 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:44:30.064 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:44:30.103 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:44:30.152 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:44:30.152 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:44:30.153 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:30.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:44:30.160 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:44:30.161 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:30.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:44:30.167 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:44:30.167 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:30.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:44:30.207 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:44:30.207 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:44:30.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:44:30.216 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:44:30.216 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:30.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44d4838640 1 Processor -- start 2024-04-25T13:44:30.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44d4838640 1 -- start start 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44d4838640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44d0161f90 0x7f44d00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44d4838640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 0x7f44d015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44d4838640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44d00d0d70 con 0x7f44d00d0510 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44d4838640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f44d00d1050 con 0x7f44d0161f90 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44d4838640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f44d006dca0 con 0x7f44d015ec40 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44ce59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44d0161f90 0x7f44d00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44ce59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f44d0161f90 0x7f44d00cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60754/0 (socket says 172.21.15.80:60754) 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44ce59c640 1 -- 172.21.15.80:0/2930987192 learned_addr learned my addr 172.21.15.80:0/2930987192 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44ced9d640 1 --2- 172.21.15.80:0/2930987192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 0x7f44d015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44ce59c640 1 -- 172.21.15.80:0/2930987192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 msgr2=0x7f44d015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44ce59c640 1 --2- 172.21.15.80:0/2930987192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 0x7f44d015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44ce59c640 1 -- 172.21.15.80:0/2930987192 >> v1:172.21.15.87:6789/0 conn(0x7f44d00d0510 legacy=0x7f44d015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.396+0000 7f44ce59c640 1 -- 172.21.15.80:0/2930987192 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d006df80 con 0x7f44d0161f90 2024-04-25T13:44:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.400+0000 7f44ce59c640 1 --2- 172.21.15.80:0/2930987192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d0161f90 0x7f44d00cffd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009e50 tx=0x7f44c801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be2f33ed2143d87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.400+0000 7f44cd59a640 1 -- 172.21.15.80:0/2930987192 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80a5730 con 0x7f44d0161f90 2024-04-25T13:44:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.400+0000 7f44cd59a640 1 -- 172.21.15.80:0/2930987192 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c80a58d0 con 0x7f44d0161f90 2024-04-25T13:44:30.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.400+0000 7f44cd59a640 1 -- 172.21.15.80:0/2930987192 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80a5ca0 con 0x7f44d0161f90 2024-04-25T13:44:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- 172.21.15.80:0/2930987192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d0161f90 msgr2=0x7f44d00cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 --2- 172.21.15.80:0/2930987192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d0161f90 0x7f44d00cffd0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009e50 tx=0x7f44c801c940 comp rx=0 tx=0).stop 2024-04-25T13:44:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- 172.21.15.80:0/2930987192 shutdown_connections 2024-04-25T13:44:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 --2- 172.21.15.80:0/2930987192 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 0x7f44d015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 --2- 172.21.15.80:0/2930987192 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d0161f90 0x7f44d00cffd0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- 172.21.15.80:0/2930987192 >> 172.21.15.80:0/2930987192 conn(0x7f44d006d080 msgr2=0x7f44d00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- 172.21.15.80:0/2930987192 shutdown_connections 2024-04-25T13:44:30.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- 172.21.15.80:0/2930987192 wait complete. 2024-04-25T13:44:30.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 Processor -- start 2024-04-25T13:44:30.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- start start 2024-04-25T13:44:30.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d00d0510 0x7f44d0162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 0x7f44d014e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44d0161f90 0x7f44d00af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f44d006f4b0 con 0x7f44d00d0510 2024-04-25T13:44:30.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f44d006f630 con 0x7f44d0161f90 2024-04-25T13:44:30.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44d4838640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f44d00d0920 con 0x7f44d015ec40 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44ce59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d00d0510 0x7f44d0162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44ce59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d00d0510 0x7f44d0162ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60770/0 (socket says 172.21.15.80:60770) 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.404+0000 7f44ce59c640 1 -- 172.21.15.80:0/1059593887 learned_addr learned my addr 172.21.15.80:0/1059593887 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44ced9d640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44d0161f90 0x7f44d00af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44cdd9b640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 0x7f44d014e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44ce59c640 1 -- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 msgr2=0x7f44d014e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44ce59c640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 0x7f44d014e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44ce59c640 1 -- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44d0161f90 msgr2=0x7f44d00af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44ce59c640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44d0161f90 0x7f44d00af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44ce59c640 1 -- 172.21.15.80:0/1059593887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d014e6e0 con 0x7f44d00d0510 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44ce59c640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d00d0510 0x7f44d0162ca0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f44c801a5d0 tx=0x7f44c8019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44beffd640 1 -- 172.21.15.80:0/1059593887 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80b8670 con 0x7f44d00d0510 2024-04-25T13:44:30.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44beffd640 1 -- 172.21.15.80:0/1059593887 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44c80be070 con 0x7f44d00d0510 2024-04-25T13:44:30.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d01f96d0 con 0x7f44d00d0510 2024-04-25T13:44:30.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44beffd640 1 -- 172.21.15.80:0/1059593887 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44c80b8a30 con 0x7f44d00d0510 2024-04-25T13:44:30.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d00723b0 con 0x7f44d00d0510 2024-04-25T13:44:30.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d006a3a0 con 0x7f44d00d0510 2024-04-25T13:44:30.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44beffd640 1 -- 172.21.15.80:0/1059593887 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f44c80b8bd0 con 0x7f44d00d0510 2024-04-25T13:44:30.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44beffd640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44b006f400 0x7f44b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44beffd640 1 -- 172.21.15.80:0/1059593887 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7f44c812aab0 con 0x7f44d00d0510 2024-04-25T13:44:30.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.408+0000 7f44cdd9b640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44b006f400 0x7f44b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.412+0000 7f44cdd9b640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44b006f400 0x7f44b0071910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f44d015bd00 tx=0x7f44c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.412+0000 7f44beffd640 1 -- 172.21.15.80:0/1059593887 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f44c8016620 con 0x7f44d00d0510 2024-04-25T13:44:30.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f44d0161740 con 0x7f44d00d0510 2024-04-25T13:44:30.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44beffd640 1 -- 172.21.15.80:0/1059593887 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+3869 (secure 0 0 0) 0x7f44c80f7d80 con 0x7f44d00d0510 2024-04-25T13:44:30.595 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:30.595 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":35,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5099,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1185745761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1185745761},{"type":"v1","addr":"172.21.15.80:6835","nonce":1185745761}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5488,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2524166918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2524166918},{"type":"v1","addr":"172.21.15.87:6835","nonce":2524166918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":5491,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3637133191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3637133191},{"type":"v1","addr":"172.21.15.87:6837","nonce":3637133191}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2471048535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2471048535},{"type":"v1","addr":"172.21.15.80:6834","nonce":2471048535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":35}],"filesystems":[]} 2024-04-25T13:44:30.595 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 35 2024-04-25T13:44:30.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44b006f400 msgr2=0x7f44b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44b006f400 0x7f44b0071910 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f44d015bd00 tx=0x7f44c4009040 comp rx=0 tx=0).stop 2024-04-25T13:44:30.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d00d0510 msgr2=0x7f44d0162ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d00d0510 0x7f44d0162ca0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f44c801a5d0 tx=0x7f44c8019740 comp rx=0 tx=0).stop 2024-04-25T13:44:30.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 shutdown_connections 2024-04-25T13:44:30.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f44b006f400 0x7f44b0071910 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f44d0161f90 0x7f44d00af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f44d015ec40 0x7f44d014e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 --2- 172.21.15.80:0/1059593887 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f44d00d0510 0x7f44d0162ca0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 >> 172.21.15.80:0/1059593887 conn(0x7f44d006d080 msgr2=0x7f44d00b14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 shutdown_connections 2024-04-25T13:44:30.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.596+0000 7f44d4838640 1 -- 172.21.15.80:0/1059593887 wait complete. 2024-04-25T13:44:30.616 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:44:30.616 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:44:30.616 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:44:30.616 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:44:30.617 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:44:30.617 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:44:30.617 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:44:30.617 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:44:30.617 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:44:30.617 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:44:30.617 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:44:30.618 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:44:30.618 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:44:30.618 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:44:30.618 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:44:30.618 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:44:30.632 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:44:30.632 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:44:30.633 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:44:30.633 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:44:30.635 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:44:30.636 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:44:30.637 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:44:30.637 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:44:30.675 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:44:30.675 DEBUG:teuthology.parallel:result is None 2024-04-25T13:44:30.676 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:44:30.676 DEBUG:teuthology.parallel:result is None 2024-04-25T13:44:30.678 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:44:30.679 DEBUG:teuthology.parallel:result is None 2024-04-25T13:44:30.680 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:44:30.680 DEBUG:teuthology.parallel:result is None 2024-04-25T13:44:30.680 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:44:30.763 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:44:30.768 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:44:30.771 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:44:30.775 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:44:30.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d3e3a7640 1 Processor -- start 2024-04-25T13:44:30.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d3e3a7640 1 -- start start 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d3e3a7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3d38073c90 0x7f3d3806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d3e3a7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d3806f2e0 0x7f3d3806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d3e3a7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d380b69f0 con 0x7f3d380b65e0 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d3e3a7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3d380b6c80 con 0x7f3d38073c90 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d3e3a7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d380b6f60 con 0x7f3d3806f2e0 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d37fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3d380b65e0 0x7f3d38073580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55918/0 (socket says 172.21.15.80:55918) 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d37fff640 1 -- 172.21.15.80:0/2312432058 learned_addr learned my addr 172.21.15.80:0/2312432058 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d377fe640 1 --2- 172.21.15.80:0/2312432058 >> v2:172.21.15.80:3300/0 conn(0x7f3d38073c90 0x7f3d3806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d3c944640 1 --2- 172.21.15.80:0/2312432058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d3806f2e0 0x7f3d3806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d36ffd640 1 -- 172.21.15.80:0/2312432058 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3820036123 0 0) 0x7f3d380b69f0 con 0x7f3d380b65e0 2024-04-25T13:44:30.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d36ffd640 1 -- 172.21.15.80:0/2312432058 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d20003530 con 0x7f3d380b65e0 2024-04-25T13:44:30.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d377fe640 1 -- 172.21.15.80:0/2312432058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d3806f2e0 msgr2=0x7f3d3806f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:30.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d377fe640 1 --2- 172.21.15.80:0/2312432058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d3806f2e0 0x7f3d3806f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d377fe640 1 -- 172.21.15.80:0/2312432058 >> v1:172.21.15.80:6789/0 conn(0x7f3d380b65e0 legacy=0x7f3d38073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.824+0000 7f3d377fe640 1 -- 172.21.15.80:0/2312432058 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d380b7240 con 0x7f3d38073c90 2024-04-25T13:44:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d377fe640 1 --2- 172.21.15.80:0/2312432058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d38073c90 0x7f3d3806eda0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0097c0 tx=0x7f3d2c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47943a198ad2f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d36ffd640 1 -- 172.21.15.80:0/2312432058 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0b0070 con 0x7f3d38073c90 2024-04-25T13:44:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d36ffd640 1 -- 172.21.15.80:0/2312432058 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d2c004230 con 0x7f3d38073c90 2024-04-25T13:44:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d36ffd640 1 -- 172.21.15.80:0/2312432058 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c003620 con 0x7f3d38073c90 2024-04-25T13:44:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/2312432058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d38073c90 msgr2=0x7f3d3806eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/2312432058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d38073c90 0x7f3d3806eda0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0097c0 tx=0x7f3d2c01c350 comp rx=0 tx=0).stop 2024-04-25T13:44:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/2312432058 shutdown_connections 2024-04-25T13:44:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/2312432058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d3806f2e0 0x7f3d3806f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/2312432058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d38073c90 0x7f3d3806eda0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/2312432058 >> 172.21.15.80:0/2312432058 conn(0x7f3d380d3500 msgr2=0x7f3d38072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/2312432058 shutdown_connections 2024-04-25T13:44:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.828+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/2312432058 wait complete. 2024-04-25T13:44:30.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 Processor -- start 2024-04-25T13:44:30.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 -- start start 2024-04-25T13:44:30.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d3806f2e0 0x7f3d38159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d380b65e0 0x7f3d38152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d381fcd50 0x7f3d38153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3d38160450 con 0x7f3d3806f2e0 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d380afeb0 con 0x7f3d380b65e0 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d381605d0 con 0x7f3d381fcd50 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d3806f2e0 0x7f3d38159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d3806f2e0 0x7f3d38159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60852/0 (socket says 172.21.15.80:60852) 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 -- 172.21.15.80:0/1943760320 learned_addr learned my addr 172.21.15.80:0/1943760320 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d377fe640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d380b65e0 0x7f3d38152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3c944640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d381fcd50 0x7f3d38153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 -- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d381fcd50 msgr2=0x7f3d38153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d381fcd50 0x7f3d38153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 -- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d380b65e0 msgr2=0x7f3d38152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d380b65e0 0x7f3d38152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 -- 172.21.15.80:0/1943760320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d381586b0 con 0x7f3d3806f2e0 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d377fe640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d380b65e0 0x7f3d38152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3c944640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d381fcd50 0x7f3d38153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d37fff640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d3806f2e0 0x7f3d38159c20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c3f0 tx=0x7f3d2800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d34ff9640 1 -- 172.21.15.80:0/1943760320 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d28017070 con 0x7f3d3806f2e0 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d34ff9640 1 -- 172.21.15.80:0/1943760320 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d28005cf0 con 0x7f3d3806f2e0 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d34ff9640 1 -- 172.21.15.80:0/1943760320 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d280072f0 con 0x7f3d3806f2e0 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d38158bb0 con 0x7f3d3806f2e0 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d38159170 con 0x7f3d3806f2e0 2024-04-25T13:44:30.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.832+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3806a3a0 con 0x7f3d3806f2e0 2024-04-25T13:44:30.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.836+0000 7f3d34ff9640 1 -- 172.21.15.80:0/1943760320 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3d2801a070 con 0x7f3d3806f2e0 2024-04-25T13:44:30.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.836+0000 7f3d34ff9640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d2406f400 0x7f3d24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:30.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.836+0000 7f3d34ff9640 1 -- 172.21.15.80:0/1943760320 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 4943+0+0 (secure 0 0 0) 0x7f3d2808e7f0 con 0x7f3d3806f2e0 2024-04-25T13:44:30.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.836+0000 7f3d377fe640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d2406f400 0x7f3d24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:30.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.836+0000 7f3d377fe640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d2406f400 0x7f3d24071910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0ad000 tx=0x7f3d2c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:30.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:30.840+0000 7f3d34ff9640 1 -- 172.21.15.80:0/1943760320 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d28021020 con 0x7f3d3806f2e0 2024-04-25T13:44:31.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:31.012+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f3d38069360 con 0x7f3d3806f2e0 2024-04-25T13:44:32.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.072+0000 7f3d34ff9640 1 -- 172.21.15.80:0/1943760320 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) ==== 97+0+0 (secure 0 0 0) 0x7f3d2805bac0 con 0x7f3d3806f2e0 2024-04-25T13:44:32.071 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d2406f400 msgr2=0x7f3d24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d2406f400 0x7f3d24071910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0ad000 tx=0x7f3d2c0a6000 comp rx=0 tx=0).stop 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d3806f2e0 msgr2=0x7f3d38159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d3806f2e0 0x7f3d38159c20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f3d2800c3f0 tx=0x7f3d2800c8b0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 shutdown_connections 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d2406f400 0x7f3d24071910 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d381fcd50 0x7f3d38153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d380b65e0 0x7f3d38152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 --2- 172.21.15.80:0/1943760320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d3806f2e0 0x7f3d38159c20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 >> 172.21.15.80:0/1943760320 conn(0x7f3d380d3500 msgr2=0x7f3d38073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 shutdown_connections 2024-04-25T13:44:32.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.076+0000 7f3d3e3a7640 1 -- 172.21.15.80:0/1943760320 wait complete. 2024-04-25T13:44:32.101 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:44:32.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d42d7c640 1 Processor -- start 2024-04-25T13:44:32.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d42d7c640 1 -- start start 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d42d7c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c161f90 0x7f5d3c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d42d7c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d3c0d0510 0x7f5d3c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d42d7c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d3c0d0d70 con 0x7f5d3c15ea70 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d42d7c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5d3c0d1050 con 0x7f5d3c0d0510 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d42d7c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3c06dca0 con 0x7f5d3c161f90 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d3bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d3c0d0510 0x7f5d3c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d3bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d3c0d0510 0x7f5d3c15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56846/0 (socket says 172.21.15.80:56846) 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d3bfff640 1 -- 172.21.15.80:0/3362404960 learned_addr learned my addr 172.21.15.80:0/3362404960 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:32.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d41319640 1 --1- 172.21.15.80:0/3362404960 >> v1:172.21.15.80:6789/0 conn(0x7f5d3c15ea70 0x7f5d3c15ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:60316/0 (socket says 172.21.15.80:60316) 2024-04-25T13:44:32.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d40b18640 1 --2- 172.21.15.80:0/3362404960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c161f90 0x7f5d3c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d3bfff640 1 -- 172.21.15.80:0/3362404960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c161f90 msgr2=0x7f5d3c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d3bfff640 1 --2- 172.21.15.80:0/3362404960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c161f90 0x7f5d3c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d3bfff640 1 -- 172.21.15.80:0/3362404960 >> v1:172.21.15.80:6789/0 conn(0x7f5d3c15ea70 legacy=0x7f5d3c15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.256+0000 7f5d3bfff640 1 -- 172.21.15.80:0/3362404960 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c06df80 con 0x7f5d3c0d0510 2024-04-25T13:44:32.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d3bfff640 1 --2- 172.21.15.80:0/3362404960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c15e530 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009e50 tx=0x7f5d2c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ae42eca8a0ae6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d3b7fe640 1 -- 172.21.15.80:0/3362404960 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0a5730 con 0x7f5d3c0d0510 2024-04-25T13:44:32.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d3b7fe640 1 -- 172.21.15.80:0/3362404960 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d2c0a58d0 con 0x7f5d3c0d0510 2024-04-25T13:44:32.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d3b7fe640 1 -- 172.21.15.80:0/3362404960 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c0a5ca0 con 0x7f5d3c0d0510 2024-04-25T13:44:32.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3362404960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 msgr2=0x7f5d3c15e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3362404960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c15e530 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009e50 tx=0x7f5d2c01c940 comp rx=0 tx=0).stop 2024-04-25T13:44:32.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3362404960 shutdown_connections 2024-04-25T13:44:32.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3362404960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c15e530 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3362404960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c161f90 0x7f5d3c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3362404960 >> 172.21.15.80:0/3362404960 conn(0x7f5d3c06d080 msgr2=0x7f5d3c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3362404960 shutdown_connections 2024-04-25T13:44:32.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.260+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3362404960 wait complete. 2024-04-25T13:44:32.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 Processor -- start 2024-04-25T13:44:32.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 -- start start 2024-04-25T13:44:32.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c141560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c15ea70 0x7f5d3c1448e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d3c161f90 0x7f5d3c1466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c06f630 con 0x7f5d3c0d0510 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3c06f4b0 con 0x7f5d3c161f90 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5d3c0d0920 con 0x7f5d3c15ea70 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c141560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c141560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56858/0 (socket says 172.21.15.80:56858) 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 -- 172.21.15.80:0/3376925124 learned_addr learned my addr 172.21.15.80:0/3376925124 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d3bfff640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c15ea70 0x7f5d3c1448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d41319640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d3c161f90 0x7f5d3c1466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 -- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c15ea70 msgr2=0x7f5d3c1448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c15ea70 0x7f5d3c1448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 -- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d3c161f90 msgr2=0x7f5d3c1466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d3c161f90 0x7f5d3c1466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 -- 172.21.15.80:0/3376925124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d3c15b600 con 0x7f5d3c0d0510 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d3bfff640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c15ea70 0x7f5d3c1448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:32.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d40b18640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c141560 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000d5f0 tx=0x7f5d3000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d397fa640 1 -- 172.21.15.80:0/3376925124 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30004240 con 0x7f5d3c0d0510 2024-04-25T13:44:32.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d397fa640 1 -- 172.21.15.80:0/3376925124 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d300043e0 con 0x7f5d3c0d0510 2024-04-25T13:44:32.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d397fa640 1 -- 172.21.15.80:0/3376925124 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d30003840 con 0x7f5d3c0d0510 2024-04-25T13:44:32.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d3c1475d0 con 0x7f5d3c0d0510 2024-04-25T13:44:32.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.264+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d3c15b8a0 con 0x7f5d3c0d0510 2024-04-25T13:44:32.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.268+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d3c06a3a0 con 0x7f5d3c0d0510 2024-04-25T13:44:32.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.268+0000 7f5d397fa640 1 -- 172.21.15.80:0/3376925124 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5d300039e0 con 0x7f5d3c0d0510 2024-04-25T13:44:32.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.268+0000 7f5d397fa640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d2806f400 0x7f5d28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.268+0000 7f5d397fa640 1 -- 172.21.15.80:0/3376925124 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f5d3008f190 con 0x7f5d3c0d0510 2024-04-25T13:44:32.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.268+0000 7f5d3bfff640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d2806f400 0x7f5d28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.268+0000 7f5d3bfff640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d2806f400 0x7f5d28071910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0ae000 tx=0x7f5d2c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.272+0000 7f5d397fa640 1 -- 172.21.15.80:0/3376925124 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d3005c460 con 0x7f5d3c0d0510 2024-04-25T13:44:32.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.488+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f5d3c1615f0 con 0x7f5d3c0d0510 2024-04-25T13:44:32.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d397fa640 1 -- 172.21.15.80:0/3376925124 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7f5d3005bd60 con 0x7f5d3c0d0510 2024-04-25T13:44:32.490 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.491 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:44:32.492 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.493 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:44:32.494 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:44:32.495 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.496 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:44:32.497 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:44:32.498 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.499 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.500 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.501 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:44:32.502 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:44:32.503 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:44:32.503 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:44:32.503 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.503 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:32.503 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:44:32.503 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d2806f400 msgr2=0x7f5d28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d2806f400 0x7f5d28071910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c0ae000 tx=0x7f5d2c0a7070 comp rx=0 tx=0).stop 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 msgr2=0x7f5d3c141560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c141560 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f5d3000d5f0 tx=0x7f5d3000dab0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 shutdown_connections 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d2806f400 0x7f5d28071910 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d3c161f90 0x7f5d3c1466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d3c15ea70 0x7f5d3c1448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 --2- 172.21.15.80:0/3376925124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d3c0d0510 0x7f5d3c141560 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 >> 172.21.15.80:0/3376925124 conn(0x7f5d3c06d080 msgr2=0x7f5d3c160860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 shutdown_connections 2024-04-25T13:44:32.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.492+0000 7f5d42d7c640 1 -- 172.21.15.80:0/3376925124 wait complete. 2024-04-25T13:44:32.513 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:32.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.664+0000 7eff8c183640 1 Processor -- start 2024-04-25T13:44:32.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.664+0000 7eff8c183640 1 -- start start 2024-04-25T13:44:32.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff8c183640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7eff840ce730 0x7eff840ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff8c183640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff84073710 con 0x7eff840b0ed0 2024-04-25T13:44:32.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff8c183640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7eff840739f0 con 0x7eff8406e490 2024-04-25T13:44:32.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff8c183640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7eff84073cd0 con 0x7eff840ce730 2024-04-25T13:44:32.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff89f1f640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7eff840b0ed0 0x7eff8415c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:60318/0 (socket says 172.21.15.80:60318) 2024-04-25T13:44:32.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff89f1f640 1 -- 172.21.15.80:0/2156281734 learned_addr learned my addr 172.21.15.80:0/2156281734 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:32.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff8971e640 1 --2- 172.21.15.80:0/2156281734 >> v2:172.21.15.80:3300/0 conn(0x7eff840ce730 0x7eff840ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff88f1d640 1 -- 172.21.15.80:0/2156281734 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985291805 0 0) 0x7eff84073710 con 0x7eff840b0ed0 2024-04-25T13:44:32.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff88f1d640 1 -- 172.21.15.80:0/2156281734 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7eff6c003530 con 0x7eff840b0ed0 2024-04-25T13:44:32.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff8971e640 1 -- 172.21.15.80:0/2156281734 >> v1:172.21.15.87:6789/0 conn(0x7eff8406e490 legacy=0x7eff8406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:32.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff8971e640 1 -- 172.21.15.80:0/2156281734 >> v1:172.21.15.80:6789/0 conn(0x7eff840b0ed0 legacy=0x7eff8415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.668+0000 7eff8971e640 1 -- 172.21.15.80:0/2156281734 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff84073fb0 con 0x7eff840ce730 2024-04-25T13:44:32.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8971e640 1 --2- 172.21.15.80:0/2156281734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff840ce730 0x7eff840ceb40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7eff780097c0 tx=0x7eff7801fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9efc5fcf1a384a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff88f1d640 1 -- 172.21.15.80:0/2156281734 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff780b6070 con 0x7eff840ce730 2024-04-25T13:44:32.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff88f1d640 1 -- 172.21.15.80:0/2156281734 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff780a9aa0 con 0x7eff840ce730 2024-04-25T13:44:32.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff88f1d640 1 -- 172.21.15.80:0/2156281734 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff780b1440 con 0x7eff840ce730 2024-04-25T13:44:32.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 -- 172.21.15.80:0/2156281734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff840ce730 msgr2=0x7eff840ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 --2- 172.21.15.80:0/2156281734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff840ce730 0x7eff840ceb40 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7eff780097c0 tx=0x7eff7801fda0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 -- 172.21.15.80:0/2156281734 shutdown_connections 2024-04-25T13:44:32.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 --2- 172.21.15.80:0/2156281734 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff840ce730 0x7eff840ceb40 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 -- 172.21.15.80:0/2156281734 >> 172.21.15.80:0/2156281734 conn(0x7eff8406d080 msgr2=0x7eff840b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 -- 172.21.15.80:0/2156281734 shutdown_connections 2024-04-25T13:44:32.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 -- 172.21.15.80:0/2156281734 wait complete. 2024-04-25T13:44:32.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 Processor -- start 2024-04-25T13:44:32.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.672+0000 7eff8c183640 1 -- start start 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff8406e490 0x7eff840b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff840b0ed0 0x7eff840b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff840ce730 0x7eff8415cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7eff8406dc00 con 0x7eff8406e490 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7eff840733a0 con 0x7eff840ce730 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7eff8406dd80 con 0x7eff840b0ed0 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff8406e490 0x7eff840b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff8406e490 0x7eff840b27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56876/0 (socket says 172.21.15.80:56876) 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 -- 172.21.15.80:0/2208161024 learned_addr learned my addr 172.21.15.80:0/2208161024 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8971e640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff840b0ed0 0x7eff840b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 -- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff840b0ed0 msgr2=0x7eff840b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff840b0ed0 0x7eff840b2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 -- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff840ce730 msgr2=0x7eff8415cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff840ce730 0x7eff8415cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 -- 172.21.15.80:0/2208161024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff841590c0 con 0x7eff8406e490 2024-04-25T13:44:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff89f1f640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff8406e490 0x7eff840b27a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c420 tx=0x7eff7400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff827fc640 1 -- 172.21.15.80:0/2208161024 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff74017070 con 0x7eff8406e490 2024-04-25T13:44:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff827fc640 1 -- 172.21.15.80:0/2208161024 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7eff74005cf0 con 0x7eff8406e490 2024-04-25T13:44:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff841593a0 con 0x7eff8406e490 2024-04-25T13:44:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff827fc640 1 -- 172.21.15.80:0/2208161024 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff740072d0 con 0x7eff8406e490 2024-04-25T13:44:32.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff84159960 con 0x7eff8406e490 2024-04-25T13:44:32.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8406a3a0 con 0x7eff8406e490 2024-04-25T13:44:32.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff827fc640 1 -- 172.21.15.80:0/2208161024 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7eff7401a070 con 0x7eff8406e490 2024-04-25T13:44:32.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff827fc640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff7006f400 0x7eff70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:32.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff827fc640 1 -- 172.21.15.80:0/2208161024 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7eff7408e5c0 con 0x7eff8406e490 2024-04-25T13:44:32.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.676+0000 7eff8971e640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff7006f400 0x7eff70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:32.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.680+0000 7eff8971e640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff7006f400 0x7eff70071910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff780b3000 tx=0x7eff780ab070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:32.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.680+0000 7eff827fc640 1 -- 172.21.15.80:0/2208161024 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7eff7405b320 con 0x7eff8406e490 2024-04-25T13:44:32.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.844+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7eff8406f880 con 0x7eff8406e490 2024-04-25T13:44:32.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.844+0000 7eff827fc640 1 -- 172.21.15.80:0/2208161024 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) ==== 74+0+12819 (secure 0 0 0) 0x7eff7405f620 con 0x7eff8406e490 2024-04-25T13:44:32.844 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:32.844 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":78,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:44:32.070526+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:32.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff7006f400 msgr2=0x7eff70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff7006f400 0x7eff70071910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7eff780b3000 tx=0x7eff780ab070 comp rx=0 tx=0).stop 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff8406e490 msgr2=0x7eff840b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff8406e490 0x7eff840b27a0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7eff7400c420 tx=0x7eff7400c8e0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 shutdown_connections 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7eff7006f400 0x7eff70071910 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7eff840ce730 0x7eff8415cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7eff840b0ed0 0x7eff840b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 --2- 172.21.15.80:0/2208161024 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7eff8406e490 0x7eff840b27a0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 >> 172.21.15.80:0/2208161024 conn(0x7eff8406d080 msgr2=0x7eff840d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 shutdown_connections 2024-04-25T13:44:32.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:32.848+0000 7eff8c183640 1 -- 172.21.15.80:0/2208161024 wait complete. 2024-04-25T13:44:32.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:44:32.867 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:44:33.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd98fc6640 1 Processor -- start 2024-04-25T13:44:33.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd98fc6640 1 -- start start 2024-04-25T13:44:33.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd98fc6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efd940b5330 0x7efd940d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd98fc6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd940d0ff0 0x7efd9406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd98fc6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd940b5740 con 0x7efd9406f2f0 2024-04-25T13:44:33.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd98fc6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efd9406f700 con 0x7efd940d0ff0 2024-04-25T13:44:33.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd98fc6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efd941629b0 con 0x7efd940b5330 2024-04-25T13:44:33.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd91d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd940d0ff0 0x7efd9406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd91d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd940d0ff0 0x7efd9406edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56886/0 (socket says 172.21.15.80:56886) 2024-04-25T13:44:33.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd91d9b640 1 -- 172.21.15.80:0/1014015433 learned_addr learned my addr 172.21.15.80:0/1014015433 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:33.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd9259c640 1 --2- 172.21.15.80:0/1014015433 >> v2:172.21.15.87:3300/0 conn(0x7efd940b5330 0x7efd940d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd91d9b640 1 -- 172.21.15.80:0/1014015433 >> v2:172.21.15.87:3300/0 conn(0x7efd940b5330 msgr2=0x7efd940d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:33.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd91d9b640 1 --2- 172.21.15.80:0/1014015433 >> v2:172.21.15.87:3300/0 conn(0x7efd940b5330 0x7efd940d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd91d9b640 1 -- 172.21.15.80:0/1014015433 >> v1:172.21.15.87:6789/0 conn(0x7efd9406f2f0 legacy=0x7efd94161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:33.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.024+0000 7efd91d9b640 1 -- 172.21.15.80:0/1014015433 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9406dd10 con 0x7efd940d0ff0 2024-04-25T13:44:33.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd91d9b640 1 --2- 172.21.15.80:0/1014015433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940d0ff0 0x7efd9406edb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7efd940701f0 tx=0x7efd7c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3afe426529b20ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd9159a640 1 -- 172.21.15.80:0/1014015433 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0b0070 con 0x7efd940d0ff0 2024-04-25T13:44:33.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd9159a640 1 -- 172.21.15.80:0/1014015433 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c0ab600 con 0x7efd940d0ff0 2024-04-25T13:44:33.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd9159a640 1 -- 172.21.15.80:0/1014015433 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0ab9d0 con 0x7efd940d0ff0 2024-04-25T13:44:33.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd98fc6640 1 -- 172.21.15.80:0/1014015433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940d0ff0 msgr2=0x7efd9406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd98fc6640 1 --2- 172.21.15.80:0/1014015433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940d0ff0 0x7efd9406edb0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7efd940701f0 tx=0x7efd7c01c450 comp rx=0 tx=0).stop 2024-04-25T13:44:33.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd98fc6640 1 -- 172.21.15.80:0/1014015433 shutdown_connections 2024-04-25T13:44:33.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd98fc6640 1 --2- 172.21.15.80:0/1014015433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940d0ff0 0x7efd9406edb0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd98fc6640 1 --2- 172.21.15.80:0/1014015433 >> v2:172.21.15.87:3300/0 conn(0x7efd940b5330 0x7efd940d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd98fc6640 1 -- 172.21.15.80:0/1014015433 >> 172.21.15.80:0/1014015433 conn(0x7efd940b4160 msgr2=0x7efd940d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:33.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd98fc6640 1 -- 172.21.15.80:0/1014015433 shutdown_connections 2024-04-25T13:44:33.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.028+0000 7efd98fc6640 1 -- 172.21.15.80:0/1014015433 wait complete. 2024-04-25T13:44:33.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 Processor -- start 2024-04-25T13:44:33.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 -- start start 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd9406f2f0 0x7efd9415bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940b5330 0x7efd94135080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd940d0ff0 0x7efd940af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efd940b0cd0 con 0x7efd940b5330 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efd9406e890 con 0x7efd940d0ff0 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efd940b0e50 con 0x7efd9406f2f0 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940b5330 0x7efd94135080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940b5330 0x7efd94135080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56890/0 (socket says 172.21.15.80:56890) 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 -- 172.21.15.80:0/348436165 learned_addr learned my addr 172.21.15.80:0/348436165 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd9259c640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd9406f2f0 0x7efd9415bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd92d9d640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd940d0ff0 0x7efd940af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 -- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd9406f2f0 msgr2=0x7efd9415bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd9406f2f0 0x7efd9415bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 -- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd940d0ff0 msgr2=0x7efd940af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd940d0ff0 0x7efd940af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 -- 172.21.15.80:0/348436165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd9415d6a0 con 0x7efd940b5330 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd92d9d640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd940d0ff0 0x7efd940af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd91d9b640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940b5330 0x7efd94135080 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efd7c01a080 tx=0x7efd7c0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd8affd640 1 -- 172.21.15.80:0/348436165 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0b9070 con 0x7efd940b5330 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd8affd640 1 -- 172.21.15.80:0/348436165 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd7c0b4420 con 0x7efd940b5330 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd9415c600 con 0x7efd940b5330 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd8affd640 1 -- 172.21.15.80:0/348436165 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c0b0040 con 0x7efd940b5330 2024-04-25T13:44:33.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.032+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd9415cbc0 con 0x7efd940b5330 2024-04-25T13:44:33.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.036+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd9415c370 con 0x7efd940b5330 2024-04-25T13:44:33.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.036+0000 7efd8affd640 1 -- 172.21.15.80:0/348436165 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efd7c0bc070 con 0x7efd940b5330 2024-04-25T13:44:33.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.036+0000 7efd8affd640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd7406f400 0x7efd74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.036+0000 7efd8affd640 1 -- 172.21.15.80:0/348436165 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7efd7c128280 con 0x7efd940b5330 2024-04-25T13:44:33.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.036+0000 7efd9259c640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd7406f400 0x7efd74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.036+0000 7efd9259c640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd7406f400 0x7efd74071910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b930 tx=0x7efd8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.040+0000 7efd8affd640 1 -- 172.21.15.80:0/348436165 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efd7c0c3080 con 0x7efd940b5330 2024-04-25T13:44:33.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.196+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efd9406a3a0 con 0x7efd940b5330 2024-04-25T13:44:33.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.196+0000 7efd8affd640 1 -- 172.21.15.80:0/348436165 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v78) ==== 93+0+10 (secure 0 0 0) 0x7efd7c0160d0 con 0x7efd940b5330 2024-04-25T13:44:33.197 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:44:33.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd7406f400 msgr2=0x7efd74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd7406f400 0x7efd74071910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b930 tx=0x7efd8c008040 comp rx=0 tx=0).stop 2024-04-25T13:44:33.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940b5330 msgr2=0x7efd94135080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940b5330 0x7efd94135080 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7efd7c01a080 tx=0x7efd7c0191f0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 shutdown_connections 2024-04-25T13:44:33.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efd7406f400 0x7efd74071910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd940d0ff0 0x7efd940af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd940b5330 0x7efd94135080 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 --2- 172.21.15.80:0/348436165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd9406f2f0 0x7efd9415bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 >> 172.21.15.80:0/348436165 conn(0x7efd940b4160 msgr2=0x7efd940d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:33.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 shutdown_connections 2024-04-25T13:44:33.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.200+0000 7efd98fc6640 1 -- 172.21.15.80:0/348436165 wait complete. 2024-04-25T13:44:33.217 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:44:33.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc714b640 1 Processor -- start 2024-04-25T13:44:33.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc714b640 1 -- start start 2024-04-25T13:44:33.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc714b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcfc00b8cd0 0x7fcfc00b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc714b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc00d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc714b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcfc00cfaa0 0x7fcfc00d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc714b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc015e8c0 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc714b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcfc00ce8d0 con 0x7fcfc00b8cd0 2024-04-25T13:44:33.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc714b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc00cebb0 con 0x7fcfc00cf690 2024-04-25T13:44:33.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc56e8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcfc00cfaa0 0x7fcfc00d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc56e8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcfc00cfaa0 0x7fcfc00d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56898/0 (socket says 172.21.15.80:56898) 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc56e8640 1 -- 172.21.15.80:0/4072431418 learned_addr learned my addr 172.21.15.80:0/4072431418 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfbffff640 1 --2- 172.21.15.80:0/4072431418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc00d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc4ee7640 1 --2- 172.21.15.80:0/4072431418 >> v2:172.21.15.87:3300/0 conn(0x7fcfc00b8cd0 0x7fcfc00b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc56e8640 1 -- 172.21.15.80:0/4072431418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 msgr2=0x7fcfc00d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc56e8640 1 --2- 172.21.15.80:0/4072431418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc00d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc56e8640 1 -- 172.21.15.80:0/4072431418 >> v2:172.21.15.87:3300/0 conn(0x7fcfc00b8cd0 msgr2=0x7fcfc00b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc56e8640 1 --2- 172.21.15.80:0/4072431418 >> v2:172.21.15.87:3300/0 conn(0x7fcfc00b8cd0 0x7fcfc00b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc56e8640 1 -- 172.21.15.80:0/4072431418 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc00cee90 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.352+0000 7fcfc4ee7640 1 --2- 172.21.15.80:0/4072431418 >> v2:172.21.15.87:3300/0 conn(0x7fcfc00b8cd0 0x7fcfc00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:33.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc56e8640 1 --2- 172.21.15.80:0/4072431418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc00d0930 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00b6660 tx=0x7fcfb801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff8767b4275d16c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfbf7fe640 1 -- 172.21.15.80:0/4072431418 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80a56f0 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfbf7fe640 1 -- 172.21.15.80:0/4072431418 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb80a5890 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfbf7fe640 1 -- 172.21.15.80:0/4072431418 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80a5c60 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 -- 172.21.15.80:0/4072431418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 msgr2=0x7fcfc00d0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 --2- 172.21.15.80:0/4072431418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc00d0930 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00b6660 tx=0x7fcfb801c960 comp rx=0 tx=0).stop 2024-04-25T13:44:33.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 -- 172.21.15.80:0/4072431418 shutdown_connections 2024-04-25T13:44:33.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 --2- 172.21.15.80:0/4072431418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc00d0930 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 --2- 172.21.15.80:0/4072431418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc00d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 --2- 172.21.15.80:0/4072431418 >> v2:172.21.15.87:3300/0 conn(0x7fcfc00b8cd0 0x7fcfc00b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 -- 172.21.15.80:0/4072431418 >> 172.21.15.80:0/4072431418 conn(0x7fcfc00d3ad0 msgr2=0x7fcfc00d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:33.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 -- 172.21.15.80:0/4072431418 shutdown_connections 2024-04-25T13:44:33.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.356+0000 7fcfc714b640 1 -- 172.21.15.80:0/4072431418 wait complete. 2024-04-25T13:44:33.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc714b640 1 Processor -- start 2024-04-25T13:44:33.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc714b640 1 -- start start 2024-04-25T13:44:33.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc714b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00b8cd0 0x7fcfc0160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc714b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc714b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc01fd470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc714b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0162130 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc714b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcfc0161fb0 con 0x7fcfc00b8cd0 2024-04-25T13:44:33.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc714b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcfc01622b0 con 0x7fcfc00cf690 2024-04-25T13:44:33.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc56e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc01fd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc56e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc01fd470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56910/0 (socket says 172.21.15.80:56910) 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc56e8640 1 -- 172.21.15.80:0/835397536 learned_addr learned my addr 172.21.15.80:0/835397536 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc4ee7640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00b8cd0 0x7fcfc0160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfbffff640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc56e8640 1 -- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 msgr2=0x7fcfc0155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc56e8640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc0155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc56e8640 1 -- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00b8cd0 msgr2=0x7fcfc0160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc56e8640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00b8cd0 0x7fcfc0160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfc56e8640 1 -- 172.21.15.80:0/835397536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfc0167790 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.360+0000 7fcfbffff640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfc56e8640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc01fd470 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcfb801d200 tx=0x7fcfb8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfbd7fa640 1 -- 172.21.15.80:0/835397536 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb80bc020 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfbd7fa640 1 -- 172.21.15.80:0/835397536 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcfb80b8690 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfc0167a70 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfbd7fa640 1 -- 172.21.15.80:0/835397536 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb8009510 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfc0168030 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfc0167500 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfbd7fa640 1 -- 172.21.15.80:0/835397536 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcfb80bf020 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfbd7fa640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcfac06f400 0x7fcfac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfbd7fa640 1 -- 172.21.15.80:0/835397536 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7fcfb8129ed0 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfc4ee7640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcfac06f400 0x7fcfac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.364+0000 7fcfc4ee7640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcfac06f400 0x7fcfac071910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b990 tx=0x7fcfb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.368+0000 7fcfbd7fa640 1 -- 172.21.15.80:0/835397536 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcfb80c6030 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.520+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcfc00b9220 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.520+0000 7fcfbd7fa640 1 -- 172.21.15.80:0/835397536 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) ==== 84+0+65 (secure 0 0 0) 0x7fcfb80165e0 con 0x7fcfc00cfaa0 2024-04-25T13:44:33.520 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:33.520 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:44:33.520 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:44:33.520 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:44:33.520 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:44:33.520 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:33.520 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcfac06f400 msgr2=0x7fcfac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcfac06f400 0x7fcfac071910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b990 tx=0x7fcfb0008040 comp rx=0 tx=0).stop 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 msgr2=0x7fcfc01fd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc01fd470 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fcfb801d200 tx=0x7fcfb8019700 comp rx=0 tx=0).stop 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 shutdown_connections 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcfac06f400 0x7fcfac071910 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcfc00cfaa0 0x7fcfc01fd470 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcfc00cf690 0x7fcfc0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 --2- 172.21.15.80:0/835397536 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcfc00b8cd0 0x7fcfc0160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 >> 172.21.15.80:0/835397536 conn(0x7fcfc00d3ad0 msgr2=0x7fcfc0070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 shutdown_connections 2024-04-25T13:44:33.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.524+0000 7fcfc714b640 1 -- 172.21.15.80:0/835397536 wait complete. 2024-04-25T13:44:33.542 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:44:33.542 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:44:33.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5e369640 1 Processor -- start 2024-04-25T13:44:33.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5e369640 1 -- start start 2024-04-25T13:44:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5e369640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4a580b0f10 0x7f4a580d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5e369640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4a58161d10 0x7f4a58162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5e369640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a580b1320 con 0x7f4a580ce2a0 2024-04-25T13:44:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5e369640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4a580b5820 con 0x7f4a58161d10 2024-04-25T13:44:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5e369640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4a580d13f0 con 0x7f4a580b0f10 2024-04-25T13:44:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5c906640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4a58161d10 0x7f4a58162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5c906640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4a58161d10 0x7f4a58162120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56912/0 (socket says 172.21.15.80:56912) 2024-04-25T13:44:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.692+0000 7f4a5c906640 1 -- 172.21.15.80:0/62704884 learned_addr learned my addr 172.21.15.80:0/62704884 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a57fff640 1 --2- 172.21.15.80:0/62704884 >> v2:172.21.15.87:3300/0 conn(0x7f4a580b0f10 0x7f4a580d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5c906640 1 -- 172.21.15.80:0/62704884 >> v2:172.21.15.87:3300/0 conn(0x7f4a580b0f10 msgr2=0x7f4a580d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5c906640 1 --2- 172.21.15.80:0/62704884 >> v2:172.21.15.87:3300/0 conn(0x7f4a580b0f10 0x7f4a580d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5c906640 1 -- 172.21.15.80:0/62704884 >> v1:172.21.15.87:6789/0 conn(0x7f4a580ce2a0 legacy=0x7f4a580ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5c906640 1 -- 172.21.15.80:0/62704884 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a580d1150 con 0x7f4a58161d10 2024-04-25T13:44:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a57fff640 1 --2- 172.21.15.80:0/62704884 >> v2:172.21.15.87:3300/0 conn(0x7f4a580b0f10 0x7f4a580d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:33.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5c906640 1 --2- 172.21.15.80:0/62704884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a58162120 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4a580713a0 tx=0x7f4a5001c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13c49f9954ad19c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a56ffd640 1 -- 172.21.15.80:0/62704884 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500a5580 con 0x7f4a58161d10 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a56ffd640 1 -- 172.21.15.80:0/62704884 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a500a5720 con 0x7f4a58161d10 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a56ffd640 1 -- 172.21.15.80:0/62704884 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500a5af0 con 0x7f4a58161d10 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5e369640 1 -- 172.21.15.80:0/62704884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 msgr2=0x7f4a58162120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5e369640 1 --2- 172.21.15.80:0/62704884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a58162120 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f4a580713a0 tx=0x7f4a5001c7d0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5e369640 1 -- 172.21.15.80:0/62704884 shutdown_connections 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5e369640 1 --2- 172.21.15.80:0/62704884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a58162120 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5e369640 1 --2- 172.21.15.80:0/62704884 >> v2:172.21.15.87:3300/0 conn(0x7f4a580b0f10 0x7f4a580d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5e369640 1 -- 172.21.15.80:0/62704884 >> 172.21.15.80:0/62704884 conn(0x7f4a580b35c0 msgr2=0x7f4a580b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:33.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5e369640 1 -- 172.21.15.80:0/62704884 shutdown_connections 2024-04-25T13:44:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.696+0000 7f4a5e369640 1 -- 172.21.15.80:0/62704884 wait complete. 2024-04-25T13:44:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 Processor -- start 2024-04-25T13:44:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 -- start start 2024-04-25T13:44:33.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a580b0f10 0x7f4a58151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a580ce2a0 0x7f4a58159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a5815d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5815e720 con 0x7f4a58161d10 2024-04-25T13:44:33.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a580b5300 con 0x7f4a580b0f10 2024-04-25T13:44:33.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a58162a70 con 0x7f4a580ce2a0 2024-04-25T13:44:33.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a5815d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a5815d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56922/0 (socket says 172.21.15.80:56922) 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 -- 172.21.15.80:0/3268403465 learned_addr learned my addr 172.21.15.80:0/3268403465 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 -- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a580ce2a0 msgr2=0x7f4a58159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a577fe640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a580ce2a0 0x7f4a58159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a57fff640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a580b0f10 0x7f4a58151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a580ce2a0 0x7f4a58159f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 -- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a580b0f10 msgr2=0x7f4a58151d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a580b0f10 0x7f4a58151d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 -- 172.21.15.80:0/3268403465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581575e0 con 0x7f4a58161d10 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a57fff640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a580b0f10 0x7f4a58151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5c906640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a5815d710 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4a500af040 tx=0x7f4a50019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a54ff9640 1 -- 172.21.15.80:0/3268403465 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500b8470 con 0x7f4a58161d10 2024-04-25T13:44:33.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a54ff9640 1 -- 172.21.15.80:0/3268403465 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a500b8630 con 0x7f4a58161d10 2024-04-25T13:44:33.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a581578c0 con 0x7f4a58161d10 2024-04-25T13:44:33.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a54ff9640 1 -- 172.21.15.80:0/3268403465 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500b8a20 con 0x7f4a58161d10 2024-04-25T13:44:33.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.700+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a58157e80 con 0x7f4a58161d10 2024-04-25T13:44:33.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.704+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a58157350 con 0x7f4a58161d10 2024-04-25T13:44:33.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.704+0000 7f4a54ff9640 1 -- 172.21.15.80:0/3268403465 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4a500c0030 con 0x7f4a58161d10 2024-04-25T13:44:33.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.704+0000 7f4a54ff9640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:33.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.704+0000 7f4a54ff9640 1 -- 172.21.15.80:0/3268403465 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 4857+0+0 (secure 0 0 0) 0x7f4a5012a7f0 con 0x7f4a58161d10 2024-04-25T13:44:33.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.704+0000 7f4a57fff640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:33.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.704+0000 7f4a57fff640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4a580d28d0 tx=0x7f4a48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:33.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.708+0000 7f4a54ff9640 1 -- 172.21.15.80:0/3268403465 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a500c7020 con 0x7f4a58161d10 2024-04-25T13:44:33.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:33.868+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4a5806a440 con 0x7f4a58161d10 2024-04-25T13:44:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.544+0000 7f4a54ff9640 1 -- 172.21.15.80:0/3268403465 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) ==== 138+0+0 (secure 0 0 0) 0x7f4a580b1320 con 0x7f4a58161d10 2024-04-25T13:44:34.543 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:44:34.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 msgr2=0x7f4a44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:34.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f4a580d28d0 tx=0x7f4a48008040 comp rx=0 tx=0).stop 2024-04-25T13:44:34.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 msgr2=0x7f4a5815d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:34.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a5815d710 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f4a500af040 tx=0x7f4a50019570 comp rx=0 tx=0).stop 2024-04-25T13:44:34.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 shutdown_connections 2024-04-25T13:44:34.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a58161d10 0x7f4a5815d710 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a580ce2a0 0x7f4a58159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 --2- 172.21.15.80:0/3268403465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a580b0f10 0x7f4a58151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 >> 172.21.15.80:0/3268403465 conn(0x7f4a580b35c0 msgr2=0x7f4a580d1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:34.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 shutdown_connections 2024-04-25T13:44:34.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.548+0000 7f4a5e369640 1 -- 172.21.15.80:0/3268403465 wait complete. 2024-04-25T13:44:34.577 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:44:34.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe05e18f640 1 Processor -- start 2024-04-25T13:44:34.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe05e18f640 1 -- start start 2024-04-25T13:44:34.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe05e18f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0580b9240 0x7fe0580cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:34.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe05e18f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580d04e0 0x7fe0580d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:34.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe05e18f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0580d0e30 con 0x7fe0580d1a60 2024-04-25T13:44:34.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe05e18f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe0580b6920 con 0x7fe0580b9240 2024-04-25T13:44:34.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe05e18f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe0580cfb20 con 0x7fe0580d04e0 2024-04-25T13:44:34.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe0577fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0580b9240 0x7fe0580cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:34.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe057fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe0580d1a60 0x7fe0580d1e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:60322/0 (socket says 172.21.15.80:60322) 2024-04-25T13:44:34.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe057fff640 1 -- 172.21.15.80:0/3574213340 learned_addr learned my addr 172.21.15.80:0/3574213340 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:34.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe056ffd640 1 --2- 172.21.15.80:0/3574213340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580d04e0 0x7fe0580d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:34.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe0577fe640 1 -- 172.21.15.80:0/3574213340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580d04e0 msgr2=0x7fe0580d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:34.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe0577fe640 1 --2- 172.21.15.80:0/3574213340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580d04e0 0x7fe0580d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe0577fe640 1 -- 172.21.15.80:0/3574213340 >> v1:172.21.15.80:6789/0 conn(0x7fe0580d1a60 legacy=0x7fe0580d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:34.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.740+0000 7fe0577fe640 1 -- 172.21.15.80:0/3574213340 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0580b6b70 con 0x7fe0580b9240 2024-04-25T13:44:34.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe0577fe640 1 --2- 172.21.15.80:0/3574213340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580b9240 0x7fe0580cf1b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe0480097a0 tx=0x7fe04801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d6838ad3543d70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:34.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe0567fc640 1 -- 172.21.15.80:0/3574213340 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0480b1070 con 0x7fe0580b9240 2024-04-25T13:44:34.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe0567fc640 1 -- 172.21.15.80:0/3574213340 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0480ac620 con 0x7fe0580b9240 2024-04-25T13:44:34.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe0567fc640 1 -- 172.21.15.80:0/3574213340 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0480ac9f0 con 0x7fe0580b9240 2024-04-25T13:44:34.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe05e18f640 1 -- 172.21.15.80:0/3574213340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580b9240 msgr2=0x7fe0580cf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:34.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe05e18f640 1 --2- 172.21.15.80:0/3574213340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580b9240 0x7fe0580cf1b0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe0480097a0 tx=0x7fe04801c3c0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe05e18f640 1 -- 172.21.15.80:0/3574213340 shutdown_connections 2024-04-25T13:44:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe05e18f640 1 --2- 172.21.15.80:0/3574213340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580d04e0 0x7fe0580d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe05e18f640 1 --2- 172.21.15.80:0/3574213340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580b9240 0x7fe0580cf1b0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe05e18f640 1 -- 172.21.15.80:0/3574213340 >> 172.21.15.80:0/3574213340 conn(0x7fe0580d3ad0 msgr2=0x7fe0580d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe05e18f640 1 -- 172.21.15.80:0/3574213340 shutdown_connections 2024-04-25T13:44:34.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.744+0000 7fe05e18f640 1 -- 172.21.15.80:0/3574213340 wait complete. 2024-04-25T13:44:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 Processor -- start 2024-04-25T13:44:34.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 -- start start 2024-04-25T13:44:34.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b9240 0x7fe058152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d04e0 0x7fe058154920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580d1a60 0x7fe058154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe05816b780 con 0x7fe0580d04e0 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0580cf900 con 0x7fe0580d1a60 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe05816b900 con 0x7fe0580b9240 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d04e0 0x7fe058154920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d04e0 0x7fe058154920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56944/0 (socket says 172.21.15.80:56944) 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 -- 172.21.15.80:0/2605075257 learned_addr learned my addr 172.21.15.80:0/2605075257 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe0577fe640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b9240 0x7fe058152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe057fff640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580d1a60 0x7fe058154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:34.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 -- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b9240 msgr2=0x7fe058152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b9240 0x7fe058152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 -- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580d1a60 msgr2=0x7fe058154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580d1a60 0x7fe058154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 -- 172.21.15.80:0/2605075257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0580cae60 con 0x7fe0580d04e0 2024-04-25T13:44:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe056ffd640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d04e0 0x7fe058154920 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00d5f0 tx=0x7fe04c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe03ffff640 1 -- 172.21.15.80:0/2605075257 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0125f0 con 0x7fe0580d04e0 2024-04-25T13:44:34.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe03ffff640 1 -- 172.21.15.80:0/2605075257 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04c019070 con 0x7fe0580d04e0 2024-04-25T13:44:34.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe058161170 con 0x7fe0580d04e0 2024-04-25T13:44:34.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe03ffff640 1 -- 172.21.15.80:0/2605075257 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0129b0 con 0x7fe0580d04e0 2024-04-25T13:44:34.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe058160a10 con 0x7fe0580d04e0 2024-04-25T13:44:34.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.748+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05806a3a0 con 0x7fe0580d04e0 2024-04-25T13:44:34.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.752+0000 7fe03ffff640 1 -- 172.21.15.80:0/2605075257 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe04c012b50 con 0x7fe0580d04e0 2024-04-25T13:44:34.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.752+0000 7fe03ffff640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f430 0x7fe044071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:34.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.752+0000 7fe03ffff640 1 -- 172.21.15.80:0/2605075257 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 5227+0+0 (secure 0 0 0) 0x7fe04c08feb0 con 0x7fe0580d04e0 2024-04-25T13:44:34.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.752+0000 7fe0577fe640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f430 0x7fe044071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:34.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.752+0000 7fe0577fe640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f430 0x7fe044071940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe0480ae000 tx=0x7fe048008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:34.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.756+0000 7fe03ffff640 1 -- 172.21.15.80:0/2605075257 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe04c05d0f0 con 0x7fe0580d04e0 2024-04-25T13:44:34.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:34.920+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe058074c30 con 0x7fe0580d04e0 2024-04-25T13:44:35.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.556+0000 7fe03ffff640 1 -- 172.21.15.80:0/2605075257 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) ==== 170+0+0 (secure 0 0 0) 0x7fe04c05c9f0 con 0x7fe0580d04e0 2024-04-25T13:44:35.556 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:44:35.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f430 msgr2=0x7fe044071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:35.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f430 0x7fe044071940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fe0480ae000 tx=0x7fe048008040 comp rx=0 tx=0).stop 2024-04-25T13:44:35.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d04e0 msgr2=0x7fe058154920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:35.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d04e0 0x7fe058154920 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00d5f0 tx=0x7fe04c00dab0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 shutdown_connections 2024-04-25T13:44:35.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f430 0x7fe044071940 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580d1a60 0x7fe058154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d04e0 0x7fe058154920 secure :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe04c00d5f0 tx=0x7fe04c00dab0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 --2- 172.21.15.80:0/2605075257 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b9240 0x7fe058152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 >> 172.21.15.80:0/2605075257 conn(0x7fe0580d3ad0 msgr2=0x7fe058070ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:35.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 shutdown_connections 2024-04-25T13:44:35.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.560+0000 7fe05e18f640 1 -- 172.21.15.80:0/2605075257 wait complete. 2024-04-25T13:44:35.591 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:44:35.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c32542640 1 Processor -- start 2024-04-25T13:44:35.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c32542640 1 -- start start 2024-04-25T13:44:35.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c32542640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c2c071e60 0x7f1c2c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c32542640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c161170 0x7f1c2c0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c32542640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c2c06fbf0 con 0x7f1c2c073c40 2024-04-25T13:44:35.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c32542640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1c2c0b1350 con 0x7f1c2c071e60 2024-04-25T13:44:35.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c32542640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c06f9a0 con 0x7f1c2c161170 2024-04-25T13:44:35.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c2b7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1c2c073c40 0x7f1c2c160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:60336/0 (socket says 172.21.15.80:60336) 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.740+0000 7f1c2b7fe640 1 -- 172.21.15.80:0/1759772320 learned_addr learned my addr 172.21.15.80:0/1759772320 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c30adf640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c161170 0x7f1c2c0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/1759772320 >> v2:172.21.15.87:3300/0 conn(0x7f1c2c071e60 0x7f1c2c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1120703870 0 0) 0x7f1c2c06fbf0 con 0x7f1c2c073c40 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1c18003530 con 0x7f1c2c073c40 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1278630584 0 0) 0x7f1c18003530 con 0x7f1c2c073c40 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c161170 msgr2=0x7f1c2c0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c161170 0x7f1c2c0b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 >> v2:172.21.15.87:3300/0 conn(0x7f1c2c071e60 msgr2=0x7f1c2c1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 --2- 172.21.15.80:0/1759772320 >> v2:172.21.15.87:3300/0 conn(0x7f1c2c071e60 0x7f1c2c1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c0b0a20 con 0x7f1c2c073c40 2024-04-25T13:44:35.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1c14001680 con 0x7f1c2c073c40 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 >> v1:172.21.15.80:6789/0 conn(0x7f1c2c073c40 legacy=0x7f1c2c160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c180039c0 0x7f1c18003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c18004810 0x7f1c18004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c18005160 0x7f1c1800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c0b1350 con 0x7f1c180039c0 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c06f9a0 con 0x7f1c18005160 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c1800bf10 con 0x7f1c18004810 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c180039c0 0x7f1c18003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2b7fe640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c18004810 0x7f1c18004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c30adf640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c18005160 0x7f1c1800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2bfff640 1 -- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c18004810 msgr2=0x7f1c18004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:35.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c18004810 0x7f1c18004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2bfff640 1 -- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c18005160 msgr2=0x7f1c1800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:35.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c18005160 0x7f1c1800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2bfff640 1 -- 172.21.15.80:0/1759772320 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c18003530 con 0x7f1c180039c0 2024-04-25T13:44:35.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c30adf640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c18005160 0x7f1c1800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c180039c0 0x7f1c18003e30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009ae0 tx=0x7f1c1c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605cd810dd68915b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0b2070 con 0x7f1c180039c0 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0a5570 con 0x7f1c180039c0 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.744+0000 7f1c2affd640 1 -- 172.21.15.80:0/1759772320 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0096b0 con 0x7f1c180039c0 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- 172.21.15.80:0/1759772320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c180039c0 msgr2=0x7f1c18003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 --2- 172.21.15.80:0/1759772320 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c180039c0 0x7f1c18003e30 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c009ae0 tx=0x7f1c1c01cab0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c30adf640 1 -- 172.21.15.80:0/1759772320 reap_dead start 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- 172.21.15.80:0/1759772320 shutdown_connections 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- 172.21.15.80:0/1759772320 >> 172.21.15.80:0/1759772320 conn(0x7f1c2c0b4160 msgr2=0x7f1c2c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- 172.21.15.80:0/1759772320 shutdown_connections 2024-04-25T13:44:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- 172.21.15.80:0/1759772320 wait complete. 2024-04-25T13:44:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 Processor -- start 2024-04-25T13:44:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- start start 2024-04-25T13:44:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c180039c0 0x7f1c2c14fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c0d0fc0 0x7f1c2c0d0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c14b400 0x7f1c2c152c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c161170 con 0x7f1c2c14b400 2024-04-25T13:44:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c2c074050 con 0x7f1c180039c0 2024-04-25T13:44:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c32542640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2c1612f0 con 0x7f1c2c0d0fc0 2024-04-25T13:44:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c30adf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c14b400 0x7f1c2c152c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c30adf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c14b400 0x7f1c2c152c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56968/0 (socket says 172.21.15.80:56968) 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.748+0000 7f1c30adf640 1 -- 172.21.15.80:0/3385905575 learned_addr learned my addr 172.21.15.80:0/3385905575 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c180039c0 0x7f1c2c14fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c2b7fe640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c0d0fc0 0x7f1c2c0d0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c30adf640 1 -- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c0d0fc0 msgr2=0x7f1c2c0d0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c30adf640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c0d0fc0 0x7f1c2c0d0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c30adf640 1 -- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c180039c0 msgr2=0x7f1c2c14fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c30adf640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c180039c0 0x7f1c2c14fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c30adf640 1 -- 172.21.15.80:0/3385905575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2c0cf540 con 0x7f1c2c14b400 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c180039c0 0x7f1c2c14fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c30adf640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c14b400 0x7f1c2c152c50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c161a70 tx=0x7f1c2400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c28ff9640 1 -- 172.21.15.80:0/3385905575 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c2401c070 con 0x7f1c2c14b400 2024-04-25T13:44:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c28ff9640 1 -- 172.21.15.80:0/3385905575 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c24017400 con 0x7f1c2c14b400 2024-04-25T13:44:35.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c2c0cf820 con 0x7f1c2c14b400 2024-04-25T13:44:35.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c28ff9640 1 -- 172.21.15.80:0/3385905575 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c240177f0 con 0x7f1c2c14b400 2024-04-25T13:44:35.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c2c14b810 con 0x7f1c2c14b400 2024-04-25T13:44:35.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2c06a3a0 con 0x7f1c2c14b400 2024-04-25T13:44:35.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c28ff9640 1 -- 172.21.15.80:0/3385905575 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1c2401f070 con 0x7f1c2c14b400 2024-04-25T13:44:35.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c28ff9640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c0c06f430 0x7f1c0c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:35.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.752+0000 7f1c28ff9640 1 -- 172.21.15.80:0/3385905575 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5609+0+0 (secure 0 0 0) 0x7f1c2408f830 con 0x7f1c2c14b400 2024-04-25T13:44:35.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.756+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c0c06f430 0x7f1c0c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:35.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.756+0000 7f1c2bfff640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c0c06f430 0x7f1c0c071940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0af000 tx=0x7f1c1c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.756+0000 7f1c28ff9640 1 -- 172.21.15.80:0/3385905575 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c24026080 con 0x7f1c2c14b400 2024-04-25T13:44:35.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:35.956+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f1c2c160200 con 0x7f1c2c14b400 2024-04-25T13:44:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.608+0000 7f1c28ff9640 1 -- 172.21.15.80:0/3385905575 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v37) ==== 174+0+0 (secure 0 0 0) 0x7f1c24012070 con 0x7f1c2c14b400 2024-04-25T13:44:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T13:44:36.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c0c06f430 msgr2=0x7f1c0c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:36.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c0c06f430 0x7f1c0c071940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0af000 tx=0x7f1c1c0046d0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c14b400 msgr2=0x7f1c2c152c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c14b400 0x7f1c2c152c50 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c161a70 tx=0x7f1c2400f4a0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 shutdown_connections 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c0c06f430 0x7f1c0c071940 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c2c14b400 0x7f1c2c152c50 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c2c0d0fc0 0x7f1c2c0d0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 --2- 172.21.15.80:0/3385905575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c180039c0 0x7f1c2c14fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 >> 172.21.15.80:0/3385905575 conn(0x7f1c2c0b4160 msgr2=0x7f1c2c15d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 shutdown_connections 2024-04-25T13:44:36.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.612+0000 7f1c32542640 1 -- 172.21.15.80:0/3385905575 wait complete. 2024-04-25T13:44:36.637 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:36.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.784+0000 7fd31f288640 1 Processor -- start 2024-04-25T13:44:36.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.784+0000 7fd31f288640 1 -- start start 2024-04-25T13:44:36.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31f288640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd318161f90 0x7fd3180cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:36.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31f288640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 0x7fd31815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:36.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31f288640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd3180d0d70 con 0x7fd3180d0510 2024-04-25T13:44:36.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31f288640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd3180d1050 con 0x7fd318161f90 2024-04-25T13:44:36.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31f288640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd31806dca0 con 0x7fd31815ec40 2024-04-25T13:44:36.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31d024640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd318161f90 0x7fd3180cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:36.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31d024640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd318161f90 0x7fd3180cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56978/0 (socket says 172.21.15.80:56978) 2024-04-25T13:44:36.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31d024640 1 -- 172.21.15.80:0/1163295957 learned_addr learned my addr 172.21.15.80:0/1163295957 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:36.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31d825640 1 --2- 172.21.15.80:0/1163295957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 0x7fd31815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:36.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31d024640 1 -- 172.21.15.80:0/1163295957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 msgr2=0x7fd31815f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:36.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31d024640 1 --2- 172.21.15.80:0/1163295957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 0x7fd31815f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31d024640 1 -- 172.21.15.80:0/1163295957 >> v1:172.21.15.87:6789/0 conn(0x7fd3180d0510 legacy=0x7fd31815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:36.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.788+0000 7fd31d024640 1 -- 172.21.15.80:0/1163295957 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31806df80 con 0x7fd318161f90 2024-04-25T13:44:36.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31d024640 1 --2- 172.21.15.80:0/1163295957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd318161f90 0x7fd3180cffd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd314009870 tx=0x7fd31401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da8cc70d36b4884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:36.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd30ffff640 1 -- 172.21.15.80:0/1163295957 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140b1070 con 0x7fd318161f90 2024-04-25T13:44:36.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd30ffff640 1 -- 172.21.15.80:0/1163295957 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3140ac600 con 0x7fd318161f90 2024-04-25T13:44:36.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd30ffff640 1 -- 172.21.15.80:0/1163295957 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140ac9d0 con 0x7fd318161f90 2024-04-25T13:44:36.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31f288640 1 -- 172.21.15.80:0/1163295957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd318161f90 msgr2=0x7fd3180cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:36.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31f288640 1 --2- 172.21.15.80:0/1163295957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd318161f90 0x7fd3180cffd0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fd314009870 tx=0x7fd31401c470 comp rx=0 tx=0).stop 2024-04-25T13:44:36.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31f288640 1 -- 172.21.15.80:0/1163295957 shutdown_connections 2024-04-25T13:44:36.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31f288640 1 --2- 172.21.15.80:0/1163295957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 0x7fd31815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31f288640 1 --2- 172.21.15.80:0/1163295957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd318161f90 0x7fd3180cffd0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31f288640 1 -- 172.21.15.80:0/1163295957 >> 172.21.15.80:0/1163295957 conn(0x7fd31806d080 msgr2=0x7fd3180b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:36.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31f288640 1 -- 172.21.15.80:0/1163295957 shutdown_connections 2024-04-25T13:44:36.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.792+0000 7fd31f288640 1 -- 172.21.15.80:0/1163295957 wait complete. 2024-04-25T13:44:36.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 Processor -- start 2024-04-25T13:44:36.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 -- start start 2024-04-25T13:44:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3180d0510 0x7fd31815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 0x7fd3180ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd318161f90 0x7fd3180ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd31806f4b0 con 0x7fd3180d0510 2024-04-25T13:44:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd31806f630 con 0x7fd318161f90 2024-04-25T13:44:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd3180d0920 con 0x7fd31815ec40 2024-04-25T13:44:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3180d0510 0x7fd31815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3180d0510 0x7fd31815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56982/0 (socket says 172.21.15.80:56982) 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 -- 172.21.15.80:0/1902571463 learned_addr learned my addr 172.21.15.80:0/1902571463 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d825640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd318161f90 0x7fd3180ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 -- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 msgr2=0x7fd3180ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31c823640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 0x7fd3180ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 0x7fd3180ac9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 -- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd318161f90 msgr2=0x7fd3180ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd318161f90 0x7fd3180ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 -- 172.21.15.80:0/1902571463 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3180ab4a0 con 0x7fd3180d0510 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d825640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd318161f90 0x7fd3180ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31d024640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3180d0510 0x7fd31815bf30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd31401a100 tx=0x7fd31401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd30dffb640 1 -- 172.21.15.80:0/1902571463 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140bb020 con 0x7fd3180d0510 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd30dffb640 1 -- 172.21.15.80:0/1902571463 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3140b7420 con 0x7fd3180d0510 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 -- 172.21.15.80:0/1902571463 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3180ab910 con 0x7fd3180d0510 2024-04-25T13:44:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd30dffb640 1 -- 172.21.15.80:0/1902571463 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3140b1040 con 0x7fd3180d0510 2024-04-25T13:44:36.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd31f288640 1 -- 172.21.15.80:0/1902571463 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3180abed0 con 0x7fd3180d0510 2024-04-25T13:44:36.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.796+0000 7fd3037fe640 1 -- 172.21.15.80:0/1902571463 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2f00052a0 con 0x7fd3180d0510 2024-04-25T13:44:36.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.800+0000 7fd30dffb640 1 -- 172.21.15.80:0/1902571463 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd3140be020 con 0x7fd3180d0510 2024-04-25T13:44:36.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.800+0000 7fd30dffb640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2fc06f400 0x7fd2fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:36.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.800+0000 7fd30dffb640 1 -- 172.21.15.80:0/1902571463 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7fd314129870 con 0x7fd3180d0510 2024-04-25T13:44:36.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.800+0000 7fd31c823640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2fc06f400 0x7fd2fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:36.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.800+0000 7fd31c823640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2fc06f400 0x7fd2fc071910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd318164b80 tx=0x7fd308009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:36.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.804+0000 7fd30dffb640 1 -- 172.21.15.80:0/1902571463 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd3140c5030 con 0x7fd3180d0510 2024-04-25T13:44:36.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.964+0000 7fd3037fe640 1 -- 172.21.15.80:0/1902571463 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2f0005860 con 0x7fd3180d0510 2024-04-25T13:44:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd30dffb640 1 -- 172.21.15.80:0/1902571463 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16609 (secure 0 0 0) 0x7fd314016150 con 0x7fd3180d0510 2024-04-25T13:44:36.966 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:36.967 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:44:35.958485+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:44:33.879562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T13:44:34.932724+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:36.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 -- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2fc06f400 msgr2=0x7fd2fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:36.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2fc06f400 0x7fd2fc071910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd318164b80 tx=0x7fd308009040 comp rx=0 tx=0).stop 2024-04-25T13:44:36.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 -- 172.21.15.80:0/1902571463 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3180d0510 msgr2=0x7fd31815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:36.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3180d0510 0x7fd31815bf30 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fd31401a100 tx=0x7fd31401ad10 comp rx=0 tx=0).stop 2024-04-25T13:44:36.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 -- 172.21.15.80:0/1902571463 shutdown_connections 2024-04-25T13:44:36.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2fc06f400 0x7fd2fc071910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd318161f90 0x7fd3180ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd31815ec40 0x7fd3180ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 --2- 172.21.15.80:0/1902571463 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd3180d0510 0x7fd31815bf30 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:36.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 -- 172.21.15.80:0/1902571463 >> 172.21.15.80:0/1902571463 conn(0x7fd31806d080 msgr2=0x7fd31815dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:36.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 -- 172.21.15.80:0/1902571463 shutdown_connections 2024-04-25T13:44:36.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:36.968+0000 7fd3037fe640 1 -- 172.21.15.80:0/1902571463 wait complete. 2024-04-25T13:44:36.989 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:37.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a3a87640 1 Processor -- start 2024-04-25T13:44:37.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a3a87640 1 -- start start 2024-04-25T13:44:37.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a3a87640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff69c161240 0x7ff69c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a3a87640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff69c161650 0x7ff69c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a3a87640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff69c06fcf0 con 0x7ff69c06f5a0 2024-04-25T13:44:37.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a3a87640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff69c073f10 con 0x7ff69c161650 2024-04-25T13:44:37.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a3a87640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff69c0b0c60 con 0x7ff69c161240 2024-04-25T13:44:37.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a2024640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff69c161650 0x7ff69c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a1823640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff69c06f5a0 0x7ff69c160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:60346/0 (socket says 172.21.15.80:60346) 2024-04-25T13:44:37.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a1823640 1 -- 172.21.15.80:0/114768735 learned_addr learned my addr 172.21.15.80:0/114768735 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:37.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a1022640 1 --2- 172.21.15.80:0/114768735 >> v2:172.21.15.87:3300/0 conn(0x7ff69c161240 0x7ff69c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a2024640 1 -- 172.21.15.80:0/114768735 >> v2:172.21.15.87:3300/0 conn(0x7ff69c161240 msgr2=0x7ff69c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:37.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a2024640 1 --2- 172.21.15.80:0/114768735 >> v2:172.21.15.87:3300/0 conn(0x7ff69c161240 0x7ff69c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a2024640 1 -- 172.21.15.80:0/114768735 >> v1:172.21.15.80:6789/0 conn(0x7ff69c06f5a0 legacy=0x7ff69c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a2024640 1 -- 172.21.15.80:0/114768735 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c0b0f40 con 0x7ff69c161650 2024-04-25T13:44:37.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a0821640 1 -- 172.21.15.80:0/114768735 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4045723901 0 0) 0x7ff69c06fcf0 con 0x7ff69c06f5a0 2024-04-25T13:44:37.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.144+0000 7ff6a1022640 1 --2- 172.21.15.80:0/114768735 >> v2:172.21.15.87:3300/0 conn(0x7ff69c161240 0x7ff69c0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:37.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a2024640 1 --2- 172.21.15.80:0/114768735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c161650 0x7ff69c0728e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff69c06ff70 tx=0x7ff69401c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be7fa930647d8a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:37.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a0821640 1 -- 172.21.15.80:0/114768735 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940b2070 con 0x7ff69c161650 2024-04-25T13:44:37.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a0821640 1 -- 172.21.15.80:0/114768735 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6940a57e0 con 0x7ff69c161650 2024-04-25T13:44:37.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a0821640 1 -- 172.21.15.80:0/114768735 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6940a5bb0 con 0x7ff69c161650 2024-04-25T13:44:37.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 -- 172.21.15.80:0/114768735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c161650 msgr2=0x7ff69c0728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/114768735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c161650 0x7ff69c0728e0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff69c06ff70 tx=0x7ff69401c8a0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 -- 172.21.15.80:0/114768735 shutdown_connections 2024-04-25T13:44:37.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/114768735 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c161650 0x7ff69c0728e0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/114768735 >> v2:172.21.15.87:3300/0 conn(0x7ff69c161240 0x7ff69c0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 -- 172.21.15.80:0/114768735 >> 172.21.15.80:0/114768735 conn(0x7ff69c06d080 msgr2=0x7ff69c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:37.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 -- 172.21.15.80:0/114768735 shutdown_connections 2024-04-25T13:44:37.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 -- 172.21.15.80:0/114768735 wait complete. 2024-04-25T13:44:37.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.148+0000 7ff6a3a87640 1 Processor -- start 2024-04-25T13:44:37.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 -- start start 2024-04-25T13:44:37.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c06f5a0 0x7ff69c154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff69c161240 0x7ff69c1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff69c161650 0x7ff69c0d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c16a260 con 0x7ff69c06f5a0 2024-04-25T13:44:37.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c16a0e0 con 0x7ff69c161650 2024-04-25T13:44:37.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff69c16a3e0 con 0x7ff69c161240 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c06f5a0 0x7ff69c154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c06f5a0 0x7ff69c154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:56998/0 (socket says 172.21.15.80:56998) 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 -- 172.21.15.80:0/2501221880 learned_addr learned my addr 172.21.15.80:0/2501221880 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 -- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff69c161240 msgr2=0x7ff69c1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1022640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff69c161240 0x7ff69c1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a2024640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff69c161650 0x7ff69c0d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff69c161240 0x7ff69c1412d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 -- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff69c161650 msgr2=0x7ff69c0d0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff69c161650 0x7ff69c0d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 -- 172.21.15.80:0/2501221880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c15eb00 con 0x7ff69c06f5a0 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a1823640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c06f5a0 0x7ff69c154e40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c450 tx=0x7ff69800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6927fc640 1 -- 172.21.15.80:0/2501221880 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff698018070 con 0x7ff69c06f5a0 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6927fc640 1 -- 172.21.15.80:0/2501221880 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6980074e0 con 0x7ff69c06f5a0 2024-04-25T13:44:37.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c15da60 con 0x7ff69c06f5a0 2024-04-25T13:44:37.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6927fc640 1 -- 172.21.15.80:0/2501221880 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6980049f0 con 0x7ff69c06f5a0 2024-04-25T13:44:37.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c15e020 con 0x7ff69c06f5a0 2024-04-25T13:44:37.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c06a3a0 con 0x7ff69c06f5a0 2024-04-25T13:44:37.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.152+0000 7ff6927fc640 1 -- 172.21.15.80:0/2501221880 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff69801b070 con 0x7ff69c06f5a0 2024-04-25T13:44:37.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.156+0000 7ff6927fc640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff68006f400 0x7ff680071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.156+0000 7ff6927fc640 1 -- 172.21.15.80:0/2501221880 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7ff69808e590 con 0x7ff69c06f5a0 2024-04-25T13:44:37.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.156+0000 7ff6a1022640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff68006f400 0x7ff680071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.156+0000 7ff6a1022640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff68006f400 0x7ff680071910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006fd0 tx=0x7ff68c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:37.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.160+0000 7ff6927fc640 1 -- 172.21.15.80:0/2501221880 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff69c15eb00 con 0x7ff69c06f5a0 2024-04-25T13:44:37.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.320+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff69c0739c0 con 0x7ff69c06f5a0 2024-04-25T13:44:37.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6927fc640 1 -- 172.21.15.80:0/2501221880 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+16609 (secure 0 0 0) 0x7ff69805b510 con 0x7ff69c06f5a0 2024-04-25T13:44:37.323 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:37.323 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:44:35.958485+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:44:33.879562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T13:44:34.932724+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:37.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff68006f400 msgr2=0x7ff680071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff68006f400 0x7ff680071910 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff68c006fd0 tx=0x7ff68c008040 comp rx=0 tx=0).stop 2024-04-25T13:44:37.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c06f5a0 msgr2=0x7ff69c154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c06f5a0 0x7ff69c154e40 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7ff69800c450 tx=0x7ff69800c910 comp rx=0 tx=0).stop 2024-04-25T13:44:37.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 shutdown_connections 2024-04-25T13:44:37.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff68006f400 0x7ff680071910 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff69c161650 0x7ff69c0d0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff69c161240 0x7ff69c1412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.324+0000 7ff6a3a87640 1 --2- 172.21.15.80:0/2501221880 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff69c06f5a0 0x7ff69c154e40 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.328+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 >> 172.21.15.80:0/2501221880 conn(0x7ff69c06d080 msgr2=0x7ff69c15c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:37.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.328+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 shutdown_connections 2024-04-25T13:44:37.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.328+0000 7ff6a3a87640 1 -- 172.21.15.80:0/2501221880 wait complete. 2024-04-25T13:44:37.349 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:44:37.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a07bb3640 1 Processor -- start 2024-04-25T13:44:37.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a07bb3640 1 -- start start 2024-04-25T13:44:37.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a07bb3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5a00071e60 0x7f5a001574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a07bb3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5a00161170 0x7f5a000b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a07bb3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5a0006fbf0 con 0x7f5a00073c40 2024-04-25T13:44:37.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a07bb3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5a000b1350 con 0x7f5a00071e60 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a07bb3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5a0006f9a0 con 0x7f5a00161170 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0594f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5a00071e60 0x7f5a001574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0594f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5a00071e60 0x7f5a001574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57004/0 (socket says 172.21.15.80:57004) 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0594f640 1 -- 172.21.15.80:0/1922873190 learned_addr learned my addr 172.21.15.80:0/1922873190 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a06150640 1 --2- 172.21.15.80:0/1922873190 >> v2:172.21.15.87:3300/0 conn(0x7f5a00161170 0x7f5a000b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0494d640 1 -- 172.21.15.80:0/1922873190 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3232308734 0 0) 0x7f5a0006fbf0 con 0x7f5a00073c40 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0494d640 1 -- 172.21.15.80:0/1922873190 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f59e8003530 con 0x7f5a00073c40 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0594f640 1 -- 172.21.15.80:0/1922873190 >> v2:172.21.15.87:3300/0 conn(0x7f5a00161170 msgr2=0x7f5a000b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0594f640 1 --2- 172.21.15.80:0/1922873190 >> v2:172.21.15.87:3300/0 conn(0x7f5a00161170 0x7f5a000b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0594f640 1 -- 172.21.15.80:0/1922873190 >> v1:172.21.15.80:6789/0 conn(0x7f5a00073c40 legacy=0x7f5a00160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.504+0000 7f5a0594f640 1 -- 172.21.15.80:0/1922873190 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a000b0a20 con 0x7f5a00071e60 2024-04-25T13:44:37.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a0594f640 1 --2- 172.21.15.80:0/1922873190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00071e60 0x7f5a001574c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5a00161a50 tx=0x7f59f801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add9cfcbb64df2d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:37.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a0494d640 1 -- 172.21.15.80:0/1922873190 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80ab610 con 0x7f5a00071e60 2024-04-25T13:44:37.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a0494d640 1 -- 172.21.15.80:0/1922873190 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f80b0070 con 0x7f5a00071e60 2024-04-25T13:44:37.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a0494d640 1 -- 172.21.15.80:0/1922873190 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f80ab9d0 con 0x7f5a00071e60 2024-04-25T13:44:37.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a07bb3640 1 -- 172.21.15.80:0/1922873190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00071e60 msgr2=0x7f5a001574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/1922873190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00071e60 0x7f5a001574c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f5a00161a50 tx=0x7f59f801c520 comp rx=0 tx=0).stop 2024-04-25T13:44:37.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a07bb3640 1 -- 172.21.15.80:0/1922873190 shutdown_connections 2024-04-25T13:44:37.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/1922873190 >> v2:172.21.15.87:3300/0 conn(0x7f5a00161170 0x7f5a000b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/1922873190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00071e60 0x7f5a001574c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a07bb3640 1 -- 172.21.15.80:0/1922873190 >> 172.21.15.80:0/1922873190 conn(0x7f5a000b4160 msgr2=0x7f5a000d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:37.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a07bb3640 1 -- 172.21.15.80:0/1922873190 shutdown_connections 2024-04-25T13:44:37.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.508+0000 7f5a07bb3640 1 -- 172.21.15.80:0/1922873190 wait complete. 2024-04-25T13:44:37.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 Processor -- start 2024-04-25T13:44:37.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 -- start start 2024-04-25T13:44:37.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a00071e60 0x7f5a00154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00073c40 0x7f5a000aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a000acd60 0x7f5a000ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00169bb0 con 0x7f5a00073c40 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5a00074050 con 0x7f5a00071e60 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5a00169d30 con 0x7f5a000acd60 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00073c40 0x7f5a000aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00073c40 0x7f5a000aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57020/0 (socket says 172.21.15.80:57020) 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 -- 172.21.15.80:0/2434434175 learned_addr learned my addr 172.21.15.80:0/2434434175 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 -- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a000acd60 msgr2=0x7f5a000ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0594f640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a00071e60 0x7f5a00154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a06150640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a000acd60 0x7f5a000ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a000acd60 0x7f5a000ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 -- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a00071e60 msgr2=0x7f5a00154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a00071e60 0x7f5a00154ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 -- 172.21.15.80:0/2434434175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a001679d0 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a0514e640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00073c40 0x7f5a000aee00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f59f400c460 tx=0x7f59f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f59fe7fc640 1 -- 172.21.15.80:0/2434434175 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f40052d0 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f59fe7fc640 1 -- 172.21.15.80:0/2434434175 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59f4005490 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f59fe7fc640 1 -- 172.21.15.80:0/2434434175 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f4012850 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a001574c0 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a000af8a0 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.512+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0006a3a0 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.516+0000 7f59fe7fc640 1 -- 172.21.15.80:0/2434434175 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f59f40129f0 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.516+0000 7f59fe7fc640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59ec06f400 0x7f59ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.516+0000 7f59fe7fc640 1 -- 172.21.15.80:0/2434434175 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5709+0+0 (secure 0 0 0) 0x7f59f408edd0 con 0x7f5a00073c40 2024-04-25T13:44:37.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.516+0000 7f5a0594f640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59ec06f400 0x7f59ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:37.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.516+0000 7f5a0594f640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59ec06f400 0x7f59ec071910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f59f80af000 tx=0x7f59f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:37.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.520+0000 7f59fe7fc640 1 -- 172.21.15.80:0/2434434175 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59f405bde0 con 0x7f5a00073c40 2024-04-25T13:44:37.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:37.712+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5a000b4cb0 con 0x7f5a00073c40 2024-04-25T13:44:38.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.640+0000 7f59fe7fc640 1 -- 172.21.15.80:0/2434434175 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v40) ==== 118+0+0 (secure 0 0 0) 0x7f59f405b6e0 con 0x7f5a00073c40 2024-04-25T13:44:38.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59ec06f400 msgr2=0x7f59ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:38.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59ec06f400 0x7f59ec071910 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f59f80af000 tx=0x7f59f8008040 comp rx=0 tx=0).stop 2024-04-25T13:44:38.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00073c40 msgr2=0x7f5a000aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00073c40 0x7f5a000aee00 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f59f400c460 tx=0x7f59f400c920 comp rx=0 tx=0).stop 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 shutdown_connections 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f59ec06f400 0x7f59ec071910 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5a000acd60 0x7f5a000ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5a00073c40 0x7f5a000aee00 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 --2- 172.21.15.80:0/2434434175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5a00071e60 0x7f5a00154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 >> 172.21.15.80:0/2434434175 conn(0x7f5a000b4160 msgr2=0x7f5a001672c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 shutdown_connections 2024-04-25T13:44:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.644+0000 7f5a07bb3640 1 -- 172.21.15.80:0/2434434175 wait complete. 2024-04-25T13:44:38.668 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:44:38.668 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:44:38.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f3596b3b640 1 Processor -- start 2024-04-25T13:44:38.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f3596b3b640 1 -- start start 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f3596b3b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35901574c0 0x7f3590073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f3596b3b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f359006fbf0 con 0x7f3590071e60 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f3596b3b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35900b0a20 con 0x7f3590161170 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f3596b3b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f35900b1350 con 0x7f35901574c0 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f35950d8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35901574c0 0x7f3590073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f35950d8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35901574c0 0x7f3590073c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57032/0 (socket says 172.21.15.80:57032) 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f35950d8640 1 -- 172.21.15.80:0/3670099777 learned_addr learned my addr 172.21.15.80:0/3670099777 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f358f7fe640 1 -- 172.21.15.80:0/3670099777 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3638118896 0 0) 0x7f359006fbf0 con 0x7f3590071e60 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f358f7fe640 1 -- 172.21.15.80:0/3670099777 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3578003530 con 0x7f3590071e60 2024-04-25T13:44:38.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f35950d8640 1 -- 172.21.15.80:0/3670099777 >> v1:172.21.15.87:6789/0 conn(0x7f3590161170 legacy=0x7f35900b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:38.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f35950d8640 1 -- 172.21.15.80:0/3670099777 >> v1:172.21.15.80:6789/0 conn(0x7f3590071e60 legacy=0x7f3590160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:38.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.820+0000 7f35950d8640 1 -- 172.21.15.80:0/3670099777 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35900b0d00 con 0x7f35901574c0 2024-04-25T13:44:38.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f35950d8640 1 --2- 172.21.15.80:0/3670099777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35901574c0 0x7f3590073c40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f35900b14d0 tx=0x7f358801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c5646a0f73f8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:38.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f358f7fe640 1 -- 172.21.15.80:0/3670099777 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880b0070 con 0x7f35901574c0 2024-04-25T13:44:38.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f358f7fe640 1 -- 172.21.15.80:0/3670099777 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35880ab600 con 0x7f35901574c0 2024-04-25T13:44:38.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f358f7fe640 1 -- 172.21.15.80:0/3670099777 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880ab9b0 con 0x7f35901574c0 2024-04-25T13:44:38.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 -- 172.21.15.80:0/3670099777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35901574c0 msgr2=0x7f3590073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:38.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 --2- 172.21.15.80:0/3670099777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35901574c0 0x7f3590073c40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f35900b14d0 tx=0x7f358801c540 comp rx=0 tx=0).stop 2024-04-25T13:44:38.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 -- 172.21.15.80:0/3670099777 shutdown_connections 2024-04-25T13:44:38.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 --2- 172.21.15.80:0/3670099777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35901574c0 0x7f3590073c40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:38.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 -- 172.21.15.80:0/3670099777 >> 172.21.15.80:0/3670099777 conn(0x7f35900b4160 msgr2=0x7f35900d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:38.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 -- 172.21.15.80:0/3670099777 shutdown_connections 2024-04-25T13:44:38.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 -- 172.21.15.80:0/3670099777 wait complete. 2024-04-25T13:44:38.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 Processor -- start 2024-04-25T13:44:38.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.824+0000 7f3596b3b640 1 -- start start 2024-04-25T13:44:38.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3590071e60 0x7f359014c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35901574c0 0x7f3590155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3590155670 0x7f359014cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f35900b56c0 con 0x7f3590155670 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f359016b7e0 con 0x7f35901574c0 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f359014e280 con 0x7f3590071e60 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3590155670 0x7f359014cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3590155670 0x7f359014cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57042/0 (socket says 172.21.15.80:57042) 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 -- 172.21.15.80:0/3147316205 learned_addr learned my addr 172.21.15.80:0/3147316205 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f358ffff640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35901574c0 0x7f3590155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 -- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3590071e60 msgr2=0x7f359014c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35948d7640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3590071e60 0x7f359014c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3590071e60 0x7f359014c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:38.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 -- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35901574c0 msgr2=0x7f3590155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35901574c0 0x7f3590155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 -- 172.21.15.80:0/3147316205 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359015f270 con 0x7f3590155670 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f358ffff640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35901574c0 0x7f3590155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35948d7640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3590071e60 0x7f359014c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f35950d8640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3590155670 0x7f359014cfe0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f35880ae040 tx=0x7f358801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f358d7fa640 1 -- 172.21.15.80:0/3147316205 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880b0040 con 0x7f3590155670 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f358d7fa640 1 -- 172.21.15.80:0/3147316205 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35880bb070 con 0x7f3590155670 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3590155a80 con 0x7f3590155670 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f358d7fa640 1 -- 172.21.15.80:0/3147316205 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35880abbd0 con 0x7f3590155670 2024-04-25T13:44:38.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3590152090 con 0x7f3590155670 2024-04-25T13:44:38.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359006a3a0 con 0x7f3590155670 2024-04-25T13:44:38.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.828+0000 7f358d7fa640 1 -- 172.21.15.80:0/3147316205 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f35880ab4c0 con 0x7f3590155670 2024-04-25T13:44:38.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.832+0000 7f358d7fa640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357c06f400 0x7f357c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:38.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.832+0000 7f358d7fa640 1 -- 172.21.15.80:0/3147316205 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f3588128e80 con 0x7f3590155670 2024-04-25T13:44:38.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.832+0000 7f35948d7640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357c06f400 0x7f357c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:38.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.832+0000 7f35948d7640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357c06f400 0x7f357c071910 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3580001570 tx=0x7f3580009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:38.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:38.836+0000 7f358d7fa640 1 -- 172.21.15.80:0/3147316205 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3590155a80 con 0x7f3590155670 2024-04-25T13:44:39.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.016+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f35900b33f0 con 0x7f3590155670 2024-04-25T13:44:39.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.640+0000 7f358d7fa640 1 -- 172.21.15.80:0/3147316205 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v41) ==== 122+0+0 (secure 0 0 0) 0x7f3588016130 con 0x7f3590155670 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357c06f400 msgr2=0x7f357c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357c06f400 0x7f357c071910 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3580001570 tx=0x7f3580009040 comp rx=0 tx=0).stop 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3590155670 msgr2=0x7f359014cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3590155670 0x7f359014cfe0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f35880ae040 tx=0x7f358801ad80 comp rx=0 tx=0).stop 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 shutdown_connections 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357c06f400 0x7f357c071910 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3590155670 0x7f359014cfe0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35901574c0 0x7f3590155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 --2- 172.21.15.80:0/3147316205 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3590071e60 0x7f359014c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 >> 172.21.15.80:0/3147316205 conn(0x7f35900b4160 msgr2=0x7f359015d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:39.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 shutdown_connections 2024-04-25T13:44:39.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.644+0000 7f3596b3b640 1 -- 172.21.15.80:0/3147316205 wait complete. 2024-04-25T13:44:39.669 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:44:39.670 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:39.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d7584640 1 Processor -- start 2024-04-25T13:44:39.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d7584640 1 -- start start 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d7584640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 0x7f13d00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d7584640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13d015ec40 0x7f13d015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d7584640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d00d0d70 con 0x7f13d00d0510 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d7584640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f13d00d1050 con 0x7f13d015ec40 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d7584640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13d006dca0 con 0x7f13d0161f90 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d4b1f640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f13d00d0510 0x7f13d015e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:60376/0 (socket says 172.21.15.80:60376) 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d4b1f640 1 -- 172.21.15.80:0/377834465 learned_addr learned my addr 172.21.15.80:0/377834465 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5b21640 1 --2- 172.21.15.80:0/377834465 >> v2:172.21.15.87:3300/0 conn(0x7f13d015ec40 0x7f13d015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5320640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 0x7f13d00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530018896 0 0) 0x7f13d00d0d70 con 0x7f13d00d0510 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13bc003530 con 0x7f13d00d0510 2024-04-25T13:44:39.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 978529166 0 0) 0x7f13bc003530 con 0x7f13d00d0510 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 msgr2=0x7f13d00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 0x7f13d00cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 >> v2:172.21.15.87:3300/0 conn(0x7f13d015ec40 msgr2=0x7f13d015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 --2- 172.21.15.80:0/377834465 >> v2:172.21.15.87:3300/0 conn(0x7f13d015ec40 0x7f13d015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d006df80 con 0x7f13d00d0510 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f13c0001680 con 0x7f13d00d0510 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 >> v1:172.21.15.80:6789/0 conn(0x7f13d00d0510 legacy=0x7f13d015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13bc0039c0 0x7f13bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 0x7f13bc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13bc004d50 0x7f13bc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13d00d1050 con 0x7f13bc004d50 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13d006dca0 con 0x7f13bc0039c0 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13bc00ba50 con 0x7f13d0161f90 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5b21640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13bc004d50 0x7f13bc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5320640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13bc0039c0 0x7f13bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d4b1f640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 0x7f13bc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5b21640 1 -- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 msgr2=0x7f13bc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5b21640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 0x7f13bc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5b21640 1 -- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13bc0039c0 msgr2=0x7f13bc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5b21640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13bc0039c0 0x7f13bc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5b21640 1 -- 172.21.15.80:0/377834465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc003530 con 0x7f13bc004d50 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d5320640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13bc0039c0 0x7f13bc003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:39.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.828+0000 7f13d4b1f640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0161f90 0x7f13bc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:39.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13d5b21640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13bc004d50 0x7f13bc005160 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009fc0 tx=0x7f13c801ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7a9d88820e98c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:39.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c80b2070 con 0x7f13bc004d50 2024-04-25T13:44:39.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c80a5cd0 con 0x7f13bc004d50 2024-04-25T13:44:39.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13c7fff640 1 -- 172.21.15.80:0/377834465 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8009c10 con 0x7f13bc004d50 2024-04-25T13:44:39.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13d7584640 1 -- 172.21.15.80:0/377834465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13bc004d50 msgr2=0x7f13bc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:39.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13d7584640 1 --2- 172.21.15.80:0/377834465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13bc004d50 0x7f13bc005160 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f13c8009fc0 tx=0x7f13c801ccc0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13d5b21640 1 -- 172.21.15.80:0/377834465 reap_dead start 2024-04-25T13:44:39.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13d7584640 1 -- 172.21.15.80:0/377834465 shutdown_connections 2024-04-25T13:44:39.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13d7584640 1 -- 172.21.15.80:0/377834465 >> 172.21.15.80:0/377834465 conn(0x7f13d006d080 msgr2=0x7f13d00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:39.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13d7584640 1 -- 172.21.15.80:0/377834465 shutdown_connections 2024-04-25T13:44:39.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.832+0000 7f13d7584640 1 -- 172.21.15.80:0/377834465 wait complete. 2024-04-25T13:44:39.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 Processor -- start 2024-04-25T13:44:39.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 -- start start 2024-04-25T13:44:39.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d0146210 0x7f13d00ac720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0142da0 0x7f13d0160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d01431b0 0x7f13d01610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0162520 con 0x7f13d0146210 2024-04-25T13:44:39.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13d01623a0 con 0x7f13d01431b0 2024-04-25T13:44:39.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13d0144620 con 0x7f13d0142da0 2024-04-25T13:44:39.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d0146210 0x7f13d00ac720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d0146210 0x7f13d00ac720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57062/0 (socket says 172.21.15.80:57062) 2024-04-25T13:44:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 -- 172.21.15.80:0/705734369 learned_addr learned my addr 172.21.15.80:0/705734369 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d4b1f640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0142da0 0x7f13d0160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5b21640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d01431b0 0x7f13d01610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 -- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0142da0 msgr2=0x7f13d0160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0142da0 0x7f13d0160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 -- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d01431b0 msgr2=0x7f13d01610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d01431b0 0x7f13d01610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 -- 172.21.15.80:0/705734369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d0146790 con 0x7f13d0146210 2024-04-25T13:44:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d4b1f640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0142da0 0x7f13d0160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5b21640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d01431b0 0x7f13d01610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d5320640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d0146210 0x7f13d00ac720 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f13cc010870 tx=0x7f13cc010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13c5ffb640 1 -- 172.21.15.80:0/705734369 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00f040 con 0x7f13d0146210 2024-04-25T13:44:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13c5ffb640 1 -- 172.21.15.80:0/705734369 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc00eb60 con 0x7f13d0146210 2024-04-25T13:44:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.836+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d0146a70 con 0x7f13d0146210 2024-04-25T13:44:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.840+0000 7f13c5ffb640 1 -- 172.21.15.80:0/705734369 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc01c400 con 0x7f13d0146210 2024-04-25T13:44:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.840+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d0147030 con 0x7f13d0146210 2024-04-25T13:44:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.840+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d006a3a0 con 0x7f13d0146210 2024-04-25T13:44:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.840+0000 7f13c5ffb640 1 -- 172.21.15.80:0/705734369 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13cc01c5a0 con 0x7f13d0146210 2024-04-25T13:44:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.840+0000 7f13c5ffb640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b406f400 0x7f13b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.840+0000 7f13c5ffb640 1 -- 172.21.15.80:0/705734369 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f13cc08f930 con 0x7f13d0146210 2024-04-25T13:44:39.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.840+0000 7f13d4b1f640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b406f400 0x7f13b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:39.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.840+0000 7f13d4b1f640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b406f400 0x7f13b4071910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c10 tx=0x7f13c0001120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:39.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:39.844+0000 7f13c5ffb640 1 -- 172.21.15.80:0/705734369 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13cc018030 con 0x7f13d0146210 2024-04-25T13:44:40.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.028+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13d015f720 con 0x7f13d0146210 2024-04-25T13:44:40.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.028+0000 7f13c5ffb640 1 -- 172.21.15.80:0/705734369 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5180 (secure 0 0 0) 0x7f13cc05c8b0 con 0x7f13d0146210 2024-04-25T13:44:40.030 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:40.030 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2082291876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2082291876},{"type":"v1","addr":"172.21.15.87:6835","nonce":2082291876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1337331528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1337331528},{"type":"v1","addr":"172.21.15.80:6833","nonce":1337331528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2521189874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2521189874},{"type":"v1","addr":"172.21.15.80:6835","nonce":2521189874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:44:35.958393+0000","modified":"2024-04-25T13:44:39.019142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/642879744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":642879744},{"type":"v1","addr":"172.21.15.87:6837","nonce":642879744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":5}]} 2024-04-25T13:44:40.030 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 41 2024-04-25T13:44:40.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b406f400 msgr2=0x7f13b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b406f400 0x7f13b4071910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c10 tx=0x7f13c0001120 comp rx=0 tx=0).stop 2024-04-25T13:44:40.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d0146210 msgr2=0x7f13d00ac720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d0146210 0x7f13d00ac720 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f13cc010870 tx=0x7f13cc010d30 comp rx=0 tx=0).stop 2024-04-25T13:44:40.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 shutdown_connections 2024-04-25T13:44:40.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b406f400 0x7f13b4071910 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d01431b0 0x7f13d01610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d0142da0 0x7f13d0160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 --2- 172.21.15.80:0/705734369 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d0146210 0x7f13d00ac720 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 >> 172.21.15.80:0/705734369 conn(0x7f13d006d080 msgr2=0x7f13d015fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 shutdown_connections 2024-04-25T13:44:40.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.032+0000 7f13d7584640 1 -- 172.21.15.80:0/705734369 wait complete. 2024-04-25T13:44:40.050 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:44:40.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.184+0000 7fad37658640 1 Processor -- start 2024-04-25T13:44:40.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.184+0000 7fad37658640 1 -- start start 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad37658640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fad30161650 0x7fad30072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad37658640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad3006fcf0 con 0x7fad3006f5a0 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad37658640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad300b0c60 con 0x7fad30161240 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad37658640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fad30073f10 con 0x7fad30161650 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad353f4640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fad3006f5a0 0x7fad30160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:60382/0 (socket says 172.21.15.80:60382) 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad353f4640 1 -- 172.21.15.80:0/38740936 learned_addr learned my addr 172.21.15.80:0/38740936 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad35bf5640 1 --2- 172.21.15.80:0/38740936 >> v2:172.21.15.87:3300/0 conn(0x7fad30161650 0x7fad30072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3293489655 0 0) 0x7fad3006fcf0 con 0x7fad3006f5a0 2024-04-25T13:44:40.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fad1c003530 con 0x7fad3006f5a0 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2021859651 0 0) 0x7fad1c003530 con 0x7fad3006f5a0 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 >> v2:172.21.15.87:3300/0 conn(0x7fad30161650 msgr2=0x7fad30072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 --2- 172.21.15.80:0/38740936 >> v2:172.21.15.87:3300/0 conn(0x7fad30161650 0x7fad30072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 >> v1:172.21.15.87:6789/0 conn(0x7fad30161240 legacy=0x7fad300723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad300b0f40 con 0x7fad3006f5a0 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fad2c001680 con 0x7fad3006f5a0 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 >> v1:172.21.15.80:6789/0 conn(0x7fad3006f5a0 legacy=0x7fad30160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad1c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad1c004810 0x7fad1c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad1c005260 0x7fad1c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad30073f10 con 0x7fad1c0039c0 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c00c1f0 con 0x7fad1c004810 2024-04-25T13:44:40.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad1c00c3f0 con 0x7fad1c005260 2024-04-25T13:44:40.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad353f4640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad1c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad353f4640 1 -- 172.21.15.80:0/38740936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad1c005260 msgr2=0x7fad1c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad353f4640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad1c005260 0x7fad1c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad353f4640 1 -- 172.21.15.80:0/38740936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad1c004810 msgr2=0x7fad1c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad353f4640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad1c004810 0x7fad1c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad353f4640 1 -- 172.21.15.80:0/38740936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c003530 con 0x7fad1c0039c0 2024-04-25T13:44:40.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.188+0000 7fad35bf5640 1 -- 172.21.15.80:0/38740936 reap_dead start 2024-04-25T13:44:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad353f4640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad1c003e30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fad300b6e70 tx=0x7fad2c01cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2251a0fbb36843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c007d40 con 0x7fad1c0039c0 2024-04-25T13:44:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c0a9070 con 0x7fad1c0039c0 2024-04-25T13:44:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad27fff640 1 -- 172.21.15.80:0/38740936 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c005d60 con 0x7fad1c0039c0 2024-04-25T13:44:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad37658640 1 -- 172.21.15.80:0/38740936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 msgr2=0x7fad1c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad37658640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad1c003e30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fad300b6e70 tx=0x7fad2c01cc20 comp rx=0 tx=0).stop 2024-04-25T13:44:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad37658640 1 -- 172.21.15.80:0/38740936 shutdown_connections 2024-04-25T13:44:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad37658640 1 --2- 172.21.15.80:0/38740936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad1c003e30 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad37658640 1 -- 172.21.15.80:0/38740936 >> 172.21.15.80:0/38740936 conn(0x7fad3006d080 msgr2=0x7fad300b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad37658640 1 -- 172.21.15.80:0/38740936 shutdown_connections 2024-04-25T13:44:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.192+0000 7fad37658640 1 -- 172.21.15.80:0/38740936 wait complete. 2024-04-25T13:44:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 Processor -- start 2024-04-25T13:44:40.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 -- start start 2024-04-25T13:44:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad30156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad30158a50 0x7fad30152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad3006e490 0x7fad3006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fad300b5560 con 0x7fad1c0039c0 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fad300b53e0 con 0x7fad3006e490 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fad300b56e0 con 0x7fad30158a50 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad30156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad30156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57086/0 (socket says 172.21.15.80:57086) 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 -- 172.21.15.80:0/1348553732 learned_addr learned my addr 172.21.15.80:0/1348553732 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 -- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad30158a50 msgr2=0x7fad30152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad30158a50 0x7fad30152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 -- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad3006e490 msgr2=0x7fad3006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad3006e490 0x7fad3006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 -- 172.21.15.80:0/1348553732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3006ede0 con 0x7fad1c0039c0 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad353f4640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad30156040 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fad2c01c690 tx=0x7fad1c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad25ffb640 1 -- 172.21.15.80:0/1348553732 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c007140 con 0x7fad1c0039c0 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad25ffb640 1 -- 172.21.15.80:0/1348553732 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad2c0072e0 con 0x7fad1c0039c0 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad25ffb640 1 -- 172.21.15.80:0/1348553732 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c0076d0 con 0x7fad1c0039c0 2024-04-25T13:44:40.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad301591a0 con 0x7fad1c0039c0 2024-04-25T13:44:40.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad30159760 con 0x7fad1c0039c0 2024-04-25T13:44:40.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.196+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad3006a3a0 con 0x7fad1c0039c0 2024-04-25T13:44:40.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.200+0000 7fad25ffb640 1 -- 172.21.15.80:0/1348553732 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fad2c0b6030 con 0x7fad1c0039c0 2024-04-25T13:44:40.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.200+0000 7fad25ffb640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad1406f430 0x7fad14071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.200+0000 7fad25ffb640 1 -- 172.21.15.80:0/1348553732 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fad2c1287a0 con 0x7fad1c0039c0 2024-04-25T13:44:40.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.200+0000 7fad34bf3640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad1406f430 0x7fad14071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.200+0000 7fad34bf3640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad1406f430 0x7fad14071940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fad20007fd0 tx=0x7fad200079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.204+0000 7fad25ffb640 1 -- 172.21.15.80:0/1348553732 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fad3006ede0 con 0x7fad1c0039c0 2024-04-25T13:44:40.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.364+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad30152ed0 con 0x7fad1c0039c0 2024-04-25T13:44:40.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad25ffb640 1 -- 172.21.15.80:0/1348553732 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16596 (secure 0 0 0) 0x7fad2c016930 con 0x7fad1c0039c0 2024-04-25T13:44:40.366 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:40.366 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":82,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:44:37.561936+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:44:33.879562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T13:44:34.932724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad1406f430 msgr2=0x7fad14071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad1406f430 0x7fad14071940 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fad20007fd0 tx=0x7fad200079e0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 msgr2=0x7fad30156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad30156040 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fad2c01c690 tx=0x7fad1c005840 comp rx=0 tx=0).stop 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 shutdown_connections 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fad1406f430 0x7fad14071940 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad3006e490 0x7fad3006e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fad30158a50 0x7fad30152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 --2- 172.21.15.80:0/1348553732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fad1c0039c0 0x7fad30156040 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 >> 172.21.15.80:0/1348553732 conn(0x7fad3006d080 msgr2=0x7fad300723a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 shutdown_connections 2024-04-25T13:44:40.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.368+0000 7fad37658640 1 -- 172.21.15.80:0/1348553732 wait complete. 2024-04-25T13:44:40.390 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:44:40.390 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:44:40.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f6339640 1 Processor -- start 2024-04-25T13:44:40.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f6339640 1 -- start start 2024-04-25T13:44:40.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f6339640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 0x7f87f0073360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f6339640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f87f00ce6a0 0x7f87f00ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f6339640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f00b5f10 con 0x7f87f006e490 2024-04-25T13:44:40.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f6339640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f87f0161320 con 0x7f87f00ce6a0 2024-04-25T13:44:40.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f6339640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87f015ccd0 con 0x7f87f00b0ed0 2024-04-25T13:44:40.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f48d6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f87f00ce6a0 0x7f87f00ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f48d6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f87f00ce6a0 0x7f87f00ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57094/0 (socket says 172.21.15.80:57094) 2024-04-25T13:44:40.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f48d6640 1 -- 172.21.15.80:0/3576543521 learned_addr learned my addr 172.21.15.80:0/3576543521 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:40.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87effff640 1 --2- 172.21.15.80:0/3576543521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 0x7f87f0073360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f48d6640 1 -- 172.21.15.80:0/3576543521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 msgr2=0x7f87f0073360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f48d6640 1 --2- 172.21.15.80:0/3576543521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 0x7f87f0073360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f48d6640 1 -- 172.21.15.80:0/3576543521 >> v1:172.21.15.87:6789/0 conn(0x7f87f006e490 legacy=0x7f87f006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.532+0000 7f87f48d6640 1 -- 172.21.15.80:0/3576543521 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f00738a0 con 0x7f87f00ce6a0 2024-04-25T13:44:40.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f48d6640 1 --2- 172.21.15.80:0/3576543521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f00ceab0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d150 tx=0x7f87e8021440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bd70f95994ca2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87eeffd640 1 -- 172.21.15.80:0/3576543521 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80b8070 con 0x7f87f00ce6a0 2024-04-25T13:44:40.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87eeffd640 1 -- 172.21.15.80:0/3576543521 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e80b3410 con 0x7f87f00ce6a0 2024-04-25T13:44:40.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87eeffd640 1 -- 172.21.15.80:0/3576543521 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80b37e0 con 0x7f87f00ce6a0 2024-04-25T13:44:40.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 -- 172.21.15.80:0/3576543521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 msgr2=0x7f87f00ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 --2- 172.21.15.80:0/3576543521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f00ceab0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f87e800d150 tx=0x7f87e8021440 comp rx=0 tx=0).stop 2024-04-25T13:44:40.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 -- 172.21.15.80:0/3576543521 shutdown_connections 2024-04-25T13:44:40.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 --2- 172.21.15.80:0/3576543521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f00ceab0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 --2- 172.21.15.80:0/3576543521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 0x7f87f0073360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 -- 172.21.15.80:0/3576543521 >> 172.21.15.80:0/3576543521 conn(0x7f87f006d080 msgr2=0x7f87f00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 -- 172.21.15.80:0/3576543521 shutdown_connections 2024-04-25T13:44:40.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 -- 172.21.15.80:0/3576543521 wait complete. 2024-04-25T13:44:40.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 Processor -- start 2024-04-25T13:44:40.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.536+0000 7f87f6339640 1 -- start start 2024-04-25T13:44:40.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87f006e490 0x7f87f00d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 0x7f87f014e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f01539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0160b20 con 0x7f87f00ce6a0 2024-04-25T13:44:40.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0160840 con 0x7f87f006e490 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87f0160e00 con 0x7f87f00b0ed0 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f01539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f01539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57096/0 (socket says 172.21.15.80:57096) 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 -- 172.21.15.80:0/2139176439 learned_addr learned my addr 172.21.15.80:0/2139176439 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87effff640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87f006e490 0x7f87f00d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 -- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 msgr2=0x7f87f014e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87ef7fe640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 0x7f87f014e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 0x7f87f014e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 -- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87f006e490 msgr2=0x7f87f00d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87f006e490 0x7f87f00d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 -- 172.21.15.80:0/2139176439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f015df90 con 0x7f87f00ce6a0 2024-04-25T13:44:40.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f48d6640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f01539d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f87e801faf0 tx=0x7f87e801e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87ecff9640 1 -- 172.21.15.80:0/2139176439 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e80b8040 con 0x7f87f00ce6a0 2024-04-25T13:44:40.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87ecff9640 1 -- 172.21.15.80:0/2139176439 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87e80b3e20 con 0x7f87f00ce6a0 2024-04-25T13:44:40.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f015e270 con 0x7f87f00ce6a0 2024-04-25T13:44:40.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87ecff9640 1 -- 172.21.15.80:0/2139176439 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87e800b720 con 0x7f87f00ce6a0 2024-04-25T13:44:40.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f015e830 con 0x7f87f00ce6a0 2024-04-25T13:44:40.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f006a3a0 con 0x7f87f00ce6a0 2024-04-25T13:44:40.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.540+0000 7f87ecff9640 1 -- 172.21.15.80:0/2139176439 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f87e80c7020 con 0x7f87f00ce6a0 2024-04-25T13:44:40.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.544+0000 7f87ecff9640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87d006f430 0x7f87d0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.544+0000 7f87ecff9640 1 -- 172.21.15.80:0/2139176439 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f87e812ee10 con 0x7f87f00ce6a0 2024-04-25T13:44:40.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.544+0000 7f87effff640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87d006f430 0x7f87d0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.544+0000 7f87effff640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87d006f430 0x7f87d0071940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f87e00096f0 tx=0x7f87e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.548+0000 7f87ecff9640 1 -- 172.21.15.80:0/2139176439 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87e80ce020 con 0x7f87f00ce6a0 2024-04-25T13:44:40.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.692+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f87f006f880 con 0x7f87d006f430 2024-04-25T13:44:40.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.712+0000 7f87ecff9640 1 -- 172.21.15.80:0/2139176439 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247527 (secure 0 0 0) 0x7f87f006f880 con 0x7f87d006f430 2024-04-25T13:44:40.714 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:40.716 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87d006f430 msgr2=0x7f87d0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87d006f430 0x7f87d0071940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f87e00096f0 tx=0x7f87e0008040 comp rx=0 tx=0).stop 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 msgr2=0x7f87f01539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f01539d0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f87e801faf0 tx=0x7f87e801e1e0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 shutdown_connections 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87d006f430 0x7f87d0071940 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87f00ce6a0 0x7f87f01539d0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87f00b0ed0 0x7f87f014e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 --2- 172.21.15.80:0/2139176439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87f006e490 0x7f87f00d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 >> 172.21.15.80:0/2139176439 conn(0x7f87f006d080 msgr2=0x7f87f00b2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 shutdown_connections 2024-04-25T13:44:40.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.716+0000 7f87f6339640 1 -- 172.21.15.80:0/2139176439 wait complete. 2024-04-25T13:44:40.734 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-04-25T13:44:40.185849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4522,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557977,"statfs":{"total":1600000000,"available":1595368916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.637042"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.605172+0000","last_change":"2024-04-25T13:44:36.604793+0000","last_active":"2024-04-25T13:44:36.605172+0000","last_peered":"2024-04-25T13:44:36.605172+0000","last_clean":"2024-04-25T13:44:36.605172+0000","last_became_active":"2024-04-25T13:44:36.594137+0000","last_became_peered":"2024-04-25T13:44:36.594137+0000","last_unstale":"2024-04-25T13:44:36.605172+0000","last_undegraded":"2024-04-25T13:44:36.605172+0000","last_fullsized":"2024-04-25T13:44:36.605172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:54.035463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.614638+0000","last_change":"2024-04-25T13:44:36.614415+0000","last_active":"2024-04-25T13:44:36.614638+0000","last_peered":"2024-04-25T13:44:36.614638+0000","last_clean":"2024-04-25T13:44:36.614638+0000","last_became_active":"2024-04-25T13:44:36.613878+0000","last_became_peered":"2024-04-25T13:44:36.613878+0000","last_unstale":"2024-04-25T13:44:36.614638+0000","last_undegraded":"2024-04-25T13:44:36.614638+0000","last_fullsized":"2024-04-25T13:44:36.614638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:10.668771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.596472+0000","last_change":"2024-04-25T13:44:36.594890+0000","last_active":"2024-04-25T13:44:36.596472+0000","last_peered":"2024-04-25T13:44:36.596472+0000","last_clean":"2024-04-25T13:44:36.596472+0000","last_became_active":"2024-04-25T13:44:36.591213+0000","last_became_peered":"2024-04-25T13:44:36.591213+0000","last_unstale":"2024-04-25T13:44:36.596472+0000","last_undegraded":"2024-04-25T13:44:36.596472+0000","last_fullsized":"2024-04-25T13:44:36.596472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:59.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.594258+0000","last_change":"2024-04-25T13:44:36.593830+0000","last_active":"2024-04-25T13:44:36.594258+0000","last_peered":"2024-04-25T13:44:36.594258+0000","last_clean":"2024-04-25T13:44:36.594258+0000","last_became_active":"2024-04-25T13:44:36.590366+0000","last_became_peered":"2024-04-25T13:44:36.590366+0000","last_unstale":"2024-04-25T13:44:36.594258+0000","last_undegraded":"2024-04-25T13:44:36.594258+0000","last_fullsized":"2024-04-25T13:44:36.594258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:30.989638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.583729+0000","last_change":"2024-04-25T13:44:36.583254+0000","last_active":"2024-04-25T13:44:36.583729+0000","last_peered":"2024-04-25T13:44:36.583729+0000","last_clean":"2024-04-25T13:44:36.583729+0000","last_became_active":"2024-04-25T13:44:36.581185+0000","last_became_peered":"2024-04-25T13:44:36.581185+0000","last_unstale":"2024-04-25T13:44:36.583729+0000","last_undegraded":"2024-04-25T13:44:36.583729+0000","last_fullsized":"2024-04-25T13:44:36.583729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.886088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.615095+0000","last_change":"2024-04-25T13:44:36.612698+0000","last_active":"2024-04-25T13:44:36.615095+0000","last_peered":"2024-04-25T13:44:36.615095+0000","last_clean":"2024-04-25T13:44:36.615095+0000","last_became_active":"2024-04-25T13:44:36.609084+0000","last_became_peered":"2024-04-25T13:44:36.609084+0000","last_unstale":"2024-04-25T13:44:36.615095+0000","last_undegraded":"2024-04-25T13:44:36.615095+0000","last_fullsized":"2024-04-25T13:44:36.615095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:18.434464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.616995+0000","last_change":"2024-04-25T13:44:36.616773+0000","last_active":"2024-04-25T13:44:36.616995+0000","last_peered":"2024-04-25T13:44:36.616995+0000","last_clean":"2024-04-25T13:44:36.616995+0000","last_became_active":"2024-04-25T13:44:36.596021+0000","last_became_peered":"2024-04-25T13:44:36.596021+0000","last_unstale":"2024-04-25T13:44:36.616995+0000","last_undegraded":"2024-04-25T13:44:36.616995+0000","last_fullsized":"2024-04-25T13:44:36.616995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:10.192526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592049+0000","last_change":"2024-04-25T13:44:36.591777+0000","last_active":"2024-04-25T13:44:36.592049+0000","last_peered":"2024-04-25T13:44:36.592049+0000","last_clean":"2024-04-25T13:44:36.592049+0000","last_became_active":"2024-04-25T13:44:36.590176+0000","last_became_peered":"2024-04-25T13:44:36.590176+0000","last_unstale":"2024-04-25T13:44:36.592049+0000","last_undegraded":"2024-04-25T13:44:36.592049+0000","last_fullsized":"2024-04-25T13:44:36.592049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:30.007608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.587097+0000","last_change":"2024-04-25T13:44:36.586812+0000","last_active":"2024-04-25T13:44:36.587097+0000","last_peered":"2024-04-25T13:44:36.587097+0000","last_clean":"2024-04-25T13:44:36.587097+0000","last_became_active":"2024-04-25T13:44:36.586075+0000","last_became_peered":"2024-04-25T13:44:36.586075+0000","last_unstale":"2024-04-25T13:44:36.587097+0000","last_undegraded":"2024-04-25T13:44:36.587097+0000","last_fullsized":"2024-04-25T13:44:36.587097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:17.526462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567516+0000","last_change":"2024-04-25T13:44:36.611721+0000","last_active":"2024-04-25T13:44:37.567516+0000","last_peered":"2024-04-25T13:44:37.567516+0000","last_clean":"2024-04-25T13:44:37.567516+0000","last_became_active":"2024-04-25T13:44:36.606078+0000","last_became_peered":"2024-04-25T13:44:36.606078+0000","last_unstale":"2024-04-25T13:44:37.567516+0000","last_undegraded":"2024-04-25T13:44:37.567516+0000","last_fullsized":"2024-04-25T13:44:37.567516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:56.938190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.608050+0000","last_change":"2024-04-25T13:44:36.607453+0000","last_active":"2024-04-25T13:44:36.608050+0000","last_peered":"2024-04-25T13:44:36.608050+0000","last_clean":"2024-04-25T13:44:36.608050+0000","last_became_active":"2024-04-25T13:44:36.601092+0000","last_became_peered":"2024-04-25T13:44:36.601092+0000","last_unstale":"2024-04-25T13:44:36.608050+0000","last_undegraded":"2024-04-25T13:44:36.608050+0000","last_fullsized":"2024-04-25T13:44:36.608050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:53.079863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.577492+0000","last_change":"2024-04-25T13:44:36.576437+0000","last_active":"2024-04-25T13:44:36.577492+0000","last_peered":"2024-04-25T13:44:36.577492+0000","last_clean":"2024-04-25T13:44:36.577492+0000","last_became_active":"2024-04-25T13:44:36.573263+0000","last_became_peered":"2024-04-25T13:44:36.573263+0000","last_unstale":"2024-04-25T13:44:36.577492+0000","last_undegraded":"2024-04-25T13:44:36.577492+0000","last_fullsized":"2024-04-25T13:44:36.577492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:25.250083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568126+0000","last_change":"2024-04-25T13:44:36.596821+0000","last_active":"2024-04-25T13:44:37.568126+0000","last_peered":"2024-04-25T13:44:37.568126+0000","last_clean":"2024-04-25T13:44:37.568126+0000","last_became_active":"2024-04-25T13:44:36.593794+0000","last_became_peered":"2024-04-25T13:44:36.593794+0000","last_unstale":"2024-04-25T13:44:37.568126+0000","last_undegraded":"2024-04-25T13:44:37.568126+0000","last_fullsized":"2024-04-25T13:44:37.568126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:58.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.583667+0000","last_change":"2024-04-25T13:44:36.582851+0000","last_active":"2024-04-25T13:44:36.583667+0000","last_peered":"2024-04-25T13:44:36.583667+0000","last_clean":"2024-04-25T13:44:36.583667+0000","last_became_active":"2024-04-25T13:44:36.582114+0000","last_became_peered":"2024-04-25T13:44:36.582114+0000","last_unstale":"2024-04-25T13:44:36.583667+0000","last_undegraded":"2024-04-25T13:44:36.583667+0000","last_fullsized":"2024-04-25T13:44:36.583667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:34.749825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568601+0000","last_change":"2024-04-25T13:44:36.597632+0000","last_active":"2024-04-25T13:44:37.568601+0000","last_peered":"2024-04-25T13:44:37.568601+0000","last_clean":"2024-04-25T13:44:37.568601+0000","last_became_active":"2024-04-25T13:44:36.591108+0000","last_became_peered":"2024-04-25T13:44:36.591108+0000","last_unstale":"2024-04-25T13:44:37.568601+0000","last_undegraded":"2024-04-25T13:44:37.568601+0000","last_fullsized":"2024-04-25T13:44:37.568601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:47.222008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568240+0000","last_change":"2024-04-25T13:44:36.610758+0000","last_active":"2024-04-25T13:44:37.568240+0000","last_peered":"2024-04-25T13:44:37.568240+0000","last_clean":"2024-04-25T13:44:37.568240+0000","last_became_active":"2024-04-25T13:44:36.610373+0000","last_became_peered":"2024-04-25T13:44:36.610373+0000","last_unstale":"2024-04-25T13:44:37.568240+0000","last_undegraded":"2024-04-25T13:44:37.568240+0000","last_fullsized":"2024-04-25T13:44:37.568240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:37.460337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.595871+0000","last_change":"2024-04-25T13:44:36.595459+0000","last_active":"2024-04-25T13:44:36.595871+0000","last_peered":"2024-04-25T13:44:36.595871+0000","last_clean":"2024-04-25T13:44:36.595871+0000","last_became_active":"2024-04-25T13:44:36.593269+0000","last_became_peered":"2024-04-25T13:44:36.593269+0000","last_unstale":"2024-04-25T13:44:36.595871+0000","last_undegraded":"2024-04-25T13:44:36.595871+0000","last_fullsized":"2024-04-25T13:44:36.595871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:43.041990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.599393+0000","last_change":"2024-04-25T13:44:36.598869+0000","last_active":"2024-04-25T13:44:36.599393+0000","last_peered":"2024-04-25T13:44:36.599393+0000","last_clean":"2024-04-25T13:44:36.599393+0000","last_became_active":"2024-04-25T13:44:36.591722+0000","last_became_peered":"2024-04-25T13:44:36.591722+0000","last_unstale":"2024-04-25T13:44:36.599393+0000","last_undegraded":"2024-04-25T13:44:36.599393+0000","last_fullsized":"2024-04-25T13:44:36.599393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:16.664003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592956+0000","last_change":"2024-04-25T13:44:36.592646+0000","last_active":"2024-04-25T13:44:36.592956+0000","last_peered":"2024-04-25T13:44:36.592956+0000","last_clean":"2024-04-25T13:44:36.592956+0000","last_became_active":"2024-04-25T13:44:36.590618+0000","last_became_peered":"2024-04-25T13:44:36.590618+0000","last_unstale":"2024-04-25T13:44:36.592956+0000","last_undegraded":"2024-04-25T13:44:36.592956+0000","last_fullsized":"2024-04-25T13:44:36.592956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:49.804159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568879+0000","last_change":"2024-04-25T13:44:36.591998+0000","last_active":"2024-04-25T13:44:37.568879+0000","last_peered":"2024-04-25T13:44:37.568879+0000","last_clean":"2024-04-25T13:44:37.568879+0000","last_became_active":"2024-04-25T13:44:36.588140+0000","last_became_peered":"2024-04-25T13:44:36.588140+0000","last_unstale":"2024-04-25T13:44:37.568879+0000","last_undegraded":"2024-04-25T13:44:37.568879+0000","last_fullsized":"2024-04-25T13:44:37.568879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:55.876469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567241+0000","last_change":"2024-04-25T13:44:36.609461+0000","last_active":"2024-04-25T13:44:37.567241+0000","last_peered":"2024-04-25T13:44:37.567241+0000","last_clean":"2024-04-25T13:44:37.567241+0000","last_became_active":"2024-04-25T13:44:36.608790+0000","last_became_peered":"2024-04-25T13:44:36.608790+0000","last_unstale":"2024-04-25T13:44:37.567241+0000","last_undegraded":"2024-04-25T13:44:37.567241+0000","last_fullsized":"2024-04-25T13:44:37.567241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:28.592867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567824+0000","last_change":"2024-04-25T13:44:36.612797+0000","last_active":"2024-04-25T13:44:37.567824+0000","last_peered":"2024-04-25T13:44:37.567824+0000","last_clean":"2024-04-25T13:44:37.567824+0000","last_became_active":"2024-04-25T13:44:36.608957+0000","last_became_peered":"2024-04-25T13:44:36.608957+0000","last_unstale":"2024-04-25T13:44:37.567824+0000","last_undegraded":"2024-04-25T13:44:37.567824+0000","last_fullsized":"2024-04-25T13:44:37.567824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:44.801001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.583828+0000","last_change":"2024-04-25T13:44:36.581586+0000","last_active":"2024-04-25T13:44:36.583828+0000","last_peered":"2024-04-25T13:44:36.583828+0000","last_clean":"2024-04-25T13:44:36.583828+0000","last_became_active":"2024-04-25T13:44:36.580818+0000","last_became_peered":"2024-04-25T13:44:36.580818+0000","last_unstale":"2024-04-25T13:44:36.583828+0000","last_undegraded":"2024-04-25T13:44:36.583828+0000","last_fullsized":"2024-04-25T13:44:36.583828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:23.328034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.595470+0000","last_change":"2024-04-25T13:44:36.594619+0000","last_active":"2024-04-25T13:44:36.595470+0000","last_peered":"2024-04-25T13:44:36.595470+0000","last_clean":"2024-04-25T13:44:36.595470+0000","last_became_active":"2024-04-25T13:44:36.593907+0000","last_became_peered":"2024-04-25T13:44:36.593907+0000","last_unstale":"2024-04-25T13:44:36.595470+0000","last_undegraded":"2024-04-25T13:44:36.595470+0000","last_fullsized":"2024-04-25T13:44:36.595470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567970+0000","last_change":"2024-04-25T13:44:36.613666+0000","last_active":"2024-04-25T13:44:37.567970+0000","last_peered":"2024-04-25T13:44:37.567970+0000","last_clean":"2024-04-25T13:44:37.567970+0000","last_became_active":"2024-04-25T13:44:36.610791+0000","last_became_peered":"2024-04-25T13:44:36.610791+0000","last_unstale":"2024-04-25T13:44:37.567970+0000","last_undegraded":"2024-04-25T13:44:37.567970+0000","last_fullsized":"2024-04-25T13:44:37.567970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:03.226950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592048+0000","last_change":"2024-04-25T13:44:36.591716+0000","last_active":"2024-04-25T13:44:36.592048+0000","last_peered":"2024-04-25T13:44:36.592048+0000","last_clean":"2024-04-25T13:44:36.592048+0000","last_became_active":"2024-04-25T13:44:36.591003+0000","last_became_peered":"2024-04-25T13:44:36.591003+0000","last_unstale":"2024-04-25T13:44:36.592048+0000","last_undegraded":"2024-04-25T13:44:36.592048+0000","last_fullsized":"2024-04-25T13:44:36.592048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:09.615310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.596599+0000","last_change":"2024-04-25T13:44:36.595878+0000","last_active":"2024-04-25T13:44:36.596599+0000","last_peered":"2024-04-25T13:44:36.596599+0000","last_clean":"2024-04-25T13:44:36.596599+0000","last_became_active":"2024-04-25T13:44:36.595203+0000","last_became_peered":"2024-04-25T13:44:36.595203+0000","last_unstale":"2024-04-25T13:44:36.596599+0000","last_undegraded":"2024-04-25T13:44:36.596599+0000","last_fullsized":"2024-04-25T13:44:36.596599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:37.948968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.566643+0000","last_change":"2024-04-25T13:44:36.600721+0000","last_active":"2024-04-25T13:44:37.566643+0000","last_peered":"2024-04-25T13:44:37.566643+0000","last_clean":"2024-04-25T13:44:37.566643+0000","last_became_active":"2024-04-25T13:44:36.599892+0000","last_became_peered":"2024-04-25T13:44:36.599892+0000","last_unstale":"2024-04-25T13:44:37.566643+0000","last_undegraded":"2024-04-25T13:44:37.566643+0000","last_fullsized":"2024-04-25T13:44:37.566643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:05.758222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.589960+0000","last_change":"2024-04-25T13:44:36.589663+0000","last_active":"2024-04-25T13:44:36.589960+0000","last_peered":"2024-04-25T13:44:36.589960+0000","last_clean":"2024-04-25T13:44:36.589960+0000","last_became_active":"2024-04-25T13:44:36.588967+0000","last_became_peered":"2024-04-25T13:44:36.588967+0000","last_unstale":"2024-04-25T13:44:36.589960+0000","last_undegraded":"2024-04-25T13:44:36.589960+0000","last_fullsized":"2024-04-25T13:44:36.589960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:47.273886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.587172+0000","last_change":"2024-04-25T13:44:36.586834+0000","last_active":"2024-04-25T13:44:36.587172+0000","last_peered":"2024-04-25T13:44:36.587172+0000","last_clean":"2024-04-25T13:44:36.587172+0000","last_became_active":"2024-04-25T13:44:36.585982+0000","last_became_peered":"2024-04-25T13:44:36.585982+0000","last_unstale":"2024-04-25T13:44:36.587172+0000","last_undegraded":"2024-04-25T13:44:36.587172+0000","last_fullsized":"2024-04-25T13:44:36.587172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:24.950626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.590081+0000","last_change":"2024-04-25T13:44:36.589728+0000","last_active":"2024-04-25T13:44:36.590081+0000","last_peered":"2024-04-25T13:44:36.590081+0000","last_clean":"2024-04-25T13:44:36.590081+0000","last_became_active":"2024-04-25T13:44:36.588869+0000","last_became_peered":"2024-04-25T13:44:36.588869+0000","last_unstale":"2024-04-25T13:44:36.590081+0000","last_undegraded":"2024-04-25T13:44:36.590081+0000","last_fullsized":"2024-04-25T13:44:36.590081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:35.747145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.600600+0000","last_change":"2024-04-25T13:44:36.600129+0000","last_active":"2024-04-25T13:44:36.600600+0000","last_peered":"2024-04-25T13:44:36.600600+0000","last_clean":"2024-04-25T13:44:36.600600+0000","last_became_active":"2024-04-25T13:44:36.599389+0000","last_became_peered":"2024-04-25T13:44:36.599389+0000","last_unstale":"2024-04-25T13:44:36.600600+0000","last_undegraded":"2024-04-25T13:44:36.600600+0000","last_fullsized":"2024-04-25T13:44:36.600600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:17.177903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.602217+0000","last_change":"2024-04-25T13:44:36.601755+0000","last_active":"2024-04-25T13:44:36.602217+0000","last_peered":"2024-04-25T13:44:36.602217+0000","last_clean":"2024-04-25T13:44:36.602217+0000","last_became_active":"2024-04-25T13:44:36.599573+0000","last_became_peered":"2024-04-25T13:44:36.599573+0000","last_unstale":"2024-04-25T13:44:36.602217+0000","last_undegraded":"2024-04-25T13:44:36.602217+0000","last_fullsized":"2024-04-25T13:44:36.602217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:56.925889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.622835+0000","last_change":"2024-04-25T13:44:35.591054+0000","last_active":"2024-04-25T13:44:36.622835+0000","last_peered":"2024-04-25T13:44:36.622835+0000","last_clean":"2024-04-25T13:44:36.622835+0000","last_became_active":"2024-04-25T13:44:35.590343+0000","last_became_peered":"2024-04-25T13:44:35.590343+0000","last_unstale":"2024-04-25T13:44:36.622835+0000","last_undegraded":"2024-04-25T13:44:36.622835+0000","last_fullsized":"2024-04-25T13:44:36.622835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:14.514741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.585822+0000","last_change":"2024-04-25T13:44:36.585172+0000","last_active":"2024-04-25T13:44:36.585822+0000","last_peered":"2024-04-25T13:44:36.585822+0000","last_clean":"2024-04-25T13:44:36.585822+0000","last_became_active":"2024-04-25T13:44:36.584271+0000","last_became_peered":"2024-04-25T13:44:36.584271+0000","last_unstale":"2024-04-25T13:44:36.585822+0000","last_undegraded":"2024-04-25T13:44:36.585822+0000","last_fullsized":"2024-04-25T13:44:36.585822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:27.553217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568395+0000","last_change":"2024-04-25T13:44:35.575627+0000","last_active":"2024-04-25T13:44:37.568395+0000","last_peered":"2024-04-25T13:44:37.568395+0000","last_clean":"2024-04-25T13:44:37.568395+0000","last_became_active":"2024-04-25T13:44:35.575156+0000","last_became_peered":"2024-04-25T13:44:35.575156+0000","last_unstale":"2024-04-25T13:44:37.568395+0000","last_undegraded":"2024-04-25T13:44:37.568395+0000","last_fullsized":"2024-04-25T13:44:37.568395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:19.115196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.612238+0000","last_change":"2024-04-25T13:44:36.610902+0000","last_active":"2024-04-25T13:44:36.612238+0000","last_peered":"2024-04-25T13:44:36.612238+0000","last_clean":"2024-04-25T13:44:36.612238+0000","last_became_active":"2024-04-25T13:44:36.607536+0000","last_became_peered":"2024-04-25T13:44:36.607536+0000","last_unstale":"2024-04-25T13:44:36.612238+0000","last_undegraded":"2024-04-25T13:44:36.612238+0000","last_fullsized":"2024-04-25T13:44:36.612238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:56.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568495+0000","last_change":"2024-04-25T13:44:35.595532+0000","last_active":"2024-04-25T13:44:37.568495+0000","last_peered":"2024-04-25T13:44:37.568495+0000","last_clean":"2024-04-25T13:44:37.568495+0000","last_became_active":"2024-04-25T13:44:35.594157+0000","last_became_peered":"2024-04-25T13:44:35.594157+0000","last_unstale":"2024-04-25T13:44:37.568495+0000","last_undegraded":"2024-04-25T13:44:37.568495+0000","last_fullsized":"2024-04-25T13:44:37.568495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:47.198139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.584318+0000","last_change":"2024-04-25T13:44:36.583441+0000","last_active":"2024-04-25T13:44:36.584318+0000","last_peered":"2024-04-25T13:44:36.584318+0000","last_clean":"2024-04-25T13:44:36.584318+0000","last_became_active":"2024-04-25T13:44:36.582707+0000","last_became_peered":"2024-04-25T13:44:36.582707+0000","last_unstale":"2024-04-25T13:44:36.584318+0000","last_undegraded":"2024-04-25T13:44:36.584318+0000","last_fullsized":"2024-04-25T13:44:36.584318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.281041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.624758+0000","last_change":"2024-04-25T13:44:35.600018+0000","last_active":"2024-04-25T13:44:36.624758+0000","last_peered":"2024-04-25T13:44:36.624758+0000","last_clean":"2024-04-25T13:44:36.624758+0000","last_became_active":"2024-04-25T13:44:35.597655+0000","last_became_peered":"2024-04-25T13:44:35.597655+0000","last_unstale":"2024-04-25T13:44:36.624758+0000","last_undegraded":"2024-04-25T13:44:36.624758+0000","last_fullsized":"2024-04-25T13:44:36.624758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:26.870545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.586733+0000","last_change":"2024-04-25T13:44:36.586390+0000","last_active":"2024-04-25T13:44:36.586733+0000","last_peered":"2024-04-25T13:44:36.586733+0000","last_clean":"2024-04-25T13:44:36.586733+0000","last_became_active":"2024-04-25T13:44:36.585611+0000","last_became_peered":"2024-04-25T13:44:36.585611+0000","last_unstale":"2024-04-25T13:44:36.586733+0000","last_undegraded":"2024-04-25T13:44:36.586733+0000","last_fullsized":"2024-04-25T13:44:36.586733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:38.433758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.622701+0000","last_change":"2024-04-25T13:44:35.571834+0000","last_active":"2024-04-25T13:44:36.622701+0000","last_peered":"2024-04-25T13:44:36.622701+0000","last_clean":"2024-04-25T13:44:36.622701+0000","last_became_active":"2024-04-25T13:44:35.571501+0000","last_became_peered":"2024-04-25T13:44:35.571501+0000","last_unstale":"2024-04-25T13:44:36.622701+0000","last_undegraded":"2024-04-25T13:44:36.622701+0000","last_fullsized":"2024-04-25T13:44:36.622701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:26.236173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568342+0000","last_change":"2024-04-25T13:44:36.612364+0000","last_active":"2024-04-25T13:44:37.568342+0000","last_peered":"2024-04-25T13:44:37.568342+0000","last_clean":"2024-04-25T13:44:37.568342+0000","last_became_active":"2024-04-25T13:44:36.607520+0000","last_became_peered":"2024-04-25T13:44:36.607520+0000","last_unstale":"2024-04-25T13:44:37.568342+0000","last_undegraded":"2024-04-25T13:44:37.568342+0000","last_fullsized":"2024-04-25T13:44:37.568342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:52.967931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.621607+0000","last_change":"2024-04-25T13:44:35.597254+0000","last_active":"2024-04-25T13:44:36.621607+0000","last_peered":"2024-04-25T13:44:36.621607+0000","last_clean":"2024-04-25T13:44:36.621607+0000","last_became_active":"2024-04-25T13:44:35.593852+0000","last_became_peered":"2024-04-25T13:44:35.593852+0000","last_unstale":"2024-04-25T13:44:36.621607+0000","last_undegraded":"2024-04-25T13:44:36.621607+0000","last_fullsized":"2024-04-25T13:44:36.621607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.384285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.607790+0000","last_change":"2024-04-25T13:44:36.607567+0000","last_active":"2024-04-25T13:44:36.607790+0000","last_peered":"2024-04-25T13:44:36.607790+0000","last_clean":"2024-04-25T13:44:36.607790+0000","last_became_active":"2024-04-25T13:44:36.607044+0000","last_became_peered":"2024-04-25T13:44:36.607044+0000","last_unstale":"2024-04-25T13:44:36.607790+0000","last_undegraded":"2024-04-25T13:44:36.607790+0000","last_fullsized":"2024-04-25T13:44:36.607790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:31.652704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.566594+0000","last_change":"2024-04-25T13:44:35.572213+0000","last_active":"2024-04-25T13:44:36.566594+0000","last_peered":"2024-04-25T13:44:36.566594+0000","last_clean":"2024-04-25T13:44:36.566594+0000","last_became_active":"2024-04-25T13:44:35.571904+0000","last_became_peered":"2024-04-25T13:44:35.571904+0000","last_unstale":"2024-04-25T13:44:36.566594+0000","last_undegraded":"2024-04-25T13:44:36.566594+0000","last_fullsized":"2024-04-25T13:44:36.566594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:14.510930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568132+0000","last_change":"2024-04-25T13:44:36.595986+0000","last_active":"2024-04-25T13:44:37.568132+0000","last_peered":"2024-04-25T13:44:37.568132+0000","last_clean":"2024-04-25T13:44:37.568132+0000","last_became_active":"2024-04-25T13:44:36.595350+0000","last_became_peered":"2024-04-25T13:44:36.595350+0000","last_unstale":"2024-04-25T13:44:37.568132+0000","last_undegraded":"2024-04-25T13:44:37.568132+0000","last_fullsized":"2024-04-25T13:44:37.568132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:07.945298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.569031+0000","last_change":"2024-04-25T13:44:35.565632+0000","last_active":"2024-04-25T13:44:36.569031+0000","last_peered":"2024-04-25T13:44:36.569031+0000","last_clean":"2024-04-25T13:44:36.569031+0000","last_became_active":"2024-04-25T13:44:35.564264+0000","last_became_peered":"2024-04-25T13:44:35.564264+0000","last_unstale":"2024-04-25T13:44:36.569031+0000","last_undegraded":"2024-04-25T13:44:36.569031+0000","last_fullsized":"2024-04-25T13:44:36.569031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:13.272951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.620263+0000","last_change":"2024-04-25T13:44:36.620003+0000","last_active":"2024-04-25T13:44:36.620263+0000","last_peered":"2024-04-25T13:44:36.620263+0000","last_clean":"2024-04-25T13:44:36.620263+0000","last_became_active":"2024-04-25T13:44:36.619209+0000","last_became_peered":"2024-04-25T13:44:36.619209+0000","last_unstale":"2024-04-25T13:44:36.620263+0000","last_undegraded":"2024-04-25T13:44:36.620263+0000","last_fullsized":"2024-04-25T13:44:36.620263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:58.286534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568610+0000","last_change":"2024-04-25T13:44:35.574184+0000","last_active":"2024-04-25T13:44:37.568610+0000","last_peered":"2024-04-25T13:44:37.568610+0000","last_clean":"2024-04-25T13:44:37.568610+0000","last_became_active":"2024-04-25T13:44:35.573366+0000","last_became_peered":"2024-04-25T13:44:35.573366+0000","last_unstale":"2024-04-25T13:44:37.568610+0000","last_undegraded":"2024-04-25T13:44:37.568610+0000","last_fullsized":"2024-04-25T13:44:37.568610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:58.821551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.609048+0000","last_change":"2024-04-25T13:44:36.608623+0000","last_active":"2024-04-25T13:44:36.609048+0000","last_peered":"2024-04-25T13:44:36.609048+0000","last_clean":"2024-04-25T13:44:36.609048+0000","last_became_active":"2024-04-25T13:44:36.602565+0000","last_became_peered":"2024-04-25T13:44:36.602565+0000","last_unstale":"2024-04-25T13:44:36.609048+0000","last_undegraded":"2024-04-25T13:44:36.609048+0000","last_fullsized":"2024-04-25T13:44:36.609048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:48.037163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.597892+0000","last_change":"2024-04-25T13:44:35.598272+0000","last_active":"2024-04-25T13:44:36.597892+0000","last_peered":"2024-04-25T13:44:36.597892+0000","last_clean":"2024-04-25T13:44:36.597892+0000","last_became_active":"2024-04-25T13:44:35.594752+0000","last_became_peered":"2024-04-25T13:44:35.594752+0000","last_unstale":"2024-04-25T13:44:36.597892+0000","last_undegraded":"2024-04-25T13:44:36.597892+0000","last_fullsized":"2024-04-25T13:44:36.597892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.199875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.607890+0000","last_change":"2024-04-25T13:44:36.607580+0000","last_active":"2024-04-25T13:44:36.607890+0000","last_peered":"2024-04-25T13:44:36.607890+0000","last_clean":"2024-04-25T13:44:36.607890+0000","last_became_active":"2024-04-25T13:44:36.606885+0000","last_became_peered":"2024-04-25T13:44:36.606885+0000","last_unstale":"2024-04-25T13:44:36.607890+0000","last_undegraded":"2024-04-25T13:44:36.607890+0000","last_fullsized":"2024-04-25T13:44:36.607890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:54.174567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.624474+0000","last_change":"2024-04-25T13:44:35.591433+0000","last_active":"2024-04-25T13:44:36.624474+0000","last_peered":"2024-04-25T13:44:36.624474+0000","last_clean":"2024-04-25T13:44:36.624474+0000","last_became_active":"2024-04-25T13:44:35.590616+0000","last_became_peered":"2024-04-25T13:44:35.590616+0000","last_unstale":"2024-04-25T13:44:36.624474+0000","last_undegraded":"2024-04-25T13:44:36.624474+0000","last_fullsized":"2024-04-25T13:44:36.624474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:56.748763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.571155+0000","last_change":"2024-04-25T13:44:36.570641+0000","last_active":"2024-04-25T13:44:36.571155+0000","last_peered":"2024-04-25T13:44:36.571155+0000","last_clean":"2024-04-25T13:44:36.571155+0000","last_became_active":"2024-04-25T13:44:36.569938+0000","last_became_peered":"2024-04-25T13:44:36.569938+0000","last_unstale":"2024-04-25T13:44:36.571155+0000","last_undegraded":"2024-04-25T13:44:36.571155+0000","last_fullsized":"2024-04-25T13:44:36.571155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.181080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"81'6","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.626804+0000","last_change":"2024-04-25T13:44:35.563616+0000","last_active":"2024-04-25T13:44:36.626804+0000","last_peered":"2024-04-25T13:44:36.626804+0000","last_clean":"2024-04-25T13:44:36.626804+0000","last_became_active":"2024-04-25T13:44:35.563329+0000","last_became_peered":"2024-04-25T13:44:35.563329+0000","last_unstale":"2024-04-25T13:44:36.626804+0000","last_undegraded":"2024-04-25T13:44:36.626804+0000","last_fullsized":"2024-04-25T13:44:36.626804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:56.290232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.605966+0000","last_change":"2024-04-25T13:44:36.605628+0000","last_active":"2024-04-25T13:44:36.605966+0000","last_peered":"2024-04-25T13:44:36.605966+0000","last_clean":"2024-04-25T13:44:36.605966+0000","last_became_active":"2024-04-25T13:44:36.595132+0000","last_became_peered":"2024-04-25T13:44:36.595132+0000","last_unstale":"2024-04-25T13:44:36.605966+0000","last_undegraded":"2024-04-25T13:44:36.605966+0000","last_fullsized":"2024-04-25T13:44:36.605966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:21.264823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.623156+0000","last_change":"2024-04-25T13:44:35.584058+0000","last_active":"2024-04-25T13:44:36.623156+0000","last_peered":"2024-04-25T13:44:36.623156+0000","last_clean":"2024-04-25T13:44:36.623156+0000","last_became_active":"2024-04-25T13:44:35.583810+0000","last_became_peered":"2024-04-25T13:44:35.583810+0000","last_unstale":"2024-04-25T13:44:36.623156+0000","last_undegraded":"2024-04-25T13:44:36.623156+0000","last_fullsized":"2024-04-25T13:44:36.623156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:17.672688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.615653+0000","last_change":"2024-04-25T13:44:36.615173+0000","last_active":"2024-04-25T13:44:36.615653+0000","last_peered":"2024-04-25T13:44:36.615653+0000","last_clean":"2024-04-25T13:44:36.615653+0000","last_became_active":"2024-04-25T13:44:36.614710+0000","last_became_peered":"2024-04-25T13:44:36.614710+0000","last_unstale":"2024-04-25T13:44:36.615653+0000","last_undegraded":"2024-04-25T13:44:36.615653+0000","last_fullsized":"2024-04-25T13:44:36.615653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:46.290346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.584945+0000","last_change":"2024-04-25T13:44:35.599646+0000","last_active":"2024-04-25T13:44:36.584945+0000","last_peered":"2024-04-25T13:44:36.584945+0000","last_clean":"2024-04-25T13:44:36.584945+0000","last_became_active":"2024-04-25T13:44:35.595707+0000","last_became_peered":"2024-04-25T13:44:35.595707+0000","last_unstale":"2024-04-25T13:44:36.584945+0000","last_undegraded":"2024-04-25T13:44:36.584945+0000","last_fullsized":"2024-04-25T13:44:36.584945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.164922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568415+0000","last_change":"2024-04-25T13:44:36.598534+0000","last_active":"2024-04-25T13:44:37.568415+0000","last_peered":"2024-04-25T13:44:37.568415+0000","last_clean":"2024-04-25T13:44:37.568415+0000","last_became_active":"2024-04-25T13:44:36.595921+0000","last_became_peered":"2024-04-25T13:44:36.595921+0000","last_unstale":"2024-04-25T13:44:37.568415+0000","last_undegraded":"2024-04-25T13:44:37.568415+0000","last_fullsized":"2024-04-25T13:44:37.568415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:06.047866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592338+0000","last_change":"2024-04-25T13:44:35.582895+0000","last_active":"2024-04-25T13:44:36.592338+0000","last_peered":"2024-04-25T13:44:36.592338+0000","last_clean":"2024-04-25T13:44:36.592338+0000","last_became_active":"2024-04-25T13:44:35.582078+0000","last_became_peered":"2024-04-25T13:44:35.582078+0000","last_unstale":"2024-04-25T13:44:36.592338+0000","last_undegraded":"2024-04-25T13:44:36.592338+0000","last_fullsized":"2024-04-25T13:44:36.592338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:15.821050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.595114+0000","last_change":"2024-04-25T13:44:36.594727+0000","last_active":"2024-04-25T13:44:36.595114+0000","last_peered":"2024-04-25T13:44:36.595114+0000","last_clean":"2024-04-25T13:44:36.595114+0000","last_became_active":"2024-04-25T13:44:36.592124+0000","last_became_peered":"2024-04-25T13:44:36.592124+0000","last_unstale":"2024-04-25T13:44:36.595114+0000","last_undegraded":"2024-04-25T13:44:36.595114+0000","last_fullsized":"2024-04-25T13:44:36.595114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:09.758689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567039+0000","last_change":"2024-04-25T13:44:35.593747+0000","last_active":"2024-04-25T13:44:37.567039+0000","last_peered":"2024-04-25T13:44:37.567039+0000","last_clean":"2024-04-25T13:44:37.567039+0000","last_became_active":"2024-04-25T13:44:35.592369+0000","last_became_peered":"2024-04-25T13:44:35.592369+0000","last_unstale":"2024-04-25T13:44:37.567039+0000","last_undegraded":"2024-04-25T13:44:37.567039+0000","last_fullsized":"2024-04-25T13:44:37.567039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:08.089896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568272+0000","last_change":"2024-04-25T13:44:36.610438+0000","last_active":"2024-04-25T13:44:37.568272+0000","last_peered":"2024-04-25T13:44:37.568272+0000","last_clean":"2024-04-25T13:44:37.568272+0000","last_became_active":"2024-04-25T13:44:36.601469+0000","last_became_peered":"2024-04-25T13:44:36.601469+0000","last_unstale":"2024-04-25T13:44:37.568272+0000","last_undegraded":"2024-04-25T13:44:37.568272+0000","last_fullsized":"2024-04-25T13:44:37.568272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.388942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.620938+0000","last_change":"2024-04-25T13:44:35.593729+0000","last_active":"2024-04-25T13:44:36.620938+0000","last_peered":"2024-04-25T13:44:36.620938+0000","last_clean":"2024-04-25T13:44:36.620938+0000","last_became_active":"2024-04-25T13:44:35.590063+0000","last_became_peered":"2024-04-25T13:44:35.590063+0000","last_unstale":"2024-04-25T13:44:36.620938+0000","last_undegraded":"2024-04-25T13:44:36.620938+0000","last_fullsized":"2024-04-25T13:44:36.620938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:51.680146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.598592+0000","last_change":"2024-04-25T13:44:36.598246+0000","last_active":"2024-04-25T13:44:36.598592+0000","last_peered":"2024-04-25T13:44:36.598592+0000","last_clean":"2024-04-25T13:44:36.598592+0000","last_became_active":"2024-04-25T13:44:36.597484+0000","last_became_peered":"2024-04-25T13:44:36.597484+0000","last_unstale":"2024-04-25T13:44:36.598592+0000","last_undegraded":"2024-04-25T13:44:36.598592+0000","last_fullsized":"2024-04-25T13:44:36.598592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:48.913915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568741+0000","last_change":"2024-04-25T13:44:35.587553+0000","last_active":"2024-04-25T13:44:37.568741+0000","last_peered":"2024-04-25T13:44:37.568741+0000","last_clean":"2024-04-25T13:44:37.568741+0000","last_became_active":"2024-04-25T13:44:35.586934+0000","last_became_peered":"2024-04-25T13:44:35.586934+0000","last_unstale":"2024-04-25T13:44:37.568741+0000","last_undegraded":"2024-04-25T13:44:37.568741+0000","last_fullsized":"2024-04-25T13:44:37.568741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:17.362081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.579086+0000","last_change":"2024-04-25T13:44:36.577063+0000","last_active":"2024-04-25T13:44:36.579086+0000","last_peered":"2024-04-25T13:44:36.579086+0000","last_clean":"2024-04-25T13:44:36.579086+0000","last_became_active":"2024-04-25T13:44:36.574179+0000","last_became_peered":"2024-04-25T13:44:36.574179+0000","last_unstale":"2024-04-25T13:44:36.579086+0000","last_undegraded":"2024-04-25T13:44:36.579086+0000","last_fullsized":"2024-04-25T13:44:36.579086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:47.488620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.625911+0000","last_change":"2024-04-25T13:44:35.599066+0000","last_active":"2024-04-25T13:44:36.625911+0000","last_peered":"2024-04-25T13:44:36.625911+0000","last_clean":"2024-04-25T13:44:36.625911+0000","last_became_active":"2024-04-25T13:44:35.594831+0000","last_became_peered":"2024-04-25T13:44:35.594831+0000","last_unstale":"2024-04-25T13:44:36.625911+0000","last_undegraded":"2024-04-25T13:44:36.625911+0000","last_fullsized":"2024-04-25T13:44:36.625911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:41.095783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.603875+0000","last_change":"2024-04-25T13:44:36.603649+0000","last_active":"2024-04-25T13:44:36.603875+0000","last_peered":"2024-04-25T13:44:36.603875+0000","last_clean":"2024-04-25T13:44:36.603875+0000","last_became_active":"2024-04-25T13:44:36.603191+0000","last_became_peered":"2024-04-25T13:44:36.603191+0000","last_unstale":"2024-04-25T13:44:36.603875+0000","last_undegraded":"2024-04-25T13:44:36.603875+0000","last_fullsized":"2024-04-25T13:44:36.603875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:33.156836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.574103+0000","last_change":"2024-04-25T13:44:35.592543+0000","last_active":"2024-04-25T13:44:36.574103+0000","last_peered":"2024-04-25T13:44:36.574103+0000","last_clean":"2024-04-25T13:44:36.574103+0000","last_became_active":"2024-04-25T13:44:35.591827+0000","last_became_peered":"2024-04-25T13:44:35.591827+0000","last_unstale":"2024-04-25T13:44:36.574103+0000","last_undegraded":"2024-04-25T13:44:36.574103+0000","last_fullsized":"2024-04-25T13:44:36.574103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:22.922936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.596532+0000","last_change":"2024-04-25T13:44:36.595619+0000","last_active":"2024-04-25T13:44:36.596532+0000","last_peered":"2024-04-25T13:44:36.596532+0000","last_clean":"2024-04-25T13:44:36.596532+0000","last_became_active":"2024-04-25T13:44:36.593828+0000","last_became_peered":"2024-04-25T13:44:36.593828+0000","last_unstale":"2024-04-25T13:44:36.596532+0000","last_undegraded":"2024-04-25T13:44:36.596532+0000","last_fullsized":"2024-04-25T13:44:36.596532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:36.621561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.585941+0000","last_change":"2024-04-25T13:44:35.591439+0000","last_active":"2024-04-25T13:44:36.585941+0000","last_peered":"2024-04-25T13:44:36.585941+0000","last_clean":"2024-04-25T13:44:36.585941+0000","last_became_active":"2024-04-25T13:44:35.588118+0000","last_became_peered":"2024-04-25T13:44:35.588118+0000","last_unstale":"2024-04-25T13:44:36.585941+0000","last_undegraded":"2024-04-25T13:44:36.585941+0000","last_fullsized":"2024-04-25T13:44:36.585941+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.816878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.593666+0000","last_change":"2024-04-25T13:44:36.593336+0000","last_active":"2024-04-25T13:44:36.593666+0000","last_peered":"2024-04-25T13:44:36.593666+0000","last_clean":"2024-04-25T13:44:36.593666+0000","last_became_active":"2024-04-25T13:44:36.592625+0000","last_became_peered":"2024-04-25T13:44:36.592625+0000","last_unstale":"2024-04-25T13:44:36.593666+0000","last_undegraded":"2024-04-25T13:44:36.593666+0000","last_fullsized":"2024-04-25T13:44:36.593666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.883555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.582467+0000","last_change":"2024-04-25T13:44:35.584228+0000","last_active":"2024-04-25T13:44:36.582467+0000","last_peered":"2024-04-25T13:44:36.582467+0000","last_clean":"2024-04-25T13:44:36.582467+0000","last_became_active":"2024-04-25T13:44:35.583376+0000","last_became_peered":"2024-04-25T13:44:35.583376+0000","last_unstale":"2024-04-25T13:44:36.582467+0000","last_undegraded":"2024-04-25T13:44:36.582467+0000","last_fullsized":"2024-04-25T13:44:36.582467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:44.373304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.589573+0000","last_change":"2024-04-25T13:44:36.589034+0000","last_active":"2024-04-25T13:44:36.589573+0000","last_peered":"2024-04-25T13:44:36.589573+0000","last_clean":"2024-04-25T13:44:36.589573+0000","last_became_active":"2024-04-25T13:44:36.588199+0000","last_became_peered":"2024-04-25T13:44:36.588199+0000","last_unstale":"2024-04-25T13:44:36.589573+0000","last_undegraded":"2024-04-25T13:44:36.589573+0000","last_fullsized":"2024-04-25T13:44:36.589573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:09.811361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568178+0000","last_change":"2024-04-25T13:44:35.573257+0000","last_active":"2024-04-25T13:44:37.568178+0000","last_peered":"2024-04-25T13:44:37.568178+0000","last_clean":"2024-04-25T13:44:37.568178+0000","last_became_active":"2024-04-25T13:44:35.572965+0000","last_became_peered":"2024-04-25T13:44:35.572965+0000","last_unstale":"2024-04-25T13:44:37.568178+0000","last_undegraded":"2024-04-25T13:44:37.568178+0000","last_fullsized":"2024-04-25T13:44:37.568178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:55.363028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568701+0000","last_change":"2024-04-25T13:44:36.586632+0000","last_active":"2024-04-25T13:44:37.568701+0000","last_peered":"2024-04-25T13:44:37.568701+0000","last_clean":"2024-04-25T13:44:37.568701+0000","last_became_active":"2024-04-25T13:44:36.584513+0000","last_became_peered":"2024-04-25T13:44:36.584513+0000","last_unstale":"2024-04-25T13:44:37.568701+0000","last_undegraded":"2024-04-25T13:44:37.568701+0000","last_fullsized":"2024-04-25T13:44:37.568701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:52.930285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.623164+0000","last_change":"2024-04-25T13:44:35.597653+0000","last_active":"2024-04-25T13:44:36.623164+0000","last_peered":"2024-04-25T13:44:36.623164+0000","last_clean":"2024-04-25T13:44:36.623164+0000","last_became_active":"2024-04-25T13:44:35.593659+0000","last_became_peered":"2024-04-25T13:44:35.593659+0000","last_unstale":"2024-04-25T13:44:36.623164+0000","last_undegraded":"2024-04-25T13:44:36.623164+0000","last_fullsized":"2024-04-25T13:44:36.623164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:27.782046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.616548+0000","last_change":"2024-04-25T13:44:36.616317+0000","last_active":"2024-04-25T13:44:36.616548+0000","last_peered":"2024-04-25T13:44:36.616548+0000","last_clean":"2024-04-25T13:44:36.616548+0000","last_became_active":"2024-04-25T13:44:36.594389+0000","last_became_peered":"2024-04-25T13:44:36.594389+0000","last_unstale":"2024-04-25T13:44:36.616548+0000","last_undegraded":"2024-04-25T13:44:36.616548+0000","last_fullsized":"2024-04-25T13:44:36.616548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:31.453872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.579523+0000","last_change":"2024-04-25T13:44:35.571317+0000","last_active":"2024-04-25T13:44:36.579523+0000","last_peered":"2024-04-25T13:44:36.579523+0000","last_clean":"2024-04-25T13:44:36.579523+0000","last_became_active":"2024-04-25T13:44:35.570522+0000","last_became_peered":"2024-04-25T13:44:35.570522+0000","last_unstale":"2024-04-25T13:44:36.579523+0000","last_undegraded":"2024-04-25T13:44:36.579523+0000","last_fullsized":"2024-04-25T13:44:36.579523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:16.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.597910+0000","last_change":"2024-04-25T13:44:36.597555+0000","last_active":"2024-04-25T13:44:36.597910+0000","last_peered":"2024-04-25T13:44:36.597910+0000","last_clean":"2024-04-25T13:44:36.597910+0000","last_became_active":"2024-04-25T13:44:36.593594+0000","last_became_peered":"2024-04-25T13:44:36.593594+0000","last_unstale":"2024-04-25T13:44:36.597910+0000","last_undegraded":"2024-04-25T13:44:36.597910+0000","last_fullsized":"2024-04-25T13:44:36.597910+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:40.804684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.568860+0000","last_change":"2024-04-25T13:44:35.573295+0000","last_active":"2024-04-25T13:44:36.568860+0000","last_peered":"2024-04-25T13:44:36.568860+0000","last_clean":"2024-04-25T13:44:36.568860+0000","last_became_active":"2024-04-25T13:44:35.572952+0000","last_became_peered":"2024-04-25T13:44:35.572952+0000","last_unstale":"2024-04-25T13:44:36.568860+0000","last_undegraded":"2024-04-25T13:44:36.568860+0000","last_fullsized":"2024-04-25T13:44:36.568860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.923840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.616502+0000","last_change":"2024-04-25T13:44:36.616339+0000","last_active":"2024-04-25T13:44:36.616502+0000","last_peered":"2024-04-25T13:44:36.616502+0000","last_clean":"2024-04-25T13:44:36.616502+0000","last_became_active":"2024-04-25T13:44:36.616070+0000","last_became_peered":"2024-04-25T13:44:36.616070+0000","last_unstale":"2024-04-25T13:44:36.616502+0000","last_undegraded":"2024-04-25T13:44:36.616502+0000","last_fullsized":"2024-04-25T13:44:36.616502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:14.167705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568978+0000","last_change":"2024-04-25T13:44:35.582760+0000","last_active":"2024-04-25T13:44:37.568978+0000","last_peered":"2024-04-25T13:44:37.568978+0000","last_clean":"2024-04-25T13:44:37.568978+0000","last_became_active":"2024-04-25T13:44:35.582522+0000","last_became_peered":"2024-04-25T13:44:35.582522+0000","last_unstale":"2024-04-25T13:44:37.568978+0000","last_undegraded":"2024-04-25T13:44:37.568978+0000","last_fullsized":"2024-04-25T13:44:37.568978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:27.450229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.594530+0000","last_change":"2024-04-25T13:44:36.593968+0000","last_active":"2024-04-25T13:44:36.594530+0000","last_peered":"2024-04-25T13:44:36.594530+0000","last_clean":"2024-04-25T13:44:36.594530+0000","last_became_active":"2024-04-25T13:44:36.591147+0000","last_became_peered":"2024-04-25T13:44:36.591147+0000","last_unstale":"2024-04-25T13:44:36.594530+0000","last_undegraded":"2024-04-25T13:44:36.594530+0000","last_fullsized":"2024-04-25T13:44:36.594530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.494335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.567221+0000","last_change":"2024-04-25T13:44:35.577004+0000","last_active":"2024-04-25T13:44:36.567221+0000","last_peered":"2024-04-25T13:44:36.567221+0000","last_clean":"2024-04-25T13:44:36.567221+0000","last_became_active":"2024-04-25T13:44:35.576740+0000","last_became_peered":"2024-04-25T13:44:35.576740+0000","last_unstale":"2024-04-25T13:44:36.567221+0000","last_undegraded":"2024-04-25T13:44:36.567221+0000","last_fullsized":"2024-04-25T13:44:36.567221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:36.243968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.600129+0000","last_change":"2024-04-25T13:44:36.599772+0000","last_active":"2024-04-25T13:44:36.600129+0000","last_peered":"2024-04-25T13:44:36.600129+0000","last_clean":"2024-04-25T13:44:36.600129+0000","last_became_active":"2024-04-25T13:44:36.599065+0000","last_became_peered":"2024-04-25T13:44:36.599065+0000","last_unstale":"2024-04-25T13:44:36.600129+0000","last_undegraded":"2024-04-25T13:44:36.600129+0000","last_fullsized":"2024-04-25T13:44:36.600129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:04.891827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.566294+0000","last_change":"2024-04-25T13:44:35.572664+0000","last_active":"2024-04-25T13:44:36.566294+0000","last_peered":"2024-04-25T13:44:36.566294+0000","last_clean":"2024-04-25T13:44:36.566294+0000","last_became_active":"2024-04-25T13:44:35.572360+0000","last_became_peered":"2024-04-25T13:44:35.572360+0000","last_unstale":"2024-04-25T13:44:36.566294+0000","last_undegraded":"2024-04-25T13:44:36.566294+0000","last_fullsized":"2024-04-25T13:44:36.566294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:51.574092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592344+0000","last_change":"2024-04-25T13:44:36.591861+0000","last_active":"2024-04-25T13:44:36.592344+0000","last_peered":"2024-04-25T13:44:36.592344+0000","last_clean":"2024-04-25T13:44:36.592344+0000","last_became_active":"2024-04-25T13:44:36.589278+0000","last_became_peered":"2024-04-25T13:44:36.589278+0000","last_unstale":"2024-04-25T13:44:36.592344+0000","last_undegraded":"2024-04-25T13:44:36.592344+0000","last_fullsized":"2024-04-25T13:44:36.592344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:45.176351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567353+0000","last_change":"2024-04-25T13:44:35.593098+0000","last_active":"2024-04-25T13:44:37.567353+0000","last_peered":"2024-04-25T13:44:37.567353+0000","last_clean":"2024-04-25T13:44:37.567353+0000","last_became_active":"2024-04-25T13:44:35.591252+0000","last_became_peered":"2024-04-25T13:44:35.591252+0000","last_unstale":"2024-04-25T13:44:37.567353+0000","last_undegraded":"2024-04-25T13:44:37.567353+0000","last_fullsized":"2024-04-25T13:44:37.567353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:37.975492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.591111+0000","last_change":"2024-04-25T13:44:36.590855+0000","last_active":"2024-04-25T13:44:36.591111+0000","last_peered":"2024-04-25T13:44:36.591111+0000","last_clean":"2024-04-25T13:44:36.591111+0000","last_became_active":"2024-04-25T13:44:36.589329+0000","last_became_peered":"2024-04-25T13:44:36.589329+0000","last_unstale":"2024-04-25T13:44:36.591111+0000","last_undegraded":"2024-04-25T13:44:36.591111+0000","last_fullsized":"2024-04-25T13:44:36.591111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.392079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.585486+0000","last_change":"2024-04-25T13:44:35.592001+0000","last_active":"2024-04-25T13:44:36.585486+0000","last_peered":"2024-04-25T13:44:36.585486+0000","last_clean":"2024-04-25T13:44:36.585486+0000","last_became_active":"2024-04-25T13:44:35.589326+0000","last_became_peered":"2024-04-25T13:44:35.589326+0000","last_unstale":"2024-04-25T13:44:36.585486+0000","last_undegraded":"2024-04-25T13:44:36.585486+0000","last_fullsized":"2024-04-25T13:44:36.585486+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:01.052795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.599700+0000","last_change":"2024-04-25T13:44:36.598876+0000","last_active":"2024-04-25T13:44:36.599700+0000","last_peered":"2024-04-25T13:44:36.599700+0000","last_clean":"2024-04-25T13:44:36.599700+0000","last_became_active":"2024-04-25T13:44:36.598035+0000","last_became_peered":"2024-04-25T13:44:36.598035+0000","last_unstale":"2024-04-25T13:44:36.599700+0000","last_undegraded":"2024-04-25T13:44:36.599700+0000","last_fullsized":"2024-04-25T13:44:36.599700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:02.083840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.621491+0000","last_change":"2024-04-25T13:44:35.594518+0000","last_active":"2024-04-25T13:44:36.621491+0000","last_peered":"2024-04-25T13:44:36.621491+0000","last_clean":"2024-04-25T13:44:36.621491+0000","last_became_active":"2024-04-25T13:44:35.593036+0000","last_became_peered":"2024-04-25T13:44:35.593036+0000","last_unstale":"2024-04-25T13:44:36.621491+0000","last_undegraded":"2024-04-25T13:44:36.621491+0000","last_fullsized":"2024-04-25T13:44:36.621491+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:01.613154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":230,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.587207+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:44:36.587207+0000","last_peered":"2024-04-25T13:44:36.587207+0000","last_clean":"2024-04-25T13:44:36.587207+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:44:36.587207+0000","last_undegraded":"2024-04-25T13:44:36.587207+0000","last_fullsized":"2024-04-25T13:44:36.587207+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:50.852505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":996,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198979900,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":419,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194893,"statfs":{"total":200000000,"available":199570792,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":419,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194893,"statfs":{"total":200000000,"available":199570794,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":425,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194887,"statfs":{"total":200000000,"available":199564514,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":419,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194893,"statfs":{"total":200000000,"available":199570772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":420,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194891,"statfs":{"total":200000000,"available":199569272,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":421,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194891,"statfs":{"total":200000000,"available":199568656,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1001,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194310,"statfs":{"total":200000000,"available":198974216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:44:40.740 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:44:40.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.924+0000 7faebc423640 1 Processor -- start 2024-04-25T13:44:40.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.924+0000 7faebc423640 1 -- start start 2024-04-25T13:44:40.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faebc423640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faeb406f5a0 0x7faeb4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faebc423640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 0x7faeb4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faebc423640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faeb40b4bd0 0x7faeb40b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faebc423640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7faeb41619c0 con 0x7faeb40b4bd0 2024-04-25T13:44:40.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faebc423640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7faeb4073f10 con 0x7faeb406f5a0 2024-04-25T13:44:40.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faebc423640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faeb406fcf0 con 0x7faeb4161070 2024-04-25T13:44:40.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba9c0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faeb40b4bd0 0x7faeb40b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba9c0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faeb40b4bd0 0x7faeb40b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57098/0 (socket says 172.21.15.80:57098) 2024-04-25T13:44:40.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba9c0640 1 -- 172.21.15.80:0/3342996854 learned_addr learned my addr 172.21.15.80:0/3342996854 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:40.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba1bf640 1 --2- 172.21.15.80:0/3342996854 >> v2:172.21.15.87:3300/0 conn(0x7faeb406f5a0 0x7faeb4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba9c0640 1 -- 172.21.15.80:0/3342996854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 msgr2=0x7faeb4161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeb99be640 1 --2- 172.21.15.80:0/3342996854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 0x7faeb4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba9c0640 1 --2- 172.21.15.80:0/3342996854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 0x7faeb4161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba9c0640 1 -- 172.21.15.80:0/3342996854 >> v2:172.21.15.87:3300/0 conn(0x7faeb406f5a0 msgr2=0x7faeb4160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:40.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba9c0640 1 --2- 172.21.15.80:0/3342996854 >> v2:172.21.15.87:3300/0 conn(0x7faeb406f5a0 0x7faeb4160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.928+0000 7faeba9c0640 1 -- 172.21.15.80:0/3342996854 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb40b0c60 con 0x7faeb40b4bd0 2024-04-25T13:44:40.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faeba9c0640 1 --2- 172.21.15.80:0/3342996854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb40b4fe0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7faeb4071e30 tx=0x7faea401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2939ed0c8f0c98b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faeb91bd640 1 -- 172.21.15.80:0/3342996854 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea40a56d0 con 0x7faeb40b4bd0 2024-04-25T13:44:40.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faeb91bd640 1 -- 172.21.15.80:0/3342996854 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea40a5870 con 0x7faeb40b4bd0 2024-04-25T13:44:40.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faeb91bd640 1 -- 172.21.15.80:0/3342996854 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea40a5c40 con 0x7faeb40b4bd0 2024-04-25T13:44:40.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 -- 172.21.15.80:0/3342996854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 msgr2=0x7faeb40b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 --2- 172.21.15.80:0/3342996854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb40b4fe0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7faeb4071e30 tx=0x7faea401c960 comp rx=0 tx=0).stop 2024-04-25T13:44:40.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 -- 172.21.15.80:0/3342996854 shutdown_connections 2024-04-25T13:44:40.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 --2- 172.21.15.80:0/3342996854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb40b4fe0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 --2- 172.21.15.80:0/3342996854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 0x7faeb4161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 --2- 172.21.15.80:0/3342996854 >> v2:172.21.15.87:3300/0 conn(0x7faeb406f5a0 0x7faeb4160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 -- 172.21.15.80:0/3342996854 >> 172.21.15.80:0/3342996854 conn(0x7faeb406d080 msgr2=0x7faeb40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:40.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 -- 172.21.15.80:0/3342996854 shutdown_connections 2024-04-25T13:44:40.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 -- 172.21.15.80:0/3342996854 wait complete. 2024-04-25T13:44:40.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 Processor -- start 2024-04-25T13:44:40.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.932+0000 7faebc423640 1 -- start start 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb406f5a0 0x7faeb4156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb4153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 0x7faeb40cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faeb4072bb0 con 0x7faeb40b4bd0 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faeb40b5930 con 0x7faeb406f5a0 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faeb4072d30 con 0x7faeb4161070 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb4153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb4153fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57110/0 (socket says 172.21.15.80:57110) 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 -- 172.21.15.80:0/4262435269 learned_addr learned my addr 172.21.15.80:0/4262435269 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeba1bf640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb406f5a0 0x7faeb4156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeba9c0640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 0x7faeb40cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 -- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 msgr2=0x7faeb40cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 0x7faeb40cefb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 -- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb406f5a0 msgr2=0x7faeb4156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb406f5a0 0x7faeb4156ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 -- 172.21.15.80:0/4262435269 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faeb40ab540 con 0x7faeb40b4bd0 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb99be640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb4153fd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7faea800c7f0 tx=0x7faea800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb2ffd640 1 -- 172.21.15.80:0/4262435269 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8015070 con 0x7faeb40b4bd0 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb2ffd640 1 -- 172.21.15.80:0/4262435269 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faea8007820 con 0x7faeb40b4bd0 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faeb40ab7e0 con 0x7faeb40b4bd0 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb2ffd640 1 -- 172.21.15.80:0/4262435269 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faea8007c10 con 0x7faeb40b4bd0 2024-04-25T13:44:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faeb4155ed0 con 0x7faeb40b4bd0 2024-04-25T13:44:40.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faeb406a3a0 con 0x7faeb40b4bd0 2024-04-25T13:44:40.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb2ffd640 1 -- 172.21.15.80:0/4262435269 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faea8018070 con 0x7faeb40b4bd0 2024-04-25T13:44:40.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb2ffd640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faea006f400 0x7faea0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:40.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.936+0000 7faeb2ffd640 1 -- 172.21.15.80:0/4262435269 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7faea808f330 con 0x7faeb40b4bd0 2024-04-25T13:44:40.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.940+0000 7faeba1bf640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faea006f400 0x7faea0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:40.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.940+0000 7faeba1bf640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faea006f400 0x7faea0071910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faeb40b6e70 tx=0x7faeac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:40.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:40.944+0000 7faeb2ffd640 1 -- 172.21.15.80:0/4262435269 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faea805c2b0 con 0x7faeb40b4bd0 2024-04-25T13:44:41.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.092+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faeb40739c0 con 0x7faea006f400 2024-04-25T13:44:41.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.108+0000 7faeb2ffd640 1 -- 172.21.15.80:0/4262435269 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247527 (secure 0 0 0) 0x7faeb40739c0 con 0x7faea006f400 2024-04-25T13:44:41.110 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:41.112 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faea006f400 msgr2=0x7faea0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faea006f400 0x7faea0071910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7faeb40b6e70 tx=0x7faeac008040 comp rx=0 tx=0).stop 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 msgr2=0x7faeb4153fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb4153fd0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7faea800c7f0 tx=0x7faea800ccb0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 shutdown_connections 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faea006f400 0x7faea0071910 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faeb4161070 0x7faeb40cefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faeb40b4bd0 0x7faeb4153fd0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 --2- 172.21.15.80:0/4262435269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faeb406f5a0 0x7faeb4156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 >> 172.21.15.80:0/4262435269 conn(0x7faeb406d080 msgr2=0x7faeb40b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:41.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 shutdown_connections 2024-04-25T13:44:41.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.112+0000 7faebc423640 1 -- 172.21.15.80:0/4262435269 wait complete. 2024-04-25T13:44:41.130 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-04-25T13:44:40.185849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":123,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4522,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557977,"statfs":{"total":1600000000,"available":1595368916,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.637042"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.605172+0000","last_change":"2024-04-25T13:44:36.604793+0000","last_active":"2024-04-25T13:44:36.605172+0000","last_peered":"2024-04-25T13:44:36.605172+0000","last_clean":"2024-04-25T13:44:36.605172+0000","last_became_active":"2024-04-25T13:44:36.594137+0000","last_became_peered":"2024-04-25T13:44:36.594137+0000","last_unstale":"2024-04-25T13:44:36.605172+0000","last_undegraded":"2024-04-25T13:44:36.605172+0000","last_fullsized":"2024-04-25T13:44:36.605172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:54.035463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.614638+0000","last_change":"2024-04-25T13:44:36.614415+0000","last_active":"2024-04-25T13:44:36.614638+0000","last_peered":"2024-04-25T13:44:36.614638+0000","last_clean":"2024-04-25T13:44:36.614638+0000","last_became_active":"2024-04-25T13:44:36.613878+0000","last_became_peered":"2024-04-25T13:44:36.613878+0000","last_unstale":"2024-04-25T13:44:36.614638+0000","last_undegraded":"2024-04-25T13:44:36.614638+0000","last_fullsized":"2024-04-25T13:44:36.614638+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:10.668771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.596472+0000","last_change":"2024-04-25T13:44:36.594890+0000","last_active":"2024-04-25T13:44:36.596472+0000","last_peered":"2024-04-25T13:44:36.596472+0000","last_clean":"2024-04-25T13:44:36.596472+0000","last_became_active":"2024-04-25T13:44:36.591213+0000","last_became_peered":"2024-04-25T13:44:36.591213+0000","last_unstale":"2024-04-25T13:44:36.596472+0000","last_undegraded":"2024-04-25T13:44:36.596472+0000","last_fullsized":"2024-04-25T13:44:36.596472+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:59.877094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.594258+0000","last_change":"2024-04-25T13:44:36.593830+0000","last_active":"2024-04-25T13:44:36.594258+0000","last_peered":"2024-04-25T13:44:36.594258+0000","last_clean":"2024-04-25T13:44:36.594258+0000","last_became_active":"2024-04-25T13:44:36.590366+0000","last_became_peered":"2024-04-25T13:44:36.590366+0000","last_unstale":"2024-04-25T13:44:36.594258+0000","last_undegraded":"2024-04-25T13:44:36.594258+0000","last_fullsized":"2024-04-25T13:44:36.594258+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:30.989638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.583729+0000","last_change":"2024-04-25T13:44:36.583254+0000","last_active":"2024-04-25T13:44:36.583729+0000","last_peered":"2024-04-25T13:44:36.583729+0000","last_clean":"2024-04-25T13:44:36.583729+0000","last_became_active":"2024-04-25T13:44:36.581185+0000","last_became_peered":"2024-04-25T13:44:36.581185+0000","last_unstale":"2024-04-25T13:44:36.583729+0000","last_undegraded":"2024-04-25T13:44:36.583729+0000","last_fullsized":"2024-04-25T13:44:36.583729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:14.886088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.615095+0000","last_change":"2024-04-25T13:44:36.612698+0000","last_active":"2024-04-25T13:44:36.615095+0000","last_peered":"2024-04-25T13:44:36.615095+0000","last_clean":"2024-04-25T13:44:36.615095+0000","last_became_active":"2024-04-25T13:44:36.609084+0000","last_became_peered":"2024-04-25T13:44:36.609084+0000","last_unstale":"2024-04-25T13:44:36.615095+0000","last_undegraded":"2024-04-25T13:44:36.615095+0000","last_fullsized":"2024-04-25T13:44:36.615095+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:18.434464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.616995+0000","last_change":"2024-04-25T13:44:36.616773+0000","last_active":"2024-04-25T13:44:36.616995+0000","last_peered":"2024-04-25T13:44:36.616995+0000","last_clean":"2024-04-25T13:44:36.616995+0000","last_became_active":"2024-04-25T13:44:36.596021+0000","last_became_peered":"2024-04-25T13:44:36.596021+0000","last_unstale":"2024-04-25T13:44:36.616995+0000","last_undegraded":"2024-04-25T13:44:36.616995+0000","last_fullsized":"2024-04-25T13:44:36.616995+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:10.192526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592049+0000","last_change":"2024-04-25T13:44:36.591777+0000","last_active":"2024-04-25T13:44:36.592049+0000","last_peered":"2024-04-25T13:44:36.592049+0000","last_clean":"2024-04-25T13:44:36.592049+0000","last_became_active":"2024-04-25T13:44:36.590176+0000","last_became_peered":"2024-04-25T13:44:36.590176+0000","last_unstale":"2024-04-25T13:44:36.592049+0000","last_undegraded":"2024-04-25T13:44:36.592049+0000","last_fullsized":"2024-04-25T13:44:36.592049+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:30.007608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.587097+0000","last_change":"2024-04-25T13:44:36.586812+0000","last_active":"2024-04-25T13:44:36.587097+0000","last_peered":"2024-04-25T13:44:36.587097+0000","last_clean":"2024-04-25T13:44:36.587097+0000","last_became_active":"2024-04-25T13:44:36.586075+0000","last_became_peered":"2024-04-25T13:44:36.586075+0000","last_unstale":"2024-04-25T13:44:36.587097+0000","last_undegraded":"2024-04-25T13:44:36.587097+0000","last_fullsized":"2024-04-25T13:44:36.587097+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:17.526462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567516+0000","last_change":"2024-04-25T13:44:36.611721+0000","last_active":"2024-04-25T13:44:37.567516+0000","last_peered":"2024-04-25T13:44:37.567516+0000","last_clean":"2024-04-25T13:44:37.567516+0000","last_became_active":"2024-04-25T13:44:36.606078+0000","last_became_peered":"2024-04-25T13:44:36.606078+0000","last_unstale":"2024-04-25T13:44:37.567516+0000","last_undegraded":"2024-04-25T13:44:37.567516+0000","last_fullsized":"2024-04-25T13:44:37.567516+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:56.938190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.608050+0000","last_change":"2024-04-25T13:44:36.607453+0000","last_active":"2024-04-25T13:44:36.608050+0000","last_peered":"2024-04-25T13:44:36.608050+0000","last_clean":"2024-04-25T13:44:36.608050+0000","last_became_active":"2024-04-25T13:44:36.601092+0000","last_became_peered":"2024-04-25T13:44:36.601092+0000","last_unstale":"2024-04-25T13:44:36.608050+0000","last_undegraded":"2024-04-25T13:44:36.608050+0000","last_fullsized":"2024-04-25T13:44:36.608050+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:53.079863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.577492+0000","last_change":"2024-04-25T13:44:36.576437+0000","last_active":"2024-04-25T13:44:36.577492+0000","last_peered":"2024-04-25T13:44:36.577492+0000","last_clean":"2024-04-25T13:44:36.577492+0000","last_became_active":"2024-04-25T13:44:36.573263+0000","last_became_peered":"2024-04-25T13:44:36.573263+0000","last_unstale":"2024-04-25T13:44:36.577492+0000","last_undegraded":"2024-04-25T13:44:36.577492+0000","last_fullsized":"2024-04-25T13:44:36.577492+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:25.250083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568126+0000","last_change":"2024-04-25T13:44:36.596821+0000","last_active":"2024-04-25T13:44:37.568126+0000","last_peered":"2024-04-25T13:44:37.568126+0000","last_clean":"2024-04-25T13:44:37.568126+0000","last_became_active":"2024-04-25T13:44:36.593794+0000","last_became_peered":"2024-04-25T13:44:36.593794+0000","last_unstale":"2024-04-25T13:44:37.568126+0000","last_undegraded":"2024-04-25T13:44:37.568126+0000","last_fullsized":"2024-04-25T13:44:37.568126+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:58.066349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.583667+0000","last_change":"2024-04-25T13:44:36.582851+0000","last_active":"2024-04-25T13:44:36.583667+0000","last_peered":"2024-04-25T13:44:36.583667+0000","last_clean":"2024-04-25T13:44:36.583667+0000","last_became_active":"2024-04-25T13:44:36.582114+0000","last_became_peered":"2024-04-25T13:44:36.582114+0000","last_unstale":"2024-04-25T13:44:36.583667+0000","last_undegraded":"2024-04-25T13:44:36.583667+0000","last_fullsized":"2024-04-25T13:44:36.583667+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:34.749825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568601+0000","last_change":"2024-04-25T13:44:36.597632+0000","last_active":"2024-04-25T13:44:37.568601+0000","last_peered":"2024-04-25T13:44:37.568601+0000","last_clean":"2024-04-25T13:44:37.568601+0000","last_became_active":"2024-04-25T13:44:36.591108+0000","last_became_peered":"2024-04-25T13:44:36.591108+0000","last_unstale":"2024-04-25T13:44:37.568601+0000","last_undegraded":"2024-04-25T13:44:37.568601+0000","last_fullsized":"2024-04-25T13:44:37.568601+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:47.222008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568240+0000","last_change":"2024-04-25T13:44:36.610758+0000","last_active":"2024-04-25T13:44:37.568240+0000","last_peered":"2024-04-25T13:44:37.568240+0000","last_clean":"2024-04-25T13:44:37.568240+0000","last_became_active":"2024-04-25T13:44:36.610373+0000","last_became_peered":"2024-04-25T13:44:36.610373+0000","last_unstale":"2024-04-25T13:44:37.568240+0000","last_undegraded":"2024-04-25T13:44:37.568240+0000","last_fullsized":"2024-04-25T13:44:37.568240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:37.460337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.595871+0000","last_change":"2024-04-25T13:44:36.595459+0000","last_active":"2024-04-25T13:44:36.595871+0000","last_peered":"2024-04-25T13:44:36.595871+0000","last_clean":"2024-04-25T13:44:36.595871+0000","last_became_active":"2024-04-25T13:44:36.593269+0000","last_became_peered":"2024-04-25T13:44:36.593269+0000","last_unstale":"2024-04-25T13:44:36.595871+0000","last_undegraded":"2024-04-25T13:44:36.595871+0000","last_fullsized":"2024-04-25T13:44:36.595871+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:43.041990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.599393+0000","last_change":"2024-04-25T13:44:36.598869+0000","last_active":"2024-04-25T13:44:36.599393+0000","last_peered":"2024-04-25T13:44:36.599393+0000","last_clean":"2024-04-25T13:44:36.599393+0000","last_became_active":"2024-04-25T13:44:36.591722+0000","last_became_peered":"2024-04-25T13:44:36.591722+0000","last_unstale":"2024-04-25T13:44:36.599393+0000","last_undegraded":"2024-04-25T13:44:36.599393+0000","last_fullsized":"2024-04-25T13:44:36.599393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:16.664003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592956+0000","last_change":"2024-04-25T13:44:36.592646+0000","last_active":"2024-04-25T13:44:36.592956+0000","last_peered":"2024-04-25T13:44:36.592956+0000","last_clean":"2024-04-25T13:44:36.592956+0000","last_became_active":"2024-04-25T13:44:36.590618+0000","last_became_peered":"2024-04-25T13:44:36.590618+0000","last_unstale":"2024-04-25T13:44:36.592956+0000","last_undegraded":"2024-04-25T13:44:36.592956+0000","last_fullsized":"2024-04-25T13:44:36.592956+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:49.804159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568879+0000","last_change":"2024-04-25T13:44:36.591998+0000","last_active":"2024-04-25T13:44:37.568879+0000","last_peered":"2024-04-25T13:44:37.568879+0000","last_clean":"2024-04-25T13:44:37.568879+0000","last_became_active":"2024-04-25T13:44:36.588140+0000","last_became_peered":"2024-04-25T13:44:36.588140+0000","last_unstale":"2024-04-25T13:44:37.568879+0000","last_undegraded":"2024-04-25T13:44:37.568879+0000","last_fullsized":"2024-04-25T13:44:37.568879+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:55.876469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567241+0000","last_change":"2024-04-25T13:44:36.609461+0000","last_active":"2024-04-25T13:44:37.567241+0000","last_peered":"2024-04-25T13:44:37.567241+0000","last_clean":"2024-04-25T13:44:37.567241+0000","last_became_active":"2024-04-25T13:44:36.608790+0000","last_became_peered":"2024-04-25T13:44:36.608790+0000","last_unstale":"2024-04-25T13:44:37.567241+0000","last_undegraded":"2024-04-25T13:44:37.567241+0000","last_fullsized":"2024-04-25T13:44:37.567241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:28.592867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567824+0000","last_change":"2024-04-25T13:44:36.612797+0000","last_active":"2024-04-25T13:44:37.567824+0000","last_peered":"2024-04-25T13:44:37.567824+0000","last_clean":"2024-04-25T13:44:37.567824+0000","last_became_active":"2024-04-25T13:44:36.608957+0000","last_became_peered":"2024-04-25T13:44:36.608957+0000","last_unstale":"2024-04-25T13:44:37.567824+0000","last_undegraded":"2024-04-25T13:44:37.567824+0000","last_fullsized":"2024-04-25T13:44:37.567824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:44.801001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.583828+0000","last_change":"2024-04-25T13:44:36.581586+0000","last_active":"2024-04-25T13:44:36.583828+0000","last_peered":"2024-04-25T13:44:36.583828+0000","last_clean":"2024-04-25T13:44:36.583828+0000","last_became_active":"2024-04-25T13:44:36.580818+0000","last_became_peered":"2024-04-25T13:44:36.580818+0000","last_unstale":"2024-04-25T13:44:36.583828+0000","last_undegraded":"2024-04-25T13:44:36.583828+0000","last_fullsized":"2024-04-25T13:44:36.583828+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:23.328034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.595470+0000","last_change":"2024-04-25T13:44:36.594619+0000","last_active":"2024-04-25T13:44:36.595470+0000","last_peered":"2024-04-25T13:44:36.595470+0000","last_clean":"2024-04-25T13:44:36.595470+0000","last_became_active":"2024-04-25T13:44:36.593907+0000","last_became_peered":"2024-04-25T13:44:36.593907+0000","last_unstale":"2024-04-25T13:44:36.595470+0000","last_undegraded":"2024-04-25T13:44:36.595470+0000","last_fullsized":"2024-04-25T13:44:36.595470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:30.623221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567970+0000","last_change":"2024-04-25T13:44:36.613666+0000","last_active":"2024-04-25T13:44:37.567970+0000","last_peered":"2024-04-25T13:44:37.567970+0000","last_clean":"2024-04-25T13:44:37.567970+0000","last_became_active":"2024-04-25T13:44:36.610791+0000","last_became_peered":"2024-04-25T13:44:36.610791+0000","last_unstale":"2024-04-25T13:44:37.567970+0000","last_undegraded":"2024-04-25T13:44:37.567970+0000","last_fullsized":"2024-04-25T13:44:37.567970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:58:03.226950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592048+0000","last_change":"2024-04-25T13:44:36.591716+0000","last_active":"2024-04-25T13:44:36.592048+0000","last_peered":"2024-04-25T13:44:36.592048+0000","last_clean":"2024-04-25T13:44:36.592048+0000","last_became_active":"2024-04-25T13:44:36.591003+0000","last_became_peered":"2024-04-25T13:44:36.591003+0000","last_unstale":"2024-04-25T13:44:36.592048+0000","last_undegraded":"2024-04-25T13:44:36.592048+0000","last_fullsized":"2024-04-25T13:44:36.592048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:09.615310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.596599+0000","last_change":"2024-04-25T13:44:36.595878+0000","last_active":"2024-04-25T13:44:36.596599+0000","last_peered":"2024-04-25T13:44:36.596599+0000","last_clean":"2024-04-25T13:44:36.596599+0000","last_became_active":"2024-04-25T13:44:36.595203+0000","last_became_peered":"2024-04-25T13:44:36.595203+0000","last_unstale":"2024-04-25T13:44:36.596599+0000","last_undegraded":"2024-04-25T13:44:36.596599+0000","last_fullsized":"2024-04-25T13:44:36.596599+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:37.948968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.566643+0000","last_change":"2024-04-25T13:44:36.600721+0000","last_active":"2024-04-25T13:44:37.566643+0000","last_peered":"2024-04-25T13:44:37.566643+0000","last_clean":"2024-04-25T13:44:37.566643+0000","last_became_active":"2024-04-25T13:44:36.599892+0000","last_became_peered":"2024-04-25T13:44:36.599892+0000","last_unstale":"2024-04-25T13:44:37.566643+0000","last_undegraded":"2024-04-25T13:44:37.566643+0000","last_fullsized":"2024-04-25T13:44:37.566643+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:05.758222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.589960+0000","last_change":"2024-04-25T13:44:36.589663+0000","last_active":"2024-04-25T13:44:36.589960+0000","last_peered":"2024-04-25T13:44:36.589960+0000","last_clean":"2024-04-25T13:44:36.589960+0000","last_became_active":"2024-04-25T13:44:36.588967+0000","last_became_peered":"2024-04-25T13:44:36.588967+0000","last_unstale":"2024-04-25T13:44:36.589960+0000","last_undegraded":"2024-04-25T13:44:36.589960+0000","last_fullsized":"2024-04-25T13:44:36.589960+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:47.273886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.587172+0000","last_change":"2024-04-25T13:44:36.586834+0000","last_active":"2024-04-25T13:44:36.587172+0000","last_peered":"2024-04-25T13:44:36.587172+0000","last_clean":"2024-04-25T13:44:36.587172+0000","last_became_active":"2024-04-25T13:44:36.585982+0000","last_became_peered":"2024-04-25T13:44:36.585982+0000","last_unstale":"2024-04-25T13:44:36.587172+0000","last_undegraded":"2024-04-25T13:44:36.587172+0000","last_fullsized":"2024-04-25T13:44:36.587172+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:24.950626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.590081+0000","last_change":"2024-04-25T13:44:36.589728+0000","last_active":"2024-04-25T13:44:36.590081+0000","last_peered":"2024-04-25T13:44:36.590081+0000","last_clean":"2024-04-25T13:44:36.590081+0000","last_became_active":"2024-04-25T13:44:36.588869+0000","last_became_peered":"2024-04-25T13:44:36.588869+0000","last_unstale":"2024-04-25T13:44:36.590081+0000","last_undegraded":"2024-04-25T13:44:36.590081+0000","last_fullsized":"2024-04-25T13:44:36.590081+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:35.747145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.600600+0000","last_change":"2024-04-25T13:44:36.600129+0000","last_active":"2024-04-25T13:44:36.600600+0000","last_peered":"2024-04-25T13:44:36.600600+0000","last_clean":"2024-04-25T13:44:36.600600+0000","last_became_active":"2024-04-25T13:44:36.599389+0000","last_became_peered":"2024-04-25T13:44:36.599389+0000","last_unstale":"2024-04-25T13:44:36.600600+0000","last_undegraded":"2024-04-25T13:44:36.600600+0000","last_fullsized":"2024-04-25T13:44:36.600600+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:17.177903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.602217+0000","last_change":"2024-04-25T13:44:36.601755+0000","last_active":"2024-04-25T13:44:36.602217+0000","last_peered":"2024-04-25T13:44:36.602217+0000","last_clean":"2024-04-25T13:44:36.602217+0000","last_became_active":"2024-04-25T13:44:36.599573+0000","last_became_peered":"2024-04-25T13:44:36.599573+0000","last_unstale":"2024-04-25T13:44:36.602217+0000","last_undegraded":"2024-04-25T13:44:36.602217+0000","last_fullsized":"2024-04-25T13:44:36.602217+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:56.925889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"81'4","reported_seq":16,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.622835+0000","last_change":"2024-04-25T13:44:35.591054+0000","last_active":"2024-04-25T13:44:36.622835+0000","last_peered":"2024-04-25T13:44:36.622835+0000","last_clean":"2024-04-25T13:44:36.622835+0000","last_became_active":"2024-04-25T13:44:35.590343+0000","last_became_peered":"2024-04-25T13:44:35.590343+0000","last_unstale":"2024-04-25T13:44:36.622835+0000","last_undegraded":"2024-04-25T13:44:36.622835+0000","last_fullsized":"2024-04-25T13:44:36.622835+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:14.514741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.585822+0000","last_change":"2024-04-25T13:44:36.585172+0000","last_active":"2024-04-25T13:44:36.585822+0000","last_peered":"2024-04-25T13:44:36.585822+0000","last_clean":"2024-04-25T13:44:36.585822+0000","last_became_active":"2024-04-25T13:44:36.584271+0000","last_became_peered":"2024-04-25T13:44:36.584271+0000","last_unstale":"2024-04-25T13:44:36.585822+0000","last_undegraded":"2024-04-25T13:44:36.585822+0000","last_fullsized":"2024-04-25T13:44:36.585822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:27.553217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"81'1","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568395+0000","last_change":"2024-04-25T13:44:35.575627+0000","last_active":"2024-04-25T13:44:37.568395+0000","last_peered":"2024-04-25T13:44:37.568395+0000","last_clean":"2024-04-25T13:44:37.568395+0000","last_became_active":"2024-04-25T13:44:35.575156+0000","last_became_peered":"2024-04-25T13:44:35.575156+0000","last_unstale":"2024-04-25T13:44:37.568395+0000","last_undegraded":"2024-04-25T13:44:37.568395+0000","last_fullsized":"2024-04-25T13:44:37.568395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:19.115196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.612238+0000","last_change":"2024-04-25T13:44:36.610902+0000","last_active":"2024-04-25T13:44:36.612238+0000","last_peered":"2024-04-25T13:44:36.612238+0000","last_clean":"2024-04-25T13:44:36.612238+0000","last_became_active":"2024-04-25T13:44:36.607536+0000","last_became_peered":"2024-04-25T13:44:36.607536+0000","last_unstale":"2024-04-25T13:44:36.612238+0000","last_undegraded":"2024-04-25T13:44:36.612238+0000","last_fullsized":"2024-04-25T13:44:36.612238+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:56.253931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568495+0000","last_change":"2024-04-25T13:44:35.595532+0000","last_active":"2024-04-25T13:44:37.568495+0000","last_peered":"2024-04-25T13:44:37.568495+0000","last_clean":"2024-04-25T13:44:37.568495+0000","last_became_active":"2024-04-25T13:44:35.594157+0000","last_became_peered":"2024-04-25T13:44:35.594157+0000","last_unstale":"2024-04-25T13:44:37.568495+0000","last_undegraded":"2024-04-25T13:44:37.568495+0000","last_fullsized":"2024-04-25T13:44:37.568495+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:47.198139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.584318+0000","last_change":"2024-04-25T13:44:36.583441+0000","last_active":"2024-04-25T13:44:36.584318+0000","last_peered":"2024-04-25T13:44:36.584318+0000","last_clean":"2024-04-25T13:44:36.584318+0000","last_became_active":"2024-04-25T13:44:36.582707+0000","last_became_peered":"2024-04-25T13:44:36.582707+0000","last_unstale":"2024-04-25T13:44:36.584318+0000","last_undegraded":"2024-04-25T13:44:36.584318+0000","last_fullsized":"2024-04-25T13:44:36.584318+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:26:39.281041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.624758+0000","last_change":"2024-04-25T13:44:35.600018+0000","last_active":"2024-04-25T13:44:36.624758+0000","last_peered":"2024-04-25T13:44:36.624758+0000","last_clean":"2024-04-25T13:44:36.624758+0000","last_became_active":"2024-04-25T13:44:35.597655+0000","last_became_peered":"2024-04-25T13:44:35.597655+0000","last_unstale":"2024-04-25T13:44:36.624758+0000","last_undegraded":"2024-04-25T13:44:36.624758+0000","last_fullsized":"2024-04-25T13:44:36.624758+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:26.870545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.586733+0000","last_change":"2024-04-25T13:44:36.586390+0000","last_active":"2024-04-25T13:44:36.586733+0000","last_peered":"2024-04-25T13:44:36.586733+0000","last_clean":"2024-04-25T13:44:36.586733+0000","last_became_active":"2024-04-25T13:44:36.585611+0000","last_became_peered":"2024-04-25T13:44:36.585611+0000","last_unstale":"2024-04-25T13:44:36.586733+0000","last_undegraded":"2024-04-25T13:44:36.586733+0000","last_fullsized":"2024-04-25T13:44:36.586733+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:38.433758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.622701+0000","last_change":"2024-04-25T13:44:35.571834+0000","last_active":"2024-04-25T13:44:36.622701+0000","last_peered":"2024-04-25T13:44:36.622701+0000","last_clean":"2024-04-25T13:44:36.622701+0000","last_became_active":"2024-04-25T13:44:35.571501+0000","last_became_peered":"2024-04-25T13:44:35.571501+0000","last_unstale":"2024-04-25T13:44:36.622701+0000","last_undegraded":"2024-04-25T13:44:36.622701+0000","last_fullsized":"2024-04-25T13:44:36.622701+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:36:26.236173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568342+0000","last_change":"2024-04-25T13:44:36.612364+0000","last_active":"2024-04-25T13:44:37.568342+0000","last_peered":"2024-04-25T13:44:37.568342+0000","last_clean":"2024-04-25T13:44:37.568342+0000","last_became_active":"2024-04-25T13:44:36.607520+0000","last_became_peered":"2024-04-25T13:44:36.607520+0000","last_unstale":"2024-04-25T13:44:37.568342+0000","last_undegraded":"2024-04-25T13:44:37.568342+0000","last_fullsized":"2024-04-25T13:44:37.568342+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:52.967931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.621607+0000","last_change":"2024-04-25T13:44:35.597254+0000","last_active":"2024-04-25T13:44:36.621607+0000","last_peered":"2024-04-25T13:44:36.621607+0000","last_clean":"2024-04-25T13:44:36.621607+0000","last_became_active":"2024-04-25T13:44:35.593852+0000","last_became_peered":"2024-04-25T13:44:35.593852+0000","last_unstale":"2024-04-25T13:44:36.621607+0000","last_undegraded":"2024-04-25T13:44:36.621607+0000","last_fullsized":"2024-04-25T13:44:36.621607+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:13.384285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.607790+0000","last_change":"2024-04-25T13:44:36.607567+0000","last_active":"2024-04-25T13:44:36.607790+0000","last_peered":"2024-04-25T13:44:36.607790+0000","last_clean":"2024-04-25T13:44:36.607790+0000","last_became_active":"2024-04-25T13:44:36.607044+0000","last_became_peered":"2024-04-25T13:44:36.607044+0000","last_unstale":"2024-04-25T13:44:36.607790+0000","last_undegraded":"2024-04-25T13:44:36.607790+0000","last_fullsized":"2024-04-25T13:44:36.607790+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:31.652704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.566594+0000","last_change":"2024-04-25T13:44:35.572213+0000","last_active":"2024-04-25T13:44:36.566594+0000","last_peered":"2024-04-25T13:44:36.566594+0000","last_clean":"2024-04-25T13:44:36.566594+0000","last_became_active":"2024-04-25T13:44:35.571904+0000","last_became_peered":"2024-04-25T13:44:35.571904+0000","last_unstale":"2024-04-25T13:44:36.566594+0000","last_undegraded":"2024-04-25T13:44:36.566594+0000","last_fullsized":"2024-04-25T13:44:36.566594+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:14.510930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568132+0000","last_change":"2024-04-25T13:44:36.595986+0000","last_active":"2024-04-25T13:44:37.568132+0000","last_peered":"2024-04-25T13:44:37.568132+0000","last_clean":"2024-04-25T13:44:37.568132+0000","last_became_active":"2024-04-25T13:44:36.595350+0000","last_became_peered":"2024-04-25T13:44:36.595350+0000","last_unstale":"2024-04-25T13:44:37.568132+0000","last_undegraded":"2024-04-25T13:44:37.568132+0000","last_fullsized":"2024-04-25T13:44:37.568132+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:07.945298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.569031+0000","last_change":"2024-04-25T13:44:35.565632+0000","last_active":"2024-04-25T13:44:36.569031+0000","last_peered":"2024-04-25T13:44:36.569031+0000","last_clean":"2024-04-25T13:44:36.569031+0000","last_became_active":"2024-04-25T13:44:35.564264+0000","last_became_peered":"2024-04-25T13:44:35.564264+0000","last_unstale":"2024-04-25T13:44:36.569031+0000","last_undegraded":"2024-04-25T13:44:36.569031+0000","last_fullsized":"2024-04-25T13:44:36.569031+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:58:13.272951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.620263+0000","last_change":"2024-04-25T13:44:36.620003+0000","last_active":"2024-04-25T13:44:36.620263+0000","last_peered":"2024-04-25T13:44:36.620263+0000","last_clean":"2024-04-25T13:44:36.620263+0000","last_became_active":"2024-04-25T13:44:36.619209+0000","last_became_peered":"2024-04-25T13:44:36.619209+0000","last_unstale":"2024-04-25T13:44:36.620263+0000","last_undegraded":"2024-04-25T13:44:36.620263+0000","last_fullsized":"2024-04-25T13:44:36.620263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:58.286534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568610+0000","last_change":"2024-04-25T13:44:35.574184+0000","last_active":"2024-04-25T13:44:37.568610+0000","last_peered":"2024-04-25T13:44:37.568610+0000","last_clean":"2024-04-25T13:44:37.568610+0000","last_became_active":"2024-04-25T13:44:35.573366+0000","last_became_peered":"2024-04-25T13:44:35.573366+0000","last_unstale":"2024-04-25T13:44:37.568610+0000","last_undegraded":"2024-04-25T13:44:37.568610+0000","last_fullsized":"2024-04-25T13:44:37.568610+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:58.821551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.609048+0000","last_change":"2024-04-25T13:44:36.608623+0000","last_active":"2024-04-25T13:44:36.609048+0000","last_peered":"2024-04-25T13:44:36.609048+0000","last_clean":"2024-04-25T13:44:36.609048+0000","last_became_active":"2024-04-25T13:44:36.602565+0000","last_became_peered":"2024-04-25T13:44:36.602565+0000","last_unstale":"2024-04-25T13:44:36.609048+0000","last_undegraded":"2024-04-25T13:44:36.609048+0000","last_fullsized":"2024-04-25T13:44:36.609048+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:24:48.037163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.597892+0000","last_change":"2024-04-25T13:44:35.598272+0000","last_active":"2024-04-25T13:44:36.597892+0000","last_peered":"2024-04-25T13:44:36.597892+0000","last_clean":"2024-04-25T13:44:36.597892+0000","last_became_active":"2024-04-25T13:44:35.594752+0000","last_became_peered":"2024-04-25T13:44:35.594752+0000","last_unstale":"2024-04-25T13:44:36.597892+0000","last_undegraded":"2024-04-25T13:44:36.597892+0000","last_fullsized":"2024-04-25T13:44:36.597892+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:03.199875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.607890+0000","last_change":"2024-04-25T13:44:36.607580+0000","last_active":"2024-04-25T13:44:36.607890+0000","last_peered":"2024-04-25T13:44:36.607890+0000","last_clean":"2024-04-25T13:44:36.607890+0000","last_became_active":"2024-04-25T13:44:36.606885+0000","last_became_peered":"2024-04-25T13:44:36.606885+0000","last_unstale":"2024-04-25T13:44:36.607890+0000","last_undegraded":"2024-04-25T13:44:36.607890+0000","last_fullsized":"2024-04-25T13:44:36.607890+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:54.174567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"81'3","reported_seq":15,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.624474+0000","last_change":"2024-04-25T13:44:35.591433+0000","last_active":"2024-04-25T13:44:36.624474+0000","last_peered":"2024-04-25T13:44:36.624474+0000","last_clean":"2024-04-25T13:44:36.624474+0000","last_became_active":"2024-04-25T13:44:35.590616+0000","last_became_peered":"2024-04-25T13:44:35.590616+0000","last_unstale":"2024-04-25T13:44:36.624474+0000","last_undegraded":"2024-04-25T13:44:36.624474+0000","last_fullsized":"2024-04-25T13:44:36.624474+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:56.748763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.571155+0000","last_change":"2024-04-25T13:44:36.570641+0000","last_active":"2024-04-25T13:44:36.571155+0000","last_peered":"2024-04-25T13:44:36.571155+0000","last_clean":"2024-04-25T13:44:36.571155+0000","last_became_active":"2024-04-25T13:44:36.569938+0000","last_became_peered":"2024-04-25T13:44:36.569938+0000","last_unstale":"2024-04-25T13:44:36.571155+0000","last_undegraded":"2024-04-25T13:44:36.571155+0000","last_fullsized":"2024-04-25T13:44:36.571155+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:13.181080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"81'6","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.626804+0000","last_change":"2024-04-25T13:44:35.563616+0000","last_active":"2024-04-25T13:44:36.626804+0000","last_peered":"2024-04-25T13:44:36.626804+0000","last_clean":"2024-04-25T13:44:36.626804+0000","last_became_active":"2024-04-25T13:44:35.563329+0000","last_became_peered":"2024-04-25T13:44:35.563329+0000","last_unstale":"2024-04-25T13:44:36.626804+0000","last_undegraded":"2024-04-25T13:44:36.626804+0000","last_fullsized":"2024-04-25T13:44:36.626804+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:56.290232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.605966+0000","last_change":"2024-04-25T13:44:36.605628+0000","last_active":"2024-04-25T13:44:36.605966+0000","last_peered":"2024-04-25T13:44:36.605966+0000","last_clean":"2024-04-25T13:44:36.605966+0000","last_became_active":"2024-04-25T13:44:36.595132+0000","last_became_peered":"2024-04-25T13:44:36.595132+0000","last_unstale":"2024-04-25T13:44:36.605966+0000","last_undegraded":"2024-04-25T13:44:36.605966+0000","last_fullsized":"2024-04-25T13:44:36.605966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:21.264823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.623156+0000","last_change":"2024-04-25T13:44:35.584058+0000","last_active":"2024-04-25T13:44:36.623156+0000","last_peered":"2024-04-25T13:44:36.623156+0000","last_clean":"2024-04-25T13:44:36.623156+0000","last_became_active":"2024-04-25T13:44:35.583810+0000","last_became_peered":"2024-04-25T13:44:35.583810+0000","last_unstale":"2024-04-25T13:44:36.623156+0000","last_undegraded":"2024-04-25T13:44:36.623156+0000","last_fullsized":"2024-04-25T13:44:36.623156+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:17.672688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.615653+0000","last_change":"2024-04-25T13:44:36.615173+0000","last_active":"2024-04-25T13:44:36.615653+0000","last_peered":"2024-04-25T13:44:36.615653+0000","last_clean":"2024-04-25T13:44:36.615653+0000","last_became_active":"2024-04-25T13:44:36.614710+0000","last_became_peered":"2024-04-25T13:44:36.614710+0000","last_unstale":"2024-04-25T13:44:36.615653+0000","last_undegraded":"2024-04-25T13:44:36.615653+0000","last_fullsized":"2024-04-25T13:44:36.615653+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:59:46.290346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.584945+0000","last_change":"2024-04-25T13:44:35.599646+0000","last_active":"2024-04-25T13:44:36.584945+0000","last_peered":"2024-04-25T13:44:36.584945+0000","last_clean":"2024-04-25T13:44:36.584945+0000","last_became_active":"2024-04-25T13:44:35.595707+0000","last_became_peered":"2024-04-25T13:44:35.595707+0000","last_unstale":"2024-04-25T13:44:36.584945+0000","last_undegraded":"2024-04-25T13:44:36.584945+0000","last_fullsized":"2024-04-25T13:44:36.584945+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:26.164922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568415+0000","last_change":"2024-04-25T13:44:36.598534+0000","last_active":"2024-04-25T13:44:37.568415+0000","last_peered":"2024-04-25T13:44:37.568415+0000","last_clean":"2024-04-25T13:44:37.568415+0000","last_became_active":"2024-04-25T13:44:36.595921+0000","last_became_peered":"2024-04-25T13:44:36.595921+0000","last_unstale":"2024-04-25T13:44:37.568415+0000","last_undegraded":"2024-04-25T13:44:37.568415+0000","last_fullsized":"2024-04-25T13:44:37.568415+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:06.047866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592338+0000","last_change":"2024-04-25T13:44:35.582895+0000","last_active":"2024-04-25T13:44:36.592338+0000","last_peered":"2024-04-25T13:44:36.592338+0000","last_clean":"2024-04-25T13:44:36.592338+0000","last_became_active":"2024-04-25T13:44:35.582078+0000","last_became_peered":"2024-04-25T13:44:35.582078+0000","last_unstale":"2024-04-25T13:44:36.592338+0000","last_undegraded":"2024-04-25T13:44:36.592338+0000","last_fullsized":"2024-04-25T13:44:36.592338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:15.821050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.595114+0000","last_change":"2024-04-25T13:44:36.594727+0000","last_active":"2024-04-25T13:44:36.595114+0000","last_peered":"2024-04-25T13:44:36.595114+0000","last_clean":"2024-04-25T13:44:36.595114+0000","last_became_active":"2024-04-25T13:44:36.592124+0000","last_became_peered":"2024-04-25T13:44:36.592124+0000","last_unstale":"2024-04-25T13:44:36.595114+0000","last_undegraded":"2024-04-25T13:44:36.595114+0000","last_fullsized":"2024-04-25T13:44:36.595114+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:09.758689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"81'8","reported_seq":22,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567039+0000","last_change":"2024-04-25T13:44:35.593747+0000","last_active":"2024-04-25T13:44:37.567039+0000","last_peered":"2024-04-25T13:44:37.567039+0000","last_clean":"2024-04-25T13:44:37.567039+0000","last_became_active":"2024-04-25T13:44:35.592369+0000","last_became_peered":"2024-04-25T13:44:35.592369+0000","last_unstale":"2024-04-25T13:44:37.567039+0000","last_undegraded":"2024-04-25T13:44:37.567039+0000","last_fullsized":"2024-04-25T13:44:37.567039+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:08.089896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568272+0000","last_change":"2024-04-25T13:44:36.610438+0000","last_active":"2024-04-25T13:44:37.568272+0000","last_peered":"2024-04-25T13:44:37.568272+0000","last_clean":"2024-04-25T13:44:37.568272+0000","last_became_active":"2024-04-25T13:44:36.601469+0000","last_became_peered":"2024-04-25T13:44:36.601469+0000","last_unstale":"2024-04-25T13:44:37.568272+0000","last_undegraded":"2024-04-25T13:44:37.568272+0000","last_fullsized":"2024-04-25T13:44:37.568272+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:56:04.388942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.620938+0000","last_change":"2024-04-25T13:44:35.593729+0000","last_active":"2024-04-25T13:44:36.620938+0000","last_peered":"2024-04-25T13:44:36.620938+0000","last_clean":"2024-04-25T13:44:36.620938+0000","last_became_active":"2024-04-25T13:44:35.590063+0000","last_became_peered":"2024-04-25T13:44:35.590063+0000","last_unstale":"2024-04-25T13:44:36.620938+0000","last_undegraded":"2024-04-25T13:44:36.620938+0000","last_fullsized":"2024-04-25T13:44:36.620938+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:51.680146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.598592+0000","last_change":"2024-04-25T13:44:36.598246+0000","last_active":"2024-04-25T13:44:36.598592+0000","last_peered":"2024-04-25T13:44:36.598592+0000","last_clean":"2024-04-25T13:44:36.598592+0000","last_became_active":"2024-04-25T13:44:36.597484+0000","last_became_peered":"2024-04-25T13:44:36.597484+0000","last_unstale":"2024-04-25T13:44:36.598592+0000","last_undegraded":"2024-04-25T13:44:36.598592+0000","last_fullsized":"2024-04-25T13:44:36.598592+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:15:48.913915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"81'2","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568741+0000","last_change":"2024-04-25T13:44:35.587553+0000","last_active":"2024-04-25T13:44:37.568741+0000","last_peered":"2024-04-25T13:44:37.568741+0000","last_clean":"2024-04-25T13:44:37.568741+0000","last_became_active":"2024-04-25T13:44:35.586934+0000","last_became_peered":"2024-04-25T13:44:35.586934+0000","last_unstale":"2024-04-25T13:44:37.568741+0000","last_undegraded":"2024-04-25T13:44:37.568741+0000","last_fullsized":"2024-04-25T13:44:37.568741+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:07:17.362081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.579086+0000","last_change":"2024-04-25T13:44:36.577063+0000","last_active":"2024-04-25T13:44:36.579086+0000","last_peered":"2024-04-25T13:44:36.579086+0000","last_clean":"2024-04-25T13:44:36.579086+0000","last_became_active":"2024-04-25T13:44:36.574179+0000","last_became_peered":"2024-04-25T13:44:36.574179+0000","last_unstale":"2024-04-25T13:44:36.579086+0000","last_undegraded":"2024-04-25T13:44:36.579086+0000","last_fullsized":"2024-04-25T13:44:36.579086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:47.488620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"81'1","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.625911+0000","last_change":"2024-04-25T13:44:35.599066+0000","last_active":"2024-04-25T13:44:36.625911+0000","last_peered":"2024-04-25T13:44:36.625911+0000","last_clean":"2024-04-25T13:44:36.625911+0000","last_became_active":"2024-04-25T13:44:35.594831+0000","last_became_peered":"2024-04-25T13:44:35.594831+0000","last_unstale":"2024-04-25T13:44:36.625911+0000","last_undegraded":"2024-04-25T13:44:36.625911+0000","last_fullsized":"2024-04-25T13:44:36.625911+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:41.095783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.603875+0000","last_change":"2024-04-25T13:44:36.603649+0000","last_active":"2024-04-25T13:44:36.603875+0000","last_peered":"2024-04-25T13:44:36.603875+0000","last_clean":"2024-04-25T13:44:36.603875+0000","last_became_active":"2024-04-25T13:44:36.603191+0000","last_became_peered":"2024-04-25T13:44:36.603191+0000","last_unstale":"2024-04-25T13:44:36.603875+0000","last_undegraded":"2024-04-25T13:44:36.603875+0000","last_fullsized":"2024-04-25T13:44:36.603875+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:33.156836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.574103+0000","last_change":"2024-04-25T13:44:35.592543+0000","last_active":"2024-04-25T13:44:36.574103+0000","last_peered":"2024-04-25T13:44:36.574103+0000","last_clean":"2024-04-25T13:44:36.574103+0000","last_became_active":"2024-04-25T13:44:35.591827+0000","last_became_peered":"2024-04-25T13:44:35.591827+0000","last_unstale":"2024-04-25T13:44:36.574103+0000","last_undegraded":"2024-04-25T13:44:36.574103+0000","last_fullsized":"2024-04-25T13:44:36.574103+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:22.922936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.596532+0000","last_change":"2024-04-25T13:44:36.595619+0000","last_active":"2024-04-25T13:44:36.596532+0000","last_peered":"2024-04-25T13:44:36.596532+0000","last_clean":"2024-04-25T13:44:36.596532+0000","last_became_active":"2024-04-25T13:44:36.593828+0000","last_became_peered":"2024-04-25T13:44:36.593828+0000","last_unstale":"2024-04-25T13:44:36.596532+0000","last_undegraded":"2024-04-25T13:44:36.596532+0000","last_fullsized":"2024-04-25T13:44:36.596532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:36.621561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.585941+0000","last_change":"2024-04-25T13:44:35.591439+0000","last_active":"2024-04-25T13:44:36.585941+0000","last_peered":"2024-04-25T13:44:36.585941+0000","last_clean":"2024-04-25T13:44:36.585941+0000","last_became_active":"2024-04-25T13:44:35.588118+0000","last_became_peered":"2024-04-25T13:44:35.588118+0000","last_unstale":"2024-04-25T13:44:36.585941+0000","last_undegraded":"2024-04-25T13:44:36.585941+0000","last_fullsized":"2024-04-25T13:44:36.585941+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.816878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.593666+0000","last_change":"2024-04-25T13:44:36.593336+0000","last_active":"2024-04-25T13:44:36.593666+0000","last_peered":"2024-04-25T13:44:36.593666+0000","last_clean":"2024-04-25T13:44:36.593666+0000","last_became_active":"2024-04-25T13:44:36.592625+0000","last_became_peered":"2024-04-25T13:44:36.592625+0000","last_unstale":"2024-04-25T13:44:36.593666+0000","last_undegraded":"2024-04-25T13:44:36.593666+0000","last_fullsized":"2024-04-25T13:44:36.593666+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.883555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.582467+0000","last_change":"2024-04-25T13:44:35.584228+0000","last_active":"2024-04-25T13:44:36.582467+0000","last_peered":"2024-04-25T13:44:36.582467+0000","last_clean":"2024-04-25T13:44:36.582467+0000","last_became_active":"2024-04-25T13:44:35.583376+0000","last_became_peered":"2024-04-25T13:44:35.583376+0000","last_unstale":"2024-04-25T13:44:36.582467+0000","last_undegraded":"2024-04-25T13:44:36.582467+0000","last_fullsized":"2024-04-25T13:44:36.582467+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:10:44.373304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.589573+0000","last_change":"2024-04-25T13:44:36.589034+0000","last_active":"2024-04-25T13:44:36.589573+0000","last_peered":"2024-04-25T13:44:36.589573+0000","last_clean":"2024-04-25T13:44:36.589573+0000","last_became_active":"2024-04-25T13:44:36.588199+0000","last_became_peered":"2024-04-25T13:44:36.588199+0000","last_unstale":"2024-04-25T13:44:36.589573+0000","last_undegraded":"2024-04-25T13:44:36.589573+0000","last_fullsized":"2024-04-25T13:44:36.589573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:09.811361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568178+0000","last_change":"2024-04-25T13:44:35.573257+0000","last_active":"2024-04-25T13:44:37.568178+0000","last_peered":"2024-04-25T13:44:37.568178+0000","last_clean":"2024-04-25T13:44:37.568178+0000","last_became_active":"2024-04-25T13:44:35.572965+0000","last_became_peered":"2024-04-25T13:44:35.572965+0000","last_unstale":"2024-04-25T13:44:37.568178+0000","last_undegraded":"2024-04-25T13:44:37.568178+0000","last_fullsized":"2024-04-25T13:44:37.568178+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:55.363028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568701+0000","last_change":"2024-04-25T13:44:36.586632+0000","last_active":"2024-04-25T13:44:37.568701+0000","last_peered":"2024-04-25T13:44:37.568701+0000","last_clean":"2024-04-25T13:44:37.568701+0000","last_became_active":"2024-04-25T13:44:36.584513+0000","last_became_peered":"2024-04-25T13:44:36.584513+0000","last_unstale":"2024-04-25T13:44:37.568701+0000","last_undegraded":"2024-04-25T13:44:37.568701+0000","last_fullsized":"2024-04-25T13:44:37.568701+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:52.930285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.623164+0000","last_change":"2024-04-25T13:44:35.597653+0000","last_active":"2024-04-25T13:44:36.623164+0000","last_peered":"2024-04-25T13:44:36.623164+0000","last_clean":"2024-04-25T13:44:36.623164+0000","last_became_active":"2024-04-25T13:44:35.593659+0000","last_became_peered":"2024-04-25T13:44:35.593659+0000","last_unstale":"2024-04-25T13:44:36.623164+0000","last_undegraded":"2024-04-25T13:44:36.623164+0000","last_fullsized":"2024-04-25T13:44:36.623164+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:10:27.782046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.616548+0000","last_change":"2024-04-25T13:44:36.616317+0000","last_active":"2024-04-25T13:44:36.616548+0000","last_peered":"2024-04-25T13:44:36.616548+0000","last_clean":"2024-04-25T13:44:36.616548+0000","last_became_active":"2024-04-25T13:44:36.594389+0000","last_became_peered":"2024-04-25T13:44:36.594389+0000","last_unstale":"2024-04-25T13:44:36.616548+0000","last_undegraded":"2024-04-25T13:44:36.616548+0000","last_fullsized":"2024-04-25T13:44:36.616548+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:31.453872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.579523+0000","last_change":"2024-04-25T13:44:35.571317+0000","last_active":"2024-04-25T13:44:36.579523+0000","last_peered":"2024-04-25T13:44:36.579523+0000","last_clean":"2024-04-25T13:44:36.579523+0000","last_became_active":"2024-04-25T13:44:35.570522+0000","last_became_peered":"2024-04-25T13:44:35.570522+0000","last_unstale":"2024-04-25T13:44:36.579523+0000","last_undegraded":"2024-04-25T13:44:36.579523+0000","last_fullsized":"2024-04-25T13:44:36.579523+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:03:16.712314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.597910+0000","last_change":"2024-04-25T13:44:36.597555+0000","last_active":"2024-04-25T13:44:36.597910+0000","last_peered":"2024-04-25T13:44:36.597910+0000","last_clean":"2024-04-25T13:44:36.597910+0000","last_became_active":"2024-04-25T13:44:36.593594+0000","last_became_peered":"2024-04-25T13:44:36.593594+0000","last_unstale":"2024-04-25T13:44:36.597910+0000","last_undegraded":"2024-04-25T13:44:36.597910+0000","last_fullsized":"2024-04-25T13:44:36.597910+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:39:40.804684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.568860+0000","last_change":"2024-04-25T13:44:35.573295+0000","last_active":"2024-04-25T13:44:36.568860+0000","last_peered":"2024-04-25T13:44:36.568860+0000","last_clean":"2024-04-25T13:44:36.568860+0000","last_became_active":"2024-04-25T13:44:35.572952+0000","last_became_peered":"2024-04-25T13:44:35.572952+0000","last_unstale":"2024-04-25T13:44:36.568860+0000","last_undegraded":"2024-04-25T13:44:36.568860+0000","last_fullsized":"2024-04-25T13:44:36.568860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:17.923840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.616502+0000","last_change":"2024-04-25T13:44:36.616339+0000","last_active":"2024-04-25T13:44:36.616502+0000","last_peered":"2024-04-25T13:44:36.616502+0000","last_clean":"2024-04-25T13:44:36.616502+0000","last_became_active":"2024-04-25T13:44:36.616070+0000","last_became_peered":"2024-04-25T13:44:36.616070+0000","last_unstale":"2024-04-25T13:44:36.616502+0000","last_undegraded":"2024-04-25T13:44:36.616502+0000","last_fullsized":"2024-04-25T13:44:36.616502+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:38:14.167705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.568978+0000","last_change":"2024-04-25T13:44:35.582760+0000","last_active":"2024-04-25T13:44:37.568978+0000","last_peered":"2024-04-25T13:44:37.568978+0000","last_clean":"2024-04-25T13:44:37.568978+0000","last_became_active":"2024-04-25T13:44:35.582522+0000","last_became_peered":"2024-04-25T13:44:35.582522+0000","last_unstale":"2024-04-25T13:44:37.568978+0000","last_undegraded":"2024-04-25T13:44:37.568978+0000","last_fullsized":"2024-04-25T13:44:37.568978+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:27.450229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.594530+0000","last_change":"2024-04-25T13:44:36.593968+0000","last_active":"2024-04-25T13:44:36.594530+0000","last_peered":"2024-04-25T13:44:36.594530+0000","last_clean":"2024-04-25T13:44:36.594530+0000","last_became_active":"2024-04-25T13:44:36.591147+0000","last_became_peered":"2024-04-25T13:44:36.591147+0000","last_unstale":"2024-04-25T13:44:36.594530+0000","last_undegraded":"2024-04-25T13:44:36.594530+0000","last_fullsized":"2024-04-25T13:44:36.594530+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:30.494335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.567221+0000","last_change":"2024-04-25T13:44:35.577004+0000","last_active":"2024-04-25T13:44:36.567221+0000","last_peered":"2024-04-25T13:44:36.567221+0000","last_clean":"2024-04-25T13:44:36.567221+0000","last_became_active":"2024-04-25T13:44:35.576740+0000","last_became_peered":"2024-04-25T13:44:35.576740+0000","last_unstale":"2024-04-25T13:44:36.567221+0000","last_undegraded":"2024-04-25T13:44:36.567221+0000","last_fullsized":"2024-04-25T13:44:36.567221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:36.243968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.600129+0000","last_change":"2024-04-25T13:44:36.599772+0000","last_active":"2024-04-25T13:44:36.600129+0000","last_peered":"2024-04-25T13:44:36.600129+0000","last_clean":"2024-04-25T13:44:36.600129+0000","last_became_active":"2024-04-25T13:44:36.599065+0000","last_became_peered":"2024-04-25T13:44:36.599065+0000","last_unstale":"2024-04-25T13:44:36.600129+0000","last_undegraded":"2024-04-25T13:44:36.600129+0000","last_fullsized":"2024-04-25T13:44:36.600129+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:04.891827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.566294+0000","last_change":"2024-04-25T13:44:35.572664+0000","last_active":"2024-04-25T13:44:36.566294+0000","last_peered":"2024-04-25T13:44:36.566294+0000","last_clean":"2024-04-25T13:44:36.566294+0000","last_became_active":"2024-04-25T13:44:35.572360+0000","last_became_peered":"2024-04-25T13:44:35.572360+0000","last_unstale":"2024-04-25T13:44:36.566294+0000","last_undegraded":"2024-04-25T13:44:36.566294+0000","last_fullsized":"2024-04-25T13:44:36.566294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:51.574092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.592344+0000","last_change":"2024-04-25T13:44:36.591861+0000","last_active":"2024-04-25T13:44:36.592344+0000","last_peered":"2024-04-25T13:44:36.592344+0000","last_clean":"2024-04-25T13:44:36.592344+0000","last_became_active":"2024-04-25T13:44:36.589278+0000","last_became_peered":"2024-04-25T13:44:36.589278+0000","last_unstale":"2024-04-25T13:44:36.592344+0000","last_undegraded":"2024-04-25T13:44:36.592344+0000","last_fullsized":"2024-04-25T13:44:36.592344+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:45.176351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-25T13:44:37.567353+0000","last_change":"2024-04-25T13:44:35.593098+0000","last_active":"2024-04-25T13:44:37.567353+0000","last_peered":"2024-04-25T13:44:37.567353+0000","last_clean":"2024-04-25T13:44:37.567353+0000","last_became_active":"2024-04-25T13:44:35.591252+0000","last_became_peered":"2024-04-25T13:44:35.591252+0000","last_unstale":"2024-04-25T13:44:37.567353+0000","last_undegraded":"2024-04-25T13:44:37.567353+0000","last_fullsized":"2024-04-25T13:44:37.567353+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:37.975492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.591111+0000","last_change":"2024-04-25T13:44:36.590855+0000","last_active":"2024-04-25T13:44:36.591111+0000","last_peered":"2024-04-25T13:44:36.591111+0000","last_clean":"2024-04-25T13:44:36.591111+0000","last_became_active":"2024-04-25T13:44:36.589329+0000","last_became_peered":"2024-04-25T13:44:36.589329+0000","last_unstale":"2024-04-25T13:44:36.591111+0000","last_undegraded":"2024-04-25T13:44:36.591111+0000","last_fullsized":"2024-04-25T13:44:36.591111+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:07.392079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.585486+0000","last_change":"2024-04-25T13:44:35.592001+0000","last_active":"2024-04-25T13:44:36.585486+0000","last_peered":"2024-04-25T13:44:36.585486+0000","last_clean":"2024-04-25T13:44:36.585486+0000","last_became_active":"2024-04-25T13:44:35.589326+0000","last_became_peered":"2024-04-25T13:44:35.589326+0000","last_unstale":"2024-04-25T13:44:36.585486+0000","last_undegraded":"2024-04-25T13:44:36.585486+0000","last_fullsized":"2024-04-25T13:44:36.585486+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:01.052795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.599700+0000","last_change":"2024-04-25T13:44:36.598876+0000","last_active":"2024-04-25T13:44:36.599700+0000","last_peered":"2024-04-25T13:44:36.599700+0000","last_clean":"2024-04-25T13:44:36.599700+0000","last_became_active":"2024-04-25T13:44:36.598035+0000","last_became_peered":"2024-04-25T13:44:36.598035+0000","last_unstale":"2024-04-25T13:44:36.599700+0000","last_undegraded":"2024-04-25T13:44:36.599700+0000","last_fullsized":"2024-04-25T13:44:36.599700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:35.546381+0000","last_clean_scrub_stamp":"2024-04-25T13:44:35.546381+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:02.083840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"81'2","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.621491+0000","last_change":"2024-04-25T13:44:35.594518+0000","last_active":"2024-04-25T13:44:36.621491+0000","last_peered":"2024-04-25T13:44:36.621491+0000","last_clean":"2024-04-25T13:44:36.621491+0000","last_became_active":"2024-04-25T13:44:35.593036+0000","last_became_peered":"2024-04-25T13:44:35.593036+0000","last_unstale":"2024-04-25T13:44:36.621491+0000","last_undegraded":"2024-04-25T13:44:36.621491+0000","last_fullsized":"2024-04-25T13:44:36.621491+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:44:34.532761+0000","last_clean_scrub_stamp":"2024-04-25T13:44:34.532761+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:01.613154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":230,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-25T13:44:36.587207+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:44:36.587207+0000","last_peered":"2024-04-25T13:44:36.587207+0000","last_clean":"2024-04-25T13:44:36.587207+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:44:36.587207+0000","last_undegraded":"2024-04-25T13:44:36.587207+0000","last_fullsized":"2024-04-25T13:44:36.587207+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:50.852505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":996,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194316,"statfs":{"total":200000000,"available":198979900,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":419,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194893,"statfs":{"total":200000000,"available":199570792,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":419,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194893,"statfs":{"total":200000000,"available":199570794,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574902,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":425,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194887,"statfs":{"total":200000000,"available":199564514,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574902,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":419,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194893,"statfs":{"total":200000000,"available":199570772,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":420,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194891,"statfs":{"total":200000000,"available":199569272,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":421,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194891,"statfs":{"total":200000000,"available":199568656,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1001,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194310,"statfs":{"total":200000000,"available":198974216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:44:41.134 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:44:41.134 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:44:41.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f894354e640 1 Processor -- start 2024-04-25T13:44:41.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f894354e640 1 -- start start 2024-04-25T13:44:41.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f894354e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f893c0d2e80 0x7f893c1607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f894354e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d59b0 0x7f893c0d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f894354e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f893c0d4330 0x7f893c0d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f894354e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f893c0d6300 con 0x7f893c0d4330 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f894354e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f893c161060 con 0x7f893c0d2e80 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f894354e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f893c161340 con 0x7f893c0d59b0 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8941aeb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f893c0d4330 0x7f893c0d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8941aeb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f893c0d4330 0x7f893c0d4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41524/0 (socket says 172.21.15.80:41524) 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8941aeb640 1 -- 172.21.15.80:0/2100037090 learned_addr learned my addr 172.21.15.80:0/2100037090 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8940ae9640 1 --2- 172.21.15.80:0/2100037090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d59b0 0x7f893c0d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f89412ea640 1 --2- 172.21.15.80:0/2100037090 >> v2:172.21.15.87:3300/0 conn(0x7f893c0d2e80 0x7f893c1607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8941aeb640 1 -- 172.21.15.80:0/2100037090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d59b0 msgr2=0x7f893c0d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8941aeb640 1 --2- 172.21.15.80:0/2100037090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d59b0 0x7f893c0d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8941aeb640 1 -- 172.21.15.80:0/2100037090 >> v2:172.21.15.87:3300/0 conn(0x7f893c0d2e80 msgr2=0x7f893c1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:41.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8941aeb640 1 --2- 172.21.15.80:0/2100037090 >> v2:172.21.15.87:3300/0 conn(0x7f893c0d2e80 0x7f893c1607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.332+0000 7f8941aeb640 1 -- 172.21.15.80:0/2100037090 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c0d2680 con 0x7f893c0d4330 2024-04-25T13:44:41.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f8941aeb640 1 --2- 172.21.15.80:0/2100037090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d4330 0x7f893c0d4740 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f893c08ec00 tx=0x7f892c01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8853c041c3996ed0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:41.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f893bfff640 1 -- 172.21.15.80:0/2100037090 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0aa430 con 0x7f893c0d4330 2024-04-25T13:44:41.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f893bfff640 1 -- 172.21.15.80:0/2100037090 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f892c0af070 con 0x7f893c0d4330 2024-04-25T13:44:41.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f893bfff640 1 -- 172.21.15.80:0/2100037090 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f892c0aa7f0 con 0x7f893c0d4330 2024-04-25T13:44:41.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 -- 172.21.15.80:0/2100037090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d4330 msgr2=0x7f893c0d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 --2- 172.21.15.80:0/2100037090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d4330 0x7f893c0d4740 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f893c08ec00 tx=0x7f892c01c4f0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 -- 172.21.15.80:0/2100037090 shutdown_connections 2024-04-25T13:44:41.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 --2- 172.21.15.80:0/2100037090 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d4330 0x7f893c0d4740 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 --2- 172.21.15.80:0/2100037090 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d59b0 0x7f893c0d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 --2- 172.21.15.80:0/2100037090 >> v2:172.21.15.87:3300/0 conn(0x7f893c0d2e80 0x7f893c1607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 -- 172.21.15.80:0/2100037090 >> 172.21.15.80:0/2100037090 conn(0x7f893c0737a0 msgr2=0x7f893c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:41.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 -- 172.21.15.80:0/2100037090 shutdown_connections 2024-04-25T13:44:41.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.336+0000 7f894354e640 1 -- 172.21.15.80:0/2100037090 wait complete. 2024-04-25T13:44:41.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 Processor -- start 2024-04-25T13:44:41.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 -- start start 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d2e80 0x7f893c14cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d4330 0x7f893c15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893c0d59b0 0x7f893c0ba810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f893c0d4e00 con 0x7f893c0d2e80 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f893c0d4c80 con 0x7f893c0d59b0 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f893c0d3290 con 0x7f893c0d4330 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d2e80 0x7f893c14cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d2e80 0x7f893c14cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41532/0 (socket says 172.21.15.80:41532) 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 -- 172.21.15.80:0/3381682471 learned_addr learned my addr 172.21.15.80:0/3381682471 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f8941aeb640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893c0d59b0 0x7f893c0ba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f8940ae9640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d4330 0x7f893c15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 -- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d4330 msgr2=0x7f893c15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d4330 0x7f893c15cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 -- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893c0d59b0 msgr2=0x7f893c0ba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893c0d59b0 0x7f893c0ba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 -- 172.21.15.80:0/3381682471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f893c15c2d0 con 0x7f893c0d2e80 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f89412ea640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d2e80 0x7f893c14cf10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8934007e60 tx=0x7f893400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f8939ffb640 1 -- 172.21.15.80:0/3381682471 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f893401a070 con 0x7f893c0d2e80 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f8939ffb640 1 -- 172.21.15.80:0/3381682471 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8934003ba0 con 0x7f893c0d2e80 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f893c1591a0 con 0x7f893c0d2e80 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f8939ffb640 1 -- 172.21.15.80:0/3381682471 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8934004030 con 0x7f893c0d2e80 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f893c159720 con 0x7f893c0d2e80 2024-04-25T13:44:41.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.340+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f893c06a3a0 con 0x7f893c0d2e80 2024-04-25T13:44:41.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.344+0000 7f8939ffb640 1 -- 172.21.15.80:0/3381682471 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f893401d070 con 0x7f893c0d2e80 2024-04-25T13:44:41.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.344+0000 7f8939ffb640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892806f400 0x7f8928071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.344+0000 7f8939ffb640 1 -- 172.21.15.80:0/3381682471 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f893408f3d0 con 0x7f893c0d2e80 2024-04-25T13:44:41.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.344+0000 7f8940ae9640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892806f400 0x7f8928071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.344+0000 7f8940ae9640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892806f400 0x7f8928071910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f893c14d760 tx=0x7f8930008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:41.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.348+0000 7f8939ffb640 1 -- 172.21.15.80:0/3381682471 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f893405bc70 con 0x7f893c0d2e80 2024-04-25T13:44:41.531 INFO:tasks.ceph.mds.b.smithi087.stderr:2024-04-25T13:44:41.529+0000 7f9b63ba9640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:44:41.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.564+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f893c0bbb90 con 0x7f893c0d2e80 2024-04-25T13:44:41.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.568+0000 7f8939ffb640 1 -- 172.21.15.80:0/3381682471 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f893405ff70 con 0x7f893c0d2e80 2024-04-25T13:44:41.568 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:44:41.568 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:44:41.568 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:44:41.568 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:44:41.568 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:44:41.569 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892806f400 msgr2=0x7f8928071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892806f400 0x7f8928071910 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f893c14d760 tx=0x7f8930008040 comp rx=0 tx=0).stop 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d2e80 msgr2=0x7f893c14cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d2e80 0x7f893c14cf10 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f8934007e60 tx=0x7f893400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 shutdown_connections 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f892806f400 0x7f8928071910 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f893c0d59b0 0x7f893c0ba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f893c0d4330 0x7f893c15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 --2- 172.21.15.80:0/3381682471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f893c0d2e80 0x7f893c14cf10 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 >> 172.21.15.80:0/3381682471 conn(0x7f893c0737a0 msgr2=0x7f893c15eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:41.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 shutdown_connections 2024-04-25T13:44:41.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.572+0000 7f894354e640 1 -- 172.21.15.80:0/3381682471 wait complete. 2024-04-25T13:44:41.590 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:41.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a73d60640 1 Processor -- start 2024-04-25T13:44:41.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a73d60640 1 -- start start 2024-04-25T13:44:41.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a73d60640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c06f6f0 0x7f0a6c06fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a73d60640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6c070040 con 0x7f0a6c0ccd70 2024-04-25T13:44:41.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a73d60640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6c074430 con 0x7f0a6c0d0f30 2024-04-25T13:44:41.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a73d60640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6c073940 con 0x7f0a6c06f6f0 2024-04-25T13:44:41.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a722fd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0a6c0ccd70 0x7f0a6c0cd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59564/0 (socket says 172.21.15.80:59564) 2024-04-25T13:44:41.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a722fd640 1 -- 172.21.15.80:0/3652377194 learned_addr learned my addr 172.21.15.80:0/3652377194 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:41.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a71afc640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c06f6f0 0x7f0a6c06fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1449106683 0 0) 0x7f0a640014a0 con 0x7f0a6c0ccd70 2024-04-25T13:44:41.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a58003530 con 0x7f0a6c0ccd70 2024-04-25T13:44:41.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3925691666 0 0) 0x7f0a58003530 con 0x7f0a6c0ccd70 2024-04-25T13:44:41.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c06f6f0 msgr2=0x7f0a6c06fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:41.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c06f6f0 0x7f0a6c06fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 >> v1:172.21.15.87:6789/0 conn(0x7f0a6c0d0f30 legacy=0x7f0a6c0d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c073b90 con 0x7f0a6c0ccd70 2024-04-25T13:44:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0a64001bd0 con 0x7f0a6c0ccd70 2024-04-25T13:44:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 >> v1:172.21.15.80:6789/0 conn(0x7f0a6c0ccd70 legacy=0x7f0a6c0cd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a58003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.732+0000 7f0a70afa640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c06f6f0 0x7f0a58004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a70afa640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a58004d50 0x7f0a58005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c073940 con 0x7f0a580039f0 2024-04-25T13:44:41.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0a5800bb90 con 0x7f0a58004d50 2024-04-25T13:44:41.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0a5800bd90 con 0x7f0a6c06f6f0 2024-04-25T13:44:41.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a71afc640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a58003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a71afc640 1 -- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c06f6f0 msgr2=0x7f0a58004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:41.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a71afc640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c06f6f0 0x7f0a58004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a71afc640 1 -- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a58004d50 msgr2=0x7f0a58005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a71afc640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a58004d50 0x7f0a58005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a71afc640 1 -- 172.21.15.80:0/3652377194 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a58003530 con 0x7f0a580039f0 2024-04-25T13:44:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a71afc640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a58003e60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009ab0 tx=0x7f0a6801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4e7643f1332f425 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680096b0 con 0x7f0a580039f0 2024-04-25T13:44:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a680a6470 con 0x7f0a580039f0 2024-04-25T13:44:41.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a70afa640 1 -- 172.21.15.80:0/3652377194 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680a6820 con 0x7f0a580039f0 2024-04-25T13:44:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a73d60640 1 -- 172.21.15.80:0/3652377194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 msgr2=0x7f0a58003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a73d60640 1 --2- 172.21.15.80:0/3652377194 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a58003e60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f0a68009ab0 tx=0x7f0a6801ca90 comp rx=0 tx=0).stop 2024-04-25T13:44:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a722fd640 1 -- 172.21.15.80:0/3652377194 reap_dead start 2024-04-25T13:44:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a73d60640 1 -- 172.21.15.80:0/3652377194 shutdown_connections 2024-04-25T13:44:41.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a73d60640 1 -- 172.21.15.80:0/3652377194 >> 172.21.15.80:0/3652377194 conn(0x7f0a6c0d20d0 msgr2=0x7f0a6c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:41.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a73d60640 1 -- 172.21.15.80:0/3652377194 shutdown_connections 2024-04-25T13:44:41.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.736+0000 7f0a73d60640 1 -- 172.21.15.80:0/3652377194 wait complete. 2024-04-25T13:44:41.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 Processor -- start 2024-04-25T13:44:41.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 -- start start 2024-04-25T13:44:41.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a6c167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c1f7bf0 0x7f0a6c0b6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a6c1f8000 0x7f0a6c0b7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c0af040 con 0x7f0a580039f0 2024-04-25T13:44:41.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0a6c0aeec0 con 0x7f0a6c1f8000 2024-04-25T13:44:41.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0a6c161ae0 con 0x7f0a6c1f7bf0 2024-04-25T13:44:41.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a6c167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a6c167070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41558/0 (socket says 172.21.15.80:41558) 2024-04-25T13:44:41.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 -- 172.21.15.80:0/249283227 learned_addr learned my addr 172.21.15.80:0/249283227 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:41.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a722fd640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a6c1f8000 0x7f0a6c0b7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a712fb640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c1f7bf0 0x7f0a6c0b6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 -- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c1f7bf0 msgr2=0x7f0a6c0b6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c1f7bf0 0x7f0a6c0b6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 -- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a6c1f8000 msgr2=0x7f0a6c0b7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a6c1f8000 0x7f0a6c0b7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 -- 172.21.15.80:0/249283227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6c164640 con 0x7f0a580039f0 2024-04-25T13:44:41.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a71afc640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a6c167070 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0a6801c8c0 tx=0x7f0a68019830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:41.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a627fc640 1 -- 172.21.15.80:0/249283227 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680a6ca0 con 0x7f0a580039f0 2024-04-25T13:44:41.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a627fc640 1 -- 172.21.15.80:0/249283227 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a680a6e40 con 0x7f0a580039f0 2024-04-25T13:44:41.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a6c164920 con 0x7f0a580039f0 2024-04-25T13:44:41.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a627fc640 1 -- 172.21.15.80:0/249283227 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a680a7650 con 0x7f0a580039f0 2024-04-25T13:44:41.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.740+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a6c15de00 con 0x7f0a580039f0 2024-04-25T13:44:41.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.744+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a6c164e40 con 0x7f0a580039f0 2024-04-25T13:44:41.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.744+0000 7f0a627fc640 1 -- 172.21.15.80:0/249283227 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a680a77f0 con 0x7f0a580039f0 2024-04-25T13:44:41.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.744+0000 7f0a627fc640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5006f400 0x7f0a50071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:41.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.744+0000 7f0a627fc640 1 -- 172.21.15.80:0/249283227 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f0a68129040 con 0x7f0a580039f0 2024-04-25T13:44:41.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.744+0000 7f0a712fb640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5006f400 0x7f0a50071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:41.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.744+0000 7f0a712fb640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5006f400 0x7f0a50071910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0b7b00 tx=0x7f0a5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:41.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.748+0000 7f0a627fc640 1 -- 172.21.15.80:0/249283227 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0a680bf060 con 0x7f0a580039f0 2024-04-25T13:44:41.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0a6c15dbf0 con 0x7f0a580039f0 2024-04-25T13:44:41.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a627fc640 1 -- 172.21.15.80:0/249283227 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5180 (secure 0 0 0) 0x7f0a680ac020 con 0x7f0a580039f0 2024-04-25T13:44:41.939 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:41.939 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2082291876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2082291876},{"type":"v1","addr":"172.21.15.87:6835","nonce":2082291876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1337331528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1337331528},{"type":"v1","addr":"172.21.15.80:6833","nonce":1337331528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2521189874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2521189874},{"type":"v1","addr":"172.21.15.80:6835","nonce":2521189874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:44:35.958393+0000","modified":"2024-04-25T13:44:39.019142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/642879744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":642879744},{"type":"v1","addr":"172.21.15.87:6837","nonce":642879744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":5}]} 2024-04-25T13:44:41.939 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 41 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5006f400 msgr2=0x7f0a50071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5006f400 0x7f0a50071910 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c0b7b00 tx=0x7f0a5c009040 comp rx=0 tx=0).stop 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 msgr2=0x7f0a6c167070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a6c167070 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0a6801c8c0 tx=0x7f0a68019830 comp rx=0 tx=0).stop 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 shutdown_connections 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5006f400 0x7f0a50071910 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a6c1f8000 0x7f0a6c0b7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6c1f7bf0 0x7f0a6c0b6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 --2- 172.21.15.80:0/249283227 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a580039f0 0x7f0a6c167070 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:41.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.940+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 >> 172.21.15.80:0/249283227 conn(0x7f0a6c0d20d0 msgr2=0x7f0a6c09ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:41.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.944+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 shutdown_connections 2024-04-25T13:44:41.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:41.944+0000 7f0a73d60640 1 -- 172.21.15.80:0/249283227 wait complete. 2024-04-25T13:44:42.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:44:35.958393+0000', 'modified': '2024-04-25T13:44:39.019142+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6627}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6627': {'gid': 6627, 'name': 'b', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/642879744', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6836', 'nonce': 642879744}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 642879744}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6627, 'qdb_cluster': [6627]} 2024-04-25T13:44:42.362 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:44:42.363 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:44:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.516+0000 7f8ed3ede640 1 Processor -- start 2024-04-25T13:44:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.516+0000 7f8ed3ede640 1 -- start start 2024-04-25T13:44:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed3ede640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc161f90 0x7f8ecc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed3ede640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8ecc0d0510 0x7f8ecc15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed3ede640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ecc0d0d70 con 0x7f8ecc15ea70 2024-04-25T13:44:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed3ede640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8ecc0d1050 con 0x7f8ecc0d0510 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed3ede640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc06dca0 con 0x7f8ecc161f90 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed247b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8ecc15ea70 0x7f8ecc15ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59578/0 (socket says 172.21.15.80:59578) 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed247b640 1 -- 172.21.15.80:0/81019107 learned_addr learned my addr 172.21.15.80:0/81019107 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1479640 1 --2- 172.21.15.80:0/81019107 >> v2:172.21.15.87:3300/0 conn(0x7f8ecc0d0510 0x7f8ecc15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc161f90 0x7f8ecc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 333582432 0 0) 0x7f8ecc0d0d70 con 0x7f8ecc15ea70 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8eb8003530 con 0x7f8ecc15ea70 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 265200933 0 0) 0x7f8eb8003530 con 0x7f8ecc15ea70 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc161f90 msgr2=0x7f8ecc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc161f90 0x7f8ecc0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 >> v2:172.21.15.87:3300/0 conn(0x7f8ecc0d0510 msgr2=0x7f8ecc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 --2- 172.21.15.80:0/81019107 >> v2:172.21.15.87:3300/0 conn(0x7f8ecc0d0510 0x7f8ecc15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc06df80 con 0x7f8ecc15ea70 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f8ec4001a00 con 0x7f8ecc15ea70 2024-04-25T13:44:42.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 >> v1:172.21.15.80:6789/0 conn(0x7f8ecc15ea70 legacy=0x7f8ecc15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc161f90 0x7f8eb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8eb8004d50 0x7f8eb8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc0d1050 con 0x7f8eb80039c0 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc06dca0 con 0x7f8eb8004d50 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb800ba50 con 0x7f8ecc161f90 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1479640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc161f90 0x7f8eb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed247b640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8eb8004d50 0x7f8eb8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc161f90 msgr2=0x7f8eb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc161f90 0x7f8eb8004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8eb8004d50 msgr2=0x7f8eb8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8eb8004d50 0x7f8eb8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/81019107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb8003530 con 0x7f8eb80039c0 2024-04-25T13:44:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.520+0000 7f8ed247b640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8eb8004d50 0x7f8eb8005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003e30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc147e20 tx=0x7f8ec801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642a3a631017389b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80096b0 con 0x7f8eb80039c0 2024-04-25T13:44:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec80a64c0 con 0x7f8eb80039c0 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed0c78640 1 -- 172.21.15.80:0/81019107 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80a6870 con 0x7f8eb80039c0 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed3ede640 1 -- 172.21.15.80:0/81019107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eb80039c0 msgr2=0x7f8eb8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed3ede640 1 --2- 172.21.15.80:0/81019107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8eb80039c0 0x7f8eb8003e30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc147e20 tx=0x7f8ec801c9d0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed247b640 1 -- 172.21.15.80:0/81019107 reap_dead start 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed3ede640 1 -- 172.21.15.80:0/81019107 shutdown_connections 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed3ede640 1 -- 172.21.15.80:0/81019107 >> 172.21.15.80:0/81019107 conn(0x7f8ecc06d080 msgr2=0x7f8ecc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed3ede640 1 -- 172.21.15.80:0/81019107 shutdown_connections 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed3ede640 1 -- 172.21.15.80:0/81019107 wait complete. 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed3ede640 1 Processor -- start 2024-04-25T13:44:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.524+0000 7f8ed3ede640 1 -- start start 2024-04-25T13:44:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed3ede640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ecc1681b0 0x7f8ecc1676a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed3ede640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc159b50 0x7f8ecc165c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed3ede640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc1f7760 0x7f8ecc1f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed3ede640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc06f630 con 0x7f8ecc1681b0 2024-04-25T13:44:42.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed3ede640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc06f4b0 con 0x7f8ecc159b50 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed3ede640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc0d0920 con 0x7f8ecc1f7760 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ecc1681b0 0x7f8ecc1676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ecc1681b0 0x7f8ecc1676a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41576/0 (socket says 172.21.15.80:41576) 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/1965069827 learned_addr learned my addr 172.21.15.80:0/1965069827 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed247b640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc1f7760 0x7f8ecc1f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1479640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc159b50 0x7f8ecc165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc1f7760 msgr2=0x7f8ecc1f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc1f7760 0x7f8ecc1f7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc159b50 msgr2=0x7f8ecc165c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc159b50 0x7f8ecc165c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/1965069827 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ecc166600 con 0x7f8ecc1681b0 2024-04-25T13:44:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ecc1681b0 0x7f8ecc1676a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8ec801ceb0 tx=0x7f8eb8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ec27fc640 1 -- 172.21.15.80:0/1965069827 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80a6670 con 0x7f8ecc1681b0 2024-04-25T13:44:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ec27fc640 1 -- 172.21.15.80:0/1965069827 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec80bf070 con 0x7f8ecc1681b0 2024-04-25T13:44:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed3ede640 1 -- 172.21.15.80:0/1965069827 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ecc1668e0 con 0x7f8ecc1681b0 2024-04-25T13:44:42.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ec27fc640 1 -- 172.21.15.80:0/1965069827 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80a6a70 con 0x7f8ecc1681b0 2024-04-25T13:44:42.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed3ede640 1 -- 172.21.15.80:0/1965069827 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ecc166ea0 con 0x7f8ecc1681b0 2024-04-25T13:44:42.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ec27fc640 1 -- 172.21.15.80:0/1965069827 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8ec80a6c10 con 0x7f8ecc1681b0 2024-04-25T13:44:42.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ec27fc640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8eb006f430 0x7f8eb0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ec27fc640 1 -- 172.21.15.80:0/1965069827 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8ec81298d0 con 0x7f8ecc1681b0 2024-04-25T13:44:42.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.528+0000 7f8ed1479640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8eb006f430 0x7f8eb0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 Processor -- start 2024-04-25T13:44:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- start start 2024-04-25T13:44:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1479640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8eb006f430 0x7f8eb0071940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc15f6b0 tx=0x7f8ebc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616a7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616a8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 0x55af616a8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55af61517160 con 0x55af61549c20 2024-04-25T13:44:42.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55af614ace20 con 0x55af6112e280 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55af61465a20 con 0x55af61535ea0 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616a7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616a7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41578/0 (socket says 172.21.15.80:41578) 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/3462730417 learned_addr learned my addr 172.21.15.80:0/3462730417 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1479640 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 msgr2=0x55af616a8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616a8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed247b640 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 0x55af616a8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 msgr2=0x55af616a8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 0x55af616a8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/3462730417 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af6146a100 con 0x55af61549c20 2024-04-25T13:44:42.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1479640 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616a7ee0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80f60e0 tx=0x7f8ec80f5bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504e600dd1d04705 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8eaf7fe640 1 -- 172.21.15.80:0/3462730417 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80f6150 con 0x55af61549c20 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8eaf7fe640 1 -- 172.21.15.80:0/3462730417 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec80fa500 con 0x55af61549c20 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8eaf7fe640 1 -- 172.21.15.80:0/3462730417 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec80fa2d0 con 0x55af61549c20 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- 172.21.15.80:0/3462730417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 msgr2=0x55af616a7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616a7ee0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80f60e0 tx=0x7f8ec80f5bd0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- 172.21.15.80:0/3462730417 shutdown_connections 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 0x55af616a8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3462730417 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616a7ee0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- 172.21.15.80:0/3462730417 >> 172.21.15.80:0/3462730417 conn(0x55af6116dff0 msgr2=0x55af6165d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- 172.21.15.80:0/3462730417 shutdown_connections 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.532+0000 7f8edb23b000 1 -- 172.21.15.80:0/3462730417 wait complete. 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 -- start start 2024-04-25T13:44:42.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 Processor -- start 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 0x55af616aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616aef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af6166a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55af614ba220 con 0x55af61549c20 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55af61450ea0 con 0x55af6112e280 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55af614a7b00 con 0x55af61535ea0 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 10 client.0 _ll_register_callbacks cb 0x55af614b9dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af6166a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af6166a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41586/0 (socket says 172.21.15.80:41586) 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 -- 172.21.15.80:0/4033392692 learned_addr learned my addr 172.21.15.80:0/4033392692 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:42.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 0x55af616aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed1479640 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616aef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 -- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 msgr2=0x55af616aef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616aef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 -- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 msgr2=0x55af616aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 0x55af616aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 -- 172.21.15.80:0/4033392692 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af6144c500 con 0x55af61549c20 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed247b640 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af6166a0c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0d17b0 tx=0x7f8ec400fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3df6a7712131537 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8eacff9640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8eacff9640 1 -- 172.21.15.80:0/4033392692 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400d960 con 0x55af61549c20 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8eacff9640 1 -- 172.21.15.80:0/4033392692 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8ec400db00 con 0x55af61549c20 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 -- 172.21.15.80:0/4033392692 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec80fa500 con 0x55af61549c20 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8eacff9640 1 -- 172.21.15.80:0/4033392692 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8ec4020050 con 0x55af61549c20 2024-04-25T13:44:42.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 10 client.6768 fetch_fsmap learned FSMap version 41 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 -- 172.21.15.80:0/4033392692 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x55af61432ea0 con 0x55af61549c20 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8eacff9640 1 -- 172.21.15.80:0/4033392692 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec80fa500 con 0x55af61549c20 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8eacff9640 1 -- 172.21.15.80:0/4033392692 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 41) ==== 3494+0+0 (secure 0 0 0) 0x7f8ec4023020 con 0x55af61549c20 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 10 client.6768 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 10 client.6768 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 20 client.6768 populate_metadata read hostname 'smithi080' 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] conn(0x55af611527d0 0x55af6166ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 4 client.6768 mds_command: new command op to 6627 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8edb23b000 1 -- 172.21.15.80:0/4033392692 --> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af61445be0 con 0x55af611527d0 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.536+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] conn(0x55af611527d0 0x55af6166ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8eacff9640 10 client.6768 ms_handle_connect on v2:172.21.15.87:6836/642879744 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] conn(0x55af611527d0 0x55af6166ae60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c510c4a28b80bbc server_cookie=530d969ff602c60a in_seq=0 out_seq=0 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8eacff9640 1 -- 172.21.15.80:0/4033392692 <== mds.0 v2:172.21.15.87:6836/642879744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55af61445be0 con 0x55af611527d0 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8eacff9640 10 client.6768 handle_command_reply: tid=0 2024-04-25T13:44:42.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 client.6768 shutdown 2024-04-25T13:44:42.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 -- 172.21.15.80:0/4033392692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 msgr2=0x55af6166a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af6166a0c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0d17b0 tx=0x7f8ec400fba0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 -- 172.21.15.80:0/4033392692 shutdown_connections 2024-04-25T13:44:42.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] conn(0x55af611527d0 0x55af6166ae60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af6166a0c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af616aef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 --2- 172.21.15.80:0/4033392692 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af6112e280 0x55af616aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 -- 172.21.15.80:0/4033392692 >> 172.21.15.80:0/4033392692 conn(0x55af6116dff0 msgr2=0x55af6165dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:42.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8eacff9640 0 client.6768 ms_handle_reset on v2:172.21.15.87:6836/642879744 2024-04-25T13:44:42.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 -- 172.21.15.80:0/4033392692 shutdown_connections 2024-04-25T13:44:42.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 1 -- 172.21.15.80:0/4033392692 wait complete. 2024-04-25T13:44:42.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.540+0000 7f8edb23b000 20 client.6768 trim_cache size 0 max 16384 2024-04-25T13:44:42.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8edb23b000 1 Processor -- start 2024-04-25T13:44:42.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8edb23b000 1 -- start start 2024-04-25T13:44:42.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af6112e280 0x55af616bdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61535ea0 0x55af616be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8edb23b000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc203c60 con 0x55af61549c20 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8edb23b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ecc202ff0 con 0x55af61535ea0 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8edb23b000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8ecc203610 con 0x55af6112e280 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616be650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41598/0 (socket says 172.21.15.80:41598) 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 -- 172.21.15.80:0/2265236008 learned_addr learned my addr 172.21.15.80:0/2265236008 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 -- 172.21.15.80:0/2265236008 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af6112e280 msgr2=0x55af616bdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 --2- 172.21.15.80:0/2265236008 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af6112e280 0x55af616bdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed1479640 1 --2- 172.21.15.80:0/2265236008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61535ea0 0x55af616be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 -- 172.21.15.80:0/2265236008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61535ea0 msgr2=0x55af616be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 --2- 172.21.15.80:0/2265236008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61535ea0 0x55af616be110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 -- 172.21.15.80:0/2265236008 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af61661840 con 0x55af61549c20 2024-04-25T13:44:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8ed247b640 1 --2- 172.21.15.80:0/2265236008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616be650 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4002240 tx=0x7f8ec4000f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448500c5ddd89678 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8eaeffd640 1 -- 172.21.15.80:0/2265236008 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec400d800 con 0x55af61549c20 2024-04-25T13:44:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8eaeffd640 1 -- 172.21.15.80:0/2265236008 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec4018070 con 0x55af61549c20 2024-04-25T13:44:42.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.560+0000 7f8eaeffd640 1 -- 172.21.15.80:0/2265236008 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4007330 con 0x55af61549c20 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- 172.21.15.80:0/2265236008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 msgr2=0x55af616be650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 --2- 172.21.15.80:0/2265236008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616be650 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4002240 tx=0x7f8ec4000f50 comp rx=0 tx=0).stop 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- 172.21.15.80:0/2265236008 shutdown_connections 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 --2- 172.21.15.80:0/2265236008 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af61549c20 0x55af616be650 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 --2- 172.21.15.80:0/2265236008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61535ea0 0x55af616be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 --2- 172.21.15.80:0/2265236008 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af6112e280 0x55af616bdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- 172.21.15.80:0/2265236008 >> 172.21.15.80:0/2265236008 conn(0x55af6116dff0 msgr2=0x55af616adaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- 172.21.15.80:0/2265236008 shutdown_connections 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- 172.21.15.80:0/2265236008 wait complete. 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- start start 2024-04-25T13:44:42.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 Processor -- start 2024-04-25T13:44:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af6112e280 0x55af6171e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af6171ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61549c20 0x55af616b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55af614ac130 con 0x55af6112e280 2024-04-25T13:44:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55af614427d0 con 0x55af61549c20 2024-04-25T13:44:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55af614b1450 con 0x55af61535ea0 2024-04-25T13:44:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 10 client.0 _ll_register_callbacks cb 0x7f8ec40012c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:44:42.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af6112e280 0x55af6171e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af6112e280 0x55af6171e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41610/0 (socket says 172.21.15.80:41610) 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/3018964509 learned_addr learned my addr 172.21.15.80:0/3018964509 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1479640 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af6171ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 msgr2=0x55af6171ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed247b640 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61549c20 0x55af616b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af6171ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61549c20 msgr2=0x55af616b2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61549c20 0x55af616b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 -- 172.21.15.80:0/3018964509 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af61465c70 con 0x55af6112e280 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8ed1c7a640 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af6112e280 0x55af6171e790 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80f28a0 tx=0x7f8ec8127cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b628f4dd1d26a6ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8eacff9640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8eacff9640 1 -- 172.21.15.80:0/3018964509 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8127d70 con 0x55af6112e280 2024-04-25T13:44:42.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8eacff9640 1 -- 172.21.15.80:0/3018964509 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f8ec80f53b0 con 0x55af6112e280 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- 172.21.15.80:0/3018964509 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec4018070 con 0x55af6112e280 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8eacff9640 1 -- 172.21.15.80:0/3018964509 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f8ec80fa500 con 0x55af6112e280 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 10 client.6774 fetch_fsmap learned FSMap version 41 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- 172.21.15.80:0/3018964509 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x55af6152f180 con 0x55af6112e280 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8eacff9640 1 -- 172.21.15.80:0/3018964509 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ec4018070 con 0x55af6112e280 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8eacff9640 1 -- 172.21.15.80:0/3018964509 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 41) ==== 3494+0+0 (secure 0 0 0) 0x7f8ec80f1ac0 con 0x55af6112e280 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 10 client.6774 fetch_fsmap finished waiting for FSMap version 41 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 10 client.6774 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 20 client.6774 populate_metadata read hostname 'smithi080' 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] conn(0x55af61140830 0x55af616b3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 4 client.6774 mds_command: new command op to 6627 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.564+0000 7f8edb23b000 1 -- 172.21.15.80:0/3018964509 --> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] -- command(tid 0: {"prefix": "status"}) -- 0x55af614baa30 con 0x55af61140830 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed1479640 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] conn(0x55af61140830 0x55af616b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8eacff9640 10 client.6774 ms_handle_connect on v2:172.21.15.87:6836/642879744 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed1479640 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] conn(0x55af61140830 0x55af616b3400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cf0877712d9becb server_cookie=b448e5daeab4486c in_seq=0 out_seq=0 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8eacff9640 1 -- 172.21.15.80:0/3018964509 <== mds.0 v2:172.21.15.87:6836/642879744 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55af614baa30 con 0x55af61140830 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8eacff9640 10 client.6774 handle_command_reply: tid=0 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 client.6774 shutdown 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 -- 172.21.15.80:0/3018964509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af6112e280 msgr2=0x55af6171e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af6112e280 0x55af6171e790 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80f28a0 tx=0x7f8ec8127cd0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 -- 172.21.15.80:0/3018964509 shutdown_connections 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:6836/642879744,v1:172.21.15.87:6837/642879744] conn(0x55af61140830 0x55af616b3400 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55af61549c20 0x55af616b2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8eacff9640 0 client.6774 ms_handle_reset on v2:172.21.15.87:6836/642879744 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55af61535ea0 0x55af6171ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 --2- 172.21.15.80:0/3018964509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55af6112e280 0x55af6171e790 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 -- 172.21.15.80:0/3018964509 >> 172.21.15.80:0/3018964509 conn(0x55af6116dff0 msgr2=0x55af61664610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:42.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 -- 172.21.15.80:0/3018964509 shutdown_connections 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 1 -- 172.21.15.80:0/3018964509 wait complete. 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8edb23b000 20 client.6774 trim_cache size 0 max 16384 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 6627, 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:44:42.571 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 5.9584671809999996, 2024-04-25T13:44:42.572 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 41, 2024-04-25T13:44:42.572 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 82, 2024-04-25T13:44:42.572 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 82, 2024-04-25T13:44:42.572 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 11.789062093 2024-04-25T13:44:42.572 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 -- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8eb006f430 msgr2=0x7f8eb0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8eb006f430 0x7f8eb0071940 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc15f6b0 tx=0x7f8ebc008040 comp rx=0 tx=0).stop 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 -- 172.21.15.80:0/1965069827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ecc1681b0 msgr2=0x7f8ecc1676a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ecc1681b0 0x7f8ecc1676a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f8ec801ceb0 tx=0x7f8eb8005840 comp rx=0 tx=0).stop 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 -- 172.21.15.80:0/1965069827 shutdown_connections 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8eb006f430 0x7f8eb0071940 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ecc1f7760 0x7f8ecc1f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ecc159b50 0x7f8ecc165c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 --2- 172.21.15.80:0/1965069827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ecc1681b0 0x7f8ecc1676a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 -- 172.21.15.80:0/1965069827 >> 172.21.15.80:0/1965069827 conn(0x7f8ecc06d080 msgr2=0x7f8ecc15aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:42.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 -- 172.21.15.80:0/1965069827 shutdown_connections 2024-04-25T13:44:42.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.568+0000 7f8ed3ede640 1 -- 172.21.15.80:0/1965069827 wait complete. 2024-04-25T13:44:42.585 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:44:42.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4086f34640 1 Processor -- start 2024-04-25T13:44:42.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4086f34640 1 -- start start 2024-04-25T13:44:42.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4086f34640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4080073f00 0x7f4080074310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4086f34640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4080157ce0 0x7f40801580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4086f34640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40800cd5f0 con 0x7f408015e960 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4086f34640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f408006e9f0 con 0x7f4080073f00 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4086f34640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f408015eeb0 con 0x7f4080157ce0 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4084cd0640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f408015e960 0x7f40800ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59586/0 (socket says 172.21.15.80:59586) 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f4084cd0640 1 -- 172.21.15.80:0/3201290278 learned_addr learned my addr 172.21.15.80:0/3201290278 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f407ffff640 1 --2- 172.21.15.80:0/3201290278 >> v2:172.21.15.87:3300/0 conn(0x7f4080073f00 0x7f4080074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2426455755 0 0) 0x7f4068001120 con 0x7f408015e960 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f40854d1640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4080157ce0 0x7f40801580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.716+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f406c003530 con 0x7f408015e960 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1418774046 0 0) 0x7f406c003530 con 0x7f408015e960 2024-04-25T13:44:42.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4080157ce0 msgr2=0x7f40801580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4080157ce0 0x7f40801580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 >> v2:172.21.15.87:3300/0 conn(0x7f4080073f00 msgr2=0x7f4080074310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 --2- 172.21.15.80:0/3201290278 >> v2:172.21.15.87:3300/0 conn(0x7f4080073f00 0x7f4080074310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408015f190 con 0x7f408015e960 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4068001850 con 0x7f408015e960 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 >> v1:172.21.15.80:6789/0 conn(0x7f408015e960 legacy=0x7f40800ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4080157ce0 0x7f406c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406c003f60 0x7f406c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406c004d50 0x7f406c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f408006e9f0 con 0x7f406c003f60 2024-04-25T13:44:42.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f408015eeb0 con 0x7f406c004d50 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f406c00bb00 con 0x7f4080157ce0 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407ffff640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406c003f60 0x7f406c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f4084cd0640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4080157ce0 0x7f406c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f40854d1640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406c004d50 0x7f406c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407ffff640 1 -- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4080157ce0 msgr2=0x7f406c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407ffff640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4080157ce0 0x7f406c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407ffff640 1 -- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406c004d50 msgr2=0x7f406c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407ffff640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406c004d50 0x7f406c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407ffff640 1 -- 172.21.15.80:0/3201290278 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406c003530 con 0x7f406c003f60 2024-04-25T13:44:42.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f40854d1640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406c004d50 0x7f406c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:42.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407ffff640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406c003f60 0x7f406c004810 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4070009ae0 tx=0x7f407001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8674bbe29a23181 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700b2070 con 0x7f406c003f60 2024-04-25T13:44:42.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40700a5640 con 0x7f406c003f60 2024-04-25T13:44:42.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f407f7fe640 1 -- 172.21.15.80:0/3201290278 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700096b0 con 0x7f406c003f60 2024-04-25T13:44:42.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f4086f34640 1 -- 172.21.15.80:0/3201290278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406c003f60 msgr2=0x7f406c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f4086f34640 1 --2- 172.21.15.80:0/3201290278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406c003f60 0x7f406c004810 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f4070009ae0 tx=0x7f407001ca50 comp rx=0 tx=0).stop 2024-04-25T13:44:42.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f40854d1640 1 -- 172.21.15.80:0/3201290278 reap_dead start 2024-04-25T13:44:42.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f4086f34640 1 -- 172.21.15.80:0/3201290278 shutdown_connections 2024-04-25T13:44:42.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f4086f34640 1 -- 172.21.15.80:0/3201290278 >> 172.21.15.80:0/3201290278 conn(0x7f40800d20d0 msgr2=0x7f40800d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:42.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f4086f34640 1 -- 172.21.15.80:0/3201290278 shutdown_connections 2024-04-25T13:44:42.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.720+0000 7f4086f34640 1 -- 172.21.15.80:0/3201290278 wait complete. 2024-04-25T13:44:42.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 Processor -- start 2024-04-25T13:44:42.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 -- start start 2024-04-25T13:44:42.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f406c003f60 0x7f40801f3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40800b6ba0 0x7f40800b6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408009e3e0 0x7f408009e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4080074850 con 0x7f40800b6ba0 2024-04-25T13:44:42.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f408015f470 con 0x7f408009e3e0 2024-04-25T13:44:42.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40801580f0 con 0x7f406c003f60 2024-04-25T13:44:42.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40800b6ba0 0x7f40800b6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40800b6ba0 0x7f40800b6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41620/0 (socket says 172.21.15.80:41620) 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 -- 172.21.15.80:0/508573169 learned_addr learned my addr 172.21.15.80:0/508573169 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f40854d1640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408009e3e0 0x7f408009e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4084cd0640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f406c003f60 0x7f40801f3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 -- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f406c003f60 msgr2=0x7f40801f3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f406c003f60 0x7f40801f3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 -- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408009e3e0 msgr2=0x7f408009e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408009e3e0 0x7f408009e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 -- 172.21.15.80:0/508573169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40800b74f0 con 0x7f40800b6ba0 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407ffff640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40800b6ba0 0x7f40800b6fb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4070009ae0 tx=0x7f406c005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407d7fa640 1 -- 172.21.15.80:0/508573169 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40700b2040 con 0x7f40800b6ba0 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407d7fa640 1 -- 172.21.15.80:0/508573169 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40700a5df0 con 0x7f40800b6ba0 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408009d340 con 0x7f40800b6ba0 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f407d7fa640 1 -- 172.21.15.80:0/508573169 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4070004020 con 0x7f40800b6ba0 2024-04-25T13:44:42.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.724+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f408009d900 con 0x7f40800b6ba0 2024-04-25T13:44:42.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.728+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f408006a3a0 con 0x7f40800b6ba0 2024-04-25T13:44:42.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.728+0000 7f407d7fa640 1 -- 172.21.15.80:0/508573169 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f40700c0030 con 0x7f40800b6ba0 2024-04-25T13:44:42.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.728+0000 7f407d7fa640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f406006f400 0x7f4060071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:42.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.728+0000 7f407d7fa640 1 -- 172.21.15.80:0/508573169 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f40701290e0 con 0x7f40800b6ba0 2024-04-25T13:44:42.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.728+0000 7f4084cd0640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f406006f400 0x7f4060071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:42.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.728+0000 7f4084cd0640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f406006f400 0x7f4060071910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4068001a40 tx=0x7f4068001120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:42.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.732+0000 7f407d7fa640 1 -- 172.21.15.80:0/508573169 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f40700c7060 con 0x7f40800b6ba0 2024-04-25T13:44:42.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.916+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f40800d2900 con 0x7f40800b6ba0 2024-04-25T13:44:42.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.916+0000 7f407d7fa640 1 -- 172.21.15.80:0/508573169 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) ==== 53+0+73 (secure 0 0 0) 0x7f40800b74f0 con 0x7f40800b6ba0 2024-04-25T13:44:42.916 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:44:42.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f406006f400 msgr2=0x7f4060071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f406006f400 0x7f4060071910 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4068001a40 tx=0x7f4068001120 comp rx=0 tx=0).stop 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40800b6ba0 msgr2=0x7f40800b6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40800b6ba0 0x7f40800b6fb0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f4070009ae0 tx=0x7f406c005840 comp rx=0 tx=0).stop 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 shutdown_connections 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f406006f400 0x7f4060071910 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f408009e3e0 0x7f408009e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40800b6ba0 0x7f40800b6fb0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 --2- 172.21.15.80:0/508573169 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f406c003f60 0x7f40801f3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 >> 172.21.15.80:0/508573169 conn(0x7f40800d20d0 msgr2=0x7f40801574b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 shutdown_connections 2024-04-25T13:44:42.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:42.920+0000 7f4086f34640 1 -- 172.21.15.80:0/508573169 wait complete. 2024-04-25T13:44:42.936 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:44:42.936 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:44:42.936 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:44:42.937 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:44:42.937 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:44:42.937 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:44:42.937 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:44:42.937 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:44:42.937 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:44:42.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:42.938 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:44:42.944 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:44:42.944 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:44:42.944 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:44:42.944 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:42.944 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:44:42.944 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:42.944 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:44:42.944 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42285sec preferred_lft 42285sec 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:44:42.945 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:44:42.946 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:44:42.947 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:44:42.947 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:42.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:42.947 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:44:42.993 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:44:42.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:42.994 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:44:43.046 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:44:43.046 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:44:43.046 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:44:43.046 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:44:43.046 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:44:43.046 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:44:43.047 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:44:43.047 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:44:43.047 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:44:43.047 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:44:43.047 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:44:43.047 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:44:43.047 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:44:43.047 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:44:43.048 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:44:43.048 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:44:43.048 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:44:43.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:44:43.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:44:43.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:43.165 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:44:43.214 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:44:43.214 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:44:43.214 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:44:43.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:43.214 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:44:43.270 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:44:43.270 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:44:43.270 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:44:43.318 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:44:43.319 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:44:43.319 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:44:43.366 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:44:43.423 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:44:43.470 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:44:43.471 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:44:43.471 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:44:43.471 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:44:43.471 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:44:43.471 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:44:43.472 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:44:43.472 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:44:43.472 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:44:43.472 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:44:43.472 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:44:43.472 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:44:43.472 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:44:43.473 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:44:43.473 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:44:43.473 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:44:43.473 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:44:43.473 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:44:43.473 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:44:43.474 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:44:43.474 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:44:43.474 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:44:43.474 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:44:43.474 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:44:43.474 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:44:43.475 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:44:43.475 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:44:43.475 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:44:43.475 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:44:43.475 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:44:43.475 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:44:43.476 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:44:43.476 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:44:43.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:43.478 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:44:43.526 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:44:43.527 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:44:43.570 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:44:43.584 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:44:43.630 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:44:43.630 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:44:43.630 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:44:43.630 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:44:43.630 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:44:43.630 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:44:43.630 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:44:43.631 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:44:43.632 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:44:43.632 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:44:43.632 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:44:43.632 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:44:43.632 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:44:43.632 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:44:43.632 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:44:43.633 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:44:43.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:43.634 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:44:43.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:44:43.680+0000 7f6fc8d27580 -1 init, newargv = 0x7f6fac005830 newargc=15 2024-04-25T13:44:43.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[33451]: starting ceph client 2024-04-25T13:44:43.686 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[33451]: starting fuse 2024-04-25T13:44:44.778 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:44:44.795 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:44:44.843 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:44:44.844 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:44:44.845 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:44:44.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:44.846 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:44:44.895 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:44:44.895 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:44:44.895 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:44:44.896 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:44:44.897 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:44:45.018 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.33451.asok 2024-04-25T13:44:45.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:44:45.030 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.33451.asok 2024-04-25T13:44:45.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:45.030 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.33451.asok status 2024-04-25T13:44:45.209 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:44:45.209 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:44:45.209 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:44:45.209 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "33451", 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 6786, 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 6786 2024-04-25T13:44:45.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3575631055 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3575631055 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.6786 192.168.0.1:0/3575631055", 2024-04-25T13:44:45.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/3575631055", 2024-04-25T13:44:45.212 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:44:45.212 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 41, 2024-04-25T13:44:45.212 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 82, 2024-04-25T13:44:45.212 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:44:45.212 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:44:45.212 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:44:45.212 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:44:45.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:45.228 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:44:45.228 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:44:45.235 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:44:45.235 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:44:45.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:45.235 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:44:45.292 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:44:45.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f7432336640 1 Processor -- start 2024-04-25T13:44:45.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f7432336640 1 -- start start 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f7432336640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f742c160400 0x7f742c068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f7432336640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f742c068ee0 con 0x7f742c164870 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f7432336640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f742c06ece0 con 0x7f742c068ad0 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f7432336640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f742c0d0bf0 con 0x7f742c160400 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f742b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f742c160400 0x7f742c068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f742b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f742c160400 0x7f742c068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41632/0 (socket says 172.21.15.80:41632) 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f742b7fe640 1 -- 172.21.15.80:0/92062144 learned_addr learned my addr 172.21.15.80:0/92062144 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.488+0000 7f742bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f742c164870 0x7f742c0d04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59598/0 (socket says 172.21.15.80:59598) 2024-04-25T13:44:45.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f742b7fe640 1 -- 172.21.15.80:0/92062144 >> v1:172.21.15.87:6789/0 conn(0x7f742c068ad0 legacy=0x7f742c06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:45.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f742b7fe640 1 -- 172.21.15.80:0/92062144 >> v1:172.21.15.80:6789/0 conn(0x7f742c164870 legacy=0x7f742c0d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:45.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f742b7fe640 1 -- 172.21.15.80:0/92062144 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c06efc0 con 0x7f742c160400 2024-04-25T13:44:45.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f742affd640 1 -- 172.21.15.80:0/92062144 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 738153076 0 0) 0x7f741c001120 con 0x7f742c164870 2024-04-25T13:44:45.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f742b7fe640 1 --2- 172.21.15.80:0/92062144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c160400 0x7f742c068590 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7420009870 tx=0x7f742001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a596cc4808229bdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:45.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f742affd640 1 -- 172.21.15.80:0/92062144 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200b1070 con 0x7f742c160400 2024-04-25T13:44:45.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f742affd640 1 -- 172.21.15.80:0/92062144 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f74200ac640 con 0x7f742c160400 2024-04-25T13:44:45.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f742affd640 1 -- 172.21.15.80:0/92062144 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74200aca10 con 0x7f742c160400 2024-04-25T13:44:45.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f7432336640 1 -- 172.21.15.80:0/92062144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c160400 msgr2=0x7f742c068590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:45.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f7432336640 1 --2- 172.21.15.80:0/92062144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c160400 0x7f742c068590 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7420009870 tx=0x7f742001c400 comp rx=0 tx=0).stop 2024-04-25T13:44:45.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f7432336640 1 -- 172.21.15.80:0/92062144 shutdown_connections 2024-04-25T13:44:45.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f7432336640 1 --2- 172.21.15.80:0/92062144 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c160400 0x7f742c068590 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:45.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f7432336640 1 -- 172.21.15.80:0/92062144 >> 172.21.15.80:0/92062144 conn(0x7f742c0cf390 msgr2=0x7f742c0cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:45.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f7432336640 1 -- 172.21.15.80:0/92062144 shutdown_connections 2024-04-25T13:44:45.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.492+0000 7f7432336640 1 -- 172.21.15.80:0/92062144 wait complete. 2024-04-25T13:44:45.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.496+0000 7f7432336640 1 Processor -- start 2024-04-25T13:44:45.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.496+0000 7f7432336640 1 -- start start 2024-04-25T13:44:45.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.496+0000 7f7432336640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f742c068ad0 0x7f742c15ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:45.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7432336640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f742c160400 0x7f742c148280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:45.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7432336640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c164870 0x7f742c15de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:45.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7432336640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f742c06c5f0 con 0x7f742c164870 2024-04-25T13:44:45.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7432336640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f742c070d10 con 0x7f742c160400 2024-04-25T13:44:45.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7432336640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f742c16ec30 con 0x7f742c068ad0 2024-04-25T13:44:45.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c164870 0x7f742c15de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:45.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c164870 0x7f742c15de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41642/0 (socket says 172.21.15.80:41642) 2024-04-25T13:44:45.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 -- 172.21.15.80:0/2531105073 learned_addr learned my addr 172.21.15.80:0/2531105073 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f742b7fe640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f742c160400 0x7f742c148280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f742bfff640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f742c068ad0 0x7f742c15ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 -- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f742c068ad0 msgr2=0x7f742c15ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f742c068ad0 0x7f742c15ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 -- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f742c160400 msgr2=0x7f742c148280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f742c160400 0x7f742c148280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 -- 172.21.15.80:0/2531105073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f742c1638a0 con 0x7f742c164870 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f742bfff640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f742c068ad0 0x7f742c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f742b7fe640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f742c160400 0x7f742c148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f74308d3640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c164870 0x7f742c15de70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f742400b9a0 tx=0x7f742400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7428ff9640 1 -- 172.21.15.80:0/2531105073 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424019070 con 0x7f742c164870 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7428ff9640 1 -- 172.21.15.80:0/2531105073 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f742400cba0 con 0x7f742c164870 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f742c162330 con 0x7f742c164870 2024-04-25T13:44:45.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7428ff9640 1 -- 172.21.15.80:0/2531105073 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424014440 con 0x7f742c164870 2024-04-25T13:44:45.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f742c1628f0 con 0x7f742c164870 2024-04-25T13:44:45.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f742c06a3a0 con 0x7f742c164870 2024-04-25T13:44:45.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7428ff9640 1 -- 172.21.15.80:0/2531105073 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f742401c070 con 0x7f742c164870 2024-04-25T13:44:45.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7428ff9640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f741806f400 0x7f7418071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:45.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.500+0000 7f7428ff9640 1 -- 172.21.15.80:0/2531105073 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f742408f040 con 0x7f742c164870 2024-04-25T13:44:45.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.504+0000 7f742bfff640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f741806f400 0x7f7418071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:45.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.504+0000 7f742bfff640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f741806f400 0x7f7418071910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f741c007fd0 tx=0x7f741c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:45.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.504+0000 7f7428ff9640 1 -- 172.21.15.80:0/2531105073 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f742405c010 con 0x7f742c164870 2024-04-25T13:44:45.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.704+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f742c163630 con 0x7f742c164870 2024-04-25T13:44:45.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7428ff9640 1 -- 172.21.15.80:0/2531105073 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v41) ==== 53+0+73 (secure 0 0 0) 0x7f742405b910 con 0x7f742c164870 2024-04-25T13:44:45.706 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:44:45.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f741806f400 msgr2=0x7f7418071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:45.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f741806f400 0x7f7418071910 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f741c007fd0 tx=0x7f741c009040 comp rx=0 tx=0).stop 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c164870 msgr2=0x7f742c15de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c164870 0x7f742c15de70 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f742400b9a0 tx=0x7f742400be60 comp rx=0 tx=0).stop 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 shutdown_connections 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f741806f400 0x7f7418071910 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f742c164870 0x7f742c15de70 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f742c160400 0x7f742c148280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 --2- 172.21.15.80:0/2531105073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f742c068ad0 0x7f742c15ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 >> 172.21.15.80:0/2531105073 conn(0x7f742c0cf390 msgr2=0x7f742c15b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 shutdown_connections 2024-04-25T13:44:45.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:45.708+0000 7f7432336640 1 -- 172.21.15.80:0/2531105073 wait complete. 2024-04-25T13:44:45.727 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:44:45.727 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:44:45.727 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:44:45.727 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:44:45.727 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:44:45.727 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:44:45.728 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:44:45.728 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:44:45.728 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:44:45.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:45.728 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:44:45.735 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:44:45.735 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:44:45.735 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:44:45.735 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:45.735 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:44:45.736 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:45.736 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:44:45.736 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:45.736 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:44:45.736 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:44:45.736 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:45.736 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:44:45.736 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42283sec preferred_lft 42283sec 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:44:45.737 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:44:45.738 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:44:45.739 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:44:45.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:45.739 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:44:45.792 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:44:45.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:45.793 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:44:45.847 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:44:45.847 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:44:45.847 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:44:45.847 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:44:45.848 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:44:45.848 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:44:45.848 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:44:45.848 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:44:45.848 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:44:45.848 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:44:45.848 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:44:45.848 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:44:45.848 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:44:45.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:44:45.848 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:44:45.849 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:44:45.849 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:44:45.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:44:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:44:45.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:44:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:44:45.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:45.973 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:44:45.979 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:44:45.979 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:44:45.980 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:44:45.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:45.980 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:44:46.034 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:44:46.034 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:44:46.035 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:44:46.082 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:44:46.083 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:44:46.083 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:44:46.130 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:44:46.188 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:44:46.235 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:44:46.236 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:44:46.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:44:46.238 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:44:46.238 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:44:46.238 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:44:46.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:46.239 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:44:46.286 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:44:46.287 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:44:46.331 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:44:46.345 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:44:46.390 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:44:46.390 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:44:46.391 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:44:46.392 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:44:46.393 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:44:46.393 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:44:46.393 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:44:46.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:46.394 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:44:46.441 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:44:46.437+0000 7f29527be580 -1 init, newargv = 0x7f293c001090 newargc=15 2024-04-25T13:44:46.442 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24761]: starting ceph client 2024-04-25T13:44:46.446 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24761]: starting fuse 2024-04-25T13:44:47.543 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:44:47.559 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:44:47.609 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:44:47.609 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:44:47.609 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:44:47.610 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:44:47.611 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:44:47.612 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:44:47.612 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:44:47.612 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:44:47.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:47.613 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:44:47.659 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:44:47.659 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:44:47.659 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:44:47.659 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:44:47.659 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:44:47.659 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:44:47.659 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:44:47.659 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:44:47.659 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:44:47.660 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:44:47.661 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:44:47.661 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:44:47.661 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:44:47.661 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:44:47.784 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.24761.asok 2024-04-25T13:44:47.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:44:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:44:47.796 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.24761.asok 2024-04-25T13:44:47.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:47.796 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.24761.asok status 2024-04-25T13:44:47.972 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "24761", 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5785, 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:44:47.973 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5785 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1666016735 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:44:47.974 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1666016735 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5785 192.168.0.1:0/1666016735", 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/1666016735", 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 41, 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 82, 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:44:47.975 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:44:47.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:47.988 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:44:47.988 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:44:47.995 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:44:47.995 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:44:47.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:44:47.995 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:44:48.049 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:44:48.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3fce78640 1 Processor -- start 2024-04-25T13:44:48.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3fce78640 1 -- start start 2024-04-25T13:44:48.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3fce78640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3f81644a0 0x7fe3f80b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3fce78640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f806ea30 con 0x7fe3f80731e0 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3fce78640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3f806ed10 con 0x7fe3f80b5450 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3fce78640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f815acb0 con 0x7fe3f81644a0 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f659c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe3f80731e0 0x7fe3f8163d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59600/0 (socket says 172.21.15.80:59600) 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f659c640 1 -- 172.21.15.80:0/306021709 learned_addr learned my addr 172.21.15.80:0/306021709 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f5d9b640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3f81644a0 0x7fe3f80b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487224273 0 0) 0x7fe3f806ea30 con 0x7fe3f80731e0 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe3e0003530 con 0x7fe3f80731e0 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3015870225 0 0) 0x7fe3e0003530 con 0x7fe3f80731e0 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3f81644a0 msgr2=0x7fe3f80b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3f81644a0 0x7fe3f80b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 >> v1:172.21.15.87:6789/0 conn(0x7fe3f80b5450 legacy=0x7fe3f80b5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:48.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f806eff0 con 0x7fe3f80731e0 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fe3f0001a00 con 0x7fe3f80731e0 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 >> v1:172.21.15.80:6789/0 conn(0x7fe3f80731e0 legacy=0x7fe3f8163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3e00039f0 0x7fe3e0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3e0004810 0x7fe3e0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e00051b0 0x7fe3e000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.208+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f815acb0 con 0x7fe3e0004810 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e000bf10 con 0x7fe3e00039f0 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e000c110 con 0x7fe3e00051b0 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f5d9b640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3e0004810 0x7fe3e0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f659c640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3e00039f0 0x7fe3e0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f6d9d640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e00051b0 0x7fe3e000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f5d9b640 1 -- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e00051b0 msgr2=0x7fe3e000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f5d9b640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e00051b0 0x7fe3e000b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f5d9b640 1 -- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3e00039f0 msgr2=0x7fe3e0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:44:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f5d9b640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3e00039f0 0x7fe3e0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f5d9b640 1 -- 172.21.15.80:0/306021709 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e0003530 con 0x7fe3e0004810 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f5d9b640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3e0004810 0x7fe3e0004c20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009ab0 tx=0x7fe3ec01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b9f02743a50fd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0b2070 con 0x7fe3e0004810 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3ec0a5550 con 0x7fe3e0004810 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f559a640 1 -- 172.21.15.80:0/306021709 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0096b0 con 0x7fe3e0004810 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3fce78640 1 -- 172.21.15.80:0/306021709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3e0004810 msgr2=0x7fe3e0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3fce78640 1 --2- 172.21.15.80:0/306021709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3e0004810 0x7fe3e0004c20 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009ab0 tx=0x7fe3ec01ca70 comp rx=0 tx=0).stop 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3f6d9d640 1 -- 172.21.15.80:0/306021709 reap_dead start 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3fce78640 1 -- 172.21.15.80:0/306021709 shutdown_connections 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3fce78640 1 -- 172.21.15.80:0/306021709 >> 172.21.15.80:0/306021709 conn(0x7fe3f806d080 msgr2=0x7fe3f80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3fce78640 1 -- 172.21.15.80:0/306021709 shutdown_connections 2024-04-25T13:44:48.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.212+0000 7fe3fce78640 1 -- 172.21.15.80:0/306021709 wait complete. 2024-04-25T13:44:48.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 Processor -- start 2024-04-25T13:44:48.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 -- start start 2024-04-25T13:44:48.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e0004810 0x7fe3f8147e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:48.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f81472a0 0x7fe3f8158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:48.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f8158ab0 0x7fe3f815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:48.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f806e650 con 0x7fe3f8158ab0 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8164b70 con 0x7fe3f81472a0 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f806e7d0 con 0x7fe3e0004810 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f8158ab0 0x7fe3f815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f8158ab0 0x7fe3f815c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41654/0 (socket says 172.21.15.80:41654) 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 -- 172.21.15.80:0/2731630085 learned_addr learned my addr 172.21.15.80:0/2731630085 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f5d9b640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f81472a0 0x7fe3f8158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f659c640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e0004810 0x7fe3f8147e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 -- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e0004810 msgr2=0x7fe3f8147e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e0004810 0x7fe3f8147e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 -- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f81472a0 msgr2=0x7fe3f8158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f81472a0 0x7fe3f8158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 -- 172.21.15.80:0/2731630085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f8148380 con 0x7fe3f8158ab0 2024-04-25T13:44:48.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3f6d9d640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f8158ab0 0x7fe3f815c2e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8071090 tx=0x7fe3e800d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3e6ffd640 1 -- 172.21.15.80:0/2731630085 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8018070 con 0x7fe3f8158ab0 2024-04-25T13:44:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3e6ffd640 1 -- 172.21.15.80:0/2731630085 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3e8003600 con 0x7fe3f8158ab0 2024-04-25T13:44:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f815b240 con 0x7fe3f8158ab0 2024-04-25T13:44:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3e6ffd640 1 -- 172.21.15.80:0/2731630085 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e80039d0 con 0x7fe3f8158ab0 2024-04-25T13:44:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.216+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f815b800 con 0x7fe3f8158ab0 2024-04-25T13:44:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.220+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f815bf20 con 0x7fe3f8158ab0 2024-04-25T13:44:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.220+0000 7fe3e6ffd640 1 -- 172.21.15.80:0/2731630085 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe3e801b070 con 0x7fe3f8158ab0 2024-04-25T13:44:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.220+0000 7fe3e6ffd640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe3d806f430 0x7fe3d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:44:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.220+0000 7fe3e6ffd640 1 -- 172.21.15.80:0/2731630085 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7fe3e808e860 con 0x7fe3f8158ab0 2024-04-25T13:44:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.220+0000 7fe3f659c640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe3d806f430 0x7fe3d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:44:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.220+0000 7fe3f659c640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe3d806f430 0x7fe3d8071940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0002060 tx=0x7fe3f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:44:48.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.224+0000 7fe3e6ffd640 1 -- 172.21.15.80:0/2731630085 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe3e805b7e0 con 0x7fe3f8158ab0 2024-04-25T13:44:48.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.412+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3f815afd0 con 0x7fe3f8158ab0 2024-04-25T13:44:48.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.412+0000 7fe3e6ffd640 1 -- 172.21.15.80:0/2731630085 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5180 (secure 0 0 0) 0x7fe3e808e020 con 0x7fe3f8158ab0 2024-04-25T13:44:48.412 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:44:48.412 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2082291876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2082291876},{"type":"v1","addr":"172.21.15.87:6835","nonce":2082291876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1337331528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1337331528},{"type":"v1","addr":"172.21.15.80:6833","nonce":1337331528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2521189874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2521189874},{"type":"v1","addr":"172.21.15.80:6835","nonce":2521189874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:44:35.958393+0000","modified":"2024-04-25T13:44:39.019142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/642879744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":642879744},{"type":"v1","addr":"172.21.15.87:6837","nonce":642879744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":5}]} 2024-04-25T13:44:48.412 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 41 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe3d806f430 msgr2=0x7fe3d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe3d806f430 0x7fe3d8071940 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0002060 tx=0x7fe3f0009040 comp rx=0 tx=0).stop 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f8158ab0 msgr2=0x7fe3f815c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f8158ab0 0x7fe3f815c2e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8071090 tx=0x7fe3e800d890 comp rx=0 tx=0).stop 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 shutdown_connections 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe3d806f430 0x7fe3d8071940 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3f8158ab0 0x7fe3f815c2e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe3f81472a0 0x7fe3f8158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 --2- 172.21.15.80:0/2731630085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3e0004810 0x7fe3f8147e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 >> 172.21.15.80:0/2731630085 conn(0x7fe3f806d080 msgr2=0x7fe3f80af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:44:48.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 shutdown_connections 2024-04-25T13:44:48.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:44:48.416+0000 7fe3fce78640 1 -- 172.21.15.80:0/2731630085 wait complete. 2024-04-25T13:44:48.433 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-04-25T13:44:48.611 INFO:teuthology.orchestra.run.smithi087.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":6627,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.987107682,"mdsmap_epoch":41,"osdmap_epoch":82,"osdmap_epoch_barrier":82,"uptime":17.817704645999999} 2024-04-25T13:44:48.611 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 6627, "mdsmap_epoch": 41, "osdmap_epoch": 82, "osdmap_epoch_barrier": 82, "rank_uptime": 11.987107682, "state": "up:active", "uptime": 17.817704646, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:44:48.612 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-25T13:44:48.777 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_mon_report_interval":"5"} 2024-04-25T13:44:48.777 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-25T13:44:48.778 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-04-25T13:44:48.925 INFO:teuthology.orchestra.run.smithi080.stdout:{"mon_tick_interval":"5"} 2024-04-25T13:44:48.926 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-04-25T13:44:48.926 INFO:tasks.cephfs.test_full:pool capacity 557049408, 531MB should be enough to fill it 2024-04-25T13:44:48.926 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:44:48.926 DEBUG:teuthology.orchestra.run.smithi080:> import time 2024-04-25T13:44:48.926 DEBUG:teuthology.orchestra.run.smithi080:> import datetime 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> # Write some buffered data through before going full, all should be well 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> print("writing some data through which we expect to succeed") 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> bytes = 0 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(f) 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-04-25T13:44:48.927 DEBUG:teuthology.orchestra.run.smithi080:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> # buffered IO, the error should always only appear from fsync and not 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> # from write 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> full = False 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> for n in range(0, int(531 * 1.1)): 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> try: 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> print("wrote bytes via buffered write, moving on to fsync") 2024-04-25T13:44:48.928 DEBUG:teuthology.orchestra.run.smithi080:> except OSError as e: 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> if True: 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> raise 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> else: 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> full = True 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> break 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> try: 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(f) 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> print("fsync'"'"'ed successfully") 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> except OSError as e: 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> full = True 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> break 2024-04-25T13:44:48.929 DEBUG:teuthology.orchestra.run.smithi080:> else: 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> if n > 531 * 0.9: 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> # Be cautious in the last region where we expect to hit 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> time.sleep(15.0) 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> if not full: 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-04-25T13:44:48.930 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> # close() should not raise an error because we already caught it in 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> # since then because all IOs got cancelled on the full flag. 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> print("calling close") 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> os.close(f) 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> print("close() did not raise error") 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-04-25T13:44:48.931 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:44:51.640 INFO:teuthology.orchestra.run.smithi080.stdout:writing some data through which we expect to succeed 2024-04-25T13:44:51.640 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 1.00 MB 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 2.00 MB 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 3.00 MB 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 4.00 MB 2024-04-25T13:44:51.641 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 5.00 MB 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 6.00 MB 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 7.00 MB 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 8.00 MB 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.642 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 9.00 MB 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 10.00 MB 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 11.00 MB 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 12.00 MB 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 13.00 MB 2024-04-25T13:44:51.643 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 14.00 MB 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 15.00 MB 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 16.00 MB 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 17.00 MB 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 18.00 MB 2024-04-25T13:44:51.644 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 19.00 MB 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 20.00 MB 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 21.00 MB 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 22.00 MB 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 23.00 MB 2024-04-25T13:44:51.645 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 24.00 MB 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 25.00 MB 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 26.00 MB 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 27.00 MB 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.646 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 28.00 MB 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 29.00 MB 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 30.00 MB 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 31.00 MB 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 32.00 MB 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.647 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 33.00 MB 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 34.00 MB 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 35.00 MB 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 36.00 MB 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 37.00 MB 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.648 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 38.00 MB 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 39.00 MB 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 40.00 MB 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 41.00 MB 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 42.00 MB 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.649 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 43.00 MB 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 44.00 MB 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 45.00 MB 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 46.00 MB 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 47.00 MB 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.650 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 48.00 MB 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 49.00 MB 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 50.00 MB 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 51.00 MB 2024-04-25T13:44:51.651 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 52.00 MB 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 53.00 MB 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 54.00 MB 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.652 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 55.00 MB 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 56.00 MB 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 57.00 MB 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 58.00 MB 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 59.00 MB 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.653 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 60.00 MB 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 61.00 MB 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 62.00 MB 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 63.00 MB 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 64.00 MB 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.654 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 65.00 MB 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 66.00 MB 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 67.00 MB 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 68.00 MB 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.655 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 69.00 MB 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 70.00 MB 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 71.00 MB 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 72.00 MB 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.657 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 73.00 MB 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 74.00 MB 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 75.00 MB 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.658 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 76.00 MB 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 77.00 MB 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 78.00 MB 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 79.00 MB 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:51.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 80.00 MB 2024-04-25T13:44:52.493 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.493 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.493 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 81.00 MB 2024-04-25T13:44:52.493 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.493 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.493 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 82.00 MB 2024-04-25T13:44:52.493 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.494 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.494 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 83.00 MB 2024-04-25T13:44:52.494 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.494 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.494 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 84.00 MB 2024-04-25T13:44:52.494 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.495 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.495 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 85.00 MB 2024-04-25T13:44:52.495 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.495 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.495 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 86.00 MB 2024-04-25T13:44:52.495 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.495 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 87.00 MB 2024-04-25T13:44:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 88.00 MB 2024-04-25T13:44:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 89.00 MB 2024-04-25T13:44:52.496 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 90.00 MB 2024-04-25T13:44:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 91.00 MB 2024-04-25T13:44:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.497 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 92.00 MB 2024-04-25T13:44:52.498 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.498 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.498 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 93.00 MB 2024-04-25T13:44:52.498 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.499 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.499 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 94.00 MB 2024-04-25T13:44:52.499 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.499 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.499 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 95.00 MB 2024-04-25T13:44:52.499 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.499 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.500 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 96.00 MB 2024-04-25T13:44:52.500 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.500 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.500 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 97.00 MB 2024-04-25T13:44:52.500 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.500 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.501 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 98.00 MB 2024-04-25T13:44:52.501 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.501 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.501 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 99.00 MB 2024-04-25T13:44:52.501 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.501 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.501 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 100.00 MB 2024-04-25T13:44:52.501 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.502 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.502 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 101.00 MB 2024-04-25T13:44:52.502 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.502 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.502 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 102.00 MB 2024-04-25T13:44:52.502 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.503 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.503 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 103.00 MB 2024-04-25T13:44:52.503 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.503 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.503 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 104.00 MB 2024-04-25T13:44:52.503 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.503 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.503 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 105.00 MB 2024-04-25T13:44:52.504 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.504 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.504 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 106.00 MB 2024-04-25T13:44:52.504 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.504 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.505 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 107.00 MB 2024-04-25T13:44:52.505 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.505 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.505 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 108.00 MB 2024-04-25T13:44:52.505 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.505 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.506 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 109.00 MB 2024-04-25T13:44:52.506 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.506 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.506 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 110.00 MB 2024-04-25T13:44:52.506 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.506 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.506 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 111.00 MB 2024-04-25T13:44:52.506 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 112.00 MB 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 113.00 MB 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 114.00 MB 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 115.00 MB 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.507 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 116.00 MB 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 117.00 MB 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 118.00 MB 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 119.00 MB 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 120.00 MB 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.508 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 121.00 MB 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 122.00 MB 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 123.00 MB 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 124.00 MB 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 125.00 MB 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.509 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 126.00 MB 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 127.00 MB 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 128.00 MB 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 129.00 MB 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 130.00 MB 2024-04-25T13:44:52.510 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 131.00 MB 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 132.00 MB 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 133.00 MB 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 134.00 MB 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 135.00 MB 2024-04-25T13:44:52.511 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 136.00 MB 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 137.00 MB 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 138.00 MB 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 139.00 MB 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 140.00 MB 2024-04-25T13:44:52.512 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 141.00 MB 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 142.00 MB 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 143.00 MB 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 144.00 MB 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 145.00 MB 2024-04-25T13:44:52.513 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 146.00 MB 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 147.00 MB 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 148.00 MB 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 149.00 MB 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.514 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 150.00 MB 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 151.00 MB 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 152.00 MB 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 153.00 MB 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 154.00 MB 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.515 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 155.00 MB 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 156.00 MB 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 157.00 MB 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 158.00 MB 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 159.00 MB 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:52.516 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:52.517 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 160.00 MB 2024-04-25T13:44:52.517 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.292 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.292 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 161.00 MB 2024-04-25T13:44:53.292 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 162.00 MB 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 163.00 MB 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 164.00 MB 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 165.00 MB 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.293 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 166.00 MB 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 167.00 MB 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 168.00 MB 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 169.00 MB 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 170.00 MB 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.294 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.295 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 171.00 MB 2024-04-25T13:44:53.295 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.295 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.295 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 172.00 MB 2024-04-25T13:44:53.295 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.295 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.295 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 173.00 MB 2024-04-25T13:44:53.295 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.296 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.296 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 174.00 MB 2024-04-25T13:44:53.296 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.296 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.296 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 175.00 MB 2024-04-25T13:44:53.296 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.296 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.297 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 176.00 MB 2024-04-25T13:44:53.297 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.297 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.297 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 177.00 MB 2024-04-25T13:44:53.297 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.297 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.297 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 178.00 MB 2024-04-25T13:44:53.298 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.298 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.298 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 179.00 MB 2024-04-25T13:44:53.298 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.298 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.298 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 180.00 MB 2024-04-25T13:44:53.299 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.299 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.299 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 181.00 MB 2024-04-25T13:44:53.299 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.299 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.299 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 182.00 MB 2024-04-25T13:44:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 183.00 MB 2024-04-25T13:44:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 184.00 MB 2024-04-25T13:44:53.300 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.301 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.301 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 185.00 MB 2024-04-25T13:44:53.301 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.301 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.301 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 186.00 MB 2024-04-25T13:44:53.301 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.301 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.301 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 187.00 MB 2024-04-25T13:44:53.302 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.302 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.302 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 188.00 MB 2024-04-25T13:44:53.302 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.302 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.302 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 189.00 MB 2024-04-25T13:44:53.302 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.303 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.303 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 190.00 MB 2024-04-25T13:44:53.303 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.303 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.303 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 191.00 MB 2024-04-25T13:44:53.303 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.304 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.304 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 192.00 MB 2024-04-25T13:44:53.304 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.304 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.304 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 193.00 MB 2024-04-25T13:44:53.304 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.305 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.305 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 194.00 MB 2024-04-25T13:44:53.305 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.305 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.305 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 195.00 MB 2024-04-25T13:44:53.305 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.305 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 196.00 MB 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 197.00 MB 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 198.00 MB 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 199.00 MB 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.306 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 200.00 MB 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 201.00 MB 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 202.00 MB 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 203.00 MB 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 204.00 MB 2024-04-25T13:44:53.307 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 205.00 MB 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 206.00 MB 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 207.00 MB 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 208.00 MB 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.308 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 209.00 MB 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 210.00 MB 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 211.00 MB 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 212.00 MB 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 213.00 MB 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.309 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 214.00 MB 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 215.00 MB 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 216.00 MB 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 217.00 MB 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 218.00 MB 2024-04-25T13:44:53.310 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 219.00 MB 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 220.00 MB 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 221.00 MB 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 222.00 MB 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.311 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 223.00 MB 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 224.00 MB 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 225.00 MB 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 226.00 MB 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 227.00 MB 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.312 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 228.00 MB 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 229.00 MB 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 230.00 MB 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 231.00 MB 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 232.00 MB 2024-04-25T13:44:53.313 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 233.00 MB 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 234.00 MB 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 235.00 MB 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 236.00 MB 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.314 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 237.00 MB 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 238.00 MB 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 239.00 MB 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 240.00 MB 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:53.315 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.046 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 241.00 MB 2024-04-25T13:44:54.046 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 242.00 MB 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 243.00 MB 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 244.00 MB 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 245.00 MB 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.047 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 246.00 MB 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 247.00 MB 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 248.00 MB 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 249.00 MB 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 250.00 MB 2024-04-25T13:44:54.048 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 251.00 MB 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 252.00 MB 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 253.00 MB 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 254.00 MB 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 255.00 MB 2024-04-25T13:44:54.049 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 256.00 MB 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 257.00 MB 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 258.00 MB 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 259.00 MB 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.050 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 260.00 MB 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 261.00 MB 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 262.00 MB 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 263.00 MB 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.051 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 264.00 MB 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 265.00 MB 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 266.00 MB 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 267.00 MB 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 268.00 MB 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.052 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 269.00 MB 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 270.00 MB 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 271.00 MB 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 272.00 MB 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 273.00 MB 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.053 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 274.00 MB 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 275.00 MB 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 276.00 MB 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 277.00 MB 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 278.00 MB 2024-04-25T13:44:54.054 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 279.00 MB 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 280.00 MB 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 281.00 MB 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 282.00 MB 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 283.00 MB 2024-04-25T13:44:54.055 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 284.00 MB 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 285.00 MB 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 286.00 MB 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 287.00 MB 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 288.00 MB 2024-04-25T13:44:54.056 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 289.00 MB 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 290.00 MB 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 291.00 MB 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 292.00 MB 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 293.00 MB 2024-04-25T13:44:54.057 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 294.00 MB 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 295.00 MB 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 296.00 MB 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 297.00 MB 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.058 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 298.00 MB 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 299.00 MB 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 300.00 MB 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 301.00 MB 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 302.00 MB 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.059 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 303.00 MB 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 304.00 MB 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 305.00 MB 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 306.00 MB 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 307.00 MB 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.060 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 308.00 MB 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 309.00 MB 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 310.00 MB 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 311.00 MB 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 312.00 MB 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.061 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 313.00 MB 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 314.00 MB 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 315.00 MB 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 316.00 MB 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 317.00 MB 2024-04-25T13:44:54.062 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 318.00 MB 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 319.00 MB 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 320.00 MB 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.063 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 321.00 MB 2024-04-25T13:44:54.822 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 322.00 MB 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 323.00 MB 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 324.00 MB 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.823 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 325.00 MB 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 326.00 MB 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 327.00 MB 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.824 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 328.00 MB 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 329.00 MB 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 330.00 MB 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 331.00 MB 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.825 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 332.00 MB 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 333.00 MB 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 334.00 MB 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.826 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 335.00 MB 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 336.00 MB 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 337.00 MB 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 338.00 MB 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.827 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 339.00 MB 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 340.00 MB 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 341.00 MB 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 342.00 MB 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 343.00 MB 2024-04-25T13:44:54.828 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 344.00 MB 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 345.00 MB 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 346.00 MB 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 347.00 MB 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.829 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 348.00 MB 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 349.00 MB 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 350.00 MB 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 351.00 MB 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 352.00 MB 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.830 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 353.00 MB 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 354.00 MB 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 355.00 MB 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 356.00 MB 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 357.00 MB 2024-04-25T13:44:54.831 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 358.00 MB 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 359.00 MB 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 360.00 MB 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.832 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 361.00 MB 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 362.00 MB 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 363.00 MB 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 364.00 MB 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 365.00 MB 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.833 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 366.00 MB 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 367.00 MB 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 368.00 MB 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 369.00 MB 2024-04-25T13:44:54.834 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 370.00 MB 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 371.00 MB 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 372.00 MB 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 373.00 MB 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.835 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 374.00 MB 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 375.00 MB 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 376.00 MB 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 377.00 MB 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 378.00 MB 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.836 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 379.00 MB 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 380.00 MB 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 381.00 MB 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 382.00 MB 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 383.00 MB 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.837 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 384.00 MB 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 385.00 MB 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 386.00 MB 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 387.00 MB 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 388.00 MB 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.838 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 389.00 MB 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 390.00 MB 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 391.00 MB 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 392.00 MB 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.839 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 393.00 MB 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 394.00 MB 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 395.00 MB 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 396.00 MB 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 397.00 MB 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.840 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 398.00 MB 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 399.00 MB 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 400.00 MB 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:44:54.841 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 401.00 MB 2024-04-25T13:44:57.315 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:44:57.316+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-04-25T13:44:59.083 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:44:59.086 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:45:06.181 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:45:06.184+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2024-04-25T13:45:10.656 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.656 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 402.00 MB 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 403.00 MB 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 404.00 MB 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 405.00 MB 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.657 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 406.00 MB 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 407.00 MB 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 408.00 MB 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 409.00 MB 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 410.00 MB 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.658 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 411.00 MB 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 412.00 MB 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 413.00 MB 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 414.00 MB 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 415.00 MB 2024-04-25T13:45:10.659 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 416.00 MB 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 417.00 MB 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 418.00 MB 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 419.00 MB 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 420.00 MB 2024-04-25T13:45:10.660 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 421.00 MB 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 422.00 MB 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 423.00 MB 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 424.00 MB 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 425.00 MB 2024-04-25T13:45:10.661 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 426.00 MB 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 427.00 MB 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 428.00 MB 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 429.00 MB 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.662 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 430.00 MB 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 431.00 MB 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 432.00 MB 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 433.00 MB 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 434.00 MB 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.663 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 435.00 MB 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 436.00 MB 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 437.00 MB 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 438.00 MB 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 439.00 MB 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.664 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 440.00 MB 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 441.00 MB 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 442.00 MB 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 443.00 MB 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 444.00 MB 2024-04-25T13:45:10.665 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 445.00 MB 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 446.00 MB 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 447.00 MB 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 448.00 MB 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 449.00 MB 2024-04-25T13:45:10.666 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 450.00 MB 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 451.00 MB 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 452.00 MB 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 453.00 MB 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.667 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 454.00 MB 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 455.00 MB 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 456.00 MB 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 457.00 MB 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 458.00 MB 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.668 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 459.00 MB 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 460.00 MB 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 461.00 MB 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 462.00 MB 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 463.00 MB 2024-04-25T13:45:10.669 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 464.00 MB 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 465.00 MB 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 466.00 MB 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 467.00 MB 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.670 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 468.00 MB 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 469.00 MB 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 470.00 MB 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 471.00 MB 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 472.00 MB 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.671 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 473.00 MB 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 474.00 MB 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 475.00 MB 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 476.00 MB 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.672 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 477.00 MB 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 478.00 MB 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 479.00 MB 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:Reached fullness after 480.00 MB 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:calling close 2024-04-25T13:45:10.673 INFO:teuthology.orchestra.run.smithi080.stdout:close() did not raise error 2024-04-25T13:45:10.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:45:10.675 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:45:10.675 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:45:10.714 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:45:10.763 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:45:10.763 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:45:10.764 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:10.764 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:10.769 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:10.769 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:10.818 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:11.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f8a02640 1 Processor -- start 2024-04-25T13:45:11.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f8a02640 1 -- start start 2024-04-25T13:45:11.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f8a02640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f19f40cc720 0x7f19f4073e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f8a02640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40743d0 0x7f19f40cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f8a02640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f19f40ce010 0x7f19f40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f8a02640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f19f415ea50 con 0x7f19f40cc720 2024-04-25T13:45:11.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f8a02640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f19f40ce960 con 0x7f19f40ce010 2024-04-25T13:45:11.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f8a02640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f19f40cd6c0 con 0x7f19f40743d0 2024-04-25T13:45:11.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f19f40cc720 0x7f19f4073e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f19f40cc720 0x7f19f4073e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49316/0 (socket says 172.21.15.80:49316) 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f259c640 1 -- 172.21.15.80:0/1281680010 learned_addr learned my addr 172.21.15.80:0/1281680010 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.019+0000 7f19f1d9b640 1 --2- 172.21.15.80:0/1281680010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40743d0 0x7f19f40cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f2d9d640 1 --2- 172.21.15.80:0/1281680010 >> v2:172.21.15.87:3300/0 conn(0x7f19f40ce010 0x7f19f40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f259c640 1 -- 172.21.15.80:0/1281680010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40743d0 msgr2=0x7f19f40cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f259c640 1 --2- 172.21.15.80:0/1281680010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40743d0 0x7f19f40cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f259c640 1 -- 172.21.15.80:0/1281680010 >> v2:172.21.15.87:3300/0 conn(0x7f19f40ce010 msgr2=0x7f19f40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f259c640 1 --2- 172.21.15.80:0/1281680010 >> v2:172.21.15.87:3300/0 conn(0x7f19f40ce010 0x7f19f40ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f259c640 1 -- 172.21.15.80:0/1281680010 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f40729e0 con 0x7f19f40cc720 2024-04-25T13:45:11.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f259c640 1 --2- 172.21.15.80:0/1281680010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40cc720 0x7f19f4073e90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f19f4077120 tx=0x7f19e801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d354c3fbf6ce041b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f159a640 1 -- 172.21.15.80:0/1281680010 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e80ab620 con 0x7f19f40cc720 2024-04-25T13:45:11.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f159a640 1 -- 172.21.15.80:0/1281680010 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e80b0070 con 0x7f19f40cc720 2024-04-25T13:45:11.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f159a640 1 -- 172.21.15.80:0/1281680010 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e80ab9e0 con 0x7f19f40cc720 2024-04-25T13:45:11.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 -- 172.21.15.80:0/1281680010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40cc720 msgr2=0x7f19f4073e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 --2- 172.21.15.80:0/1281680010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40cc720 0x7f19f4073e90 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f19f4077120 tx=0x7f19e801c460 comp rx=0 tx=0).stop 2024-04-25T13:45:11.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 -- 172.21.15.80:0/1281680010 shutdown_connections 2024-04-25T13:45:11.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 --2- 172.21.15.80:0/1281680010 >> v2:172.21.15.87:3300/0 conn(0x7f19f40ce010 0x7f19f40ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 --2- 172.21.15.80:0/1281680010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40743d0 0x7f19f40cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 --2- 172.21.15.80:0/1281680010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40cc720 0x7f19f4073e90 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 -- 172.21.15.80:0/1281680010 >> 172.21.15.80:0/1281680010 conn(0x7f19f40d20d0 msgr2=0x7f19f40d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:11.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 -- 172.21.15.80:0/1281680010 shutdown_connections 2024-04-25T13:45:11.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.023+0000 7f19f8a02640 1 -- 172.21.15.80:0/1281680010 wait complete. 2024-04-25T13:45:11.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 Processor -- start 2024-04-25T13:45:11.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 -- start start 2024-04-25T13:45:11.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40743d0 0x7f19f40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19f40cc720 0x7f19f409f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40ce010 0x7f19f40ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f19f40ccb30 con 0x7f19f40743d0 2024-04-25T13:45:11.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f19f406e710 con 0x7f19f40cc720 2024-04-25T13:45:11.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f19f40747e0 con 0x7f19f40ce010 2024-04-25T13:45:11.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40743d0 0x7f19f40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40743d0 0x7f19f40b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49320/0 (socket says 172.21.15.80:49320) 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 -- 172.21.15.80:0/325081970 learned_addr learned my addr 172.21.15.80:0/325081970 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f2d9d640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40ce010 0x7f19f40ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f1d9b640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19f40cc720 0x7f19f409f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 -- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40ce010 msgr2=0x7f19f40ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40ce010 0x7f19f40ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 -- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19f40cc720 msgr2=0x7f19f409f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19f40cc720 0x7f19f409f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 -- 172.21.15.80:0/325081970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19f40b7670 con 0x7f19f40743d0 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f259c640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40743d0 0x7f19f40b4cf0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f19e801ab10 tx=0x7f19e801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19e2ffd640 1 -- 172.21.15.80:0/325081970 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e80abda0 con 0x7f19f40743d0 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19e2ffd640 1 -- 172.21.15.80:0/325081970 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e80be3f0 con 0x7f19f40743d0 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19f40b7950 con 0x7f19f40743d0 2024-04-25T13:45:11.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19e2ffd640 1 -- 172.21.15.80:0/325081970 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19e80be7e0 con 0x7f19f40743d0 2024-04-25T13:45:11.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.027+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19f409e8b0 con 0x7f19f40743d0 2024-04-25T13:45:11.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.031+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19f406a3a0 con 0x7f19f40743d0 2024-04-25T13:45:11.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.031+0000 7f19e2ffd640 1 -- 172.21.15.80:0/325081970 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f19e80b43f0 con 0x7f19f40743d0 2024-04-25T13:45:11.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.031+0000 7f19e2ffd640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d406f400 0x7f19d4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.031+0000 7f19e2ffd640 1 -- 172.21.15.80:0/325081970 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f19e8129700 con 0x7f19f40743d0 2024-04-25T13:45:11.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.031+0000 7f19e2ffd640 1 -- 172.21.15.80:0/325081970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7f19d4076fe0 con 0x7f19f40743d0 2024-04-25T13:45:11.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.031+0000 7f19f1d9b640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d406f400 0x7f19d4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.031+0000 7f19f1d9b640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d406f400 0x7f19d4071910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f19f40d2860 tx=0x7f19e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.035+0000 7f19e2ffd640 1 -- 172.21.15.80:0/325081970 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f19e80c4060 con 0x7f19f40743d0 2024-04-25T13:45:11.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.195+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f19f40ce4c0 con 0x7f19f40743d0 2024-04-25T13:45:11.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.195+0000 7f19e2ffd640 1 -- 172.21.15.80:0/325081970 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16639 (secure 0 0 0) 0x7f19e8016110 con 0x7f19f40743d0 2024-04-25T13:45:11.197 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:11.197 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":87,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:01.362921+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:44:33.879562+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T13:44:34.932724+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d406f400 msgr2=0x7f19d4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d406f400 0x7f19d4071910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f19f40d2860 tx=0x7f19e4008040 comp rx=0 tx=0).stop 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40743d0 msgr2=0x7f19f40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40743d0 0x7f19f40b4cf0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f19e801ab10 tx=0x7f19e801aca0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 shutdown_connections 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19d406f400 0x7f19d4071910 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19f40ce010 0x7f19f40ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f19f40cc720 0x7f19f409f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 --2- 172.21.15.80:0/325081970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f19f40743d0 0x7f19f40b4cf0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 >> 172.21.15.80:0/325081970 conn(0x7f19f40d20d0 msgr2=0x7f19f409ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:11.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 shutdown_connections 2024-04-25T13:45:11.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.199+0000 7f19f8a02640 1 -- 172.21.15.80:0/325081970 wait complete. 2024-04-25T13:45:11.219 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:44:33.879562+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:44:34.932724+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:45:11.220 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:45:11.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.355+0000 7fc0f2cee640 1 Processor -- start 2024-04-25T13:45:11.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.355+0000 7fc0f2cee640 1 -- start start 2024-04-25T13:45:11.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f2cee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ec071e60 0x7fc0ec1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f2cee640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ec06fbf0 con 0x7fc0ec073c40 2024-04-25T13:45:11.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f2cee640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc0ec0b0a20 con 0x7fc0ec161170 2024-04-25T13:45:11.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f2cee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec0b1350 con 0x7fc0ec071e60 2024-04-25T13:45:11.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e3fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc0ec073c40 0x7fc0ec160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38464/0 (socket says 172.21.15.80:38464) 2024-04-25T13:45:11.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e3fff640 1 -- 172.21.15.80:0/177496429 learned_addr learned my addr 172.21.15.80:0/177496429 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:11.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 788070476 0 0) 0x7fc0ec06fbf0 con 0x7fc0ec073c40 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f0a8a640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ec071e60 0x7fc0ec1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc0d4003530 con 0x7fc0ec073c40 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1007263984 0 0) 0x7fc0d4003530 con 0x7fc0ec073c40 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ec071e60 msgr2=0x7fc0ec1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ec071e60 0x7fc0ec1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 >> v1:172.21.15.87:6789/0 conn(0x7fc0ec161170 legacy=0x7fc0ec0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0b0d00 con 0x7fc0ec073c40 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc0dc001680 con 0x7fc0ec073c40 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 >> v1:172.21.15.80:6789/0 conn(0x7fc0ec073c40 legacy=0x7fc0ec160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0d40039f0 0x7fc0d4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ec071e60 0x7fc0d4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0d4004d50 0x7fc0d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec0b1350 con 0x7fc0d40039f0 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d400ff70 con 0x7fc0d4004d50 2024-04-25T13:45:11.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d4010120 con 0x7fc0ec071e60 2024-04-25T13:45:11.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f0a8a640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0d40039f0 0x7fc0d4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f128b640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0d4004d50 0x7fc0d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f0a8a640 1 -- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ec071e60 msgr2=0x7fc0d4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:11.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f0a8a640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0ec071e60 0x7fc0d4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f0a8a640 1 -- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0d4004d50 msgr2=0x7fc0d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:11.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f0a8a640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0d4004d50 0x7fc0d4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f0a8a640 1 -- 172.21.15.80:0/177496429 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d4003530 con 0x7fc0d40039f0 2024-04-25T13:45:11.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.359+0000 7fc0f128b640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0d4004d50 0x7fc0d4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:11.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f0a8a640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0d40039f0 0x7fc0d4003e80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8006ae0 tx=0x7fc0e801cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f702a0d53f138db9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e80ab070 con 0x7fc0d40039f0 2024-04-25T13:45:11.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e80a5ac0 con 0x7fc0d40039f0 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0e37fe640 1 -- 172.21.15.80:0/177496429 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800bd60 con 0x7fc0d40039f0 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f2cee640 1 -- 172.21.15.80:0/177496429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0d40039f0 msgr2=0x7fc0d4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f2cee640 1 --2- 172.21.15.80:0/177496429 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0d40039f0 0x7fc0d4003e80 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8006ae0 tx=0x7fc0e801cd20 comp rx=0 tx=0).stop 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f128b640 1 -- 172.21.15.80:0/177496429 reap_dead start 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f2cee640 1 -- 172.21.15.80:0/177496429 shutdown_connections 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f2cee640 1 -- 172.21.15.80:0/177496429 >> 172.21.15.80:0/177496429 conn(0x7fc0ec0b4160 msgr2=0x7fc0ec0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f2cee640 1 -- 172.21.15.80:0/177496429 shutdown_connections 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f2cee640 1 -- 172.21.15.80:0/177496429 wait complete. 2024-04-25T13:45:11.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f2cee640 1 Processor -- start 2024-04-25T13:45:11.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.363+0000 7fc0f2cee640 1 -- start start 2024-04-25T13:45:11.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0d40039f0 0x7fc0ec0ada70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ec072e20 0x7fc0ec0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ec0b00f0 0x7fc0ec1f7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec0b56c0 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ec074050 con 0x7fc0ec072e20 2024-04-25T13:45:11.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc0ec1f6a60 con 0x7fc0d40039f0 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ec0b00f0 0x7fc0ec1f7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ec0b00f0 0x7fc0ec1f7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49996/0 (socket says 172.21.15.80:49996) 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 -- 172.21.15.80:0/3961115904 learned_addr learned my addr 172.21.15.80:0/3961115904 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f0a8a640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0d40039f0 0x7fc0ec0ada70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0e3fff640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ec072e20 0x7fc0ec0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 -- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0d40039f0 msgr2=0x7fc0ec0ada70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0d40039f0 0x7fc0ec0ada70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 -- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ec072e20 msgr2=0x7fc0ec0afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ec072e20 0x7fc0ec0afbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 -- 172.21.15.80:0/3961115904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ec0ae370 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0e3fff640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ec072e20 0x7fc0ec0afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:11.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f128b640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ec0b00f0 0x7fc0ec1f7e30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0b12c0 tx=0x7fc0e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0e17fa640 1 -- 172.21.15.80:0/3961115904 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4006d60 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0e17fa640 1 -- 172.21.15.80:0/3961115904 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0e4003980 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ec0ae650 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0e17fa640 1 -- 172.21.15.80:0/3961115904 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0e4003d70 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ec0aec10 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0ec06a3a0 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.367+0000 7fc0e17fa640 1 -- 172.21.15.80:0/3961115904 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc0e401b020 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.371+0000 7fc0e17fa640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc0d806f400 0x7fc0d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.371+0000 7fc0e17fa640 1 -- 172.21.15.80:0/3961115904 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fc0e408ed90 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.371+0000 7fc0e17fa640 1 -- 172.21.15.80:0/3961115904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7fc0d8076f80 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.371+0000 7fc0f0a8a640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc0d806f400 0x7fc0d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.371+0000 7fc0f0a8a640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc0d806f400 0x7fc0d8071910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80b0000 tx=0x7fc0e80046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.375+0000 7fc0e17fa640 1 -- 172.21.15.80:0/3961115904 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc0e405bda0 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.531+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc0ec0ae100 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.531+0000 7fc0e17fa640 1 -- 172.21.15.80:0/3961115904 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fc0e405b6a0 con 0x7fc0ec0b00f0 2024-04-25T13:45:11.533 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:45:11.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc0d806f400 msgr2=0x7fc0d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc0d806f400 0x7fc0d8071910 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc0e80b0000 tx=0x7fc0e80046d0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ec0b00f0 msgr2=0x7fc0ec1f7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ec0b00f0 0x7fc0ec1f7e30 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fc0ec0b12c0 tx=0x7fc0e400dc00 comp rx=0 tx=0).stop 2024-04-25T13:45:11.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 shutdown_connections 2024-04-25T13:45:11.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc0d806f400 0x7fc0d8071910 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc0ec0b00f0 0x7fc0ec1f7e30 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc0ec072e20 0x7fc0ec0afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 --2- 172.21.15.80:0/3961115904 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc0d40039f0 0x7fc0ec0ada70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 >> 172.21.15.80:0/3961115904 conn(0x7fc0ec0b4160 msgr2=0x7fc0ec0d0f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:11.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 shutdown_connections 2024-04-25T13:45:11.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.535+0000 7fc0f2cee640 1 -- 172.21.15.80:0/3961115904 wait complete. 2024-04-25T13:45:11.554 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:45:11.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff18611a640 1 Processor -- start 2024-04-25T13:45:11.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff18611a640 1 -- start start 2024-04-25T13:45:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff18611a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1800b5330 0x7ff1800d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff18611a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1800d0ff0 0x7ff18006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff18611a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1800b5740 con 0x7ff18006f2f0 2024-04-25T13:45:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff18611a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff18006f700 con 0x7ff1800b5330 2024-04-25T13:45:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff18611a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff1801629b0 con 0x7ff1800d0ff0 2024-04-25T13:45:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1800b5330 0x7ff1800d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff18006f2f0 0x7ff180161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38472/0 (socket says 172.21.15.80:38472) 2024-04-25T13:45:11.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17f7fe640 1 --2- 172.21.15.80:0/377112290 >> v2:172.21.15.80:3300/0 conn(0x7ff1800b5330 0x7ff1800d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49998/0 (socket says 172.21.15.80:49998) 2024-04-25T13:45:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17ffff640 1 -- 172.21.15.80:0/377112290 learned_addr learned my addr 172.21.15.80:0/377112290 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17effd640 1 --2- 172.21.15.80:0/377112290 >> v2:172.21.15.87:3300/0 conn(0x7ff1800d0ff0 0x7ff18006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17f7fe640 1 -- 172.21.15.80:0/377112290 >> v2:172.21.15.87:3300/0 conn(0x7ff1800d0ff0 msgr2=0x7ff18006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17f7fe640 1 --2- 172.21.15.80:0/377112290 >> v2:172.21.15.87:3300/0 conn(0x7ff1800d0ff0 0x7ff18006edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17f7fe640 1 -- 172.21.15.80:0/377112290 >> v1:172.21.15.80:6789/0 conn(0x7ff18006f2f0 legacy=0x7ff180161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17e7fc640 1 -- 172.21.15.80:0/377112290 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 122475384 0 0) 0x7ff1800b5740 con 0x7ff18006f2f0 2024-04-25T13:45:11.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.695+0000 7ff17f7fe640 1 -- 172.21.15.80:0/377112290 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff18006dd10 con 0x7ff1800b5330 2024-04-25T13:45:11.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff17f7fe640 1 --2- 172.21.15.80:0/377112290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800b5330 0x7ff1800d0ab0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff180074290 tx=0x7ff17801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51bfb024b8bef40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff17e7fc640 1 -- 172.21.15.80:0/377112290 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1780b0070 con 0x7ff1800b5330 2024-04-25T13:45:11.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff17e7fc640 1 -- 172.21.15.80:0/377112290 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1780ab600 con 0x7ff1800b5330 2024-04-25T13:45:11.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff17e7fc640 1 -- 172.21.15.80:0/377112290 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1780ab9d0 con 0x7ff1800b5330 2024-04-25T13:45:11.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff18611a640 1 -- 172.21.15.80:0/377112290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800b5330 msgr2=0x7ff1800d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff18611a640 1 --2- 172.21.15.80:0/377112290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800b5330 0x7ff1800d0ab0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7ff180074290 tx=0x7ff17801c3f0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff18611a640 1 -- 172.21.15.80:0/377112290 shutdown_connections 2024-04-25T13:45:11.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff18611a640 1 --2- 172.21.15.80:0/377112290 >> v2:172.21.15.87:3300/0 conn(0x7ff1800d0ff0 0x7ff18006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff18611a640 1 --2- 172.21.15.80:0/377112290 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800b5330 0x7ff1800d0ab0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff18611a640 1 -- 172.21.15.80:0/377112290 >> 172.21.15.80:0/377112290 conn(0x7ff1800b4160 msgr2=0x7ff1800d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:11.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff18611a640 1 -- 172.21.15.80:0/377112290 shutdown_connections 2024-04-25T13:45:11.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.699+0000 7ff18611a640 1 -- 172.21.15.80:0/377112290 wait complete. 2024-04-25T13:45:11.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 Processor -- start 2024-04-25T13:45:11.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 -- start start 2024-04-25T13:45:11.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff18006f2f0 0x7ff1801438b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1800b5330 0x7ff18015e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800d0ff0 0x7ff18015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff1800b0cd0 con 0x7ff1800d0ff0 2024-04-25T13:45:11.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff18006e890 con 0x7ff1800b5330 2024-04-25T13:45:11.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff1800b0e50 con 0x7ff18006f2f0 2024-04-25T13:45:11.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800d0ff0 0x7ff18015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800d0ff0 0x7ff18015ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50008/0 (socket says 172.21.15.80:50008) 2024-04-25T13:45:11.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 -- 172.21.15.80:0/1837225679 learned_addr learned my addr 172.21.15.80:0/1837225679 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:11.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17effd640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1800b5330 0x7ff18015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17f7fe640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff18006f2f0 0x7ff1801438b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 -- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff18006f2f0 msgr2=0x7ff1801438b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff18006f2f0 0x7ff1801438b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 -- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1800b5330 msgr2=0x7ff18015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1800b5330 0x7ff18015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 -- 172.21.15.80:0/1837225679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1800b0480 con 0x7ff1800d0ff0 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff17ffff640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800d0ff0 0x7ff18015ee40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff17000c730 tx=0x7ff17000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff167fff640 1 -- 172.21.15.80:0/1837225679 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff170018070 con 0x7ff1800d0ff0 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff167fff640 1 -- 172.21.15.80:0/1837225679 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff17000ce40 con 0x7ff1800d0ff0 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff167fff640 1 -- 172.21.15.80:0/1837225679 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff17000a860 con 0x7ff1800d0ff0 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1801448d0 con 0x7ff1800d0ff0 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff18015b700 con 0x7ff1800d0ff0 2024-04-25T13:45:11.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.703+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff18006a3a0 con 0x7ff1800d0ff0 2024-04-25T13:45:11.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.707+0000 7ff167fff640 1 -- 172.21.15.80:0/1837225679 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff17001b070 con 0x7ff1800d0ff0 2024-04-25T13:45:11.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.707+0000 7ff167fff640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16c06f400 0x7ff16c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:11.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.707+0000 7ff167fff640 1 -- 172.21.15.80:0/1837225679 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7ff17008e7e0 con 0x7ff1800d0ff0 2024-04-25T13:45:11.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.707+0000 7ff167fff640 1 -- 172.21.15.80:0/1837225679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7ff16c076f80 con 0x7ff1800d0ff0 2024-04-25T13:45:11.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.707+0000 7ff17f7fe640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16c06f400 0x7ff16c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:11.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.707+0000 7ff17f7fe640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16c06f400 0x7ff16c071910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff1780ad000 tx=0x7ff178008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:11.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.711+0000 7ff167fff640 1 -- 172.21.15.80:0/1837225679 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff17005b760 con 0x7ff1800d0ff0 2024-04-25T13:45:11.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.863+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff18006fd00 con 0x7ff1800d0ff0 2024-04-25T13:45:11.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff167fff640 1 -- 172.21.15.80:0/1837225679 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7ff17005b060 con 0x7ff1800d0ff0 2024-04-25T13:45:11.866 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16c06f400 msgr2=0x7ff16c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16c06f400 0x7ff16c071910 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff1780ad000 tx=0x7ff178008040 comp rx=0 tx=0).stop 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800d0ff0 msgr2=0x7ff18015ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800d0ff0 0x7ff18015ee40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff17000c730 tx=0x7ff17000cbf0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 shutdown_connections 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff16c06f400 0x7ff16c071910 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1800d0ff0 0x7ff18015ee40 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1800b5330 0x7ff18015e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 --2- 172.21.15.80:0/1837225679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff18006f2f0 0x7ff1801438b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 >> 172.21.15.80:0/1837225679 conn(0x7ff1800b4160 msgr2=0x7ff1801474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 shutdown_connections 2024-04-25T13:45:11.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:11.867+0000 7ff18611a640 1 -- 172.21.15.80:0/1837225679 wait complete. 2024-04-25T13:45:11.886 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:45:12.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f297761f640 1 Processor -- start 2024-04-25T13:45:12.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f297761f640 1 -- start start 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f297761f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f297006edb0 0x7f297006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f297761f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2970161b60 0x7f2970161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f297761f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29701624b0 con 0x7f29700b5330 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f297761f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f297006f700 con 0x7f2970161b60 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f297761f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f29701629b0 con 0x7f297006edb0 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f29753bb640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f29700b5330 0x7f29700d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38488/0 (socket says 172.21.15.80:38488) 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f29753bb640 1 -- 172.21.15.80:0/4220978156 learned_addr learned my addr 172.21.15.80:0/4220978156 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1544045093 0 0) 0x7f296c001120 con 0x7f29700b5330 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f2974bba640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f297006edb0 0x7f297006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f295c003530 con 0x7f29700b5330 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.031+0000 7f2975bbc640 1 --2- 172.21.15.80:0/4220978156 >> v2:172.21.15.87:3300/0 conn(0x7f2970161b60 0x7f2970161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2266712894 0 0) 0x7f295c003530 con 0x7f29700b5330 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f297006edb0 msgr2=0x7f297006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f297006edb0 0x7f297006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 >> v2:172.21.15.87:3300/0 conn(0x7f2970161b60 msgr2=0x7f2970161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 --2- 172.21.15.80:0/4220978156 >> v2:172.21.15.87:3300/0 conn(0x7f2970161b60 0x7f2970161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29700b5740 con 0x7f29700b5330 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f296c001850 con 0x7f29700b5330 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 --2- 172.21.15.80:0/4220978156 >> v2:172.21.15.87:3300/0 conn(0x7f2970161b60 0x7f2970161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 >> v1:172.21.15.80:6789/0 conn(0x7f29700b5330 legacy=0x7f29700d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f297006edb0 0x7f295c0039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f295c003f40 0x7f295c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c004d50 0x7f295c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f297006f700 con 0x7f295c004d50 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f29701629b0 con 0x7f295c003f40 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f295c00bb60 con 0x7f297006edb0 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c004d50 0x7f295c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2974bba640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f295c003f40 0x7f295c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 -- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f297006edb0 msgr2=0x7f295c0039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f297006edb0 0x7f295c0039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 -- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f295c003f40 msgr2=0x7f295c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f295c003f40 0x7f295c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 -- 172.21.15.80:0/4220978156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295c003530 con 0x7f295c004d50 2024-04-25T13:45:12.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c004d50 0x7f295c005160 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f29680073b0 tx=0x7f296801ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05b8fc3950bff39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29680b2070 con 0x7f295c004d50 2024-04-25T13:45:12.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29680a5d60 con 0x7f295c004d50 2024-04-25T13:45:12.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2967fff640 1 -- 172.21.15.80:0/4220978156 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2968009c80 con 0x7f295c004d50 2024-04-25T13:45:12.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f297761f640 1 -- 172.21.15.80:0/4220978156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c004d50 msgr2=0x7f295c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f297761f640 1 --2- 172.21.15.80:0/4220978156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f295c004d50 0x7f295c005160 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f29680073b0 tx=0x7f296801ce60 comp rx=0 tx=0).stop 2024-04-25T13:45:12.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f2975bbc640 1 -- 172.21.15.80:0/4220978156 reap_dead start 2024-04-25T13:45:12.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f297761f640 1 -- 172.21.15.80:0/4220978156 shutdown_connections 2024-04-25T13:45:12.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f297761f640 1 -- 172.21.15.80:0/4220978156 >> 172.21.15.80:0/4220978156 conn(0x7f29700b4160 msgr2=0x7f29700d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:12.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f297761f640 1 -- 172.21.15.80:0/4220978156 shutdown_connections 2024-04-25T13:45:12.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.035+0000 7f297761f640 1 -- 172.21.15.80:0/4220978156 wait complete. 2024-04-25T13:45:12.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 Processor -- start 2024-04-25T13:45:12.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 -- start start 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29701416e0 0x7f297013e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29701445d0 0x7f29701449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29701f46d0 0x7f29701f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f297006e2d0 con 0x7f29701416e0 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f297006dff0 con 0x7f29701f46d0 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f297006e5b0 con 0x7f29701445d0 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29701416e0 0x7f297013e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29701416e0 0x7f297013e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50024/0 (socket says 172.21.15.80:50024) 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 -- 172.21.15.80:0/495894094 learned_addr learned my addr 172.21.15.80:0/495894094 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 -- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29701445d0 msgr2=0x7f29701449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f2974bba640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29701445d0 0x7f29701449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29701445d0 0x7f29701449e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 -- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29701f46d0 msgr2=0x7f29701f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29701f46d0 0x7f29701f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 -- 172.21.15.80:0/495894094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2970141af0 con 0x7f29701416e0 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f29753bb640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29701416e0 0x7f297013e850 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f296c00f3a0 tx=0x7f296c00f860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f2965ffb640 1 -- 172.21.15.80:0/495894094 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c007d00 con 0x7f29701416e0 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f2965ffb640 1 -- 172.21.15.80:0/495894094 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f296c00a0d0 con 0x7f29701416e0 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f297013efc0 con 0x7f29701416e0 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f2965ffb640 1 -- 172.21.15.80:0/495894094 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f296c00a500 con 0x7f29701416e0 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.039+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f297013f580 con 0x7f29701416e0 2024-04-25T13:45:12.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.043+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f297006a3a0 con 0x7f29701416e0 2024-04-25T13:45:12.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.043+0000 7f2965ffb640 1 -- 172.21.15.80:0/495894094 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f296c014070 con 0x7f29701416e0 2024-04-25T13:45:12.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.043+0000 7f2965ffb640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f295406f400 0x7f2954071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.043+0000 7f2965ffb640 1 -- 172.21.15.80:0/495894094 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f296c091c20 con 0x7f29701416e0 2024-04-25T13:45:12.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.043+0000 7f2965ffb640 1 -- 172.21.15.80:0/495894094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7f2954076fe0 con 0x7f29701416e0 2024-04-25T13:45:12.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.043+0000 7f2974bba640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f295406f400 0x7f2954071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.043+0000 7f2974bba640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f295406f400 0x7f2954071910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f29600098d0 tx=0x7f2960008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.047+0000 7f2965ffb640 1 -- 172.21.15.80:0/495894094 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29701624b0 con 0x7f29701416e0 2024-04-25T13:45:12.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.195+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f297006f460 con 0x7f29701416e0 2024-04-25T13:45:12.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.195+0000 7f2965ffb640 1 -- 172.21.15.80:0/495894094 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f296c05ec30 con 0x7f29701416e0 2024-04-25T13:45:12.194 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:45:12.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.195+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f295406f400 msgr2=0x7f2954071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.195+0000 7f297761f640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f295406f400 0x7f2954071910 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f29600098d0 tx=0x7f2960008040 comp rx=0 tx=0).stop 2024-04-25T13:45:12.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.195+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29701416e0 msgr2=0x7f297013e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.195+0000 7f297761f640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29701416e0 0x7f297013e850 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f296c00f3a0 tx=0x7f296c00f860 comp rx=0 tx=0).stop 2024-04-25T13:45:12.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.199+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 shutdown_connections 2024-04-25T13:45:12.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.199+0000 7f297761f640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f295406f400 0x7f2954071910 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.199+0000 7f297761f640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29701f46d0 0x7f29701f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.199+0000 7f297761f640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29701445d0 0x7f29701449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.199+0000 7f297761f640 1 --2- 172.21.15.80:0/495894094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29701416e0 0x7f297013e850 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.199+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 >> 172.21.15.80:0/495894094 conn(0x7f29700b4160 msgr2=0x7f29700b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:12.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.199+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 shutdown_connections 2024-04-25T13:45:12.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.199+0000 7f297761f640 1 -- 172.21.15.80:0/495894094 wait complete. 2024-04-25T13:45:12.215 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:45:12.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.347+0000 7f05035af640 1 Processor -- start 2024-04-25T13:45:12.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.347+0000 7f05035af640 1 -- start start 2024-04-25T13:45:12.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f05035af640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04fc15e080 0x7f04fc160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f05035af640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04fc161420 con 0x7f04fc0701f0 2024-04-25T13:45:12.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f05035af640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04fc0b1f80 con 0x7f04fc161010 2024-04-25T13:45:12.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f05035af640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f04fc0b2260 con 0x7f04fc15e080 2024-04-25T13:45:12.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f0500b4a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f04fc15e080 0x7f04fc160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f050134b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f04fc0701f0 0x7f04fc0b6400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38504/0 (socket says 172.21.15.80:38504) 2024-04-25T13:45:12.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f050134b640 1 -- 172.21.15.80:0/2354184291 learned_addr learned my addr 172.21.15.80:0/2354184291 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:12.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f0500b4a640 1 -- 172.21.15.80:0/2354184291 >> v1:172.21.15.87:6789/0 conn(0x7f04fc161010 legacy=0x7f04fc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:12.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f0500b4a640 1 -- 172.21.15.80:0/2354184291 >> v1:172.21.15.80:6789/0 conn(0x7f04fc0701f0 legacy=0x7f04fc0b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f0500b4a640 1 -- 172.21.15.80:0/2354184291 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc0b5920 con 0x7f04fc15e080 2024-04-25T13:45:12.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f0500b4a640 1 --2- 172.21.15.80:0/2354184291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc15e080 0x7f04fc160ad0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0097c0 tx=0x7f04ec01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88405b4fd1566ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f04f3fff640 1 -- 172.21.15.80:0/2354184291 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0ab5f0 con 0x7f04fc15e080 2024-04-25T13:45:12.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f04f3fff640 1 -- 172.21.15.80:0/2354184291 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04ec0b0070 con 0x7f04fc15e080 2024-04-25T13:45:12.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.351+0000 7f04f3fff640 1 -- 172.21.15.80:0/2354184291 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04ec0ab9b0 con 0x7f04fc15e080 2024-04-25T13:45:12.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- 172.21.15.80:0/2354184291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc15e080 msgr2=0x7f04fc160ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 --2- 172.21.15.80:0/2354184291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc15e080 0x7f04fc160ad0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0097c0 tx=0x7f04ec01c3d0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- 172.21.15.80:0/2354184291 shutdown_connections 2024-04-25T13:45:12.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 --2- 172.21.15.80:0/2354184291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc15e080 0x7f04fc160ad0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- 172.21.15.80:0/2354184291 >> 172.21.15.80:0/2354184291 conn(0x7f04fc0d2ad0 msgr2=0x7f04fc0d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:12.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- 172.21.15.80:0/2354184291 shutdown_connections 2024-04-25T13:45:12.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- 172.21.15.80:0/2354184291 wait complete. 2024-04-25T13:45:12.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 Processor -- start 2024-04-25T13:45:12.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- start start 2024-04-25T13:45:12.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc0701f0 0x7f04fc1458d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc15e080 0x7f04fc1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc161010 0x7f04fc0b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc070780 con 0x7f04fc0701f0 2024-04-25T13:45:12.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04fc070600 con 0x7f04fc161010 2024-04-25T13:45:12.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f05035af640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f04fc16b690 con 0x7f04fc15e080 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f050134b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc0701f0 0x7f04fc1458d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f050134b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc0701f0 0x7f04fc1458d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50054/0 (socket says 172.21.15.80:50054) 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f050134b640 1 -- 172.21.15.80:0/2472800891 learned_addr learned my addr 172.21.15.80:0/2472800891 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f0500b4a640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc15e080 0x7f04fc1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f0501b4c640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc161010 0x7f04fc0b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f050134b640 1 -- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc15e080 msgr2=0x7f04fc1568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f050134b640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc15e080 0x7f04fc1568a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f050134b640 1 -- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc161010 msgr2=0x7f04fc0b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f050134b640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc161010 0x7f04fc0b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f050134b640 1 -- 172.21.15.80:0/2472800891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04fc154bc0 con 0x7f04fc0701f0 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f0501b4c640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc161010 0x7f04fc0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.355+0000 7f0500b4a640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc15e080 0x7f04fc1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f050134b640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc0701f0 0x7f04fc1458d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c370 tx=0x7f04f800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f04f1ffb640 1 -- 172.21.15.80:0/2472800891 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8017070 con 0x7f04fc0701f0 2024-04-25T13:45:12.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f04f1ffb640 1 -- 172.21.15.80:0/2472800891 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04f8005cf0 con 0x7f04fc0701f0 2024-04-25T13:45:12.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04fc1fdd90 con 0x7f04fc0701f0 2024-04-25T13:45:12.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f04f1ffb640 1 -- 172.21.15.80:0/2472800891 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f80072f0 con 0x7f04fc0701f0 2024-04-25T13:45:12.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04fc0afd80 con 0x7f04fc0701f0 2024-04-25T13:45:12.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04fc06a3a0 con 0x7f04fc0701f0 2024-04-25T13:45:12.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f04f1ffb640 1 -- 172.21.15.80:0/2472800891 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f04f801a070 con 0x7f04fc0701f0 2024-04-25T13:45:12.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f04f1ffb640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f400 0x7f04e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f04f1ffb640 1 -- 172.21.15.80:0/2472800891 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f04f808e5a0 con 0x7f04fc0701f0 2024-04-25T13:45:12.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f04f1ffb640 1 -- 172.21.15.80:0/2472800891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7f04e0076fe0 con 0x7f04fc0701f0 2024-04-25T13:45:12.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f0500b4a640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f400 0x7f04e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.359+0000 7f0500b4a640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f400 0x7f04e0071910 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0af000 tx=0x7f04ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.363+0000 7f04f1ffb640 1 -- 172.21.15.80:0/2472800891 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f04fc161420 con 0x7f04fc0701f0 2024-04-25T13:45:12.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.523+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f04fc161950 con 0x7f04fc0701f0 2024-04-25T13:45:12.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.523+0000 7f04f1ffb640 1 -- 172.21.15.80:0/2472800891 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) ==== 80+0+0 (secure 0 0 0) 0x7f04f805b520 con 0x7f04fc0701f0 2024-04-25T13:45:12.525 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f400 msgr2=0x7f04e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f400 0x7f04e0071910 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f04ec0af000 tx=0x7f04ec008040 comp rx=0 tx=0).stop 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc0701f0 msgr2=0x7f04fc1458d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc0701f0 0x7f04fc1458d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c370 tx=0x7f04f800c830 comp rx=0 tx=0).stop 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 shutdown_connections 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f04e006f400 0x7f04e0071910 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f04fc161010 0x7f04fc0b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04fc15e080 0x7f04fc1568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 --2- 172.21.15.80:0/2472800891 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04fc0701f0 0x7f04fc1458d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 >> 172.21.15.80:0/2472800891 conn(0x7f04fc0d2ad0 msgr2=0x7f04fc0d2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:12.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 shutdown_connections 2024-04-25T13:45:12.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.527+0000 7f05035af640 1 -- 172.21.15.80:0/2472800891 wait complete. 2024-04-25T13:45:12.545 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:45:12.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:12.545 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:45:12.557 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[33451]: fuse finished with error 0 and tester_r 0 2024-04-25T13:45:12.558 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:45:12.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:12.558 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:45:12.676 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:45:12.677 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:45:12.677 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:12.687 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:45:12.687 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:12.736 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:12.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268ed23640 1 Processor -- start 2024-04-25T13:45:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268ed23640 1 -- start start 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268ed23640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26881605b0 0x7f268806f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268ed23640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f268806fc30 0x7f26880d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268ed23640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26880cd210 con 0x7f26880d11c0 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268ed23640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f26880cc720 con 0x7f26881605b0 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268ed23640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2688070040 con 0x7f268806fc30 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f2687fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26881605b0 0x7f268806f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268cabf640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f26880d11c0 0x7f2688072e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38516/0 (socket says 172.21.15.80:38516) 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f2687fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26881605b0 0x7f268806f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50062/0 (socket says 172.21.15.80:50062) 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.891+0000 7f268cabf640 1 -- 172.21.15.80:0/3111357002 learned_addr learned my addr 172.21.15.80:0/3111357002 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268d2c0640 1 --2- 172.21.15.80:0/3111357002 >> v2:172.21.15.87:3300/0 conn(0x7f268806fc30 0x7f26880d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f2687fff640 1 -- 172.21.15.80:0/3111357002 >> v2:172.21.15.87:3300/0 conn(0x7f268806fc30 msgr2=0x7f26880d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:12.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f2687fff640 1 --2- 172.21.15.80:0/3111357002 >> v2:172.21.15.87:3300/0 conn(0x7f268806fc30 0x7f26880d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f2687fff640 1 -- 172.21.15.80:0/3111357002 >> v1:172.21.15.80:6789/0 conn(0x7f26880d11c0 legacy=0x7f2688072e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f2687fff640 1 -- 172.21.15.80:0/3111357002 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26880a0030 con 0x7f26881605b0 2024-04-25T13:45:12.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268d2c0640 1 --2- 172.21.15.80:0/3111357002 >> v2:172.21.15.87:3300/0 conn(0x7f268806fc30 0x7f26880d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f2687fff640 1 --2- 172.21.15.80:0/3111357002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268806f6f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f26880d2080 tx=0x7f267001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=669481ada54af61e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f26877fe640 1 -- 172.21.15.80:0/3111357002 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26700b0070 con 0x7f26881605b0 2024-04-25T13:45:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f26877fe640 1 -- 172.21.15.80:0/3111357002 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26700ab600 con 0x7f26881605b0 2024-04-25T13:45:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f26877fe640 1 -- 172.21.15.80:0/3111357002 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26700ab9d0 con 0x7f26881605b0 2024-04-25T13:45:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268ed23640 1 -- 172.21.15.80:0/3111357002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 msgr2=0x7f268806f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268ed23640 1 --2- 172.21.15.80:0/3111357002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268806f6f0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f26880d2080 tx=0x7f267001c3c0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268ed23640 1 -- 172.21.15.80:0/3111357002 shutdown_connections 2024-04-25T13:45:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268ed23640 1 --2- 172.21.15.80:0/3111357002 >> v2:172.21.15.87:3300/0 conn(0x7f268806fc30 0x7f26880d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268ed23640 1 --2- 172.21.15.80:0/3111357002 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268806f6f0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268ed23640 1 -- 172.21.15.80:0/3111357002 >> 172.21.15.80:0/3111357002 conn(0x7f26880d20d0 msgr2=0x7f26880d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268ed23640 1 -- 172.21.15.80:0/3111357002 shutdown_connections 2024-04-25T13:45:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.895+0000 7f268ed23640 1 -- 172.21.15.80:0/3111357002 wait complete. 2024-04-25T13:45:12.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268ed23640 1 Processor -- start 2024-04-25T13:45:12.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268ed23640 1 -- start start 2024-04-25T13:45:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268ed23640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268806fc30 0x7f268809f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268ed23640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26880d11c0 0x7f26881483c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268ed23640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268815c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268ed23640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26881609c0 con 0x7f26881605b0 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268ed23640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f26880cfdd0 con 0x7f26880d11c0 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268ed23640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2688160b40 con 0x7f268806fc30 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268d2c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268815c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268d2c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268815c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50070/0 (socket says 172.21.15.80:50070) 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268d2c0640 1 -- 172.21.15.80:0/1440414749 learned_addr learned my addr 172.21.15.80:0/1440414749 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268cabf640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268806fc30 0x7f268809f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f2687fff640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26880d11c0 0x7f26881483c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268d2c0640 1 -- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268806fc30 msgr2=0x7f268809f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268d2c0640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268806fc30 0x7f268809f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268d2c0640 1 -- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26880d11c0 msgr2=0x7f26881483c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268d2c0640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26880d11c0 0x7f26881483c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.899+0000 7f268d2c0640 1 -- 172.21.15.80:0/1440414749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268809e910 con 0x7f26881605b0 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f268d2c0640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268815c1c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f268000c820 tx=0x7f268000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f26857fa640 1 -- 172.21.15.80:0/1440414749 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680015020 con 0x7f26881605b0 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f26857fa640 1 -- 172.21.15.80:0/1440414749 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2680004240 con 0x7f26881605b0 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268809ebf0 con 0x7f26881605b0 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f26857fa640 1 -- 172.21.15.80:0/1440414749 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2680003620 con 0x7f26881605b0 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f268809f1b0 con 0x7f26881605b0 2024-04-25T13:45:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f268806a3a0 con 0x7f26881605b0 2024-04-25T13:45:12.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f26857fa640 1 -- 172.21.15.80:0/1440414749 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2680018020 con 0x7f26881605b0 2024-04-25T13:45:12.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f26857fa640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f266806f400 0x7f2668071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:12.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f26857fa640 1 -- 172.21.15.80:0/1440414749 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f268008f340 con 0x7f26881605b0 2024-04-25T13:45:12.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f26857fa640 1 -- 172.21.15.80:0/1440414749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7f2668076f80 con 0x7f26881605b0 2024-04-25T13:45:12.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f268cabf640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f266806f400 0x7f2668071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:12.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.903+0000 7f268cabf640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f266806f400 0x7f2668071910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f26780012f0 tx=0x7f2678009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:12.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:12.907+0000 7f26857fa640 1 -- 172.21.15.80:0/1440414749 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f268001f020 con 0x7f26881605b0 2024-04-25T13:45:13.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.075+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2688076b20 con 0x7f26881605b0 2024-04-25T13:45:13.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f26857fa640 1 -- 172.21.15.80:0/1440414749 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) ==== 74+0+16639 (secure 0 0 0) 0x7f268005c230 con 0x7f26881605b0 2024-04-25T13:45:13.078 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:13.078 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":87,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:01.362921+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:44:33.879562+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T13:44:34.932724+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f266806f400 msgr2=0x7f2668071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f266806f400 0x7f2668071910 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f26780012f0 tx=0x7f2678009040 comp rx=0 tx=0).stop 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 msgr2=0x7f268815c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268815c1c0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f268000c820 tx=0x7f268000cce0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 shutdown_connections 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f266806f400 0x7f2668071910 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26881605b0 0x7f268815c1c0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26880d11c0 0x7f26881483c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 --2- 172.21.15.80:0/1440414749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f268806fc30 0x7f268809f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 >> 172.21.15.80:0/1440414749 conn(0x7f26880d20d0 msgr2=0x7f268815f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 shutdown_connections 2024-04-25T13:45:13.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.079+0000 7f268ed23640 1 -- 172.21.15.80:0/1440414749 wait complete. 2024-04-25T13:45:13.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:44:33.879562+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:44:34.932724+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:45:13.101 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:45:13.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.255+0000 7fe9b2c67640 1 Processor -- start 2024-04-25T13:45:13.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.255+0000 7fe9b2c67640 1 -- start start 2024-04-25T13:45:13.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.255+0000 7fe9b2c67640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac06edb0 0x7fe9ac06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.255+0000 7fe9b2c67640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9ac161b60 0x7fe9ac161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.255+0000 7fe9b2c67640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9ac1624b0 con 0x7fe9ac0b5330 2024-04-25T13:45:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.255+0000 7fe9b2c67640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe9ac06f700 con 0x7fe9ac161b60 2024-04-25T13:45:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.255+0000 7fe9b2c67640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac1629b0 con 0x7fe9ac06edb0 2024-04-25T13:45:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9abfff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac06edb0 0x7fe9ac06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9b1204640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe9ac161b60 0x7fe9ac161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9abfff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac06edb0 0x7fe9ac06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:49518/0 (socket says 172.21.15.80:49518) 2024-04-25T13:45:13.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9abfff640 1 -- 172.21.15.80:0/2480865422 learned_addr learned my addr 172.21.15.80:0/2480865422 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:13.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9b1204640 1 -- 172.21.15.80:0/2480865422 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac06edb0 msgr2=0x7fe9ac06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:13.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9b1204640 1 --2- 172.21.15.80:0/2480865422 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac06edb0 0x7fe9ac06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9b1204640 1 -- 172.21.15.80:0/2480865422 >> v1:172.21.15.87:6789/0 conn(0x7fe9ac0b5330 legacy=0x7fe9ac0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9b1204640 1 -- 172.21.15.80:0/2480865422 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac0b5740 con 0x7fe9ac161b60 2024-04-25T13:45:13.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9abfff640 1 --2- 172.21.15.80:0/2480865422 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac06edb0 0x7fe9ac06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:13.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9b1204640 1 --2- 172.21.15.80:0/2480865422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac161b60 0x7fe9ac161f70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009d90 tx=0x7fe9a401c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4256e98926a1e73f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9ab7fe640 1 -- 172.21.15.80:0/2480865422 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40b2070 con 0x7fe9ac161b60 2024-04-25T13:45:13.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9ab7fe640 1 -- 172.21.15.80:0/2480865422 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9a40a5970 con 0x7fe9ac161b60 2024-04-25T13:45:13.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.259+0000 7fe9ab7fe640 1 -- 172.21.15.80:0/2480865422 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9a40a5d40 con 0x7fe9ac161b60 2024-04-25T13:45:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2480865422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac161b60 msgr2=0x7fe9ac161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2480865422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac161b60 0x7fe9ac161f70 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe9a4009d90 tx=0x7fe9a401c8b0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2480865422 shutdown_connections 2024-04-25T13:45:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2480865422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac161b60 0x7fe9ac161f70 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2480865422 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac06edb0 0x7fe9ac06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2480865422 >> 172.21.15.80:0/2480865422 conn(0x7fe9ac0b4160 msgr2=0x7fe9ac0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:13.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2480865422 shutdown_connections 2024-04-25T13:45:13.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2480865422 wait complete. 2024-04-25T13:45:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 Processor -- start 2024-04-25T13:45:13.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.263+0000 7fe9b2c67640 1 -- start start 2024-04-25T13:45:13.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac06edb0 0x7fe9ac157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac0b5330 0x7fe9ac15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac161b60 0x7fe9ac0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac06e890 con 0x7fe9ac06edb0 2024-04-25T13:45:13.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ac06e5b0 con 0x7fe9ac0b5330 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ac0b0cd0 con 0x7fe9ac161b60 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac06edb0 0x7fe9ac157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac06edb0 0x7fe9ac157830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50076/0 (socket says 172.21.15.80:50076) 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 -- 172.21.15.80:0/2627639133 learned_addr learned my addr 172.21.15.80:0/2627639133 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b1204640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac161b60 0x7fe9ac0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9abfff640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac0b5330 0x7fe9ac15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 -- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac161b60 msgr2=0x7fe9ac0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac161b60 0x7fe9ac0af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 -- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac0b5330 msgr2=0x7fe9ac15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac0b5330 0x7fe9ac15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 -- 172.21.15.80:0/2627639133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ac1f98d0 con 0x7fe9ac06edb0 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9abfff640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac0b5330 0x7fe9ac15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b0a03640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac06edb0 0x7fe9ac157830 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c3a0 tx=0x7fe99c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9a97fa640 1 -- 172.21.15.80:0/2627639133 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c0052d0 con 0x7fe9ac06edb0 2024-04-25T13:45:13.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9a97fa640 1 -- 172.21.15.80:0/2627639133 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe99c005470 con 0x7fe9ac06edb0 2024-04-25T13:45:13.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9a97fa640 1 -- 172.21.15.80:0/2627639133 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe99c007260 con 0x7fe9ac06edb0 2024-04-25T13:45:13.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ac15ec80 con 0x7fe9ac06edb0 2024-04-25T13:45:13.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ac15f240 con 0x7fe9ac06edb0 2024-04-25T13:45:13.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.267+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ac15f480 con 0x7fe9ac06edb0 2024-04-25T13:45:13.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.271+0000 7fe9a97fa640 1 -- 172.21.15.80:0/2627639133 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe99c007400 con 0x7fe9ac06edb0 2024-04-25T13:45:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.271+0000 7fe9a97fa640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe99806f400 0x7fe998071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.271+0000 7fe9a97fa640 1 -- 172.21.15.80:0/2627639133 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7fe99c08e6c0 con 0x7fe9ac06edb0 2024-04-25T13:45:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.271+0000 7fe9a97fa640 1 -- 172.21.15.80:0/2627639133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7fe998076f80 con 0x7fe9ac06edb0 2024-04-25T13:45:13.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.271+0000 7fe9abfff640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe99806f400 0x7fe998071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.271+0000 7fe9abfff640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe99806f400 0x7fe998071910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0b0470 tx=0x7fe9a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.275+0000 7fe9a97fa640 1 -- 172.21.15.80:0/2627639133 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe9ac1624b0 con 0x7fe9ac06edb0 2024-04-25T13:45:13.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.439+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe9ac06a3a0 con 0x7fe9ac06edb0 2024-04-25T13:45:13.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.439+0000 7fe9a97fa640 1 -- 172.21.15.80:0/2627639133 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) ==== 93+0+10 (secure 0 0 0) 0x7fe99c05af60 con 0x7fe9ac06edb0 2024-04-25T13:45:13.439 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe99806f400 msgr2=0x7fe998071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe99806f400 0x7fe998071910 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe9ac0b0470 tx=0x7fe9a0008040 comp rx=0 tx=0).stop 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac06edb0 msgr2=0x7fe9ac157830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac06edb0 0x7fe9ac157830 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe99c00c3a0 tx=0x7fe99c00c860 comp rx=0 tx=0).stop 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 shutdown_connections 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe99806f400 0x7fe998071910 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe9ac161b60 0x7fe9ac0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe9ac0b5330 0x7fe9ac15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 --2- 172.21.15.80:0/2627639133 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe9ac06edb0 0x7fe9ac157830 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 >> 172.21.15.80:0/2627639133 conn(0x7fe9ac0b4160 msgr2=0x7fe9ac15f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 shutdown_connections 2024-04-25T13:45:13.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.443+0000 7fe9b2c67640 1 -- 172.21.15.80:0/2627639133 wait complete. 2024-04-25T13:45:13.462 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:45:13.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f9600519640 1 Processor -- start 2024-04-25T13:45:13.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f9600519640 1 -- start start 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f9600519640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f95f8073080 0x7f95f8073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f9600519640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95f806eb80 0x7f95f80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f9600519640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95f80737f0 con 0x7f95f815c710 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f9600519640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f95f80b53d0 con 0x7f95f806eb80 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f9600519640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f95f8160b20 con 0x7f95f8073080 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f95feab6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95f806eb80 0x7f95f80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f95fdab4640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f95f815c710 0x7f95f806e470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38532/0 (socket says 172.21.15.80:38532) 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f95fdab4640 1 -- 172.21.15.80:0/3438281069 learned_addr learned my addr 172.21.15.80:0/3438281069 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f95fe2b5640 1 --2- 172.21.15.80:0/3438281069 >> v2:172.21.15.87:3300/0 conn(0x7f95f8073080 0x7f95f8073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f95feab6640 1 -- 172.21.15.80:0/3438281069 >> v2:172.21.15.87:3300/0 conn(0x7f95f8073080 msgr2=0x7f95f8073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:13.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f95feab6640 1 --2- 172.21.15.80:0/3438281069 >> v2:172.21.15.87:3300/0 conn(0x7f95f8073080 0x7f95f8073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f95feab6640 1 -- 172.21.15.80:0/3438281069 >> v1:172.21.15.80:6789/0 conn(0x7f95f815c710 legacy=0x7f95f806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.619+0000 7f95feab6640 1 -- 172.21.15.80:0/3438281069 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f81604d0 con 0x7f95f806eb80 2024-04-25T13:45:13.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f95feab6640 1 --2- 172.21.15.80:0/3438281069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f806eb80 0x7f95f80ce630 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f95f8071630 tx=0x7f95f001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe04edbbf34319a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f95fd2b3640 1 -- 172.21.15.80:0/3438281069 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f00a66f0 con 0x7f95f806eb80 2024-04-25T13:45:13.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f95fd2b3640 1 -- 172.21.15.80:0/3438281069 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f00a6890 con 0x7f95f806eb80 2024-04-25T13:45:13.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f95fd2b3640 1 -- 172.21.15.80:0/3438281069 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f00a6c60 con 0x7f95f806eb80 2024-04-25T13:45:13.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f9600519640 1 -- 172.21.15.80:0/3438281069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f806eb80 msgr2=0x7f95f80ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f9600519640 1 --2- 172.21.15.80:0/3438281069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f806eb80 0x7f95f80ce630 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f95f8071630 tx=0x7f95f001c8a0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f9600519640 1 -- 172.21.15.80:0/3438281069 shutdown_connections 2024-04-25T13:45:13.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f9600519640 1 --2- 172.21.15.80:0/3438281069 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f806eb80 0x7f95f80ce630 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f9600519640 1 --2- 172.21.15.80:0/3438281069 >> v2:172.21.15.87:3300/0 conn(0x7f95f8073080 0x7f95f8073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f9600519640 1 -- 172.21.15.80:0/3438281069 >> 172.21.15.80:0/3438281069 conn(0x7f95f80b4160 msgr2=0x7f95f80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:13.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f9600519640 1 -- 172.21.15.80:0/3438281069 shutdown_connections 2024-04-25T13:45:13.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.623+0000 7f9600519640 1 -- 172.21.15.80:0/3438281069 wait complete. 2024-04-25T13:45:13.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f9600519640 1 Processor -- start 2024-04-25T13:45:13.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f9600519640 1 -- start start 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f9600519640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f8073080 0x7f95f81460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f9600519640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95f815c710 0x7f95f8148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f9600519640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95f81474f0 0x7f95f814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f9600519640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f95f80cecf0 con 0x7f95f8073080 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f9600519640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f95f80ceb70 con 0x7f95f81474f0 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f9600519640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f95f80cee70 con 0x7f95f815c710 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f8073080 0x7f95f81460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f8073080 0x7f95f81460c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50102/0 (socket says 172.21.15.80:50102) 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 -- 172.21.15.80:0/2965860806 learned_addr learned my addr 172.21.15.80:0/2965860806 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 -- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95f815c710 msgr2=0x7f95f8148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95f815c710 0x7f95f8148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 -- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95f81474f0 msgr2=0x7f95f814dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95feab6640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95f81474f0 0x7f95f814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95f81474f0 0x7f95f814dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 -- 172.21.15.80:0/2965860806 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f8148780 con 0x7f95f8073080 2024-04-25T13:45:13.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95fe2b5640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f8073080 0x7f95f81460c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f95f400d600 tx=0x7f95f400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95eeffd640 1 -- 172.21.15.80:0/2965860806 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4004240 con 0x7f95f8073080 2024-04-25T13:45:13.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.627+0000 7f95eeffd640 1 -- 172.21.15.80:0/2965860806 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95f40043e0 con 0x7f95f8073080 2024-04-25T13:45:13.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f814cf50 con 0x7f95f8073080 2024-04-25T13:45:13.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f95eeffd640 1 -- 172.21.15.80:0/2965860806 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f4003840 con 0x7f95f8073080 2024-04-25T13:45:13.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f814d510 con 0x7f95f8073080 2024-04-25T13:45:13.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95f806a3a0 con 0x7f95f8073080 2024-04-25T13:45:13.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f95eeffd640 1 -- 172.21.15.80:0/2965860806 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95f40039e0 con 0x7f95f8073080 2024-04-25T13:45:13.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f95eeffd640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95dc06f430 0x7f95dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f95eeffd640 1 -- 172.21.15.80:0/2965860806 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f95f408f5d0 con 0x7f95f8073080 2024-04-25T13:45:13.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f95eeffd640 1 -- 172.21.15.80:0/2965860806 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7f95dc076fb0 con 0x7f95f8073080 2024-04-25T13:45:13.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f95fdab4640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95dc06f430 0x7f95dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.631+0000 7f95fdab4640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95dc06f430 0x7f95dc071940 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f95e8000f70 tx=0x7f95e8007a90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.639+0000 7f95eeffd640 1 -- 172.21.15.80:0/2965860806 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95f405c420 con 0x7f95f8073080 2024-04-25T13:45:13.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.795+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95f806f840 con 0x7f95f8073080 2024-04-25T13:45:13.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f95eeffd640 1 -- 172.21.15.80:0/2965860806 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) ==== 104+0+11 (secure 0 0 0) 0x7f95f405bd20 con 0x7f95f8073080 2024-04-25T13:45:13.797 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95dc06f430 msgr2=0x7f95dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95dc06f430 0x7f95dc071940 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f95e8000f70 tx=0x7f95e8007a90 comp rx=0 tx=0).stop 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f8073080 msgr2=0x7f95f81460c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f8073080 0x7f95f81460c0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f95f400d600 tx=0x7f95f400dac0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 shutdown_connections 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95dc06f430 0x7f95dc071940 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95f81474f0 0x7f95f814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95f815c710 0x7f95f8148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 --2- 172.21.15.80:0/2965860806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95f8073080 0x7f95f81460c0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 >> 172.21.15.80:0/2965860806 conn(0x7f95f80b4160 msgr2=0x7f95f806e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:13.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 shutdown_connections 2024-04-25T13:45:13.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.799+0000 7f9600519640 1 -- 172.21.15.80:0/2965860806 wait complete. 2024-04-25T13:45:13.819 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:45:13.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0334680640 1 Processor -- start 2024-04-25T13:45:13.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0334680640 1 -- start start 2024-04-25T13:45:13.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0334680640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f032c0d09c0 0x7f032c0d0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0334680640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0b9260 0x7f032c0b9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0334680640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f032c0d18b0 con 0x7f032c0d26b0 2024-04-25T13:45:13.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0334680640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f032c0d56c0 con 0x7f032c0d09c0 2024-04-25T13:45:13.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0334680640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f032c0d1310 con 0x7f032c0b9260 2024-04-25T13:45:13.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033241c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f032c0d26b0 0x7f032c0d2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38534/0 (socket says 172.21.15.80:38534) 2024-04-25T13:45:13.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033241c640 1 -- 172.21.15.80:0/1496038215 learned_addr learned my addr 172.21.15.80:0/1496038215 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1030033298 0 0) 0x7f0328001120 con 0x7f032c0d26b0 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0332c1d640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0b9260 0x7f032c0b9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0318003530 con 0x7f032c0d26b0 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0331c1b640 1 --2- 172.21.15.80:0/1496038215 >> v2:172.21.15.87:3300/0 conn(0x7f032c0d09c0 0x7f032c0d0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 188636534 0 0) 0x7f0318003530 con 0x7f032c0d26b0 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0b9260 msgr2=0x7f032c0b9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0b9260 0x7f032c0b9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 >> v2:172.21.15.87:3300/0 conn(0x7f032c0d09c0 msgr2=0x7f032c0d0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 --2- 172.21.15.80:0/1496038215 >> v2:172.21.15.87:3300/0 conn(0x7f032c0d09c0 0x7f032c0d0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c0d15d0 con 0x7f032c0d26b0 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0328001850 con 0x7f032c0d26b0 2024-04-25T13:45:13.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 >> v1:172.21.15.80:6789/0 conn(0x7f032c0d26b0 legacy=0x7f032c0d2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f0318003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0318004810 0x7f0318004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0b9260 0x7f0318005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f032c0d56c0 con 0x7f03180039c0 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f032c0d1310 con 0x7f0318004810 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f031800b9d0 con 0x7f032c0b9260 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033241c640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f0318003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0331c1b640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0318004810 0x7f0318004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033241c640 1 -- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0b9260 msgr2=0x7f0318005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033241c640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0b9260 0x7f0318005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033241c640 1 -- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0318004810 msgr2=0x7f0318004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:13.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033241c640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0318004810 0x7f0318004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f033241c640 1 -- 172.21.15.80:0/1496038215 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0318003530 con 0x7f03180039c0 2024-04-25T13:45:13.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.975+0000 7f0331c1b640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0318004810 0x7f0318004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f033241c640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f0318003e30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f032800c490 tx=0x7f032801cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6329e9cb9cc26549 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032800c0e0 con 0x7f03180039c0 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03280a9020 con 0x7f03180039c0 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f033141a640 1 -- 172.21.15.80:0/1496038215 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328005ae0 con 0x7f03180039c0 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f0334680640 1 -- 172.21.15.80:0/1496038215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 msgr2=0x7f0318003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f0334680640 1 --2- 172.21.15.80:0/1496038215 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f0318003e30 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f032800c490 tx=0x7f032801cca0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f0332c1d640 1 -- 172.21.15.80:0/1496038215 reap_dead start 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f0334680640 1 -- 172.21.15.80:0/1496038215 shutdown_connections 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f0334680640 1 -- 172.21.15.80:0/1496038215 >> 172.21.15.80:0/1496038215 conn(0x7f032c0b77b0 msgr2=0x7f032c0b7bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f0334680640 1 -- 172.21.15.80:0/1496038215 shutdown_connections 2024-04-25T13:45:13.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.979+0000 7f0334680640 1 -- 172.21.15.80:0/1496038215 wait complete. 2024-04-25T13:45:13.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 Processor -- start 2024-04-25T13:45:13.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 -- start start 2024-04-25T13:45:13.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f032c15be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f032c14ef80 0x7f032c14f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0ba940 0x7f032c0bad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f032c160630 con 0x7f03180039c0 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f032c1604b0 con 0x7f032c14ef80 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f032c16ad90 con 0x7f032c0ba940 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f032c15be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f032c15be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50116/0 (socket says 172.21.15.80:50116) 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 -- 172.21.15.80:0/4187478758 learned_addr learned my addr 172.21.15.80:0/4187478758 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0331c1b640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f032c14ef80 0x7f032c14f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0332c1d640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0ba940 0x7f032c0bad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 -- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0ba940 msgr2=0x7f032c0bad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0ba940 0x7f032c0bad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 -- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f032c14ef80 msgr2=0x7f032c14f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f032c14ef80 0x7f032c14f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 -- 172.21.15.80:0/4187478758 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032c15c350 con 0x7f03180039c0 2024-04-25T13:45:13.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0331c1b640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f032c14ef80 0x7f032c14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:13.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0332c1d640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0ba940 0x7f032c0bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:13.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f033241c640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f032c15be10 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f03280ab040 tx=0x7f0318005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0322ffd640 1 -- 172.21.15.80:0/4187478758 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280a5a70 con 0x7f03180039c0 2024-04-25T13:45:13.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0322ffd640 1 -- 172.21.15.80:0/4187478758 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03280b3030 con 0x7f03180039c0 2024-04-25T13:45:13.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032c0bb290 con 0x7f03180039c0 2024-04-25T13:45:13.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0322ffd640 1 -- 172.21.15.80:0/4187478758 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280047f0 con 0x7f03180039c0 2024-04-25T13:45:13.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f032c161120 con 0x7f03180039c0 2024-04-25T13:45:13.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.983+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f032c06a3a0 con 0x7f03180039c0 2024-04-25T13:45:13.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.987+0000 7f0322ffd640 1 -- 172.21.15.80:0/4187478758 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0328004990 con 0x7f03180039c0 2024-04-25T13:45:13.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.987+0000 7f0322ffd640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f031006f400 0x7f0310071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:13.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.987+0000 7f0322ffd640 1 -- 172.21.15.80:0/4187478758 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f0328128760 con 0x7f03180039c0 2024-04-25T13:45:13.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.987+0000 7f0322ffd640 1 -- 172.21.15.80:0/4187478758 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7f0310076fe0 con 0x7f03180039c0 2024-04-25T13:45:13.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.987+0000 7f0331c1b640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f031006f400 0x7f0310071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:13.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.987+0000 7f0331c1b640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f031006f400 0x7f0310071910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f031c009160 tx=0x7f031c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:13.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:13.991+0000 7f0322ffd640 1 -- 172.21.15.80:0/4187478758 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f032c0bb290 con 0x7f03180039c0 2024-04-25T13:45:14.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f032c15c840 con 0x7f03180039c0 2024-04-25T13:45:14.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0322ffd640 1 -- 172.21.15.80:0/4187478758 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) ==== 100+0+11 (secure 0 0 0) 0x7f032c15c350 con 0x7f03180039c0 2024-04-25T13:45:14.154 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:45:14.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f031006f400 msgr2=0x7f0310071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f031006f400 0x7f0310071910 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f031c009160 tx=0x7f031c008040 comp rx=0 tx=0).stop 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 msgr2=0x7f032c15be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f032c15be10 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f03280ab040 tx=0x7f0318005840 comp rx=0 tx=0).stop 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 shutdown_connections 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f031006f400 0x7f0310071910 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f032c0ba940 0x7f032c0bad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f032c14ef80 0x7f032c14f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 --2- 172.21.15.80:0/4187478758 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03180039c0 0x7f032c15be10 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 >> 172.21.15.80:0/4187478758 conn(0x7f032c0b77b0 msgr2=0x7f032c15e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 shutdown_connections 2024-04-25T13:45:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.155+0000 7f0334680640 1 -- 172.21.15.80:0/4187478758 wait complete. 2024-04-25T13:45:14.178 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:45:14.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33db6b5640 1 Processor -- start 2024-04-25T13:45:14.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33db6b5640 1 -- start start 2024-04-25T13:45:14.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33db6b5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f33d406f5a0 0x7f33d4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33db6b5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33d406fcf0 con 0x7f33d40733b0 2024-04-25T13:45:14.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33db6b5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33d40b59f0 con 0x7f33d4161070 2024-04-25T13:45:14.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33db6b5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f33d4073f10 con 0x7f33d406f5a0 2024-04-25T13:45:14.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33d9451640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f33d406f5a0 0x7f33d4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33d9c52640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f33d40733b0 0x7f33d40b52e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38550/0 (socket says 172.21.15.80:38550) 2024-04-25T13:45:14.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33d9c52640 1 -- 172.21.15.80:0/3560445985 learned_addr learned my addr 172.21.15.80:0/3560445985 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:14.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33d9451640 1 -- 172.21.15.80:0/3560445985 >> v1:172.21.15.87:6789/0 conn(0x7f33d4161070 legacy=0x7f33d40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33d9451640 1 -- 172.21.15.80:0/3560445985 >> v1:172.21.15.80:6789/0 conn(0x7f33d40733b0 legacy=0x7f33d40b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.331+0000 7f33d9451640 1 -- 172.21.15.80:0/3560445985 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d40b0c60 con 0x7f33d406f5a0 2024-04-25T13:45:14.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33d9451640 1 --2- 172.21.15.80:0/3560445985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d4160b30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f33d00097c0 tx=0x7f33d001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942f52938cf1bcae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:14.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33cbfff640 1 -- 172.21.15.80:0/3560445985 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d00b0070 con 0x7f33d406f5a0 2024-04-25T13:45:14.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33cbfff640 1 -- 172.21.15.80:0/3560445985 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d00ab600 con 0x7f33d406f5a0 2024-04-25T13:45:14.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33cbfff640 1 -- 172.21.15.80:0/3560445985 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d00ab9d0 con 0x7f33d406f5a0 2024-04-25T13:45:14.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33db6b5640 1 -- 172.21.15.80:0/3560445985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 msgr2=0x7f33d4160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33db6b5640 1 --2- 172.21.15.80:0/3560445985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d4160b30 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f33d00097c0 tx=0x7f33d001c410 comp rx=0 tx=0).stop 2024-04-25T13:45:14.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33db6b5640 1 -- 172.21.15.80:0/3560445985 shutdown_connections 2024-04-25T13:45:14.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33db6b5640 1 --2- 172.21.15.80:0/3560445985 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d4160b30 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33db6b5640 1 -- 172.21.15.80:0/3560445985 >> 172.21.15.80:0/3560445985 conn(0x7f33d406d080 msgr2=0x7f33d40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:14.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33db6b5640 1 -- 172.21.15.80:0/3560445985 shutdown_connections 2024-04-25T13:45:14.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.339+0000 7f33db6b5640 1 -- 172.21.15.80:0/3560445985 wait complete. 2024-04-25T13:45:14.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33db6b5640 1 Processor -- start 2024-04-25T13:45:14.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33db6b5640 1 -- start start 2024-04-25T13:45:14.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33db6b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d414c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33db6b5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33d40733b0 0x7f33d41460c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33db6b5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33d4161070 0x7f33d4157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33db6b5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f33d416b6f0 con 0x7f33d406f5a0 2024-04-25T13:45:14.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33db6b5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f33d4161890 con 0x7f33d4161070 2024-04-25T13:45:14.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33db6b5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f33d416b870 con 0x7f33d40733b0 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9451640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d414c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9451640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d414c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50136/0 (socket says 172.21.15.80:50136) 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9451640 1 -- 172.21.15.80:0/261051972 learned_addr learned my addr 172.21.15.80:0/261051972 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d8c50640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33d40733b0 0x7f33d41460c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9c52640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33d4161070 0x7f33d4157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9451640 1 -- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33d40733b0 msgr2=0x7f33d41460c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9451640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33d40733b0 0x7f33d41460c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9451640 1 -- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33d4161070 msgr2=0x7f33d4157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9451640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33d4161070 0x7f33d4157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.343+0000 7f33d9451640 1 -- 172.21.15.80:0/261051972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d414b1f0 con 0x7f33d406f5a0 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33d9451640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d414c890 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f33d00ae040 tx=0x7f33d001ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:14.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d00b3810 con 0x7f33d406f5a0 2024-04-25T13:45:14.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33d00b39d0 con 0x7f33d406f5a0 2024-04-25T13:45:14.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d00b3dc0 con 0x7f33d406f5a0 2024-04-25T13:45:14.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d4146840 con 0x7f33d406f5a0 2024-04-25T13:45:14.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d414b4d0 con 0x7f33d406f5a0 2024-04-25T13:45:14.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d406a3a0 con 0x7f33d406f5a0 2024-04-25T13:45:14.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f33d00ab4c0 con 0x7f33d406f5a0 2024-04-25T13:45:14.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33c9ffb640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f33b806f430 0x7f33b8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5709+0+0 (secure 0 0 0) 0x7f33d012bd40 con 0x7f33d406f5a0 2024-04-25T13:45:14.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=88}) -- 0x7f33b8076fb0 con 0x7f33d406f5a0 2024-04-25T13:45:14.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.347+0000 7f33d8c50640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f33b806f430 0x7f33b8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.351+0000 7f33d8c50640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f33b806f430 0x7f33b8071940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f33d40b0630 tx=0x7f33c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:14.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.351+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f33d0016090 con 0x7f33d406f5a0 2024-04-25T13:45:14.420 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:45:14.419+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2024-04-25T13:45:14.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.431+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 <== mon.0 v2:172.21.15.80:3300/0 7 ==== osd_map(88..88 src has 1..88) ==== 304+0+0 (secure 0 0 0) 0x7f33d00f49c0 con 0x7f33d406f5a0 2024-04-25T13:45:14.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.431+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=89}) -- 0x7f33b80778d0 con 0x7f33d406f5a0 2024-04-25T13:45:14.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.527+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f33d40739c0 con 0x7f33d406f5a0 2024-04-25T13:45:14.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.527+0000 7f33c9ffb640 1 -- 172.21.15.80:0/261051972 <== mon.0 v2:172.21.15.80:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v88) ==== 80+0+0 (secure 0 0 0) 0x7f33d00f8e30 con 0x7f33d406f5a0 2024-04-25T13:45:14.528 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:45:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f33b806f430 msgr2=0x7f33b8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f33b806f430 0x7f33b8071940 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f33d40b0630 tx=0x7f33c4009040 comp rx=0 tx=0).stop 2024-04-25T13:45:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 msgr2=0x7f33d414c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d414c890 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f33d00ae040 tx=0x7f33d001ac50 comp rx=0 tx=0).stop 2024-04-25T13:45:14.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 shutdown_connections 2024-04-25T13:45:14.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f33b806f430 0x7f33b8071940 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33d4161070 0x7f33d4157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f33d40733b0 0x7f33d41460c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 --2- 172.21.15.80:0/261051972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f33d406f5a0 0x7f33d414c890 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 >> 172.21.15.80:0/261051972 conn(0x7f33d406d080 msgr2=0x7f33d40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:14.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 shutdown_connections 2024-04-25T13:45:14.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.531+0000 7f33db6b5640 1 -- 172.21.15.80:0/261051972 wait complete. 2024-04-25T13:45:14.550 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:45:14.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:14.550 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:45:14.562 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24761]: fuse finished with error 0 and tester_r 0 2024-04-25T13:45:14.564 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:45:14.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:14.564 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:45:14.639 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:45:14.639 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:14.639 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:14.645 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:45:14.645 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:45:14.645 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:45:14.645 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:14.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:45:14.687 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:14.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.843+0000 7f693957e640 1 Processor -- start 2024-04-25T13:45:14.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.843+0000 7f693957e640 1 -- start start 2024-04-25T13:45:14.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f693957e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6934161f90 0x7f69340cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f693957e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69340d0510 0x7f693415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f693957e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f693415ea70 0x7f693415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f693957e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f69340d0d70 con 0x7f693415ea70 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f693957e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f69340d1050 con 0x7f6934161f90 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f693957e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f693406dca0 con 0x7f69340d0510 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f693415ea70 0x7f693415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f693415ea70 0x7f693415ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50144/0 (socket says 172.21.15.80:50144) 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 -- 172.21.15.80:0/1707737790 learned_addr learned my addr 172.21.15.80:0/1707737790 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f6932ffd640 1 --2- 172.21.15.80:0/1707737790 >> v2:172.21.15.87:3300/0 conn(0x7f6934161f90 0x7f69340cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69327fc640 1 --2- 172.21.15.80:0/1707737790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69340d0510 0x7f693415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 -- 172.21.15.80:0/1707737790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69340d0510 msgr2=0x7f693415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 --2- 172.21.15.80:0/1707737790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69340d0510 0x7f693415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 -- 172.21.15.80:0/1707737790 >> v2:172.21.15.87:3300/0 conn(0x7f6934161f90 msgr2=0x7f69340cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:14.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 --2- 172.21.15.80:0/1707737790 >> v2:172.21.15.87:3300/0 conn(0x7f6934161f90 0x7f69340cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 -- 172.21.15.80:0/1707737790 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f693406df80 con 0x7f693415ea70 2024-04-25T13:45:14.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f6932ffd640 1 --2- 172.21.15.80:0/1707737790 >> v2:172.21.15.87:3300/0 conn(0x7f6934161f90 0x7f69340cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:14.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f69337fe640 1 --2- 172.21.15.80:0/1707737790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f693415ea70 0x7f693415ee80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f69340d17b0 tx=0x7f692c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f5b1de070c5b2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:14.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f6931ffb640 1 -- 172.21.15.80:0/1707737790 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0a5770 con 0x7f693415ea70 2024-04-25T13:45:14.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f6931ffb640 1 -- 172.21.15.80:0/1707737790 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692c0a5910 con 0x7f693415ea70 2024-04-25T13:45:14.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f6931ffb640 1 -- 172.21.15.80:0/1707737790 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692c0a5ce0 con 0x7f693415ea70 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f693957e640 1 -- 172.21.15.80:0/1707737790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f693415ea70 msgr2=0x7f693415ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.847+0000 7f693957e640 1 --2- 172.21.15.80:0/1707737790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f693415ea70 0x7f693415ee80 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f69340d17b0 tx=0x7f692c01c930 comp rx=0 tx=0).stop 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 -- 172.21.15.80:0/1707737790 shutdown_connections 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 --2- 172.21.15.80:0/1707737790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f693415ea70 0x7f693415ee80 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 --2- 172.21.15.80:0/1707737790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f69340d0510 0x7f693415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 --2- 172.21.15.80:0/1707737790 >> v2:172.21.15.87:3300/0 conn(0x7f6934161f90 0x7f69340cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 -- 172.21.15.80:0/1707737790 >> 172.21.15.80:0/1707737790 conn(0x7f693406d080 msgr2=0x7f69340b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 -- 172.21.15.80:0/1707737790 shutdown_connections 2024-04-25T13:45:14.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 -- 172.21.15.80:0/1707737790 wait complete. 2024-04-25T13:45:14.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 Processor -- start 2024-04-25T13:45:14.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 -- start start 2024-04-25T13:45:14.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69340d0510 0x7f6934167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f693415ea70 0x7f69341fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6934161f90 0x7f69341fa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f69341447a0 con 0x7f69340d0510 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6934144620 con 0x7f693415ea70 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f693957e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f69340d0920 con 0x7f6934161f90 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f6932ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69340d0510 0x7f6934167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f6932ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69340d0510 0x7f6934167ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50158/0 (socket says 172.21.15.80:50158) 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f6932ffd640 1 -- 172.21.15.80:0/1607062393 learned_addr learned my addr 172.21.15.80:0/1607062393 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f69327fc640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f693415ea70 0x7f69341fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f69337fe640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6934161f90 0x7f69341fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f6932ffd640 1 -- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6934161f90 msgr2=0x7f69341fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f6932ffd640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6934161f90 0x7f69341fa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f6932ffd640 1 -- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f693415ea70 msgr2=0x7f69341fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:14.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f6932ffd640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f693415ea70 0x7f69341fa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f6932ffd640 1 -- 172.21.15.80:0/1607062393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69341fb7a0 con 0x7f69340d0510 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.851+0000 7f69327fc640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f693415ea70 0x7f69341fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f6932ffd640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69340d0510 0x7f6934167ae0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f692400c820 tx=0x7f692400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f691b7fe640 1 -- 172.21.15.80:0/1607062393 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924015070 con 0x7f69340d0510 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f691b7fe640 1 -- 172.21.15.80:0/1607062393 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6924007a30 con 0x7f69340d0510 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6934145970 con 0x7f69340d0510 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f691b7fe640 1 -- 172.21.15.80:0/1607062393 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924007e20 con 0x7f69340d0510 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6934166180 con 0x7f69340d0510 2024-04-25T13:45:14.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f693406a3a0 con 0x7f69340d0510 2024-04-25T13:45:14.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f691b7fe640 1 -- 172.21.15.80:0/1607062393 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6924018070 con 0x7f69340d0510 2024-04-25T13:45:14.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f691b7fe640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f692006f400 0x7f6920071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:14.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f691b7fe640 1 -- 172.21.15.80:0/1607062393 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5709+0+0 (secure 0 0 0) 0x7f692408f150 con 0x7f69340d0510 2024-04-25T13:45:14.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f691b7fe640 1 -- 172.21.15.80:0/1607062393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=89}) -- 0x7f6920076f80 con 0x7f69340d0510 2024-04-25T13:45:14.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f69327fc640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f692006f400 0x7f6920071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:14.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.855+0000 7f69327fc640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f692006f400 0x7f6920071910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f69340ce9a0 tx=0x7f6928008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:14.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:14.863+0000 7f691b7fe640 1 -- 172.21.15.80:0/1607062393 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f69341fb7a0 con 0x7f69340d0510 2024-04-25T13:45:15.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69340b5280 con 0x7f69340d0510 2024-04-25T13:45:15.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f691b7fe640 1 -- 172.21.15.80:0/1607062393 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5180 (secure 0 0 0) 0x7f692405c0d0 con 0x7f69340d0510 2024-04-25T13:45:15.058 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:15.058 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2082291876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2082291876},{"type":"v1","addr":"172.21.15.87:6835","nonce":2082291876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1337331528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1337331528},{"type":"v1","addr":"172.21.15.80:6833","nonce":1337331528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2521189874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2521189874},{"type":"v1","addr":"172.21.15.80:6835","nonce":2521189874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:44:35.958393+0000","modified":"2024-04-25T13:44:39.019142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/642879744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":642879744},{"type":"v1","addr":"172.21.15.87:6837","nonce":642879744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":5}]} 2024-04-25T13:45:15.058 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 41 2024-04-25T13:45:15.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f692006f400 msgr2=0x7f6920071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f692006f400 0x7f6920071910 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f69340ce9a0 tx=0x7f6928008040 comp rx=0 tx=0).stop 2024-04-25T13:45:15.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69340d0510 msgr2=0x7f6934167ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69340d0510 0x7f6934167ae0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f692400c820 tx=0x7f692400cce0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 shutdown_connections 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f692006f400 0x7f6920071910 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6934161f90 0x7f69341fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f693415ea70 0x7f69341fa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 --2- 172.21.15.80:0/1607062393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f69340d0510 0x7f6934167ae0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 >> 172.21.15.80:0/1607062393 conn(0x7f693406d080 msgr2=0x7f6934168120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 shutdown_connections 2024-04-25T13:45:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.059+0000 7f693957e640 1 -- 172.21.15.80:0/1607062393 wait complete. 2024-04-25T13:45:15.080 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:15.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8ea5e2640 1 Processor -- start 2024-04-25T13:45:15.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8ea5e2640 1 -- start start 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8ea5e2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e406f5a0 0x7fb8e4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8ea5e2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8e4161070 0x7fb8e4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8ea5e2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e406fcf0 con 0x7fb8e40b4bd0 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8ea5e2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e41619c0 con 0x7fb8e4161070 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8ea5e2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e4073f10 con 0x7fb8e406f5a0 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8e3fff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e406f5a0 0x7fb8e4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8e37fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8e4161070 0x7fb8e4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8e3fff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e406f5a0 0x7fb8e4160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:49606/0 (socket says 172.21.15.80:49606) 2024-04-25T13:45:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8e8b7f640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb8e40b4bd0 0x7fb8e40b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:49416/0 (socket says 172.21.15.80:49416) 2024-04-25T13:45:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.215+0000 7fb8e3fff640 1 -- 172.21.15.80:0/3518865092 learned_addr learned my addr 172.21.15.80:0/3518865092 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8e3fff640 1 -- 172.21.15.80:0/3518865092 >> v2:172.21.15.87:3300/0 conn(0x7fb8e4161070 msgr2=0x7fb8e4161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8e3fff640 1 --2- 172.21.15.80:0/3518865092 >> v2:172.21.15.87:3300/0 conn(0x7fb8e4161070 0x7fb8e4161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8e3fff640 1 -- 172.21.15.80:0/3518865092 >> v1:172.21.15.87:6789/0 conn(0x7fb8e40b4bd0 legacy=0x7fb8e40b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8e3fff640 1 -- 172.21.15.80:0/3518865092 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e40b0c60 con 0x7fb8e406f5a0 2024-04-25T13:45:15.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8e3fff640 1 --2- 172.21.15.80:0/3518865092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e406f5a0 0x7fb8e4160b30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009870 tx=0x7fb8dc01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9b57de14de4377d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8e2ffd640 1 -- 172.21.15.80:0/3518865092 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0b1070 con 0x7fb8e406f5a0 2024-04-25T13:45:15.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8e2ffd640 1 -- 172.21.15.80:0/3518865092 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8dc0ac620 con 0x7fb8e406f5a0 2024-04-25T13:45:15.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8e2ffd640 1 -- 172.21.15.80:0/3518865092 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0ac9f0 con 0x7fb8e406f5a0 2024-04-25T13:45:15.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/3518865092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e406f5a0 msgr2=0x7fb8e4160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/3518865092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e406f5a0 0x7fb8e4160b30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009870 tx=0x7fb8dc01c4d0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/3518865092 shutdown_connections 2024-04-25T13:45:15.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/3518865092 >> v2:172.21.15.87:3300/0 conn(0x7fb8e4161070 0x7fb8e4161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/3518865092 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e406f5a0 0x7fb8e4160b30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/3518865092 >> 172.21.15.80:0/3518865092 conn(0x7fb8e406d080 msgr2=0x7fb8e40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:15.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/3518865092 shutdown_connections 2024-04-25T13:45:15.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.219+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/3518865092 wait complete. 2024-04-25T13:45:15.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8ea5e2640 1 Processor -- start 2024-04-25T13:45:15.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8ea5e2640 1 -- start start 2024-04-25T13:45:15.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8ea5e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e406f5a0 0x7fb8e4158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8ea5e2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40b4bd0 0x7fb8e4150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8ea5e2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161070 0x7fb8e4153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8ea5e2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4168850 con 0x7fb8e406f5a0 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8ea5e2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e41686d0 con 0x7fb8e40b4bd0 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8ea5e2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e41689d0 con 0x7fb8e4161070 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8e3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e406f5a0 0x7fb8e4158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8e3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e406f5a0 0x7fb8e4158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50172/0 (socket says 172.21.15.80:50172) 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.223+0000 7fb8e3fff640 1 -- 172.21.15.80:0/1195445443 learned_addr learned my addr 172.21.15.80:0/1195445443 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e37fe640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40b4bd0 0x7fb8e4150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e8b7f640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161070 0x7fb8e4153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e3fff640 1 -- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161070 msgr2=0x7fb8e4153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e3fff640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161070 0x7fb8e4153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e3fff640 1 -- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40b4bd0 msgr2=0x7fb8e4150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e3fff640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40b4bd0 0x7fb8e4150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e3fff640 1 -- 172.21.15.80:0/1195445443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e41567f0 con 0x7fb8e406f5a0 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e37fe640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40b4bd0 0x7fb8e4150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e3fff640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e406f5a0 0x7fb8e4158390 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0af040 tx=0x7fb8dc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e0ff9640 1 -- 172.21.15.80:0/1195445443 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0bb020 con 0x7fb8e406f5a0 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e0ff9640 1 -- 172.21.15.80:0/1195445443 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8dc0b7420 con 0x7fb8e406f5a0 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e4156a90 con 0x7fb8e406f5a0 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e0ff9640 1 -- 172.21.15.80:0/1195445443 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc0b1040 con 0x7fb8e406f5a0 2024-04-25T13:45:15.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e4157050 con 0x7fb8e406f5a0 2024-04-25T13:45:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e406a3a0 con 0x7fb8e406f5a0 2024-04-25T13:45:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e0ff9640 1 -- 172.21.15.80:0/1195445443 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb8dc0be020 con 0x7fb8e406f5a0 2024-04-25T13:45:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e0ff9640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8c406f400 0x7fb8c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e0ff9640 1 -- 172.21.15.80:0/1195445443 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5709+0+0 (secure 0 0 0) 0x7fb8dc129900 con 0x7fb8e406f5a0 2024-04-25T13:45:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e0ff9640 1 -- 172.21.15.80:0/1195445443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=89}) -- 0x7fb8c4076f80 con 0x7fb8e406f5a0 2024-04-25T13:45:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.227+0000 7fb8e37fe640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8c406f400 0x7fb8c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.231+0000 7fb8e37fe640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8c406f400 0x7fb8c4071910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40cdb80 tx=0x7fb8cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.231+0000 7fb8e0ff9640 1 -- 172.21.15.80:0/1195445443 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8dc0c5030 con 0x7fb8e406f5a0 2024-04-25T13:45:15.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.391+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8e40739c0 con 0x7fb8e406f5a0 2024-04-25T13:45:15.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.395+0000 7fb8e0ff9640 1 -- 172.21.15.80:0/1195445443 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+16625 (secure 0 0 0) 0x7fb8dc016150 con 0x7fb8e406f5a0 2024-04-25T13:45:15.395 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:15.395 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":88,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:14.423928+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:44:33.879562+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T13:44:34.932724+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","full","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:15.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8c406f400 msgr2=0x7fb8c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8c406f400 0x7fb8c4071910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fb8e40cdb80 tx=0x7fb8cc008040 comp rx=0 tx=0).stop 2024-04-25T13:45:15.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e406f5a0 msgr2=0x7fb8e4158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e406f5a0 0x7fb8e4158390 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0af040 tx=0x7fb8dc01ad10 comp rx=0 tx=0).stop 2024-04-25T13:45:15.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 shutdown_connections 2024-04-25T13:45:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8c406f400 0x7fb8c4071910 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161070 0x7fb8e4153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40b4bd0 0x7fb8e4150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 --2- 172.21.15.80:0/1195445443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e406f5a0 0x7fb8e4158390 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 >> 172.21.15.80:0/1195445443 conn(0x7fb8e406d080 msgr2=0x7fb8e40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 shutdown_connections 2024-04-25T13:45:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.399+0000 7fb8ea5e2640 1 -- 172.21.15.80:0/1195445443 wait complete. 2024-04-25T13:45:15.417 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:44:33.879562+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:44:34.932724+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:45:15.417 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:45:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39efb57640 1 Processor -- start 2024-04-25T13:45:15.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39efb57640 1 -- start start 2024-04-25T13:45:15.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39efb57640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f39e8073ad0 0x7f39e806e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39efb57640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e806edc0 con 0x7f39e806e9b0 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39efb57640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39e80737f0 con 0x7f39e8073080 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39efb57640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f39e80b53d0 con 0x7f39e8073ad0 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ee0f4640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f39e806e9b0 0x7f39e80ce630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38560/0 (socket says 172.21.15.80:38560) 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ee0f4640 1 -- 172.21.15.80:0/3444110067 learned_addr learned my addr 172.21.15.80:0/3444110067 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ed0f2640 1 --2- 172.21.15.80:0/3444110067 >> v2:172.21.15.87:3300/0 conn(0x7f39e8073ad0 0x7f39e806e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3700151846 0 0) 0x7f39e00014a0 con 0x7f39e806e9b0 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f39d4003530 con 0x7f39e806e9b0 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 893393773 0 0) 0x7f39d4003530 con 0x7f39e806e9b0 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 >> v2:172.21.15.87:3300/0 conn(0x7f39e8073ad0 msgr2=0x7f39e806e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 --2- 172.21.15.80:0/3444110067 >> v2:172.21.15.87:3300/0 conn(0x7f39e8073ad0 0x7f39e806e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 >> v1:172.21.15.87:6789/0 conn(0x7f39e8073080 legacy=0x7f39e815c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e81604d0 con 0x7f39e806e9b0 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f39e0001bd0 con 0x7f39e806e9b0 2024-04-25T13:45:15.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.571+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 >> v1:172.21.15.80:6789/0 conn(0x7f39e806e9b0 legacy=0x7f39e80ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4003a10 0x7f39d4003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39d4004810 0x7f39d4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39d40052b0 0x7f39d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39e80b53d0 con 0x7f39d4004810 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39d400c240 con 0x7f39d4003a10 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39d400c440 con 0x7f39d40052b0 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed0f2640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39d4004810 0x7f39d4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ee0f4640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39d40052b0 0x7f39d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed8f3640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4003a10 0x7f39d4003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed0f2640 1 -- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39d40052b0 msgr2=0x7f39d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed0f2640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39d40052b0 0x7f39d400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed0f2640 1 -- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4003a10 msgr2=0x7f39d4003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed0f2640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4003a10 0x7f39d4003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed0f2640 1 -- 172.21.15.80:0/3444110067 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d4003530 con 0x7f39d4004810 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed8f3640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4003a10 0x7f39d4003e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:15.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ee0f4640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39d40052b0 0x7f39d400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:15.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ee0f4640 1 -- 172.21.15.80:0/3444110067 reap_dead start 2024-04-25T13:45:15.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ed0f2640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39d4004810 0x7f39d4004c20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f39d800aae0 tx=0x7f39d801dfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addd846b9cce091b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d80b2070 con 0x7f39d4004810 2024-04-25T13:45:15.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39d80ad600 con 0x7f39d4004810 2024-04-25T13:45:15.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39ec8f1640 1 -- 172.21.15.80:0/3444110067 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d800b040 con 0x7f39d4004810 2024-04-25T13:45:15.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39efb57640 1 -- 172.21.15.80:0/3444110067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39d4004810 msgr2=0x7f39d4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39efb57640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39d4004810 0x7f39d4004c20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f39d800aae0 tx=0x7f39d801dfc0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39efb57640 1 -- 172.21.15.80:0/3444110067 shutdown_connections 2024-04-25T13:45:15.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39efb57640 1 --2- 172.21.15.80:0/3444110067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39d4004810 0x7f39d4004c20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39efb57640 1 -- 172.21.15.80:0/3444110067 >> 172.21.15.80:0/3444110067 conn(0x7f39e80b4160 msgr2=0x7f39e80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39efb57640 1 -- 172.21.15.80:0/3444110067 shutdown_connections 2024-04-25T13:45:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.575+0000 7f39efb57640 1 -- 172.21.15.80:0/3444110067 wait complete. 2024-04-25T13:45:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 Processor -- start 2024-04-25T13:45:15.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 -- start start 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4004810 0x7f39e8154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39e8151e70 0x7f39e814a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39e8156440 0x7f39e8156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f39e815cb20 con 0x7f39e8156440 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8073ee0 con 0x7f39d4004810 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f39e8147ee0 con 0x7f39e8151e70 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39e8156440 0x7f39e8156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39e8156440 0x7f39e8156850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50192/0 (socket says 172.21.15.80:50192) 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 -- 172.21.15.80:0/680882344 learned_addr learned my addr 172.21.15.80:0/680882344 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:15.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ed0f2640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39e8151e70 0x7f39e814a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 -- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39e8151e70 msgr2=0x7f39e814a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39e8151e70 0x7f39e814a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 -- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4004810 msgr2=0x7f39e8154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ed8f3640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4004810 0x7f39e8154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4004810 0x7f39e8154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 -- 172.21.15.80:0/680882344 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e815ef50 con 0x7f39e8156440 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ed8f3640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4004810 0x7f39e8154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39ee0f4640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39e8156440 0x7f39e8156850 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f39e0001360 tx=0x7f39e0002200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39de7fc640 1 -- 172.21.15.80:0/680882344 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0004d70 con 0x7f39e8156440 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39de7fc640 1 -- 172.21.15.80:0/680882344 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39e000bb10 con 0x7f39e8156440 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e8156d90 con 0x7f39e8156440 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.579+0000 7f39de7fc640 1 -- 172.21.15.80:0/680882344 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0007070 con 0x7f39e8156440 2024-04-25T13:45:15.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.583+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e815d3f0 con 0x7f39e8156440 2024-04-25T13:45:15.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.583+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e815cea0 con 0x7f39e8156440 2024-04-25T13:45:15.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.583+0000 7f39de7fc640 1 -- 172.21.15.80:0/680882344 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f39e001b020 con 0x7f39e8156440 2024-04-25T13:45:15.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.583+0000 7f39de7fc640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f39cc06f400 0x7f39cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.583+0000 7f39de7fc640 1 -- 172.21.15.80:0/680882344 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f39e008f370 con 0x7f39e8156440 2024-04-25T13:45:15.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.583+0000 7f39de7fc640 1 -- 172.21.15.80:0/680882344 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f39cc076fe0 con 0x7f39e8156440 2024-04-25T13:45:15.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.583+0000 7f39ed8f3640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f39cc06f400 0x7f39cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.583+0000 7f39ed8f3640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f39cc06f400 0x7f39cc071910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f39e8160290 tx=0x7f39e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.587+0000 7f39de7fc640 1 -- 172.21.15.80:0/680882344 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f39e8156d90 con 0x7f39e8156440 2024-04-25T13:45:15.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f39e8152370 con 0x7f39e8156440 2024-04-25T13:45:15.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39de7fc640 1 -- 172.21.15.80:0/680882344 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f39e815ef50 con 0x7f39e8156440 2024-04-25T13:45:15.746 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f39cc06f400 msgr2=0x7f39cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f39cc06f400 0x7f39cc071910 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f39e8160290 tx=0x7f39e4009040 comp rx=0 tx=0).stop 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39e8156440 msgr2=0x7f39e8156850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39e8156440 0x7f39e8156850 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f39e0001360 tx=0x7f39e0002200 comp rx=0 tx=0).stop 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 shutdown_connections 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f39cc06f400 0x7f39cc071910 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39e8156440 0x7f39e8156850 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39e8151e70 0x7f39e814a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 --2- 172.21.15.80:0/680882344 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f39d4004810 0x7f39e8154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 >> 172.21.15.80:0/680882344 conn(0x7f39e80b4160 msgr2=0x7f39e80d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 shutdown_connections 2024-04-25T13:45:15.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.747+0000 7f39efb57640 1 -- 172.21.15.80:0/680882344 wait complete. 2024-04-25T13:45:15.766 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:45:15.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.903+0000 7f0d3dfd8640 1 Processor -- start 2024-04-25T13:45:15.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.903+0000 7f0d3dfd8640 1 -- start start 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d3dfd8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d38162270 0x7f0d3806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d3dfd8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d380b5740 con 0x7f0d380b5330 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d3dfd8640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d3806dd10 con 0x7f0d3806edb0 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d3dfd8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0d3806f700 con 0x7f0d38162270 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d377fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0d380b5330 0x7f0d380d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38574/0 (socket says 172.21.15.80:38574) 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d377fe640 1 -- 172.21.15.80:0/3646086433 learned_addr learned my addr 172.21.15.80:0/3646086433 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d37fff640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d38162270 0x7f0d3806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 864202042 0 0) 0x7f0d380b5740 con 0x7f0d380b5330 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d24003530 con 0x7f0d380b5330 2024-04-25T13:45:15.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 56722677 0 0) 0x7f0d24003530 con 0x7f0d380b5330 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d38162270 msgr2=0x7f0d3806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d38162270 0x7f0d3806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 >> v1:172.21.15.87:6789/0 conn(0x7f0d3806edb0 legacy=0x7f0d38161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d3806dff0 con 0x7f0d380b5330 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0d28001680 con 0x7f0d380b5330 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 >> v1:172.21.15.80:6789/0 conn(0x7f0d380b5330 legacy=0x7f0d380d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d240039f0 0x7f0d24003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d24004810 0x7f0d24004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d24005160 0x7f0d2400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0d3806f700 con 0x7f0d240039f0 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0d2400bfc0 con 0x7f0d24005160 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0d2400c1c0 con 0x7f0d24004810 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d377fe640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d240039f0 0x7f0d24003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d2effd640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d24004810 0x7f0d24004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d37fff640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d24005160 0x7f0d2400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d377fe640 1 -- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d24004810 msgr2=0x7f0d24004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:15.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d377fe640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d24004810 0x7f0d24004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d377fe640 1 -- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d24005160 msgr2=0x7f0d2400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:15.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d377fe640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d24005160 0x7f0d2400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d377fe640 1 -- 172.21.15.80:0/3646086433 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d24003530 con 0x7f0d240039f0 2024-04-25T13:45:15.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.907+0000 7f0d2effd640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d24004810 0x7f0d24004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:15.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d377fe640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d240039f0 0x7f0d24003e60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800c240 tx=0x7f0d2801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef2b2d443acedec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28007d20 con 0x7f0d240039f0 2024-04-25T13:45:15.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d280a9070 con 0x7f0d240039f0 2024-04-25T13:45:15.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d36ffd640 1 -- 172.21.15.80:0/3646086433 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28005d60 con 0x7f0d240039f0 2024-04-25T13:45:15.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3646086433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d240039f0 msgr2=0x7f0d24003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d3dfd8640 1 --2- 172.21.15.80:0/3646086433 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d240039f0 0x7f0d24003e60 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f0d2800c240 tx=0x7f0d2801cc80 comp rx=0 tx=0).stop 2024-04-25T13:45:15.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d37fff640 1 -- 172.21.15.80:0/3646086433 reap_dead start 2024-04-25T13:45:15.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3646086433 shutdown_connections 2024-04-25T13:45:15.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3646086433 >> 172.21.15.80:0/3646086433 conn(0x7f0d380b4160 msgr2=0x7f0d380d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:15.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3646086433 shutdown_connections 2024-04-25T13:45:15.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3646086433 wait complete. 2024-04-25T13:45:15.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d3dfd8640 1 Processor -- start 2024-04-25T13:45:15.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.911+0000 7f0d3dfd8640 1 -- start start 2024-04-25T13:45:15.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d380ac290 0x7f0d3815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d381447c0 0x7f0d3815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d38144bd0 0x7f0d3815b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0d380b0e50 con 0x7f0d380ac290 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0d380b0cd0 con 0x7f0d38144bd0 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0d380d11c0 con 0x7f0d381447c0 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d380ac290 0x7f0d3815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d380ac290 0x7f0d3815a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50218/0 (socket says 172.21.15.80:50218) 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 -- 172.21.15.80:0/3058601906 learned_addr learned my addr 172.21.15.80:0/3058601906 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d2effd640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d381447c0 0x7f0d3815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d37fff640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d38144bd0 0x7f0d3815b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 -- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d381447c0 msgr2=0x7f0d3815b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d381447c0 0x7f0d3815b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 -- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d38144bd0 msgr2=0x7f0d3815b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d38144bd0 0x7f0d3815b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 -- 172.21.15.80:0/3058601906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d380ac6a0 con 0x7f0d380ac290 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d37fff640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d38144bd0 0x7f0d3815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d377fe640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d380ac290 0x7f0d3815a260 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0d2801cab0 tx=0x7f0d24005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d34ff9640 1 -- 172.21.15.80:0/3058601906 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28007d20 con 0x7f0d380ac290 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d34ff9640 1 -- 172.21.15.80:0/3058601906 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d280048b0 con 0x7f0d380ac290 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d380ac920 con 0x7f0d380ac290 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d34ff9640 1 -- 172.21.15.80:0/3058601906 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d28004c60 con 0x7f0d380ac290 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d380acee0 con 0x7f0d380ac290 2024-04-25T13:45:15.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d3806a3a0 con 0x7f0d380ac290 2024-04-25T13:45:15.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d34ff9640 1 -- 172.21.15.80:0/3058601906 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0d28007180 con 0x7f0d380ac290 2024-04-25T13:45:15.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d34ff9640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0d1806f400 0x7f0d18071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:15.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d34ff9640 1 -- 172.21.15.80:0/3058601906 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f0d281289e0 con 0x7f0d380ac290 2024-04-25T13:45:15.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d34ff9640 1 -- 172.21.15.80:0/3058601906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f0d18076f80 con 0x7f0d380ac290 2024-04-25T13:45:15.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.915+0000 7f0d2effd640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0d1806f400 0x7f0d18071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:15.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.919+0000 7f0d2effd640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0d1806f400 0x7f0d18071910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0d20000f70 tx=0x7f0d20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:15.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:15.923+0000 7f0d34ff9640 1 -- 172.21.15.80:0/3058601906 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0d380ac6a0 con 0x7f0d380ac290 2024-04-25T13:45:16.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.067+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f0d3806fd00 con 0x7f0d380ac290 2024-04-25T13:45:16.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.067+0000 7f0d34ff9640 1 -- 172.21.15.80:0/3058601906 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f0d28016990 con 0x7f0d380ac290 2024-04-25T13:45:16.067 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:45:16.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0d1806f400 msgr2=0x7f0d18071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0d1806f400 0x7f0d18071910 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0d20000f70 tx=0x7f0d20009040 comp rx=0 tx=0).stop 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d380ac290 msgr2=0x7f0d3815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d380ac290 0x7f0d3815a260 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f0d2801cab0 tx=0x7f0d24005840 comp rx=0 tx=0).stop 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 shutdown_connections 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0d1806f400 0x7f0d18071910 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0d38144bd0 0x7f0d3815b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0d381447c0 0x7f0d3815b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 --2- 172.21.15.80:0/3058601906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0d380ac290 0x7f0d3815a260 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 >> 172.21.15.80:0/3058601906 conn(0x7f0d380b4160 msgr2=0x7f0d380b5740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 shutdown_connections 2024-04-25T13:45:16.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.071+0000 7f0d3dfd8640 1 -- 172.21.15.80:0/3058601906 wait complete. 2024-04-25T13:45:16.088 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:45:16.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f61962b0640 1 Processor -- start 2024-04-25T13:45:16.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f61962b0640 1 -- start start 2024-04-25T13:45:16.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f61962b0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6190071e60 0x7f61901574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f61962b0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6190073c40 0x7f6190160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f61962b0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61901613b0 con 0x7f6190160fa0 2024-04-25T13:45:16.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f61962b0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f61900b1350 con 0x7f6190073c40 2024-04-25T13:45:16.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f61962b0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f619006f9a0 con 0x7f6190071e60 2024-04-25T13:45:16.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f618f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6190073c40 0x7f6190160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f619484d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6190160fa0 0x7f61900b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38586/0 (socket says 172.21.15.80:38586) 2024-04-25T13:45:16.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f619484d640 1 -- 172.21.15.80:0/517211490 learned_addr learned my addr 172.21.15.80:0/517211490 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:16.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f618ffff640 1 --2- 172.21.15.80:0/517211490 >> v2:172.21.15.87:3300/0 conn(0x7f6190071e60 0x7f61901574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f618f7fe640 1 -- 172.21.15.80:0/517211490 >> v2:172.21.15.87:3300/0 conn(0x7f6190071e60 msgr2=0x7f61901574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:16.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f618f7fe640 1 --2- 172.21.15.80:0/517211490 >> v2:172.21.15.87:3300/0 conn(0x7f6190071e60 0x7f61901574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f618f7fe640 1 -- 172.21.15.80:0/517211490 >> v1:172.21.15.80:6789/0 conn(0x7f6190160fa0 legacy=0x7f61900b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f618f7fe640 1 -- 172.21.15.80:0/517211490 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619006fbf0 con 0x7f6190073c40 2024-04-25T13:45:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.227+0000 7f618effd640 1 -- 172.21.15.80:0/517211490 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2194730442 0 0) 0x7f6188001120 con 0x7f6190160fa0 2024-04-25T13:45:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f618f7fe640 1 --2- 172.21.15.80:0/517211490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f6190160a60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6190071630 tx=0x7f617801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=280f69d6f79e498a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f618effd640 1 -- 172.21.15.80:0/517211490 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61780b0070 con 0x7f6190073c40 2024-04-25T13:45:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f618effd640 1 -- 172.21.15.80:0/517211490 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61780ab600 con 0x7f6190073c40 2024-04-25T13:45:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f618effd640 1 -- 172.21.15.80:0/517211490 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61780ab9d0 con 0x7f6190073c40 2024-04-25T13:45:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f61962b0640 1 -- 172.21.15.80:0/517211490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 msgr2=0x7f6190160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f61962b0640 1 --2- 172.21.15.80:0/517211490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f6190160a60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f6190071630 tx=0x7f617801c480 comp rx=0 tx=0).stop 2024-04-25T13:45:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f61962b0640 1 -- 172.21.15.80:0/517211490 shutdown_connections 2024-04-25T13:45:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f61962b0640 1 --2- 172.21.15.80:0/517211490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f6190160a60 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f61962b0640 1 --2- 172.21.15.80:0/517211490 >> v2:172.21.15.87:3300/0 conn(0x7f6190071e60 0x7f61901574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f61962b0640 1 -- 172.21.15.80:0/517211490 >> 172.21.15.80:0/517211490 conn(0x7f61900b4160 msgr2=0x7f61900d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f61962b0640 1 -- 172.21.15.80:0/517211490 shutdown_connections 2024-04-25T13:45:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.231+0000 7f61962b0640 1 -- 172.21.15.80:0/517211490 wait complete. 2024-04-25T13:45:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 Processor -- start 2024-04-25T13:45:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 -- start start 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6190071e60 0x7f6190151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f619015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6190160fa0 0x7f619015fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6190074050 con 0x7f6190073c40 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6190072270 con 0x7f6190160fa0 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f61900b56c0 con 0x7f6190071e60 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f619015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f619015f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50242/0 (socket says 172.21.15.80:50242) 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 -- 172.21.15.80:0/1693973521 learned_addr learned my addr 172.21.15.80:0/1693973521 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618ffff640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6190071e60 0x7f6190151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f619484d640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6190160fa0 0x7f619015fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 -- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6190071e60 msgr2=0x7f6190151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6190071e60 0x7f6190151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 -- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6190160fa0 msgr2=0x7f619015fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6190160fa0 0x7f619015fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 -- 172.21.15.80:0/1693973521 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6190153520 con 0x7f6190073c40 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618ffff640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6190071e60 0x7f6190151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f619484d640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6190160fa0 0x7f619015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618f7fe640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f619015f7e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f617801a0b0 tx=0x7f6178019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618cff9640 1 -- 172.21.15.80:0/1693973521 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61780b0040 con 0x7f6190073c40 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618cff9640 1 -- 172.21.15.80:0/1693973521 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61780ba070 con 0x7f6190073c40 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6190151fd0 con 0x7f6190073c40 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f618cff9640 1 -- 172.21.15.80:0/1693973521 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61780b4870 con 0x7f6190073c40 2024-04-25T13:45:16.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901f64f0 con 0x7f6190073c40 2024-04-25T13:45:16.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.235+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61901f5f80 con 0x7f6190073c40 2024-04-25T13:45:16.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.239+0000 7f618cff9640 1 -- 172.21.15.80:0/1693973521 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f61780ab4c0 con 0x7f6190073c40 2024-04-25T13:45:16.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.239+0000 7f618cff9640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f617006f400 0x7f6170071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.239+0000 7f618cff9640 1 -- 172.21.15.80:0/1693973521 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5709+0+0 (secure 0 0 0) 0x7f6178129960 con 0x7f6190073c40 2024-04-25T13:45:16.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.239+0000 7f618cff9640 1 -- 172.21.15.80:0/1693973521 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=90}) -- 0x7f6170076f80 con 0x7f6190073c40 2024-04-25T13:45:16.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.239+0000 7f618ffff640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f617006f400 0x7f6170071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.239+0000 7f618ffff640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f617006f400 0x7f6170071910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6180009f50 tx=0x7f618000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.243+0000 7f618cff9640 1 -- 172.21.15.80:0/1693973521 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6190153520 con 0x7f6190073c40 2024-04-25T13:45:16.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.391+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f619006a3a0 con 0x7f6190073c40 2024-04-25T13:45:16.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.391+0000 7f618cff9640 1 -- 172.21.15.80:0/1693973521 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f6178016070 con 0x7f6190073c40 2024-04-25T13:45:16.392 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f617006f400 msgr2=0x7f6170071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f617006f400 0x7f6170071910 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6180009f50 tx=0x7f618000b040 comp rx=0 tx=0).stop 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 msgr2=0x7f619015f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f619015f7e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f617801a0b0 tx=0x7f6178019220 comp rx=0 tx=0).stop 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 shutdown_connections 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f617006f400 0x7f6170071910 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6190160fa0 0x7f619015fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6190073c40 0x7f619015f7e0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 --2- 172.21.15.80:0/1693973521 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6190071e60 0x7f6190151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 >> 172.21.15.80:0/1693973521 conn(0x7f61900b4160 msgr2=0x7f6190072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:16.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 shutdown_connections 2024-04-25T13:45:16.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.395+0000 7f61962b0640 1 -- 172.21.15.80:0/1693973521 wait complete. 2024-04-25T13:45:16.414 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.547+0000 7fb0875ad640 1 Processor -- start 2024-04-25T13:45:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.547+0000 7fb0875ad640 1 -- start start 2024-04-25T13:45:16.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb0875ad640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb080161630 0x7fb08006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb0875ad640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb08006cf90 con 0x7fb08015d630 2024-04-25T13:45:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb0875ad640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb08006d270 con 0x7fb0800cfb60 2024-04-25T13:45:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb0875ad640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb080143920 con 0x7fb080161630 2024-04-25T13:45:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb085349640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb080161630 0x7fb08006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb085b4a640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb08015d630 0x7fb08015da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38598/0 (socket says 172.21.15.80:38598) 2024-04-25T13:45:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb085b4a640 1 -- 172.21.15.80:0/2802069278 learned_addr learned my addr 172.21.15.80:0/2802069278 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb085349640 1 -- 172.21.15.80:0/2802069278 >> v1:172.21.15.87:6789/0 conn(0x7fb0800cfb60 legacy=0x7fb0800cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb085349640 1 -- 172.21.15.80:0/2802069278 >> v1:172.21.15.80:6789/0 conn(0x7fb08015d630 legacy=0x7fb08015da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb085349640 1 -- 172.21.15.80:0/2802069278 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb080160e30 con 0x7fb080161630 2024-04-25T13:45:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.551+0000 7fb085349640 1 --2- 172.21.15.80:0/2802069278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb08006c5f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0097c0 tx=0x7fb07c01ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef30ffe0b71d9fe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb077fff640 1 -- 172.21.15.80:0/2802069278 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0b5070 con 0x7fb080161630 2024-04-25T13:45:16.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb077fff640 1 -- 172.21.15.80:0/2802069278 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07c0a8950 con 0x7fb080161630 2024-04-25T13:45:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb077fff640 1 -- 172.21.15.80:0/2802069278 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07c0a8d20 con 0x7fb080161630 2024-04-25T13:45:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- 172.21.15.80:0/2802069278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 msgr2=0x7fb08006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 --2- 172.21.15.80:0/2802069278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb08006c5f0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0097c0 tx=0x7fb07c01ec50 comp rx=0 tx=0).stop 2024-04-25T13:45:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- 172.21.15.80:0/2802069278 shutdown_connections 2024-04-25T13:45:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 --2- 172.21.15.80:0/2802069278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb08006c5f0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- 172.21.15.80:0/2802069278 >> 172.21.15.80:0/2802069278 conn(0x7fb0800d33a0 msgr2=0x7fb0800cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- 172.21.15.80:0/2802069278 shutdown_connections 2024-04-25T13:45:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- 172.21.15.80:0/2802069278 wait complete. 2024-04-25T13:45:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 Processor -- start 2024-04-25T13:45:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- start start 2024-04-25T13:45:16.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0800cfb60 0x7fb08013fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb08015d630 0x7fb08015a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb0800c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb080161a40 con 0x7fb080161630 2024-04-25T13:45:16.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb08015e150 con 0x7fb0800cfb60 2024-04-25T13:45:16.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.555+0000 7fb0875ad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb080158550 con 0x7fb08015d630 2024-04-25T13:45:16.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb0800c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb0800c7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50254/0 (socket says 172.21.15.80:50254) 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 -- 172.21.15.80:0/2372504483 learned_addr learned my addr 172.21.15.80:0/2372504483 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085349640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0800cfb60 0x7fb08013fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb084b48640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb08015d630 0x7fb08015a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 -- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb08015d630 msgr2=0x7fb08015a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb08015d630 0x7fb08015a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 -- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0800cfb60 msgr2=0x7fb08013fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0800cfb60 0x7fb08013fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 -- 172.21.15.80:0/2372504483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08015abb0 con 0x7fb080161630 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb084b48640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb08015d630 0x7fb08015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085b4a640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb0800c7da0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb07800caa0 tx=0x7fb07800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb075ffb640 1 -- 172.21.15.80:0/2372504483 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078017070 con 0x7fb080161630 2024-04-25T13:45:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb075ffb640 1 -- 172.21.15.80:0/2372504483 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb07800ac00 con 0x7fb080161630 2024-04-25T13:45:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb080156450 con 0x7fb080161630 2024-04-25T13:45:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb075ffb640 1 -- 172.21.15.80:0/2372504483 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb078012400 con 0x7fb080161630 2024-04-25T13:45:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb080156a10 con 0x7fb080161630 2024-04-25T13:45:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb08006a3a0 con 0x7fb080161630 2024-04-25T13:45:16.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb075ffb640 1 -- 172.21.15.80:0/2372504483 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb07801a070 con 0x7fb080161630 2024-04-25T13:45:16.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb075ffb640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb06406f400 0x7fb064071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb075ffb640 1 -- 172.21.15.80:0/2372504483 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fb07808ef10 con 0x7fb080161630 2024-04-25T13:45:16.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.559+0000 7fb085349640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb06406f400 0x7fb064071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.563+0000 7fb085349640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb06406f400 0x7fb064071910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0b2000 tx=0x7fb07c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.563+0000 7fb075ffb640 1 -- 172.21.15.80:0/2372504483 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb07805b7b0 con 0x7fb080161630 2024-04-25T13:45:16.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.751+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0800d0c40 con 0x7fb080161630 2024-04-25T13:45:16.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.751+0000 7fb075ffb640 1 -- 172.21.15.80:0/2372504483 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5180 (secure 0 0 0) 0x7fb07805fab0 con 0x7fb080161630 2024-04-25T13:45:16.752 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:16.752 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2082291876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2082291876},{"type":"v1","addr":"172.21.15.87:6835","nonce":2082291876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1337331528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1337331528},{"type":"v1","addr":"172.21.15.80:6833","nonce":1337331528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2521189874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2521189874},{"type":"v1","addr":"172.21.15.80:6835","nonce":2521189874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:44:35.958393+0000","modified":"2024-04-25T13:44:39.019142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/642879744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":642879744},{"type":"v1","addr":"172.21.15.87:6837","nonce":642879744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":5}]} 2024-04-25T13:45:16.753 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 41 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb06406f400 msgr2=0x7fb064071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb06406f400 0x7fb064071910 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fb07c0b2000 tx=0x7fb07c008040 comp rx=0 tx=0).stop 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 msgr2=0x7fb0800c7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb0800c7da0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fb07800caa0 tx=0x7fb07800cf60 comp rx=0 tx=0).stop 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 shutdown_connections 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb06406f400 0x7fb064071910 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb080161630 0x7fb0800c7da0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb08015d630 0x7fb08015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 --2- 172.21.15.80:0/2372504483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0800cfb60 0x7fb08013fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 >> 172.21.15.80:0/2372504483 conn(0x7fb0800d33a0 msgr2=0x7fb08015b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:16.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 shutdown_connections 2024-04-25T13:45:16.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.755+0000 7fb0875ad640 1 -- 172.21.15.80:0/2372504483 wait complete. 2024-04-25T13:45:16.773 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:16.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.923+0000 7fb34be4b640 1 Processor -- start 2024-04-25T13:45:16.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.923+0000 7fb34be4b640 1 -- start start 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb34be4b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344161f90 0x7fb3440cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb34be4b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3440d0510 0x7fb34415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb34be4b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb34415ea70 0x7fb34415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb34be4b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb3440d0d70 con 0x7fb3440d0510 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb34be4b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb3440d1050 con 0x7fb34415ea70 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb34be4b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb34406dca0 con 0x7fb344161f90 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb3493e6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3440d0510 0x7fb34415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb3493e6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3440d0510 0x7fb34415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50262/0 (socket says 172.21.15.80:50262) 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb3493e6640 1 -- 172.21.15.80:0/3498088095 learned_addr learned my addr 172.21.15.80:0/3498088095 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb34a3e8640 1 --2- 172.21.15.80:0/3498088095 >> v2:172.21.15.87:3300/0 conn(0x7fb34415ea70 0x7fb34415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb349be7640 1 --2- 172.21.15.80:0/3498088095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344161f90 0x7fb3440cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb3493e6640 1 -- 172.21.15.80:0/3498088095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344161f90 msgr2=0x7fb3440cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:16.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb3493e6640 1 --2- 172.21.15.80:0/3498088095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344161f90 0x7fb3440cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb3493e6640 1 -- 172.21.15.80:0/3498088095 >> v2:172.21.15.87:3300/0 conn(0x7fb34415ea70 msgr2=0x7fb34415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:16.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb3493e6640 1 --2- 172.21.15.80:0/3498088095 >> v2:172.21.15.87:3300/0 conn(0x7fb34415ea70 0x7fb34415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.927+0000 7fb3493e6640 1 -- 172.21.15.80:0/3498088095 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34406df80 con 0x7fb3440d0510 2024-04-25T13:45:16.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb3493e6640 1 --2- 172.21.15.80:0/3498088095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3440d0510 0x7fb34415e530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb3440d1260 tx=0x7fb33401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6305b1aa8991489a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb348be5640 1 -- 172.21.15.80:0/3498088095 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3340b1070 con 0x7fb3440d0510 2024-04-25T13:45:16.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb348be5640 1 -- 172.21.15.80:0/3498088095 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3340a8050 con 0x7fb3440d0510 2024-04-25T13:45:16.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb348be5640 1 -- 172.21.15.80:0/3498088095 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3340ac860 con 0x7fb3440d0510 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 -- 172.21.15.80:0/3498088095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3440d0510 msgr2=0x7fb34415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 --2- 172.21.15.80:0/3498088095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3440d0510 0x7fb34415e530 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fb3440d1260 tx=0x7fb33401c460 comp rx=0 tx=0).stop 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 -- 172.21.15.80:0/3498088095 shutdown_connections 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 --2- 172.21.15.80:0/3498088095 >> v2:172.21.15.87:3300/0 conn(0x7fb34415ea70 0x7fb34415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 --2- 172.21.15.80:0/3498088095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3440d0510 0x7fb34415e530 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 --2- 172.21.15.80:0/3498088095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb344161f90 0x7fb3440cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 -- 172.21.15.80:0/3498088095 >> 172.21.15.80:0/3498088095 conn(0x7fb34406d080 msgr2=0x7fb3440b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 -- 172.21.15.80:0/3498088095 shutdown_connections 2024-04-25T13:45:16.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 -- 172.21.15.80:0/3498088095 wait complete. 2024-04-25T13:45:16.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 Processor -- start 2024-04-25T13:45:16.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.931+0000 7fb34be4b640 1 -- start start 2024-04-25T13:45:16.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3440d0510 0x7fb34415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34415ea70 0x7fb34413f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb344161f90 0x7fb3441404f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb3441447a0 con 0x7fb344161f90 2024-04-25T13:45:16.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb344144620 con 0x7fb3440d0510 2024-04-25T13:45:16.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3440d0920 con 0x7fb34415ea70 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb344161f90 0x7fb3441404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb344161f90 0x7fb3441404f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50274/0 (socket says 172.21.15.80:50274) 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 -- 172.21.15.80:0/1096527679 learned_addr learned my addr 172.21.15.80:0/1096527679 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb349be7640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3440d0510 0x7fb34415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb3493e6640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34415ea70 0x7fb34413f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 -- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34415ea70 msgr2=0x7fb34413f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34415ea70 0x7fb34413f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 -- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3440d0510 msgr2=0x7fb34415d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3440d0510 0x7fb34415d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 -- 172.21.15.80:0/1096527679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb34413dca0 con 0x7fb344161f90 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34a3e8640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb344161f90 0x7fb3441404f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00ef00 tx=0x7fb33c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb33a7fc640 1 -- 172.21.15.80:0/1096527679 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c018020 con 0x7fb344161f90 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb33a7fc640 1 -- 172.21.15.80:0/1096527679 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb33c00bd20 con 0x7fb344161f90 2024-04-25T13:45:16.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb33a7fc640 1 -- 172.21.15.80:0/1096527679 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33c014640 con 0x7fb344161f90 2024-04-25T13:45:16.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb34413df40 con 0x7fb344161f90 2024-04-25T13:45:16.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb34413e500 con 0x7fb344161f90 2024-04-25T13:45:16.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.935+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34406a3a0 con 0x7fb344161f90 2024-04-25T13:45:16.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.939+0000 7fb33a7fc640 1 -- 172.21.15.80:0/1096527679 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb33c01b020 con 0x7fb344161f90 2024-04-25T13:45:16.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.939+0000 7fb33a7fc640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb32806f430 0x7fb328071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:16.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.939+0000 7fb33a7fc640 1 -- 172.21.15.80:0/1096527679 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7fb33c08f830 con 0x7fb344161f90 2024-04-25T13:45:16.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.939+0000 7fb349be7640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb32806f430 0x7fb328071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:16.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.939+0000 7fb349be7640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb32806f430 0x7fb328071940 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb34406fa20 tx=0x7fb340008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:16.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:16.943+0000 7fb33a7fc640 1 -- 172.21.15.80:0/1096527679 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb33c022070 con 0x7fb344161f90 2024-04-25T13:45:17.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3441615a0 con 0x7fb344161f90 2024-04-25T13:45:17.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb33a7fc640 1 -- 172.21.15.80:0/1096527679 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16596 (secure 0 0 0) 0x7fb33c05c680 con 0x7fb344161f90 2024-04-25T13:45:17.107 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":90,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:16.441559+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:44:33.879562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T13:44:34.932724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb32806f430 msgr2=0x7fb328071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb32806f430 0x7fb328071940 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb34406fa20 tx=0x7fb340008040 comp rx=0 tx=0).stop 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb344161f90 msgr2=0x7fb3441404f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb344161f90 0x7fb3441404f0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fb33c00ef00 tx=0x7fb33c00b480 comp rx=0 tx=0).stop 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 shutdown_connections 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb32806f430 0x7fb328071940 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb344161f90 0x7fb3441404f0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb34415ea70 0x7fb34413f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 --2- 172.21.15.80:0/1096527679 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3440d0510 0x7fb34415d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.107+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 >> 172.21.15.80:0/1096527679 conn(0x7fb34406d080 msgr2=0x7fb3440757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.111+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 shutdown_connections 2024-04-25T13:45:17.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.111+0000 7fb34be4b640 1 -- 172.21.15.80:0/1096527679 wait complete. 2024-04-25T13:45:17.129 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:45:17.129 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:17.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.271+0000 7f5737b57640 1 Processor -- start 2024-04-25T13:45:17.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.271+0000 7f5737b57640 1 -- start start 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f5737b57640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57300d2e80 0x7f57301607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f5737b57640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57300d59b0 0x7f57300d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f5737b57640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57300d6300 con 0x7f57300d4330 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f5737b57640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5730161060 con 0x7f57300d2e80 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f5737b57640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5730161340 con 0x7f57300d59b0 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f57358f3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57300d2e80 0x7f57301607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f57358f3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57300d2e80 0x7f57301607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50282/0 (socket says 172.21.15.80:50282) 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f57358f3640 1 -- 172.21.15.80:0/2363948611 learned_addr learned my addr 172.21.15.80:0/2363948611 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f57350f2640 1 --2- 172.21.15.80:0/2363948611 >> v2:172.21.15.87:3300/0 conn(0x7f57300d59b0 0x7f57300d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f57358f3640 1 -- 172.21.15.80:0/2363948611 >> v2:172.21.15.87:3300/0 conn(0x7f57300d59b0 msgr2=0x7f57300d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f57358f3640 1 --2- 172.21.15.80:0/2363948611 >> v2:172.21.15.87:3300/0 conn(0x7f57300d59b0 0x7f57300d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f57358f3640 1 -- 172.21.15.80:0/2363948611 >> v1:172.21.15.87:6789/0 conn(0x7f57300d4330 legacy=0x7f57300d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:17.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.275+0000 7f57358f3640 1 -- 172.21.15.80:0/2363948611 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57300d2680 con 0x7f57300d2e80 2024-04-25T13:45:17.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f57358f3640 1 --2- 172.21.15.80:0/2363948611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d2e80 0x7f57301607a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f573008ec00 tx=0x7f572001c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68062278356b11a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:17.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f57348f1640 1 -- 172.21.15.80:0/2363948611 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57200a5510 con 0x7f57300d2e80 2024-04-25T13:45:17.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f57348f1640 1 -- 172.21.15.80:0/2363948611 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57200a56b0 con 0x7f57300d2e80 2024-04-25T13:45:17.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f57348f1640 1 -- 172.21.15.80:0/2363948611 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57200a5a80 con 0x7f57300d2e80 2024-04-25T13:45:17.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 -- 172.21.15.80:0/2363948611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d2e80 msgr2=0x7f57301607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 --2- 172.21.15.80:0/2363948611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d2e80 0x7f57301607a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f573008ec00 tx=0x7f572001c740 comp rx=0 tx=0).stop 2024-04-25T13:45:17.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 -- 172.21.15.80:0/2363948611 shutdown_connections 2024-04-25T13:45:17.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 --2- 172.21.15.80:0/2363948611 >> v2:172.21.15.87:3300/0 conn(0x7f57300d59b0 0x7f57300d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 --2- 172.21.15.80:0/2363948611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d2e80 0x7f57301607a0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 -- 172.21.15.80:0/2363948611 >> 172.21.15.80:0/2363948611 conn(0x7f57300737a0 msgr2=0x7f573006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:17.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 -- 172.21.15.80:0/2363948611 shutdown_connections 2024-04-25T13:45:17.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 -- 172.21.15.80:0/2363948611 wait complete. 2024-04-25T13:45:17.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 Processor -- start 2024-04-25T13:45:17.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.279+0000 7f5737b57640 1 -- start start 2024-04-25T13:45:17.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d4330 0x7f57300bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57300d59b0 0x7f57301520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573014db40 0x7f57300b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57300d3410 con 0x7f57300d4330 2024-04-25T13:45:17.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57300d3290 con 0x7f57300d59b0 2024-04-25T13:45:17.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f573015df60 con 0x7f573014db40 2024-04-25T13:45:17.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d4330 0x7f57300bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d4330 0x7f57300bb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50296/0 (socket says 172.21.15.80:50296) 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 -- 172.21.15.80:0/2161152499 learned_addr learned my addr 172.21.15.80:0/2161152499 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 -- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573014db40 msgr2=0x7f57300b93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573014db40 0x7f57300b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57350f2640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57300d59b0 0x7f57301520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 -- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57300d59b0 msgr2=0x7f57301520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57300d59b0 0x7f57301520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 -- 172.21.15.80:0/2161152499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5730158370 con 0x7f57300d4330 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57358f3640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d4330 0x7f57300bb2d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f572001adf0 tx=0x7f57200194e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f572e7fc640 1 -- 172.21.15.80:0/2161152499 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57200b8470 con 0x7f57300d4330 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f572e7fc640 1 -- 172.21.15.80:0/2161152499 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57200be070 con 0x7f57300d4330 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5730158650 con 0x7f57300d4330 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f572e7fc640 1 -- 172.21.15.80:0/2161152499 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57200b8890 con 0x7f57300d4330 2024-04-25T13:45:17.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57300bb810 con 0x7f57300d4330 2024-04-25T13:45:17.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f573006a3a0 con 0x7f57300d4330 2024-04-25T13:45:17.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f572e7fc640 1 -- 172.21.15.80:0/2161152499 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57200b8a30 con 0x7f57300d4330 2024-04-25T13:45:17.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f572e7fc640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571c06f400 0x7f571c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f572e7fc640 1 -- 172.21.15.80:0/2161152499 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5709+0+0 (secure 0 0 0) 0x7f572012ac40 con 0x7f57300d4330 2024-04-25T13:45:17.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.283+0000 7f57350f2640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571c06f400 0x7f571c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.287+0000 7f57350f2640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571c06f400 0x7f571c071910 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5730163010 tx=0x7f5724008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:17.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.287+0000 7f572e7fc640 1 -- 172.21.15.80:0/2161152499 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57300d6300 con 0x7f57300d4330 2024-04-25T13:45:17.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.475+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57300d4110 con 0x7f57300d4330 2024-04-25T13:45:17.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.475+0000 7f572e7fc640 1 -- 172.21.15.80:0/2161152499 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5180 (secure 0 0 0) 0x7f57200f7ca0 con 0x7f57300d4330 2024-04-25T13:45:17.476 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:17.476 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2082291876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2082291876},{"type":"v1","addr":"172.21.15.87:6835","nonce":2082291876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1337331528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1337331528},{"type":"v1","addr":"172.21.15.80:6833","nonce":1337331528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2521189874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2521189874},{"type":"v1","addr":"172.21.15.80:6835","nonce":2521189874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:44:35.958393+0000","modified":"2024-04-25T13:44:39.019142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/642879744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":642879744},{"type":"v1","addr":"172.21.15.87:6837","nonce":642879744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":5}]} 2024-04-25T13:45:17.476 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 41 2024-04-25T13:45:17.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571c06f400 msgr2=0x7f571c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571c06f400 0x7f571c071910 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5730163010 tx=0x7f5724008040 comp rx=0 tx=0).stop 2024-04-25T13:45:17.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d4330 msgr2=0x7f57300bb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d4330 0x7f57300bb2d0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f572001adf0 tx=0x7f57200194e0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 shutdown_connections 2024-04-25T13:45:17.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f571c06f400 0x7f571c071910 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f573014db40 0x7f57300b93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57300d59b0 0x7f57301520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 --2- 172.21.15.80:0/2161152499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57300d4330 0x7f57300bb2d0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 >> 172.21.15.80:0/2161152499 conn(0x7f57300737a0 msgr2=0x7f5730162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:17.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 shutdown_connections 2024-04-25T13:45:17.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.479+0000 7f5737b57640 1 -- 172.21.15.80:0/2161152499 wait complete. 2024-04-25T13:45:17.496 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:17.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.639+0000 7fa53bfc4640 1 Processor -- start 2024-04-25T13:45:17.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53bfc4640 1 -- start start 2024-04-25T13:45:17.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53bfc4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa5340b4eb0 0x7fa5341603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53bfc4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa5340b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53bfc4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5340b80d0 con 0x7fa5340d0ab0 2024-04-25T13:45:17.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53bfc4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa5340ba400 con 0x7fa5340b4eb0 2024-04-25T13:45:17.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53bfc4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa5340d48a0 con 0x7fa5340b7780 2024-04-25T13:45:17.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa539d60640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa5340b4eb0 0x7fa5341603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53a561640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa5340d0ab0 0x7fa5340d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38608/0 (socket says 172.21.15.80:38608) 2024-04-25T13:45:17.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53a561640 1 -- 172.21.15.80:0/1525670478 learned_addr learned my addr 172.21.15.80:0/1525670478 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:17.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53955f640 1 --2- 172.21.15.80:0/1525670478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa5340b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa539d60640 1 -- 172.21.15.80:0/1525670478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 msgr2=0x7fa5340b7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:17.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa539d60640 1 --2- 172.21.15.80:0/1525670478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa5340b7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa539d60640 1 -- 172.21.15.80:0/1525670478 >> v1:172.21.15.80:6789/0 conn(0x7fa5340d0ab0 legacy=0x7fa5340d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa539d60640 1 -- 172.21.15.80:0/1525670478 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5340d4b80 con 0x7fa5340b4eb0 2024-04-25T13:45:17.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.643+0000 7fa53955f640 1 --2- 172.21.15.80:0/1525670478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa5340b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:17.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa539d60640 1 --2- 172.21.15.80:0/1525670478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341603c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa5300097c0 tx=0x7fa53001c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1e6ae7257f57636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:17.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa538d5e640 1 -- 172.21.15.80:0/1525670478 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5300b0070 con 0x7fa5340b4eb0 2024-04-25T13:45:17.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa538d5e640 1 -- 172.21.15.80:0/1525670478 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5300ab600 con 0x7fa5340b4eb0 2024-04-25T13:45:17.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa538d5e640 1 -- 172.21.15.80:0/1525670478 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5300ab9d0 con 0x7fa5340b4eb0 2024-04-25T13:45:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 -- 172.21.15.80:0/1525670478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 msgr2=0x7fa5341603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/1525670478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341603c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa5300097c0 tx=0x7fa53001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 -- 172.21.15.80:0/1525670478 shutdown_connections 2024-04-25T13:45:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/1525670478 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa5340b7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/1525670478 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341603c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 -- 172.21.15.80:0/1525670478 >> 172.21.15.80:0/1525670478 conn(0x7fa53406c9f0 msgr2=0x7fa53406ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:17.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 -- 172.21.15.80:0/1525670478 shutdown_connections 2024-04-25T13:45:17.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 -- 172.21.15.80:0/1525670478 wait complete. 2024-04-25T13:45:17.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 Processor -- start 2024-04-25T13:45:17.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.647+0000 7fa53bfc4640 1 -- start start 2024-04-25T13:45:17.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa534162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5340d0ab0 0x7fa53415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa5340d5940 con 0x7fa5340b4eb0 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa534160900 con 0x7fa5340d0ab0 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa534162260 con 0x7fa5340b7780 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50314/0 (socket says 172.21.15.80:50314) 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 -- 172.21.15.80:0/3299795743 learned_addr learned my addr 172.21.15.80:0/3299795743 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53a561640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5340d0ab0 0x7fa53415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53955f640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa534162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 -- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 msgr2=0x7fa534162ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa534162ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 -- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5340d0ab0 msgr2=0x7fa53415d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5340d0ab0 0x7fa53415d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 -- 172.21.15.80:0/3299795743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5341569e0 con 0x7fa5340b4eb0 2024-04-25T13:45:17.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53a561640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5340d0ab0 0x7fa53415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:17.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa539d60640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341561b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa53001a0d0 tx=0x7fa530019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:17.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa52a7fc640 1 -- 172.21.15.80:0/3299795743 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5300b0040 con 0x7fa5340b4eb0 2024-04-25T13:45:17.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa52a7fc640 1 -- 172.21.15.80:0/3299795743 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5300b4410 con 0x7fa5340b4eb0 2024-04-25T13:45:17.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa534156cc0 con 0x7fa5340b4eb0 2024-04-25T13:45:17.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa52a7fc640 1 -- 172.21.15.80:0/3299795743 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5300b47e0 con 0x7fa5340b4eb0 2024-04-25T13:45:17.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa534157280 con 0x7fa5340b4eb0 2024-04-25T13:45:17.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa53406a3a0 con 0x7fa5340b4eb0 2024-04-25T13:45:17.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa52a7fc640 1 -- 172.21.15.80:0/3299795743 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa5300bc020 con 0x7fa5340b4eb0 2024-04-25T13:45:17.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa52a7fc640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa51806f400 0x7fa518071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa52a7fc640 1 -- 172.21.15.80:0/3299795743 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5767+0+0 (secure 0 0 0) 0x7fa530128750 con 0x7fa5340b4eb0 2024-04-25T13:45:17.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.651+0000 7fa53955f640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa51806f400 0x7fa518071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.655+0000 7fa53955f640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa51806f400 0x7fa518071910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa524006fd0 tx=0x7fa524008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:17.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.655+0000 7fa52a7fc640 1 -- 172.21.15.80:0/3299795743 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5300c3020 con 0x7fa5340b4eb0 2024-04-25T13:45:17.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa534160b40 con 0x7fa5340b4eb0 2024-04-25T13:45:17.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa52a7fc640 1 -- 172.21.15.80:0/3299795743 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5180 (secure 0 0 0) 0x7fa530016090 con 0x7fa5340b4eb0 2024-04-25T13:45:17.838 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:17.838 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":41,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2082291876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2082291876},{"type":"v1","addr":"172.21.15.87:6835","nonce":2082291876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1337331528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1337331528},{"type":"v1","addr":"172.21.15.80:6833","nonce":1337331528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2521189874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2521189874},{"type":"v1","addr":"172.21.15.80:6835","nonce":2521189874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:44:35.958393+0000","modified":"2024-04-25T13:44:39.019142+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6627},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6627":{"gid":6627,"name":"b","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/642879744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":642879744},{"type":"v1","addr":"172.21.15.87:6837","nonce":642879744}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6627,"qdb_cluster":[6627]},"id":5}]} 2024-04-25T13:45:17.838 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 41 2024-04-25T13:45:17.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa51806f400 msgr2=0x7fa518071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa51806f400 0x7fa518071910 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa524006fd0 tx=0x7fa524008040 comp rx=0 tx=0).stop 2024-04-25T13:45:17.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 msgr2=0x7fa5341561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341561b0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa53001a0d0 tx=0x7fa530019240 comp rx=0 tx=0).stop 2024-04-25T13:45:17.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 shutdown_connections 2024-04-25T13:45:17.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa51806f400 0x7fa518071910 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5340d0ab0 0x7fa53415d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5340b7780 0x7fa534162ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 --2- 172.21.15.80:0/3299795743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5340b4eb0 0x7fa5341561b0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 >> 172.21.15.80:0/3299795743 conn(0x7fa53406c9f0 msgr2=0x7fa5340d6450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:17.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 shutdown_connections 2024-04-25T13:45:17.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.839+0000 7fa53bfc4640 1 -- 172.21.15.80:0/3299795743 wait complete. 2024-04-25T13:45:17.859 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:17.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.991+0000 7feba7fa5640 1 Processor -- start 2024-04-25T13:45:17.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.991+0000 7feba7fa5640 1 -- start start 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba7fa5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7feba00d2e80 0x7feba01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba7fa5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feba00d59b0 0x7feba00d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba7fa5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feba00d6300 con 0x7feba00d4330 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba7fa5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7feba0161060 con 0x7feba00d2e80 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba7fa5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feba0161340 con 0x7feba00d59b0 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba5d41640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7feba00d2e80 0x7feba01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba6542640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7feba00d4330 0x7feba00d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38612/0 (socket says 172.21.15.80:38612) 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba6542640 1 -- 172.21.15.80:0/3034601951 learned_addr learned my addr 172.21.15.80:0/3034601951 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba5540640 1 --2- 172.21.15.80:0/3034601951 >> v2:172.21.15.87:3300/0 conn(0x7feba00d59b0 0x7feba00d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba5d41640 1 -- 172.21.15.80:0/3034601951 >> v2:172.21.15.87:3300/0 conn(0x7feba00d59b0 msgr2=0x7feba00d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba5d41640 1 --2- 172.21.15.80:0/3034601951 >> v2:172.21.15.87:3300/0 conn(0x7feba00d59b0 0x7feba00d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba5d41640 1 -- 172.21.15.80:0/3034601951 >> v1:172.21.15.80:6789/0 conn(0x7feba00d4330 legacy=0x7feba00d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba5d41640 1 -- 172.21.15.80:0/3034601951 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba00d2680 con 0x7feba00d2e80 2024-04-25T13:45:17.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.995+0000 7feba4d3f640 1 -- 172.21.15.80:0/3034601951 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1270530964 0 0) 0x7feb980014a0 con 0x7feba00d4330 2024-04-25T13:45:17.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba5d41640 1 --2- 172.21.15.80:0/3034601951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba01607a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7feba00c9670 tx=0x7feb9c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63ec2f143e5e8685 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:17.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba4d3f640 1 -- 172.21.15.80:0/3034601951 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0b0070 con 0x7feba00d2e80 2024-04-25T13:45:17.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba4d3f640 1 -- 172.21.15.80:0/3034601951 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c0ab600 con 0x7feba00d2e80 2024-04-25T13:45:17.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba4d3f640 1 -- 172.21.15.80:0/3034601951 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0ab9d0 con 0x7feba00d2e80 2024-04-25T13:45:17.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 -- 172.21.15.80:0/3034601951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 msgr2=0x7feba01607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:17.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 --2- 172.21.15.80:0/3034601951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba01607a0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7feba00c9670 tx=0x7feb9c01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 -- 172.21.15.80:0/3034601951 shutdown_connections 2024-04-25T13:45:17.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 --2- 172.21.15.80:0/3034601951 >> v2:172.21.15.87:3300/0 conn(0x7feba00d59b0 0x7feba00d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 --2- 172.21.15.80:0/3034601951 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba01607a0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:17.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 -- 172.21.15.80:0/3034601951 >> 172.21.15.80:0/3034601951 conn(0x7feba00737a0 msgr2=0x7feba006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:17.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 -- 172.21.15.80:0/3034601951 shutdown_connections 2024-04-25T13:45:17.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 -- 172.21.15.80:0/3034601951 wait complete. 2024-04-25T13:45:18.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 Processor -- start 2024-04-25T13:45:18.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:17.999+0000 7feba7fa5640 1 -- start start 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba0155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feba00d4330 0x7feba0156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feba00d59b0 0x7feba00cef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feba00d3410 con 0x7feba00d2e80 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feba00d3290 con 0x7feba00d4330 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feba015df60 con 0x7feba00d59b0 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba0155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba0155c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50340/0 (socket says 172.21.15.80:50340) 2024-04-25T13:45:18.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 -- 172.21.15.80:0/752646315 learned_addr learned my addr 172.21.15.80:0/752646315 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5540640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feba00d4330 0x7feba0156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 -- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feba00d59b0 msgr2=0x7feba00cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba6542640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feba00d59b0 0x7feba00cef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feba00d59b0 0x7feba00cef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 -- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feba00d4330 msgr2=0x7feba0156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feba00d4330 0x7feba0156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 -- 172.21.15.80:0/752646315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba00ce8a0 con 0x7feba00d2e80 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5540640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feba00d4330 0x7feba0156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba5d41640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba0155c30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0ae040 tx=0x7feb9c01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feb967fc640 1 -- 172.21.15.80:0/752646315 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0b0050 con 0x7feba00d2e80 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feb967fc640 1 -- 172.21.15.80:0/752646315 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb9c0bc070 con 0x7feba00d2e80 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba0157770 con 0x7feba00d2e80 2024-04-25T13:45:18.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feb967fc640 1 -- 172.21.15.80:0/752646315 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9c0b5830 con 0x7feba00d2e80 2024-04-25T13:45:18.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba0157490 con 0x7feba00d2e80 2024-04-25T13:45:18.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba006a3a0 con 0x7feba00d2e80 2024-04-25T13:45:18.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feb967fc640 1 -- 172.21.15.80:0/752646315 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feb9c0b59d0 con 0x7feba00d2e80 2024-04-25T13:45:18.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feb967fc640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb8406f400 0x7feb84071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.003+0000 7feb967fc640 1 -- 172.21.15.80:0/752646315 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5767+0+0 (secure 0 0 0) 0x7feb9c129b70 con 0x7feba00d2e80 2024-04-25T13:45:18.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.007+0000 7feba5540640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb8406f400 0x7feb84071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.007+0000 7feba5540640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb8406f400 0x7feb84071910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feba00cfce0 tx=0x7feb90008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:18.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.011+0000 7feb967fc640 1 -- 172.21.15.80:0/752646315 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feba00ce8a0 con 0x7feba00d2e80 2024-04-25T13:45:18.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.155+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feba016af00 con 0x7feba00d2e80 2024-04-25T13:45:18.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.159+0000 7feb967fc640 1 -- 172.21.15.80:0/752646315 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+16694 (secure 0 0 0) 0x7feb9c016070 con 0x7feba00d2e80 2024-04-25T13:45:18.160 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:18.160 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":91,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:17.449622+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:44:33.879562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T13:44:34.932724+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1e","mappings":[{"from":6,"to":0}]},{"pgid":"11.28","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb8406f400 msgr2=0x7feb84071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb8406f400 0x7feb84071910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7feba00cfce0 tx=0x7feb90008040 comp rx=0 tx=0).stop 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 msgr2=0x7feba0155c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba0155c30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7feb9c0ae040 tx=0x7feb9c01ac30 comp rx=0 tx=0).stop 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 shutdown_connections 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feb8406f400 0x7feb84071910 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feba00d59b0 0x7feba00cef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feba00d4330 0x7feba0156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 --2- 172.21.15.80:0/752646315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feba00d2e80 0x7feba0155c30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 >> 172.21.15.80:0/752646315 conn(0x7feba00737a0 msgr2=0x7feba0162f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 shutdown_connections 2024-04-25T13:45:18.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.163+0000 7feba7fa5640 1 -- 172.21.15.80:0/752646315 wait complete. 2024-04-25T13:45:18.180 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:45:18.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.331+0000 7fa1875b4640 1 Processor -- start 2024-04-25T13:45:18.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.331+0000 7fa1875b4640 1 -- start start 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.331+0000 7fa1875b4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa18006ebb0 0x7fa18006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.331+0000 7fa1875b4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1800d0af0 con 0x7fa1800d06e0 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.331+0000 7fa1875b4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1800d0d70 con 0x7fa180161f90 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.331+0000 7fa1875b4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa1800d1050 con 0x7fa18006ebb0 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.331+0000 7fa184b4f640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa1800d06e0 0x7fa18006e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38620/0 (socket says 172.21.15.80:38620) 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.331+0000 7fa184b4f640 1 -- 172.21.15.80:0/1668464251 learned_addr learned my addr 172.21.15.80:0/1668464251 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa185b51640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa18006ebb0 0x7fa18006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4222690889 0 0) 0x7fa170001120 con 0x7fa1800d06e0 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa16c003530 con 0x7fa1800d06e0 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1244097689 0 0) 0x7fa16c003530 con 0x7fa1800d06e0 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa18006ebb0 msgr2=0x7fa18006efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:18.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa18006ebb0 0x7fa18006efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 >> v1:172.21.15.87:6789/0 conn(0x7fa180161f90 legacy=0x7fa1800cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa18006dca0 con 0x7fa1800d06e0 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fa170001850 con 0x7fa1800d06e0 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 >> v1:172.21.15.80:6789/0 conn(0x7fa1800d06e0 legacy=0x7fa18006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa18006ebb0 0x7fa16c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa16c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c004d50 0x7fa16c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1800d1050 con 0x7fa16c003f50 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa16c00bb90 con 0x7fa16c004d50 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa16c00bd90 con 0x7fa18006ebb0 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa184b4f640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa16c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa185b51640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c004d50 0x7fa16c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa185350640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa18006ebb0 0x7fa16c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa184b4f640 1 -- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa18006ebb0 msgr2=0x7fa16c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:18.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa184b4f640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa18006ebb0 0x7fa16c0039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa184b4f640 1 -- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c004d50 msgr2=0x7fa16c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:18.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa184b4f640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa16c004d50 0x7fa16c005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa184b4f640 1 -- 172.21.15.80:0/1668464251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16c003530 con 0x7fa16c003f50 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa184b4f640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa16c004810 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa17000c4c0 tx=0x7fa17001cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f671b3caebe2a89b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17000c0e0 con 0x7fa16c003f50 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1700a9020 con 0x7fa16c003f50 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa177fff640 1 -- 172.21.15.80:0/1668464251 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170005b20 con 0x7fa16c003f50 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa1875b4640 1 -- 172.21.15.80:0/1668464251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 msgr2=0x7fa16c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.335+0000 7fa1875b4640 1 --2- 172.21.15.80:0/1668464251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa16c004810 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fa17000c4c0 tx=0x7fa17001cc30 comp rx=0 tx=0).stop 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185b51640 1 -- 172.21.15.80:0/1668464251 reap_dead start 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 -- 172.21.15.80:0/1668464251 shutdown_connections 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 -- 172.21.15.80:0/1668464251 >> 172.21.15.80:0/1668464251 conn(0x7fa18006d080 msgr2=0x7fa1800b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:18.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 -- 172.21.15.80:0/1668464251 shutdown_connections 2024-04-25T13:45:18.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 -- 172.21.15.80:0/1668464251 wait complete. 2024-04-25T13:45:18.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 Processor -- start 2024-04-25T13:45:18.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 -- start start 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa180140280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1801407c0 0x7fa180148080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1801485c0 0x7fa1801489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1801447a0 con 0x7fa16c003f50 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa180144620 con 0x7fa1801407c0 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa1875b4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa180169d00 con 0x7fa1801485c0 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa180140280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa180140280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50358/0 (socket says 172.21.15.80:50358) 2024-04-25T13:45:18.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 -- 172.21.15.80:0/1317846203 learned_addr learned my addr 172.21.15.80:0/1317846203 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa184b4f640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1801407c0 0x7fa180148080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185b51640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1801485c0 0x7fa1801489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 -- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1801485c0 msgr2=0x7fa1801489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1801485c0 0x7fa1801489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 -- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1801407c0 msgr2=0x7fa180148080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1801407c0 0x7fa180148080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 -- 172.21.15.80:0/1317846203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa180143e20 con 0x7fa16c003f50 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.339+0000 7fa185350640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa180140280 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00d580 tx=0x7fa17c00da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa175ffb640 1 -- 172.21.15.80:0/1317846203 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c003620 con 0x7fa16c003f50 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa175ffb640 1 -- 172.21.15.80:0/1317846203 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa17c017070 con 0x7fa16c003f50 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa180140bd0 con 0x7fa16c003f50 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa175ffb640 1 -- 172.21.15.80:0/1317846203 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c0039e0 con 0x7fa16c003f50 2024-04-25T13:45:18.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa180149640 con 0x7fa16c003f50 2024-04-25T13:45:18.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa18006a3a0 con 0x7fa16c003f50 2024-04-25T13:45:18.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa175ffb640 1 -- 172.21.15.80:0/1317846203 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa17c003b80 con 0x7fa16c003f50 2024-04-25T13:45:18.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa175ffb640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa16406f400 0x7fa164071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa175ffb640 1 -- 172.21.15.80:0/1317846203 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5767+0+0 (secure 0 0 0) 0x7fa17c08fb20 con 0x7fa16c003f50 2024-04-25T13:45:18.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa184b4f640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa16406f400 0x7fa164071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.343+0000 7fa184b4f640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa16406f400 0x7fa164071910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa1700aa000 tx=0x7fa1800d0f40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:18.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.347+0000 7fa175ffb640 1 -- 172.21.15.80:0/1317846203 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa180140bd0 con 0x7fa16c003f50 2024-04-25T13:45:18.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.531+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa1800b5960 con 0x7fa16c003f50 2024-04-25T13:45:18.533 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:45:18.535+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:45:18.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.535+0000 7fa175ffb640 1 -- 172.21.15.80:0/1317846203 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) ==== 161+0+0 (secure 0 0 0) 0x7fa17c05cb40 con 0x7fa16c003f50 2024-04-25T13:45:18.536 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:45:18.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa16406f400 msgr2=0x7fa164071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa16406f400 0x7fa164071910 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa1700aa000 tx=0x7fa1800d0f40 comp rx=0 tx=0).stop 2024-04-25T13:45:18.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 msgr2=0x7fa180140280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa180140280 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00d580 tx=0x7fa17c00da40 comp rx=0 tx=0).stop 2024-04-25T13:45:18.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 shutdown_connections 2024-04-25T13:45:18.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa16406f400 0x7fa164071910 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1801485c0 0x7fa1801489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1801407c0 0x7fa180148080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 --2- 172.21.15.80:0/1317846203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa16c003f50 0x7fa180140280 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 >> 172.21.15.80:0/1317846203 conn(0x7fa18006d080 msgr2=0x7fa1801436c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:18.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 shutdown_connections 2024-04-25T13:45:18.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.539+0000 7fa1875b4640 1 -- 172.21.15.80:0/1317846203 wait complete. 2024-04-25T13:45:18.560 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:45:18.640 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:45:18.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.707+0000 7f5ec2c8e640 1 Processor -- start 2024-04-25T13:45:18.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.707+0000 7f5ec2c8e640 1 -- start start 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec2c8e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ebc06cf80 0x7f5ebc0cf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec2c8e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5ebc0cf9e0 0x7f5ebc0d02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec2c8e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ebc06d560 con 0x7f5ebc0d0830 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec2c8e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0d28d0 con 0x7f5ebc06cf80 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec2c8e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5ebc0d2bb0 con 0x7f5ebc0cf9e0 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec0a2a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ebc06cf80 0x7f5ebc0cf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec122b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5ebc0d0830 0x7f5ebc0ce0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38632/0 (socket says 172.21.15.80:38632) 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec122b640 1 -- 172.21.15.80:0/836103927 learned_addr learned my addr 172.21.15.80:0/836103927 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ebbfff640 1 --2- 172.21.15.80:0/836103927 >> v2:172.21.15.87:3300/0 conn(0x7f5ebc0cf9e0 0x7f5ebc0d02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec0a2a640 1 -- 172.21.15.80:0/836103927 >> v2:172.21.15.87:3300/0 conn(0x7f5ebc0cf9e0 msgr2=0x7f5ebc0d02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:18.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec0a2a640 1 --2- 172.21.15.80:0/836103927 >> v2:172.21.15.87:3300/0 conn(0x7f5ebc0cf9e0 0x7f5ebc0d02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec0a2a640 1 -- 172.21.15.80:0/836103927 >> v1:172.21.15.80:6789/0 conn(0x7f5ebc0d0830 legacy=0x7f5ebc0ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec0a2a640 1 -- 172.21.15.80:0/836103927 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc0b6240 con 0x7f5ebc06cf80 2024-04-25T13:45:18.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ebb7fe640 1 -- 172.21.15.80:0/836103927 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1955934797 0 0) 0x7f5ebc06d560 con 0x7f5ebc0d0830 2024-04-25T13:45:18.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ec0a2a640 1 --2- 172.21.15.80:0/836103927 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc06cf80 0x7f5ebc0cf4a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc072a80 tx=0x7f5eac01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9840e0c135ff522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:18.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ebb7fe640 1 -- 172.21.15.80:0/836103927 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0b0070 con 0x7f5ebc06cf80 2024-04-25T13:45:18.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ebb7fe640 1 -- 172.21.15.80:0/836103927 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eac0ab400 con 0x7f5ebc06cf80 2024-04-25T13:45:18.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.711+0000 7f5ebb7fe640 1 -- 172.21.15.80:0/836103927 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eac0ab7d0 con 0x7f5ebc06cf80 2024-04-25T13:45:18.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/836103927 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc06cf80 msgr2=0x7f5ebc0cf4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/836103927 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc06cf80 0x7f5ebc0cf4a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc072a80 tx=0x7f5eac01c460 comp rx=0 tx=0).stop 2024-04-25T13:45:18.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/836103927 shutdown_connections 2024-04-25T13:45:18.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/836103927 >> v2:172.21.15.87:3300/0 conn(0x7f5ebc0cf9e0 0x7f5ebc0d02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/836103927 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc06cf80 0x7f5ebc0cf4a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/836103927 >> 172.21.15.80:0/836103927 conn(0x7f5ebc0b5580 msgr2=0x7f5ebc0b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:18.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/836103927 shutdown_connections 2024-04-25T13:45:18.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/836103927 wait complete. 2024-04-25T13:45:18.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 Processor -- start 2024-04-25T13:45:18.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- start start 2024-04-25T13:45:18.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ebc06cf80 0x7f5ebc160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ebc0cf9e0 0x7f5ebc15a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc0d0830 0x7f5ebc161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d30d0 con 0x7f5ebc0d0830 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0b4df0 con 0x7f5ebc06cf80 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec2c8e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc0d3250 con 0x7f5ebc0cf9e0 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec122b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc0d0830 0x7f5ebc161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec122b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc0d0830 0x7f5ebc161bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50378/0 (socket says 172.21.15.80:50378) 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec122b640 1 -- 172.21.15.80:0/701149883 learned_addr learned my addr 172.21.15.80:0/701149883 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ec0a2a640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ebc06cf80 0x7f5ebc160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.715+0000 7f5ebbfff640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ebc0cf9e0 0x7f5ebc15a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec122b640 1 -- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ebc0cf9e0 msgr2=0x7f5ebc15a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec122b640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ebc0cf9e0 0x7f5ebc15a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec122b640 1 -- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ebc06cf80 msgr2=0x7f5ebc160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec122b640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ebc06cf80 0x7f5ebc160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec122b640 1 -- 172.21.15.80:0/701149883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc160b10 con 0x7f5ebc0d0830 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec0a2a640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ebc06cf80 0x7f5ebc160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec122b640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc0d0830 0x7f5ebc161bb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400c700 tx=0x7f5eb400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5eb97fa640 1 -- 172.21.15.80:0/701149883 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4018070 con 0x7f5ebc0d0830 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5eb97fa640 1 -- 172.21.15.80:0/701149883 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5eb400ce10 con 0x7f5ebc0d0830 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5eb97fa640 1 -- 172.21.15.80:0/701149883 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb400a800 con 0x7f5ebc0d0830 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc160df0 con 0x7f5ebc0d0830 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc1613b0 con 0x7f5ebc0d0830 2024-04-25T13:45:18.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc06a3a0 con 0x7f5ebc0d0830 2024-04-25T13:45:18.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5eb97fa640 1 -- 172.21.15.80:0/701149883 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5eb401b070 con 0x7f5ebc0d0830 2024-04-25T13:45:18.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5eb97fa640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ea806f430 0x7f5ea8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:18.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5eb97fa640 1 -- 172.21.15.80:0/701149883 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5853+0+0 (secure 0 0 0) 0x7f5eb408ead0 con 0x7f5ebc0d0830 2024-04-25T13:45:18.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec0a2a640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ea806f430 0x7f5ea8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:18.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.719+0000 7f5ec0a2a640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ea806f430 0x7f5ea8071940 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0ad000 tx=0x7f5eac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:18.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.723+0000 7f5eb97fa640 1 -- 172.21.15.80:0/701149883 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5eb408ee20 con 0x7f5ebc0d0830 2024-04-25T13:45:18.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:18.923+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5ebc0cf4d0 con 0x7f5ebc0d0830 2024-04-25T13:45:19.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.539+0000 7f5eb97fa640 1 -- 172.21.15.80:0/701149883 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7f5eb405ba10 con 0x7f5ebc0d0830 2024-04-25T13:45:19.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ea806f430 msgr2=0x7f5ea8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:19.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ea806f430 0x7f5ea8071940 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5eac0ad000 tx=0x7f5eac008040 comp rx=0 tx=0).stop 2024-04-25T13:45:19.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc0d0830 msgr2=0x7f5ebc161bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:19.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc0d0830 0x7f5ebc161bb0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5eb400c700 tx=0x7f5eb400cbc0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 shutdown_connections 2024-04-25T13:45:19.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ea806f430 0x7f5ea8071940 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ebc0d0830 0x7f5ebc161bb0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ebc0cf9e0 0x7f5ebc15a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 --2- 172.21.15.80:0/701149883 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ebc06cf80 0x7f5ebc160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 >> 172.21.15.80:0/701149883 conn(0x7f5ebc0b5580 msgr2=0x7f5ebc0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:19.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 shutdown_connections 2024-04-25T13:45:19.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.543+0000 7f5ec2c8e640 1 -- 172.21.15.80:0/701149883 wait complete. 2024-04-25T13:45:19.567 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:45:19.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.723+0000 7ff16b2eb640 1 Processor -- start 2024-04-25T13:45:19.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.723+0000 7ff16b2eb640 1 -- start start 2024-04-25T13:45:19.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff16b2eb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1640b09a0 0x7ff1641602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff16b2eb640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff164160820 0x7ff1640ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff16b2eb640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1640b4a60 con 0x7ff16406fb30 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff16b2eb640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff164161270 con 0x7ff1640b09a0 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff16b2eb640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff1640b0db0 con 0x7ff164160820 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff168886640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1640b09a0 0x7ff1641602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff168886640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1640b09a0 0x7ff1641602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50380/0 (socket says 172.21.15.80:50380) 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff168886640 1 -- 172.21.15.80:0/2302440232 learned_addr learned my addr 172.21.15.80:0/2302440232 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff169888640 1 --2- 172.21.15.80:0/2302440232 >> v2:172.21.15.87:3300/0 conn(0x7ff164160820 0x7ff1640ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff168886640 1 -- 172.21.15.80:0/2302440232 >> v2:172.21.15.87:3300/0 conn(0x7ff164160820 msgr2=0x7ff1640ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff168886640 1 --2- 172.21.15.80:0/2302440232 >> v2:172.21.15.87:3300/0 conn(0x7ff164160820 0x7ff1640ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff168886640 1 -- 172.21.15.80:0/2302440232 >> v1:172.21.15.87:6789/0 conn(0x7ff16406fb30 legacy=0x7ff1640b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:19.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.727+0000 7ff168886640 1 -- 172.21.15.80:0/2302440232 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff164160f90 con 0x7ff1640b09a0 2024-04-25T13:45:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff168886640 1 --2- 172.21.15.80:0/2302440232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1640b09a0 0x7ff1641602e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff164071cb0 tx=0x7ff15401c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edffa5351d73a695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff15bfff640 1 -- 172.21.15.80:0/2302440232 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1540ab5f0 con 0x7ff1640b09a0 2024-04-25T13:45:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff15bfff640 1 -- 172.21.15.80:0/2302440232 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1540b0070 con 0x7ff1640b09a0 2024-04-25T13:45:19.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff15bfff640 1 -- 172.21.15.80:0/2302440232 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1540ab9b0 con 0x7ff1640b09a0 2024-04-25T13:45:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 -- 172.21.15.80:0/2302440232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1640b09a0 msgr2=0x7ff1641602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/2302440232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1640b09a0 0x7ff1641602e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7ff164071cb0 tx=0x7ff15401c4e0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 -- 172.21.15.80:0/2302440232 shutdown_connections 2024-04-25T13:45:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/2302440232 >> v2:172.21.15.87:3300/0 conn(0x7ff164160820 0x7ff1640ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/2302440232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1640b09a0 0x7ff1641602e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 -- 172.21.15.80:0/2302440232 >> 172.21.15.80:0/2302440232 conn(0x7ff1640b35c0 msgr2=0x7ff1640b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 -- 172.21.15.80:0/2302440232 shutdown_connections 2024-04-25T13:45:19.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 -- 172.21.15.80:0/2302440232 wait complete. 2024-04-25T13:45:19.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 Processor -- start 2024-04-25T13:45:19.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.731+0000 7ff16b2eb640 1 -- start start 2024-04-25T13:45:19.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16406fb30 0x7ff16415a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:19.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1640b09a0 0x7ff1641522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:19.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff164160820 0x7ff1640b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640d0950 con 0x7ff164160820 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff1640b1180 con 0x7ff1640b09a0 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff1640d0ad0 con 0x7ff16406fb30 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff164160820 0x7ff1640b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff164160820 0x7ff1640b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50384/0 (socket says 172.21.15.80:50384) 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 -- 172.21.15.80:0/146782850 learned_addr learned my addr 172.21.15.80:0/146782850 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169087640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16406fb30 0x7ff16415a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff168886640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1640b09a0 0x7ff1641522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 -- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16406fb30 msgr2=0x7ff16415a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16406fb30 0x7ff16415a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 -- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1640b09a0 msgr2=0x7ff1641522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1640b09a0 0x7ff1641522c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 -- 172.21.15.80:0/146782850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16415ce80 con 0x7ff164160820 2024-04-25T13:45:19.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169087640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16406fb30 0x7ff16415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:19.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff169888640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff164160820 0x7ff1640b2450 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009fd0 tx=0x7ff15c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:19.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff159ffb640 1 -- 172.21.15.80:0/146782850 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c014420 con 0x7ff164160820 2024-04-25T13:45:19.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff159ffb640 1 -- 172.21.15.80:0/146782850 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff15c019070 con 0x7ff164160820 2024-04-25T13:45:19.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16415a990 con 0x7ff164160820 2024-04-25T13:45:19.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff159ffb640 1 -- 172.21.15.80:0/146782850 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c014800 con 0x7ff164160820 2024-04-25T13:45:19.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff16415d400 con 0x7ff164160820 2024-04-25T13:45:19.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff16415a700 con 0x7ff164160820 2024-04-25T13:45:19.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.735+0000 7ff159ffb640 1 -- 172.21.15.80:0/146782850 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff15c0149a0 con 0x7ff164160820 2024-04-25T13:45:19.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.739+0000 7ff159ffb640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff14806f400 0x7ff148071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:19.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.739+0000 7ff159ffb640 1 -- 172.21.15.80:0/146782850 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5853+0+0 (secure 0 0 0) 0x7ff15c090620 con 0x7ff164160820 2024-04-25T13:45:19.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.739+0000 7ff169087640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff14806f400 0x7ff148071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:19.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.739+0000 7ff169087640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff14806f400 0x7ff148071910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff164071a20 tx=0x7ff160008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:19.741 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.743+0000 7ff159ffb640 1 -- 172.21.15.80:0/146782850 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff15c05d560 con 0x7ff164160820 2024-04-25T13:45:19.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:19.903+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff16406a440 con 0x7ff164160820 2024-04-25T13:45:20.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.547+0000 7ff159ffb640 1 -- 172.21.15.80:0/146782850 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7ff16406a440 con 0x7ff164160820 2024-04-25T13:45:20.548 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:45:20.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff14806f400 msgr2=0x7ff148071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff14806f400 0x7ff148071910 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff164071a20 tx=0x7ff160008040 comp rx=0 tx=0).stop 2024-04-25T13:45:20.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff164160820 msgr2=0x7ff1640b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff164160820 0x7ff1640b2450 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009fd0 tx=0x7ff15c00e4b0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 shutdown_connections 2024-04-25T13:45:20.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff14806f400 0x7ff148071910 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff164160820 0x7ff1640b2450 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1640b09a0 0x7ff1641522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 --2- 172.21.15.80:0/146782850 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff16406fb30 0x7ff16415a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 >> 172.21.15.80:0/146782850 conn(0x7ff1640b35c0 msgr2=0x7ff164166b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:20.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 shutdown_connections 2024-04-25T13:45:20.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.551+0000 7ff16b2eb640 1 -- 172.21.15.80:0/146782850 wait complete. 2024-04-25T13:45:20.577 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:45:20.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afe832640 1 Processor -- start 2024-04-25T13:45:20.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afe832640 1 -- start start 2024-04-25T13:45:20.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afe832640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80b48c0 0x7f9af8161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afe832640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9af80d0840 0x7f9af80d0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afe832640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9af80ce220 0x7f9af80ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afe832640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9af80b0bc0 con 0x7f9af80ce220 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afe832640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9af80d1190 con 0x7f9af80d0840 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afe832640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9af80b0e10 con 0x7f9af80b48c0 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afcdcf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9af80ce220 0x7f9af80ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afcdcf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9af80ce220 0x7f9af80ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50398/0 (socket says 172.21.15.80:50398) 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afcdcf640 1 -- 172.21.15.80:0/191411447 learned_addr learned my addr 172.21.15.80:0/191411447 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9af7fff640 1 --2- 172.21.15.80:0/191411447 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80b48c0 0x7f9af8161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9af77fe640 1 --2- 172.21.15.80:0/191411447 >> v2:172.21.15.87:3300/0 conn(0x7f9af80d0840 0x7f9af80d0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afcdcf640 1 -- 172.21.15.80:0/191411447 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80b48c0 msgr2=0x7f9af8161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afcdcf640 1 --2- 172.21.15.80:0/191411447 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80b48c0 0x7f9af8161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afcdcf640 1 -- 172.21.15.80:0/191411447 >> v2:172.21.15.87:3300/0 conn(0x7f9af80d0840 msgr2=0x7f9af80d0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afcdcf640 1 --2- 172.21.15.80:0/191411447 >> v2:172.21.15.87:3300/0 conn(0x7f9af80d0840 0x7f9af80d0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9afcdcf640 1 -- 172.21.15.80:0/191411447 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af80b10f0 con 0x7f9af80ce220 2024-04-25T13:45:20.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.735+0000 7f9af77fe640 1 --2- 172.21.15.80:0/191411447 >> v2:172.21.15.87:3300/0 conn(0x7f9af80d0840 0x7f9af80d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:20.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afcdcf640 1 --2- 172.21.15.80:0/191411447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af80ce630 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9af8071aa0 tx=0x7f9af001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7def90137dfb698f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9af6ffd640 1 -- 172.21.15.80:0/191411447 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00a55e0 con 0x7f9af80ce220 2024-04-25T13:45:20.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9af6ffd640 1 -- 172.21.15.80:0/191411447 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9af00a5780 con 0x7f9af80ce220 2024-04-25T13:45:20.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9af6ffd640 1 -- 172.21.15.80:0/191411447 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9af00a5b50 con 0x7f9af80ce220 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 -- 172.21.15.80:0/191411447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 msgr2=0x7f9af80ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 --2- 172.21.15.80:0/191411447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af80ce630 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f9af8071aa0 tx=0x7f9af001c960 comp rx=0 tx=0).stop 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 -- 172.21.15.80:0/191411447 shutdown_connections 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 --2- 172.21.15.80:0/191411447 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af80ce630 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 --2- 172.21.15.80:0/191411447 >> v2:172.21.15.87:3300/0 conn(0x7f9af80d0840 0x7f9af80d0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 --2- 172.21.15.80:0/191411447 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80b48c0 0x7f9af8161600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 -- 172.21.15.80:0/191411447 >> 172.21.15.80:0/191411447 conn(0x7f9af80b35c0 msgr2=0x7f9af80b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 -- 172.21.15.80:0/191411447 shutdown_connections 2024-04-25T13:45:20.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 -- 172.21.15.80:0/191411447 wait complete. 2024-04-25T13:45:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.739+0000 7f9afe832640 1 Processor -- start 2024-04-25T13:45:20.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 -- start start 2024-04-25T13:45:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af80b48c0 0x7f9af815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af814e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80d0840 0x7f9af8150b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9af806fc30 con 0x7f9af80ce220 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9af815dd30 con 0x7f9af80b48c0 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9af81517a0 con 0x7f9af80d0840 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af814e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af814e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50410/0 (socket says 172.21.15.80:50410) 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 -- 172.21.15.80:0/3229338713 learned_addr learned my addr 172.21.15.80:0/3229338713 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afcdcf640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80d0840 0x7f9af8150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af7fff640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af80b48c0 0x7f9af815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 -- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80d0840 msgr2=0x7f9af8150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80d0840 0x7f9af8150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 -- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af80b48c0 msgr2=0x7f9af815c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af80b48c0 0x7f9af815c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 -- 172.21.15.80:0/3229338713 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af814fed0 con 0x7f9af80ce220 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afcdcf640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80d0840 0x7f9af8150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af7fff640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af80b48c0 0x7f9af815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af77fe640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af814e8d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c850 tx=0x7f9aec00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af4ff9640 1 -- 172.21.15.80:0/3229338713 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec015070 con 0x7f9af80ce220 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af4ff9640 1 -- 172.21.15.80:0/3229338713 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aec007a30 con 0x7f9af80ce220 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af815b930 con 0x7f9af80ce220 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9af4ff9640 1 -- 172.21.15.80:0/3229338713 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aec007e20 con 0x7f9af80ce220 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af815bef0 con 0x7f9af80ce220 2024-04-25T13:45:20.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.743+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9af806a440 con 0x7f9af80ce220 2024-04-25T13:45:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.747+0000 7f9af4ff9640 1 -- 172.21.15.80:0/3229338713 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9aec018070 con 0x7f9af80ce220 2024-04-25T13:45:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.747+0000 7f9af4ff9640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ae406f430 0x7f9ae4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.747+0000 7f9af4ff9640 1 -- 172.21.15.80:0/3229338713 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5415+0+0 (secure 0 0 0) 0x7f9aec08f340 con 0x7f9af80ce220 2024-04-25T13:45:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.747+0000 7f9af7fff640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ae406f430 0x7f9ae4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:20.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.747+0000 7f9af7fff640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ae406f430 0x7f9ae4071940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9af80cf2f0 tx=0x7f9ae8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:20.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.751+0000 7f9af4ff9640 1 -- 172.21.15.80:0/3229338713 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9af814fed0 con 0x7f9af80ce220 2024-04-25T13:45:20.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:20.907+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9af815e6c0 con 0x7f9af80ce220 2024-04-25T13:45:21.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.563+0000 7f9af4ff9640 1 -- 172.21.15.80:0/3229338713 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f9aec00ae20 con 0x7f9af80ce220 2024-04-25T13:45:21.563 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:45:21.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ae406f430 msgr2=0x7f9ae4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:21.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ae406f430 0x7f9ae4071940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9af80cf2f0 tx=0x7f9ae8008040 comp rx=0 tx=0).stop 2024-04-25T13:45:21.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 msgr2=0x7f9af814e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af814e8d0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f9aec00c850 tx=0x7f9aec00cd10 comp rx=0 tx=0).stop 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 shutdown_connections 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ae406f430 0x7f9ae4071940 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9af80d0840 0x7f9af8150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9af80ce220 0x7f9af814e8d0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 --2- 172.21.15.80:0/3229338713 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9af80b48c0 0x7f9af815c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 >> 172.21.15.80:0/3229338713 conn(0x7f9af80b35c0 msgr2=0x7f9af80b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 shutdown_connections 2024-04-25T13:45:21.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.567+0000 7f9afe832640 1 -- 172.21.15.80:0/3229338713 wait complete. 2024-04-25T13:45:21.594 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:45:21.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f71832d0640 1 Processor -- start 2024-04-25T13:45:21.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f71832d0640 1 -- start start 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f71832d0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f717c0ceb60 0x7f717c0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f71832d0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f717c0cf4b0 0x7f717c0cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f71832d0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f717c0b0ae0 con 0x7f717c06f080 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f71832d0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f717c0cfe00 con 0x7f717c0cf4b0 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f71832d0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f717c0727b0 con 0x7f717c0ceb60 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f7182acf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f717c0cf4b0 0x7f717c0cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f7182acf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f717c0cf4b0 0x7f717c0cf8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35870/0 (socket says 172.21.15.80:35870) 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f7182acf640 1 -- 172.21.15.80:0/1558131969 learned_addr learned my addr 172.21.15.80:0/1558131969 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f7181acd640 1 --2- 172.21.15.80:0/1558131969 >> v2:172.21.15.87:3300/0 conn(0x7f717c0ceb60 0x7f717c0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:21.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f7182acf640 1 -- 172.21.15.80:0/1558131969 >> v2:172.21.15.87:3300/0 conn(0x7f717c0ceb60 msgr2=0x7f717c0cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:21.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f7182acf640 1 --2- 172.21.15.80:0/1558131969 >> v2:172.21.15.87:3300/0 conn(0x7f717c0ceb60 0x7f717c0cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f7182acf640 1 -- 172.21.15.80:0/1558131969 >> v1:172.21.15.87:6789/0 conn(0x7f717c06f080 legacy=0x7f717c0b5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:21.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.747+0000 7f7182acf640 1 -- 172.21.15.80:0/1558131969 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c072a90 con 0x7f717c0cf4b0 2024-04-25T13:45:21.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f7182acf640 1 --2- 172.21.15.80:0/1558131969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0cf4b0 0x7f717c0cf8c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f717c0b1290 tx=0x7f717401c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a151fb2a153bcace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:21.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71812cc640 1 -- 172.21.15.80:0/1558131969 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71740b0070 con 0x7f717c0cf4b0 2024-04-25T13:45:21.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71812cc640 1 -- 172.21.15.80:0/1558131969 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71740ab600 con 0x7f717c0cf4b0 2024-04-25T13:45:21.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71812cc640 1 -- 172.21.15.80:0/1558131969 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71740ab9d0 con 0x7f717c0cf4b0 2024-04-25T13:45:21.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71832d0640 1 -- 172.21.15.80:0/1558131969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0cf4b0 msgr2=0x7f717c0cf8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:21.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71832d0640 1 --2- 172.21.15.80:0/1558131969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0cf4b0 0x7f717c0cf8c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f717c0b1290 tx=0x7f717401c300 comp rx=0 tx=0).stop 2024-04-25T13:45:21.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71832d0640 1 -- 172.21.15.80:0/1558131969 shutdown_connections 2024-04-25T13:45:21.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71832d0640 1 --2- 172.21.15.80:0/1558131969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0cf4b0 0x7f717c0cf8c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71832d0640 1 --2- 172.21.15.80:0/1558131969 >> v2:172.21.15.87:3300/0 conn(0x7f717c0ceb60 0x7f717c0cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71832d0640 1 -- 172.21.15.80:0/1558131969 >> 172.21.15.80:0/1558131969 conn(0x7f717c06cdb0 msgr2=0x7f717c0b3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:21.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71832d0640 1 -- 172.21.15.80:0/1558131969 shutdown_connections 2024-04-25T13:45:21.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.751+0000 7f71832d0640 1 -- 172.21.15.80:0/1558131969 wait complete. 2024-04-25T13:45:21.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 Processor -- start 2024-04-25T13:45:21.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 -- start start 2024-04-25T13:45:21.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f717c06f080 0x7f717c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:21.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0ceb60 0x7f717c144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:21.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f717c0cf4b0 0x7f717c0ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:21.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f717c167d00 con 0x7f717c0ceb60 2024-04-25T13:45:21.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f717c167b80 con 0x7f717c0cf4b0 2024-04-25T13:45:21.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f717c167e80 con 0x7f717c06f080 2024-04-25T13:45:21.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0ceb60 0x7f717c144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:21.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0ceb60 0x7f717c144110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35880/0 (socket says 172.21.15.80:35880) 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 -- 172.21.15.80:0/4037595714 learned_addr learned my addr 172.21.15.80:0/4037595714 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71822ce640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f717c06f080 0x7f717c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7182acf640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f717c0cf4b0 0x7f717c0ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 -- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f717c06f080 msgr2=0x7f717c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f717c06f080 0x7f717c15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 -- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f717c0cf4b0 msgr2=0x7f717c0ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f717c0cf4b0 0x7f717c0ab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 -- 172.21.15.80:0/4037595714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f717c073df0 con 0x7f717c0ceb60 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7182acf640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f717c0cf4b0 0x7f717c0ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7181acd640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0ceb60 0x7f717c144110 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f716c00d800 tx=0x7f716c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7172ffd640 1 -- 172.21.15.80:0/4037595714 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c00b6c0 con 0x7f717c0ceb60 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7172ffd640 1 -- 172.21.15.80:0/4037595714 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f716c019070 con 0x7f717c0ceb60 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f7172ffd640 1 -- 172.21.15.80:0/4037595714 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716c00ba80 con 0x7f717c0ceb60 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717c0b0280 con 0x7f717c0ceb60 2024-04-25T13:45:21.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717c0ac500 con 0x7f717c0ceb60 2024-04-25T13:45:21.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.755+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f717c06a2c0 con 0x7f717c0ceb60 2024-04-25T13:45:21.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.759+0000 7f7172ffd640 1 -- 172.21.15.80:0/4037595714 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f716c00bc20 con 0x7f717c0ceb60 2024-04-25T13:45:21.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.759+0000 7f7172ffd640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f716006f400 0x7f7160071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:21.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.759+0000 7f7172ffd640 1 -- 172.21.15.80:0/4037595714 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f716c08f530 con 0x7f717c0ceb60 2024-04-25T13:45:21.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.759+0000 7f71822ce640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f716006f400 0x7f7160071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:21.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.759+0000 7f71822ce640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f716006f400 0x7f7160071910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f717c06fe40 tx=0x7f71780094d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:21.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.763+0000 7f7172ffd640 1 -- 172.21.15.80:0/4037595714 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f716c05c690 con 0x7f717c0ceb60 2024-04-25T13:45:21.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.923+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f717c0b19d0 con 0x7f717c0ceb60 2024-04-25T13:45:21.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.931+0000 7f7172ffd640 1 -- 172.21.15.80:0/4037595714 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) ==== 104+0+0 (secure 0 0 0) 0x7f716c05bf90 con 0x7f717c0ceb60 2024-04-25T13:45:21.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f716006f400 msgr2=0x7f7160071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:21.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f716006f400 0x7f7160071910 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f717c06fe40 tx=0x7f71780094d0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0ceb60 msgr2=0x7f717c144110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:21.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0ceb60 0x7f717c144110 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f716c00d800 tx=0x7f716c00dcc0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 shutdown_connections 2024-04-25T13:45:21.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f716006f400 0x7f7160071910 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f717c0cf4b0 0x7f717c0ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f717c0ceb60 0x7f717c144110 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 --2- 172.21.15.80:0/4037595714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f717c06f080 0x7f717c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:21.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 >> 172.21.15.80:0/4037595714 conn(0x7f717c06cdb0 msgr2=0x7f717c06d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:21.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 shutdown_connections 2024-04-25T13:45:21.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:21.935+0000 7f71832d0640 1 -- 172.21.15.80:0/4037595714 wait complete. 2024-04-25T13:45:21.956 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2024-04-25T13:45:22.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.099+0000 7f5b5adb0640 1 Processor -- start 2024-04-25T13:45:22.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.099+0000 7f5b5adb0640 1 -- start start 2024-04-25T13:45:22.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b5adb0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b54161f90 0x7f5b540cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b5adb0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b540d0510 0x7f5b5415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b5adb0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b540d0d70 con 0x7f5b5415ea70 2024-04-25T13:45:22.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b5adb0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5b540d1050 con 0x7f5b54161f90 2024-04-25T13:45:22.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b5adb0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5406dca0 con 0x7f5b540d0510 2024-04-25T13:45:22.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b58b4c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b54161f90 0x7f5b540cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b5934d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5b5415ea70 0x7f5b5415ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51580/0 (socket says 172.21.15.80:51580) 2024-04-25T13:45:22.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b5934d640 1 -- 172.21.15.80:0/2574160062 learned_addr learned my addr 172.21.15.80:0/2574160062 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:22.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b53fff640 1 --2- 172.21.15.80:0/2574160062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b540d0510 0x7f5b5415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b58b4c640 1 -- 172.21.15.80:0/2574160062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b540d0510 msgr2=0x7f5b5415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:22.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b58b4c640 1 --2- 172.21.15.80:0/2574160062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b540d0510 0x7f5b5415e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b58b4c640 1 -- 172.21.15.80:0/2574160062 >> v1:172.21.15.80:6789/0 conn(0x7f5b5415ea70 legacy=0x7f5b5415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b537fe640 1 -- 172.21.15.80:0/2574160062 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 257437528 0 0) 0x7f5b540d0d70 con 0x7f5b5415ea70 2024-04-25T13:45:22.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.103+0000 7f5b58b4c640 1 -- 172.21.15.80:0/2574160062 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b5406df80 con 0x7f5b54161f90 2024-04-25T13:45:22.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b58b4c640 1 --2- 172.21.15.80:0/2574160062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b54161f90 0x7f5b540cffd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009d90 tx=0x7f5b4401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c6e53d5ecfd3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:22.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b537fe640 1 -- 172.21.15.80:0/2574160062 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440b2070 con 0x7f5b54161f90 2024-04-25T13:45:22.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b537fe640 1 -- 172.21.15.80:0/2574160062 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b440a5870 con 0x7f5b54161f90 2024-04-25T13:45:22.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b537fe640 1 -- 172.21.15.80:0/2574160062 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b440a5c40 con 0x7f5b54161f90 2024-04-25T13:45:22.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 -- 172.21.15.80:0/2574160062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b54161f90 msgr2=0x7f5b540cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/2574160062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b54161f90 0x7f5b540cffd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f5b44009d90 tx=0x7f5b4401c880 comp rx=0 tx=0).stop 2024-04-25T13:45:22.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 -- 172.21.15.80:0/2574160062 shutdown_connections 2024-04-25T13:45:22.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/2574160062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b540d0510 0x7f5b5415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/2574160062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b54161f90 0x7f5b540cffd0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 -- 172.21.15.80:0/2574160062 >> 172.21.15.80:0/2574160062 conn(0x7f5b5406d080 msgr2=0x7f5b540b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:22.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 -- 172.21.15.80:0/2574160062 shutdown_connections 2024-04-25T13:45:22.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 -- 172.21.15.80:0/2574160062 wait complete. 2024-04-25T13:45:22.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 Processor -- start 2024-04-25T13:45:22.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.107+0000 7f5b5adb0640 1 -- start start 2024-04-25T13:45:22.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b540d0510 0x7f5b5414e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5415ea70 0x7f5b540ab530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b54161f90 0x7f5b541670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5406f4b0 con 0x7f5b5415ea70 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5406f630 con 0x7f5b540d0510 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b540d0920 con 0x7f5b54161f90 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5415ea70 0x7f5b540ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5415ea70 0x7f5b540ab530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35902/0 (socket says 172.21.15.80:35902) 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 -- 172.21.15.80:0/60212498 learned_addr learned my addr 172.21.15.80:0/60212498 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5934d640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b54161f90 0x7f5b541670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b58b4c640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b540d0510 0x7f5b5414e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 -- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b54161f90 msgr2=0x7f5b541670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b54161f90 0x7f5b541670b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 -- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b540d0510 msgr2=0x7f5b5414e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b540d0510 0x7f5b5414e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 -- 172.21.15.80:0/60212498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b54073d80 con 0x7f5b5415ea70 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5934d640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b54161f90 0x7f5b541670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b58b4c640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b540d0510 0x7f5b5414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b53fff640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5415ea70 0x7f5b540ab530 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00d5f0 tx=0x7f5b3c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b517fa640 1 -- 172.21.15.80:0/60212498 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0125f0 con 0x7f5b5415ea70 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b517fa640 1 -- 172.21.15.80:0/60212498 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b3c019070 con 0x7f5b5415ea70 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b54160130 con 0x7f5b5415ea70 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b517fa640 1 -- 172.21.15.80:0/60212498 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3c0129b0 con 0x7f5b5415ea70 2024-04-25T13:45:22.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b541606f0 con 0x7f5b5415ea70 2024-04-25T13:45:22.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b54073af0 con 0x7f5b5415ea70 2024-04-25T13:45:22.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.111+0000 7f5b517fa640 1 -- 172.21.15.80:0/60212498 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5b3c012b50 con 0x7f5b5415ea70 2024-04-25T13:45:22.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.115+0000 7f5b517fa640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b3406f400 0x7f5b34071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.115+0000 7f5b517fa640 1 -- 172.21.15.80:0/60212498 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f5b3c08fe00 con 0x7f5b5415ea70 2024-04-25T13:45:22.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.115+0000 7f5b58b4c640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b3406f400 0x7f5b34071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.115+0000 7f5b58b4c640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b3406f400 0x7f5b34071910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b440af000 tx=0x7f5b440a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:22.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.115+0000 7f5b517fa640 1 -- 172.21.15.80:0/60212498 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b3c05d160 con 0x7f5b5415ea70 2024-04-25T13:45:22.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.335+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) -- 0x7f5b5406a3a0 con 0x7f5b5415ea70 2024-04-25T13:45:22.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b517fa640 1 -- 172.21.15.80:0/60212498 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v262) ==== 133+0+0 (secure 0 0 0) 0x7f5b3c05ca60 con 0x7f5b5415ea70 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b3406f400 msgr2=0x7f5b34071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b3406f400 0x7f5b34071910 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b440af000 tx=0x7f5b440a7070 comp rx=0 tx=0).stop 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5415ea70 msgr2=0x7f5b540ab530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5415ea70 0x7f5b540ab530 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c00d5f0 tx=0x7f5b3c00dab0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 shutdown_connections 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b3406f400 0x7f5b34071910 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b54161f90 0x7f5b541670b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b5415ea70 0x7f5b540ab530 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 --2- 172.21.15.80:0/60212498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b540d0510 0x7f5b5414e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 >> 172.21.15.80:0/60212498 conn(0x7f5b5406d080 msgr2=0x7f5b540b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 shutdown_connections 2024-04-25T13:45:22.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.567+0000 7f5b5adb0640 1 -- 172.21.15.80:0/60212498 wait complete. 2024-04-25T13:45:22.593 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-04-25T13:45:22.593 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2024-04-25T13:45:22.594 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:22.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.739+0000 7fe54d958640 1 Processor -- start 2024-04-25T13:45:22.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.739+0000 7fe54d958640 1 -- start start 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe54d958640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe54806f5a0 0x7fe548160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe54d958640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5480733b0 0x7fe548161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe54d958640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe54806fcf0 con 0x7fe548161070 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe54d958640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe5481619c0 con 0x7fe54806f5a0 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe54d958640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe548073f10 con 0x7fe5480733b0 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe546ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe54806f5a0 0x7fe548160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe546ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe54806f5a0 0x7fe548160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35912/0 (socket says 172.21.15.80:35912) 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe546ffd640 1 -- 172.21.15.80:0/1886423742 learned_addr learned my addr 172.21.15.80:0/1886423742 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe5477fe640 1 --2- 172.21.15.80:0/1886423742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5480733b0 0x7fe548161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe546ffd640 1 -- 172.21.15.80:0/1886423742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5480733b0 msgr2=0x7fe548161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:22.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe546ffd640 1 --2- 172.21.15.80:0/1886423742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5480733b0 0x7fe548161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe546ffd640 1 -- 172.21.15.80:0/1886423742 >> v1:172.21.15.87:6789/0 conn(0x7fe548161070 legacy=0x7fe5480b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:22.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe546ffd640 1 -- 172.21.15.80:0/1886423742 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5480b0c60 con 0x7fe54806f5a0 2024-04-25T13:45:22.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.743+0000 7fe5477fe640 1 --2- 172.21.15.80:0/1886423742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5480733b0 0x7fe548161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:22.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe546ffd640 1 --2- 172.21.15.80:0/1886423742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54806f5a0 0x7fe548160b30 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe54000bb40 tx=0x7fe54001eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdff207a7960e2fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:22.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe545ffb640 1 -- 172.21.15.80:0/1886423742 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5400b4070 con 0x7fe54806f5a0 2024-04-25T13:45:22.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe545ffb640 1 -- 172.21.15.80:0/1886423742 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe5400a79e0 con 0x7fe54806f5a0 2024-04-25T13:45:22.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe545ffb640 1 -- 172.21.15.80:0/1886423742 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5400a7db0 con 0x7fe54806f5a0 2024-04-25T13:45:22.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 -- 172.21.15.80:0/1886423742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54806f5a0 msgr2=0x7fe548160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 --2- 172.21.15.80:0/1886423742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54806f5a0 0x7fe548160b30 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe54000bb40 tx=0x7fe54001eb40 comp rx=0 tx=0).stop 2024-04-25T13:45:22.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 -- 172.21.15.80:0/1886423742 shutdown_connections 2024-04-25T13:45:22.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 --2- 172.21.15.80:0/1886423742 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5480733b0 0x7fe548161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 --2- 172.21.15.80:0/1886423742 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54806f5a0 0x7fe548160b30 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 -- 172.21.15.80:0/1886423742 >> 172.21.15.80:0/1886423742 conn(0x7fe54806d080 msgr2=0x7fe5480b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:22.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 -- 172.21.15.80:0/1886423742 shutdown_connections 2024-04-25T13:45:22.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 -- 172.21.15.80:0/1886423742 wait complete. 2024-04-25T13:45:22.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 Processor -- start 2024-04-25T13:45:22.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.747+0000 7fe54d958640 1 -- start start 2024-04-25T13:45:22.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54806f5a0 0x7fe54814d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5480733b0 0x7fe548158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe548161070 0x7fe548158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe54806e610 con 0x7fe5480733b0 2024-04-25T13:45:22.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe54806e490 con 0x7fe54806f5a0 2024-04-25T13:45:22.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe54806e790 con 0x7fe548161070 2024-04-25T13:45:22.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5480733b0 0x7fe548158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5480733b0 0x7fe548158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35922/0 (socket says 172.21.15.80:35922) 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 -- 172.21.15.80:0/4047353833 learned_addr learned my addr 172.21.15.80:0/4047353833 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5477fe640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe548161070 0x7fe548158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 -- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe548161070 msgr2=0x7fe548158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe548161070 0x7fe548158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 -- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54806f5a0 msgr2=0x7fe54814d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54806f5a0 0x7fe54814d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 -- 172.21.15.80:0/4047353833 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54814d5f0 con 0x7fe5480733b0 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe5467fc640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5480733b0 0x7fe548158430 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00b440 tx=0x7fe53c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe52ffff640 1 -- 172.21.15.80:0/4047353833 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c015020 con 0x7fe5480733b0 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe52ffff640 1 -- 172.21.15.80:0/4047353833 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe53c011890 con 0x7fe5480733b0 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54815e350 con 0x7fe5480733b0 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe52ffff640 1 -- 172.21.15.80:0/4047353833 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53c011c80 con 0x7fe5480733b0 2024-04-25T13:45:22.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe548157c30 con 0x7fe5480733b0 2024-04-25T13:45:22.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54806a3a0 con 0x7fe5480733b0 2024-04-25T13:45:22.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe52ffff640 1 -- 172.21.15.80:0/4047353833 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe53c018020 con 0x7fe5480733b0 2024-04-25T13:45:22.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe52ffff640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe53406f400 0x7fe534071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:22.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe52ffff640 1 -- 172.21.15.80:0/4047353833 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7fe53c08f0e0 con 0x7fe5480733b0 2024-04-25T13:45:22.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.751+0000 7fe546ffd640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe53406f400 0x7fe534071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:22.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.755+0000 7fe546ffd640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe53406f400 0x7fe534071910 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe5400b1000 tx=0x7fe540004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:22.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.755+0000 7fe52ffff640 1 -- 172.21.15.80:0/4047353833 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe53c01f020 con 0x7fe5480733b0 2024-04-25T13:45:22.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.911+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe54815e0e0 con 0x7fe5480733b0 2024-04-25T13:45:22.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe52ffff640 1 -- 172.21.15.80:0/4047353833 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+12947 (secure 0 0 0) 0x7fe53c00d040 con 0x7fe5480733b0 2024-04-25T13:45:22.913 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:22.913 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":95,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:21.551373+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/642879744":"2024-04-26T13:45:18.536681+0000","172.21.15.87:6836/642879744":"2024-04-26T13:45:18.536681+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe53406f400 msgr2=0x7fe534071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe53406f400 0x7fe534071910 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fe5400b1000 tx=0x7fe540004740 comp rx=0 tx=0).stop 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5480733b0 msgr2=0x7fe548158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5480733b0 0x7fe548158430 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fe53c00b440 tx=0x7fe53c00b900 comp rx=0 tx=0).stop 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 shutdown_connections 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe53406f400 0x7fe534071910 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe548161070 0x7fe548158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5480733b0 0x7fe548158430 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 --2- 172.21.15.80:0/4047353833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54806f5a0 0x7fe54814d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 >> 172.21.15.80:0/4047353833 conn(0x7fe54806d080 msgr2=0x7fe5480b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 shutdown_connections 2024-04-25T13:45:22.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:22.915+0000 7fe54d958640 1 -- 172.21.15.80:0/4047353833 wait complete. 2024-04-25T13:45:22.935 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:45:22.935 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:45:23.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.087+0000 7f161b150640 1 Processor -- start 2024-04-25T13:45:23.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.087+0000 7f161b150640 1 -- start start 2024-04-25T13:45:23.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f161b150640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 0x7f1614160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f161b150640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1614161160 0x7f16140d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f161b150640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1614161570 con 0x7f16140d2170 2024-04-25T13:45:23.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f161b150640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1614070ea0 con 0x7f1614161160 2024-04-25T13:45:23.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f161b150640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1614070290 con 0x7f161406fb90 2024-04-25T13:45:23.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1613fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1614161160 0x7f16140d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1613fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1614161160 0x7f16140d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35932/0 (socket says 172.21.15.80:35932) 2024-04-25T13:45:23.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1613fff640 1 -- 172.21.15.80:0/1064706229 learned_addr learned my addr 172.21.15.80:0/1064706229 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:23.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1618eec640 1 --2- 172.21.15.80:0/1064706229 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 0x7f1614160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1613fff640 1 -- 172.21.15.80:0/1064706229 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 msgr2=0x7f1614160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:23.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1613fff640 1 --2- 172.21.15.80:0/1064706229 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 0x7f1614160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1613fff640 1 -- 172.21.15.80:0/1064706229 >> v1:172.21.15.87:6789/0 conn(0x7f16140d2170 legacy=0x7f16140b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:23.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1613fff640 1 -- 172.21.15.80:0/1064706229 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1614070570 con 0x7f1614161160 2024-04-25T13:45:23.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f1613fff640 1 --2- 172.21.15.80:0/1064706229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1614161160 0x7f16140d1c30 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1608009870 tx=0x7f160801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b4c0ad24db944b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:23.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f16137fe640 1 -- 172.21.15.80:0/1064706229 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080b0070 con 0x7f1614161160 2024-04-25T13:45:23.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f16137fe640 1 -- 172.21.15.80:0/1064706229 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16080ab600 con 0x7f1614161160 2024-04-25T13:45:23.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.091+0000 7f16137fe640 1 -- 172.21.15.80:0/1064706229 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080ab9d0 con 0x7f1614161160 2024-04-25T13:45:23.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- 172.21.15.80:0/1064706229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1614161160 msgr2=0x7f16140d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 --2- 172.21.15.80:0/1064706229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1614161160 0x7f16140d1c30 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1608009870 tx=0x7f160801c280 comp rx=0 tx=0).stop 2024-04-25T13:45:23.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- 172.21.15.80:0/1064706229 shutdown_connections 2024-04-25T13:45:23.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 --2- 172.21.15.80:0/1064706229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1614161160 0x7f16140d1c30 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 --2- 172.21.15.80:0/1064706229 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 0x7f1614160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- 172.21.15.80:0/1064706229 >> 172.21.15.80:0/1064706229 conn(0x7f1614073790 msgr2=0x7f1614073ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:23.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- 172.21.15.80:0/1064706229 shutdown_connections 2024-04-25T13:45:23.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- 172.21.15.80:0/1064706229 wait complete. 2024-04-25T13:45:23.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 Processor -- start 2024-04-25T13:45:23.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- start start 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 0x7f16141435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16140d2170 0x7f161414c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614161160 0x7f16140b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f161416a140 con 0x7f16140d2170 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1614169fc0 con 0x7f1614161160 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f161b150640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f161416a2c0 con 0x7f161406fb90 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f1613fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16140d2170 0x7f161414c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f1613fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16140d2170 0x7f161414c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35942/0 (socket says 172.21.15.80:35942) 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.095+0000 7f1613fff640 1 -- 172.21.15.80:0/3592679801 learned_addr learned my addr 172.21.15.80:0/3592679801 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f1618eec640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 0x7f16141435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f1613fff640 1 -- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 msgr2=0x7f16141435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f1613fff640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 0x7f16141435a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f1613fff640 1 -- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614161160 msgr2=0x7f16140b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f16196ed640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614161160 0x7f16140b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f1613fff640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614161160 0x7f16140b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f1613fff640 1 -- 172.21.15.80:0/3592679801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16141516f0 con 0x7f16140d2170 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f16196ed640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614161160 0x7f16140b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f1613fff640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16140d2170 0x7f161414c430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f16080ae040 tx=0x7f1608019020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f16117fa640 1 -- 172.21.15.80:0/3592679801 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080ba020 con 0x7f16140d2170 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f16117fa640 1 -- 172.21.15.80:0/3592679801 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16080b6620 con 0x7f16140d2170 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1614151410 con 0x7f16140d2170 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f16117fa640 1 -- 172.21.15.80:0/3592679801 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16080b0040 con 0x7f16140d2170 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16140b2a60 con 0x7f16140d2170 2024-04-25T13:45:23.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f161406a3a0 con 0x7f16140d2170 2024-04-25T13:45:23.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f16117fa640 1 -- 172.21.15.80:0/3592679801 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f16080bd020 con 0x7f16140d2170 2024-04-25T13:45:23.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f16117fa640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f160006f400 0x7f1600071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f16117fa640 1 -- 172.21.15.80:0/3592679801 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f1608128d90 con 0x7f16140d2170 2024-04-25T13:45:23.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.099+0000 7f1618eec640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f160006f400 0x7f1600071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.103+0000 7f1618eec640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f160006f400 0x7f1600071910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16040096f0 tx=0x7f1604008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:23.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.107+0000 7f16117fa640 1 -- 172.21.15.80:0/3592679801 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1614161570 con 0x7f16140d2170 2024-04-25T13:45:23.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1614075fb0 con 0x7f16140d2170 2024-04-25T13:45:23.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f16117fa640 1 -- 172.21.15.80:0/3592679801 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) ==== 93+0+10 (secure 0 0 0) 0x7f16080f6010 con 0x7f16140d2170 2024-04-25T13:45:23.266 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f160006f400 msgr2=0x7f1600071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f160006f400 0x7f1600071910 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f16040096f0 tx=0x7f1604008040 comp rx=0 tx=0).stop 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16140d2170 msgr2=0x7f161414c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16140d2170 0x7f161414c430 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f16080ae040 tx=0x7f1608019020 comp rx=0 tx=0).stop 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 shutdown_connections 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f160006f400 0x7f1600071910 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1614161160 0x7f16140b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16140d2170 0x7f161414c430 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 --2- 172.21.15.80:0/3592679801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f161406fb90 0x7f16141435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 >> 172.21.15.80:0/3592679801 conn(0x7f1614073790 msgr2=0x7f1614148280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.267+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 shutdown_connections 2024-04-25T13:45:23.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.271+0000 7f161b150640 1 -- 172.21.15.80:0/3592679801 wait complete. 2024-04-25T13:45:23.289 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-04-25T13:45:23.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf89ccf640 1 Processor -- start 2024-04-25T13:45:23.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf89ccf640 1 -- start start 2024-04-25T13:45:23.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf89ccf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf840cd760 0x7fcf840cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf89ccf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcf8415eeb0 0x7fcf840ce090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf89ccf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf840cc420 con 0x7fcf840ce5d0 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf89ccf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcf840ce9e0 con 0x7fcf8415eeb0 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf89ccf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8406dd00 con 0x7fcf840cd760 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf894ce640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fcf840ce5d0 0x7fcf840c6900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51588/0 (socket says 172.21.15.80:51588) 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf83fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcf8415eeb0 0x7fcf840ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf894ce640 1 -- 172.21.15.80:0/1892364928 learned_addr learned my addr 172.21.15.80:0/1892364928 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.431+0000 7fcf88ccd640 1 --2- 172.21.15.80:0/1892364928 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf840cd760 0x7fcf840cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf83fff640 1 -- 172.21.15.80:0/1892364928 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf840cd760 msgr2=0x7fcf840cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf83fff640 1 --2- 172.21.15.80:0/1892364928 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf840cd760 0x7fcf840cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf83fff640 1 -- 172.21.15.80:0/1892364928 >> v1:172.21.15.80:6789/0 conn(0x7fcf840ce5d0 legacy=0x7fcf840c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf83fff640 1 -- 172.21.15.80:0/1892364928 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8406dfe0 con 0x7fcf8415eeb0 2024-04-25T13:45:23.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf83fff640 1 --2- 172.21.15.80:0/1892364928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf8415eeb0 0x7fcf840ce090 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcf780097c0 tx=0x7fcf7801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5af1cabc04d6756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:23.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf837fe640 1 -- 172.21.15.80:0/1892364928 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf780b0070 con 0x7fcf8415eeb0 2024-04-25T13:45:23.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf837fe640 1 -- 172.21.15.80:0/1892364928 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf780ab600 con 0x7fcf8415eeb0 2024-04-25T13:45:23.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf837fe640 1 -- 172.21.15.80:0/1892364928 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf780ab9d0 con 0x7fcf8415eeb0 2024-04-25T13:45:23.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1892364928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf8415eeb0 msgr2=0x7fcf840ce090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1892364928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf8415eeb0 0x7fcf840ce090 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fcf780097c0 tx=0x7fcf7801c4a0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1892364928 shutdown_connections 2024-04-25T13:45:23.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1892364928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf8415eeb0 0x7fcf840ce090 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1892364928 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf840cd760 0x7fcf840cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1892364928 >> 172.21.15.80:0/1892364928 conn(0x7fcf840b34a0 msgr2=0x7fcf840b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:23.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1892364928 shutdown_connections 2024-04-25T13:45:23.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.435+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1892364928 wait complete. 2024-04-25T13:45:23.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 Processor -- start 2024-04-25T13:45:23.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 -- start start 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf840cd760 0x7fcf8414aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf840ce5d0 0x7fcf841523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf8415eeb0 0x7fcf84152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8415f2c0 con 0x7fcf840ce5d0 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcf8416b5d0 con 0x7fcf840cd760 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcf8415b7d0 con 0x7fcf8415eeb0 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf840ce5d0 0x7fcf841523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf840ce5d0 0x7fcf841523f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35960/0 (socket says 172.21.15.80:35960) 2024-04-25T13:45:23.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 -- 172.21.15.80:0/1333179786 learned_addr learned my addr 172.21.15.80:0/1333179786 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf894ce640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf8415eeb0 0x7fcf84152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf88ccd640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf840cd760 0x7fcf8414aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 -- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf8415eeb0 msgr2=0x7fcf84152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf8415eeb0 0x7fcf84152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 -- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf840cd760 msgr2=0x7fcf8414aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf840cd760 0x7fcf8414aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 -- 172.21.15.80:0/1333179786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf8414a130 con 0x7fcf840ce5d0 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf894ce640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf8415eeb0 0x7fcf84152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf83fff640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf840ce5d0 0x7fcf841523f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcf7801a0d0 tx=0x7fcf78019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf817fa640 1 -- 172.21.15.80:0/1333179786 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf780b0040 con 0x7fcf840ce5d0 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf817fa640 1 -- 172.21.15.80:0/1333179786 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf780ab400 con 0x7fcf840ce5d0 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf8414a410 con 0x7fcf840ce5d0 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf817fa640 1 -- 172.21.15.80:0/1333179786 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf780b4630 con 0x7fcf840ce5d0 2024-04-25T13:45:23.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.439+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf8414a9d0 con 0x7fcf840ce5d0 2024-04-25T13:45:23.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.443+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf8406a2c0 con 0x7fcf840ce5d0 2024-04-25T13:45:23.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.443+0000 7fcf817fa640 1 -- 172.21.15.80:0/1333179786 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcf780bc020 con 0x7fcf840ce5d0 2024-04-25T13:45:23.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.443+0000 7fcf817fa640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf7006f430 0x7fcf70071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:23.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.443+0000 7fcf817fa640 1 -- 172.21.15.80:0/1333179786 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7fcf78128370 con 0x7fcf840ce5d0 2024-04-25T13:45:23.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.443+0000 7fcf88ccd640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf7006f430 0x7fcf70071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:23.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.443+0000 7fcf88ccd640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf7006f430 0x7fcf70071940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcf840cec60 tx=0x7fcf74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:23.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.447+0000 7fcf817fa640 1 -- 172.21.15.80:0/1333179786 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcf780c3020 con 0x7fcf840ce5d0 2024-04-25T13:45:23.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:23.651+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) -- 0x7fcf84159460 con 0x7fcf840ce5d0 2024-04-25T13:45:24.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.575+0000 7fcf817fa640 1 -- 172.21.15.80:0/1333179786 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v264) ==== 140+0+0 (secure 0 0 0) 0x7fcf78016090 con 0x7fcf840ce5d0 2024-04-25T13:45:24.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf7006f430 msgr2=0x7fcf70071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:24.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf7006f430 0x7fcf70071940 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fcf840cec60 tx=0x7fcf74008040 comp rx=0 tx=0).stop 2024-04-25T13:45:24.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf840ce5d0 msgr2=0x7fcf841523f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:24.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf840ce5d0 0x7fcf841523f0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fcf7801a0d0 tx=0x7fcf78019240 comp rx=0 tx=0).stop 2024-04-25T13:45:24.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 shutdown_connections 2024-04-25T13:45:24.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcf7006f430 0x7fcf70071940 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcf8415eeb0 0x7fcf84152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcf840ce5d0 0x7fcf841523f0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 --2- 172.21.15.80:0/1333179786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf840cd760 0x7fcf8414aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 >> 172.21.15.80:0/1333179786 conn(0x7fcf840b34a0 msgr2=0x7fcf841606e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:24.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 shutdown_connections 2024-04-25T13:45:24.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.579+0000 7fcf89ccf640 1 -- 172.21.15.80:0/1333179786 wait complete. 2024-04-25T13:45:24.603 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2024-04-25T13:45:24.770 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_objectstore":"memstore"} 2024-04-25T13:45:24.771 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2024-04-25T13:45:24.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:45:24.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d7fa2640 1 Processor -- start 2024-04-25T13:45:24.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d7fa2640 1 -- start start 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d7fa2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f87d0071e60 0x7f87d01574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d7fa2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f87d0073c40 0x7f87d0160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d7fa2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87d01613b0 con 0x7f87d0160fa0 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d7fa2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f87d00b1350 con 0x7f87d0073c40 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d7fa2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f87d006f9a0 con 0x7f87d0071e60 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d553d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f87d0073c40 0x7f87d0160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d553d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f87d0073c40 0x7f87d0160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35974/0 (socket says 172.21.15.80:35974) 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d553d640 1 -- 172.21.15.80:0/962198193 learned_addr learned my addr 172.21.15.80:0/962198193 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d5d3e640 1 --2- 172.21.15.80:0/962198193 >> v2:172.21.15.87:3300/0 conn(0x7f87d0071e60 0x7f87d01574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d553d640 1 -- 172.21.15.80:0/962198193 >> v2:172.21.15.87:3300/0 conn(0x7f87d0071e60 msgr2=0x7f87d01574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d553d640 1 --2- 172.21.15.80:0/962198193 >> v2:172.21.15.87:3300/0 conn(0x7f87d0071e60 0x7f87d01574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d553d640 1 -- 172.21.15.80:0/962198193 >> v1:172.21.15.87:6789/0 conn(0x7f87d0160fa0 legacy=0x7f87d00b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:24.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.931+0000 7f87d553d640 1 -- 172.21.15.80:0/962198193 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d006fbf0 con 0x7f87d0073c40 2024-04-25T13:45:24.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d553d640 1 --2- 172.21.15.80:0/962198193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d0160a60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f87d0071630 tx=0x7f87c001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410d9625c1c3e0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:24.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d4d3c640 1 -- 172.21.15.80:0/962198193 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00b4070 con 0x7f87d0073c40 2024-04-25T13:45:24.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d4d3c640 1 -- 172.21.15.80:0/962198193 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c00a7ab0 con 0x7f87d0073c40 2024-04-25T13:45:24.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d4d3c640 1 -- 172.21.15.80:0/962198193 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00af440 con 0x7f87d0073c40 2024-04-25T13:45:24.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d7fa2640 1 -- 172.21.15.80:0/962198193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 msgr2=0x7f87d0160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:24.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/962198193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d0160a60 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f87d0071630 tx=0x7f87c001ebf0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d7fa2640 1 -- 172.21.15.80:0/962198193 shutdown_connections 2024-04-25T13:45:24.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/962198193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d0160a60 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/962198193 >> v2:172.21.15.87:3300/0 conn(0x7f87d0071e60 0x7f87d01574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d7fa2640 1 -- 172.21.15.80:0/962198193 >> 172.21.15.80:0/962198193 conn(0x7f87d00b4160 msgr2=0x7f87d00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:24.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d7fa2640 1 -- 172.21.15.80:0/962198193 shutdown_connections 2024-04-25T13:45:24.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.935+0000 7f87d7fa2640 1 -- 172.21.15.80:0/962198193 wait complete. 2024-04-25T13:45:24.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 Processor -- start 2024-04-25T13:45:24.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 -- start start 2024-04-25T13:45:24.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87d0071e60 0x7f87d0141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:24.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d01663b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:24.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d0160fa0 0x7f87d01668f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:24.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0074050 con 0x7f87d0073c40 2024-04-25T13:45:24.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f87d0072270 con 0x7f87d0160fa0 2024-04-25T13:45:24.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f87d00b56c0 con 0x7f87d0071e60 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d01663b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d01663b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35976/0 (socket says 172.21.15.80:35976) 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 -- 172.21.15.80:0/1845154790 learned_addr learned my addr 172.21.15.80:0/1845154790 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d5d3e640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87d0071e60 0x7f87d0141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 -- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87d0071e60 msgr2=0x7f87d0141d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87d0071e60 0x7f87d0141d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 -- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d0160fa0 msgr2=0x7f87d01668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d653f640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d0160fa0 0x7f87d01668f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d0160fa0 0x7f87d01668f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 -- 172.21.15.80:0/1845154790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d0142630 con 0x7f87d0073c40 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d553d640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d01663b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f87c001b800 tx=0x7f87c001b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87c67fc640 1 -- 172.21.15.80:0/1845154790 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00b4040 con 0x7f87d0073c40 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87c67fc640 1 -- 172.21.15.80:0/1845154790 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c00bf070 con 0x7f87d0073c40 2024-04-25T13:45:24.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d0142910 con 0x7f87d0073c40 2024-04-25T13:45:24.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87c67fc640 1 -- 172.21.15.80:0/1845154790 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c00ba6a0 con 0x7f87d0073c40 2024-04-25T13:45:24.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d0142ed0 con 0x7f87d0073c40 2024-04-25T13:45:24.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.939+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d01423a0 con 0x7f87d0073c40 2024-04-25T13:45:24.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.943+0000 7f87c67fc640 1 -- 172.21.15.80:0/1845154790 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f87c00ba840 con 0x7f87d0073c40 2024-04-25T13:45:24.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.943+0000 7f87c67fc640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87b406f400 0x7f87b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:24.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.943+0000 7f87c67fc640 1 -- 172.21.15.80:0/1845154790 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f87c012c5d0 con 0x7f87d0073c40 2024-04-25T13:45:24.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.943+0000 7f87d5d3e640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87b406f400 0x7f87b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:24.942 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.943+0000 7f87d5d3e640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87b406f400 0x7f87b4071910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b930 tx=0x7f87cc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:24.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:24.947+0000 7f87c67fc640 1 -- 172.21.15.80:0/1845154790 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f87c00187e0 con 0x7f87d0073c40 2024-04-25T13:45:25.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.111+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f87d006a3a0 con 0x7f87d0073c40 2024-04-25T13:45:25.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.115+0000 7f87c67fc640 1 -- 172.21.15.80:0/1845154790 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f87c00f9850 con 0x7f87d0073c40 2024-04-25T13:45:25.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87b406f400 msgr2=0x7f87b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:25.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87b406f400 0x7f87b4071910 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f87cc00b930 tx=0x7f87cc008040 comp rx=0 tx=0).stop 2024-04-25T13:45:25.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 msgr2=0x7f87d01663b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:25.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d01663b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f87c001b800 tx=0x7f87c001b990 comp rx=0 tx=0).stop 2024-04-25T13:45:25.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 shutdown_connections 2024-04-25T13:45:25.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f87b406f400 0x7f87b4071910 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f87d0160fa0 0x7f87d01668f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f87d0073c40 0x7f87d01663b0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 --2- 172.21.15.80:0/1845154790 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f87d0071e60 0x7f87d0141d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 >> 172.21.15.80:0/1845154790 conn(0x7f87d00b4160 msgr2=0x7f87d00b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:25.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 shutdown_connections 2024-04-25T13:45:25.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.119+0000 7f87d7fa2640 1 -- 172.21.15.80:0/1845154790 wait complete. 2024-04-25T13:45:25.138 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:45:25.138 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:45:25.187 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:45:25.236 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:45:25.236 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:25.236 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:25.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:45:25.242 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:45:25.242 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:25.247 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:45:25.247 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:25.247 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:25.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:45:25.291 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:45:25.291 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:25.297 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:45:25.297 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:45:25.297 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:25.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f9610640 1 Processor -- start 2024-04-25T13:45:25.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f9610640 1 -- start start 2024-04-25T13:45:25.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f9610640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 0x7f57f4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f9610640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57f40733b0 0x7f57f4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f9610640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f406fcf0 con 0x7f57f4161070 2024-04-25T13:45:25.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f9610640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f57f41619c0 con 0x7f57f40733b0 2024-04-25T13:45:25.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f9610640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57f4073f10 con 0x7f57f406f5a0 2024-04-25T13:45:25.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57f40733b0 0x7f57f4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f57f40733b0 0x7f57f4161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35984/0 (socket says 172.21.15.80:35984) 2024-04-25T13:45:25.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f37fe640 1 -- 172.21.15.80:0/4225607174 learned_addr learned my addr 172.21.15.80:0/4225607174 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:25.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f2ffd640 1 --2- 172.21.15.80:0/4225607174 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 0x7f57f4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f37fe640 1 -- 172.21.15.80:0/4225607174 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 msgr2=0x7f57f4160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:25.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f37fe640 1 --2- 172.21.15.80:0/4225607174 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 0x7f57f4160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f37fe640 1 -- 172.21.15.80:0/4225607174 >> v1:172.21.15.87:6789/0 conn(0x7f57f4161070 legacy=0x7f57f40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:25.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.491+0000 7f57f37fe640 1 -- 172.21.15.80:0/4225607174 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f40b0c60 con 0x7f57f40733b0 2024-04-25T13:45:25.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f37fe640 1 --2- 172.21.15.80:0/4225607174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40733b0 0x7f57f4161480 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e80 tx=0x7f57ec01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=493cafa0396af5f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:25.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f27fc640 1 -- 172.21.15.80:0/4225607174 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0a55d0 con 0x7f57f40733b0 2024-04-25T13:45:25.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f27fc640 1 -- 172.21.15.80:0/4225607174 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec0a5770 con 0x7f57f40733b0 2024-04-25T13:45:25.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f27fc640 1 -- 172.21.15.80:0/4225607174 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0a5b40 con 0x7f57f40733b0 2024-04-25T13:45:25.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 -- 172.21.15.80:0/4225607174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40733b0 msgr2=0x7f57f4161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 --2- 172.21.15.80:0/4225607174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40733b0 0x7f57f4161480 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009e80 tx=0x7f57ec01c940 comp rx=0 tx=0).stop 2024-04-25T13:45:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 -- 172.21.15.80:0/4225607174 shutdown_connections 2024-04-25T13:45:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 --2- 172.21.15.80:0/4225607174 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f40733b0 0x7f57f4161480 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 --2- 172.21.15.80:0/4225607174 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 0x7f57f4160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 -- 172.21.15.80:0/4225607174 >> 172.21.15.80:0/4225607174 conn(0x7f57f406d080 msgr2=0x7f57f40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 -- 172.21.15.80:0/4225607174 shutdown_connections 2024-04-25T13:45:25.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 -- 172.21.15.80:0/4225607174 wait complete. 2024-04-25T13:45:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 Processor -- start 2024-04-25T13:45:25.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.495+0000 7f57f9610640 1 -- start start 2024-04-25T13:45:25.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 0x7f57f4153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40733b0 0x7f57f414e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f4161070 0x7f57f41512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57f406e610 con 0x7f57f4161070 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57f406e490 con 0x7f57f40733b0 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57f406e790 con 0x7f57f406f5a0 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f4161070 0x7f57f41512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f4161070 0x7f57f41512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:35988/0 (socket says 172.21.15.80:35988) 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 -- 172.21.15.80:0/2389397609 learned_addr learned my addr 172.21.15.80:0/2389397609 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f2ffd640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 0x7f57f4153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57ea7fc640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40733b0 0x7f57f414e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 -- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 msgr2=0x7f57f4153f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 0x7f57f4153f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 -- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40733b0 msgr2=0x7f57f414e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40733b0 0x7f57f414e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 -- 172.21.15.80:0/2389397609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f40d08d0 con 0x7f57f4161070 2024-04-25T13:45:25.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f37fe640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f4161070 0x7f57f41512a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f57ec01ce20 tx=0x7f57ec0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:25.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57ebfff640 1 -- 172.21.15.80:0/2389397609 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0ba020 con 0x7f57f4161070 2024-04-25T13:45:25.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57ebfff640 1 -- 172.21.15.80:0/2389397609 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec0b6660 con 0x7f57f4161070 2024-04-25T13:45:25.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f40d0bb0 con 0x7f57f4161070 2024-04-25T13:45:25.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57ebfff640 1 -- 172.21.15.80:0/2389397609 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec009510 con 0x7f57f4161070 2024-04-25T13:45:25.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f415e8c0 con 0x7f57f4161070 2024-04-25T13:45:25.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f406a3a0 con 0x7f57f4161070 2024-04-25T13:45:25.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.499+0000 7f57ebfff640 1 -- 172.21.15.80:0/2389397609 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57ec0bd020 con 0x7f57f4161070 2024-04-25T13:45:25.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.503+0000 7f57ebfff640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d806f430 0x7f57d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.503+0000 7f57ebfff640 1 -- 172.21.15.80:0/2389397609 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7f57ec129210 con 0x7f57f4161070 2024-04-25T13:45:25.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.503+0000 7f57f2ffd640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d806f430 0x7f57d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.503+0000 7f57f2ffd640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d806f430 0x7f57d8071940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57f40b6e70 tx=0x7f57dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:25.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.507+0000 7f57ebfff640 1 -- 172.21.15.80:0/2389397609 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57ec0c5020 con 0x7f57f4161070 2024-04-25T13:45:25.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.703+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57f40739c0 con 0x7f57f4161070 2024-04-25T13:45:25.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.703+0000 7f57ebfff640 1 -- 172.21.15.80:0/2389397609 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+3869 (secure 0 0 0) 0x7f57ec0165c0 con 0x7f57f4161070 2024-04-25T13:45:25.704 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:25.704 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5713,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2082291876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2082291876},{"type":"v1","addr":"172.21.15.87:6835","nonce":2082291876}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":5875,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4088424705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4088424705},{"type":"v1","addr":"172.21.15.87:6837","nonce":4088424705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43},{"gid":6612,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1337331528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1337331528},{"type":"v1","addr":"172.21.15.80:6833","nonce":1337331528}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36},{"gid":6618,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2521189874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2521189874},{"type":"v1","addr":"172.21.15.80:6835","nonce":2521189874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":36}],"filesystems":[]} 2024-04-25T13:45:25.705 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 43 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d806f430 msgr2=0x7f57d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d806f430 0x7f57d8071940 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f57f40b6e70 tx=0x7f57dc008040 comp rx=0 tx=0).stop 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f4161070 msgr2=0x7f57f41512a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f4161070 0x7f57f41512a0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f57ec01ce20 tx=0x7f57ec0196e0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 shutdown_connections 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d806f430 0x7f57d8071940 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57f4161070 0x7f57f41512a0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f40733b0 0x7f57f414e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 --2- 172.21.15.80:0/2389397609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f406f5a0 0x7f57f4153f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 >> 172.21.15.80:0/2389397609 conn(0x7f57f406d080 msgr2=0x7f57f4155660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 shutdown_connections 2024-04-25T13:45:25.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.707+0000 7f57f9610640 1 -- 172.21.15.80:0/2389397609 wait complete. 2024-04-25T13:45:25.727 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:45:25.727 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:45:25.727 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:45:25.727 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:45:25.728 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:45:25.728 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:45:25.728 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:45:25.728 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:45:25.728 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:45:25.728 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:45:25.729 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:45:25.729 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:45:25.729 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:45:25.729 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:45:25.729 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:45:25.729 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:45:25.742 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:45:25.743 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:45:25.744 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:45:25.744 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:45:25.747 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:45:25.748 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:45:25.750 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:45:25.750 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:45:25.789 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:45:25.790 DEBUG:teuthology.parallel:result is None 2024-04-25T13:45:25.790 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:45:25.791 DEBUG:teuthology.parallel:result is None 2024-04-25T13:45:25.799 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:45:25.847 DEBUG:teuthology.parallel:result is None 2024-04-25T13:45:25.848 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:45:25.849 DEBUG:teuthology.parallel:result is None 2024-04-25T13:45:25.849 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:45:25.887 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:45:25.887 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:45:25.903 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:45:25.942 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:45:25.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab18d84640 1 Processor -- start 2024-04-25T13:45:25.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab18d84640 1 -- start start 2024-04-25T13:45:25.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab18d84640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fab140b4950 0x7fab141603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab18d84640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fab140b93e0 0x7fab140d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab18d84640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140d2940 0x7fab140d4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab18d84640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fab14164e80 con 0x7fab140b93e0 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab18d84640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fab14164270 con 0x7fab140b4950 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab18d84640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fab14164550 con 0x7fab140d2940 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab11d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fab140b93e0 0x7fab140d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab11d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fab140b93e0 0x7fab140d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36036/0 (socket says 172.21.15.80:36036) 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab11d9b640 1 -- 172.21.15.80:0/28620040 learned_addr learned my addr 172.21.15.80:0/28620040 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.995+0000 7fab1259c640 1 --2- 172.21.15.80:0/28620040 >> v2:172.21.15.87:3300/0 conn(0x7fab140b4950 0x7fab141603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab12d9d640 1 --2- 172.21.15.80:0/28620040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140d2940 0x7fab140d4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab11d9b640 1 -- 172.21.15.80:0/28620040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140d2940 msgr2=0x7fab140d4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab11d9b640 1 --2- 172.21.15.80:0/28620040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140d2940 0x7fab140d4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab11d9b640 1 -- 172.21.15.80:0/28620040 >> v2:172.21.15.87:3300/0 conn(0x7fab140b4950 msgr2=0x7fab141603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:25.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab11d9b640 1 --2- 172.21.15.80:0/28620040 >> v2:172.21.15.87:3300/0 conn(0x7fab140b4950 0x7fab141603f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:25.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab11d9b640 1 -- 172.21.15.80:0/28620040 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab14164830 con 0x7fab140b93e0 2024-04-25T13:45:25.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab11d9b640 1 --2- 172.21.15.80:0/28620040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140b93e0 0x7fab140d2400 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fab1406dcc0 tx=0x7fab0801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a36bae415750e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:25.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab1159a640 1 -- 172.21.15.80:0/28620040 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080ab5f0 con 0x7fab140b93e0 2024-04-25T13:45:25.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab1159a640 1 -- 172.21.15.80:0/28620040 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab080b0070 con 0x7fab140b93e0 2024-04-25T13:45:25.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab1159a640 1 -- 172.21.15.80:0/28620040 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab080ab9b0 con 0x7fab140b93e0 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 -- 172.21.15.80:0/28620040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140b93e0 msgr2=0x7fab140d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 --2- 172.21.15.80:0/28620040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140b93e0 0x7fab140d2400 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fab1406dcc0 tx=0x7fab0801c520 comp rx=0 tx=0).stop 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 -- 172.21.15.80:0/28620040 shutdown_connections 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 --2- 172.21.15.80:0/28620040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140d2940 0x7fab140d4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 --2- 172.21.15.80:0/28620040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140b93e0 0x7fab140d2400 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 --2- 172.21.15.80:0/28620040 >> v2:172.21.15.87:3300/0 conn(0x7fab140b4950 0x7fab141603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 -- 172.21.15.80:0/28620040 >> 172.21.15.80:0/28620040 conn(0x7fab1406c5f0 msgr2=0x7fab1406ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 -- 172.21.15.80:0/28620040 shutdown_connections 2024-04-25T13:45:26.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:25.999+0000 7fab18d84640 1 -- 172.21.15.80:0/28620040 wait complete. 2024-04-25T13:45:26.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 Processor -- start 2024-04-25T13:45:26.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 -- start start 2024-04-25T13:45:26.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab140b4950 0x7fab1415f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140b93e0 0x7fab140cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140d2940 0x7fab14157c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fab140b77b0 con 0x7fab140d2940 2024-04-25T13:45:26.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fab14163f00 con 0x7fab140b4950 2024-04-25T13:45:26.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fab140b7930 con 0x7fab140b93e0 2024-04-25T13:45:26.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140d2940 0x7fab14157c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140d2940 0x7fab14157c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36040/0 (socket says 172.21.15.80:36040) 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 -- 172.21.15.80:0/3795480055 learned_addr learned my addr 172.21.15.80:0/3795480055 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab11d9b640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140b93e0 0x7fab140cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab1259c640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab140b4950 0x7fab1415f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 -- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140b93e0 msgr2=0x7fab140cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140b93e0 0x7fab140cf3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 -- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab140b4950 msgr2=0x7fab1415f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab140b4950 0x7fab1415f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 -- 172.21.15.80:0/3795480055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab1415dae0 con 0x7fab140d2940 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab12d9d640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140d2940 0x7fab14157c90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fab0c007d70 tx=0x7fab0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:26.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab02ffd640 1 -- 172.21.15.80:0/3795480055 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c014420 con 0x7fab140d2940 2024-04-25T13:45:26.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab02ffd640 1 -- 172.21.15.80:0/3795480055 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fab0c019070 con 0x7fab140d2940 2024-04-25T13:45:26.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab1415ddc0 con 0x7fab140d2940 2024-04-25T13:45:26.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab02ffd640 1 -- 172.21.15.80:0/3795480055 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0c014820 con 0x7fab140d2940 2024-04-25T13:45:26.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.003+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab1415e380 con 0x7fab140d2940 2024-04-25T13:45:26.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.007+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1406a3a0 con 0x7fab140d2940 2024-04-25T13:45:26.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.007+0000 7fab02ffd640 1 -- 172.21.15.80:0/3795480055 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fab0c0149c0 con 0x7fab140d2940 2024-04-25T13:45:26.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.007+0000 7fab02ffd640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faaf406f400 0x7faaf4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.007+0000 7fab02ffd640 1 -- 172.21.15.80:0/3795480055 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 4943+0+0 (secure 0 0 0) 0x7fab0c08f810 con 0x7fab140d2940 2024-04-25T13:45:26.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.007+0000 7fab1259c640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faaf406f400 0x7faaf4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.007+0000 7fab1259c640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faaf406f400 0x7faaf4071910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fab0400b990 tx=0x7fab04008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:26.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.011+0000 7fab02ffd640 1 -- 172.21.15.80:0/3795480055 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fab0c05ca00 con 0x7fab140d2940 2024-04-25T13:45:26.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.171+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fab140d42c0 con 0x7fab140d2940 2024-04-25T13:45:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.603+0000 7fab02ffd640 1 -- 172.21.15.80:0/3795480055 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) ==== 97+0+0 (secure 0 0 0) 0x7fab0c05c300 con 0x7fab140d2940 2024-04-25T13:45:26.602 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:45:26.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faaf406f400 msgr2=0x7faaf4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:26.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faaf406f400 0x7faaf4071910 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fab0400b990 tx=0x7fab04008040 comp rx=0 tx=0).stop 2024-04-25T13:45:26.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140d2940 msgr2=0x7fab14157c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:26.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140d2940 0x7fab14157c90 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fab0c007d70 tx=0x7fab0c00e4b0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 shutdown_connections 2024-04-25T13:45:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faaf406f400 0x7faaf4071910 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fab140d2940 0x7fab14157c90 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fab140b93e0 0x7fab140cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 --2- 172.21.15.80:0/3795480055 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fab140b4950 0x7fab1415f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 >> 172.21.15.80:0/3795480055 conn(0x7fab1406c5f0 msgr2=0x7fab140b5900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 shutdown_connections 2024-04-25T13:45:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.607+0000 7fab18d84640 1 -- 172.21.15.80:0/3795480055 wait complete. 2024-04-25T13:45:26.633 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:45:26.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd8c30640 1 Processor -- start 2024-04-25T13:45:26.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd8c30640 1 -- start start 2024-04-25T13:45:26.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd8c30640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9cd40d59b0 0x7f9cd40d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd8c30640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9cd40d4330 0x7f9cd40d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd8c30640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cd40d6300 con 0x7f9cd40d2e80 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd8c30640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd4161060 con 0x7f9cd40d59b0 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd8c30640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9cd4161340 con 0x7f9cd40d4330 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd1d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9cd40d59b0 0x7f9cd40d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd1d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9cd40d59b0 0x7f9cd40d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36050/0 (socket says 172.21.15.80:36050) 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd1d9b640 1 -- 172.21.15.80:0/546542517 learned_addr learned my addr 172.21.15.80:0/546542517 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd2d9d640 1 --2- 172.21.15.80:0/546542517 >> v2:172.21.15.87:3300/0 conn(0x7f9cd40d4330 0x7f9cd40d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd1d9b640 1 -- 172.21.15.80:0/546542517 >> v2:172.21.15.87:3300/0 conn(0x7f9cd40d4330 msgr2=0x7f9cd40d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd1d9b640 1 --2- 172.21.15.80:0/546542517 >> v2:172.21.15.87:3300/0 conn(0x7f9cd40d4330 0x7f9cd40d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd1d9b640 1 -- 172.21.15.80:0/546542517 >> v1:172.21.15.87:6789/0 conn(0x7f9cd40d2e80 legacy=0x7f9cd41607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd1d9b640 1 -- 172.21.15.80:0/546542517 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd40d2680 con 0x7f9cd40d59b0 2024-04-25T13:45:26.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.827+0000 7f9cd2d9d640 1 --2- 172.21.15.80:0/546542517 >> v2:172.21.15.87:3300/0 conn(0x7f9cd40d4330 0x7f9cd40d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:26.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd1d9b640 1 --2- 172.21.15.80:0/546542517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d59b0 0x7f9cd40d5dc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40c9670 tx=0x7f9cc801c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea4a9c81e7c1523 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:26.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd159a640 1 -- 172.21.15.80:0/546542517 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80b0070 con 0x7f9cd40d59b0 2024-04-25T13:45:26.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd159a640 1 -- 172.21.15.80:0/546542517 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc80ab600 con 0x7f9cd40d59b0 2024-04-25T13:45:26.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd159a640 1 -- 172.21.15.80:0/546542517 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80ab9d0 con 0x7f9cd40d59b0 2024-04-25T13:45:26.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd8c30640 1 -- 172.21.15.80:0/546542517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d59b0 msgr2=0x7f9cd40d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:26.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/546542517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d59b0 0x7f9cd40d5dc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40c9670 tx=0x7f9cc801c270 comp rx=0 tx=0).stop 2024-04-25T13:45:26.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd8c30640 1 -- 172.21.15.80:0/546542517 shutdown_connections 2024-04-25T13:45:26.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/546542517 >> v2:172.21.15.87:3300/0 conn(0x7f9cd40d4330 0x7f9cd40d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/546542517 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d59b0 0x7f9cd40d5dc0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd8c30640 1 -- 172.21.15.80:0/546542517 >> 172.21.15.80:0/546542517 conn(0x7f9cd40737a0 msgr2=0x7f9cd406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:26.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd8c30640 1 -- 172.21.15.80:0/546542517 shutdown_connections 2024-04-25T13:45:26.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.831+0000 7f9cd8c30640 1 -- 172.21.15.80:0/546542517 wait complete. 2024-04-25T13:45:26.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 Processor -- start 2024-04-25T13:45:26.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 -- start start 2024-04-25T13:45:26.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cd40d2e80 0x7f9cd4156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d4330 0x7f9cd4155700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cd40d59b0 0x7f9cd40cae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40d3410 con 0x7f9cd40d4330 2024-04-25T13:45:26.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40d3290 con 0x7f9cd40d59b0 2024-04-25T13:45:26.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd415df60 con 0x7f9cd40d2e80 2024-04-25T13:45:26.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d4330 0x7f9cd4155700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d4330 0x7f9cd4155700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36062/0 (socket says 172.21.15.80:36062) 2024-04-25T13:45:26.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 -- 172.21.15.80:0/1814299549 learned_addr learned my addr 172.21.15.80:0/1814299549 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:26.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd2d9d640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cd40d59b0 0x7f9cd40cae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd259c640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cd40d2e80 0x7f9cd4156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 -- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cd40d2e80 msgr2=0x7f9cd4156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:26.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cd40d2e80 0x7f9cd4156df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 -- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cd40d59b0 msgr2=0x7f9cd40cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:26.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cd40d59b0 0x7f9cd40cae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:26.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 -- 172.21.15.80:0/1814299549 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd4155c40 con 0x7f9cd40d4330 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd259c640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cd40d2e80 0x7f9cd4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd2d9d640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cd40d59b0 0x7f9cd40cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd1d9b640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d4330 0x7f9cd4155700 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80ae040 tx=0x7f9cc801aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cc2ffd640 1 -- 172.21.15.80:0/1814299549 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80b0040 con 0x7f9cd40d4330 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cc2ffd640 1 -- 172.21.15.80:0/1814299549 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cc80bc070 con 0x7f9cd40d4330 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cc2ffd640 1 -- 172.21.15.80:0/1814299549 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80b6860 con 0x7f9cd40d4330 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd4156030 con 0x7f9cd40d4330 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd41565f0 con 0x7f9cd40d4330 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.835+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd41fddd0 con 0x7f9cd40d4330 2024-04-25T13:45:26.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.839+0000 7f9cc2ffd640 1 -- 172.21.15.80:0/1814299549 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9cc80b6a00 con 0x7f9cd40d4330 2024-04-25T13:45:26.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.839+0000 7f9cc2ffd640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9cb406f400 0x7f9cb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:26.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.839+0000 7f9cc2ffd640 1 -- 172.21.15.80:0/1814299549 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f9cc81291a0 con 0x7f9cd40d4330 2024-04-25T13:45:26.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.839+0000 7f9cd259c640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9cb406f400 0x7f9cb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:26.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.839+0000 7f9cd259c640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9cb406f400 0x7f9cb4071910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40b8fd0 tx=0x7f9cc4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:26.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:26.843+0000 7f9cc2ffd640 1 -- 172.21.15.80:0/1814299549 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9cd40d6300 con 0x7f9cd40d4330 2024-04-25T13:45:27.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.055+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f9cd406a3a0 con 0x7f9cd40d4330 2024-04-25T13:45:27.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.055+0000 7f9cc2ffd640 1 -- 172.21.15.80:0/1814299549 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f9cc80f6500 con 0x7f9cd40d4330 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.055 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:45:27.056 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.057 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:45:27.058 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:45:27.059 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.060 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:45:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:45:27.062 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.064 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:45:27.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:45:27.067 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:45:27.067 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.067 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:27.067 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:27.067 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:45:27.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9cb406f400 msgr2=0x7f9cb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9cb406f400 0x7f9cb4071910 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40b8fd0 tx=0x7f9cc4008040 comp rx=0 tx=0).stop 2024-04-25T13:45:27.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d4330 msgr2=0x7f9cd4155700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d4330 0x7f9cd4155700 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f9cc80ae040 tx=0x7f9cc801aab0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 shutdown_connections 2024-04-25T13:45:27.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9cb406f400 0x7f9cb4071910 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cd40d59b0 0x7f9cd40cae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cd40d4330 0x7f9cd4155700 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 --2- 172.21.15.80:0/1814299549 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cd40d2e80 0x7f9cd4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 >> 172.21.15.80:0/1814299549 conn(0x7f9cd40737a0 msgr2=0x7f9cd415fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:27.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 shutdown_connections 2024-04-25T13:45:27.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.059+0000 7f9cd8c30640 1 -- 172.21.15.80:0/1814299549 wait complete. 2024-04-25T13:45:27.077 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:27.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe30622c640 1 Processor -- start 2024-04-25T13:45:27.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe30622c640 1 -- start start 2024-04-25T13:45:27.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe30622c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3000731e0 0x7fe300163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe30622c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe3001642d0 0x7fe300160720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe30622c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3001646e0 con 0x7fe300160c60 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe30622c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe30015acb0 con 0x7fe3001642d0 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe30622c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe30006ea30 con 0x7fe3000731e0 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe300160c60 0x7fe300161070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51606/0 (socket says 172.21.15.80:51606) 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fffff640 1 -- 172.21.15.80:0/817735018 learned_addr learned my addr 172.21.15.80:0/817735018 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2feffd640 1 --2- 172.21.15.80:0/817735018 >> v2:172.21.15.87:3300/0 conn(0x7fe3001642d0 0x7fe300160720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2ff7fe640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3000731e0 0x7fe300163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3789913864 0 0) 0x7fe2f00014a0 con 0x7fe300160c60 2024-04-25T13:45:27.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2ec003530 con 0x7fe300160c60 2024-04-25T13:45:27.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1173900111 0 0) 0x7fe2ec003530 con 0x7fe300160c60 2024-04-25T13:45:27.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3000731e0 msgr2=0x7fe300163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:27.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3000731e0 0x7fe300163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 >> v2:172.21.15.87:3300/0 conn(0x7fe3001642d0 msgr2=0x7fe300160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:27.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 --2- 172.21.15.80:0/817735018 >> v2:172.21.15.87:3300/0 conn(0x7fe3001642d0 0x7fe300160720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30006ed10 con 0x7fe300160c60 2024-04-25T13:45:27.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fe2f0001bd0 con 0x7fe300160c60 2024-04-25T13:45:27.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 >> v1:172.21.15.80:6789/0 conn(0x7fe300160c60 legacy=0x7fe300161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec0039c0 0x7fe2ec003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2ec004810 0x7fe2ec004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3000731e0 0x7fe2ec005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe30015acb0 con 0x7fe2ec004810 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe30006ea30 con 0x7fe2ec0039c0 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe2ec00b9d0 con 0x7fe3000731e0 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.223+0000 7fe2feffd640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2ec004810 0x7fe2ec004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2ff7fe640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec0039c0 0x7fe2ec003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2feffd640 1 -- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3000731e0 msgr2=0x7fe2ec005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2feffd640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe3000731e0 0x7fe2ec005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2feffd640 1 -- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec0039c0 msgr2=0x7fe2ec003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:27.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2feffd640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec0039c0 0x7fe2ec003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2feffd640 1 -- 172.21.15.80:0/817735018 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2ec003530 con 0x7fe2ec004810 2024-04-25T13:45:27.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2feffd640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2ec004810 0x7fe2ec004c20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe3000b6ce0 tx=0x7fe2e801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46328e2ca95f2cf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80b2070 con 0x7fe2ec004810 2024-04-25T13:45:27.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2e80a5580 con 0x7fe2ec004810 2024-04-25T13:45:27.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2fe7fc640 1 -- 172.21.15.80:0/817735018 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2e80096b0 con 0x7fe2ec004810 2024-04-25T13:45:27.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe30622c640 1 -- 172.21.15.80:0/817735018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2ec004810 msgr2=0x7fe2ec004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe30622c640 1 --2- 172.21.15.80:0/817735018 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2ec004810 0x7fe2ec004c20 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fe3000b6ce0 tx=0x7fe2e801c990 comp rx=0 tx=0).stop 2024-04-25T13:45:27.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe2fffff640 1 -- 172.21.15.80:0/817735018 reap_dead start 2024-04-25T13:45:27.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe30622c640 1 -- 172.21.15.80:0/817735018 shutdown_connections 2024-04-25T13:45:27.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe30622c640 1 -- 172.21.15.80:0/817735018 >> 172.21.15.80:0/817735018 conn(0x7fe30006d080 msgr2=0x7fe3000b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:27.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe30622c640 1 -- 172.21.15.80:0/817735018 shutdown_connections 2024-04-25T13:45:27.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.227+0000 7fe30622c640 1 -- 172.21.15.80:0/817735018 wait complete. 2024-04-25T13:45:27.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 Processor -- start 2024-04-25T13:45:27.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 -- start start 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec004810 0x7fe30014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe30015d060 0x7fe30015d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3000b2560 0x7fe3000b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe300164b70 con 0x7fe3000b2560 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe30006f2d0 con 0x7fe2ec004810 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe30006e650 con 0x7fe30015d060 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3000b2560 0x7fe3000b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3000b2560 0x7fe3000b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36090/0 (socket says 172.21.15.80:36090) 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 -- 172.21.15.80:0/3176633690 learned_addr learned my addr 172.21.15.80:0/3176633690 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:27.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2ff7fe640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec004810 0x7fe30014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2feffd640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe30015d060 0x7fe30015d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 -- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe30015d060 msgr2=0x7fe30015d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe30015d060 0x7fe30015d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 -- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec004810 msgr2=0x7fe30014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec004810 0x7fe30014f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 -- 172.21.15.80:0/3176633690 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe300152a30 con 0x7fe3000b2560 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2fffff640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3000b2560 0x7fe3000b2970 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe300163cd0 tx=0x7fe2f000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2f7fff640 1 -- 172.21.15.80:0/3176633690 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000da20 con 0x7fe3000b2560 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2f7fff640 1 -- 172.21.15.80:0/3176633690 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f000dc00 con 0x7fe3000b2560 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3000b2eb0 con 0x7fe3000b2560 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe2f7fff640 1 -- 172.21.15.80:0/3176633690 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000fe50 con 0x7fe3000b2560 2024-04-25T13:45:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.231+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe300151c70 con 0x7fe3000b2560 2024-04-25T13:45:27.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.235+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3001514b0 con 0x7fe3000b2560 2024-04-25T13:45:27.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.235+0000 7fe2f7fff640 1 -- 172.21.15.80:0/3176633690 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2f0014070 con 0x7fe3000b2560 2024-04-25T13:45:27.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.235+0000 7fe2f7fff640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2e006f400 0x7fe2e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.235+0000 7fe2f7fff640 1 -- 172.21.15.80:0/3176633690 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7fe2f0091800 con 0x7fe3000b2560 2024-04-25T13:45:27.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.235+0000 7fe2ff7fe640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2e006f400 0x7fe2e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.235+0000 7fe2ff7fe640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2e006f400 0x7fe2e0071910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80090e0 tx=0x7fe2f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.239+0000 7fe2f7fff640 1 -- 172.21.15.80:0/3176633690 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe3001514b0 con 0x7fe3000b2560 2024-04-25T13:45:27.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.399+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe300151720 con 0x7fe3000b2560 2024-04-25T13:45:27.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe2f7fff640 1 -- 172.21.15.80:0/3176633690 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v99) ==== 74+0+12820 (secure 0 0 0) 0x7fe3000b2eb0 con 0x7fe3000b2560 2024-04-25T13:45:27.401 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:27.401 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":99,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:26.618015+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:27.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2e006f400 msgr2=0x7fe2e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2e006f400 0x7fe2e0071910 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe2f80090e0 tx=0x7fe2f8008040 comp rx=0 tx=0).stop 2024-04-25T13:45:27.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3000b2560 msgr2=0x7fe3000b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3000b2560 0x7fe3000b2970 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fe300163cd0 tx=0x7fe2f000fc00 comp rx=0 tx=0).stop 2024-04-25T13:45:27.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 shutdown_connections 2024-04-25T13:45:27.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2e006f400 0x7fe2e0071910 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe3000b2560 0x7fe3000b2970 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe30015d060 0x7fe30015d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 --2- 172.21.15.80:0/3176633690 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2ec004810 0x7fe30014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 >> 172.21.15.80:0/3176633690 conn(0x7fe30006d080 msgr2=0x7fe3000b4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:27.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 shutdown_connections 2024-04-25T13:45:27.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.403+0000 7fe30622c640 1 -- 172.21.15.80:0/3176633690 wait complete. 2024-04-25T13:45:27.422 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '90', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:45:27.422 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:45:27.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f889de5f640 1 Processor -- start 2024-04-25T13:45:27.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f889de5f640 1 -- start start 2024-04-25T13:45:27.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f889de5f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f88980b5330 0x7f88980d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f889de5f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f88980d0ff0 0x7f889806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f889de5f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88980b5740 con 0x7f889806f2f0 2024-04-25T13:45:27.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f889de5f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f889806f700 con 0x7f88980d0ff0 2024-04-25T13:45:27.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f889de5f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f88981629b0 con 0x7f88980b5330 2024-04-25T13:45:27.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f8896ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f88980d0ff0 0x7f889806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f8896ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f88980d0ff0 0x7f889806edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36102/0 (socket says 172.21.15.80:36102) 2024-04-25T13:45:27.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f8896ffd640 1 -- 172.21.15.80:0/71803364 learned_addr learned my addr 172.21.15.80:0/71803364 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:27.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f88977fe640 1 --2- 172.21.15.80:0/71803364 >> v2:172.21.15.87:3300/0 conn(0x7f88980b5330 0x7f88980d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f8896ffd640 1 -- 172.21.15.80:0/71803364 >> v2:172.21.15.87:3300/0 conn(0x7f88980b5330 msgr2=0x7f88980d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:27.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f8896ffd640 1 --2- 172.21.15.80:0/71803364 >> v2:172.21.15.87:3300/0 conn(0x7f88980b5330 0x7f88980d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f8896ffd640 1 -- 172.21.15.80:0/71803364 >> v1:172.21.15.87:6789/0 conn(0x7f889806f2f0 legacy=0x7f8898161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:27.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.563+0000 7f8896ffd640 1 -- 172.21.15.80:0/71803364 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889806dd10 con 0x7f88980d0ff0 2024-04-25T13:45:27.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f8896ffd640 1 --2- 172.21.15.80:0/71803364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88980d0ff0 0x7f889806edb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f88980701f0 tx=0x7f888001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d2d3c0514ca89ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f88967fc640 1 -- 172.21.15.80:0/71803364 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88800b0070 con 0x7f88980d0ff0 2024-04-25T13:45:27.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f88967fc640 1 -- 172.21.15.80:0/71803364 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88800ab600 con 0x7f88980d0ff0 2024-04-25T13:45:27.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f88967fc640 1 -- 172.21.15.80:0/71803364 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88800ab9d0 con 0x7f88980d0ff0 2024-04-25T13:45:27.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f889de5f640 1 -- 172.21.15.80:0/71803364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88980d0ff0 msgr2=0x7f889806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f889de5f640 1 --2- 172.21.15.80:0/71803364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88980d0ff0 0x7f889806edb0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f88980701f0 tx=0x7f888001c450 comp rx=0 tx=0).stop 2024-04-25T13:45:27.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f889de5f640 1 -- 172.21.15.80:0/71803364 shutdown_connections 2024-04-25T13:45:27.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f889de5f640 1 --2- 172.21.15.80:0/71803364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88980d0ff0 0x7f889806edb0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f889de5f640 1 --2- 172.21.15.80:0/71803364 >> v2:172.21.15.87:3300/0 conn(0x7f88980b5330 0x7f88980d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f889de5f640 1 -- 172.21.15.80:0/71803364 >> 172.21.15.80:0/71803364 conn(0x7f88980b4160 msgr2=0x7f88980d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:27.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f889de5f640 1 -- 172.21.15.80:0/71803364 shutdown_connections 2024-04-25T13:45:27.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.567+0000 7f889de5f640 1 -- 172.21.15.80:0/71803364 wait complete. 2024-04-25T13:45:27.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 Processor -- start 2024-04-25T13:45:27.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 -- start start 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f889806f2f0 0x7f8898132ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88980b5330 0x7f889815a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88980d0ff0 0x7f889815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f88980b0cd0 con 0x7f889806f2f0 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f889806e890 con 0x7f88980b5330 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f88980b0e50 con 0x7f88980d0ff0 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f889806f2f0 0x7f8898132ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f889806f2f0 0x7f8898132ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36110/0 (socket says 172.21.15.80:36110) 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 -- 172.21.15.80:0/3127327435 learned_addr learned my addr 172.21.15.80:0/3127327435 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:27.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f8896ffd640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88980b5330 0x7f889815a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f8897fff640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88980d0ff0 0x7f889815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 -- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88980d0ff0 msgr2=0x7f889815aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88980d0ff0 0x7f889815aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 -- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88980b5330 msgr2=0x7f889815a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88980b5330 0x7f889815a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 -- 172.21.15.80:0/3127327435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889815c210 con 0x7f889806f2f0 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f8896ffd640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88980b5330 0x7f889815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f88977fe640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f889806f2f0 0x7f8898132ec0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f888800d5f0 tx=0x7f888800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f888ffff640 1 -- 172.21.15.80:0/3127327435 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88880125f0 con 0x7f889806f2f0 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f888ffff640 1 -- 172.21.15.80:0/3127327435 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8888019070 con 0x7f889806f2f0 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f888ffff640 1 -- 172.21.15.80:0/3127327435 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88880129b0 con 0x7f889806f2f0 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8898141d70 con 0x7f889806f2f0 2024-04-25T13:45:27.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8898142330 con 0x7f889806f2f0 2024-04-25T13:45:27.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.571+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889806a3a0 con 0x7f889806f2f0 2024-04-25T13:45:27.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.575+0000 7f888ffff640 1 -- 172.21.15.80:0/3127327435 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8888012b50 con 0x7f889806f2f0 2024-04-25T13:45:27.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.575+0000 7f888ffff640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887806f400 0x7f8878071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.575+0000 7f888ffff640 1 -- 172.21.15.80:0/3127327435 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f888808f370 con 0x7f889806f2f0 2024-04-25T13:45:27.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.575+0000 7f8896ffd640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887806f400 0x7f8878071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.575+0000 7f8896ffd640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887806f400 0x7f8878071910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8898147cb0 tx=0x7f8880008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.579+0000 7f888ffff640 1 -- 172.21.15.80:0/3127327435 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f888805c640 con 0x7f889806f2f0 2024-04-25T13:45:27.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.727+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f88980d0810 con 0x7f889806f2f0 2024-04-25T13:45:27.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.727+0000 7f888ffff640 1 -- 172.21.15.80:0/3127327435 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v99) ==== 93+0+10 (secure 0 0 0) 0x7f888805bf40 con 0x7f889806f2f0 2024-04-25T13:45:27.729 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:45:27.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887806f400 msgr2=0x7f8878071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887806f400 0x7f8878071910 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f8898147cb0 tx=0x7f8880008040 comp rx=0 tx=0).stop 2024-04-25T13:45:27.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f889806f2f0 msgr2=0x7f8898132ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f889806f2f0 0x7f8898132ec0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f888800d5f0 tx=0x7f888800dab0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 shutdown_connections 2024-04-25T13:45:27.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887806f400 0x7f8878071910 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88980d0ff0 0x7f889815aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88980b5330 0x7f889815a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 --2- 172.21.15.80:0/3127327435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f889806f2f0 0x7f8898132ec0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 >> 172.21.15.80:0/3127327435 conn(0x7f88980b4160 msgr2=0x7f88980d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:27.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 shutdown_connections 2024-04-25T13:45:27.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.731+0000 7f889de5f640 1 -- 172.21.15.80:0/3127327435 wait complete. 2024-04-25T13:45:27.750 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:45:27.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.883+0000 7f406e03a640 1 Processor -- start 2024-04-25T13:45:27.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.883+0000 7f406e03a640 1 -- start start 2024-04-25T13:45:27.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f406e03a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f406806f5a0 0x7f4068160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f406e03a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4068161070 0x7f4068161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f406e03a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406806fcf0 con 0x7f40680b4bd0 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f406e03a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f40681619c0 con 0x7f406806f5a0 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f406e03a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4068073f10 con 0x7f4068161070 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f40677fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f406806f5a0 0x7f4068160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f4067fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f40680b4bd0 0x7f40680b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51612/0 (socket says 172.21.15.80:51612) 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f40677fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f406806f5a0 0x7f4068160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36116/0 (socket says 172.21.15.80:36116) 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f4067fff640 1 -- 172.21.15.80:0/2135015006 learned_addr learned my addr 172.21.15.80:0/2135015006 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f4066ffd640 1 --2- 172.21.15.80:0/2135015006 >> v2:172.21.15.87:3300/0 conn(0x7f4068161070 0x7f4068161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f40677fe640 1 -- 172.21.15.80:0/2135015006 >> v2:172.21.15.87:3300/0 conn(0x7f4068161070 msgr2=0x7f4068161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f40677fe640 1 --2- 172.21.15.80:0/2135015006 >> v2:172.21.15.87:3300/0 conn(0x7f4068161070 0x7f4068161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f40677fe640 1 -- 172.21.15.80:0/2135015006 >> v1:172.21.15.80:6789/0 conn(0x7f40680b4bd0 legacy=0x7f40680b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f40677fe640 1 -- 172.21.15.80:0/2135015006 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40680b0c60 con 0x7f406806f5a0 2024-04-25T13:45:27.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.887+0000 7f40667fc640 1 -- 172.21.15.80:0/2135015006 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3579746698 0 0) 0x7f406806fcf0 con 0x7f40680b4bd0 2024-04-25T13:45:27.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f40677fe640 1 --2- 172.21.15.80:0/2135015006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406806f5a0 0x7f4068160b30 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f406806ff70 tx=0x7f405801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ac58954b2996c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f40667fc640 1 -- 172.21.15.80:0/2135015006 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580b0070 con 0x7f406806f5a0 2024-04-25T13:45:27.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f40667fc640 1 -- 172.21.15.80:0/2135015006 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40580ab600 con 0x7f406806f5a0 2024-04-25T13:45:27.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f40667fc640 1 -- 172.21.15.80:0/2135015006 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40580ab9d0 con 0x7f406806f5a0 2024-04-25T13:45:27.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 -- 172.21.15.80:0/2135015006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406806f5a0 msgr2=0x7f4068160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 --2- 172.21.15.80:0/2135015006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406806f5a0 0x7f4068160b30 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f406806ff70 tx=0x7f405801c3d0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 -- 172.21.15.80:0/2135015006 shutdown_connections 2024-04-25T13:45:27.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 --2- 172.21.15.80:0/2135015006 >> v2:172.21.15.87:3300/0 conn(0x7f4068161070 0x7f4068161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 --2- 172.21.15.80:0/2135015006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f406806f5a0 0x7f4068160b30 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 -- 172.21.15.80:0/2135015006 >> 172.21.15.80:0/2135015006 conn(0x7f406806d080 msgr2=0x7f40680b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:27.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 -- 172.21.15.80:0/2135015006 shutdown_connections 2024-04-25T13:45:27.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 -- 172.21.15.80:0/2135015006 wait complete. 2024-04-25T13:45:27.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 Processor -- start 2024-04-25T13:45:27.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.891+0000 7f406e03a640 1 -- start start 2024-04-25T13:45:27.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406806f5a0 0x7f4068150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40680b4bd0 0x7f4068153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4068161070 0x7f40680d00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f406816a860 con 0x7f40680b4bd0 2024-04-25T13:45:27.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f406816a6e0 con 0x7f406806f5a0 2024-04-25T13:45:27.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f406816a9e0 con 0x7f4068161070 2024-04-25T13:45:27.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40680b4bd0 0x7f4068153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40680b4bd0 0x7f4068153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36132/0 (socket says 172.21.15.80:36132) 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 -- 172.21.15.80:0/2108494336 learned_addr learned my addr 172.21.15.80:0/2108494336 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 -- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4068161070 msgr2=0x7f40680d00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f40677fe640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406806f5a0 0x7f4068150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4068161070 0x7f40680d00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 -- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406806f5a0 msgr2=0x7f4068150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406806f5a0 0x7f4068150e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 -- 172.21.15.80:0/2108494336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406813fb80 con 0x7f40680b4bd0 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f4066ffd640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40680b4bd0 0x7f4068153ce0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f405000d5f0 tx=0x7f405000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f405ffff640 1 -- 172.21.15.80:0/2108494336 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500125f0 con 0x7f40680b4bd0 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f405ffff640 1 -- 172.21.15.80:0/2108494336 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4050019070 con 0x7f40680b4bd0 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f405ffff640 1 -- 172.21.15.80:0/2108494336 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40500129b0 con 0x7f40680b4bd0 2024-04-25T13:45:27.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4068151e60 con 0x7f40680b4bd0 2024-04-25T13:45:27.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4068158f60 con 0x7f40680b4bd0 2024-04-25T13:45:27.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f406806a3a0 con 0x7f40680b4bd0 2024-04-25T13:45:27.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f405ffff640 1 -- 172.21.15.80:0/2108494336 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4050012b50 con 0x7f40680b4bd0 2024-04-25T13:45:27.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f405ffff640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f404806f430 0x7f4048071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:27.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.895+0000 7f405ffff640 1 -- 172.21.15.80:0/2108494336 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7f405008f2d0 con 0x7f40680b4bd0 2024-04-25T13:45:27.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.899+0000 7f40677fe640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f404806f430 0x7f4048071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:27.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.899+0000 7f40677fe640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f404806f430 0x7f4048071940 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f40580ad000 tx=0x7f40580a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:27.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:27.903+0000 7f405ffff640 1 -- 172.21.15.80:0/2108494336 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f405005c5a0 con 0x7f40680b4bd0 2024-04-25T13:45:28.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4068151bb0 con 0x7f40680b4bd0 2024-04-25T13:45:28.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f405ffff640 1 -- 172.21.15.80:0/2108494336 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v99) ==== 84+0+65 (secure 0 0 0) 0x7f405005bea0 con 0x7f40680b4bd0 2024-04-25T13:45:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:28.050 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:45:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:45:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:45:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:28.050 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f404806f430 msgr2=0x7f4048071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f404806f430 0x7f4048071940 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f40580ad000 tx=0x7f40580a6000 comp rx=0 tx=0).stop 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40680b4bd0 msgr2=0x7f4068153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40680b4bd0 0x7f4068153ce0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f405000d5f0 tx=0x7f405000dab0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 shutdown_connections 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f404806f430 0x7f4048071940 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4068161070 0x7f40680d00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40680b4bd0 0x7f4068153ce0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 --2- 172.21.15.80:0/2108494336 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f406806f5a0 0x7f4068150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 >> 172.21.15.80:0/2108494336 conn(0x7f406806d080 msgr2=0x7f40680d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:28.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 shutdown_connections 2024-04-25T13:45:28.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.051+0000 7f406e03a640 1 -- 172.21.15.80:0/2108494336 wait complete. 2024-04-25T13:45:28.070 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:45:28.070 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:45:28.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd13a495640 1 Processor -- start 2024-04-25T13:45:28.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd13a495640 1 -- start start 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd13a495640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd13406cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd13a495640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd134164780 0x7fd134164b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd13a495640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd1340b4e00 con 0x7fd1341650d0 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd13a495640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd13406d840 con 0x7fd134164780 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd13a495640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd1340b50e0 con 0x7fd13406c9d0 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd133fff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd13406cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd1337fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd134164780 0x7fd134164b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd133fff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd13406cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:36556/0 (socket says 172.21.15.80:36556) 2024-04-25T13:45:28.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.219+0000 7fd133fff640 1 -- 172.21.15.80:0/1264004060 learned_addr learned my addr 172.21.15.80:0/1264004060 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:28.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd133fff640 1 -- 172.21.15.80:0/1264004060 >> v2:172.21.15.87:3300/0 conn(0x7fd134164780 msgr2=0x7fd134164b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:28.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd133fff640 1 --2- 172.21.15.80:0/1264004060 >> v2:172.21.15.87:3300/0 conn(0x7fd134164780 0x7fd134164b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd133fff640 1 -- 172.21.15.80:0/1264004060 >> v1:172.21.15.87:6789/0 conn(0x7fd1341650d0 legacy=0x7fd1340b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd133fff640 1 -- 172.21.15.80:0/1264004060 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd13406d4d0 con 0x7fd13406c9d0 2024-04-25T13:45:28.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd132ffd640 1 -- 172.21.15.80:0/1264004060 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383051318 0 0) 0x7fd1340b4e00 con 0x7fd1341650d0 2024-04-25T13:45:28.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd1337fe640 1 --2- 172.21.15.80:0/1264004060 >> v2:172.21.15.87:3300/0 conn(0x7fd134164780 0x7fd134164b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:45:28.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd133fff640 1 --2- 172.21.15.80:0/1264004060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd13406cde0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd124009870 tx=0x7fd12401c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3104f9c3373fb0f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:28.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd132ffd640 1 -- 172.21.15.80:0/1264004060 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240b2070 con 0x7fd13406c9d0 2024-04-25T13:45:28.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd132ffd640 1 -- 172.21.15.80:0/1264004060 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd1240a9070 con 0x7fd13406c9d0 2024-04-25T13:45:28.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.223+0000 7fd132ffd640 1 -- 172.21.15.80:0/1264004060 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1240ad880 con 0x7fd13406c9d0 2024-04-25T13:45:28.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- 172.21.15.80:0/1264004060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 msgr2=0x7fd13406cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 --2- 172.21.15.80:0/1264004060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd13406cde0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd124009870 tx=0x7fd12401c560 comp rx=0 tx=0).stop 2024-04-25T13:45:28.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- 172.21.15.80:0/1264004060 shutdown_connections 2024-04-25T13:45:28.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 --2- 172.21.15.80:0/1264004060 >> v2:172.21.15.87:3300/0 conn(0x7fd134164780 0x7fd134164b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 --2- 172.21.15.80:0/1264004060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd13406cde0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- 172.21.15.80:0/1264004060 >> 172.21.15.80:0/1264004060 conn(0x7fd1340d3ac0 msgr2=0x7fd1340d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:28.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- 172.21.15.80:0/1264004060 shutdown_connections 2024-04-25T13:45:28.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- 172.21.15.80:0/1264004060 wait complete. 2024-04-25T13:45:28.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 Processor -- start 2024-04-25T13:45:28.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- start start 2024-04-25T13:45:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd134071370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134164780 0x7fd1341552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1341650d0 0x7fd1340b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd1340b8780 con 0x7fd1341650d0 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd1340b8600 con 0x7fd134164780 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd13a495640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd13416d9e0 con 0x7fd13406c9d0 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1341650d0 0x7fd1340b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1341650d0 0x7fd1340b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36134/0 (socket says 172.21.15.80:36134) 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 -- 172.21.15.80:0/3651734884 learned_addr learned my addr 172.21.15.80:0/3651734884 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd1337fe640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134164780 0x7fd1341552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd133fff640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd134071370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 -- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 msgr2=0x7fd134071370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd134071370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 -- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134164780 msgr2=0x7fd1341552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134164780 0x7fd1341552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 -- 172.21.15.80:0/3651734884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd134155830 con 0x7fd1341650d0 2024-04-25T13:45:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd1337fe640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134164780 0x7fd1341552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.227+0000 7fd138a32640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1341650d0 0x7fd1340b1d10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00c780 tx=0x7fd12c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd130ff9640 1 -- 172.21.15.80:0/3651734884 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c003db0 con 0x7fd1341650d0 2024-04-25T13:45:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd130ff9640 1 -- 172.21.15.80:0/3651734884 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd12c00ce30 con 0x7fd1341650d0 2024-04-25T13:45:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd134155fe0 con 0x7fd1341650d0 2024-04-25T13:45:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd130ff9640 1 -- 172.21.15.80:0/3651734884 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd12c005290 con 0x7fd1341650d0 2024-04-25T13:45:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1341565a0 con 0x7fd1341650d0 2024-04-25T13:45:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd13406a440 con 0x7fd1341650d0 2024-04-25T13:45:28.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd130ff9640 1 -- 172.21.15.80:0/3651734884 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd12c005430 con 0x7fd1341650d0 2024-04-25T13:45:28.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd130ff9640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12006f400 0x7fd120071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd130ff9640 1 -- 172.21.15.80:0/3651734884 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 4857+0+0 (secure 0 0 0) 0x7fd12c08ee50 con 0x7fd1341650d0 2024-04-25T13:45:28.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd133fff640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12006f400 0x7fd120071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.231+0000 7fd133fff640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12006f400 0x7fd120071910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd1240af000 tx=0x7fd124008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:28.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.239+0000 7fd130ff9640 1 -- 172.21.15.80:0/3651734884 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd12c05c010 con 0x7fd1341650d0 2024-04-25T13:45:28.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.387+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd134076070 con 0x7fd1341650d0 2024-04-25T13:45:28.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.611+0000 7fd130ff9640 1 -- 172.21.15.80:0/3651734884 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v100) ==== 138+0+0 (secure 0 0 0) 0x7fd12c064090 con 0x7fd1341650d0 2024-04-25T13:45:28.612 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:45:28.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12006f400 msgr2=0x7fd120071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12006f400 0x7fd120071910 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd1240af000 tx=0x7fd124008040 comp rx=0 tx=0).stop 2024-04-25T13:45:28.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1341650d0 msgr2=0x7fd1340b1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1341650d0 0x7fd1340b1d10 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00c780 tx=0x7fd12c00cc40 comp rx=0 tx=0).stop 2024-04-25T13:45:28.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 shutdown_connections 2024-04-25T13:45:28.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd12006f400 0x7fd120071910 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd1341650d0 0x7fd1340b1d10 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd134164780 0x7fd1341552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 --2- 172.21.15.80:0/3651734884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd13406c9d0 0x7fd134071370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 >> 172.21.15.80:0/3651734884 conn(0x7fd1340d3ac0 msgr2=0x7fd134160710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:28.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.619+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 shutdown_connections 2024-04-25T13:45:28.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.623+0000 7fd13a495640 1 -- 172.21.15.80:0/3651734884 wait complete. 2024-04-25T13:45:28.644 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:45:28.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.803+0000 7f062750b640 1 Processor -- start 2024-04-25T13:45:28.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f062750b640 1 -- start start 2024-04-25T13:45:28.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f062750b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 0x7f06200723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f062750b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f062006fcf0 con 0x7f062006f5a0 2024-04-25T13:45:28.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f062750b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06200b0c60 con 0x7f0620161650 2024-04-25T13:45:28.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f062750b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0620073f10 con 0x7f0620161240 2024-04-25T13:45:28.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f06252a7640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f062006f5a0 0x7f0620160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51618/0 (socket says 172.21.15.80:51618) 2024-04-25T13:45:28.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f06252a7640 1 -- 172.21.15.80:0/3164681782 learned_addr learned my addr 172.21.15.80:0/3164681782 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2249089620 0 0) 0x7f062006fcf0 con 0x7f062006f5a0 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f060c003530 con 0x7f062006f5a0 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0624aa6640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 0x7f06200723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2017084032 0 0) 0x7f060c003530 con 0x7f062006f5a0 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 msgr2=0x7f06200723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 0x7f06200723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 >> v1:172.21.15.87:6789/0 conn(0x7f0620161650 legacy=0x7f06200728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06200b0f40 con 0x7f062006f5a0 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f061c001680 con 0x7f062006f5a0 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0624aa6640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 0x7f06200723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:45:28.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 >> v1:172.21.15.80:6789/0 conn(0x7f062006f5a0 legacy=0x7f0620160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f060c0039f0 0x7f060c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f060c004810 0x7f060c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 0x7f060c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0620073f10 con 0x7f060c004810 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f060c005560 con 0x7f060c0039f0 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f060c00bf10 con 0x7f0620161240 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0624aa6640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f060c004810 0x7f060c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0625aa8640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 0x7f060c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f06252a7640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f060c0039f0 0x7f060c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0624aa6640 1 -- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 msgr2=0x7f060c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:28.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0624aa6640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0620161240 0x7f060c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0624aa6640 1 -- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f060c0039f0 msgr2=0x7f060c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:28.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0624aa6640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f060c0039f0 0x7f060c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.807+0000 7f0624aa6640 1 -- 172.21.15.80:0/3164681782 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f060c003530 con 0x7f060c004810 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f0624aa6640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f060c004810 0x7f060c004c20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f06100098d0 tx=0x7f0620073800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a368ddede5e363f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061000f040 con 0x7f060c004810 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06100b0070 con 0x7f060c004810 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f0617fff640 1 -- 172.21.15.80:0/3164681782 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06100ab820 con 0x7f060c004810 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f062750b640 1 -- 172.21.15.80:0/3164681782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f060c004810 msgr2=0x7f060c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f062750b640 1 --2- 172.21.15.80:0/3164681782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f060c004810 0x7f060c004c20 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f06100098d0 tx=0x7f0620073800 comp rx=0 tx=0).stop 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f0625aa8640 1 -- 172.21.15.80:0/3164681782 reap_dead start 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f062750b640 1 -- 172.21.15.80:0/3164681782 shutdown_connections 2024-04-25T13:45:28.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f062750b640 1 -- 172.21.15.80:0/3164681782 >> 172.21.15.80:0/3164681782 conn(0x7f062006d080 msgr2=0x7f06200b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:28.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f062750b640 1 -- 172.21.15.80:0/3164681782 shutdown_connections 2024-04-25T13:45:28.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.811+0000 7f062750b640 1 -- 172.21.15.80:0/3164681782 wait complete. 2024-04-25T13:45:28.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 Processor -- start 2024-04-25T13:45:28.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 -- start start 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f060c004810 0x7f0620149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0620158540 0x7f0620158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0620166f00 0x7f0620167310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0620169260 con 0x7f0620166f00 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f06201690e0 con 0x7f0620158540 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f06201693e0 con 0x7f060c004810 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0620166f00 0x7f0620167310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0620166f00 0x7f0620167310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36150/0 (socket says 172.21.15.80:36150) 2024-04-25T13:45:28.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 -- 172.21.15.80:0/2670663613 learned_addr learned my addr 172.21.15.80:0/2670663613 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0624aa6640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0620158540 0x7f0620158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f06252a7640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f060c004810 0x7f0620149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 -- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f060c004810 msgr2=0x7f0620149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f060c004810 0x7f0620149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 -- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0620158540 msgr2=0x7f0620158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0620158540 0x7f0620158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 -- 172.21.15.80:0/2670663613 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0620167850 con 0x7f0620166f00 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0625aa8640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0620166f00 0x7f0620167310 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0620070290 tx=0x7f061800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0615ffb640 1 -- 172.21.15.80:0/2670663613 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618003980 con 0x7f0620166f00 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0615ffb640 1 -- 172.21.15.80:0/2670663613 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0618018070 con 0x7f0620166f00 2024-04-25T13:45:28.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f0615ffb640 1 -- 172.21.15.80:0/2670663613 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0618003d60 con 0x7f0620166f00 2024-04-25T13:45:28.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06201f9f40 con 0x7f0620166f00 2024-04-25T13:45:28.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.815+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06201fa500 con 0x7f0620166f00 2024-04-25T13:45:28.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.819+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0620158f30 con 0x7f0620166f00 2024-04-25T13:45:28.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.819+0000 7f0615ffb640 1 -- 172.21.15.80:0/2670663613 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f06180050d0 con 0x7f0620166f00 2024-04-25T13:45:28.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.819+0000 7f0615ffb640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f060406f400 0x7f0604071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:28.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.819+0000 7f0615ffb640 1 -- 172.21.15.80:0/2670663613 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5227+0+0 (secure 0 0 0) 0x7f061808f760 con 0x7f0620166f00 2024-04-25T13:45:28.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.819+0000 7f06252a7640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f060406f400 0x7f0604071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:28.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.819+0000 7f06252a7640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f060406f400 0x7f0604071910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f061c001ce0 tx=0x7f061c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:28.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.823+0000 7f0615ffb640 1 -- 172.21.15.80:0/2670663613 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0620167850 con 0x7f0620166f00 2024-04-25T13:45:28.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:28.975+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f062006a3a0 con 0x7f0620166f00 2024-04-25T13:45:29.217 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:45:29.220 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:45:29.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.615+0000 7f0615ffb640 1 -- 172.21.15.80:0/2670663613 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v101) ==== 170+0+0 (secure 0 0 0) 0x7f061805c9a0 con 0x7f0620166f00 2024-04-25T13:45:29.617 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:45:29.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f060406f400 msgr2=0x7f0604071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:29.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f060406f400 0x7f0604071910 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f061c001ce0 tx=0x7f061c009040 comp rx=0 tx=0).stop 2024-04-25T13:45:29.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0620166f00 msgr2=0x7f0620167310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:29.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0620166f00 0x7f0620167310 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f0620070290 tx=0x7f061800ec10 comp rx=0 tx=0).stop 2024-04-25T13:45:29.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 shutdown_connections 2024-04-25T13:45:29.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f060406f400 0x7f0604071910 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0620166f00 0x7f0620167310 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0620158540 0x7f0620158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 --2- 172.21.15.80:0/2670663613 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f060c004810 0x7f0620149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 >> 172.21.15.80:0/2670663613 conn(0x7f062006d080 msgr2=0x7f06200cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:29.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 shutdown_connections 2024-04-25T13:45:29.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.623+0000 7f062750b640 1 -- 172.21.15.80:0/2670663613 wait complete. 2024-04-25T13:45:29.654 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:45:29.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d43a5640 1 Processor -- start 2024-04-25T13:45:29.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d43a5640 1 -- start start 2024-04-25T13:45:29.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d43a5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f18cc164910 0x7f18cc0b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:29.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d43a5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18cc0ce270 con 0x7f18cc0cf310 2024-04-25T13:45:29.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d43a5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18cc0ce550 con 0x7f18cc0c6a50 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d43a5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f18cc0b8870 con 0x7f18cc164910 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d2942640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f18cc0cf310 0x7f18cc0cf720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51630/0 (socket says 172.21.15.80:51630) 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d2942640 1 -- 172.21.15.80:0/2886604992 learned_addr learned my addr 172.21.15.80:0/2886604992 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d2141640 1 --2- 172.21.15.80:0/2886604992 >> v2:172.21.15.87:3300/0 conn(0x7f18cc164910 0x7f18cc0b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 258602060 0 0) 0x7f18cc0ce270 con 0x7f18cc0cf310 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.803+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f18b8003530 con 0x7f18cc0cf310 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1587013097 0 0) 0x7f18b8003530 con 0x7f18cc0cf310 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 >> v2:172.21.15.87:3300/0 conn(0x7f18cc164910 msgr2=0x7f18cc0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 --2- 172.21.15.80:0/2886604992 >> v2:172.21.15.87:3300/0 conn(0x7f18cc164910 0x7f18cc0b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 >> v1:172.21.15.87:6789/0 conn(0x7f18cc0c6a50 legacy=0x7f18cc0c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:29.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc0ce830 con 0x7f18cc0cf310 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f18c8001680 con 0x7f18cc0cf310 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 >> v1:172.21.15.80:6789/0 conn(0x7f18cc0cf310 legacy=0x7f18cc0cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b80039c0 0x7f18b8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b8004810 0x7f18b8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b8005260 0x7f18b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc0b8870 con 0x7f18b80039c0 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18b800c1f0 con 0x7f18b8005260 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18b800c3f0 con 0x7f18b8004810 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2141640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b80039c0 0x7f18b8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2942640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b8005260 0x7f18b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2141640 1 -- 172.21.15.80:0/2886604992 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b8004810 msgr2=0x7f18b8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2141640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b8004810 0x7f18b8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2141640 1 -- 172.21.15.80:0/2886604992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b8005260 msgr2=0x7f18b800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:29.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2141640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18b8005260 0x7f18b800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2942640 1 -- 172.21.15.80:0/2886604992 reap_dead start 2024-04-25T13:45:29.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2141640 1 -- 172.21.15.80:0/2886604992 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b8003530 con 0x7f18b80039c0 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d2141640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b80039c0 0x7f18b8003e30 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009ae0 tx=0x7f18c401ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27482a58c88832ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c40b2070 con 0x7f18b80039c0 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c40a5550 con 0x7f18b80039c0 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d113f640 1 -- 172.21.15.80:0/2886604992 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c40096b0 con 0x7f18b80039c0 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d43a5640 1 -- 172.21.15.80:0/2886604992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b80039c0 msgr2=0x7f18b8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d43a5640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b80039c0 0x7f18b8003e30 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f18c4009ae0 tx=0x7f18c401ca70 comp rx=0 tx=0).stop 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d43a5640 1 -- 172.21.15.80:0/2886604992 shutdown_connections 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d43a5640 1 --2- 172.21.15.80:0/2886604992 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18b80039c0 0x7f18b8003e30 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d43a5640 1 -- 172.21.15.80:0/2886604992 >> 172.21.15.80:0/2886604992 conn(0x7f18cc0d3a80 msgr2=0x7f18cc0d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:29.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.807+0000 7f18d43a5640 1 -- 172.21.15.80:0/2886604992 shutdown_connections 2024-04-25T13:45:29.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 -- 172.21.15.80:0/2886604992 wait complete. 2024-04-25T13:45:29.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 Processor -- start 2024-04-25T13:45:29.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 -- start start 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b80039c0 0x7f18cc0b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18cc0b4f90 0x7f18cc163d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18cc1fe040 0x7f18cc1fe450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc06e490 con 0x7f18cc0b4f90 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18cc0cff30 con 0x7f18cc1fe040 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d43a5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18cc06e610 con 0x7f18b80039c0 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18cc0b4f90 0x7f18cc163d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18cc0b4f90 0x7f18cc163d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36166/0 (socket says 172.21.15.80:36166) 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 -- 172.21.15.80:0/3612384213 learned_addr learned my addr 172.21.15.80:0/3612384213 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:29.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d2141640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b80039c0 0x7f18cc0b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d2942640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18cc1fe040 0x7f18cc1fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 -- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b80039c0 msgr2=0x7f18cc0b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b80039c0 0x7f18cc0b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 -- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18cc1fe040 msgr2=0x7f18cc1fe450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18cc1fe040 0x7f18cc1fe450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 -- 172.21.15.80:0/3612384213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18cc0b5950 con 0x7f18cc0b4f90 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.811+0000 7f18d1940640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18cc0b4f90 0x7f18cc163d50 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f18c000c800 tx=0x7f18c000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18beffd640 1 -- 172.21.15.80:0/3612384213 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c0003b80 con 0x7f18cc0b4f90 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18beffd640 1 -- 172.21.15.80:0/3612384213 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18c0003d40 con 0x7f18cc0b4f90 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18cc0b5c30 con 0x7f18cc0b4f90 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18beffd640 1 -- 172.21.15.80:0/3612384213 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c000aa90 con 0x7f18cc0b4f90 2024-04-25T13:45:29.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18cc1feb10 con 0x7f18cc0b4f90 2024-04-25T13:45:29.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18cc06a3a0 con 0x7f18cc0b4f90 2024-04-25T13:45:29.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18beffd640 1 -- 172.21.15.80:0/3612384213 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f18c000ac30 con 0x7f18cc0b4f90 2024-04-25T13:45:29.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18beffd640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f18b006f400 0x7f18b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:29.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18beffd640 1 -- 172.21.15.80:0/3612384213 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5609+0+0 (secure 0 0 0) 0x7f18c008f190 con 0x7f18cc0b4f90 2024-04-25T13:45:29.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18d2141640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f18b006f400 0x7f18b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:29.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.815+0000 7f18d2141640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f18b006f400 0x7f18b0071910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f18c40af000 tx=0x7f18c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:29.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:29.819+0000 7f18beffd640 1 -- 172.21.15.80:0/3612384213 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f18c005c170 con 0x7f18cc0b4f90 2024-04-25T13:45:30.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.003+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f18cc071d70 con 0x7f18cc0b4f90 2024-04-25T13:45:30.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.695+0000 7f18beffd640 1 -- 172.21.15.80:0/3612384213 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v46) ==== 174+0+0 (secure 0 0 0) 0x7f18c005ba70 con 0x7f18cc0b4f90 2024-04-25T13:45:30.696 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T13:45:30.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f18b006f400 msgr2=0x7f18b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:30.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f18b006f400 0x7f18b0071910 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f18c40af000 tx=0x7f18c4008040 comp rx=0 tx=0).stop 2024-04-25T13:45:30.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18cc0b4f90 msgr2=0x7f18cc163d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:30.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18cc0b4f90 0x7f18cc163d50 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f18c000c800 tx=0x7f18c000ccc0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 shutdown_connections 2024-04-25T13:45:30.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f18b006f400 0x7f18b0071910 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18cc1fe040 0x7f18cc1fe450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18cc0b4f90 0x7f18cc163d50 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 --2- 172.21.15.80:0/3612384213 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18b80039c0 0x7f18cc0b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 >> 172.21.15.80:0/3612384213 conn(0x7f18cc0d3a80 msgr2=0x7f18cc070fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:30.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 shutdown_connections 2024-04-25T13:45:30.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.703+0000 7f18d43a5640 1 -- 172.21.15.80:0/3612384213 wait complete. 2024-04-25T13:45:30.731 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:30.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.871+0000 7f3dffcd7640 1 Processor -- start 2024-04-25T13:45:30.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.871+0000 7f3dffcd7640 1 -- start start 2024-04-25T13:45:30.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dffcd7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3df806f5a0 0x7f3df8160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dffcd7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3df80733b0 0x7f3df8161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dffcd7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3df806fcf0 con 0x7f3df8161070 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dffcd7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3df81619c0 con 0x7f3df806f5a0 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dffcd7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3df8073f10 con 0x7f3df80733b0 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dfda73640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3df806f5a0 0x7f3df8160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dfda73640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3df806f5a0 0x7f3df8160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36168/0 (socket says 172.21.15.80:36168) 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dfda73640 1 -- 172.21.15.80:0/988088611 learned_addr learned my addr 172.21.15.80:0/988088611 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dfda73640 1 -- 172.21.15.80:0/988088611 >> v2:172.21.15.87:3300/0 conn(0x7f3df80733b0 msgr2=0x7f3df8161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dfda73640 1 --2- 172.21.15.80:0/988088611 >> v2:172.21.15.87:3300/0 conn(0x7f3df80733b0 0x7f3df8161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dfda73640 1 -- 172.21.15.80:0/988088611 >> v1:172.21.15.87:6789/0 conn(0x7f3df8161070 legacy=0x7f3df80b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:30.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.875+0000 7f3dfda73640 1 -- 172.21.15.80:0/988088611 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df80b0c60 con 0x7f3df806f5a0 2024-04-25T13:45:30.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dfda73640 1 --2- 172.21.15.80:0/988088611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df8160b30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3df8070270 tx=0x7f3df401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f5a67ec1350d426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:30.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dfca71640 1 -- 172.21.15.80:0/988088611 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40b0070 con 0x7f3df806f5a0 2024-04-25T13:45:30.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dfca71640 1 -- 172.21.15.80:0/988088611 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df40ab600 con 0x7f3df806f5a0 2024-04-25T13:45:30.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dfca71640 1 -- 172.21.15.80:0/988088611 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40ab9d0 con 0x7f3df806f5a0 2024-04-25T13:45:30.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 -- 172.21.15.80:0/988088611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 msgr2=0x7f3df8160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:30.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/988088611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df8160b30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3df8070270 tx=0x7f3df401c470 comp rx=0 tx=0).stop 2024-04-25T13:45:30.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 -- 172.21.15.80:0/988088611 shutdown_connections 2024-04-25T13:45:30.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/988088611 >> v2:172.21.15.87:3300/0 conn(0x7f3df80733b0 0x7f3df8161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/988088611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df8160b30 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 -- 172.21.15.80:0/988088611 >> 172.21.15.80:0/988088611 conn(0x7f3df806d080 msgr2=0x7f3df80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:30.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 -- 172.21.15.80:0/988088611 shutdown_connections 2024-04-25T13:45:30.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 -- 172.21.15.80:0/988088611 wait complete. 2024-04-25T13:45:30.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 Processor -- start 2024-04-25T13:45:30.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.879+0000 7f3dffcd7640 1 -- start start 2024-04-25T13:45:30.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df814fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:30.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3df80733b0 0x7f3df815db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:30.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3df8161070 0x7f3df81590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:30.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3df816a050 con 0x7f3df806f5a0 2024-04-25T13:45:30.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3df8169ed0 con 0x7f3df8161070 2024-04-25T13:45:30.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3df816a1d0 con 0x7f3df80733b0 2024-04-25T13:45:30.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df814fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:30.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df814fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36180/0 (socket says 172.21.15.80:36180) 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 -- 172.21.15.80:0/2013948863 learned_addr learned my addr 172.21.15.80:0/2013948863 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 -- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3df80733b0 msgr2=0x7f3df815db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3df80733b0 0x7f3df815db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 -- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3df8161070 msgr2=0x7f3df81590b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfe274640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3df8161070 0x7f3df81590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3df8161070 0x7f3df81590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 -- 172.21.15.80:0/2013948863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df814eb10 con 0x7f3df806f5a0 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dfda73640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df814fbb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3df40097f0 tx=0x7f3df401acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dee7fc640 1 -- 172.21.15.80:0/2013948863 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40b0040 con 0x7f3df806f5a0 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dee7fc640 1 -- 172.21.15.80:0/2013948863 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3df40bc070 con 0x7f3df806f5a0 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df814edf0 con 0x7f3df806f5a0 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dee7fc640 1 -- 172.21.15.80:0/2013948863 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df40b6880 con 0x7f3df806f5a0 2024-04-25T13:45:30.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df814f3b0 con 0x7f3df806f5a0 2024-04-25T13:45:30.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df806a3a0 con 0x7f3df806f5a0 2024-04-25T13:45:30.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dee7fc640 1 -- 172.21.15.80:0/2013948863 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3df40b6a20 con 0x7f3df806f5a0 2024-04-25T13:45:30.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dee7fc640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3ddc06f430 0x7f3ddc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:30.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.883+0000 7f3dee7fc640 1 -- 172.21.15.80:0/2013948863 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f3df412a190 con 0x7f3df806f5a0 2024-04-25T13:45:30.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.887+0000 7f3dfd272640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3ddc06f430 0x7f3ddc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:30.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.887+0000 7f3dfd272640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3ddc06f430 0x7f3ddc071940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3df81faf50 tx=0x7f3de8006bb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:30.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:30.887+0000 7f3dee7fc640 1 -- 172.21.15.80:0/2013948863 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3df40160f0 con 0x7f3df806f5a0 2024-04-25T13:45:31.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.051+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3df80739c0 con 0x7f3df806f5a0 2024-04-25T13:45:31.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.051+0000 7f3dee7fc640 1 -- 172.21.15.80:0/2013948863 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16612 (secure 0 0 0) 0x7f3df40f7160 con 0x7f3df806f5a0 2024-04-25T13:45:31.052 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:31.052 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":102,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:30.006637+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:45:28.400550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T13:45:28.988401+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3ddc06f430 msgr2=0x7f3ddc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3ddc06f430 0x7f3ddc071940 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3df81faf50 tx=0x7f3de8006bb0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 msgr2=0x7f3df814fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df814fbb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f3df40097f0 tx=0x7f3df401acb0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 shutdown_connections 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3ddc06f430 0x7f3ddc071940 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3df8161070 0x7f3df81590b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3df80733b0 0x7f3df815db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 --2- 172.21.15.80:0/2013948863 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3df806f5a0 0x7f3df814fbb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 >> 172.21.15.80:0/2013948863 conn(0x7f3df806d080 msgr2=0x7f3df80cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:31.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 shutdown_connections 2024-04-25T13:45:31.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.055+0000 7f3dffcd7640 1 -- 172.21.15.80:0/2013948863 wait complete. 2024-04-25T13:45:31.074 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f563b57c640 1 Processor -- start 2024-04-25T13:45:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f563b57c640 1 -- start start 2024-04-25T13:45:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f563b57c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d2e80 0x7f56341607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f563b57c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f56340d59b0 0x7f56340d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f563b57c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56340d4330 0x7f56340d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f563b57c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f56340d6300 con 0x7f56340d4330 2024-04-25T13:45:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f563b57c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5634161060 con 0x7f56340d59b0 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f563b57c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5634161340 con 0x7f56340d2e80 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639b19640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56340d4330 0x7f56340d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639b19640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56340d4330 0x7f56340d4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48822/0 (socket says 172.21.15.80:48822) 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639b19640 1 -- 172.21.15.80:0/3222371979 learned_addr learned my addr 172.21.15.80:0/3222371979 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639b19640 1 -- 172.21.15.80:0/3222371979 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d2e80 msgr2=0x7f56341607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5638b17640 1 --2- 172.21.15.80:0/3222371979 >> v2:172.21.15.87:3300/0 conn(0x7f56340d59b0 0x7f56340d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639318640 1 --2- 172.21.15.80:0/3222371979 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d2e80 0x7f56341607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639b19640 1 --2- 172.21.15.80:0/3222371979 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d2e80 0x7f56341607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639b19640 1 -- 172.21.15.80:0/3222371979 >> v2:172.21.15.87:3300/0 conn(0x7f56340d59b0 msgr2=0x7f56340d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639b19640 1 --2- 172.21.15.80:0/3222371979 >> v2:172.21.15.87:3300/0 conn(0x7f56340d59b0 0x7f56340d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.219+0000 7f5639b19640 1 -- 172.21.15.80:0/3222371979 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56340d2680 con 0x7f56340d4330 2024-04-25T13:45:31.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f5639b19640 1 --2- 172.21.15.80:0/3222371979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d4330 0x7f56340d4740 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f56340d3720 tx=0x7f562c01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16d266d043a9be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:31.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f562bfff640 1 -- 172.21.15.80:0/3222371979 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c0a57d0 con 0x7f56340d4330 2024-04-25T13:45:31.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f562bfff640 1 -- 172.21.15.80:0/3222371979 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c0a5970 con 0x7f56340d4330 2024-04-25T13:45:31.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f562bfff640 1 -- 172.21.15.80:0/3222371979 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c0a5d40 con 0x7f56340d4330 2024-04-25T13:45:31.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 -- 172.21.15.80:0/3222371979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d4330 msgr2=0x7f56340d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 --2- 172.21.15.80:0/3222371979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d4330 0x7f56340d4740 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f56340d3720 tx=0x7f562c01c930 comp rx=0 tx=0).stop 2024-04-25T13:45:31.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 -- 172.21.15.80:0/3222371979 shutdown_connections 2024-04-25T13:45:31.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 --2- 172.21.15.80:0/3222371979 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d4330 0x7f56340d4740 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 --2- 172.21.15.80:0/3222371979 >> v2:172.21.15.87:3300/0 conn(0x7f56340d59b0 0x7f56340d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 --2- 172.21.15.80:0/3222371979 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d2e80 0x7f56341607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 -- 172.21.15.80:0/3222371979 >> 172.21.15.80:0/3222371979 conn(0x7f56340737a0 msgr2=0x7f563406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:31.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 -- 172.21.15.80:0/3222371979 shutdown_connections 2024-04-25T13:45:31.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 -- 172.21.15.80:0/3222371979 wait complete. 2024-04-25T13:45:31.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 Processor -- start 2024-04-25T13:45:31.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.223+0000 7f563b57c640 1 -- start start 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d2e80 0x7f5634158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d4330 0x7f563414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56340d59b0 0x7f56340b9870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f56340d4e00 con 0x7f56340d2e80 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f56340d4c80 con 0x7f56340d59b0 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f56340d3290 con 0x7f56340d4330 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d2e80 0x7f5634158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d2e80 0x7f5634158420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48834/0 (socket says 172.21.15.80:48834) 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 -- 172.21.15.80:0/2486197405 learned_addr learned my addr 172.21.15.80:0/2486197405 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:31.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5638b17640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d4330 0x7f563414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639b19640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56340d59b0 0x7f56340b9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 -- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d4330 msgr2=0x7f563414ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d4330 0x7f563414ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 -- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56340d59b0 msgr2=0x7f56340b9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56340d59b0 0x7f56340b9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 -- 172.21.15.80:0/2486197405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563415cfe0 con 0x7f56340d2e80 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639b19640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56340d59b0 0x7f56340b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5639318640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d2e80 0x7f5634158420 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f563000c7f0 tx=0x7f563000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5629ffb640 1 -- 172.21.15.80:0/2486197405 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630004230 con 0x7f56340d2e80 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5629ffb640 1 -- 172.21.15.80:0/2486197405 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56300043d0 con 0x7f56340d2e80 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563415d2c0 con 0x7f56340d2e80 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5629ffb640 1 -- 172.21.15.80:0/2486197405 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5630003860 con 0x7f56340d2e80 2024-04-25T13:45:31.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56340b9350 con 0x7f56340d2e80 2024-04-25T13:45:31.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563406a3a0 con 0x7f56340d2e80 2024-04-25T13:45:31.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.227+0000 7f5629ffb640 1 -- 172.21.15.80:0/2486197405 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5630003c10 con 0x7f56340d2e80 2024-04-25T13:45:31.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.231+0000 7f5629ffb640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f561806f430 0x7f5618071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.231+0000 7f5629ffb640 1 -- 172.21.15.80:0/2486197405 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7f563008f3c0 con 0x7f56340d2e80 2024-04-25T13:45:31.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.231+0000 7f5638b17640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f561806f430 0x7f5618071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.231+0000 7f5638b17640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f561806f430 0x7f5618071940 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f563415ca50 tx=0x7f5624008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:31.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.235+0000 7f5629ffb640 1 -- 172.21.15.80:0/2486197405 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5630015020 con 0x7f56340d2e80 2024-04-25T13:45:31.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.395+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56340bbb90 con 0x7f56340d2e80 2024-04-25T13:45:31.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f5629ffb640 1 -- 172.21.15.80:0/2486197405 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+16612 (secure 0 0 0) 0x7f563005c340 con 0x7f56340d2e80 2024-04-25T13:45:31.398 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:31.398 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":102,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:30.006637+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:45:28.400550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T13:45:28.988401+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:31.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f561806f430 msgr2=0x7f5618071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f561806f430 0x7f5618071940 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f563415ca50 tx=0x7f5624008040 comp rx=0 tx=0).stop 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d2e80 msgr2=0x7f5634158420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d2e80 0x7f5634158420 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f563000c7f0 tx=0x7f563000ccb0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 shutdown_connections 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f561806f430 0x7f5618071940 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56340d59b0 0x7f56340b9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56340d4330 0x7f563414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 --2- 172.21.15.80:0/2486197405 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56340d2e80 0x7f5634158420 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.399+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 >> 172.21.15.80:0/2486197405 conn(0x7f56340737a0 msgr2=0x7f563415a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.403+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 shutdown_connections 2024-04-25T13:45:31.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.403+0000 7f563b57c640 1 -- 172.21.15.80:0/2486197405 wait complete. 2024-04-25T13:45:31.421 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:45:31.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3fdb09640 1 Processor -- start 2024-04-25T13:45:31.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3fdb09640 1 -- start start 2024-04-25T13:45:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3fdb09640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3f80b5330 0x7fb3f80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3fdb09640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 0x7fb3f8161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3fdb09640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f81624b0 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3fdb09640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f81629b0 con 0x7fb3f80b5330 2024-04-25T13:45:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3fdb09640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80b5740 con 0x7fb3f8161b60 2024-04-25T13:45:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3f77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3f80b5330 0x7fb3f80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3f6ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb3f80d0ff0 0x7fb3f806edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51394/0 (socket says 172.21.15.80:51394) 2024-04-25T13:45:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/3371647384 learned_addr learned my addr 172.21.15.80:0/3371647384 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.571+0000 7fb3f77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3f80b5330 0x7fb3f80d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48850/0 (socket says 172.21.15.80:48850) 2024-04-25T13:45:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f7fff640 1 --2- 172.21.15.80:0/3371647384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 0x7fb3f8161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f77fe640 1 -- 172.21.15.80:0/3371647384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 msgr2=0x7fb3f8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/3371647384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 0x7fb3f8161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f77fe640 1 -- 172.21.15.80:0/3371647384 >> v1:172.21.15.80:6789/0 conn(0x7fb3f80d0ff0 legacy=0x7fb3f806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f77fe640 1 -- 172.21.15.80:0/3371647384 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f806dd10 con 0x7fb3f80b5330 2024-04-25T13:45:31.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/3371647384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80b5330 0x7fb3f80d0ab0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80097c0 tx=0x7fb3e801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f687c2ffbc0b2d04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:31.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f67fc640 1 -- 172.21.15.80:0/3371647384 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80b0070 con 0x7fb3f80b5330 2024-04-25T13:45:31.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f67fc640 1 -- 172.21.15.80:0/3371647384 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3e80ab600 con 0x7fb3f80b5330 2024-04-25T13:45:31.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3f67fc640 1 -- 172.21.15.80:0/3371647384 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80ab9d0 con 0x7fb3f80b5330 2024-04-25T13:45:31.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3fdb09640 1 -- 172.21.15.80:0/3371647384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80b5330 msgr2=0x7fb3f80d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/3371647384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80b5330 0x7fb3f80d0ab0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80097c0 tx=0x7fb3e801c410 comp rx=0 tx=0).stop 2024-04-25T13:45:31.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3fdb09640 1 -- 172.21.15.80:0/3371647384 shutdown_connections 2024-04-25T13:45:31.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/3371647384 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 0x7fb3f8161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/3371647384 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80b5330 0x7fb3f80d0ab0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3fdb09640 1 -- 172.21.15.80:0/3371647384 >> 172.21.15.80:0/3371647384 conn(0x7fb3f80b4160 msgr2=0x7fb3f80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:31.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3fdb09640 1 -- 172.21.15.80:0/3371647384 shutdown_connections 2024-04-25T13:45:31.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.575+0000 7fb3fdb09640 1 -- 172.21.15.80:0/3371647384 wait complete. 2024-04-25T13:45:31.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3fdb09640 1 Processor -- start 2024-04-25T13:45:31.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3fdb09640 1 -- start start 2024-04-25T13:45:31.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3fdb09640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b5330 0x7fb3f81f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3fdb09640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80d0ff0 0x7fb3f8144390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3fdb09640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 0x7fb3f80af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3fdb09640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80b0cd0 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3fdb09640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f806e890 con 0x7fb3f80b5330 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3fdb09640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80b0e50 con 0x7fb3f8161b60 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3f6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80d0ff0 0x7fb3f8144390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3f6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80d0ff0 0x7fb3f8144390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48860/0 (socket says 172.21.15.80:48860) 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.579+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/1392739687 learned_addr learned my addr 172.21.15.80:0/1392739687 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f7fff640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 0x7fb3f80af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b5330 0x7fb3f81f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 msgr2=0x7fb3f80af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f6ffd640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 0x7fb3f80af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b5330 msgr2=0x7fb3f81f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f6ffd640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b5330 0x7fb3f81f98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/1392739687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f8168640 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f6ffd640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80d0ff0 0x7fb3f8144390 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00c3a0 tx=0x7fb3ec00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:31.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3dffff640 1 -- 172.21.15.80:0/1392739687 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec017070 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3dffff640 1 -- 172.21.15.80:0/1392739687 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ec005cf0 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f8168920 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3dffff640 1 -- 172.21.15.80:0/1392739687 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0072f0 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f815c470 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f806a3a0 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3dffff640 1 -- 172.21.15.80:0/1392739687 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb3ec01a070 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3dffff640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:31.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3dffff640 1 -- 172.21.15.80:0/1392739687 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5709+0+0 (secure 0 0 0) 0x7fb3ec08e5e0 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.583+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:31.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.587+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80ad000 tx=0x7fb3e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:31.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.591+0000 7fb3dffff640 1 -- 172.21.15.80:0/1392739687 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb3f81624b0 con 0x7fb3f80d0ff0 2024-04-25T13:45:31.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:31.795+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb3f806fd00 con 0x7fb3f80d0ff0 2024-04-25T13:45:32.607 INFO:tasks.ceph.mds.a.smithi080.stderr:2024-04-25T13:45:32.607+0000 7fcff08ad640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:45:32.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.679+0000 7fb3dffff640 1 -- 172.21.15.80:0/1392739687 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v48) ==== 118+0+0 (secure 0 0 0) 0x7fb3ec05b4d0 con 0x7fb3f80d0ff0 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 msgr2=0x7fb3e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb3e80ad000 tx=0x7fb3e8008040 comp rx=0 tx=0).stop 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80d0ff0 msgr2=0x7fb3f8144390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80d0ff0 0x7fb3f8144390 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb3ec00c3a0 tx=0x7fb3ec00c860 comp rx=0 tx=0).stop 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 shutdown_connections 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f8161b60 0x7fb3f80af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f80d0ff0 0x7fb3f8144390 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 --2- 172.21.15.80:0/1392739687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b5330 0x7fb3f81f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 >> 172.21.15.80:0/1392739687 conn(0x7fb3f80b4160 msgr2=0x7fb3f8160ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 shutdown_connections 2024-04-25T13:45:32.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.683+0000 7fb3fdb09640 1 -- 172.21.15.80:0/1392739687 wait complete. 2024-04-25T13:45:32.706 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:45:32.706 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:45:32.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2eeaa88640 1 Processor -- start 2024-04-25T13:45:32.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2eeaa88640 1 -- start start 2024-04-25T13:45:32.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2eeaa88640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2ee4068ca0 0x7f2ee40b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:32.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2eeaa88640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee40b7d90 con 0x7f2ee40b65f0 2024-04-25T13:45:32.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2eeaa88640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ee40b8070 con 0x7f2ee40b11f0 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2eeaa88640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2ee40b1b40 con 0x7f2ee4068ca0 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee8824640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2ee40b65f0 0x7f2ee406f150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51404/0 (socket says 172.21.15.80:51404) 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee8824640 1 -- 172.21.15.80:0/804479423 learned_addr learned my addr 172.21.15.80:0/804479423 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee9025640 1 --2- 172.21.15.80:0/804479423 >> v2:172.21.15.87:3300/0 conn(0x7f2ee4068ca0 0x7f2ee40b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2948556752 0 0) 0x7f2ee40b7d90 con 0x7f2ee40b65f0 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2ed0003530 con 0x7f2ee40b65f0 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4033609281 0 0) 0x7f2ed0003530 con 0x7f2ee40b65f0 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 >> v2:172.21.15.87:3300/0 conn(0x7f2ee4068ca0 msgr2=0x7f2ee40b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 --2- 172.21.15.80:0/804479423 >> v2:172.21.15.87:3300/0 conn(0x7f2ee4068ca0 0x7f2ee40b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 >> v1:172.21.15.87:6789/0 conn(0x7f2ee40b11f0 legacy=0x7f2ee4068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee40b0710 con 0x7f2ee40b65f0 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2ed4001a00 con 0x7f2ee40b65f0 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 >> v1:172.21.15.80:6789/0 conn(0x7f2ee40b65f0 legacy=0x7f2ee406f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:32.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed00039c0 0x7f2ed0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0004810 0x7f2ed0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0005260 0x7f2ed000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed000c1c0 con 0x7f2ed0005260 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ed000c370 con 0x7f2ed00039c0 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ed000c570 con 0x7f2ed0004810 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee9025640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0005260 0x7f2ed000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee8824640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed00039c0 0x7f2ed0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee9025640 1 -- 172.21.15.80:0/804479423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0004810 msgr2=0x7f2ed0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee3fff640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0004810 0x7f2ed0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee9025640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ed0004810 0x7f2ed0004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee9025640 1 -- 172.21.15.80:0/804479423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed00039c0 msgr2=0x7f2ed0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee9025640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed00039c0 0x7f2ed0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.859+0000 7f2ee9025640 1 -- 172.21.15.80:0/804479423 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ed0003530 con 0x7f2ed0005260 2024-04-25T13:45:32.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2ee9025640 1 -- 172.21.15.80:0/804479423 reap_dead start 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2ee9025640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0005260 0x7f2ed000b9d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009ab0 tx=0x7f2edc01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab23a75c36f53154 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0096b0 con 0x7f2ed0005260 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2edc004020 con 0x7f2ed0005260 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2ee37fe640 1 -- 172.21.15.80:0/804479423 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc004410 con 0x7f2ed0005260 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2eeaa88640 1 -- 172.21.15.80:0/804479423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0005260 msgr2=0x7f2ed000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2eeaa88640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0005260 0x7f2ed000b9d0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009ab0 tx=0x7f2edc01ca90 comp rx=0 tx=0).stop 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2eeaa88640 1 -- 172.21.15.80:0/804479423 shutdown_connections 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2eeaa88640 1 --2- 172.21.15.80:0/804479423 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ed0005260 0x7f2ed000b9d0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2eeaa88640 1 -- 172.21.15.80:0/804479423 >> 172.21.15.80:0/804479423 conn(0x7f2ee4074000 msgr2=0x7f2ee406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2eeaa88640 1 -- 172.21.15.80:0/804479423 shutdown_connections 2024-04-25T13:45:32.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.863+0000 7f2eeaa88640 1 -- 172.21.15.80:0/804479423 wait complete. 2024-04-25T13:45:32.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 Processor -- start 2024-04-25T13:45:32.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 -- start start 2024-04-25T13:45:32.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed0005260 0x7f2ee4145f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:32.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ee4160b50 0x7f2ee4070020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:32.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ee4070560 0x7f2ee4153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40b6a00 con 0x7f2ee4070560 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee40b0d60 con 0x7f2ed0005260 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee40b6b80 con 0x7f2ee4160b50 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ee4070560 0x7f2ee4153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ee4070560 0x7f2ee4153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48870/0 (socket says 172.21.15.80:48870) 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 -- 172.21.15.80:0/2322627040 learned_addr learned my addr 172.21.15.80:0/2322627040 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee3fff640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ee4160b50 0x7f2ee4070020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee8824640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed0005260 0x7f2ee4145f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 -- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ee4160b50 msgr2=0x7f2ee4070020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ee4160b50 0x7f2ee4070020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 -- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed0005260 msgr2=0x7f2ee4145f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed0005260 0x7f2ee4145f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 -- 172.21.15.80:0/2322627040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee41464a0 con 0x7f2ee4070560 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee8824640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed0005260 0x7f2ee4145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:32.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee9025640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ee4070560 0x7f2ee4153300 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01d330 tx=0x7f2ed0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:32.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee17fa640 1 -- 172.21.15.80:0/2322627040 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0bb070 con 0x7f2ee4070560 2024-04-25T13:45:32.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee17fa640 1 -- 172.21.15.80:0/2322627040 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2edc003a50 con 0x7f2ee4070560 2024-04-25T13:45:32.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2ee17fa640 1 -- 172.21.15.80:0/2322627040 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc0b2070 con 0x7f2ee4070560 2024-04-25T13:45:32.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee4154440 con 0x7f2ee4070560 2024-04-25T13:45:32.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee4154a00 con 0x7f2ee4070560 2024-04-25T13:45:32.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.867+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee4151a90 con 0x7f2ee4070560 2024-04-25T13:45:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.871+0000 7f2ee17fa640 1 -- 172.21.15.80:0/2322627040 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2edc0be070 con 0x7f2ee4070560 2024-04-25T13:45:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.871+0000 7f2ee17fa640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ec406f430 0x7f2ec4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.871+0000 7f2ee17fa640 1 -- 172.21.15.80:0/2322627040 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f2edc1289e0 con 0x7f2ee4070560 2024-04-25T13:45:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.871+0000 7f2ee8824640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ec406f430 0x7f2ec4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:32.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.871+0000 7f2ee8824640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ec406f430 0x7f2ec4071940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001590 tx=0x7f2ed40014e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:32.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:32.875+0000 7f2ee17fa640 1 -- 172.21.15.80:0/2322627040 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2edc0167a0 con 0x7f2ee4070560 2024-04-25T13:45:33.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.055+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2ee40b5e60 con 0x7f2ee4070560 2024-04-25T13:45:33.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.683+0000 7f2ee17fa640 1 -- 172.21.15.80:0/2322627040 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v49) ==== 122+0+0 (secure 0 0 0) 0x7f2edc0f58d0 con 0x7f2ee4070560 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ec406f430 msgr2=0x7f2ec4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ec406f430 0x7f2ec4071940 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001590 tx=0x7f2ed40014e0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ee4070560 msgr2=0x7f2ee4153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ee4070560 0x7f2ee4153300 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f2edc01d330 tx=0x7f2ed0005840 comp rx=0 tx=0).stop 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 shutdown_connections 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ec406f430 0x7f2ec4071940 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ee4070560 0x7f2ee4153300 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ee4160b50 0x7f2ee4070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 --2- 172.21.15.80:0/2322627040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ed0005260 0x7f2ee4145f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 >> 172.21.15.80:0/2322627040 conn(0x7f2ee4074000 msgr2=0x7f2ee415d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:33.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 shutdown_connections 2024-04-25T13:45:33.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.687+0000 7f2eeaa88640 1 -- 172.21.15.80:0/2322627040 wait complete. 2024-04-25T13:45:33.711 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:45:33.711 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:33.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff046c2f640 1 Processor -- start 2024-04-25T13:45:33.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff046c2f640 1 -- start start 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff046c2f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff0400b2d30 0x7ff0400d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff046c2f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff0400d0350 0x7ff0400d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff046c2f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff04006cbe0 0x7ff04006cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff046c2f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff0400d1120 con 0x7ff0400b2d30 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff046c2f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff0400d0ca0 con 0x7ff0400d0350 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff046c2f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff0400d30b0 con 0x7ff04006cbe0 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0449cb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff0400b2d30 0x7ff0400d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0449cb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff0400b2d30 0x7ff0400d1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48880/0 (socket says 172.21.15.80:48880) 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0449cb640 1 -- 172.21.15.80:0/509993827 learned_addr learned my addr 172.21.15.80:0/509993827 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0451cc640 1 --2- 172.21.15.80:0/509993827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff04006cbe0 0x7ff04006cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff037fff640 1 --2- 172.21.15.80:0/509993827 >> v2:172.21.15.87:3300/0 conn(0x7ff0400d0350 0x7ff0400d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0449cb640 1 -- 172.21.15.80:0/509993827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff04006cbe0 msgr2=0x7ff04006cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0449cb640 1 --2- 172.21.15.80:0/509993827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff04006cbe0 0x7ff04006cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0449cb640 1 -- 172.21.15.80:0/509993827 >> v2:172.21.15.87:3300/0 conn(0x7ff0400d0350 msgr2=0x7ff0400d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0449cb640 1 --2- 172.21.15.80:0/509993827 >> v2:172.21.15.87:3300/0 conn(0x7ff0400d0350 0x7ff0400d0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.859+0000 7ff0449cb640 1 -- 172.21.15.80:0/509993827 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0400ce1c0 con 0x7ff0400b2d30 2024-04-25T13:45:33.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff0449cb640 1 --2- 172.21.15.80:0/509993827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400b2d30 0x7ff0400d1dd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff040069670 tx=0x7ff03c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b73301ce0ed2a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff0377fe640 1 -- 172.21.15.80:0/509993827 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0b1070 con 0x7ff0400b2d30 2024-04-25T13:45:33.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff0377fe640 1 -- 172.21.15.80:0/509993827 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff03c0ac630 con 0x7ff0400b2d30 2024-04-25T13:45:33.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff0377fe640 1 -- 172.21.15.80:0/509993827 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c0a8050 con 0x7ff0400b2d30 2024-04-25T13:45:33.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 -- 172.21.15.80:0/509993827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400b2d30 msgr2=0x7ff0400d1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 --2- 172.21.15.80:0/509993827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400b2d30 0x7ff0400d1dd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff040069670 tx=0x7ff03c01c520 comp rx=0 tx=0).stop 2024-04-25T13:45:33.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 -- 172.21.15.80:0/509993827 shutdown_connections 2024-04-25T13:45:33.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 --2- 172.21.15.80:0/509993827 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff04006cbe0 0x7ff04006cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 --2- 172.21.15.80:0/509993827 >> v2:172.21.15.87:3300/0 conn(0x7ff0400d0350 0x7ff0400d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 --2- 172.21.15.80:0/509993827 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400b2d30 0x7ff0400d1dd0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 -- 172.21.15.80:0/509993827 >> 172.21.15.80:0/509993827 conn(0x7ff0400d3ad0 msgr2=0x7ff0400d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:33.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 -- 172.21.15.80:0/509993827 shutdown_connections 2024-04-25T13:45:33.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.863+0000 7ff046c2f640 1 -- 172.21.15.80:0/509993827 wait complete. 2024-04-25T13:45:33.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 Processor -- start 2024-04-25T13:45:33.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 -- start start 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff04006cbe0 0x7ff0400cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400d0350 0x7ff0401643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff040165050 0x7ff0400c8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff04006d940 con 0x7ff0400d0350 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff0400d3390 con 0x7ff04006cbe0 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff0400cf4b0 con 0x7ff040165050 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400d0350 0x7ff0401643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400d0350 0x7ff0401643d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48884/0 (socket says 172.21.15.80:48884) 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 -- 172.21.15.80:0/820501264 learned_addr learned my addr 172.21.15.80:0/820501264 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff0451cc640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff040165050 0x7ff0400c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff0449cb640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff04006cbe0 0x7ff0400cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 -- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff040165050 msgr2=0x7ff0400c8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff040165050 0x7ff0400c8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 -- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff04006cbe0 msgr2=0x7ff0400cb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff04006cbe0 0x7ff0400cb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 -- 172.21.15.80:0/820501264 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff040164910 con 0x7ff0400d0350 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff0451cc640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff040165050 0x7ff0400c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff0449cb640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff04006cbe0 0x7ff0400cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff037fff640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400d0350 0x7ff0401643d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff03000d620 tx=0x7ff03000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff0357fa640 1 -- 172.21.15.80:0/820501264 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03000b620 con 0x7ff0400d0350 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff0357fa640 1 -- 172.21.15.80:0/820501264 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff030019070 con 0x7ff0400d0350 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0400c77a0 con 0x7ff0400d0350 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff0357fa640 1 -- 172.21.15.80:0/820501264 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03000b9e0 con 0x7ff0400d0350 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.867+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0400c7a40 con 0x7ff0400d0350 2024-04-25T13:45:33.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.871+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff04006a3a0 con 0x7ff0400d0350 2024-04-25T13:45:33.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.871+0000 7ff0357fa640 1 -- 172.21.15.80:0/820501264 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff03000bb80 con 0x7ff0400d0350 2024-04-25T13:45:33.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.871+0000 7ff0357fa640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff02c06f400 0x7ff02c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:33.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.871+0000 7ff0357fa640 1 -- 172.21.15.80:0/820501264 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7ff03008f6a0 con 0x7ff0400d0350 2024-04-25T13:45:33.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.871+0000 7ff0449cb640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff02c06f400 0x7ff02c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:33.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.871+0000 7ff0449cb640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff02c06f400 0x7ff02c071910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0ae000 tx=0x7ff03c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:33.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:33.875+0000 7ff0357fa640 1 -- 172.21.15.80:0/820501264 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff03005c620 con 0x7ff0400d0350 2024-04-25T13:45:34.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.055+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff040074c30 con 0x7ff0400d0350 2024-04-25T13:45:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.055+0000 7ff0357fa640 1 -- 172.21.15.80:0/820501264 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7ff03005bf20 con 0x7ff0400d0350 2024-04-25T13:45:34.056 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:34.056 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:45:34.056 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:45:34.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff02c06f400 msgr2=0x7ff02c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff02c06f400 0x7ff02c071910 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff03c0ae000 tx=0x7ff03c0a7000 comp rx=0 tx=0).stop 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400d0350 msgr2=0x7ff0401643d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400d0350 0x7ff0401643d0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff03000d620 tx=0x7ff03000dae0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 shutdown_connections 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff02c06f400 0x7ff02c071910 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff040165050 0x7ff0400c8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff0400d0350 0x7ff0401643d0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 --2- 172.21.15.80:0/820501264 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff04006cbe0 0x7ff0400cb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 >> 172.21.15.80:0/820501264 conn(0x7ff0400d3ad0 msgr2=0x7ff04016a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 shutdown_connections 2024-04-25T13:45:34.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.059+0000 7ff046c2f640 1 -- 172.21.15.80:0/820501264 wait complete. 2024-04-25T13:45:34.076 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:45:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.223+0000 7f2ecaa0d640 1 Processor -- start 2024-04-25T13:45:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.223+0000 7f2ecaa0d640 1 -- start start 2024-04-25T13:45:34.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ecaa0d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec406f5a0 0x7f2ec4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ecaa0d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ec40733b0 0x7f2ec4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ecaa0d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ec406fcf0 con 0x7f2ec4161070 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ecaa0d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2ec41619c0 con 0x7f2ec40733b0 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ecaa0d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec4073f10 con 0x7f2ec406f5a0 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ec8faa640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ec40733b0 0x7f2ec4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ebb7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2ec4161070 0x7f2ec40b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51416/0 (socket says 172.21.15.80:51416) 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ebb7fe640 1 -- 172.21.15.80:0/3081524413 learned_addr learned my addr 172.21.15.80:0/3081524413 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ec8faa640 1 --2- 172.21.15.80:0/3081524413 >> v2:172.21.15.80:3300/0 conn(0x7f2ec40733b0 0x7f2ec4161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48894/0 (socket says 172.21.15.80:48894) 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ec3fff640 1 --2- 172.21.15.80:0/3081524413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec406f5a0 0x7f2ec4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ec8faa640 1 -- 172.21.15.80:0/3081524413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec406f5a0 msgr2=0x7f2ec4160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ec37fe640 1 -- 172.21.15.80:0/3081524413 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 85120440 0 0) 0x7f2ec406fcf0 con 0x7f2ec4161070 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ec8faa640 1 --2- 172.21.15.80:0/3081524413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec406f5a0 0x7f2ec4160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ec8faa640 1 -- 172.21.15.80:0/3081524413 >> v1:172.21.15.80:6789/0 conn(0x7f2ec4161070 legacy=0x7f2ec40b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.227+0000 7f2ec8faa640 1 -- 172.21.15.80:0/3081524413 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec40b0c60 con 0x7f2ec40733b0 2024-04-25T13:45:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ec8faa640 1 --2- 172.21.15.80:0/3081524413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec4161480 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009dc0 tx=0x7f2ebc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d083ab5c26c302f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ec37fe640 1 -- 172.21.15.80:0/3081524413 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0b2070 con 0x7f2ec40733b0 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ec37fe640 1 -- 172.21.15.80:0/3081524413 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ebc0a57b0 con 0x7f2ec40733b0 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ec37fe640 1 -- 172.21.15.80:0/3081524413 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc0a5b80 con 0x7f2ec40733b0 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/3081524413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 msgr2=0x7f2ec4161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/3081524413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec4161480 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009dc0 tx=0x7f2ebc01c8e0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/3081524413 shutdown_connections 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/3081524413 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec4161480 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/3081524413 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec406f5a0 0x7f2ec4160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/3081524413 >> 172.21.15.80:0/3081524413 conn(0x7f2ec406d080 msgr2=0x7f2ec40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/3081524413 shutdown_connections 2024-04-25T13:45:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/3081524413 wait complete. 2024-04-25T13:45:34.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 Processor -- start 2024-04-25T13:45:34.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.231+0000 7f2ecaa0d640 1 -- start start 2024-04-25T13:45:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec406f5a0 0x7f2ec4154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec415e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec4161070 0x7f2ec41f9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec406e610 con 0x7f2ec40733b0 2024-04-25T13:45:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec406e490 con 0x7f2ec406f5a0 2024-04-25T13:45:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ec406e790 con 0x7f2ec4161070 2024-04-25T13:45:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec415e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48902/0 (socket says 172.21.15.80:48902) 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 -- 172.21.15.80:0/913786490 learned_addr learned my addr 172.21.15.80:0/913786490 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ec3fff640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec406f5a0 0x7f2ec4154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ec8faa640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec4161070 0x7f2ec41f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 -- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec4161070 msgr2=0x7f2ec41f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec4161070 0x7f2ec41f9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 -- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec406f5a0 msgr2=0x7f2ec4154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec406f5a0 0x7f2ec4154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 -- 172.21.15.80:0/913786490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec40b42c0 con 0x7f2ec40733b0 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ec3fff640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec406f5a0 0x7f2ec4154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ebb7fe640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec415e130 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4071e30 tx=0x7f2eac00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ec17fa640 1 -- 172.21.15.80:0/913786490 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac0074e0 con 0x7f2ec40733b0 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ec17fa640 1 -- 172.21.15.80:0/913786490 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2eac0047f0 con 0x7f2ec40733b0 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec41fa390 con 0x7f2ec40733b0 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ec17fa640 1 -- 172.21.15.80:0/913786490 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eac007680 con 0x7f2ec40733b0 2024-04-25T13:45:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec415d650 con 0x7f2ec40733b0 2024-04-25T13:45:34.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec41fa100 con 0x7f2ec40733b0 2024-04-25T13:45:34.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ec17fa640 1 -- 172.21.15.80:0/913786490 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2eac011070 con 0x7f2ec40733b0 2024-04-25T13:45:34.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.235+0000 7f2ec17fa640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f430 0x7f2ea4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.239+0000 7f2ec17fa640 1 -- 172.21.15.80:0/913786490 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f2eac08f2c0 con 0x7f2ec40733b0 2024-04-25T13:45:34.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.239+0000 7f2ec3fff640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f430 0x7f2ea4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.239+0000 7f2ec3fff640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f430 0x7f2ea4071940 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40b6e70 tx=0x7f2eb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:34.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.239+0000 7f2ec17fa640 1 -- 172.21.15.80:0/913786490 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2eac05c320 con 0x7f2ec40733b0 2024-04-25T13:45:34.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.403+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ec406a3a0 con 0x7f2ec40733b0 2024-04-25T13:45:34.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ec17fa640 1 -- 172.21.15.80:0/913786490 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16599 (secure 0 0 0) 0x7f2eac05bc20 con 0x7f2ec40733b0 2024-04-25T13:45:34.405 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:34.405 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":103,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:31.659793+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"90","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:45:28.400550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T13:45:28.988401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f430 msgr2=0x7f2ea4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f430 0x7f2ea4071940 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2ec40b6e70 tx=0x7f2eb4008040 comp rx=0 tx=0).stop 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 msgr2=0x7f2ec415e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec415e130 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f2ec4071e30 tx=0x7f2eac00c820 comp rx=0 tx=0).stop 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 shutdown_connections 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ea406f430 0x7f2ea4071940 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ec4161070 0x7f2ec41f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ec40733b0 0x7f2ec415e130 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 --2- 172.21.15.80:0/913786490 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ec406f5a0 0x7f2ec4154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 >> 172.21.15.80:0/913786490 conn(0x7f2ec406d080 msgr2=0x7f2ec40b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:34.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 shutdown_connections 2024-04-25T13:45:34.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.407+0000 7f2ecaa0d640 1 -- 172.21.15.80:0/913786490 wait complete. 2024-04-25T13:45:34.427 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:45:34.427 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:45:34.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324ccdf640 1 Processor -- start 2024-04-25T13:45:34.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324ccdf640 1 -- start start 2024-04-25T13:45:34.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324ccdf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 0x7f324806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324ccdf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f324815d630 0x7f324815da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324ccdf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f324806cf90 con 0x7f32480cfb60 2024-04-25T13:45:34.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324ccdf640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3248143920 con 0x7f324815d630 2024-04-25T13:45:34.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324ccdf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f324806f350 con 0x7f3248161630 2024-04-25T13:45:34.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f3245d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f32480cfb60 0x7f32480cff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51428/0 (socket says 172.21.15.80:51428) 2024-04-25T13:45:34.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f3245d9b640 1 -- 172.21.15.80:0/2670657206 learned_addr learned my addr 172.21.15.80:0/2670657206 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:34.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f3246d9d640 1 --2- 172.21.15.80:0/2670657206 >> v2:172.21.15.87:3300/0 conn(0x7f324815d630 0x7f324815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324659c640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 0x7f324806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3456927992 0 0) 0x7f324806cf90 con 0x7f32480cfb60 2024-04-25T13:45:34.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.579+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3234003530 con 0x7f32480cfb60 2024-04-25T13:45:34.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 27154479 0 0) 0x7f3234003530 con 0x7f32480cfb60 2024-04-25T13:45:34.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 msgr2=0x7f324806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:34.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 0x7f324806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 >> v2:172.21.15.87:3300/0 conn(0x7f324815d630 msgr2=0x7f324815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 --2- 172.21.15.80:0/2670657206 >> v2:172.21.15.87:3300/0 conn(0x7f324815d630 0x7f324815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f324806d270 con 0x7f32480cfb60 2024-04-25T13:45:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f3230001a00 con 0x7f32480cfb60 2024-04-25T13:45:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 >> v1:172.21.15.80:6789/0 conn(0x7f32480cfb60 legacy=0x7f32480cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f3234003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3234004810 0x7f3234004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 0x7f3234005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3248143920 con 0x7f32340039c0 2024-04-25T13:45:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f323400b9d0 con 0x7f3234004810 2024-04-25T13:45:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f323400bbd0 con 0x7f3248161630 2024-04-25T13:45:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324659c640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f3234003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f3245d9b640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3234004810 0x7f3234004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f3246d9d640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 0x7f3234005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324659c640 1 -- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 msgr2=0x7f3234005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:34.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324659c640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 0x7f3234005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324659c640 1 -- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3234004810 msgr2=0x7f3234004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:34.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324659c640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3234004810 0x7f3234004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324659c640 1 -- 172.21.15.80:0/2670657206 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3234003530 con 0x7f32340039c0 2024-04-25T13:45:34.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f3246d9d640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3248161630 0x7f3234005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:34.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324659c640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f3234003e30 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3240009ae0 tx=0x7f324001ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6101404aae4460c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:34.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400b2070 con 0x7f32340039c0 2024-04-25T13:45:34.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32400a5570 con 0x7f32340039c0 2024-04-25T13:45:34.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324559a640 1 -- 172.21.15.80:0/2670657206 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400096b0 con 0x7f32340039c0 2024-04-25T13:45:34.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324ccdf640 1 -- 172.21.15.80:0/2670657206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 msgr2=0x7f3234003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f324ccdf640 1 --2- 172.21.15.80:0/2670657206 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f3234003e30 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f3240009ae0 tx=0x7f324001ca90 comp rx=0 tx=0).stop 2024-04-25T13:45:34.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.583+0000 7f3246d9d640 1 -- 172.21.15.80:0/2670657206 reap_dead start 2024-04-25T13:45:34.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 -- 172.21.15.80:0/2670657206 shutdown_connections 2024-04-25T13:45:34.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 -- 172.21.15.80:0/2670657206 >> 172.21.15.80:0/2670657206 conn(0x7f32480d33a0 msgr2=0x7f32480cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:34.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 -- 172.21.15.80:0/2670657206 shutdown_connections 2024-04-25T13:45:34.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 -- 172.21.15.80:0/2670657206 wait complete. 2024-04-25T13:45:34.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 Processor -- start 2024-04-25T13:45:34.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 -- start start 2024-04-25T13:45:34.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f32481f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32481426f0 0x7f3248160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32480d13b0 0x7f32480d17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3248161a40 con 0x7f32340039c0 2024-04-25T13:45:34.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3248161110 con 0x7f32480d13b0 2024-04-25T13:45:34.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324ccdf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3248070d50 con 0x7f32481426f0 2024-04-25T13:45:34.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f32481f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f32481f6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48918/0 (socket says 172.21.15.80:48918) 2024-04-25T13:45:34.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324659c640 1 -- 172.21.15.80:0/1170063681 learned_addr learned my addr 172.21.15.80:0/1170063681 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:34.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f3246d9d640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32480d13b0 0x7f32480d17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f3245d9b640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32481426f0 0x7f3248160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324659c640 1 -- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32481426f0 msgr2=0x7f3248160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324659c640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32481426f0 0x7f3248160170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324659c640 1 -- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32480d13b0 msgr2=0x7f32480d17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324659c640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32480d13b0 0x7f32480d17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f324659c640 1 -- 172.21.15.80:0/1170063681 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32480d1d00 con 0x7f32340039c0 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.587+0000 7f3246d9d640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32480d13b0 0x7f32480d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f324659c640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f32481f6a70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f32400b0040 tx=0x7f3234005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f323effd640 1 -- 172.21.15.80:0/1170063681 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32400b2040 con 0x7f32340039c0 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f323effd640 1 -- 172.21.15.80:0/1170063681 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32400a5d20 con 0x7f32340039c0 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32481f72c0 con 0x7f32340039c0 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f323effd640 1 -- 172.21.15.80:0/1170063681 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3240004260 con 0x7f32340039c0 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32481f7880 con 0x7f32340039c0 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f324806a3a0 con 0x7f32340039c0 2024-04-25T13:45:34.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f323effd640 1 -- 172.21.15.80:0/1170063681 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f32400c1030 con 0x7f32340039c0 2024-04-25T13:45:34.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f323effd640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f322806f430 0x7f3228071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f323effd640 1 -- 172.21.15.80:0/1170063681 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f32401290d0 con 0x7f32340039c0 2024-04-25T13:45:34.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f3245d9b640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f322806f430 0x7f3228071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.591+0000 7f3245d9b640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f322806f430 0x7f3228071940 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3230004c00 tx=0x7f3230004b90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:34.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.595+0000 7f323effd640 1 -- 172.21.15.80:0/1170063681 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32480d1d00 con 0x7f32340039c0 2024-04-25T13:45:34.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.755+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32480d0c40 con 0x7f322806f430 2024-04-25T13:45:34.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.771+0000 7f323effd640 1 -- 172.21.15.80:0/1170063681 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247981 (secure 0 0 0) 0x7f32480d0c40 con 0x7f322806f430 2024-04-25T13:45:34.771 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:34.776 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f322806f430 msgr2=0x7f3228071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f322806f430 0x7f3228071940 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f3230004c00 tx=0x7f3230004b90 comp rx=0 tx=0).stop 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 msgr2=0x7f32481f6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f32481f6a70 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f32400b0040 tx=0x7f3234005840 comp rx=0 tx=0).stop 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 shutdown_connections 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f322806f430 0x7f3228071940 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32480d13b0 0x7f32480d17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32481426f0 0x7f3248160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 --2- 172.21.15.80:0/1170063681 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32340039c0 0x7f32481f6a70 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 >> 172.21.15.80:0/1170063681 conn(0x7f32480d33a0 msgr2=0x7f324815ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:34.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 shutdown_connections 2024-04-25T13:45:34.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.775+0000 7f324ccdf640 1 -- 172.21.15.80:0/1170063681 wait complete. 2024-04-25T13:45:34.796 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-04-25T13:45:34.206621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556954,"statfs":{"total":1600000000,"available":1594321436,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.590620"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679635+0000","last_change":"2024-04-25T13:45:30.706530+0000","last_active":"2024-04-25T13:45:31.679635+0000","last_peered":"2024-04-25T13:45:31.679635+0000","last_clean":"2024-04-25T13:45:31.679635+0000","last_became_active":"2024-04-25T13:45:30.705786+0000","last_became_peered":"2024-04-25T13:45:30.705786+0000","last_unstale":"2024-04-25T13:45:31.679635+0000","last_undegraded":"2024-04-25T13:45:31.679635+0000","last_fullsized":"2024-04-25T13:45:31.679635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:37.514261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677415+0000","last_change":"2024-04-25T13:45:30.706054+0000","last_active":"2024-04-25T13:45:31.677415+0000","last_peered":"2024-04-25T13:45:31.677415+0000","last_clean":"2024-04-25T13:45:31.677415+0000","last_became_active":"2024-04-25T13:45:30.704852+0000","last_became_peered":"2024-04-25T13:45:30.704852+0000","last_unstale":"2024-04-25T13:45:31.677415+0000","last_undegraded":"2024-04-25T13:45:31.677415+0000","last_fullsized":"2024-04-25T13:45:31.677415+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:49.045396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678183+0000","last_change":"2024-04-25T13:45:30.691197+0000","last_active":"2024-04-25T13:45:31.678183+0000","last_peered":"2024-04-25T13:45:31.678183+0000","last_clean":"2024-04-25T13:45:31.678183+0000","last_became_active":"2024-04-25T13:45:30.690270+0000","last_became_peered":"2024-04-25T13:45:30.690270+0000","last_unstale":"2024-04-25T13:45:31.678183+0000","last_undegraded":"2024-04-25T13:45:31.678183+0000","last_fullsized":"2024-04-25T13:45:31.678183+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:58.509848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082426+0000","last_change":"2024-04-25T13:45:30.703626+0000","last_active":"2024-04-25T13:45:32.082426+0000","last_peered":"2024-04-25T13:45:32.082426+0000","last_clean":"2024-04-25T13:45:32.082426+0000","last_became_active":"2024-04-25T13:45:30.702976+0000","last_became_peered":"2024-04-25T13:45:30.702976+0000","last_unstale":"2024-04-25T13:45:32.082426+0000","last_undegraded":"2024-04-25T13:45:32.082426+0000","last_fullsized":"2024-04-25T13:45:32.082426+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:21.089644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673356+0000","last_change":"2024-04-25T13:45:30.693814+0000","last_active":"2024-04-25T13:45:31.673356+0000","last_peered":"2024-04-25T13:45:31.673356+0000","last_clean":"2024-04-25T13:45:31.673356+0000","last_became_active":"2024-04-25T13:45:30.692997+0000","last_became_peered":"2024-04-25T13:45:30.692997+0000","last_unstale":"2024-04-25T13:45:31.673356+0000","last_undegraded":"2024-04-25T13:45:31.673356+0000","last_fullsized":"2024-04-25T13:45:31.673356+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:18.356937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678531+0000","last_change":"2024-04-25T13:45:30.695054+0000","last_active":"2024-04-25T13:45:31.678531+0000","last_peered":"2024-04-25T13:45:31.678531+0000","last_clean":"2024-04-25T13:45:31.678531+0000","last_became_active":"2024-04-25T13:45:30.694118+0000","last_became_peered":"2024-04-25T13:45:30.694118+0000","last_unstale":"2024-04-25T13:45:31.678531+0000","last_undegraded":"2024-04-25T13:45:31.678531+0000","last_fullsized":"2024-04-25T13:45:31.678531+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:57.495389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082798+0000","last_change":"2024-04-25T13:45:30.687364+0000","last_active":"2024-04-25T13:45:32.082798+0000","last_peered":"2024-04-25T13:45:32.082798+0000","last_clean":"2024-04-25T13:45:32.082798+0000","last_became_active":"2024-04-25T13:45:30.686618+0000","last_became_peered":"2024-04-25T13:45:30.686618+0000","last_unstale":"2024-04-25T13:45:32.082798+0000","last_undegraded":"2024-04-25T13:45:32.082798+0000","last_fullsized":"2024-04-25T13:45:32.082798+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:20.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677353+0000","last_change":"2024-04-25T13:45:30.688301+0000","last_active":"2024-04-25T13:45:31.677353+0000","last_peered":"2024-04-25T13:45:31.677353+0000","last_clean":"2024-04-25T13:45:31.677353+0000","last_became_active":"2024-04-25T13:45:30.686603+0000","last_became_peered":"2024-04-25T13:45:30.686603+0000","last_unstale":"2024-04-25T13:45:31.677353+0000","last_undegraded":"2024-04-25T13:45:31.677353+0000","last_fullsized":"2024-04-25T13:45:31.677353+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:39.897707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677125+0000","last_change":"2024-04-25T13:45:30.685224+0000","last_active":"2024-04-25T13:45:31.677125+0000","last_peered":"2024-04-25T13:45:31.677125+0000","last_clean":"2024-04-25T13:45:31.677125+0000","last_became_active":"2024-04-25T13:45:30.684513+0000","last_became_peered":"2024-04-25T13:45:30.684513+0000","last_unstale":"2024-04-25T13:45:31.677125+0000","last_undegraded":"2024-04-25T13:45:31.677125+0000","last_fullsized":"2024-04-25T13:45:31.677125+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:13.327393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679635+0000","last_change":"2024-04-25T13:45:30.690502+0000","last_active":"2024-04-25T13:45:31.679635+0000","last_peered":"2024-04-25T13:45:31.679635+0000","last_clean":"2024-04-25T13:45:31.679635+0000","last_became_active":"2024-04-25T13:45:30.689683+0000","last_became_peered":"2024-04-25T13:45:30.689683+0000","last_unstale":"2024-04-25T13:45:31.679635+0000","last_undegraded":"2024-04-25T13:45:31.679635+0000","last_fullsized":"2024-04-25T13:45:31.679635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:37.350256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680027+0000","last_change":"2024-04-25T13:45:30.692178+0000","last_active":"2024-04-25T13:45:31.680027+0000","last_peered":"2024-04-25T13:45:31.680027+0000","last_clean":"2024-04-25T13:45:31.680027+0000","last_became_active":"2024-04-25T13:45:30.686925+0000","last_became_peered":"2024-04-25T13:45:30.686925+0000","last_unstale":"2024-04-25T13:45:31.680027+0000","last_undegraded":"2024-04-25T13:45:31.680027+0000","last_fullsized":"2024-04-25T13:45:31.680027+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:52.080827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678492+0000","last_change":"2024-04-25T13:45:30.690575+0000","last_active":"2024-04-25T13:45:31.678492+0000","last_peered":"2024-04-25T13:45:31.678492+0000","last_clean":"2024-04-25T13:45:31.678492+0000","last_became_active":"2024-04-25T13:45:30.689761+0000","last_became_peered":"2024-04-25T13:45:30.689761+0000","last_unstale":"2024-04-25T13:45:31.678492+0000","last_undegraded":"2024-04-25T13:45:31.678492+0000","last_fullsized":"2024-04-25T13:45:31.678492+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:15.675006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678694+0000","last_change":"2024-04-25T13:45:30.702023+0000","last_active":"2024-04-25T13:45:31.678694+0000","last_peered":"2024-04-25T13:45:31.678694+0000","last_clean":"2024-04-25T13:45:31.678694+0000","last_became_active":"2024-04-25T13:45:30.701220+0000","last_became_peered":"2024-04-25T13:45:30.701220+0000","last_unstale":"2024-04-25T13:45:31.678694+0000","last_undegraded":"2024-04-25T13:45:31.678694+0000","last_fullsized":"2024-04-25T13:45:31.678694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:32.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082722+0000","last_change":"2024-04-25T13:45:30.703325+0000","last_active":"2024-04-25T13:45:32.082722+0000","last_peered":"2024-04-25T13:45:32.082722+0000","last_clean":"2024-04-25T13:45:32.082722+0000","last_became_active":"2024-04-25T13:45:30.702178+0000","last_became_peered":"2024-04-25T13:45:30.702178+0000","last_unstale":"2024-04-25T13:45:32.082722+0000","last_undegraded":"2024-04-25T13:45:32.082722+0000","last_fullsized":"2024-04-25T13:45:32.082722+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:30.258376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678267+0000","last_change":"2024-04-25T13:45:30.687069+0000","last_active":"2024-04-25T13:45:31.678267+0000","last_peered":"2024-04-25T13:45:31.678267+0000","last_clean":"2024-04-25T13:45:31.678267+0000","last_became_active":"2024-04-25T13:45:30.686238+0000","last_became_peered":"2024-04-25T13:45:30.686238+0000","last_unstale":"2024-04-25T13:45:31.678267+0000","last_undegraded":"2024-04-25T13:45:31.678267+0000","last_fullsized":"2024-04-25T13:45:31.678267+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.772364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673327+0000","last_change":"2024-04-25T13:45:30.705163+0000","last_active":"2024-04-25T13:45:31.673327+0000","last_peered":"2024-04-25T13:45:31.673327+0000","last_clean":"2024-04-25T13:45:31.673327+0000","last_became_active":"2024-04-25T13:45:30.703622+0000","last_became_peered":"2024-04-25T13:45:30.703622+0000","last_unstale":"2024-04-25T13:45:31.673327+0000","last_undegraded":"2024-04-25T13:45:31.673327+0000","last_fullsized":"2024-04-25T13:45:31.673327+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:28.377812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673097+0000","last_change":"2024-04-25T13:45:30.696845+0000","last_active":"2024-04-25T13:45:31.673097+0000","last_peered":"2024-04-25T13:45:31.673097+0000","last_clean":"2024-04-25T13:45:31.673097+0000","last_became_active":"2024-04-25T13:45:30.695427+0000","last_became_peered":"2024-04-25T13:45:30.695427+0000","last_unstale":"2024-04-25T13:45:31.673097+0000","last_undegraded":"2024-04-25T13:45:31.673097+0000","last_fullsized":"2024-04-25T13:45:31.673097+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:42.226278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083405+0000","last_change":"2024-04-25T13:45:30.695731+0000","last_active":"2024-04-25T13:45:32.083405+0000","last_peered":"2024-04-25T13:45:32.083405+0000","last_clean":"2024-04-25T13:45:32.083405+0000","last_became_active":"2024-04-25T13:45:30.694534+0000","last_became_peered":"2024-04-25T13:45:30.694534+0000","last_unstale":"2024-04-25T13:45:32.083405+0000","last_undegraded":"2024-04-25T13:45:32.083405+0000","last_fullsized":"2024-04-25T13:45:32.083405+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:24.938460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.672981+0000","last_change":"2024-04-25T13:45:30.693893+0000","last_active":"2024-04-25T13:45:31.672981+0000","last_peered":"2024-04-25T13:45:31.672981+0000","last_clean":"2024-04-25T13:45:31.672981+0000","last_became_active":"2024-04-25T13:45:30.692463+0000","last_became_peered":"2024-04-25T13:45:30.692463+0000","last_unstale":"2024-04-25T13:45:31.672981+0000","last_undegraded":"2024-04-25T13:45:31.672981+0000","last_fullsized":"2024-04-25T13:45:31.672981+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.239491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083431+0000","last_change":"2024-04-25T13:45:30.699370+0000","last_active":"2024-04-25T13:45:32.083431+0000","last_peered":"2024-04-25T13:45:32.083431+0000","last_clean":"2024-04-25T13:45:32.083431+0000","last_became_active":"2024-04-25T13:45:30.689218+0000","last_became_peered":"2024-04-25T13:45:30.689218+0000","last_unstale":"2024-04-25T13:45:32.083431+0000","last_undegraded":"2024-04-25T13:45:32.083431+0000","last_fullsized":"2024-04-25T13:45:32.083431+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:02.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082888+0000","last_change":"2024-04-25T13:45:30.700740+0000","last_active":"2024-04-25T13:45:32.082888+0000","last_peered":"2024-04-25T13:45:32.082888+0000","last_clean":"2024-04-25T13:45:32.082888+0000","last_became_active":"2024-04-25T13:45:30.700202+0000","last_became_peered":"2024-04-25T13:45:30.700202+0000","last_unstale":"2024-04-25T13:45:32.082888+0000","last_undegraded":"2024-04-25T13:45:32.082888+0000","last_fullsized":"2024-04-25T13:45:32.082888+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:12.354500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679052+0000","last_change":"2024-04-25T13:45:30.705540+0000","last_active":"2024-04-25T13:45:31.679052+0000","last_peered":"2024-04-25T13:45:31.679052+0000","last_clean":"2024-04-25T13:45:31.679052+0000","last_became_active":"2024-04-25T13:45:30.698005+0000","last_became_peered":"2024-04-25T13:45:30.698005+0000","last_unstale":"2024-04-25T13:45:31.679052+0000","last_undegraded":"2024-04-25T13:45:31.679052+0000","last_fullsized":"2024-04-25T13:45:31.679052+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:10.359947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680072+0000","last_change":"2024-04-25T13:45:30.701179+0000","last_active":"2024-04-25T13:45:31.680072+0000","last_peered":"2024-04-25T13:45:31.680072+0000","last_clean":"2024-04-25T13:45:31.680072+0000","last_became_active":"2024-04-25T13:45:30.699936+0000","last_became_peered":"2024-04-25T13:45:30.699936+0000","last_unstale":"2024-04-25T13:45:31.680072+0000","last_undegraded":"2024-04-25T13:45:31.680072+0000","last_fullsized":"2024-04-25T13:45:31.680072+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.748580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082974+0000","last_change":"2024-04-25T13:45:30.695984+0000","last_active":"2024-04-25T13:45:32.082974+0000","last_peered":"2024-04-25T13:45:32.082974+0000","last_clean":"2024-04-25T13:45:32.082974+0000","last_became_active":"2024-04-25T13:45:30.695237+0000","last_became_peered":"2024-04-25T13:45:30.695237+0000","last_unstale":"2024-04-25T13:45:32.082974+0000","last_undegraded":"2024-04-25T13:45:32.082974+0000","last_fullsized":"2024-04-25T13:45:32.082974+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:07.521157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677622+0000","last_change":"2024-04-25T13:45:30.696311+0000","last_active":"2024-04-25T13:45:31.677622+0000","last_peered":"2024-04-25T13:45:31.677622+0000","last_clean":"2024-04-25T13:45:31.677622+0000","last_became_active":"2024-04-25T13:45:30.693211+0000","last_became_peered":"2024-04-25T13:45:30.693211+0000","last_unstale":"2024-04-25T13:45:31.677622+0000","last_undegraded":"2024-04-25T13:45:31.677622+0000","last_fullsized":"2024-04-25T13:45:31.677622+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:56.913967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677342+0000","last_change":"2024-04-25T13:45:30.690523+0000","last_active":"2024-04-25T13:45:31.677342+0000","last_peered":"2024-04-25T13:45:31.677342+0000","last_clean":"2024-04-25T13:45:31.677342+0000","last_became_active":"2024-04-25T13:45:30.689476+0000","last_became_peered":"2024-04-25T13:45:30.689476+0000","last_unstale":"2024-04-25T13:45:31.677342+0000","last_undegraded":"2024-04-25T13:45:31.677342+0000","last_fullsized":"2024-04-25T13:45:31.677342+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:22.843557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674805+0000","last_change":"2024-04-25T13:45:30.681991+0000","last_active":"2024-04-25T13:45:31.674805+0000","last_peered":"2024-04-25T13:45:31.674805+0000","last_clean":"2024-04-25T13:45:31.674805+0000","last_became_active":"2024-04-25T13:45:30.680907+0000","last_became_peered":"2024-04-25T13:45:30.680907+0000","last_unstale":"2024-04-25T13:45:31.674805+0000","last_undegraded":"2024-04-25T13:45:31.674805+0000","last_fullsized":"2024-04-25T13:45:31.674805+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:59.836485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680407+0000","last_change":"2024-04-25T13:45:30.702740+0000","last_active":"2024-04-25T13:45:31.680407+0000","last_peered":"2024-04-25T13:45:31.680407+0000","last_clean":"2024-04-25T13:45:31.680407+0000","last_became_active":"2024-04-25T13:45:30.702130+0000","last_became_peered":"2024-04-25T13:45:30.702130+0000","last_unstale":"2024-04-25T13:45:31.680407+0000","last_undegraded":"2024-04-25T13:45:31.680407+0000","last_fullsized":"2024-04-25T13:45:31.680407+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:11.447092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677573+0000","last_change":"2024-04-25T13:45:30.701142+0000","last_active":"2024-04-25T13:45:31.677573+0000","last_peered":"2024-04-25T13:45:31.677573+0000","last_clean":"2024-04-25T13:45:31.677573+0000","last_became_active":"2024-04-25T13:45:30.694528+0000","last_became_peered":"2024-04-25T13:45:30.694528+0000","last_unstale":"2024-04-25T13:45:31.677573+0000","last_undegraded":"2024-04-25T13:45:31.677573+0000","last_fullsized":"2024-04-25T13:45:31.677573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:40.837981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678050+0000","last_change":"2024-04-25T13:45:30.703510+0000","last_active":"2024-04-25T13:45:31.678050+0000","last_peered":"2024-04-25T13:45:31.678050+0000","last_clean":"2024-04-25T13:45:31.678050+0000","last_became_active":"2024-04-25T13:45:30.703136+0000","last_became_peered":"2024-04-25T13:45:30.703136+0000","last_unstale":"2024-04-25T13:45:31.678050+0000","last_undegraded":"2024-04-25T13:45:31.678050+0000","last_fullsized":"2024-04-25T13:45:31.678050+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:59.671556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673758+0000","last_change":"2024-04-25T13:45:30.669877+0000","last_active":"2024-04-25T13:45:31.673758+0000","last_peered":"2024-04-25T13:45:31.673758+0000","last_clean":"2024-04-25T13:45:31.673758+0000","last_became_active":"2024-04-25T13:45:30.668924+0000","last_became_peered":"2024-04-25T13:45:30.668924+0000","last_unstale":"2024-04-25T13:45:31.673758+0000","last_undegraded":"2024-04-25T13:45:31.673758+0000","last_fullsized":"2024-04-25T13:45:31.673758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:26.937469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678932+0000","last_change":"2024-04-25T13:45:30.705353+0000","last_active":"2024-04-25T13:45:31.678932+0000","last_peered":"2024-04-25T13:45:31.678932+0000","last_clean":"2024-04-25T13:45:31.678932+0000","last_became_active":"2024-04-25T13:45:30.704104+0000","last_became_peered":"2024-04-25T13:45:30.704104+0000","last_unstale":"2024-04-25T13:45:31.678932+0000","last_undegraded":"2024-04-25T13:45:31.678932+0000","last_fullsized":"2024-04-25T13:45:31.678932+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:11.200209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.672518+0000","last_change":"2024-04-25T13:45:30.698654+0000","last_active":"2024-04-25T13:45:31.672518+0000","last_peered":"2024-04-25T13:45:31.672518+0000","last_clean":"2024-04-25T13:45:31.672518+0000","last_became_active":"2024-04-25T13:45:30.697869+0000","last_became_peered":"2024-04-25T13:45:30.697869+0000","last_unstale":"2024-04-25T13:45:31.672518+0000","last_undegraded":"2024-04-25T13:45:31.672518+0000","last_fullsized":"2024-04-25T13:45:31.672518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:54.902916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680909+0000","last_change":"2024-04-25T13:45:29.651480+0000","last_active":"2024-04-25T13:45:31.680909+0000","last_peered":"2024-04-25T13:45:31.680909+0000","last_clean":"2024-04-25T13:45:31.680909+0000","last_became_active":"2024-04-25T13:45:29.648634+0000","last_became_peered":"2024-04-25T13:45:29.648634+0000","last_unstale":"2024-04-25T13:45:31.680909+0000","last_undegraded":"2024-04-25T13:45:31.680909+0000","last_fullsized":"2024-04-25T13:45:31.680909+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:08.126814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680745+0000","last_change":"2024-04-25T13:45:30.690629+0000","last_active":"2024-04-25T13:45:31.680745+0000","last_peered":"2024-04-25T13:45:31.680745+0000","last_clean":"2024-04-25T13:45:31.680745+0000","last_became_active":"2024-04-25T13:45:30.689897+0000","last_became_peered":"2024-04-25T13:45:30.689897+0000","last_unstale":"2024-04-25T13:45:31.680745+0000","last_undegraded":"2024-04-25T13:45:31.680745+0000","last_fullsized":"2024-04-25T13:45:31.680745+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:32.191945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673590+0000","last_change":"2024-04-25T13:45:29.655908+0000","last_active":"2024-04-25T13:45:31.673590+0000","last_peered":"2024-04-25T13:45:31.673590+0000","last_clean":"2024-04-25T13:45:31.673590+0000","last_became_active":"2024-04-25T13:45:29.654175+0000","last_became_peered":"2024-04-25T13:45:29.654175+0000","last_unstale":"2024-04-25T13:45:31.673590+0000","last_undegraded":"2024-04-25T13:45:31.673590+0000","last_fullsized":"2024-04-25T13:45:31.673590+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:50.178589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680126+0000","last_change":"2024-04-25T13:45:30.693131+0000","last_active":"2024-04-25T13:45:31.680126+0000","last_peered":"2024-04-25T13:45:31.680126+0000","last_clean":"2024-04-25T13:45:31.680126+0000","last_became_active":"2024-04-25T13:45:30.692346+0000","last_became_peered":"2024-04-25T13:45:30.692346+0000","last_unstale":"2024-04-25T13:45:31.680126+0000","last_undegraded":"2024-04-25T13:45:31.680126+0000","last_fullsized":"2024-04-25T13:45:31.680126+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:52.544084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673998+0000","last_change":"2024-04-25T13:45:29.644224+0000","last_active":"2024-04-25T13:45:31.673998+0000","last_peered":"2024-04-25T13:45:31.673998+0000","last_clean":"2024-04-25T13:45:31.673998+0000","last_became_active":"2024-04-25T13:45:29.643837+0000","last_became_peered":"2024-04-25T13:45:29.643837+0000","last_unstale":"2024-04-25T13:45:31.673998+0000","last_undegraded":"2024-04-25T13:45:31.673998+0000","last_fullsized":"2024-04-25T13:45:31.673998+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:25.953536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679152+0000","last_change":"2024-04-25T13:45:30.702919+0000","last_active":"2024-04-25T13:45:31.679152+0000","last_peered":"2024-04-25T13:45:31.679152+0000","last_clean":"2024-04-25T13:45:31.679152+0000","last_became_active":"2024-04-25T13:45:30.701546+0000","last_became_peered":"2024-04-25T13:45:30.701546+0000","last_unstale":"2024-04-25T13:45:31.679152+0000","last_undegraded":"2024-04-25T13:45:31.679152+0000","last_fullsized":"2024-04-25T13:45:31.679152+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:09.751236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673969+0000","last_change":"2024-04-25T13:45:29.649695+0000","last_active":"2024-04-25T13:45:31.673969+0000","last_peered":"2024-04-25T13:45:31.673969+0000","last_clean":"2024-04-25T13:45:31.673969+0000","last_became_active":"2024-04-25T13:45:29.647943+0000","last_became_peered":"2024-04-25T13:45:29.647943+0000","last_unstale":"2024-04-25T13:45:31.673969+0000","last_undegraded":"2024-04-25T13:45:31.673969+0000","last_fullsized":"2024-04-25T13:45:31.673969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:00.051313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677763+0000","last_change":"2024-04-25T13:45:30.697727+0000","last_active":"2024-04-25T13:45:31.677763+0000","last_peered":"2024-04-25T13:45:31.677763+0000","last_clean":"2024-04-25T13:45:31.677763+0000","last_became_active":"2024-04-25T13:45:30.694967+0000","last_became_peered":"2024-04-25T13:45:30.694967+0000","last_unstale":"2024-04-25T13:45:31.677763+0000","last_undegraded":"2024-04-25T13:45:31.677763+0000","last_fullsized":"2024-04-25T13:45:31.677763+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:10.381166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674206+0000","last_change":"2024-04-25T13:45:29.657837+0000","last_active":"2024-04-25T13:45:31.674206+0000","last_peered":"2024-04-25T13:45:31.674206+0000","last_clean":"2024-04-25T13:45:31.674206+0000","last_became_active":"2024-04-25T13:45:29.655285+0000","last_became_peered":"2024-04-25T13:45:29.655285+0000","last_unstale":"2024-04-25T13:45:31.674206+0000","last_undegraded":"2024-04-25T13:45:31.674206+0000","last_fullsized":"2024-04-25T13:45:31.674206+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:07.291221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673640+0000","last_change":"2024-04-25T13:45:30.696311+0000","last_active":"2024-04-25T13:45:31.673640+0000","last_peered":"2024-04-25T13:45:31.673640+0000","last_clean":"2024-04-25T13:45:31.673640+0000","last_became_active":"2024-04-25T13:45:30.693804+0000","last_became_peered":"2024-04-25T13:45:30.693804+0000","last_unstale":"2024-04-25T13:45:31.673640+0000","last_undegraded":"2024-04-25T13:45:31.673640+0000","last_fullsized":"2024-04-25T13:45:31.673640+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:11.191176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.675271+0000","last_change":"2024-04-25T13:45:29.650556+0000","last_active":"2024-04-25T13:45:31.675271+0000","last_peered":"2024-04-25T13:45:31.675271+0000","last_clean":"2024-04-25T13:45:31.675271+0000","last_became_active":"2024-04-25T13:45:29.648785+0000","last_became_peered":"2024-04-25T13:45:29.648785+0000","last_unstale":"2024-04-25T13:45:31.675271+0000","last_undegraded":"2024-04-25T13:45:31.675271+0000","last_fullsized":"2024-04-25T13:45:31.675271+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:16.820752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677305+0000","last_change":"2024-04-25T13:45:30.697016+0000","last_active":"2024-04-25T13:45:31.677305+0000","last_peered":"2024-04-25T13:45:31.677305+0000","last_clean":"2024-04-25T13:45:31.677305+0000","last_became_active":"2024-04-25T13:45:30.695606+0000","last_became_peered":"2024-04-25T13:45:30.695606+0000","last_unstale":"2024-04-25T13:45:31.677305+0000","last_undegraded":"2024-04-25T13:45:31.677305+0000","last_fullsized":"2024-04-25T13:45:31.677305+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:10.706220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674053+0000","last_change":"2024-04-25T13:45:29.647907+0000","last_active":"2024-04-25T13:45:31.674053+0000","last_peered":"2024-04-25T13:45:31.674053+0000","last_clean":"2024-04-25T13:45:31.674053+0000","last_became_active":"2024-04-25T13:45:29.643831+0000","last_became_peered":"2024-04-25T13:45:29.643831+0000","last_unstale":"2024-04-25T13:45:31.674053+0000","last_undegraded":"2024-04-25T13:45:31.674053+0000","last_fullsized":"2024-04-25T13:45:31.674053+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:04.554430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678729+0000","last_change":"2024-04-25T13:45:30.686896+0000","last_active":"2024-04-25T13:45:31.678729+0000","last_peered":"2024-04-25T13:45:31.678729+0000","last_clean":"2024-04-25T13:45:31.678729+0000","last_became_active":"2024-04-25T13:45:30.685147+0000","last_became_peered":"2024-04-25T13:45:30.685147+0000","last_unstale":"2024-04-25T13:45:31.678729+0000","last_undegraded":"2024-04-25T13:45:31.678729+0000","last_fullsized":"2024-04-25T13:45:31.678729+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:40.404306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678321+0000","last_change":"2024-04-25T13:45:29.654859+0000","last_active":"2024-04-25T13:45:31.678321+0000","last_peered":"2024-04-25T13:45:31.678321+0000","last_clean":"2024-04-25T13:45:31.678321+0000","last_became_active":"2024-04-25T13:45:29.654627+0000","last_became_peered":"2024-04-25T13:45:29.654627+0000","last_unstale":"2024-04-25T13:45:31.678321+0000","last_undegraded":"2024-04-25T13:45:31.678321+0000","last_fullsized":"2024-04-25T13:45:31.678321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:39.921074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677891+0000","last_change":"2024-04-25T13:45:30.690268+0000","last_active":"2024-04-25T13:45:31.677891+0000","last_peered":"2024-04-25T13:45:31.677891+0000","last_clean":"2024-04-25T13:45:31.677891+0000","last_became_active":"2024-04-25T13:45:30.688767+0000","last_became_peered":"2024-04-25T13:45:30.688767+0000","last_unstale":"2024-04-25T13:45:31.677891+0000","last_undegraded":"2024-04-25T13:45:31.677891+0000","last_fullsized":"2024-04-25T13:45:31.677891+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.407223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680462+0000","last_change":"2024-04-25T13:45:29.637734+0000","last_active":"2024-04-25T13:45:31.680462+0000","last_peered":"2024-04-25T13:45:31.680462+0000","last_clean":"2024-04-25T13:45:31.680462+0000","last_became_active":"2024-04-25T13:45:29.637008+0000","last_became_peered":"2024-04-25T13:45:29.637008+0000","last_unstale":"2024-04-25T13:45:31.680462+0000","last_undegraded":"2024-04-25T13:45:31.680462+0000","last_fullsized":"2024-04-25T13:45:31.680462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:46.023590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678968+0000","last_change":"2024-04-25T13:45:30.688496+0000","last_active":"2024-04-25T13:45:31.678968+0000","last_peered":"2024-04-25T13:45:31.678968+0000","last_clean":"2024-04-25T13:45:31.678968+0000","last_became_active":"2024-04-25T13:45:30.687435+0000","last_became_peered":"2024-04-25T13:45:30.687435+0000","last_unstale":"2024-04-25T13:45:31.678968+0000","last_undegraded":"2024-04-25T13:45:31.678968+0000","last_fullsized":"2024-04-25T13:45:31.678968+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:35.944250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679220+0000","last_change":"2024-04-25T13:45:29.658342+0000","last_active":"2024-04-25T13:45:31.679220+0000","last_peered":"2024-04-25T13:45:31.679220+0000","last_clean":"2024-04-25T13:45:31.679220+0000","last_became_active":"2024-04-25T13:45:29.657471+0000","last_became_peered":"2024-04-25T13:45:29.657471+0000","last_unstale":"2024-04-25T13:45:31.679220+0000","last_undegraded":"2024-04-25T13:45:31.679220+0000","last_fullsized":"2024-04-25T13:45:31.679220+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:52.005814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673402+0000","last_change":"2024-04-25T13:45:30.697544+0000","last_active":"2024-04-25T13:45:31.673402+0000","last_peered":"2024-04-25T13:45:31.673402+0000","last_clean":"2024-04-25T13:45:31.673402+0000","last_became_active":"2024-04-25T13:45:30.695390+0000","last_became_peered":"2024-04-25T13:45:30.695390+0000","last_unstale":"2024-04-25T13:45:31.673402+0000","last_undegraded":"2024-04-25T13:45:31.673402+0000","last_fullsized":"2024-04-25T13:45:31.673402+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:13.576086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.681274+0000","last_change":"2024-04-25T13:45:29.650779+0000","last_active":"2024-04-25T13:45:31.681274+0000","last_peered":"2024-04-25T13:45:31.681274+0000","last_clean":"2024-04-25T13:45:31.681274+0000","last_became_active":"2024-04-25T13:45:29.647654+0000","last_became_peered":"2024-04-25T13:45:29.647654+0000","last_unstale":"2024-04-25T13:45:31.681274+0000","last_undegraded":"2024-04-25T13:45:31.681274+0000","last_fullsized":"2024-04-25T13:45:31.681274+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:20.529136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674188+0000","last_change":"2024-04-25T13:45:30.669004+0000","last_active":"2024-04-25T13:45:31.674188+0000","last_peered":"2024-04-25T13:45:31.674188+0000","last_clean":"2024-04-25T13:45:31.674188+0000","last_became_active":"2024-04-25T13:45:30.667971+0000","last_became_peered":"2024-04-25T13:45:30.667971+0000","last_unstale":"2024-04-25T13:45:31.674188+0000","last_undegraded":"2024-04-25T13:45:31.674188+0000","last_fullsized":"2024-04-25T13:45:31.674188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:12.150857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677457+0000","last_change":"2024-04-25T13:45:29.644819+0000","last_active":"2024-04-25T13:45:31.677457+0000","last_peered":"2024-04-25T13:45:31.677457+0000","last_clean":"2024-04-25T13:45:31.677457+0000","last_became_active":"2024-04-25T13:45:29.644062+0000","last_became_peered":"2024-04-25T13:45:29.644062+0000","last_unstale":"2024-04-25T13:45:31.677457+0000","last_undegraded":"2024-04-25T13:45:31.677457+0000","last_fullsized":"2024-04-25T13:45:31.677457+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:13.936553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673234+0000","last_change":"2024-04-25T13:45:30.671342+0000","last_active":"2024-04-25T13:45:31.673234+0000","last_peered":"2024-04-25T13:45:31.673234+0000","last_clean":"2024-04-25T13:45:31.673234+0000","last_became_active":"2024-04-25T13:45:30.670419+0000","last_became_peered":"2024-04-25T13:45:30.670419+0000","last_unstale":"2024-04-25T13:45:31.673234+0000","last_undegraded":"2024-04-25T13:45:31.673234+0000","last_fullsized":"2024-04-25T13:45:31.673234+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.530084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678613+0000","last_change":"2024-04-25T13:45:29.632367+0000","last_active":"2024-04-25T13:45:31.678613+0000","last_peered":"2024-04-25T13:45:31.678613+0000","last_clean":"2024-04-25T13:45:31.678613+0000","last_became_active":"2024-04-25T13:45:29.632075+0000","last_became_peered":"2024-04-25T13:45:29.632075+0000","last_unstale":"2024-04-25T13:45:31.678613+0000","last_undegraded":"2024-04-25T13:45:31.678613+0000","last_fullsized":"2024-04-25T13:45:31.678613+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:21.139156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678910+0000","last_change":"2024-04-25T13:45:30.697671+0000","last_active":"2024-04-25T13:45:31.678910+0000","last_peered":"2024-04-25T13:45:31.678910+0000","last_clean":"2024-04-25T13:45:31.678910+0000","last_became_active":"2024-04-25T13:45:30.696487+0000","last_became_peered":"2024-04-25T13:45:30.696487+0000","last_unstale":"2024-04-25T13:45:31.678910+0000","last_undegraded":"2024-04-25T13:45:31.678910+0000","last_fullsized":"2024-04-25T13:45:31.678910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:12.924977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679952+0000","last_change":"2024-04-25T13:45:29.628699+0000","last_active":"2024-04-25T13:45:31.679952+0000","last_peered":"2024-04-25T13:45:31.679952+0000","last_clean":"2024-04-25T13:45:31.679952+0000","last_became_active":"2024-04-25T13:45:29.628403+0000","last_became_peered":"2024-04-25T13:45:29.628403+0000","last_unstale":"2024-04-25T13:45:31.679952+0000","last_undegraded":"2024-04-25T13:45:31.679952+0000","last_fullsized":"2024-04-25T13:45:31.679952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:33.811942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677397+0000","last_change":"2024-04-25T13:45:30.696857+0000","last_active":"2024-04-25T13:45:31.677397+0000","last_peered":"2024-04-25T13:45:31.677397+0000","last_clean":"2024-04-25T13:45:31.677397+0000","last_became_active":"2024-04-25T13:45:30.693590+0000","last_became_peered":"2024-04-25T13:45:30.693590+0000","last_unstale":"2024-04-25T13:45:31.677397+0000","last_undegraded":"2024-04-25T13:45:31.677397+0000","last_fullsized":"2024-04-25T13:45:31.677397+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:12.940703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678942+0000","last_change":"2024-04-25T13:45:29.657645+0000","last_active":"2024-04-25T13:45:31.678942+0000","last_peered":"2024-04-25T13:45:31.678942+0000","last_clean":"2024-04-25T13:45:31.678942+0000","last_became_active":"2024-04-25T13:45:29.656926+0000","last_became_peered":"2024-04-25T13:45:29.656926+0000","last_unstale":"2024-04-25T13:45:31.678942+0000","last_undegraded":"2024-04-25T13:45:31.678942+0000","last_fullsized":"2024-04-25T13:45:31.678942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:06.311229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679294+0000","last_change":"2024-04-25T13:45:30.705158+0000","last_active":"2024-04-25T13:45:31.679294+0000","last_peered":"2024-04-25T13:45:31.679294+0000","last_clean":"2024-04-25T13:45:31.679294+0000","last_became_active":"2024-04-25T13:45:30.693070+0000","last_became_peered":"2024-04-25T13:45:30.693070+0000","last_unstale":"2024-04-25T13:45:31.679294+0000","last_undegraded":"2024-04-25T13:45:31.679294+0000","last_fullsized":"2024-04-25T13:45:31.679294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:31.204287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673257+0000","last_change":"2024-04-25T13:45:29.643984+0000","last_active":"2024-04-25T13:45:31.673257+0000","last_peered":"2024-04-25T13:45:31.673257+0000","last_clean":"2024-04-25T13:45:31.673257+0000","last_became_active":"2024-04-25T13:45:29.643681+0000","last_became_peered":"2024-04-25T13:45:29.643681+0000","last_unstale":"2024-04-25T13:45:31.673257+0000","last_undegraded":"2024-04-25T13:45:31.673257+0000","last_fullsized":"2024-04-25T13:45:31.673257+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:13.782451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679394+0000","last_change":"2024-04-25T13:45:30.685429+0000","last_active":"2024-04-25T13:45:31.679394+0000","last_peered":"2024-04-25T13:45:31.679394+0000","last_clean":"2024-04-25T13:45:31.679394+0000","last_became_active":"2024-04-25T13:45:30.684572+0000","last_became_peered":"2024-04-25T13:45:30.684572+0000","last_unstale":"2024-04-25T13:45:31.679394+0000","last_undegraded":"2024-04-25T13:45:31.679394+0000","last_fullsized":"2024-04-25T13:45:31.679394+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:54.664690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678330+0000","last_change":"2024-04-25T13:45:29.649805+0000","last_active":"2024-04-25T13:45:31.678330+0000","last_peered":"2024-04-25T13:45:31.678330+0000","last_clean":"2024-04-25T13:45:31.678330+0000","last_became_active":"2024-04-25T13:45:29.647510+0000","last_became_peered":"2024-04-25T13:45:29.647510+0000","last_unstale":"2024-04-25T13:45:31.678330+0000","last_undegraded":"2024-04-25T13:45:31.678330+0000","last_fullsized":"2024-04-25T13:45:31.678330+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:10.935408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678162+0000","last_change":"2024-04-25T13:45:30.692116+0000","last_active":"2024-04-25T13:45:31.678162+0000","last_peered":"2024-04-25T13:45:31.678162+0000","last_clean":"2024-04-25T13:45:31.678162+0000","last_became_active":"2024-04-25T13:45:30.691130+0000","last_became_peered":"2024-04-25T13:45:30.691130+0000","last_unstale":"2024-04-25T13:45:31.678162+0000","last_undegraded":"2024-04-25T13:45:31.678162+0000","last_fullsized":"2024-04-25T13:45:31.678162+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:23.282257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674350+0000","last_change":"2024-04-25T13:45:29.655132+0000","last_active":"2024-04-25T13:45:31.674350+0000","last_peered":"2024-04-25T13:45:31.674350+0000","last_clean":"2024-04-25T13:45:31.674350+0000","last_became_active":"2024-04-25T13:45:29.654779+0000","last_became_peered":"2024-04-25T13:45:29.654779+0000","last_unstale":"2024-04-25T13:45:31.674350+0000","last_undegraded":"2024-04-25T13:45:31.674350+0000","last_fullsized":"2024-04-25T13:45:31.674350+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:13.766677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674113+0000","last_change":"2024-04-25T13:45:30.688391+0000","last_active":"2024-04-25T13:45:31.674113+0000","last_peered":"2024-04-25T13:45:31.674113+0000","last_clean":"2024-04-25T13:45:31.674113+0000","last_became_active":"2024-04-25T13:45:30.686897+0000","last_became_peered":"2024-04-25T13:45:30.686897+0000","last_unstale":"2024-04-25T13:45:31.674113+0000","last_undegraded":"2024-04-25T13:45:31.674113+0000","last_fullsized":"2024-04-25T13:45:31.674113+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:07.607757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674010+0000","last_change":"2024-04-25T13:45:29.640247+0000","last_active":"2024-04-25T13:45:31.674010+0000","last_peered":"2024-04-25T13:45:31.674010+0000","last_clean":"2024-04-25T13:45:31.674010+0000","last_became_active":"2024-04-25T13:45:29.639918+0000","last_became_peered":"2024-04-25T13:45:29.639918+0000","last_unstale":"2024-04-25T13:45:31.674010+0000","last_undegraded":"2024-04-25T13:45:31.674010+0000","last_fullsized":"2024-04-25T13:45:31.674010+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:40.653372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083213+0000","last_change":"2024-04-25T13:45:30.700435+0000","last_active":"2024-04-25T13:45:32.083213+0000","last_peered":"2024-04-25T13:45:32.083213+0000","last_clean":"2024-04-25T13:45:32.083213+0000","last_became_active":"2024-04-25T13:45:30.700040+0000","last_became_peered":"2024-04-25T13:45:30.700040+0000","last_unstale":"2024-04-25T13:45:32.083213+0000","last_undegraded":"2024-04-25T13:45:32.083213+0000","last_fullsized":"2024-04-25T13:45:32.083213+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:08.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679578+0000","last_change":"2024-04-25T13:45:29.642203+0000","last_active":"2024-04-25T13:45:31.679578+0000","last_peered":"2024-04-25T13:45:31.679578+0000","last_clean":"2024-04-25T13:45:31.679578+0000","last_became_active":"2024-04-25T13:45:29.639686+0000","last_became_peered":"2024-04-25T13:45:29.639686+0000","last_unstale":"2024-04-25T13:45:31.679578+0000","last_undegraded":"2024-04-25T13:45:31.679578+0000","last_fullsized":"2024-04-25T13:45:31.679578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:39.330999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677812+0000","last_change":"2024-04-25T13:45:30.688919+0000","last_active":"2024-04-25T13:45:31.677812+0000","last_peered":"2024-04-25T13:45:31.677812+0000","last_clean":"2024-04-25T13:45:31.677812+0000","last_became_active":"2024-04-25T13:45:30.687551+0000","last_became_peered":"2024-04-25T13:45:30.687551+0000","last_unstale":"2024-04-25T13:45:31.677812+0000","last_undegraded":"2024-04-25T13:45:31.677812+0000","last_fullsized":"2024-04-25T13:45:31.677812+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.268012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083400+0000","last_change":"2024-04-25T13:45:29.649230+0000","last_active":"2024-04-25T13:45:32.083400+0000","last_peered":"2024-04-25T13:45:32.083400+0000","last_clean":"2024-04-25T13:45:32.083400+0000","last_became_active":"2024-04-25T13:45:29.648967+0000","last_became_peered":"2024-04-25T13:45:29.648967+0000","last_unstale":"2024-04-25T13:45:32.083400+0000","last_undegraded":"2024-04-25T13:45:32.083400+0000","last_fullsized":"2024-04-25T13:45:32.083400+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:31.820121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083341+0000","last_change":"2024-04-25T13:45:30.693729+0000","last_active":"2024-04-25T13:45:32.083341+0000","last_peered":"2024-04-25T13:45:32.083341+0000","last_clean":"2024-04-25T13:45:32.083341+0000","last_became_active":"2024-04-25T13:45:30.686562+0000","last_became_peered":"2024-04-25T13:45:30.686562+0000","last_unstale":"2024-04-25T13:45:32.083341+0000","last_undegraded":"2024-04-25T13:45:32.083341+0000","last_fullsized":"2024-04-25T13:45:32.083341+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:53.056414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083922+0000","last_change":"2024-04-25T13:45:29.645317+0000","last_active":"2024-04-25T13:45:32.083922+0000","last_peered":"2024-04-25T13:45:32.083922+0000","last_clean":"2024-04-25T13:45:32.083922+0000","last_became_active":"2024-04-25T13:45:29.644926+0000","last_became_peered":"2024-04-25T13:45:29.644926+0000","last_unstale":"2024-04-25T13:45:32.083922+0000","last_undegraded":"2024-04-25T13:45:32.083922+0000","last_fullsized":"2024-04-25T13:45:32.083922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:57.383144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678098+0000","last_change":"2024-04-25T13:45:30.683001+0000","last_active":"2024-04-25T13:45:31.678098+0000","last_peered":"2024-04-25T13:45:31.678098+0000","last_clean":"2024-04-25T13:45:31.678098+0000","last_became_active":"2024-04-25T13:45:30.681093+0000","last_became_peered":"2024-04-25T13:45:30.681093+0000","last_unstale":"2024-04-25T13:45:31.678098+0000","last_undegraded":"2024-04-25T13:45:31.678098+0000","last_fullsized":"2024-04-25T13:45:31.678098+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:10.170509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680127+0000","last_change":"2024-04-25T13:45:29.641572+0000","last_active":"2024-04-25T13:45:31.680127+0000","last_peered":"2024-04-25T13:45:31.680127+0000","last_clean":"2024-04-25T13:45:31.680127+0000","last_became_active":"2024-04-25T13:45:29.641234+0000","last_became_peered":"2024-04-25T13:45:29.641234+0000","last_unstale":"2024-04-25T13:45:31.680127+0000","last_undegraded":"2024-04-25T13:45:31.680127+0000","last_fullsized":"2024-04-25T13:45:31.680127+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.198681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674865+0000","last_change":"2024-04-25T13:45:30.682389+0000","last_active":"2024-04-25T13:45:31.674865+0000","last_peered":"2024-04-25T13:45:31.674865+0000","last_clean":"2024-04-25T13:45:31.674865+0000","last_became_active":"2024-04-25T13:45:30.681572+0000","last_became_peered":"2024-04-25T13:45:30.681572+0000","last_unstale":"2024-04-25T13:45:31.674865+0000","last_undegraded":"2024-04-25T13:45:31.674865+0000","last_fullsized":"2024-04-25T13:45:31.674865+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:37.659326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679922+0000","last_change":"2024-04-25T13:45:29.638340+0000","last_active":"2024-04-25T13:45:31.679922+0000","last_peered":"2024-04-25T13:45:31.679922+0000","last_clean":"2024-04-25T13:45:31.679922+0000","last_became_active":"2024-04-25T13:45:29.637980+0000","last_became_peered":"2024-04-25T13:45:29.637980+0000","last_unstale":"2024-04-25T13:45:31.679922+0000","last_undegraded":"2024-04-25T13:45:31.679922+0000","last_fullsized":"2024-04-25T13:45:31.679922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:52.934013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679941+0000","last_change":"2024-04-25T13:45:30.682364+0000","last_active":"2024-04-25T13:45:31.679941+0000","last_peered":"2024-04-25T13:45:31.679941+0000","last_clean":"2024-04-25T13:45:31.679941+0000","last_became_active":"2024-04-25T13:45:30.681621+0000","last_became_peered":"2024-04-25T13:45:30.681621+0000","last_unstale":"2024-04-25T13:45:31.679941+0000","last_undegraded":"2024-04-25T13:45:31.679941+0000","last_fullsized":"2024-04-25T13:45:31.679941+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:20.503554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.706155+0000","last_change":"2024-04-25T13:45:29.630611+0000","last_active":"2024-04-25T13:45:31.706155+0000","last_peered":"2024-04-25T13:45:31.706155+0000","last_clean":"2024-04-25T13:45:31.706155+0000","last_became_active":"2024-04-25T13:45:29.630291+0000","last_became_peered":"2024-04-25T13:45:29.630291+0000","last_unstale":"2024-04-25T13:45:31.706155+0000","last_undegraded":"2024-04-25T13:45:31.706155+0000","last_fullsized":"2024-04-25T13:45:31.706155+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:35.420073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680056+0000","last_change":"2024-04-25T13:45:30.694116+0000","last_active":"2024-04-25T13:45:31.680056+0000","last_peered":"2024-04-25T13:45:31.680056+0000","last_clean":"2024-04-25T13:45:31.680056+0000","last_became_active":"2024-04-25T13:45:30.692877+0000","last_became_peered":"2024-04-25T13:45:30.692877+0000","last_unstale":"2024-04-25T13:45:31.680056+0000","last_undegraded":"2024-04-25T13:45:31.680056+0000","last_fullsized":"2024-04-25T13:45:31.680056+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:59.092680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678472+0000","last_change":"2024-04-25T13:45:29.648292+0000","last_active":"2024-04-25T13:45:31.678472+0000","last_peered":"2024-04-25T13:45:31.678472+0000","last_clean":"2024-04-25T13:45:31.678472+0000","last_became_active":"2024-04-25T13:45:29.646541+0000","last_became_peered":"2024-04-25T13:45:29.646541+0000","last_unstale":"2024-04-25T13:45:31.678472+0000","last_undegraded":"2024-04-25T13:45:31.678472+0000","last_fullsized":"2024-04-25T13:45:31.678472+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:11.823282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673210+0000","last_change":"2024-04-25T13:45:30.668538+0000","last_active":"2024-04-25T13:45:31.673210+0000","last_peered":"2024-04-25T13:45:31.673210+0000","last_clean":"2024-04-25T13:45:31.673210+0000","last_became_active":"2024-04-25T13:45:30.667706+0000","last_became_peered":"2024-04-25T13:45:30.667706+0000","last_unstale":"2024-04-25T13:45:31.673210+0000","last_undegraded":"2024-04-25T13:45:31.673210+0000","last_fullsized":"2024-04-25T13:45:31.673210+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:47.886928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680283+0000","last_change":"2024-04-25T13:45:29.644241+0000","last_active":"2024-04-25T13:45:31.680283+0000","last_peered":"2024-04-25T13:45:31.680283+0000","last_clean":"2024-04-25T13:45:31.680283+0000","last_became_active":"2024-04-25T13:45:29.642947+0000","last_became_peered":"2024-04-25T13:45:29.642947+0000","last_unstale":"2024-04-25T13:45:31.680283+0000","last_undegraded":"2024-04-25T13:45:31.680283+0000","last_fullsized":"2024-04-25T13:45:31.680283+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:49.524576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678443+0000","last_change":"2024-04-25T13:45:30.695035+0000","last_active":"2024-04-25T13:45:31.678443+0000","last_peered":"2024-04-25T13:45:31.678443+0000","last_clean":"2024-04-25T13:45:31.678443+0000","last_became_active":"2024-04-25T13:45:30.693490+0000","last_became_peered":"2024-04-25T13:45:30.693490+0000","last_unstale":"2024-04-25T13:45:31.678443+0000","last_undegraded":"2024-04-25T13:45:31.678443+0000","last_fullsized":"2024-04-25T13:45:31.678443+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:11.419064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674153+0000","last_change":"2024-04-25T13:45:29.650026+0000","last_active":"2024-04-25T13:45:31.674153+0000","last_peered":"2024-04-25T13:45:31.674153+0000","last_clean":"2024-04-25T13:45:31.674153+0000","last_became_active":"2024-04-25T13:45:29.648521+0000","last_became_peered":"2024-04-25T13:45:29.648521+0000","last_unstale":"2024-04-25T13:45:31.674153+0000","last_undegraded":"2024-04-25T13:45:31.674153+0000","last_fullsized":"2024-04-25T13:45:31.674153+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:19.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673676+0000","last_change":"2024-04-25T13:45:30.698784+0000","last_active":"2024-04-25T13:45:31.673676+0000","last_peered":"2024-04-25T13:45:31.673676+0000","last_clean":"2024-04-25T13:45:31.673676+0000","last_became_active":"2024-04-25T13:45:30.697750+0000","last_became_peered":"2024-04-25T13:45:30.697750+0000","last_unstale":"2024-04-25T13:45:31.673676+0000","last_undegraded":"2024-04-25T13:45:31.673676+0000","last_fullsized":"2024-04-25T13:45:31.673676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:43.751783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.675141+0000","last_change":"2024-04-25T13:45:29.656792+0000","last_active":"2024-04-25T13:45:31.675141+0000","last_peered":"2024-04-25T13:45:31.675141+0000","last_clean":"2024-04-25T13:45:31.675141+0000","last_became_active":"2024-04-25T13:45:29.654359+0000","last_became_peered":"2024-04-25T13:45:29.654359+0000","last_unstale":"2024-04-25T13:45:31.675141+0000","last_undegraded":"2024-04-25T13:45:31.675141+0000","last_fullsized":"2024-04-25T13:45:31.675141+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:23.975386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677341+0000","last_change":"2024-04-25T13:45:30.688245+0000","last_active":"2024-04-25T13:45:31.677341+0000","last_peered":"2024-04-25T13:45:31.677341+0000","last_clean":"2024-04-25T13:45:31.677341+0000","last_became_active":"2024-04-25T13:45:30.687211+0000","last_became_peered":"2024-04-25T13:45:30.687211+0000","last_unstale":"2024-04-25T13:45:31.677341+0000","last_undegraded":"2024-04-25T13:45:31.677341+0000","last_fullsized":"2024-04-25T13:45:31.677341+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:06.219560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.681413+0000","last_change":"2024-04-25T13:45:29.649893+0000","last_active":"2024-04-25T13:45:31.681413+0000","last_peered":"2024-04-25T13:45:31.681413+0000","last_clean":"2024-04-25T13:45:31.681413+0000","last_became_active":"2024-04-25T13:45:29.644888+0000","last_became_peered":"2024-04-25T13:45:29.644888+0000","last_unstale":"2024-04-25T13:45:31.681413+0000","last_undegraded":"2024-04-25T13:45:31.681413+0000","last_fullsized":"2024-04-25T13:45:31.681413+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:27.639484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.681124+0000","last_change":"2024-04-25T13:45:30.692692+0000","last_active":"2024-04-25T13:45:31.681124+0000","last_peered":"2024-04-25T13:45:31.681124+0000","last_clean":"2024-04-25T13:45:31.681124+0000","last_became_active":"2024-04-25T13:45:30.691903+0000","last_became_peered":"2024-04-25T13:45:30.691903+0000","last_unstale":"2024-04-25T13:45:31.681124+0000","last_undegraded":"2024-04-25T13:45:31.681124+0000","last_fullsized":"2024-04-25T13:45:31.681124+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:45.339239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673933+0000","last_change":"2024-04-25T13:45:29.654966+0000","last_active":"2024-04-25T13:45:31.673933+0000","last_peered":"2024-04-25T13:45:31.673933+0000","last_clean":"2024-04-25T13:45:31.673933+0000","last_became_active":"2024-04-25T13:45:29.653430+0000","last_became_peered":"2024-04-25T13:45:29.653430+0000","last_unstale":"2024-04-25T13:45:31.673933+0000","last_undegraded":"2024-04-25T13:45:31.673933+0000","last_fullsized":"2024-04-25T13:45:31.673933+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:45.655478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679623+0000","last_change":"2024-04-25T13:45:30.705782+0000","last_active":"2024-04-25T13:45:31.679623+0000","last_peered":"2024-04-25T13:45:31.679623+0000","last_clean":"2024-04-25T13:45:31.679623+0000","last_became_active":"2024-04-25T13:45:30.704147+0000","last_became_peered":"2024-04-25T13:45:30.704147+0000","last_unstale":"2024-04-25T13:45:31.679623+0000","last_undegraded":"2024-04-25T13:45:31.679623+0000","last_fullsized":"2024-04-25T13:45:31.679623+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:24.556348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679462+0000","last_change":"2024-04-25T13:45:29.629109+0000","last_active":"2024-04-25T13:45:31.679462+0000","last_peered":"2024-04-25T13:45:31.679462+0000","last_clean":"2024-04-25T13:45:31.679462+0000","last_became_active":"2024-04-25T13:45:29.628332+0000","last_became_peered":"2024-04-25T13:45:29.628332+0000","last_unstale":"2024-04-25T13:45:31.679462+0000","last_undegraded":"2024-04-25T13:45:31.679462+0000","last_fullsized":"2024-04-25T13:45:31.679462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:23.113730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":274,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677529+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:45:31.677529+0000","last_peered":"2024-04-25T13:45:31.677529+0000","last_clean":"2024-04-25T13:45:31.677529+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:45:31.677529+0000","last_undegraded":"2024-04-25T13:45:31.677529+0000","last_fullsized":"2024-04-25T13:45:31.677529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:11.594314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194187,"statfs":{"total":200000000,"available":198847936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":549,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194763,"statfs":{"total":200000000,"available":199437680,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":550,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194762,"statfs":{"total":200000000,"available":199436740,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199438374,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":550,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194762,"statfs":{"total":200000000,"available":199436762,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":549,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194763,"statfs":{"total":200000000,"available":199437792,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194763,"statfs":{"total":200000000,"available":199438180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194187,"statfs":{"total":200000000,"available":198847972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:45:34.801 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:45:34.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b3490640 1 Processor -- start 2024-04-25T13:45:34.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b3490640 1 -- start start 2024-04-25T13:45:34.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b3490640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38ac0d06e0 0x7f38ac06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b3490640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 0x7f38ac06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b3490640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38ac0d0af0 con 0x7f38ac161f90 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b3490640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f38ac0d0d70 con 0x7f38ac0d06e0 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b3490640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f38ac0d1050 con 0x7f38ac06e9e0 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b0a2b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38ac0d06e0 0x7f38ac06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b0a2b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38ac0d06e0 0x7f38ac06e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48930/0 (socket says 172.21.15.80:48930) 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b122c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f38ac161f90 0x7f38ac0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51444/0 (socket says 172.21.15.80:51444) 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b0a2b640 1 -- 172.21.15.80:0/1191077851 learned_addr learned my addr 172.21.15.80:0/1191077851 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b1a2d640 1 --2- 172.21.15.80:0/1191077851 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 0x7f38ac06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b0a2b640 1 -- 172.21.15.80:0/1191077851 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 msgr2=0x7f38ac06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b0a2b640 1 --2- 172.21.15.80:0/1191077851 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 0x7f38ac06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b0a2b640 1 -- 172.21.15.80:0/1191077851 >> v1:172.21.15.80:6789/0 conn(0x7f38ac161f90 legacy=0x7f38ac0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.987+0000 7f38b0a2b640 1 -- 172.21.15.80:0/1191077851 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac06dca0 con 0x7f38ac0d06e0 2024-04-25T13:45:34.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b0a2b640 1 --2- 172.21.15.80:0/1191077851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac0d06e0 0x7f38ac06e4a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f389c009870 tx=0x7f389c01c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6757e52ddcf853 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:34.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38a3fff640 1 -- 172.21.15.80:0/1191077851 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0b0070 con 0x7f38ac0d06e0 2024-04-25T13:45:34.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38a3fff640 1 -- 172.21.15.80:0/1191077851 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f389c0ab600 con 0x7f38ac0d06e0 2024-04-25T13:45:34.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38a3fff640 1 -- 172.21.15.80:0/1191077851 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f389c0ab9d0 con 0x7f38ac0d06e0 2024-04-25T13:45:34.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b3490640 1 -- 172.21.15.80:0/1191077851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac0d06e0 msgr2=0x7f38ac06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b3490640 1 --2- 172.21.15.80:0/1191077851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac0d06e0 0x7f38ac06e4a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f389c009870 tx=0x7f389c01c330 comp rx=0 tx=0).stop 2024-04-25T13:45:34.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b3490640 1 -- 172.21.15.80:0/1191077851 shutdown_connections 2024-04-25T13:45:34.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b3490640 1 --2- 172.21.15.80:0/1191077851 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 0x7f38ac06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b3490640 1 --2- 172.21.15.80:0/1191077851 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac0d06e0 0x7f38ac06e4a0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b3490640 1 -- 172.21.15.80:0/1191077851 >> 172.21.15.80:0/1191077851 conn(0x7f38ac06d080 msgr2=0x7f38ac0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:34.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b3490640 1 -- 172.21.15.80:0/1191077851 shutdown_connections 2024-04-25T13:45:34.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.991+0000 7f38b3490640 1 -- 172.21.15.80:0/1191077851 wait complete. 2024-04-25T13:45:34.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 Processor -- start 2024-04-25T13:45:34.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 -- start start 2024-04-25T13:45:34.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 0x7f38ac1650d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ac0d06e0 0x7f38ac145de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac161f90 0x7f38ac167020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac1447a0 con 0x7f38ac161f90 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f38ac144620 con 0x7f38ac0d06e0 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f38ac16b510 con 0x7f38ac06e9e0 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac161f90 0x7f38ac167020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac161f90 0x7f38ac167020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48944/0 (socket says 172.21.15.80:48944) 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 -- 172.21.15.80:0/1735530914 learned_addr learned my addr 172.21.15.80:0/1735530914 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 -- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 msgr2=0x7f38ac1650d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b122c640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 0x7f38ac1650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b0a2b640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ac0d06e0 0x7f38ac145de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 0x7f38ac1650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 -- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ac0d06e0 msgr2=0x7f38ac145de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ac0d06e0 0x7f38ac145de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 -- 172.21.15.80:0/1735530914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ac162ca0 con 0x7f38ac161f90 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b0a2b640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ac0d06e0 0x7f38ac145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b1a2d640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac161f90 0x7f38ac167020 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f38a4007d80 tx=0x7f38a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38a1ffb640 1 -- 172.21.15.80:0/1735530914 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4014420 con 0x7f38ac161f90 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38a1ffb640 1 -- 172.21.15.80:0/1735530914 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38a4019070 con 0x7f38ac161f90 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ac162f60 con 0x7f38ac161f90 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38a1ffb640 1 -- 172.21.15.80:0/1735530914 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38a4014800 con 0x7f38ac161f90 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.995+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ac163520 con 0x7f38ac161f90 2024-04-25T13:45:34.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.999+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ac165950 con 0x7f38ac161f90 2024-04-25T13:45:34.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.999+0000 7f38a1ffb640 1 -- 172.21.15.80:0/1735530914 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f38a40149a0 con 0x7f38ac161f90 2024-04-25T13:45:34.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.999+0000 7f38a1ffb640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f389006f400 0x7f3890071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:34.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.999+0000 7f38a1ffb640 1 -- 172.21.15.80:0/1735530914 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f38a408fc10 con 0x7f38ac161f90 2024-04-25T13:45:34.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.999+0000 7f38b122c640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f389006f400 0x7f3890071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:34.999+0000 7f38b122c640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f389006f400 0x7f3890071910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38a8000f70 tx=0x7f38a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:35.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.003+0000 7f38a1ffb640 1 -- 172.21.15.80:0/1735530914 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38a405ca60 con 0x7f38ac161f90 2024-04-25T13:45:35.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.155+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38ac06a3a0 con 0x7f389006f400 2024-04-25T13:45:35.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.175+0000 7f38a1ffb640 1 -- 172.21.15.80:0/1735530914 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247981 (secure 0 0 0) 0x7f38ac06a3a0 con 0x7f389006f400 2024-04-25T13:45:35.176 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:35.178 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f389006f400 msgr2=0x7f3890071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f389006f400 0x7f3890071910 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38a8000f70 tx=0x7f38a8009040 comp rx=0 tx=0).stop 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac161f90 msgr2=0x7f38ac167020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac161f90 0x7f38ac167020 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f38a4007d80 tx=0x7f38a400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 shutdown_connections 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f389006f400 0x7f3890071910 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38ac161f90 0x7f38ac167020 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38ac0d06e0 0x7f38ac145de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 --2- 172.21.15.80:0/1735530914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f38ac06e9e0 0x7f38ac1650d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 >> 172.21.15.80:0/1735530914 conn(0x7f38ac06d080 msgr2=0x7f38ac0b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:35.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 shutdown_connections 2024-04-25T13:45:35.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.179+0000 7f38b3490640 1 -- 172.21.15.80:0/1735530914 wait complete. 2024-04-25T13:45:35.196 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-04-25T13:45:34.206621+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556954,"statfs":{"total":1600000000,"available":1594321436,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.590620"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679635+0000","last_change":"2024-04-25T13:45:30.706530+0000","last_active":"2024-04-25T13:45:31.679635+0000","last_peered":"2024-04-25T13:45:31.679635+0000","last_clean":"2024-04-25T13:45:31.679635+0000","last_became_active":"2024-04-25T13:45:30.705786+0000","last_became_peered":"2024-04-25T13:45:30.705786+0000","last_unstale":"2024-04-25T13:45:31.679635+0000","last_undegraded":"2024-04-25T13:45:31.679635+0000","last_fullsized":"2024-04-25T13:45:31.679635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:37.514261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677415+0000","last_change":"2024-04-25T13:45:30.706054+0000","last_active":"2024-04-25T13:45:31.677415+0000","last_peered":"2024-04-25T13:45:31.677415+0000","last_clean":"2024-04-25T13:45:31.677415+0000","last_became_active":"2024-04-25T13:45:30.704852+0000","last_became_peered":"2024-04-25T13:45:30.704852+0000","last_unstale":"2024-04-25T13:45:31.677415+0000","last_undegraded":"2024-04-25T13:45:31.677415+0000","last_fullsized":"2024-04-25T13:45:31.677415+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:49.045396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678183+0000","last_change":"2024-04-25T13:45:30.691197+0000","last_active":"2024-04-25T13:45:31.678183+0000","last_peered":"2024-04-25T13:45:31.678183+0000","last_clean":"2024-04-25T13:45:31.678183+0000","last_became_active":"2024-04-25T13:45:30.690270+0000","last_became_peered":"2024-04-25T13:45:30.690270+0000","last_unstale":"2024-04-25T13:45:31.678183+0000","last_undegraded":"2024-04-25T13:45:31.678183+0000","last_fullsized":"2024-04-25T13:45:31.678183+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:58.509848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082426+0000","last_change":"2024-04-25T13:45:30.703626+0000","last_active":"2024-04-25T13:45:32.082426+0000","last_peered":"2024-04-25T13:45:32.082426+0000","last_clean":"2024-04-25T13:45:32.082426+0000","last_became_active":"2024-04-25T13:45:30.702976+0000","last_became_peered":"2024-04-25T13:45:30.702976+0000","last_unstale":"2024-04-25T13:45:32.082426+0000","last_undegraded":"2024-04-25T13:45:32.082426+0000","last_fullsized":"2024-04-25T13:45:32.082426+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:21.089644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673356+0000","last_change":"2024-04-25T13:45:30.693814+0000","last_active":"2024-04-25T13:45:31.673356+0000","last_peered":"2024-04-25T13:45:31.673356+0000","last_clean":"2024-04-25T13:45:31.673356+0000","last_became_active":"2024-04-25T13:45:30.692997+0000","last_became_peered":"2024-04-25T13:45:30.692997+0000","last_unstale":"2024-04-25T13:45:31.673356+0000","last_undegraded":"2024-04-25T13:45:31.673356+0000","last_fullsized":"2024-04-25T13:45:31.673356+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:18.356937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678531+0000","last_change":"2024-04-25T13:45:30.695054+0000","last_active":"2024-04-25T13:45:31.678531+0000","last_peered":"2024-04-25T13:45:31.678531+0000","last_clean":"2024-04-25T13:45:31.678531+0000","last_became_active":"2024-04-25T13:45:30.694118+0000","last_became_peered":"2024-04-25T13:45:30.694118+0000","last_unstale":"2024-04-25T13:45:31.678531+0000","last_undegraded":"2024-04-25T13:45:31.678531+0000","last_fullsized":"2024-04-25T13:45:31.678531+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:57.495389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082798+0000","last_change":"2024-04-25T13:45:30.687364+0000","last_active":"2024-04-25T13:45:32.082798+0000","last_peered":"2024-04-25T13:45:32.082798+0000","last_clean":"2024-04-25T13:45:32.082798+0000","last_became_active":"2024-04-25T13:45:30.686618+0000","last_became_peered":"2024-04-25T13:45:30.686618+0000","last_unstale":"2024-04-25T13:45:32.082798+0000","last_undegraded":"2024-04-25T13:45:32.082798+0000","last_fullsized":"2024-04-25T13:45:32.082798+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:20.048150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677353+0000","last_change":"2024-04-25T13:45:30.688301+0000","last_active":"2024-04-25T13:45:31.677353+0000","last_peered":"2024-04-25T13:45:31.677353+0000","last_clean":"2024-04-25T13:45:31.677353+0000","last_became_active":"2024-04-25T13:45:30.686603+0000","last_became_peered":"2024-04-25T13:45:30.686603+0000","last_unstale":"2024-04-25T13:45:31.677353+0000","last_undegraded":"2024-04-25T13:45:31.677353+0000","last_fullsized":"2024-04-25T13:45:31.677353+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:39.897707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677125+0000","last_change":"2024-04-25T13:45:30.685224+0000","last_active":"2024-04-25T13:45:31.677125+0000","last_peered":"2024-04-25T13:45:31.677125+0000","last_clean":"2024-04-25T13:45:31.677125+0000","last_became_active":"2024-04-25T13:45:30.684513+0000","last_became_peered":"2024-04-25T13:45:30.684513+0000","last_unstale":"2024-04-25T13:45:31.677125+0000","last_undegraded":"2024-04-25T13:45:31.677125+0000","last_fullsized":"2024-04-25T13:45:31.677125+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:13.327393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679635+0000","last_change":"2024-04-25T13:45:30.690502+0000","last_active":"2024-04-25T13:45:31.679635+0000","last_peered":"2024-04-25T13:45:31.679635+0000","last_clean":"2024-04-25T13:45:31.679635+0000","last_became_active":"2024-04-25T13:45:30.689683+0000","last_became_peered":"2024-04-25T13:45:30.689683+0000","last_unstale":"2024-04-25T13:45:31.679635+0000","last_undegraded":"2024-04-25T13:45:31.679635+0000","last_fullsized":"2024-04-25T13:45:31.679635+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:28:37.350256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680027+0000","last_change":"2024-04-25T13:45:30.692178+0000","last_active":"2024-04-25T13:45:31.680027+0000","last_peered":"2024-04-25T13:45:31.680027+0000","last_clean":"2024-04-25T13:45:31.680027+0000","last_became_active":"2024-04-25T13:45:30.686925+0000","last_became_peered":"2024-04-25T13:45:30.686925+0000","last_unstale":"2024-04-25T13:45:31.680027+0000","last_undegraded":"2024-04-25T13:45:31.680027+0000","last_fullsized":"2024-04-25T13:45:31.680027+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:54:52.080827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678492+0000","last_change":"2024-04-25T13:45:30.690575+0000","last_active":"2024-04-25T13:45:31.678492+0000","last_peered":"2024-04-25T13:45:31.678492+0000","last_clean":"2024-04-25T13:45:31.678492+0000","last_became_active":"2024-04-25T13:45:30.689761+0000","last_became_peered":"2024-04-25T13:45:30.689761+0000","last_unstale":"2024-04-25T13:45:31.678492+0000","last_undegraded":"2024-04-25T13:45:31.678492+0000","last_fullsized":"2024-04-25T13:45:31.678492+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:15.675006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678694+0000","last_change":"2024-04-25T13:45:30.702023+0000","last_active":"2024-04-25T13:45:31.678694+0000","last_peered":"2024-04-25T13:45:31.678694+0000","last_clean":"2024-04-25T13:45:31.678694+0000","last_became_active":"2024-04-25T13:45:30.701220+0000","last_became_peered":"2024-04-25T13:45:30.701220+0000","last_unstale":"2024-04-25T13:45:31.678694+0000","last_undegraded":"2024-04-25T13:45:31.678694+0000","last_fullsized":"2024-04-25T13:45:31.678694+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:32.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082722+0000","last_change":"2024-04-25T13:45:30.703325+0000","last_active":"2024-04-25T13:45:32.082722+0000","last_peered":"2024-04-25T13:45:32.082722+0000","last_clean":"2024-04-25T13:45:32.082722+0000","last_became_active":"2024-04-25T13:45:30.702178+0000","last_became_peered":"2024-04-25T13:45:30.702178+0000","last_unstale":"2024-04-25T13:45:32.082722+0000","last_undegraded":"2024-04-25T13:45:32.082722+0000","last_fullsized":"2024-04-25T13:45:32.082722+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:30.258376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678267+0000","last_change":"2024-04-25T13:45:30.687069+0000","last_active":"2024-04-25T13:45:31.678267+0000","last_peered":"2024-04-25T13:45:31.678267+0000","last_clean":"2024-04-25T13:45:31.678267+0000","last_became_active":"2024-04-25T13:45:30.686238+0000","last_became_peered":"2024-04-25T13:45:30.686238+0000","last_unstale":"2024-04-25T13:45:31.678267+0000","last_undegraded":"2024-04-25T13:45:31.678267+0000","last_fullsized":"2024-04-25T13:45:31.678267+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:47.772364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673327+0000","last_change":"2024-04-25T13:45:30.705163+0000","last_active":"2024-04-25T13:45:31.673327+0000","last_peered":"2024-04-25T13:45:31.673327+0000","last_clean":"2024-04-25T13:45:31.673327+0000","last_became_active":"2024-04-25T13:45:30.703622+0000","last_became_peered":"2024-04-25T13:45:30.703622+0000","last_unstale":"2024-04-25T13:45:31.673327+0000","last_undegraded":"2024-04-25T13:45:31.673327+0000","last_fullsized":"2024-04-25T13:45:31.673327+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:28.377812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673097+0000","last_change":"2024-04-25T13:45:30.696845+0000","last_active":"2024-04-25T13:45:31.673097+0000","last_peered":"2024-04-25T13:45:31.673097+0000","last_clean":"2024-04-25T13:45:31.673097+0000","last_became_active":"2024-04-25T13:45:30.695427+0000","last_became_peered":"2024-04-25T13:45:30.695427+0000","last_unstale":"2024-04-25T13:45:31.673097+0000","last_undegraded":"2024-04-25T13:45:31.673097+0000","last_fullsized":"2024-04-25T13:45:31.673097+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:42.226278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083405+0000","last_change":"2024-04-25T13:45:30.695731+0000","last_active":"2024-04-25T13:45:32.083405+0000","last_peered":"2024-04-25T13:45:32.083405+0000","last_clean":"2024-04-25T13:45:32.083405+0000","last_became_active":"2024-04-25T13:45:30.694534+0000","last_became_peered":"2024-04-25T13:45:30.694534+0000","last_unstale":"2024-04-25T13:45:32.083405+0000","last_undegraded":"2024-04-25T13:45:32.083405+0000","last_fullsized":"2024-04-25T13:45:32.083405+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:24.938460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.672981+0000","last_change":"2024-04-25T13:45:30.693893+0000","last_active":"2024-04-25T13:45:31.672981+0000","last_peered":"2024-04-25T13:45:31.672981+0000","last_clean":"2024-04-25T13:45:31.672981+0000","last_became_active":"2024-04-25T13:45:30.692463+0000","last_became_peered":"2024-04-25T13:45:30.692463+0000","last_unstale":"2024-04-25T13:45:31.672981+0000","last_undegraded":"2024-04-25T13:45:31.672981+0000","last_fullsized":"2024-04-25T13:45:31.672981+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:39.239491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083431+0000","last_change":"2024-04-25T13:45:30.699370+0000","last_active":"2024-04-25T13:45:32.083431+0000","last_peered":"2024-04-25T13:45:32.083431+0000","last_clean":"2024-04-25T13:45:32.083431+0000","last_became_active":"2024-04-25T13:45:30.689218+0000","last_became_peered":"2024-04-25T13:45:30.689218+0000","last_unstale":"2024-04-25T13:45:32.083431+0000","last_undegraded":"2024-04-25T13:45:32.083431+0000","last_fullsized":"2024-04-25T13:45:32.083431+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:35:02.832053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082888+0000","last_change":"2024-04-25T13:45:30.700740+0000","last_active":"2024-04-25T13:45:32.082888+0000","last_peered":"2024-04-25T13:45:32.082888+0000","last_clean":"2024-04-25T13:45:32.082888+0000","last_became_active":"2024-04-25T13:45:30.700202+0000","last_became_peered":"2024-04-25T13:45:30.700202+0000","last_unstale":"2024-04-25T13:45:32.082888+0000","last_undegraded":"2024-04-25T13:45:32.082888+0000","last_fullsized":"2024-04-25T13:45:32.082888+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:12.354500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679052+0000","last_change":"2024-04-25T13:45:30.705540+0000","last_active":"2024-04-25T13:45:31.679052+0000","last_peered":"2024-04-25T13:45:31.679052+0000","last_clean":"2024-04-25T13:45:31.679052+0000","last_became_active":"2024-04-25T13:45:30.698005+0000","last_became_peered":"2024-04-25T13:45:30.698005+0000","last_unstale":"2024-04-25T13:45:31.679052+0000","last_undegraded":"2024-04-25T13:45:31.679052+0000","last_fullsized":"2024-04-25T13:45:31.679052+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:10.359947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680072+0000","last_change":"2024-04-25T13:45:30.701179+0000","last_active":"2024-04-25T13:45:31.680072+0000","last_peered":"2024-04-25T13:45:31.680072+0000","last_clean":"2024-04-25T13:45:31.680072+0000","last_became_active":"2024-04-25T13:45:30.699936+0000","last_became_peered":"2024-04-25T13:45:30.699936+0000","last_unstale":"2024-04-25T13:45:31.680072+0000","last_undegraded":"2024-04-25T13:45:31.680072+0000","last_fullsized":"2024-04-25T13:45:31.680072+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:02.748580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.082974+0000","last_change":"2024-04-25T13:45:30.695984+0000","last_active":"2024-04-25T13:45:32.082974+0000","last_peered":"2024-04-25T13:45:32.082974+0000","last_clean":"2024-04-25T13:45:32.082974+0000","last_became_active":"2024-04-25T13:45:30.695237+0000","last_became_peered":"2024-04-25T13:45:30.695237+0000","last_unstale":"2024-04-25T13:45:32.082974+0000","last_undegraded":"2024-04-25T13:45:32.082974+0000","last_fullsized":"2024-04-25T13:45:32.082974+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:07.521157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677622+0000","last_change":"2024-04-25T13:45:30.696311+0000","last_active":"2024-04-25T13:45:31.677622+0000","last_peered":"2024-04-25T13:45:31.677622+0000","last_clean":"2024-04-25T13:45:31.677622+0000","last_became_active":"2024-04-25T13:45:30.693211+0000","last_became_peered":"2024-04-25T13:45:30.693211+0000","last_unstale":"2024-04-25T13:45:31.677622+0000","last_undegraded":"2024-04-25T13:45:31.677622+0000","last_fullsized":"2024-04-25T13:45:31.677622+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:56.913967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677342+0000","last_change":"2024-04-25T13:45:30.690523+0000","last_active":"2024-04-25T13:45:31.677342+0000","last_peered":"2024-04-25T13:45:31.677342+0000","last_clean":"2024-04-25T13:45:31.677342+0000","last_became_active":"2024-04-25T13:45:30.689476+0000","last_became_peered":"2024-04-25T13:45:30.689476+0000","last_unstale":"2024-04-25T13:45:31.677342+0000","last_undegraded":"2024-04-25T13:45:31.677342+0000","last_fullsized":"2024-04-25T13:45:31.677342+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:22.843557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674805+0000","last_change":"2024-04-25T13:45:30.681991+0000","last_active":"2024-04-25T13:45:31.674805+0000","last_peered":"2024-04-25T13:45:31.674805+0000","last_clean":"2024-04-25T13:45:31.674805+0000","last_became_active":"2024-04-25T13:45:30.680907+0000","last_became_peered":"2024-04-25T13:45:30.680907+0000","last_unstale":"2024-04-25T13:45:31.674805+0000","last_undegraded":"2024-04-25T13:45:31.674805+0000","last_fullsized":"2024-04-25T13:45:31.674805+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:50:59.836485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680407+0000","last_change":"2024-04-25T13:45:30.702740+0000","last_active":"2024-04-25T13:45:31.680407+0000","last_peered":"2024-04-25T13:45:31.680407+0000","last_clean":"2024-04-25T13:45:31.680407+0000","last_became_active":"2024-04-25T13:45:30.702130+0000","last_became_peered":"2024-04-25T13:45:30.702130+0000","last_unstale":"2024-04-25T13:45:31.680407+0000","last_undegraded":"2024-04-25T13:45:31.680407+0000","last_fullsized":"2024-04-25T13:45:31.680407+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:11.447092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677573+0000","last_change":"2024-04-25T13:45:30.701142+0000","last_active":"2024-04-25T13:45:31.677573+0000","last_peered":"2024-04-25T13:45:31.677573+0000","last_clean":"2024-04-25T13:45:31.677573+0000","last_became_active":"2024-04-25T13:45:30.694528+0000","last_became_peered":"2024-04-25T13:45:30.694528+0000","last_unstale":"2024-04-25T13:45:31.677573+0000","last_undegraded":"2024-04-25T13:45:31.677573+0000","last_fullsized":"2024-04-25T13:45:31.677573+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:40.837981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678050+0000","last_change":"2024-04-25T13:45:30.703510+0000","last_active":"2024-04-25T13:45:31.678050+0000","last_peered":"2024-04-25T13:45:31.678050+0000","last_clean":"2024-04-25T13:45:31.678050+0000","last_became_active":"2024-04-25T13:45:30.703136+0000","last_became_peered":"2024-04-25T13:45:30.703136+0000","last_unstale":"2024-04-25T13:45:31.678050+0000","last_undegraded":"2024-04-25T13:45:31.678050+0000","last_fullsized":"2024-04-25T13:45:31.678050+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:59.671556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673758+0000","last_change":"2024-04-25T13:45:30.669877+0000","last_active":"2024-04-25T13:45:31.673758+0000","last_peered":"2024-04-25T13:45:31.673758+0000","last_clean":"2024-04-25T13:45:31.673758+0000","last_became_active":"2024-04-25T13:45:30.668924+0000","last_became_peered":"2024-04-25T13:45:30.668924+0000","last_unstale":"2024-04-25T13:45:31.673758+0000","last_undegraded":"2024-04-25T13:45:31.673758+0000","last_fullsized":"2024-04-25T13:45:31.673758+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:07:26.937469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678932+0000","last_change":"2024-04-25T13:45:30.705353+0000","last_active":"2024-04-25T13:45:31.678932+0000","last_peered":"2024-04-25T13:45:31.678932+0000","last_clean":"2024-04-25T13:45:31.678932+0000","last_became_active":"2024-04-25T13:45:30.704104+0000","last_became_peered":"2024-04-25T13:45:30.704104+0000","last_unstale":"2024-04-25T13:45:31.678932+0000","last_undegraded":"2024-04-25T13:45:31.678932+0000","last_fullsized":"2024-04-25T13:45:31.678932+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:11.200209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.672518+0000","last_change":"2024-04-25T13:45:30.698654+0000","last_active":"2024-04-25T13:45:31.672518+0000","last_peered":"2024-04-25T13:45:31.672518+0000","last_clean":"2024-04-25T13:45:31.672518+0000","last_became_active":"2024-04-25T13:45:30.697869+0000","last_became_peered":"2024-04-25T13:45:30.697869+0000","last_unstale":"2024-04-25T13:45:31.672518+0000","last_undegraded":"2024-04-25T13:45:31.672518+0000","last_fullsized":"2024-04-25T13:45:31.672518+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:54.902916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"102'4","reported_seq":18,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680909+0000","last_change":"2024-04-25T13:45:29.651480+0000","last_active":"2024-04-25T13:45:31.680909+0000","last_peered":"2024-04-25T13:45:31.680909+0000","last_clean":"2024-04-25T13:45:31.680909+0000","last_became_active":"2024-04-25T13:45:29.648634+0000","last_became_peered":"2024-04-25T13:45:29.648634+0000","last_unstale":"2024-04-25T13:45:31.680909+0000","last_undegraded":"2024-04-25T13:45:31.680909+0000","last_fullsized":"2024-04-25T13:45:31.680909+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:08.126814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680745+0000","last_change":"2024-04-25T13:45:30.690629+0000","last_active":"2024-04-25T13:45:31.680745+0000","last_peered":"2024-04-25T13:45:31.680745+0000","last_clean":"2024-04-25T13:45:31.680745+0000","last_became_active":"2024-04-25T13:45:30.689897+0000","last_became_peered":"2024-04-25T13:45:30.689897+0000","last_unstale":"2024-04-25T13:45:31.680745+0000","last_undegraded":"2024-04-25T13:45:31.680745+0000","last_fullsized":"2024-04-25T13:45:31.680745+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:32.191945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673590+0000","last_change":"2024-04-25T13:45:29.655908+0000","last_active":"2024-04-25T13:45:31.673590+0000","last_peered":"2024-04-25T13:45:31.673590+0000","last_clean":"2024-04-25T13:45:31.673590+0000","last_became_active":"2024-04-25T13:45:29.654175+0000","last_became_peered":"2024-04-25T13:45:29.654175+0000","last_unstale":"2024-04-25T13:45:31.673590+0000","last_undegraded":"2024-04-25T13:45:31.673590+0000","last_fullsized":"2024-04-25T13:45:31.673590+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:50.178589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680126+0000","last_change":"2024-04-25T13:45:30.693131+0000","last_active":"2024-04-25T13:45:31.680126+0000","last_peered":"2024-04-25T13:45:31.680126+0000","last_clean":"2024-04-25T13:45:31.680126+0000","last_became_active":"2024-04-25T13:45:30.692346+0000","last_became_peered":"2024-04-25T13:45:30.692346+0000","last_unstale":"2024-04-25T13:45:31.680126+0000","last_undegraded":"2024-04-25T13:45:31.680126+0000","last_fullsized":"2024-04-25T13:45:31.680126+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:52.544084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673998+0000","last_change":"2024-04-25T13:45:29.644224+0000","last_active":"2024-04-25T13:45:31.673998+0000","last_peered":"2024-04-25T13:45:31.673998+0000","last_clean":"2024-04-25T13:45:31.673998+0000","last_became_active":"2024-04-25T13:45:29.643837+0000","last_became_peered":"2024-04-25T13:45:29.643837+0000","last_unstale":"2024-04-25T13:45:31.673998+0000","last_undegraded":"2024-04-25T13:45:31.673998+0000","last_fullsized":"2024-04-25T13:45:31.673998+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:25.953536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679152+0000","last_change":"2024-04-25T13:45:30.702919+0000","last_active":"2024-04-25T13:45:31.679152+0000","last_peered":"2024-04-25T13:45:31.679152+0000","last_clean":"2024-04-25T13:45:31.679152+0000","last_became_active":"2024-04-25T13:45:30.701546+0000","last_became_peered":"2024-04-25T13:45:30.701546+0000","last_unstale":"2024-04-25T13:45:31.679152+0000","last_undegraded":"2024-04-25T13:45:31.679152+0000","last_fullsized":"2024-04-25T13:45:31.679152+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:09.751236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673969+0000","last_change":"2024-04-25T13:45:29.649695+0000","last_active":"2024-04-25T13:45:31.673969+0000","last_peered":"2024-04-25T13:45:31.673969+0000","last_clean":"2024-04-25T13:45:31.673969+0000","last_became_active":"2024-04-25T13:45:29.647943+0000","last_became_peered":"2024-04-25T13:45:29.647943+0000","last_unstale":"2024-04-25T13:45:31.673969+0000","last_undegraded":"2024-04-25T13:45:31.673969+0000","last_fullsized":"2024-04-25T13:45:31.673969+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:00.051313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677763+0000","last_change":"2024-04-25T13:45:30.697727+0000","last_active":"2024-04-25T13:45:31.677763+0000","last_peered":"2024-04-25T13:45:31.677763+0000","last_clean":"2024-04-25T13:45:31.677763+0000","last_became_active":"2024-04-25T13:45:30.694967+0000","last_became_peered":"2024-04-25T13:45:30.694967+0000","last_unstale":"2024-04-25T13:45:31.677763+0000","last_undegraded":"2024-04-25T13:45:31.677763+0000","last_fullsized":"2024-04-25T13:45:31.677763+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:01:10.381166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674206+0000","last_change":"2024-04-25T13:45:29.657837+0000","last_active":"2024-04-25T13:45:31.674206+0000","last_peered":"2024-04-25T13:45:31.674206+0000","last_clean":"2024-04-25T13:45:31.674206+0000","last_became_active":"2024-04-25T13:45:29.655285+0000","last_became_peered":"2024-04-25T13:45:29.655285+0000","last_unstale":"2024-04-25T13:45:31.674206+0000","last_undegraded":"2024-04-25T13:45:31.674206+0000","last_fullsized":"2024-04-25T13:45:31.674206+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:07.291221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673640+0000","last_change":"2024-04-25T13:45:30.696311+0000","last_active":"2024-04-25T13:45:31.673640+0000","last_peered":"2024-04-25T13:45:31.673640+0000","last_clean":"2024-04-25T13:45:31.673640+0000","last_became_active":"2024-04-25T13:45:30.693804+0000","last_became_peered":"2024-04-25T13:45:30.693804+0000","last_unstale":"2024-04-25T13:45:31.673640+0000","last_undegraded":"2024-04-25T13:45:31.673640+0000","last_fullsized":"2024-04-25T13:45:31.673640+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:11.191176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.675271+0000","last_change":"2024-04-25T13:45:29.650556+0000","last_active":"2024-04-25T13:45:31.675271+0000","last_peered":"2024-04-25T13:45:31.675271+0000","last_clean":"2024-04-25T13:45:31.675271+0000","last_became_active":"2024-04-25T13:45:29.648785+0000","last_became_peered":"2024-04-25T13:45:29.648785+0000","last_unstale":"2024-04-25T13:45:31.675271+0000","last_undegraded":"2024-04-25T13:45:31.675271+0000","last_fullsized":"2024-04-25T13:45:31.675271+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:16.820752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677305+0000","last_change":"2024-04-25T13:45:30.697016+0000","last_active":"2024-04-25T13:45:31.677305+0000","last_peered":"2024-04-25T13:45:31.677305+0000","last_clean":"2024-04-25T13:45:31.677305+0000","last_became_active":"2024-04-25T13:45:30.695606+0000","last_became_peered":"2024-04-25T13:45:30.695606+0000","last_unstale":"2024-04-25T13:45:31.677305+0000","last_undegraded":"2024-04-25T13:45:31.677305+0000","last_fullsized":"2024-04-25T13:45:31.677305+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:34:10.706220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674053+0000","last_change":"2024-04-25T13:45:29.647907+0000","last_active":"2024-04-25T13:45:31.674053+0000","last_peered":"2024-04-25T13:45:31.674053+0000","last_clean":"2024-04-25T13:45:31.674053+0000","last_became_active":"2024-04-25T13:45:29.643831+0000","last_became_peered":"2024-04-25T13:45:29.643831+0000","last_unstale":"2024-04-25T13:45:31.674053+0000","last_undegraded":"2024-04-25T13:45:31.674053+0000","last_fullsized":"2024-04-25T13:45:31.674053+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:04.554430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678729+0000","last_change":"2024-04-25T13:45:30.686896+0000","last_active":"2024-04-25T13:45:31.678729+0000","last_peered":"2024-04-25T13:45:31.678729+0000","last_clean":"2024-04-25T13:45:31.678729+0000","last_became_active":"2024-04-25T13:45:30.685147+0000","last_became_peered":"2024-04-25T13:45:30.685147+0000","last_unstale":"2024-04-25T13:45:31.678729+0000","last_undegraded":"2024-04-25T13:45:31.678729+0000","last_fullsized":"2024-04-25T13:45:31.678729+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:40.404306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678321+0000","last_change":"2024-04-25T13:45:29.654859+0000","last_active":"2024-04-25T13:45:31.678321+0000","last_peered":"2024-04-25T13:45:31.678321+0000","last_clean":"2024-04-25T13:45:31.678321+0000","last_became_active":"2024-04-25T13:45:29.654627+0000","last_became_peered":"2024-04-25T13:45:29.654627+0000","last_unstale":"2024-04-25T13:45:31.678321+0000","last_undegraded":"2024-04-25T13:45:31.678321+0000","last_fullsized":"2024-04-25T13:45:31.678321+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:39.921074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677891+0000","last_change":"2024-04-25T13:45:30.690268+0000","last_active":"2024-04-25T13:45:31.677891+0000","last_peered":"2024-04-25T13:45:31.677891+0000","last_clean":"2024-04-25T13:45:31.677891+0000","last_became_active":"2024-04-25T13:45:30.688767+0000","last_became_peered":"2024-04-25T13:45:30.688767+0000","last_unstale":"2024-04-25T13:45:31.677891+0000","last_undegraded":"2024-04-25T13:45:31.677891+0000","last_fullsized":"2024-04-25T13:45:31.677891+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:20.407223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680462+0000","last_change":"2024-04-25T13:45:29.637734+0000","last_active":"2024-04-25T13:45:31.680462+0000","last_peered":"2024-04-25T13:45:31.680462+0000","last_clean":"2024-04-25T13:45:31.680462+0000","last_became_active":"2024-04-25T13:45:29.637008+0000","last_became_peered":"2024-04-25T13:45:29.637008+0000","last_unstale":"2024-04-25T13:45:31.680462+0000","last_undegraded":"2024-04-25T13:45:31.680462+0000","last_fullsized":"2024-04-25T13:45:31.680462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:46.023590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678968+0000","last_change":"2024-04-25T13:45:30.688496+0000","last_active":"2024-04-25T13:45:31.678968+0000","last_peered":"2024-04-25T13:45:31.678968+0000","last_clean":"2024-04-25T13:45:31.678968+0000","last_became_active":"2024-04-25T13:45:30.687435+0000","last_became_peered":"2024-04-25T13:45:30.687435+0000","last_unstale":"2024-04-25T13:45:31.678968+0000","last_undegraded":"2024-04-25T13:45:31.678968+0000","last_fullsized":"2024-04-25T13:45:31.678968+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:35.944250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679220+0000","last_change":"2024-04-25T13:45:29.658342+0000","last_active":"2024-04-25T13:45:31.679220+0000","last_peered":"2024-04-25T13:45:31.679220+0000","last_clean":"2024-04-25T13:45:31.679220+0000","last_became_active":"2024-04-25T13:45:29.657471+0000","last_became_peered":"2024-04-25T13:45:29.657471+0000","last_unstale":"2024-04-25T13:45:31.679220+0000","last_undegraded":"2024-04-25T13:45:31.679220+0000","last_fullsized":"2024-04-25T13:45:31.679220+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:52.005814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673402+0000","last_change":"2024-04-25T13:45:30.697544+0000","last_active":"2024-04-25T13:45:31.673402+0000","last_peered":"2024-04-25T13:45:31.673402+0000","last_clean":"2024-04-25T13:45:31.673402+0000","last_became_active":"2024-04-25T13:45:30.695390+0000","last_became_peered":"2024-04-25T13:45:30.695390+0000","last_unstale":"2024-04-25T13:45:31.673402+0000","last_undegraded":"2024-04-25T13:45:31.673402+0000","last_fullsized":"2024-04-25T13:45:31.673402+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:13.576086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"102'3","reported_seq":17,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.681274+0000","last_change":"2024-04-25T13:45:29.650779+0000","last_active":"2024-04-25T13:45:31.681274+0000","last_peered":"2024-04-25T13:45:31.681274+0000","last_clean":"2024-04-25T13:45:31.681274+0000","last_became_active":"2024-04-25T13:45:29.647654+0000","last_became_peered":"2024-04-25T13:45:29.647654+0000","last_unstale":"2024-04-25T13:45:31.681274+0000","last_undegraded":"2024-04-25T13:45:31.681274+0000","last_fullsized":"2024-04-25T13:45:31.681274+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:20.529136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674188+0000","last_change":"2024-04-25T13:45:30.669004+0000","last_active":"2024-04-25T13:45:31.674188+0000","last_peered":"2024-04-25T13:45:31.674188+0000","last_clean":"2024-04-25T13:45:31.674188+0000","last_became_active":"2024-04-25T13:45:30.667971+0000","last_became_peered":"2024-04-25T13:45:30.667971+0000","last_unstale":"2024-04-25T13:45:31.674188+0000","last_undegraded":"2024-04-25T13:45:31.674188+0000","last_fullsized":"2024-04-25T13:45:31.674188+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:12.150857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"102'6","reported_seq":20,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677457+0000","last_change":"2024-04-25T13:45:29.644819+0000","last_active":"2024-04-25T13:45:31.677457+0000","last_peered":"2024-04-25T13:45:31.677457+0000","last_clean":"2024-04-25T13:45:31.677457+0000","last_became_active":"2024-04-25T13:45:29.644062+0000","last_became_peered":"2024-04-25T13:45:29.644062+0000","last_unstale":"2024-04-25T13:45:31.677457+0000","last_undegraded":"2024-04-25T13:45:31.677457+0000","last_fullsized":"2024-04-25T13:45:31.677457+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:13.936553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673234+0000","last_change":"2024-04-25T13:45:30.671342+0000","last_active":"2024-04-25T13:45:31.673234+0000","last_peered":"2024-04-25T13:45:31.673234+0000","last_clean":"2024-04-25T13:45:31.673234+0000","last_became_active":"2024-04-25T13:45:30.670419+0000","last_became_peered":"2024-04-25T13:45:30.670419+0000","last_unstale":"2024-04-25T13:45:31.673234+0000","last_undegraded":"2024-04-25T13:45:31.673234+0000","last_fullsized":"2024-04-25T13:45:31.673234+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:35.530084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678613+0000","last_change":"2024-04-25T13:45:29.632367+0000","last_active":"2024-04-25T13:45:31.678613+0000","last_peered":"2024-04-25T13:45:31.678613+0000","last_clean":"2024-04-25T13:45:31.678613+0000","last_became_active":"2024-04-25T13:45:29.632075+0000","last_became_peered":"2024-04-25T13:45:29.632075+0000","last_unstale":"2024-04-25T13:45:31.678613+0000","last_undegraded":"2024-04-25T13:45:31.678613+0000","last_fullsized":"2024-04-25T13:45:31.678613+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:21.139156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678910+0000","last_change":"2024-04-25T13:45:30.697671+0000","last_active":"2024-04-25T13:45:31.678910+0000","last_peered":"2024-04-25T13:45:31.678910+0000","last_clean":"2024-04-25T13:45:31.678910+0000","last_became_active":"2024-04-25T13:45:30.696487+0000","last_became_peered":"2024-04-25T13:45:30.696487+0000","last_unstale":"2024-04-25T13:45:31.678910+0000","last_undegraded":"2024-04-25T13:45:31.678910+0000","last_fullsized":"2024-04-25T13:45:31.678910+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:12.924977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679952+0000","last_change":"2024-04-25T13:45:29.628699+0000","last_active":"2024-04-25T13:45:31.679952+0000","last_peered":"2024-04-25T13:45:31.679952+0000","last_clean":"2024-04-25T13:45:31.679952+0000","last_became_active":"2024-04-25T13:45:29.628403+0000","last_became_peered":"2024-04-25T13:45:29.628403+0000","last_unstale":"2024-04-25T13:45:31.679952+0000","last_undegraded":"2024-04-25T13:45:31.679952+0000","last_fullsized":"2024-04-25T13:45:31.679952+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:41:33.811942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677397+0000","last_change":"2024-04-25T13:45:30.696857+0000","last_active":"2024-04-25T13:45:31.677397+0000","last_peered":"2024-04-25T13:45:31.677397+0000","last_clean":"2024-04-25T13:45:31.677397+0000","last_became_active":"2024-04-25T13:45:30.693590+0000","last_became_peered":"2024-04-25T13:45:30.693590+0000","last_unstale":"2024-04-25T13:45:31.677397+0000","last_undegraded":"2024-04-25T13:45:31.677397+0000","last_fullsized":"2024-04-25T13:45:31.677397+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:12.940703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678942+0000","last_change":"2024-04-25T13:45:29.657645+0000","last_active":"2024-04-25T13:45:31.678942+0000","last_peered":"2024-04-25T13:45:31.678942+0000","last_clean":"2024-04-25T13:45:31.678942+0000","last_became_active":"2024-04-25T13:45:29.656926+0000","last_became_peered":"2024-04-25T13:45:29.656926+0000","last_unstale":"2024-04-25T13:45:31.678942+0000","last_undegraded":"2024-04-25T13:45:31.678942+0000","last_fullsized":"2024-04-25T13:45:31.678942+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:06.311229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679294+0000","last_change":"2024-04-25T13:45:30.705158+0000","last_active":"2024-04-25T13:45:31.679294+0000","last_peered":"2024-04-25T13:45:31.679294+0000","last_clean":"2024-04-25T13:45:31.679294+0000","last_became_active":"2024-04-25T13:45:30.693070+0000","last_became_peered":"2024-04-25T13:45:30.693070+0000","last_unstale":"2024-04-25T13:45:31.679294+0000","last_undegraded":"2024-04-25T13:45:31.679294+0000","last_fullsized":"2024-04-25T13:45:31.679294+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:31.204287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"102'8","reported_seq":22,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673257+0000","last_change":"2024-04-25T13:45:29.643984+0000","last_active":"2024-04-25T13:45:31.673257+0000","last_peered":"2024-04-25T13:45:31.673257+0000","last_clean":"2024-04-25T13:45:31.673257+0000","last_became_active":"2024-04-25T13:45:29.643681+0000","last_became_peered":"2024-04-25T13:45:29.643681+0000","last_unstale":"2024-04-25T13:45:31.673257+0000","last_undegraded":"2024-04-25T13:45:31.673257+0000","last_fullsized":"2024-04-25T13:45:31.673257+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:13.782451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679394+0000","last_change":"2024-04-25T13:45:30.685429+0000","last_active":"2024-04-25T13:45:31.679394+0000","last_peered":"2024-04-25T13:45:31.679394+0000","last_clean":"2024-04-25T13:45:31.679394+0000","last_became_active":"2024-04-25T13:45:30.684572+0000","last_became_peered":"2024-04-25T13:45:30.684572+0000","last_unstale":"2024-04-25T13:45:31.679394+0000","last_undegraded":"2024-04-25T13:45:31.679394+0000","last_fullsized":"2024-04-25T13:45:31.679394+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:54.664690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678330+0000","last_change":"2024-04-25T13:45:29.649805+0000","last_active":"2024-04-25T13:45:31.678330+0000","last_peered":"2024-04-25T13:45:31.678330+0000","last_clean":"2024-04-25T13:45:31.678330+0000","last_became_active":"2024-04-25T13:45:29.647510+0000","last_became_peered":"2024-04-25T13:45:29.647510+0000","last_unstale":"2024-04-25T13:45:31.678330+0000","last_undegraded":"2024-04-25T13:45:31.678330+0000","last_fullsized":"2024-04-25T13:45:31.678330+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:10.935408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678162+0000","last_change":"2024-04-25T13:45:30.692116+0000","last_active":"2024-04-25T13:45:31.678162+0000","last_peered":"2024-04-25T13:45:31.678162+0000","last_clean":"2024-04-25T13:45:31.678162+0000","last_became_active":"2024-04-25T13:45:30.691130+0000","last_became_peered":"2024-04-25T13:45:30.691130+0000","last_unstale":"2024-04-25T13:45:31.678162+0000","last_undegraded":"2024-04-25T13:45:31.678162+0000","last_fullsized":"2024-04-25T13:45:31.678162+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:23.282257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674350+0000","last_change":"2024-04-25T13:45:29.655132+0000","last_active":"2024-04-25T13:45:31.674350+0000","last_peered":"2024-04-25T13:45:31.674350+0000","last_clean":"2024-04-25T13:45:31.674350+0000","last_became_active":"2024-04-25T13:45:29.654779+0000","last_became_peered":"2024-04-25T13:45:29.654779+0000","last_unstale":"2024-04-25T13:45:31.674350+0000","last_undegraded":"2024-04-25T13:45:31.674350+0000","last_fullsized":"2024-04-25T13:45:31.674350+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:02:13.766677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674113+0000","last_change":"2024-04-25T13:45:30.688391+0000","last_active":"2024-04-25T13:45:31.674113+0000","last_peered":"2024-04-25T13:45:31.674113+0000","last_clean":"2024-04-25T13:45:31.674113+0000","last_became_active":"2024-04-25T13:45:30.686897+0000","last_became_peered":"2024-04-25T13:45:30.686897+0000","last_unstale":"2024-04-25T13:45:31.674113+0000","last_undegraded":"2024-04-25T13:45:31.674113+0000","last_fullsized":"2024-04-25T13:45:31.674113+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:07.607757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674010+0000","last_change":"2024-04-25T13:45:29.640247+0000","last_active":"2024-04-25T13:45:31.674010+0000","last_peered":"2024-04-25T13:45:31.674010+0000","last_clean":"2024-04-25T13:45:31.674010+0000","last_became_active":"2024-04-25T13:45:29.639918+0000","last_became_peered":"2024-04-25T13:45:29.639918+0000","last_unstale":"2024-04-25T13:45:31.674010+0000","last_undegraded":"2024-04-25T13:45:31.674010+0000","last_fullsized":"2024-04-25T13:45:31.674010+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:40.653372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083213+0000","last_change":"2024-04-25T13:45:30.700435+0000","last_active":"2024-04-25T13:45:32.083213+0000","last_peered":"2024-04-25T13:45:32.083213+0000","last_clean":"2024-04-25T13:45:32.083213+0000","last_became_active":"2024-04-25T13:45:30.700040+0000","last_became_peered":"2024-04-25T13:45:30.700040+0000","last_unstale":"2024-04-25T13:45:32.083213+0000","last_undegraded":"2024-04-25T13:45:32.083213+0000","last_fullsized":"2024-04-25T13:45:32.083213+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:08.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679578+0000","last_change":"2024-04-25T13:45:29.642203+0000","last_active":"2024-04-25T13:45:31.679578+0000","last_peered":"2024-04-25T13:45:31.679578+0000","last_clean":"2024-04-25T13:45:31.679578+0000","last_became_active":"2024-04-25T13:45:29.639686+0000","last_became_peered":"2024-04-25T13:45:29.639686+0000","last_unstale":"2024-04-25T13:45:31.679578+0000","last_undegraded":"2024-04-25T13:45:31.679578+0000","last_fullsized":"2024-04-25T13:45:31.679578+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:39.330999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677812+0000","last_change":"2024-04-25T13:45:30.688919+0000","last_active":"2024-04-25T13:45:31.677812+0000","last_peered":"2024-04-25T13:45:31.677812+0000","last_clean":"2024-04-25T13:45:31.677812+0000","last_became_active":"2024-04-25T13:45:30.687551+0000","last_became_peered":"2024-04-25T13:45:30.687551+0000","last_unstale":"2024-04-25T13:45:31.677812+0000","last_undegraded":"2024-04-25T13:45:31.677812+0000","last_fullsized":"2024-04-25T13:45:31.677812+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:06.268012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083400+0000","last_change":"2024-04-25T13:45:29.649230+0000","last_active":"2024-04-25T13:45:32.083400+0000","last_peered":"2024-04-25T13:45:32.083400+0000","last_clean":"2024-04-25T13:45:32.083400+0000","last_became_active":"2024-04-25T13:45:29.648967+0000","last_became_peered":"2024-04-25T13:45:29.648967+0000","last_unstale":"2024-04-25T13:45:32.083400+0000","last_undegraded":"2024-04-25T13:45:32.083400+0000","last_fullsized":"2024-04-25T13:45:32.083400+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:31.820121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083341+0000","last_change":"2024-04-25T13:45:30.693729+0000","last_active":"2024-04-25T13:45:32.083341+0000","last_peered":"2024-04-25T13:45:32.083341+0000","last_clean":"2024-04-25T13:45:32.083341+0000","last_became_active":"2024-04-25T13:45:30.686562+0000","last_became_peered":"2024-04-25T13:45:30.686562+0000","last_unstale":"2024-04-25T13:45:32.083341+0000","last_undegraded":"2024-04-25T13:45:32.083341+0000","last_fullsized":"2024-04-25T13:45:32.083341+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:53.056414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:32.083922+0000","last_change":"2024-04-25T13:45:29.645317+0000","last_active":"2024-04-25T13:45:32.083922+0000","last_peered":"2024-04-25T13:45:32.083922+0000","last_clean":"2024-04-25T13:45:32.083922+0000","last_became_active":"2024-04-25T13:45:29.644926+0000","last_became_peered":"2024-04-25T13:45:29.644926+0000","last_unstale":"2024-04-25T13:45:32.083922+0000","last_undegraded":"2024-04-25T13:45:32.083922+0000","last_fullsized":"2024-04-25T13:45:32.083922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:57.383144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678098+0000","last_change":"2024-04-25T13:45:30.683001+0000","last_active":"2024-04-25T13:45:31.678098+0000","last_peered":"2024-04-25T13:45:31.678098+0000","last_clean":"2024-04-25T13:45:31.678098+0000","last_became_active":"2024-04-25T13:45:30.681093+0000","last_became_peered":"2024-04-25T13:45:30.681093+0000","last_unstale":"2024-04-25T13:45:31.678098+0000","last_undegraded":"2024-04-25T13:45:31.678098+0000","last_fullsized":"2024-04-25T13:45:31.678098+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:10.170509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680127+0000","last_change":"2024-04-25T13:45:29.641572+0000","last_active":"2024-04-25T13:45:31.680127+0000","last_peered":"2024-04-25T13:45:31.680127+0000","last_clean":"2024-04-25T13:45:31.680127+0000","last_became_active":"2024-04-25T13:45:29.641234+0000","last_became_peered":"2024-04-25T13:45:29.641234+0000","last_unstale":"2024-04-25T13:45:31.680127+0000","last_undegraded":"2024-04-25T13:45:31.680127+0000","last_fullsized":"2024-04-25T13:45:31.680127+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:29.198681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674865+0000","last_change":"2024-04-25T13:45:30.682389+0000","last_active":"2024-04-25T13:45:31.674865+0000","last_peered":"2024-04-25T13:45:31.674865+0000","last_clean":"2024-04-25T13:45:31.674865+0000","last_became_active":"2024-04-25T13:45:30.681572+0000","last_became_peered":"2024-04-25T13:45:30.681572+0000","last_unstale":"2024-04-25T13:45:31.674865+0000","last_undegraded":"2024-04-25T13:45:31.674865+0000","last_fullsized":"2024-04-25T13:45:31.674865+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:37.659326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"102'2","reported_seq":16,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679922+0000","last_change":"2024-04-25T13:45:29.638340+0000","last_active":"2024-04-25T13:45:31.679922+0000","last_peered":"2024-04-25T13:45:31.679922+0000","last_clean":"2024-04-25T13:45:31.679922+0000","last_became_active":"2024-04-25T13:45:29.637980+0000","last_became_peered":"2024-04-25T13:45:29.637980+0000","last_unstale":"2024-04-25T13:45:31.679922+0000","last_undegraded":"2024-04-25T13:45:31.679922+0000","last_fullsized":"2024-04-25T13:45:31.679922+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:52:52.934013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679941+0000","last_change":"2024-04-25T13:45:30.682364+0000","last_active":"2024-04-25T13:45:31.679941+0000","last_peered":"2024-04-25T13:45:31.679941+0000","last_clean":"2024-04-25T13:45:31.679941+0000","last_became_active":"2024-04-25T13:45:30.681621+0000","last_became_peered":"2024-04-25T13:45:30.681621+0000","last_unstale":"2024-04-25T13:45:31.679941+0000","last_undegraded":"2024-04-25T13:45:31.679941+0000","last_fullsized":"2024-04-25T13:45:31.679941+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:20.503554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"103'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.706155+0000","last_change":"2024-04-25T13:45:29.630611+0000","last_active":"2024-04-25T13:45:31.706155+0000","last_peered":"2024-04-25T13:45:31.706155+0000","last_clean":"2024-04-25T13:45:31.706155+0000","last_became_active":"2024-04-25T13:45:29.630291+0000","last_became_peered":"2024-04-25T13:45:29.630291+0000","last_unstale":"2024-04-25T13:45:31.706155+0000","last_undegraded":"2024-04-25T13:45:31.706155+0000","last_fullsized":"2024-04-25T13:45:31.706155+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:35.420073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680056+0000","last_change":"2024-04-25T13:45:30.694116+0000","last_active":"2024-04-25T13:45:31.680056+0000","last_peered":"2024-04-25T13:45:31.680056+0000","last_clean":"2024-04-25T13:45:31.680056+0000","last_became_active":"2024-04-25T13:45:30.692877+0000","last_became_peered":"2024-04-25T13:45:30.692877+0000","last_unstale":"2024-04-25T13:45:31.680056+0000","last_undegraded":"2024-04-25T13:45:31.680056+0000","last_fullsized":"2024-04-25T13:45:31.680056+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:59.092680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678472+0000","last_change":"2024-04-25T13:45:29.648292+0000","last_active":"2024-04-25T13:45:31.678472+0000","last_peered":"2024-04-25T13:45:31.678472+0000","last_clean":"2024-04-25T13:45:31.678472+0000","last_became_active":"2024-04-25T13:45:29.646541+0000","last_became_peered":"2024-04-25T13:45:29.646541+0000","last_unstale":"2024-04-25T13:45:31.678472+0000","last_undegraded":"2024-04-25T13:45:31.678472+0000","last_fullsized":"2024-04-25T13:45:31.678472+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:11.823282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673210+0000","last_change":"2024-04-25T13:45:30.668538+0000","last_active":"2024-04-25T13:45:31.673210+0000","last_peered":"2024-04-25T13:45:31.673210+0000","last_clean":"2024-04-25T13:45:31.673210+0000","last_became_active":"2024-04-25T13:45:30.667706+0000","last_became_peered":"2024-04-25T13:45:30.667706+0000","last_unstale":"2024-04-25T13:45:31.673210+0000","last_undegraded":"2024-04-25T13:45:31.673210+0000","last_fullsized":"2024-04-25T13:45:31.673210+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:12:47.886928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.680283+0000","last_change":"2024-04-25T13:45:29.644241+0000","last_active":"2024-04-25T13:45:31.680283+0000","last_peered":"2024-04-25T13:45:31.680283+0000","last_clean":"2024-04-25T13:45:31.680283+0000","last_became_active":"2024-04-25T13:45:29.642947+0000","last_became_peered":"2024-04-25T13:45:29.642947+0000","last_unstale":"2024-04-25T13:45:31.680283+0000","last_undegraded":"2024-04-25T13:45:31.680283+0000","last_fullsized":"2024-04-25T13:45:31.680283+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:49.524576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.678443+0000","last_change":"2024-04-25T13:45:30.695035+0000","last_active":"2024-04-25T13:45:31.678443+0000","last_peered":"2024-04-25T13:45:31.678443+0000","last_clean":"2024-04-25T13:45:31.678443+0000","last_became_active":"2024-04-25T13:45:30.693490+0000","last_became_peered":"2024-04-25T13:45:30.693490+0000","last_unstale":"2024-04-25T13:45:31.678443+0000","last_undegraded":"2024-04-25T13:45:31.678443+0000","last_fullsized":"2024-04-25T13:45:31.678443+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:11.419064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.674153+0000","last_change":"2024-04-25T13:45:29.650026+0000","last_active":"2024-04-25T13:45:31.674153+0000","last_peered":"2024-04-25T13:45:31.674153+0000","last_clean":"2024-04-25T13:45:31.674153+0000","last_became_active":"2024-04-25T13:45:29.648521+0000","last_became_peered":"2024-04-25T13:45:29.648521+0000","last_unstale":"2024-04-25T13:45:31.674153+0000","last_undegraded":"2024-04-25T13:45:31.674153+0000","last_fullsized":"2024-04-25T13:45:31.674153+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:19.876603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673676+0000","last_change":"2024-04-25T13:45:30.698784+0000","last_active":"2024-04-25T13:45:31.673676+0000","last_peered":"2024-04-25T13:45:31.673676+0000","last_clean":"2024-04-25T13:45:31.673676+0000","last_became_active":"2024-04-25T13:45:30.697750+0000","last_became_peered":"2024-04-25T13:45:30.697750+0000","last_unstale":"2024-04-25T13:45:31.673676+0000","last_undegraded":"2024-04-25T13:45:31.673676+0000","last_fullsized":"2024-04-25T13:45:31.673676+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:47:43.751783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.675141+0000","last_change":"2024-04-25T13:45:29.656792+0000","last_active":"2024-04-25T13:45:31.675141+0000","last_peered":"2024-04-25T13:45:31.675141+0000","last_clean":"2024-04-25T13:45:31.675141+0000","last_became_active":"2024-04-25T13:45:29.654359+0000","last_became_peered":"2024-04-25T13:45:29.654359+0000","last_unstale":"2024-04-25T13:45:31.675141+0000","last_undegraded":"2024-04-25T13:45:31.675141+0000","last_fullsized":"2024-04-25T13:45:31.675141+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:23.975386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677341+0000","last_change":"2024-04-25T13:45:30.688245+0000","last_active":"2024-04-25T13:45:31.677341+0000","last_peered":"2024-04-25T13:45:31.677341+0000","last_clean":"2024-04-25T13:45:31.677341+0000","last_became_active":"2024-04-25T13:45:30.687211+0000","last_became_peered":"2024-04-25T13:45:30.687211+0000","last_unstale":"2024-04-25T13:45:31.677341+0000","last_undegraded":"2024-04-25T13:45:31.677341+0000","last_fullsized":"2024-04-25T13:45:31.677341+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:24:06.219560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.681413+0000","last_change":"2024-04-25T13:45:29.649893+0000","last_active":"2024-04-25T13:45:31.681413+0000","last_peered":"2024-04-25T13:45:31.681413+0000","last_clean":"2024-04-25T13:45:31.681413+0000","last_became_active":"2024-04-25T13:45:29.644888+0000","last_became_peered":"2024-04-25T13:45:29.644888+0000","last_unstale":"2024-04-25T13:45:31.681413+0000","last_undegraded":"2024-04-25T13:45:31.681413+0000","last_fullsized":"2024-04-25T13:45:31.681413+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:27.639484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.681124+0000","last_change":"2024-04-25T13:45:30.692692+0000","last_active":"2024-04-25T13:45:31.681124+0000","last_peered":"2024-04-25T13:45:31.681124+0000","last_clean":"2024-04-25T13:45:31.681124+0000","last_became_active":"2024-04-25T13:45:30.691903+0000","last_became_peered":"2024-04-25T13:45:30.691903+0000","last_unstale":"2024-04-25T13:45:31.681124+0000","last_undegraded":"2024-04-25T13:45:31.681124+0000","last_fullsized":"2024-04-25T13:45:31.681124+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:45.339239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":14,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.673933+0000","last_change":"2024-04-25T13:45:29.654966+0000","last_active":"2024-04-25T13:45:31.673933+0000","last_peered":"2024-04-25T13:45:31.673933+0000","last_clean":"2024-04-25T13:45:31.673933+0000","last_became_active":"2024-04-25T13:45:29.653430+0000","last_became_peered":"2024-04-25T13:45:29.653430+0000","last_unstale":"2024-04-25T13:45:31.673933+0000","last_undegraded":"2024-04-25T13:45:31.673933+0000","last_fullsized":"2024-04-25T13:45:31.673933+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:22:45.655478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":12,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679623+0000","last_change":"2024-04-25T13:45:30.705782+0000","last_active":"2024-04-25T13:45:31.679623+0000","last_peered":"2024-04-25T13:45:31.679623+0000","last_clean":"2024-04-25T13:45:31.679623+0000","last_became_active":"2024-04-25T13:45:30.704147+0000","last_became_peered":"2024-04-25T13:45:30.704147+0000","last_unstale":"2024-04-25T13:45:31.679623+0000","last_undegraded":"2024-04-25T13:45:31.679623+0000","last_fullsized":"2024-04-25T13:45:31.679623+0000","mapping_epoch":101,"log_start":"0'0","ondisk_log_start":"0'0","created":101,"last_epoch_clean":102,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:29.615360+0000","last_clean_scrub_stamp":"2024-04-25T13:45:29.615360+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:24.556348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"102'1","reported_seq":15,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.679462+0000","last_change":"2024-04-25T13:45:29.629109+0000","last_active":"2024-04-25T13:45:31.679462+0000","last_peered":"2024-04-25T13:45:31.679462+0000","last_clean":"2024-04-25T13:45:31.679462+0000","last_became_active":"2024-04-25T13:45:29.628332+0000","last_became_peered":"2024-04-25T13:45:29.628332+0000","last_unstale":"2024-04-25T13:45:31.679462+0000","last_undegraded":"2024-04-25T13:45:31.679462+0000","last_fullsized":"2024-04-25T13:45:31.679462+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:45:28.605527+0000","last_clean_scrub_stamp":"2024-04-25T13:45:28.605527+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:54:23.113730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":274,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-25T13:45:31.677529+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:45:31.677529+0000","last_peered":"2024-04-25T13:45:31.677529+0000","last_clean":"2024-04-25T13:45:31.677529+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:45:31.677529+0000","last_undegraded":"2024-04-25T13:45:31.677529+0000","last_fullsized":"2024-04-25T13:45:31.677529+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:11.594314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574913,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194187,"statfs":{"total":200000000,"available":198847936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":549,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194763,"statfs":{"total":200000000,"available":199437680,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574913,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":550,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194762,"statfs":{"total":200000000,"available":199436740,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574913,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194764,"statfs":{"total":200000000,"available":199438374,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":550,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194762,"statfs":{"total":200000000,"available":199436762,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574913,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":549,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194763,"statfs":{"total":200000000,"available":199437792,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":548,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194763,"statfs":{"total":200000000,"available":199438180,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574913,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1125,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194187,"statfs":{"total":200000000,"available":198847972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:45:35.200 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:45:35.200 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:45:35.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.383+0000 7f47ef02a640 1 Processor -- start 2024-04-25T13:45:35.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.383+0000 7f47ef02a640 1 -- start start 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47ef02a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 0x7f47e80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47ef02a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47e80d0510 0x7f47e815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47ef02a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47e80d0d70 con 0x7f47e815ea70 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47ef02a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f47e80d1050 con 0x7f47e80d0510 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47ef02a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f47e806dca0 con 0x7f47e8161f90 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47e80d0510 0x7f47e815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f47e80d0510 0x7f47e815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48956/0 (socket says 172.21.15.80:48956) 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e7fff640 1 -- 172.21.15.80:0/3689831504 learned_addr learned my addr 172.21.15.80:0/3689831504 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:35.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47ecdc6640 1 --2- 172.21.15.80:0/3689831504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 0x7f47e80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e7fff640 1 -- 172.21.15.80:0/3689831504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 msgr2=0x7f47e80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:35.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e7fff640 1 --2- 172.21.15.80:0/3689831504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 0x7f47e80cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e7fff640 1 -- 172.21.15.80:0/3689831504 >> v1:172.21.15.87:6789/0 conn(0x7f47e815ea70 legacy=0x7f47e815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:35.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e7fff640 1 -- 172.21.15.80:0/3689831504 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e806df80 con 0x7f47e80d0510 2024-04-25T13:45:35.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47ecdc6640 1 --2- 172.21.15.80:0/3689831504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 0x7f47e80cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:35.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e7fff640 1 --2- 172.21.15.80:0/3689831504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e815e530 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009870 tx=0x7f47dc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30226b61871bb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:35.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e77fe640 1 -- 172.21.15.80:0/3689831504 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0b0070 con 0x7f47e80d0510 2024-04-25T13:45:35.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e77fe640 1 -- 172.21.15.80:0/3689831504 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47dc0ab600 con 0x7f47e80d0510 2024-04-25T13:45:35.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.387+0000 7f47e77fe640 1 -- 172.21.15.80:0/3689831504 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47dc0ab9d0 con 0x7f47e80d0510 2024-04-25T13:45:35.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 -- 172.21.15.80:0/3689831504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 msgr2=0x7f47e815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 --2- 172.21.15.80:0/3689831504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e815e530 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f47dc009870 tx=0x7f47dc01c430 comp rx=0 tx=0).stop 2024-04-25T13:45:35.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 -- 172.21.15.80:0/3689831504 shutdown_connections 2024-04-25T13:45:35.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 --2- 172.21.15.80:0/3689831504 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e815e530 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 --2- 172.21.15.80:0/3689831504 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 0x7f47e80cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 -- 172.21.15.80:0/3689831504 >> 172.21.15.80:0/3689831504 conn(0x7f47e806d080 msgr2=0x7f47e80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:35.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 -- 172.21.15.80:0/3689831504 shutdown_connections 2024-04-25T13:45:35.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 -- 172.21.15.80:0/3689831504 wait complete. 2024-04-25T13:45:35.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 Processor -- start 2024-04-25T13:45:35.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.391+0000 7f47ef02a640 1 -- start start 2024-04-25T13:45:35.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e8140450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47e815ea70 0x7f47e815c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 0x7f47e815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f47e806f630 con 0x7f47e80d0510 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f47e806f4b0 con 0x7f47e815ea70 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f47e80d0920 con 0x7f47e8161f90 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e8140450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e8140450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48968/0 (socket says 172.21.15.80:48968) 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 -- 172.21.15.80:0/244051840 learned_addr learned my addr 172.21.15.80:0/244051840 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47e7fff640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47e815ea70 0x7f47e815c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 -- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 msgr2=0x7f47e815cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 0x7f47e815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 -- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47e815ea70 msgr2=0x7f47e815c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47e815ea70 0x7f47e815c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 -- 172.21.15.80:0/244051840 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e80ab4a0 con 0x7f47e80d0510 2024-04-25T13:45:35.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ecdc6640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e8140450 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f47d800c7f0 tx=0x7f47d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:35.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47e57fa640 1 -- 172.21.15.80:0/244051840 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8015070 con 0x7f47e80d0510 2024-04-25T13:45:35.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47e57fa640 1 -- 172.21.15.80:0/244051840 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47d8007820 con 0x7f47e80d0510 2024-04-25T13:45:35.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e80ab9a0 con 0x7f47e80d0510 2024-04-25T13:45:35.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47e57fa640 1 -- 172.21.15.80:0/244051840 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47d8007c10 con 0x7f47e80d0510 2024-04-25T13:45:35.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e80abf60 con 0x7f47e80d0510 2024-04-25T13:45:35.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47e57fa640 1 -- 172.21.15.80:0/244051840 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f47d8018070 con 0x7f47e80d0510 2024-04-25T13:45:35.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.395+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e806a3a0 con 0x7f47e80d0510 2024-04-25T13:45:35.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.399+0000 7f47e57fa640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47d406f430 0x7f47d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.399+0000 7f47e57fa640 1 -- 172.21.15.80:0/244051840 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f47d808f300 con 0x7f47e80d0510 2024-04-25T13:45:35.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.399+0000 7f47e7fff640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47d406f430 0x7f47d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.399+0000 7f47e7fff640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47d406f430 0x7f47d4071940 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0ad000 tx=0x7f47dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:35.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.403+0000 7f47e57fa640 1 -- 172.21.15.80:0/244051840 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f47d805c1f0 con 0x7f47e80d0510 2024-04-25T13:45:35.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.619+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f47e80b4620 con 0x7f47e80d0510 2024-04-25T13:45:35.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.623+0000 7f47e57fa640 1 -- 172.21.15.80:0/244051840 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f47d805baf0 con 0x7f47e80d0510 2024-04-25T13:45:35.624 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:45:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:45:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:45:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:45:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:45:35.625 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:45:35.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47d406f430 msgr2=0x7f47d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47d406f430 0x7f47d4071940 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f47dc0ad000 tx=0x7f47dc008040 comp rx=0 tx=0).stop 2024-04-25T13:45:35.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 msgr2=0x7f47e8140450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e8140450 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f47d800c7f0 tx=0x7f47d800ccb0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 shutdown_connections 2024-04-25T13:45:35.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f47d406f430 0x7f47d4071940 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f47e8161f90 0x7f47e815cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f47e815ea70 0x7f47e815c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 --2- 172.21.15.80:0/244051840 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f47e80d0510 0x7f47e8140450 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 >> 172.21.15.80:0/244051840 conn(0x7f47e806d080 msgr2=0x7f47e80757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:35.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 shutdown_connections 2024-04-25T13:45:35.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.627+0000 7f47ef02a640 1 -- 172.21.15.80:0/244051840 wait complete. 2024-04-25T13:45:35.645 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:35.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f729b7f8640 1 Processor -- start 2024-04-25T13:45:35.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f729b7f8640 1 -- start start 2024-04-25T13:45:35.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f729b7f8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f729406f5a0 0x7f7294160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f729b7f8640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7294161070 0x7f7294161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f729b7f8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 0x7f72940b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f729b7f8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f72941619c0 con 0x7f729406f5a0 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f729b7f8640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7294073f10 con 0x7f7294161070 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f729b7f8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f729406fcf0 con 0x7f72940b4bd0 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f7299594640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f729406f5a0 0x7f7294160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f7299594640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f729406f5a0 0x7f7294160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48978/0 (socket says 172.21.15.80:48978) 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f7299594640 1 -- 172.21.15.80:0/2514025882 learned_addr learned my addr 172.21.15.80:0/2514025882 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f7299594640 1 -- 172.21.15.80:0/2514025882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 msgr2=0x7f72940b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f7299594640 1 --2- 172.21.15.80:0/2514025882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 0x7f72940b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f7299594640 1 -- 172.21.15.80:0/2514025882 >> v2:172.21.15.87:3300/0 conn(0x7f7294161070 msgr2=0x7f7294161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f7299594640 1 --2- 172.21.15.80:0/2514025882 >> v2:172.21.15.87:3300/0 conn(0x7f7294161070 0x7f7294161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.795+0000 7f7299594640 1 -- 172.21.15.80:0/2514025882 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72940b0c60 con 0x7f729406f5a0 2024-04-25T13:45:35.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.799+0000 7f7299594640 1 --2- 172.21.15.80:0/2514025882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f7294160b30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f729406ff70 tx=0x7f729001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88899771e08020e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:35.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.799+0000 7f728bfff640 1 -- 172.21.15.80:0/2514025882 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72900b2070 con 0x7f729406f5a0 2024-04-25T13:45:35.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.799+0000 7f728bfff640 1 -- 172.21.15.80:0/2514025882 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72900a9070 con 0x7f729406f5a0 2024-04-25T13:45:35.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.799+0000 7f728bfff640 1 -- 172.21.15.80:0/2514025882 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72900ad880 con 0x7f729406f5a0 2024-04-25T13:45:35.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 -- 172.21.15.80:0/2514025882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 msgr2=0x7f7294160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2514025882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f7294160b30 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f729406ff70 tx=0x7f729001c4b0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 -- 172.21.15.80:0/2514025882 shutdown_connections 2024-04-25T13:45:35.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2514025882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 0x7f72940b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2514025882 >> v2:172.21.15.87:3300/0 conn(0x7f7294161070 0x7f7294161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2514025882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f7294160b30 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 -- 172.21.15.80:0/2514025882 >> 172.21.15.80:0/2514025882 conn(0x7f729406d080 msgr2=0x7f72940b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:35.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 -- 172.21.15.80:0/2514025882 shutdown_connections 2024-04-25T13:45:35.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 -- 172.21.15.80:0/2514025882 wait complete. 2024-04-25T13:45:35.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 Processor -- start 2024-04-25T13:45:35.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.803+0000 7f729b7f8640 1 -- start start 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f729415be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 0x7f7294151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7294161070 0x7f7294153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7294072bb0 con 0x7f729406f5a0 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f72940b5930 con 0x7f7294161070 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7294072d30 con 0x7f72940b4bd0 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f729415be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f729415be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48980/0 (socket says 172.21.15.80:48980) 2024-04-25T13:45:35.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 -- 172.21.15.80:0/2140586091 learned_addr learned my addr 172.21.15.80:0/2140586091 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7298d93640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 0x7f7294151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299d95640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7294161070 0x7f7294153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 -- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 msgr2=0x7f7294151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 0x7f7294151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 -- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7294161070 msgr2=0x7f7294153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7294161070 0x7f7294153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 -- 172.21.15.80:0/2140586091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729415ad60 con 0x7f729406f5a0 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7299594640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f729415be00 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f72900b0040 tx=0x7f729001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7289ffb640 1 -- 172.21.15.80:0/2140586091 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72900bc020 con 0x7f729406f5a0 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7289ffb640 1 -- 172.21.15.80:0/2140586091 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72900b8420 con 0x7f729406f5a0 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729415b040 con 0x7f729406f5a0 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7289ffb640 1 -- 172.21.15.80:0/2140586091 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72900b2040 con 0x7f729406f5a0 2024-04-25T13:45:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729415b600 con 0x7f729406f5a0 2024-04-25T13:45:35.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729406a3a0 con 0x7f729406f5a0 2024-04-25T13:45:35.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7289ffb640 1 -- 172.21.15.80:0/2140586091 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f72900bf020 con 0x7f729406f5a0 2024-04-25T13:45:35.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7289ffb640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f727806f430 0x7f7278071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:35.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.807+0000 7f7289ffb640 1 -- 172.21.15.80:0/2140586091 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f7290129d80 con 0x7f729406f5a0 2024-04-25T13:45:35.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.811+0000 7f7298d93640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f727806f430 0x7f7278071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:35.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.811+0000 7f7298d93640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f727806f430 0x7f7278071940 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7284006fd0 tx=0x7f7284008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:35.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.811+0000 7f7289ffb640 1 -- 172.21.15.80:0/2140586091 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f72900c6030 con 0x7f729406f5a0 2024-04-25T13:45:35.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.995+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72940739c0 con 0x7f729406f5a0 2024-04-25T13:45:35.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.995+0000 7f7289ffb640 1 -- 172.21.15.80:0/2140586091 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7f7290016130 con 0x7f729406f5a0 2024-04-25T13:45:35.995 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:35.995 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:45:35.995 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f727806f430 msgr2=0x7f7278071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f727806f430 0x7f7278071940 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7284006fd0 tx=0x7f7284008040 comp rx=0 tx=0).stop 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 msgr2=0x7f729415be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f729415be00 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f72900b0040 tx=0x7f729001acf0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 shutdown_connections 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f727806f430 0x7f7278071940 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7294161070 0x7f7294153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f72940b4bd0 0x7f7294151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 --2- 172.21.15.80:0/2140586091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f729406f5a0 0x7f729415be00 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:35.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 >> 172.21.15.80:0/2140586091 conn(0x7f729406d080 msgr2=0x7f72940b2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 shutdown_connections 2024-04-25T13:45:35.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:35.999+0000 7f729b7f8640 1 -- 172.21.15.80:0/2140586091 wait complete. 2024-04-25T13:45:36.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:45:30.006542+0000', 'modified': '2024-04-25T13:45:33.060919+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7044}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7044': {'gid': 7044, 'name': 'a', 'rank': 0, 'incarnation': 46, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6835/817443596', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6833', 'nonce': 817443596}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 817443596}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7044, 'qdb_cluster': [7044]} 2024-04-25T13:45:36.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:45:36.419 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T13:45:36.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.579+0000 7f6476ad5640 1 Processor -- start 2024-04-25T13:45:36.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.579+0000 7f6476ad5640 1 -- start start 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6476ad5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64700d04e0 0x7f64700d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6476ad5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64700d0e30 con 0x7f64700b9240 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6476ad5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64700cfb20 con 0x7f64700b7650 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6476ad5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f64700b6920 con 0x7f64700d04e0 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6474871640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f64700b9240 0x7f64700cf1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51458/0 (socket says 172.21.15.80:51458) 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6474871640 1 -- 172.21.15.80:0/1908904179 learned_addr learned my addr 172.21.15.80:0/1908904179 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6467fff640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64700d04e0 0x7f64700d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2688212071 0 0) 0x7f646c0012a0 con 0x7f64700b9240 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6458003530 con 0x7f64700b9240 2024-04-25T13:45:36.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3980489300 0 0) 0x7f6458003530 con 0x7f64700b9240 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64700d04e0 msgr2=0x7f64700d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64700d04e0 0x7f64700d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 >> v1:172.21.15.87:6789/0 conn(0x7f64700b7650 legacy=0x7f64700b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64700b6b70 con 0x7f64700b9240 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f646c0019d0 con 0x7f64700b9240 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 >> v1:172.21.15.80:6789/0 conn(0x7f64700b9240 legacy=0x7f64700cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64580039f0 0x7f6458003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6458004810 0x7f6458004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64700d04e0 0x7f64580052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f64700b6920 con 0x7f6458004810 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f645800bb90 con 0x7f64580039f0 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f645800bd90 con 0x7f64700d04e0 2024-04-25T13:45:36.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6467fff640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6458004810 0x7f6458004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6474871640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64580039f0 0x7f6458003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6467fff640 1 -- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64700d04e0 msgr2=0x7f64580052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6467fff640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f64700d04e0 0x7f64580052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6467fff640 1 -- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64580039f0 msgr2=0x7f6458003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6467fff640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64580039f0 0x7f6458003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.583+0000 7f6467fff640 1 -- 172.21.15.80:0/1908904179 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6458003530 con 0x7f6458004810 2024-04-25T13:45:36.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f6467fff640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6458004810 0x7f6458004c20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6460009ae0 tx=0x7f646001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254ce0471efdd315 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64600096b0 con 0x7f6458004810 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64600a6490 con 0x7f6458004810 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f64677fe640 1 -- 172.21.15.80:0/1908904179 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64700b8980 con 0x7f6458004810 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f6476ad5640 1 -- 172.21.15.80:0/1908904179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6458004810 msgr2=0x7f6458004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f6476ad5640 1 --2- 172.21.15.80:0/1908904179 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6458004810 0x7f6458004c20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f6460009ae0 tx=0x7f646001cab0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f6475072640 1 -- 172.21.15.80:0/1908904179 reap_dead start 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f6476ad5640 1 -- 172.21.15.80:0/1908904179 shutdown_connections 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f6476ad5640 1 -- 172.21.15.80:0/1908904179 >> 172.21.15.80:0/1908904179 conn(0x7f64700d3ad0 msgr2=0x7f64700d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f6476ad5640 1 -- 172.21.15.80:0/1908904179 shutdown_connections 2024-04-25T13:45:36.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.587+0000 7f6476ad5640 1 -- 172.21.15.80:0/1908904179 wait complete. 2024-04-25T13:45:36.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 Processor -- start 2024-04-25T13:45:36.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 -- start start 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6458004810 0x7f64701614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6470167f50 0x7f647014e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647015ed80 0x7f647015f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f647016a300 con 0x7f6470167f50 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f647016a180 con 0x7f647015ed80 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f647016a480 con 0x7f6458004810 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6470167f50 0x7f647014e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6470167f50 0x7f647014e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48996/0 (socket says 172.21.15.80:48996) 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 -- 172.21.15.80:0/3605857944 learned_addr learned my addr 172.21.15.80:0/3605857944 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6474871640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6458004810 0x7f64701614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6475072640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647015ed80 0x7f647015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 -- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6458004810 msgr2=0x7f64701614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6458004810 0x7f64701614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 -- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647015ed80 msgr2=0x7f647015f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647015ed80 0x7f647015f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 -- 172.21.15.80:0/3605857944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647015f6d0 con 0x7f6470167f50 2024-04-25T13:45:36.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6467fff640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6470167f50 0x7f647014e800 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6460009ae0 tx=0x7f6458005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f64657fa640 1 -- 172.21.15.80:0/3605857944 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64600a6bc0 con 0x7f6470167f50 2024-04-25T13:45:36.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f64657fa640 1 -- 172.21.15.80:0/3605857944 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64600a6d60 con 0x7f6470167f50 2024-04-25T13:45:36.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 -- 172.21.15.80:0/3605857944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64701581b0 con 0x7f6470167f50 2024-04-25T13:45:36.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f64657fa640 1 -- 172.21.15.80:0/3605857944 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64600a76b0 con 0x7f6470167f50 2024-04-25T13:45:36.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.591+0000 7f6476ad5640 1 -- 172.21.15.80:0/3605857944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64701576d0 con 0x7f6470167f50 2024-04-25T13:45:36.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f64657fa640 1 -- 172.21.15.80:0/3605857944 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f64600a7850 con 0x7f6470167f50 2024-04-25T13:45:36.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f64657fa640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f645c06f400 0x7f645c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f64657fa640 1 -- 172.21.15.80:0/3605857944 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f64601297f0 con 0x7f6470167f50 2024-04-25T13:45:36.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6474871640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f645c06f400 0x7f645c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f647de32000 1 Processor -- start 2024-04-25T13:45:36.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f647de32000 1 -- start start 2024-04-25T13:45:36.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6474871640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f645c06f400 0x7f645c071910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f646c001440 tx=0x7f646c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f647de32000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acceee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f647de32000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab5cea0 0x55839accf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f647de32000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839a755280 0x55839accf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f647de32000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55839ab3e160 con 0x55839ab5cea0 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f647de32000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55839aad3e20 con 0x55839a755280 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f647de32000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55839aa8ca20 con 0x55839ab70c20 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab5cea0 0x55839accf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab5cea0 0x55839accf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49008/0 (socket says 172.21.15.80:49008) 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 -- 172.21.15.80:0/874229232 learned_addr learned my addr 172.21.15.80:0/874229232 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 -- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 msgr2=0x55839acceee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6474871640 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acceee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6475072640 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839a755280 0x55839accf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acceee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 -- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839a755280 msgr2=0x55839accf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839a755280 0x55839accf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 -- 172.21.15.80:0/874229232 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839aa91100 con 0x55839ab5cea0 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6474871640 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f6467fff640 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab5cea0 0x55839accf420 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6460127d40 tx=0x7f64600f59c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfa6f3bb151f6e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.595+0000 7f64567fc640 1 -- 172.21.15.80:0/874229232 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6460125070 con 0x55839ab5cea0 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f64567fc640 1 -- 172.21.15.80:0/874229232 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64600f6770 con 0x55839ab5cea0 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f64567fc640 1 -- 172.21.15.80:0/874229232 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646010f030 con 0x55839ab5cea0 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- 172.21.15.80:0/874229232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab5cea0 msgr2=0x55839accf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab5cea0 0x55839accf420 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f6460127d40 tx=0x7f64600f59c0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- 172.21.15.80:0/874229232 shutdown_connections 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839a755280 0x55839accf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab5cea0 0x55839accf420 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 --2- 172.21.15.80:0/874229232 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acceee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- 172.21.15.80:0/874229232 >> 172.21.15.80:0/874229232 conn(0x55839a794ff0 msgr2=0x55839ac844c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- 172.21.15.80:0/874229232 shutdown_connections 2024-04-25T13:45:36.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- 172.21.15.80:0/874229232 wait complete. 2024-04-25T13:45:36.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- start start 2024-04-25T13:45:36.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 Processor -- start 2024-04-25T13:45:36.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839ac910c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55839aae1220 con 0x55839a755280 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55839aa77ea0 con 0x55839ab5cea0 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55839aaceb00 con 0x55839ab70c20 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f647de32000 10 client.0 _ll_register_callbacks cb 0x55839aae0dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6474871640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6474871640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49022/0 (socket says 172.21.15.80:49022) 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6474871640 1 -- 172.21.15.80:0/1572103836 learned_addr learned my addr 172.21.15.80:0/1572103836 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:36.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6474871640 1 -- 172.21.15.80:0/1572103836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 msgr2=0x55839ac910c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6474871640 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839ac910c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6474871640 1 -- 172.21.15.80:0/1572103836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 msgr2=0x55839acd5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6467fff640 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6474871640 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.599+0000 7f6474871640 1 -- 172.21.15.80:0/1572103836 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55839aa73500 con 0x55839a755280 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f6474871640 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd5a40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6470076e80 tx=0x7f646c0078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=628db25dff3d5682 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 1 -- 172.21.15.80:0/1572103836 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c01a070 con 0x55839a755280 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 1 -- 172.21.15.80:0/1572103836 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f646c023720 con 0x55839a755280 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 -- 172.21.15.80:0/1572103836 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64600f6770 con 0x55839a755280 2024-04-25T13:45:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 1 -- 172.21.15.80:0/1572103836 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f646c023920 con 0x55839a755280 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 10 client.7188 fetch_fsmap learned FSMap version 49 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 -- 172.21.15.80:0/1572103836 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55839aa59ea0 con 0x55839a755280 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 1 -- 172.21.15.80:0/1572103836 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f64600f6770 con 0x55839a755280 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 1 -- 172.21.15.80:0/1572103836 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f646c020070 con 0x55839a755280 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 10 client.7188 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 10 client.7188 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 20 client.7188 populate_metadata read hostname 'smithi080' 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] conn(0x55839a7797d0 0x55839ac91e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 4 client.7188 mds_command: new command op to 7044 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 -- 172.21.15.80:0/1572103836 --> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55839aa6cbe0 con 0x55839a7797d0 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f6467fff640 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] conn(0x55839a7797d0 0x55839ac91e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 10 client.7188 ms_handle_connect on v2:172.21.15.80:6833/817443596 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f6467fff640 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] conn(0x55839a7797d0 0x55839ac91e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed572f943890425e server_cookie=d184e2231024ae85 in_seq=0 out_seq=0 2024-04-25T13:45:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 1 -- 172.21.15.80:0/1572103836 <== mds.0 v2:172.21.15.80:6833/817443596 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55839aa6cbe0 con 0x55839a7797d0 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 10 client.7188 handle_command_reply: tid=0 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 client.7188 shutdown 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 -- 172.21.15.80:0/1572103836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 msgr2=0x55839acd5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd5a40 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f6470076e80 tx=0x7f646c0078b0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 -- 172.21.15.80:0/1572103836 shutdown_connections 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] conn(0x55839a7797d0 0x55839ac91e30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839ac910c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f644f7fe640 0 client.7188 ms_handle_reset on v2:172.21.15.80:6833/817443596 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 --2- 172.21.15.80:0/1572103836 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd5a40 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 -- 172.21.15.80:0/1572103836 >> 172.21.15.80:0/1572103836 conn(0x55839a794ff0 msgr2=0x55839ac84c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 -- 172.21.15.80:0/1572103836 shutdown_connections 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 1 -- 172.21.15.80:0/1572103836 wait complete. 2024-04-25T13:45:36.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.603+0000 7f647de32000 20 client.7188 trim_cache size 0 max 16384 2024-04-25T13:45:36.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 Processor -- start 2024-04-25T13:45:36.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 -- start start 2024-04-25T13:45:36.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839a755280 0x55839acdaa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acdafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab70c20 0x55839ace4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6470168c50 con 0x55839ab70c20 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f64700cb740 con 0x55839ab5cea0 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f64700cd970 con 0x55839a755280 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab70c20 0x55839ace4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab70c20 0x55839ace4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49038/0 (socket says 172.21.15.80:49038) 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 -- 172.21.15.80:0/3654928035 learned_addr learned my addr 172.21.15.80:0/3654928035 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 -- 172.21.15.80:0/3654928035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839a755280 msgr2=0x55839acdaa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 --2- 172.21.15.80:0/3654928035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839a755280 0x55839acdaa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 -- 172.21.15.80:0/3654928035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 msgr2=0x55839acdafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 --2- 172.21.15.80:0/3654928035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acdafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 -- 172.21.15.80:0/3654928035 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55839ac88840 con 0x55839ab70c20 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6475072640 1 --2- 172.21.15.80:0/3654928035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab70c20 0x55839ace4bd0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6470073e90 tx=0x7f6468005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6b5049611af71ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6455ffb640 1 -- 172.21.15.80:0/3654928035 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468001850 con 0x55839ab70c20 2024-04-25T13:45:36.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6455ffb640 1 -- 172.21.15.80:0/3654928035 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468018070 con 0x55839ab70c20 2024-04-25T13:45:36.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f6455ffb640 1 -- 172.21.15.80:0/3654928035 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6468001c30 con 0x55839ab70c20 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 -- 172.21.15.80:0/3654928035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab70c20 msgr2=0x55839ace4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.623+0000 7f647de32000 1 --2- 172.21.15.80:0/3654928035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab70c20 0x55839ace4bd0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f6470073e90 tx=0x7f6468005f90 comp rx=0 tx=0).stop 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- 172.21.15.80:0/3654928035 shutdown_connections 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 --2- 172.21.15.80:0/3654928035 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839ab70c20 0x55839ace4bd0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 --2- 172.21.15.80:0/3654928035 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acdafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 --2- 172.21.15.80:0/3654928035 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839a755280 0x55839acdaa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- 172.21.15.80:0/3654928035 >> 172.21.15.80:0/3654928035 conn(0x55839a794ff0 msgr2=0x55839acd4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- 172.21.15.80:0/3654928035 shutdown_connections 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- 172.21.15.80:0/3654928035 wait complete. 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- start start 2024-04-25T13:45:36.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 Processor -- start 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acda3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55839aad3130 con 0x55839a755280 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55839aa697d0 con 0x55839ab5cea0 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55839aad8450 con 0x55839ab70c20 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 10 client.0 _ll_register_callbacks cb 0x7f646c007760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49044/0 (socket says 172.21.15.80:49044) 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 -- 172.21.15.80:0/4096024779 learned_addr learned my addr 172.21.15.80:0/4096024779 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6475072640 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acda3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6467fff640 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 -- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 msgr2=0x55839acda3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acda3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 -- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 msgr2=0x55839acd9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 -- 172.21.15.80:0/4096024779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55839ac91430 con 0x55839a755280 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6467fff640 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6474871640 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd9940 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f646c0011e0 tx=0x7f646c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a70ce7d1e47ae9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f644f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f644f7fe640 1 -- 172.21.15.80:0/4096024779 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c03a060 con 0x55839a755280 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f644f7fe640 1 -- 172.21.15.80:0/4096024779 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f646c013570 con 0x55839a755280 2024-04-25T13:45:36.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- 172.21.15.80:0/4096024779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6468018070 con 0x55839a755280 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f644f7fe640 1 -- 172.21.15.80:0/4096024779 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) ==== 24+0+0 (secure 0 0 0) 0x7f646c040050 con 0x55839a755280 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 10 client.7194 fetch_fsmap learned FSMap version 49 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- 172.21.15.80:0/4096024779 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=2+}) -- 0x55839aa8cc70 con 0x55839a755280 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f644f7fe640 1 -- 172.21.15.80:0/4096024779 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6468018070 con 0x55839a755280 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f644f7fe640 1 -- 172.21.15.80:0/4096024779 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 49) ==== 3494+0+0 (secure 0 0 0) 0x7f646c043020 con 0x55839a755280 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 10 client.7194 fetch_fsmap finished waiting for FSMap version 49 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 10 client.7194 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 20 client.7194 populate_metadata read hostname 'smithi080' 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] conn(0x55839a767830 0x55839ad489a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 4 client.7194 mds_command: new command op to 7044 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f647de32000 1 -- 172.21.15.80:0/4096024779 --> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] -- command(tid 0: {"prefix": "status"}) -- 0x55839aae1a30 con 0x55839a767830 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.627+0000 7f6467fff640 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] conn(0x55839a767830 0x55839ad489a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f644f7fe640 10 client.7194 ms_handle_connect on v2:172.21.15.80:6833/817443596 2024-04-25T13:45:36.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6467fff640 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] conn(0x55839a767830 0x55839ad489a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf2f1f1100ba561 server_cookie=2a7ee46f379383f9 in_seq=0 out_seq=0 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f644f7fe640 1 -- 172.21.15.80:0/4096024779 <== mds.0 v2:172.21.15.80:6833/817443596 1 ==== command_reply(tid 0: 0 ) ==== 8+0+320 (crc 0 0 0) 0x55839aae1a30 con 0x55839a767830 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f644f7fe640 10 client.7194 handle_command_reply: tid=0 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 client.7194 shutdown 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 -- 172.21.15.80:0/4096024779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 msgr2=0x55839acd9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd9940 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f646c0011e0 tx=0x7f646c014000 comp rx=0 tx=0).stop 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 -- 172.21.15.80:0/4096024779 shutdown_connections 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.80:6833/817443596,v1:172.21.15.80:6835/817443596] conn(0x55839a767830 0x55839ad489a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55839ab70c20 0x55839acda3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55839ab5cea0 0x55839acd9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f644f7fe640 0 client.7194 ms_handle_reset on v2:172.21.15.80:6833/817443596 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 --2- 172.21.15.80:0/4096024779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55839a755280 0x55839acd9940 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 -- 172.21.15.80:0/4096024779 >> 172.21.15.80:0/4096024779 conn(0x55839a794ff0 msgr2=0x55839ac8b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 -- 172.21.15.80:0/4096024779 shutdown_connections 2024-04-25T13:45:36.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 1 -- 172.21.15.80:0/4096024779 wait complete. 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f647de32000 20 client.7194 trim_cache size 0 max 16384 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7044, 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 5.932859959, 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 49, 2024-04-25T13:45:36.635 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 103, 2024-04-25T13:45:36.636 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 103, 2024-04-25T13:45:36.636 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 10.740139272 2024-04-25T13:45:36.636 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:45:36.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 -- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f645c06f400 msgr2=0x7f645c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f645c06f400 0x7f645c071910 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f646c001440 tx=0x7f646c009040 comp rx=0 tx=0).stop 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 -- 172.21.15.80:0/3605857944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6470167f50 msgr2=0x7f647014e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6470167f50 0x7f647014e800 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f6460009ae0 tx=0x7f6458005840 comp rx=0 tx=0).stop 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 -- 172.21.15.80:0/3605857944 shutdown_connections 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f645c06f400 0x7f645c071910 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f647015ed80 0x7f647015f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6470167f50 0x7f647014e800 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 --2- 172.21.15.80:0/3605857944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6458004810 0x7f64701614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 -- 172.21.15.80:0/3605857944 >> 172.21.15.80:0/3605857944 conn(0x7f64700d3ad0 msgr2=0x7f6470155f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 -- 172.21.15.80:0/3605857944 shutdown_connections 2024-04-25T13:45:36.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.631+0000 7f6476ad5640 1 -- 172.21.15.80:0/3605857944 wait complete. 2024-04-25T13:45:36.651 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:45:36.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc3d0b640 1 Processor -- start 2024-04-25T13:45:36.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc3d0b640 1 -- start start 2024-04-25T13:45:36.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc3d0b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fafbc161240 0x7fafbc0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc3d0b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc06fcf0 con 0x7fafbc161650 2024-04-25T13:45:36.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc3d0b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafbc0b0c60 con 0x7fafbc06f5a0 2024-04-25T13:45:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc3d0b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fafbc073f10 con 0x7fafbc161240 2024-04-25T13:45:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc22a8640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fafbc161650 0x7fafbc0728e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51470/0 (socket says 172.21.15.80:51470) 2024-04-25T13:45:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc12a6640 1 --2- 172.21.15.80:0/4241485793 >> v2:172.21.15.87:3300/0 conn(0x7fafbc161240 0x7fafbc0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc22a8640 1 -- 172.21.15.80:0/4241485793 learned_addr learned my addr 172.21.15.80:0/4241485793 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:36.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc0aa5640 1 -- 172.21.15.80:0/4241485793 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3276838009 0 0) 0x7fafbc06fcf0 con 0x7fafbc161650 2024-04-25T13:45:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc0aa5640 1 -- 172.21.15.80:0/4241485793 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafa8003530 con 0x7fafbc161650 2024-04-25T13:45:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc12a6640 1 -- 172.21.15.80:0/4241485793 >> v1:172.21.15.87:6789/0 conn(0x7fafbc06f5a0 legacy=0x7fafbc160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc12a6640 1 -- 172.21.15.80:0/4241485793 >> v1:172.21.15.80:6789/0 conn(0x7fafbc161650 legacy=0x7fafbc0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.795+0000 7fafc12a6640 1 -- 172.21.15.80:0/4241485793 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc0b0f40 con 0x7fafbc161240 2024-04-25T13:45:36.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc12a6640 1 --2- 172.21.15.80:0/4241485793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc161240 0x7fafbc0723a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fafb80097c0 tx=0x7fafb801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1782b2dcadbe2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc0aa5640 1 -- 172.21.15.80:0/4241485793 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80b0070 con 0x7fafbc161240 2024-04-25T13:45:36.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc0aa5640 1 -- 172.21.15.80:0/4241485793 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb80ab600 con 0x7fafbc161240 2024-04-25T13:45:36.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc0aa5640 1 -- 172.21.15.80:0/4241485793 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80ab9b0 con 0x7fafbc161240 2024-04-25T13:45:36.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc3d0b640 1 -- 172.21.15.80:0/4241485793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc161240 msgr2=0x7fafbc0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc3d0b640 1 --2- 172.21.15.80:0/4241485793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc161240 0x7fafbc0723a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fafb80097c0 tx=0x7fafb801c410 comp rx=0 tx=0).stop 2024-04-25T13:45:36.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc3d0b640 1 -- 172.21.15.80:0/4241485793 shutdown_connections 2024-04-25T13:45:36.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc3d0b640 1 --2- 172.21.15.80:0/4241485793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc161240 0x7fafbc0723a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc3d0b640 1 -- 172.21.15.80:0/4241485793 >> 172.21.15.80:0/4241485793 conn(0x7fafbc06d080 msgr2=0x7fafbc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:36.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc3d0b640 1 -- 172.21.15.80:0/4241485793 shutdown_connections 2024-04-25T13:45:36.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.799+0000 7fafc3d0b640 1 -- 172.21.15.80:0/4241485793 wait complete. 2024-04-25T13:45:36.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 Processor -- start 2024-04-25T13:45:36.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 -- start start 2024-04-25T13:45:36.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc06f5a0 0x7fafbc15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafbc161240 0x7fafbc153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafbc161650 0x7fafbc153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc16b270 con 0x7fafbc161240 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc16b0f0 con 0x7fafbc06f5a0 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fafbc16b3f0 con 0x7fafbc161650 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafbc161240 0x7fafbc153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafbc161240 0x7fafbc153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49050/0 (socket says 172.21.15.80:49050) 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 -- 172.21.15.80:0/2290005673 learned_addr learned my addr 172.21.15.80:0/2290005673 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc1aa7640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc06f5a0 0x7fafbc15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc22a8640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafbc161650 0x7fafbc153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 -- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafbc161650 msgr2=0x7fafbc153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafbc161650 0x7fafbc153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 -- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc06f5a0 msgr2=0x7fafbc15fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc06f5a0 0x7fafbc15fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 -- 172.21.15.80:0/2290005673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc14b7f0 con 0x7fafbc161240 2024-04-25T13:45:36.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc12a6640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafbc161240 0x7fafbc153670 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fafb8019020 tx=0x7fafb80191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafb27fc640 1 -- 172.21.15.80:0/2290005673 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80b0070 con 0x7fafbc161240 2024-04-25T13:45:36.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafb27fc640 1 -- 172.21.15.80:0/2290005673 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb80b1040 con 0x7fafbc161240 2024-04-25T13:45:36.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc14bad0 con 0x7fafbc161240 2024-04-25T13:45:36.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafb27fc640 1 -- 172.21.15.80:0/2290005673 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80abdc0 con 0x7fafbc161240 2024-04-25T13:45:36.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.803+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc14c090 con 0x7fafbc161240 2024-04-25T13:45:36.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.807+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafbc06a3a0 con 0x7fafbc161240 2024-04-25T13:45:36.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.807+0000 7fafb27fc640 1 -- 172.21.15.80:0/2290005673 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fafb80bc060 con 0x7fafbc161240 2024-04-25T13:45:36.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.807+0000 7fafb27fc640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa006f430 0x7fafa0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:36.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.807+0000 7fafb27fc640 1 -- 172.21.15.80:0/2290005673 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7fafb81283b0 con 0x7fafbc161240 2024-04-25T13:45:36.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.807+0000 7fafc1aa7640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa006f430 0x7fafa0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:36.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.807+0000 7fafc1aa7640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa006f430 0x7fafa0071940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0b6e70 tx=0x7fafac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:36.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.811+0000 7fafb27fc640 1 -- 172.21.15.80:0/2290005673 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fafb8016090 con 0x7fafbc161240 2024-04-25T13:45:36.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.999+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fafbc160600 con 0x7fafbc161240 2024-04-25T13:45:36.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:36.999+0000 7fafb27fc640 1 -- 172.21.15.80:0/2290005673 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7fafb80f5320 con 0x7fafbc161240 2024-04-25T13:45:36.999 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:45:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa006f430 msgr2=0x7fafa0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa006f430 0x7fafa0071940 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fafbc0b6e70 tx=0x7fafac009040 comp rx=0 tx=0).stop 2024-04-25T13:45:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafbc161240 msgr2=0x7fafbc153670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafbc161240 0x7fafbc153670 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fafb8019020 tx=0x7fafb80191b0 comp rx=0 tx=0).stop 2024-04-25T13:45:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 shutdown_connections 2024-04-25T13:45:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa006f430 0x7fafa0071940 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafbc161650 0x7fafbc153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafbc161240 0x7fafbc153670 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 --2- 172.21.15.80:0/2290005673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafbc06f5a0 0x7fafbc15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 >> 172.21.15.80:0/2290005673 conn(0x7fafbc06d080 msgr2=0x7fafbc0cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 shutdown_connections 2024-04-25T13:45:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37.003+0000 7fafc3d0b640 1 -- 172.21.15.80:0/2290005673 wait complete. 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:45:37.020 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:45:37.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:37.021 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:37.027 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42231sec preferred_lft 42231sec 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:45:37.028 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:37.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:37.030 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:45:37.080 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:45:37.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:37.082 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:45:37.133 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:45:37.134 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:45:37.134 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:45:37.134 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:45:37.134 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:45:37.134 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:45:37.134 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:45:37.134 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:45:37.134 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:45:37.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:45:37.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:45:37.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:37.255 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:45:37.303 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:45:37.303 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:45:37.304 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:45:37.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:37.304 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:45:37.357 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:45:37.357 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:45:37.357 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:45:37.405 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:45:37.405 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:45:37.405 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:45:37.453 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:45:37.510 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:45:37.558 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:45:37.558 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:45:37.558 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:45:37.558 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:45:37.558 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:45:37.558 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:45:37.558 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:45:37.559 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:45:37.560 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:45:37.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:37.561 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:45:37.609 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:45:37.610 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:45:37.653 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:45:37.668 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:45:37.714 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:45:37.714 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:45:37.714 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:45:37.714 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:45:37.714 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:45:37.715 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:45:37.716 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:45:37.717 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:45:37.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:37.717 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:45:37.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:45:37.763+0000 7fd4ad6fd580 -1 init, newargv = 0x7fd49c000ee0 newargc=15 2024-04-25T13:45:37.763 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[35046]: starting ceph client 2024-04-25T13:45:37.768 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[35046]: starting fuse 2024-04-25T13:45:38.865 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:45:38.882 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:45:38.930 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:45:38.931 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:45:38.932 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:45:38.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:38.933 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:45:38.981 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:45:38.981 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:45:38.982 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:45:38.983 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:45:39.106 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.35046.asok 2024-04-25T13:45:39.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:45:39.117 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35046.asok 2024-04-25T13:45:39.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:39.117 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35046.asok status 2024-04-25T13:45:39.297 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:45:39.297 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "35046", 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7209, 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:45:39.298 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 7209 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1879350086 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1879350086 2024-04-25T13:45:39.299 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.7209 192.168.0.1:0/1879350086", 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1879350086", 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 49, 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 103, 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:45:39.300 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:45:39.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:39.315 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:45:39.315 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:45:39.321 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:45:39.321 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:45:39.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:39.321 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:45:39.377 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:45:39.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb010987640 1 Processor -- start 2024-04-25T13:45:39.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb010987640 1 -- start start 2024-04-25T13:45:39.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb010987640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c161240 0x7fb00c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb010987640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb00c06fcf0 con 0x7fb00c06f5a0 2024-04-25T13:45:39.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb010987640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb00c0b0c60 con 0x7fb00c161650 2024-04-25T13:45:39.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb010987640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb00c073f10 con 0x7fb00c161240 2024-04-25T13:45:39.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb00a59c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb00c06f5a0 0x7fb00c160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51486/0 (socket says 172.21.15.80:51486) 2024-04-25T13:45:39.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb00a59c640 1 -- 172.21.15.80:0/3734954599 learned_addr learned my addr 172.21.15.80:0/3734954599 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:39.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb001d9b640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c161240 0x7fb00c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1644994411 0 0) 0x7fb00c06fcf0 con 0x7fb00c06f5a0 2024-04-25T13:45:39.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.575+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7faff8003530 con 0x7fb00c06f5a0 2024-04-25T13:45:39.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2213518192 0 0) 0x7faff8003530 con 0x7fb00c06f5a0 2024-04-25T13:45:39.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c161240 msgr2=0x7fb00c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c161240 0x7fb00c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 >> v1:172.21.15.87:6789/0 conn(0x7fb00c161650 legacy=0x7fb00c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c0b0f40 con 0x7fb00c06f5a0 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7faffc001680 con 0x7fb00c06f5a0 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 >> v1:172.21.15.80:6789/0 conn(0x7fb00c06f5a0 legacy=0x7fb00c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faff80039f0 0x7faff8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff8004810 0x7faff8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c161240 0x7faff80052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c073f10 con 0x7faff80039f0 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faff800bb90 con 0x7faff8004810 2024-04-25T13:45:39.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faff800bd90 con 0x7fb00c161240 2024-04-25T13:45:39.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb00a59c640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faff80039f0 0x7faff8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb001d9b640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff8004810 0x7faff8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb00ad9d640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c161240 0x7faff80052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb00a59c640 1 -- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c161240 msgr2=0x7faff80052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb00a59c640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c161240 0x7faff80052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb00a59c640 1 -- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff8004810 msgr2=0x7faff8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:39.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb00a59c640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faff8004810 0x7faff8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.580 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb00a59c640 1 -- 172.21.15.80:0/3734954599 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff8003530 con 0x7faff80039f0 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb00a59c640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faff80039f0 0x7faff8003e60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0b6e70 tx=0x7fb00c073800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44cc9e09180596c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc007d40 con 0x7faff80039f0 2024-04-25T13:45:39.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faffc0a9020 con 0x7faff80039f0 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.579+0000 7fb009d9b640 1 -- 172.21.15.80:0/3734954599 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faffc005ae0 con 0x7faff80039f0 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 -- 172.21.15.80:0/3734954599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faff80039f0 msgr2=0x7faff8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 --2- 172.21.15.80:0/3734954599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faff80039f0 0x7faff8003e60 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0b6e70 tx=0x7fb00c073800 comp rx=0 tx=0).stop 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb00ad9d640 1 -- 172.21.15.80:0/3734954599 reap_dead start 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 -- 172.21.15.80:0/3734954599 shutdown_connections 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 -- 172.21.15.80:0/3734954599 >> 172.21.15.80:0/3734954599 conn(0x7fb00c06d080 msgr2=0x7fb00c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 -- 172.21.15.80:0/3734954599 shutdown_connections 2024-04-25T13:45:39.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 -- 172.21.15.80:0/3734954599 wait complete. 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 Processor -- start 2024-04-25T13:45:39.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 -- start start 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c15f1c0 0x7fb00c1588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb00c15d5b0 0x7fb00c15d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb00c15b690 0x7fb00c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.583+0000 7fb010987640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c169260 con 0x7fb00c15d5b0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb010987640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb00c1690e0 con 0x7fb00c15b690 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb010987640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb00c1693e0 con 0x7fb00c15f1c0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb00c15d5b0 0x7fb00c15d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb00c15d5b0 0x7fb00c15d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49058/0 (socket says 172.21.15.80:49058) 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 -- 172.21.15.80:0/2397393952 learned_addr learned my addr 172.21.15.80:0/2397393952 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb00a59c640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c15f1c0 0x7fb00c1588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb00ad9d640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb00c15b690 0x7fb00c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 -- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c15f1c0 msgr2=0x7fb00c1588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c15f1c0 0x7fb00c1588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 -- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb00c15b690 msgr2=0x7fb00c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb00c15b690 0x7fb00c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 -- 172.21.15.80:0/2397393952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb00c15bfe0 con 0x7fb00c15d5b0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb00ad9d640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb00c15b690 0x7fb00c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb00a59c640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c15f1c0 0x7fb00c1588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb001d9b640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb00c15d5b0 0x7fb00c15d9c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7faff40115e0 tx=0x7faff4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb0037fe640 1 -- 172.21.15.80:0/2397393952 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff400f8f0 con 0x7fb00c15d5b0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb0037fe640 1 -- 172.21.15.80:0/2397393952 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faff400fa90 con 0x7fb00c15d5b0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb00c1fb5c0 con 0x7fb00c15d5b0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb0037fe640 1 -- 172.21.15.80:0/2397393952 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff400fe40 con 0x7fb00c15d5b0 2024-04-25T13:45:39.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb00c15e400 con 0x7fb00c15d5b0 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb00c06a3a0 con 0x7fb00c15d5b0 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb0037fe640 1 -- 172.21.15.80:0/2397393952 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faff4015460 con 0x7fb00c15d5b0 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb0037fe640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafec06f400 0x7fafec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb0037fe640 1 -- 172.21.15.80:0/2397393952 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7faff408fb40 con 0x7fb00c15d5b0 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.587+0000 7fb00a59c640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafec06f400 0x7fafec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:39.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.591+0000 7fb00a59c640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafec06f400 0x7fafec071910 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faffc0aa000 tx=0x7faffc001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:39.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.595+0000 7fb0037fe640 1 -- 172.21.15.80:0/2397393952 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb00c15bfe0 con 0x7fb00c15d5b0 2024-04-25T13:45:39.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb00c0739c0 con 0x7fb00c15d5b0 2024-04-25T13:45:39.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb0037fe640 1 -- 172.21.15.80:0/2397393952 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v49) ==== 53+0+73 (secure 0 0 0) 0x7faff4019030 con 0x7fb00c15d5b0 2024-04-25T13:45:39.793 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafec06f400 msgr2=0x7fafec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafec06f400 0x7fafec071910 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faffc0aa000 tx=0x7faffc001a50 comp rx=0 tx=0).stop 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb00c15d5b0 msgr2=0x7fb00c15d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb00c15d5b0 0x7fb00c15d9c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7faff40115e0 tx=0x7faff4011aa0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 shutdown_connections 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafec06f400 0x7fafec071910 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb00c15b690 0x7fb00c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb00c15d5b0 0x7fb00c15d9c0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 --2- 172.21.15.80:0/2397393952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb00c15f1c0 0x7fb00c1588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 >> 172.21.15.80:0/2397393952 conn(0x7fb00c06d080 msgr2=0x7fb00c0728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:39.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 shutdown_connections 2024-04-25T13:45:39.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:39.795+0000 7fb010987640 1 -- 172.21.15.80:0/2397393952 wait complete. 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:45:39.816 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:45:39.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:39.817 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:45:39.824 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:45:39.824 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:45:39.824 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:45:39.824 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:39.824 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:45:39.824 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:39.824 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42229sec preferred_lft 42229sec 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:45:39.825 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:45:39.826 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:45:39.827 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:45:39.827 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:39.827 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:45:39.827 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:45:39.827 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:45:39.827 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:45:39.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:39.827 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:45:39.877 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:45:39.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:39.879 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:45:39.932 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:45:39.932 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:45:39.932 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:45:39.932 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:45:39.932 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:45:39.932 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:45:39.932 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:45:39.933 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:45:39.933 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:45:39.933 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:45:39.933 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:45:39.933 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:45:39.933 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:45:39.933 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:45:39.933 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:45:39.933 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:45:39.933 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:45:40.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:45:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:45:40.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:45:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:45:40.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:40.055 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:45:40.061 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:45:40.061 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:45:40.061 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:45:40.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:40.061 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:45:40.115 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:45:40.116 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:45:40.116 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:45:40.163 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:45:40.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:45:40.163 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:45:40.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:45:40.267 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:45:40.315 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:45:40.315 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:45:40.315 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:45:40.316 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:45:40.316 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:45:40.316 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:45:40.316 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:45:40.316 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:45:40.316 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:45:40.317 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:45:40.317 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:45:40.317 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:45:40.317 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:45:40.317 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:45:40.317 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:45:40.318 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:45:40.318 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:45:40.318 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:45:40.318 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:45:40.318 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:45:40.318 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:45:40.319 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:45:40.319 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:45:40.319 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:45:40.319 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:45:40.319 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:45:40.319 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:45:40.320 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:45:40.320 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:45:40.320 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:45:40.320 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:45:40.320 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:45:40.320 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:45:40.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:40.322 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:45:40.370 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:45:40.370 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:45:40.415 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:45:40.431 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:45:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:45:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:45:40.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:40.481 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:45:40.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:45:40.516+0000 7fbe29988580 -1 init, newargv = 0x7fbe1801b520 newargc=15 2024-04-25T13:45:40.521 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24943]: starting ceph client 2024-04-25T13:45:40.527 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24943]: starting fuse 2024-04-25T13:45:41.626 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:45:41.643 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:45:41.691 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:45:41.691 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:45:41.691 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:45:41.691 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:45:41.691 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:45:41.691 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:45:41.692 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:45:41.693 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:45:41.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:41.694 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:45:41.742 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:45:41.742 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:45:41.742 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:45:41.742 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:45:41.742 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:45:41.742 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:45:41.743 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:45:41.744 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:45:41.744 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:45:41.744 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:45:41.744 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:45:41.744 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:45:41.744 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:45:41.744 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:45:41.864 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.24943.asok 2024-04-25T13:45:41.868 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:45:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:45:41.876 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.24943.asok 2024-04-25T13:45:41.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:41.877 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.24943.asok status 2024-04-25T13:45:42.057 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:45:42.057 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:45:42.057 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:45:42.057 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:45:42.058 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:45:42.058 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:45:42.058 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:45:42.058 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "24943", 2024-04-25T13:45:42.058 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:45:42.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:45:42.058 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5546, 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5546 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:45:42.059 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:45:42.060 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:45:42.060 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2470310345 2024-04-25T13:45:42.060 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:45:42.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:45:42.060 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:45:42.060 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:45:42.060 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:45:42.060 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2470310345 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5546 192.168.0.1:0/2470310345", 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/2470310345", 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 49, 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 103, 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:45:42.061 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:45:42.062 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:45:42.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:42.074 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:45:42.074 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:45:42.081 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:45:42.081 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:45:42.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:42.082 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:45:42.136 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:42.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.295+0000 7f888d8ed640 1 Processor -- start 2024-04-25T13:45:42.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.295+0000 7f888d8ed640 1 -- start start 2024-04-25T13:45:42.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f888d8ed640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f888806f5a0 0x7f8888160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f888d8ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88880733b0 0x7f8888161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f888d8ed640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f888806fcf0 con 0x7f8888161070 2024-04-25T13:45:42.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f888d8ed640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f88881619c0 con 0x7f888806f5a0 2024-04-25T13:45:42.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f888d8ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8888073f10 con 0x7f88880733b0 2024-04-25T13:45:42.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88867fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8888161070 0x7f88880b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:44070/0 (socket says 172.21.15.80:44070) 2024-04-25T13:45:42.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88867fc640 1 -- 172.21.15.80:0/2767415833 learned_addr learned my addr 172.21.15.80:0/2767415833 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:42.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8886ffd640 1 --2- 172.21.15.80:0/2767415833 >> v2:172.21.15.87:3300/0 conn(0x7f888806f5a0 0x7f8888160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:42.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2907435975 0 0) 0x7f888806fcf0 con 0x7f8888161070 2024-04-25T13:45:42.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88877fe640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88880733b0 0x7f8888161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:42.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8870005740 con 0x7f8888161070 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2829093258 0 0) 0x7f8870005740 con 0x7f8888161070 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88880733b0 msgr2=0x7f8888161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88880733b0 0x7f8888161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 >> v2:172.21.15.87:3300/0 conn(0x7f888806f5a0 msgr2=0x7f8888160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 --2- 172.21.15.80:0/2767415833 >> v2:172.21.15.87:3300/0 conn(0x7f888806f5a0 0x7f8888160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88880b0c60 con 0x7f8888161070 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f887c001680 con 0x7f8888161070 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 >> v1:172.21.15.80:6789/0 conn(0x7f8888161070 legacy=0x7f88880b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8870005c00 0x7f8870006070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88880733b0 0x7f8870006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8870006f60 0x7f8870007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f88881619c0 con 0x7f8870006f60 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8888073f10 con 0x7f8870005c00 2024-04-25T13:45:42.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f887000dc90 con 0x7f88880733b0 2024-04-25T13:45:42.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88877fe640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8870006f60 0x7f8870007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:42.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88877fe640 1 -- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88880733b0 msgr2=0x7f8870006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:42.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88877fe640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f88880733b0 0x7f8870006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88877fe640 1 -- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8870005c00 msgr2=0x7f8870006070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:42.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88877fe640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8870005c00 0x7f8870006070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.299+0000 7f88877fe640 1 -- 172.21.15.80:0/2767415833 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8870005740 con 0x7f8870006f60 2024-04-25T13:45:42.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f88877fe640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8870006f60 0x7f8870007370 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8888070290 tx=0x7f888001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c3c90d827b58aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:42.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88800b2070 con 0x7f8870006f60 2024-04-25T13:45:42.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88800a5c30 con 0x7f8870006f60 2024-04-25T13:45:42.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f8885ffb640 1 -- 172.21.15.80:0/2767415833 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880009c10 con 0x7f8870006f60 2024-04-25T13:45:42.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f888d8ed640 1 -- 172.21.15.80:0/2767415833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8870006f60 msgr2=0x7f8870007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:42.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f888d8ed640 1 --2- 172.21.15.80:0/2767415833 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8870006f60 0x7f8870007370 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f8888070290 tx=0x7f888001cc20 comp rx=0 tx=0).stop 2024-04-25T13:45:42.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f88877fe640 1 -- 172.21.15.80:0/2767415833 reap_dead start 2024-04-25T13:45:42.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f888d8ed640 1 -- 172.21.15.80:0/2767415833 shutdown_connections 2024-04-25T13:45:42.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f888d8ed640 1 -- 172.21.15.80:0/2767415833 >> 172.21.15.80:0/2767415833 conn(0x7f888806d080 msgr2=0x7f88880b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:42.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f888d8ed640 1 -- 172.21.15.80:0/2767415833 shutdown_connections 2024-04-25T13:45:42.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.303+0000 7f888d8ed640 1 -- 172.21.15.80:0/2767415833 wait complete. 2024-04-25T13:45:42.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 Processor -- start 2024-04-25T13:45:42.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 -- start start 2024-04-25T13:45:42.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88881f1f70 0x7f88881f2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88880ae4c0 0x7f888815b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f888815be20 0x7f888814ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f888806f9b0 con 0x7f88881f1f70 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f88880b1220 con 0x7f88880ae4c0 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8888161480 con 0x7f888815be20 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88881f1f70 0x7f88881f2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88881f1f70 0x7f88881f2aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58360/0 (socket says 172.21.15.80:58360) 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 -- 172.21.15.80:0/271868193 learned_addr learned my addr 172.21.15.80:0/271868193 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:42.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f88867fc640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88880ae4c0 0x7f888815b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f88877fe640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f888815be20 0x7f888814ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 -- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f888815be20 msgr2=0x7f888814ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f888815be20 0x7f888814ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 -- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88880ae4c0 msgr2=0x7f888815b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88880ae4c0 0x7f888815b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 -- 172.21.15.80:0/271868193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888815c230 con 0x7f88881f1f70 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f8886ffd640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88881f1f70 0x7f88881f2aa0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f887800e7c0 tx=0x7f887800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f886ffff640 1 -- 172.21.15.80:0/271868193 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8878017070 con 0x7f88881f1f70 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f886ffff640 1 -- 172.21.15.80:0/271868193 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f887800ee70 con 0x7f88881f1f70 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f886ffff640 1 -- 172.21.15.80:0/271868193 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887800cb00 con 0x7f88881f1f70 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88880aecc0 con 0x7f88881f1f70 2024-04-25T13:45:42.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.307+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88880af280 con 0x7f88881f1f70 2024-04-25T13:45:42.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.311+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888806a3a0 con 0x7f88881f1f70 2024-04-25T13:45:42.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.311+0000 7f886ffff640 1 -- 172.21.15.80:0/271868193 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f887801a070 con 0x7f88881f1f70 2024-04-25T13:45:42.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.311+0000 7f886ffff640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887406f400 0x7f8874071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:42.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.311+0000 7f886ffff640 1 -- 172.21.15.80:0/271868193 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5709+0+0 (secure 0 0 0) 0x7f887808efd0 con 0x7f88881f1f70 2024-04-25T13:45:42.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.311+0000 7f88867fc640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887406f400 0x7f8874071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:42.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.311+0000 7f88867fc640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887406f400 0x7f8874071910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f887c000f70 tx=0x7f887c00c040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:42.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.315+0000 7f886ffff640 1 -- 172.21.15.80:0/271868193 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f888815c230 con 0x7f88881f1f70 2024-04-25T13:45:42.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.511+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88880739c0 con 0x7f88881f1f70 2024-04-25T13:45:42.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.511+0000 7f886ffff640 1 -- 172.21.15.80:0/271868193 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7f887805bf50 con 0x7f88881f1f70 2024-04-25T13:45:42.511 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:42.511 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:45:42.511 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887406f400 msgr2=0x7f8874071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887406f400 0x7f8874071910 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f887c000f70 tx=0x7f887c00c040 comp rx=0 tx=0).stop 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88881f1f70 msgr2=0x7f88881f2aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88881f1f70 0x7f88881f2aa0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f887800e7c0 tx=0x7f887800ec80 comp rx=0 tx=0).stop 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 shutdown_connections 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f887406f400 0x7f8874071910 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f888815be20 0x7f888814ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f88880ae4c0 0x7f888815b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 --2- 172.21.15.80:0/271868193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f88881f1f70 0x7f88881f2aa0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 >> 172.21.15.80:0/271868193 conn(0x7f888806d080 msgr2=0x7f88880b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 shutdown_connections 2024-04-25T13:45:42.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:42.515+0000 7f888d8ed640 1 -- 172.21.15.80:0/271868193 wait complete. 2024-04-25T13:45:42.533 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:45:42.684 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":7044,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.973251298999999,"mdsmap_epoch":49,"osdmap_epoch":103,"osdmap_epoch_barrier":103,"uptime":16.780540206000001} 2024-04-25T13:45:42.685 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 7044, "mdsmap_epoch": 49, "osdmap_epoch": 103, "osdmap_epoch_barrier": 103, "rank_uptime": 11.973251299, "state": "up:active", "uptime": 16.780540206, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:45:42.685 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-25T13:45:42.845 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_mon_report_interval":"5"} 2024-04-25T13:45:42.846 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-25T13:45:42.846 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2024-04-25T13:45:42.846 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:42.846 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:42.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:42.852 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2024-04-25T13:45:45.908 INFO:teuthology.orchestra.run.smithi080.stderr:265+0 records in 2024-04-25T13:45:45.908 INFO:teuthology.orchestra.run.smithi080.stderr:265+0 records out 2024-04-25T13:45:45.908 INFO:teuthology.orchestra.run.smithi080.stderr:277872640 bytes (278 MB, 265 MiB) copied, 3.00291 s, 92.5 MB/s 2024-04-25T13:45:45.909 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:45.909 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:45.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:45.915 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2024-04-25T13:45:48.274 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:45:48.275+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2024-04-25T13:45:48.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:45:48.362 INFO:teuthology.orchestra.run.smithi080.stderr:dd: error writing 'large_file_b': No space left on device 2024-04-25T13:45:48.362 INFO:teuthology.orchestra.run.smithi080.stderr:dd: closing output file 'large_file_b': No space left on device 2024-04-25T13:45:48.362 INFO:tasks.cephfs.test_full:Writing file B failed (full status happened already) 2024-04-25T13:45:48.363 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:45:48.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c4548640 1 Processor -- start 2024-04-25T13:45:48.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c4548640 1 -- start start 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c4548640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc160790 0x7fc1bc068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c4548640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc1bc068ad0 0x7fc1bc06cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c4548640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1bc06c690 con 0x7fc1bc0c88e0 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c4548640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc1bc0c7ee0 con 0x7fc1bc068ad0 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c4548640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc1bc068ee0 con 0x7fc1bc160790 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c2ae5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc1bc068ad0 0x7fc1bc06cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c22e4640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc1bc0c88e0 0x7fc1bc0c77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:44084/0 (socket says 172.21.15.80:44084) 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c22e4640 1 -- 172.21.15.80:0/1260930037 learned_addr learned my addr 172.21.15.80:0/1260930037 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:48.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c1ae3640 1 --2- 172.21.15.80:0/1260930037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc160790 0x7fc1bc068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:48.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c2ae5640 1 -- 172.21.15.80:0/1260930037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc160790 msgr2=0x7fc1bc068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:48.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c2ae5640 1 --2- 172.21.15.80:0/1260930037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc160790 0x7fc1bc068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c2ae5640 1 -- 172.21.15.80:0/1260930037 >> v1:172.21.15.80:6789/0 conn(0x7fc1bc0c88e0 legacy=0x7fc1bc0c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:48.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.519+0000 7fc1c2ae5640 1 -- 172.21.15.80:0/1260930037 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc160ba0 con 0x7fc1bc068ad0 2024-04-25T13:45:48.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c2ae5640 1 --2- 172.21.15.80:0/1260930037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc068ad0 0x7fc1bc06cb40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009d90 tx=0x7fc1b401c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cc7f0b48221069 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:48.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c12e2640 1 -- 172.21.15.80:0/1260930037 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40a6700 con 0x7fc1bc068ad0 2024-04-25T13:45:48.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c12e2640 1 -- 172.21.15.80:0/1260930037 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b40a68a0 con 0x7fc1bc068ad0 2024-04-25T13:45:48.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c12e2640 1 -- 172.21.15.80:0/1260930037 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40a6c70 con 0x7fc1bc068ad0 2024-04-25T13:45:48.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c4548640 1 -- 172.21.15.80:0/1260930037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc068ad0 msgr2=0x7fc1bc06cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:48.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c4548640 1 --2- 172.21.15.80:0/1260930037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc068ad0 0x7fc1bc06cb40 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009d90 tx=0x7fc1b401c800 comp rx=0 tx=0).stop 2024-04-25T13:45:48.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c4548640 1 -- 172.21.15.80:0/1260930037 shutdown_connections 2024-04-25T13:45:48.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c4548640 1 --2- 172.21.15.80:0/1260930037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc068ad0 0x7fc1bc06cb40 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c4548640 1 --2- 172.21.15.80:0/1260930037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc160790 0x7fc1bc068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c4548640 1 -- 172.21.15.80:0/1260930037 >> 172.21.15.80:0/1260930037 conn(0x7fc1bc0cf390 msgr2=0x7fc1bc0cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:48.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c4548640 1 -- 172.21.15.80:0/1260930037 shutdown_connections 2024-04-25T13:45:48.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.523+0000 7fc1c4548640 1 -- 172.21.15.80:0/1260930037 wait complete. 2024-04-25T13:45:48.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 Processor -- start 2024-04-25T13:45:48.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 -- start start 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1bc068ad0 0x7fc1bc0b3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc0c88e0 0x7fc1bc14f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc160790 0x7fc1bc0b24d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc06d490 con 0x7fc1bc160790 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc1616e0 con 0x7fc1bc068ad0 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc1bc06d610 con 0x7fc1bc0c88e0 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc160790 0x7fc1bc0b24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc160790 0x7fc1bc0b24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58372/0 (socket says 172.21.15.80:58372) 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 -- 172.21.15.80:0/3400632513 learned_addr learned my addr 172.21.15.80:0/3400632513 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c1ae3640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc0c88e0 0x7fc1bc14f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:48.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c22e4640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1bc068ad0 0x7fc1bc0b3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 -- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc0c88e0 msgr2=0x7fc1bc14f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc0c88e0 0x7fc1bc14f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 -- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1bc068ad0 msgr2=0x7fc1bc0b3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1bc068ad0 0x7fc1bc0b3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 -- 172.21.15.80:0/3400632513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc14f7a0 con 0x7fc1bc160790 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c2ae5640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc160790 0x7fc1bc0b24d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc1b401d0a0 tx=0x7fc1b40195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1b2ffd640 1 -- 172.21.15.80:0/3400632513 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40a66c0 con 0x7fc1bc160790 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1b2ffd640 1 -- 172.21.15.80:0/3400632513 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1b40ba680 con 0x7fc1bc160790 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc155510 con 0x7fc1bc160790 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1b2ffd640 1 -- 172.21.15.80:0/3400632513 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b40baa30 con 0x7fc1bc160790 2024-04-25T13:45:48.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.527+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc14e4a0 con 0x7fc1bc160790 2024-04-25T13:45:48.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.531+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1bc154750 con 0x7fc1bc160790 2024-04-25T13:45:48.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.531+0000 7fc1b2ffd640 1 -- 172.21.15.80:0/3400632513 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc1b40c2020 con 0x7fc1bc160790 2024-04-25T13:45:48.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.531+0000 7fc1b2ffd640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1a006f400 0x7fc1a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:48.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.531+0000 7fc1b2ffd640 1 -- 172.21.15.80:0/3400632513 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7fc1b412ae40 con 0x7fc1bc160790 2024-04-25T13:45:48.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.531+0000 7fc1b2ffd640 1 -- 172.21.15.80:0/3400632513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=105}) -- 0x7fc1a0076f80 con 0x7fc1bc160790 2024-04-25T13:45:48.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.531+0000 7fc1c22e4640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1a006f400 0x7fc1a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:48.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.531+0000 7fc1c22e4640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1a006f400 0x7fc1a0071910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8000f70 tx=0x7fc1b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:48.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.535+0000 7fc1b2ffd640 1 -- 172.21.15.80:0/3400632513 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc1b4016480 con 0x7fc1bc160790 2024-04-25T13:45:48.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.695+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fc1bc0c9000 con 0x7fc1bc160790 2024-04-25T13:45:48.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.699+0000 7fc1b2ffd640 1 -- 172.21.15.80:0/3400632513 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) ==== 81+0+34128 (secure 0 0 0) 0x7fc1b40f7dd0 con 0x7fc1bc160790 2024-04-25T13:45:48.698 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:48.698 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 104, 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:45:48.277654+0000", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:45:48.699 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 13, 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.700 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:48.701 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:48.702 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:48.703 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.704 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:45:48.705 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:48.706 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 12, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.400550+0000", 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:48.707 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:48.708 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:48.709 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:48.710 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:48.711 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.712 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 13, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.988401+0000", 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:48.713 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.714 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:48.715 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:48.716 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:45:48.717 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:48.718 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:48.719 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:48.723 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:48.724 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.725 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:45:48.727 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:48.728 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.729 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.730 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.731 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:45:48.732 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:48.733 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:48.734 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:48.735 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.736 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.737 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.738 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:48.739 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:48.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.741 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.742 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:45:48.742 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:48.742 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.742 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.742 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.742 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:48.742 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.742 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.743 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:45:48.744 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:48.745 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:45:48.746 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:48.747 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:48.748 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.749 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.750 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.750 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:45:48.750 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:45:48.750 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:45:48.750 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:45:48.750 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:48.750 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:48.750 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:48.751 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.752 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:45:48.753 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.754 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.755 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:45:48.756 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.757 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.758 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.759 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:48.760 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:48.761 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:48.762 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:48.763 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:45:48.764 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:48.765 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:48.766 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:45:48.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:48.768 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:48.769 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:45:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.699+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1a006f400 msgr2=0x7fc1a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.699+0000 7fc1c4548640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1a006f400 0x7fc1a0071910 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8000f70 tx=0x7fc1b8009040 comp rx=0 tx=0).stop 2024-04-25T13:45:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.699+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc160790 msgr2=0x7fc1bc0b24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.699+0000 7fc1c4548640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc160790 0x7fc1bc0b24d0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fc1b401d0a0 tx=0x7fc1b40195a0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.703+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 shutdown_connections 2024-04-25T13:45:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.703+0000 7fc1c4548640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc1a006f400 0x7fc1a0071910 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.703+0000 7fc1c4548640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc1bc160790 0x7fc1bc0b24d0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.703+0000 7fc1c4548640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc1bc0c88e0 0x7fc1bc14f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.703+0000 7fc1c4548640 1 --2- 172.21.15.80:0/3400632513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1bc068ad0 0x7fc1bc0b3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.703+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 >> 172.21.15.80:0/3400632513 conn(0x7fc1bc0cf390 msgr2=0x7fc1bc0cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.703+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 shutdown_connections 2024-04-25T13:45:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:48.703+0000 7fc1c4548640 1 -- 172.21.15.80:0/3400632513 wait complete. 2024-04-25T13:45:48.773 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:48.773 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:48.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:48.783 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2024-04-25T13:45:48.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:45:48.870 INFO:teuthology.orchestra.run.smithi080.stderr:dd: error writing 'large_file_b': No space left on device 2024-04-25T13:45:48.870 INFO:teuthology.orchestra.run.smithi080.stderr:1+0 records in 2024-04-25T13:45:48.870 INFO:teuthology.orchestra.run.smithi080.stderr:0+0 records out 2024-04-25T13:45:48.871 INFO:teuthology.orchestra.run.smithi080.stderr:0 bytes copied, 0.00442208 s, 0.0 kB/s 2024-04-25T13:45:48.871 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:45:49.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.019+0000 7f95d1d3c640 1 Processor -- start 2024-04-25T13:45:49.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.019+0000 7f95d1d3c640 1 -- start start 2024-04-25T13:45:49.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95d1d3c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95cc0b0ed0 0x7f95cc15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95d1d3c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95cc15cda0 con 0x7f95cc0ce730 2024-04-25T13:45:49.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95d1d3c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95cc073710 con 0x7f95cc06eba0 2024-04-25T13:45:49.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95d1d3c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f95cc0739f0 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95cb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f95cc0b0ed0 0x7f95cc15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95caffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f95cc0ce730 0x7f95cc06e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:44098/0 (socket says 172.21.15.80:44098) 2024-04-25T13:45:49.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95caffd640 1 -- 172.21.15.80:0/784106229 learned_addr learned my addr 172.21.15.80:0/784106229 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:45:49.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95cb7fe640 1 -- 172.21.15.80:0/784106229 >> v1:172.21.15.87:6789/0 conn(0x7f95cc06eba0 legacy=0x7f95cc1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:49.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95ca7fc640 1 -- 172.21.15.80:0/784106229 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2156776852 0 0) 0x7f95cc15cda0 con 0x7f95cc0ce730 2024-04-25T13:45:49.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95cb7fe640 1 -- 172.21.15.80:0/784106229 >> v1:172.21.15.80:6789/0 conn(0x7f95cc0ce730 legacy=0x7f95cc06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.023+0000 7f95cb7fe640 1 -- 172.21.15.80:0/784106229 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc073cd0 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95cb7fe640 1 --2- 172.21.15.80:0/784106229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc15c860 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0097c0 tx=0x7f95bc01ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb6f7bb43ab0a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:49.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95ca7fc640 1 -- 172.21.15.80:0/784106229 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0b5070 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95ca7fc640 1 -- 172.21.15.80:0/784106229 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc0a8990 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95ca7fc640 1 -- 172.21.15.80:0/784106229 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0a8d60 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 -- 172.21.15.80:0/784106229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 msgr2=0x7f95cc15c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 --2- 172.21.15.80:0/784106229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc15c860 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0097c0 tx=0x7f95bc01ec90 comp rx=0 tx=0).stop 2024-04-25T13:45:49.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 -- 172.21.15.80:0/784106229 shutdown_connections 2024-04-25T13:45:49.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 --2- 172.21.15.80:0/784106229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc15c860 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 -- 172.21.15.80:0/784106229 >> 172.21.15.80:0/784106229 conn(0x7f95cc06d080 msgr2=0x7f95cc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:49.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 -- 172.21.15.80:0/784106229 shutdown_connections 2024-04-25T13:45:49.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 -- 172.21.15.80:0/784106229 wait complete. 2024-04-25T13:45:49.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 Processor -- start 2024-04-25T13:45:49.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.027+0000 7f95d1d3c640 1 -- start start 2024-04-25T13:45:49.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc14cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95cc0ce730 0x7f95cc14d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95cc151910 0x7f95cc15e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc0733a0 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f95cc0b12e0 con 0x7f95cc0ce730 2024-04-25T13:45:49.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f95cc0ceb40 con 0x7f95cc151910 2024-04-25T13:45:49.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc14cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc14cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58396/0 (socket says 172.21.15.80:58396) 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 -- 172.21.15.80:0/92496813 learned_addr learned my addr 172.21.15.80:0/92496813 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cbfff640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95cc151910 0x7f95cc15e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95caffd640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95cc0ce730 0x7f95cc14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 -- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95cc151910 msgr2=0x7f95cc15e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95cc151910 0x7f95cc15e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 -- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95cc0ce730 msgr2=0x7f95cc14d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95cc0ce730 0x7f95cc14d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 -- 172.21.15.80:0/92496813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95cc15cfa0 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95cb7fe640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc14cc10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0097c0 tx=0x7f95bc01ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95b3fff640 1 -- 172.21.15.80:0/92496813 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0bf070 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95b3fff640 1 -- 172.21.15.80:0/92496813 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95bc0b0820 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95cc15d280 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95b3fff640 1 -- 172.21.15.80:0/92496813 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc0b5040 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95cc15d840 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.031+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95cc06a3a0 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.035+0000 7f95b3fff640 1 -- 172.21.15.80:0/92496813 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95bc0c2070 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.035+0000 7f95b3fff640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95b806f430 0x7f95b8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.035+0000 7f95b3fff640 1 -- 172.21.15.80:0/92496813 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f95bc12bc00 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.035+0000 7f95b3fff640 1 -- 172.21.15.80:0/92496813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=105}) -- 0x7f95b8077010 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.035+0000 7f95caffd640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95b806f430 0x7f95b8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.035+0000 7f95caffd640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95b806f430 0x7f95b8071940 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0b4a70 tx=0x7f95c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:49.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.039+0000 7f95b3fff640 1 -- 172.21.15.80:0/92496813 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95bc0187f0 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.199+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f95cc06f880 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.199+0000 7f95b3fff640 1 -- 172.21.15.80:0/92496813 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v104) ==== 81+0+34128 (secure 0 0 0) 0x7f95bc0f8b70 con 0x7f95cc0b0ed0 2024-04-25T13:45:49.200 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:49.200 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:45:49.200 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 104, 2024-04-25T13:45:49.200 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:45:49.200 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:45:49.200 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:45:48.277654+0000", 2024-04-25T13:45:49.200 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:45:49.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 13, 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:45:49.202 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:49.203 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:49.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:49.205 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:49.206 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:49.207 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 12, 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.400550+0000", 2024-04-25T13:45:49.208 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:45:49.209 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:49.210 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:49.211 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 13, 2024-04-25T13:45:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.988401+0000", 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:45:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:49.216 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:49.217 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:49.218 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:45:49.219 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.220 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:49.221 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.222 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.223 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.225 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:49.226 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.227 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:49.231 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:49.232 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:49.233 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:49.234 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:49.235 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:49.236 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.237 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.238 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:49.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:49.242 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:49.243 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.244 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.245 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.246 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.247 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:49.248 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.249 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.250 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.251 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.252 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:45:49.253 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.254 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.255 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:45:49.256 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:49.257 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:49.258 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:49.259 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.260 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:49.261 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:49.262 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:45:49.263 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:49.264 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.265 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:49.266 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.267 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:49.268 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:45:49.269 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:49.271 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95b806f430 msgr2=0x7f95b8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95b806f430 0x7f95b8071940 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f95cc0b4a70 tx=0x7f95c0009040 comp rx=0 tx=0).stop 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 msgr2=0x7f95cc14cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc14cc10 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0097c0 tx=0x7f95bc01ba30 comp rx=0 tx=0).stop 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 shutdown_connections 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95b806f430 0x7f95b8071940 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95cc151910 0x7f95cc15e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f95cc0ce730 0x7f95cc14d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 --2- 172.21.15.80:0/92496813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f95cc0b0ed0 0x7f95cc14cc10 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 >> 172.21.15.80:0/92496813 conn(0x7f95cc06d080 msgr2=0x7f95cc0d0a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 shutdown_connections 2024-04-25T13:45:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.203+0000 7f95d1d3c640 1 -- 172.21.15.80:0/92496813 wait complete. 2024-04-25T13:45:49.274 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:45:49.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.427+0000 7f135ec73640 1 Processor -- start 2024-04-25T13:45:49.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.427+0000 7f135ec73640 1 -- start start 2024-04-25T13:45:49.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ec73640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1358161f90 0x7f13580cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ec73640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13580d0510 0x7f135815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ec73640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f135815ea70 0x7f135815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ec73640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f13580d0d70 con 0x7f1358161f90 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ec73640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f13580d1050 con 0x7f13580d0510 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ec73640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f135806dca0 con 0x7f135815ea70 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ca0f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1358161f90 0x7f13580cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ca0f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1358161f90 0x7f13580cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58398/0 (socket says 172.21.15.80:58398) 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ca0f640 1 -- 172.21.15.80:0/2419185771 learned_addr learned my addr 172.21.15.80:0/2419185771 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ca0f640 1 -- 172.21.15.80:0/2419185771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f135815ea70 msgr2=0x7f135815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ca0f640 1 --2- 172.21.15.80:0/2419185771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f135815ea70 0x7f135815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ca0f640 1 -- 172.21.15.80:0/2419185771 >> v2:172.21.15.87:3300/0 conn(0x7f13580d0510 msgr2=0x7f135815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ca0f640 1 --2- 172.21.15.80:0/2419185771 >> v2:172.21.15.87:3300/0 conn(0x7f13580d0510 0x7f135815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.431+0000 7f135ca0f640 1 -- 172.21.15.80:0/2419185771 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135806df80 con 0x7f1358161f90 2024-04-25T13:45:49.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ca0f640 1 --2- 172.21.15.80:0/2419185771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1358161f90 0x7f13580cffd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1358074290 tx=0x7f134801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc2b89bdec4f71d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:49.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f13577fe640 1 -- 172.21.15.80:0/2419185771 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480b1070 con 0x7f1358161f90 2024-04-25T13:45:49.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f13577fe640 1 -- 172.21.15.80:0/2419185771 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13480a8050 con 0x7f1358161f90 2024-04-25T13:45:49.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f13577fe640 1 -- 172.21.15.80:0/2419185771 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480ac860 con 0x7f1358161f90 2024-04-25T13:45:49.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 -- 172.21.15.80:0/2419185771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1358161f90 msgr2=0x7f13580cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 --2- 172.21.15.80:0/2419185771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1358161f90 0x7f13580cffd0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f1358074290 tx=0x7f134801c460 comp rx=0 tx=0).stop 2024-04-25T13:45:49.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 -- 172.21.15.80:0/2419185771 shutdown_connections 2024-04-25T13:45:49.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 --2- 172.21.15.80:0/2419185771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f135815ea70 0x7f135815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 --2- 172.21.15.80:0/2419185771 >> v2:172.21.15.87:3300/0 conn(0x7f13580d0510 0x7f135815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 --2- 172.21.15.80:0/2419185771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1358161f90 0x7f13580cffd0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 -- 172.21.15.80:0/2419185771 >> 172.21.15.80:0/2419185771 conn(0x7f135806d080 msgr2=0x7f13580b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:49.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 -- 172.21.15.80:0/2419185771 shutdown_connections 2024-04-25T13:45:49.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 -- 172.21.15.80:0/2419185771 wait complete. 2024-04-25T13:45:49.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 Processor -- start 2024-04-25T13:45:49.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.435+0000 7f135ec73640 1 -- start start 2024-04-25T13:45:49.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13580d0510 0x7f13581426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f135815ea70 0x7f135815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1358161f90 0x7f135815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13581447a0 con 0x7f13580d0510 2024-04-25T13:45:49.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1358144620 con 0x7f135815ea70 2024-04-25T13:45:49.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13580d0920 con 0x7f1358161f90 2024-04-25T13:45:49.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13580d0510 0x7f13581426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13580d0510 0x7f13581426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58412/0 (socket says 172.21.15.80:58412) 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 -- 172.21.15.80:0/2452553091 learned_addr learned my addr 172.21.15.80:0/2452553091 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f1357fff640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f135815ea70 0x7f135815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135d210640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1358161f90 0x7f135815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 -- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1358161f90 msgr2=0x7f135815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1358161f90 0x7f135815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 -- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f135815ea70 msgr2=0x7f135815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f135815ea70 0x7f135815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 -- 172.21.15.80:0/2452553091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f135813f2d0 con 0x7f13580d0510 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ca0f640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13580d0510 0x7f13581426d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1348000c10 tx=0x7f134801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f13557fa640 1 -- 172.21.15.80:0/2452553091 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480b1040 con 0x7f13580d0510 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f13557fa640 1 -- 172.21.15.80:0/2452553091 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13480bd070 con 0x7f13580d0510 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1358071f20 con 0x7f13580d0510 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f13557fa640 1 -- 172.21.15.80:0/2452553091 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13480b7660 con 0x7f13580d0510 2024-04-25T13:45:49.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f135813ed10 con 0x7f13580d0510 2024-04-25T13:45:49.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f135806a3a0 con 0x7f13580d0510 2024-04-25T13:45:49.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f13557fa640 1 -- 172.21.15.80:0/2452553091 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13480b7800 con 0x7f13580d0510 2024-04-25T13:45:49.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.439+0000 7f13557fa640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134406f400 0x7f1344071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:49.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.443+0000 7f13557fa640 1 -- 172.21.15.80:0/2452553091 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f134812a230 con 0x7f13580d0510 2024-04-25T13:45:49.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.443+0000 7f13557fa640 1 -- 172.21.15.80:0/2452553091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=105}) -- 0x7f1344076fe0 con 0x7f13580d0510 2024-04-25T13:45:49.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.443+0000 7f1357fff640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134406f400 0x7f1344071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:49.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.443+0000 7f1357fff640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134406f400 0x7f1344071910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f134c006fd0 tx=0x7f134c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:49.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.443+0000 7f13557fa640 1 -- 172.21.15.80:0/2452553091 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1348016110 con 0x7f13580d0510 2024-04-25T13:45:49.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.643+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13580b09e0 con 0x7f13580d0510 2024-04-25T13:45:49.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.643+0000 7f13557fa640 1 -- 172.21.15.80:0/2452553091 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7f13480f7290 con 0x7f13580d0510 2024-04-25T13:45:49.643 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:49.643 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:45:49.644 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134406f400 msgr2=0x7f1344071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134406f400 0x7f1344071910 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f134c006fd0 tx=0x7f134c008040 comp rx=0 tx=0).stop 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13580d0510 msgr2=0x7f13581426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13580d0510 0x7f13581426d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f1348000c10 tx=0x7f134801aca0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 shutdown_connections 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f134406f400 0x7f1344071910 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1358161f90 0x7f135815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f135815ea70 0x7f135815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 --2- 172.21.15.80:0/2452553091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13580d0510 0x7f13581426d0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 >> 172.21.15.80:0/2452553091 conn(0x7f135806d080 msgr2=0x7f135815f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:49.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 shutdown_connections 2024-04-25T13:45:49.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:49.647+0000 7f135ec73640 1 -- 172.21.15.80:0/2452553091 wait complete. 2024-04-25T13:45:49.665 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:45:49.834 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":7044,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.121614727000001,"mdsmap_epoch":49,"osdmap_epoch":104,"osdmap_epoch_barrier":104,"uptime":23.928898295} 2024-04-25T13:45:49.834 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 7044, "mdsmap_epoch": 49, "osdmap_epoch": 104, "osdmap_epoch_barrier": 104, "rank_uptime": 19.121614727, "state": "up:active", "uptime": 23.928898295, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:45:49.834 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:45:49.835 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:45:49.835 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:45:49.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:49.842 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2024-04-25T13:45:49.891 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:45:49.891+0000 7fd48effd640 -1 client.7209 handle_client_reply no pending request on tid 13 safe is:1 2024-04-25T13:45:49.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:45:49.892 INFO:teuthology.orchestra.run.smithi080.stderr:dd: failed to open 'small_file_1': Permission denied 2024-04-25T13:45:49.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:49.892 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-04-25T13:45:49.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:45:49.947 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-04-25T13:45:50.032 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:45:50.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66ea462640 1 Processor -- start 2024-04-25T13:45:50.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66ea462640 1 -- start start 2024-04-25T13:45:50.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66ea462640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f66e40d2e80 0x7f66e41607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:50.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66ea462640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:50.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66ea462640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e40d6300 con 0x7f66e40d4330 2024-04-25T13:45:50.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66ea462640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f66e4161060 con 0x7f66e40d2e80 2024-04-25T13:45:50.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66ea462640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66e4161340 con 0x7f66e40d59b0 2024-04-25T13:45:50.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66e3fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f66e40d2e80 0x7f66e41607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:50.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66e3fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f66e40d2e80 0x7f66e41607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58422/0 (socket says 172.21.15.80:58422) 2024-04-25T13:45:50.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66e3fff640 1 -- 172.21.15.80:0/1856940245 learned_addr learned my addr 172.21.15.80:0/1856940245 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:50.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66dbfff640 1 --2- 172.21.15.80:0/1856940245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:50.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66e3fff640 1 -- 172.21.15.80:0/1856940245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 msgr2=0x7f66e40d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:50.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66e3fff640 1 --2- 172.21.15.80:0/1856940245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66e3fff640 1 -- 172.21.15.80:0/1856940245 >> v1:172.21.15.87:6789/0 conn(0x7f66e40d4330 legacy=0x7f66e40d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:50.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66e3fff640 1 -- 172.21.15.80:0/1856940245 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e40d2680 con 0x7f66e40d2e80 2024-04-25T13:45:50.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.235+0000 7f66dbfff640 1 --2- 172.21.15.80:0/1856940245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:50.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66e3fff640 1 --2- 172.21.15.80:0/1856940245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d2e80 0x7f66e41607a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009870 tx=0x7f66d401c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876ec7bad63c81df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:50.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66e37fe640 1 -- 172.21.15.80:0/1856940245 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40b0070 con 0x7f66e40d2e80 2024-04-25T13:45:50.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66e37fe640 1 -- 172.21.15.80:0/1856940245 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d40ab600 con 0x7f66e40d2e80 2024-04-25T13:45:50.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66e37fe640 1 -- 172.21.15.80:0/1856940245 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d40ab9d0 con 0x7f66e40d2e80 2024-04-25T13:45:50.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66ea462640 1 -- 172.21.15.80:0/1856940245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d2e80 msgr2=0x7f66e41607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:50.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66ea462640 1 --2- 172.21.15.80:0/1856940245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d2e80 0x7f66e41607a0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f66d4009870 tx=0x7f66d401c340 comp rx=0 tx=0).stop 2024-04-25T13:45:50.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66ea462640 1 -- 172.21.15.80:0/1856940245 shutdown_connections 2024-04-25T13:45:50.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66ea462640 1 --2- 172.21.15.80:0/1856940245 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66ea462640 1 --2- 172.21.15.80:0/1856940245 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d2e80 0x7f66e41607a0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66ea462640 1 -- 172.21.15.80:0/1856940245 >> 172.21.15.80:0/1856940245 conn(0x7f66e40737a0 msgr2=0x7f66e406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:50.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66ea462640 1 -- 172.21.15.80:0/1856940245 shutdown_connections 2024-04-25T13:45:50.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.239+0000 7f66ea462640 1 -- 172.21.15.80:0/1856940245 wait complete. 2024-04-25T13:45:50.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 Processor -- start 2024-04-25T13:45:50.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 -- start start 2024-04-25T13:45:50.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e40d2e80 0x7f66e4155550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:50.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d4330 0x7f66e40cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:50.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:50.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f66e40d3410 con 0x7f66e40d4330 2024-04-25T13:45:50.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f66e40d3290 con 0x7f66e40d2e80 2024-04-25T13:45:50.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66e415df60 con 0x7f66e40d59b0 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d4330 0x7f66e40cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d4330 0x7f66e40cea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58424/0 (socket says 172.21.15.80:58424) 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 -- 172.21.15.80:0/2797295472 learned_addr learned my addr 172.21.15.80:0/2797295472 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66e3fff640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e40d2e80 0x7f66e4155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66e89ff640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 -- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 msgr2=0x7f66e40cef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40cef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 -- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e40d2e80 msgr2=0x7f66e4155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e40d2e80 0x7f66e4155550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 -- 172.21.15.80:0/2797295472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e4155a90 con 0x7f66e40d4330 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66dbfff640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d4330 0x7f66e40cea10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00c820 tx=0x7f66cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc015070 con 0x7f66e40d4330 2024-04-25T13:45:50.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66cc007a30 con 0x7f66e40d4330 2024-04-25T13:45:50.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e4156880 con 0x7f66e40d4330 2024-04-25T13:45:50.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66cc007e40 con 0x7f66e40d4330 2024-04-25T13:45:50.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.243+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e4156e40 con 0x7f66e40d4330 2024-04-25T13:45:50.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.247+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e4157040 con 0x7f66e40d4330 2024-04-25T13:45:50.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.247+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f66cc018070 con 0x7f66e40d4330 2024-04-25T13:45:50.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.247+0000 7f66e17fa640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f66c406f400 0x7f66c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:50.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.247+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f66cc08f250 con 0x7f66e40d4330 2024-04-25T13:45:50.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.247+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=105}) -- 0x7f66c4076fe0 con 0x7f66e40d4330 2024-04-25T13:45:50.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.247+0000 7f66e3fff640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f66c406f400 0x7f66c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:50.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.247+0000 7f66e3fff640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f66c406f400 0x7f66c4071910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f66d40ad000 tx=0x7f66d40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:50.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.251+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f66cc05c140 con 0x7f66e40d4330 2024-04-25T13:45:50.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.335+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 <== mon.0 v2:172.21.15.80:3300/0 7 ==== osd_map(105..105 src has 1..105) ==== 320+0+0 (secure 0 0 0) 0x7f66cc0c1e70 con 0x7f66e40d4330 2024-04-25T13:45:50.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.335+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=106}) -- 0x7f66c40777f0 con 0x7f66e40d4330 2024-04-25T13:45:50.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.419+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f66e406a3a0 con 0x7f66e40d4330 2024-04-25T13:45:50.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66e17fa640 1 -- 172.21.15.80:0/2797295472 <== mon.0 v2:172.21.15.80:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v105) ==== 81+0+34232 (secure 0 0 0) 0x7f66cc05ba40 con 0x7f66e40d4330 2024-04-25T13:45:50.422 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:50.422 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:45:50.422 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 105, 2024-04-25T13:45:50.422 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:45:50.422 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:45:50.333976+0000", 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:45:50.423 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 13, 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:45:50.424 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:50.425 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:50.426 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:50.427 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:50.428 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.429 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.430 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 12, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.400550+0000", 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:50.431 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:50.432 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:50.433 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:50.434 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:50.435 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:45:50.436 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 13, 2024-04-25T13:45:50.437 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.988401+0000", 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:50.438 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:50.439 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:50.440 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:50.441 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.442 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:50.443 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:50.444 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:45:50.445 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.446 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:45:50.447 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:45:50.448 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:50.449 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:50.450 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.451 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.452 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.453 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.454 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:45:50.455 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.456 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.457 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.458 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:50.459 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:45:50.460 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:50.461 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:50.462 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:45:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:45:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:50.466 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:50.467 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:50.468 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:45:50.469 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:45:50.470 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:45:50.471 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull", 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.472 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.473 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.474 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.475 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.476 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:50.477 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:50.478 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:45:50.479 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:45:50.480 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:45:50.481 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.482 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.483 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.484 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.485 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.486 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.487 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:45:50.488 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:50.489 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:50.490 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:50.491 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:50.492 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:50.493 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:45:50.494 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:45:50.495 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:45:50.496 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:45:50.497 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:45:50.497 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:45:50.497 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:50.497 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:45:50.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f66c406f400 msgr2=0x7f66c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f66c406f400 0x7f66c4071910 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f66d40ad000 tx=0x7f66d40a6000 comp rx=0 tx=0).stop 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d4330 msgr2=0x7f66e40cea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d4330 0x7f66e40cea10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f66cc00c820 tx=0x7f66cc00cce0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 shutdown_connections 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f66c406f400 0x7f66c4071910 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66e40d59b0 0x7f66e40cef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f66e40d4330 0x7f66e40cea10 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 --2- 172.21.15.80:0/2797295472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f66e40d2e80 0x7f66e4155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 >> 172.21.15.80:0/2797295472 conn(0x7f66e40737a0 msgr2=0x7f66e4162250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 shutdown_connections 2024-04-25T13:45:50.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:50.423+0000 7f66ea462640 1 -- 172.21.15.80:0/2797295472 wait complete. 2024-04-25T13:45:50.500 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:45:54.367 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:45:54.367+0000 7fa2297e1640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2024-04-25T13:45:55.500 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:45:55.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b6ff0640 1 Processor -- start 2024-04-25T13:45:55.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b6ff0640 1 -- start start 2024-04-25T13:45:55.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b6ff0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f20b0161f90 0x7f20b00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:55.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b6ff0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b00d0510 0x7f20b015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:55.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b6ff0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20b00d0d70 con 0x7f20b015ea70 2024-04-25T13:45:55.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b6ff0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f20b00d1050 con 0x7f20b0161f90 2024-04-25T13:45:55.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b6ff0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20b006dca0 con 0x7f20b00d0510 2024-04-25T13:45:55.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b4d8c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f20b0161f90 0x7f20b00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:55.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b4d8c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f20b0161f90 0x7f20b00cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50812/0 (socket says 172.21.15.80:50812) 2024-04-25T13:45:55.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b4d8c640 1 -- 172.21.15.80:0/2407394027 learned_addr learned my addr 172.21.15.80:0/2407394027 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:55.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b4d8c640 1 -- 172.21.15.80:0/2407394027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b00d0510 msgr2=0x7f20b015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:55.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20affff640 1 --2- 172.21.15.80:0/2407394027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b00d0510 0x7f20b015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:55.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b4d8c640 1 --2- 172.21.15.80:0/2407394027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b00d0510 0x7f20b015e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b4d8c640 1 -- 172.21.15.80:0/2407394027 >> v1:172.21.15.87:6789/0 conn(0x7f20b015ea70 legacy=0x7f20b015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:45:55.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.663+0000 7f20b4d8c640 1 -- 172.21.15.80:0/2407394027 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b006df80 con 0x7f20b0161f90 2024-04-25T13:45:55.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b4d8c640 1 --2- 172.21.15.80:0/2407394027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b00cffd0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009870 tx=0x7f20a001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b57863d10fe51eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:55.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20af7fe640 1 -- 172.21.15.80:0/2407394027 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00b0070 con 0x7f20b0161f90 2024-04-25T13:45:55.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20af7fe640 1 -- 172.21.15.80:0/2407394027 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a00ab600 con 0x7f20b0161f90 2024-04-25T13:45:55.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20af7fe640 1 -- 172.21.15.80:0/2407394027 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a00ab9d0 con 0x7f20b0161f90 2024-04-25T13:45:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2407394027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 msgr2=0x7f20b00cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2407394027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b00cffd0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009870 tx=0x7f20a001c430 comp rx=0 tx=0).stop 2024-04-25T13:45:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2407394027 shutdown_connections 2024-04-25T13:45:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2407394027 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b00d0510 0x7f20b015e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2407394027 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b00cffd0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2407394027 >> 172.21.15.80:0/2407394027 conn(0x7f20b006d080 msgr2=0x7f20b00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2407394027 shutdown_connections 2024-04-25T13:45:55.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.667+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2407394027 wait complete. 2024-04-25T13:45:55.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b6ff0640 1 Processor -- start 2024-04-25T13:45:55.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b6ff0640 1 -- start start 2024-04-25T13:45:55.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b6ff0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b00d0510 0x7f20b015b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:55.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b6ff0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b015ea70 0x7f20b0140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:55.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b6ff0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b015a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b6ff0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f20b006f4b0 con 0x7f20b0161f90 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b6ff0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f20b006f630 con 0x7f20b00d0510 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b6ff0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20b00d0920 con 0x7f20b015ea70 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b558d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b015a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b558d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b015a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50814/0 (socket says 172.21.15.80:50814) 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.671+0000 7f20b558d640 1 -- 172.21.15.80:0/2335718510 learned_addr learned my addr 172.21.15.80:0/2335718510 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b4d8c640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b00d0510 0x7f20b015b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20affff640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b015ea70 0x7f20b0140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b558d640 1 -- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b015ea70 msgr2=0x7f20b0140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b558d640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b015ea70 0x7f20b0140bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b558d640 1 -- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b00d0510 msgr2=0x7f20b015b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b558d640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b00d0510 0x7f20b015b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b558d640 1 -- 172.21.15.80:0/2335718510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b015b7a0 con 0x7f20b0161f90 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b4d8c640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b00d0510 0x7f20b015b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20affff640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b015ea70 0x7f20b0140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b558d640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b015a4a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f20a800be90 tx=0x7f20a800bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20ad7fa640 1 -- 172.21.15.80:0/2335718510 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8019070 con 0x7f20b0161f90 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20ad7fa640 1 -- 172.21.15.80:0/2335718510 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20a8014440 con 0x7f20b0161f90 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b0160d10 con 0x7f20b0161f90 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20ad7fa640 1 -- 172.21.15.80:0/2335718510 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8014870 con 0x7f20b0161f90 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b0163580 con 0x7f20b0161f90 2024-04-25T13:45:55.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b006a3a0 con 0x7f20b0161f90 2024-04-25T13:45:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20ad7fa640 1 -- 172.21.15.80:0/2335718510 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f20a801c070 con 0x7f20b0161f90 2024-04-25T13:45:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20ad7fa640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f209c06f400 0x7f209c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:45:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20ad7fa640 1 -- 172.21.15.80:0/2335718510 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5709+0+0 (secure 0 0 0) 0x7f20a808f1f0 con 0x7f20b0161f90 2024-04-25T13:45:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20ad7fa640 1 -- 172.21.15.80:0/2335718510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=109}) -- 0x7f209c076f80 con 0x7f20b0161f90 2024-04-25T13:45:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.675+0000 7f20b4d8c640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f209c06f400 0x7f209c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:45:55.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.679+0000 7f20b4d8c640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f209c06f400 0x7f209c071910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ad000 tx=0x7f20a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:45:55.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.683+0000 7f20ad7fa640 1 -- 172.21.15.80:0/2335718510 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f20a805c170 con 0x7f20b0161f90 2024-04-25T13:45:55.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.843+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f20b00ceef0 con 0x7f20b0161f90 2024-04-25T13:45:55.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.847+0000 7f20ad7fa640 1 -- 172.21.15.80:0/2335718510 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v108) ==== 81+0+34192 (secure 0 0 0) 0x7f20a805ba70 con 0x7f20b0161f90 2024-04-25T13:45:55.847 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:45:55.847 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:45:55.847 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 108, 2024-04-25T13:45:55.847 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:45:55.847 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:45:55.847 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:45:55.382605+0000", 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:45:55.848 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 13, 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:55.849 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:55.850 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:55.851 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:55.852 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:55.853 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:55.854 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.855 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 12, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.400550+0000", 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:55.856 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:55.857 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:55.858 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.859 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.860 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:55.861 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 13, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.988401+0000", 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 3, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full", 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:45:55.862 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:45:55.863 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "104", 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:45:55.864 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:45:55.865 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:45:55.866 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:45:55.867 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:55.868 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:55.869 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:45:55.870 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:45:55.871 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:45:55.872 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.873 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.874 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.875 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:45:55.876 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.877 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.878 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.879 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.880 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:55.881 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.882 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.883 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:55.884 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:55.885 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.886 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.887 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:55.888 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:55.889 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:45:55.890 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:55.891 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:55.892 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.893 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull", 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:55.894 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:45:55.895 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.896 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:45:55.897 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:45:55.898 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:55.899 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:45:55.900 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:55.901 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.902 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.903 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "full", 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.904 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.905 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:45:55.906 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:55.907 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.908 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull", 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:45:55.909 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.910 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:55.911 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:55.912 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:55.913 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:45:55.914 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:45:55.915 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:45:55.916 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:45:55.917 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f209c06f400 msgr2=0x7f209c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f209c06f400 0x7f209c071910 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f20a00ad000 tx=0x7f20a0008040 comp rx=0 tx=0).stop 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 msgr2=0x7f20b015a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b015a4a0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f20a800be90 tx=0x7f20a800bf80 comp rx=0 tx=0).stop 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 shutdown_connections 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f209c06f400 0x7f209c071910 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f20b0161f90 0x7f20b015a4a0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20b015ea70 0x7f20b0140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 --2- 172.21.15.80:0/2335718510 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20b00d0510 0x7f20b015b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 >> 172.21.15.80:0/2335718510 conn(0x7f20b006d080 msgr2=0x7f20b00b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 shutdown_connections 2024-04-25T13:45:55.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:45:55.851+0000 7f20b6ff0640 1 -- 172.21.15.80:0/2335718510 wait complete. 2024-04-25T13:45:55.920 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:45:59.271 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:45:59.274 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:46:00.921 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:46:01.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.083+0000 7fd7bb010640 1 Processor -- start 2024-04-25T13:46:01.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.083+0000 7fd7bb010640 1 -- start start 2024-04-25T13:46:01.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7bb010640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7b4161f90 0x7fd7b40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7bb010640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7b415ec40 0x7fd7b415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7bb010640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7b40d0d70 con 0x7fd7b40d0510 2024-04-25T13:46:01.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7bb010640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b40d1050 con 0x7fd7b415ec40 2024-04-25T13:46:01.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7bb010640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd7b406dca0 con 0x7fd7b4161f90 2024-04-25T13:46:01.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b95ad640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7b415ec40 0x7fd7b415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b3fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd7b40d0510 0x7fd7b415e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45858/0 (socket says 172.21.15.80:45858) 2024-04-25T13:46:01.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b95ad640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7b415ec40 0x7fd7b415f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38094/0 (socket says 172.21.15.80:38094) 2024-04-25T13:46:01.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b3fff640 1 -- 172.21.15.80:0/2983749797 learned_addr learned my addr 172.21.15.80:0/2983749797 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:01.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b8dac640 1 --2- 172.21.15.80:0/2983749797 >> v2:172.21.15.87:3300/0 conn(0x7fd7b4161f90 0x7fd7b40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b95ad640 1 -- 172.21.15.80:0/2983749797 >> v2:172.21.15.87:3300/0 conn(0x7fd7b4161f90 msgr2=0x7fd7b40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:01.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b95ad640 1 --2- 172.21.15.80:0/2983749797 >> v2:172.21.15.87:3300/0 conn(0x7fd7b4161f90 0x7fd7b40cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b95ad640 1 -- 172.21.15.80:0/2983749797 >> v1:172.21.15.80:6789/0 conn(0x7fd7b40d0510 legacy=0x7fd7b415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.087+0000 7fd7b95ad640 1 -- 172.21.15.80:0/2983749797 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b406df80 con 0x7fd7b415ec40 2024-04-25T13:46:01.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7b95ad640 1 --2- 172.21.15.80:0/2983749797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b415ec40 0x7fd7b415f050 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40d17b0 tx=0x7fd7ac01c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c444f9430780b02e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7b37fe640 1 -- 172.21.15.80:0/2983749797 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0b2070 con 0x7fd7b415ec40 2024-04-25T13:46:01.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7b37fe640 1 -- 172.21.15.80:0/2983749797 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7ac0a5830 con 0x7fd7b415ec40 2024-04-25T13:46:01.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7b37fe640 1 -- 172.21.15.80:0/2983749797 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0a5c00 con 0x7fd7b415ec40 2024-04-25T13:46:01.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 -- 172.21.15.80:0/2983749797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b415ec40 msgr2=0x7fd7b415f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 --2- 172.21.15.80:0/2983749797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b415ec40 0x7fd7b415f050 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40d17b0 tx=0x7fd7ac01c840 comp rx=0 tx=0).stop 2024-04-25T13:46:01.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 -- 172.21.15.80:0/2983749797 shutdown_connections 2024-04-25T13:46:01.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 --2- 172.21.15.80:0/2983749797 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b415ec40 0x7fd7b415f050 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 --2- 172.21.15.80:0/2983749797 >> v2:172.21.15.87:3300/0 conn(0x7fd7b4161f90 0x7fd7b40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 -- 172.21.15.80:0/2983749797 >> 172.21.15.80:0/2983749797 conn(0x7fd7b406d080 msgr2=0x7fd7b40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:01.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 -- 172.21.15.80:0/2983749797 shutdown_connections 2024-04-25T13:46:01.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 -- 172.21.15.80:0/2983749797 wait complete. 2024-04-25T13:46:01.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.091+0000 7fd7bb010640 1 Processor -- start 2024-04-25T13:46:01.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 -- start start 2024-04-25T13:46:01.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b40d0510 0x7fd7b41436b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7b415ec40 0x7fd7b40abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7b4161f90 0x7fd7b40b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b406f4b0 con 0x7fd7b40d0510 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b406f630 con 0x7fd7b4161f90 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd7b40d0920 con 0x7fd7b415ec40 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b40d0510 0x7fd7b41436b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b40d0510 0x7fd7b41436b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38102/0 (socket says 172.21.15.80:38102) 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 -- 172.21.15.80:0/1383593908 learned_addr learned my addr 172.21.15.80:0/1383593908 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b3fff640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7b415ec40 0x7fd7b40abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 -- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7b415ec40 msgr2=0x7fd7b40abfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7b415ec40 0x7fd7b40abfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 -- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7b4161f90 msgr2=0x7fd7b40b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b95ad640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7b4161f90 0x7fd7b40b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7b4161f90 0x7fd7b40b1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 -- 172.21.15.80:0/1383593908 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b41fa7a0 con 0x7fd7b40d0510 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b3fff640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7b415ec40 0x7fd7b40abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b8dac640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b40d0510 0x7fd7b41436b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400d5f0 tx=0x7fd7a400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b17fa640 1 -- 172.21.15.80:0/1383593908 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4004240 con 0x7fd7b40d0510 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b17fa640 1 -- 172.21.15.80:0/1383593908 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7a40043e0 con 0x7fd7b40d0510 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b41f99e0 con 0x7fd7b40d0510 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b17fa640 1 -- 172.21.15.80:0/1383593908 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4003840 con 0x7fd7b40d0510 2024-04-25T13:46:01.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b41f9fa0 con 0x7fd7b40d0510 2024-04-25T13:46:01.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7b406a3a0 con 0x7fd7b40d0510 2024-04-25T13:46:01.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.095+0000 7fd7b17fa640 1 -- 172.21.15.80:0/1383593908 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7a40039e0 con 0x7fd7b40d0510 2024-04-25T13:46:01.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.099+0000 7fd7b17fa640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd7a006f400 0x7fd7a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.099+0000 7fd7b17fa640 1 -- 172.21.15.80:0/1383593908 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fd7a408f560 con 0x7fd7b40d0510 2024-04-25T13:46:01.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.099+0000 7fd7b3fff640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd7a006f400 0x7fd7a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.099+0000 7fd7b3fff640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd7a006f400 0x7fd7a0071910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd7b41684b0 tx=0x7fd7a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.103+0000 7fd7b17fa640 1 -- 172.21.15.80:0/1383593908 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd7a405c450 con 0x7fd7b40d0510 2024-04-25T13:46:01.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.263+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fd7b4069cb0 con 0x7fd7b40d0510 2024-04-25T13:46:01.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.267+0000 7fd7b17fa640 1 -- 172.21.15.80:0/1383593908 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v110) ==== 81+0+34089 (secure 0 0 0) 0x7fd7a405bd50 con 0x7fd7b40d0510 2024-04-25T13:46:01.267 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:01.267 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 110, 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:45:57.407731+0000", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:46:01.268 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 13, 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:46:01.269 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:46:01.270 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.271 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:46:01.272 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:46:01.273 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.274 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.275 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 12, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.400550+0000", 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:46:01.276 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.277 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:46:01.278 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:46:01.279 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:46:01.280 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.281 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 13, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.988401+0000", 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:46:01.282 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:46:01.283 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:46:01.284 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:46:01.285 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:46:01.286 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.287 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.288 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.289 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.290 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:46:01.291 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.292 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.293 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.294 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.295 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.296 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:46:01.297 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:46:01.298 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:46:01.299 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.300 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.301 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.302 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:46:01.303 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.304 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.305 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.306 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.307 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.308 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.309 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.310 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.311 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.312 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.313 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.314 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.315 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.316 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.317 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.318 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.319 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:46:01.320 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.321 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.322 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.323 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:46:01.324 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.325 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.326 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.327 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.328 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.329 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.330 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.331 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.332 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.333 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.334 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.335 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.336 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:46:01.337 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:46:01.338 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:46:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:46:01.339 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.339 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:01.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd7a006f400 msgr2=0x7fd7a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd7a006f400 0x7fd7a0071910 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd7b41684b0 tx=0x7fd7a8009040 comp rx=0 tx=0).stop 2024-04-25T13:46:01.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b40d0510 msgr2=0x7fd7b41436b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b40d0510 0x7fd7b41436b0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400d5f0 tx=0x7fd7a400dab0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 shutdown_connections 2024-04-25T13:46:01.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd7a006f400 0x7fd7a0071910 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7b4161f90 0x7fd7b40b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7b415ec40 0x7fd7b40abfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 --2- 172.21.15.80:0/1383593908 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7b40d0510 0x7fd7b41436b0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 >> 172.21.15.80:0/1383593908 conn(0x7fd7b406d080 msgr2=0x7fd7b40d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:01.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 shutdown_connections 2024-04-25T13:46:01.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.271+0000 7fd7bb010640 1 -- 172.21.15.80:0/1383593908 wait complete. 2024-04-25T13:46:01.342 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-25T13:46:01.342 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:46:01.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b85ba640 1 Processor -- start 2024-04-25T13:46:01.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b85ba640 1 -- start start 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b85ba640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62b00d2e80 0x7f62b01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b85ba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d59b0 0x7f62b00d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b85ba640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62b00d6300 con 0x7f62b00d4330 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b85ba640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f62b0161060 con 0x7f62b00d2e80 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b85ba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f62b0161340 con 0x7f62b00d59b0 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b6356640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62b00d2e80 0x7f62b01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b5b55640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d59b0 0x7f62b00d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b6b57640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f62b00d4330 0x7f62b00d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:56454/0 (socket says 172.21.15.80:56454) 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b6b57640 1 -- 172.21.15.80:0/2503506940 learned_addr learned my addr 172.21.15.80:0/2503506940 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:01.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b6356640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f62b00d2e80 0x7f62b01607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:48584/0 (socket says 172.21.15.80:48584) 2024-04-25T13:46:01.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b5b55640 1 -- 172.21.15.80:0/2503506940 >> v2:172.21.15.87:3300/0 conn(0x7f62b00d2e80 msgr2=0x7f62b01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:01.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b5b55640 1 --2- 172.21.15.80:0/2503506940 >> v2:172.21.15.87:3300/0 conn(0x7f62b00d2e80 0x7f62b01607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b5b55640 1 -- 172.21.15.80:0/2503506940 >> v1:172.21.15.87:6789/0 conn(0x7f62b00d4330 legacy=0x7f62b00d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b5b55640 1 -- 172.21.15.80:0/2503506940 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b00d2680 con 0x7f62b00d59b0 2024-04-25T13:46:01.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.495+0000 7f62b6356640 1 --2- 172.21.15.80:0/2503506940 >> v2:172.21.15.87:3300/0 conn(0x7f62b00d2e80 0x7f62b01607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:01.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b5b55640 1 --2- 172.21.15.80:0/2503506940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d59b0 0x7f62b00d5dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f62a00097c0 tx=0x7f62a001c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cdbf08423a81587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b5354640 1 -- 172.21.15.80:0/2503506940 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a00b1070 con 0x7f62b00d59b0 2024-04-25T13:46:01.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b5354640 1 -- 172.21.15.80:0/2503506940 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a00ac410 con 0x7f62b00d59b0 2024-04-25T13:46:01.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b5354640 1 -- 172.21.15.80:0/2503506940 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a00ac7e0 con 0x7f62b00d59b0 2024-04-25T13:46:01.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b85ba640 1 -- 172.21.15.80:0/2503506940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d59b0 msgr2=0x7f62b00d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b85ba640 1 --2- 172.21.15.80:0/2503506940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d59b0 0x7f62b00d5dc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f62a00097c0 tx=0x7f62a001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b85ba640 1 -- 172.21.15.80:0/2503506940 shutdown_connections 2024-04-25T13:46:01.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b85ba640 1 --2- 172.21.15.80:0/2503506940 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d59b0 0x7f62b00d5dc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b85ba640 1 --2- 172.21.15.80:0/2503506940 >> v2:172.21.15.87:3300/0 conn(0x7f62b00d2e80 0x7f62b01607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b85ba640 1 -- 172.21.15.80:0/2503506940 >> 172.21.15.80:0/2503506940 conn(0x7f62b00737a0 msgr2=0x7f62b006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:01.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b85ba640 1 -- 172.21.15.80:0/2503506940 shutdown_connections 2024-04-25T13:46:01.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.499+0000 7f62b85ba640 1 -- 172.21.15.80:0/2503506940 wait complete. 2024-04-25T13:46:01.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b85ba640 1 Processor -- start 2024-04-25T13:46:01.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b85ba640 1 -- start start 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b85ba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b00d2e80 0x7f62b00c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b85ba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d4330 0x7f62b0156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b85ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b00d59b0 0x7f62b00c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b85ba640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00d3410 con 0x7f62b00d2e80 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b85ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f62b00d3290 con 0x7f62b00d59b0 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b85ba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f62b015df60 con 0x7f62b00d4330 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b6356640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b00d2e80 0x7f62b00c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b6356640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b00d2e80 0x7f62b00c9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38116/0 (socket says 172.21.15.80:38116) 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.503+0000 7f62b6356640 1 -- 172.21.15.80:0/3612329251 learned_addr learned my addr 172.21.15.80:0/3612329251 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:01.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b5b55640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d4330 0x7f62b0156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b6b57640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b00d59b0 0x7f62b00c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b6356640 1 -- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d4330 msgr2=0x7f62b0156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b6356640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d4330 0x7f62b0156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b6356640 1 -- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b00d59b0 msgr2=0x7f62b00c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b6356640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b00d59b0 0x7f62b00c9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b6356640 1 -- 172.21.15.80:0/3612329251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b00cd6d0 con 0x7f62b00d2e80 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b6356640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b00d2e80 0x7f62b00c9050 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00e960 tx=0x7f62ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62a6ffd640 1 -- 172.21.15.80:0/3612329251 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac017070 con 0x7f62b00d2e80 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62a6ffd640 1 -- 172.21.15.80:0/3612329251 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62ac00ca00 con 0x7f62b00d2e80 2024-04-25T13:46:01.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62a6ffd640 1 -- 172.21.15.80:0/3612329251 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62ac00cdd0 con 0x7f62b00d2e80 2024-04-25T13:46:01.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b00cdbd0 con 0x7f62b00d2e80 2024-04-25T13:46:01.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b00ce190 con 0x7f62b00d2e80 2024-04-25T13:46:01.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b006a3a0 con 0x7f62b00d2e80 2024-04-25T13:46:01.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62a6ffd640 1 -- 172.21.15.80:0/3612329251 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f62ac01a070 con 0x7f62b00d2e80 2024-04-25T13:46:01.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62a6ffd640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f629406f400 0x7f6294071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62a6ffd640 1 -- 172.21.15.80:0/3612329251 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f62ac08f920 con 0x7f62b00d2e80 2024-04-25T13:46:01.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.507+0000 7f62b5b55640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f629406f400 0x7f6294071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.511+0000 7f62b5b55640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f629406f400 0x7f6294071910 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f62a00ae000 tx=0x7f62a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.511+0000 7f62a6ffd640 1 -- 172.21.15.80:0/3612329251 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f62ac05c8f0 con 0x7f62b00d2e80 2024-04-25T13:46:01.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.679+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f62b00bbb90 con 0x7f62b00d2e80 2024-04-25T13:46:01.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62a6ffd640 1 -- 172.21.15.80:0/3612329251 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v110) ==== 81+0+34089 (secure 0 0 0) 0x7f62ac05c1f0 con 0x7f62b00d2e80 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 110, 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:45:57.407731+0000", 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:46:01.681 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:46:01.682 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 13, 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:46:01.683 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:46:01.684 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:46:01.685 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:46:01.686 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:46:01.687 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:46:01.688 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:46:01.689 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 12, 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.400550+0000", 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:46:01.690 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:46:01.691 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:46:01.692 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:46:01.693 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.694 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:46:01.695 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:46:01.696 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.697 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 13, 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:45:28.988401+0000", 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:46:01.698 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:46:01.699 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:46:01.700 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:46:01.701 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:46:01.702 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:46:01.703 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:46:01.704 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:46:01.705 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.706 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.707 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.708 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.709 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:46:01.710 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.712 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.713 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.714 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:46:01.715 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:46:01.717 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.718 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:46:01.719 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:46:01.722 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:46:01.723 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:46:01.724 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.725 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.726 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.727 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.728 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:46:01.729 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.730 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.731 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.733 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.734 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.735 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.736 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.737 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.738 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.739 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:46:01.740 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.742 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:46:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:46:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:46:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:46:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.743 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.744 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.745 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.746 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.747 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.748 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:46:01.749 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:46:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:46:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:46:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:46:01.750 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:46:01.751 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 101, 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.753 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.754 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.755 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:46:01.756 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.757 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.758 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:46:01.759 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.760 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.761 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.762 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:01.763 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:46:01.764 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.765 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:46:01.766 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:01.767 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f629406f400 msgr2=0x7f6294071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f629406f400 0x7f6294071910 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f62a00ae000 tx=0x7f62a0008040 comp rx=0 tx=0).stop 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b00d2e80 msgr2=0x7f62b00c9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b00d2e80 0x7f62b00c9050 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00e960 tx=0x7f62ac00ee20 comp rx=0 tx=0).stop 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 shutdown_connections 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f629406f400 0x7f6294071910 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f62b00d59b0 0x7f62b00c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f62b00d4330 0x7f62b0156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 --2- 172.21.15.80:0/3612329251 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f62b00d2e80 0x7f62b00c9050 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 >> 172.21.15.80:0/3612329251 conn(0x7f62b00737a0 msgr2=0x7f62b0162150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:01.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 shutdown_connections 2024-04-25T13:46:01.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.683+0000 7f62b85ba640 1 -- 172.21.15.80:0/3612329251 wait complete. 2024-04-25T13:46:01.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:01.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.919+0000 7fc9736e6640 1 Processor -- start 2024-04-25T13:46:01.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.919+0000 7fc9736e6640 1 -- start start 2024-04-25T13:46:01.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc9736e6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc96c0d59b0 0x7fc96c0d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc9736e6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c0d6300 con 0x7fc96c0d2e80 2024-04-25T13:46:01.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc9736e6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc96c161060 con 0x7fc96c0d4330 2024-04-25T13:46:01.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc9736e6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc96c161340 con 0x7fc96c0d59b0 2024-04-25T13:46:01.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc970c81640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc96c0d59b0 0x7fc96c0d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc971482640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc96c0d2e80 0x7fc96c1607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45864/0 (socket says 172.21.15.80:45864) 2024-04-25T13:46:01.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc971482640 1 -- 172.21.15.80:0/1350842620 learned_addr learned my addr 172.21.15.80:0/1350842620 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:01.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc970c81640 1 -- 172.21.15.80:0/1350842620 >> v1:172.21.15.87:6789/0 conn(0x7fc96c0d4330 legacy=0x7fc96c0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:01.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc970c81640 1 -- 172.21.15.80:0/1350842620 >> v1:172.21.15.80:6789/0 conn(0x7fc96c0d2e80 legacy=0x7fc96c1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc963fff640 1 -- 172.21.15.80:0/1350842620 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 171900067 0 0) 0x7fc968001120 con 0x7fc96c0d2e80 2024-04-25T13:46:01.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc970c81640 1 -- 172.21.15.80:0/1350842620 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c0d2680 con 0x7fc96c0d59b0 2024-04-25T13:46:01.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.923+0000 7fc970c81640 1 --2- 172.21.15.80:0/1350842620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d59b0 0x7fc96c0d5dc0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0097c0 tx=0x7fc95c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acb7aa91e96f11b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc963fff640 1 -- 172.21.15.80:0/1350842620 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0b0070 con 0x7fc96c0d59b0 2024-04-25T13:46:01.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc963fff640 1 -- 172.21.15.80:0/1350842620 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c0ab600 con 0x7fc96c0d59b0 2024-04-25T13:46:01.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc963fff640 1 -- 172.21.15.80:0/1350842620 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0a4e60 con 0x7fc96c0d59b0 2024-04-25T13:46:01.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 -- 172.21.15.80:0/1350842620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d59b0 msgr2=0x7fc96c0d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 --2- 172.21.15.80:0/1350842620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d59b0 0x7fc96c0d5dc0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0097c0 tx=0x7fc95c01c3d0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 -- 172.21.15.80:0/1350842620 shutdown_connections 2024-04-25T13:46:01.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 --2- 172.21.15.80:0/1350842620 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d59b0 0x7fc96c0d5dc0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 -- 172.21.15.80:0/1350842620 >> 172.21.15.80:0/1350842620 conn(0x7fc96c0737a0 msgr2=0x7fc96c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:01.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 -- 172.21.15.80:0/1350842620 shutdown_connections 2024-04-25T13:46:01.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 -- 172.21.15.80:0/1350842620 wait complete. 2024-04-25T13:46:01.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 Processor -- start 2024-04-25T13:46:01.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.927+0000 7fc9736e6640 1 -- start start 2024-04-25T13:46:01.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc96c0d2e80 0x7fc96c0c9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d4330 0x7fc96c1592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc96c0d59b0 0x7fc96c0b9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c15e0e0 con 0x7fc96c0d4330 2024-04-25T13:46:01.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc96c15df60 con 0x7fc96c0d59b0 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc96c16e950 con 0x7fc96c0d2e80 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d4330 0x7fc96c1592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d4330 0x7fc96c1592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38136/0 (socket says 172.21.15.80:38136) 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 -- 172.21.15.80:0/2881614313 learned_addr learned my addr 172.21.15.80:0/2881614313 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 -- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc96c0d2e80 msgr2=0x7fc96c0c9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc971482640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc96c0d2e80 0x7fc96c0c9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc971c83640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc96c0d59b0 0x7fc96c0b9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc96c0d2e80 0x7fc96c0c9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 -- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc96c0d59b0 msgr2=0x7fc96c0b9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc96c0d59b0 0x7fc96c0b9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:01.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 -- 172.21.15.80:0/2881614313 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc96c0ca020 con 0x7fc96c0d4330 2024-04-25T13:46:01.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc970c81640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d4330 0x7fc96c1592e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0ae040 tx=0x7fc95c01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc961ffb640 1 -- 172.21.15.80:0/2881614313 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0bd070 con 0x7fc96c0d4330 2024-04-25T13:46:01.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc961ffb640 1 -- 172.21.15.80:0/2881614313 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc95c0a4e60 con 0x7fc96c0d4330 2024-04-25T13:46:01.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc961ffb640 1 -- 172.21.15.80:0/2881614313 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc95c0b0040 con 0x7fc96c0d4330 2024-04-25T13:46:01.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc96c0ca300 con 0x7fc96c0d4330 2024-04-25T13:46:01.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc96c0ca8c0 con 0x7fc96c0d4330 2024-04-25T13:46:01.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96c06a3a0 con 0x7fc96c0d4330 2024-04-25T13:46:01.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc961ffb640 1 -- 172.21.15.80:0/2881614313 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc95c004230 con 0x7fc96c0d4330 2024-04-25T13:46:01.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc961ffb640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc95006f430 0x7fc950071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:01.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc961ffb640 1 -- 172.21.15.80:0/2881614313 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fc95c128d50 con 0x7fc96c0d4330 2024-04-25T13:46:01.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.931+0000 7fc971482640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc95006f430 0x7fc950071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:01.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.935+0000 7fc971482640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc95006f430 0x7fc950071940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc968000f70 tx=0x7fc968009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:01.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:01.939+0000 7fc961ffb640 1 -- 172.21.15.80:0/2881614313 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc95c12c030 con 0x7fc96c0d4330 2024-04-25T13:46:02.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc96c0b8930 con 0x7fc96c0d4330 2024-04-25T13:46:02.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc961ffb640 1 -- 172.21.15.80:0/2881614313 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7fc95c016050 con 0x7fc96c0d4330 2024-04-25T13:46:02.125 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:02.125 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:46:02.125 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:46:02.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc95006f430 msgr2=0x7fc950071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:02.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc95006f430 0x7fc950071940 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc968000f70 tx=0x7fc968009040 comp rx=0 tx=0).stop 2024-04-25T13:46:02.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d4330 msgr2=0x7fc96c1592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:02.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d4330 0x7fc96c1592e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fc95c0ae040 tx=0x7fc95c01ac10 comp rx=0 tx=0).stop 2024-04-25T13:46:02.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 shutdown_connections 2024-04-25T13:46:02.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc95006f430 0x7fc950071940 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc96c0d59b0 0x7fc96c0b9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc96c0d4330 0x7fc96c1592e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 --2- 172.21.15.80:0/2881614313 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc96c0d2e80 0x7fc96c0c9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 >> 172.21.15.80:0/2881614313 conn(0x7fc96c0737a0 msgr2=0x7fc96c160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:02.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 shutdown_connections 2024-04-25T13:46:02.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.127+0000 7fc9736e6640 1 -- 172.21.15.80:0/2881614313 wait complete. 2024-04-25T13:46:02.145 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:46:02.294 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":7044,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.582884425,"mdsmap_epoch":49,"osdmap_epoch":110,"osdmap_epoch_barrier":104,"uptime":36.390178339000002} 2024-04-25T13:46:02.294 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 7044, "mdsmap_epoch": 49, "osdmap_epoch": 110, "osdmap_epoch_barrier": 104, "rank_uptime": 31.582884425, "state": "up:active", "uptime": 36.390178339, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:46:02.295 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:46:02.295 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:02.295 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:02.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:02.301 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-04-25T13:46:02.615 INFO:teuthology.orchestra.run.smithi080.stderr:50+0 records in 2024-04-25T13:46:02.615 INFO:teuthology.orchestra.run.smithi080.stderr:50+0 records out 2024-04-25T13:46:02.615 INFO:teuthology.orchestra.run.smithi080.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.264175 s, 198 MB/s 2024-04-25T13:46:02.616 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:46:02.616 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:46:02.658 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:46:02.711 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:46:02.711 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:46:02.711 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:02.712 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:02.717 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:02.717 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:02.765 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:02.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f081340b640 1 Processor -- start 2024-04-25T13:46:02.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f081340b640 1 -- start start 2024-04-25T13:46:02.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f081340b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c161f90 0x7f080c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:02.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f081340b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f080c15ec40 0x7f080c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:02.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f081340b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080c0d0d70 con 0x7f080c0d0510 2024-04-25T13:46:02.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f081340b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f080c0d1050 con 0x7f080c15ec40 2024-04-25T13:46:02.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f081340b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f080c06dca0 con 0x7f080c161f90 2024-04-25T13:46:02.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f08119a8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f080c15ec40 0x7f080c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:02.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f08119a8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f080c15ec40 0x7f080c15f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38150/0 (socket says 172.21.15.80:38150) 2024-04-25T13:46:02.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f08119a8640 1 -- 172.21.15.80:0/4251259226 learned_addr learned my addr 172.21.15.80:0/4251259226 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:02.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.959+0000 7f08111a7640 1 --2- 172.21.15.80:0/4251259226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c161f90 0x7f080c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:02.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f08119a8640 1 -- 172.21.15.80:0/4251259226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c161f90 msgr2=0x7f080c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:02.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f08119a8640 1 --2- 172.21.15.80:0/4251259226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c161f90 0x7f080c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f08119a8640 1 -- 172.21.15.80:0/4251259226 >> v1:172.21.15.87:6789/0 conn(0x7f080c0d0510 legacy=0x7f080c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:02.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f08119a8640 1 -- 172.21.15.80:0/4251259226 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c06df80 con 0x7f080c15ec40 2024-04-25T13:46:02.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f08119a8640 1 --2- 172.21.15.80:0/4251259226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c15ec40 0x7f080c15f050 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0804009e50 tx=0x7f080401c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8599631c1c10906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:02.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f0803fff640 1 -- 172.21.15.80:0/4251259226 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040a56b0 con 0x7f080c15ec40 2024-04-25T13:46:02.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f0803fff640 1 -- 172.21.15.80:0/4251259226 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08040a5850 con 0x7f080c15ec40 2024-04-25T13:46:02.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f0803fff640 1 -- 172.21.15.80:0/4251259226 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040a5c20 con 0x7f080c15ec40 2024-04-25T13:46:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f081340b640 1 -- 172.21.15.80:0/4251259226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c15ec40 msgr2=0x7f080c15f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f081340b640 1 --2- 172.21.15.80:0/4251259226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c15ec40 0x7f080c15f050 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0804009e50 tx=0x7f080401c900 comp rx=0 tx=0).stop 2024-04-25T13:46:02.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f081340b640 1 -- 172.21.15.80:0/4251259226 shutdown_connections 2024-04-25T13:46:02.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f081340b640 1 --2- 172.21.15.80:0/4251259226 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c15ec40 0x7f080c15f050 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f081340b640 1 --2- 172.21.15.80:0/4251259226 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c161f90 0x7f080c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f081340b640 1 -- 172.21.15.80:0/4251259226 >> 172.21.15.80:0/4251259226 conn(0x7f080c06d080 msgr2=0x7f080c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:02.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f081340b640 1 -- 172.21.15.80:0/4251259226 shutdown_connections 2024-04-25T13:46:02.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.963+0000 7f081340b640 1 -- 172.21.15.80:0/4251259226 wait complete. 2024-04-25T13:46:02.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 Processor -- start 2024-04-25T13:46:02.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 -- start start 2024-04-25T13:46:02.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c0d0510 0x7f080c167460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:02.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f080c15ec40 0x7f080c142060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:02.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c161f90 0x7f080c1425a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:02.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f080c06f630 con 0x7f080c161f90 2024-04-25T13:46:02.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f080c06f4b0 con 0x7f080c15ec40 2024-04-25T13:46:02.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f080c0d0920 con 0x7f080c0d0510 2024-04-25T13:46:02.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c161f90 0x7f080c1425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c161f90 0x7f080c1425a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38152/0 (socket says 172.21.15.80:38152) 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 -- 172.21.15.80:0/277183696 learned_addr learned my addr 172.21.15.80:0/277183696 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08111a7640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c0d0510 0x7f080c167460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 -- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c0d0510 msgr2=0x7f080c167460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c0d0510 0x7f080c167460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 -- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f080c15ec40 msgr2=0x7f080c142060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08109a6640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f080c15ec40 0x7f080c142060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f080c15ec40 0x7f080c142060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 -- 172.21.15.80:0/277183696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c164090 con 0x7f080c161f90 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08111a7640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c0d0510 0x7f080c167460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f08119a8640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c161f90 0x7f080c1425a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f08040af040 tx=0x7f0804019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f0801ffb640 1 -- 172.21.15.80:0/277183696 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040bc020 con 0x7f080c161f90 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f0801ffb640 1 -- 172.21.15.80:0/277183696 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08040b8670 con 0x7f080c161f90 2024-04-25T13:46:02.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c164370 con 0x7f080c161f90 2024-04-25T13:46:02.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f0801ffb640 1 -- 172.21.15.80:0/277183696 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040b8a40 con 0x7f080c161f90 2024-04-25T13:46:02.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.967+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c164ae0 con 0x7f080c161f90 2024-04-25T13:46:02.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.971+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c06a3a0 con 0x7f080c161f90 2024-04-25T13:46:02.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.971+0000 7f0801ffb640 1 -- 172.21.15.80:0/277183696 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f08040bf020 con 0x7f080c161f90 2024-04-25T13:46:02.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.971+0000 7f0801ffb640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07f006f430 0x7f07f0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:02.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.971+0000 7f0801ffb640 1 -- 172.21.15.80:0/277183696 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f0804129de0 con 0x7f080c161f90 2024-04-25T13:46:02.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.971+0000 7f08111a7640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07f006f430 0x7f07f0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:02.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.971+0000 7f08111a7640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07f006f430 0x7f07f0071940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f080c06fa20 tx=0x7f0808008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:02.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:02.975+0000 7f0801ffb640 1 -- 172.21.15.80:0/277183696 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f08040165e0 con 0x7f080c161f90 2024-04-25T13:46:03.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.135+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f080c1626a0 con 0x7f080c161f90 2024-04-25T13:46:03.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.139+0000 7f0801ffb640 1 -- 172.21.15.80:0/277183696 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16600 (secure 0 0 0) 0x7f08040f6d60 con 0x7f080c161f90 2024-04-25T13:46:03.139 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:03.139 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":110,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:57.407731+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:45:28.400550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T13:45:28.988401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07f006f430 msgr2=0x7f07f0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07f006f430 0x7f07f0071940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f080c06fa20 tx=0x7f0808008040 comp rx=0 tx=0).stop 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c161f90 msgr2=0x7f080c1425a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c161f90 0x7f080c1425a0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f08040af040 tx=0x7f0804019700 comp rx=0 tx=0).stop 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 shutdown_connections 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07f006f430 0x7f07f0071940 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080c161f90 0x7f080c1425a0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f080c15ec40 0x7f080c142060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 --2- 172.21.15.80:0/277183696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080c0d0510 0x7f080c167460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 >> 172.21.15.80:0/277183696 conn(0x7f080c06d080 msgr2=0x7f080c160950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 shutdown_connections 2024-04-25T13:46:03.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.143+0000 7f081340b640 1 -- 172.21.15.80:0/277183696 wait complete. 2024-04-25T13:46:03.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:45:28.400550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:45:28.988401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:46:03.160 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:46:03.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.315+0000 7f9258159640 1 Processor -- start 2024-04-25T13:46:03.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.315+0000 7f9258159640 1 -- start start 2024-04-25T13:46:03.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9258159640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92500b5330 0x7f92500d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9258159640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 0x7f9250161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9258159640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92501624b0 con 0x7f92500d0ff0 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9258159640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f92501629b0 con 0x7f92500b5330 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9258159640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92500b5740 con 0x7f9250161b60 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9255ef5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92500b5330 0x7f92500d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9255ef5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92500b5330 0x7f92500d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38154/0 (socket says 172.21.15.80:38154) 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9255ef5640 1 -- 172.21.15.80:0/2405922938 learned_addr learned my addr 172.21.15.80:0/2405922938 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f92566f6640 1 --2- 172.21.15.80:0/2405922938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 0x7f9250161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9255ef5640 1 -- 172.21.15.80:0/2405922938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 msgr2=0x7f9250161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9255ef5640 1 --2- 172.21.15.80:0/2405922938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 0x7f9250161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9255ef5640 1 -- 172.21.15.80:0/2405922938 >> v1:172.21.15.87:6789/0 conn(0x7f92500d0ff0 legacy=0x7f925006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.319+0000 7f9255ef5640 1 -- 172.21.15.80:0/2405922938 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925006dd10 con 0x7f92500b5330 2024-04-25T13:46:03.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9255ef5640 1 --2- 172.21.15.80:0/2405922938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500b5330 0x7f92500d0ab0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f924c009870 tx=0x7f924c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415316eb66c02a88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:03.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9254ef3640 1 -- 172.21.15.80:0/2405922938 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0b1070 con 0x7f92500b5330 2024-04-25T13:46:03.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9254ef3640 1 -- 172.21.15.80:0/2405922938 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924c0ac620 con 0x7f92500b5330 2024-04-25T13:46:03.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9254ef3640 1 -- 172.21.15.80:0/2405922938 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0ac9f0 con 0x7f92500b5330 2024-04-25T13:46:03.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 -- 172.21.15.80:0/2405922938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500b5330 msgr2=0x7f92500d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 --2- 172.21.15.80:0/2405922938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500b5330 0x7f92500d0ab0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f924c009870 tx=0x7f924c01c4d0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 -- 172.21.15.80:0/2405922938 shutdown_connections 2024-04-25T13:46:03.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 --2- 172.21.15.80:0/2405922938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 0x7f9250161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 --2- 172.21.15.80:0/2405922938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500b5330 0x7f92500d0ab0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 -- 172.21.15.80:0/2405922938 >> 172.21.15.80:0/2405922938 conn(0x7f92500b4160 msgr2=0x7f92500d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:03.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 -- 172.21.15.80:0/2405922938 shutdown_connections 2024-04-25T13:46:03.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 -- 172.21.15.80:0/2405922938 wait complete. 2024-04-25T13:46:03.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 Processor -- start 2024-04-25T13:46:03.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.323+0000 7f9258159640 1 -- start start 2024-04-25T13:46:03.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500d0ff0 0x7f9250143fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 0x7f9250073740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92500af8b0 0x7f92500afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f92500b0cd0 con 0x7f92500d0ff0 2024-04-25T13:46:03.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f925006e890 con 0x7f92500af8b0 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92500b0e50 con 0x7f9250161b60 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500d0ff0 0x7f9250143fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500d0ff0 0x7f9250143fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38160/0 (socket says 172.21.15.80:38160) 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 -- 172.21.15.80:0/3027589285 learned_addr learned my addr 172.21.15.80:0/3027589285 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f92566f6640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92500af8b0 0x7f92500afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f92556f4640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 0x7f9250073740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 -- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 msgr2=0x7f9250073740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 0x7f9250073740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 -- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92500af8b0 msgr2=0x7f92500afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92500af8b0 0x7f92500afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 -- 172.21.15.80:0/3027589285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f925015f460 con 0x7f92500d0ff0 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9255ef5640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500d0ff0 0x7f9250143fb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f924c0af040 tx=0x7f924c019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:03.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f92467fc640 1 -- 172.21.15.80:0/3027589285 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0bc070 con 0x7f92500d0ff0 2024-04-25T13:46:03.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f92467fc640 1 -- 172.21.15.80:0/3027589285 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f924c0b7430 con 0x7f92500d0ff0 2024-04-25T13:46:03.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f92467fc640 1 -- 172.21.15.80:0/3027589285 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f924c0b1040 con 0x7f92500d0ff0 2024-04-25T13:46:03.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92500b0200 con 0x7f92500d0ff0 2024-04-25T13:46:03.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9250072a10 con 0x7f92500d0ff0 2024-04-25T13:46:03.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f925006a3a0 con 0x7f92500d0ff0 2024-04-25T13:46:03.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f92467fc640 1 -- 172.21.15.80:0/3027589285 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f924c0bf070 con 0x7f92500d0ff0 2024-04-25T13:46:03.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.327+0000 7f92467fc640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f923406f400 0x7f9234071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.331+0000 7f92467fc640 1 -- 172.21.15.80:0/3027589285 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f924c128820 con 0x7f92500d0ff0 2024-04-25T13:46:03.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.331+0000 7f92556f4640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f923406f400 0x7f9234071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.331+0000 7f92556f4640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f923406f400 0x7f9234071910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9250162320 tx=0x7f9240009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:03.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.335+0000 7f92467fc640 1 -- 172.21.15.80:0/3027589285 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92500b0200 con 0x7f92500d0ff0 2024-04-25T13:46:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.495+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f925006fd00 con 0x7f92500d0ff0 2024-04-25T13:46:03.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.495+0000 7f92467fc640 1 -- 172.21.15.80:0/3027589285 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f924c016150 con 0x7f92500d0ff0 2024-04-25T13:46:03.497 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:46:03.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f923406f400 msgr2=0x7f9234071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f923406f400 0x7f9234071910 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9250162320 tx=0x7f9240009040 comp rx=0 tx=0).stop 2024-04-25T13:46:03.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500d0ff0 msgr2=0x7f9250143fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500d0ff0 0x7f9250143fb0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f924c0af040 tx=0x7f924c019270 comp rx=0 tx=0).stop 2024-04-25T13:46:03.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 shutdown_connections 2024-04-25T13:46:03.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f923406f400 0x7f9234071910 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92500af8b0 0x7f92500afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9250161b60 0x7f9250073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 --2- 172.21.15.80:0/3027589285 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92500d0ff0 0x7f9250143fb0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 >> 172.21.15.80:0/3027589285 conn(0x7f92500b4160 msgr2=0x7f92500b2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:03.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 shutdown_connections 2024-04-25T13:46:03.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.499+0000 7f9258159640 1 -- 172.21.15.80:0/3027589285 wait complete. 2024-04-25T13:46:03.519 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:46:03.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4ff1567640 1 Processor -- start 2024-04-25T13:46:03.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4ff1567640 1 -- start start 2024-04-25T13:46:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4ff1567640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4fec071e60 0x7f4fec1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4ff1567640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fec06fbf0 con 0x7f4fec161170 2024-04-25T13:46:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4ff1567640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4fec0b0a20 con 0x7f4fec073c40 2024-04-25T13:46:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4ff1567640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4fec0b1350 con 0x7f4fec071e60 2024-04-25T13:46:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feb7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4fec161170 0x7f4fec0b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45872/0 (socket says 172.21.15.80:45872) 2024-04-25T13:46:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feb7fe640 1 -- 172.21.15.80:0/1528269468 learned_addr learned my addr 172.21.15.80:0/1528269468 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feaffd640 1 --2- 172.21.15.80:0/1528269468 >> v2:172.21.15.87:3300/0 conn(0x7f4fec071e60 0x7f4fec1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 27083185 0 0) 0x7f4fec06fbf0 con 0x7f4fec161170 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4fd4003530 con 0x7f4fec161170 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3085491091 0 0) 0x7f4fd4003530 con 0x7f4fec161170 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 >> v2:172.21.15.87:3300/0 conn(0x7f4fec071e60 msgr2=0x7f4fec1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 --2- 172.21.15.80:0/1528269468 >> v2:172.21.15.87:3300/0 conn(0x7f4fec071e60 0x7f4fec1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 >> v1:172.21.15.87:6789/0 conn(0x7f4fec073c40 legacy=0x7f4fec160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec0b0d00 con 0x7f4fec161170 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4fe4001a00 con 0x7f4fec161170 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 >> v1:172.21.15.80:6789/0 conn(0x7f4fec161170 legacy=0x7f4fec0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fd40039c0 0x7f4fd4003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4004810 0x7f4fd4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd4005280 0x7f4fd400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec0b1350 con 0x7f4fd40039c0 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd400c1a0 con 0x7f4fd4004810 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd400c3a0 con 0x7f4fd4005280 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feaffd640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fd40039c0 0x7f4fd4003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feb7fe640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd4005280 0x7f4fd400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fea7fc640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4004810 0x7f4fd4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feaffd640 1 -- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd4005280 msgr2=0x7f4fd400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feaffd640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd4005280 0x7f4fd400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feaffd640 1 -- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4004810 msgr2=0x7f4fd4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feaffd640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4004810 0x7f4fd4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feaffd640 1 -- 172.21.15.80:0/1528269468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd4003530 con 0x7f4fd40039c0 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feb7fe640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd4005280 0x7f4fd400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:03.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4fea7fc640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fd4004810 0x7f4fd4004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:03.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.683+0000 7f4feb7fe640 1 -- 172.21.15.80:0/1528269468 reap_dead start 2024-04-25T13:46:03.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4feaffd640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fd40039c0 0x7f4fd4003e50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009ae0 tx=0x7f4fdc01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c152f5ed55d6b44c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:03.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0096b0 con 0x7f4fd40039c0 2024-04-25T13:46:03.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0a6450 con 0x7f4fd40039c0 2024-04-25T13:46:03.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4fe9ffb640 1 -- 172.21.15.80:0/1528269468 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc0a6840 con 0x7f4fd40039c0 2024-04-25T13:46:03.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4ff1567640 1 -- 172.21.15.80:0/1528269468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fd40039c0 msgr2=0x7f4fd4003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4ff1567640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fd40039c0 0x7f4fd4003e50 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc009ae0 tx=0x7f4fdc01ca70 comp rx=0 tx=0).stop 2024-04-25T13:46:03.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4ff1567640 1 -- 172.21.15.80:0/1528269468 shutdown_connections 2024-04-25T13:46:03.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4ff1567640 1 --2- 172.21.15.80:0/1528269468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fd40039c0 0x7f4fd4003e50 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4ff1567640 1 -- 172.21.15.80:0/1528269468 >> 172.21.15.80:0/1528269468 conn(0x7f4fec0b4160 msgr2=0x7f4fec0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:03.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4ff1567640 1 -- 172.21.15.80:0/1528269468 shutdown_connections 2024-04-25T13:46:03.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.687+0000 7f4ff1567640 1 -- 172.21.15.80:0/1528269468 wait complete. 2024-04-25T13:46:03.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 Processor -- start 2024-04-25T13:46:03.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 -- start start 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd40039c0 0x7f4fec1f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fec1475e0 0x7f4fec141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fec152fa0 0x7f4fec1f9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec0b4ba0 con 0x7f4fec152fa0 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec06f9a0 con 0x7f4fec1475e0 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4fec0b4d20 con 0x7f4fd40039c0 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fec152fa0 0x7f4fec1f9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fec152fa0 0x7f4fec1f9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38180/0 (socket says 172.21.15.80:38180) 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 -- 172.21.15.80:0/343711583 learned_addr learned my addr 172.21.15.80:0/343711583 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feaffd640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd40039c0 0x7f4fec1f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4fea7fc640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fec1475e0 0x7f4fec141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 -- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd40039c0 msgr2=0x7f4fec1f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd40039c0 0x7f4fec1f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 -- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fec1475e0 msgr2=0x7f4fec141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fec1475e0 0x7f4fec141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 -- 172.21.15.80:0/343711583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec1f65c0 con 0x7f4fec152fa0 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4fea7fc640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fec1475e0 0x7f4fec141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4feb7fe640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fec152fa0 0x7f4fec1f9d20 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4fec071df0 tx=0x7f4fe400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4fd37fe640 1 -- 172.21.15.80:0/343711583 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4004d70 con 0x7f4fec152fa0 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4fd37fe640 1 -- 172.21.15.80:0/343711583 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4fe400bb20 con 0x7f4fec152fa0 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4fd37fe640 1 -- 172.21.15.80:0/343711583 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4007070 con 0x7f4fec152fa0 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec151f00 con 0x7f4fec152fa0 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.691+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec1521e0 con 0x7f4fec152fa0 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.695+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fec06a3a0 con 0x7f4fec152fa0 2024-04-25T13:46:03.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.695+0000 7f4fd37fe640 1 -- 172.21.15.80:0/343711583 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4fe401b020 con 0x7f4fec152fa0 2024-04-25T13:46:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.695+0000 7f4fd37fe640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4fd806f430 0x7f4fd8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.695+0000 7f4fd37fe640 1 -- 172.21.15.80:0/343711583 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f4fe408f3b0 con 0x7f4fec152fa0 2024-04-25T13:46:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.695+0000 7f4feaffd640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4fd806f430 0x7f4fd8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:03.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.695+0000 7f4feaffd640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4fd806f430 0x7f4fd8071940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0ad000 tx=0x7f4fdc0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:03.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.699+0000 7f4fd37fe640 1 -- 172.21.15.80:0/343711583 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4fec1f65c0 con 0x7f4fec152fa0 2024-04-25T13:46:03.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.855+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4fec15e160 con 0x7f4fec152fa0 2024-04-25T13:46:03.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4fd37fe640 1 -- 172.21.15.80:0/343711583 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7f4fe405c2a0 con 0x7f4fec152fa0 2024-04-25T13:46:03.856 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:46:03.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4fd806f430 msgr2=0x7f4fd8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4fd806f430 0x7f4fd8071940 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc0ad000 tx=0x7f4fdc0046d0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fec152fa0 msgr2=0x7f4fec1f9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fec152fa0 0x7f4fec1f9d20 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4fec071df0 tx=0x7f4fe400b4a0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 shutdown_connections 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4fd806f430 0x7f4fd8071940 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4fec152fa0 0x7f4fec1f9d20 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4fec1475e0 0x7f4fec141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 --2- 172.21.15.80:0/343711583 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4fd40039c0 0x7f4fec1f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 >> 172.21.15.80:0/343711583 conn(0x7f4fec0b4160 msgr2=0x7f4fec0b2010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 shutdown_connections 2024-04-25T13:46:03.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:03.859+0000 7f4ff1567640 1 -- 172.21.15.80:0/343711583 wait complete. 2024-04-25T13:46:03.876 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:46:04.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.031+0000 7fb0322a6640 1 Processor -- start 2024-04-25T13:46:04.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.031+0000 7fb0322a6640 1 -- start start 2024-04-25T13:46:04.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb0322a6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c161170 0x7fb02c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb0322a6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb02c073c40 0x7fb02c0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb0322a6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb02c06fbf0 con 0x7fb02c071e60 2024-04-25T13:46:04.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb0322a6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb02c0b1350 con 0x7fb02c073c40 2024-04-25T13:46:04.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb0322a6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c06f9a0 con 0x7fb02c161170 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb02c071e60 0x7fb02c160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45886/0 (socket says 172.21.15.80:45886) 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 -- 172.21.15.80:0/4165020662 learned_addr learned my addr 172.21.15.80:0/4165020662 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02b7fe640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c161170 0x7fb02c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb030843640 1 --2- 172.21.15.80:0/4165020662 >> v2:172.21.15.87:3300/0 conn(0x7fb02c073c40 0x7fb02c0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2431885695 0 0) 0x7fb02c06fbf0 con 0x7fb02c071e60 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb014003530 con 0x7fb02c071e60 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 592642128 0 0) 0x7fb014003530 con 0x7fb02c071e60 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c161170 msgr2=0x7fb02c1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c161170 0x7fb02c1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 >> v2:172.21.15.87:3300/0 conn(0x7fb02c073c40 msgr2=0x7fb02c0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:04.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 --2- 172.21.15.80:0/4165020662 >> v2:172.21.15.87:3300/0 conn(0x7fb02c073c40 0x7fb02c0b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c0b0a20 con 0x7fb02c071e60 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb01c001680 con 0x7fb02c071e60 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 >> v1:172.21.15.80:6789/0 conn(0x7fb02c071e60 legacy=0x7fb02c160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0140039c0 0x7fb014003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb014004810 0x7fb014004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014005160 0x7fb01400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c0b1350 con 0x7fb0140039c0 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c06f9a0 con 0x7fb014005160 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb01400bf10 con 0x7fb014004810 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0140039c0 0x7fb014003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb030843640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014005160 0x7fb01400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02b7fe640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb014004810 0x7fb014004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 -- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb014004810 msgr2=0x7fb014004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:04.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb014004810 0x7fb014004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 -- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014005160 msgr2=0x7fb01400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:04.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014005160 0x7fb01400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 -- 172.21.15.80:0/4165020662 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb014003530 con 0x7fb0140039c0 2024-04-25T13:46:04.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02b7fe640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb014004810 0x7fb014004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:04.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb030843640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb014005160 0x7fb01400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:04.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02bfff640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0140039c0 0x7fb014003e30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c240 tx=0x7fb01c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e4a593dcd5f85d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.035+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c007d40 con 0x7fb0140039c0 2024-04-25T13:46:04.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb01c0a9020 con 0x7fb0140039c0 2024-04-25T13:46:04.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb02affd640 1 -- 172.21.15.80:0/4165020662 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c0a5b70 con 0x7fb0140039c0 2024-04-25T13:46:04.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 -- 172.21.15.80:0/4165020662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0140039c0 msgr2=0x7fb014003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 --2- 172.21.15.80:0/4165020662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0140039c0 0x7fb014003e30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00c240 tx=0x7fb01c01cc70 comp rx=0 tx=0).stop 2024-04-25T13:46:04.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb030843640 1 -- 172.21.15.80:0/4165020662 reap_dead start 2024-04-25T13:46:04.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 -- 172.21.15.80:0/4165020662 shutdown_connections 2024-04-25T13:46:04.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 -- 172.21.15.80:0/4165020662 >> 172.21.15.80:0/4165020662 conn(0x7fb02c0b4160 msgr2=0x7fb02c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:04.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 -- 172.21.15.80:0/4165020662 shutdown_connections 2024-04-25T13:46:04.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 -- 172.21.15.80:0/4165020662 wait complete. 2024-04-25T13:46:04.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 Processor -- start 2024-04-25T13:46:04.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 -- start start 2024-04-25T13:46:04.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0140039c0 0x7fb02c15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.039+0000 7fb0322a6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c14a7b0 0x7fb02c147210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb0322a6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02c06e840 0x7fb02c06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb0322a6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c161170 con 0x7fb02c06e840 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb0322a6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c074050 con 0x7fb0140039c0 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb0322a6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb02c1612f0 con 0x7fb02c14a7b0 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02c06e840 0x7fb02c06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02c06e840 0x7fb02c06ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38196/0 (socket says 172.21.15.80:38196) 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 -- 172.21.15.80:0/364632923 learned_addr learned my addr 172.21.15.80:0/364632923 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb02b7fe640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c14a7b0 0x7fb02c147210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb02bfff640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0140039c0 0x7fb02c15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 -- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c14a7b0 msgr2=0x7fb02c147210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c14a7b0 0x7fb02c147210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 -- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0140039c0 msgr2=0x7fb02c15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0140039c0 0x7fb02c15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 -- 172.21.15.80:0/364632923 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c06f190 con 0x7fb02c06e840 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb02bfff640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0140039c0 0x7fb02c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb030843640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02c06e840 0x7fb02c06ec50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb02c161a70 tx=0x7fb02400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb028ff9640 1 -- 172.21.15.80:0/364632923 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024016620 con 0x7fb02c06e840 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb028ff9640 1 -- 172.21.15.80:0/364632923 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb02401b070 con 0x7fb02c06e840 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb028ff9640 1 -- 172.21.15.80:0/364632923 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb024016a20 con 0x7fb02c06e840 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c15d460 con 0x7fb02c06e840 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c15da20 con 0x7fb02c06e840 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02c15d1d0 con 0x7fb02c06e840 2024-04-25T13:46:04.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb028ff9640 1 -- 172.21.15.80:0/364632923 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb024016bc0 con 0x7fb02c06e840 2024-04-25T13:46:04.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb028ff9640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb01806f430 0x7fb018071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb028ff9640 1 -- 172.21.15.80:0/364632923 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fb02408fff0 con 0x7fb02c06e840 2024-04-25T13:46:04.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.043+0000 7fb02bfff640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb01806f430 0x7fb018071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.047+0000 7fb02bfff640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb01806f430 0x7fb018071940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0aa000 tx=0x7fb02c06fdc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.047+0000 7fb028ff9640 1 -- 172.21.15.80:0/364632923 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb024025020 con 0x7fb02c06e840 2024-04-25T13:46:04.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.199+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb02c06a3a0 con 0x7fb02c06e840 2024-04-25T13:46:04.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.199+0000 7fb028ff9640 1 -- 172.21.15.80:0/364632923 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7fb02c06f190 con 0x7fb02c06e840 2024-04-25T13:46:04.199 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb01806f430 msgr2=0x7fb018071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb01806f430 0x7fb018071940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb01c0aa000 tx=0x7fb02c06fdc0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02c06e840 msgr2=0x7fb02c06ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02c06e840 0x7fb02c06ec50 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fb02c161a70 tx=0x7fb02400f520 comp rx=0 tx=0).stop 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 shutdown_connections 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb01806f430 0x7fb018071940 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02c06e840 0x7fb02c06ec50 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02c14a7b0 0x7fb02c147210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 --2- 172.21.15.80:0/364632923 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0140039c0 0x7fb02c15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 >> 172.21.15.80:0/364632923 conn(0x7fb02c0b4160 msgr2=0x7fb02c0aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:04.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 shutdown_connections 2024-04-25T13:46:04.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.203+0000 7fb0322a6640 1 -- 172.21.15.80:0/364632923 wait complete. 2024-04-25T13:46:04.218 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:46:04.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60fc6be640 1 Processor -- start 2024-04-25T13:46:04.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60fc6be640 1 -- start start 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60fc6be640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 0x7f60f406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60fc6be640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60f40cfb60 0x7f60f40cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60fc6be640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f60f40d04b0 con 0x7f60f415d630 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60fc6be640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f60f4143920 con 0x7f60f40cfb60 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60fc6be640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f60f406f350 con 0x7f60f4161630 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60f9c59640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60f40cfb60 0x7f60f40cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60f9c59640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f60f40cfb60 0x7f60f40cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38202/0 (socket says 172.21.15.80:38202) 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60f9c59640 1 -- 172.21.15.80:0/3440974037 learned_addr learned my addr 172.21.15.80:0/3440974037 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60fa45a640 1 --2- 172.21.15.80:0/3440974037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 0x7f60f406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60f9c59640 1 -- 172.21.15.80:0/3440974037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 msgr2=0x7f60f406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:04.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60f9c59640 1 --2- 172.21.15.80:0/3440974037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 0x7f60f406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60f9c59640 1 -- 172.21.15.80:0/3440974037 >> v1:172.21.15.87:6789/0 conn(0x7f60f415d630 legacy=0x7f60f415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:04.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.355+0000 7f60f9c59640 1 -- 172.21.15.80:0/3440974037 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f406cf90 con 0x7f60f40cfb60 2024-04-25T13:46:04.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60f9c59640 1 --2- 172.21.15.80:0/3440974037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f40cfb60 0x7f60f40cff70 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009870 tx=0x7f60e401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59905d624385c871 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60f9458640 1 -- 172.21.15.80:0/3440974037 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e40b5070 con 0x7f60f40cfb60 2024-04-25T13:46:04.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60f9458640 1 -- 172.21.15.80:0/3440974037 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e40a87f0 con 0x7f60f40cfb60 2024-04-25T13:46:04.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60f9458640 1 -- 172.21.15.80:0/3440974037 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e40a8bc0 con 0x7f60f40cfb60 2024-04-25T13:46:04.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60fc6be640 1 -- 172.21.15.80:0/3440974037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f40cfb60 msgr2=0x7f60f40cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60fc6be640 1 --2- 172.21.15.80:0/3440974037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f40cfb60 0x7f60f40cff70 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009870 tx=0x7f60e401eb30 comp rx=0 tx=0).stop 2024-04-25T13:46:04.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60fc6be640 1 -- 172.21.15.80:0/3440974037 shutdown_connections 2024-04-25T13:46:04.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60fc6be640 1 --2- 172.21.15.80:0/3440974037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f40cfb60 0x7f60f40cff70 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60fc6be640 1 --2- 172.21.15.80:0/3440974037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 0x7f60f406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60fc6be640 1 -- 172.21.15.80:0/3440974037 >> 172.21.15.80:0/3440974037 conn(0x7f60f40d33a0 msgr2=0x7f60f40cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:04.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60fc6be640 1 -- 172.21.15.80:0/3440974037 shutdown_connections 2024-04-25T13:46:04.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.359+0000 7f60fc6be640 1 -- 172.21.15.80:0/3440974037 wait complete. 2024-04-25T13:46:04.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 Processor -- start 2024-04-25T13:46:04.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 -- start start 2024-04-25T13:46:04.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60f40cfb60 0x7f60f413b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f415d630 0x7f60f4136680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 0x7f60f413f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4161110 con 0x7f60f415d630 2024-04-25T13:46:04.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4160e30 con 0x7f60f40cfb60 2024-04-25T13:46:04.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f60f4161a40 con 0x7f60f4161630 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f415d630 0x7f60f4136680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f415d630 0x7f60f4136680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38218/0 (socket says 172.21.15.80:38218) 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 -- 172.21.15.80:0/2805220333 learned_addr learned my addr 172.21.15.80:0/2805220333 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fa45a640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60f40cfb60 0x7f60f413b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fac5b640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 0x7f60f413f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 -- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 msgr2=0x7f60f413f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 0x7f60f413f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 -- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60f40cfb60 msgr2=0x7f60f413b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60f40cfb60 0x7f60f413b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 -- 172.21.15.80:0/2805220333 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f415b040 con 0x7f60f415d630 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60f9c59640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f415d630 0x7f60f4136680 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f60e40b3040 tx=0x7f60e401b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60eaffd640 1 -- 172.21.15.80:0/2805220333 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e40c0070 con 0x7f60f415d630 2024-04-25T13:46:04.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60eaffd640 1 -- 172.21.15.80:0/2805220333 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f60e40b0930 con 0x7f60f415d630 2024-04-25T13:46:04.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60eaffd640 1 -- 172.21.15.80:0/2805220333 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e40b5070 con 0x7f60f415d630 2024-04-25T13:46:04.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f4137300 con 0x7f60f415d630 2024-04-25T13:46:04.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f41378c0 con 0x7f60f415d630 2024-04-25T13:46:04.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.363+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f406a3a0 con 0x7f60f415d630 2024-04-25T13:46:04.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.367+0000 7f60eaffd640 1 -- 172.21.15.80:0/2805220333 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f60e40c3070 con 0x7f60f415d630 2024-04-25T13:46:04.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.367+0000 7f60eaffd640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f60d806f400 0x7f60d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.367+0000 7f60eaffd640 1 -- 172.21.15.80:0/2805220333 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f60e412c1c0 con 0x7f60f415d630 2024-04-25T13:46:04.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.367+0000 7f60fa45a640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f60d806f400 0x7f60d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.367+0000 7f60fa45a640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f60d806f400 0x7f60d8071910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f60f0003fd0 tx=0x7f60f00039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.371+0000 7f60eaffd640 1 -- 172.21.15.80:0/2805220333 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f60f40d04b0 con 0x7f60f415d630 2024-04-25T13:46:04.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.539+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f60f40d0c40 con 0x7f60f415d630 2024-04-25T13:46:04.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.539+0000 7f60eaffd640 1 -- 172.21.15.80:0/2805220333 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) ==== 80+0+0 (secure 0 0 0) 0x7f60e40f90b0 con 0x7f60f415d630 2024-04-25T13:46:04.539 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:46:04.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f60d806f400 msgr2=0x7f60d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f60d806f400 0x7f60d8071910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f60f0003fd0 tx=0x7f60f00039a0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f415d630 msgr2=0x7f60f4136680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f415d630 0x7f60f4136680 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f60e40b3040 tx=0x7f60e401b8d0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 shutdown_connections 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f60d806f400 0x7f60d8071910 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f60f4161630 0x7f60f413f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f60f415d630 0x7f60f4136680 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 --2- 172.21.15.80:0/2805220333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f60f40cfb60 0x7f60f413b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 >> 172.21.15.80:0/2805220333 conn(0x7f60f40d33a0 msgr2=0x7f60f415b640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 shutdown_connections 2024-04-25T13:46:04.542 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.543+0000 7f60fc6be640 1 -- 172.21.15.80:0/2805220333 wait complete. 2024-04-25T13:46:04.560 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:46:04.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:04.560 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:04.572 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[35046]: fuse finished with error 0 and tester_r 0 2024-04-25T13:46:04.576 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:46:04.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:04.577 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:46:04.741 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:46:04.742 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:46:04.742 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:04.748 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:46:04.748 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:04.798 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:04.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.947+0000 7fd7a8419640 1 Processor -- start 2024-04-25T13:46:04.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.947+0000 7fd7a8419640 1 -- start start 2024-04-25T13:46:04.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a8419640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7a015bcc0 0x7fd7a00b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a8419640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7a00b7d10 0x7fd7a00b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a8419640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a0068590 0x7fd7a00689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a8419640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a00b2210 con 0x7fd7a015bcc0 2024-04-25T13:46:04.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a8419640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd7a00b2440 con 0x7fd7a00b7d10 2024-04-25T13:46:04.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a8419640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a015f170 con 0x7fd7a0068590 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a61b5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7a015bcc0 0x7fd7a00b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a61b5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7a015bcc0 0x7fd7a00b1cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38230/0 (socket says 172.21.15.80:38230) 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a61b5640 1 -- 172.21.15.80:0/1798343862 learned_addr learned my addr 172.21.15.80:0/1798343862 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a69b6640 1 --2- 172.21.15.80:0/1798343862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a0068590 0x7fd7a00689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a59b4640 1 --2- 172.21.15.80:0/1798343862 >> v2:172.21.15.87:3300/0 conn(0x7fd7a00b7d10 0x7fd7a00b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a61b5640 1 -- 172.21.15.80:0/1798343862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a0068590 msgr2=0x7fd7a00689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a61b5640 1 --2- 172.21.15.80:0/1798343862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a0068590 0x7fd7a00689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a61b5640 1 -- 172.21.15.80:0/1798343862 >> v2:172.21.15.87:3300/0 conn(0x7fd7a00b7d10 msgr2=0x7fd7a00b8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:04.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a61b5640 1 --2- 172.21.15.80:0/1798343862 >> v2:172.21.15.87:3300/0 conn(0x7fd7a00b7d10 0x7fd7a00b8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.951+0000 7fd7a61b5640 1 -- 172.21.15.80:0/1798343862 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a006c9b0 con 0x7fd7a015bcc0 2024-04-25T13:46:04.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a61b5640 1 --2- 172.21.15.80:0/1798343862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a015bcc0 0x7fd7a00b1cd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0071e80 tx=0x7fd79c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad7351c3262f2a79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a51b3640 1 -- 172.21.15.80:0/1798343862 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0b0070 con 0x7fd7a015bcc0 2024-04-25T13:46:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a51b3640 1 -- 172.21.15.80:0/1798343862 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79c0ab600 con 0x7fd7a015bcc0 2024-04-25T13:46:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a51b3640 1 -- 172.21.15.80:0/1798343862 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0ab9d0 con 0x7fd7a015bcc0 2024-04-25T13:46:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 -- 172.21.15.80:0/1798343862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a015bcc0 msgr2=0x7fd7a00b1cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1798343862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a015bcc0 0x7fd7a00b1cd0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0071e80 tx=0x7fd79c01c520 comp rx=0 tx=0).stop 2024-04-25T13:46:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 -- 172.21.15.80:0/1798343862 shutdown_connections 2024-04-25T13:46:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1798343862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a0068590 0x7fd7a00689a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1798343862 >> v2:172.21.15.87:3300/0 conn(0x7fd7a00b7d10 0x7fd7a00b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1798343862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a015bcc0 0x7fd7a00b1cd0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 -- 172.21.15.80:0/1798343862 >> 172.21.15.80:0/1798343862 conn(0x7fd7a00cf450 msgr2=0x7fd7a00cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:04.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 -- 172.21.15.80:0/1798343862 shutdown_connections 2024-04-25T13:46:04.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 -- 172.21.15.80:0/1798343862 wait complete. 2024-04-25T13:46:04.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 Processor -- start 2024-04-25T13:46:04.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.955+0000 7fd7a8419640 1 -- start start 2024-04-25T13:46:04.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a0068590 0x7fd7a01561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7a00b7d10 0x7fd7a00ab530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a015bcc0 0x7fd7a0159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a015c0d0 con 0x7fd7a0068590 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a015f600 con 0x7fd7a00b7d10 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd7a00692f0 con 0x7fd7a015bcc0 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a0068590 0x7fd7a01561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a0068590 0x7fd7a01561f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38234/0 (socket says 172.21.15.80:38234) 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 -- 172.21.15.80:0/1996924738 learned_addr learned my addr 172.21.15.80:0/1996924738 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a59b4640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7a00b7d10 0x7fd7a00ab530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 -- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a015bcc0 msgr2=0x7fd7a0159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a015bcc0 0x7fd7a0159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 -- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7a00b7d10 msgr2=0x7fd7a00ab530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7a00b7d10 0x7fd7a00ab530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 -- 172.21.15.80:0/1996924738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a0161b50 con 0x7fd7a0068590 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a61b5640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a0068590 0x7fd7a01561f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0ae040 tx=0x7fd79c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd796ffd640 1 -- 172.21.15.80:0/1996924738 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0b0040 con 0x7fd7a0068590 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd796ffd640 1 -- 172.21.15.80:0/1996924738 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd79c0b9070 con 0x7fd7a0068590 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a00b7640 con 0x7fd7a0068590 2024-04-25T13:46:04.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd796ffd640 1 -- 172.21.15.80:0/1996924738 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79c0b4660 con 0x7fd7a0068590 2024-04-25T13:46:04.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a01581f0 con 0x7fd7a0068590 2024-04-25T13:46:04.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7a006a3a0 con 0x7fd7a0068590 2024-04-25T13:46:04.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd796ffd640 1 -- 172.21.15.80:0/1996924738 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd79c0b4800 con 0x7fd7a0068590 2024-04-25T13:46:04.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.959+0000 7fd796ffd640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd78406f400 0x7fd784071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:04.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.963+0000 7fd796ffd640 1 -- 172.21.15.80:0/1996924738 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fd79c1299b0 con 0x7fd7a0068590 2024-04-25T13:46:04.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.963+0000 7fd7a59b4640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd78406f400 0x7fd784071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:04.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.963+0000 7fd7a59b4640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd78406f400 0x7fd784071910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0167fc0 tx=0x7fd790008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:04.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:04.963+0000 7fd796ffd640 1 -- 172.21.15.80:0/1996924738 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd7a006c9b0 con 0x7fd7a0068590 2024-04-25T13:46:05.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.127+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7a006dee0 con 0x7fd7a0068590 2024-04-25T13:46:05.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd796ffd640 1 -- 172.21.15.80:0/1996924738 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16600 (secure 0 0 0) 0x7fd79c016110 con 0x7fd7a0068590 2024-04-25T13:46:05.129 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:05.130 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":110,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:57.407731+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:45:28.400550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T13:45:28.988401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:05.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd78406f400 msgr2=0x7fd784071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd78406f400 0x7fd784071910 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0167fc0 tx=0x7fd790008040 comp rx=0 tx=0).stop 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a0068590 msgr2=0x7fd7a01561f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a0068590 0x7fd7a01561f0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fd79c0ae040 tx=0x7fd79c0192c0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 shutdown_connections 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd78406f400 0x7fd784071910 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7a015bcc0 0x7fd7a0159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7a00b7d10 0x7fd7a00ab530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 --2- 172.21.15.80:0/1996924738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7a0068590 0x7fd7a01561f0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 >> 172.21.15.80:0/1996924738 conn(0x7fd7a00cf450 msgr2=0x7fd7a006d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 shutdown_connections 2024-04-25T13:46:05.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.131+0000 7fd7a8419640 1 -- 172.21.15.80:0/1996924738 wait complete. 2024-04-25T13:46:05.152 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:45:28.400550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:45:28.988401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:46:05.152 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:46:05.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.303+0000 7f57e3810640 1 Processor -- start 2024-04-25T13:46:05.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.303+0000 7f57e3810640 1 -- start start 2024-04-25T13:46:05.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e3810640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 0x7f57dc06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e3810640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57dc161b60 0x7f57dc161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e3810640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57dc1624b0 con 0x7f57dc0b5330 2024-04-25T13:46:05.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e3810640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f57dc06f700 con 0x7f57dc161b60 2024-04-25T13:46:05.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e3810640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc1629b0 con 0x7f57dc06edb0 2024-04-25T13:46:05.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e1dad640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57dc161b60 0x7f57dc161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e0dab640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 0x7f57dc06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e1dad640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57dc161b60 0x7f57dc161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:48730/0 (socket says 172.21.15.80:48730) 2024-04-25T13:46:05.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e1dad640 1 -- 172.21.15.80:0/1030561030 learned_addr learned my addr 172.21.15.80:0/1030561030 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:05.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e1dad640 1 -- 172.21.15.80:0/1030561030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 msgr2=0x7f57dc06f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:05.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e1dad640 1 --2- 172.21.15.80:0/1030561030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 0x7f57dc06f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e1dad640 1 -- 172.21.15.80:0/1030561030 >> v1:172.21.15.87:6789/0 conn(0x7f57dc0b5330 legacy=0x7f57dc0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.307+0000 7f57e1dad640 1 -- 172.21.15.80:0/1030561030 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc0b5740 con 0x7f57dc161b60 2024-04-25T13:46:05.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e1dad640 1 --2- 172.21.15.80:0/1030561030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc161b60 0x7f57dc161f70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009e50 tx=0x7f57d401c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da5e03fc3b3accf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:05.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57d3fff640 1 -- 172.21.15.80:0/1030561030 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d40a5830 con 0x7f57dc161b60 2024-04-25T13:46:05.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57d3fff640 1 -- 172.21.15.80:0/1030561030 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d40a59d0 con 0x7f57dc161b60 2024-04-25T13:46:05.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57d3fff640 1 -- 172.21.15.80:0/1030561030 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d40a5da0 con 0x7f57dc161b60 2024-04-25T13:46:05.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e3810640 1 -- 172.21.15.80:0/1030561030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc161b60 msgr2=0x7f57dc161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e3810640 1 --2- 172.21.15.80:0/1030561030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc161b60 0x7f57dc161f70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009e50 tx=0x7f57d401c970 comp rx=0 tx=0).stop 2024-04-25T13:46:05.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e3810640 1 -- 172.21.15.80:0/1030561030 shutdown_connections 2024-04-25T13:46:05.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e3810640 1 --2- 172.21.15.80:0/1030561030 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc161b60 0x7f57dc161f70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e3810640 1 --2- 172.21.15.80:0/1030561030 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 0x7f57dc06f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e3810640 1 -- 172.21.15.80:0/1030561030 >> 172.21.15.80:0/1030561030 conn(0x7f57dc0b4160 msgr2=0x7f57dc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:05.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e3810640 1 -- 172.21.15.80:0/1030561030 shutdown_connections 2024-04-25T13:46:05.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.311+0000 7f57e3810640 1 -- 172.21.15.80:0/1030561030 wait complete. 2024-04-25T13:46:05.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 Processor -- start 2024-04-25T13:46:05.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 -- start start 2024-04-25T13:46:05.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 0x7f57dc13b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0b5330 0x7f57dc140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57dc161b60 0x7f57dc140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc06e890 con 0x7f57dc161b60 2024-04-25T13:46:05.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57dc06e5b0 con 0x7f57dc0b5330 2024-04-25T13:46:05.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57dc0b0cd0 con 0x7f57dc06edb0 2024-04-25T13:46:05.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57dc161b60 0x7f57dc140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57dc161b60 0x7f57dc140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38250/0 (socket says 172.21.15.80:38250) 2024-04-25T13:46:05.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 -- 172.21.15.80:0/2172750010 learned_addr learned my addr 172.21.15.80:0/2172750010 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:05.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e0dab640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0b5330 0x7f57dc140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e15ac640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 0x7f57dc13b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 -- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 msgr2=0x7f57dc13b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 0x7f57dc13b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 -- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0b5330 msgr2=0x7f57dc140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0b5330 0x7f57dc140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 -- 172.21.15.80:0/2172750010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57dc13b7b0 con 0x7f57dc161b60 2024-04-25T13:46:05.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e1dad640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57dc161b60 0x7f57dc140590 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009e50 tx=0x7f57d4019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:05.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57d1ffb640 1 -- 172.21.15.80:0/2172750010 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d40bc020 con 0x7f57dc161b60 2024-04-25T13:46:05.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57d1ffb640 1 -- 172.21.15.80:0/2172750010 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57d40b8670 con 0x7f57dc161b60 2024-04-25T13:46:05.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57d1ffb640 1 -- 172.21.15.80:0/2172750010 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d40b8a20 con 0x7f57dc161b60 2024-04-25T13:46:05.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57dc159410 con 0x7f57dc161b60 2024-04-25T13:46:05.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57dc1599d0 con 0x7f57dc161b60 2024-04-25T13:46:05.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.315+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57dc06a3a0 con 0x7f57dc161b60 2024-04-25T13:46:05.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.319+0000 7f57d1ffb640 1 -- 172.21.15.80:0/2172750010 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57d40bf020 con 0x7f57dc161b60 2024-04-25T13:46:05.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.319+0000 7f57d1ffb640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57c006f430 0x7f57c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.319+0000 7f57d1ffb640 1 -- 172.21.15.80:0/2172750010 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f57d412a400 con 0x7f57dc161b60 2024-04-25T13:46:05.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.319+0000 7f57e15ac640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57c006f430 0x7f57c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.319+0000 7f57e15ac640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57c006f430 0x7f57c0071940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f57d80015b0 tx=0x7f57d8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:05.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.323+0000 7f57d1ffb640 1 -- 172.21.15.80:0/2172750010 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57d4016620 con 0x7f57dc161b60 2024-04-25T13:46:05.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.483+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f57dc1591a0 con 0x7f57dc161b60 2024-04-25T13:46:05.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57d1ffb640 1 -- 172.21.15.80:0/2172750010 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f57d40f7460 con 0x7f57dc161b60 2024-04-25T13:46:05.484 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57c006f430 msgr2=0x7f57c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57c006f430 0x7f57c0071940 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f57d80015b0 tx=0x7f57d8009040 comp rx=0 tx=0).stop 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57dc161b60 msgr2=0x7f57dc140590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57dc161b60 0x7f57dc140590 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f57d4009e50 tx=0x7f57d4019710 comp rx=0 tx=0).stop 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 shutdown_connections 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57c006f430 0x7f57c0071940 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57dc161b60 0x7f57dc140590 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57dc0b5330 0x7f57dc140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 --2- 172.21.15.80:0/2172750010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57dc06edb0 0x7f57dc13b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 >> 172.21.15.80:0/2172750010 conn(0x7f57dc0b4160 msgr2=0x7f57dc06cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 shutdown_connections 2024-04-25T13:46:05.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.487+0000 7f57e3810640 1 -- 172.21.15.80:0/2172750010 wait complete. 2024-04-25T13:46:05.507 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:46:05.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae9ae01640 1 Processor -- start 2024-04-25T13:46:05.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae9ae01640 1 -- start start 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae9ae01640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fae94162270 0x7fae9406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae9ae01640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae940b5740 con 0x7fae940b5330 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae9ae01640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae9406dd10 con 0x7fae9406edb0 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae9ae01640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fae9406f700 con 0x7fae94162270 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae98b9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fae940b5330 0x7fae940d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45900/0 (socket says 172.21.15.80:45900) 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae98b9d640 1 -- 172.21.15.80:0/2321205287 learned_addr learned my addr 172.21.15.80:0/2321205287 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae9939e640 1 --2- 172.21.15.80:0/2321205287 >> v2:172.21.15.87:3300/0 conn(0x7fae94162270 0x7fae9406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2352578066 0 0) 0x7fae940b5740 con 0x7fae940b5330 2024-04-25T13:46:05.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae7c003530 con 0x7fae940b5330 2024-04-25T13:46:05.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 618226736 0 0) 0x7fae7c003530 con 0x7fae940b5330 2024-04-25T13:46:05.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 >> v2:172.21.15.87:3300/0 conn(0x7fae94162270 msgr2=0x7fae9406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:05.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 --2- 172.21.15.80:0/2321205287 >> v2:172.21.15.87:3300/0 conn(0x7fae94162270 0x7fae9406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 >> v1:172.21.15.87:6789/0 conn(0x7fae9406edb0 legacy=0x7fae94161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9406dff0 con 0x7fae940b5330 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fae84001680 con 0x7fae940b5330 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 >> v1:172.21.15.80:6789/0 conn(0x7fae940b5330 legacy=0x7fae940d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c0039c0 0x7fae7c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae7c004810 0x7fae7c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae7c005260 0x7fae7c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fae9406f700 con 0x7fae7c005260 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fae7c00c1f0 con 0x7fae7c0039c0 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fae7c00c3f0 con 0x7fae7c004810 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.663+0000 7fae9939e640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae7c005260 0x7fae7c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae98b9d640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c0039c0 0x7fae7c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae93fff640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae7c004810 0x7fae7c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9939e640 1 -- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae7c004810 msgr2=0x7fae7c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9939e640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae7c004810 0x7fae7c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9939e640 1 -- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c0039c0 msgr2=0x7fae7c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:05.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9939e640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c0039c0 0x7fae7c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9939e640 1 -- 172.21.15.80:0/2321205287 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae7c003530 con 0x7fae7c005260 2024-04-25T13:46:05.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae93fff640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae7c004810 0x7fae7c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:05.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9939e640 1 -- 172.21.15.80:0/2321205287 reap_dead start 2024-04-25T13:46:05.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9939e640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae7c005260 0x7fae7c00b9d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009fc0 tx=0x7fae8c01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7087eec8167d197c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:05.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c009c10 con 0x7fae7c005260 2024-04-25T13:46:05.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8c0a5bc0 con 0x7fae7c005260 2024-04-25T13:46:05.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae937fe640 1 -- 172.21.15.80:0/2321205287 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0039a0 con 0x7fae7c005260 2024-04-25T13:46:05.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9ae01640 1 -- 172.21.15.80:0/2321205287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae7c005260 msgr2=0x7fae7c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9ae01640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae7c005260 0x7fae7c00b9d0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7fae8c009fc0 tx=0x7fae8c01ccc0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9ae01640 1 -- 172.21.15.80:0/2321205287 shutdown_connections 2024-04-25T13:46:05.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9ae01640 1 --2- 172.21.15.80:0/2321205287 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae7c005260 0x7fae7c00b9d0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9ae01640 1 -- 172.21.15.80:0/2321205287 >> 172.21.15.80:0/2321205287 conn(0x7fae940b4160 msgr2=0x7fae940d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:05.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9ae01640 1 -- 172.21.15.80:0/2321205287 shutdown_connections 2024-04-25T13:46:05.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.667+0000 7fae9ae01640 1 -- 172.21.15.80:0/2321205287 wait complete. 2024-04-25T13:46:05.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 Processor -- start 2024-04-25T13:46:05.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 -- start start 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c005260 0x7fae94158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae9415da50 0x7fae9415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae94141ab0 0x7fae94141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fae9406e5b0 con 0x7fae94141ab0 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fae941629b0 con 0x7fae7c005260 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fae9406e890 con 0x7fae9415da50 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae94141ab0 0x7fae94141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae94141ab0 0x7fae94141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38266/0 (socket says 172.21.15.80:38266) 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 -- 172.21.15.80:0/447527768 learned_addr learned my addr 172.21.15.80:0/447527768 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:05.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae93fff640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae9415da50 0x7fae9415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae98b9d640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c005260 0x7fae94158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 -- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae9415da50 msgr2=0x7fae9415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae9415da50 0x7fae9415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 -- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c005260 msgr2=0x7fae94158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c005260 0x7fae94158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 -- 172.21.15.80:0/447527768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae9415e3a0 con 0x7fae94141ab0 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae98b9d640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c005260 0x7fae94158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9939e640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae94141ab0 0x7fae94141ec0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fae8c01c730 tx=0x7fae8c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae917fa640 1 -- 172.21.15.80:0/447527768 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c009c10 con 0x7fae94141ab0 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae917fa640 1 -- 172.21.15.80:0/447527768 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae8c0bd070 con 0x7fae94141ab0 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae941590d0 con 0x7fae94141ab0 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae917fa640 1 -- 172.21.15.80:0/447527768 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8c0039a0 con 0x7fae94141ab0 2024-04-25T13:46:05.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.671+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae94142fe0 con 0x7fae94141ab0 2024-04-25T13:46:05.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.675+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae94159680 con 0x7fae94141ab0 2024-04-25T13:46:05.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.675+0000 7fae917fa640 1 -- 172.21.15.80:0/447527768 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fae8c003b40 con 0x7fae94141ab0 2024-04-25T13:46:05.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.675+0000 7fae917fa640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae8006f400 0x7fae80071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:05.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.675+0000 7fae917fa640 1 -- 172.21.15.80:0/447527768 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fae8c12b610 con 0x7fae94141ab0 2024-04-25T13:46:05.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.675+0000 7fae98b9d640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae8006f400 0x7fae80071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:05.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.675+0000 7fae98b9d640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae8006f400 0x7fae80071910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fae84001210 tx=0x7fae840011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:05.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.679+0000 7fae917fa640 1 -- 172.21.15.80:0/447527768 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fae8c0c7020 con 0x7fae94141ab0 2024-04-25T13:46:05.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.839+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fae9406a3a0 con 0x7fae94141ab0 2024-04-25T13:46:05.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.839+0000 7fae917fa640 1 -- 172.21.15.80:0/447527768 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7fae9415e3a0 con 0x7fae94141ab0 2024-04-25T13:46:05.838 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:46:05.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.839+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae8006f400 msgr2=0x7fae80071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.839+0000 7fae9ae01640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae8006f400 0x7fae80071910 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fae84001210 tx=0x7fae840011a0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae94141ab0 msgr2=0x7fae94141ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:05.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae94141ab0 0x7fae94141ec0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7fae8c01c730 tx=0x7fae8c009750 comp rx=0 tx=0).stop 2024-04-25T13:46:05.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 shutdown_connections 2024-04-25T13:46:05.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae8006f400 0x7fae80071910 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae94141ab0 0x7fae94141ec0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae9415da50 0x7fae9415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 --2- 172.21.15.80:0/447527768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae7c005260 0x7fae94158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:05.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 >> 172.21.15.80:0/447527768 conn(0x7fae940b4160 msgr2=0x7fae94161040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:05.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 shutdown_connections 2024-04-25T13:46:05.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:05.843+0000 7fae9ae01640 1 -- 172.21.15.80:0/447527768 wait complete. 2024-04-25T13:46:05.861 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:46:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8cfa3640 1 Processor -- start 2024-04-25T13:46:06.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8cfa3640 1 -- start start 2024-04-25T13:46:06.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8cfa3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c880d3ed0 0x7f7c880d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8cfa3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c880ba770 0x7f7c880b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8cfa3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c880b92a0 con 0x7f7c880d2ad0 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8cfa3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7c880d4c10 con 0x7f7c880d3ed0 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8cfa3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7c880d4ef0 con 0x7f7c880ba770 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8659c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c880d3ed0 0x7f7c880d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c85d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7c880d2ad0 0x7f7c880b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45912/0 (socket says 172.21.15.80:45912) 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c85d9b640 1 -- 172.21.15.80:0/4055067007 learned_addr learned my addr 172.21.15.80:0/4055067007 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c86d9d640 1 --2- 172.21.15.80:0/4055067007 >> v2:172.21.15.87:3300/0 conn(0x7f7c880ba770 0x7f7c880b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8559a640 1 -- 172.21.15.80:0/4055067007 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1376091018 0 0) 0x7f7c7c001120 con 0x7f7c880d2ad0 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8559a640 1 -- 172.21.15.80:0/4055067007 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c70003530 con 0x7f7c880d2ad0 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8659c640 1 -- 172.21.15.80:0/4055067007 >> v2:172.21.15.87:3300/0 conn(0x7f7c880ba770 msgr2=0x7f7c880b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8659c640 1 --2- 172.21.15.80:0/4055067007 >> v2:172.21.15.87:3300/0 conn(0x7f7c880ba770 0x7f7c880b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8659c640 1 -- 172.21.15.80:0/4055067007 >> v1:172.21.15.80:6789/0 conn(0x7f7c880d2ad0 legacy=0x7f7c880b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.015+0000 7f7c8659c640 1 -- 172.21.15.80:0/4055067007 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c880d51d0 con 0x7f7c880d3ed0 2024-04-25T13:46:06.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8659c640 1 --2- 172.21.15.80:0/4055067007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880d3ed0 0x7f7c880d2590 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f7c880d1d20 tx=0x7f7c7801c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e3fe4e1d0b7271 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8559a640 1 -- 172.21.15.80:0/4055067007 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780b0070 con 0x7f7c880d3ed0 2024-04-25T13:46:06.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8559a640 1 -- 172.21.15.80:0/4055067007 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c78004230 con 0x7f7c880d3ed0 2024-04-25T13:46:06.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8559a640 1 -- 172.21.15.80:0/4055067007 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78003620 con 0x7f7c880d3ed0 2024-04-25T13:46:06.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/4055067007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880d3ed0 msgr2=0x7f7c880d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/4055067007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880d3ed0 0x7f7c880d2590 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f7c880d1d20 tx=0x7f7c7801c330 comp rx=0 tx=0).stop 2024-04-25T13:46:06.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/4055067007 shutdown_connections 2024-04-25T13:46:06.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/4055067007 >> v2:172.21.15.87:3300/0 conn(0x7f7c880ba770 0x7f7c880b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/4055067007 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880d3ed0 0x7f7c880d2590 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/4055067007 >> 172.21.15.80:0/4055067007 conn(0x7f7c880737b0 msgr2=0x7f7c8806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:06.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/4055067007 shutdown_connections 2024-04-25T13:46:06.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.019+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/4055067007 wait complete. 2024-04-25T13:46:06.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 Processor -- start 2024-04-25T13:46:06.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 -- start start 2024-04-25T13:46:06.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880ba770 0x7f7c88169da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c880d2ad0 0x7f7c88155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c880d3ed0 0x7f7c88155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7c880b8320 con 0x7f7c880ba770 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c880b84a0 con 0x7f7c880d2ad0 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c88160620 con 0x7f7c880d3ed0 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880ba770 0x7f7c88169da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880ba770 0x7f7c88169da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38276/0 (socket says 172.21.15.80:38276) 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 -- 172.21.15.80:0/2174551023 learned_addr learned my addr 172.21.15.80:0/2174551023 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c85d9b640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c880d2ad0 0x7f7c88155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c86d9d640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c880d3ed0 0x7f7c88155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 -- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c880d3ed0 msgr2=0x7f7c88155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c880d3ed0 0x7f7c88155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 -- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c880d2ad0 msgr2=0x7f7c88155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c880d2ad0 0x7f7c88155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 -- 172.21.15.80:0/2174551023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8815d460 con 0x7f7c880ba770 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c85d9b640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c880d2ad0 0x7f7c88155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8659c640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880ba770 0x7f7c88169da0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f7c780ae040 tx=0x7f7c780190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c76ffd640 1 -- 172.21.15.80:0/2174551023 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780b9070 con 0x7f7c880ba770 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c76ffd640 1 -- 172.21.15.80:0/2174551023 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c78003dd0 con 0x7f7c880ba770 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8815d740 con 0x7f7c880ba770 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c76ffd640 1 -- 172.21.15.80:0/2174551023 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c780b0040 con 0x7f7c880ba770 2024-04-25T13:46:06.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.023+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c8815f2a0 con 0x7f7c880ba770 2024-04-25T13:46:06.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.027+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c8806a3a0 con 0x7f7c880ba770 2024-04-25T13:46:06.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.027+0000 7f7c76ffd640 1 -- 172.21.15.80:0/2174551023 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7c780bc070 con 0x7f7c880ba770 2024-04-25T13:46:06.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.027+0000 7f7c76ffd640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c6806f400 0x7f7c68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.027+0000 7f7c76ffd640 1 -- 172.21.15.80:0/2174551023 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f7c781293a0 con 0x7f7c880ba770 2024-04-25T13:46:06.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.027+0000 7f7c85d9b640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c6806f400 0x7f7c68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.027+0000 7f7c85d9b640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c6806f400 0x7f7c68071910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c007fd0 tx=0x7f7c7c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.031+0000 7f7c76ffd640 1 -- 172.21.15.80:0/2174551023 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c78016070 con 0x7f7c880ba770 2024-04-25T13:46:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.183+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7c880768c0 con 0x7f7c880ba770 2024-04-25T13:46:06.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.183+0000 7f7c76ffd640 1 -- 172.21.15.80:0/2174551023 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f7c780f6320 con 0x7f7c880ba770 2024-04-25T13:46:06.182 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:46:06.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.183+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c6806f400 msgr2=0x7f7c68071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.183+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c6806f400 0x7f7c68071910 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c007fd0 tx=0x7f7c7c009040 comp rx=0 tx=0).stop 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.183+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880ba770 msgr2=0x7f7c88169da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.183+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880ba770 0x7f7c88169da0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f7c780ae040 tx=0x7f7c780190d0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.187+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 shutdown_connections 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.187+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c6806f400 0x7f7c68071910 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.187+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c880d3ed0 0x7f7c88155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.187+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c880d2ad0 0x7f7c88155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.187+0000 7f7c8cfa3640 1 --2- 172.21.15.80:0/2174551023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c880ba770 0x7f7c88169da0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.187+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 >> 172.21.15.80:0/2174551023 conn(0x7f7c880737b0 msgr2=0x7f7c88159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.187+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 shutdown_connections 2024-04-25T13:46:06.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.187+0000 7f7c8cfa3640 1 -- 172.21.15.80:0/2174551023 wait complete. 2024-04-25T13:46:06.203 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:46:06.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5ed5e640 1 Processor -- start 2024-04-25T13:46:06.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5ed5e640 1 -- start start 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5ed5e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fce5806f5a0 0x7fce58160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5ed5e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580733b0 0x7fce58161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5ed5e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce5806fcf0 con 0x7fce58161070 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5ed5e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fce581619c0 con 0x7fce5806f5a0 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5ed5e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce58073f10 con 0x7fce580733b0 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5cafa640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fce5806f5a0 0x7fce58160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce5d2fb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580733b0 0x7fce58161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce57fff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fce58161070 0x7fce580b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:56504/0 (socket says 172.21.15.80:56504) 2024-04-25T13:46:06.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.351+0000 7fce57fff640 1 -- 172.21.15.80:0/4087494396 learned_addr learned my addr 172.21.15.80:0/4087494396 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce5cafa640 1 -- 172.21.15.80:0/4087494396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580733b0 msgr2=0x7fce58161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce5cafa640 1 --2- 172.21.15.80:0/4087494396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580733b0 0x7fce58161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce5cafa640 1 -- 172.21.15.80:0/4087494396 >> v1:172.21.15.87:6789/0 conn(0x7fce58161070 legacy=0x7fce580b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce5cafa640 1 -- 172.21.15.80:0/4087494396 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce580b0c60 con 0x7fce5806f5a0 2024-04-25T13:46:06.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce5d2fb640 1 --2- 172.21.15.80:0/4087494396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580733b0 0x7fce58161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:06.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce5cafa640 1 --2- 172.21.15.80:0/4087494396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 0x7fce58160b30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fce480097f0 tx=0x7fce4801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fb6f08427f66a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce577fe640 1 -- 172.21.15.80:0/4087494396 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480b0070 con 0x7fce5806f5a0 2024-04-25T13:46:06.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce577fe640 1 -- 172.21.15.80:0/4087494396 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce480ab600 con 0x7fce5806f5a0 2024-04-25T13:46:06.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.355+0000 7fce577fe640 1 -- 172.21.15.80:0/4087494396 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480ab9d0 con 0x7fce5806f5a0 2024-04-25T13:46:06.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 -- 172.21.15.80:0/4087494396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 msgr2=0x7fce58160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/4087494396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 0x7fce58160b30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fce480097f0 tx=0x7fce4801c410 comp rx=0 tx=0).stop 2024-04-25T13:46:06.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 -- 172.21.15.80:0/4087494396 shutdown_connections 2024-04-25T13:46:06.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/4087494396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580733b0 0x7fce58161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/4087494396 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 0x7fce58160b30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 -- 172.21.15.80:0/4087494396 >> 172.21.15.80:0/4087494396 conn(0x7fce5806d080 msgr2=0x7fce580b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 -- 172.21.15.80:0/4087494396 shutdown_connections 2024-04-25T13:46:06.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 -- 172.21.15.80:0/4087494396 wait complete. 2024-04-25T13:46:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 Processor -- start 2024-04-25T13:46:06.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.359+0000 7fce5ed5e640 1 -- start start 2024-04-25T13:46:06.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 0x7fce5813e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce580733b0 0x7fce581fad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce58161070 0x7fce5815c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce5816a050 con 0x7fce580733b0 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce58169ed0 con 0x7fce5806f5a0 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce5816a1d0 con 0x7fce58161070 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce580733b0 0x7fce581fad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce580733b0 0x7fce581fad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38288/0 (socket says 172.21.15.80:38288) 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 -- 172.21.15.80:0/1606992494 learned_addr learned my addr 172.21.15.80:0/1606992494 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5cafa640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 0x7fce5813e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5d2fb640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce58161070 0x7fce5815c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 -- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce58161070 msgr2=0x7fce5815c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce58161070 0x7fce5815c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 -- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 msgr2=0x7fce5813e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 0x7fce5813e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 -- 172.21.15.80:0/1606992494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce581fba60 con 0x7fce580733b0 2024-04-25T13:46:06.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce57fff640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce580733b0 0x7fce581fad30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c370 tx=0x7fce4c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce557fa640 1 -- 172.21.15.80:0/1606992494 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c017070 con 0x7fce580733b0 2024-04-25T13:46:06.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce557fa640 1 -- 172.21.15.80:0/1606992494 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce4c005cf0 con 0x7fce580733b0 2024-04-25T13:46:06.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce557fa640 1 -- 172.21.15.80:0/1606992494 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c0072f0 con 0x7fce580733b0 2024-04-25T13:46:06.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5814e0a0 con 0x7fce580733b0 2024-04-25T13:46:06.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5815d950 con 0x7fce580733b0 2024-04-25T13:46:06.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce581fb7d0 con 0x7fce580733b0 2024-04-25T13:46:06.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce557fa640 1 -- 172.21.15.80:0/1606992494 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fce4c01a070 con 0x7fce580733b0 2024-04-25T13:46:06.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.363+0000 7fce557fa640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce4406f400 0x7fce44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.367+0000 7fce557fa640 1 -- 172.21.15.80:0/1606992494 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fce4c08e920 con 0x7fce580733b0 2024-04-25T13:46:06.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.367+0000 7fce5cafa640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce4406f400 0x7fce44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.367+0000 7fce5cafa640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce4406f400 0x7fce44071910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fce480ad000 tx=0x7fce480a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.371+0000 7fce557fa640 1 -- 172.21.15.80:0/1606992494 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce4c021020 con 0x7fce580733b0 2024-04-25T13:46:06.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fce5806a3a0 con 0x7fce580733b0 2024-04-25T13:46:06.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce557fa640 1 -- 172.21.15.80:0/1606992494 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v110) ==== 80+0+0 (secure 0 0 0) 0x7fce4c05ba10 con 0x7fce580733b0 2024-04-25T13:46:06.529 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:46:06.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce4406f400 msgr2=0x7fce44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce4406f400 0x7fce44071910 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fce480ad000 tx=0x7fce480a6000 comp rx=0 tx=0).stop 2024-04-25T13:46:06.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce580733b0 msgr2=0x7fce581fad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce580733b0 0x7fce581fad30 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00c370 tx=0x7fce4c00c830 comp rx=0 tx=0).stop 2024-04-25T13:46:06.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 shutdown_connections 2024-04-25T13:46:06.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce4406f400 0x7fce44071910 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce58161070 0x7fce5815c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce580733b0 0x7fce581fad30 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 --2- 172.21.15.80:0/1606992494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce5806f5a0 0x7fce5813e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 >> 172.21.15.80:0/1606992494 conn(0x7fce5806d080 msgr2=0x7fce580cffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:06.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 shutdown_connections 2024-04-25T13:46:06.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.531+0000 7fce5ed5e640 1 -- 172.21.15.80:0/1606992494 wait complete. 2024-04-25T13:46:06.550 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:46:06.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:06.551 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:06.563 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[24943]: fuse finished with error 0 and tester_r 0 2024-04-25T13:46:06.565 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:46:06.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:06.565 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:46:06.721 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:46:06.721 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:06.721 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:06.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:46:06.727 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:46:06.727 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:46:06.728 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:06.771 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:46:06.771 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:06.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eaff4b640 1 Processor -- start 2024-04-25T13:46:06.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eaff4b640 1 -- start start 2024-04-25T13:46:06.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eaff4b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea806f5a0 0x7f3ea8160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eaff4b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3ea8161070 0x7f3ea8161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eaff4b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ea806fcf0 con 0x7f3ea80b4bd0 2024-04-25T13:46:06.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eaff4b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3ea81619c0 con 0x7f3ea8161070 2024-04-25T13:46:06.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eaff4b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea8073f10 con 0x7f3ea806f5a0 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3ead4e6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3ea8161070 0x7f3ea8161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3ead4e6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3ea8161070 0x7f3ea8161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38296/0 (socket says 172.21.15.80:38296) 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3ead4e6640 1 -- 172.21.15.80:0/2146747913 learned_addr learned my addr 172.21.15.80:0/2146747913 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eadce7640 1 --2- 172.21.15.80:0/2146747913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea806f5a0 0x7f3ea8160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3ead4e6640 1 -- 172.21.15.80:0/2146747913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea806f5a0 msgr2=0x7f3ea8160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3ead4e6640 1 --2- 172.21.15.80:0/2146747913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea806f5a0 0x7f3ea8160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3ead4e6640 1 -- 172.21.15.80:0/2146747913 >> v1:172.21.15.87:6789/0 conn(0x7f3ea80b4bd0 legacy=0x7f3ea80b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3ead4e6640 1 -- 172.21.15.80:0/2146747913 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea80b0c60 con 0x7f3ea8161070 2024-04-25T13:46:06.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.923+0000 7f3eadce7640 1 --2- 172.21.15.80:0/2146747913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea806f5a0 0x7f3ea8160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:06.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3ead4e6640 1 --2- 172.21.15.80:0/2146747913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea8161070 0x7f3ea8161480 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3e980098a0 tx=0x7f3e9801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=584b78f6a4b8bd91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eacce5640 1 -- 172.21.15.80:0/2146747913 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980b0070 con 0x7f3ea8161070 2024-04-25T13:46:06.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eacce5640 1 -- 172.21.15.80:0/2146747913 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3e980ab600 con 0x7f3ea8161070 2024-04-25T13:46:06.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eacce5640 1 -- 172.21.15.80:0/2146747913 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e980ab9d0 con 0x7f3ea8161070 2024-04-25T13:46:06.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2146747913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea8161070 msgr2=0x7f3ea8161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2146747913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea8161070 0x7f3ea8161480 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f3e980098a0 tx=0x7f3e9801c470 comp rx=0 tx=0).stop 2024-04-25T13:46:06.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2146747913 shutdown_connections 2024-04-25T13:46:06.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2146747913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea8161070 0x7f3ea8161480 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2146747913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea806f5a0 0x7f3ea8160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2146747913 >> 172.21.15.80:0/2146747913 conn(0x7f3ea806d080 msgr2=0x7f3ea80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:06.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2146747913 shutdown_connections 2024-04-25T13:46:06.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.927+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2146747913 wait complete. 2024-04-25T13:46:06.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 Processor -- start 2024-04-25T13:46:06.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 -- start start 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea806f5a0 0x7f3ea815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea80b4bd0 0x7f3ea814cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea8161070 0x7f3ea806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea816a860 con 0x7f3ea806f5a0 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea816a6e0 con 0x7f3ea80b4bd0 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea816a9e0 con 0x7f3ea8161070 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea806f5a0 0x7f3ea815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea806f5a0 0x7f3ea815ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38298/0 (socket says 172.21.15.80:38298) 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 -- 172.21.15.80:0/2544001270 learned_addr learned my addr 172.21.15.80:0/2544001270 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3ead4e6640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea80b4bd0 0x7f3ea814cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eae4e8640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea8161070 0x7f3ea806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 -- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea8161070 msgr2=0x7f3ea806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea8161070 0x7f3ea806e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 -- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea80b4bd0 msgr2=0x7f3ea814cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea80b4bd0 0x7f3ea814cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 -- 172.21.15.80:0/2544001270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea813bad0 con 0x7f3ea806f5a0 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eadce7640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea806f5a0 0x7f3ea815ff30 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c7f0 tx=0x7f3ea400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3e9e7fc640 1 -- 172.21.15.80:0/2544001270 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4015070 con 0x7f3ea806f5a0 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3e9e7fc640 1 -- 172.21.15.80:0/2544001270 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ea40077c0 con 0x7f3ea806f5a0 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea813e8a0 con 0x7f3ea806f5a0 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3e9e7fc640 1 -- 172.21.15.80:0/2544001270 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4007b70 con 0x7f3ea806f5a0 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.931+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea814d110 con 0x7f3ea806f5a0 2024-04-25T13:46:06.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.935+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea806a3a0 con 0x7f3ea806f5a0 2024-04-25T13:46:06.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.935+0000 7f3e9e7fc640 1 -- 172.21.15.80:0/2544001270 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3ea4018070 con 0x7f3ea806f5a0 2024-04-25T13:46:06.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.935+0000 7f3e9e7fc640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3e8c06f400 0x7f3e8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:06.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.935+0000 7f3e9e7fc640 1 -- 172.21.15.80:0/2544001270 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f3ea408f400 con 0x7f3ea806f5a0 2024-04-25T13:46:06.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.935+0000 7f3ead4e6640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3e8c06f400 0x7f3e8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:06.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.935+0000 7f3ead4e6640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3e8c06f400 0x7f3e8c071910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3e980ad000 tx=0x7f3e980a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:06.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:06.939+0000 7f3e9e7fc640 1 -- 172.21.15.80:0/2544001270 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ea813e8a0 con 0x7f3ea806f5a0 2024-04-25T13:46:07.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ea80b5410 con 0x7f3ea806f5a0 2024-04-25T13:46:07.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3e9e7fc640 1 -- 172.21.15.80:0/2544001270 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7f3ea405c2f0 con 0x7f3ea806f5a0 2024-04-25T13:46:07.121 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:07.122 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:46:07.122 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:46:07.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3e8c06f400 msgr2=0x7f3e8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3e8c06f400 0x7f3e8c071910 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3e980ad000 tx=0x7f3e980a6000 comp rx=0 tx=0).stop 2024-04-25T13:46:07.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea806f5a0 msgr2=0x7f3ea815ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea806f5a0 0x7f3ea815ff30 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400c7f0 tx=0x7f3ea400ccb0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 shutdown_connections 2024-04-25T13:46:07.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3e8c06f400 0x7f3e8c071910 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3ea8161070 0x7f3ea806e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3ea80b4bd0 0x7f3ea814cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 --2- 172.21.15.80:0/2544001270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3ea806f5a0 0x7f3ea815ff30 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 >> 172.21.15.80:0/2544001270 conn(0x7f3ea806d080 msgr2=0x7f3ea8155b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:07.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 shutdown_connections 2024-04-25T13:46:07.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.123+0000 7f3eaff4b640 1 -- 172.21.15.80:0/2544001270 wait complete. 2024-04-25T13:46:07.145 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:07.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f971eb15640 1 Processor -- start 2024-04-25T13:46:07.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f971eb15640 1 -- start start 2024-04-25T13:46:07.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f971eb15640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f97180c71b0 0x7f97180ce770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f971eb15640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9718164b90 con 0x7f97180d0350 2024-04-25T13:46:07.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f971eb15640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97180d1360 con 0x7f97180cecb0 2024-04-25T13:46:07.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f971eb15640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9718074cd0 con 0x7f97180c71b0 2024-04-25T13:46:07.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f971c8b1640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f97180d0350 0x7f97180c6aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45918/0 (socket says 172.21.15.80:45918) 2024-04-25T13:46:07.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f971c8b1640 1 -- 172.21.15.80:0/1090909794 learned_addr learned my addr 172.21.15.80:0/1090909794 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:07.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f9717fff640 1 --2- 172.21.15.80:0/1090909794 >> v2:172.21.15.87:3300/0 conn(0x7f97180c71b0 0x7f97180ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2244029158 0 0) 0x7f9718164b90 con 0x7f97180d0350 2024-04-25T13:46:07.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.291+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9704003530 con 0x7f97180d0350 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1095981901 0 0) 0x7f9704003530 con 0x7f97180d0350 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 >> v2:172.21.15.87:3300/0 conn(0x7f97180c71b0 msgr2=0x7f97180ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 --2- 172.21.15.80:0/1090909794 >> v2:172.21.15.87:3300/0 conn(0x7f97180c71b0 0x7f97180ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 >> v1:172.21.15.87:6789/0 conn(0x7f97180cecb0 legacy=0x7f97180cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97180d0ac0 con 0x7f97180d0350 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f9708001800 con 0x7f97180d0350 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 >> v1:172.21.15.80:6789/0 conn(0x7f97180d0350 legacy=0x7f97180c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97040039c0 0x7f9704003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9704004810 0x7f9704004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9704005260 0x7f970400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9718074cd0 con 0x7f9704004810 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f970400c1f0 con 0x7f9704005260 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f970400c3f0 con 0x7f97040039c0 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f9717fff640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9704004810 0x7f9704004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971d0b2640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9704005260 0x7f970400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971c8b1640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97040039c0 0x7f9704003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f9717fff640 1 -- 172.21.15.80:0/1090909794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97040039c0 msgr2=0x7f9704003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f9717fff640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f97040039c0 0x7f9704003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f9717fff640 1 -- 172.21.15.80:0/1090909794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9704005260 msgr2=0x7f970400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f9717fff640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9704005260 0x7f970400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f9717fff640 1 -- 172.21.15.80:0/1090909794 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9704003530 con 0x7f9704004810 2024-04-25T13:46:07.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971d0b2640 1 -- 172.21.15.80:0/1090909794 reap_dead start 2024-04-25T13:46:07.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f9717fff640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9704004810 0x7f9704004c20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9700009fc0 tx=0x7f970001cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5942f8249257ca35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:07.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97000b2070 con 0x7f9704004810 2024-04-25T13:46:07.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97000a5a70 con 0x7f9704004810 2024-04-25T13:46:07.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f97177fe640 1 -- 172.21.15.80:0/1090909794 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9700009c10 con 0x7f9704004810 2024-04-25T13:46:07.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971eb15640 1 -- 172.21.15.80:0/1090909794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9704004810 msgr2=0x7f9704004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971eb15640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9704004810 0x7f9704004c20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9700009fc0 tx=0x7f970001cc80 comp rx=0 tx=0).stop 2024-04-25T13:46:07.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971eb15640 1 -- 172.21.15.80:0/1090909794 shutdown_connections 2024-04-25T13:46:07.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971eb15640 1 --2- 172.21.15.80:0/1090909794 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9704004810 0x7f9704004c20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971eb15640 1 -- 172.21.15.80:0/1090909794 >> 172.21.15.80:0/1090909794 conn(0x7f97180d3ad0 msgr2=0x7f97180d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:07.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.295+0000 7f971eb15640 1 -- 172.21.15.80:0/1090909794 shutdown_connections 2024-04-25T13:46:07.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 -- 172.21.15.80:0/1090909794 wait complete. 2024-04-25T13:46:07.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 Processor -- start 2024-04-25T13:46:07.298 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 -- start start 2024-04-25T13:46:07.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9704004810 0x7f9718202d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971815fb40 0x7f9718162620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97180b45c0 0x7f97180b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f97180c6eb0 con 0x7f97180b45c0 2024-04-25T13:46:07.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f97180d0760 con 0x7f971815fb40 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971eb15640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f97180c7030 con 0x7f9704004810 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97180b45c0 0x7f97180b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97180b45c0 0x7f97180b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38304/0 (socket says 172.21.15.80:38304) 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 -- 172.21.15.80:0/2248431484 learned_addr learned my addr 172.21.15.80:0/2248431484 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f9717fff640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971815fb40 0x7f9718162620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971c8b1640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9704004810 0x7f9718202d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 -- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9704004810 msgr2=0x7f9718202d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9704004810 0x7f9718202d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 -- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971815fb40 msgr2=0x7f9718162620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971815fb40 0x7f9718162620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 -- 172.21.15.80:0/2248431484 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97180b4f10 con 0x7f97180b45c0 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.299+0000 7f971d0b2640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97180b45c0 0x7f97180b49d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f97180d3a60 tx=0x7f971000e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f97157fa640 1 -- 172.21.15.80:0/2248431484 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710019070 con 0x7f97180b45c0 2024-04-25T13:46:07.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f97157fa640 1 -- 172.21.15.80:0/2248431484 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9710005280 con 0x7f97180b45c0 2024-04-25T13:46:07.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97181fefb0 con 0x7f97180b45c0 2024-04-25T13:46:07.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f97157fa640 1 -- 172.21.15.80:0/2248431484 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9710007030 con 0x7f97180b45c0 2024-04-25T13:46:07.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971815ff50 con 0x7f97180b45c0 2024-04-25T13:46:07.302 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97180c8df0 con 0x7f97180b45c0 2024-04-25T13:46:07.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f97157fa640 1 -- 172.21.15.80:0/2248431484 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f971001c070 con 0x7f97180b45c0 2024-04-25T13:46:07.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f97157fa640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96f806f400 0x7f96f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f97157fa640 1 -- 172.21.15.80:0/2248431484 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f971008e4c0 con 0x7f97180b45c0 2024-04-25T13:46:07.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.303+0000 7f971c8b1640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96f806f400 0x7f96f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.307+0000 7f971c8b1640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96f806f400 0x7f96f8071910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f97080011e0 tx=0x7f9708009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:07.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.307+0000 7f97157fa640 1 -- 172.21.15.80:0/2248431484 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f97180b4f10 con 0x7f97180b45c0 2024-04-25T13:46:07.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.471+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97181fed40 con 0x7f97180b45c0 2024-04-25T13:46:07.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.471+0000 7f97157fa640 1 -- 172.21.15.80:0/2248431484 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16600 (secure 0 0 0) 0x7f971005b520 con 0x7f97180b45c0 2024-04-25T13:46:07.471 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:07.471 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":110,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:57.407731+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:45:28.400550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T13:45:28.988401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96f806f400 msgr2=0x7f96f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96f806f400 0x7f96f8071910 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f97080011e0 tx=0x7f9708009040 comp rx=0 tx=0).stop 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97180b45c0 msgr2=0x7f97180b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97180b45c0 0x7f97180b49d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f97180d3a60 tx=0x7f971000e820 comp rx=0 tx=0).stop 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 shutdown_connections 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96f806f400 0x7f96f8071910 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f97180b45c0 0x7f97180b49d0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971815fb40 0x7f9718162620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 --2- 172.21.15.80:0/2248431484 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9704004810 0x7f9718202d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 >> 172.21.15.80:0/2248431484 conn(0x7f97180d3ad0 msgr2=0x7f97180d1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 shutdown_connections 2024-04-25T13:46:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.475+0000 7f971eb15640 1 -- 172.21.15.80:0/2248431484 wait complete. 2024-04-25T13:46:07.493 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:45:28.400550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:45:28.988401+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:46:07.494 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:46:07.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614eef3640 1 Processor -- start 2024-04-25T13:46:07.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614eef3640 1 -- start start 2024-04-25T13:46:07.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614eef3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6148071e60 0x7f61481574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614eef3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6148161170 0x7f61480b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614eef3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f614806fbf0 con 0x7f6148073c40 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614eef3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f61480b1350 con 0x7f6148161170 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614eef3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f614806f9a0 con 0x7f6148071e60 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614d490640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6148161170 0x7f61480b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614d490640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6148161170 0x7f61480b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38310/0 (socket says 172.21.15.80:38310) 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614d490640 1 -- 172.21.15.80:0/293278996 learned_addr learned my addr 172.21.15.80:0/293278996 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614cc8f640 1 --2- 172.21.15.80:0/293278996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6148071e60 0x7f61481574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614d490640 1 -- 172.21.15.80:0/293278996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6148071e60 msgr2=0x7f61481574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614d490640 1 --2- 172.21.15.80:0/293278996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6148071e60 0x7f61481574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614d490640 1 -- 172.21.15.80:0/293278996 >> v1:172.21.15.87:6789/0 conn(0x7f6148073c40 legacy=0x7f6148160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614d490640 1 -- 172.21.15.80:0/293278996 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61480b0a20 con 0x7f6148161170 2024-04-25T13:46:07.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.651+0000 7f614cc8f640 1 --2- 172.21.15.80:0/293278996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6148071e60 0x7f61481574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:07.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614d490640 1 --2- 172.21.15.80:0/293278996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148161170 0x7f61480b4ba0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f6140009870 tx=0x7f614001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72668dc48c3245e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:07.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f61477fe640 1 -- 172.21.15.80:0/293278996 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61400b1070 con 0x7f6148161170 2024-04-25T13:46:07.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f61477fe640 1 -- 172.21.15.80:0/293278996 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61400ac630 con 0x7f6148161170 2024-04-25T13:46:07.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f61477fe640 1 -- 172.21.15.80:0/293278996 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61400a8050 con 0x7f6148161170 2024-04-25T13:46:07.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614eef3640 1 -- 172.21.15.80:0/293278996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148161170 msgr2=0x7f61480b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614eef3640 1 --2- 172.21.15.80:0/293278996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148161170 0x7f61480b4ba0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f6140009870 tx=0x7f614001c520 comp rx=0 tx=0).stop 2024-04-25T13:46:07.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614eef3640 1 -- 172.21.15.80:0/293278996 shutdown_connections 2024-04-25T13:46:07.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614eef3640 1 --2- 172.21.15.80:0/293278996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148161170 0x7f61480b4ba0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614eef3640 1 --2- 172.21.15.80:0/293278996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6148071e60 0x7f61481574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614eef3640 1 -- 172.21.15.80:0/293278996 >> 172.21.15.80:0/293278996 conn(0x7f61480b4160 msgr2=0x7f61480d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:07.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614eef3640 1 -- 172.21.15.80:0/293278996 shutdown_connections 2024-04-25T13:46:07.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.655+0000 7f614eef3640 1 -- 172.21.15.80:0/293278996 wait complete. 2024-04-25T13:46:07.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 Processor -- start 2024-04-25T13:46:07.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 -- start start 2024-04-25T13:46:07.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148071e60 0x7f6148142150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148073c40 0x7f6148151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f614815d580 0x7f614815d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6148169bb0 con 0x7f6148073c40 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6148074050 con 0x7f6148071e60 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6148169d30 con 0x7f614815d580 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148073c40 0x7f6148151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148073c40 0x7f6148151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38318/0 (socket says 172.21.15.80:38318) 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 -- 172.21.15.80:0/2000458393 learned_addr learned my addr 172.21.15.80:0/2000458393 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614cc8f640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148071e60 0x7f6148142150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614d490640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f614815d580 0x7f614815d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 -- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f614815d580 msgr2=0x7f614815d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f614815d580 0x7f614815d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 -- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148071e60 msgr2=0x7f6148142150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148071e60 0x7f6148142150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 -- 172.21.15.80:0/2000458393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61481506c0 con 0x7f6148073c40 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614cc8f640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148071e60 0x7f6148142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f6147fff640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148073c40 0x7f6148151fc0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f613000b3e0 tx=0x7f613000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f61457fa640 1 -- 172.21.15.80:0/2000458393 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130015020 con 0x7f6148073c40 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f61457fa640 1 -- 172.21.15.80:0/2000458393 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6130011830 con 0x7f6148073c40 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614815ded0 con 0x7f6148073c40 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f61457fa640 1 -- 172.21.15.80:0/2000458393 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6130011c60 con 0x7f6148073c40 2024-04-25T13:46:07.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.659+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6148152bd0 con 0x7f6148073c40 2024-04-25T13:46:07.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.663+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6148152920 con 0x7f6148073c40 2024-04-25T13:46:07.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.663+0000 7f61457fa640 1 -- 172.21.15.80:0/2000458393 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6130018020 con 0x7f6148073c40 2024-04-25T13:46:07.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.663+0000 7f61457fa640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f612806f400 0x7f6128071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.663+0000 7f61457fa640 1 -- 172.21.15.80:0/2000458393 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f613008f470 con 0x7f6148073c40 2024-04-25T13:46:07.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.663+0000 7f614cc8f640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f612806f400 0x7f6128071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.663+0000 7f614cc8f640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f612806f400 0x7f6128071910 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6138007f40 tx=0x7f6138007490 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:07.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.667+0000 7f61457fa640 1 -- 172.21.15.80:0/2000458393 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f613001f030 con 0x7f6148073c40 2024-04-25T13:46:07.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.823+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f614806a3a0 con 0x7f6148073c40 2024-04-25T13:46:07.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.823+0000 7f61457fa640 1 -- 172.21.15.80:0/2000458393 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) ==== 93+0+10 (secure 0 0 0) 0x7f614815ded0 con 0x7f6148073c40 2024-04-25T13:46:07.824 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:46:07.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f612806f400 msgr2=0x7f6128071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f612806f400 0x7f6128071910 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6138007f40 tx=0x7f6138007490 comp rx=0 tx=0).stop 2024-04-25T13:46:07.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148073c40 msgr2=0x7f6148151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148073c40 0x7f6148151fc0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f613000b3e0 tx=0x7f613000b8a0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 shutdown_connections 2024-04-25T13:46:07.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f612806f400 0x7f6128071910 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f614815d580 0x7f614815d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6148073c40 0x7f6148151fc0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 --2- 172.21.15.80:0/2000458393 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6148071e60 0x7f6148142150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 >> 172.21.15.80:0/2000458393 conn(0x7f61480b4160 msgr2=0x7f6148150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:07.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 shutdown_connections 2024-04-25T13:46:07.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.827+0000 7f614eef3640 1 -- 172.21.15.80:0/2000458393 wait complete. 2024-04-25T13:46:07.845 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:46:07.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f636ba4c640 1 Processor -- start 2024-04-25T13:46:07.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f636ba4c640 1 -- start start 2024-04-25T13:46:07.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f636ba4c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f63640b5330 0x7f63640d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f636ba4c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63640d0ff0 0x7f636406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f636ba4c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63640b5740 con 0x7f636406f2f0 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f636ba4c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f636406f700 con 0x7f63640b5330 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f636ba4c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63641629b0 con 0x7f63640d0ff0 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f63697e8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f63640b5330 0x7f63640d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f6369fe9640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f636406f2f0 0x7f6364161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45924/0 (socket says 172.21.15.80:45924) 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f6369fe9640 1 -- 172.21.15.80:0/3214345993 learned_addr learned my addr 172.21.15.80:0/3214345993 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f6368fe7640 1 --2- 172.21.15.80:0/3214345993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63640d0ff0 0x7f636406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f63697e8640 1 -- 172.21.15.80:0/3214345993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63640d0ff0 msgr2=0x7f636406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f63697e8640 1 --2- 172.21.15.80:0/3214345993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63640d0ff0 0x7f636406edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f63697e8640 1 -- 172.21.15.80:0/3214345993 >> v1:172.21.15.80:6789/0 conn(0x7f636406f2f0 legacy=0x7f6364161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f635bfff640 1 -- 172.21.15.80:0/3214345993 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 512654049 0 0) 0x7f63640b5740 con 0x7f636406f2f0 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f63697e8640 1 -- 172.21.15.80:0/3214345993 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636406dd10 con 0x7f63640b5330 2024-04-25T13:46:07.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.987+0000 7f6368fe7640 1 --2- 172.21.15.80:0/3214345993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63640d0ff0 0x7f636406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:07.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f63697e8640 1 --2- 172.21.15.80:0/3214345993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640d0ab0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f63600097c0 tx=0x7f636001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b713630bd9d90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:07.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f635bfff640 1 -- 172.21.15.80:0/3214345993 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63600b0070 con 0x7f63640b5330 2024-04-25T13:46:07.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f635bfff640 1 -- 172.21.15.80:0/3214345993 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63600ab600 con 0x7f63640b5330 2024-04-25T13:46:07.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f635bfff640 1 -- 172.21.15.80:0/3214345993 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63600ab9d0 con 0x7f63640b5330 2024-04-25T13:46:07.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 -- 172.21.15.80:0/3214345993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 msgr2=0x7f63640d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 --2- 172.21.15.80:0/3214345993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640d0ab0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f63600097c0 tx=0x7f636001c410 comp rx=0 tx=0).stop 2024-04-25T13:46:07.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 -- 172.21.15.80:0/3214345993 shutdown_connections 2024-04-25T13:46:07.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 --2- 172.21.15.80:0/3214345993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f63640d0ff0 0x7f636406edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 --2- 172.21.15.80:0/3214345993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640d0ab0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 -- 172.21.15.80:0/3214345993 >> 172.21.15.80:0/3214345993 conn(0x7f63640b4160 msgr2=0x7f63640d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:07.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 -- 172.21.15.80:0/3214345993 shutdown_connections 2024-04-25T13:46:07.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 -- 172.21.15.80:0/3214345993 wait complete. 2024-04-25T13:46:07.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.991+0000 7f636ba4c640 1 Processor -- start 2024-04-25T13:46:07.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 -- start start 2024-04-25T13:46:07.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f636406f2f0 0x7f6364143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63640d0ff0 0x7f636415e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f63640b0cd0 con 0x7f63640b5330 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f636406e890 con 0x7f63640d0ff0 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f63640b0e50 con 0x7f636406f2f0 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38328/0 (socket says 172.21.15.80:38328) 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 -- 172.21.15.80:0/281375761 learned_addr learned my addr 172.21.15.80:0/281375761 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 -- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f636406f2f0 msgr2=0x7f6364143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f636406f2f0 0x7f6364143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 -- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63640d0ff0 msgr2=0x7f636415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6369fe9640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63640d0ff0 0x7f636415e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63640d0ff0 0x7f636415e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:07.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 -- 172.21.15.80:0/281375761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636415afc0 con 0x7f63640b5330 2024-04-25T13:46:07.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6368fe7640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640ab4a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f635400c7f0 tx=0x7f635400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:07.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6359ffb640 1 -- 172.21.15.80:0/281375761 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354015070 con 0x7f63640b5330 2024-04-25T13:46:07.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6359ffb640 1 -- 172.21.15.80:0/281375761 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6354007820 con 0x7f63640b5330 2024-04-25T13:46:07.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6364159f20 con 0x7f63640b5330 2024-04-25T13:46:07.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f6359ffb640 1 -- 172.21.15.80:0/281375761 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6354007c10 con 0x7f63640b5330 2024-04-25T13:46:07.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636415a4e0 con 0x7f63640b5330 2024-04-25T13:46:07.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.995+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636406a3a0 con 0x7f63640b5330 2024-04-25T13:46:07.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.999+0000 7f6359ffb640 1 -- 172.21.15.80:0/281375761 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6354018070 con 0x7f63640b5330 2024-04-25T13:46:07.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.999+0000 7f6359ffb640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f634806f400 0x7f6348071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:07.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.999+0000 7f6359ffb640 1 -- 172.21.15.80:0/281375761 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f635408f8b0 con 0x7f63640b5330 2024-04-25T13:46:07.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.999+0000 7f63697e8640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f634806f400 0x7f6348071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:07.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:07.999+0000 7f63697e8640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f634806f400 0x7f6348071910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f63600ad000 tx=0x7f6360008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:08.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.003+0000 7f6359ffb640 1 -- 172.21.15.80:0/281375761 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f635405c780 con 0x7f63640b5330 2024-04-25T13:46:08.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.147+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f636406fd00 con 0x7f63640b5330 2024-04-25T13:46:08.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f6359ffb640 1 -- 172.21.15.80:0/281375761 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) ==== 104+0+11 (secure 0 0 0) 0x7f635405c080 con 0x7f63640b5330 2024-04-25T13:46:08.148 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f634806f400 msgr2=0x7f6348071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f634806f400 0x7f6348071910 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f63600ad000 tx=0x7f6360008040 comp rx=0 tx=0).stop 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 msgr2=0x7f63640ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640ab4a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f635400c7f0 tx=0x7f635400ccb0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 shutdown_connections 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f634806f400 0x7f6348071910 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f63640d0ff0 0x7f636415e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f63640b5330 0x7f63640ab4a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 --2- 172.21.15.80:0/281375761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f636406f2f0 0x7f6364143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 >> 172.21.15.80:0/281375761 conn(0x7f63640b4160 msgr2=0x7f63640b5930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:08.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 shutdown_connections 2024-04-25T13:46:08.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.151+0000 7f636ba4c640 1 -- 172.21.15.80:0/281375761 wait complete. 2024-04-25T13:46:08.169 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:46:08.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.323+0000 7f9702582640 1 Processor -- start 2024-04-25T13:46:08.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.323+0000 7f9702582640 1 -- start start 2024-04-25T13:46:08.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f9702582640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96fc0b5330 0x7f96fc0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f9702582640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f96fc161b60 0x7f96fc161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f9702582640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96fc1624b0 con 0x7f96fc0d0ff0 2024-04-25T13:46:08.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f9702582640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f96fc1629b0 con 0x7f96fc161b60 2024-04-25T13:46:08.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f9702582640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0b5740 con 0x7f96fc0b5330 2024-04-25T13:46:08.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fb7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f96fc0d0ff0 0x7f96fc06edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45938/0 (socket says 172.21.15.80:45938) 2024-04-25T13:46:08.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fb7fe640 1 -- 172.21.15.80:0/91607252 learned_addr learned my addr 172.21.15.80:0/91607252 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:08.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fbfff640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96fc0b5330 0x7f96fc0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f9700b1f640 1 --2- 172.21.15.80:0/91607252 >> v2:172.21.15.87:3300/0 conn(0x7f96fc161b60 0x7f96fc161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3455895873 0 0) 0x7f96f0001120 con 0x7f96fc0d0ff0 2024-04-25T13:46:08.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96e4003530 con 0x7f96fc0d0ff0 2024-04-25T13:46:08.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 32043706 0 0) 0x7f96e4003530 con 0x7f96fc0d0ff0 2024-04-25T13:46:08.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96fc0b5330 msgr2=0x7f96fc0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:08.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96fc0b5330 0x7f96fc0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 >> v2:172.21.15.87:3300/0 conn(0x7f96fc161b60 msgr2=0x7f96fc161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 --2- 172.21.15.80:0/91607252 >> v2:172.21.15.87:3300/0 conn(0x7f96fc161b60 0x7f96fc161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc06dd10 con 0x7f96fc0d0ff0 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f96f0001850 con 0x7f96fc0d0ff0 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 >> v1:172.21.15.80:6789/0 conn(0x7f96fc0d0ff0 legacy=0x7f96fc06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96e40039c0 0x7f96e4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96fc0b5330 0x7f96e4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96e4004d50 0x7f96e4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc1629b0 con 0x7f96e40039c0 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc0b5740 con 0x7f96e4004d50 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f96e400ba50 con 0x7f96fc0b5330 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fbfff640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96e40039c0 0x7f96e4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fb7fe640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96fc0b5330 0x7f96e4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fbfff640 1 -- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96fc0b5330 msgr2=0x7f96e4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:08.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fbfff640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96fc0b5330 0x7f96e4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fbfff640 1 -- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96e4004d50 msgr2=0x7f96e4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:08.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fbfff640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96e4004d50 0x7f96e4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.327+0000 7f96fbfff640 1 -- 172.21.15.80:0/91607252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e4003530 con 0x7f96e40039c0 2024-04-25T13:46:08.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f96fbfff640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96e40039c0 0x7f96e4003e30 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009ae0 tx=0x7f96ec01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb52e7e432e016b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:08.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0096b0 con 0x7f96e40039c0 2024-04-25T13:46:08.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ec0a6470 con 0x7f96e40039c0 2024-04-25T13:46:08.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f96faffd640 1 -- 172.21.15.80:0/91607252 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96ec0a6860 con 0x7f96e40039c0 2024-04-25T13:46:08.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9702582640 1 -- 172.21.15.80:0/91607252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96e40039c0 msgr2=0x7f96e4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9702582640 1 --2- 172.21.15.80:0/91607252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96e40039c0 0x7f96e4003e30 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f96ec009ae0 tx=0x7f96ec01ca90 comp rx=0 tx=0).stop 2024-04-25T13:46:08.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9700b1f640 1 -- 172.21.15.80:0/91607252 reap_dead start 2024-04-25T13:46:08.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9702582640 1 -- 172.21.15.80:0/91607252 shutdown_connections 2024-04-25T13:46:08.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9702582640 1 -- 172.21.15.80:0/91607252 >> 172.21.15.80:0/91607252 conn(0x7f96fc0b4160 msgr2=0x7f96fc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:08.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9702582640 1 -- 172.21.15.80:0/91607252 shutdown_connections 2024-04-25T13:46:08.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9702582640 1 -- 172.21.15.80:0/91607252 wait complete. 2024-04-25T13:46:08.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9702582640 1 Processor -- start 2024-04-25T13:46:08.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.331+0000 7f9702582640 1 -- start start 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96e40039c0 0x7f96fc0ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96fc15b6b0 0x7f96fc15bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96fc1fb050 0x7f96fc1fb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc0b0cd0 con 0x7f96fc1fb050 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f96fc06e890 con 0x7f96fc15b6b0 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f96fc0b0e50 con 0x7f96e40039c0 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96fc1fb050 0x7f96fc1fb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96fc1fb050 0x7f96fc1fb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38344/0 (socket says 172.21.15.80:38344) 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 -- 172.21.15.80:0/1303111079 learned_addr learned my addr 172.21.15.80:0/1303111079 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f96fb7fe640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96fc15b6b0 0x7f96fc15bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f96fbfff640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96e40039c0 0x7f96fc0ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 -- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96e40039c0 msgr2=0x7f96fc0ab4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96e40039c0 0x7f96fc0ab4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 -- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96fc15b6b0 msgr2=0x7f96fc15bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96fc15b6b0 0x7f96fc15bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 -- 172.21.15.80:0/1303111079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96fc168d20 con 0x7f96fc1fb050 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9700b1f640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96fc1fb050 0x7f96fc1fb460 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f96fc147cd0 tx=0x7f96f400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f96f8ff9640 1 -- 172.21.15.80:0/1303111079 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f400bd10 con 0x7f96fc1fb050 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f96f8ff9640 1 -- 172.21.15.80:0/1303111079 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f401a070 con 0x7f96fc1fb050 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f96f8ff9640 1 -- 172.21.15.80:0/1303111079 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f4014640 con 0x7f96fc1fb050 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96fc15dd90 con 0x7f96fc1fb050 2024-04-25T13:46:08.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96fc15e350 con 0x7f96fc1fb050 2024-04-25T13:46:08.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96fc06a3a0 con 0x7f96fc1fb050 2024-04-25T13:46:08.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f96f8ff9640 1 -- 172.21.15.80:0/1303111079 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f96f4013070 con 0x7f96fc1fb050 2024-04-25T13:46:08.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f96f8ff9640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96e806f400 0x7f96e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.335+0000 7f96f8ff9640 1 -- 172.21.15.80:0/1303111079 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f96f408f360 con 0x7f96fc1fb050 2024-04-25T13:46:08.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.339+0000 7f96fbfff640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96e806f400 0x7f96e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.339+0000 7f96fbfff640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96e806f400 0x7f96e8071910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0ad000 tx=0x7f96ec0b2040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:08.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.343+0000 7f96f8ff9640 1 -- 172.21.15.80:0/1303111079 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f96f405c2e0 con 0x7f96fc1fb050 2024-04-25T13:46:08.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.495+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f96fc0d19f0 con 0x7f96fc1fb050 2024-04-25T13:46:08.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.495+0000 7f96f8ff9640 1 -- 172.21.15.80:0/1303111079 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) ==== 100+0+11 (secure 0 0 0) 0x7f96f40642d0 con 0x7f96fc1fb050 2024-04-25T13:46:08.495 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:46:08.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96e806f400 msgr2=0x7f96e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96e806f400 0x7f96e8071910 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f96ec0ad000 tx=0x7f96ec0b2040 comp rx=0 tx=0).stop 2024-04-25T13:46:08.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96fc1fb050 msgr2=0x7f96fc1fb460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96fc1fb050 0x7f96fc1fb460 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f96fc147cd0 tx=0x7f96f400b480 comp rx=0 tx=0).stop 2024-04-25T13:46:08.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 shutdown_connections 2024-04-25T13:46:08.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96e806f400 0x7f96e8071910 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96fc1fb050 0x7f96fc1fb460 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96fc15b6b0 0x7f96fc15bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 --2- 172.21.15.80:0/1303111079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96e40039c0 0x7f96fc0ab4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 >> 172.21.15.80:0/1303111079 conn(0x7f96fc0b4160 msgr2=0x7f96fc0b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:08.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 shutdown_connections 2024-04-25T13:46:08.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.499+0000 7f9702582640 1 -- 172.21.15.80:0/1303111079 wait complete. 2024-04-25T13:46:08.516 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:08.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.651+0000 7fd8b162a640 1 Processor -- start 2024-04-25T13:46:08.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.651+0000 7fd8b162a640 1 -- start start 2024-04-25T13:46:08.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8b162a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd8ac161f90 0x7fd8ac0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8b162a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac15ec40 0x7fd8ac15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8b162a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8ac0d0d70 con 0x7fd8ac0d0510 2024-04-25T13:46:08.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8b162a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd8ac0d1050 con 0x7fd8ac161f90 2024-04-25T13:46:08.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8b162a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac06dca0 con 0x7fd8ac15ec40 2024-04-25T13:46:08.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8aaffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd8ac161f90 0x7fd8ac0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8aa7fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd8ac0d0510 0x7fd8ac15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45944/0 (socket says 172.21.15.80:45944) 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8aaffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd8ac161f90 0x7fd8ac0cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38348/0 (socket says 172.21.15.80:38348) 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8aa7fc640 1 -- 172.21.15.80:0/589047898 learned_addr learned my addr 172.21.15.80:0/589047898 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8ab7fe640 1 --2- 172.21.15.80:0/589047898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac15ec40 0x7fd8ac15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8aaffd640 1 -- 172.21.15.80:0/589047898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac15ec40 msgr2=0x7fd8ac15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8aaffd640 1 --2- 172.21.15.80:0/589047898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac15ec40 0x7fd8ac15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8aaffd640 1 -- 172.21.15.80:0/589047898 >> v1:172.21.15.80:6789/0 conn(0x7fd8ac0d0510 legacy=0x7fd8ac15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8a9ffb640 1 -- 172.21.15.80:0/589047898 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3347282402 0 0) 0x7fd8ac0d0d70 con 0x7fd8ac0d0510 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8aaffd640 1 -- 172.21.15.80:0/589047898 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac06df80 con 0x7fd8ac161f90 2024-04-25T13:46:08.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.655+0000 7fd8ab7fe640 1 --2- 172.21.15.80:0/589047898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac15ec40 0x7fd8ac15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:08.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8aaffd640 1 --2- 172.21.15.80:0/589047898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac161f90 0x7fd8ac0cffd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0097c0 tx=0x7fd89c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa44b91582975390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:08.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8a9ffb640 1 -- 172.21.15.80:0/589047898 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0b0070 con 0x7fd8ac161f90 2024-04-25T13:46:08.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8a9ffb640 1 -- 172.21.15.80:0/589047898 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd89c0ab600 con 0x7fd8ac161f90 2024-04-25T13:46:08.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8a9ffb640 1 -- 172.21.15.80:0/589047898 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd89c0ab9d0 con 0x7fd8ac161f90 2024-04-25T13:46:08.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 -- 172.21.15.80:0/589047898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac161f90 msgr2=0x7fd8ac0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 --2- 172.21.15.80:0/589047898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac161f90 0x7fd8ac0cffd0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0097c0 tx=0x7fd89c01c3b0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 -- 172.21.15.80:0/589047898 shutdown_connections 2024-04-25T13:46:08.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 --2- 172.21.15.80:0/589047898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac15ec40 0x7fd8ac15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 --2- 172.21.15.80:0/589047898 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac161f90 0x7fd8ac0cffd0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 -- 172.21.15.80:0/589047898 >> 172.21.15.80:0/589047898 conn(0x7fd8ac06d080 msgr2=0x7fd8ac0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:08.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 -- 172.21.15.80:0/589047898 shutdown_connections 2024-04-25T13:46:08.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 -- 172.21.15.80:0/589047898 wait complete. 2024-04-25T13:46:08.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 Processor -- start 2024-04-25T13:46:08.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.659+0000 7fd8b162a640 1 -- start start 2024-04-25T13:46:08.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac0d0510 0x7fd8ac141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac15ec40 0x7fd8ac161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8ac161f90 0x7fd8ac0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac06f4b0 con 0x7fd8ac15ec40 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd8ac06f630 con 0x7fd8ac161f90 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd8ac0d0920 con 0x7fd8ac0d0510 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac15ec40 0x7fd8ac161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac15ec40 0x7fd8ac161070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38360/0 (socket says 172.21.15.80:38360) 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 -- 172.21.15.80:0/1129576624 learned_addr learned my addr 172.21.15.80:0/1129576624 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8ab7fe640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8ac161f90 0x7fd8ac0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aaffd640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac0d0510 0x7fd8ac141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 -- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac0d0510 msgr2=0x7fd8ac141480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac0d0510 0x7fd8ac141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 -- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8ac161f90 msgr2=0x7fd8ac0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8ac161f90 0x7fd8ac0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 -- 172.21.15.80:0/1129576624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8ac1586d0 con 0x7fd8ac15ec40 2024-04-25T13:46:08.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8ab7fe640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8ac161f90 0x7fd8ac0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:08.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aa7fc640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac15ec40 0x7fd8ac161070 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c480 tx=0x7fd8a000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:08.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8937fe640 1 -- 172.21.15.80:0/1129576624 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0018070 con 0x7fd8ac15ec40 2024-04-25T13:46:08.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8937fe640 1 -- 172.21.15.80:0/1129576624 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8a0007500 con 0x7fd8ac15ec40 2024-04-25T13:46:08.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8ac15a530 con 0x7fd8ac15ec40 2024-04-25T13:46:08.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8937fe640 1 -- 172.21.15.80:0/1129576624 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8a0004a30 con 0x7fd8ac15ec40 2024-04-25T13:46:08.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8ac1583f0 con 0x7fd8ac15ec40 2024-04-25T13:46:08.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8ac157e60 con 0x7fd8ac15ec40 2024-04-25T13:46:08.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8937fe640 1 -- 172.21.15.80:0/1129576624 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd8a001b070 con 0x7fd8ac15ec40 2024-04-25T13:46:08.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8937fe640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd89806f400 0x7fd898071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:08.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8937fe640 1 -- 172.21.15.80:0/1129576624 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fd8a008e510 con 0x7fd8ac15ec40 2024-04-25T13:46:08.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.663+0000 7fd8aaffd640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd89806f400 0x7fd898071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:08.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.667+0000 7fd8aaffd640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd89806f400 0x7fd898071910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0ad000 tx=0x7fd89c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:08.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.667+0000 7fd8937fe640 1 -- 172.21.15.80:0/1129576624 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd8a005b500 con 0x7fd8ac15ec40 2024-04-25T13:46:08.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8ac06a3a0 con 0x7fd8ac15ec40 2024-04-25T13:46:08.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8937fe640 1 -- 172.21.15.80:0/1129576624 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7fd8a005ae00 con 0x7fd8ac15ec40 2024-04-25T13:46:08.849 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:08.849 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:46:08.850 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd89806f400 msgr2=0x7fd898071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd89806f400 0x7fd898071910 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd89c0ad000 tx=0x7fd89c0a6000 comp rx=0 tx=0).stop 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac15ec40 msgr2=0x7fd8ac161070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac15ec40 0x7fd8ac161070 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd8a000c480 tx=0x7fd8a000c940 comp rx=0 tx=0).stop 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 shutdown_connections 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd89806f400 0x7fd898071910 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd8ac161f90 0x7fd8ac0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd8ac15ec40 0x7fd8ac161070 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 --2- 172.21.15.80:0/1129576624 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd8ac0d0510 0x7fd8ac141480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 >> 172.21.15.80:0/1129576624 conn(0x7fd8ac06d080 msgr2=0x7fd8ac0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 shutdown_connections 2024-04-25T13:46:08.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:08.851+0000 7fd8b162a640 1 -- 172.21.15.80:0/1129576624 wait complete. 2024-04-25T13:46:08.870 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:09.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.003+0000 7ffb45e55640 1 Processor -- start 2024-04-25T13:46:09.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.003+0000 7ffb45e55640 1 -- start start 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb45e55640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ffb400d2e80 0x7ffb401607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb45e55640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffb400d59b0 0x7ffb400d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb45e55640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffb400d6300 con 0x7ffb400d4330 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb45e55640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ffb40161060 con 0x7ffb400d59b0 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb45e55640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ffb40161340 con 0x7ffb400d2e80 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3effd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffb400d59b0 0x7ffb400d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3effd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffb400d59b0 0x7ffb400d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38366/0 (socket says 172.21.15.80:38366) 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3effd640 1 -- 172.21.15.80:0/3343192443 learned_addr learned my addr 172.21.15.80:0/3343192443 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3f7fe640 1 --2- 172.21.15.80:0/3343192443 >> v2:172.21.15.87:3300/0 conn(0x7ffb400d2e80 0x7ffb401607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3effd640 1 -- 172.21.15.80:0/3343192443 >> v2:172.21.15.87:3300/0 conn(0x7ffb400d2e80 msgr2=0x7ffb401607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3effd640 1 --2- 172.21.15.80:0/3343192443 >> v2:172.21.15.87:3300/0 conn(0x7ffb400d2e80 0x7ffb401607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3effd640 1 -- 172.21.15.80:0/3343192443 >> v1:172.21.15.87:6789/0 conn(0x7ffb400d4330 legacy=0x7ffb400d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:09.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3effd640 1 -- 172.21.15.80:0/3343192443 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb400d2680 con 0x7ffb400d59b0 2024-04-25T13:46:09.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3effd640 1 --2- 172.21.15.80:0/3343192443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb400d5dc0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ffb400c9670 tx=0x7ffb3401c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b68b5d741529708 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3e7fc640 1 -- 172.21.15.80:0/3343192443 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb340b0070 con 0x7ffb400d59b0 2024-04-25T13:46:09.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3e7fc640 1 -- 172.21.15.80:0/3343192443 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb340ab600 con 0x7ffb400d59b0 2024-04-25T13:46:09.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.007+0000 7ffb3e7fc640 1 -- 172.21.15.80:0/3343192443 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb340ab9d0 con 0x7ffb400d59b0 2024-04-25T13:46:09.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- 172.21.15.80:0/3343192443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 msgr2=0x7ffb400d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 --2- 172.21.15.80:0/3343192443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb400d5dc0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7ffb400c9670 tx=0x7ffb3401c270 comp rx=0 tx=0).stop 2024-04-25T13:46:09.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- 172.21.15.80:0/3343192443 shutdown_connections 2024-04-25T13:46:09.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 --2- 172.21.15.80:0/3343192443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb400d5dc0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 --2- 172.21.15.80:0/3343192443 >> v2:172.21.15.87:3300/0 conn(0x7ffb400d2e80 0x7ffb401607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- 172.21.15.80:0/3343192443 >> 172.21.15.80:0/3343192443 conn(0x7ffb400737a0 msgr2=0x7ffb4006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:09.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- 172.21.15.80:0/3343192443 shutdown_connections 2024-04-25T13:46:09.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- 172.21.15.80:0/3343192443 wait complete. 2024-04-25T13:46:09.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 Processor -- start 2024-04-25T13:46:09.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- start start 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb400d2e80 0x7ffb400c9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb400d4330 0x7ffb40157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb40143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ffb400d3410 con 0x7ffb400d59b0 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffb400d3290 con 0x7ffb400d2e80 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb45e55640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ffb4015df60 con 0x7ffb400d4330 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb3ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb40143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb3ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb40143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38372/0 (socket says 172.21.15.80:38372) 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.011+0000 7ffb3ffff640 1 -- 172.21.15.80:0/2116885006 learned_addr learned my addr 172.21.15.80:0/2116885006 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:09.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3f7fe640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb400d2e80 0x7ffb400c9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3effd640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb400d4330 0x7ffb40157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3ffff640 1 -- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb400d4330 msgr2=0x7ffb40157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3ffff640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb400d4330 0x7ffb40157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3ffff640 1 -- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb400d2e80 msgr2=0x7ffb400c9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3ffff640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb400d2e80 0x7ffb400c9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3ffff640 1 -- 172.21.15.80:0/2116885006 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4015a140 con 0x7ffb400d59b0 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3effd640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb400d4330 0x7ffb40157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3f7fe640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb400d2e80 0x7ffb400c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3ffff640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb40143f00 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b9e0 tx=0x7ffb3800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb27fff640 1 -- 172.21.15.80:0/2116885006 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb38003b80 con 0x7ffb400d59b0 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb27fff640 1 -- 172.21.15.80:0/2116885006 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb38003d20 con 0x7ffb400d59b0 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb27fff640 1 -- 172.21.15.80:0/2116885006 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb3800cb60 con 0x7ffb400d59b0 2024-04-25T13:46:09.014 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4015a420 con 0x7ffb400d59b0 2024-04-25T13:46:09.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb400cd700 con 0x7ffb400d59b0 2024-04-25T13:46:09.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb400cd4f0 con 0x7ffb400d59b0 2024-04-25T13:46:09.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb27fff640 1 -- 172.21.15.80:0/2116885006 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffb38006d10 con 0x7ffb400d59b0 2024-04-25T13:46:09.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb27fff640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb2c06f400 0x7ffb2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb27fff640 1 -- 172.21.15.80:0/2116885006 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7ffb3808ef10 con 0x7ffb400d59b0 2024-04-25T13:46:09.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.015+0000 7ffb3f7fe640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb2c06f400 0x7ffb2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.019+0000 7ffb3f7fe640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb2c06f400 0x7ffb2c071910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ffb400b8fd0 tx=0x7ffb30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.019+0000 7ffb27fff640 1 -- 172.21.15.80:0/2116885006 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffb3805bd80 con 0x7ffb400d59b0 2024-04-25T13:46:09.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.179+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffb4006a3a0 con 0x7ffb400d59b0 2024-04-25T13:46:09.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.179+0000 7ffb27fff640 1 -- 172.21.15.80:0/2116885006 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16600 (secure 0 0 0) 0x7ffb3805b680 con 0x7ffb400d59b0 2024-04-25T13:46:09.180 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:09.180 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":110,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:57.407731+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:45:28.400550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T13:45:28.988401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:09.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb2c06f400 msgr2=0x7ffb2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb2c06f400 0x7ffb2c071910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7ffb400b8fd0 tx=0x7ffb30008040 comp rx=0 tx=0).stop 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 msgr2=0x7ffb40143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb40143f00 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800b9e0 tx=0x7ffb3800bea0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 shutdown_connections 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffb2c06f400 0x7ffb2c071910 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffb400d59b0 0x7ffb40143f00 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffb400d4330 0x7ffb40157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 --2- 172.21.15.80:0/2116885006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb400d2e80 0x7ffb400c9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 >> 172.21.15.80:0/2116885006 conn(0x7ffb400737a0 msgr2=0x7ffb4015fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 shutdown_connections 2024-04-25T13:46:09.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.183+0000 7ffb45e55640 1 -- 172.21.15.80:0/2116885006 wait complete. 2024-04-25T13:46:09.202 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:46:09.202 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:09.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.335+0000 7f899bb9b640 1 Processor -- start 2024-04-25T13:46:09.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.335+0000 7f899bb9b640 1 -- start start 2024-04-25T13:46:09.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899bb9b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f899406f5a0 0x7f8994160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899bb9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89940733b0 0x7f8994161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899bb9b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f899406fcf0 con 0x7f8994161070 2024-04-25T13:46:09.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899bb9b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f89941619c0 con 0x7f89940733b0 2024-04-25T13:46:09.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899bb9b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8994073f10 con 0x7f899406f5a0 2024-04-25T13:46:09.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899a138640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89940733b0 0x7f8994161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899a138640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f89940733b0 0x7f8994161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38374/0 (socket says 172.21.15.80:38374) 2024-04-25T13:46:09.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899a138640 1 -- 172.21.15.80:0/553105113 learned_addr learned my addr 172.21.15.80:0/553105113 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:09.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f8999937640 1 --2- 172.21.15.80:0/553105113 >> v2:172.21.15.87:3300/0 conn(0x7f899406f5a0 0x7f8994160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899a138640 1 -- 172.21.15.80:0/553105113 >> v2:172.21.15.87:3300/0 conn(0x7f899406f5a0 msgr2=0x7f8994160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:09.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899a138640 1 --2- 172.21.15.80:0/553105113 >> v2:172.21.15.87:3300/0 conn(0x7f899406f5a0 0x7f8994160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899a138640 1 -- 172.21.15.80:0/553105113 >> v1:172.21.15.87:6789/0 conn(0x7f8994161070 legacy=0x7f89940b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:09.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.339+0000 7f899a138640 1 -- 172.21.15.80:0/553105113 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89940b0c60 con 0x7f89940733b0 2024-04-25T13:46:09.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899a138640 1 --2- 172.21.15.80:0/553105113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940733b0 0x7f8994161480 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8994071e30 tx=0x7f899001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e46183bdeaa7885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f8998935640 1 -- 172.21.15.80:0/553105113 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900a55f0 con 0x7f89940733b0 2024-04-25T13:46:09.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f8998935640 1 -- 172.21.15.80:0/553105113 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89900a5790 con 0x7f89940733b0 2024-04-25T13:46:09.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f8998935640 1 -- 172.21.15.80:0/553105113 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89900a5b60 con 0x7f89940733b0 2024-04-25T13:46:09.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 -- 172.21.15.80:0/553105113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940733b0 msgr2=0x7f8994161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 --2- 172.21.15.80:0/553105113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940733b0 0x7f8994161480 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f8994071e30 tx=0x7f899001c920 comp rx=0 tx=0).stop 2024-04-25T13:46:09.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 -- 172.21.15.80:0/553105113 shutdown_connections 2024-04-25T13:46:09.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 --2- 172.21.15.80:0/553105113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89940733b0 0x7f8994161480 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 --2- 172.21.15.80:0/553105113 >> v2:172.21.15.87:3300/0 conn(0x7f899406f5a0 0x7f8994160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 -- 172.21.15.80:0/553105113 >> 172.21.15.80:0/553105113 conn(0x7f899406d080 msgr2=0x7f89940b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:09.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 -- 172.21.15.80:0/553105113 shutdown_connections 2024-04-25T13:46:09.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 -- 172.21.15.80:0/553105113 wait complete. 2024-04-25T13:46:09.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 Processor -- start 2024-04-25T13:46:09.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.343+0000 7f899bb9b640 1 -- start start 2024-04-25T13:46:09.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f899406f5a0 0x7f8994148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89940733b0 0x7f89940af9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8994161070 0x7f89940aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f899416a050 con 0x7f899406f5a0 2024-04-25T13:46:09.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8994169ed0 con 0x7f8994161070 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f899416a1d0 con 0x7f89940733b0 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f899406f5a0 0x7f8994148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f899406f5a0 0x7f8994148330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38384/0 (socket says 172.21.15.80:38384) 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 -- 172.21.15.80:0/1250809011 learned_addr learned my addr 172.21.15.80:0/1250809011 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 -- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89940733b0 msgr2=0x7f89940af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89940733b0 0x7f89940af9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 -- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8994161070 msgr2=0x7f89940aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8994161070 0x7f89940aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 -- 172.21.15.80:0/1250809011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89940abf50 con 0x7f899406f5a0 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999937640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f899406f5a0 0x7f8994148330 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c820 tx=0x7f898c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f898a7fc640 1 -- 172.21.15.80:0/1250809011 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c004030 con 0x7f899406f5a0 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f898a7fc640 1 -- 172.21.15.80:0/1250809011 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f898c0041d0 con 0x7f899406f5a0 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89940ac230 con 0x7f899406f5a0 2024-04-25T13:46:09.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f898a7fc640 1 -- 172.21.15.80:0/1250809011 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c007830 con 0x7f899406f5a0 2024-04-25T13:46:09.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89940ac7f0 con 0x7f899406f5a0 2024-04-25T13:46:09.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899406a3a0 con 0x7f899406f5a0 2024-04-25T13:46:09.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f898a7fc640 1 -- 172.21.15.80:0/1250809011 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f898c0079d0 con 0x7f899406f5a0 2024-04-25T13:46:09.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f898a7fc640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f897806f400 0x7f8978071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f898a7fc640 1 -- 172.21.15.80:0/1250809011 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f898c08f240 con 0x7f899406f5a0 2024-04-25T13:46:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.347+0000 7f8999136640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f897806f400 0x7f8978071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.351+0000 7f8999136640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f897806f400 0x7f8978071910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f89940b3950 tx=0x7f8984006bb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.355+0000 7f898a7fc640 1 -- 172.21.15.80:0/1250809011 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f898c015020 con 0x7f899406f5a0 2024-04-25T13:46:09.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.543+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89940739c0 con 0x7f899406f5a0 2024-04-25T13:46:09.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.543+0000 7f898a7fc640 1 -- 172.21.15.80:0/1250809011 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7f898c05c130 con 0x7f899406f5a0 2024-04-25T13:46:09.544 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:09.544 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:46:09.546 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:46:09.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f897806f400 msgr2=0x7f8978071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f897806f400 0x7f8978071910 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f89940b3950 tx=0x7f8984006bb0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f899406f5a0 msgr2=0x7f8994148330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f899406f5a0 0x7f8994148330 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f898c00c820 tx=0x7f898c00cce0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 shutdown_connections 2024-04-25T13:46:09.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f897806f400 0x7f8978071910 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8994161070 0x7f89940aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89940733b0 0x7f89940af9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 --2- 172.21.15.80:0/1250809011 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f899406f5a0 0x7f8994148330 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 >> 172.21.15.80:0/1250809011 conn(0x7f899406d080 msgr2=0x7f89940b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:09.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 shutdown_connections 2024-04-25T13:46:09.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.547+0000 7f899bb9b640 1 -- 172.21.15.80:0/1250809011 wait complete. 2024-04-25T13:46:09.565 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14ca72640 1 Processor -- start 2024-04-25T13:46:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14ca72640 1 -- start start 2024-04-25T13:46:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14ca72640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe148161f90 0x7fe1480cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14ca72640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1480d0510 0x7fe14815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14ca72640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1480d0d70 con 0x7fe14815ea70 2024-04-25T13:46:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14ca72640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe1480d1050 con 0x7fe1480d0510 2024-04-25T13:46:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14ca72640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe14806dca0 con 0x7fe148161f90 2024-04-25T13:46:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14659c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe148161f90 0x7fe1480cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe145d9b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe1480d0510 0x7fe14815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe146d9d640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fe14815ea70 0x7fe14815ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:56548/0 (socket says 172.21.15.80:56548) 2024-04-25T13:46:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe146d9d640 1 -- 172.21.15.80:0/1429272766 learned_addr learned my addr 172.21.15.80:0/1429272766 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe145d9b640 1 -- 172.21.15.80:0/1429272766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe148161f90 msgr2=0x7fe1480cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe145d9b640 1 --2- 172.21.15.80:0/1429272766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe148161f90 0x7fe1480cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe145d9b640 1 -- 172.21.15.80:0/1429272766 >> v1:172.21.15.87:6789/0 conn(0x7fe14815ea70 legacy=0x7fe14815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe145d9b640 1 -- 172.21.15.80:0/1429272766 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe14806df80 con 0x7fe1480d0510 2024-04-25T13:46:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.703+0000 7fe14659c640 1 --2- 172.21.15.80:0/1429272766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe148161f90 0x7fe1480cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:09.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe145d9b640 1 --2- 172.21.15.80:0/1429272766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1480d0510 0x7fe14815e530 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0097c0 tx=0x7fe13c01c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a20599afb345ee9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14559a640 1 -- 172.21.15.80:0/1429272766 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0b0070 con 0x7fe1480d0510 2024-04-25T13:46:09.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14559a640 1 -- 172.21.15.80:0/1429272766 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13c0ab600 con 0x7fe1480d0510 2024-04-25T13:46:09.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14559a640 1 -- 172.21.15.80:0/1429272766 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13c0ab9d0 con 0x7fe1480d0510 2024-04-25T13:46:09.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14ca72640 1 -- 172.21.15.80:0/1429272766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1480d0510 msgr2=0x7fe14815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14ca72640 1 --2- 172.21.15.80:0/1429272766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1480d0510 0x7fe14815e530 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0097c0 tx=0x7fe13c01c3b0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14ca72640 1 -- 172.21.15.80:0/1429272766 shutdown_connections 2024-04-25T13:46:09.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14ca72640 1 --2- 172.21.15.80:0/1429272766 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1480d0510 0x7fe14815e530 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14ca72640 1 --2- 172.21.15.80:0/1429272766 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe148161f90 0x7fe1480cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14ca72640 1 -- 172.21.15.80:0/1429272766 >> 172.21.15.80:0/1429272766 conn(0x7fe14806d080 msgr2=0x7fe1480b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:09.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14ca72640 1 -- 172.21.15.80:0/1429272766 shutdown_connections 2024-04-25T13:46:09.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.707+0000 7fe14ca72640 1 -- 172.21.15.80:0/1429272766 wait complete. 2024-04-25T13:46:09.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 Processor -- start 2024-04-25T13:46:09.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 -- start start 2024-04-25T13:46:09.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1480d0510 0x7fe14813fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe14815ea70 0x7fe1481404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe148161f90 0x7fe1481409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe14806f630 con 0x7fe1480d0510 2024-04-25T13:46:09.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe14806f4b0 con 0x7fe148161f90 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1480d0920 con 0x7fe14815ea70 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1480d0510 0x7fe14813fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1480d0510 0x7fe14813fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38388/0 (socket says 172.21.15.80:38388) 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 -- 172.21.15.80:0/600311505 learned_addr learned my addr 172.21.15.80:0/600311505 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe145d9b640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe14815ea70 0x7fe1481404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 -- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe14815ea70 msgr2=0x7fe1481404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe146d9d640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe148161f90 0x7fe1481409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe14815ea70 0x7fe1481404a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 -- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe148161f90 msgr2=0x7fe1481409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe148161f90 0x7fe1481409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 -- 172.21.15.80:0/600311505 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe148145280 con 0x7fe1480d0510 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14659c640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1480d0510 0x7fe14813fe00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe13800e900 tx=0x7fe13800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe136ffd640 1 -- 172.21.15.80:0/600311505 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe138017070 con 0x7fe1480d0510 2024-04-25T13:46:09.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe136ffd640 1 -- 172.21.15.80:0/600311505 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe13800ca00 con 0x7fe1480d0510 2024-04-25T13:46:09.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe136ffd640 1 -- 172.21.15.80:0/600311505 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13800cdb0 con 0x7fe1480d0510 2024-04-25T13:46:09.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1481456f0 con 0x7fe1480d0510 2024-04-25T13:46:09.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.711+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe148145cb0 con 0x7fe1480d0510 2024-04-25T13:46:09.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.715+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe14806a3a0 con 0x7fe1480d0510 2024-04-25T13:46:09.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.715+0000 7fe136ffd640 1 -- 172.21.15.80:0/600311505 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe13801a070 con 0x7fe1480d0510 2024-04-25T13:46:09.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.715+0000 7fe136ffd640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe12806f430 0x7fe128071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:09.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.715+0000 7fe136ffd640 1 -- 172.21.15.80:0/600311505 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fe13808f160 con 0x7fe1480d0510 2024-04-25T13:46:09.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.715+0000 7fe145d9b640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe12806f430 0x7fe128071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:09.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.715+0000 7fe145d9b640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe12806f430 0x7fe128071940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0ad000 tx=0x7fe13c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:09.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.719+0000 7fe136ffd640 1 -- 172.21.15.80:0/600311505 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe13805c050 con 0x7fe1480d0510 2024-04-25T13:46:09.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.903+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe1480b64a0 con 0x7fe1480d0510 2024-04-25T13:46:09.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.903+0000 7fe136ffd640 1 -- 172.21.15.80:0/600311505 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5180 (secure 0 0 0) 0x7fe13805b950 con 0x7fe1480d0510 2024-04-25T13:46:09.903 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:09.903 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":49,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:45:30.006542+0000","modified":"2024-04-25T13:45:33.060919+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7044},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7044":{"gid":7044,"name":"a","rank":0,"incarnation":46,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/817443596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":817443596},{"type":"v1","addr":"172.21.15.80:6835","nonce":817443596}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7044,"qdb_cluster":[7044]},"id":6}]} 2024-04-25T13:46:09.904 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 49 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe12806f430 msgr2=0x7fe128071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe12806f430 0x7fe128071940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe13c0ad000 tx=0x7fe13c008040 comp rx=0 tx=0).stop 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1480d0510 msgr2=0x7fe14813fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1480d0510 0x7fe14813fe00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fe13800e900 tx=0x7fe13800edc0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 shutdown_connections 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe12806f430 0x7fe128071940 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe148161f90 0x7fe1481409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe14815ea70 0x7fe1481404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 --2- 172.21.15.80:0/600311505 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1480d0510 0x7fe14813fe00 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 >> 172.21.15.80:0/600311505 conn(0x7fe14806d080 msgr2=0x7fe1480b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 shutdown_connections 2024-04-25T13:46:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:09.907+0000 7fe14ca72640 1 -- 172.21.15.80:0/600311505 wait complete. 2024-04-25T13:46:09.924 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:10.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd74acb4640 1 Processor -- start 2024-04-25T13:46:10.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd74acb4640 1 -- start start 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd74acb4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7440d59b0 0x7fd7440d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd74acb4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd7440d6300 con 0x7fd7440d2e80 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd74acb4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd744161060 con 0x7fd7440d4330 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd74acb4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd744161340 con 0x7fd7440d59b0 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd748a50640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd7440d2e80 0x7fd7441607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45948/0 (socket says 172.21.15.80:45948) 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd748a50640 1 -- 172.21.15.80:0/2312211358 learned_addr learned my addr 172.21.15.80:0/2312211358 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd743fff640 1 --2- 172.21.15.80:0/2312211358 >> v2:172.21.15.87:3300/0 conn(0x7fd7440d59b0 0x7fd7440d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2687873687 0 0) 0x7fd738001120 con 0x7fd7440d2e80 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd734003530 con 0x7fd7440d2e80 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 468518368 0 0) 0x7fd734003530 con 0x7fd7440d2e80 2024-04-25T13:46:10.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 >> v2:172.21.15.87:3300/0 conn(0x7fd7440d59b0 msgr2=0x7fd7440d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 --2- 172.21.15.80:0/2312211358 >> v2:172.21.15.87:3300/0 conn(0x7fd7440d59b0 0x7fd7440d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 >> v1:172.21.15.87:6789/0 conn(0x7fd7440d4330 legacy=0x7fd7440d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7440d2680 con 0x7fd7440d2e80 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fd738001870 con 0x7fd7440d2e80 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 >> v1:172.21.15.80:6789/0 conn(0x7fd7440d2e80 legacy=0x7fd7441607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7340039c0 0x7fd734003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd734004810 0x7fd734004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd73400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd744161340 con 0x7fd734005260 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd73400c1f0 con 0x7fd734004810 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd73400c3f0 con 0x7fd7340039c0 2024-04-25T13:46:10.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd749251640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd73400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd743fff640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd734004810 0x7fd734004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd748a50640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7340039c0 0x7fd734003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd749251640 1 -- 172.21.15.80:0/2312211358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7340039c0 msgr2=0x7fd734003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:10.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd749251640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7340039c0 0x7fd734003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd749251640 1 -- 172.21.15.80:0/2312211358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd734004810 msgr2=0x7fd734004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:10.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd749251640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd734004810 0x7fd734004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.071+0000 7fd749251640 1 -- 172.21.15.80:0/2312211358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd734003530 con 0x7fd734005260 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd749251640 1 -- 172.21.15.80:0/2312211358 reap_dead start 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd749251640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd73400b9d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fd73c001e40 tx=0x7fd73c01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8fb965d76c9286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c001a90 con 0x7fd734005260 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73c003b80 con 0x7fd734005260 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd7437fe640 1 -- 172.21.15.80:0/2312211358 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c005020 con 0x7fd734005260 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd74acb4640 1 -- 172.21.15.80:0/2312211358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 msgr2=0x7fd73400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd74acb4640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd73400b9d0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fd73c001e40 tx=0x7fd73c01c760 comp rx=0 tx=0).stop 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd74acb4640 1 -- 172.21.15.80:0/2312211358 shutdown_connections 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd74acb4640 1 --2- 172.21.15.80:0/2312211358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd73400b9d0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd74acb4640 1 -- 172.21.15.80:0/2312211358 >> 172.21.15.80:0/2312211358 conn(0x7fd7440737a0 msgr2=0x7fd74406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:10.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd74acb4640 1 -- 172.21.15.80:0/2312211358 shutdown_connections 2024-04-25T13:46:10.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.075+0000 7fd74acb4640 1 -- 172.21.15.80:0/2312211358 wait complete. 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 Processor -- start 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 -- start start 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd744154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7440bb760 0x7fd7440cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd74415c3e0 0x7fd74415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd7440d3290 con 0x7fd734005260 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd744161620 con 0x7fd7440bb760 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd7440d3410 con 0x7fd74415c3e0 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd744154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd744154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38410/0 (socket says 172.21.15.80:38410) 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 -- 172.21.15.80:0/2036273487 learned_addr learned my addr 172.21.15.80:0/2036273487 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd749251640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd74415c3e0 0x7fd74415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd743fff640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7440bb760 0x7fd7440cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 -- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd74415c3e0 msgr2=0x7fd74415c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd74415c3e0 0x7fd74415c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 -- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7440bb760 msgr2=0x7fd7440cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7440bb760 0x7fd7440cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 -- 172.21.15.80:0/2036273487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd74415cd30 con 0x7fd734005260 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd743fff640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7440bb760 0x7fd7440cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd748a50640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd744154bb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd738011560 tx=0x7fd738011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd7417fa640 1 -- 172.21.15.80:0/2036273487 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd738010040 con 0x7fd734005260 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd7417fa640 1 -- 172.21.15.80:0/2036273487 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd73801a070 con 0x7fd734005260 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd7417fa640 1 -- 172.21.15.80:0/2036273487 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73800fae0 con 0x7fd734005260 2024-04-25T13:46:10.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7441facb0 con 0x7fd734005260 2024-04-25T13:46:10.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7441fa380 con 0x7fd734005260 2024-04-25T13:46:10.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.079+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd74406a3a0 con 0x7fd734005260 2024-04-25T13:46:10.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.083+0000 7fd7417fa640 1 -- 172.21.15.80:0/2036273487 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd73800fc80 con 0x7fd734005260 2024-04-25T13:46:10.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.083+0000 7fd7417fa640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd72806f400 0x7fd728071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.083+0000 7fd7417fa640 1 -- 172.21.15.80:0/2036273487 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7fd738092d20 con 0x7fd734005260 2024-04-25T13:46:10.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.083+0000 7fd743fff640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd72806f400 0x7fd728071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.083+0000 7fd743fff640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd72806f400 0x7fd728071910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd7440d24a0 tx=0x7fd730008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.087+0000 7fd7417fa640 1 -- 172.21.15.80:0/2036273487 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd74415cd30 con 0x7fd734005260 2024-04-25T13:46:10.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.235+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7440ccc90 con 0x7fd734005260 2024-04-25T13:46:10.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd7417fa640 1 -- 172.21.15.80:0/2036273487 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) ==== 74+0+16600 (secure 0 0 0) 0x7fd7440d6300 con 0x7fd734005260 2024-04-25T13:46:10.237 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:10.237 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":110,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:45:57.407731+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:45:28.400550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T13:45:28.988401+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:10.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd72806f400 msgr2=0x7fd728071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd72806f400 0x7fd728071910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd7440d24a0 tx=0x7fd730008040 comp rx=0 tx=0).stop 2024-04-25T13:46:10.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 msgr2=0x7fd744154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd744154bb0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fd738011560 tx=0x7fd738011a20 comp rx=0 tx=0).stop 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 shutdown_connections 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd72806f400 0x7fd728071910 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd74415c3e0 0x7fd74415c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7440bb760 0x7fd7440cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 --2- 172.21.15.80:0/2036273487 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd734005260 0x7fd744154bb0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 >> 172.21.15.80:0/2036273487 conn(0x7fd7440737a0 msgr2=0x7fd744156030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 shutdown_connections 2024-04-25T13:46:10.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.239+0000 7fd74acb4640 1 -- 172.21.15.80:0/2036273487 wait complete. 2024-04-25T13:46:10.259 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:46:10.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5109738640 1 Processor -- start 2024-04-25T13:46:10.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5109738640 1 -- start start 2024-04-25T13:46:10.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5109738640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5104161f90 0x7f51040cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5109738640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f510415ec40 0x7f510415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5109738640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51040d0d70 con 0x7f51040d0510 2024-04-25T13:46:10.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5109738640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f51040d1050 con 0x7f5104161f90 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5109738640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f510406dca0 con 0x7f510415ec40 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f51027fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f51040d0510 0x7f510415e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45958/0 (socket says 172.21.15.80:45958) 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5102ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5104161f90 0x7f51040cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f51027fc640 1 -- 172.21.15.80:0/1687228096 learned_addr learned my addr 172.21.15.80:0/1687228096 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f51037fe640 1 --2- 172.21.15.80:0/1687228096 >> v2:172.21.15.87:3300/0 conn(0x7f510415ec40 0x7f510415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5101ffb640 1 -- 172.21.15.80:0/1687228096 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 991249084 0 0) 0x7f51040d0d70 con 0x7f51040d0510 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5101ffb640 1 -- 172.21.15.80:0/1687228096 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50ec003530 con 0x7f51040d0510 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5102ffd640 1 -- 172.21.15.80:0/1687228096 >> v2:172.21.15.87:3300/0 conn(0x7f510415ec40 msgr2=0x7f510415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5102ffd640 1 --2- 172.21.15.80:0/1687228096 >> v2:172.21.15.87:3300/0 conn(0x7f510415ec40 0x7f510415f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5102ffd640 1 -- 172.21.15.80:0/1687228096 >> v1:172.21.15.80:6789/0 conn(0x7f51040d0510 legacy=0x7f510415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.411+0000 7f5102ffd640 1 -- 172.21.15.80:0/1687228096 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510406df80 con 0x7f5104161f90 2024-04-25T13:46:10.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5102ffd640 1 --2- 172.21.15.80:0/1687228096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040cffd0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f51040b4b80 tx=0x7f50f401c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301df087babd9252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5101ffb640 1 -- 172.21.15.80:0/1687228096 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f40b2070 con 0x7f5104161f90 2024-04-25T13:46:10.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5101ffb640 1 -- 172.21.15.80:0/1687228096 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f40a5830 con 0x7f5104161f90 2024-04-25T13:46:10.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5101ffb640 1 -- 172.21.15.80:0/1687228096 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f40a5c00 con 0x7f5104161f90 2024-04-25T13:46:10.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5109738640 1 -- 172.21.15.80:0/1687228096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 msgr2=0x7f51040cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5109738640 1 --2- 172.21.15.80:0/1687228096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040cffd0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f51040b4b80 tx=0x7f50f401c840 comp rx=0 tx=0).stop 2024-04-25T13:46:10.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5109738640 1 -- 172.21.15.80:0/1687228096 shutdown_connections 2024-04-25T13:46:10.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5109738640 1 --2- 172.21.15.80:0/1687228096 >> v2:172.21.15.87:3300/0 conn(0x7f510415ec40 0x7f510415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5109738640 1 --2- 172.21.15.80:0/1687228096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040cffd0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5109738640 1 -- 172.21.15.80:0/1687228096 >> 172.21.15.80:0/1687228096 conn(0x7f510406d080 msgr2=0x7f51040b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:10.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5109738640 1 -- 172.21.15.80:0/1687228096 shutdown_connections 2024-04-25T13:46:10.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.415+0000 7f5109738640 1 -- 172.21.15.80:0/1687228096 wait complete. 2024-04-25T13:46:10.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 Processor -- start 2024-04-25T13:46:10.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 -- start start 2024-04-25T13:46:10.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51040d0510 0x7f51041574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f510415ec40 0x7f51041584f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f510406f4b0 con 0x7f5104161f90 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f510406f630 con 0x7f51040d0510 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51040d0920 con 0x7f510415ec40 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38420/0 (socket says 172.21.15.80:38420) 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 -- 172.21.15.80:0/2875452980 learned_addr learned my addr 172.21.15.80:0/2875452980 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 -- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f510415ec40 msgr2=0x7f51041584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f510415ec40 0x7f51041584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 -- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51040d0510 msgr2=0x7f51041574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51040d0510 0x7f51041574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 -- 172.21.15.80:0/2875452980 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510415b6e0 con 0x7f5104161f90 2024-04-25T13:46:10.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f51037fe640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040b5fa0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00d550 tx=0x7f50fc00da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f50eb7fe640 1 -- 172.21.15.80:0/2875452980 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0125f0 con 0x7f5104161f90 2024-04-25T13:46:10.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f50eb7fe640 1 -- 172.21.15.80:0/2875452980 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50fc019070 con 0x7f5104161f90 2024-04-25T13:46:10.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f50eb7fe640 1 -- 172.21.15.80:0/2875452980 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0129d0 con 0x7f5104161f90 2024-04-25T13:46:10.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510415b9c0 con 0x7f5104161f90 2024-04-25T13:46:10.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510415bf80 con 0x7f5104161f90 2024-04-25T13:46:10.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.419+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510406a3a0 con 0x7f5104161f90 2024-04-25T13:46:10.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.423+0000 7f50eb7fe640 1 -- 172.21.15.80:0/2875452980 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f50fc012b70 con 0x7f5104161f90 2024-04-25T13:46:10.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.423+0000 7f50eb7fe640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50f006f400 0x7f50f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.423+0000 7f50eb7fe640 1 -- 172.21.15.80:0/2875452980 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5709+0+0 (secure 0 0 0) 0x7f50fc08f5b0 con 0x7f5104161f90 2024-04-25T13:46:10.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.423+0000 7f5102ffd640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50f006f400 0x7f50f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.423+0000 7f5102ffd640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50f006f400 0x7f50f0071910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f50f40af000 tx=0x7f50f4004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.427+0000 7f50eb7fe640 1 -- 172.21.15.80:0/2875452980 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f50fc05c5c0 con 0x7f5104161f90 2024-04-25T13:46:10.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.627+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f51040b08a0 con 0x7f5104161f90 2024-04-25T13:46:10.628 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:46:10.627+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:46:10.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.631+0000 7f50eb7fe640 1 -- 172.21.15.80:0/2875452980 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7f50fc05bec0 con 0x7f5104161f90 2024-04-25T13:46:10.630 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50f006f400 msgr2=0x7f50f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50f006f400 0x7f50f0071910 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f50f40af000 tx=0x7f50f4004ac0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 msgr2=0x7f51040b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040b5fa0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00d550 tx=0x7f50fc00da10 comp rx=0 tx=0).stop 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 shutdown_connections 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f50f006f400 0x7f50f0071910 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5104161f90 0x7f51040b5fa0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f510415ec40 0x7f51041584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 --2- 172.21.15.80:0/2875452980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51040d0510 0x7f51041574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 >> 172.21.15.80:0/2875452980 conn(0x7f510406d080 msgr2=0x7f510415e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 shutdown_connections 2024-04-25T13:46:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.639+0000 7f5109738640 1 -- 172.21.15.80:0/2875452980 wait complete. 2024-04-25T13:46:10.640 INFO:tasks.ceph.mds.a.smithi080.stderr: -9761> 2024-04-25T13:45:32.607+0000 7fcff08ad640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:46:10.660 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:46:10.713 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:46:10.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.807+0000 7f991b664640 1 Processor -- start 2024-04-25T13:46:10.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.807+0000 7f991b664640 1 -- start start 2024-04-25T13:46:10.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f991b664640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9914164ad0 0x7f99140b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f991b664640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140b7d00 0x7f99140d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f991b664640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99140d4380 con 0x7f99140d2990 2024-04-25T13:46:10.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f991b664640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f99140b8110 con 0x7f9914164ad0 2024-04-25T13:46:10.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f991b664640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9914164640 con 0x7f99140b7d00 2024-04-25T13:46:10.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9919c01640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f99140d2990 0x7f9914160960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45962/0 (socket says 172.21.15.80:45962) 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9919c01640 1 -- 172.21.15.80:0/2838004005 learned_addr learned my addr 172.21.15.80:0/2838004005 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9918bff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140b7d00 0x7f99140d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9919400640 1 --2- 172.21.15.80:0/2838004005 >> v2:172.21.15.87:3300/0 conn(0x7f9914164ad0 0x7f99140b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3917803326 0 0) 0x7f99140d4380 con 0x7f99140d2990 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9900003530 con 0x7f99140d2990 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1054051781 0 0) 0x7f9900003530 con 0x7f99140d2990 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140b7d00 msgr2=0x7f99140d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140b7d00 0x7f99140d2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 >> v2:172.21.15.87:3300/0 conn(0x7f9914164ad0 msgr2=0x7f99140b77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 --2- 172.21.15.80:0/2838004005 >> v2:172.21.15.87:3300/0 conn(0x7f9914164ad0 0x7f99140b77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99140b8610 con 0x7f99140d2990 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f990c001a00 con 0x7f99140d2990 2024-04-25T13:46:10.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 >> v1:172.21.15.80:6789/0 conn(0x7f99140d2990 legacy=0x7f9914160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99000039c0 0x7f9900003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f9900004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140b7d00 0x7f9900005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f99140b8110 con 0x7f9900004810 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9914164640 con 0x7f99000039c0 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f990000ba80 con 0x7f99140b7d00 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9918bff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f9900004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9919400640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99000039c0 0x7f9900003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9919c01640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140b7d00 0x7f9900005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9918bff640 1 -- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140b7d00 msgr2=0x7f9900005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9918bff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140b7d00 0x7f9900005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9918bff640 1 -- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99000039c0 msgr2=0x7f9900003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9918bff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99000039c0 0x7f9900003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.811+0000 7f9918bff640 1 -- 172.21.15.80:0/2838004005 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9900003530 con 0x7f9900004810 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f9918bff640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f9900004c20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9904009ae0 tx=0x7f990401ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c112c857ebf60653 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040096b0 con 0x7f9900004810 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99040a6470 con 0x7f9900004810 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f990bfff640 1 -- 172.21.15.80:0/2838004005 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99040a6860 con 0x7f9900004810 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f991b664640 1 -- 172.21.15.80:0/2838004005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 msgr2=0x7f9900004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f991b664640 1 --2- 172.21.15.80:0/2838004005 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f9900004c20 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f9904009ae0 tx=0x7f990401ca90 comp rx=0 tx=0).stop 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f9919c01640 1 -- 172.21.15.80:0/2838004005 reap_dead start 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f991b664640 1 -- 172.21.15.80:0/2838004005 shutdown_connections 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f991b664640 1 -- 172.21.15.80:0/2838004005 >> 172.21.15.80:0/2838004005 conn(0x7f9914073570 msgr2=0x7f991406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f991b664640 1 -- 172.21.15.80:0/2838004005 shutdown_connections 2024-04-25T13:46:10.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f991b664640 1 -- 172.21.15.80:0/2838004005 wait complete. 2024-04-25T13:46:10.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f991b664640 1 Processor -- start 2024-04-25T13:46:10.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.815+0000 7f991b664640 1 -- start start 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f991415a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140bace0 0x7f99140bb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991415caa0 0x7f991415ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f991416b0d0 con 0x7f9900004810 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f99140d2da0 con 0x7f991415caa0 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f991416b250 con 0x7f99140bace0 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f991415a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f991415a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38460/0 (socket says 172.21.15.80:38460) 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 -- 172.21.15.80:0/166664165 learned_addr learned my addr 172.21.15.80:0/166664165 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:10.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9918bff640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140bace0 0x7f99140bb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919c01640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991415caa0 0x7f991415ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 -- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140bace0 msgr2=0x7f99140bb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140bace0 0x7f99140bb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 -- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991415caa0 msgr2=0x7f991415ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991415caa0 0x7f991415ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 -- 172.21.15.80:0/166664165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f991415d3f0 con 0x7f9900004810 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9919400640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f991415a0f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f99100116d0 tx=0x7f9910011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9909ffb640 1 -- 172.21.15.80:0/166664165 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991000f9c0 con 0x7f9900004810 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9909ffb640 1 -- 172.21.15.80:0/166664165 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f991000fb60 con 0x7f9900004810 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99141fd3c0 con 0x7f9900004810 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9909ffb640 1 -- 172.21.15.80:0/166664165 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9910015460 con 0x7f9900004810 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99141fd980 con 0x7f9900004810 2024-04-25T13:46:10.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f991406a3a0 con 0x7f9900004810 2024-04-25T13:46:10.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9909ffb640 1 -- 172.21.15.80:0/166664165 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9910015600 con 0x7f9900004810 2024-04-25T13:46:10.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9909ffb640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98f806f400 0x7f98f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:10.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9909ffb640 1 -- 172.21.15.80:0/166664165 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7f991008fd30 con 0x7f9900004810 2024-04-25T13:46:10.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.819+0000 7f9918bff640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98f806f400 0x7f98f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:10.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.823+0000 7f9918bff640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98f806f400 0x7f98f8071910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9904009ae0 tx=0x7f99040046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:10.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:10.827+0000 7f9909ffb640 1 -- 172.21.15.80:0/166664165 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f991415d3f0 con 0x7f9900004810 2024-04-25T13:46:11.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.023+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f991416a200 con 0x7f9900004810 2024-04-25T13:46:11.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.635+0000 7f9909ffb640 1 -- 172.21.15.80:0/166664165 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v51) ==== 104+0+0 (secure 0 0 0) 0x7f9910019030 con 0x7f9900004810 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.635+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98f806f400 msgr2=0x7f98f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.635+0000 7f991b664640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98f806f400 0x7f98f8071910 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f9904009ae0 tx=0x7f99040046d0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.635+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 msgr2=0x7f991415a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.635+0000 7f991b664640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f991415a0f0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f99100116d0 tx=0x7f9910011b90 comp rx=0 tx=0).stop 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.639+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 shutdown_connections 2024-04-25T13:46:11.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.639+0000 7f991b664640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98f806f400 0x7f98f8071910 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.639+0000 7f991b664640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f991415caa0 0x7f991415ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.639+0000 7f991b664640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99140bace0 0x7f99140bb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.639+0000 7f991b664640 1 --2- 172.21.15.80:0/166664165 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9900004810 0x7f991415a0f0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.639+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 >> 172.21.15.80:0/166664165 conn(0x7f9914073570 msgr2=0x7f991415ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.639+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 shutdown_connections 2024-04-25T13:46:11.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.639+0000 7f991b664640 1 -- 172.21.15.80:0/166664165 wait complete. 2024-04-25T13:46:11.660 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:46:11.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.819+0000 7fd2d3792640 1 Processor -- start 2024-04-25T13:46:11.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.819+0000 7fd2d3792640 1 -- start start 2024-04-25T13:46:11.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.819+0000 7fd2d3792640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd2cc0b0f10 0x7fd2cc0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.819+0000 7fd2d3792640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd2cc0ce2a0 0x7fd2cc0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.819+0000 7fd2d3792640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd2cc0b1320 con 0x7fd2cc0cf730 2024-04-25T13:46:11.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.819+0000 7fd2d3792640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd2cc0b5820 con 0x7fd2cc0b0f10 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.819+0000 7fd2d3792640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd2cc0d13f0 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d152e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd2cc0b0f10 0x7fd2cc0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d1d2f640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd2cc0cf730 0x7fd2cc0cfb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48856/0 (socket says 172.21.15.80:48856) 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d1d2f640 1 -- 172.21.15.80:0/1867776317 learned_addr learned my addr 172.21.15.80:0/1867776317 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d0d2d640 1 --2- 172.21.15.80:0/1867776317 >> v2:172.21.15.87:3300/0 conn(0x7fd2cc0ce2a0 0x7fd2cc0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d152e640 1 -- 172.21.15.80:0/1867776317 >> v2:172.21.15.87:3300/0 conn(0x7fd2cc0ce2a0 msgr2=0x7fd2cc0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d152e640 1 --2- 172.21.15.80:0/1867776317 >> v2:172.21.15.87:3300/0 conn(0x7fd2cc0ce2a0 0x7fd2cc0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d152e640 1 -- 172.21.15.80:0/1867776317 >> v1:172.21.15.80:6789/0 conn(0x7fd2cc0cf730 legacy=0x7fd2cc0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d152e640 1 -- 172.21.15.80:0/1867776317 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc0cebf0 con 0x7fd2cc0b0f10 2024-04-25T13:46:11.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d152e640 1 --2- 172.21.15.80:0/1867776317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0b0f10 0x7fd2cc0d0c10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0b4600 tx=0x7fd2c801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec7fbc5dba5cc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2c3fff640 1 -- 172.21.15.80:0/1867776317 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80b0070 con 0x7fd2cc0b0f10 2024-04-25T13:46:11.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2c3fff640 1 -- 172.21.15.80:0/1867776317 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2c80ab600 con 0x7fd2cc0b0f10 2024-04-25T13:46:11.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2c3fff640 1 -- 172.21.15.80:0/1867776317 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80ab9d0 con 0x7fd2cc0b0f10 2024-04-25T13:46:11.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d3792640 1 -- 172.21.15.80:0/1867776317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0b0f10 msgr2=0x7fd2cc0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d3792640 1 --2- 172.21.15.80:0/1867776317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0b0f10 0x7fd2cc0d0c10 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0b4600 tx=0x7fd2c801c310 comp rx=0 tx=0).stop 2024-04-25T13:46:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d3792640 1 -- 172.21.15.80:0/1867776317 shutdown_connections 2024-04-25T13:46:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d3792640 1 --2- 172.21.15.80:0/1867776317 >> v2:172.21.15.87:3300/0 conn(0x7fd2cc0ce2a0 0x7fd2cc0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d3792640 1 --2- 172.21.15.80:0/1867776317 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0b0f10 0x7fd2cc0d0c10 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d3792640 1 -- 172.21.15.80:0/1867776317 >> 172.21.15.80:0/1867776317 conn(0x7fd2cc0b35c0 msgr2=0x7fd2cc0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d3792640 1 -- 172.21.15.80:0/1867776317 shutdown_connections 2024-04-25T13:46:11.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.823+0000 7fd2d3792640 1 -- 172.21.15.80:0/1867776317 wait complete. 2024-04-25T13:46:11.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 Processor -- start 2024-04-25T13:46:11.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 -- start start 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2cc0b0f10 0x7fd2cc152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0ce2a0 0x7fd2cc1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2cc0cf730 0x7fd2cc159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc0b4d40 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd2cc0d1150 con 0x7fd2cc0cf730 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd2cc0b5020 con 0x7fd2cc0b0f10 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0ce2a0 0x7fd2cc1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0ce2a0 0x7fd2cc1557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47814/0 (socket says 172.21.15.80:47814) 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 -- 172.21.15.80:0/2824719705 learned_addr learned my addr 172.21.15.80:0/2824719705 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d1d2f640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2cc0cf730 0x7fd2cc159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 -- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2cc0b0f10 msgr2=0x7fd2cc152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2cc0b0f10 0x7fd2cc152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 -- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2cc0cf730 msgr2=0x7fd2cc159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2cc0cf730 0x7fd2cc159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 -- 172.21.15.80:0/2824719705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2cc1f7e50 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d1d2f640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2cc0cf730 0x7fd2cc159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d0d2d640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0ce2a0 0x7fd2cc1557c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00d620 tx=0x7fd2bc00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2c1ffb640 1 -- 172.21.15.80:0/2824719705 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00b620 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2c1ffb640 1 -- 172.21.15.80:0/2824719705 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2bc019070 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2c1ffb640 1 -- 172.21.15.80:0/2824719705 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00b9e0 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2cc1f8130 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.827+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2cc1f86f0 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.831+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2cc156110 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.831+0000 7fd2c1ffb640 1 -- 172.21.15.80:0/2824719705 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd2bc00bb80 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.831+0000 7fd2c1ffb640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2b006f400 0x7fd2b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.831+0000 7fd2c1ffb640 1 -- 172.21.15.80:0/2824719705 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5795+0+0 (secure 0 0 0) 0x7fd2bc08f700 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.831+0000 7fd2d152e640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2b006f400 0x7fd2b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:11.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.831+0000 7fd2d152e640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2b006f400 0x7fd2b0071910 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80ad000 tx=0x7fd2c8004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:11.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.835+0000 7fd2c1ffb640 1 -- 172.21.15.80:0/2824719705 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd2bc05c620 con 0x7fd2cc0ce2a0 2024-04-25T13:46:11.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:11.999+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd2cc1f8b30 con 0x7fd2cc0ce2a0 2024-04-25T13:46:12.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.643+0000 7fd2c1ffb640 1 -- 172.21.15.80:0/2824719705 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v112) ==== 181+0+0 (secure 0 0 0) 0x7fd2bc05bf20 con 0x7fd2cc0ce2a0 2024-04-25T13:46:12.643 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:46:12.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2b006f400 msgr2=0x7fd2b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2b006f400 0x7fd2b0071910 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80ad000 tx=0x7fd2c8004740 comp rx=0 tx=0).stop 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0ce2a0 msgr2=0x7fd2cc1557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0ce2a0 0x7fd2cc1557c0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00d620 tx=0x7fd2bc00dae0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 shutdown_connections 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd2b006f400 0x7fd2b0071910 secure :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fd2c80ad000 tx=0x7fd2c8004740 comp rx=0 tx=0).stop 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd2cc0cf730 0x7fd2cc159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd2cc0ce2a0 0x7fd2cc1557c0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 --2- 172.21.15.80:0/2824719705 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd2cc0b0f10 0x7fd2cc152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 >> 172.21.15.80:0/2824719705 conn(0x7fd2cc0b35c0 msgr2=0x7fd2cc15b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 shutdown_connections 2024-04-25T13:46:12.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.647+0000 7fd2d3792640 1 -- 172.21.15.80:0/2824719705 wait complete. 2024-04-25T13:46:12.671 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:46:12.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123c5c3640 1 Processor -- start 2024-04-25T13:46:12.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123c5c3640 1 -- start start 2024-04-25T13:46:12.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123c5c3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f123406cb60 0x7f123406cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123c5c3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f123406d560 con 0x7f12340ce0c0 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123c5c3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f12340d28d0 con 0x7f12340cf410 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123c5c3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f12340cff30 con 0x7f123406cb60 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123ab60640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f12340ce0c0 0x7f12340ce4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48860/0 (socket says 172.21.15.80:48860) 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123ab60640 1 -- 172.21.15.80:0/718091687 learned_addr learned my addr 172.21.15.80:0/718091687 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3944339816 0 0) 0x7f123406d560 con 0x7f12340ce0c0 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123a35f640 1 --2- 172.21.15.80:0/718091687 >> v2:172.21.15.87:3300/0 conn(0x7f123406cb60 0x7f123406cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1220003530 con 0x7f12340ce0c0 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1959272209 0 0) 0x7f1220003530 con 0x7f12340ce0c0 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 >> v2:172.21.15.87:3300/0 conn(0x7f123406cb60 msgr2=0x7f123406cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.823+0000 7f123935d640 1 --2- 172.21.15.80:0/718091687 >> v2:172.21.15.87:3300/0 conn(0x7f123406cb60 0x7f123406cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 >> v1:172.21.15.87:6789/0 conn(0x7f12340cf410 legacy=0x7f12340cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12340d2bb0 con 0x7f12340ce0c0 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f122c001a00 con 0x7f12340ce0c0 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 >> v1:172.21.15.80:6789/0 conn(0x7f12340ce0c0 legacy=0x7f12340ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12200039c0 0x7f1220003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1220004810 0x7f1220004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f122000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f12340cff30 con 0x7f12200051d0 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f122000c170 con 0x7f12200039c0 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f122000c370 con 0x7f1220004810 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123ab60640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f122000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123a35f640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12200039c0 0x7f1220003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f1239b5e640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1220004810 0x7f1220004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123ab60640 1 -- 172.21.15.80:0/718091687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1220004810 msgr2=0x7f1220004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123ab60640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1220004810 0x7f1220004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123ab60640 1 -- 172.21.15.80:0/718091687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12200039c0 msgr2=0x7f1220003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:12.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123ab60640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12200039c0 0x7f1220003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123ab60640 1 -- 172.21.15.80:0/718091687 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1220003530 con 0x7f12200051d0 2024-04-25T13:46:12.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123ab60640 1 -- 172.21.15.80:0/718091687 reap_dead start 2024-04-25T13:46:12.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123ab60640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f122000b9d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f12340b4a90 tx=0x7f122c01d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96238d27178427d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c00a4a0 con 0x7f12200051d0 2024-04-25T13:46:12.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f122c0ad650 con 0x7f12200051d0 2024-04-25T13:46:12.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123935d640 1 -- 172.21.15.80:0/718091687 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f122c0ada40 con 0x7f12200051d0 2024-04-25T13:46:12.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123c5c3640 1 -- 172.21.15.80:0/718091687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 msgr2=0x7f122000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123c5c3640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f122000b9d0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f12340b4a90 tx=0x7f122c01d2a0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123c5c3640 1 -- 172.21.15.80:0/718091687 shutdown_connections 2024-04-25T13:46:12.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123c5c3640 1 --2- 172.21.15.80:0/718091687 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f122000b9d0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123c5c3640 1 -- 172.21.15.80:0/718091687 >> 172.21.15.80:0/718091687 conn(0x7f12340b5580 msgr2=0x7f12340b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:12.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123c5c3640 1 -- 172.21.15.80:0/718091687 shutdown_connections 2024-04-25T13:46:12.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.827+0000 7f123c5c3640 1 -- 172.21.15.80:0/718091687 wait complete. 2024-04-25T13:46:12.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 Processor -- start 2024-04-25T13:46:12.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 -- start start 2024-04-25T13:46:12.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f12341ddb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340ca990 0x7f12340b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340cada0 0x7f12340b2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f12340d30d0 con 0x7f12200051d0 2024-04-25T13:46:12.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f12340b4df0 con 0x7f12340cada0 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f12340d3250 con 0x7f12340ca990 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f12341ddb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f12341ddb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47828/0 (socket says 172.21.15.80:47828) 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 -- 172.21.15.80:0/2024508953 learned_addr learned my addr 172.21.15.80:0/2024508953 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123ab60640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340cada0 0x7f12340b2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f1239b5e640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340ca990 0x7f12340b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 -- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340ca990 msgr2=0x7f12340b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340ca990 0x7f12340b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 -- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340cada0 msgr2=0x7f12340b2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340cada0 0x7f12340b2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 -- 172.21.15.80:0/2024508953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12340cd6b0 con 0x7f12200051d0 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123ab60640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340cada0 0x7f12340b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123a35f640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f12341ddb60 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1230011560 tx=0x7f1230011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f122affd640 1 -- 172.21.15.80:0/2024508953 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230015870 con 0x7f12200051d0 2024-04-25T13:46:12.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f122affd640 1 -- 172.21.15.80:0/2024508953 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1230015a10 con 0x7f12200051d0 2024-04-25T13:46:12.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12340cd990 con 0x7f12200051d0 2024-04-25T13:46:12.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f122affd640 1 -- 172.21.15.80:0/2024508953 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1230015de0 con 0x7f12200051d0 2024-04-25T13:46:12.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.831+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12340c8810 con 0x7f12200051d0 2024-04-25T13:46:12.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.835+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12340cdeb0 con 0x7f12200051d0 2024-04-25T13:46:12.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.835+0000 7f122affd640 1 -- 172.21.15.80:0/2024508953 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f123001d400 con 0x7f12200051d0 2024-04-25T13:46:12.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.835+0000 7f122affd640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f121806f400 0x7f1218071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:12.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.835+0000 7f122affd640 1 -- 172.21.15.80:0/2024508953 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5357+0+0 (secure 0 0 0) 0x7f123008f9e0 con 0x7f12200051d0 2024-04-25T13:46:12.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.835+0000 7f1239b5e640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f121806f400 0x7f1218071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:12.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.835+0000 7f1239b5e640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f121806f400 0x7f1218071910 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1224007fd0 tx=0x7f1224009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:12.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.839+0000 7f122affd640 1 -- 172.21.15.80:0/2024508953 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f12340cdeb0 con 0x7f12200051d0 2024-04-25T13:46:12.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:12.991+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f123406a440 con 0x7f12200051d0 2024-04-25T13:46:13.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.647+0000 7f122affd640 1 -- 172.21.15.80:0/2024508953 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v113) ==== 169+0+0 (secure 0 0 0) 0x7f1230019030 con 0x7f12200051d0 2024-04-25T13:46:13.646 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:46:13.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f121806f400 msgr2=0x7f1218071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f121806f400 0x7f1218071910 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f1224007fd0 tx=0x7f1224009040 comp rx=0 tx=0).stop 2024-04-25T13:46:13.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 msgr2=0x7f12341ddb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f12341ddb60 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f1230011560 tx=0x7f1230011a20 comp rx=0 tx=0).stop 2024-04-25T13:46:13.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 shutdown_connections 2024-04-25T13:46:13.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f121806f400 0x7f1218071910 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12340cada0 0x7f12340b2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f12340ca990 0x7f12340b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 --2- 172.21.15.80:0/2024508953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f12200051d0 0x7f12341ddb60 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 >> 172.21.15.80:0/2024508953 conn(0x7f12340b5580 msgr2=0x7f12340d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 shutdown_connections 2024-04-25T13:46:13.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.651+0000 7f123c5c3640 1 -- 172.21.15.80:0/2024508953 wait complete. 2024-04-25T13:46:13.681 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:46:13.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.835+0000 7f9cb2d23640 1 Processor -- start 2024-04-25T13:46:13.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.835+0000 7f9cb2d23640 1 -- start start 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cb2d23640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9cac06edb0 0x7f9cac06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cb2d23640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9cac161b60 0x7f9cac161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cb2d23640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cac1624b0 con 0x7f9cac0b5330 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cb2d23640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9cac06f700 con 0x7f9cac06edb0 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cb2d23640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9cac1629b0 con 0x7f9cac161b60 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cabfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9cac06edb0 0x7f9cac06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cabfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9cac06edb0 0x7f9cac06f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47842/0 (socket says 172.21.15.80:47842) 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cabfff640 1 -- 172.21.15.80:0/2315258792 learned_addr learned my addr 172.21.15.80:0/2315258792 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cb12c0640 1 --2- 172.21.15.80:0/2315258792 >> v2:172.21.15.87:3300/0 conn(0x7f9cac161b60 0x7f9cac161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cabfff640 1 -- 172.21.15.80:0/2315258792 >> v2:172.21.15.87:3300/0 conn(0x7f9cac161b60 msgr2=0x7f9cac161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cabfff640 1 --2- 172.21.15.80:0/2315258792 >> v2:172.21.15.87:3300/0 conn(0x7f9cac161b60 0x7f9cac161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cabfff640 1 -- 172.21.15.80:0/2315258792 >> v1:172.21.15.87:6789/0 conn(0x7f9cac0b5330 legacy=0x7f9cac0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cabfff640 1 -- 172.21.15.80:0/2315258792 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac0b5740 con 0x7f9cac06edb0 2024-04-25T13:46:13.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.839+0000 7f9cabfff640 1 --2- 172.21.15.80:0/2315258792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac06edb0 0x7f9cac06f1c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9cac147cb0 tx=0x7f9ca001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7d6f1e001fe031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cab7fe640 1 -- 172.21.15.80:0/2315258792 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00b0070 con 0x7f9cac06edb0 2024-04-25T13:46:13.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cab7fe640 1 -- 172.21.15.80:0/2315258792 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca00ab600 con 0x7f9cac06edb0 2024-04-25T13:46:13.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cab7fe640 1 -- 172.21.15.80:0/2315258792 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca00ab9d0 con 0x7f9cac06edb0 2024-04-25T13:46:13.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 -- 172.21.15.80:0/2315258792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac06edb0 msgr2=0x7f9cac06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/2315258792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac06edb0 0x7f9cac06f1c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f9cac147cb0 tx=0x7f9ca001c460 comp rx=0 tx=0).stop 2024-04-25T13:46:13.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 -- 172.21.15.80:0/2315258792 shutdown_connections 2024-04-25T13:46:13.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/2315258792 >> v2:172.21.15.87:3300/0 conn(0x7f9cac161b60 0x7f9cac161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/2315258792 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac06edb0 0x7f9cac06f1c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 -- 172.21.15.80:0/2315258792 >> 172.21.15.80:0/2315258792 conn(0x7f9cac0b4160 msgr2=0x7f9cac0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:13.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 -- 172.21.15.80:0/2315258792 shutdown_connections 2024-04-25T13:46:13.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 -- 172.21.15.80:0/2315258792 wait complete. 2024-04-25T13:46:13.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 Processor -- start 2024-04-25T13:46:13.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.843+0000 7f9cb2d23640 1 -- start start 2024-04-25T13:46:13.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cac06edb0 0x7f9cac1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cac0b5330 0x7f9cac152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac161b60 0x7f9cac153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac06e890 con 0x7f9cac161b60 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9cac06e5b0 con 0x7f9cac06edb0 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9cac0b0cd0 con 0x7f9cac0b5330 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac161b60 0x7f9cac153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac161b60 0x7f9cac153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47858/0 (socket says 172.21.15.80:47858) 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 -- 172.21.15.80:0/1966169270 learned_addr learned my addr 172.21.15.80:0/1966169270 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cabfff640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cac0b5330 0x7f9cac152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb0abf640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cac06edb0 0x7f9cac1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 -- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cac0b5330 msgr2=0x7f9cac152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cac0b5330 0x7f9cac152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 -- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cac06edb0 msgr2=0x7f9cac1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cac06edb0 0x7f9cac1412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:13.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 -- 172.21.15.80:0/1966169270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cac14f620 con 0x7f9cac161b60 2024-04-25T13:46:13.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cabfff640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cac0b5330 0x7f9cac152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:13.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb12c0640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac161b60 0x7f9cac153290 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4007e90 tx=0x7f9ca400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9ca97fa640 1 -- 172.21.15.80:0/1966169270 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4014420 con 0x7f9cac161b60 2024-04-25T13:46:13.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9ca97fa640 1 -- 172.21.15.80:0/1966169270 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca4019070 con 0x7f9cac161b60 2024-04-25T13:46:13.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cac14f900 con 0x7f9cac161b60 2024-04-25T13:46:13.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9ca97fa640 1 -- 172.21.15.80:0/1966169270 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca4014800 con 0x7f9cac161b60 2024-04-25T13:46:13.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cac14fec0 con 0x7f9cac161b60 2024-04-25T13:46:13.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cac15d0b0 con 0x7f9cac161b60 2024-04-25T13:46:13.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9ca97fa640 1 -- 172.21.15.80:0/1966169270 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9ca40149a0 con 0x7f9cac161b60 2024-04-25T13:46:13.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9ca97fa640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9806f400 0x7f9c98071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:13.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9ca97fa640 1 -- 172.21.15.80:0/1966169270 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4943+0+0 (secure 0 0 0) 0x7f9ca408f870 con 0x7f9cac161b60 2024-04-25T13:46:13.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.847+0000 7f9cb0abf640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9806f400 0x7f9c98071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:13.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.851+0000 7f9cb0abf640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9806f400 0x7f9c98071910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0015b0 tx=0x7f9c9c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:13.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:13.855+0000 7f9ca97fa640 1 -- 172.21.15.80:0/1966169270 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ca405ca60 con 0x7f9cac161b60 2024-04-25T13:46:14.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.003+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f9cac06a3a0 con 0x7f9cac161b60 2024-04-25T13:46:14.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9ca97fa640 1 -- 172.21.15.80:0/1966169270 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f9ca405c360 con 0x7f9cac161b60 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9806f400 msgr2=0x7f9c98071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9806f400 0x7f9c98071910 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9c9c0015b0 tx=0x7f9c9c009040 comp rx=0 tx=0).stop 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac161b60 msgr2=0x7f9cac153290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac161b60 0x7f9cac153290 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4007e90 tx=0x7f9ca400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 shutdown_connections 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9806f400 0x7f9c98071910 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cac161b60 0x7f9cac153290 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cac0b5330 0x7f9cac152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 --2- 172.21.15.80:0/1966169270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cac06edb0 0x7f9cac1412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 >> 172.21.15.80:0/1966169270 conn(0x7f9cac0b4160 msgr2=0x7f9cac0d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 shutdown_connections 2024-04-25T13:46:14.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.011+0000 7f9cb2d23640 1 -- 172.21.15.80:0/1966169270 wait complete. 2024-04-25T13:46:14.029 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2024-04-25T13:46:14.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.183+0000 7f23e2f42640 1 Processor -- start 2024-04-25T13:46:14.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.183+0000 7f23e2f42640 1 -- start start 2024-04-25T13:46:14.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e2f42640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f23dc06f5a0 0x7f23dc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e2f42640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc161070 0x7f23dc161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e2f42640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23dc06fcf0 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e2f42640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f23dc1619c0 con 0x7f23dc06f5a0 2024-04-25T13:46:14.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e2f42640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc073f10 con 0x7f23dc161070 2024-04-25T13:46:14.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23dbfff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc161070 0x7f23dc161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e0cde640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f23dc06f5a0 0x7f23dc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e14df640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f23dc0b4bd0 0x7f23dc0b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:40814/0 (socket says 172.21.15.80:40814) 2024-04-25T13:46:14.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e14df640 1 -- 172.21.15.80:0/3754373557 learned_addr learned my addr 172.21.15.80:0/3754373557 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:14.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e0cde640 1 --2- 172.21.15.80:0/3754373557 >> v2:172.21.15.87:3300/0 conn(0x7f23dc06f5a0 0x7f23dc160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:53998/0 (socket says 172.21.15.80:53998) 2024-04-25T13:46:14.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e0cde640 1 -- 172.21.15.80:0/3754373557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc161070 msgr2=0x7f23dc161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:14.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e0cde640 1 --2- 172.21.15.80:0/3754373557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc161070 0x7f23dc161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e0cde640 1 -- 172.21.15.80:0/3754373557 >> v1:172.21.15.87:6789/0 conn(0x7f23dc0b4bd0 legacy=0x7f23dc0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23e0cde640 1 -- 172.21.15.80:0/3754373557 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc0b0c60 con 0x7f23dc06f5a0 2024-04-25T13:46:14.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.187+0000 7f23dbfff640 1 --2- 172.21.15.80:0/3754373557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc161070 0x7f23dc161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:46:14.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e0cde640 1 --2- 172.21.15.80:0/3754373557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc06f5a0 0x7f23dc160b30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23c40097f0 tx=0x7f23c401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72891dddc5ecf9e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:14.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23db7fe640 1 -- 172.21.15.80:0/3754373557 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40b0070 con 0x7f23dc06f5a0 2024-04-25T13:46:14.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23db7fe640 1 -- 172.21.15.80:0/3754373557 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23c40ab600 con 0x7f23dc06f5a0 2024-04-25T13:46:14.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23db7fe640 1 -- 172.21.15.80:0/3754373557 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c40ab9d0 con 0x7f23dc06f5a0 2024-04-25T13:46:14.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e2f42640 1 -- 172.21.15.80:0/3754373557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc06f5a0 msgr2=0x7f23dc160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3754373557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc06f5a0 0x7f23dc160b30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f23c40097f0 tx=0x7f23c401c410 comp rx=0 tx=0).stop 2024-04-25T13:46:14.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e2f42640 1 -- 172.21.15.80:0/3754373557 shutdown_connections 2024-04-25T13:46:14.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3754373557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc161070 0x7f23dc161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3754373557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc06f5a0 0x7f23dc160b30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e2f42640 1 -- 172.21.15.80:0/3754373557 >> 172.21.15.80:0/3754373557 conn(0x7f23dc06d080 msgr2=0x7f23dc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:14.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e2f42640 1 -- 172.21.15.80:0/3754373557 shutdown_connections 2024-04-25T13:46:14.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.191+0000 7f23e2f42640 1 -- 172.21.15.80:0/3754373557 wait complete. 2024-04-25T13:46:14.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 Processor -- start 2024-04-25T13:46:14.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 -- start start 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc06f5a0 0x7f23dc1fb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23dc0b4bd0 0x7f23dc1fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc161070 0x7f23dc15d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc16a860 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc16a6e0 con 0x7f23dc161070 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc16a9e0 con 0x7f23dc06f5a0 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23dc0b4bd0 0x7f23dc1fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23dc0b4bd0 0x7f23dc1fbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47866/0 (socket says 172.21.15.80:47866) 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 -- 172.21.15.80:0/3828925525 learned_addr learned my addr 172.21.15.80:0/3828925525 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e0cde640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc06f5a0 0x7f23dc1fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e14df640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc161070 0x7f23dc15d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 -- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc06f5a0 msgr2=0x7f23dc1fb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc06f5a0 0x7f23dc1fb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 -- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc161070 msgr2=0x7f23dc15d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc161070 0x7f23dc15d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 -- 172.21.15.80:0/3828925525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc14e2b0 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23dbfff640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23dc0b4bd0 0x7f23dc1fbdb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00e9b0 tx=0x7f23cc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23d97fa640 1 -- 172.21.15.80:0/3828925525 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc017070 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23d97fa640 1 -- 172.21.15.80:0/3828925525 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23cc00ca00 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23d97fa640 1 -- 172.21.15.80:0/3828925525 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc00cdd0 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc15e8c0 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.196 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc14e830 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.195+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23dc15e630 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.199+0000 7f23d97fa640 1 -- 172.21.15.80:0/3828925525 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f23cc01a070 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.199+0000 7f23d97fa640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23bc06f400 0x7f23bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.199+0000 7f23d97fa640 1 -- 172.21.15.80:0/3828925525 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4943+0+0 (secure 0 0 0) 0x7f23cc08f540 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.199+0000 7f23e0cde640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23bc06f400 0x7f23bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.199+0000 7f23e0cde640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23bc06f400 0x7f23bc071910 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ad000 tx=0x7f23c40a4f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:14.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.203+0000 7f23d97fa640 1 -- 172.21.15.80:0/3828925525 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f23cc05c8a0 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.415+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) -- 0x7f23dc06a3a0 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23d97fa640 1 -- 172.21.15.80:0/3828925525 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v308) ==== 137+0+0 (secure 0 0 0) 0x7f23cc05c1a0 con 0x7f23dc0b4bd0 2024-04-25T13:46:14.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23bc06f400 msgr2=0x7f23bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23bc06f400 0x7f23bc071910 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f23c40ad000 tx=0x7f23c40a4f90 comp rx=0 tx=0).stop 2024-04-25T13:46:14.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23dc0b4bd0 msgr2=0x7f23dc1fbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23dc0b4bd0 0x7f23dc1fbdb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00e9b0 tx=0x7f23cc00ee70 comp rx=0 tx=0).stop 2024-04-25T13:46:14.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 shutdown_connections 2024-04-25T13:46:14.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23bc06f400 0x7f23bc071910 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23dc161070 0x7f23dc15d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23dc0b4bd0 0x7f23dc1fbdb0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 --2- 172.21.15.80:0/3828925525 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f23dc06f5a0 0x7f23dc1fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 >> 172.21.15.80:0/3828925525 conn(0x7f23dc06d080 msgr2=0x7f23dc0d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:14.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 shutdown_connections 2024-04-25T13:46:14.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.651+0000 7f23e2f42640 1 -- 172.21.15.80:0/3828925525 wait complete. 2024-04-25T13:46:14.673 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2024-04-25T13:46:14.673 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2024-04-25T13:46:14.674 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2024-04-25T13:46:14.674 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:14.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb9165640 1 Processor -- start 2024-04-25T13:46:14.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb9165640 1 -- start start 2024-04-25T13:46:14.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb9165640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3bb40d10b0 0x7f3bb40d4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb9165640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3bb40c7150 0x7f3bb416e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb9165640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bb40b3680 con 0x7f3bb4164960 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb9165640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb40d02f0 con 0x7f3bb40c7150 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb9165640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3bb40d05d0 con 0x7f3bb40d10b0 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb359e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3bb40c7150 0x7f3bb416e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb359e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3bb40c7150 0x7f3bb416e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47868/0 (socket says 172.21.15.80:47868) 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb259c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3bb4164960 0x7f3bb40c6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48862/0 (socket says 172.21.15.80:48862) 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb359e640 1 -- 172.21.15.80:0/1560993483 learned_addr learned my addr 172.21.15.80:0/1560993483 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb2d9d640 1 --2- 172.21.15.80:0/1560993483 >> v2:172.21.15.87:3300/0 conn(0x7f3bb40d10b0 0x7f3bb40d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb359e640 1 -- 172.21.15.80:0/1560993483 >> v2:172.21.15.87:3300/0 conn(0x7f3bb40d10b0 msgr2=0x7f3bb40d4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb359e640 1 --2- 172.21.15.80:0/1560993483 >> v2:172.21.15.87:3300/0 conn(0x7f3bb40d10b0 0x7f3bb40d4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb359e640 1 -- 172.21.15.80:0/1560993483 >> v1:172.21.15.80:6789/0 conn(0x7f3bb4164960 legacy=0x7f3bb40c6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb359e640 1 -- 172.21.15.80:0/1560993483 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb40d08b0 con 0x7f3bb40c7150 2024-04-25T13:46:14.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.823+0000 7f3bb1d9b640 1 -- 172.21.15.80:0/1560993483 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4152532771 0 0) 0x7f3bb40b3680 con 0x7f3bb4164960 2024-04-25T13:46:14.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb359e640 1 --2- 172.21.15.80:0/1560993483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40c7150 0x7f3bb416e530 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4068fd0 tx=0x7f3bac01d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557e4ad156cc335a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:14.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb1d9b640 1 -- 172.21.15.80:0/1560993483 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0a57d0 con 0x7f3bb40c7150 2024-04-25T13:46:14.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb1d9b640 1 -- 172.21.15.80:0/1560993483 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bac0a5970 con 0x7f3bb40c7150 2024-04-25T13:46:14.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb1d9b640 1 -- 172.21.15.80:0/1560993483 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bac0a5d40 con 0x7f3bb40c7150 2024-04-25T13:46:14.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb9165640 1 -- 172.21.15.80:0/1560993483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40c7150 msgr2=0x7f3bb416e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb9165640 1 --2- 172.21.15.80:0/1560993483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40c7150 0x7f3bb416e530 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4068fd0 tx=0x7f3bac01d450 comp rx=0 tx=0).stop 2024-04-25T13:46:14.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb9165640 1 -- 172.21.15.80:0/1560993483 shutdown_connections 2024-04-25T13:46:14.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb9165640 1 --2- 172.21.15.80:0/1560993483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40c7150 0x7f3bb416e530 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb9165640 1 --2- 172.21.15.80:0/1560993483 >> v2:172.21.15.87:3300/0 conn(0x7f3bb40d10b0 0x7f3bb40d4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb9165640 1 -- 172.21.15.80:0/1560993483 >> 172.21.15.80:0/1560993483 conn(0x7f3bb40d3a70 msgr2=0x7f3bb40d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:14.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb9165640 1 -- 172.21.15.80:0/1560993483 shutdown_connections 2024-04-25T13:46:14.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.827+0000 7f3bb9165640 1 -- 172.21.15.80:0/1560993483 wait complete. 2024-04-25T13:46:14.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 Processor -- start 2024-04-25T13:46:14.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 -- start start 2024-04-25T13:46:14.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb40c7150 0x7f3bb40c9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40d10b0 0x7f3bb40b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3bb4164960 0x7f3bb40ccf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4171620 con 0x7f3bb40d10b0 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41714a0 con 0x7f3bb40c7150 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb41717a0 con 0x7f3bb4164960 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40d10b0 0x7f3bb40b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40d10b0 0x7f3bb40b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47882/0 (socket says 172.21.15.80:47882) 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 -- 172.21.15.80:0/2336650360 learned_addr learned my addr 172.21.15.80:0/2336650360 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 -- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3bb4164960 msgr2=0x7f3bb40ccf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3bb4164960 0x7f3bb40ccf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb2d9d640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb40c7150 0x7f3bb40c9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 -- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb40c7150 msgr2=0x7f3bb40c9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb40c7150 0x7f3bb40c9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 -- 172.21.15.80:0/2336650360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb4201920 con 0x7f3bb40d10b0 2024-04-25T13:46:14.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb259c640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40d10b0 0x7f3bb40b1980 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c480 tx=0x7f3ba000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:14.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bab7fe640 1 -- 172.21.15.80:0/2336650360 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0018070 con 0x7f3bb40d10b0 2024-04-25T13:46:14.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bab7fe640 1 -- 172.21.15.80:0/2336650360 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ba0007500 con 0x7f3bb40d10b0 2024-04-25T13:46:14.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bab7fe640 1 -- 172.21.15.80:0/2336650360 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0004a30 con 0x7f3bb40d10b0 2024-04-25T13:46:14.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb4201c00 con 0x7f3bb40d10b0 2024-04-25T13:46:14.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb41fecf0 con 0x7f3bb40d10b0 2024-04-25T13:46:14.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.831+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb406a3a0 con 0x7f3bb40d10b0 2024-04-25T13:46:14.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.835+0000 7f3bab7fe640 1 -- 172.21.15.80:0/2336650360 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3ba001b070 con 0x7f3bb40d10b0 2024-04-25T13:46:14.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.835+0000 7f3bab7fe640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b9c06f400 0x7f3b9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:14.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.835+0000 7f3bab7fe640 1 -- 172.21.15.80:0/2336650360 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4943+0+0 (secure 0 0 0) 0x7f3ba008e4c0 con 0x7f3bb40d10b0 2024-04-25T13:46:14.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.835+0000 7f3bb2d9d640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b9c06f400 0x7f3b9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:14.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.835+0000 7f3bb2d9d640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b9c06f400 0x7f3b9c071910 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4168450 tx=0x7f3ba4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:14.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.839+0000 7f3bab7fe640 1 -- 172.21.15.80:0/2336650360 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3ba005b8b0 con 0x7f3bb40d10b0 2024-04-25T13:46:14.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.999+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bb4201680 con 0x7f3bb40d10b0 2024-04-25T13:46:15.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:14.999+0000 7f3bab7fe640 1 -- 172.21.15.80:0/2336650360 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+12957 (secure 0 0 0) 0x7f3ba005b1b0 con 0x7f3bb40d10b0 2024-04-25T13:46:15.002 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:15.002 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":113,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:13.647113+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/817443596":"2024-04-26T13:46:10.632050+0000","172.21.15.80:6833/817443596":"2024-04-26T13:46:10.632050+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b9c06f400 msgr2=0x7f3b9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b9c06f400 0x7f3b9c071910 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4168450 tx=0x7f3ba4008040 comp rx=0 tx=0).stop 2024-04-25T13:46:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40d10b0 msgr2=0x7f3bb40b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40d10b0 0x7f3bb40b1980 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f3ba000c480 tx=0x7f3ba000c940 comp rx=0 tx=0).stop 2024-04-25T13:46:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 shutdown_connections 2024-04-25T13:46:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b9c06f400 0x7f3b9c071910 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3bb4164960 0x7f3bb40ccf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3bb40d10b0 0x7f3bb40b1980 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 --2- 172.21.15.80:0/2336650360 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3bb40c7150 0x7f3bb40c9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 >> 172.21.15.80:0/2336650360 conn(0x7f3bb40d3a70 msgr2=0x7f3bb40b4870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:15.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 shutdown_connections 2024-04-25T13:46:15.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.003+0000 7f3bb9165640 1 -- 172.21.15.80:0/2336650360 wait complete. 2024-04-25T13:46:15.021 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:46:15.021 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:46:15.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc84e175640 1 Processor -- start 2024-04-25T13:46:15.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc84e175640 1 -- start start 2024-04-25T13:46:15.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc84e175640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc8481648f0 0x7fc848168500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc84e175640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84806ead0 0x7fc84806eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc84e175640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8480d0f20 con 0x7fc848168a40 2024-04-25T13:46:15.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc84e175640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc8480d1200 con 0x7fc8481648f0 2024-04-25T13:46:15.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc84e175640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc8480d14e0 con 0x7fc84806ead0 2024-04-25T13:46:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc8477fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc8481648f0 0x7fc848168500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc847fff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84806ead0 0x7fc84806eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc8477fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc8481648f0 0x7fc848168500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:54034/0 (socket says 172.21.15.80:54034) 2024-04-25T13:46:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc8477fe640 1 -- 172.21.15.80:0/1987453199 learned_addr learned my addr 172.21.15.80:0/1987453199 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.167+0000 7fc846ffd640 1 --1- 172.21.15.80:0/1987453199 >> v1:172.21.15.87:6789/0 conn(0x7fc848168a40 0x7fc848168e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:40816/0 (socket says 172.21.15.80:40816) 2024-04-25T13:46:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc847fff640 1 -- 172.21.15.80:0/1987453199 >> v2:172.21.15.87:3300/0 conn(0x7fc8481648f0 msgr2=0x7fc848168500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc847fff640 1 --2- 172.21.15.80:0/1987453199 >> v2:172.21.15.87:3300/0 conn(0x7fc8481648f0 0x7fc848168500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc847fff640 1 -- 172.21.15.80:0/1987453199 >> v1:172.21.15.87:6789/0 conn(0x7fc848168a40 legacy=0x7fc848168e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc847fff640 1 -- 172.21.15.80:0/1987453199 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8480d17c0 con 0x7fc84806ead0 2024-04-25T13:46:15.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc847fff640 1 --2- 172.21.15.80:0/1987453199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84806ead0 0x7fc84806eee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8380097c0 tx=0x7fc83801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=382fd65ab885443d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:15.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc8467fc640 1 -- 172.21.15.80:0/1987453199 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8380b0070 con 0x7fc84806ead0 2024-04-25T13:46:15.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc8467fc640 1 -- 172.21.15.80:0/1987453199 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8380ab600 con 0x7fc84806ead0 2024-04-25T13:46:15.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc8467fc640 1 -- 172.21.15.80:0/1987453199 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8380ab9d0 con 0x7fc84806ead0 2024-04-25T13:46:15.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc84e175640 1 -- 172.21.15.80:0/1987453199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84806ead0 msgr2=0x7fc84806eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc84e175640 1 --2- 172.21.15.80:0/1987453199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84806ead0 0x7fc84806eee0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc8380097c0 tx=0x7fc83801c410 comp rx=0 tx=0).stop 2024-04-25T13:46:15.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc84e175640 1 -- 172.21.15.80:0/1987453199 shutdown_connections 2024-04-25T13:46:15.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc84e175640 1 --2- 172.21.15.80:0/1987453199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84806ead0 0x7fc84806eee0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc84e175640 1 --2- 172.21.15.80:0/1987453199 >> v2:172.21.15.87:3300/0 conn(0x7fc8481648f0 0x7fc848168500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc84e175640 1 -- 172.21.15.80:0/1987453199 >> 172.21.15.80:0/1987453199 conn(0x7fc8480b3cc0 msgr2=0x7fc8480b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:15.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc84e175640 1 -- 172.21.15.80:0/1987453199 shutdown_connections 2024-04-25T13:46:15.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.171+0000 7fc84e175640 1 -- 172.21.15.80:0/1987453199 wait complete. 2024-04-25T13:46:15.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 Processor -- start 2024-04-25T13:46:15.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 -- start start 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84806ead0 0x7fc848150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8481648f0 0x7fc84809d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848168a40 0x7fc8480afff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc848171bd0 con 0x7fc848168a40 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc848171a50 con 0x7fc84806ead0 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc848171d50 con 0x7fc8481648f0 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848168a40 0x7fc8480afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848168a40 0x7fc8480afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47892/0 (socket says 172.21.15.80:47892) 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 -- 172.21.15.80:0/1843543586 learned_addr learned my addr 172.21.15.80:0/1843543586 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc8477fe640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84806ead0 0x7fc848150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc846ffd640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8481648f0 0x7fc84809d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 -- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8481648f0 msgr2=0x7fc84809d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8481648f0 0x7fc84809d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 -- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84806ead0 msgr2=0x7fc848150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84806ead0 0x7fc848150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 -- 172.21.15.80:0/1843543586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84815f770 con 0x7fc848168a40 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc847fff640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848168a40 0x7fc8480afff0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc8380097c0 tx=0x7fc8380191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc82ffff640 1 -- 172.21.15.80:0/1843543586 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8380ba070 con 0x7fc848168a40 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc82ffff640 1 -- 172.21.15.80:0/1843543586 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8380b43f0 con 0x7fc848168a40 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc82ffff640 1 -- 172.21.15.80:0/1843543586 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8380b0040 con 0x7fc848168a40 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84815e6d0 con 0x7fc848168a40 2024-04-25T13:46:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.175+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc84815ec90 con 0x7fc848168a40 2024-04-25T13:46:15.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.179+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc84806a3a0 con 0x7fc848168a40 2024-04-25T13:46:15.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.179+0000 7fc82ffff640 1 -- 172.21.15.80:0/1843543586 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc8380bd070 con 0x7fc848168a40 2024-04-25T13:46:15.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.179+0000 7fc82ffff640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc83406f430 0x7fc834071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.179+0000 7fc82ffff640 1 -- 172.21.15.80:0/1843543586 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4943+0+0 (secure 0 0 0) 0x7fc838128390 con 0x7fc848168a40 2024-04-25T13:46:15.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.179+0000 7fc8477fe640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc83406f430 0x7fc834071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.179+0000 7fc8477fe640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc83406f430 0x7fc834071940 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc840009a40 tx=0x7fc840008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:15.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.183+0000 7fc82ffff640 1 -- 172.21.15.80:0/1843543586 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc8380c4080 con 0x7fc848168a40 2024-04-25T13:46:15.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.327+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc848074620 con 0x7fc848168a40 2024-04-25T13:46:15.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc82ffff640 1 -- 172.21.15.80:0/1843543586 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) ==== 93+0+10 (secure 0 0 0) 0x7fc838016090 con 0x7fc848168a40 2024-04-25T13:46:15.329 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc83406f430 msgr2=0x7fc834071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc83406f430 0x7fc834071940 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc840009a40 tx=0x7fc840008040 comp rx=0 tx=0).stop 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848168a40 msgr2=0x7fc8480afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848168a40 0x7fc8480afff0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fc8380097c0 tx=0x7fc8380191b0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 shutdown_connections 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc83406f430 0x7fc834071940 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc848168a40 0x7fc8480afff0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8481648f0 0x7fc84809d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 --2- 172.21.15.80:0/1843543586 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84806ead0 0x7fc848150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 >> 172.21.15.80:0/1843543586 conn(0x7fc8480b3cc0 msgr2=0x7fc848160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:15.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 shutdown_connections 2024-04-25T13:46:15.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.331+0000 7fc84e175640 1 -- 172.21.15.80:0/1843543586 wait complete. 2024-04-25T13:46:15.349 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-04-25T13:46:15.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9af15c0640 1 Processor -- start 2024-04-25T13:46:15.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9af15c0640 1 -- start start 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9af15c0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec1639e0 0x7f9aec0d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9af15c0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9aec0b9770 0x7f9aec0b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9af15c0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9aec0d4e00 con 0x7f9aec0b7760 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9af15c0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9aec0d1a70 con 0x7f9aec0b9770 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9af15c0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9aec0d5a10 con 0x7f9aec1639e0 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9aeb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9aec0b9770 0x7f9aec0b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9aeb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9aec0b9770 0x7f9aec0b9b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47902/0 (socket says 172.21.15.80:47902) 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9aeb7fe640 1 -- 172.21.15.80:0/2351588263 learned_addr learned my addr 172.21.15.80:0/2351588263 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:15.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9aeaffd640 1 --2- 172.21.15.80:0/2351588263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec1639e0 0x7f9aec0d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9aeb7fe640 1 -- 172.21.15.80:0/2351588263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec1639e0 msgr2=0x7f9aec0d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:15.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9aeb7fe640 1 --2- 172.21.15.80:0/2351588263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec1639e0 0x7f9aec0d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9aeb7fe640 1 -- 172.21.15.80:0/2351588263 >> v1:172.21.15.87:6789/0 conn(0x7f9aec0b7760 legacy=0x7f9aec0b7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:15.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.491+0000 7f9aeb7fe640 1 -- 172.21.15.80:0/2351588263 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec0d50e0 con 0x7f9aec0b9770 2024-04-25T13:46:15.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9aeb7fe640 1 --2- 172.21.15.80:0/2351588263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec0b9770 0x7f9aec0b9b80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009e50 tx=0x7f9ae401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856920dbdc362656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:15.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9ae9ffb640 1 -- 172.21.15.80:0/2351588263 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40a5670 con 0x7f9aec0b9770 2024-04-25T13:46:15.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9ae9ffb640 1 -- 172.21.15.80:0/2351588263 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae40a5810 con 0x7f9aec0b9770 2024-04-25T13:46:15.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9ae9ffb640 1 -- 172.21.15.80:0/2351588263 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40a5be0 con 0x7f9aec0b9770 2024-04-25T13:46:15.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 -- 172.21.15.80:0/2351588263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec0b9770 msgr2=0x7f9aec0b9b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 --2- 172.21.15.80:0/2351588263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec0b9770 0x7f9aec0b9b80 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009e50 tx=0x7f9ae401c9f0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 -- 172.21.15.80:0/2351588263 shutdown_connections 2024-04-25T13:46:15.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 --2- 172.21.15.80:0/2351588263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec0b9770 0x7f9aec0b9b80 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 --2- 172.21.15.80:0/2351588263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec1639e0 0x7f9aec0d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 -- 172.21.15.80:0/2351588263 >> 172.21.15.80:0/2351588263 conn(0x7f9aec06c9f0 msgr2=0x7f9aec06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:15.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 -- 172.21.15.80:0/2351588263 shutdown_connections 2024-04-25T13:46:15.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 -- 172.21.15.80:0/2351588263 wait complete. 2024-04-25T13:46:15.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.495+0000 7f9af15c0640 1 Processor -- start 2024-04-25T13:46:15.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 -- start start 2024-04-25T13:46:15.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec0b7760 0x7f9aec153670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9aec0b9770 0x7f9aec0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec1639e0 0x7f9aec156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec16d1f0 con 0x7f9aec1639e0 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9aec16d070 con 0x7f9aec0b9770 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9aec16d370 con 0x7f9aec0b7760 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec1639e0 0x7f9aec156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec1639e0 0x7f9aec156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47914/0 (socket says 172.21.15.80:47914) 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 -- 172.21.15.80:0/1289653274 learned_addr learned my addr 172.21.15.80:0/1289653274 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeaffd640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec0b7760 0x7f9aec153670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aea7fc640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9aec0b9770 0x7f9aec0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 -- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec0b7760 msgr2=0x7f9aec153670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec0b7760 0x7f9aec153670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 -- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9aec0b9770 msgr2=0x7f9aec0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9aec0b9770 0x7f9aec0cecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 -- 172.21.15.80:0/1289653274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9aec0cf820 con 0x7f9aec1639e0 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9aeb7fe640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec1639e0 0x7f9aec156650 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009e50 tx=0x7f9ae4019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9ad37fe640 1 -- 172.21.15.80:0/1289653274 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40b8670 con 0x7f9aec1639e0 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9ad37fe640 1 -- 172.21.15.80:0/1289653274 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ae40be070 con 0x7f9aec1639e0 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9aec0cfb00 con 0x7f9aec1639e0 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9ad37fe640 1 -- 172.21.15.80:0/1289653274 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae40b8a10 con 0x7f9aec1639e0 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9aec0d00c0 con 0x7f9aec1639e0 2024-04-25T13:46:15.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.499+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9aec06a3a0 con 0x7f9aec1639e0 2024-04-25T13:46:15.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.503+0000 7f9ad37fe640 1 -- 172.21.15.80:0/1289653274 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9ae40b8bb0 con 0x7f9aec1639e0 2024-04-25T13:46:15.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.503+0000 7f9ad37fe640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ad806f400 0x7f9ad8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:15.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.503+0000 7f9ad37fe640 1 -- 172.21.15.80:0/1289653274 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4943+0+0 (secure 0 0 0) 0x7f9ae412abf0 con 0x7f9aec1639e0 2024-04-25T13:46:15.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.503+0000 7f9aeaffd640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ad806f400 0x7f9ad8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:15.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.503+0000 7f9aeaffd640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ad806f400 0x7f9ad8071910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00b7c0 tx=0x7f9adc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:15.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.507+0000 7f9ad37fe640 1 -- 172.21.15.80:0/1289653274 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ae40165e0 con 0x7f9aec1639e0 2024-04-25T13:46:15.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:15.711+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) -- 0x7f9aec0cf540 con 0x7f9aec1639e0 2024-04-25T13:46:16.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.659+0000 7f9ad37fe640 1 -- 172.21.15.80:0/1289653274 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v310) ==== 131+0+0 (secure 0 0 0) 0x7f9ae40f7ec0 con 0x7f9aec1639e0 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ad806f400 msgr2=0x7f9ad8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ad806f400 0x7f9ad8071910 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00b7c0 tx=0x7f9adc008040 comp rx=0 tx=0).stop 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec1639e0 msgr2=0x7f9aec156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec1639e0 0x7f9aec156650 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f9ae4009e50 tx=0x7f9ae4019790 comp rx=0 tx=0).stop 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 shutdown_connections 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ad806f400 0x7f9ad8071910 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9aec1639e0 0x7f9aec156650 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9aec0b9770 0x7f9aec0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 --2- 172.21.15.80:0/1289653274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9aec0b7760 0x7f9aec153670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 >> 172.21.15.80:0/1289653274 conn(0x7f9aec06c9f0 msgr2=0x7f9aec0b57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:16.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 shutdown_connections 2024-04-25T13:46:16.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.663+0000 7f9af15c0640 1 -- 172.21.15.80:0/1289653274 wait complete. 2024-04-25T13:46:16.688 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:46:16.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648c2db640 1 Processor -- start 2024-04-25T13:46:16.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648c2db640 1 -- start start 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648c2db640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 0x7f64841574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648c2db640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6484073c40 0x7f6484160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648c2db640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6484160fa0 0x7f64840b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648c2db640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f64840b1350 con 0x7f6484160fa0 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648c2db640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f648406f9a0 con 0x7f6484073c40 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648c2db640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f64841613b0 con 0x7f6484071e60 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a878640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6484160fa0 0x7f64840b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a878640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6484160fa0 0x7f64840b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47928/0 (socket says 172.21.15.80:47928) 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a878640 1 -- 172.21.15.80:0/210124184 learned_addr learned my addr 172.21.15.80:0/210124184 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:16.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a077640 1 --2- 172.21.15.80:0/210124184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 0x7f64841574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:16.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f6489876640 1 --2- 172.21.15.80:0/210124184 >> v2:172.21.15.87:3300/0 conn(0x7f6484073c40 0x7f6484160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:16.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a878640 1 -- 172.21.15.80:0/210124184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 msgr2=0x7f64841574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:16.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a878640 1 --2- 172.21.15.80:0/210124184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 0x7f64841574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a878640 1 -- 172.21.15.80:0/210124184 >> v2:172.21.15.87:3300/0 conn(0x7f6484073c40 msgr2=0x7f6484160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:16.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a878640 1 --2- 172.21.15.80:0/210124184 >> v2:172.21.15.87:3300/0 conn(0x7f6484073c40 0x7f6484160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.847+0000 7f648a878640 1 -- 172.21.15.80:0/210124184 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648406fbf0 con 0x7f6484160fa0 2024-04-25T13:46:16.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648a878640 1 --2- 172.21.15.80:0/210124184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484160fa0 0x7f64840b4ba0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6484070290 tx=0x7f647c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3db7fa14fc54edd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:16.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f6489075640 1 -- 172.21.15.80:0/210124184 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c0a5760 con 0x7f6484160fa0 2024-04-25T13:46:16.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f6489075640 1 -- 172.21.15.80:0/210124184 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f647c0a5900 con 0x7f6484160fa0 2024-04-25T13:46:16.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f6489075640 1 -- 172.21.15.80:0/210124184 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647c0a5cd0 con 0x7f6484160fa0 2024-04-25T13:46:16.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 -- 172.21.15.80:0/210124184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484160fa0 msgr2=0x7f64840b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:16.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 --2- 172.21.15.80:0/210124184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484160fa0 0x7f64840b4ba0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f6484070290 tx=0x7f647c01c9f0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 -- 172.21.15.80:0/210124184 shutdown_connections 2024-04-25T13:46:16.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 --2- 172.21.15.80:0/210124184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484160fa0 0x7f64840b4ba0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 --2- 172.21.15.80:0/210124184 >> v2:172.21.15.87:3300/0 conn(0x7f6484073c40 0x7f6484160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 --2- 172.21.15.80:0/210124184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 0x7f64841574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 -- 172.21.15.80:0/210124184 >> 172.21.15.80:0/210124184 conn(0x7f64840b4160 msgr2=0x7f64840d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:16.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 -- 172.21.15.80:0/210124184 shutdown_connections 2024-04-25T13:46:16.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.851+0000 7f648c2db640 1 -- 172.21.15.80:0/210124184 wait complete. 2024-04-25T13:46:16.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 Processor -- start 2024-04-25T13:46:16.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 -- start start 2024-04-25T13:46:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 0x7f6484151bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484073c40 0x7f6484158010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6484160fa0 0x7f6484158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6484074050 con 0x7f6484073c40 2024-04-25T13:46:16.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6484072270 con 0x7f6484160fa0 2024-04-25T13:46:16.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f648415d470 con 0x7f6484071e60 2024-04-25T13:46:16.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484073c40 0x7f6484158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:16.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484073c40 0x7f6484158010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47936/0 (socket says 172.21.15.80:47936) 2024-04-25T13:46:16.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 -- 172.21.15.80:0/4143396315 learned_addr learned my addr 172.21.15.80:0/4143396315 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:16.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648a077640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 0x7f6484151bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:16.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 -- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 msgr2=0x7f6484151bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:16.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 0x7f6484151bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648a878640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6484160fa0 0x7f6484158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 -- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6484160fa0 msgr2=0x7f6484158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6484160fa0 0x7f6484158550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 -- 172.21.15.80:0/4143396315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6484152110 con 0x7f6484073c40 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6489876640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484073c40 0x7f6484158010 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f648000e870 tx=0x7f648000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6476ffd640 1 -- 172.21.15.80:0/4143396315 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480017070 con 0x7f6484073c40 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6476ffd640 1 -- 172.21.15.80:0/4143396315 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f648000c940 con 0x7f6484073c40 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64840ae950 con 0x7f6484073c40 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f6476ffd640 1 -- 172.21.15.80:0/4143396315 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648000cd30 con 0x7f6484073c40 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.855+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64840aee50 con 0x7f6484073c40 2024-04-25T13:46:16.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.859+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f648406a3a0 con 0x7f6484073c40 2024-04-25T13:46:16.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.859+0000 7f6476ffd640 1 -- 172.21.15.80:0/4143396315 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f648001a070 con 0x7f6484073c40 2024-04-25T13:46:16.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.859+0000 7f6476ffd640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:16.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.859+0000 7f6476ffd640 1 -- 172.21.15.80:0/4143396315 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4943+0+0 (secure 0 0 0) 0x7f648008eeb0 con 0x7f6484073c40 2024-04-25T13:46:16.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.859+0000 7f648a077640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:16.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.859+0000 7f648a077640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6478009670 tx=0x7f6478008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:16.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:16.863+0000 7f6476ffd640 1 -- 172.21.15.80:0/4143396315 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f648005c0a0 con 0x7f6484073c40 2024-04-25T13:46:17.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.015+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f648415cb80 con 0x7f6484073c40 2024-04-25T13:46:17.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.019+0000 7f6476ffd640 1 -- 172.21.15.80:0/4143396315 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7f648005b9a0 con 0x7f6484073c40 2024-04-25T13:46:17.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 msgr2=0x7f6468071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:17.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6478009670 tx=0x7f6478008040 comp rx=0 tx=0).stop 2024-04-25T13:46:17.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484073c40 msgr2=0x7f6484158010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:17.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484073c40 0x7f6484158010 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f648000e870 tx=0x7f648000ec70 comp rx=0 tx=0).stop 2024-04-25T13:46:17.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 shutdown_connections 2024-04-25T13:46:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f646806f400 0x7f6468071910 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6484160fa0 0x7f6484158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6484073c40 0x7f6484158010 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 --2- 172.21.15.80:0/4143396315 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6484071e60 0x7f6484151bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 >> 172.21.15.80:0/4143396315 conn(0x7f64840b4160 msgr2=0x7f64840723f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 shutdown_connections 2024-04-25T13:46:17.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.023+0000 7f648c2db640 1 -- 172.21.15.80:0/4143396315 wait complete. 2024-04-25T13:46:17.039 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:46:17.040 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:46:17.077 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:46:17.123 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:46:17.123 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:17.123 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:17.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:46:17.129 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:46:17.129 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:17.170 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:46:17.170 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:17.170 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:17.176 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:46:17.176 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:46:17.176 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:17.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:46:17.222 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:46:17.222 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:17.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.359+0000 7fb961b7a640 1 Processor -- start 2024-04-25T13:46:17.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.359+0000 7fb961b7a640 1 -- start start 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb961b7a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0b2d30 0x7fb95c0d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb961b7a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb95c0ceca0 0x7fb95c0cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb961b7a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb95c0d30b0 con 0x7fb95c0d0350 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb961b7a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb95c0d1120 con 0x7fb95c0ceca0 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb961b7a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb95c0ce1c0 con 0x7fb95c0b2d30 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb95c0ceca0 0x7fb95c0cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb95c0d0350 0x7fb95c0d0760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48874/0 (socket says 172.21.15.80:48874) 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95affd640 1 -- 172.21.15.80:0/2969130182 learned_addr learned my addr 172.21.15.80:0/2969130182 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95b7fe640 1 --2- 172.21.15.80:0/2969130182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0b2d30 0x7fb95c0d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95bfff640 1 -- 172.21.15.80:0/2969130182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0b2d30 msgr2=0x7fb95c0d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:17.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95bfff640 1 --2- 172.21.15.80:0/2969130182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0b2d30 0x7fb95c0d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95bfff640 1 -- 172.21.15.80:0/2969130182 >> v1:172.21.15.80:6789/0 conn(0x7fb95c0d0350 legacy=0x7fb95c0d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:17.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95bfff640 1 -- 172.21.15.80:0/2969130182 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c0ce4a0 con 0x7fb95c0ceca0 2024-04-25T13:46:17.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95b7fe640 1 --2- 172.21.15.80:0/2969130182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0b2d30 0x7fb95c0d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:17.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.363+0000 7fb95bfff640 1 --2- 172.21.15.80:0/2969130182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0ceca0 0x7fb95c0cf0b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb954009d90 tx=0x7fb95401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c3d053b4389f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:17.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb95a7fc640 1 -- 172.21.15.80:0/2969130182 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540b2070 con 0x7fb95c0ceca0 2024-04-25T13:46:17.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb95a7fc640 1 -- 172.21.15.80:0/2969130182 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb9540a5920 con 0x7fb95c0ceca0 2024-04-25T13:46:17.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb95a7fc640 1 -- 172.21.15.80:0/2969130182 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9540a5cf0 con 0x7fb95c0ceca0 2024-04-25T13:46:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 -- 172.21.15.80:0/2969130182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0ceca0 msgr2=0x7fb95c0cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 --2- 172.21.15.80:0/2969130182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0ceca0 0x7fb95c0cf0b0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fb954009d90 tx=0x7fb95401c990 comp rx=0 tx=0).stop 2024-04-25T13:46:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 -- 172.21.15.80:0/2969130182 shutdown_connections 2024-04-25T13:46:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 --2- 172.21.15.80:0/2969130182 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0ceca0 0x7fb95c0cf0b0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 --2- 172.21.15.80:0/2969130182 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0b2d30 0x7fb95c0d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 -- 172.21.15.80:0/2969130182 >> 172.21.15.80:0/2969130182 conn(0x7fb95c0d3ad0 msgr2=0x7fb95c0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 -- 172.21.15.80:0/2969130182 shutdown_connections 2024-04-25T13:46:17.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 -- 172.21.15.80:0/2969130182 wait complete. 2024-04-25T13:46:17.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 Processor -- start 2024-04-25T13:46:17.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.367+0000 7fb961b7a640 1 -- start start 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0b2d30 0x7fb95c155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0ceca0 0x7fb95c158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb95c0d0350 0x7fb95c15f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c16d2f0 con 0x7fb95c0b2d30 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb95c16d170 con 0x7fb95c0d0350 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb95c16d470 con 0x7fb95c0ceca0 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0b2d30 0x7fb95c155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0b2d30 0x7fb95c155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:47962/0 (socket says 172.21.15.80:47962) 2024-04-25T13:46:17.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 -- 172.21.15.80:0/3167598108 learned_addr learned my addr 172.21.15.80:0/3167598108 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 -- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0ceca0 msgr2=0x7fb95c158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95affd640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0ceca0 0x7fb95c158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95bfff640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb95c0d0350 0x7fb95c15f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0ceca0 0x7fb95c158e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 -- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb95c0d0350 msgr2=0x7fb95c15f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb95c0d0350 0x7fb95c15f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 -- 172.21.15.80:0/3167598108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb95c0ca520 con 0x7fb95c0b2d30 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95b7fe640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0b2d30 0x7fb95c155ca0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c7f0 tx=0x7fb94c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb943fff640 1 -- 172.21.15.80:0/3167598108 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c005ba0 con 0x7fb95c0b2d30 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb943fff640 1 -- 172.21.15.80:0/3167598108 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb94c005d40 con 0x7fb95c0b2d30 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb95c0b7b80 con 0x7fb95c0b2d30 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb943fff640 1 -- 172.21.15.80:0/3167598108 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94c006270 con 0x7fb95c0b2d30 2024-04-25T13:46:17.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb95c0ca000 con 0x7fb95c0b2d30 2024-04-25T13:46:17.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb95c06a3a0 con 0x7fb95c0b2d30 2024-04-25T13:46:17.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb943fff640 1 -- 172.21.15.80:0/3167598108 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb94c006620 con 0x7fb95c0b2d30 2024-04-25T13:46:17.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb943fff640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb94806f400 0x7fb948071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:17.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb943fff640 1 -- 172.21.15.80:0/3167598108 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 4943+0+0 (secure 0 0 0) 0x7fb94c08f180 con 0x7fb95c0b2d30 2024-04-25T13:46:17.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.371+0000 7fb95affd640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb94806f400 0x7fb948071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:17.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.375+0000 7fb95affd640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb94806f400 0x7fb948071910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb950000f70 tx=0x7fb950009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:17.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.375+0000 7fb943fff640 1 -- 172.21.15.80:0/3167598108 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb94c015020 con 0x7fb95c0b2d30 2024-04-25T13:46:17.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb95c0c73b0 con 0x7fb95c0b2d30 2024-04-25T13:46:17.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb943fff640 1 -- 172.21.15.80:0/3167598108 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+3866 (secure 0 0 0) 0x7fb94c05c400 con 0x7fb95c0b2d30 2024-04-25T13:46:17.557 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:17.557 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":51,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5483,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2563102725","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2563102725},{"type":"v1","addr":"172.21.15.87:6835","nonce":2563102725}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":44},{"gid":5489,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/3145767980","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":3145767980},{"type":"v1","addr":"172.21.15.87:6837","nonce":3145767980}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7041,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/2001394182","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":2001394182},{"type":"v1","addr":"172.21.15.80:6834","nonce":2001394182}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":45},{"gid":7419,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/949883249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":949883249},{"type":"v1","addr":"172.21.15.80:6835","nonce":949883249}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[]} 2024-04-25T13:46:17.557 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 51 2024-04-25T13:46:17.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb94806f400 msgr2=0x7fb948071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:17.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb94806f400 0x7fb948071910 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb950000f70 tx=0x7fb950009040 comp rx=0 tx=0).stop 2024-04-25T13:46:17.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0b2d30 msgr2=0x7fb95c155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:17.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0b2d30 0x7fb95c155ca0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fb94c00c7f0 tx=0x7fb94c00ccb0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 shutdown_connections 2024-04-25T13:46:17.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb94806f400 0x7fb948071910 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb95c0d0350 0x7fb95c15f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb95c0ceca0 0x7fb95c158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 --2- 172.21.15.80:0/3167598108 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb95c0b2d30 0x7fb95c155ca0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:17.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 >> 172.21.15.80:0/3167598108 conn(0x7fb95c0d3ad0 msgr2=0x7fb95c0d0760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:17.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 shutdown_connections 2024-04-25T13:46:17.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:17.559+0000 7fb961b7a640 1 -- 172.21.15.80:0/3167598108 wait complete. 2024-04-25T13:46:17.579 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:46:17.579 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:46:17.580 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:46:17.580 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:46:17.580 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:46:17.580 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:46:17.580 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:46:17.580 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:46:17.580 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:46:17.580 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:46:17.581 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:46:17.581 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:46:17.581 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:46:17.581 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:46:17.581 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:46:17.581 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:46:17.595 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:46:17.595 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:46:17.599 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:46:17.599 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:46:17.604 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:46:17.604 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:46:17.637 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:46:17.638 DEBUG:teuthology.parallel:result is None 2024-04-25T13:46:17.646 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:46:17.646 DEBUG:teuthology.parallel:result is None 2024-04-25T13:46:17.647 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:46:17.648 DEBUG:teuthology.parallel:result is None 2024-04-25T13:46:17.738 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:46:17.750 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:46:17.750 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:46:18.775 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T13:46:23.595 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:46:23.595 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:46:23.598 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:46:23.598 DEBUG:teuthology.parallel:result is None 2024-04-25T13:46:23.599 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:46:23.700 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:46:23.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0d524e640 1 Processor -- start 2024-04-25T13:46:23.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0d524e640 1 -- start start 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0d524e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe0d00d2e80 0x7fe0d01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0d524e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d0161060 con 0x7fe0d00d4330 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0d524e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0d0161340 con 0x7fe0d00d59b0 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0d524e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe0d00d2680 con 0x7fe0d00d2e80 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0cf59e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe0d00d4330 0x7fe0d00d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38328/0 (socket says 172.21.15.80:38328) 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0cf59e640 1 -- 172.21.15.80:0/3101605675 learned_addr learned my addr 172.21.15.80:0/3101605675 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0ced9d640 1 --2- 172.21.15.80:0/3101605675 >> v2:172.21.15.87:3300/0 conn(0x7fe0d00d2e80 0x7fe0d01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1978377123 0 0) 0x7fe0d0161060 con 0x7fe0d00d4330 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.759+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe0bc003530 con 0x7fe0d00d4330 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2336128246 0 0) 0x7fe0bc003530 con 0x7fe0d00d4330 2024-04-25T13:46:23.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 >> v2:172.21.15.87:3300/0 conn(0x7fe0d00d2e80 msgr2=0x7fe0d01607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 --2- 172.21.15.80:0/3101605675 >> v2:172.21.15.87:3300/0 conn(0x7fe0d00d2e80 0x7fe0d01607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 >> v1:172.21.15.87:6789/0 conn(0x7fe0d00d59b0 legacy=0x7fe0d00d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d00d2960 con 0x7fe0d00d4330 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fe0c8001a20 con 0x7fe0d00d4330 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 >> v1:172.21.15.80:6789/0 conn(0x7fe0d00d4330 legacy=0x7fe0d00d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0bc0039c0 0x7fe0bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0bc004810 0x7fe0bc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00d2680 con 0x7fe0bc005260 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c1f0 con 0x7fe0bc0039c0 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe0bc00c3f0 con 0x7fe0bc004810 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cf59e640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:23.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0ced9d640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0bc0039c0 0x7fe0bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:23.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0ce59c640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0bc004810 0x7fe0bc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:23.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cf59e640 1 -- 172.21.15.80:0/3101605675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0bc004810 msgr2=0x7fe0bc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:23.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cf59e640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0bc004810 0x7fe0bc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:23.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cf59e640 1 -- 172.21.15.80:0/3101605675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0bc0039c0 msgr2=0x7fe0bc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:23.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cf59e640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0bc0039c0 0x7fe0bc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:23.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cf59e640 1 -- 172.21.15.80:0/3101605675 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0bc003530 con 0x7fe0bc005260 2024-04-25T13:46:23.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cf59e640 1 -- 172.21.15.80:0/3101605675 reap_dead start 2024-04-25T13:46:23.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cf59e640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0bc00b9d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0164240 tx=0x7fe0c801fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cb5cf02957d2171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:23.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c800a4c0 con 0x7fe0bc005260 2024-04-25T13:46:23.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0c80aa4c0 con 0x7fe0bc005260 2024-04-25T13:46:23.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0cdd9b640 1 -- 172.21.15.80:0/3101605675 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80aa890 con 0x7fe0bc005260 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0d524e640 1 -- 172.21.15.80:0/3101605675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 msgr2=0x7fe0bc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0d524e640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0bc00b9d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0164240 tx=0x7fe0c801fb20 comp rx=0 tx=0).stop 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0d524e640 1 -- 172.21.15.80:0/3101605675 shutdown_connections 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0d524e640 1 --2- 172.21.15.80:0/3101605675 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0bc00b9d0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0d524e640 1 -- 172.21.15.80:0/3101605675 >> 172.21.15.80:0/3101605675 conn(0x7fe0d00737a0 msgr2=0x7fe0d006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0d524e640 1 -- 172.21.15.80:0/3101605675 shutdown_connections 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.763+0000 7fe0d524e640 1 -- 172.21.15.80:0/3101605675 wait complete. 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 Processor -- start 2024-04-25T13:46:23.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 -- start start 2024-04-25T13:46:23.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0d0157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:23.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0d015ab00 0x7fe0d015b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:23.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0d015b570 0x7fe0d015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00d3410 con 0x7fe0bc005260 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00d3290 con 0x7fe0d015ab00 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d015df60 con 0x7fe0d015b570 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0d0157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0d0157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57390/0 (socket says 172.21.15.80:57390) 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 -- 172.21.15.80:0/99296818 learned_addr learned my addr 172.21.15.80:0/99296818 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ce59c640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0d015ab00 0x7fe0d015b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0cf59e640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0d015b570 0x7fe0d015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 -- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0d015b570 msgr2=0x7fe0d015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0d015b570 0x7fe0d015f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 -- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0d015ab00 msgr2=0x7fe0d015b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0d015ab00 0x7fe0d015b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 -- 172.21.15.80:0/99296818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d0159d80 con 0x7fe0bc005260 2024-04-25T13:46:23.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ce59c640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0d015ab00 0x7fe0d015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:23.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0ced9d640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0d0157320 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8011590 tx=0x7fe0b8011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:23.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0c77fe640 1 -- 172.21.15.80:0/99296818 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8015870 con 0x7fe0bc005260 2024-04-25T13:46:23.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0c77fe640 1 -- 172.21.15.80:0/99296818 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b8015a10 con 0x7fe0bc005260 2024-04-25T13:46:23.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.767+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d00ceaa0 con 0x7fe0bc005260 2024-04-25T13:46:23.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.771+0000 7fe0c77fe640 1 -- 172.21.15.80:0/99296818 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8015dc0 con 0x7fe0bc005260 2024-04-25T13:46:23.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.771+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d015a300 con 0x7fe0bc005260 2024-04-25T13:46:23.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.771+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d006a3a0 con 0x7fe0bc005260 2024-04-25T13:46:23.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.771+0000 7fe0c77fe640 1 -- 172.21.15.80:0/99296818 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe0b8013400 con 0x7fe0bc005260 2024-04-25T13:46:23.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.771+0000 7fe0c77fe640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe0b006f430 0x7fe0b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:23.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.771+0000 7fe0c77fe640 1 -- 172.21.15.80:0/99296818 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4943+0+0 (secure 0 0 0) 0x7fe0b808f7e0 con 0x7fe0bc005260 2024-04-25T13:46:23.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.771+0000 7fe0ce59c640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe0b006f430 0x7fe0b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:23.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.771+0000 7fe0ce59c640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe0b006f430 0x7fe0b0071940 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0000c10 tx=0x7fe0c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:23.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.775+0000 7fe0c77fe640 1 -- 172.21.15.80:0/99296818 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe0b8019030 con 0x7fe0bc005260 2024-04-25T13:46:23.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:23.947+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe0d015d7b0 con 0x7fe0bc005260 2024-04-25T13:46:25.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0c77fe640 1 -- 172.21.15.80:0/99296818 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v118) ==== 97+0+0 (secure 0 0 0) 0x7fe0b805cb40 con 0x7fe0bc005260 2024-04-25T13:46:25.284 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:46:25.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe0b006f430 msgr2=0x7fe0b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe0b006f430 0x7fe0b0071940 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0000c10 tx=0x7fe0c0009040 comp rx=0 tx=0).stop 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 msgr2=0x7fe0d0157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0d0157320 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe0b8011590 tx=0x7fe0b8011a50 comp rx=0 tx=0).stop 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 shutdown_connections 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe0b006f430 0x7fe0b0071940 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0d015b570 0x7fe0d015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0d015ab00 0x7fe0d015b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 --2- 172.21.15.80:0/99296818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0bc005260 0x7fe0d0157320 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 >> 172.21.15.80:0/99296818 conn(0x7fe0d00737a0 msgr2=0x7fe0d0159420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 shutdown_connections 2024-04-25T13:46:25.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.287+0000 7fe0d524e640 1 -- 172.21.15.80:0/99296818 wait complete. 2024-04-25T13:46:25.316 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:46:25.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8cc86640 1 Processor -- start 2024-04-25T13:46:25.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8cc86640 1 -- start start 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8cc86640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe880d59b0 0x7efe880d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8cc86640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe880d6300 con 0x7efe880d2e80 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8cc86640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe88161060 con 0x7efe880d4330 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8cc86640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efe88161340 con 0x7efe880d59b0 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe85d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe880d59b0 0x7efe880d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8659c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7efe880d2e80 0x7efe881607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38342/0 (socket says 172.21.15.80:38342) 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8659c640 1 -- 172.21.15.80:0/3680231969 learned_addr learned my addr 172.21.15.80:0/3680231969 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe85d9b640 1 -- 172.21.15.80:0/3680231969 >> v1:172.21.15.87:6789/0 conn(0x7efe880d4330 legacy=0x7efe880d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe85d9b640 1 -- 172.21.15.80:0/3680231969 >> v1:172.21.15.80:6789/0 conn(0x7efe880d2e80 legacy=0x7efe881607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe85d9b640 1 -- 172.21.15.80:0/3680231969 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe880d2680 con 0x7efe880d59b0 2024-04-25T13:46:25.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.471+0000 7efe8559a640 1 -- 172.21.15.80:0/3680231969 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1043806537 0 0) 0x7efe7c001120 con 0x7efe880d2e80 2024-04-25T13:46:25.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe85d9b640 1 --2- 172.21.15.80:0/3680231969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d59b0 0x7efe880d5dc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7efe780097c0 tx=0x7efe7801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59cbe3994a68b994 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:25.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8559a640 1 -- 172.21.15.80:0/3680231969 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe780b0070 con 0x7efe880d59b0 2024-04-25T13:46:25.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8559a640 1 -- 172.21.15.80:0/3680231969 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe780ab600 con 0x7efe880d59b0 2024-04-25T13:46:25.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8559a640 1 -- 172.21.15.80:0/3680231969 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe780a4e60 con 0x7efe880d59b0 2024-04-25T13:46:25.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8cc86640 1 -- 172.21.15.80:0/3680231969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d59b0 msgr2=0x7efe880d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8cc86640 1 --2- 172.21.15.80:0/3680231969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d59b0 0x7efe880d5dc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7efe780097c0 tx=0x7efe7801c3d0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8cc86640 1 -- 172.21.15.80:0/3680231969 shutdown_connections 2024-04-25T13:46:25.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8cc86640 1 --2- 172.21.15.80:0/3680231969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d59b0 0x7efe880d5dc0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8cc86640 1 -- 172.21.15.80:0/3680231969 >> 172.21.15.80:0/3680231969 conn(0x7efe880737a0 msgr2=0x7efe8806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:25.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8cc86640 1 -- 172.21.15.80:0/3680231969 shutdown_connections 2024-04-25T13:46:25.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8cc86640 1 -- 172.21.15.80:0/3680231969 wait complete. 2024-04-25T13:46:25.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.475+0000 7efe8cc86640 1 Processor -- start 2024-04-25T13:46:25.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 -- start start 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d2e80 0x7efe8815c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe880d4330 0x7efe881525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880d59b0 0x7efe88152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efe8815e0e0 con 0x7efe880d2e80 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe8815df60 con 0x7efe880d4330 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe8816e950 con 0x7efe880d59b0 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d2e80 0x7efe8815c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d2e80 0x7efe8815c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57410/0 (socket says 172.21.15.80:57410) 2024-04-25T13:46:25.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 -- 172.21.15.80:0/3512858471 learned_addr learned my addr 172.21.15.80:0/3512858471 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 -- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880d59b0 msgr2=0x7efe88152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe86d9d640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880d59b0 0x7efe88152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe85d9b640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe880d4330 0x7efe881525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880d59b0 0x7efe88152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 -- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe880d4330 msgr2=0x7efe881525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe880d4330 0x7efe881525b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 -- 172.21.15.80:0/3512858471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe88152af0 con 0x7efe880d2e80 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8659c640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d2e80 0x7efe8815c9d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00c3e0 tx=0x7efe7c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe76ffd640 1 -- 172.21.15.80:0/3512858471 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c017070 con 0x7efe880d2e80 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe76ffd640 1 -- 172.21.15.80:0/3512858471 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe7c005cf0 con 0x7efe880d2e80 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8815b930 con 0x7efe880d2e80 2024-04-25T13:46:25.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe76ffd640 1 -- 172.21.15.80:0/3512858471 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7c007310 con 0x7efe880d2e80 2024-04-25T13:46:25.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8815bef0 con 0x7efe880d2e80 2024-04-25T13:46:25.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.479+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8806a3a0 con 0x7efe880d2e80 2024-04-25T13:46:25.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.483+0000 7efe76ffd640 1 -- 172.21.15.80:0/3512858471 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe7c01a070 con 0x7efe880d2e80 2024-04-25T13:46:25.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.483+0000 7efe76ffd640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6806f400 0x7efe68071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.483+0000 7efe76ffd640 1 -- 172.21.15.80:0/3512858471 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7efe7c08e6e0 con 0x7efe880d2e80 2024-04-25T13:46:25.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.483+0000 7efe85d9b640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6806f400 0x7efe68071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.483+0000 7efe85d9b640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6806f400 0x7efe68071910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efe780ad000 tx=0x7efe78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:25.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.487+0000 7efe76ffd640 1 -- 172.21.15.80:0/3512858471 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe7c021020 con 0x7efe880d2e80 2024-04-25T13:46:25.488 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T13:46:25.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.711+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7efe880bbb90 con 0x7efe880d2e80 2024-04-25T13:46:25.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.711+0000 7efe76ffd640 1 -- 172.21.15.80:0/3512858471 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7efe880d6300 con 0x7efe880d2e80 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:46:25.710 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.711 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:46:25.712 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.713 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:46:25.714 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.715 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:46:25.716 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:46:25.717 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.718 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.719 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:46:25.720 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:46:25.721 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.722 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:25.723 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:46:25.723 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:25.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6806f400 msgr2=0x7efe68071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6806f400 0x7efe68071910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7efe780ad000 tx=0x7efe78008040 comp rx=0 tx=0).stop 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d2e80 msgr2=0x7efe8815c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d2e80 0x7efe8815c9d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7efe7c00c3e0 tx=0x7efe7c00c8a0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 shutdown_connections 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6806f400 0x7efe68071910 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe880d59b0 0x7efe88152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe880d4330 0x7efe881525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 --2- 172.21.15.80:0/3512858471 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe880d2e80 0x7efe8815c9d0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 >> 172.21.15.80:0/3512858471 conn(0x7efe880737a0 msgr2=0x7efe8815e2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 shutdown_connections 2024-04-25T13:46:25.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.715+0000 7efe8cc86640 1 -- 172.21.15.80:0/3512858471 wait complete. 2024-04-25T13:46:25.733 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:25.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.883+0000 7f4a429d0640 1 Processor -- start 2024-04-25T13:46:25.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.883+0000 7f4a429d0640 1 -- start start 2024-04-25T13:46:25.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a429d0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4a3c06f5a0 0x7f4a3c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a429d0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a3c06fcf0 con 0x7f4a3c0733b0 2024-04-25T13:46:25.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a429d0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a3c0b59f0 con 0x7f4a3c161070 2024-04-25T13:46:25.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a429d0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4a3c073f10 con 0x7f4a3c06f5a0 2024-04-25T13:46:25.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a40f6d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4a3c0733b0 0x7f4a3c0b52e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38358/0 (socket says 172.21.15.80:38358) 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a40f6d640 1 -- 172.21.15.80:0/2307222187 learned_addr learned my addr 172.21.15.80:0/2307222187 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3bfff640 1 --2- 172.21.15.80:0/2307222187 >> v2:172.21.15.87:3300/0 conn(0x7f4a3c06f5a0 0x7f4a3c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2317127020 0 0) 0x7f4a3c06fcf0 con 0x7f4a3c0733b0 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4a28003530 con 0x7f4a3c0733b0 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4154081947 0 0) 0x7f4a28003530 con 0x7f4a3c0733b0 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 >> v2:172.21.15.87:3300/0 conn(0x7f4a3c06f5a0 msgr2=0x7f4a3c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 --2- 172.21.15.80:0/2307222187 >> v2:172.21.15.87:3300/0 conn(0x7f4a3c06f5a0 0x7f4a3c160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 >> v1:172.21.15.87:6789/0 conn(0x7f4a3c161070 legacy=0x7f4a3c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c0b0c60 con 0x7f4a3c0733b0 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4a34001a00 con 0x7f4a3c0733b0 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 >> v1:172.21.15.80:6789/0 conn(0x7f4a3c0733b0 legacy=0x7f4a3c0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a280039c0 0x7f4a28003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a28004810 0x7f4a28004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a28005260 0x7f4a2800dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c073f10 con 0x7f4a28004810 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a2800e400 con 0x7f4a280039c0 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a2800e600 con 0x7f4a28005260 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a337fe640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a28004810 0x7f4a28004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3bfff640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a280039c0 0x7f4a28003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a40f6d640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a28005260 0x7f4a2800dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a337fe640 1 -- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a28005260 msgr2=0x7f4a2800dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a337fe640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a28005260 0x7f4a2800dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a337fe640 1 -- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a280039c0 msgr2=0x7f4a28003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a337fe640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a280039c0 0x7f4a28003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a337fe640 1 -- 172.21.15.80:0/2307222187 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a28003530 con 0x7f4a28004810 2024-04-25T13:46:25.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a3bfff640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a280039c0 0x7f4a28003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:25.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.887+0000 7f4a40f6d640 1 -- 172.21.15.80:0/2307222187 reap_dead start 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a337fe640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a28004810 0x7f4a28004c20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f4a24001780 tx=0x7f4a2401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56602599ab0a0bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a240013a0 con 0x7f4a28004810 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a240052f0 con 0x7f4a28004810 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a3b7fe640 1 -- 172.21.15.80:0/2307222187 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a24003840 con 0x7f4a28004810 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a429d0640 1 -- 172.21.15.80:0/2307222187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a28004810 msgr2=0x7f4a28004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a429d0640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a28004810 0x7f4a28004c20 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f4a24001780 tx=0x7f4a2401c400 comp rx=0 tx=0).stop 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a429d0640 1 -- 172.21.15.80:0/2307222187 shutdown_connections 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a429d0640 1 --2- 172.21.15.80:0/2307222187 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a28004810 0x7f4a28004c20 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a429d0640 1 -- 172.21.15.80:0/2307222187 >> 172.21.15.80:0/2307222187 conn(0x7f4a3c06d080 msgr2=0x7f4a3c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a429d0640 1 -- 172.21.15.80:0/2307222187 shutdown_connections 2024-04-25T13:46:25.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.891+0000 7f4a429d0640 1 -- 172.21.15.80:0/2307222187 wait complete. 2024-04-25T13:46:25.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 Processor -- start 2024-04-25T13:46:25.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 -- start start 2024-04-25T13:46:25.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a28004810 0x7f4a3c1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a3c158a50 0x7f4a3c14b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a3c1f6030 0x7f4a3c1f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c0b53e0 con 0x7f4a3c1f6030 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a3c161890 con 0x7f4a28004810 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a3c0b5560 con 0x7f4a3c158a50 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a3c1f6030 0x7f4a3c1f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a3c1f6030 0x7f4a3c1f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57436/0 (socket says 172.21.15.80:57436) 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 -- 172.21.15.80:0/1394971819 learned_addr learned my addr 172.21.15.80:0/1394971819 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a337fe640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a3c158a50 0x7f4a3c14b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a3bfff640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a28004810 0x7f4a3c1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 -- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a3c158a50 msgr2=0x7f4a3c14b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a3c158a50 0x7f4a3c14b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 -- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a28004810 msgr2=0x7f4a3c1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a28004810 0x7f4a3c1546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 -- 172.21.15.80:0/1394971819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3c156c50 con 0x7f4a3c1f6030 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a3bfff640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a28004810 0x7f4a3c1546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a40f6d640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a3c1f6030 0x7f4a3c1f6440 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0b4970 tx=0x7f4a3400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a397fa640 1 -- 172.21.15.80:0/1394971819 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3400ec40 con 0x7f4a3c1f6030 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a397fa640 1 -- 172.21.15.80:0/1394971819 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a3400ede0 con 0x7f4a3c1f6030 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3c156f30 con 0x7f4a3c1f6030 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a397fa640 1 -- 172.21.15.80:0/1394971819 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a34014670 con 0x7f4a3c1f6030 2024-04-25T13:46:25.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a3c1574f0 con 0x7f4a3c1f6030 2024-04-25T13:46:25.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.895+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a3c06a3a0 con 0x7f4a3c1f6030 2024-04-25T13:46:25.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.899+0000 7f4a397fa640 1 -- 172.21.15.80:0/1394971819 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4a34014810 con 0x7f4a3c1f6030 2024-04-25T13:46:25.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.899+0000 7f4a397fa640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a1c06f400 0x7f4a1c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:25.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.899+0000 7f4a397fa640 1 -- 172.21.15.80:0/1394971819 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7f4a3408ed60 con 0x7f4a3c1f6030 2024-04-25T13:46:25.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.899+0000 7f4a3bfff640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a1c06f400 0x7f4a1c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:25.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.899+0000 7f4a3bfff640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a1c06f400 0x7f4a1c071910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c06b930 tx=0x7f4a2c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:25.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:25.903+0000 7f4a397fa640 1 -- 172.21.15.80:0/1394971819 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a3405c0d0 con 0x7f4a3c1f6030 2024-04-25T13:46:26.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a3c0739c0 con 0x7f4a3c1f6030 2024-04-25T13:46:26.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a397fa640 1 -- 172.21.15.80:0/1394971819 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+12830 (secure 0 0 0) 0x7f4a3405b9d0 con 0x7f4a3c1f6030 2024-04-25T13:46:26.065 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:26.065 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":118,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:25.285726+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:26.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a1c06f400 msgr2=0x7f4a1c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a1c06f400 0x7f4a1c071910 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c06b930 tx=0x7f4a2c008040 comp rx=0 tx=0).stop 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a3c1f6030 msgr2=0x7f4a3c1f6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a3c1f6030 0x7f4a3c1f6440 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0b4970 tx=0x7f4a3400e530 comp rx=0 tx=0).stop 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 shutdown_connections 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a1c06f400 0x7f4a1c071910 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a3c1f6030 0x7f4a3c1f6440 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a3c158a50 0x7f4a3c14b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 --2- 172.21.15.80:0/1394971819 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a28004810 0x7f4a3c1546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 >> 172.21.15.80:0/1394971819 conn(0x7f4a3c06d080 msgr2=0x7f4a3c1552c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 shutdown_connections 2024-04-25T13:46:26.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.067+0000 7f4a429d0640 1 -- 172.21.15.80:0/1394971819 wait complete. 2024-04-25T13:46:26.088 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:46:26.089 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:46:26.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.243+0000 7f3558433640 1 Processor -- start 2024-04-25T13:46:26.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.243+0000 7f3558433640 1 -- start start 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f3558433640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3550161270 0x7f35500724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f3558433640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35500b08c0 0x7f3550072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f3558433640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3550161680 con 0x7f3550072a00 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f3558433640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f35500d12e0 con 0x7f35500b08c0 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f3558433640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3550073350 con 0x7f3550161270 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f35569d0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35500b08c0 0x7f3550072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f35569d0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35500b08c0 0x7f3550072e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57438/0 (socket says 172.21.15.80:57438) 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f35569d0640 1 -- 172.21.15.80:0/1636248143 learned_addr learned my addr 172.21.15.80:0/1636248143 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:26.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f35561cf640 1 --2- 172.21.15.80:0/1636248143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3550161270 0x7f35500724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f35569d0640 1 -- 172.21.15.80:0/1636248143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3550161270 msgr2=0x7f35500724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f35569d0640 1 --2- 172.21.15.80:0/1636248143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3550161270 0x7f35500724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f35569d0640 1 -- 172.21.15.80:0/1636248143 >> v1:172.21.15.87:6789/0 conn(0x7f3550072a00 legacy=0x7f355006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.247+0000 7f35569d0640 1 -- 172.21.15.80:0/1636248143 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3550163de0 con 0x7f35500b08c0 2024-04-25T13:46:26.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f35569d0640 1 --2- 172.21.15.80:0/1636248143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35500b08c0 0x7f3550072e10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3548009e80 tx=0x7f354801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee20eb22adda65de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f35551cd640 1 -- 172.21.15.80:0/1636248143 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480a5650 con 0x7f35500b08c0 2024-04-25T13:46:26.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f35551cd640 1 -- 172.21.15.80:0/1636248143 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35480a57f0 con 0x7f35500b08c0 2024-04-25T13:46:26.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f35551cd640 1 -- 172.21.15.80:0/1636248143 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480a5bc0 con 0x7f35500b08c0 2024-04-25T13:46:26.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f3558433640 1 -- 172.21.15.80:0/1636248143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35500b08c0 msgr2=0x7f3550072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f3558433640 1 --2- 172.21.15.80:0/1636248143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35500b08c0 0x7f3550072e10 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f3548009e80 tx=0x7f354801c9d0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f3558433640 1 -- 172.21.15.80:0/1636248143 shutdown_connections 2024-04-25T13:46:26.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f3558433640 1 --2- 172.21.15.80:0/1636248143 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35500b08c0 0x7f3550072e10 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f3558433640 1 --2- 172.21.15.80:0/1636248143 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3550161270 0x7f35500724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f3558433640 1 -- 172.21.15.80:0/1636248143 >> 172.21.15.80:0/1636248143 conn(0x7f35500b4160 msgr2=0x7f35500d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f3558433640 1 -- 172.21.15.80:0/1636248143 shutdown_connections 2024-04-25T13:46:26.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.251+0000 7f3558433640 1 -- 172.21.15.80:0/1636248143 wait complete. 2024-04-25T13:46:26.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 Processor -- start 2024-04-25T13:46:26.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 -- start start 2024-04-25T13:46:26.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3550072a00 0x7f3550157c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3550161270 0x7f355014be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f355015af20 0x7f355015d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f355016d300 con 0x7f3550072a00 2024-04-25T13:46:26.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f355016d180 con 0x7f3550161270 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f355016d480 con 0x7f355015af20 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3550072a00 0x7f3550157c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3550072a00 0x7f3550157c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57448/0 (socket says 172.21.15.80:57448) 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 -- 172.21.15.80:0/33766680 learned_addr learned my addr 172.21.15.80:0/33766680 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35559ce640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3550161270 0x7f355014be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35569d0640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f355015af20 0x7f355015d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 -- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f355015af20 msgr2=0x7f355015d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f355015af20 0x7f355015d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 -- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3550161270 msgr2=0x7f355014be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3550161270 0x7f355014be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 -- 172.21.15.80:0/33766680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355015b8c0 con 0x7f3550072a00 2024-04-25T13:46:26.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f35561cf640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3550072a00 0x7f3550157c80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f354c00d5f0 tx=0x7f354c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3546ffd640 1 -- 172.21.15.80:0/33766680 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c0125f0 con 0x7f3550072a00 2024-04-25T13:46:26.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3546ffd640 1 -- 172.21.15.80:0/33766680 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f354c019070 con 0x7f3550072a00 2024-04-25T13:46:26.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3550151320 con 0x7f3550072a00 2024-04-25T13:46:26.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3546ffd640 1 -- 172.21.15.80:0/33766680 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c0129b0 con 0x7f3550072a00 2024-04-25T13:46:26.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f355015bb60 con 0x7f3550072a00 2024-04-25T13:46:26.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.255+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f355006a3a0 con 0x7f3550072a00 2024-04-25T13:46:26.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.259+0000 7f3546ffd640 1 -- 172.21.15.80:0/33766680 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f354c012b50 con 0x7f3550072a00 2024-04-25T13:46:26.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.259+0000 7f3546ffd640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f353406f400 0x7f3534071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.259+0000 7f3546ffd640 1 -- 172.21.15.80:0/33766680 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7f354c08fd80 con 0x7f3550072a00 2024-04-25T13:46:26.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.259+0000 7f35559ce640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f353406f400 0x7f3534071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.259+0000 7f35559ce640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f353406f400 0x7f3534071910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f35501fd130 tx=0x7f3540008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.263+0000 7f3546ffd640 1 -- 172.21.15.80:0/33766680 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f354c05d130 con 0x7f3550072a00 2024-04-25T13:46:26.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3550161050 con 0x7f3550072a00 2024-04-25T13:46:26.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3546ffd640 1 -- 172.21.15.80:0/33766680 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f354c05ca30 con 0x7f3550072a00 2024-04-25T13:46:26.425 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:46:26.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f353406f400 msgr2=0x7f3534071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f353406f400 0x7f3534071910 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f35501fd130 tx=0x7f3540008040 comp rx=0 tx=0).stop 2024-04-25T13:46:26.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3550072a00 msgr2=0x7f3550157c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3550072a00 0x7f3550157c80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f354c00d5f0 tx=0x7f354c00dab0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 shutdown_connections 2024-04-25T13:46:26.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f353406f400 0x7f3534071910 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f355015af20 0x7f355015d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3550161270 0x7f355014be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.427+0000 7f3558433640 1 --2- 172.21.15.80:0/33766680 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3550072a00 0x7f3550157c80 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.431+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 >> 172.21.15.80:0/33766680 conn(0x7f35500b4160 msgr2=0x7f355015e460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.431+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 shutdown_connections 2024-04-25T13:46:26.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.431+0000 7f3558433640 1 -- 172.21.15.80:0/33766680 wait complete. 2024-04-25T13:46:26.448 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:46:26.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.595+0000 7f34c1b44640 1 Processor -- start 2024-04-25T13:46:26.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.595+0000 7f34c1b44640 1 -- start start 2024-04-25T13:46:26.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34c1b44640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f34bc0b5a00 0x7f34bc141100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34c1b44640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f34bc162710 0x7f34bc06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34c1b44640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34c1b44640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f34bc06df50 con 0x7f34bc162710 2024-04-25T13:46:26.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34c1b44640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f34bc0b62a0 con 0x7f34bc0b5a00 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34c1b44640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc0b5fc0 con 0x7f34bc06db40 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34baffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f34bc162710 0x7f34bc06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34baffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f34bc162710 0x7f34bc06d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57458/0 (socket says 172.21.15.80:57458) 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34baffd640 1 -- 172.21.15.80:0/3702093852 learned_addr learned my addr 172.21.15.80:0/3702093852 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34bb7fe640 1 --2- 172.21.15.80:0/3702093852 >> v2:172.21.15.87:3300/0 conn(0x7f34bc0b5a00 0x7f34bc141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34bbfff640 1 --2- 172.21.15.80:0/3702093852 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34baffd640 1 -- 172.21.15.80:0/3702093852 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 msgr2=0x7f34bc068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34baffd640 1 --2- 172.21.15.80:0/3702093852 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34baffd640 1 -- 172.21.15.80:0/3702093852 >> v2:172.21.15.87:3300/0 conn(0x7f34bc0b5a00 msgr2=0x7f34bc141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34baffd640 1 --2- 172.21.15.80:0/3702093852 >> v2:172.21.15.87:3300/0 conn(0x7f34bc0b5a00 0x7f34bc141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.599+0000 7f34baffd640 1 -- 172.21.15.80:0/3702093852 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc15e900 con 0x7f34bc162710 2024-04-25T13:46:26.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34baffd640 1 --2- 172.21.15.80:0/3702093852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc162710 0x7f34bc06d600 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0729f0 tx=0x7f34b001c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28348bb582aeef81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34ba7fc640 1 -- 172.21.15.80:0/3702093852 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00b0070 con 0x7f34bc162710 2024-04-25T13:46:26.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34ba7fc640 1 -- 172.21.15.80:0/3702093852 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b00ab600 con 0x7f34bc162710 2024-04-25T13:46:26.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34ba7fc640 1 -- 172.21.15.80:0/3702093852 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00ab9d0 con 0x7f34bc162710 2024-04-25T13:46:26.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 -- 172.21.15.80:0/3702093852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc162710 msgr2=0x7f34bc06d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 --2- 172.21.15.80:0/3702093852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc162710 0x7f34bc06d600 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f34bc0729f0 tx=0x7f34b001c260 comp rx=0 tx=0).stop 2024-04-25T13:46:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 -- 172.21.15.80:0/3702093852 shutdown_connections 2024-04-25T13:46:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 --2- 172.21.15.80:0/3702093852 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 --2- 172.21.15.80:0/3702093852 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc162710 0x7f34bc06d600 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 --2- 172.21.15.80:0/3702093852 >> v2:172.21.15.87:3300/0 conn(0x7f34bc0b5a00 0x7f34bc141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 -- 172.21.15.80:0/3702093852 >> 172.21.15.80:0/3702093852 conn(0x7f34bc072ed0 msgr2=0x7f34bc0732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 -- 172.21.15.80:0/3702093852 shutdown_connections 2024-04-25T13:46:26.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 -- 172.21.15.80:0/3702093852 wait complete. 2024-04-25T13:46:26.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 Processor -- start 2024-04-25T13:46:26.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.603+0000 7f34c1b44640 1 -- start start 2024-04-25T13:46:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc0b5a00 0x7f34bc13caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34bc162710 0x7f34bc13d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc143970 con 0x7f34bc0b5a00 2024-04-25T13:46:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc1437f0 con 0x7f34bc162710 2024-04-25T13:46:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f34bc143af0 con 0x7f34bc06db40 2024-04-25T13:46:26.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc0b5a00 0x7f34bc13caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc0b5a00 0x7f34bc13caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57472/0 (socket says 172.21.15.80:57472) 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 -- 172.21.15.80:0/1175692332 learned_addr learned my addr 172.21.15.80:0/1175692332 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34bbfff640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34bc162710 0x7f34bc13d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34bb7fe640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 -- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 msgr2=0x7f34bc157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 -- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34bc162710 msgr2=0x7f34bc13d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34bc162710 0x7f34bc13d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 -- 172.21.15.80:0/1175692332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc143510 con 0x7f34bc0b5a00 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34bbfff640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34bc162710 0x7f34bc13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:26.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34bb7fe640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:26.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34baffd640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc0b5a00 0x7f34bc13caf0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f34b001cb00 tx=0x7f34b0019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34a3fff640 1 -- 172.21.15.80:0/1175692332 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00ba070 con 0x7f34bc0b5a00 2024-04-25T13:46:26.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34a3fff640 1 -- 172.21.15.80:0/1175692332 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f34b00b4640 con 0x7f34bc0b5a00 2024-04-25T13:46:26.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc0b3080 con 0x7f34bc0b5a00 2024-04-25T13:46:26.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34a3fff640 1 -- 172.21.15.80:0/1175692332 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b00b0040 con 0x7f34bc0b5a00 2024-04-25T13:46:26.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc0b3640 con 0x7f34bc0b5a00 2024-04-25T13:46:26.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.607+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34bc06a3a0 con 0x7f34bc0b5a00 2024-04-25T13:46:26.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.611+0000 7f34a3fff640 1 -- 172.21.15.80:0/1175692332 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f34b00bd070 con 0x7f34bc0b5a00 2024-04-25T13:46:26.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.611+0000 7f34a3fff640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34a806f430 0x7f34a8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.611+0000 7f34a3fff640 1 -- 172.21.15.80:0/1175692332 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7f34b0128130 con 0x7f34bc0b5a00 2024-04-25T13:46:26.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.611+0000 7f34bb7fe640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34a806f430 0x7f34a8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.611+0000 7f34bb7fe640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34a806f430 0x7f34a8071940 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f34bc06eca0 tx=0x7f34ac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.615+0000 7f34a3fff640 1 -- 172.21.15.80:0/1175692332 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f34b0016110 con 0x7f34bc0b5a00 2024-04-25T13:46:26.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.775+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f34bc1432a0 con 0x7f34bc0b5a00 2024-04-25T13:46:26.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.775+0000 7f34a3fff640 1 -- 172.21.15.80:0/1175692332 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v118) ==== 84+0+65 (secure 0 0 0) 0x7f34b00f54e0 con 0x7f34bc0b5a00 2024-04-25T13:46:26.775 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:26.775 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:46:26.775 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:46:26.775 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:46:26.775 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:46:26.776 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:26.776 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:46:26.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34a806f430 msgr2=0x7f34a8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34a806f430 0x7f34a8071940 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f34bc06eca0 tx=0x7f34ac008040 comp rx=0 tx=0).stop 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc0b5a00 msgr2=0x7f34bc13caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc0b5a00 0x7f34bc13caf0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f34b001cb00 tx=0x7f34b0019000 comp rx=0 tx=0).stop 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 shutdown_connections 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f34a806f430 0x7f34a8071940 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f34bc162710 0x7f34bc13d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f34bc0b5a00 0x7f34bc13caf0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 --2- 172.21.15.80:0/1175692332 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f34bc06db40 0x7f34bc157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 >> 172.21.15.80:0/1175692332 conn(0x7f34bc072ed0 msgr2=0x7f34bc0d1130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 shutdown_connections 2024-04-25T13:46:26.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.779+0000 7f34c1b44640 1 -- 172.21.15.80:0/1175692332 wait complete. 2024-04-25T13:46:26.797 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:46:26.797 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:46:26.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.951+0000 7fd4f3fc9640 1 Processor -- start 2024-04-25T13:46:26.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.951+0000 7fd4f3fc9640 1 -- start start 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f3fc9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd4ec0d0f50 0x7fd4ec161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f3fc9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ec0b0e10 con 0x7fd4ec0b48c0 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f3fc9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd4ec0b10f0 con 0x7fd4ec15d520 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f3fc9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd4ec0b0bc0 con 0x7fd4ec0d0f50 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1d65640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd4ec0b48c0 0x7fd4ec0d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38370/0 (socket says 172.21.15.80:38370) 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1d65640 1 -- 172.21.15.80:0/3716231900 learned_addr learned my addr 172.21.15.80:0/3716231900 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1564640 1 --2- 172.21.15.80:0/3716231900 >> v2:172.21.15.87:3300/0 conn(0x7fd4ec0d0f50 0x7fd4ec161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1875891834 0 0) 0x7fd4ec0b0e10 con 0x7fd4ec0b48c0 2024-04-25T13:46:26.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd4d8003530 con 0x7fd4ec0b48c0 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 677600188 0 0) 0x7fd4d8003530 con 0x7fd4ec0b48c0 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 >> v2:172.21.15.87:3300/0 conn(0x7fd4ec0d0f50 msgr2=0x7fd4ec161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 --2- 172.21.15.80:0/3716231900 >> v2:172.21.15.87:3300/0 conn(0x7fd4ec0d0f50 0x7fd4ec161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 >> v1:172.21.15.87:6789/0 conn(0x7fd4ec15d520 legacy=0x7fd4ec15d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec06f710 con 0x7fd4ec0b48c0 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fd4e8001680 con 0x7fd4ec0b48c0 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 >> v1:172.21.15.80:6789/0 conn(0x7fd4ec0b48c0 legacy=0x7fd4ec0d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d80039c0 0x7fd4d8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4d8004810 0x7fd4d8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8005260 0x7fd4d800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec0b0bc0 con 0x7fd4d8004810 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d800c1f0 con 0x7fd4d8005260 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d800c3f0 con 0x7fd4d80039c0 2024-04-25T13:46:26.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1564640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4d8004810 0x7fd4d8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f2566640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8005260 0x7fd4d800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1564640 1 -- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d80039c0 msgr2=0x7fd4d8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1d65640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d80039c0 0x7fd4d8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1564640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4d80039c0 0x7fd4d8003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1564640 1 -- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8005260 msgr2=0x7fd4d800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1564640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8005260 0x7fd4d800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f1564640 1 -- 172.21.15.80:0/3716231900 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d8003530 con 0x7fd4d8004810 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f2566640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8005260 0x7fd4d800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:26.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.955+0000 7fd4f2566640 1 -- 172.21.15.80:0/3716231900 reap_dead start 2024-04-25T13:46:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f1564640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4d8004810 0x7fd4d8004c20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009ae0 tx=0x7fd4dc01d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56642018b55f7171 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0b2070 con 0x7fd4d8004810 2024-04-25T13:46:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4dc0a5550 con 0x7fd4d8004810 2024-04-25T13:46:26.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f0d63640 1 -- 172.21.15.80:0/3716231900 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0096b0 con 0x7fd4d8004810 2024-04-25T13:46:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/3716231900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4d8004810 msgr2=0x7fd4d8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4d8004810 0x7fd4d8004c20 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009ae0 tx=0x7fd4dc01d4d0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/3716231900 shutdown_connections 2024-04-25T13:46:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 --2- 172.21.15.80:0/3716231900 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4d8004810 0x7fd4d8004c20 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/3716231900 >> 172.21.15.80:0/3716231900 conn(0x7fd4ec0b35c0 msgr2=0x7fd4ec0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/3716231900 shutdown_connections 2024-04-25T13:46:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/3716231900 wait complete. 2024-04-25T13:46:26.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 Processor -- start 2024-04-25T13:46:26.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.959+0000 7fd4f3fc9640 1 -- start start 2024-04-25T13:46:26.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8004810 0x7fd4ec159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec14b710 0x7fd4ec1f9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4ec0cf730 0x7fd4ec0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec15dab0 con 0x7fd4ec0cf730 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ec15d930 con 0x7fd4d8004810 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd4ec15dc30 con 0x7fd4ec14b710 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4ec0cf730 0x7fd4ec0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4ec0cf730 0x7fd4ec0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57488/0 (socket says 172.21.15.80:57488) 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 -- 172.21.15.80:0/465157943 learned_addr learned my addr 172.21.15.80:0/465157943 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f1d65640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8004810 0x7fd4ec159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f1564640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec14b710 0x7fd4ec1f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 -- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec14b710 msgr2=0x7fd4ec1f9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec14b710 0x7fd4ec1f9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 -- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8004810 msgr2=0x7fd4ec159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8004810 0x7fd4ec159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:26.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 -- 172.21.15.80:0/465157943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ec0d0080 con 0x7fd4ec0cf730 2024-04-25T13:46:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f1d65640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8004810 0x7fd4ec159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f2566640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4ec0cf730 0x7fd4ec0cfb40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0b6310 tx=0x7fd4e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4e27fc640 1 -- 172.21.15.80:0/465157943 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4018070 con 0x7fd4ec0cf730 2024-04-25T13:46:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4e27fc640 1 -- 172.21.15.80:0/465157943 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4e4003ba0 con 0x7fd4ec0cf730 2024-04-25T13:46:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4e27fc640 1 -- 172.21.15.80:0/465157943 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4005070 con 0x7fd4ec0cf730 2024-04-25T13:46:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ec154dc0 con 0x7fd4ec0cf730 2024-04-25T13:46:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ec148ef0 con 0x7fd4ec0cf730 2024-04-25T13:46:26.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.963+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4ec159ef0 con 0x7fd4ec0cf730 2024-04-25T13:46:26.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.967+0000 7fd4e27fc640 1 -- 172.21.15.80:0/465157943 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4e401b070 con 0x7fd4ec0cf730 2024-04-25T13:46:26.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.967+0000 7fd4e27fc640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4d006f400 0x7fd4d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:26.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.967+0000 7fd4e27fc640 1 -- 172.21.15.80:0/465157943 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 4857+0+0 (secure 0 0 0) 0x7fd4e408ea10 con 0x7fd4ec0cf730 2024-04-25T13:46:26.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.967+0000 7fd4f1d65640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4d006f400 0x7fd4d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:26.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.967+0000 7fd4f1d65640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4d006f400 0x7fd4d0071910 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8001ec0 tx=0x7fd4e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:26.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:26.971+0000 7fd4e27fc640 1 -- 172.21.15.80:0/465157943 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd4e4022020 con 0x7fd4ec0cf730 2024-04-25T13:46:27.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.123+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fd4ec154f90 con 0x7fd4ec0cf730 2024-04-25T13:46:27.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4e27fc640 1 -- 172.21.15.80:0/465157943 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v119) ==== 138+0+0 (secure 0 0 0) 0x7fd4ec0d0080 con 0x7fd4ec0cf730 2024-04-25T13:46:27.316 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:46:27.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4d006f400 msgr2=0x7fd4d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:27.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4d006f400 0x7fd4d0071910 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8001ec0 tx=0x7fd4e8009040 comp rx=0 tx=0).stop 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4ec0cf730 msgr2=0x7fd4ec0cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4ec0cf730 0x7fd4ec0cfb40 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec0b6310 tx=0x7fd4e400dc00 comp rx=0 tx=0).stop 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 shutdown_connections 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd4d006f400 0x7fd4d0071910 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd4ec0cf730 0x7fd4ec0cfb40 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4ec14b710 0x7fd4ec1f9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 --2- 172.21.15.80:0/465157943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4d8004810 0x7fd4ec159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 >> 172.21.15.80:0/465157943 conn(0x7fd4ec0b35c0 msgr2=0x7fd4ec15f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 shutdown_connections 2024-04-25T13:46:27.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.319+0000 7fd4f3fc9640 1 -- 172.21.15.80:0/465157943 wait complete. 2024-04-25T13:46:27.344 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:46:27.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.503+0000 7fd57e1ae640 1 Processor -- start 2024-04-25T13:46:27.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.503+0000 7fd57e1ae640 1 -- start start 2024-04-25T13:46:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd57e1ae640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd57806f5a0 0x7fd578160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd57e1ae640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd578161070 0x7fd578161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd57e1ae640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd57806fcf0 con 0x7fd5780b4bd0 2024-04-25T13:46:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd57e1ae640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd5781619c0 con 0x7fd578161070 2024-04-25T13:46:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd57e1ae640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd578073f10 con 0x7fd57806f5a0 2024-04-25T13:46:27.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd56ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd578161070 0x7fd578161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:27.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd577fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd5780b4bd0 0x7fd5780b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38386/0 (socket says 172.21.15.80:38386) 2024-04-25T13:46:27.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd577fff640 1 -- 172.21.15.80:0/3898858612 learned_addr learned my addr 172.21.15.80:0/3898858612 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:27.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd5777fe640 1 --2- 172.21.15.80:0/3898858612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd57806f5a0 0x7fd578160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:27.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd56ffff640 1 -- 172.21.15.80:0/3898858612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd57806f5a0 msgr2=0x7fd578160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:27.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd56ffff640 1 --2- 172.21.15.80:0/3898858612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd57806f5a0 0x7fd578160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd56ffff640 1 -- 172.21.15.80:0/3898858612 >> v1:172.21.15.80:6789/0 conn(0x7fd5780b4bd0 legacy=0x7fd5780b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:27.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd576ffd640 1 -- 172.21.15.80:0/3898858612 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4235161875 0 0) 0x7fd57806fcf0 con 0x7fd5780b4bd0 2024-04-25T13:46:27.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd56ffff640 1 -- 172.21.15.80:0/3898858612 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5780b0c60 con 0x7fd578161070 2024-04-25T13:46:27.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd56ffff640 1 --2- 172.21.15.80:0/3898858612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd578161480 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd5600097f0 tx=0x7fd56001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5e70c31104790b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:27.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd576ffd640 1 -- 172.21.15.80:0/3898858612 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5600b0070 con 0x7fd578161070 2024-04-25T13:46:27.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd576ffd640 1 -- 172.21.15.80:0/3898858612 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5600ab600 con 0x7fd578161070 2024-04-25T13:46:27.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.507+0000 7fd576ffd640 1 -- 172.21.15.80:0/3898858612 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5600ab9d0 con 0x7fd578161070 2024-04-25T13:46:27.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 -- 172.21.15.80:0/3898858612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 msgr2=0x7fd578161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:27.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/3898858612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd578161480 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd5600097f0 tx=0x7fd56001c410 comp rx=0 tx=0).stop 2024-04-25T13:46:27.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 -- 172.21.15.80:0/3898858612 shutdown_connections 2024-04-25T13:46:27.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/3898858612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd578161480 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/3898858612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd57806f5a0 0x7fd578160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 -- 172.21.15.80:0/3898858612 >> 172.21.15.80:0/3898858612 conn(0x7fd57806d080 msgr2=0x7fd5780b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:27.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 -- 172.21.15.80:0/3898858612 shutdown_connections 2024-04-25T13:46:27.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 -- 172.21.15.80:0/3898858612 wait complete. 2024-04-25T13:46:27.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 Processor -- start 2024-04-25T13:46:27.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.511+0000 7fd57e1ae640 1 -- start start 2024-04-25T13:46:27.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd57806f5a0 0x7fd57815f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:27.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5780b4bd0 0x7fd578155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:27.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd5780b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:27.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd57816a860 con 0x7fd578161070 2024-04-25T13:46:27.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd57816a6e0 con 0x7fd57806f5a0 2024-04-25T13:46:27.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd57816a9e0 con 0x7fd5780b4bd0 2024-04-25T13:46:27.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd5780b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:27.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd5780b0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57500/0 (socket says 172.21.15.80:57500) 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 -- 172.21.15.80:0/814025600 learned_addr learned my addr 172.21.15.80:0/814025600 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd56ffff640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5780b4bd0 0x7fd578155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd5777fe640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd57806f5a0 0x7fd57815f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 -- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5780b4bd0 msgr2=0x7fd578155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5780b4bd0 0x7fd578155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 -- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd57806f5a0 msgr2=0x7fd57815f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd57806f5a0 0x7fd57815f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 -- 172.21.15.80:0/814025600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57815f550 con 0x7fd578161070 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd577fff640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd5780b0120 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c6a0 tx=0x7fd57000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd574ff9640 1 -- 172.21.15.80:0/814025600 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570018070 con 0x7fd578161070 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd574ff9640 1 -- 172.21.15.80:0/814025600 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd570004d70 con 0x7fd578161070 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57815f990 con 0x7fd578161070 2024-04-25T13:46:27.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd574ff9640 1 -- 172.21.15.80:0/814025600 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57000a880 con 0x7fd578161070 2024-04-25T13:46:27.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57815ff50 con 0x7fd578161070 2024-04-25T13:46:27.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57806a3a0 con 0x7fd578161070 2024-04-25T13:46:27.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd574ff9640 1 -- 172.21.15.80:0/814025600 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd57001b070 con 0x7fd578161070 2024-04-25T13:46:27.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd574ff9640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd55806f400 0x7fd558071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:27.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.515+0000 7fd574ff9640 1 -- 172.21.15.80:0/814025600 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5227+0+0 (secure 0 0 0) 0x7fd57008e870 con 0x7fd578161070 2024-04-25T13:46:27.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.519+0000 7fd5777fe640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd55806f400 0x7fd558071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:27.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.519+0000 7fd5777fe640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd55806f400 0x7fd558071910 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd5780b6e70 tx=0x7fd568008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:27.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.523+0000 7fd574ff9640 1 -- 172.21.15.80:0/814025600 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd57005ba20 con 0x7fd578161070 2024-04-25T13:46:27.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:27.683+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd57815f720 con 0x7fd578161070 2024-04-25T13:46:28.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.331+0000 7fd574ff9640 1 -- 172.21.15.80:0/814025600 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v120) ==== 170+0+0 (secure 0 0 0) 0x7fd57005b320 con 0x7fd578161070 2024-04-25T13:46:28.329 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:46:28.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd55806f400 msgr2=0x7fd558071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:28.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd55806f400 0x7fd558071910 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd5780b6e70 tx=0x7fd568008040 comp rx=0 tx=0).stop 2024-04-25T13:46:28.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 msgr2=0x7fd5780b0120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd5780b0120 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd57000c6a0 tx=0x7fd57000cb60 comp rx=0 tx=0).stop 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 shutdown_connections 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd55806f400 0x7fd558071910 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd578161070 0x7fd5780b0120 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5780b4bd0 0x7fd578155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 --2- 172.21.15.80:0/814025600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd57806f5a0 0x7fd57815f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 >> 172.21.15.80:0/814025600 conn(0x7fd57806d080 msgr2=0x7fd5780b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 shutdown_connections 2024-04-25T13:46:28.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.335+0000 7fd57e1ae640 1 -- 172.21.15.80:0/814025600 wait complete. 2024-04-25T13:46:28.365 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:46:28.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb45b9e640 1 Processor -- start 2024-04-25T13:46:28.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb45b9e640 1 -- start start 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb45b9e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb400738e0 0x7fcb400d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb45b9e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb400d0010 con 0x7fcb400734d0 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb45b9e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb40074020 con 0x7fcb400ceec0 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb45b9e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcb40074300 con 0x7fcb400738e0 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb3f7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fcb400734d0 0x7fcb400ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38398/0 (socket says 172.21.15.80:38398) 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb3f7fe640 1 -- 172.21.15.80:0/2017723946 learned_addr learned my addr 172.21.15.80:0/2017723946 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb3ffff640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb400738e0 0x7fcb400d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681502253 0 0) 0x7fcb400d0010 con 0x7fcb400734d0 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.515+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcb28003530 con 0x7fcb400734d0 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3441977138 0 0) 0x7fcb28003530 con 0x7fcb400734d0 2024-04-25T13:46:28.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb400738e0 msgr2=0x7fcb400d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb400738e0 0x7fcb400d0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 >> v1:172.21.15.87:6789/0 conn(0x7fcb400ceec0 legacy=0x7fcb400d0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb400745e0 con 0x7fcb400734d0 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fcb30001680 con 0x7fcb400734d0 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 >> v1:172.21.15.80:6789/0 conn(0x7fcb400734d0 legacy=0x7fcb400ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb400738e0 0x7fcb280039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb28003f70 0x7fcb28004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28004d50 0x7fcb28005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40074300 con 0x7fcb28003f70 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2800bb60 con 0x7fcb28004d50 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2800bd60 con 0x7fcb400738e0 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3effd640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb28003f70 0x7fcb28004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:28.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3ffff640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28004d50 0x7fcb28005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:28.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3effd640 1 -- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb400738e0 msgr2=0x7fcb280039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:28.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3f7fe640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb400738e0 0x7fcb280039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:28.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3effd640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb400738e0 0x7fcb280039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3effd640 1 -- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28004d50 msgr2=0x7fcb28005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:28.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3effd640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28004d50 0x7fcb28005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3effd640 1 -- 172.21.15.80:0/2017723946 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb28003530 con 0x7fcb28003f70 2024-04-25T13:46:28.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3effd640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb28003f70 0x7fcb28004810 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fcb34001780 tx=0x7fcb3401c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0f23db5bbce09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:28.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb340013a0 con 0x7fcb28003f70 2024-04-25T13:46:28.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb340a9070 con 0x7fcb28003f70 2024-04-25T13:46:28.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3e7fc640 1 -- 172.21.15.80:0/2017723946 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb340052f0 con 0x7fcb28003f70 2024-04-25T13:46:28.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb45b9e640 1 -- 172.21.15.80:0/2017723946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb28003f70 msgr2=0x7fcb28004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:28.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb45b9e640 1 --2- 172.21.15.80:0/2017723946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb28003f70 0x7fcb28004810 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fcb34001780 tx=0x7fcb3401c420 comp rx=0 tx=0).stop 2024-04-25T13:46:28.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb3ffff640 1 -- 172.21.15.80:0/2017723946 reap_dead start 2024-04-25T13:46:28.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb45b9e640 1 -- 172.21.15.80:0/2017723946 shutdown_connections 2024-04-25T13:46:28.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb45b9e640 1 -- 172.21.15.80:0/2017723946 >> 172.21.15.80:0/2017723946 conn(0x7fcb400b3cc0 msgr2=0x7fcb400b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:28.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb45b9e640 1 -- 172.21.15.80:0/2017723946 shutdown_connections 2024-04-25T13:46:28.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.519+0000 7fcb45b9e640 1 -- 172.21.15.80:0/2017723946 wait complete. 2024-04-25T13:46:28.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 Processor -- start 2024-04-25T13:46:28.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 -- start start 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28003f70 0x7fcb4009eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb40154d20 0x7fcb4009ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb4009cf80 0x7fcb400aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4015e960 con 0x7fcb40154d20 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40073cf0 con 0x7fcb28003f70 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4015eae0 con 0x7fcb4009cf80 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb40154d20 0x7fcb4009ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb40154d20 0x7fcb4009ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57508/0 (socket says 172.21.15.80:57508) 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 -- 172.21.15.80:0/109839199 learned_addr learned my addr 172.21.15.80:0/109839199 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3ffff640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb4009cf80 0x7fcb400aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:28.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3f7fe640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28003f70 0x7fcb4009eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 -- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb4009cf80 msgr2=0x7fcb400aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb4009cf80 0x7fcb400aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 -- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28003f70 msgr2=0x7fcb4009eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28003f70 0x7fcb4009eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 -- 172.21.15.80:0/109839199 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4009d390 con 0x7fcb40154d20 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3f7fe640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28003f70 0x7fcb4009eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb3effd640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb40154d20 0x7fcb4009ca40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fcb34005040 tx=0x7fcb28005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb27fff640 1 -- 172.21.15.80:0/109839199 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb340ba030 con 0x7fcb40154d20 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb27fff640 1 -- 172.21.15.80:0/109839199 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb34000ee0 con 0x7fcb40154d20 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4009f4c0 con 0x7fcb40154d20 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.523+0000 7fcb27fff640 1 -- 172.21.15.80:0/109839199 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34007040 con 0x7fcb40154d20 2024-04-25T13:46:28.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.527+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb40155960 con 0x7fcb40154d20 2024-04-25T13:46:28.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.527+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb4006a3a0 con 0x7fcb40154d20 2024-04-25T13:46:28.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.527+0000 7fcb27fff640 1 -- 172.21.15.80:0/109839199 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcb340bd070 con 0x7fcb40154d20 2024-04-25T13:46:28.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.527+0000 7fcb27fff640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb2c06f400 0x7fcb2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:28.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.527+0000 7fcb27fff640 1 -- 172.21.15.80:0/109839199 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5609+0+0 (secure 0 0 0) 0x7fcb34129070 con 0x7fcb40154d20 2024-04-25T13:46:28.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.527+0000 7fcb3f7fe640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb2c06f400 0x7fcb2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:28.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.527+0000 7fcb3f7fe640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb2c06f400 0x7fcb2c071910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcb300017f0 tx=0x7fcb30001180 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:28.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.531+0000 7fcb27fff640 1 -- 172.21.15.80:0/109839199 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcb4009d390 con 0x7fcb40154d20 2024-04-25T13:46:28.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:28.715+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcb4009f250 con 0x7fcb40154d20 2024-04-25T13:46:29.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.335+0000 7fcb27fff640 1 -- 172.21.15.80:0/109839199 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v55) ==== 174+0+0 (secure 0 0 0) 0x7fcb34016130 con 0x7fcb40154d20 2024-04-25T13:46:29.332 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb2c06f400 msgr2=0x7fcb2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb2c06f400 0x7fcb2c071910 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcb300017f0 tx=0x7fcb30001180 comp rx=0 tx=0).stop 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb40154d20 msgr2=0x7fcb4009ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb40154d20 0x7fcb4009ca40 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fcb34005040 tx=0x7fcb28005840 comp rx=0 tx=0).stop 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 shutdown_connections 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb2c06f400 0x7fcb2c071910 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb4009cf80 0x7fcb400aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb40154d20 0x7fcb4009ca40 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 --2- 172.21.15.80:0/109839199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb28003f70 0x7fcb4009eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 >> 172.21.15.80:0/109839199 conn(0x7fcb400b3cc0 msgr2=0x7fcb4006f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:29.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 shutdown_connections 2024-04-25T13:46:29.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.339+0000 7fcb45b9e640 1 -- 172.21.15.80:0/109839199 wait complete. 2024-04-25T13:46:29.377 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:29.396 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:46:29.421 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:46:29.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5bbe23640 1 Processor -- start 2024-04-25T13:46:29.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5bbe23640 1 -- start start 2024-04-25T13:46:29.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5bbe23640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa5b40ce730 0x7fa5b40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5bbe23640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b4073710 con 0x7fa5b406e490 2024-04-25T13:46:29.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5bbe23640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5b40739f0 con 0x7fa5b40b0ed0 2024-04-25T13:46:29.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5bbe23640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa5b4073cd0 con 0x7fa5b40ce730 2024-04-25T13:46:29.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5b93be640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa5b40ce730 0x7fa5b40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5b93be640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa5b40ce730 0x7fa5b40ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57520/0 (socket says 172.21.15.80:57520) 2024-04-25T13:46:29.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5ba3c0640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa5b406e490 0x7fa5b406e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38400/0 (socket says 172.21.15.80:38400) 2024-04-25T13:46:29.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5b93be640 1 -- 172.21.15.80:0/3933620311 learned_addr learned my addr 172.21.15.80:0/3933620311 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:29.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5b93be640 1 -- 172.21.15.80:0/3933620311 >> v1:172.21.15.87:6789/0 conn(0x7fa5b40b0ed0 legacy=0x7fa5b415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5b8bbd640 1 -- 172.21.15.80:0/3933620311 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1586629898 0 0) 0x7fa5b4073710 con 0x7fa5b406e490 2024-04-25T13:46:29.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5b93be640 1 -- 172.21.15.80:0/3933620311 >> v1:172.21.15.80:6789/0 conn(0x7fa5b406e490 legacy=0x7fa5b406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.523+0000 7fa5b93be640 1 -- 172.21.15.80:0/3933620311 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b4073fb0 con 0x7fa5b40ce730 2024-04-25T13:46:29.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5b93be640 1 --2- 172.21.15.80:0/3933620311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40ceb40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009870 tx=0x7fa5a401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1c02d8c78df77c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:29.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5b8bbd640 1 -- 172.21.15.80:0/3933620311 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40b5070 con 0x7fa5b40ce730 2024-04-25T13:46:29.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5b8bbd640 1 -- 172.21.15.80:0/3933620311 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5a40a8a50 con 0x7fa5b40ce730 2024-04-25T13:46:29.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5b8bbd640 1 -- 172.21.15.80:0/3933620311 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5a40a8e20 con 0x7fa5b40ce730 2024-04-25T13:46:29.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5bbe23640 1 -- 172.21.15.80:0/3933620311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 msgr2=0x7fa5b40ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5bbe23640 1 --2- 172.21.15.80:0/3933620311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40ceb40 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7fa5a4009870 tx=0x7fa5a401ed50 comp rx=0 tx=0).stop 2024-04-25T13:46:29.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5bbe23640 1 -- 172.21.15.80:0/3933620311 shutdown_connections 2024-04-25T13:46:29.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5bbe23640 1 --2- 172.21.15.80:0/3933620311 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40ceb40 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5bbe23640 1 -- 172.21.15.80:0/3933620311 >> 172.21.15.80:0/3933620311 conn(0x7fa5b406d080 msgr2=0x7fa5b40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:29.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5bbe23640 1 -- 172.21.15.80:0/3933620311 shutdown_connections 2024-04-25T13:46:29.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.527+0000 7fa5bbe23640 1 -- 172.21.15.80:0/3933620311 wait complete. 2024-04-25T13:46:29.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 Processor -- start 2024-04-25T13:46:29.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 -- start start 2024-04-25T13:46:29.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b406e490 0x7fa5b414ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b40b0ed0 0x7fa5b40af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b406dc00 con 0x7fa5b40ce730 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa5b40733a0 con 0x7fa5b40b0ed0 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa5b406dd80 con 0x7fa5b406e490 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57530/0 (socket says 172.21.15.80:57530) 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 -- 172.21.15.80:0/374571115 learned_addr learned my addr 172.21.15.80:0/374571115 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5b9bbf640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b406e490 0x7fa5b414ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5b93be640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b40b0ed0 0x7fa5b40af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 -- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b406e490 msgr2=0x7fa5b414ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b406e490 0x7fa5b414ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 -- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b40b0ed0 msgr2=0x7fa5b40af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b40b0ed0 0x7fa5b40af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 -- 172.21.15.80:0/374571115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5b41fac70 con 0x7fa5b40ce730 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5b9bbf640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b406e490 0x7fa5b414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5b93be640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b40b0ed0 0x7fa5b40af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5ba3c0640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40afdf0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00c6d0 tx=0x7fa5ac00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5aa7fc640 1 -- 172.21.15.80:0/374571115 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac018070 con 0x7fa5b40ce730 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5aa7fc640 1 -- 172.21.15.80:0/374571115 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa5ac004d70 con 0x7fa5b40ce730 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5b41faf50 con 0x7fa5b40ce730 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5aa7fc640 1 -- 172.21.15.80:0/374571115 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00a880 con 0x7fa5b40ce730 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.531+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5b41fb510 con 0x7fa5b40ce730 2024-04-25T13:46:29.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.535+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5b406a3a0 con 0x7fa5b40ce730 2024-04-25T13:46:29.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.535+0000 7fa5aa7fc640 1 -- 172.21.15.80:0/374571115 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa5ac01b070 con 0x7fa5b40ce730 2024-04-25T13:46:29.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.535+0000 7fa5aa7fc640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa59806f400 0x7fa598071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.535+0000 7fa5aa7fc640 1 -- 172.21.15.80:0/374571115 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5709+0+0 (secure 0 0 0) 0x7fa5ac08ec90 con 0x7fa5b40ce730 2024-04-25T13:46:29.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.535+0000 7fa5b9bbf640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa59806f400 0x7fa598071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.535+0000 7fa5b9bbf640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa59806f400 0x7fa598071910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0000f70 tx=0x7fa5b0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:29.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.539+0000 7fa5aa7fc640 1 -- 172.21.15.80:0/374571115 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa5ac05bcf0 con 0x7fa5b40ce730 2024-04-25T13:46:29.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.703+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa5b41faa00 con 0x7fa5b40ce730 2024-04-25T13:46:29.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.707+0000 7fa5aa7fc640 1 -- 172.21.15.80:0/374571115 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16561 (secure 0 0 0) 0x7fa5ac05b5f0 con 0x7fa5b40ce730 2024-04-25T13:46:29.706 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:29.706 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":121,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:28.718476+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.707+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa59806f400 msgr2=0x7fa598071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.707+0000 7fa5bbe23640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa59806f400 0x7fa598071910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa5b0000f70 tx=0x7fa5b0009040 comp rx=0 tx=0).stop 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.707+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 msgr2=0x7fa5b40afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.707+0000 7fa5bbe23640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40afdf0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00c6d0 tx=0x7fa5ac00cb90 comp rx=0 tx=0).stop 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.707+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 shutdown_connections 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.711+0000 7fa5bbe23640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa59806f400 0x7fa598071910 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.711+0000 7fa5bbe23640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa5b40ce730 0x7fa5b40afdf0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.711+0000 7fa5bbe23640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa5b40b0ed0 0x7fa5b40af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.711+0000 7fa5bbe23640 1 --2- 172.21.15.80:0/374571115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa5b406e490 0x7fa5b414ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.711+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 >> 172.21.15.80:0/374571115 conn(0x7fa5b406d080 msgr2=0x7fa5b4158240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:29.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.711+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 shutdown_connections 2024-04-25T13:46:29.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.711+0000 7fa5bbe23640 1 -- 172.21.15.80:0/374571115 wait complete. 2024-04-25T13:46:29.729 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:29.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.875+0000 7f7eff7a2640 1 Processor -- start 2024-04-25T13:46:29.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.875+0000 7f7eff7a2640 1 -- start start 2024-04-25T13:46:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eff7a2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7ef806ebb0 0x7f7ef806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eff7a2640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef80d0af0 con 0x7f7ef8161f90 2024-04-25T13:46:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eff7a2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ef80d0d70 con 0x7f7ef80d06e0 2024-04-25T13:46:29.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eff7a2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7ef80d1050 con 0x7f7ef806ebb0 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efd53e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7ef8161f90 0x7f7ef80cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38412/0 (socket says 172.21.15.80:38412) 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efd53e640 1 -- 172.21.15.80:0/2088131895 learned_addr learned my addr 172.21.15.80:0/2088131895 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 398063203 0 0) 0x7f7ef4001120 con 0x7f7ef8161f90 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efdd3f640 1 --2- 172.21.15.80:0/2088131895 >> v2:172.21.15.87:3300/0 conn(0x7f7ef806ebb0 0x7f7ef806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7ee4003530 con 0x7f7ef8161f90 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3141662837 0 0) 0x7f7ee4003530 con 0x7f7ef8161f90 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 >> v2:172.21.15.87:3300/0 conn(0x7f7ef806ebb0 msgr2=0x7f7ef806efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 --2- 172.21.15.80:0/2088131895 >> v2:172.21.15.87:3300/0 conn(0x7f7ef806ebb0 0x7f7ef806efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 >> v1:172.21.15.87:6789/0 conn(0x7f7ef80d06e0 legacy=0x7f7ef806e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef806dca0 con 0x7f7ef8161f90 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7ef4001850 con 0x7f7ef8161f90 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 >> v1:172.21.15.80:6789/0 conn(0x7f7ef8161f90 legacy=0x7f7ef80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee40039c0 0x7f7ee4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee4005360 0x7f7ee400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef80d1050 con 0x7f7ee40039c0 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7ee400c290 con 0x7f7ee4004810 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7ee400c490 con 0x7f7ee4005360 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efd53e640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee40039c0 0x7f7ee4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efcd3d640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efdd3f640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee4005360 0x7f7ee400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efd53e640 1 -- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee4005360 msgr2=0x7f7ee400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efd53e640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee4005360 0x7f7ee400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efd53e640 1 -- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee4004810 msgr2=0x7f7ee4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efd53e640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee4004810 0x7f7ee4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efd53e640 1 -- 172.21.15.80:0/2088131895 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ee4003530 con 0x7f7ee40039c0 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efdd3f640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ee4005360 0x7f7ee400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:29.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.879+0000 7f7efdd3f640 1 -- 172.21.15.80:0/2088131895 reap_dead start 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7efd53e640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee40039c0 0x7f7ee4003e30 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8147e20 tx=0x7f7ef401cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ed58c677094502b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef400c0e0 con 0x7f7ee40039c0 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ef40a9020 con 0x7f7ee40039c0 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eeffff640 1 -- 172.21.15.80:0/2088131895 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ef4005b00 con 0x7f7ee40039c0 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2088131895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee40039c0 msgr2=0x7f7ee4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eff7a2640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee40039c0 0x7f7ee4003e30 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f7ef8147e20 tx=0x7f7ef401cb10 comp rx=0 tx=0).stop 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2088131895 shutdown_connections 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eff7a2640 1 --2- 172.21.15.80:0/2088131895 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ee40039c0 0x7f7ee4003e30 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2088131895 >> 172.21.15.80:0/2088131895 conn(0x7f7ef806d080 msgr2=0x7f7ef80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2088131895 shutdown_connections 2024-04-25T13:46:29.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.883+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2088131895 wait complete. 2024-04-25T13:46:29.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 Processor -- start 2024-04-25T13:46:29.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 -- start start 2024-04-25T13:46:29.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee40039c0 0x7f7ef8158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ef815af10 0x7f7ef80af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ef815b320 0x7f7ef80afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef806ea30 con 0x7f7ef815af10 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef806e8b0 con 0x7f7ee40039c0 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7ef80d03a0 con 0x7f7ef815b320 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ef815af10 0x7f7ef80af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ef815af10 0x7f7ef80af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57542/0 (socket says 172.21.15.80:57542) 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 -- 172.21.15.80:0/2186058977 learned_addr learned my addr 172.21.15.80:0/2186058977 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efd53e640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee40039c0 0x7f7ef8158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efdd3f640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ef815b320 0x7f7ef80afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 -- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ef815b320 msgr2=0x7f7ef80afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ef815b320 0x7f7ef80afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 -- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee40039c0 msgr2=0x7f7ef8158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee40039c0 0x7f7ef8158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 -- 172.21.15.80:0/2186058977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef8158b50 con 0x7f7ef815af10 2024-04-25T13:46:29.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efd53e640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee40039c0 0x7f7ef8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7efcd3d640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ef815af10 0x7f7ef80af8b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e4e0 tx=0x7f7ee800e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eedffb640 1 -- 172.21.15.80:0/2186058977 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee80052e0 con 0x7f7ef815af10 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eedffb640 1 -- 172.21.15.80:0/2186058977 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ee8018070 con 0x7f7ef815af10 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef8155e70 con 0x7f7ef815af10 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eedffb640 1 -- 172.21.15.80:0/2186058977 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8005480 con 0x7f7ef815af10 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef8158e30 con 0x7f7ef815af10 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.887+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef806a3a0 con 0x7f7ef815af10 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.891+0000 7f7eedffb640 1 -- 172.21.15.80:0/2186058977 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7ee8003840 con 0x7f7ef815af10 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.891+0000 7f7eedffb640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7edc06f400 0x7f7edc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.891+0000 7f7eedffb640 1 -- 172.21.15.80:0/2186058977 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5709+0+0 (secure 0 0 0) 0x7f7ee808ee30 con 0x7f7ef815af10 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.891+0000 7f7efd53e640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7edc06f400 0x7f7edc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:29.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.891+0000 7f7efd53e640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7edc06f400 0x7f7edc071910 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40aa000 tx=0x7f7ef80d0f40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:29.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:29.895+0000 7f7eedffb640 1 -- 172.21.15.80:0/2186058977 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7ee805be40 con 0x7f7ef815af10 2024-04-25T13:46:30.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.051+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ef80b2cc0 con 0x7f7ef815af10 2024-04-25T13:46:30.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.051+0000 7f7eedffb640 1 -- 172.21.15.80:0/2186058977 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16561 (secure 0 0 0) 0x7f7ee805b740 con 0x7f7ef815af10 2024-04-25T13:46:30.050 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:30.050 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":121,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:28.718476+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:30.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7edc06f400 msgr2=0x7f7edc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:30.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7edc06f400 0x7f7edc071910 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40aa000 tx=0x7f7ef80d0f40 comp rx=0 tx=0).stop 2024-04-25T13:46:30.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ef815af10 msgr2=0x7f7ef80af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:30.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ef815af10 0x7f7ef80af8b0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f7ee800e4e0 tx=0x7f7ee800e9a0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 shutdown_connections 2024-04-25T13:46:30.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7edc06f400 0x7f7edc071910 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7ef815b320 0x7f7ef80afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7ef815af10 0x7f7ef80af8b0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 --2- 172.21.15.80:0/2186058977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7ee40039c0 0x7f7ef8158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 >> 172.21.15.80:0/2186058977 conn(0x7f7ef806d080 msgr2=0x7f7ef80b60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:30.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 shutdown_connections 2024-04-25T13:46:30.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.055+0000 7f7eff7a2640 1 -- 172.21.15.80:0/2186058977 wait complete. 2024-04-25T13:46:30.071 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:46:30.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.227+0000 7f173e3d3640 1 Processor -- start 2024-04-25T13:46:30.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.227+0000 7f173e3d3640 1 -- start start 2024-04-25T13:46:30.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f173e3d3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f17380b5330 0x7f17380d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f173e3d3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738161b60 0x7f1738161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f173e3d3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f17381624b0 con 0x7f17380d0ff0 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f173e3d3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f17381629b0 con 0x7f17380b5330 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f173e3d3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17380b5740 con 0x7f1738161b60 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f172ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f17380d0ff0 0x7f173806edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:38416/0 (socket says 172.21.15.80:38416) 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f172ffff640 1 -- 172.21.15.80:0/2566292894 learned_addr learned my addr 172.21.15.80:0/2566292894 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f173c970640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738161b60 0x7f1738161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f1737fff640 1 --2- 172.21.15.80:0/2566292894 >> v2:172.21.15.87:3300/0 conn(0x7f17380b5330 0x7f17380d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2834971875 0 0) 0x7f1720001120 con 0x7f17380d0ff0 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1724003530 con 0x7f17380d0ff0 2024-04-25T13:46:30.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3282273396 0 0) 0x7f1724003530 con 0x7f17380d0ff0 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738161b60 msgr2=0x7f1738161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738161b60 0x7f1738161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 >> v2:172.21.15.87:3300/0 conn(0x7f17380b5330 msgr2=0x7f17380d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 --2- 172.21.15.80:0/2566292894 >> v2:172.21.15.87:3300/0 conn(0x7f17380b5330 0x7f17380d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173806dd10 con 0x7f17380d0ff0 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1720001850 con 0x7f17380d0ff0 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 >> v1:172.21.15.80:6789/0 conn(0x7f17380d0ff0 legacy=0x7f173806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f1724003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738161b60 0x7f1724004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1724004d50 0x7f1724005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f1737fff640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f1724003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f17381629b0 con 0x7f17240039c0 2024-04-25T13:46:30.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f17380b5740 con 0x7f1724004d50 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f172400ba50 con 0x7f1738161b60 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f172ffff640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738161b60 0x7f1724004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f173c970640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1724004d50 0x7f1724005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f1737fff640 1 -- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738161b60 msgr2=0x7f1724004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f1737fff640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738161b60 0x7f1724004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f1737fff640 1 -- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1724004d50 msgr2=0x7f1724005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f1737fff640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1724004d50 0x7f1724005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f1737fff640 1 -- 172.21.15.80:0/2566292894 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1724003530 con 0x7f17240039c0 2024-04-25T13:46:30.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.231+0000 7f173c970640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1724004d50 0x7f1724005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:30.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f1737fff640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f1724003e30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1728009ae0 tx=0x7f172801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd764d404dbd25ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:30.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17280096b0 con 0x7f17240039c0 2024-04-25T13:46:30.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17280a6490 con 0x7f17240039c0 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f17377fe640 1 -- 172.21.15.80:0/2566292894 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17280a6880 con 0x7f17240039c0 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f173e3d3640 1 -- 172.21.15.80:0/2566292894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 msgr2=0x7f1724003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f173e3d3640 1 --2- 172.21.15.80:0/2566292894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f1724003e30 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f1728009ae0 tx=0x7f172801cab0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f173c970640 1 -- 172.21.15.80:0/2566292894 reap_dead start 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f173e3d3640 1 -- 172.21.15.80:0/2566292894 shutdown_connections 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f173e3d3640 1 -- 172.21.15.80:0/2566292894 >> 172.21.15.80:0/2566292894 conn(0x7f17380b4160 msgr2=0x7f17380d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f173e3d3640 1 -- 172.21.15.80:0/2566292894 shutdown_connections 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.235+0000 7f173e3d3640 1 -- 172.21.15.80:0/2566292894 wait complete. 2024-04-25T13:46:30.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 Processor -- start 2024-04-25T13:46:30.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 -- start start 2024-04-25T13:46:30.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f173813f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738141a60 0x7f173815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173815bab0 0x7f1738159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f17380b0cd0 con 0x7f17240039c0 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f173806e890 con 0x7f173815bab0 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f17380b0e50 con 0x7f1738141a60 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f173813f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f173813f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57548/0 (socket says 172.21.15.80:57548) 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 -- 172.21.15.80:0/2153034952 learned_addr learned my addr 172.21.15.80:0/2153034952 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f172ffff640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738141a60 0x7f173815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173c970640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173815bab0 0x7f1738159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 -- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738141a60 msgr2=0x7f173815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738141a60 0x7f173815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 -- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173815bab0 msgr2=0x7f1738159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173815bab0 0x7f1738159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:30.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 -- 172.21.15.80:0/2153034952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f173815bec0 con 0x7f17240039c0 2024-04-25T13:46:30.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f1737fff640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f173813f3d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1728009a00 tx=0x7f1724005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:30.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f17357fa640 1 -- 172.21.15.80:0/2153034952 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17280a6c60 con 0x7f17240039c0 2024-04-25T13:46:30.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f17357fa640 1 -- 172.21.15.80:0/2153034952 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f17280a6e40 con 0x7f17240039c0 2024-04-25T13:46:30.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17381469f0 con 0x7f17240039c0 2024-04-25T13:46:30.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f17357fa640 1 -- 172.21.15.80:0/2153034952 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17280a76b0 con 0x7f17240039c0 2024-04-25T13:46:30.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1738146430 con 0x7f17240039c0 2024-04-25T13:46:30.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.239+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f173806a3a0 con 0x7f17240039c0 2024-04-25T13:46:30.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.243+0000 7f17357fa640 1 -- 172.21.15.80:0/2153034952 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f17280a7850 con 0x7f17240039c0 2024-04-25T13:46:30.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.243+0000 7f17357fa640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f171806f430 0x7f1718071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:30.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.243+0000 7f17357fa640 1 -- 172.21.15.80:0/2153034952 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5709+0+0 (secure 0 0 0) 0x7f17281296f0 con 0x7f17240039c0 2024-04-25T13:46:30.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.243+0000 7f172ffff640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f171806f430 0x7f1718071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:30.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.243+0000 7f172ffff640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f171806f430 0x7f1718071940 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1720007fd0 tx=0x7f1720009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:30.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.247+0000 7f17357fa640 1 -- 172.21.15.80:0/2153034952 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f173815bec0 con 0x7f17240039c0 2024-04-25T13:46:30.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:30.447+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f17380b3330 con 0x7f17240039c0 2024-04-25T13:46:31.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.375+0000 7f17357fa640 1 -- 172.21.15.80:0/2153034952 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v58) ==== 118+0+0 (secure 0 0 0) 0x7f17280ac020 con 0x7f17240039c0 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f171806f430 msgr2=0x7f1718071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f171806f430 0x7f1718071940 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1720007fd0 tx=0x7f1720009040 comp rx=0 tx=0).stop 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 msgr2=0x7f173813f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f173813f3d0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f1728009a00 tx=0x7f1724005840 comp rx=0 tx=0).stop 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 shutdown_connections 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f171806f430 0x7f1718071940 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f173815bab0 0x7f1738159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1738141a60 0x7f173815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 --2- 172.21.15.80:0/2153034952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f17240039c0 0x7f173813f3d0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 >> 172.21.15.80:0/2153034952 conn(0x7f17380b4160 msgr2=0x7f1738147510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:31.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 shutdown_connections 2024-04-25T13:46:31.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.379+0000 7f173e3d3640 1 -- 172.21.15.80:0/2153034952 wait complete. 2024-04-25T13:46:31.400 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:46:31.400 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:46:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3c2a7640 1 Processor -- start 2024-04-25T13:46:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3c2a7640 1 -- start start 2024-04-25T13:46:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3c2a7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9e34071e60 0x7f9e341574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3c2a7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9e34073c40 0x7f9e34160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3c2a7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e341613b0 con 0x7f9e34160fa0 2024-04-25T13:46:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3c2a7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9e340b1350 con 0x7f9e34071e60 2024-04-25T13:46:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3c2a7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9e3406f9a0 con 0x7f9e34073c40 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3a844640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9e34160fa0 0x7f9e340b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52630/0 (socket says 172.21.15.80:52630) 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3a043640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9e34071e60 0x7f9e341574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3a844640 1 -- 172.21.15.80:0/2310303846 learned_addr learned my addr 172.21.15.80:0/2310303846 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e39842640 1 --2- 172.21.15.80:0/2310303846 >> v2:172.21.15.87:3300/0 conn(0x7f9e34073c40 0x7f9e34160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e39041640 1 -- 172.21.15.80:0/2310303846 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342004977 0 0) 0x7f9e2c0014a0 con 0x7f9e34160fa0 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e39041640 1 -- 172.21.15.80:0/2310303846 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e20003530 con 0x7f9e34160fa0 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3a043640 1 -- 172.21.15.80:0/2310303846 >> v2:172.21.15.87:3300/0 conn(0x7f9e34073c40 msgr2=0x7f9e34160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3a043640 1 --2- 172.21.15.80:0/2310303846 >> v2:172.21.15.87:3300/0 conn(0x7f9e34073c40 0x7f9e34160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3a043640 1 -- 172.21.15.80:0/2310303846 >> v1:172.21.15.80:6789/0 conn(0x7f9e34160fa0 legacy=0x7f9e340b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.555+0000 7f9e3a043640 1 -- 172.21.15.80:0/2310303846 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3406fbf0 con 0x7f9e34071e60 2024-04-25T13:46:31.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3a043640 1 --2- 172.21.15.80:0/2310303846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34071e60 0x7f9e341574c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9e340cecc0 tx=0x7f9e3001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940e30838eefb3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:31.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e39041640 1 -- 172.21.15.80:0/2310303846 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300b0070 con 0x7f9e34071e60 2024-04-25T13:46:31.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e39041640 1 -- 172.21.15.80:0/2310303846 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e300ab600 con 0x7f9e34071e60 2024-04-25T13:46:31.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e39041640 1 -- 172.21.15.80:0/2310303846 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e300ab9d0 con 0x7f9e34071e60 2024-04-25T13:46:31.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2310303846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34071e60 msgr2=0x7f9e341574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:31.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2310303846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34071e60 0x7f9e341574c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f9e340cecc0 tx=0x7f9e3001c480 comp rx=0 tx=0).stop 2024-04-25T13:46:31.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2310303846 shutdown_connections 2024-04-25T13:46:31.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2310303846 >> v2:172.21.15.87:3300/0 conn(0x7f9e34073c40 0x7f9e34160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2310303846 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34071e60 0x7f9e341574c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2310303846 >> 172.21.15.80:0/2310303846 conn(0x7f9e340b4160 msgr2=0x7f9e340d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2310303846 shutdown_connections 2024-04-25T13:46:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.559+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2310303846 wait complete. 2024-04-25T13:46:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 Processor -- start 2024-04-25T13:46:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 -- start start 2024-04-25T13:46:31.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e34071e60 0x7f9e34157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34073c40 0x7f9e341fa0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e34160fa0 0x7f9e341fa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9e34074050 con 0x7f9e34073c40 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9e34072270 con 0x7f9e34071e60 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9e340b56c0 con 0x7f9e34160fa0 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34073c40 0x7f9e341fa0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34073c40 0x7f9e341fa0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42934/0 (socket says 172.21.15.80:42934) 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 -- 172.21.15.80:0/2892187861 learned_addr learned my addr 172.21.15.80:0/2892187861 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 -- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e34160fa0 msgr2=0x7f9e341fa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3a844640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e34160fa0 0x7f9e341fa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3a043640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e34071e60 0x7f9e34157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e34160fa0 0x7f9e341fa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 -- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e34071e60 msgr2=0x7f9e34157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e34071e60 0x7f9e34157b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 -- 172.21.15.80:0/2892187861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e34155240 con 0x7f9e34073c40 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3a844640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e34160fa0 0x7f9e341fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e39842640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34073c40 0x7f9e341fa0c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400d620 tx=0x7f9e2400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e2affd640 1 -- 172.21.15.80:0/2892187861 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400b620 con 0x7f9e34073c40 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e2affd640 1 -- 172.21.15.80:0/2892187861 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e24019070 con 0x7f9e34073c40 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e34155520 con 0x7f9e34073c40 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e2affd640 1 -- 172.21.15.80:0/2892187861 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e2400b9e0 con 0x7f9e34073c40 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.563+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e34151430 con 0x7f9e34073c40 2024-04-25T13:46:31.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.567+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e341581a0 con 0x7f9e34073c40 2024-04-25T13:46:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.567+0000 7f9e2affd640 1 -- 172.21.15.80:0/2892187861 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e2400bb80 con 0x7f9e34073c40 2024-04-25T13:46:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.567+0000 7f9e2affd640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e1806f430 0x7f9e18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.567+0000 7f9e2affd640 1 -- 172.21.15.80:0/2892187861 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f9e2408f630 con 0x7f9e34073c40 2024-04-25T13:46:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.567+0000 7f9e3a043640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e1806f430 0x7f9e18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.567+0000 7f9e3a043640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e1806f430 0x7f9e18071940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e300ad000 tx=0x7f9e30008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:31.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.571+0000 7f9e2affd640 1 -- 172.21.15.80:0/2892187861 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9e2405c5c0 con 0x7f9e34073c40 2024-04-25T13:46:31.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:31.767+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f9e34156000 con 0x7f9e34073c40 2024-04-25T13:46:32.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.383+0000 7f9e2affd640 1 -- 172.21.15.80:0/2892187861 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v59) ==== 122+0+0 (secure 0 0 0) 0x7f9e2405bec0 con 0x7f9e34073c40 2024-04-25T13:46:32.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e1806f430 msgr2=0x7f9e18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e1806f430 0x7f9e18071940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f9e300ad000 tx=0x7f9e30008040 comp rx=0 tx=0).stop 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34073c40 msgr2=0x7f9e341fa0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34073c40 0x7f9e341fa0c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f9e2400d620 tx=0x7f9e2400dae0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 shutdown_connections 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e1806f430 0x7f9e18071940 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e34160fa0 0x7f9e341fa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e34073c40 0x7f9e341fa0c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 --2- 172.21.15.80:0/2892187861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e34071e60 0x7f9e34157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 >> 172.21.15.80:0/2892187861 conn(0x7f9e340b4160 msgr2=0x7f9e3415eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 shutdown_connections 2024-04-25T13:46:32.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.387+0000 7f9e3c2a7640 1 -- 172.21.15.80:0/2892187861 wait complete. 2024-04-25T13:46:32.408 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:46:32.408 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:32.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef82c3640 1 Processor -- start 2024-04-25T13:46:32.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef82c3640 1 -- start start 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef82c3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faef00d0350 0x7faef00c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef82c3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7faef00cf780 0x7faef01689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef82c3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faef0164b90 con 0x7faef00c6fe0 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef82c3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7faef0074cd0 con 0x7faef00d0350 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef82c3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7faef00d1360 con 0x7faef00cf780 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef605f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faef00d0350 0x7faef00c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef585e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7faef00c6fe0 0x7faef01682e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52632/0 (socket says 172.21.15.80:52632) 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.559+0000 7faef585e640 1 -- 172.21.15.80:0/693686894 learned_addr learned my addr 172.21.15.80:0/693686894 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef6860640 1 --2- 172.21.15.80:0/693686894 >> v2:172.21.15.87:3300/0 conn(0x7faef00cf780 0x7faef01689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef605f640 1 -- 172.21.15.80:0/693686894 >> v2:172.21.15.87:3300/0 conn(0x7faef00cf780 msgr2=0x7faef01689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:32.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef605f640 1 --2- 172.21.15.80:0/693686894 >> v2:172.21.15.87:3300/0 conn(0x7faef00cf780 0x7faef01689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef505d640 1 -- 172.21.15.80:0/693686894 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2195573276 0 0) 0x7faef0164b90 con 0x7faef00c6fe0 2024-04-25T13:46:32.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef605f640 1 -- 172.21.15.80:0/693686894 >> v1:172.21.15.80:6789/0 conn(0x7faef00c6fe0 legacy=0x7faef01682e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef605f640 1 -- 172.21.15.80:0/693686894 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef00c73f0 con 0x7faef00d0350 2024-04-25T13:46:32.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef6860640 1 --2- 172.21.15.80:0/693686894 >> v2:172.21.15.87:3300/0 conn(0x7faef00cf780 0x7faef01689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:32.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef605f640 1 --2- 172.21.15.80:0/693686894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef00c6aa0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7faef00b2bd0 tx=0x7faeec01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefa117e8b3f8abd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:32.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef505d640 1 -- 172.21.15.80:0/693686894 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0b0070 con 0x7faef00d0350 2024-04-25T13:46:32.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef505d640 1 -- 172.21.15.80:0/693686894 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faeec0ab600 con 0x7faef00d0350 2024-04-25T13:46:32.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.563+0000 7faef505d640 1 -- 172.21.15.80:0/693686894 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeec0ab9d0 con 0x7faef00d0350 2024-04-25T13:46:32.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- 172.21.15.80:0/693686894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 msgr2=0x7faef00c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 --2- 172.21.15.80:0/693686894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef00c6aa0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7faef00b2bd0 tx=0x7faeec01c3d0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- 172.21.15.80:0/693686894 shutdown_connections 2024-04-25T13:46:32.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 --2- 172.21.15.80:0/693686894 >> v2:172.21.15.87:3300/0 conn(0x7faef00cf780 0x7faef01689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 --2- 172.21.15.80:0/693686894 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef00c6aa0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- 172.21.15.80:0/693686894 >> 172.21.15.80:0/693686894 conn(0x7faef00d3ad0 msgr2=0x7faef00d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:32.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- 172.21.15.80:0/693686894 shutdown_connections 2024-04-25T13:46:32.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- 172.21.15.80:0/693686894 wait complete. 2024-04-25T13:46:32.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 Processor -- start 2024-04-25T13:46:32.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- start start 2024-04-25T13:46:32.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faef00c6fe0 0x7faef015d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faef00cf780 0x7faef015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef0162d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faef00d0760 con 0x7faef00d0350 2024-04-25T13:46:32.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faef00ce250 con 0x7faef00c6fe0 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef82c3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faef0168f30 con 0x7faef00cf780 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef6860640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef0162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef6860640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef0162d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42966/0 (socket says 172.21.15.80:42966) 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef6860640 1 -- 172.21.15.80:0/569760168 learned_addr learned my addr 172.21.15.80:0/569760168 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef605f640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faef00c6fe0 0x7faef015d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.567+0000 7faef585e640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faef00cf780 0x7faef015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef6860640 1 -- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faef00cf780 msgr2=0x7faef015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef6860640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faef00cf780 0x7faef015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef6860640 1 -- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faef00c6fe0 msgr2=0x7faef015d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef6860640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faef00c6fe0 0x7faef015d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef6860640 1 -- 172.21.15.80:0/569760168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef0161f70 con 0x7faef00d0350 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef6860640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef0162d30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7faee800cd80 tx=0x7faee800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faee6ffd640 1 -- 172.21.15.80:0/569760168 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee800f680 con 0x7faef00d0350 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faee6ffd640 1 -- 172.21.15.80:0/569760168 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faee800f840 con 0x7faef00d0350 2024-04-25T13:46:32.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef0162250 con 0x7faef00d0350 2024-04-25T13:46:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faee6ffd640 1 -- 172.21.15.80:0/569760168 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faee800fc30 con 0x7faef00d0350 2024-04-25T13:46:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef00b3660 con 0x7faef00d0350 2024-04-25T13:46:32.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef00b5ff0 con 0x7faef00d0350 2024-04-25T13:46:32.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faee6ffd640 1 -- 172.21.15.80:0/569760168 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faee80213f0 con 0x7faef00d0350 2024-04-25T13:46:32.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faee6ffd640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faed406f400 0x7faed4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faee6ffd640 1 -- 172.21.15.80:0/569760168 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7faee8090300 con 0x7faef00d0350 2024-04-25T13:46:32.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef605f640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faed406f400 0x7faed4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.571+0000 7faef605f640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faed406f400 0x7faed4071910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faeec0ad000 tx=0x7faeec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:32.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.575+0000 7faee6ffd640 1 -- 172.21.15.80:0/569760168 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faee8014030 con 0x7faef00d0350 2024-04-25T13:46:32.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faef00b38c0 con 0x7faef00d0350 2024-04-25T13:46:32.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faee6ffd640 1 -- 172.21.15.80:0/569760168 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5174 (secure 0 0 0) 0x7faee805d360 con 0x7faef00d0350 2024-04-25T13:46:32.765 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:32.765 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:46:28.718395+0000","modified":"2024-04-25T13:46:31.773079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/2773338889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2773338889},{"type":"v1","addr":"172.21.15.80:6835","nonce":2773338889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":7}]} 2024-04-25T13:46:32.765 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 59 2024-04-25T13:46:32.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faed406f400 msgr2=0x7faed4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faed406f400 0x7faed4071910 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7faeec0ad000 tx=0x7faeec008040 comp rx=0 tx=0).stop 2024-04-25T13:46:32.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 msgr2=0x7faef0162d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef0162d30 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7faee800cd80 tx=0x7faee800a650 comp rx=0 tx=0).stop 2024-04-25T13:46:32.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 shutdown_connections 2024-04-25T13:46:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faed406f400 0x7faed4071910 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faef00d0350 0x7faef0162d30 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faef00cf780 0x7faef015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 --2- 172.21.15.80:0/569760168 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faef00c6fe0 0x7faef015d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 >> 172.21.15.80:0/569760168 conn(0x7faef00d3ad0 msgr2=0x7faef00700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 shutdown_connections 2024-04-25T13:46:32.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.767+0000 7faef82c3640 1 -- 172.21.15.80:0/569760168 wait complete. 2024-04-25T13:46:32.786 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.927+0000 7f4b011e3640 1 Processor -- start 2024-04-25T13:46:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.927+0000 7f4b011e3640 1 -- start start 2024-04-25T13:46:32.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4b011e3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4afc161f90 0x7f4afc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4b011e3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4afc15ec40 0x7f4afc15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4b011e3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4afc0d0d70 con 0x7f4afc0d0510 2024-04-25T13:46:32.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4b011e3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4afc0d1050 con 0x7f4afc161f90 2024-04-25T13:46:32.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4b011e3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4afc06dca0 con 0x7f4afc15ec40 2024-04-25T13:46:32.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4afad9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4afc161f90 0x7f4afc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4afad9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4afc161f90 0x7f4afc0cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42970/0 (socket says 172.21.15.80:42970) 2024-04-25T13:46:32.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4afad9d640 1 -- 172.21.15.80:0/573635872 learned_addr learned my addr 172.21.15.80:0/573635872 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:32.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4afb59e640 1 --2- 172.21.15.80:0/573635872 >> v2:172.21.15.87:3300/0 conn(0x7f4afc15ec40 0x7f4afc15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4afad9d640 1 -- 172.21.15.80:0/573635872 >> v2:172.21.15.87:3300/0 conn(0x7f4afc15ec40 msgr2=0x7f4afc15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:32.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4afad9d640 1 --2- 172.21.15.80:0/573635872 >> v2:172.21.15.87:3300/0 conn(0x7f4afc15ec40 0x7f4afc15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4afad9d640 1 -- 172.21.15.80:0/573635872 >> v1:172.21.15.87:6789/0 conn(0x7f4afc0d0510 legacy=0x7f4afc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.931+0000 7f4afad9d640 1 -- 172.21.15.80:0/573635872 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc06df80 con 0x7f4afc161f90 2024-04-25T13:46:32.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4afad9d640 1 --2- 172.21.15.80:0/573635872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc161f90 0x7f4afc0cffd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0b4b80 tx=0x7f4aec01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c428f1bfa797334 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:32.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4afa59c640 1 -- 172.21.15.80:0/573635872 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0b2070 con 0x7f4afc161f90 2024-04-25T13:46:32.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4afa59c640 1 -- 172.21.15.80:0/573635872 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aec0a9070 con 0x7f4afc161f90 2024-04-25T13:46:32.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4afa59c640 1 -- 172.21.15.80:0/573635872 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0ad880 con 0x7f4afc161f90 2024-04-25T13:46:32.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 -- 172.21.15.80:0/573635872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc161f90 msgr2=0x7f4afc0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 --2- 172.21.15.80:0/573635872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc161f90 0x7f4afc0cffd0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0b4b80 tx=0x7f4aec01c430 comp rx=0 tx=0).stop 2024-04-25T13:46:32.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 -- 172.21.15.80:0/573635872 shutdown_connections 2024-04-25T13:46:32.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 --2- 172.21.15.80:0/573635872 >> v2:172.21.15.87:3300/0 conn(0x7f4afc15ec40 0x7f4afc15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 --2- 172.21.15.80:0/573635872 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc161f90 0x7f4afc0cffd0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 -- 172.21.15.80:0/573635872 >> 172.21.15.80:0/573635872 conn(0x7f4afc06d080 msgr2=0x7f4afc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:32.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 -- 172.21.15.80:0/573635872 shutdown_connections 2024-04-25T13:46:32.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 -- 172.21.15.80:0/573635872 wait complete. 2024-04-25T13:46:32.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 Processor -- start 2024-04-25T13:46:32.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.935+0000 7f4b011e3640 1 -- start start 2024-04-25T13:46:32.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc0d0510 0x7f4afc162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4afc15ec40 0x7f4afc0739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4afc161f90 0x7f4afc1411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc06f4b0 con 0x7f4afc0d0510 2024-04-25T13:46:32.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc06f630 con 0x7f4afc161f90 2024-04-25T13:46:32.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4afc0d0920 con 0x7f4afc15ec40 2024-04-25T13:46:32.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc0d0510 0x7f4afc162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc0d0510 0x7f4afc162ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42976/0 (socket says 172.21.15.80:42976) 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 -- 172.21.15.80:0/2281314085 learned_addr learned my addr 172.21.15.80:0/2281314085 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4af3fff640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4afc15ec40 0x7f4afc0739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afb59e640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4afc161f90 0x7f4afc1411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 -- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4afc15ec40 msgr2=0x7f4afc0739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4afc15ec40 0x7f4afc0739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 -- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4afc161f90 msgr2=0x7f4afc1411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4afc161f90 0x7f4afc1411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 -- 172.21.15.80:0/2281314085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc071f20 con 0x7f4afc0d0510 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afb59e640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4afc161f90 0x7f4afc1411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4afad9d640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc0d0510 0x7f4afc162ca0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0b0040 tx=0x7f4aec019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4af37fe640 1 -- 172.21.15.80:0/2281314085 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0bb070 con 0x7f4afc0d0510 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4af37fe640 1 -- 172.21.15.80:0/2281314085 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4aec0addf0 con 0x7f4afc0d0510 2024-04-25T13:46:32.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4afc072200 con 0x7f4afc0d0510 2024-04-25T13:46:32.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4af37fe640 1 -- 172.21.15.80:0/2281314085 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0b2040 con 0x7f4afc0d0510 2024-04-25T13:46:32.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4afc141710 con 0x7f4afc0d0510 2024-04-25T13:46:32.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4afc06a3a0 con 0x7f4afc0d0510 2024-04-25T13:46:32.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4af37fe640 1 -- 172.21.15.80:0/2281314085 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4aec0be070 con 0x7f4afc0d0510 2024-04-25T13:46:32.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.939+0000 7f4af37fe640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4adc06f430 0x7f4adc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:32.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.943+0000 7f4af37fe640 1 -- 172.21.15.80:0/2281314085 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f4aec1282f0 con 0x7f4afc0d0510 2024-04-25T13:46:32.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.943+0000 7f4af3fff640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4adc06f430 0x7f4adc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:32.940 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.943+0000 7f4af3fff640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4adc06f430 0x7f4adc071940 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0b20d0 tx=0x7f4ae4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:32.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:32.947+0000 7f4af37fe640 1 -- 172.21.15.80:0/2281314085 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4aec016110 con 0x7f4afc0d0510 2024-04-25T13:46:33.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.107+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4afc0769e0 con 0x7f4afc0d0510 2024-04-25T13:46:33.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.107+0000 7f4af37fe640 1 -- 172.21.15.80:0/2281314085 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) ==== 74+0+16548 (secure 0 0 0) 0x7f4aec0f5270 con 0x7f4afc0d0510 2024-04-25T13:46:33.106 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:33.107 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":122,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:30.336412+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:33.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4adc06f430 msgr2=0x7f4adc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4adc06f430 0x7f4adc071940 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0b20d0 tx=0x7f4ae4009040 comp rx=0 tx=0).stop 2024-04-25T13:46:33.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc0d0510 msgr2=0x7f4afc162ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc0d0510 0x7f4afc162ca0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0b0040 tx=0x7f4aec019230 comp rx=0 tx=0).stop 2024-04-25T13:46:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 shutdown_connections 2024-04-25T13:46:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4adc06f430 0x7f4adc071940 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4afc161f90 0x7f4afc1411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4afc15ec40 0x7f4afc0739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 --2- 172.21.15.80:0/2281314085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4afc0d0510 0x7f4afc162ca0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 >> 172.21.15.80:0/2281314085 conn(0x7f4afc06d080 msgr2=0x7f4afc0b5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 shutdown_connections 2024-04-25T13:46:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.111+0000 7f4b011e3640 1 -- 172.21.15.80:0/2281314085 wait complete. 2024-04-25T13:46:33.129 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:46:33.129 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:46:33.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.267+0000 7f2a02b57640 1 Processor -- start 2024-04-25T13:46:33.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a02b57640 1 -- start start 2024-04-25T13:46:33.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a02b57640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f29fc06f5a0 0x7f29fc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a02b57640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 0x7f29fc161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a02b57640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29fc06fcf0 con 0x7f29fc161070 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a02b57640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f29fc1619c0 con 0x7f29fc06f5a0 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a02b57640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f29fc073f10 con 0x7f29fc0733b0 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fbfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f29fc161070 0x7f29fc0b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52638/0 (socket says 172.21.15.80:52638) 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fbfff640 1 -- 172.21.15.80:0/2533864256 learned_addr learned my addr 172.21.15.80:0/2533864256 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a010f4640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 0x7f29fc161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a008f3640 1 --2- 172.21.15.80:0/2533864256 >> v2:172.21.15.87:3300/0 conn(0x7f29fc06f5a0 0x7f29fc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 704545786 0 0) 0x7f29fc06fcf0 con 0x7f29fc161070 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29e8003530 con 0x7f29fc161070 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 737847712 0 0) 0x7f29e8003530 con 0x7f29fc161070 2024-04-25T13:46:33.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 msgr2=0x7f29fc161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 0x7f29fc161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 >> v2:172.21.15.87:3300/0 conn(0x7f29fc06f5a0 msgr2=0x7f29fc160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 --2- 172.21.15.80:0/2533864256 >> v2:172.21.15.87:3300/0 conn(0x7f29fc06f5a0 0x7f29fc160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc0b0c60 con 0x7f29fc161070 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f29e4001680 con 0x7f29fc161070 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 >> v1:172.21.15.80:6789/0 conn(0x7f29fc161070 legacy=0x7f29fc0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29e80039f0 0x7f29e8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e8004810 0x7f29e8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 0x7f29e8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1619c0 con 0x7f29e80039f0 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc073f10 con 0x7f29e8004810 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f29e800bab0 con 0x7f29fc0733b0 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a008f3640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29e80039f0 0x7f29e8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fbfff640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e8004810 0x7f29e8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a010f4640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 0x7f29e8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a008f3640 1 -- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 msgr2=0x7f29e8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a008f3640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 0x7f29e8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a008f3640 1 -- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e8004810 msgr2=0x7f29e8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:33.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a008f3640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e8004810 0x7f29e8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a008f3640 1 -- 172.21.15.80:0/2533864256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e8003530 con 0x7f29e80039f0 2024-04-25T13:46:33.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f2a010f4640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc0733b0 0x7f29e8005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:33.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.271+0000 7f29fbfff640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e8004810 0x7f29e8004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:33.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f2a008f3640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29e80039f0 0x7f29e8003e60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0b6e70 tx=0x7f29ec01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce45dc7d600958da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:33.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0096b0 con 0x7f29e80039f0 2024-04-25T13:46:33.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29ec0a6410 con 0x7f29e80039f0 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f29fb7fe640 1 -- 172.21.15.80:0/2533864256 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec0a67c0 con 0x7f29e80039f0 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f2a02b57640 1 -- 172.21.15.80:0/2533864256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29e80039f0 msgr2=0x7f29e8003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f2a02b57640 1 --2- 172.21.15.80:0/2533864256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29e80039f0 0x7f29e8003e60 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0b6e70 tx=0x7f29ec01ca30 comp rx=0 tx=0).stop 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f2a010f4640 1 -- 172.21.15.80:0/2533864256 reap_dead start 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f2a02b57640 1 -- 172.21.15.80:0/2533864256 shutdown_connections 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f2a02b57640 1 -- 172.21.15.80:0/2533864256 >> 172.21.15.80:0/2533864256 conn(0x7f29fc06d080 msgr2=0x7f29fc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f2a02b57640 1 -- 172.21.15.80:0/2533864256 shutdown_connections 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.275+0000 7f2a02b57640 1 -- 172.21.15.80:0/2533864256 wait complete. 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 Processor -- start 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 -- start start 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e80039f0 0x7f29fc165970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc153f50 0x7f29fc158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc0cef70 0x7f29fc0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc168040 con 0x7f29fc0cef70 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc167ec0 con 0x7f29e80039f0 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f29fc1681c0 con 0x7f29fc153f50 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc0cef70 0x7f29fc0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc0cef70 0x7f29fc0cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42994/0 (socket says 172.21.15.80:42994) 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 -- 172.21.15.80:0/966277670 learned_addr learned my addr 172.21.15.80:0/966277670 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f29fbfff640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc153f50 0x7f29fc158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a008f3640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e80039f0 0x7f29fc165970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 -- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc153f50 msgr2=0x7f29fc158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc153f50 0x7f29fc158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 -- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e80039f0 msgr2=0x7f29fc165970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e80039f0 0x7f29fc165970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 -- 172.21.15.80:0/966277670 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc0cf8c0 con 0x7f29fc0cef70 2024-04-25T13:46:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a010f4640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc0cef70 0x7f29fc0cf380 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0701f0 tx=0x7f29f400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:33.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f29f97fa640 1 -- 172.21.15.80:0/966277670 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f401c070 con 0x7f29fc0cef70 2024-04-25T13:46:33.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f29f97fa640 1 -- 172.21.15.80:0/966277670 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29f4017400 con 0x7f29fc0cef70 2024-04-25T13:46:33.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc162a70 con 0x7f29fc0cef70 2024-04-25T13:46:33.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f29f97fa640 1 -- 172.21.15.80:0/966277670 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f40177f0 con 0x7f29fc0cef70 2024-04-25T13:46:33.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc165170 con 0x7f29fc0cef70 2024-04-25T13:46:33.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29fc06a3a0 con 0x7f29fc0cef70 2024-04-25T13:46:33.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.279+0000 7f29f97fa640 1 -- 172.21.15.80:0/966277670 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f29f401f070 con 0x7f29fc0cef70 2024-04-25T13:46:33.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.283+0000 7f29f97fa640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f29dc06f430 0x7f29dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.283+0000 7f29f97fa640 1 -- 172.21.15.80:0/966277670 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f29f408f450 con 0x7f29fc0cef70 2024-04-25T13:46:33.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.283+0000 7f2a008f3640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f29dc06f430 0x7f29dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.283+0000 7f2a008f3640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f29dc06f430 0x7f29dc071940 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0ad000 tx=0x7f29ec0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:33.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.287+0000 7f29f97fa640 1 -- 172.21.15.80:0/966277670 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29f4012070 con 0x7f29fc0cef70 2024-04-25T13:46:33.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.443+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29fc0739c0 con 0x7f29dc06f430 2024-04-25T13:46:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.463+0000 7f29f97fa640 1 -- 172.21.15.80:0/966277670 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237596 (secure 0 0 0) 0x7f29fc0739c0 con 0x7f29dc06f430 2024-04-25T13:46:33.464 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:33.465 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:46:33.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f29dc06f430 msgr2=0x7f29dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f29dc06f430 0x7f29dc071940 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0ad000 tx=0x7f29ec0046d0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc0cef70 msgr2=0x7f29fc0cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc0cef70 0x7f29fc0cf380 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0701f0 tx=0x7f29f400f4a0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 shutdown_connections 2024-04-25T13:46:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f29dc06f430 0x7f29dc071940 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f29fc0cef70 0x7f29fc0cf380 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29fc153f50 0x7f29fc158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 --2- 172.21.15.80:0/966277670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29e80039f0 0x7f29fc165970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 >> 172.21.15.80:0/966277670 conn(0x7f29fc06d080 msgr2=0x7f29fc06e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 shutdown_connections 2024-04-25T13:46:33.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.467+0000 7f2a02b57640 1 -- 172.21.15.80:0/966277670 wait complete. 2024-04-25T13:46:33.483 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-25T13:46:32.228909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6244,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556255,"statfs":{"total":1600000000,"available":1593605780,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.908101"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335260+0000","last_change":"2024-04-25T13:46:27.335260+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335260+0000","last_undegraded":"2024-04-25T13:46:27.335260+0000","last_fullsized":"2024-04-25T13:46:27.335260+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327390+0000","last_change":"2024-04-25T13:46:27.327390+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327390+0000","last_undegraded":"2024-04-25T13:46:27.327390+0000","last_fullsized":"2024-04-25T13:46:27.327390+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335626+0000","last_change":"2024-04-25T13:46:27.335626+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335626+0000","last_undegraded":"2024-04-25T13:46:27.335626+0000","last_fullsized":"2024-04-25T13:46:27.335626+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335478+0000","last_change":"2024-04-25T13:46:27.335478+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335478+0000","last_undegraded":"2024-04-25T13:46:27.335478+0000","last_fullsized":"2024-04-25T13:46:27.335478+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328593+0000","last_change":"2024-04-25T13:46:27.328593+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328593+0000","last_undegraded":"2024-04-25T13:46:27.328593+0000","last_fullsized":"2024-04-25T13:46:27.328593+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.334338+0000","last_change":"2024-04-25T13:46:27.334338+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.334338+0000","last_undegraded":"2024-04-25T13:46:27.334338+0000","last_fullsized":"2024-04-25T13:46:27.334338+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322542+0000","last_change":"2024-04-25T13:46:27.322542+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322542+0000","last_undegraded":"2024-04-25T13:46:27.322542+0000","last_fullsized":"2024-04-25T13:46:27.322542+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328199+0000","last_change":"2024-04-25T13:46:27.328199+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328199+0000","last_undegraded":"2024-04-25T13:46:27.328199+0000","last_fullsized":"2024-04-25T13:46:27.328199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.331673+0000","last_change":"2024-04-25T13:46:27.331673+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.331673+0000","last_undegraded":"2024-04-25T13:46:27.331673+0000","last_fullsized":"2024-04-25T13:46:27.331673+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327624+0000","last_change":"2024-04-25T13:46:27.327624+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327624+0000","last_undegraded":"2024-04-25T13:46:27.327624+0000","last_fullsized":"2024-04-25T13:46:27.327624+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.326248+0000","last_change":"2024-04-25T13:46:27.326248+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.326248+0000","last_undegraded":"2024-04-25T13:46:27.326248+0000","last_fullsized":"2024-04-25T13:46:27.326248+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327363+0000","last_change":"2024-04-25T13:46:27.327363+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327363+0000","last_undegraded":"2024-04-25T13:46:27.327363+0000","last_fullsized":"2024-04-25T13:46:27.327363+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.331684+0000","last_change":"2024-04-25T13:46:27.331684+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.331684+0000","last_undegraded":"2024-04-25T13:46:27.331684+0000","last_fullsized":"2024-04-25T13:46:27.331684+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.330622+0000","last_change":"2024-04-25T13:46:27.330622+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.330622+0000","last_undegraded":"2024-04-25T13:46:27.330622+0000","last_fullsized":"2024-04-25T13:46:27.330622+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328453+0000","last_change":"2024-04-25T13:46:27.328453+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328453+0000","last_undegraded":"2024-04-25T13:46:27.328453+0000","last_fullsized":"2024-04-25T13:46:27.328453+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.329692+0000","last_change":"2024-04-25T13:46:27.329692+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.329692+0000","last_undegraded":"2024-04-25T13:46:27.329692+0000","last_fullsized":"2024-04-25T13:46:27.329692+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327476+0000","last_change":"2024-04-25T13:46:27.327476+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327476+0000","last_undegraded":"2024-04-25T13:46:27.327476+0000","last_fullsized":"2024-04-25T13:46:27.327476+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322745+0000","last_change":"2024-04-25T13:46:27.322745+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322745+0000","last_undegraded":"2024-04-25T13:46:27.322745+0000","last_fullsized":"2024-04-25T13:46:27.322745+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.332762+0000","last_change":"2024-04-25T13:46:27.332762+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.332762+0000","last_undegraded":"2024-04-25T13:46:27.332762+0000","last_fullsized":"2024-04-25T13:46:27.332762+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.333810+0000","last_change":"2024-04-25T13:46:27.333810+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.333810+0000","last_undegraded":"2024-04-25T13:46:27.333810+0000","last_fullsized":"2024-04-25T13:46:27.333810+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.325007+0000","last_change":"2024-04-25T13:46:27.325007+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.325007+0000","last_undegraded":"2024-04-25T13:46:27.325007+0000","last_fullsized":"2024-04-25T13:46:27.325007+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.324891+0000","last_change":"2024-04-25T13:46:27.324891+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.324891+0000","last_undegraded":"2024-04-25T13:46:27.324891+0000","last_fullsized":"2024-04-25T13:46:27.324891+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.334766+0000","last_change":"2024-04-25T13:46:27.334766+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.334766+0000","last_undegraded":"2024-04-25T13:46:27.334766+0000","last_fullsized":"2024-04-25T13:46:27.334766+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.325007+0000","last_change":"2024-04-25T13:46:27.325007+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.325007+0000","last_undegraded":"2024-04-25T13:46:27.325007+0000","last_fullsized":"2024-04-25T13:46:27.325007+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328629+0000","last_change":"2024-04-25T13:46:27.328629+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328629+0000","last_undegraded":"2024-04-25T13:46:27.328629+0000","last_fullsized":"2024-04-25T13:46:27.328629+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327924+0000","last_change":"2024-04-25T13:46:27.327924+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327924+0000","last_undegraded":"2024-04-25T13:46:27.327924+0000","last_fullsized":"2024-04-25T13:46:27.327924+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.323952+0000","last_change":"2024-04-25T13:46:27.323952+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.323952+0000","last_undegraded":"2024-04-25T13:46:27.323952+0000","last_fullsized":"2024-04-25T13:46:27.323952+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328084+0000","last_change":"2024-04-25T13:46:27.328084+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328084+0000","last_undegraded":"2024-04-25T13:46:27.328084+0000","last_fullsized":"2024-04-25T13:46:27.328084+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322774+0000","last_change":"2024-04-25T13:46:27.322774+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322774+0000","last_undegraded":"2024-04-25T13:46:27.322774+0000","last_fullsized":"2024-04-25T13:46:27.322774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.329262+0000","last_change":"2024-04-25T13:46:27.329262+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.329262+0000","last_undegraded":"2024-04-25T13:46:27.329262+0000","last_fullsized":"2024-04-25T13:46:27.329262+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.330441+0000","last_change":"2024-04-25T13:46:27.330441+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.330441+0000","last_undegraded":"2024-04-25T13:46:27.330441+0000","last_fullsized":"2024-04-25T13:46:27.330441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.326308+0000","last_change":"2024-04-25T13:46:27.326308+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.326308+0000","last_undegraded":"2024-04-25T13:46:27.326308+0000","last_fullsized":"2024-04-25T13:46:27.326308+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":307,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T13:46:27.329125+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:46:27.329125+0000","last_peered":"2024-04-25T13:46:27.329125+0000","last_clean":"2024-04-25T13:46:27.329125+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:46:27.329125+0000","last_undegraded":"2024-04-25T13:46:27.329125+0000","last_fullsized":"2024-04-25T13:46:27.329125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194096,"statfs":{"total":200000000,"available":198755038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194096,"statfs":{"total":200000000,"available":198755038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:46:33.488 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:46:33.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.663+0000 7f4bcb7a3640 1 Processor -- start 2024-04-25T13:46:33.663 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.663+0000 7f4bcb7a3640 1 -- start start 2024-04-25T13:46:33.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bcb7a3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4bc406f5a0 0x7f4bc4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bcb7a3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4bc4161070 0x7f4bc4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bcb7a3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bc406fcf0 con 0x7f4bc40b4bd0 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bcb7a3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc41619c0 con 0x7f4bc4161070 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bcb7a3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4bc4073f10 con 0x7f4bc406f5a0 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bc8d3e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4bc4161070 0x7f4bc4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bc8d3e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4bc4161070 0x7f4bc4161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42998/0 (socket says 172.21.15.80:42998) 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bc8d3e640 1 -- 172.21.15.80:0/2169832896 learned_addr learned my addr 172.21.15.80:0/2169832896 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bc8d3e640 1 -- 172.21.15.80:0/2169832896 >> v2:172.21.15.87:3300/0 conn(0x7f4bc406f5a0 msgr2=0x7f4bc4160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bc8d3e640 1 --2- 172.21.15.80:0/2169832896 >> v2:172.21.15.87:3300/0 conn(0x7f4bc406f5a0 0x7f4bc4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bc8d3e640 1 -- 172.21.15.80:0/2169832896 >> v1:172.21.15.87:6789/0 conn(0x7f4bc40b4bd0 legacy=0x7f4bc40b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:33.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bc8d3e640 1 -- 172.21.15.80:0/2169832896 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc40b0c60 con 0x7f4bc4161070 2024-04-25T13:46:33.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.667+0000 7f4bc8d3e640 1 --2- 172.21.15.80:0/2169832896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc4161070 0x7f4bc4161480 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4071e30 tx=0x7f4bb801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc663d24c70e38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:33.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bc3fff640 1 -- 172.21.15.80:0/2169832896 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80b0070 con 0x7f4bc4161070 2024-04-25T13:46:33.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bc3fff640 1 -- 172.21.15.80:0/2169832896 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bb80ab600 con 0x7f4bc4161070 2024-04-25T13:46:33.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bc3fff640 1 -- 172.21.15.80:0/2169832896 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb80ab9d0 con 0x7f4bc4161070 2024-04-25T13:46:33.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2169832896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc4161070 msgr2=0x7f4bc4161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2169832896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc4161070 0x7f4bc4161480 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4071e30 tx=0x7f4bb801c450 comp rx=0 tx=0).stop 2024-04-25T13:46:33.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2169832896 shutdown_connections 2024-04-25T13:46:33.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2169832896 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc4161070 0x7f4bc4161480 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2169832896 >> v2:172.21.15.87:3300/0 conn(0x7f4bc406f5a0 0x7f4bc4160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2169832896 >> 172.21.15.80:0/2169832896 conn(0x7f4bc406d080 msgr2=0x7f4bc40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:33.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2169832896 shutdown_connections 2024-04-25T13:46:33.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2169832896 wait complete. 2024-04-25T13:46:33.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 Processor -- start 2024-04-25T13:46:33.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- start start 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc406f5a0 0x7f4bc4155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bc40b4bd0 0x7f4bc40ad620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bc4161070 0x7f4bc4156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc416a860 con 0x7f4bc406f5a0 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc416a6e0 con 0x7f4bc40b4bd0 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.671+0000 7f4bcb7a3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc416a9e0 con 0x7f4bc4161070 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc406f5a0 0x7f4bc4155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc406f5a0 0x7f4bc4155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43006/0 (socket says 172.21.15.80:43006) 2024-04-25T13:46:33.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 -- 172.21.15.80:0/2553842023 learned_addr learned my addr 172.21.15.80:0/2553842023 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc8d3e640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bc40b4bd0 0x7f4bc40ad620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc9d40640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bc4161070 0x7f4bc4156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 -- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bc4161070 msgr2=0x7f4bc4156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bc4161070 0x7f4bc4156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 -- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bc40b4bd0 msgr2=0x7f4bc40ad620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bc40b4bd0 0x7f4bc40ad620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 -- 172.21.15.80:0/2553842023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc40ab4a0 con 0x7f4bc406f5a0 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc8d3e640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bc40b4bd0 0x7f4bc40ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc953f640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc406f5a0 0x7f4bc4155510 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00b420 tx=0x7f4bbc00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc1ffb640 1 -- 172.21.15.80:0/2553842023 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc011830 con 0x7f4bc406f5a0 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc1ffb640 1 -- 172.21.15.80:0/2553842023 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bbc016070 con 0x7f4bc406f5a0 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc4155a50 con 0x7f4bc406f5a0 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc1ffb640 1 -- 172.21.15.80:0/2553842023 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bbc011c30 con 0x7f4bc406f5a0 2024-04-25T13:46:33.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc41fc410 con 0x7f4bc406f5a0 2024-04-25T13:46:33.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc406a3a0 con 0x7f4bc406f5a0 2024-04-25T13:46:33.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc1ffb640 1 -- 172.21.15.80:0/2553842023 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4bbc01b460 con 0x7f4bc406f5a0 2024-04-25T13:46:33.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc1ffb640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb006f430 0x7f4bb0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:33.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc1ffb640 1 -- 172.21.15.80:0/2553842023 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f4bbc08eea0 con 0x7f4bc406f5a0 2024-04-25T13:46:33.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.675+0000 7f4bc8d3e640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb006f430 0x7f4bb0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:33.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.679+0000 7f4bc8d3e640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb006f430 0x7f4bb0071940 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4070270 tx=0x7f4bb80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:33.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.683+0000 7f4bc1ffb640 1 -- 172.21.15.80:0/2553842023 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4bbc05be20 con 0x7f4bc406f5a0 2024-04-25T13:46:33.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.835+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4bc40739c0 con 0x7f4bb006f430 2024-04-25T13:46:33.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.851+0000 7f4bc1ffb640 1 -- 172.21.15.80:0/2553842023 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237596 (secure 0 0 0) 0x7f4bc40739c0 con 0x7f4bb006f430 2024-04-25T13:46:33.852 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:33.854 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:46:33.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb006f430 msgr2=0x7f4bb0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb006f430 0x7f4bb0071940 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4070270 tx=0x7f4bb80a6000 comp rx=0 tx=0).stop 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc406f5a0 msgr2=0x7f4bc4155510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc406f5a0 0x7f4bc4155510 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc00b420 tx=0x7f4bbc00b8e0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 shutdown_connections 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4bb006f430 0x7f4bb0071940 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4bc4161070 0x7f4bc4156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bc40b4bd0 0x7f4bc40ad620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 --2- 172.21.15.80:0/2553842023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bc406f5a0 0x7f4bc4155510 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 >> 172.21.15.80:0/2553842023 conn(0x7f4bc406d080 msgr2=0x7f4bc40b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 shutdown_connections 2024-04-25T13:46:33.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:33.855+0000 7f4bcb7a3640 1 -- 172.21.15.80:0/2553842023 wait complete. 2024-04-25T13:46:33.872 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-25T13:46:32.228909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6244,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556255,"statfs":{"total":1600000000,"available":1593605780,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.908101"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335260+0000","last_change":"2024-04-25T13:46:27.335260+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335260+0000","last_undegraded":"2024-04-25T13:46:27.335260+0000","last_fullsized":"2024-04-25T13:46:27.335260+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327390+0000","last_change":"2024-04-25T13:46:27.327390+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327390+0000","last_undegraded":"2024-04-25T13:46:27.327390+0000","last_fullsized":"2024-04-25T13:46:27.327390+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335626+0000","last_change":"2024-04-25T13:46:27.335626+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335626+0000","last_undegraded":"2024-04-25T13:46:27.335626+0000","last_fullsized":"2024-04-25T13:46:27.335626+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335478+0000","last_change":"2024-04-25T13:46:27.335478+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335478+0000","last_undegraded":"2024-04-25T13:46:27.335478+0000","last_fullsized":"2024-04-25T13:46:27.335478+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328593+0000","last_change":"2024-04-25T13:46:27.328593+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328593+0000","last_undegraded":"2024-04-25T13:46:27.328593+0000","last_fullsized":"2024-04-25T13:46:27.328593+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.334338+0000","last_change":"2024-04-25T13:46:27.334338+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.334338+0000","last_undegraded":"2024-04-25T13:46:27.334338+0000","last_fullsized":"2024-04-25T13:46:27.334338+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322542+0000","last_change":"2024-04-25T13:46:27.322542+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322542+0000","last_undegraded":"2024-04-25T13:46:27.322542+0000","last_fullsized":"2024-04-25T13:46:27.322542+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328199+0000","last_change":"2024-04-25T13:46:27.328199+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328199+0000","last_undegraded":"2024-04-25T13:46:27.328199+0000","last_fullsized":"2024-04-25T13:46:27.328199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.331673+0000","last_change":"2024-04-25T13:46:27.331673+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.331673+0000","last_undegraded":"2024-04-25T13:46:27.331673+0000","last_fullsized":"2024-04-25T13:46:27.331673+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327624+0000","last_change":"2024-04-25T13:46:27.327624+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327624+0000","last_undegraded":"2024-04-25T13:46:27.327624+0000","last_fullsized":"2024-04-25T13:46:27.327624+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.326248+0000","last_change":"2024-04-25T13:46:27.326248+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.326248+0000","last_undegraded":"2024-04-25T13:46:27.326248+0000","last_fullsized":"2024-04-25T13:46:27.326248+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327363+0000","last_change":"2024-04-25T13:46:27.327363+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327363+0000","last_undegraded":"2024-04-25T13:46:27.327363+0000","last_fullsized":"2024-04-25T13:46:27.327363+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.331684+0000","last_change":"2024-04-25T13:46:27.331684+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.331684+0000","last_undegraded":"2024-04-25T13:46:27.331684+0000","last_fullsized":"2024-04-25T13:46:27.331684+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.330622+0000","last_change":"2024-04-25T13:46:27.330622+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.330622+0000","last_undegraded":"2024-04-25T13:46:27.330622+0000","last_fullsized":"2024-04-25T13:46:27.330622+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328453+0000","last_change":"2024-04-25T13:46:27.328453+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328453+0000","last_undegraded":"2024-04-25T13:46:27.328453+0000","last_fullsized":"2024-04-25T13:46:27.328453+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.329692+0000","last_change":"2024-04-25T13:46:27.329692+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.329692+0000","last_undegraded":"2024-04-25T13:46:27.329692+0000","last_fullsized":"2024-04-25T13:46:27.329692+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327476+0000","last_change":"2024-04-25T13:46:27.327476+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327476+0000","last_undegraded":"2024-04-25T13:46:27.327476+0000","last_fullsized":"2024-04-25T13:46:27.327476+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322745+0000","last_change":"2024-04-25T13:46:27.322745+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322745+0000","last_undegraded":"2024-04-25T13:46:27.322745+0000","last_fullsized":"2024-04-25T13:46:27.322745+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.332762+0000","last_change":"2024-04-25T13:46:27.332762+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.332762+0000","last_undegraded":"2024-04-25T13:46:27.332762+0000","last_fullsized":"2024-04-25T13:46:27.332762+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.333810+0000","last_change":"2024-04-25T13:46:27.333810+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.333810+0000","last_undegraded":"2024-04-25T13:46:27.333810+0000","last_fullsized":"2024-04-25T13:46:27.333810+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.325007+0000","last_change":"2024-04-25T13:46:27.325007+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.325007+0000","last_undegraded":"2024-04-25T13:46:27.325007+0000","last_fullsized":"2024-04-25T13:46:27.325007+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.324891+0000","last_change":"2024-04-25T13:46:27.324891+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.324891+0000","last_undegraded":"2024-04-25T13:46:27.324891+0000","last_fullsized":"2024-04-25T13:46:27.324891+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.334766+0000","last_change":"2024-04-25T13:46:27.334766+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.334766+0000","last_undegraded":"2024-04-25T13:46:27.334766+0000","last_fullsized":"2024-04-25T13:46:27.334766+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.325007+0000","last_change":"2024-04-25T13:46:27.325007+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.325007+0000","last_undegraded":"2024-04-25T13:46:27.325007+0000","last_fullsized":"2024-04-25T13:46:27.325007+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328629+0000","last_change":"2024-04-25T13:46:27.328629+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328629+0000","last_undegraded":"2024-04-25T13:46:27.328629+0000","last_fullsized":"2024-04-25T13:46:27.328629+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327924+0000","last_change":"2024-04-25T13:46:27.327924+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327924+0000","last_undegraded":"2024-04-25T13:46:27.327924+0000","last_fullsized":"2024-04-25T13:46:27.327924+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.323952+0000","last_change":"2024-04-25T13:46:27.323952+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.323952+0000","last_undegraded":"2024-04-25T13:46:27.323952+0000","last_fullsized":"2024-04-25T13:46:27.323952+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328084+0000","last_change":"2024-04-25T13:46:27.328084+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328084+0000","last_undegraded":"2024-04-25T13:46:27.328084+0000","last_fullsized":"2024-04-25T13:46:27.328084+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322774+0000","last_change":"2024-04-25T13:46:27.322774+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322774+0000","last_undegraded":"2024-04-25T13:46:27.322774+0000","last_fullsized":"2024-04-25T13:46:27.322774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.329262+0000","last_change":"2024-04-25T13:46:27.329262+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.329262+0000","last_undegraded":"2024-04-25T13:46:27.329262+0000","last_fullsized":"2024-04-25T13:46:27.329262+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.330441+0000","last_change":"2024-04-25T13:46:27.330441+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.330441+0000","last_undegraded":"2024-04-25T13:46:27.330441+0000","last_fullsized":"2024-04-25T13:46:27.330441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.326308+0000","last_change":"2024-04-25T13:46:27.326308+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.326308+0000","last_undegraded":"2024-04-25T13:46:27.326308+0000","last_fullsized":"2024-04-25T13:46:27.326308+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":307,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T13:46:27.329125+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:46:27.329125+0000","last_peered":"2024-04-25T13:46:27.329125+0000","last_clean":"2024-04-25T13:46:27.329125+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:46:27.329125+0000","last_undegraded":"2024-04-25T13:46:27.329125+0000","last_fullsized":"2024-04-25T13:46:27.329125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194096,"statfs":{"total":200000000,"available":198755038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194096,"statfs":{"total":200000000,"available":198755038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:46:33.876 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:46:34.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fe7d2640 1 Processor -- start 2024-04-25T13:46:34.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fe7d2640 1 -- start start 2024-04-25T13:46:34.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fe7d2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3f8159f10 0x7fb3f806dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fe7d2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3f806e140 0x7fb3f806e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fe7d2640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f80b1320 con 0x7fb3f80b4ca0 2024-04-25T13:46:34.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fe7d2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f80b1040 con 0x7fb3f8159f10 2024-04-25T13:46:34.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fe7d2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f80b5450 con 0x7fb3f806e140 2024-04-25T13:46:34.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3f8159f10 0x7fb3f806dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f7fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb3f80b4ca0 0x7fb3f80734e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52642/0 (socket says 172.21.15.80:52642) 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f7fff640 1 -- 172.21.15.80:0/548398156 learned_addr learned my addr 172.21.15.80:0/548398156 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb3f8159f10 0x7fb3f806dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43010/0 (socket says 172.21.15.80:43010) 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fcd6f640 1 --2- 172.21.15.80:0/548398156 >> v2:172.21.15.87:3300/0 conn(0x7fb3f806e140 0x7fb3f806e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f77fe640 1 -- 172.21.15.80:0/548398156 >> v2:172.21.15.87:3300/0 conn(0x7fb3f806e140 msgr2=0x7fb3f806e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/548398156 >> v2:172.21.15.87:3300/0 conn(0x7fb3f806e140 0x7fb3f806e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f77fe640 1 -- 172.21.15.80:0/548398156 >> v1:172.21.15.80:6789/0 conn(0x7fb3f80b4ca0 legacy=0x7fb3f80734e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f77fe640 1 -- 172.21.15.80:0/548398156 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f80b5730 con 0x7fb3f8159f10 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/548398156 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656431615 0 0) 0x7fb3f80b1320 con 0x7fb3f80b4ca0 2024-04-25T13:46:34.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.051+0000 7fb3fcd6f640 1 --2- 172.21.15.80:0/548398156 >> v2:172.21.15.87:3300/0 conn(0x7fb3f806e140 0x7fb3f806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:34.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/548398156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f806dc00 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8160f70 tx=0x7fb3ec01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a9ed5e6b98b1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/548398156 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0b0070 con 0x7fb3f8159f10 2024-04-25T13:46:34.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/548398156 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3ec0ab600 con 0x7fb3f8159f10 2024-04-25T13:46:34.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3f6ffd640 1 -- 172.21.15.80:0/548398156 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ec0ab9d0 con 0x7fb3f8159f10 2024-04-25T13:46:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/548398156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 msgr2=0x7fb3f806dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/548398156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f806dc00 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8160f70 tx=0x7fb3ec01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/548398156 shutdown_connections 2024-04-25T13:46:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/548398156 >> v2:172.21.15.87:3300/0 conn(0x7fb3f806e140 0x7fb3f806e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/548398156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f806dc00 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/548398156 >> 172.21.15.80:0/548398156 conn(0x7fb3f806d080 msgr2=0x7fb3f80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/548398156 shutdown_connections 2024-04-25T13:46:34.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.055+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/548398156 wait complete. 2024-04-25T13:46:34.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 Processor -- start 2024-04-25T13:46:34.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 -- start start 2024-04-25T13:46:34.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f806e140 0x7fb3f815c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b4ca0 0x7fb3f8147710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f80b2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f816b690 con 0x7fb3f8159f10 2024-04-25T13:46:34.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f816b510 con 0x7fb3f80b4ca0 2024-04-25T13:46:34.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f816b810 con 0x7fb3f806e140 2024-04-25T13:46:34.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f80b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f80b2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43018/0 (socket says 172.21.15.80:43018) 2024-04-25T13:46:34.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 -- 172.21.15.80:0/1143906364 learned_addr learned my addr 172.21.15.80:0/1143906364 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:34.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b4ca0 0x7fb3f8147710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3f7fff640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f806e140 0x7fb3f815c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 -- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f806e140 msgr2=0x7fb3f815c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f806e140 0x7fb3f815c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 -- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b4ca0 msgr2=0x7fb3f8147710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b4ca0 0x7fb3f8147710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 -- 172.21.15.80:0/1143906364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f8148970 con 0x7fb3f8159f10 2024-04-25T13:46:34.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3f77fe640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b4ca0 0x7fb3f8147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:34.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fcd6f640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f80b2540 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c5c0 tx=0x7fb3f000ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3f4ff9640 1 -- 172.21.15.80:0/1143906364 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0015070 con 0x7fb3f8159f10 2024-04-25T13:46:34.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3f4ff9640 1 -- 172.21.15.80:0/1143906364 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f0011820 con 0x7fb3f8159f10 2024-04-25T13:46:34.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f8148e10 con 0x7fb3f8159f10 2024-04-25T13:46:34.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3f4ff9640 1 -- 172.21.15.80:0/1143906364 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0011c10 con 0x7fb3f8159f10 2024-04-25T13:46:34.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f8153d50 con 0x7fb3f8159f10 2024-04-25T13:46:34.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.059+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f806a3a0 con 0x7fb3f8159f10 2024-04-25T13:46:34.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.063+0000 7fb3f4ff9640 1 -- 172.21.15.80:0/1143906364 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb3f0018070 con 0x7fb3f8159f10 2024-04-25T13:46:34.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.063+0000 7fb3f4ff9640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.063+0000 7fb3f4ff9640 1 -- 172.21.15.80:0/1143906364 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fb3f008f500 con 0x7fb3f8159f10 2024-04-25T13:46:34.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.063+0000 7fb3f7fff640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.063+0000 7fb3f7fff640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8007fd0 tx=0x7fb3e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.067+0000 7fb3f4ff9640 1 -- 172.21.15.80:0/1143906364 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb3f005c4d0 con 0x7fb3f8159f10 2024-04-25T13:46:34.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.211+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb3f8149330 con 0x7fb3e406f400 2024-04-25T13:46:34.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3f4ff9640 1 -- 172.21.15.80:0/1143906364 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+237596 (secure 0 0 0) 0x7fb3e800d7e0 con 0x7fb3e406f400 2024-04-25T13:46:34.233 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:34.235 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:46:34.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 msgr2=0x7fb3e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8007fd0 tx=0x7fb3e8009040 comp rx=0 tx=0).stop 2024-04-25T13:46:34.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 msgr2=0x7fb3f80b2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f80b2540 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000c5c0 tx=0x7fb3f000ca80 comp rx=0 tx=0).stop 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 shutdown_connections 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb3e406f400 0x7fb3e4071910 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3f8159f10 0x7fb3f80b2540 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3f80b4ca0 0x7fb3f8147710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 --2- 172.21.15.80:0/1143906364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3f806e140 0x7fb3f815c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 >> 172.21.15.80:0/1143906364 conn(0x7fb3f806d080 msgr2=0x7fb3f8152a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.231+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 shutdown_connections 2024-04-25T13:46:34.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.235+0000 7fb3fe7d2640 1 -- 172.21.15.80:0/1143906364 wait complete. 2024-04-25T13:46:34.250 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-25T13:46:32.228909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6244,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556255,"statfs":{"total":1600000000,"available":1593605780,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.908101"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335260+0000","last_change":"2024-04-25T13:46:27.335260+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335260+0000","last_undegraded":"2024-04-25T13:46:27.335260+0000","last_fullsized":"2024-04-25T13:46:27.335260+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327390+0000","last_change":"2024-04-25T13:46:27.327390+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327390+0000","last_undegraded":"2024-04-25T13:46:27.327390+0000","last_fullsized":"2024-04-25T13:46:27.327390+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335626+0000","last_change":"2024-04-25T13:46:27.335626+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335626+0000","last_undegraded":"2024-04-25T13:46:27.335626+0000","last_fullsized":"2024-04-25T13:46:27.335626+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.335478+0000","last_change":"2024-04-25T13:46:27.335478+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.335478+0000","last_undegraded":"2024-04-25T13:46:27.335478+0000","last_fullsized":"2024-04-25T13:46:27.335478+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328593+0000","last_change":"2024-04-25T13:46:27.328593+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328593+0000","last_undegraded":"2024-04-25T13:46:27.328593+0000","last_fullsized":"2024-04-25T13:46:27.328593+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.334338+0000","last_change":"2024-04-25T13:46:27.334338+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.334338+0000","last_undegraded":"2024-04-25T13:46:27.334338+0000","last_fullsized":"2024-04-25T13:46:27.334338+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322542+0000","last_change":"2024-04-25T13:46:27.322542+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322542+0000","last_undegraded":"2024-04-25T13:46:27.322542+0000","last_fullsized":"2024-04-25T13:46:27.322542+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328199+0000","last_change":"2024-04-25T13:46:27.328199+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328199+0000","last_undegraded":"2024-04-25T13:46:27.328199+0000","last_fullsized":"2024-04-25T13:46:27.328199+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.331673+0000","last_change":"2024-04-25T13:46:27.331673+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.331673+0000","last_undegraded":"2024-04-25T13:46:27.331673+0000","last_fullsized":"2024-04-25T13:46:27.331673+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327624+0000","last_change":"2024-04-25T13:46:27.327624+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327624+0000","last_undegraded":"2024-04-25T13:46:27.327624+0000","last_fullsized":"2024-04-25T13:46:27.327624+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.326248+0000","last_change":"2024-04-25T13:46:27.326248+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.326248+0000","last_undegraded":"2024-04-25T13:46:27.326248+0000","last_fullsized":"2024-04-25T13:46:27.326248+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327363+0000","last_change":"2024-04-25T13:46:27.327363+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327363+0000","last_undegraded":"2024-04-25T13:46:27.327363+0000","last_fullsized":"2024-04-25T13:46:27.327363+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.331684+0000","last_change":"2024-04-25T13:46:27.331684+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.331684+0000","last_undegraded":"2024-04-25T13:46:27.331684+0000","last_fullsized":"2024-04-25T13:46:27.331684+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.330622+0000","last_change":"2024-04-25T13:46:27.330622+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.330622+0000","last_undegraded":"2024-04-25T13:46:27.330622+0000","last_fullsized":"2024-04-25T13:46:27.330622+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328453+0000","last_change":"2024-04-25T13:46:27.328453+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328453+0000","last_undegraded":"2024-04-25T13:46:27.328453+0000","last_fullsized":"2024-04-25T13:46:27.328453+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.329692+0000","last_change":"2024-04-25T13:46:27.329692+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.329692+0000","last_undegraded":"2024-04-25T13:46:27.329692+0000","last_fullsized":"2024-04-25T13:46:27.329692+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327476+0000","last_change":"2024-04-25T13:46:27.327476+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327476+0000","last_undegraded":"2024-04-25T13:46:27.327476+0000","last_fullsized":"2024-04-25T13:46:27.327476+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322745+0000","last_change":"2024-04-25T13:46:27.322745+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322745+0000","last_undegraded":"2024-04-25T13:46:27.322745+0000","last_fullsized":"2024-04-25T13:46:27.322745+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.332762+0000","last_change":"2024-04-25T13:46:27.332762+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.332762+0000","last_undegraded":"2024-04-25T13:46:27.332762+0000","last_fullsized":"2024-04-25T13:46:27.332762+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.333810+0000","last_change":"2024-04-25T13:46:27.333810+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.333810+0000","last_undegraded":"2024-04-25T13:46:27.333810+0000","last_fullsized":"2024-04-25T13:46:27.333810+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.325007+0000","last_change":"2024-04-25T13:46:27.325007+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.325007+0000","last_undegraded":"2024-04-25T13:46:27.325007+0000","last_fullsized":"2024-04-25T13:46:27.325007+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.324891+0000","last_change":"2024-04-25T13:46:27.324891+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.324891+0000","last_undegraded":"2024-04-25T13:46:27.324891+0000","last_fullsized":"2024-04-25T13:46:27.324891+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.334766+0000","last_change":"2024-04-25T13:46:27.334766+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.334766+0000","last_undegraded":"2024-04-25T13:46:27.334766+0000","last_fullsized":"2024-04-25T13:46:27.334766+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.325007+0000","last_change":"2024-04-25T13:46:27.325007+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.325007+0000","last_undegraded":"2024-04-25T13:46:27.325007+0000","last_fullsized":"2024-04-25T13:46:27.325007+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328629+0000","last_change":"2024-04-25T13:46:27.328629+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328629+0000","last_undegraded":"2024-04-25T13:46:27.328629+0000","last_fullsized":"2024-04-25T13:46:27.328629+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.327924+0000","last_change":"2024-04-25T13:46:27.327924+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.327924+0000","last_undegraded":"2024-04-25T13:46:27.327924+0000","last_fullsized":"2024-04-25T13:46:27.327924+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.323952+0000","last_change":"2024-04-25T13:46:27.323952+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.323952+0000","last_undegraded":"2024-04-25T13:46:27.323952+0000","last_fullsized":"2024-04-25T13:46:27.323952+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.328084+0000","last_change":"2024-04-25T13:46:27.328084+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.328084+0000","last_undegraded":"2024-04-25T13:46:27.328084+0000","last_fullsized":"2024-04-25T13:46:27.328084+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.322774+0000","last_change":"2024-04-25T13:46:27.322774+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.322774+0000","last_undegraded":"2024-04-25T13:46:27.322774+0000","last_fullsized":"2024-04-25T13:46:27.322774+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.329262+0000","last_change":"2024-04-25T13:46:27.329262+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.329262+0000","last_undegraded":"2024-04-25T13:46:27.329262+0000","last_fullsized":"2024-04-25T13:46:27.329262+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.330441+0000","last_change":"2024-04-25T13:46:27.330441+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.330441+0000","last_undegraded":"2024-04-25T13:46:27.330441+0000","last_fullsized":"2024-04-25T13:46:27.330441+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T13:46:28.321318+0000","last_change":"2024-04-25T13:46:28.321318+0000","last_active":"2024-04-25T13:46:28.321318+0000","last_peered":"2024-04-25T13:46:28.321318+0000","last_clean":"2024-04-25T13:46:28.321318+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:28.321318+0000","last_undegraded":"2024-04-25T13:46:28.321318+0000","last_fullsized":"2024-04-25T13:46:28.321318+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":1,"reported_epoch":119,"state":"creating+peering","last_fresh":"2024-04-25T13:46:27.326308+0000","last_change":"2024-04-25T13:46:27.326308+0000","last_active":"2024-04-25T13:46:27.309242+0000","last_peered":"2024-04-25T13:46:27.309242+0000","last_clean":"2024-04-25T13:46:27.309242+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T13:46:27.326308+0000","last_undegraded":"2024-04-25T13:46:27.326308+0000","last_fullsized":"2024-04-25T13:46:27.326308+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":307,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-04-25T13:46:27.329125+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:46:27.329125+0000","last_peered":"2024-04-25T13:46:27.329125+0000","last_clean":"2024-04-25T13:46:27.329125+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:46:27.329125+0000","last_undegraded":"2024-04-25T13:46:27.329125+0000","last_fullsized":"2024-04-25T13:46:27.329125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574924,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194096,"statfs":{"total":200000000,"available":198755038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574924,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":633,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194678,"statfs":{"total":200000000,"available":199351223,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":639,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194673,"statfs":{"total":200000000,"available":199345406,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574924,"num_pgs":11,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1215,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194096,"statfs":{"total":200000000,"available":198755038,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:46:34.253 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-04-25T13:46:34.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-04-25T13:46:34.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-04-25T13:46:34.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-04-25T13:46:34.254 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.254 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-04-25T13:46:34.255 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.256 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.257 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.258 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-04-25T13:46:34.259 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-04-25T13:46:34.260 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1f is not active+clean 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.335260+0000', 'last_change': '2024-04-25T13:46:27.335260+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.335260+0000', 'last_undegraded': '2024-04-25T13:46:27.335260+0000', 'last_fullsized': '2024-04-25T13:46:27.335260+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1e is not active+clean 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.327390+0000', 'last_change': '2024-04-25T13:46:27.327390+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.327390+0000', 'last_undegraded': '2024-04-25T13:46:27.327390+0000', 'last_fullsized': '2024-04-25T13:46:27.327390+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1d is not active+clean 2024-04-25T13:46:34.261 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.335626+0000', 'last_change': '2024-04-25T13:46:27.335626+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.335626+0000', 'last_undegraded': '2024-04-25T13:46:27.335626+0000', 'last_fullsized': '2024-04-25T13:46:27.335626+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-04-25T13:46:34.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1c is not active+clean 2024-04-25T13:46:34.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.335478+0000', 'last_change': '2024-04-25T13:46:27.335478+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.335478+0000', 'last_undegraded': '2024-04-25T13:46:27.335478+0000', 'last_fullsized': '2024-04-25T13:46:27.335478+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-04-25T13:46:34.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.262 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1b is not active+clean 2024-04-25T13:46:34.262 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.328593+0000', 'last_change': '2024-04-25T13:46:27.328593+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.328593+0000', 'last_undegraded': '2024-04-25T13:46:27.328593+0000', 'last_fullsized': '2024-04-25T13:46:27.328593+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1a is not active+clean 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.334338+0000', 'last_change': '2024-04-25T13:46:27.334338+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.334338+0000', 'last_undegraded': '2024-04-25T13:46:27.334338+0000', 'last_fullsized': '2024-04-25T13:46:27.334338+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.19 is not active+clean 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.322542+0000', 'last_change': '2024-04-25T13:46:27.322542+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.322542+0000', 'last_undegraded': '2024-04-25T13:46:27.322542+0000', 'last_fullsized': '2024-04-25T13:46:27.322542+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-04-25T13:46:34.263 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.18 is not active+clean 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.328199+0000', 'last_change': '2024-04-25T13:46:27.328199+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.328199+0000', 'last_undegraded': '2024-04-25T13:46:27.328199+0000', 'last_fullsized': '2024-04-25T13:46:27.328199+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.17 is not active+clean 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.331673+0000', 'last_change': '2024-04-25T13:46:27.331673+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.331673+0000', 'last_undegraded': '2024-04-25T13:46:27.331673+0000', 'last_fullsized': '2024-04-25T13:46:27.331673+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.16 is not active+clean 2024-04-25T13:46:34.264 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.327624+0000', 'last_change': '2024-04-25T13:46:27.327624+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.327624+0000', 'last_undegraded': '2024-04-25T13:46:27.327624+0000', 'last_fullsized': '2024-04-25T13:46:27.327624+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.15 is not active+clean 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.326248+0000', 'last_change': '2024-04-25T13:46:27.326248+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.326248+0000', 'last_undegraded': '2024-04-25T13:46:27.326248+0000', 'last_fullsized': '2024-04-25T13:46:27.326248+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.14 is not active+clean 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.327363+0000', 'last_change': '2024-04-25T13:46:27.327363+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.327363+0000', 'last_undegraded': '2024-04-25T13:46:27.327363+0000', 'last_fullsized': '2024-04-25T13:46:27.327363+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:46:34.265 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.13 is not active+clean 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.331684+0000', 'last_change': '2024-04-25T13:46:27.331684+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.331684+0000', 'last_undegraded': '2024-04-25T13:46:27.331684+0000', 'last_fullsized': '2024-04-25T13:46:27.331684+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.12 is not active+clean 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.330622+0000', 'last_change': '2024-04-25T13:46:27.330622+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.330622+0000', 'last_undegraded': '2024-04-25T13:46:27.330622+0000', 'last_fullsized': '2024-04-25T13:46:27.330622+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-04-25T13:46:34.266 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.11 is not active+clean 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.328453+0000', 'last_change': '2024-04-25T13:46:27.328453+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.328453+0000', 'last_undegraded': '2024-04-25T13:46:27.328453+0000', 'last_fullsized': '2024-04-25T13:46:27.328453+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.10 is not active+clean 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.329692+0000', 'last_change': '2024-04-25T13:46:27.329692+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.329692+0000', 'last_undegraded': '2024-04-25T13:46:27.329692+0000', 'last_fullsized': '2024-04-25T13:46:27.329692+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.e is not active+clean 2024-04-25T13:46:34.267 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.327476+0000', 'last_change': '2024-04-25T13:46:27.327476+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.327476+0000', 'last_undegraded': '2024-04-25T13:46:27.327476+0000', 'last_fullsized': '2024-04-25T13:46:27.327476+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.d is not active+clean 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.322745+0000', 'last_change': '2024-04-25T13:46:27.322745+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.322745+0000', 'last_undegraded': '2024-04-25T13:46:27.322745+0000', 'last_fullsized': '2024-04-25T13:46:27.322745+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.c is not active+clean 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.332762+0000', 'last_change': '2024-04-25T13:46:27.332762+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.332762+0000', 'last_undegraded': '2024-04-25T13:46:27.332762+0000', 'last_fullsized': '2024-04-25T13:46:27.332762+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.268 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.b is not active+clean 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.333810+0000', 'last_change': '2024-04-25T13:46:27.333810+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.333810+0000', 'last_undegraded': '2024-04-25T13:46:27.333810+0000', 'last_fullsized': '2024-04-25T13:46:27.333810+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.a is not active+clean 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.325007+0000', 'last_change': '2024-04-25T13:46:27.325007+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.325007+0000', 'last_undegraded': '2024-04-25T13:46:27.325007+0000', 'last_fullsized': '2024-04-25T13:46:27.325007+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.269 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.9 is not active+clean 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.324891+0000', 'last_change': '2024-04-25T13:46:27.324891+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.324891+0000', 'last_undegraded': '2024-04-25T13:46:27.324891+0000', 'last_fullsized': '2024-04-25T13:46:27.324891+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.8 is not active+clean 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.334766+0000', 'last_change': '2024-04-25T13:46:27.334766+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.334766+0000', 'last_undegraded': '2024-04-25T13:46:27.334766+0000', 'last_fullsized': '2024-04-25T13:46:27.334766+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.7 is not active+clean 2024-04-25T13:46:34.270 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.325007+0000', 'last_change': '2024-04-25T13:46:27.325007+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.325007+0000', 'last_undegraded': '2024-04-25T13:46:27.325007+0000', 'last_fullsized': '2024-04-25T13:46:27.325007+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.6 is not active+clean 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.328629+0000', 'last_change': '2024-04-25T13:46:27.328629+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.328629+0000', 'last_undegraded': '2024-04-25T13:46:27.328629+0000', 'last_fullsized': '2024-04-25T13:46:27.328629+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.5 is not active+clean 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.327924+0000', 'last_change': '2024-04-25T13:46:27.327924+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.327924+0000', 'last_undegraded': '2024-04-25T13:46:27.327924+0000', 'last_fullsized': '2024-04-25T13:46:27.327924+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.271 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.4 is not active+clean 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.323952+0000', 'last_change': '2024-04-25T13:46:27.323952+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.323952+0000', 'last_undegraded': '2024-04-25T13:46:27.323952+0000', 'last_fullsized': '2024-04-25T13:46:27.323952+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.3 is not active+clean 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.328084+0000', 'last_change': '2024-04-25T13:46:27.328084+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.328084+0000', 'last_undegraded': '2024-04-25T13:46:27.328084+0000', 'last_fullsized': '2024-04-25T13:46:27.328084+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-04-25T13:46:34.272 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.2 is not active+clean 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.322774+0000', 'last_change': '2024-04-25T13:46:27.322774+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.322774+0000', 'last_undegraded': '2024-04-25T13:46:27.322774+0000', 'last_fullsized': '2024-04-25T13:46:27.322774+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.1 is not active+clean 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.329262+0000', 'last_change': '2024-04-25T13:46:27.329262+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.329262+0000', 'last_undegraded': '2024-04-25T13:46:27.329262+0000', 'last_fullsized': '2024-04-25T13:46:27.329262+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.0 is not active+clean 2024-04-25T13:46:34.273 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.330441+0000', 'last_change': '2024-04-25T13:46:27.330441+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.330441+0000', 'last_undegraded': '2024-04-25T13:46:27.330441+0000', 'last_fullsized': '2024-04-25T13:46:27.330441+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T13:46:34.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-04-25T13:46:34.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T13:46:28.321318+0000', 'last_change': '2024-04-25T13:46:28.321318+0000', 'last_active': '2024-04-25T13:46:28.321318+0000', 'last_peered': '2024-04-25T13:46:28.321318+0000', 'last_clean': '2024-04-25T13:46:28.321318+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:28.321318+0000', 'last_undegraded': '2024-04-25T13:46:28.321318+0000', 'last_fullsized': '2024-04-25T13:46:28.321318+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:28.321318+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T13:46:34.274 INFO:tasks.cephfs.filesystem.ceph_manager:PG 14.f is not active+clean 2024-04-25T13:46:34.274 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '14.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 119, 'state': 'creating+peering', 'last_fresh': '2024-04-25T13:46:27.326308+0000', 'last_change': '2024-04-25T13:46:27.326308+0000', 'last_active': '2024-04-25T13:46:27.309242+0000', 'last_peered': '2024-04-25T13:46:27.309242+0000', 'last_clean': '2024-04-25T13:46:27.309242+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T13:46:27.326308+0000', 'last_undegraded': '2024-04-25T13:46:27.326308+0000', 'last_fullsized': '2024-04-25T13:46:27.326308+0000', 'mapping_epoch': 119, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 119, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'last_clean_scrub_stamp': '2024-04-25T13:46:27.309242+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T13:46:34.274 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T13:46:34.305 INFO:tasks.ceph.mds.a.smithi080.stderr:2024-04-25T13:46:34.307+0000 7f83b3cbb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:46:34.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.431+0000 7f151af5c640 1 Processor -- start 2024-04-25T13:46:34.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f151af5c640 1 -- start start 2024-04-25T13:46:34.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f151af5c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d2e80 0x7f15141607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f151af5c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f15140d59b0 0x7f15140d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f151af5c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f15140d4330 0x7f15140d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f151af5c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f15140d6300 con 0x7f15140d59b0 2024-04-25T13:46:34.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f151af5c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1514161060 con 0x7f15140d4330 2024-04-25T13:46:34.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f151af5c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1514161340 con 0x7f15140d2e80 2024-04-25T13:46:34.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1513fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f15140d59b0 0x7f15140d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1513fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f15140d59b0 0x7f15140d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43030/0 (socket says 172.21.15.80:43030) 2024-04-25T13:46:34.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1513fff640 1 -- 172.21.15.80:0/1987713650 learned_addr learned my addr 172.21.15.80:0/1987713650 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:34.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f15194f9640 1 --2- 172.21.15.80:0/1987713650 >> v2:172.21.15.87:3300/0 conn(0x7f15140d4330 0x7f15140d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1518cf8640 1 --2- 172.21.15.80:0/1987713650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d2e80 0x7f15141607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1513fff640 1 -- 172.21.15.80:0/1987713650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d2e80 msgr2=0x7f15141607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:34.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1513fff640 1 --2- 172.21.15.80:0/1987713650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d2e80 0x7f15141607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1513fff640 1 -- 172.21.15.80:0/1987713650 >> v2:172.21.15.87:3300/0 conn(0x7f15140d4330 msgr2=0x7f15140d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:34.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1513fff640 1 --2- 172.21.15.80:0/1987713650 >> v2:172.21.15.87:3300/0 conn(0x7f15140d4330 0x7f15140d4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1513fff640 1 -- 172.21.15.80:0/1987713650 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15140d2680 con 0x7f15140d59b0 2024-04-25T13:46:34.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.435+0000 7f1518cf8640 1 --2- 172.21.15.80:0/1987713650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d2e80 0x7f15141607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:34.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f1513fff640 1 --2- 172.21.15.80:0/1987713650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d59b0 0x7f15140d5dc0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f15140d6c70 tx=0x7f150001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b800b6198ea5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f15137fe640 1 -- 172.21.15.80:0/1987713650 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15000ab640 con 0x7f15140d59b0 2024-04-25T13:46:34.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f15137fe640 1 -- 172.21.15.80:0/1987713650 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15000b0070 con 0x7f15140d59b0 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f15137fe640 1 -- 172.21.15.80:0/1987713650 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15000aba00 con 0x7f15140d59b0 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 -- 172.21.15.80:0/1987713650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d59b0 msgr2=0x7f15140d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 --2- 172.21.15.80:0/1987713650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d59b0 0x7f15140d5dc0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f15140d6c70 tx=0x7f150001c460 comp rx=0 tx=0).stop 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 -- 172.21.15.80:0/1987713650 shutdown_connections 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 --2- 172.21.15.80:0/1987713650 >> v2:172.21.15.87:3300/0 conn(0x7f15140d4330 0x7f15140d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 --2- 172.21.15.80:0/1987713650 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d59b0 0x7f15140d5dc0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 --2- 172.21.15.80:0/1987713650 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d2e80 0x7f15141607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 -- 172.21.15.80:0/1987713650 >> 172.21.15.80:0/1987713650 conn(0x7f15140737a0 msgr2=0x7f151406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 -- 172.21.15.80:0/1987713650 shutdown_connections 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 -- 172.21.15.80:0/1987713650 wait complete. 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 Processor -- start 2024-04-25T13:46:34.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.439+0000 7f151af5c640 1 -- start start 2024-04-25T13:46:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d2e80 0x7f1514151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15140d4330 0x7f151415b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d59b0 0x7f1514153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f15140d4e00 con 0x7f15140d2e80 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f15140d4c80 con 0x7f15140d4330 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f15140d3290 con 0x7f15140d59b0 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d2e80 0x7f1514151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d2e80 0x7f1514151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43042/0 (socket says 172.21.15.80:43042) 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 -- 172.21.15.80:0/3544375060 learned_addr learned my addr 172.21.15.80:0/3544375060 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f15194f9640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d59b0 0x7f1514153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1513fff640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15140d4330 0x7f151415b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 -- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d59b0 msgr2=0x7f1514153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d59b0 0x7f1514153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 -- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15140d4330 msgr2=0x7f151415b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15140d4330 0x7f151415b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 -- 172.21.15.80:0/3544375060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15140cf930 con 0x7f15140d2e80 2024-04-25T13:46:34.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1513fff640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15140d4330 0x7f151415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:34.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f1518cf8640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d2e80 0x7f1514151af0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f150800c820 tx=0x7f150800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f15117fa640 1 -- 172.21.15.80:0/3544375060 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508015070 con 0x7f15140d2e80 2024-04-25T13:46:34.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f15117fa640 1 -- 172.21.15.80:0/3544375060 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1508007a30 con 0x7f15140d2e80 2024-04-25T13:46:34.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1514152030 con 0x7f15140d2e80 2024-04-25T13:46:34.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f15117fa640 1 -- 172.21.15.80:0/3544375060 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508007de0 con 0x7f15140d2e80 2024-04-25T13:46:34.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15141549d0 con 0x7f15140d2e80 2024-04-25T13:46:34.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.443+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f151406a3a0 con 0x7f15140d2e80 2024-04-25T13:46:34.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.447+0000 7f15117fa640 1 -- 172.21.15.80:0/3544375060 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1508018070 con 0x7f15140d2e80 2024-04-25T13:46:34.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.447+0000 7f15117fa640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14f806f400 0x7f14f8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.447+0000 7f15117fa640 1 -- 172.21.15.80:0/3544375060 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f150808f160 con 0x7f15140d2e80 2024-04-25T13:46:34.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.447+0000 7f1513fff640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14f806f400 0x7f14f8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.447+0000 7f1513fff640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14f806f400 0x7f14f8071910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f15000af000 tx=0x7f1500008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.451+0000 7f15117fa640 1 -- 172.21.15.80:0/3544375060 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f150805c0e0 con 0x7f15140d2e80 2024-04-25T13:46:34.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.651+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f151416af00 con 0x7f15140d2e80 2024-04-25T13:46:34.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f15117fa640 1 -- 172.21.15.80:0/3544375060 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1008 (secure 0 0 0) 0x7f15080640d0 con 0x7f15140d2e80 2024-04-25T13:46:34.652 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:34.652 INFO:teuthology.orchestra.run.smithi080.stdout:{"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":383,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":122,"num_osds":8,"num_up_osds":8,"osd_up_since":1714052417,"num_in_osds":8,"osd_in_since":1714052414,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":6577108,"bytes_avail":1593422892,"bytes_total":1600000000,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":59,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"a","status":"up:active","gid":7509}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-25T13:44:22.179930+0000","services":{}},"progress_events":{}} 2024-04-25T13:46:34.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14f806f400 msgr2=0x7f14f8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14f806f400 0x7f14f8071910 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f15000af000 tx=0x7f1500008040 comp rx=0 tx=0).stop 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d2e80 msgr2=0x7f1514151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d2e80 0x7f1514151af0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f150800c820 tx=0x7f150800cce0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 shutdown_connections 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14f806f400 0x7f14f8071910 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f15140d59b0 0x7f1514153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15140d4330 0x7f151415b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 --2- 172.21.15.80:0/3544375060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f15140d2e80 0x7f1514151af0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 >> 172.21.15.80:0/3544375060 conn(0x7f15140737a0 msgr2=0x7f15140d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 shutdown_connections 2024-04-25T13:46:34.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.655+0000 7f151af5c640 1 -- 172.21.15.80:0/3544375060 wait complete. 2024-04-25T13:46:34.673 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T13:46:34.674 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:46:34.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.863+0000 7fce3641f640 1 Processor -- start 2024-04-25T13:46:34.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.863+0000 7fce3641f640 1 -- start start 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce3641f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fce300d4330 0x7fce300d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce3641f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce30161060 con 0x7fce300d2e80 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce3641f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce30161340 con 0x7fce300d59b0 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce3641f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fce300d2680 con 0x7fce300d4330 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fce300d2e80 0x7fce301607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52650/0 (socket says 172.21.15.80:52650) 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2ffff640 1 -- 172.21.15.80:0/4132811368 learned_addr learned my addr 172.21.15.80:0/4132811368 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce349bc640 1 --2- 172.21.15.80:0/4132811368 >> v2:172.21.15.87:3300/0 conn(0x7fce300d4330 0x7fce300d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 647782226 0 0) 0x7fce30161060 con 0x7fce300d2e80 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce1c003530 con 0x7fce300d2e80 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2830122409 0 0) 0x7fce1c003530 con 0x7fce300d2e80 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 >> v2:172.21.15.87:3300/0 conn(0x7fce300d4330 msgr2=0x7fce300d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:34.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 --2- 172.21.15.80:0/4132811368 >> v2:172.21.15.87:3300/0 conn(0x7fce300d4330 0x7fce300d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 >> v1:172.21.15.87:6789/0 conn(0x7fce300d59b0 legacy=0x7fce300d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce300d2960 con 0x7fce300d2e80 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fce200016a0 con 0x7fce300d2e80 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 >> v1:172.21.15.80:6789/0 conn(0x7fce300d2e80 legacy=0x7fce301607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce1c0039c0 0x7fce1c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c004810 0x7fce1c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce1c005260 0x7fce1c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce300d2680 con 0x7fce1c005260 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce1c00c1f0 con 0x7fce1c0039c0 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce1c00c3f0 con 0x7fce1c004810 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce349bc640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce1c005260 0x7fce1c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2ffff640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce1c0039c0 0x7fce1c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2f7fe640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c004810 0x7fce1c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce349bc640 1 -- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c004810 msgr2=0x7fce1c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:34.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce349bc640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c004810 0x7fce1c004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce349bc640 1 -- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce1c0039c0 msgr2=0x7fce1c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:34.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce349bc640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce1c0039c0 0x7fce1c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce349bc640 1 -- 172.21.15.80:0/4132811368 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce1c003530 con 0x7fce1c005260 2024-04-25T13:46:34.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.867+0000 7fce2ffff640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce1c0039c0 0x7fce1c003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:34.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce349bc640 1 -- 172.21.15.80:0/4132811368 reap_dead start 2024-04-25T13:46:34.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce349bc640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce1c005260 0x7fce1c00b9d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fce28009f90 tx=0x7fce2801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2267a4d2fb701f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28009c10 con 0x7fce1c005260 2024-04-25T13:46:34.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce28003ba0 con 0x7fce1c005260 2024-04-25T13:46:34.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce2effd640 1 -- 172.21.15.80:0/4132811368 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce280a5980 con 0x7fce1c005260 2024-04-25T13:46:34.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 -- 172.21.15.80:0/4132811368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce1c005260 msgr2=0x7fce1c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce1c005260 0x7fce1c00b9d0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fce28009f90 tx=0x7fce2801cc80 comp rx=0 tx=0).stop 2024-04-25T13:46:34.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 -- 172.21.15.80:0/4132811368 shutdown_connections 2024-04-25T13:46:34.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 --2- 172.21.15.80:0/4132811368 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce1c005260 0x7fce1c00b9d0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 -- 172.21.15.80:0/4132811368 >> 172.21.15.80:0/4132811368 conn(0x7fce300737a0 msgr2=0x7fce3006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:34.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 -- 172.21.15.80:0/4132811368 shutdown_connections 2024-04-25T13:46:34.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 -- 172.21.15.80:0/4132811368 wait complete. 2024-04-25T13:46:34.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 Processor -- start 2024-04-25T13:46:34.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.871+0000 7fce3641f640 1 -- start start 2024-04-25T13:46:34.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c005260 0x7fce300c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3015a460 0x7fce3015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce3015c470 0x7fce300cdd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce300d3410 con 0x7fce3015c470 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce300d3290 con 0x7fce3015a460 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce3015df60 con 0x7fce1c005260 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce3015c470 0x7fce300cdd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce3015c470 0x7fce300cdd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43062/0 (socket says 172.21.15.80:43062) 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 -- 172.21.15.80:0/2098687435 learned_addr learned my addr 172.21.15.80:0/2098687435 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce2ffff640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c005260 0x7fce300c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce2f7fe640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3015a460 0x7fce3015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 -- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c005260 msgr2=0x7fce300c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c005260 0x7fce300c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 -- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3015a460 msgr2=0x7fce3015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:34.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3015a460 0x7fce3015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:34.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 -- 172.21.15.80:0/2098687435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3015c880 con 0x7fce3015c470 2024-04-25T13:46:34.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce349bc640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce3015c470 0x7fce300cdd10 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fce2801d520 tx=0x7fce28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce2cff9640 1 -- 172.21.15.80:0/2098687435 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28009c10 con 0x7fce3015c470 2024-04-25T13:46:34.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce2cff9640 1 -- 172.21.15.80:0/2098687435 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce280039a0 con 0x7fce3015c470 2024-04-25T13:46:34.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3015b170 con 0x7fce3015c470 2024-04-25T13:46:34.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce2cff9640 1 -- 172.21.15.80:0/2098687435 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28004260 con 0x7fce3015c470 2024-04-25T13:46:34.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce300c9820 con 0x7fce3015c470 2024-04-25T13:46:34.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3006a3a0 con 0x7fce3015c470 2024-04-25T13:46:34.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce2cff9640 1 -- 172.21.15.80:0/2098687435 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fce280bf020 con 0x7fce3015c470 2024-04-25T13:46:34.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.875+0000 7fce2cff9640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce1006f400 0x7fce10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:34.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.879+0000 7fce2cff9640 1 -- 172.21.15.80:0/2098687435 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fce28129a70 con 0x7fce3015c470 2024-04-25T13:46:34.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.879+0000 7fce2ffff640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce1006f400 0x7fce10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:34.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.879+0000 7fce2ffff640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce1006f400 0x7fce10071910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fce20001230 tx=0x7fce200011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:34.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:34.883+0000 7fce2cff9640 1 -- 172.21.15.80:0/2098687435 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce280c6020 con 0x7fce3015c470 2024-04-25T13:46:35.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.027+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fce300bbb90 con 0x7fce1006f400 2024-04-25T13:46:35.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.043+0000 7fce2cff9640 1 -- 172.21.15.80:0/2098687435 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247981 (secure 0 0 0) 0x7fce300bbb90 con 0x7fce1006f400 2024-04-25T13:46:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:35.049 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:46:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce1006f400 msgr2=0x7fce10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce1006f400 0x7fce10071910 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fce20001230 tx=0x7fce200011a0 comp rx=0 tx=0).stop 2024-04-25T13:46:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce3015c470 msgr2=0x7fce300cdd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce3015c470 0x7fce300cdd10 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fce2801d520 tx=0x7fce28009750 comp rx=0 tx=0).stop 2024-04-25T13:46:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 shutdown_connections 2024-04-25T13:46:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce1006f400 0x7fce10071910 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce3015c470 0x7fce300cdd10 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce3015a460 0x7fce3015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 --2- 172.21.15.80:0/2098687435 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce1c005260 0x7fce300c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 >> 172.21.15.80:0/2098687435 conn(0x7fce300737a0 msgr2=0x7fce300757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 shutdown_connections 2024-04-25T13:46:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:35.047+0000 7fce3641f640 1 -- 172.21.15.80:0/2098687435 wait complete. 2024-04-25T13:46:35.066 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-04-25T13:46:34.230768+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556077,"statfs":{"total":1600000000,"available":1593422892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.003275"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588009+0000","last_change":"2024-04-25T13:46:29.379199+0000","last_active":"2024-04-25T13:46:30.588009+0000","last_peered":"2024-04-25T13:46:30.588009+0000","last_clean":"2024-04-25T13:46:30.588009+0000","last_became_active":"2024-04-25T13:46:29.378876+0000","last_became_peered":"2024-04-25T13:46:29.378876+0000","last_unstale":"2024-04-25T13:46:30.588009+0000","last_undegraded":"2024-04-25T13:46:30.588009+0000","last_fullsized":"2024-04-25T13:46:30.588009+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:06.966627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341497+0000","last_change":"2024-04-25T13:46:29.359230+0000","last_active":"2024-04-25T13:46:30.341497+0000","last_peered":"2024-04-25T13:46:30.341497+0000","last_clean":"2024-04-25T13:46:30.341497+0000","last_became_active":"2024-04-25T13:46:29.358723+0000","last_became_peered":"2024-04-25T13:46:29.358723+0000","last_unstale":"2024-04-25T13:46:30.341497+0000","last_undegraded":"2024-04-25T13:46:30.341497+0000","last_fullsized":"2024-04-25T13:46:30.341497+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.753013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.587526+0000","last_change":"2024-04-25T13:46:29.361490+0000","last_active":"2024-04-25T13:46:30.587526+0000","last_peered":"2024-04-25T13:46:30.587526+0000","last_clean":"2024-04-25T13:46:30.587526+0000","last_became_active":"2024-04-25T13:46:29.361188+0000","last_became_peered":"2024-04-25T13:46:29.361188+0000","last_unstale":"2024-04-25T13:46:30.587526+0000","last_undegraded":"2024-04-25T13:46:30.587526+0000","last_fullsized":"2024-04-25T13:46:30.587526+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:44.756958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629462+0000","last_change":"2024-04-25T13:46:29.364994+0000","last_active":"2024-04-25T13:46:31.629462+0000","last_peered":"2024-04-25T13:46:31.629462+0000","last_clean":"2024-04-25T13:46:31.629462+0000","last_became_active":"2024-04-25T13:46:29.364548+0000","last_became_peered":"2024-04-25T13:46:29.364548+0000","last_unstale":"2024-04-25T13:46:31.629462+0000","last_undegraded":"2024-04-25T13:46:31.629462+0000","last_fullsized":"2024-04-25T13:46:31.629462+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:44.339514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621639+0000","last_change":"2024-04-25T13:46:29.389614+0000","last_active":"2024-04-25T13:46:30.621639+0000","last_peered":"2024-04-25T13:46:30.621639+0000","last_clean":"2024-04-25T13:46:30.621639+0000","last_became_active":"2024-04-25T13:46:29.386499+0000","last_became_peered":"2024-04-25T13:46:29.386499+0000","last_unstale":"2024-04-25T13:46:30.621639+0000","last_undegraded":"2024-04-25T13:46:30.621639+0000","last_fullsized":"2024-04-25T13:46:30.621639+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:17.092243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629561+0000","last_change":"2024-04-25T13:46:29.386033+0000","last_active":"2024-04-25T13:46:31.629561+0000","last_peered":"2024-04-25T13:46:31.629561+0000","last_clean":"2024-04-25T13:46:31.629561+0000","last_became_active":"2024-04-25T13:46:29.385379+0000","last_became_peered":"2024-04-25T13:46:29.385379+0000","last_unstale":"2024-04-25T13:46:31.629561+0000","last_undegraded":"2024-04-25T13:46:31.629561+0000","last_fullsized":"2024-04-25T13:46:31.629561+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:08.249909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629289+0000","last_change":"2024-04-25T13:46:29.385575+0000","last_active":"2024-04-25T13:46:31.629289+0000","last_peered":"2024-04-25T13:46:31.629289+0000","last_clean":"2024-04-25T13:46:31.629289+0000","last_became_active":"2024-04-25T13:46:29.363594+0000","last_became_peered":"2024-04-25T13:46:29.363594+0000","last_unstale":"2024-04-25T13:46:31.629289+0000","last_undegraded":"2024-04-25T13:46:31.629289+0000","last_fullsized":"2024-04-25T13:46:31.629289+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:00.275547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341407+0000","last_change":"2024-04-25T13:46:29.393782+0000","last_active":"2024-04-25T13:46:30.341407+0000","last_peered":"2024-04-25T13:46:30.341407+0000","last_clean":"2024-04-25T13:46:30.341407+0000","last_became_active":"2024-04-25T13:46:29.393340+0000","last_became_peered":"2024-04-25T13:46:29.393340+0000","last_unstale":"2024-04-25T13:46:30.341407+0000","last_undegraded":"2024-04-25T13:46:30.341407+0000","last_fullsized":"2024-04-25T13:46:30.341407+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:32.074485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341715+0000","last_change":"2024-04-25T13:46:29.377607+0000","last_active":"2024-04-25T13:46:30.341715+0000","last_peered":"2024-04-25T13:46:30.341715+0000","last_clean":"2024-04-25T13:46:30.341715+0000","last_became_active":"2024-04-25T13:46:29.360329+0000","last_became_peered":"2024-04-25T13:46:29.360329+0000","last_unstale":"2024-04-25T13:46:30.341715+0000","last_undegraded":"2024-04-25T13:46:30.341715+0000","last_fullsized":"2024-04-25T13:46:30.341715+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:50.475932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621917+0000","last_change":"2024-04-25T13:46:29.388662+0000","last_active":"2024-04-25T13:46:30.621917+0000","last_peered":"2024-04-25T13:46:30.621917+0000","last_clean":"2024-04-25T13:46:30.621917+0000","last_became_active":"2024-04-25T13:46:29.384118+0000","last_became_peered":"2024-04-25T13:46:29.384118+0000","last_unstale":"2024-04-25T13:46:30.621917+0000","last_undegraded":"2024-04-25T13:46:30.621917+0000","last_fullsized":"2024-04-25T13:46:30.621917+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:41.484705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630039+0000","last_change":"2024-04-25T13:46:29.370803+0000","last_active":"2024-04-25T13:46:31.630039+0000","last_peered":"2024-04-25T13:46:31.630039+0000","last_clean":"2024-04-25T13:46:31.630039+0000","last_became_active":"2024-04-25T13:46:29.369106+0000","last_became_peered":"2024-04-25T13:46:29.369106+0000","last_unstale":"2024-04-25T13:46:31.630039+0000","last_undegraded":"2024-04-25T13:46:31.630039+0000","last_fullsized":"2024-04-25T13:46:31.630039+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:59.533887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342438+0000","last_change":"2024-04-25T13:46:29.357312+0000","last_active":"2024-04-25T13:46:30.342438+0000","last_peered":"2024-04-25T13:46:30.342438+0000","last_clean":"2024-04-25T13:46:30.342438+0000","last_became_active":"2024-04-25T13:46:29.353270+0000","last_became_peered":"2024-04-25T13:46:29.353270+0000","last_unstale":"2024-04-25T13:46:30.342438+0000","last_undegraded":"2024-04-25T13:46:30.342438+0000","last_fullsized":"2024-04-25T13:46:30.342438+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:52.237940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629613+0000","last_change":"2024-04-25T13:46:29.355819+0000","last_active":"2024-04-25T13:46:31.629613+0000","last_peered":"2024-04-25T13:46:31.629613+0000","last_clean":"2024-04-25T13:46:31.629613+0000","last_became_active":"2024-04-25T13:46:29.355353+0000","last_became_peered":"2024-04-25T13:46:29.355353+0000","last_unstale":"2024-04-25T13:46:31.629613+0000","last_undegraded":"2024-04-25T13:46:31.629613+0000","last_fullsized":"2024-04-25T13:46:31.629613+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:00.696939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.346612+0000","last_change":"2024-04-25T13:46:29.360510+0000","last_active":"2024-04-25T13:46:31.346612+0000","last_peered":"2024-04-25T13:46:31.346612+0000","last_clean":"2024-04-25T13:46:31.346612+0000","last_became_active":"2024-04-25T13:46:29.359901+0000","last_became_peered":"2024-04-25T13:46:29.359901+0000","last_unstale":"2024-04-25T13:46:31.346612+0000","last_undegraded":"2024-04-25T13:46:31.346612+0000","last_fullsized":"2024-04-25T13:46:31.346612+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:29.293066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629944+0000","last_change":"2024-04-25T13:46:29.356612+0000","last_active":"2024-04-25T13:46:31.629944+0000","last_peered":"2024-04-25T13:46:31.629944+0000","last_clean":"2024-04-25T13:46:31.629944+0000","last_became_active":"2024-04-25T13:46:29.353061+0000","last_became_peered":"2024-04-25T13:46:29.353061+0000","last_unstale":"2024-04-25T13:46:31.629944+0000","last_undegraded":"2024-04-25T13:46:31.629944+0000","last_fullsized":"2024-04-25T13:46:31.629944+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:16.514862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630367+0000","last_change":"2024-04-25T13:46:29.354914+0000","last_active":"2024-04-25T13:46:31.630367+0000","last_peered":"2024-04-25T13:46:31.630367+0000","last_clean":"2024-04-25T13:46:31.630367+0000","last_became_active":"2024-04-25T13:46:29.354099+0000","last_became_peered":"2024-04-25T13:46:29.354099+0000","last_unstale":"2024-04-25T13:46:31.630367+0000","last_undegraded":"2024-04-25T13:46:31.630367+0000","last_fullsized":"2024-04-25T13:46:31.630367+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:54.809032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341960+0000","last_change":"2024-04-25T13:46:29.348116+0000","last_active":"2024-04-25T13:46:30.341960+0000","last_peered":"2024-04-25T13:46:30.341960+0000","last_clean":"2024-04-25T13:46:30.341960+0000","last_became_active":"2024-04-25T13:46:29.347746+0000","last_became_peered":"2024-04-25T13:46:29.347746+0000","last_unstale":"2024-04-25T13:46:30.341960+0000","last_undegraded":"2024-04-25T13:46:30.341960+0000","last_fullsized":"2024-04-25T13:46:30.341960+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:01.615426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629875+0000","last_change":"2024-04-25T13:46:29.355605+0000","last_active":"2024-04-25T13:46:31.629875+0000","last_peered":"2024-04-25T13:46:31.629875+0000","last_clean":"2024-04-25T13:46:31.629875+0000","last_became_active":"2024-04-25T13:46:29.354988+0000","last_became_peered":"2024-04-25T13:46:29.354988+0000","last_unstale":"2024-04-25T13:46:31.629875+0000","last_undegraded":"2024-04-25T13:46:31.629875+0000","last_fullsized":"2024-04-25T13:46:31.629875+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:29.248386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347108+0000","last_change":"2024-04-25T13:46:29.360303+0000","last_active":"2024-04-25T13:46:31.347108+0000","last_peered":"2024-04-25T13:46:31.347108+0000","last_clean":"2024-04-25T13:46:31.347108+0000","last_became_active":"2024-04-25T13:46:29.359458+0000","last_became_peered":"2024-04-25T13:46:29.359458+0000","last_unstale":"2024-04-25T13:46:31.347108+0000","last_undegraded":"2024-04-25T13:46:31.347108+0000","last_fullsized":"2024-04-25T13:46:31.347108+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:28.949462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343455+0000","last_change":"2024-04-25T13:46:29.357541+0000","last_active":"2024-04-25T13:46:30.343455+0000","last_peered":"2024-04-25T13:46:30.343455+0000","last_clean":"2024-04-25T13:46:30.343455+0000","last_became_active":"2024-04-25T13:46:29.355431+0000","last_became_peered":"2024-04-25T13:46:29.355431+0000","last_unstale":"2024-04-25T13:46:30.343455+0000","last_undegraded":"2024-04-25T13:46:30.343455+0000","last_fullsized":"2024-04-25T13:46:30.343455+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:28.115823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.346538+0000","last_change":"2024-04-25T13:46:29.379316+0000","last_active":"2024-04-25T13:46:31.346538+0000","last_peered":"2024-04-25T13:46:31.346538+0000","last_clean":"2024-04-25T13:46:31.346538+0000","last_became_active":"2024-04-25T13:46:29.379046+0000","last_became_peered":"2024-04-25T13:46:29.379046+0000","last_unstale":"2024-04-25T13:46:31.346538+0000","last_undegraded":"2024-04-25T13:46:31.346538+0000","last_fullsized":"2024-04-25T13:46:31.346538+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:50.935327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342819+0000","last_change":"2024-04-25T13:46:29.358988+0000","last_active":"2024-04-25T13:46:30.342819+0000","last_peered":"2024-04-25T13:46:30.342819+0000","last_clean":"2024-04-25T13:46:30.342819+0000","last_became_active":"2024-04-25T13:46:29.358338+0000","last_became_peered":"2024-04-25T13:46:29.358338+0000","last_unstale":"2024-04-25T13:46:30.342819+0000","last_undegraded":"2024-04-25T13:46:30.342819+0000","last_fullsized":"2024-04-25T13:46:30.342819+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:57.359133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341877+0000","last_change":"2024-04-25T13:46:29.360931+0000","last_active":"2024-04-25T13:46:30.341877+0000","last_peered":"2024-04-25T13:46:30.341877+0000","last_clean":"2024-04-25T13:46:30.341877+0000","last_became_active":"2024-04-25T13:46:29.360432+0000","last_became_peered":"2024-04-25T13:46:29.360432+0000","last_unstale":"2024-04-25T13:46:30.341877+0000","last_undegraded":"2024-04-25T13:46:30.341877+0000","last_fullsized":"2024-04-25T13:46:30.341877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:35.622752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629283+0000","last_change":"2024-04-25T13:46:29.365656+0000","last_active":"2024-04-25T13:46:31.629283+0000","last_peered":"2024-04-25T13:46:31.629283+0000","last_clean":"2024-04-25T13:46:31.629283+0000","last_became_active":"2024-04-25T13:46:29.364664+0000","last_became_peered":"2024-04-25T13:46:29.364664+0000","last_unstale":"2024-04-25T13:46:31.629283+0000","last_undegraded":"2024-04-25T13:46:31.629283+0000","last_fullsized":"2024-04-25T13:46:31.629283+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:30.471600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622149+0000","last_change":"2024-04-25T13:46:29.389093+0000","last_active":"2024-04-25T13:46:30.622149+0000","last_peered":"2024-04-25T13:46:30.622149+0000","last_clean":"2024-04-25T13:46:30.622149+0000","last_became_active":"2024-04-25T13:46:29.385645+0000","last_became_peered":"2024-04-25T13:46:29.385645+0000","last_unstale":"2024-04-25T13:46:30.622149+0000","last_undegraded":"2024-04-25T13:46:30.622149+0000","last_fullsized":"2024-04-25T13:46:30.622149+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:13.995233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341597+0000","last_change":"2024-04-25T13:46:29.350360+0000","last_active":"2024-04-25T13:46:30.341597+0000","last_peered":"2024-04-25T13:46:30.341597+0000","last_clean":"2024-04-25T13:46:30.341597+0000","last_became_active":"2024-04-25T13:46:29.350137+0000","last_became_peered":"2024-04-25T13:46:29.350137+0000","last_unstale":"2024-04-25T13:46:30.341597+0000","last_undegraded":"2024-04-25T13:46:30.341597+0000","last_fullsized":"2024-04-25T13:46:30.341597+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:11.315733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629560+0000","last_change":"2024-04-25T13:46:29.364445+0000","last_active":"2024-04-25T13:46:31.629560+0000","last_peered":"2024-04-25T13:46:31.629560+0000","last_clean":"2024-04-25T13:46:31.629560+0000","last_became_active":"2024-04-25T13:46:29.361873+0000","last_became_peered":"2024-04-25T13:46:29.361873+0000","last_unstale":"2024-04-25T13:46:31.629560+0000","last_undegraded":"2024-04-25T13:46:31.629560+0000","last_fullsized":"2024-04-25T13:46:31.629560+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:52.572388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622194+0000","last_change":"2024-04-25T13:46:29.372301+0000","last_active":"2024-04-25T13:46:30.622194+0000","last_peered":"2024-04-25T13:46:30.622194+0000","last_clean":"2024-04-25T13:46:30.622194+0000","last_became_active":"2024-04-25T13:46:29.370609+0000","last_became_peered":"2024-04-25T13:46:29.370609+0000","last_unstale":"2024-04-25T13:46:30.622194+0000","last_undegraded":"2024-04-25T13:46:30.622194+0000","last_fullsized":"2024-04-25T13:46:30.622194+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:35.503611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.340859+0000","last_change":"2024-04-25T13:46:29.347300+0000","last_active":"2024-04-25T13:46:30.340859+0000","last_peered":"2024-04-25T13:46:30.340859+0000","last_clean":"2024-04-25T13:46:30.340859+0000","last_became_active":"2024-04-25T13:46:29.347044+0000","last_became_peered":"2024-04-25T13:46:29.347044+0000","last_unstale":"2024-04-25T13:46:30.340859+0000","last_undegraded":"2024-04-25T13:46:30.340859+0000","last_fullsized":"2024-04-25T13:46:30.340859+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.145655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347498+0000","last_change":"2024-04-25T13:46:29.354560+0000","last_active":"2024-04-25T13:46:31.347498+0000","last_peered":"2024-04-25T13:46:31.347498+0000","last_clean":"2024-04-25T13:46:31.347498+0000","last_became_active":"2024-04-25T13:46:29.354213+0000","last_became_peered":"2024-04-25T13:46:29.354213+0000","last_unstale":"2024-04-25T13:46:31.347498+0000","last_undegraded":"2024-04-25T13:46:31.347498+0000","last_fullsized":"2024-04-25T13:46:31.347498+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:50.021364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629701+0000","last_change":"2024-04-25T13:46:29.362585+0000","last_active":"2024-04-25T13:46:31.629701+0000","last_peered":"2024-04-25T13:46:31.629701+0000","last_clean":"2024-04-25T13:46:31.629701+0000","last_became_active":"2024-04-25T13:46:29.362356+0000","last_became_peered":"2024-04-25T13:46:29.362356+0000","last_unstale":"2024-04-25T13:46:31.629701+0000","last_undegraded":"2024-04-25T13:46:31.629701+0000","last_fullsized":"2024-04-25T13:46:31.629701+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:36.383874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588071+0000","last_change":"2024-04-25T13:46:29.349095+0000","last_active":"2024-04-25T13:46:30.588071+0000","last_peered":"2024-04-25T13:46:30.588071+0000","last_clean":"2024-04-25T13:46:30.588071+0000","last_became_active":"2024-04-25T13:46:29.348721+0000","last_became_peered":"2024-04-25T13:46:29.348721+0000","last_unstale":"2024-04-25T13:46:30.588071+0000","last_undegraded":"2024-04-25T13:46:30.588071+0000","last_fullsized":"2024-04-25T13:46:30.588071+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:47.534623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347543+0000","last_change":"2024-04-25T13:46:29.367798+0000","last_active":"2024-04-25T13:46:31.347543+0000","last_peered":"2024-04-25T13:46:31.347543+0000","last_clean":"2024-04-25T13:46:31.347543+0000","last_became_active":"2024-04-25T13:46:29.367627+0000","last_became_peered":"2024-04-25T13:46:29.367627+0000","last_unstale":"2024-04-25T13:46:31.347543+0000","last_undegraded":"2024-04-25T13:46:31.347543+0000","last_fullsized":"2024-04-25T13:46:31.347543+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.874484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"121'4","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342643+0000","last_change":"2024-04-25T13:46:28.357817+0000","last_active":"2024-04-25T13:46:30.342643+0000","last_peered":"2024-04-25T13:46:30.342643+0000","last_clean":"2024-04-25T13:46:30.342643+0000","last_became_active":"2024-04-25T13:46:28.356033+0000","last_became_peered":"2024-04-25T13:46:28.356033+0000","last_unstale":"2024-04-25T13:46:30.342643+0000","last_undegraded":"2024-04-25T13:46:30.342643+0000","last_fullsized":"2024-04-25T13:46:30.342643+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:40:32.433741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630108+0000","last_change":"2024-04-25T13:46:29.365988+0000","last_active":"2024-04-25T13:46:31.630108+0000","last_peered":"2024-04-25T13:46:31.630108+0000","last_clean":"2024-04-25T13:46:31.630108+0000","last_became_active":"2024-04-25T13:46:29.365721+0000","last_became_peered":"2024-04-25T13:46:29.365721+0000","last_unstale":"2024-04-25T13:46:31.630108+0000","last_undegraded":"2024-04-25T13:46:31.630108+0000","last_fullsized":"2024-04-25T13:46:31.630108+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:18.995871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629398+0000","last_change":"2024-04-25T13:46:28.364069+0000","last_active":"2024-04-25T13:46:31.629398+0000","last_peered":"2024-04-25T13:46:31.629398+0000","last_clean":"2024-04-25T13:46:31.629398+0000","last_became_active":"2024-04-25T13:46:28.363317+0000","last_became_peered":"2024-04-25T13:46:28.363317+0000","last_unstale":"2024-04-25T13:46:31.629398+0000","last_undegraded":"2024-04-25T13:46:31.629398+0000","last_fullsized":"2024-04-25T13:46:31.629398+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:38.581336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588260+0000","last_change":"2024-04-25T13:46:29.352272+0000","last_active":"2024-04-25T13:46:30.588260+0000","last_peered":"2024-04-25T13:46:30.588260+0000","last_clean":"2024-04-25T13:46:30.588260+0000","last_became_active":"2024-04-25T13:46:29.351906+0000","last_became_peered":"2024-04-25T13:46:29.351906+0000","last_unstale":"2024-04-25T13:46:30.588260+0000","last_undegraded":"2024-04-25T13:46:30.588260+0000","last_fullsized":"2024-04-25T13:46:30.588260+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:09.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341444+0000","last_change":"2024-04-25T13:46:28.367229+0000","last_active":"2024-04-25T13:46:30.341444+0000","last_peered":"2024-04-25T13:46:30.341444+0000","last_clean":"2024-04-25T13:46:30.341444+0000","last_became_active":"2024-04-25T13:46:28.366847+0000","last_became_peered":"2024-04-25T13:46:28.366847+0000","last_unstale":"2024-04-25T13:46:30.341444+0000","last_undegraded":"2024-04-25T13:46:30.341444+0000","last_fullsized":"2024-04-25T13:46:30.341444+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:33.018943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343705+0000","last_change":"2024-04-25T13:46:29.365163+0000","last_active":"2024-04-25T13:46:30.343705+0000","last_peered":"2024-04-25T13:46:30.343705+0000","last_clean":"2024-04-25T13:46:30.343705+0000","last_became_active":"2024-04-25T13:46:29.363861+0000","last_became_peered":"2024-04-25T13:46:29.363861+0000","last_unstale":"2024-04-25T13:46:30.343705+0000","last_undegraded":"2024-04-25T13:46:30.343705+0000","last_fullsized":"2024-04-25T13:46:30.343705+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:37.043886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341729+0000","last_change":"2024-04-25T13:46:28.365386+0000","last_active":"2024-04-25T13:46:30.341729+0000","last_peered":"2024-04-25T13:46:30.341729+0000","last_clean":"2024-04-25T13:46:30.341729+0000","last_became_active":"2024-04-25T13:46:28.363348+0000","last_became_peered":"2024-04-25T13:46:28.363348+0000","last_unstale":"2024-04-25T13:46:30.341729+0000","last_undegraded":"2024-04-25T13:46:30.341729+0000","last_fullsized":"2024-04-25T13:46:30.341729+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:34.461750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.394721+0000","last_change":"2024-04-25T13:46:29.378357+0000","last_active":"2024-04-25T13:46:31.394721+0000","last_peered":"2024-04-25T13:46:31.394721+0000","last_clean":"2024-04-25T13:46:31.394721+0000","last_became_active":"2024-04-25T13:46:29.376081+0000","last_became_peered":"2024-04-25T13:46:29.376081+0000","last_unstale":"2024-04-25T13:46:31.394721+0000","last_undegraded":"2024-04-25T13:46:31.394721+0000","last_fullsized":"2024-04-25T13:46:31.394721+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:43.188032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588498+0000","last_change":"2024-04-25T13:46:28.361416+0000","last_active":"2024-04-25T13:46:30.588498+0000","last_peered":"2024-04-25T13:46:30.588498+0000","last_clean":"2024-04-25T13:46:30.588498+0000","last_became_active":"2024-04-25T13:46:28.361020+0000","last_became_peered":"2024-04-25T13:46:28.361020+0000","last_unstale":"2024-04-25T13:46:30.588498+0000","last_undegraded":"2024-04-25T13:46:30.588498+0000","last_fullsized":"2024-04-25T13:46:30.588498+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:54.615568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629824+0000","last_change":"2024-04-25T13:46:29.386574+0000","last_active":"2024-04-25T13:46:31.629824+0000","last_peered":"2024-04-25T13:46:31.629824+0000","last_clean":"2024-04-25T13:46:31.629824+0000","last_became_active":"2024-04-25T13:46:29.370737+0000","last_became_peered":"2024-04-25T13:46:29.370737+0000","last_unstale":"2024-04-25T13:46:31.629824+0000","last_undegraded":"2024-04-25T13:46:31.629824+0000","last_fullsized":"2024-04-25T13:46:31.629824+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:29.854734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342953+0000","last_change":"2024-04-25T13:46:28.364545+0000","last_active":"2024-04-25T13:46:30.342953+0000","last_peered":"2024-04-25T13:46:30.342953+0000","last_clean":"2024-04-25T13:46:30.342953+0000","last_became_active":"2024-04-25T13:46:28.364139+0000","last_became_peered":"2024-04-25T13:46:28.364139+0000","last_unstale":"2024-04-25T13:46:30.342953+0000","last_undegraded":"2024-04-25T13:46:30.342953+0000","last_fullsized":"2024-04-25T13:46:30.342953+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:48.068211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630370+0000","last_change":"2024-04-25T13:46:29.353318+0000","last_active":"2024-04-25T13:46:31.630370+0000","last_peered":"2024-04-25T13:46:31.630370+0000","last_clean":"2024-04-25T13:46:31.630370+0000","last_became_active":"2024-04-25T13:46:29.352744+0000","last_became_peered":"2024-04-25T13:46:29.352744+0000","last_unstale":"2024-04-25T13:46:31.630370+0000","last_undegraded":"2024-04-25T13:46:31.630370+0000","last_fullsized":"2024-04-25T13:46:31.630370+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:54.802572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622750+0000","last_change":"2024-04-25T13:46:28.363100+0000","last_active":"2024-04-25T13:46:30.622750+0000","last_peered":"2024-04-25T13:46:30.622750+0000","last_clean":"2024-04-25T13:46:30.622750+0000","last_became_active":"2024-04-25T13:46:28.361935+0000","last_became_peered":"2024-04-25T13:46:28.361935+0000","last_unstale":"2024-04-25T13:46:30.622750+0000","last_undegraded":"2024-04-25T13:46:30.622750+0000","last_fullsized":"2024-04-25T13:46:30.622750+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:29.648109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342199+0000","last_change":"2024-04-25T13:46:29.382181+0000","last_active":"2024-04-25T13:46:30.342199+0000","last_peered":"2024-04-25T13:46:30.342199+0000","last_clean":"2024-04-25T13:46:30.342199+0000","last_became_active":"2024-04-25T13:46:29.376621+0000","last_became_peered":"2024-04-25T13:46:29.376621+0000","last_unstale":"2024-04-25T13:46:30.342199+0000","last_undegraded":"2024-04-25T13:46:30.342199+0000","last_fullsized":"2024-04-25T13:46:30.342199+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:19.632433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.589048+0000","last_change":"2024-04-25T13:46:28.354914+0000","last_active":"2024-04-25T13:46:30.589048+0000","last_peered":"2024-04-25T13:46:30.589048+0000","last_clean":"2024-04-25T13:46:30.589048+0000","last_became_active":"2024-04-25T13:46:28.350518+0000","last_became_peered":"2024-04-25T13:46:28.350518+0000","last_unstale":"2024-04-25T13:46:30.589048+0000","last_undegraded":"2024-04-25T13:46:30.589048+0000","last_fullsized":"2024-04-25T13:46:30.589048+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:25.118939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342382+0000","last_change":"2024-04-25T13:46:29.346398+0000","last_active":"2024-04-25T13:46:30.342382+0000","last_peered":"2024-04-25T13:46:30.342382+0000","last_clean":"2024-04-25T13:46:30.342382+0000","last_became_active":"2024-04-25T13:46:29.346142+0000","last_became_peered":"2024-04-25T13:46:29.346142+0000","last_unstale":"2024-04-25T13:46:30.342382+0000","last_undegraded":"2024-04-25T13:46:30.342382+0000","last_fullsized":"2024-04-25T13:46:30.342382+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:29.131276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588444+0000","last_change":"2024-04-25T13:46:28.341599+0000","last_active":"2024-04-25T13:46:30.588444+0000","last_peered":"2024-04-25T13:46:30.588444+0000","last_clean":"2024-04-25T13:46:30.588444+0000","last_became_active":"2024-04-25T13:46:28.341326+0000","last_became_peered":"2024-04-25T13:46:28.341326+0000","last_unstale":"2024-04-25T13:46:30.588444+0000","last_undegraded":"2024-04-25T13:46:30.588444+0000","last_fullsized":"2024-04-25T13:46:30.588444+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:34.794066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342721+0000","last_change":"2024-04-25T13:46:29.381753+0000","last_active":"2024-04-25T13:46:30.342721+0000","last_peered":"2024-04-25T13:46:30.342721+0000","last_clean":"2024-04-25T13:46:30.342721+0000","last_became_active":"2024-04-25T13:46:29.381001+0000","last_became_peered":"2024-04-25T13:46:29.381001+0000","last_unstale":"2024-04-25T13:46:30.342721+0000","last_undegraded":"2024-04-25T13:46:30.342721+0000","last_fullsized":"2024-04-25T13:46:30.342721+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:02.375916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.628967+0000","last_change":"2024-04-25T13:46:28.347431+0000","last_active":"2024-04-25T13:46:31.628967+0000","last_peered":"2024-04-25T13:46:31.628967+0000","last_clean":"2024-04-25T13:46:31.628967+0000","last_became_active":"2024-04-25T13:46:28.346978+0000","last_became_peered":"2024-04-25T13:46:28.346978+0000","last_unstale":"2024-04-25T13:46:31.628967+0000","last_undegraded":"2024-04-25T13:46:31.628967+0000","last_fullsized":"2024-04-25T13:46:31.628967+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:49.443864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621668+0000","last_change":"2024-04-25T13:46:29.360205+0000","last_active":"2024-04-25T13:46:30.621668+0000","last_peered":"2024-04-25T13:46:30.621668+0000","last_clean":"2024-04-25T13:46:30.621668+0000","last_became_active":"2024-04-25T13:46:29.359472+0000","last_became_peered":"2024-04-25T13:46:29.359472+0000","last_unstale":"2024-04-25T13:46:30.621668+0000","last_undegraded":"2024-04-25T13:46:30.621668+0000","last_fullsized":"2024-04-25T13:46:30.621668+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:45.111194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"121'3","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629950+0000","last_change":"2024-04-25T13:46:28.366062+0000","last_active":"2024-04-25T13:46:31.629950+0000","last_peered":"2024-04-25T13:46:31.629950+0000","last_clean":"2024-04-25T13:46:31.629950+0000","last_became_active":"2024-04-25T13:46:28.365299+0000","last_became_peered":"2024-04-25T13:46:28.365299+0000","last_unstale":"2024-04-25T13:46:31.629950+0000","last_undegraded":"2024-04-25T13:46:31.629950+0000","last_fullsized":"2024-04-25T13:46:31.629950+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:36.253244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629540+0000","last_change":"2024-04-25T13:46:29.392343+0000","last_active":"2024-04-25T13:46:31.629540+0000","last_peered":"2024-04-25T13:46:31.629540+0000","last_clean":"2024-04-25T13:46:31.629540+0000","last_became_active":"2024-04-25T13:46:29.391955+0000","last_became_peered":"2024-04-25T13:46:29.391955+0000","last_unstale":"2024-04-25T13:46:31.629540+0000","last_undegraded":"2024-04-25T13:46:31.629540+0000","last_fullsized":"2024-04-25T13:46:31.629540+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:30.980780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"121'6","reported_seq":20,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342609+0000","last_change":"2024-04-25T13:46:28.356745+0000","last_active":"2024-04-25T13:46:30.342609+0000","last_peered":"2024-04-25T13:46:30.342609+0000","last_clean":"2024-04-25T13:46:30.342609+0000","last_became_active":"2024-04-25T13:46:28.355794+0000","last_became_peered":"2024-04-25T13:46:28.355794+0000","last_unstale":"2024-04-25T13:46:30.342609+0000","last_undegraded":"2024-04-25T13:46:30.342609+0000","last_fullsized":"2024-04-25T13:46:30.342609+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:06.865488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629235+0000","last_change":"2024-04-25T13:46:29.354533+0000","last_active":"2024-04-25T13:46:31.629235+0000","last_peered":"2024-04-25T13:46:31.629235+0000","last_clean":"2024-04-25T13:46:31.629235+0000","last_became_active":"2024-04-25T13:46:29.354309+0000","last_became_peered":"2024-04-25T13:46:29.354309+0000","last_unstale":"2024-04-25T13:46:31.629235+0000","last_undegraded":"2024-04-25T13:46:31.629235+0000","last_fullsized":"2024-04-25T13:46:31.629235+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:38.456980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.348038+0000","last_change":"2024-04-25T13:46:28.362182+0000","last_active":"2024-04-25T13:46:31.348038+0000","last_peered":"2024-04-25T13:46:31.348038+0000","last_clean":"2024-04-25T13:46:31.348038+0000","last_became_active":"2024-04-25T13:46:28.361659+0000","last_became_peered":"2024-04-25T13:46:28.361659+0000","last_unstale":"2024-04-25T13:46:31.348038+0000","last_undegraded":"2024-04-25T13:46:31.348038+0000","last_fullsized":"2024-04-25T13:46:31.348038+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:06.649906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588747+0000","last_change":"2024-04-25T13:46:29.362173+0000","last_active":"2024-04-25T13:46:30.588747+0000","last_peered":"2024-04-25T13:46:30.588747+0000","last_clean":"2024-04-25T13:46:30.588747+0000","last_became_active":"2024-04-25T13:46:29.361877+0000","last_became_peered":"2024-04-25T13:46:29.361877+0000","last_unstale":"2024-04-25T13:46:30.588747+0000","last_undegraded":"2024-04-25T13:46:30.588747+0000","last_fullsized":"2024-04-25T13:46:30.588747+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:52.608640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347194+0000","last_change":"2024-04-25T13:46:28.355020+0000","last_active":"2024-04-25T13:46:31.347194+0000","last_peered":"2024-04-25T13:46:31.347194+0000","last_clean":"2024-04-25T13:46:31.347194+0000","last_became_active":"2024-04-25T13:46:28.354039+0000","last_became_peered":"2024-04-25T13:46:28.354039+0000","last_unstale":"2024-04-25T13:46:31.347194+0000","last_undegraded":"2024-04-25T13:46:31.347194+0000","last_fullsized":"2024-04-25T13:46:31.347194+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:28.804890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342231+0000","last_change":"2024-04-25T13:46:29.362689+0000","last_active":"2024-04-25T13:46:30.342231+0000","last_peered":"2024-04-25T13:46:30.342231+0000","last_clean":"2024-04-25T13:46:30.342231+0000","last_became_active":"2024-04-25T13:46:29.359630+0000","last_became_peered":"2024-04-25T13:46:29.359630+0000","last_unstale":"2024-04-25T13:46:30.342231+0000","last_undegraded":"2024-04-25T13:46:30.342231+0000","last_fullsized":"2024-04-25T13:46:30.342231+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.678172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342682+0000","last_change":"2024-04-25T13:46:28.367780+0000","last_active":"2024-04-25T13:46:30.342682+0000","last_peered":"2024-04-25T13:46:30.342682+0000","last_clean":"2024-04-25T13:46:30.342682+0000","last_became_active":"2024-04-25T13:46:28.366553+0000","last_became_peered":"2024-04-25T13:46:28.366553+0000","last_unstale":"2024-04-25T13:46:30.342682+0000","last_undegraded":"2024-04-25T13:46:30.342682+0000","last_fullsized":"2024-04-25T13:46:30.342682+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:41.580565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347017+0000","last_change":"2024-04-25T13:46:29.366585+0000","last_active":"2024-04-25T13:46:31.347017+0000","last_peered":"2024-04-25T13:46:31.347017+0000","last_clean":"2024-04-25T13:46:31.347017+0000","last_became_active":"2024-04-25T13:46:29.366210+0000","last_became_peered":"2024-04-25T13:46:29.366210+0000","last_unstale":"2024-04-25T13:46:31.347017+0000","last_undegraded":"2024-04-25T13:46:31.347017+0000","last_fullsized":"2024-04-25T13:46:31.347017+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:41.918881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"121'8","reported_seq":22,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588045+0000","last_change":"2024-04-25T13:46:28.368314+0000","last_active":"2024-04-25T13:46:30.588045+0000","last_peered":"2024-04-25T13:46:30.588045+0000","last_clean":"2024-04-25T13:46:30.588045+0000","last_became_active":"2024-04-25T13:46:28.368036+0000","last_became_peered":"2024-04-25T13:46:28.368036+0000","last_unstale":"2024-04-25T13:46:30.588045+0000","last_undegraded":"2024-04-25T13:46:30.588045+0000","last_fullsized":"2024-04-25T13:46:30.588045+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.384214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622178+0000","last_change":"2024-04-25T13:46:29.382594+0000","last_active":"2024-04-25T13:46:30.622178+0000","last_peered":"2024-04-25T13:46:30.622178+0000","last_clean":"2024-04-25T13:46:30.622178+0000","last_became_active":"2024-04-25T13:46:29.382014+0000","last_became_peered":"2024-04-25T13:46:29.382014+0000","last_unstale":"2024-04-25T13:46:30.622178+0000","last_undegraded":"2024-04-25T13:46:30.622178+0000","last_fullsized":"2024-04-25T13:46:30.622178+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:43.814045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343799+0000","last_change":"2024-04-25T13:46:28.365770+0000","last_active":"2024-04-25T13:46:30.343799+0000","last_peered":"2024-04-25T13:46:30.343799+0000","last_clean":"2024-04-25T13:46:30.343799+0000","last_became_active":"2024-04-25T13:46:28.358806+0000","last_became_peered":"2024-04-25T13:46:28.358806+0000","last_unstale":"2024-04-25T13:46:30.343799+0000","last_undegraded":"2024-04-25T13:46:30.343799+0000","last_fullsized":"2024-04-25T13:46:30.343799+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:38.295721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.394593+0000","last_change":"2024-04-25T13:46:29.369548+0000","last_active":"2024-04-25T13:46:31.394593+0000","last_peered":"2024-04-25T13:46:31.394593+0000","last_clean":"2024-04-25T13:46:31.394593+0000","last_became_active":"2024-04-25T13:46:29.367940+0000","last_became_peered":"2024-04-25T13:46:29.367940+0000","last_unstale":"2024-04-25T13:46:31.394593+0000","last_undegraded":"2024-04-25T13:46:31.394593+0000","last_fullsized":"2024-04-25T13:46:31.394593+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:02.738864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621689+0000","last_change":"2024-04-25T13:46:28.350352+0000","last_active":"2024-04-25T13:46:30.621689+0000","last_peered":"2024-04-25T13:46:30.621689+0000","last_clean":"2024-04-25T13:46:30.621689+0000","last_became_active":"2024-04-25T13:46:28.349684+0000","last_became_peered":"2024-04-25T13:46:28.349684+0000","last_unstale":"2024-04-25T13:46:30.621689+0000","last_undegraded":"2024-04-25T13:46:30.621689+0000","last_fullsized":"2024-04-25T13:46:30.621689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:59.478011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630602+0000","last_change":"2024-04-25T13:46:29.384622+0000","last_active":"2024-04-25T13:46:31.630602+0000","last_peered":"2024-04-25T13:46:31.630602+0000","last_clean":"2024-04-25T13:46:31.630602+0000","last_became_active":"2024-04-25T13:46:29.370079+0000","last_became_peered":"2024-04-25T13:46:29.370079+0000","last_unstale":"2024-04-25T13:46:31.630602+0000","last_undegraded":"2024-04-25T13:46:31.630602+0000","last_fullsized":"2024-04-25T13:46:31.630602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:53.778017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343129+0000","last_change":"2024-04-25T13:46:28.369672+0000","last_active":"2024-04-25T13:46:30.343129+0000","last_peered":"2024-04-25T13:46:30.343129+0000","last_clean":"2024-04-25T13:46:30.343129+0000","last_became_active":"2024-04-25T13:46:28.368816+0000","last_became_peered":"2024-04-25T13:46:28.368816+0000","last_unstale":"2024-04-25T13:46:30.343129+0000","last_undegraded":"2024-04-25T13:46:30.343129+0000","last_fullsized":"2024-04-25T13:46:30.343129+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:44.300220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347950+0000","last_change":"2024-04-25T13:46:29.360114+0000","last_active":"2024-04-25T13:46:31.347950+0000","last_peered":"2024-04-25T13:46:31.347950+0000","last_clean":"2024-04-25T13:46:31.347950+0000","last_became_active":"2024-04-25T13:46:29.359116+0000","last_became_peered":"2024-04-25T13:46:29.359116+0000","last_unstale":"2024-04-25T13:46:31.347950+0000","last_undegraded":"2024-04-25T13:46:31.347950+0000","last_fullsized":"2024-04-25T13:46:31.347950+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:01.756809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343236+0000","last_change":"2024-04-25T13:46:28.369073+0000","last_active":"2024-04-25T13:46:30.343236+0000","last_peered":"2024-04-25T13:46:30.343236+0000","last_clean":"2024-04-25T13:46:30.343236+0000","last_became_active":"2024-04-25T13:46:28.368187+0000","last_became_peered":"2024-04-25T13:46:28.368187+0000","last_unstale":"2024-04-25T13:46:30.343236+0000","last_undegraded":"2024-04-25T13:46:30.343236+0000","last_fullsized":"2024-04-25T13:46:30.343236+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:04.696053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.346778+0000","last_change":"2024-04-25T13:46:29.366062+0000","last_active":"2024-04-25T13:46:31.346778+0000","last_peered":"2024-04-25T13:46:31.346778+0000","last_clean":"2024-04-25T13:46:31.346778+0000","last_became_active":"2024-04-25T13:46:29.363193+0000","last_became_peered":"2024-04-25T13:46:29.363193+0000","last_unstale":"2024-04-25T13:46:31.346778+0000","last_undegraded":"2024-04-25T13:46:31.346778+0000","last_fullsized":"2024-04-25T13:46:31.346778+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:29.426791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.393766+0000","last_change":"2024-04-25T13:46:28.342122+0000","last_active":"2024-04-25T13:46:31.393766+0000","last_peered":"2024-04-25T13:46:31.393766+0000","last_clean":"2024-04-25T13:46:31.393766+0000","last_became_active":"2024-04-25T13:46:28.341434+0000","last_became_peered":"2024-04-25T13:46:28.341434+0000","last_unstale":"2024-04-25T13:46:31.393766+0000","last_undegraded":"2024-04-25T13:46:31.393766+0000","last_fullsized":"2024-04-25T13:46:31.393766+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:05.178915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347005+0000","last_change":"2024-04-25T13:46:29.355378+0000","last_active":"2024-04-25T13:46:31.347005+0000","last_peered":"2024-04-25T13:46:31.347005+0000","last_clean":"2024-04-25T13:46:31.347005+0000","last_became_active":"2024-04-25T13:46:29.355122+0000","last_became_peered":"2024-04-25T13:46:29.355122+0000","last_unstale":"2024-04-25T13:46:31.347005+0000","last_undegraded":"2024-04-25T13:46:31.347005+0000","last_fullsized":"2024-04-25T13:46:31.347005+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:56.388798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.394110+0000","last_change":"2024-04-25T13:46:28.348803+0000","last_active":"2024-04-25T13:46:31.394110+0000","last_peered":"2024-04-25T13:46:31.394110+0000","last_clean":"2024-04-25T13:46:31.394110+0000","last_became_active":"2024-04-25T13:46:28.348028+0000","last_became_peered":"2024-04-25T13:46:28.348028+0000","last_unstale":"2024-04-25T13:46:31.394110+0000","last_undegraded":"2024-04-25T13:46:31.394110+0000","last_fullsized":"2024-04-25T13:46:31.394110+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:02.730612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622372+0000","last_change":"2024-04-25T13:46:29.387247+0000","last_active":"2024-04-25T13:46:30.622372+0000","last_peered":"2024-04-25T13:46:30.622372+0000","last_clean":"2024-04-25T13:46:30.622372+0000","last_became_active":"2024-04-25T13:46:29.382882+0000","last_became_peered":"2024-04-25T13:46:29.382882+0000","last_unstale":"2024-04-25T13:46:30.622372+0000","last_undegraded":"2024-04-25T13:46:30.622372+0000","last_fullsized":"2024-04-25T13:46:30.622372+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:45.318321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341912+0000","last_change":"2024-04-25T13:46:28.349758+0000","last_active":"2024-04-25T13:46:30.341912+0000","last_peered":"2024-04-25T13:46:30.341912+0000","last_clean":"2024-04-25T13:46:30.341912+0000","last_became_active":"2024-04-25T13:46:28.349053+0000","last_became_peered":"2024-04-25T13:46:28.349053+0000","last_unstale":"2024-04-25T13:46:30.341912+0000","last_undegraded":"2024-04-25T13:46:30.341912+0000","last_fullsized":"2024-04-25T13:46:30.341912+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:20.072377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342294+0000","last_change":"2024-04-25T13:46:29.360709+0000","last_active":"2024-04-25T13:46:30.342294+0000","last_peered":"2024-04-25T13:46:30.342294+0000","last_clean":"2024-04-25T13:46:30.342294+0000","last_became_active":"2024-04-25T13:46:29.360099+0000","last_became_peered":"2024-04-25T13:46:29.360099+0000","last_unstale":"2024-04-25T13:46:30.342294+0000","last_undegraded":"2024-04-25T13:46:30.342294+0000","last_fullsized":"2024-04-25T13:46:30.342294+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:18.551041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.394749+0000","last_change":"2024-04-25T13:46:28.358101+0000","last_active":"2024-04-25T13:46:31.394749+0000","last_peered":"2024-04-25T13:46:31.394749+0000","last_clean":"2024-04-25T13:46:31.394749+0000","last_became_active":"2024-04-25T13:46:28.357404+0000","last_became_peered":"2024-04-25T13:46:28.357404+0000","last_unstale":"2024-04-25T13:46:31.394749+0000","last_undegraded":"2024-04-25T13:46:31.394749+0000","last_fullsized":"2024-04-25T13:46:31.394749+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:23.195960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588398+0000","last_change":"2024-04-25T13:46:29.358029+0000","last_active":"2024-04-25T13:46:30.588398+0000","last_peered":"2024-04-25T13:46:30.588398+0000","last_clean":"2024-04-25T13:46:30.588398+0000","last_became_active":"2024-04-25T13:46:29.357649+0000","last_became_peered":"2024-04-25T13:46:29.357649+0000","last_unstale":"2024-04-25T13:46:30.588398+0000","last_undegraded":"2024-04-25T13:46:30.588398+0000","last_fullsized":"2024-04-25T13:46:30.588398+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:48.295079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"122'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343625+0000","last_change":"2024-04-25T13:46:28.368509+0000","last_active":"2024-04-25T13:46:30.343625+0000","last_peered":"2024-04-25T13:46:30.343625+0000","last_clean":"2024-04-25T13:46:30.343625+0000","last_became_active":"2024-04-25T13:46:28.367670+0000","last_became_peered":"2024-04-25T13:46:28.367670+0000","last_unstale":"2024-04-25T13:46:30.343625+0000","last_undegraded":"2024-04-25T13:46:30.343625+0000","last_fullsized":"2024-04-25T13:46:30.343625+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:35.914594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588971+0000","last_change":"2024-04-25T13:46:29.389280+0000","last_active":"2024-04-25T13:46:30.588971+0000","last_peered":"2024-04-25T13:46:30.588971+0000","last_clean":"2024-04-25T13:46:30.588971+0000","last_became_active":"2024-04-25T13:46:29.388775+0000","last_became_peered":"2024-04-25T13:46:29.388775+0000","last_unstale":"2024-04-25T13:46:30.588971+0000","last_undegraded":"2024-04-25T13:46:30.588971+0000","last_fullsized":"2024-04-25T13:46:30.588971+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:25.686534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341016+0000","last_change":"2024-04-25T13:46:28.363418+0000","last_active":"2024-04-25T13:46:30.341016+0000","last_peered":"2024-04-25T13:46:30.341016+0000","last_clean":"2024-04-25T13:46:30.341016+0000","last_became_active":"2024-04-25T13:46:28.363048+0000","last_became_peered":"2024-04-25T13:46:28.363048+0000","last_unstale":"2024-04-25T13:46:30.341016+0000","last_undegraded":"2024-04-25T13:46:30.341016+0000","last_fullsized":"2024-04-25T13:46:30.341016+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:45.906638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622601+0000","last_change":"2024-04-25T13:46:29.351416+0000","last_active":"2024-04-25T13:46:30.622601+0000","last_peered":"2024-04-25T13:46:30.622601+0000","last_clean":"2024-04-25T13:46:30.622601+0000","last_became_active":"2024-04-25T13:46:29.350701+0000","last_became_peered":"2024-04-25T13:46:29.350701+0000","last_unstale":"2024-04-25T13:46:30.622601+0000","last_undegraded":"2024-04-25T13:46:30.622601+0000","last_fullsized":"2024-04-25T13:46:30.622601+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:14.252201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622451+0000","last_change":"2024-04-25T13:46:28.368555+0000","last_active":"2024-04-25T13:46:30.622451+0000","last_peered":"2024-04-25T13:46:30.622451+0000","last_clean":"2024-04-25T13:46:30.622451+0000","last_became_active":"2024-04-25T13:46:28.366374+0000","last_became_peered":"2024-04-25T13:46:28.366374+0000","last_unstale":"2024-04-25T13:46:30.622451+0000","last_undegraded":"2024-04-25T13:46:30.622451+0000","last_fullsized":"2024-04-25T13:46:30.622451+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:04.320196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347365+0000","last_change":"2024-04-25T13:46:29.363266+0000","last_active":"2024-04-25T13:46:31.347365+0000","last_peered":"2024-04-25T13:46:31.347365+0000","last_clean":"2024-04-25T13:46:31.347365+0000","last_became_active":"2024-04-25T13:46:29.362887+0000","last_became_peered":"2024-04-25T13:46:29.362887+0000","last_unstale":"2024-04-25T13:46:31.347365+0000","last_undegraded":"2024-04-25T13:46:31.347365+0000","last_fullsized":"2024-04-25T13:46:31.347365+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:59.489360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342047+0000","last_change":"2024-04-25T13:46:28.362120+0000","last_active":"2024-04-25T13:46:30.342047+0000","last_peered":"2024-04-25T13:46:30.342047+0000","last_clean":"2024-04-25T13:46:30.342047+0000","last_became_active":"2024-04-25T13:46:28.354018+0000","last_became_peered":"2024-04-25T13:46:28.354018+0000","last_unstale":"2024-04-25T13:46:30.342047+0000","last_undegraded":"2024-04-25T13:46:30.342047+0000","last_fullsized":"2024-04-25T13:46:30.342047+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:56.290367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343212+0000","last_change":"2024-04-25T13:46:29.367925+0000","last_active":"2024-04-25T13:46:30.343212+0000","last_peered":"2024-04-25T13:46:30.343212+0000","last_clean":"2024-04-25T13:46:30.343212+0000","last_became_active":"2024-04-25T13:46:29.365620+0000","last_became_peered":"2024-04-25T13:46:29.365620+0000","last_unstale":"2024-04-25T13:46:30.343212+0000","last_undegraded":"2024-04-25T13:46:30.343212+0000","last_fullsized":"2024-04-25T13:46:30.343212+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:17.626818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621800+0000","last_change":"2024-04-25T13:46:28.359523+0000","last_active":"2024-04-25T13:46:30.621800+0000","last_peered":"2024-04-25T13:46:30.621800+0000","last_clean":"2024-04-25T13:46:30.621800+0000","last_became_active":"2024-04-25T13:46:28.356573+0000","last_became_peered":"2024-04-25T13:46:28.356573+0000","last_unstale":"2024-04-25T13:46:30.621800+0000","last_undegraded":"2024-04-25T13:46:30.621800+0000","last_fullsized":"2024-04-25T13:46:30.621800+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:40.243763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.348315+0000","last_change":"2024-04-25T13:46:29.383509+0000","last_active":"2024-04-25T13:46:31.348315+0000","last_peered":"2024-04-25T13:46:31.348315+0000","last_clean":"2024-04-25T13:46:31.348315+0000","last_became_active":"2024-04-25T13:46:29.383326+0000","last_became_peered":"2024-04-25T13:46:29.383326+0000","last_unstale":"2024-04-25T13:46:31.348315+0000","last_undegraded":"2024-04-25T13:46:31.348315+0000","last_fullsized":"2024-04-25T13:46:31.348315+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:37.302283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630188+0000","last_change":"2024-04-25T13:46:28.354915+0000","last_active":"2024-04-25T13:46:31.630188+0000","last_peered":"2024-04-25T13:46:31.630188+0000","last_clean":"2024-04-25T13:46:31.630188+0000","last_became_active":"2024-04-25T13:46:28.354528+0000","last_became_peered":"2024-04-25T13:46:28.354528+0000","last_unstale":"2024-04-25T13:46:31.630188+0000","last_undegraded":"2024-04-25T13:46:31.630188+0000","last_fullsized":"2024-04-25T13:46:31.630188+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:55.882013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621671+0000","last_change":"2024-04-25T13:46:29.372892+0000","last_active":"2024-04-25T13:46:30.621671+0000","last_peered":"2024-04-25T13:46:30.621671+0000","last_clean":"2024-04-25T13:46:30.621671+0000","last_became_active":"2024-04-25T13:46:29.371561+0000","last_became_peered":"2024-04-25T13:46:29.371561+0000","last_unstale":"2024-04-25T13:46:30.621671+0000","last_undegraded":"2024-04-25T13:46:30.621671+0000","last_fullsized":"2024-04-25T13:46:30.621671+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:10.120733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.348399+0000","last_change":"2024-04-25T13:46:28.372486+0000","last_active":"2024-04-25T13:46:31.348399+0000","last_peered":"2024-04-25T13:46:31.348399+0000","last_clean":"2024-04-25T13:46:31.348399+0000","last_became_active":"2024-04-25T13:46:28.372037+0000","last_became_peered":"2024-04-25T13:46:28.372037+0000","last_unstale":"2024-04-25T13:46:31.348399+0000","last_undegraded":"2024-04-25T13:46:31.348399+0000","last_fullsized":"2024-04-25T13:46:31.348399+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:06.292772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.346996+0000","last_change":"2024-04-25T13:46:29.363543+0000","last_active":"2024-04-25T13:46:31.346996+0000","last_peered":"2024-04-25T13:46:31.346996+0000","last_clean":"2024-04-25T13:46:31.346996+0000","last_became_active":"2024-04-25T13:46:29.363037+0000","last_became_peered":"2024-04-25T13:46:29.363037+0000","last_unstale":"2024-04-25T13:46:31.346996+0000","last_undegraded":"2024-04-25T13:46:31.346996+0000","last_fullsized":"2024-04-25T13:46:31.346996+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:23.856965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629583+0000","last_change":"2024-04-25T13:46:28.370546+0000","last_active":"2024-04-25T13:46:31.629583+0000","last_peered":"2024-04-25T13:46:31.629583+0000","last_clean":"2024-04-25T13:46:31.629583+0000","last_became_active":"2024-04-25T13:46:28.369881+0000","last_became_peered":"2024-04-25T13:46:28.369881+0000","last_unstale":"2024-04-25T13:46:31.629583+0000","last_undegraded":"2024-04-25T13:46:31.629583+0000","last_fullsized":"2024-04-25T13:46:31.629583+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:50.276099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":313,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629755+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:46:31.629755+0000","last_peered":"2024-04-25T13:46:31.629755+0000","last_clean":"2024-04-25T13:46:31.629755+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:46:31.629755+0000","last_undegraded":"2024-04-25T13:46:31.629755+0000","last_fullsized":"2024-04-25T13:46:31.629755+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574925,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1234,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194077,"statfs":{"total":200000000,"available":198735576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574925,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194653,"statfs":{"total":200000000,"available":199325404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574925,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194654,"statfs":{"total":200000000,"available":199325986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574925,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":659,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194652,"statfs":{"total":200000000,"available":199324320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":659,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194652,"statfs":{"total":200000000,"available":199324422,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574925,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194654,"statfs":{"total":200000000,"available":199326022,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194654,"statfs":{"total":200000000,"available":199326056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574925,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1235,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194077,"statfs":{"total":200000000,"available":198735106,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:46:38.070 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:46:38.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b36daa640 1 Processor -- start 2024-04-25T13:46:38.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b36daa640 1 -- start start 2024-04-25T13:46:38.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b36daa640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9b3006d000 0x7f9b30068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b36daa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b301622c0 0x7f9b30168d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b36daa640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b300d2490 con 0x7f9b30068ad0 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b36daa640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9b300d2000 con 0x7f9b3006d000 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b36daa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b30068ee0 con 0x7f9b301622c0 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b2ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9b30068ad0 0x7f9b30161bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52654/0 (socket says 172.21.15.80:52654) 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.231+0000 7f9b2ffff640 1 -- 172.21.15.80:0/2177985604 learned_addr learned my addr 172.21.15.80:0/2177985604 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b35347640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b301622c0 0x7f9b30168d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b34b46640 1 --2- 172.21.15.80:0/2177985604 >> v2:172.21.15.87:3300/0 conn(0x7f9b3006d000 0x7f9b30068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1450491704 0 0) 0x7f9b20001120 con 0x7f9b30068ad0 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b18003530 con 0x7f9b30068ad0 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4074789050 0 0) 0x7f9b18003530 con 0x7f9b30068ad0 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b301622c0 msgr2=0x7f9b30168d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:38.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b301622c0 0x7f9b30168d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 >> v2:172.21.15.87:3300/0 conn(0x7f9b3006d000 msgr2=0x7f9b30068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 --2- 172.21.15.80:0/2177985604 >> v2:172.21.15.87:3300/0 conn(0x7f9b3006d000 0x7f9b30068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b300d13f0 con 0x7f9b30068ad0 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f9b20001850 con 0x7f9b30068ad0 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 >> v1:172.21.15.80:6789/0 conn(0x7f9b30068ad0 legacy=0x7f9b30161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b180039c0 0x7f9b18003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b18004810 0x7f9b18004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b301622c0 0x7f9b18005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9b300d2000 con 0x7f9b180039c0 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30068ee0 con 0x7f9b18004810 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b1800b9d0 con 0x7f9b301622c0 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b34b46640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b180039c0 0x7f9b18003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b34b46640 1 -- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b301622c0 msgr2=0x7f9b18005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:38.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b34b46640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b301622c0 0x7f9b18005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b34b46640 1 -- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b18004810 msgr2=0x7f9b18004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:38.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b34b46640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b18004810 0x7f9b18004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b34b46640 1 -- 172.21.15.80:0/2177985604 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b18003530 con 0x7f9b180039c0 2024-04-25T13:46:38.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b34b46640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b180039c0 0x7f9b18003e30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009ab0 tx=0x7f9b2401ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=331d9fa26f410434 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:38.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b240b2070 con 0x7f9b180039c0 2024-04-25T13:46:38.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b240a5630 con 0x7f9b180039c0 2024-04-25T13:46:38.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.235+0000 7f9b2f7fe640 1 -- 172.21.15.80:0/2177985604 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b240096b0 con 0x7f9b180039c0 2024-04-25T13:46:38.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- 172.21.15.80:0/2177985604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b180039c0 msgr2=0x7f9b18003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 --2- 172.21.15.80:0/2177985604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b180039c0 0x7f9b18003e30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009ab0 tx=0x7f9b2401ca60 comp rx=0 tx=0).stop 2024-04-25T13:46:38.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 -- 172.21.15.80:0/2177985604 reap_dead start 2024-04-25T13:46:38.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- 172.21.15.80:0/2177985604 shutdown_connections 2024-04-25T13:46:38.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- 172.21.15.80:0/2177985604 >> 172.21.15.80:0/2177985604 conn(0x7f9b300d28f0 msgr2=0x7f9b300d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:38.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- 172.21.15.80:0/2177985604 shutdown_connections 2024-04-25T13:46:38.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- 172.21.15.80:0/2177985604 wait complete. 2024-04-25T13:46:38.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 Processor -- start 2024-04-25T13:46:38.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- start start 2024-04-25T13:46:38.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b180039c0 0x7f9b3014eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30149a60 0x7f9b300b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b300b0800 0x7f9b3015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3006d410 con 0x7f9b300b0800 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b300d10a0 con 0x7f9b30149a60 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b36daa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b3006fed0 con 0x7f9b180039c0 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b300b0800 0x7f9b3015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b300b0800 0x7f9b3015fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43076/0 (socket says 172.21.15.80:43076) 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 -- 172.21.15.80:0/3248996986 learned_addr learned my addr 172.21.15.80:0/3248996986 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b2ffff640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30149a60 0x7f9b300b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b34b46640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b180039c0 0x7f9b3014eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 -- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b180039c0 msgr2=0x7f9b3014eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b180039c0 0x7f9b3014eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 -- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30149a60 msgr2=0x7f9b300b02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30149a60 0x7f9b300b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b35347640 1 -- 172.21.15.80:0/3248996986 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b300b6930 con 0x7f9b300b0800 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.239+0000 7f9b34b46640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b180039c0 0x7f9b3014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:38.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b35347640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b300b0800 0x7f9b3015fd70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9b300b5910 tx=0x7f9b2800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:38.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b2d7fa640 1 -- 172.21.15.80:0/3248996986 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2800bae0 con 0x7f9b300b0800 2024-04-25T13:46:38.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b2d7fa640 1 -- 172.21.15.80:0/3248996986 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b2801a070 con 0x7f9b300b0800 2024-04-25T13:46:38.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b2d7fa640 1 -- 172.21.15.80:0/3248996986 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28014400 con 0x7f9b300b0800 2024-04-25T13:46:38.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b300b7620 con 0x7f9b300b0800 2024-04-25T13:46:38.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b300b1030 con 0x7f9b300b0800 2024-04-25T13:46:38.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b3006a3a0 con 0x7f9b300b0800 2024-04-25T13:46:38.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b2d7fa640 1 -- 172.21.15.80:0/3248996986 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b28013070 con 0x7f9b300b0800 2024-04-25T13:46:38.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b2d7fa640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b1c06f400 0x7f9b1c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b2d7fa640 1 -- 172.21.15.80:0/3248996986 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f9b2808f480 con 0x7f9b300b0800 2024-04-25T13:46:38.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.243+0000 7f9b34b46640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b1c06f400 0x7f9b1c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.247+0000 7f9b34b46640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b1c06f400 0x7f9b1c071910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9b240af000 tx=0x7f9b240046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:38.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.247+0000 7f9b2d7fa640 1 -- 172.21.15.80:0/3248996986 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b28024020 con 0x7f9b300b0800 2024-04-25T13:46:38.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.403+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9b300b1e90 con 0x7f9b1c06f400 2024-04-25T13:46:38.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.423+0000 7f9b2d7fa640 1 -- 172.21.15.80:0/3248996986 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247982 (secure 0 0 0) 0x7f9b300b1e90 con 0x7f9b1c06f400 2024-04-25T13:46:38.424 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:38.425 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:46:38.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b1c06f400 msgr2=0x7f9b1c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b1c06f400 0x7f9b1c071910 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9b240af000 tx=0x7f9b240046d0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b300b0800 msgr2=0x7f9b3015fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b300b0800 0x7f9b3015fd70 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9b300b5910 tx=0x7f9b2800b480 comp rx=0 tx=0).stop 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 shutdown_connections 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b1c06f400 0x7f9b1c071910 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b300b0800 0x7f9b3015fd70 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b30149a60 0x7f9b300b02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 --2- 172.21.15.80:0/3248996986 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b180039c0 0x7f9b3014eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 >> 172.21.15.80:0/3248996986 conn(0x7f9b300d28f0 msgr2=0x7f9b30155e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 shutdown_connections 2024-04-25T13:46:38.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.427+0000 7f9b36daa640 1 -- 172.21.15.80:0/3248996986 wait complete. 2024-04-25T13:46:38.444 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-04-25T13:46:38.232330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6422,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556077,"statfs":{"total":1600000000,"available":1593422892,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.894787"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588009+0000","last_change":"2024-04-25T13:46:29.379199+0000","last_active":"2024-04-25T13:46:30.588009+0000","last_peered":"2024-04-25T13:46:30.588009+0000","last_clean":"2024-04-25T13:46:30.588009+0000","last_became_active":"2024-04-25T13:46:29.378876+0000","last_became_peered":"2024-04-25T13:46:29.378876+0000","last_unstale":"2024-04-25T13:46:30.588009+0000","last_undegraded":"2024-04-25T13:46:30.588009+0000","last_fullsized":"2024-04-25T13:46:30.588009+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:06.966627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341497+0000","last_change":"2024-04-25T13:46:29.359230+0000","last_active":"2024-04-25T13:46:30.341497+0000","last_peered":"2024-04-25T13:46:30.341497+0000","last_clean":"2024-04-25T13:46:30.341497+0000","last_became_active":"2024-04-25T13:46:29.358723+0000","last_became_peered":"2024-04-25T13:46:29.358723+0000","last_unstale":"2024-04-25T13:46:30.341497+0000","last_undegraded":"2024-04-25T13:46:30.341497+0000","last_fullsized":"2024-04-25T13:46:30.341497+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:29.753013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.587526+0000","last_change":"2024-04-25T13:46:29.361490+0000","last_active":"2024-04-25T13:46:30.587526+0000","last_peered":"2024-04-25T13:46:30.587526+0000","last_clean":"2024-04-25T13:46:30.587526+0000","last_became_active":"2024-04-25T13:46:29.361188+0000","last_became_peered":"2024-04-25T13:46:29.361188+0000","last_unstale":"2024-04-25T13:46:30.587526+0000","last_undegraded":"2024-04-25T13:46:30.587526+0000","last_fullsized":"2024-04-25T13:46:30.587526+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:44.756958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629462+0000","last_change":"2024-04-25T13:46:29.364994+0000","last_active":"2024-04-25T13:46:31.629462+0000","last_peered":"2024-04-25T13:46:31.629462+0000","last_clean":"2024-04-25T13:46:31.629462+0000","last_became_active":"2024-04-25T13:46:29.364548+0000","last_became_peered":"2024-04-25T13:46:29.364548+0000","last_unstale":"2024-04-25T13:46:31.629462+0000","last_undegraded":"2024-04-25T13:46:31.629462+0000","last_fullsized":"2024-04-25T13:46:31.629462+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:44.339514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621639+0000","last_change":"2024-04-25T13:46:29.389614+0000","last_active":"2024-04-25T13:46:30.621639+0000","last_peered":"2024-04-25T13:46:30.621639+0000","last_clean":"2024-04-25T13:46:30.621639+0000","last_became_active":"2024-04-25T13:46:29.386499+0000","last_became_peered":"2024-04-25T13:46:29.386499+0000","last_unstale":"2024-04-25T13:46:30.621639+0000","last_undegraded":"2024-04-25T13:46:30.621639+0000","last_fullsized":"2024-04-25T13:46:30.621639+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:17.092243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629561+0000","last_change":"2024-04-25T13:46:29.386033+0000","last_active":"2024-04-25T13:46:31.629561+0000","last_peered":"2024-04-25T13:46:31.629561+0000","last_clean":"2024-04-25T13:46:31.629561+0000","last_became_active":"2024-04-25T13:46:29.385379+0000","last_became_peered":"2024-04-25T13:46:29.385379+0000","last_unstale":"2024-04-25T13:46:31.629561+0000","last_undegraded":"2024-04-25T13:46:31.629561+0000","last_fullsized":"2024-04-25T13:46:31.629561+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:45:08.249909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629289+0000","last_change":"2024-04-25T13:46:29.385575+0000","last_active":"2024-04-25T13:46:31.629289+0000","last_peered":"2024-04-25T13:46:31.629289+0000","last_clean":"2024-04-25T13:46:31.629289+0000","last_became_active":"2024-04-25T13:46:29.363594+0000","last_became_peered":"2024-04-25T13:46:29.363594+0000","last_unstale":"2024-04-25T13:46:31.629289+0000","last_undegraded":"2024-04-25T13:46:31.629289+0000","last_fullsized":"2024-04-25T13:46:31.629289+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:00.275547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341407+0000","last_change":"2024-04-25T13:46:29.393782+0000","last_active":"2024-04-25T13:46:30.341407+0000","last_peered":"2024-04-25T13:46:30.341407+0000","last_clean":"2024-04-25T13:46:30.341407+0000","last_became_active":"2024-04-25T13:46:29.393340+0000","last_became_peered":"2024-04-25T13:46:29.393340+0000","last_unstale":"2024-04-25T13:46:30.341407+0000","last_undegraded":"2024-04-25T13:46:30.341407+0000","last_fullsized":"2024-04-25T13:46:30.341407+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:32.074485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341715+0000","last_change":"2024-04-25T13:46:29.377607+0000","last_active":"2024-04-25T13:46:30.341715+0000","last_peered":"2024-04-25T13:46:30.341715+0000","last_clean":"2024-04-25T13:46:30.341715+0000","last_became_active":"2024-04-25T13:46:29.360329+0000","last_became_peered":"2024-04-25T13:46:29.360329+0000","last_unstale":"2024-04-25T13:46:30.341715+0000","last_undegraded":"2024-04-25T13:46:30.341715+0000","last_fullsized":"2024-04-25T13:46:30.341715+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:50.475932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621917+0000","last_change":"2024-04-25T13:46:29.388662+0000","last_active":"2024-04-25T13:46:30.621917+0000","last_peered":"2024-04-25T13:46:30.621917+0000","last_clean":"2024-04-25T13:46:30.621917+0000","last_became_active":"2024-04-25T13:46:29.384118+0000","last_became_peered":"2024-04-25T13:46:29.384118+0000","last_unstale":"2024-04-25T13:46:30.621917+0000","last_undegraded":"2024-04-25T13:46:30.621917+0000","last_fullsized":"2024-04-25T13:46:30.621917+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:41.484705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630039+0000","last_change":"2024-04-25T13:46:29.370803+0000","last_active":"2024-04-25T13:46:31.630039+0000","last_peered":"2024-04-25T13:46:31.630039+0000","last_clean":"2024-04-25T13:46:31.630039+0000","last_became_active":"2024-04-25T13:46:29.369106+0000","last_became_peered":"2024-04-25T13:46:29.369106+0000","last_unstale":"2024-04-25T13:46:31.630039+0000","last_undegraded":"2024-04-25T13:46:31.630039+0000","last_fullsized":"2024-04-25T13:46:31.630039+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:59.533887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342438+0000","last_change":"2024-04-25T13:46:29.357312+0000","last_active":"2024-04-25T13:46:30.342438+0000","last_peered":"2024-04-25T13:46:30.342438+0000","last_clean":"2024-04-25T13:46:30.342438+0000","last_became_active":"2024-04-25T13:46:29.353270+0000","last_became_peered":"2024-04-25T13:46:29.353270+0000","last_unstale":"2024-04-25T13:46:30.342438+0000","last_undegraded":"2024-04-25T13:46:30.342438+0000","last_fullsized":"2024-04-25T13:46:30.342438+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:52.237940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629613+0000","last_change":"2024-04-25T13:46:29.355819+0000","last_active":"2024-04-25T13:46:31.629613+0000","last_peered":"2024-04-25T13:46:31.629613+0000","last_clean":"2024-04-25T13:46:31.629613+0000","last_became_active":"2024-04-25T13:46:29.355353+0000","last_became_peered":"2024-04-25T13:46:29.355353+0000","last_unstale":"2024-04-25T13:46:31.629613+0000","last_undegraded":"2024-04-25T13:46:31.629613+0000","last_fullsized":"2024-04-25T13:46:31.629613+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:00.696939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.346612+0000","last_change":"2024-04-25T13:46:29.360510+0000","last_active":"2024-04-25T13:46:31.346612+0000","last_peered":"2024-04-25T13:46:31.346612+0000","last_clean":"2024-04-25T13:46:31.346612+0000","last_became_active":"2024-04-25T13:46:29.359901+0000","last_became_peered":"2024-04-25T13:46:29.359901+0000","last_unstale":"2024-04-25T13:46:31.346612+0000","last_undegraded":"2024-04-25T13:46:31.346612+0000","last_fullsized":"2024-04-25T13:46:31.346612+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:25:29.293066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629944+0000","last_change":"2024-04-25T13:46:29.356612+0000","last_active":"2024-04-25T13:46:31.629944+0000","last_peered":"2024-04-25T13:46:31.629944+0000","last_clean":"2024-04-25T13:46:31.629944+0000","last_became_active":"2024-04-25T13:46:29.353061+0000","last_became_peered":"2024-04-25T13:46:29.353061+0000","last_unstale":"2024-04-25T13:46:31.629944+0000","last_undegraded":"2024-04-25T13:46:31.629944+0000","last_fullsized":"2024-04-25T13:46:31.629944+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:16.514862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630367+0000","last_change":"2024-04-25T13:46:29.354914+0000","last_active":"2024-04-25T13:46:31.630367+0000","last_peered":"2024-04-25T13:46:31.630367+0000","last_clean":"2024-04-25T13:46:31.630367+0000","last_became_active":"2024-04-25T13:46:29.354099+0000","last_became_peered":"2024-04-25T13:46:29.354099+0000","last_unstale":"2024-04-25T13:46:31.630367+0000","last_undegraded":"2024-04-25T13:46:31.630367+0000","last_fullsized":"2024-04-25T13:46:31.630367+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:54.809032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341960+0000","last_change":"2024-04-25T13:46:29.348116+0000","last_active":"2024-04-25T13:46:30.341960+0000","last_peered":"2024-04-25T13:46:30.341960+0000","last_clean":"2024-04-25T13:46:30.341960+0000","last_became_active":"2024-04-25T13:46:29.347746+0000","last_became_peered":"2024-04-25T13:46:29.347746+0000","last_unstale":"2024-04-25T13:46:30.341960+0000","last_undegraded":"2024-04-25T13:46:30.341960+0000","last_fullsized":"2024-04-25T13:46:30.341960+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:01.615426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629875+0000","last_change":"2024-04-25T13:46:29.355605+0000","last_active":"2024-04-25T13:46:31.629875+0000","last_peered":"2024-04-25T13:46:31.629875+0000","last_clean":"2024-04-25T13:46:31.629875+0000","last_became_active":"2024-04-25T13:46:29.354988+0000","last_became_peered":"2024-04-25T13:46:29.354988+0000","last_unstale":"2024-04-25T13:46:31.629875+0000","last_undegraded":"2024-04-25T13:46:31.629875+0000","last_fullsized":"2024-04-25T13:46:31.629875+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:29.248386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347108+0000","last_change":"2024-04-25T13:46:29.360303+0000","last_active":"2024-04-25T13:46:31.347108+0000","last_peered":"2024-04-25T13:46:31.347108+0000","last_clean":"2024-04-25T13:46:31.347108+0000","last_became_active":"2024-04-25T13:46:29.359458+0000","last_became_peered":"2024-04-25T13:46:29.359458+0000","last_unstale":"2024-04-25T13:46:31.347108+0000","last_undegraded":"2024-04-25T13:46:31.347108+0000","last_fullsized":"2024-04-25T13:46:31.347108+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:28.949462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343455+0000","last_change":"2024-04-25T13:46:29.357541+0000","last_active":"2024-04-25T13:46:30.343455+0000","last_peered":"2024-04-25T13:46:30.343455+0000","last_clean":"2024-04-25T13:46:30.343455+0000","last_became_active":"2024-04-25T13:46:29.355431+0000","last_became_peered":"2024-04-25T13:46:29.355431+0000","last_unstale":"2024-04-25T13:46:30.343455+0000","last_undegraded":"2024-04-25T13:46:30.343455+0000","last_fullsized":"2024-04-25T13:46:30.343455+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:28.115823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.346538+0000","last_change":"2024-04-25T13:46:29.379316+0000","last_active":"2024-04-25T13:46:31.346538+0000","last_peered":"2024-04-25T13:46:31.346538+0000","last_clean":"2024-04-25T13:46:31.346538+0000","last_became_active":"2024-04-25T13:46:29.379046+0000","last_became_peered":"2024-04-25T13:46:29.379046+0000","last_unstale":"2024-04-25T13:46:31.346538+0000","last_undegraded":"2024-04-25T13:46:31.346538+0000","last_fullsized":"2024-04-25T13:46:31.346538+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:19:50.935327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342819+0000","last_change":"2024-04-25T13:46:29.358988+0000","last_active":"2024-04-25T13:46:30.342819+0000","last_peered":"2024-04-25T13:46:30.342819+0000","last_clean":"2024-04-25T13:46:30.342819+0000","last_became_active":"2024-04-25T13:46:29.358338+0000","last_became_peered":"2024-04-25T13:46:29.358338+0000","last_unstale":"2024-04-25T13:46:30.342819+0000","last_undegraded":"2024-04-25T13:46:30.342819+0000","last_fullsized":"2024-04-25T13:46:30.342819+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:57.359133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341877+0000","last_change":"2024-04-25T13:46:29.360931+0000","last_active":"2024-04-25T13:46:30.341877+0000","last_peered":"2024-04-25T13:46:30.341877+0000","last_clean":"2024-04-25T13:46:30.341877+0000","last_became_active":"2024-04-25T13:46:29.360432+0000","last_became_peered":"2024-04-25T13:46:29.360432+0000","last_unstale":"2024-04-25T13:46:30.341877+0000","last_undegraded":"2024-04-25T13:46:30.341877+0000","last_fullsized":"2024-04-25T13:46:30.341877+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:35.622752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629283+0000","last_change":"2024-04-25T13:46:29.365656+0000","last_active":"2024-04-25T13:46:31.629283+0000","last_peered":"2024-04-25T13:46:31.629283+0000","last_clean":"2024-04-25T13:46:31.629283+0000","last_became_active":"2024-04-25T13:46:29.364664+0000","last_became_peered":"2024-04-25T13:46:29.364664+0000","last_unstale":"2024-04-25T13:46:31.629283+0000","last_undegraded":"2024-04-25T13:46:31.629283+0000","last_fullsized":"2024-04-25T13:46:31.629283+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:30.471600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622149+0000","last_change":"2024-04-25T13:46:29.389093+0000","last_active":"2024-04-25T13:46:30.622149+0000","last_peered":"2024-04-25T13:46:30.622149+0000","last_clean":"2024-04-25T13:46:30.622149+0000","last_became_active":"2024-04-25T13:46:29.385645+0000","last_became_peered":"2024-04-25T13:46:29.385645+0000","last_unstale":"2024-04-25T13:46:30.622149+0000","last_undegraded":"2024-04-25T13:46:30.622149+0000","last_fullsized":"2024-04-25T13:46:30.622149+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:16:13.995233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341597+0000","last_change":"2024-04-25T13:46:29.350360+0000","last_active":"2024-04-25T13:46:30.341597+0000","last_peered":"2024-04-25T13:46:30.341597+0000","last_clean":"2024-04-25T13:46:30.341597+0000","last_became_active":"2024-04-25T13:46:29.350137+0000","last_became_peered":"2024-04-25T13:46:29.350137+0000","last_unstale":"2024-04-25T13:46:30.341597+0000","last_undegraded":"2024-04-25T13:46:30.341597+0000","last_fullsized":"2024-04-25T13:46:30.341597+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:05:11.315733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629560+0000","last_change":"2024-04-25T13:46:29.364445+0000","last_active":"2024-04-25T13:46:31.629560+0000","last_peered":"2024-04-25T13:46:31.629560+0000","last_clean":"2024-04-25T13:46:31.629560+0000","last_became_active":"2024-04-25T13:46:29.361873+0000","last_became_peered":"2024-04-25T13:46:29.361873+0000","last_unstale":"2024-04-25T13:46:31.629560+0000","last_undegraded":"2024-04-25T13:46:31.629560+0000","last_fullsized":"2024-04-25T13:46:31.629560+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:52.572388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622194+0000","last_change":"2024-04-25T13:46:29.372301+0000","last_active":"2024-04-25T13:46:30.622194+0000","last_peered":"2024-04-25T13:46:30.622194+0000","last_clean":"2024-04-25T13:46:30.622194+0000","last_became_active":"2024-04-25T13:46:29.370609+0000","last_became_peered":"2024-04-25T13:46:29.370609+0000","last_unstale":"2024-04-25T13:46:30.622194+0000","last_undegraded":"2024-04-25T13:46:30.622194+0000","last_fullsized":"2024-04-25T13:46:30.622194+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:54:35.503611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.340859+0000","last_change":"2024-04-25T13:46:29.347300+0000","last_active":"2024-04-25T13:46:30.340859+0000","last_peered":"2024-04-25T13:46:30.340859+0000","last_clean":"2024-04-25T13:46:30.340859+0000","last_became_active":"2024-04-25T13:46:29.347044+0000","last_became_peered":"2024-04-25T13:46:29.347044+0000","last_unstale":"2024-04-25T13:46:30.340859+0000","last_undegraded":"2024-04-25T13:46:30.340859+0000","last_fullsized":"2024-04-25T13:46:30.340859+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:08.145655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347498+0000","last_change":"2024-04-25T13:46:29.354560+0000","last_active":"2024-04-25T13:46:31.347498+0000","last_peered":"2024-04-25T13:46:31.347498+0000","last_clean":"2024-04-25T13:46:31.347498+0000","last_became_active":"2024-04-25T13:46:29.354213+0000","last_became_peered":"2024-04-25T13:46:29.354213+0000","last_unstale":"2024-04-25T13:46:31.347498+0000","last_undegraded":"2024-04-25T13:46:31.347498+0000","last_fullsized":"2024-04-25T13:46:31.347498+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:28:50.021364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629701+0000","last_change":"2024-04-25T13:46:29.362585+0000","last_active":"2024-04-25T13:46:31.629701+0000","last_peered":"2024-04-25T13:46:31.629701+0000","last_clean":"2024-04-25T13:46:31.629701+0000","last_became_active":"2024-04-25T13:46:29.362356+0000","last_became_peered":"2024-04-25T13:46:29.362356+0000","last_unstale":"2024-04-25T13:46:31.629701+0000","last_undegraded":"2024-04-25T13:46:31.629701+0000","last_fullsized":"2024-04-25T13:46:31.629701+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:36.383874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588071+0000","last_change":"2024-04-25T13:46:29.349095+0000","last_active":"2024-04-25T13:46:30.588071+0000","last_peered":"2024-04-25T13:46:30.588071+0000","last_clean":"2024-04-25T13:46:30.588071+0000","last_became_active":"2024-04-25T13:46:29.348721+0000","last_became_peered":"2024-04-25T13:46:29.348721+0000","last_unstale":"2024-04-25T13:46:30.588071+0000","last_undegraded":"2024-04-25T13:46:30.588071+0000","last_fullsized":"2024-04-25T13:46:30.588071+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:47.534623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347543+0000","last_change":"2024-04-25T13:46:29.367798+0000","last_active":"2024-04-25T13:46:31.347543+0000","last_peered":"2024-04-25T13:46:31.347543+0000","last_clean":"2024-04-25T13:46:31.347543+0000","last_became_active":"2024-04-25T13:46:29.367627+0000","last_became_peered":"2024-04-25T13:46:29.367627+0000","last_unstale":"2024-04-25T13:46:31.347543+0000","last_undegraded":"2024-04-25T13:46:31.347543+0000","last_fullsized":"2024-04-25T13:46:31.347543+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:10.874484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"121'4","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342643+0000","last_change":"2024-04-25T13:46:28.357817+0000","last_active":"2024-04-25T13:46:30.342643+0000","last_peered":"2024-04-25T13:46:30.342643+0000","last_clean":"2024-04-25T13:46:30.342643+0000","last_became_active":"2024-04-25T13:46:28.356033+0000","last_became_peered":"2024-04-25T13:46:28.356033+0000","last_unstale":"2024-04-25T13:46:30.342643+0000","last_undegraded":"2024-04-25T13:46:30.342643+0000","last_fullsized":"2024-04-25T13:46:30.342643+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:40:32.433741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630108+0000","last_change":"2024-04-25T13:46:29.365988+0000","last_active":"2024-04-25T13:46:31.630108+0000","last_peered":"2024-04-25T13:46:31.630108+0000","last_clean":"2024-04-25T13:46:31.630108+0000","last_became_active":"2024-04-25T13:46:29.365721+0000","last_became_peered":"2024-04-25T13:46:29.365721+0000","last_unstale":"2024-04-25T13:46:31.630108+0000","last_undegraded":"2024-04-25T13:46:31.630108+0000","last_fullsized":"2024-04-25T13:46:31.630108+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:18.995871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629398+0000","last_change":"2024-04-25T13:46:28.364069+0000","last_active":"2024-04-25T13:46:31.629398+0000","last_peered":"2024-04-25T13:46:31.629398+0000","last_clean":"2024-04-25T13:46:31.629398+0000","last_became_active":"2024-04-25T13:46:28.363317+0000","last_became_peered":"2024-04-25T13:46:28.363317+0000","last_unstale":"2024-04-25T13:46:31.629398+0000","last_undegraded":"2024-04-25T13:46:31.629398+0000","last_fullsized":"2024-04-25T13:46:31.629398+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:38.581336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588260+0000","last_change":"2024-04-25T13:46:29.352272+0000","last_active":"2024-04-25T13:46:30.588260+0000","last_peered":"2024-04-25T13:46:30.588260+0000","last_clean":"2024-04-25T13:46:30.588260+0000","last_became_active":"2024-04-25T13:46:29.351906+0000","last_became_peered":"2024-04-25T13:46:29.351906+0000","last_unstale":"2024-04-25T13:46:30.588260+0000","last_undegraded":"2024-04-25T13:46:30.588260+0000","last_fullsized":"2024-04-25T13:46:30.588260+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:09.244791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341444+0000","last_change":"2024-04-25T13:46:28.367229+0000","last_active":"2024-04-25T13:46:30.341444+0000","last_peered":"2024-04-25T13:46:30.341444+0000","last_clean":"2024-04-25T13:46:30.341444+0000","last_became_active":"2024-04-25T13:46:28.366847+0000","last_became_peered":"2024-04-25T13:46:28.366847+0000","last_unstale":"2024-04-25T13:46:30.341444+0000","last_undegraded":"2024-04-25T13:46:30.341444+0000","last_fullsized":"2024-04-25T13:46:30.341444+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:26:33.018943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343705+0000","last_change":"2024-04-25T13:46:29.365163+0000","last_active":"2024-04-25T13:46:30.343705+0000","last_peered":"2024-04-25T13:46:30.343705+0000","last_clean":"2024-04-25T13:46:30.343705+0000","last_became_active":"2024-04-25T13:46:29.363861+0000","last_became_peered":"2024-04-25T13:46:29.363861+0000","last_unstale":"2024-04-25T13:46:30.343705+0000","last_undegraded":"2024-04-25T13:46:30.343705+0000","last_fullsized":"2024-04-25T13:46:30.343705+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:37.043886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341729+0000","last_change":"2024-04-25T13:46:28.365386+0000","last_active":"2024-04-25T13:46:30.341729+0000","last_peered":"2024-04-25T13:46:30.341729+0000","last_clean":"2024-04-25T13:46:30.341729+0000","last_became_active":"2024-04-25T13:46:28.363348+0000","last_became_peered":"2024-04-25T13:46:28.363348+0000","last_unstale":"2024-04-25T13:46:30.341729+0000","last_undegraded":"2024-04-25T13:46:30.341729+0000","last_fullsized":"2024-04-25T13:46:30.341729+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:34.461750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.394721+0000","last_change":"2024-04-25T13:46:29.378357+0000","last_active":"2024-04-25T13:46:31.394721+0000","last_peered":"2024-04-25T13:46:31.394721+0000","last_clean":"2024-04-25T13:46:31.394721+0000","last_became_active":"2024-04-25T13:46:29.376081+0000","last_became_peered":"2024-04-25T13:46:29.376081+0000","last_unstale":"2024-04-25T13:46:31.394721+0000","last_undegraded":"2024-04-25T13:46:31.394721+0000","last_fullsized":"2024-04-25T13:46:31.394721+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:43.188032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588498+0000","last_change":"2024-04-25T13:46:28.361416+0000","last_active":"2024-04-25T13:46:30.588498+0000","last_peered":"2024-04-25T13:46:30.588498+0000","last_clean":"2024-04-25T13:46:30.588498+0000","last_became_active":"2024-04-25T13:46:28.361020+0000","last_became_peered":"2024-04-25T13:46:28.361020+0000","last_unstale":"2024-04-25T13:46:30.588498+0000","last_undegraded":"2024-04-25T13:46:30.588498+0000","last_fullsized":"2024-04-25T13:46:30.588498+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:54.615568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629824+0000","last_change":"2024-04-25T13:46:29.386574+0000","last_active":"2024-04-25T13:46:31.629824+0000","last_peered":"2024-04-25T13:46:31.629824+0000","last_clean":"2024-04-25T13:46:31.629824+0000","last_became_active":"2024-04-25T13:46:29.370737+0000","last_became_peered":"2024-04-25T13:46:29.370737+0000","last_unstale":"2024-04-25T13:46:31.629824+0000","last_undegraded":"2024-04-25T13:46:31.629824+0000","last_fullsized":"2024-04-25T13:46:31.629824+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:29.854734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342953+0000","last_change":"2024-04-25T13:46:28.364545+0000","last_active":"2024-04-25T13:46:30.342953+0000","last_peered":"2024-04-25T13:46:30.342953+0000","last_clean":"2024-04-25T13:46:30.342953+0000","last_became_active":"2024-04-25T13:46:28.364139+0000","last_became_peered":"2024-04-25T13:46:28.364139+0000","last_unstale":"2024-04-25T13:46:30.342953+0000","last_undegraded":"2024-04-25T13:46:30.342953+0000","last_fullsized":"2024-04-25T13:46:30.342953+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:48.068211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630370+0000","last_change":"2024-04-25T13:46:29.353318+0000","last_active":"2024-04-25T13:46:31.630370+0000","last_peered":"2024-04-25T13:46:31.630370+0000","last_clean":"2024-04-25T13:46:31.630370+0000","last_became_active":"2024-04-25T13:46:29.352744+0000","last_became_peered":"2024-04-25T13:46:29.352744+0000","last_unstale":"2024-04-25T13:46:31.630370+0000","last_undegraded":"2024-04-25T13:46:31.630370+0000","last_fullsized":"2024-04-25T13:46:31.630370+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:54.802572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622750+0000","last_change":"2024-04-25T13:46:28.363100+0000","last_active":"2024-04-25T13:46:30.622750+0000","last_peered":"2024-04-25T13:46:30.622750+0000","last_clean":"2024-04-25T13:46:30.622750+0000","last_became_active":"2024-04-25T13:46:28.361935+0000","last_became_peered":"2024-04-25T13:46:28.361935+0000","last_unstale":"2024-04-25T13:46:30.622750+0000","last_undegraded":"2024-04-25T13:46:30.622750+0000","last_fullsized":"2024-04-25T13:46:30.622750+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:29.648109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342199+0000","last_change":"2024-04-25T13:46:29.382181+0000","last_active":"2024-04-25T13:46:30.342199+0000","last_peered":"2024-04-25T13:46:30.342199+0000","last_clean":"2024-04-25T13:46:30.342199+0000","last_became_active":"2024-04-25T13:46:29.376621+0000","last_became_peered":"2024-04-25T13:46:29.376621+0000","last_unstale":"2024-04-25T13:46:30.342199+0000","last_undegraded":"2024-04-25T13:46:30.342199+0000","last_fullsized":"2024-04-25T13:46:30.342199+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:19.632433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.589048+0000","last_change":"2024-04-25T13:46:28.354914+0000","last_active":"2024-04-25T13:46:30.589048+0000","last_peered":"2024-04-25T13:46:30.589048+0000","last_clean":"2024-04-25T13:46:30.589048+0000","last_became_active":"2024-04-25T13:46:28.350518+0000","last_became_peered":"2024-04-25T13:46:28.350518+0000","last_unstale":"2024-04-25T13:46:30.589048+0000","last_undegraded":"2024-04-25T13:46:30.589048+0000","last_fullsized":"2024-04-25T13:46:30.589048+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:25.118939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342382+0000","last_change":"2024-04-25T13:46:29.346398+0000","last_active":"2024-04-25T13:46:30.342382+0000","last_peered":"2024-04-25T13:46:30.342382+0000","last_clean":"2024-04-25T13:46:30.342382+0000","last_became_active":"2024-04-25T13:46:29.346142+0000","last_became_peered":"2024-04-25T13:46:29.346142+0000","last_unstale":"2024-04-25T13:46:30.342382+0000","last_undegraded":"2024-04-25T13:46:30.342382+0000","last_fullsized":"2024-04-25T13:46:30.342382+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:29.131276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588444+0000","last_change":"2024-04-25T13:46:28.341599+0000","last_active":"2024-04-25T13:46:30.588444+0000","last_peered":"2024-04-25T13:46:30.588444+0000","last_clean":"2024-04-25T13:46:30.588444+0000","last_became_active":"2024-04-25T13:46:28.341326+0000","last_became_peered":"2024-04-25T13:46:28.341326+0000","last_unstale":"2024-04-25T13:46:30.588444+0000","last_undegraded":"2024-04-25T13:46:30.588444+0000","last_fullsized":"2024-04-25T13:46:30.588444+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:34.794066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342721+0000","last_change":"2024-04-25T13:46:29.381753+0000","last_active":"2024-04-25T13:46:30.342721+0000","last_peered":"2024-04-25T13:46:30.342721+0000","last_clean":"2024-04-25T13:46:30.342721+0000","last_became_active":"2024-04-25T13:46:29.381001+0000","last_became_peered":"2024-04-25T13:46:29.381001+0000","last_unstale":"2024-04-25T13:46:30.342721+0000","last_undegraded":"2024-04-25T13:46:30.342721+0000","last_fullsized":"2024-04-25T13:46:30.342721+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:02.375916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.628967+0000","last_change":"2024-04-25T13:46:28.347431+0000","last_active":"2024-04-25T13:46:31.628967+0000","last_peered":"2024-04-25T13:46:31.628967+0000","last_clean":"2024-04-25T13:46:31.628967+0000","last_became_active":"2024-04-25T13:46:28.346978+0000","last_became_peered":"2024-04-25T13:46:28.346978+0000","last_unstale":"2024-04-25T13:46:31.628967+0000","last_undegraded":"2024-04-25T13:46:31.628967+0000","last_fullsized":"2024-04-25T13:46:31.628967+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:49.443864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621668+0000","last_change":"2024-04-25T13:46:29.360205+0000","last_active":"2024-04-25T13:46:30.621668+0000","last_peered":"2024-04-25T13:46:30.621668+0000","last_clean":"2024-04-25T13:46:30.621668+0000","last_became_active":"2024-04-25T13:46:29.359472+0000","last_became_peered":"2024-04-25T13:46:29.359472+0000","last_unstale":"2024-04-25T13:46:30.621668+0000","last_undegraded":"2024-04-25T13:46:30.621668+0000","last_fullsized":"2024-04-25T13:46:30.621668+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:45.111194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"121'3","reported_seq":17,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629950+0000","last_change":"2024-04-25T13:46:28.366062+0000","last_active":"2024-04-25T13:46:31.629950+0000","last_peered":"2024-04-25T13:46:31.629950+0000","last_clean":"2024-04-25T13:46:31.629950+0000","last_became_active":"2024-04-25T13:46:28.365299+0000","last_became_peered":"2024-04-25T13:46:28.365299+0000","last_unstale":"2024-04-25T13:46:31.629950+0000","last_undegraded":"2024-04-25T13:46:31.629950+0000","last_fullsized":"2024-04-25T13:46:31.629950+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:36.253244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629540+0000","last_change":"2024-04-25T13:46:29.392343+0000","last_active":"2024-04-25T13:46:31.629540+0000","last_peered":"2024-04-25T13:46:31.629540+0000","last_clean":"2024-04-25T13:46:31.629540+0000","last_became_active":"2024-04-25T13:46:29.391955+0000","last_became_peered":"2024-04-25T13:46:29.391955+0000","last_unstale":"2024-04-25T13:46:31.629540+0000","last_undegraded":"2024-04-25T13:46:31.629540+0000","last_fullsized":"2024-04-25T13:46:31.629540+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:30.980780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"121'6","reported_seq":20,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342609+0000","last_change":"2024-04-25T13:46:28.356745+0000","last_active":"2024-04-25T13:46:30.342609+0000","last_peered":"2024-04-25T13:46:30.342609+0000","last_clean":"2024-04-25T13:46:30.342609+0000","last_became_active":"2024-04-25T13:46:28.355794+0000","last_became_peered":"2024-04-25T13:46:28.355794+0000","last_unstale":"2024-04-25T13:46:30.342609+0000","last_undegraded":"2024-04-25T13:46:30.342609+0000","last_fullsized":"2024-04-25T13:46:30.342609+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:57:06.865488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629235+0000","last_change":"2024-04-25T13:46:29.354533+0000","last_active":"2024-04-25T13:46:31.629235+0000","last_peered":"2024-04-25T13:46:31.629235+0000","last_clean":"2024-04-25T13:46:31.629235+0000","last_became_active":"2024-04-25T13:46:29.354309+0000","last_became_peered":"2024-04-25T13:46:29.354309+0000","last_unstale":"2024-04-25T13:46:31.629235+0000","last_undegraded":"2024-04-25T13:46:31.629235+0000","last_fullsized":"2024-04-25T13:46:31.629235+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:38.456980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.348038+0000","last_change":"2024-04-25T13:46:28.362182+0000","last_active":"2024-04-25T13:46:31.348038+0000","last_peered":"2024-04-25T13:46:31.348038+0000","last_clean":"2024-04-25T13:46:31.348038+0000","last_became_active":"2024-04-25T13:46:28.361659+0000","last_became_peered":"2024-04-25T13:46:28.361659+0000","last_unstale":"2024-04-25T13:46:31.348038+0000","last_undegraded":"2024-04-25T13:46:31.348038+0000","last_fullsized":"2024-04-25T13:46:31.348038+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:06.649906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588747+0000","last_change":"2024-04-25T13:46:29.362173+0000","last_active":"2024-04-25T13:46:30.588747+0000","last_peered":"2024-04-25T13:46:30.588747+0000","last_clean":"2024-04-25T13:46:30.588747+0000","last_became_active":"2024-04-25T13:46:29.361877+0000","last_became_peered":"2024-04-25T13:46:29.361877+0000","last_unstale":"2024-04-25T13:46:30.588747+0000","last_undegraded":"2024-04-25T13:46:30.588747+0000","last_fullsized":"2024-04-25T13:46:30.588747+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:52.608640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347194+0000","last_change":"2024-04-25T13:46:28.355020+0000","last_active":"2024-04-25T13:46:31.347194+0000","last_peered":"2024-04-25T13:46:31.347194+0000","last_clean":"2024-04-25T13:46:31.347194+0000","last_became_active":"2024-04-25T13:46:28.354039+0000","last_became_peered":"2024-04-25T13:46:28.354039+0000","last_unstale":"2024-04-25T13:46:31.347194+0000","last_undegraded":"2024-04-25T13:46:31.347194+0000","last_fullsized":"2024-04-25T13:46:31.347194+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:28.804890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342231+0000","last_change":"2024-04-25T13:46:29.362689+0000","last_active":"2024-04-25T13:46:30.342231+0000","last_peered":"2024-04-25T13:46:30.342231+0000","last_clean":"2024-04-25T13:46:30.342231+0000","last_became_active":"2024-04-25T13:46:29.359630+0000","last_became_peered":"2024-04-25T13:46:29.359630+0000","last_unstale":"2024-04-25T13:46:30.342231+0000","last_undegraded":"2024-04-25T13:46:30.342231+0000","last_fullsized":"2024-04-25T13:46:30.342231+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.678172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342682+0000","last_change":"2024-04-25T13:46:28.367780+0000","last_active":"2024-04-25T13:46:30.342682+0000","last_peered":"2024-04-25T13:46:30.342682+0000","last_clean":"2024-04-25T13:46:30.342682+0000","last_became_active":"2024-04-25T13:46:28.366553+0000","last_became_peered":"2024-04-25T13:46:28.366553+0000","last_unstale":"2024-04-25T13:46:30.342682+0000","last_undegraded":"2024-04-25T13:46:30.342682+0000","last_fullsized":"2024-04-25T13:46:30.342682+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:41.580565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347017+0000","last_change":"2024-04-25T13:46:29.366585+0000","last_active":"2024-04-25T13:46:31.347017+0000","last_peered":"2024-04-25T13:46:31.347017+0000","last_clean":"2024-04-25T13:46:31.347017+0000","last_became_active":"2024-04-25T13:46:29.366210+0000","last_became_peered":"2024-04-25T13:46:29.366210+0000","last_unstale":"2024-04-25T13:46:31.347017+0000","last_undegraded":"2024-04-25T13:46:31.347017+0000","last_fullsized":"2024-04-25T13:46:31.347017+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:41.918881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"121'8","reported_seq":22,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588045+0000","last_change":"2024-04-25T13:46:28.368314+0000","last_active":"2024-04-25T13:46:30.588045+0000","last_peered":"2024-04-25T13:46:30.588045+0000","last_clean":"2024-04-25T13:46:30.588045+0000","last_became_active":"2024-04-25T13:46:28.368036+0000","last_became_peered":"2024-04-25T13:46:28.368036+0000","last_unstale":"2024-04-25T13:46:30.588045+0000","last_undegraded":"2024-04-25T13:46:30.588045+0000","last_fullsized":"2024-04-25T13:46:30.588045+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:41.384214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622178+0000","last_change":"2024-04-25T13:46:29.382594+0000","last_active":"2024-04-25T13:46:30.622178+0000","last_peered":"2024-04-25T13:46:30.622178+0000","last_clean":"2024-04-25T13:46:30.622178+0000","last_became_active":"2024-04-25T13:46:29.382014+0000","last_became_peered":"2024-04-25T13:46:29.382014+0000","last_unstale":"2024-04-25T13:46:30.622178+0000","last_undegraded":"2024-04-25T13:46:30.622178+0000","last_fullsized":"2024-04-25T13:46:30.622178+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:43.814045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343799+0000","last_change":"2024-04-25T13:46:28.365770+0000","last_active":"2024-04-25T13:46:30.343799+0000","last_peered":"2024-04-25T13:46:30.343799+0000","last_clean":"2024-04-25T13:46:30.343799+0000","last_became_active":"2024-04-25T13:46:28.358806+0000","last_became_peered":"2024-04-25T13:46:28.358806+0000","last_unstale":"2024-04-25T13:46:30.343799+0000","last_undegraded":"2024-04-25T13:46:30.343799+0000","last_fullsized":"2024-04-25T13:46:30.343799+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:38.295721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.394593+0000","last_change":"2024-04-25T13:46:29.369548+0000","last_active":"2024-04-25T13:46:31.394593+0000","last_peered":"2024-04-25T13:46:31.394593+0000","last_clean":"2024-04-25T13:46:31.394593+0000","last_became_active":"2024-04-25T13:46:29.367940+0000","last_became_peered":"2024-04-25T13:46:29.367940+0000","last_unstale":"2024-04-25T13:46:31.394593+0000","last_undegraded":"2024-04-25T13:46:31.394593+0000","last_fullsized":"2024-04-25T13:46:31.394593+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:02.738864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621689+0000","last_change":"2024-04-25T13:46:28.350352+0000","last_active":"2024-04-25T13:46:30.621689+0000","last_peered":"2024-04-25T13:46:30.621689+0000","last_clean":"2024-04-25T13:46:30.621689+0000","last_became_active":"2024-04-25T13:46:28.349684+0000","last_became_peered":"2024-04-25T13:46:28.349684+0000","last_unstale":"2024-04-25T13:46:30.621689+0000","last_undegraded":"2024-04-25T13:46:30.621689+0000","last_fullsized":"2024-04-25T13:46:30.621689+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:59.478011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630602+0000","last_change":"2024-04-25T13:46:29.384622+0000","last_active":"2024-04-25T13:46:31.630602+0000","last_peered":"2024-04-25T13:46:31.630602+0000","last_clean":"2024-04-25T13:46:31.630602+0000","last_became_active":"2024-04-25T13:46:29.370079+0000","last_became_peered":"2024-04-25T13:46:29.370079+0000","last_unstale":"2024-04-25T13:46:31.630602+0000","last_undegraded":"2024-04-25T13:46:31.630602+0000","last_fullsized":"2024-04-25T13:46:31.630602+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:53.778017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343129+0000","last_change":"2024-04-25T13:46:28.369672+0000","last_active":"2024-04-25T13:46:30.343129+0000","last_peered":"2024-04-25T13:46:30.343129+0000","last_clean":"2024-04-25T13:46:30.343129+0000","last_became_active":"2024-04-25T13:46:28.368816+0000","last_became_peered":"2024-04-25T13:46:28.368816+0000","last_unstale":"2024-04-25T13:46:30.343129+0000","last_undegraded":"2024-04-25T13:46:30.343129+0000","last_fullsized":"2024-04-25T13:46:30.343129+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:44.300220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347950+0000","last_change":"2024-04-25T13:46:29.360114+0000","last_active":"2024-04-25T13:46:31.347950+0000","last_peered":"2024-04-25T13:46:31.347950+0000","last_clean":"2024-04-25T13:46:31.347950+0000","last_became_active":"2024-04-25T13:46:29.359116+0000","last_became_peered":"2024-04-25T13:46:29.359116+0000","last_unstale":"2024-04-25T13:46:31.347950+0000","last_undegraded":"2024-04-25T13:46:31.347950+0000","last_fullsized":"2024-04-25T13:46:31.347950+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:01.756809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343236+0000","last_change":"2024-04-25T13:46:28.369073+0000","last_active":"2024-04-25T13:46:30.343236+0000","last_peered":"2024-04-25T13:46:30.343236+0000","last_clean":"2024-04-25T13:46:30.343236+0000","last_became_active":"2024-04-25T13:46:28.368187+0000","last_became_peered":"2024-04-25T13:46:28.368187+0000","last_unstale":"2024-04-25T13:46:30.343236+0000","last_undegraded":"2024-04-25T13:46:30.343236+0000","last_fullsized":"2024-04-25T13:46:30.343236+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:08:04.696053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.346778+0000","last_change":"2024-04-25T13:46:29.366062+0000","last_active":"2024-04-25T13:46:31.346778+0000","last_peered":"2024-04-25T13:46:31.346778+0000","last_clean":"2024-04-25T13:46:31.346778+0000","last_became_active":"2024-04-25T13:46:29.363193+0000","last_became_peered":"2024-04-25T13:46:29.363193+0000","last_unstale":"2024-04-25T13:46:31.346778+0000","last_undegraded":"2024-04-25T13:46:31.346778+0000","last_fullsized":"2024-04-25T13:46:31.346778+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:29.426791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.393766+0000","last_change":"2024-04-25T13:46:28.342122+0000","last_active":"2024-04-25T13:46:31.393766+0000","last_peered":"2024-04-25T13:46:31.393766+0000","last_clean":"2024-04-25T13:46:31.393766+0000","last_became_active":"2024-04-25T13:46:28.341434+0000","last_became_peered":"2024-04-25T13:46:28.341434+0000","last_unstale":"2024-04-25T13:46:31.393766+0000","last_undegraded":"2024-04-25T13:46:31.393766+0000","last_fullsized":"2024-04-25T13:46:31.393766+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:05.178915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347005+0000","last_change":"2024-04-25T13:46:29.355378+0000","last_active":"2024-04-25T13:46:31.347005+0000","last_peered":"2024-04-25T13:46:31.347005+0000","last_clean":"2024-04-25T13:46:31.347005+0000","last_became_active":"2024-04-25T13:46:29.355122+0000","last_became_peered":"2024-04-25T13:46:29.355122+0000","last_unstale":"2024-04-25T13:46:31.347005+0000","last_undegraded":"2024-04-25T13:46:31.347005+0000","last_fullsized":"2024-04-25T13:46:31.347005+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:56.388798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.394110+0000","last_change":"2024-04-25T13:46:28.348803+0000","last_active":"2024-04-25T13:46:31.394110+0000","last_peered":"2024-04-25T13:46:31.394110+0000","last_clean":"2024-04-25T13:46:31.394110+0000","last_became_active":"2024-04-25T13:46:28.348028+0000","last_became_peered":"2024-04-25T13:46:28.348028+0000","last_unstale":"2024-04-25T13:46:31.394110+0000","last_undegraded":"2024-04-25T13:46:31.394110+0000","last_fullsized":"2024-04-25T13:46:31.394110+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:02.730612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622372+0000","last_change":"2024-04-25T13:46:29.387247+0000","last_active":"2024-04-25T13:46:30.622372+0000","last_peered":"2024-04-25T13:46:30.622372+0000","last_clean":"2024-04-25T13:46:30.622372+0000","last_became_active":"2024-04-25T13:46:29.382882+0000","last_became_peered":"2024-04-25T13:46:29.382882+0000","last_unstale":"2024-04-25T13:46:30.622372+0000","last_undegraded":"2024-04-25T13:46:30.622372+0000","last_fullsized":"2024-04-25T13:46:30.622372+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:45.318321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341912+0000","last_change":"2024-04-25T13:46:28.349758+0000","last_active":"2024-04-25T13:46:30.341912+0000","last_peered":"2024-04-25T13:46:30.341912+0000","last_clean":"2024-04-25T13:46:30.341912+0000","last_became_active":"2024-04-25T13:46:28.349053+0000","last_became_peered":"2024-04-25T13:46:28.349053+0000","last_unstale":"2024-04-25T13:46:30.341912+0000","last_undegraded":"2024-04-25T13:46:30.341912+0000","last_fullsized":"2024-04-25T13:46:30.341912+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:20.072377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342294+0000","last_change":"2024-04-25T13:46:29.360709+0000","last_active":"2024-04-25T13:46:30.342294+0000","last_peered":"2024-04-25T13:46:30.342294+0000","last_clean":"2024-04-25T13:46:30.342294+0000","last_became_active":"2024-04-25T13:46:29.360099+0000","last_became_peered":"2024-04-25T13:46:29.360099+0000","last_unstale":"2024-04-25T13:46:30.342294+0000","last_undegraded":"2024-04-25T13:46:30.342294+0000","last_fullsized":"2024-04-25T13:46:30.342294+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:18.551041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"121'2","reported_seq":16,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.394749+0000","last_change":"2024-04-25T13:46:28.358101+0000","last_active":"2024-04-25T13:46:31.394749+0000","last_peered":"2024-04-25T13:46:31.394749+0000","last_clean":"2024-04-25T13:46:31.394749+0000","last_became_active":"2024-04-25T13:46:28.357404+0000","last_became_peered":"2024-04-25T13:46:28.357404+0000","last_unstale":"2024-04-25T13:46:31.394749+0000","last_undegraded":"2024-04-25T13:46:31.394749+0000","last_fullsized":"2024-04-25T13:46:31.394749+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:23.195960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588398+0000","last_change":"2024-04-25T13:46:29.358029+0000","last_active":"2024-04-25T13:46:30.588398+0000","last_peered":"2024-04-25T13:46:30.588398+0000","last_clean":"2024-04-25T13:46:30.588398+0000","last_became_active":"2024-04-25T13:46:29.357649+0000","last_became_peered":"2024-04-25T13:46:29.357649+0000","last_unstale":"2024-04-25T13:46:30.588398+0000","last_undegraded":"2024-04-25T13:46:30.588398+0000","last_fullsized":"2024-04-25T13:46:30.588398+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:48.295079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"122'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343625+0000","last_change":"2024-04-25T13:46:28.368509+0000","last_active":"2024-04-25T13:46:30.343625+0000","last_peered":"2024-04-25T13:46:30.343625+0000","last_clean":"2024-04-25T13:46:30.343625+0000","last_became_active":"2024-04-25T13:46:28.367670+0000","last_became_peered":"2024-04-25T13:46:28.367670+0000","last_unstale":"2024-04-25T13:46:30.343625+0000","last_undegraded":"2024-04-25T13:46:30.343625+0000","last_fullsized":"2024-04-25T13:46:30.343625+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:35.914594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.588971+0000","last_change":"2024-04-25T13:46:29.389280+0000","last_active":"2024-04-25T13:46:30.588971+0000","last_peered":"2024-04-25T13:46:30.588971+0000","last_clean":"2024-04-25T13:46:30.588971+0000","last_became_active":"2024-04-25T13:46:29.388775+0000","last_became_peered":"2024-04-25T13:46:29.388775+0000","last_unstale":"2024-04-25T13:46:30.588971+0000","last_undegraded":"2024-04-25T13:46:30.588971+0000","last_fullsized":"2024-04-25T13:46:30.588971+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:25.686534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.341016+0000","last_change":"2024-04-25T13:46:28.363418+0000","last_active":"2024-04-25T13:46:30.341016+0000","last_peered":"2024-04-25T13:46:30.341016+0000","last_clean":"2024-04-25T13:46:30.341016+0000","last_became_active":"2024-04-25T13:46:28.363048+0000","last_became_peered":"2024-04-25T13:46:28.363048+0000","last_unstale":"2024-04-25T13:46:30.341016+0000","last_undegraded":"2024-04-25T13:46:30.341016+0000","last_fullsized":"2024-04-25T13:46:30.341016+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:55:45.906638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622601+0000","last_change":"2024-04-25T13:46:29.351416+0000","last_active":"2024-04-25T13:46:30.622601+0000","last_peered":"2024-04-25T13:46:30.622601+0000","last_clean":"2024-04-25T13:46:30.622601+0000","last_became_active":"2024-04-25T13:46:29.350701+0000","last_became_peered":"2024-04-25T13:46:29.350701+0000","last_unstale":"2024-04-25T13:46:30.622601+0000","last_undegraded":"2024-04-25T13:46:30.622601+0000","last_fullsized":"2024-04-25T13:46:30.622601+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:14.252201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.622451+0000","last_change":"2024-04-25T13:46:28.368555+0000","last_active":"2024-04-25T13:46:30.622451+0000","last_peered":"2024-04-25T13:46:30.622451+0000","last_clean":"2024-04-25T13:46:30.622451+0000","last_became_active":"2024-04-25T13:46:28.366374+0000","last_became_peered":"2024-04-25T13:46:28.366374+0000","last_unstale":"2024-04-25T13:46:30.622451+0000","last_undegraded":"2024-04-25T13:46:30.622451+0000","last_fullsized":"2024-04-25T13:46:30.622451+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:04.320196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.347365+0000","last_change":"2024-04-25T13:46:29.363266+0000","last_active":"2024-04-25T13:46:31.347365+0000","last_peered":"2024-04-25T13:46:31.347365+0000","last_clean":"2024-04-25T13:46:31.347365+0000","last_became_active":"2024-04-25T13:46:29.362887+0000","last_became_peered":"2024-04-25T13:46:29.362887+0000","last_unstale":"2024-04-25T13:46:31.347365+0000","last_undegraded":"2024-04-25T13:46:31.347365+0000","last_fullsized":"2024-04-25T13:46:31.347365+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:59.489360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.342047+0000","last_change":"2024-04-25T13:46:28.362120+0000","last_active":"2024-04-25T13:46:30.342047+0000","last_peered":"2024-04-25T13:46:30.342047+0000","last_clean":"2024-04-25T13:46:30.342047+0000","last_became_active":"2024-04-25T13:46:28.354018+0000","last_became_peered":"2024-04-25T13:46:28.354018+0000","last_unstale":"2024-04-25T13:46:30.342047+0000","last_undegraded":"2024-04-25T13:46:30.342047+0000","last_fullsized":"2024-04-25T13:46:30.342047+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:56.290367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.343212+0000","last_change":"2024-04-25T13:46:29.367925+0000","last_active":"2024-04-25T13:46:30.343212+0000","last_peered":"2024-04-25T13:46:30.343212+0000","last_clean":"2024-04-25T13:46:30.343212+0000","last_became_active":"2024-04-25T13:46:29.365620+0000","last_became_peered":"2024-04-25T13:46:29.365620+0000","last_unstale":"2024-04-25T13:46:30.343212+0000","last_undegraded":"2024-04-25T13:46:30.343212+0000","last_fullsized":"2024-04-25T13:46:30.343212+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:18:17.626818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621800+0000","last_change":"2024-04-25T13:46:28.359523+0000","last_active":"2024-04-25T13:46:30.621800+0000","last_peered":"2024-04-25T13:46:30.621800+0000","last_clean":"2024-04-25T13:46:30.621800+0000","last_became_active":"2024-04-25T13:46:28.356573+0000","last_became_peered":"2024-04-25T13:46:28.356573+0000","last_unstale":"2024-04-25T13:46:30.621800+0000","last_undegraded":"2024-04-25T13:46:30.621800+0000","last_fullsized":"2024-04-25T13:46:30.621800+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:52:40.243763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.348315+0000","last_change":"2024-04-25T13:46:29.383509+0000","last_active":"2024-04-25T13:46:31.348315+0000","last_peered":"2024-04-25T13:46:31.348315+0000","last_clean":"2024-04-25T13:46:31.348315+0000","last_became_active":"2024-04-25T13:46:29.383326+0000","last_became_peered":"2024-04-25T13:46:29.383326+0000","last_unstale":"2024-04-25T13:46:31.348315+0000","last_undegraded":"2024-04-25T13:46:31.348315+0000","last_fullsized":"2024-04-25T13:46:31.348315+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:00:37.302283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.630188+0000","last_change":"2024-04-25T13:46:28.354915+0000","last_active":"2024-04-25T13:46:31.630188+0000","last_peered":"2024-04-25T13:46:31.630188+0000","last_clean":"2024-04-25T13:46:31.630188+0000","last_became_active":"2024-04-25T13:46:28.354528+0000","last_became_peered":"2024-04-25T13:46:28.354528+0000","last_unstale":"2024-04-25T13:46:31.630188+0000","last_undegraded":"2024-04-25T13:46:31.630188+0000","last_fullsized":"2024-04-25T13:46:31.630188+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:55.882013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:30.621671+0000","last_change":"2024-04-25T13:46:29.372892+0000","last_active":"2024-04-25T13:46:30.621671+0000","last_peered":"2024-04-25T13:46:30.621671+0000","last_clean":"2024-04-25T13:46:30.621671+0000","last_became_active":"2024-04-25T13:46:29.371561+0000","last_became_peered":"2024-04-25T13:46:29.371561+0000","last_unstale":"2024-04-25T13:46:30.621671+0000","last_undegraded":"2024-04-25T13:46:30.621671+0000","last_fullsized":"2024-04-25T13:46:30.621671+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:33:10.120733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.348399+0000","last_change":"2024-04-25T13:46:28.372486+0000","last_active":"2024-04-25T13:46:31.348399+0000","last_peered":"2024-04-25T13:46:31.348399+0000","last_clean":"2024-04-25T13:46:31.348399+0000","last_became_active":"2024-04-25T13:46:28.372037+0000","last_became_peered":"2024-04-25T13:46:28.372037+0000","last_unstale":"2024-04-25T13:46:31.348399+0000","last_undegraded":"2024-04-25T13:46:31.348399+0000","last_fullsized":"2024-04-25T13:46:31.348399+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:06.292772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.346996+0000","last_change":"2024-04-25T13:46:29.363543+0000","last_active":"2024-04-25T13:46:31.346996+0000","last_peered":"2024-04-25T13:46:31.346996+0000","last_clean":"2024-04-25T13:46:31.346996+0000","last_became_active":"2024-04-25T13:46:29.363037+0000","last_became_peered":"2024-04-25T13:46:29.363037+0000","last_unstale":"2024-04-25T13:46:31.346996+0000","last_undegraded":"2024-04-25T13:46:31.346996+0000","last_fullsized":"2024-04-25T13:46:31.346996+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:28.321318+0000","last_clean_scrub_stamp":"2024-04-25T13:46:28.321318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:56:23.856965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"121'1","reported_seq":15,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629583+0000","last_change":"2024-04-25T13:46:28.370546+0000","last_active":"2024-04-25T13:46:31.629583+0000","last_peered":"2024-04-25T13:46:31.629583+0000","last_clean":"2024-04-25T13:46:31.629583+0000","last_became_active":"2024-04-25T13:46:28.369881+0000","last_became_peered":"2024-04-25T13:46:28.369881+0000","last_unstale":"2024-04-25T13:46:31.629583+0000","last_undegraded":"2024-04-25T13:46:31.629583+0000","last_fullsized":"2024-04-25T13:46:31.629583+0000","mapping_epoch":119,"log_start":"0'0","ondisk_log_start":"0'0","created":119,"last_epoch_clean":120,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:46:27.309242+0000","last_clean_scrub_stamp":"2024-04-25T13:46:27.309242+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:50.276099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":313,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-25T13:46:31.629755+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:46:31.629755+0000","last_peered":"2024-04-25T13:46:31.629755+0000","last_clean":"2024-04-25T13:46:31.629755+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:46:31.629755+0000","last_undegraded":"2024-04-25T13:46:31.629755+0000","last_fullsized":"2024-04-25T13:46:31.629755+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574926,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1234,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194077,"statfs":{"total":200000000,"available":198735576,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574926,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194653,"statfs":{"total":200000000,"available":199325404,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574926,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194654,"statfs":{"total":200000000,"available":199325986,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574926,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":659,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194652,"statfs":{"total":200000000,"available":199324320,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":659,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194652,"statfs":{"total":200000000,"available":199324422,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574926,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194654,"statfs":{"total":200000000,"available":199326022,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":658,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194654,"statfs":{"total":200000000,"available":199326056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574926,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1235,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194077,"statfs":{"total":200000000,"available":198735106,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:46:38.448 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:46:38.448 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:46:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.643+0000 7f897c028640 1 Processor -- start 2024-04-25T13:46:38.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.643+0000 7f897c028640 1 -- start start 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897c028640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89740b0ed0 0x7f897415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897c028640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740ce730 0x7f89740ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897c028640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f897415cda0 con 0x7f897406e490 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897c028640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8974073710 con 0x7f89740b0ed0 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897c028640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89740739f0 con 0x7f89740ce730 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897a5c5640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f897406e490 0x7f897406e8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52656/0 (socket says 172.21.15.80:52656) 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897a5c5640 1 -- 172.21.15.80:0/1636171074 learned_addr learned my addr 172.21.15.80:0/1636171074 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f89795c3640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740ce730 0x7f89740ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8979dc4640 1 --2- 172.21.15.80:0/1636171074 >> v2:172.21.15.87:3300/0 conn(0x7f89740b0ed0 0x7f897415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2351201745 0 0) 0x7f897415cda0 con 0x7f897406e490 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8960003530 con 0x7f897406e490 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1667221585 0 0) 0x7f8960003530 con 0x7f897406e490 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740ce730 msgr2=0x7f89740ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740ce730 0x7f89740ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 >> v2:172.21.15.87:3300/0 conn(0x7f89740b0ed0 msgr2=0x7f897415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 --2- 172.21.15.80:0/1636171074 >> v2:172.21.15.87:3300/0 conn(0x7f89740b0ed0 0x7f897415c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8974073cd0 con 0x7f897406e490 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f896c001a20 con 0x7f897406e490 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 >> v1:172.21.15.80:6789/0 conn(0x7f897406e490 legacy=0x7f897406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f8960003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740ce730 0x7f8960004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8960004d50 0x7f8960005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8974073710 con 0x7f89600039c0 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89740739f0 con 0x7f8960004d50 2024-04-25T13:46:38.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f896000ba50 con 0x7f89740ce730 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8979dc4640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f8960003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897a5c5640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8960004d50 0x7f8960005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f89795c3640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740ce730 0x7f8960004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8979dc4640 1 -- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740ce730 msgr2=0x7f8960004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8979dc4640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740ce730 0x7f8960004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8979dc4640 1 -- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8960004d50 msgr2=0x7f8960005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8979dc4640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8960004d50 0x7f8960005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f8979dc4640 1 -- 172.21.15.80:0/1636171074 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8960003530 con 0x7f89600039c0 2024-04-25T13:46:38.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.647+0000 7f897a5c5640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8960004d50 0x7f8960005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:38.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f8979dc4640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f8960003e30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8970009ab0 tx=0x7f897001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dbec9545a55a6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:38.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89700b2070 con 0x7f89600039c0 2024-04-25T13:46:38.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89700a55c0 con 0x7f89600039c0 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f8978dc2640 1 -- 172.21.15.80:0/1636171074 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89700096b0 con 0x7f89600039c0 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897c028640 1 -- 172.21.15.80:0/1636171074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 msgr2=0x7f8960003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897c028640 1 --2- 172.21.15.80:0/1636171074 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f8960003e30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f8970009ab0 tx=0x7f897001c9d0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897a5c5640 1 -- 172.21.15.80:0/1636171074 reap_dead start 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897c028640 1 -- 172.21.15.80:0/1636171074 shutdown_connections 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897c028640 1 -- 172.21.15.80:0/1636171074 >> 172.21.15.80:0/1636171074 conn(0x7f897406d080 msgr2=0x7f89740b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897c028640 1 -- 172.21.15.80:0/1636171074 shutdown_connections 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897c028640 1 -- 172.21.15.80:0/1636171074 wait complete. 2024-04-25T13:46:38.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897c028640 1 Processor -- start 2024-04-25T13:46:38.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.651+0000 7f897c028640 1 -- start start 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f89741575e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f897415ea30 0x7f89740b2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740b3010 0x7f89741f9d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f89740733a0 con 0x7f89600039c0 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89740b12e0 con 0x7f897415ea30 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89740ceb40 con 0x7f89740b3010 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f89741575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f89741575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43104/0 (socket says 172.21.15.80:43104) 2024-04-25T13:46:38.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 -- 172.21.15.80:0/433542001 learned_addr learned my addr 172.21.15.80:0/433542001 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897a5c5640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740b3010 0x7f89741f9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 -- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740b3010 msgr2=0x7f89741f9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740b3010 0x7f89741f9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f89795c3640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f897415ea30 0x7f89740b2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 -- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f897415ea30 msgr2=0x7f89740b2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f897415ea30 0x7f89740b2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 -- 172.21.15.80:0/433542001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89740b3420 con 0x7f89600039c0 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f8979dc4640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f89741575e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8970009ab0 tx=0x7f8960005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f896a7fc640 1 -- 172.21.15.80:0/433542001 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89700b2040 con 0x7f89600039c0 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f896a7fc640 1 -- 172.21.15.80:0/433542001 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89700a5df0 con 0x7f89600039c0 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f896a7fc640 1 -- 172.21.15.80:0/433542001 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8970004020 con 0x7f89600039c0 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f897415f460 con 0x7f89600039c0 2024-04-25T13:46:38.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89741f7cd0 con 0x7f89600039c0 2024-04-25T13:46:38.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.655+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f897406a3a0 con 0x7f89600039c0 2024-04-25T13:46:38.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.659+0000 7f896a7fc640 1 -- 172.21.15.80:0/433542001 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f89700c0030 con 0x7f89600039c0 2024-04-25T13:46:38.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.659+0000 7f896a7fc640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f895806f400 0x7f8958071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:38.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.659+0000 7f896a7fc640 1 -- 172.21.15.80:0/433542001 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f89701297f0 con 0x7f89600039c0 2024-04-25T13:46:38.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.659+0000 7f89795c3640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f895806f400 0x7f8958071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:38.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.659+0000 7f89795c3640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f895806f400 0x7f8958071910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f89740d12e0 tx=0x7f8964008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:38.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.663+0000 7f896a7fc640 1 -- 172.21.15.80:0/433542001 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89700c7060 con 0x7f89600039c0 2024-04-25T13:46:38.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.883+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f897406f880 con 0x7f89600039c0 2024-04-25T13:46:38.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f896a7fc640 1 -- 172.21.15.80:0/433542001 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f89740b3420 con 0x7f89600039c0 2024-04-25T13:46:38.885 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:46:38.885 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:46:38.885 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:46:38.885 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:46:38.885 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:46:38.886 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f895806f400 msgr2=0x7f8958071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f895806f400 0x7f8958071910 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f89740d12e0 tx=0x7f8964008040 comp rx=0 tx=0).stop 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 msgr2=0x7f89741575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f89741575e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f8970009ab0 tx=0x7f8960005840 comp rx=0 tx=0).stop 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 shutdown_connections 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f895806f400 0x7f8958071910 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89740b3010 0x7f89741f9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f897415ea30 0x7f89740b2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.887+0000 7f897c028640 1 --2- 172.21.15.80:0/433542001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89600039c0 0x7f89741575e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:38.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.891+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 >> 172.21.15.80:0/433542001 conn(0x7f897406d080 msgr2=0x7f8974158420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:38.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.891+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 shutdown_connections 2024-04-25T13:46:38.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:38.891+0000 7f897c028640 1 -- 172.21.15.80:0/433542001 wait complete. 2024-04-25T13:46:38.908 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:39.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.051+0000 7f2b4c201640 1 Processor -- start 2024-04-25T13:46:39.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.051+0000 7f2b4c201640 1 -- start start 2024-04-25T13:46:39.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b4c201640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2b4406f5a0 0x7f2b44160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b4c201640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b44161070 0x7f2b44161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b4c201640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b4406fcf0 con 0x7f2b440b4bd0 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b4c201640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2b441619c0 con 0x7f2b4406f5a0 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b4c201640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2b44073f10 con 0x7f2b44161070 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b4a79e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2b440b4bd0 0x7f2b440b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52670/0 (socket says 172.21.15.80:52670) 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b4a79e640 1 -- 172.21.15.80:0/690725085 learned_addr learned my addr 172.21.15.80:0/690725085 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b49f9d640 1 --2- 172.21.15.80:0/690725085 >> v2:172.21.15.80:3300/0 conn(0x7f2b4406f5a0 0x7f2b44160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b48f9b640 1 -- 172.21.15.80:0/690725085 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1844435960 0 0) 0x7f2b4406fcf0 con 0x7f2b440b4bd0 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b4979c640 1 --2- 172.21.15.80:0/690725085 >> v2:172.21.15.87:3300/0 conn(0x7f2b44161070 0x7f2b44161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b48f9b640 1 -- 172.21.15.80:0/690725085 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b30003530 con 0x7f2b440b4bd0 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b49f9d640 1 -- 172.21.15.80:0/690725085 >> v2:172.21.15.87:3300/0 conn(0x7f2b44161070 msgr2=0x7f2b44161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b49f9d640 1 --2- 172.21.15.80:0/690725085 >> v2:172.21.15.87:3300/0 conn(0x7f2b44161070 0x7f2b44161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b49f9d640 1 -- 172.21.15.80:0/690725085 >> v1:172.21.15.80:6789/0 conn(0x7f2b440b4bd0 legacy=0x7f2b440b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b48f9b640 1 -- 172.21.15.80:0/690725085 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1127958149 0 0) 0x7f2b30003530 con 0x7f2b440b4bd0 2024-04-25T13:46:39.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.055+0000 7f2b49f9d640 1 -- 172.21.15.80:0/690725085 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b440b0c60 con 0x7f2b4406f5a0 2024-04-25T13:46:39.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b49f9d640 1 --2- 172.21.15.80:0/690725085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b4406f5a0 0x7f2b44160b30 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2b4406ff70 tx=0x7f2b3c01c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b4f55707da9aedf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b48f9b640 1 -- 172.21.15.80:0/690725085 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c0b0070 con 0x7f2b4406f5a0 2024-04-25T13:46:39.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b48f9b640 1 -- 172.21.15.80:0/690725085 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0ab400 con 0x7f2b4406f5a0 2024-04-25T13:46:39.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b48f9b640 1 -- 172.21.15.80:0/690725085 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b40001a00 con 0x7f2b4406f5a0 2024-04-25T13:46:39.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 -- 172.21.15.80:0/690725085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b4406f5a0 msgr2=0x7f2b44160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 --2- 172.21.15.80:0/690725085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b4406f5a0 0x7f2b44160b30 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2b4406ff70 tx=0x7f2b3c01c2f0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 -- 172.21.15.80:0/690725085 shutdown_connections 2024-04-25T13:46:39.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 --2- 172.21.15.80:0/690725085 >> v2:172.21.15.87:3300/0 conn(0x7f2b44161070 0x7f2b44161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 --2- 172.21.15.80:0/690725085 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b4406f5a0 0x7f2b44160b30 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 -- 172.21.15.80:0/690725085 >> 172.21.15.80:0/690725085 conn(0x7f2b4406d080 msgr2=0x7f2b440b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 -- 172.21.15.80:0/690725085 shutdown_connections 2024-04-25T13:46:39.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 -- 172.21.15.80:0/690725085 wait complete. 2024-04-25T13:46:39.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 Processor -- start 2024-04-25T13:46:39.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.059+0000 7f2b4c201640 1 -- start start 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4406f5a0 0x7f2b44159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b440b4bd0 0x7f2b44157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b44161070 0x7f2b44157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4416a860 con 0x7f2b44161070 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b4416a6e0 con 0x7f2b4406f5a0 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b4416a9e0 con 0x7f2b440b4bd0 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b44161070 0x7f2b44157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b44161070 0x7f2b44157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43134/0 (socket says 172.21.15.80:43134) 2024-04-25T13:46:39.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 -- 172.21.15.80:0/1383003935 learned_addr learned my addr 172.21.15.80:0/1383003935 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b49f9d640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4406f5a0 0x7f2b44159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4979c640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b440b4bd0 0x7f2b44157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 -- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b440b4bd0 msgr2=0x7f2b44157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b440b4bd0 0x7f2b44157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 -- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4406f5a0 msgr2=0x7f2b44159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4406f5a0 0x7f2b44159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 -- 172.21.15.80:0/1383003935 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4415a840 con 0x7f2b44161070 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4a79e640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b44161070 0x7f2b44157f50 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000cc00 tx=0x7f2b4000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b3a7fc640 1 -- 172.21.15.80:0/1383003935 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4000ce10 con 0x7f2b44161070 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b3a7fc640 1 -- 172.21.15.80:0/1383003935 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b40012400 con 0x7f2b44161070 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4415ab20 con 0x7f2b44161070 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b3a7fc640 1 -- 172.21.15.80:0/1383003935 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b400127f0 con 0x7f2b44161070 2024-04-25T13:46:39.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b4415d650 con 0x7f2b44161070 2024-04-25T13:46:39.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b4406a3a0 con 0x7f2b44161070 2024-04-25T13:46:39.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b3a7fc640 1 -- 172.21.15.80:0/1383003935 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2b40012990 con 0x7f2b44161070 2024-04-25T13:46:39.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b3a7fc640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b2806f400 0x7f2b28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.063+0000 7f2b3a7fc640 1 -- 172.21.15.80:0/1383003935 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f2b400904e0 con 0x7f2b44161070 2024-04-25T13:46:39.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.067+0000 7f2b49f9d640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b2806f400 0x7f2b28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.067+0000 7f2b49f9d640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b2806f400 0x7f2b28071910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0ad000 tx=0x7f2b3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.067+0000 7f2b3a7fc640 1 -- 172.21.15.80:0/1383003935 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b4005d4b0 con 0x7f2b44161070 2024-04-25T13:46:39.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.247+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b440739c0 con 0x7f2b44161070 2024-04-25T13:46:39.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b3a7fc640 1 -- 172.21.15.80:0/1383003935 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5174 (secure 0 0 0) 0x7f2b4005cdb0 con 0x7f2b44161070 2024-04-25T13:46:39.248 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:39.248 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:46:28.718395+0000","modified":"2024-04-25T13:46:31.773079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/2773338889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2773338889},{"type":"v1","addr":"172.21.15.80:6835","nonce":2773338889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":7}]} 2024-04-25T13:46:39.249 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 59 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b2806f400 msgr2=0x7f2b28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b2806f400 0x7f2b28071910 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c0ad000 tx=0x7f2b3c008040 comp rx=0 tx=0).stop 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b44161070 msgr2=0x7f2b44157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b44161070 0x7f2b44157f50 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000cc00 tx=0x7f2b4000a430 comp rx=0 tx=0).stop 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 shutdown_connections 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b2806f400 0x7f2b28071910 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b44161070 0x7f2b44157f50 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b440b4bd0 0x7f2b44157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 --2- 172.21.15.80:0/1383003935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b4406f5a0 0x7f2b44159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 >> 172.21.15.80:0/1383003935 conn(0x7f2b4406d080 msgr2=0x7f2b440d1fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 shutdown_connections 2024-04-25T13:46:39.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.251+0000 7f2b4c201640 1 -- 172.21.15.80:0/1383003935 wait complete. 2024-04-25T13:46:39.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 59, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:46:28.718395+0000', 'modified': '2024-04-25T13:46:31.773079+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7509}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7509': {'gid': 7509, 'name': 'a', 'rank': 0, 'incarnation': 55, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6835/2773338889', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 2773338889}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 2773338889}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7509, 'qdb_cluster': [7509]} 2024-04-25T13:46:39.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:46:39.672 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T13:46:39.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.835+0000 7f8679256640 1 Processor -- start 2024-04-25T13:46:39.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.835+0000 7f8679256640 1 -- start start 2024-04-25T13:46:39.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f8679256640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8674161240 0x7f86740723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f8679256640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8674161650 0x7f86740728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f8679256640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f867406fcf0 con 0x7f867406f5a0 2024-04-25T13:46:39.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f8679256640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8674073f10 con 0x7f8674161240 2024-04-25T13:46:39.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f8679256640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f86740b0c60 con 0x7f8674161650 2024-04-25T13:46:39.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8674161240 0x7f86740723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867259c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8674161240 0x7f86740723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43136/0 (socket says 172.21.15.80:43136) 2024-04-25T13:46:39.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867259c640 1 -- 172.21.15.80:0/1456789808 learned_addr learned my addr 172.21.15.80:0/1456789808 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:39.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867359e640 1 --2- 172.21.15.80:0/1456789808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8674161650 0x7f86740728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867259c640 1 -- 172.21.15.80:0/1456789808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8674161650 msgr2=0x7f86740728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867259c640 1 --2- 172.21.15.80:0/1456789808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8674161650 0x7f86740728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867259c640 1 -- 172.21.15.80:0/1456789808 >> v1:172.21.15.87:6789/0 conn(0x7f867406f5a0 legacy=0x7f8674160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867259c640 1 -- 172.21.15.80:0/1456789808 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86740b0f40 con 0x7f8674161240 2024-04-25T13:46:39.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867359e640 1 --2- 172.21.15.80:0/1456789808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8674161650 0x7f86740728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:39.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f867259c640 1 --2- 172.21.15.80:0/1456789808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f86740723a0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8668009870 tx=0x7f866801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965205ac6c9f4f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f8671d9b640 1 -- 172.21.15.80:0/1456789808 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680b0070 con 0x7f8674161240 2024-04-25T13:46:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f8671d9b640 1 -- 172.21.15.80:0/1456789808 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86680ab600 con 0x7f8674161240 2024-04-25T13:46:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.839+0000 7f8671d9b640 1 -- 172.21.15.80:0/1456789808 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680ab9d0 con 0x7f8674161240 2024-04-25T13:46:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 -- 172.21.15.80:0/1456789808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 msgr2=0x7f86740723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 --2- 172.21.15.80:0/1456789808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f86740723a0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8668009870 tx=0x7f866801c490 comp rx=0 tx=0).stop 2024-04-25T13:46:39.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 -- 172.21.15.80:0/1456789808 shutdown_connections 2024-04-25T13:46:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 --2- 172.21.15.80:0/1456789808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8674161650 0x7f86740728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 --2- 172.21.15.80:0/1456789808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f86740723a0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 -- 172.21.15.80:0/1456789808 >> 172.21.15.80:0/1456789808 conn(0x7f867406d080 msgr2=0x7f86740b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 -- 172.21.15.80:0/1456789808 shutdown_connections 2024-04-25T13:46:39.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 -- 172.21.15.80:0/1456789808 wait complete. 2024-04-25T13:46:39.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 Processor -- start 2024-04-25T13:46:39.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.843+0000 7f8679256640 1 -- start start 2024-04-25T13:46:39.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8679256640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f867406f5a0 0x7f86741fda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8679256640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f867415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8679256640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8674161650 0x7f867414a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8679256640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f867416a260 con 0x7f8674161240 2024-04-25T13:46:39.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8679256640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f867416a0e0 con 0x7f8674161650 2024-04-25T13:46:39.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8679256640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f867416a3e0 con 0x7f867406f5a0 2024-04-25T13:46:39.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f867415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f867415d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43146/0 (socket says 172.21.15.80:43146) 2024-04-25T13:46:39.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 -- 172.21.15.80:0/2232573599 learned_addr learned my addr 172.21.15.80:0/2232573599 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:39.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8672d9d640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f867406f5a0 0x7f86741fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867359e640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8674161650 0x7f867414a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 -- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f867406f5a0 msgr2=0x7f86741fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f867406f5a0 0x7f86741fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 -- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8674161650 msgr2=0x7f867414a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8674161650 0x7f867414a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 -- 172.21.15.80:0/2232573599 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f867415a190 con 0x7f8674161240 2024-04-25T13:46:39.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f867259c640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f867415d5b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f86680ae040 tx=0x7f866801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f865b7fe640 1 -- 172.21.15.80:0/2232573599 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680b0040 con 0x7f8674161240 2024-04-25T13:46:39.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f865b7fe640 1 -- 172.21.15.80:0/2232573599 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86680bc070 con 0x7f8674161240 2024-04-25T13:46:39.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8679256640 1 -- 172.21.15.80:0/2232573599 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f867415bcd0 con 0x7f8674161240 2024-04-25T13:46:39.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f865b7fe640 1 -- 172.21.15.80:0/2232573599 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86680b6880 con 0x7f8674161240 2024-04-25T13:46:39.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f8679256640 1 -- 172.21.15.80:0/2232573599 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f867415dc60 con 0x7f8674161240 2024-04-25T13:46:39.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.847+0000 7f865b7fe640 1 -- 172.21.15.80:0/2232573599 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f86680ab480 con 0x7f8674161240 2024-04-25T13:46:39.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f865b7fe640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f866006f430 0x7f8660071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f865b7fe640 1 -- 172.21.15.80:0/2232573599 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f86681296f0 con 0x7f8674161240 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f8672d9d640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f866006f430 0x7f8660071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f86805b3000 1 Processor -- start 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f86805b3000 1 -- start start 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f8672d9d640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f866006f430 0x7f8660071940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f86740b6e70 tx=0x7f8664008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f86805b3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 0x5591a0a38ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f86805b3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 0x5591a0a39420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f86805b3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a39960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f86805b3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5591a08a8160 con 0x5591a04bf280 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f86805b3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5591a083de20 con 0x5591a08c6ea0 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f86805b3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5591a07f6a20 con 0x5591a08dac20 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a39960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a39960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43154/0 (socket says 172.21.15.80:43154) 2024-04-25T13:46:39.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 -- 172.21.15.80:0/3872790889 learned_addr learned my addr 172.21.15.80:0/3872790889 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f8672d9d640 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 0x5591a0a38ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867259c640 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 0x5591a0a39420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 -- 172.21.15.80:0/3872790889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 msgr2=0x5591a0a38ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 0x5591a0a38ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 -- 172.21.15.80:0/3872790889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 msgr2=0x5591a0a39420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 0x5591a0a39420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 -- 172.21.15.80:0/3872790889 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a07fb100 con 0x5591a04bf280 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f867359e640 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a39960 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f86740701f0 tx=0x7f866c00d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b0bf89451b0d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f8658ff9640 1 -- 172.21.15.80:0/3872790889 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c003ba0 con 0x5591a04bf280 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f8658ff9640 1 -- 172.21.15.80:0/3872790889 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c003d40 con 0x5591a04bf280 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.851+0000 7f8658ff9640 1 -- 172.21.15.80:0/3872790889 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f866c00f8a0 con 0x5591a04bf280 2024-04-25T13:46:39.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- 172.21.15.80:0/3872790889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 msgr2=0x5591a0a39960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a39960 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f86740701f0 tx=0x7f866c00d740 comp rx=0 tx=0).stop 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- 172.21.15.80:0/3872790889 shutdown_connections 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a39960 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 0x5591a0a39420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 --2- 172.21.15.80:0/3872790889 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 0x5591a0a38ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- 172.21.15.80:0/3872790889 >> 172.21.15.80:0/3872790889 conn(0x5591a04feff0 msgr2=0x5591a09ee4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- 172.21.15.80:0/3872790889 shutdown_connections 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- 172.21.15.80:0/3872790889 wait complete. 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- start start 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 Processor -- start 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 0x5591a0a403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a09fb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5591a084b220 con 0x5591a04bf280 2024-04-25T13:46:39.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5591a07e1ea0 con 0x5591a08dac20 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5591a0838b00 con 0x5591a08c6ea0 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 10 client.0 _ll_register_callbacks cb 0x5591a084adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a3fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43162/0 (socket says 172.21.15.80:43162) 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 -- 172.21.15.80:0/879545516 learned_addr learned my addr 172.21.15.80:0/879545516 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f867259c640 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 0x5591a0a403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f867359e640 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a09fb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 -- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 msgr2=0x5591a0a403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 0x5591a0a403a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 -- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 msgr2=0x5591a09fb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a09fb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 -- 172.21.15.80:0/879545516 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a07dd500 con 0x5591a04bf280 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f867359e640 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a09fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:39.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8672d9d640 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a3fe60 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f866400deb0 tx=0x7f8664004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62bc5e92eedb855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8651ffb640 1 -- 172.21.15.80:0/879545516 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664014e40 con 0x5591a04bf280 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- 172.21.15.80:0/879545516 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f866c003d40 con 0x5591a04bf280 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8651ffb640 1 -- 172.21.15.80:0/879545516 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f866400f040 con 0x5591a04bf280 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8651ffb640 1 -- 172.21.15.80:0/879545516 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f8664005920 con 0x5591a04bf280 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 10 client.7674 fetch_fsmap learned FSMap version 59 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- 172.21.15.80:0/879545516 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5591a07c3ea0 con 0x5591a04bf280 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8651ffb640 1 -- 172.21.15.80:0/879545516 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f866c003d40 con 0x5591a04bf280 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f8651ffb640 1 -- 172.21.15.80:0/879545516 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 59) ==== 3494+0+0 (secure 0 0 0) 0x7f8664030080 con 0x5591a04bf280 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 10 client.7674 fetch_fsmap finished waiting for FSMap version 59 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 10 client.7674 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 20 client.7674 populate_metadata read hostname 'smithi080' 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] conn(0x5591a04e37d0 0x5591a09fbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 4 client.7674 mds_command: new command op to 7509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:46:39.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.855+0000 7f86805b3000 1 -- 172.21.15.80:0/879545516 --> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591a07d6be0 con 0x5591a04e37d0 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f867259c640 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] conn(0x5591a04e37d0 0x5591a09fbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f8651ffb640 10 client.7674 ms_handle_connect on v2:172.21.15.80:6834/2773338889 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f867259c640 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] conn(0x5591a04e37d0 0x5591a09fbe60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bf64f80ffe45966 server_cookie=eecf9b45c47b39af in_seq=0 out_seq=0 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f8651ffb640 1 -- 172.21.15.80:0/879545516 <== mds.0 v2:172.21.15.80:6834/2773338889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5591a07d6be0 con 0x5591a04e37d0 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f8651ffb640 10 client.7674 handle_command_reply: tid=0 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 client.7674 shutdown 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 -- 172.21.15.80:0/879545516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 msgr2=0x5591a0a3fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a3fe60 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f866400deb0 tx=0x7f8664004690 comp rx=0 tx=0).stop 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 -- 172.21.15.80:0/879545516 shutdown_connections 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] conn(0x5591a04e37d0 0x5591a09fbe60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a09fb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f8651ffb640 0 client.7674 ms_handle_reset on v2:172.21.15.80:6834/2773338889 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 0x5591a0a403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 --2- 172.21.15.80:0/879545516 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a3fe60 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 -- 172.21.15.80:0/879545516 >> 172.21.15.80:0/879545516 conn(0x5591a04feff0 msgr2=0x5591a09eec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 -- 172.21.15.80:0/879545516 shutdown_connections 2024-04-25T13:46:39.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 1 -- 172.21.15.80:0/879545516 wait complete. 2024-04-25T13:46:39.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.859+0000 7f86805b3000 20 client.7674 trim_cache size 0 max 16384 2024-04-25T13:46:39.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.875+0000 7f86805b3000 1 Processor -- start 2024-04-25T13:46:39.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.875+0000 7f86805b3000 1 -- start start 2024-04-25T13:46:39.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a450b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 0x5591a0a4eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a0a4f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8674157910 con 0x5591a04bf280 2024-04-25T13:46:39.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8674156c70 con 0x5591a08dac20 2024-04-25T13:46:39.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f86741572c0 con 0x5591a08c6ea0 2024-04-25T13:46:39.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a450b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a450b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43178/0 (socket says 172.21.15.80:43178) 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 -- 172.21.15.80:0/2677421265 learned_addr learned my addr 172.21.15.80:0/2677421265 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 -- 172.21.15.80:0/2677421265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 msgr2=0x5591a0a4eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 --2- 172.21.15.80:0/2677421265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 0x5591a0a4eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 -- 172.21.15.80:0/2677421265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 msgr2=0x5591a0a4f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f867359e640 1 --2- 172.21.15.80:0/2677421265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a0a4f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 --2- 172.21.15.80:0/2677421265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a0a4f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 -- 172.21.15.80:0/2677421265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591a09fb430 con 0x5591a04bf280 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8672d9d640 1 --2- 172.21.15.80:0/2677421265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a450b0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f866400d530 tx=0x7f866401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=478e5d4e08e7328b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8653fff640 1 -- 172.21.15.80:0/2677421265 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664012a60 con 0x5591a04bf280 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8653fff640 1 -- 172.21.15.80:0/2677421265 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8664025070 con 0x5591a04bf280 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f8653fff640 1 -- 172.21.15.80:0/2677421265 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664012e40 con 0x5591a04bf280 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- 172.21.15.80:0/2677421265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 msgr2=0x5591a0a450b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 --2- 172.21.15.80:0/2677421265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a450b0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f866400d530 tx=0x7f866401e000 comp rx=0 tx=0).stop 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- 172.21.15.80:0/2677421265 shutdown_connections 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 --2- 172.21.15.80:0/2677421265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08dac20 0x5591a0a4f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 --2- 172.21.15.80:0/2677421265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08c6ea0 0x5591a0a4eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 --2- 172.21.15.80:0/2677421265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0a450b0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- 172.21.15.80:0/2677421265 >> 172.21.15.80:0/2677421265 conn(0x5591a04feff0 msgr2=0x5591a09f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- 172.21.15.80:0/2677421265 shutdown_connections 2024-04-25T13:46:39.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- 172.21.15.80:0/2677421265 wait complete. 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 -- start start 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.879+0000 7f86805b3000 1 Processor -- start 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0aab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 0x5591a0aab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 0x5591a0a43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x5591a083d130 con 0x5591a04bf280 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x5591a07d37d0 con 0x5591a08c6ea0 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x5591a0842450 con 0x5591a08dac20 2024-04-25T13:46:39.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 10 client.0 _ll_register_callbacks cb 0x7f866400be60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0aab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0aab380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43194/0 (socket says 172.21.15.80:43194) 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 -- 172.21.15.80:0/650518299 learned_addr learned my addr 172.21.15.80:0/650518299 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f867359e640 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 0x5591a0a43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f867259c640 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 0x5591a0aab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 -- 172.21.15.80:0/650518299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 msgr2=0x5591a0a43890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 0x5591a0a43890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 -- 172.21.15.80:0/650518299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 msgr2=0x5591a0aab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 0x5591a0aab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 -- 172.21.15.80:0/650518299 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591a07f6c70 con 0x5591a04bf280 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8672d9d640 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0aab380 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f866400d5d0 tx=0x7f8664012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a239ffdf12b5c70e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:39.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 1 -- 172.21.15.80:0/650518299 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8664038470 con 0x5591a04bf280 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 1 -- 172.21.15.80:0/650518299 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f8664038610 con 0x5591a04bf280 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- 172.21.15.80:0/650518299 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8664025070 con 0x5591a04bf280 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 1 -- 172.21.15.80:0/650518299 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) ==== 24+0+0 (secure 0 0 0) 0x7f8664049050 con 0x5591a04bf280 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 10 client.7680 fetch_fsmap learned FSMap version 59 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- 172.21.15.80:0/650518299 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=2+}) -- 0x5591a08c0180 con 0x5591a04bf280 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 1 -- 172.21.15.80:0/650518299 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8664025070 con 0x5591a04bf280 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 1 -- 172.21.15.80:0/650518299 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 59) ==== 3494+0+0 (secure 0 0 0) 0x7f866404c020 con 0x5591a04bf280 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 10 client.7680 fetch_fsmap finished waiting for FSMap version 59 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 10 client.7680 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 20 client.7680 populate_metadata read hostname 'smithi080' 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] conn(0x5591a04d1830 0x5591a0a441f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 4 client.7680 mds_command: new command op to 7509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- 172.21.15.80:0/650518299 --> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] -- command(tid 0: {"prefix": "status"}) -- 0x5591a084ba30 con 0x5591a04d1830 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f867259c640 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] conn(0x5591a04d1830 0x5591a0a441f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 10 client.7680 ms_handle_connect on v2:172.21.15.80:6834/2773338889 2024-04-25T13:46:39.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f867259c640 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] conn(0x5591a04d1830 0x5591a0a441f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92af8015b548aa5e server_cookie=9205856b08e0b341 in_seq=0 out_seq=0 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 1 -- 172.21.15.80:0/650518299 <== mds.0 v2:172.21.15.80:6834/2773338889 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5591a084ba30 con 0x5591a04d1830 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 10 client.7680 handle_command_reply: tid=0 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 client.7680 shutdown 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- 172.21.15.80:0/650518299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 msgr2=0x5591a0aab380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0aab380 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f866400d5d0 tx=0x7f8664012000 comp rx=0 tx=0).stop 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- 172.21.15.80:0/650518299 shutdown_connections 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.80:6834/2773338889,v1:172.21.15.80:6835/2773338889] conn(0x5591a04d1830 0x5591a0a441f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x5591a08dac20 0x5591a0a43890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5591a08c6ea0 0x5591a0aab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8651ffb640 0 client.7680 ms_handle_reset on v2:172.21.15.80:6834/2773338889 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 --2- 172.21.15.80:0/650518299 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x5591a04bf280 0x5591a0aab380 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- 172.21.15.80:0/650518299 >> 172.21.15.80:0/650518299 conn(0x5591a04feff0 msgr2=0x5591a0a3e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- 172.21.15.80:0/650518299 shutdown_connections 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 1 -- 172.21.15.80:0/650518299 wait complete. 2024-04-25T13:46:39.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f86805b3000 20 client.7680 trim_cache size 0 max 16384 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7509, 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 10.548748713, 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 59, 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 122, 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 122, 2024-04-25T13:46:39.887 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 16.180400457000001 2024-04-25T13:46:39.888 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:39.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 -- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f866006f430 msgr2=0x7f8660071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f866006f430 0x7f8660071940 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f86740b6e70 tx=0x7f8664008040 comp rx=0 tx=0).stop 2024-04-25T13:46:39.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 -- 172.21.15.80:0/2232573599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 msgr2=0x7f867415d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:39.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f867415d5b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f86680ae040 tx=0x7f866801acd0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 -- 172.21.15.80:0/2232573599 shutdown_connections 2024-04-25T13:46:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f866006f430 0x7f8660071940 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8674161650 0x7f867414a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8674161240 0x7f867415d5b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 --2- 172.21.15.80:0/2232573599 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f867406f5a0 0x7f86741fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 -- 172.21.15.80:0/2232573599 >> 172.21.15.80:0/2232573599 conn(0x7f867406d080 msgr2=0x7f867406ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 -- 172.21.15.80:0/2232573599 shutdown_connections 2024-04-25T13:46:39.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:39.883+0000 7f8679256640 1 -- 172.21.15.80:0/2232573599 wait complete. 2024-04-25T13:46:39.902 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:46:40.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.039+0000 7fd5f2705640 1 Processor -- start 2024-04-25T13:46:40.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.039+0000 7fd5f2705640 1 -- start start 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5f2705640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec0b0ed0 0x7fd5ec15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5f2705640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd5ec0ce730 0x7fd5ec0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5f2705640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5ec15cda0 con 0x7fd5ec06e490 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5f2705640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd5ec073710 con 0x7fd5ec0ce730 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5f2705640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ec0739f0 con 0x7fd5ec0b0ed0 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5eb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd5ec0ce730 0x7fd5ec0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5eb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd5ec0ce730 0x7fd5ec0ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43206/0 (socket says 172.21.15.80:43206) 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5eb7fe640 1 -- 172.21.15.80:0/3232718183 learned_addr learned my addr 172.21.15.80:0/3232718183 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5ebfff640 1 --2- 172.21.15.80:0/3232718183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec0b0ed0 0x7fd5ec15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5eb7fe640 1 -- 172.21.15.80:0/3232718183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec0b0ed0 msgr2=0x7fd5ec15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:40.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5eb7fe640 1 --2- 172.21.15.80:0/3232718183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec0b0ed0 0x7fd5ec15c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5eb7fe640 1 -- 172.21.15.80:0/3232718183 >> v1:172.21.15.80:6789/0 conn(0x7fd5ec06e490 legacy=0x7fd5ec06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.043+0000 7fd5eb7fe640 1 -- 172.21.15.80:0/3232718183 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec073cd0 con 0x7fd5ec0ce730 2024-04-25T13:46:40.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5eb7fe640 1 --2- 172.21.15.80:0/3232718183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0ceb40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009870 tx=0x7fd5d401ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e945c89223d2719e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:40.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5eaffd640 1 -- 172.21.15.80:0/3232718183 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40b5070 con 0x7fd5ec0ce730 2024-04-25T13:46:40.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5eaffd640 1 -- 172.21.15.80:0/3232718183 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d40a8a50 con 0x7fd5ec0ce730 2024-04-25T13:46:40.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5eaffd640 1 -- 172.21.15.80:0/3232718183 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40a8e20 con 0x7fd5ec0ce730 2024-04-25T13:46:40.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 -- 172.21.15.80:0/3232718183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 msgr2=0x7fd5ec0ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 --2- 172.21.15.80:0/3232718183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0ceb40 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fd5d4009870 tx=0x7fd5d401ed50 comp rx=0 tx=0).stop 2024-04-25T13:46:40.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 -- 172.21.15.80:0/3232718183 shutdown_connections 2024-04-25T13:46:40.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 --2- 172.21.15.80:0/3232718183 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0ceb40 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 --2- 172.21.15.80:0/3232718183 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec0b0ed0 0x7fd5ec15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 -- 172.21.15.80:0/3232718183 >> 172.21.15.80:0/3232718183 conn(0x7fd5ec06d080 msgr2=0x7fd5ec0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:40.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 -- 172.21.15.80:0/3232718183 shutdown_connections 2024-04-25T13:46:40.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 -- 172.21.15.80:0/3232718183 wait complete. 2024-04-25T13:46:40.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 Processor -- start 2024-04-25T13:46:40.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.047+0000 7fd5f2705640 1 -- start start 2024-04-25T13:46:40.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec06e490 0x7fd5ec154040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:40.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5ec0b0ed0 0x7fd5ec15ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:40.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:40.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec0733a0 con 0x7fd5ec0ce730 2024-04-25T13:46:40.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ec0b12e0 con 0x7fd5ec0b0ed0 2024-04-25T13:46:40.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ec06dc00 con 0x7fd5ec06e490 2024-04-25T13:46:40.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0b2cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43218/0 (socket says 172.21.15.80:43218) 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 -- 172.21.15.80:0/2071551683 learned_addr learned my addr 172.21.15.80:0/2071551683 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5ebfff640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec06e490 0x7fd5ec154040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5eb7fe640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5ec0b0ed0 0x7fd5ec15ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 -- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec06e490 msgr2=0x7fd5ec154040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec06e490 0x7fd5ec154040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 -- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5ec0b0ed0 msgr2=0x7fd5ec15ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5ec0b0ed0 0x7fd5ec15ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 -- 172.21.15.80:0/2071551683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ec154f80 con 0x7fd5ec0ce730 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f0ca2640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0b2cd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400c390 tx=0x7fd5e400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5e8ff9640 1 -- 172.21.15.80:0/2071551683 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4017070 con 0x7fd5ec0ce730 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5e8ff9640 1 -- 172.21.15.80:0/2071551683 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5e40052d0 con 0x7fd5ec0ce730 2024-04-25T13:46:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ec155260 con 0x7fd5ec0ce730 2024-04-25T13:46:40.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5e8ff9640 1 -- 172.21.15.80:0/2071551683 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5e4003840 con 0x7fd5ec0ce730 2024-04-25T13:46:40.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ec155820 con 0x7fd5ec0ce730 2024-04-25T13:46:40.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ec06a3a0 con 0x7fd5ec0ce730 2024-04-25T13:46:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5e8ff9640 1 -- 172.21.15.80:0/2071551683 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd5e401a070 con 0x7fd5ec0ce730 2024-04-25T13:46:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5e8ff9640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5cc06f400 0x7fd5cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5e8ff9640 1 -- 172.21.15.80:0/2071551683 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fd5e408e450 con 0x7fd5ec0ce730 2024-04-25T13:46:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.051+0000 7fd5ebfff640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5cc06f400 0x7fd5cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.055+0000 7fd5ebfff640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5cc06f400 0x7fd5cc071910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc003fd0 tx=0x7fd5dc0039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:40.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.055+0000 7fd5e8ff9640 1 -- 172.21.15.80:0/2071551683 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd5e4005480 con 0x7fd5ec0ce730 2024-04-25T13:46:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.239+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd5ec06f880 con 0x7fd5ec0ce730 2024-04-25T13:46:40.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.239+0000 7fd5e8ff9640 1 -- 172.21.15.80:0/2071551683 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) ==== 53+0+73 (secure 0 0 0) 0x7fd5e405b3d0 con 0x7fd5ec0ce730 2024-04-25T13:46:40.238 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:46:40.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5cc06f400 msgr2=0x7fd5cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5cc06f400 0x7fd5cc071910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd5dc003fd0 tx=0x7fd5dc0039a0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 msgr2=0x7fd5ec0b2cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0b2cd0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd5e400c390 tx=0x7fd5e400c790 comp rx=0 tx=0).stop 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 shutdown_connections 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd5cc06f400 0x7fd5cc071910 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5ec0ce730 0x7fd5ec0b2cd0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5ec0b0ed0 0x7fd5ec15ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 --2- 172.21.15.80:0/2071551683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5ec06e490 0x7fd5ec154040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 >> 172.21.15.80:0/2071551683 conn(0x7fd5ec06d080 msgr2=0x7fd5ec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 shutdown_connections 2024-04-25T13:46:40.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40.243+0000 7fd5f2705640 1 -- 172.21.15.80:0/2071551683 wait complete. 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:46:40.259 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:46:40.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:40.260 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:46:40.265 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:46:40.265 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:46:40.265 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:46:40.266 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42168sec preferred_lft 42168sec 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:46:40.267 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:40.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:40.268 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:46:40.316 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:46:40.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:40.317 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:46:40.370 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:46:40.370 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:46:40.370 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:46:40.370 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:46:40.371 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:46:40.371 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:46:40.371 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:46:40.371 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:46:40.371 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:46:40.371 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:46:40.371 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:46:40.371 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:46:40.371 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:46:40.371 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:46:40.372 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:46:40.372 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:46:40.372 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:46:40.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:46:40.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:40.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:40.486 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:46:40.532 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:46:40.533 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:46:40.533 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:46:40.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:40.533 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:46:40.588 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:46:40.589 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:46:40.589 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:40.636 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:46:40.637 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:46:40.637 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:40.684 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:46:40.742 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:46:40.791 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:46:40.792 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:46:40.793 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:46:40.793 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:46:40.793 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:46:40.793 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:46:40.793 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:46:40.793 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:46:40.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:40.794 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:46:40.840 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:46:40.841 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:46:40.885 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:46:40.900 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:46:40.952 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:46:40.953 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:46:40.954 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:46:40.955 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:46:40.955 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:46:40.955 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:46:40.955 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:46:40.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:40.956 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:46:40.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:46:40.995+0000 7f584e986580 -1 init, newargv = 0x7f583c001240 newargc=15 2024-04-25T13:46:40.992 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[36988]: starting ceph client 2024-04-25T13:46:40.997 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[36988]: starting fuse 2024-04-25T13:46:42.098 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:46:42.114 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:46:42.163 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:46:42.164 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:46:42.165 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:42.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:42.166 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:46:42.216 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:46:42.217 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:46:42.217 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:46:42.218 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:46:42.219 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:42.219 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:46:42.219 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:46:42.339 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:42.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:42.350 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:42.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:42.351 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36988.asok status 2024-04-25T13:46:42.527 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:42.527 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:46:42.527 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "36988", 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7698, 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:46:42.528 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 7698 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.7698 192.168.0.1:0/1083223254", 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1083223254", 2024-04-25T13:46:42.529 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:46:42.530 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 59, 2024-04-25T13:46:42.530 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 122, 2024-04-25T13:46:42.530 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:46:42.530 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:46:42.530 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:46:42.530 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:42.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:42.547 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:46:42.547 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:42.553 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:46:42.553 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:42.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:42.554 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:42.608 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:46:42.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c0a502640 1 Processor -- start 2024-04-25T13:46:42.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c0a502640 1 -- start start 2024-04-25T13:46:42.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c0a502640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c040d0270 0x7f0c0406dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:42.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c0a502640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c04163eb0 0x7f0c041642c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c0a502640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c040b2100 con 0x7f0c041614b0 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c0a502640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0c040b23e0 con 0x7f0c040d0270 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c0a502640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0c041609d0 con 0x7f0c04163eb0 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c03fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0c041614b0 0x7f0c040cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41940/0 (socket says 172.21.15.80:41940) 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c03fff640 1 -- 172.21.15.80:0/2789715474 learned_addr learned my addr 172.21.15.80:0/2789715474 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c037fe640 1 --2- 172.21.15.80:0/2789715474 >> v2:172.21.15.80:3300/0 conn(0x7f0c040d0270 0x7f0c0406dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c02ffd640 1 -- 172.21.15.80:0/2789715474 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 511246851 0 0) 0x7f0c040b2100 con 0x7f0c041614b0 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c02ffd640 1 -- 172.21.15.80:0/2789715474 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0bec003530 con 0x7f0c041614b0 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.811+0000 7f0c08a9f640 1 --2- 172.21.15.80:0/2789715474 >> v2:172.21.15.87:3300/0 conn(0x7f0c04163eb0 0x7f0c041642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c02ffd640 1 -- 172.21.15.80:0/2789715474 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1511614848 0 0) 0x7f0bec003530 con 0x7f0c041614b0 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c037fe640 1 -- 172.21.15.80:0/2789715474 >> v2:172.21.15.87:3300/0 conn(0x7f0c04163eb0 msgr2=0x7f0c041642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c037fe640 1 --2- 172.21.15.80:0/2789715474 >> v2:172.21.15.87:3300/0 conn(0x7f0c04163eb0 0x7f0c041642c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:42.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c037fe640 1 -- 172.21.15.80:0/2789715474 >> v1:172.21.15.80:6789/0 conn(0x7f0c041614b0 legacy=0x7f0c040cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:42.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c037fe640 1 -- 172.21.15.80:0/2789715474 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c04160cb0 con 0x7f0c040d0270 2024-04-25T13:46:42.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c037fe640 1 --2- 172.21.15.80:0/2789715474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c040d0270 0x7f0c0406dae0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0c04072440 tx=0x7f0bf801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccd1b29c832c486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:42.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c02ffd640 1 -- 172.21.15.80:0/2789715474 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80b0070 con 0x7f0c040d0270 2024-04-25T13:46:42.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c02ffd640 1 -- 172.21.15.80:0/2789715474 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf80ab400 con 0x7f0c040d0270 2024-04-25T13:46:42.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c02ffd640 1 -- 172.21.15.80:0/2789715474 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc001a00 con 0x7f0c040d0270 2024-04-25T13:46:42.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c0a502640 1 -- 172.21.15.80:0/2789715474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c040d0270 msgr2=0x7f0c0406dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:42.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c0a502640 1 --2- 172.21.15.80:0/2789715474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c040d0270 0x7f0c0406dae0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0c04072440 tx=0x7f0bf801c3a0 comp rx=0 tx=0).stop 2024-04-25T13:46:42.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c0a502640 1 -- 172.21.15.80:0/2789715474 shutdown_connections 2024-04-25T13:46:42.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c0a502640 1 --2- 172.21.15.80:0/2789715474 >> v2:172.21.15.87:3300/0 conn(0x7f0c04163eb0 0x7f0c041642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:42.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c0a502640 1 --2- 172.21.15.80:0/2789715474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c040d0270 0x7f0c0406dae0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:42.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c0a502640 1 -- 172.21.15.80:0/2789715474 >> 172.21.15.80:0/2789715474 conn(0x7f0c040d33a0 msgr2=0x7f0c040cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:42.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c0a502640 1 -- 172.21.15.80:0/2789715474 shutdown_connections 2024-04-25T13:46:42.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.815+0000 7f0c0a502640 1 -- 172.21.15.80:0/2789715474 wait complete. 2024-04-25T13:46:42.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 Processor -- start 2024-04-25T13:46:42.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 -- start start 2024-04-25T13:46:42.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c041614b0 0x7f0c0413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:42.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c04163eb0 0x7f0c0414d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:42.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0414f140 0x7f0c0414f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:42.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0406d3b0 con 0x7f0c04163eb0 2024-04-25T13:46:42.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0406d230 con 0x7f0c041614b0 2024-04-25T13:46:42.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0416d810 con 0x7f0c0414f140 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c04163eb0 0x7f0c0414d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c04163eb0 0x7f0c0414d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57604/0 (socket says 172.21.15.80:57604) 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 -- 172.21.15.80:0/3166582079 learned_addr learned my addr 172.21.15.80:0/3166582079 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c08a9f640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0414f140 0x7f0c0414f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c03fff640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c041614b0 0x7f0c0413f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 -- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0414f140 msgr2=0x7f0c0414f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0414f140 0x7f0c0414f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 -- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c041614b0 msgr2=0x7f0c0413f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c041614b0 0x7f0c0413f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 -- 172.21.15.80:0/3166582079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c0414eb90 con 0x7f0c04163eb0 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c037fe640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c04163eb0 0x7f0c0414d300 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80ae040 tx=0x7f0bf8019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c00ff9640 1 -- 172.21.15.80:0/3166582079 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80b0040 con 0x7f0c04163eb0 2024-04-25T13:46:42.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c00ff9640 1 -- 172.21.15.80:0/3166582079 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0bf80b9070 con 0x7f0c04163eb0 2024-04-25T13:46:42.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c0414fa90 con 0x7f0c04163eb0 2024-04-25T13:46:42.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c00ff9640 1 -- 172.21.15.80:0/3166582079 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf80b4890 con 0x7f0c04163eb0 2024-04-25T13:46:42.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.819+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c0414e1c0 con 0x7f0c04163eb0 2024-04-25T13:46:42.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.823+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c0414dc50 con 0x7f0c04163eb0 2024-04-25T13:46:42.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.823+0000 7f0c00ff9640 1 -- 172.21.15.80:0/3166582079 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0bf80b4a30 con 0x7f0c04163eb0 2024-04-25T13:46:42.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.823+0000 7f0c00ff9640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:42.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.823+0000 7f0c00ff9640 1 -- 172.21.15.80:0/3166582079 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f0bf8129cd0 con 0x7f0c04163eb0 2024-04-25T13:46:42.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.823+0000 7f0c03fff640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:42.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.823+0000 7f0c03fff640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc001570 tx=0x7f0bfc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:42.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:42.827+0000 7f0c00ff9640 1 -- 172.21.15.80:0/3166582079 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c0414fa90 con 0x7f0c04163eb0 2024-04-25T13:46:43.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.027+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c0406a3a0 con 0x7f0c04163eb0 2024-04-25T13:46:43.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.027+0000 7f0c00ff9640 1 -- 172.21.15.80:0/3166582079 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v59) ==== 53+0+73 (secure 0 0 0) 0x7f0c0414eb90 con 0x7f0c04163eb0 2024-04-25T13:46:43.025 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.027+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 msgr2=0x7f0bf0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.027+0000 7f0c0a502640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc001570 tx=0x7f0bfc009040 comp rx=0 tx=0).stop 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.027+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c04163eb0 msgr2=0x7f0c0414d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.027+0000 7f0c0a502640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c04163eb0 0x7f0c0414d300 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f0bf80ae040 tx=0x7f0bf8019140 comp rx=0 tx=0).stop 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.031+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 shutdown_connections 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.031+0000 7f0c0a502640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0bf006f400 0x7f0bf0071910 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.031+0000 7f0c0a502640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c0414f140 0x7f0c0414f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.031+0000 7f0c0a502640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c04163eb0 0x7f0c0414d300 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.031+0000 7f0c0a502640 1 --2- 172.21.15.80:0/3166582079 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c041614b0 0x7f0c0413f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:43.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.031+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 >> 172.21.15.80:0/3166582079 conn(0x7f0c040d33a0 msgr2=0x7f0c04156590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:43.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.031+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 shutdown_connections 2024-04-25T13:46:43.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:43.031+0000 7f0c0a502640 1 -- 172.21.15.80:0/3166582079 wait complete. 2024-04-25T13:46:43.047 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:46:43.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:46:43.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:46:43.048 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:46:43.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:43.048 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:46:43.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:46:43.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:46:43.048 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:46:43.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:43.048 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:46:43.055 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:46:43.055 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:43.056 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42166sec preferred_lft 42166sec 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:46:43.057 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:46:43.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:43.059 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:46:43.110 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:46:43.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:43.112 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:46:43.161 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:46:43.162 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:46:43.163 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:46:43.163 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:46:43.163 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:46:43.163 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:46:43.163 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:46:43.163 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:46:43.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:46:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:46:43.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:46:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:43.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:43.286 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:46:43.292 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:46:43.292 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:46:43.292 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:46:43.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:43.292 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:46:43.345 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:46:43.345 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:46:43.346 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:43.393 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:46:43.393 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:46:43.393 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:43.441 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:46:43.498 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:46:43.550 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:46:43.551 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:46:43.552 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:46:43.553 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:46:43.553 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:46:43.553 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:46:43.553 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:46:43.553 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:46:43.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:43.554 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:46:43.601 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:46:43.601 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:46:43.650 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:46:43.665 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:46:43.715 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:46:43.715 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:46:43.715 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:46:43.715 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:46:43.716 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:46:43.717 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:46:43.718 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:46:43.718 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:46:43.718 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:46:43.718 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:46:43.718 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:46:43.718 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:46:43.718 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:46:43.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:43.718 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:46:43.755 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:46:43.752+0000 7fe5bb192580 -1 init, newargv = 0x7fe5ac00a350 newargc=15 2024-04-25T13:46:43.756 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25098]: starting ceph client 2024-04-25T13:46:43.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25098]: starting fuse 2024-04-25T13:46:44.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:46:44.877 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:46:44.928 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:46:44.928 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:46:44.928 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:46:44.928 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:46:44.928 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:46:44.928 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:46:44.928 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:46:44.928 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:46:44.929 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:46:44.930 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:46:44.930 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:46:44.930 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:46:44.930 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:46:44.930 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:46:44.930 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:46:44.930 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:46:44.930 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:46:44.931 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:46:44.931 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:46:44.931 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:46:44.931 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:46:44.931 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:44.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:44.932 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:46:44.981 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:46:44.981 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:46:44.982 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:46:44.983 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:46:45.104 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.25098.asok 2024-04-25T13:46:45.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:46:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:45.116 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.25098.asok 2024-04-25T13:46:45.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:45.116 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.25098.asok status 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "25098", 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:46:45.305 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5759, 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5759 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2860292770 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:46:45.306 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2860292770 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5759 192.168.0.1:0/2860292770", 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/2860292770", 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 59, 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 122, 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:46:45.307 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:46:45.308 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:46:45.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:45.324 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:46:45.324 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:45.330 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:46:45.330 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:45.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:45.331 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:45.385 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:45.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f78771ba640 1 Processor -- start 2024-04-25T13:46:45.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f78771ba640 1 -- start start 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f78771ba640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78700d2c70 0x7f78700d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f78771ba640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78700b92a0 0x7f78700b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f78771ba640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78700d43f0 con 0x7f78700d5490 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f78771ba640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f78700d46d0 con 0x7f78700b92a0 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f78771ba640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f78700d49b0 con 0x7f78700d2c70 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f7875757640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78700b92a0 0x7f78700b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f7874f56640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f78700d5490 0x7f78700d2560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41956/0 (socket says 172.21.15.80:41956) 2024-04-25T13:46:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f7874f56640 1 -- 172.21.15.80:0/1634647685 learned_addr learned my addr 172.21.15.80:0/1634647685 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:45.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f786ffff640 1 --2- 172.21.15.80:0/1634647685 >> v2:172.21.15.87:3300/0 conn(0x7f78700d2c70 0x7f78700d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f7875757640 1 -- 172.21.15.80:0/1634647685 >> v2:172.21.15.87:3300/0 conn(0x7f78700d2c70 msgr2=0x7f78700d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:45.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f7875757640 1 --2- 172.21.15.80:0/1634647685 >> v2:172.21.15.87:3300/0 conn(0x7f78700d2c70 0x7f78700d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f7875757640 1 -- 172.21.15.80:0/1634647685 >> v1:172.21.15.80:6789/0 conn(0x7f78700d5490 legacy=0x7f78700d2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f7875757640 1 -- 172.21.15.80:0/1634647685 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78700d4c90 con 0x7f78700b92a0 2024-04-25T13:46:45.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.547+0000 7f786f7fe640 1 -- 172.21.15.80:0/1634647685 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2379043648 0 0) 0x7f78700d43f0 con 0x7f78700d5490 2024-04-25T13:46:45.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f7875757640 1 --2- 172.21.15.80:0/1634647685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700b92a0 0x7f78700b96b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f78700b8840 tx=0x7f786801c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1d476421ff1bf45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f786f7fe640 1 -- 172.21.15.80:0/1634647685 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78680b2070 con 0x7f78700b92a0 2024-04-25T13:46:45.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f786f7fe640 1 -- 172.21.15.80:0/1634647685 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78680a59e0 con 0x7f78700b92a0 2024-04-25T13:46:45.550 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f786f7fe640 1 -- 172.21.15.80:0/1634647685 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78680a5db0 con 0x7f78700b92a0 2024-04-25T13:46:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f78771ba640 1 -- 172.21.15.80:0/1634647685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700b92a0 msgr2=0x7f78700b96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f78771ba640 1 --2- 172.21.15.80:0/1634647685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700b92a0 0x7f78700b96b0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f78700b8840 tx=0x7f786801c900 comp rx=0 tx=0).stop 2024-04-25T13:46:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f78771ba640 1 -- 172.21.15.80:0/1634647685 shutdown_connections 2024-04-25T13:46:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f78771ba640 1 --2- 172.21.15.80:0/1634647685 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700b92a0 0x7f78700b96b0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f78771ba640 1 --2- 172.21.15.80:0/1634647685 >> v2:172.21.15.87:3300/0 conn(0x7f78700d2c70 0x7f78700d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f78771ba640 1 -- 172.21.15.80:0/1634647685 >> 172.21.15.80:0/1634647685 conn(0x7f7870073740 msgr2=0x7f787006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f78771ba640 1 -- 172.21.15.80:0/1634647685 shutdown_connections 2024-04-25T13:46:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.551+0000 7f78771ba640 1 -- 172.21.15.80:0/1634647685 wait complete. 2024-04-25T13:46:45.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 Processor -- start 2024-04-25T13:46:45.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 -- start start 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78700b92a0 0x7f78700b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700d2c70 0x7f78700ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78700d5490 0x7f78700ce920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f78700ba000 con 0x7f78700d2c70 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f78700b5140 con 0x7f78700b92a0 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f78700ba180 con 0x7f78700d5490 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700d2c70 0x7f78700ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700d2c70 0x7f78700ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57620/0 (socket says 172.21.15.80:57620) 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 -- 172.21.15.80:0/2980990001 learned_addr learned my addr 172.21.15.80:0/2980990001 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:45.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 -- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78700d5490 msgr2=0x7f78700ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78700d5490 0x7f78700ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f7874f56640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78700b92a0 0x7f78700b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 -- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78700b92a0 msgr2=0x7f78700b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78700b92a0 0x7f78700b52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 -- 172.21.15.80:0/2980990001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78700b5830 con 0x7f78700d2c70 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786ffff640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700d2c70 0x7f78700ce3e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f786400e8d0 tx=0x7f786400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786d7fa640 1 -- 172.21.15.80:0/2980990001 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7864017070 con 0x7f78700d2c70 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786d7fa640 1 -- 172.21.15.80:0/2980990001 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f786400c940 con 0x7f78700d2c70 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78700b65e0 con 0x7f78700d2c70 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f786d7fa640 1 -- 172.21.15.80:0/2980990001 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786400ccf0 con 0x7f78700d2c70 2024-04-25T13:46:45.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.555+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78700b6020 con 0x7f78700d2c70 2024-04-25T13:46:45.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.559+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f787006a3a0 con 0x7f78700d2c70 2024-04-25T13:46:45.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.559+0000 7f786d7fa640 1 -- 172.21.15.80:0/2980990001 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f786401a070 con 0x7f78700d2c70 2024-04-25T13:46:45.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.559+0000 7f786d7fa640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f785c06f430 0x7f785c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:45.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.559+0000 7f786d7fa640 1 -- 172.21.15.80:0/2980990001 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7f786408f1d0 con 0x7f78700d2c70 2024-04-25T13:46:45.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.559+0000 7f7874f56640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f785c06f430 0x7f785c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:45.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.559+0000 7f7874f56640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f785c06f430 0x7f785c071940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f78600015a0 tx=0x7f7860009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:45.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.563+0000 7f786d7fa640 1 -- 172.21.15.80:0/2980990001 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f786405c0c0 con 0x7f78700d2c70 2024-04-25T13:46:45.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.759+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f787015f9b0 con 0x7f78700d2c70 2024-04-25T13:46:45.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.759+0000 7f786d7fa640 1 -- 172.21.15.80:0/2980990001 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5174 (secure 0 0 0) 0x7f786405b9c0 con 0x7f78700d2c70 2024-04-25T13:46:45.758 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:45.758 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:46:28.718395+0000","modified":"2024-04-25T13:46:31.773079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/2773338889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2773338889},{"type":"v1","addr":"172.21.15.80:6835","nonce":2773338889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":7}]} 2024-04-25T13:46:45.759 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 59 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f785c06f430 msgr2=0x7f785c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f785c06f430 0x7f785c071940 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f78600015a0 tx=0x7f7860009040 comp rx=0 tx=0).stop 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700d2c70 msgr2=0x7f78700ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700d2c70 0x7f78700ce3e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f786400e8d0 tx=0x7f786400ecd0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 shutdown_connections 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f785c06f430 0x7f785c071940 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78700d5490 0x7f78700ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78700d2c70 0x7f78700ce3e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 --2- 172.21.15.80:0/2980990001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f78700b92a0 0x7f78700b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 >> 172.21.15.80:0/2980990001 conn(0x7f7870073740 msgr2=0x7f78700cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:45.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 shutdown_connections 2024-04-25T13:46:45.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:45.763+0000 7f78771ba640 1 -- 172.21.15.80:0/2980990001 wait complete. 2024-04-25T13:46:45.781 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:46:45.946 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":7509,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.595307357999999,"mdsmap_epoch":59,"osdmap_epoch":122,"osdmap_epoch_barrier":122,"uptime":22.226981541000001} 2024-04-25T13:46:45.947 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 7509, "mdsmap_epoch": 59, "osdmap_epoch": 122, "osdmap_epoch_barrier": 122, "rank_uptime": 16.595307358, "state": "up:active", "uptime": 22.226981541, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:46:45.948 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-25T13:46:46.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce848c9640 1 Processor -- start 2024-04-25T13:46:46.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce848c9640 1 -- start start 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce848c9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fce800d0f50 0x7fce80161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce848c9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce800b0e10 con 0x7fce8015d520 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce848c9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce800b10f0 con 0x7fce800b48c0 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce848c9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fce800b0bc0 con 0x7fce800d0f50 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce77fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fce800d0f50 0x7fce80161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce7ed9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fce8015d520 0x7fce8015d930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41968/0 (socket says 172.21.15.80:41968) 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.107+0000 7fce7ed9d640 1 -- 172.21.15.80:0/3929369600 learned_addr learned my addr 172.21.15.80:0/3929369600 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce7dd9b640 1 -- 172.21.15.80:0/3929369600 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 50068531 0 0) 0x7fce800b0e10 con 0x7fce8015d520 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce7dd9b640 1 -- 172.21.15.80:0/3929369600 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fce6c005740 con 0x7fce8015d520 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce77fff640 1 -- 172.21.15.80:0/3929369600 >> v1:172.21.15.87:6789/0 conn(0x7fce800b48c0 legacy=0x7fce800d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce77fff640 1 -- 172.21.15.80:0/3929369600 >> v1:172.21.15.80:6789/0 conn(0x7fce8015d520 legacy=0x7fce8015d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce77fff640 1 -- 172.21.15.80:0/3929369600 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8006f710 con 0x7fce800d0f50 2024-04-25T13:46:46.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce77fff640 1 --2- 172.21.15.80:0/3929369600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce800d0f50 0x7fce80161600 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fce680097c0 tx=0x7fce6801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c81e865d78b3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce7dd9b640 1 -- 172.21.15.80:0/3929369600 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce680b0070 con 0x7fce800d0f50 2024-04-25T13:46:46.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce7dd9b640 1 -- 172.21.15.80:0/3929369600 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce680ab600 con 0x7fce800d0f50 2024-04-25T13:46:46.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce7dd9b640 1 -- 172.21.15.80:0/3929369600 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce680ab9d0 con 0x7fce800d0f50 2024-04-25T13:46:46.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce848c9640 1 -- 172.21.15.80:0/3929369600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce800d0f50 msgr2=0x7fce80161600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce848c9640 1 --2- 172.21.15.80:0/3929369600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce800d0f50 0x7fce80161600 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fce680097c0 tx=0x7fce6801c410 comp rx=0 tx=0).stop 2024-04-25T13:46:46.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce848c9640 1 -- 172.21.15.80:0/3929369600 shutdown_connections 2024-04-25T13:46:46.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce848c9640 1 --2- 172.21.15.80:0/3929369600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce800d0f50 0x7fce80161600 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce848c9640 1 -- 172.21.15.80:0/3929369600 >> 172.21.15.80:0/3929369600 conn(0x7fce800b35c0 msgr2=0x7fce800b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce848c9640 1 -- 172.21.15.80:0/3929369600 shutdown_connections 2024-04-25T13:46:46.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.111+0000 7fce848c9640 1 -- 172.21.15.80:0/3929369600 wait complete. 2024-04-25T13:46:46.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce848c9640 1 Processor -- start 2024-04-25T13:46:46.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce848c9640 1 -- start start 2024-04-25T13:46:46.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce848c9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce800b48c0 0x7fce8015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce848c9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce8015d520 0x7fce80150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce848c9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce80071eb0 0x7fce800722c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce848c9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce800d00c0 con 0x7fce8015d520 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce848c9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce800cff40 con 0x7fce80071eb0 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce848c9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce800d0240 con 0x7fce800b48c0 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce8015d520 0x7fce80150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce8015d520 0x7fce80150650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57634/0 (socket says 172.21.15.80:57634) 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 -- 172.21.15.80:0/161739050 learned_addr learned my addr 172.21.15.80:0/161739050 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce7ed9d640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce80071eb0 0x7fce800722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 -- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce800b48c0 msgr2=0x7fce8015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce800b48c0 0x7fce8015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 -- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce80071eb0 msgr2=0x7fce800722c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce80071eb0 0x7fce800722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 -- 172.21.15.80:0/161739050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce80072800 con 0x7fce8015d520 2024-04-25T13:46:46.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce77fff640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce8015d520 0x7fce80150650 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fce68019020 tx=0x7fce680191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce76ffd640 1 -- 172.21.15.80:0/161739050 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce680b0040 con 0x7fce8015d520 2024-04-25T13:46:46.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce76ffd640 1 -- 172.21.15.80:0/161739050 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce680ba070 con 0x7fce8015d520 2024-04-25T13:46:46.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.115+0000 7fce76ffd640 1 -- 172.21.15.80:0/161739050 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce680b4630 con 0x7fce8015d520 2024-04-25T13:46:46.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.119+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce80072a80 con 0x7fce8015d520 2024-04-25T13:46:46.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.119+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce80073040 con 0x7fce8015d520 2024-04-25T13:46:46.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.119+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce8006a440 con 0x7fce8015d520 2024-04-25T13:46:46.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.119+0000 7fce76ffd640 1 -- 172.21.15.80:0/161739050 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fce680b47d0 con 0x7fce8015d520 2024-04-25T13:46:46.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.119+0000 7fce76ffd640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce6006f400 0x7fce60071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:46.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.119+0000 7fce76ffd640 1 -- 172.21.15.80:0/161739050 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5709+0+0 (secure 0 0 0) 0x7fce68129000 con 0x7fce8015d520 2024-04-25T13:46:46.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.119+0000 7fce7e59c640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce6006f400 0x7fce60071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:46.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.119+0000 7fce7e59c640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce6006f400 0x7fce60071910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fce70000f70 tx=0x7fce700079e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:46.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.123+0000 7fce76ffd640 1 -- 172.21.15.80:0/161739050 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce80072800 con 0x7fce8015d520 2024-04-25T13:46:46.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.287+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7fce8015c4c0 con 0x7fce8015d520 2024-04-25T13:46:46.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.443+0000 7fce76ffd640 1 -- 172.21.15.80:0/161739050 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v123) ==== 181+0+0 (secure 0 0 0) 0x7fce68016090 con 0x7fce8015d520 2024-04-25T13:46:46.441 INFO:teuthology.orchestra.run.smithi080.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce6006f400 msgr2=0x7fce60071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce6006f400 0x7fce60071910 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fce70000f70 tx=0x7fce700079e0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce8015d520 msgr2=0x7fce80150650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:46.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce8015d520 0x7fce80150650 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fce68019020 tx=0x7fce680191b0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 shutdown_connections 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce6006f400 0x7fce60071910 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce80071eb0 0x7fce800722c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce8015d520 0x7fce80150650 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 --2- 172.21.15.80:0/161739050 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce800b48c0 0x7fce8015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 >> 172.21.15.80:0/161739050 conn(0x7fce800b35c0 msgr2=0x7fce8015ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 shutdown_connections 2024-04-25T13:46:46.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46.447+0000 7fce848c9640 1 -- 172.21.15.80:0/161739050 wait complete. 2024-04-25T13:46:46.468 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:46:46.468 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:46:46.468 DEBUG:teuthology.orchestra.run.smithi080:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-25T13:46:46.468 DEBUG:teuthology.orchestra.run.smithi080:> os.fchmod(fd, 0O666) 2024-04-25T13:46:46.468 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(fd) 2024-04-25T13:46:46.468 DEBUG:teuthology.orchestra.run.smithi080:> os.close(fd) 2024-04-25T13:46:46.468 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:46:46.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:46.546 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:46:46.546 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:46:46.547 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:46:46.548 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:46:46.661 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:46.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:46.672 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:46.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:46.672 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36988.asok status 2024-04-25T13:46:46.856 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:46.856 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "36988", 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 1, 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7698, 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:46:46.857 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 7698 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:46.858 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.7698 192.168.0.1:0/1083223254", 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1083223254", 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 2, 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 59, 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 122, 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 122, 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:46:46.859 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:46.873 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2024-04-25T13:46:47.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.027+0000 7f0232094640 1 Processor -- start 2024-04-25T13:46:47.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.027+0000 7f0232094640 1 -- start start 2024-04-25T13:46:47.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f0232094640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0b0ed0 0x7f022c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f0232094640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f022c06eba0 0x7f022c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f0232094640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f022c15cda0 con 0x7f022c0ce730 2024-04-25T13:46:47.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f0232094640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f022c073710 con 0x7f022c06eba0 2024-04-25T13:46:47.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f0232094640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f022c0739f0 con 0x7f022c0b0ed0 2024-04-25T13:46:47.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f022bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f022c06eba0 0x7f022c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f022bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f022c06eba0 0x7f022c0ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57648/0 (socket says 172.21.15.80:57648) 2024-04-25T13:46:47.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f022bfff640 1 -- 172.21.15.80:0/2576151989 learned_addr learned my addr 172.21.15.80:0/2576151989 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:47.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f022b7fe640 1 --2- 172.21.15.80:0/2576151989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0b0ed0 0x7f022c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f022bfff640 1 -- 172.21.15.80:0/2576151989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0b0ed0 msgr2=0x7f022c15c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:47.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f022bfff640 1 --2- 172.21.15.80:0/2576151989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0b0ed0 0x7f022c15c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f022bfff640 1 -- 172.21.15.80:0/2576151989 >> v1:172.21.15.87:6789/0 conn(0x7f022c0ce730 legacy=0x7f022c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:47.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.031+0000 7f022bfff640 1 -- 172.21.15.80:0/2576151989 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c073cd0 con 0x7f022c06eba0 2024-04-25T13:46:47.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f022bfff640 1 --2- 172.21.15.80:0/2576151989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c06eba0 0x7f022c0ceb40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0224009e50 tx=0x7f022401f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d1a5978259ce0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:47.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f022affd640 1 -- 172.21.15.80:0/2576151989 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02240b4070 con 0x7f022c06eba0 2024-04-25T13:46:47.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f022affd640 1 -- 172.21.15.80:0/2576151989 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02240af400 con 0x7f022c06eba0 2024-04-25T13:46:47.032 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f022affd640 1 -- 172.21.15.80:0/2576151989 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02240af7d0 con 0x7f022c06eba0 2024-04-25T13:46:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 -- 172.21.15.80:0/2576151989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c06eba0 msgr2=0x7f022c0ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 --2- 172.21.15.80:0/2576151989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c06eba0 0x7f022c0ceb40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f0224009e50 tx=0x7f022401f1e0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 -- 172.21.15.80:0/2576151989 shutdown_connections 2024-04-25T13:46:47.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 --2- 172.21.15.80:0/2576151989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c06eba0 0x7f022c0ceb40 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 --2- 172.21.15.80:0/2576151989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0b0ed0 0x7f022c15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 -- 172.21.15.80:0/2576151989 >> 172.21.15.80:0/2576151989 conn(0x7f022c06d080 msgr2=0x7f022c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:47.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 -- 172.21.15.80:0/2576151989 shutdown_connections 2024-04-25T13:46:47.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 -- 172.21.15.80:0/2576151989 wait complete. 2024-04-25T13:46:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 Processor -- start 2024-04-25T13:46:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.035+0000 7f0232094640 1 -- start start 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c0b0ed0 0x7f022c140b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0ce730 0x7f022c15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f022c15a3f0 0x7f022c13df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f022c0733a0 con 0x7f022c0b0ed0 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f022c0b12e0 con 0x7f022c15a3f0 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f022c1608f0 con 0x7f022c0ce730 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c0b0ed0 0x7f022c140b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c0b0ed0 0x7f022c140b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57652/0 (socket says 172.21.15.80:57652) 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 -- 172.21.15.80:0/3017009943 learned_addr learned my addr 172.21.15.80:0/3017009943 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022bfff640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f022c15a3f0 0x7f022c13df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0223fff640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0ce730 0x7f022c15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 -- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0ce730 msgr2=0x7f022c15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0ce730 0x7f022c15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 -- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f022c15a3f0 msgr2=0x7f022c13df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f022c15a3f0 0x7f022c13df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 -- 172.21.15.80:0/3017009943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f022c15ab70 con 0x7f022c0b0ed0 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022bfff640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f022c15a3f0 0x7f022c13df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0223fff640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0ce730 0x7f022c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f022b7fe640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c0b0ed0 0x7f022c140b00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f021c00d5f0 tx=0x7f021c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0228ff9640 1 -- 172.21.15.80:0/3017009943 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c0125f0 con 0x7f022c0b0ed0 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0228ff9640 1 -- 172.21.15.80:0/3017009943 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f021c019070 con 0x7f022c0b0ed0 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0228ff9640 1 -- 172.21.15.80:0/3017009943 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f021c0129b0 con 0x7f022c0b0ed0 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f022c15ae50 con 0x7f022c0b0ed0 2024-04-25T13:46:47.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f022c15b410 con 0x7f022c0b0ed0 2024-04-25T13:46:47.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f022c06a3a0 con 0x7f022c0b0ed0 2024-04-25T13:46:47.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.039+0000 7f0228ff9640 1 -- 172.21.15.80:0/3017009943 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f021c012b50 con 0x7f022c0b0ed0 2024-04-25T13:46:47.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.043+0000 7f0228ff9640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f020c06f400 0x7f020c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.043+0000 7f0228ff9640 1 -- 172.21.15.80:0/3017009943 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5709+0+0 (secure 0 0 0) 0x7f021c08f680 con 0x7f022c0b0ed0 2024-04-25T13:46:47.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.043+0000 7f0223fff640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f020c06f400 0x7f020c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.043+0000 7f0223fff640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f020c06f400 0x7f020c071910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f022c0cdba0 tx=0x7f0214008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:47.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.047+0000 7f0228ff9640 1 -- 172.21.15.80:0/3017009943 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f021c05c690 con 0x7f022c0b0ed0 2024-04-25T13:46:47.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.215+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) -- 0x7f022c155a40 con 0x7f022c0b0ed0 2024-04-25T13:46:47.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.447+0000 7f0228ff9640 1 -- 172.21.15.80:0/3017009943 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v124) ==== 93+0+0 (secure 0 0 0) 0x7f021c05bf90 con 0x7f022c0b0ed0 2024-04-25T13:46:47.444 INFO:teuthology.orchestra.run.smithi080.stderr:pauserd,pausewr is set 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.447+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f020c06f400 msgr2=0x7f020c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.447+0000 7f0232094640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f020c06f400 0x7f020c071910 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f022c0cdba0 tx=0x7f0214008040 comp rx=0 tx=0).stop 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.447+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c0b0ed0 msgr2=0x7f022c140b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.447+0000 7f0232094640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c0b0ed0 0x7f022c140b00 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f021c00d5f0 tx=0x7f021c00dab0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.451+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 shutdown_connections 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.451+0000 7f0232094640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f020c06f400 0x7f020c071910 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.451+0000 7f0232094640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f022c15a3f0 0x7f022c13df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.451+0000 7f0232094640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f022c0ce730 0x7f022c15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.451+0000 7f0232094640 1 --2- 172.21.15.80:0/3017009943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f022c0b0ed0 0x7f022c140b00 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.451+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 >> 172.21.15.80:0/3017009943 conn(0x7f022c06d080 msgr2=0x7f022c06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:47.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.451+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 shutdown_connections 2024-04-25T13:46:47.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.451+0000 7f0232094640 1 -- 172.21.15.80:0/3017009943 wait complete. 2024-04-25T13:46:47.472 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2024-04-25T13:46:47.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.627+0000 7f96da7fc640 1 Processor -- start 2024-04-25T13:46:47.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.627+0000 7f96da7fc640 1 -- start start 2024-04-25T13:46:47.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d40b0ed0 0x7f96d415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96da7fc640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d415cda0 con 0x7f96d406eba0 2024-04-25T13:46:47.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96da7fc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96d4073710 con 0x7f96d40ce730 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96da7fc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f96d40739f0 con 0x7f96d40b0ed0 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d8d99640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f96d406eba0 0x7f96d41608f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41980/0 (socket says 172.21.15.80:41980) 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d8d99640 1 -- 172.21.15.80:0/870884455 learned_addr learned my addr 172.21.15.80:0/870884455 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d3fff640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d40b0ed0 0x7f96d415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2034852759 0 0) 0x7f96d415cda0 con 0x7f96d406eba0 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f96c0003530 con 0x7f96d406eba0 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2041871081 0 0) 0x7f96c0003530 con 0x7f96d406eba0 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d40b0ed0 msgr2=0x7f96d415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d40b0ed0 0x7f96d415c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 >> v1:172.21.15.87:6789/0 conn(0x7f96d40ce730 legacy=0x7f96d406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d4073cd0 con 0x7f96d406eba0 2024-04-25T13:46:47.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f96cc001a00 con 0x7f96d406eba0 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 >> v1:172.21.15.80:6789/0 conn(0x7f96d406eba0 legacy=0x7f96d41608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96c00039f0 0x7f96c0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d40b0ed0 0x7f96c0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96c0004d50 0x7f96c0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f96d40739f0 con 0x7f96c00039f0 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f96c000bb90 con 0x7f96c0004d50 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f96c000bd90 con 0x7f96d40b0ed0 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d3fff640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96c00039f0 0x7f96c0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96cb7fe640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d40b0ed0 0x7f96c0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d8d99640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96c0004d50 0x7f96c0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d3fff640 1 -- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d40b0ed0 msgr2=0x7f96c0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d3fff640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d40b0ed0 0x7f96c0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d3fff640 1 -- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96c0004d50 msgr2=0x7f96c0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:47.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d3fff640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96c0004d50 0x7f96c0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d3fff640 1 -- 172.21.15.80:0/870884455 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c0003530 con 0x7f96c00039f0 2024-04-25T13:46:47.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.631+0000 7f96d8d99640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96c0004d50 0x7f96c0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96d3fff640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96c00039f0 0x7f96c0003e60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009ae0 tx=0x7f96c401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766fc7a56f7b282 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40b2070 con 0x7f96c00039f0 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96c40a55c0 con 0x7f96c00039f0 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96d37fe640 1 -- 172.21.15.80:0/870884455 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c40096b0 con 0x7f96c00039f0 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96da7fc640 1 -- 172.21.15.80:0/870884455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96c00039f0 msgr2=0x7f96c0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96da7fc640 1 --2- 172.21.15.80:0/870884455 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96c00039f0 0x7f96c0003e60 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009ae0 tx=0x7f96c401c9d0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96d8d99640 1 -- 172.21.15.80:0/870884455 reap_dead start 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96da7fc640 1 -- 172.21.15.80:0/870884455 shutdown_connections 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96da7fc640 1 -- 172.21.15.80:0/870884455 >> 172.21.15.80:0/870884455 conn(0x7f96d406d080 msgr2=0x7f96d40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:47.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96da7fc640 1 -- 172.21.15.80:0/870884455 shutdown_connections 2024-04-25T13:46:47.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96da7fc640 1 -- 172.21.15.80:0/870884455 wait complete. 2024-04-25T13:46:47.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96da7fc640 1 Processor -- start 2024-04-25T13:46:47.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.635+0000 7f96da7fc640 1 -- start start 2024-04-25T13:46:47.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96d4141580 0x7f96d4146fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96d41456f0 0x7f96d415e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d415e550 0x7f96d413d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f96d40733a0 con 0x7f96d41456f0 2024-04-25T13:46:47.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f96d40b12e0 con 0x7f96d4141580 2024-04-25T13:46:47.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f96d415c860 con 0x7f96d415e550 2024-04-25T13:46:47.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96d41456f0 0x7f96d415e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96d41456f0 0x7f96d415e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57666/0 (socket says 172.21.15.80:57666) 2024-04-25T13:46:47.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 -- 172.21.15.80:0/3514241909 learned_addr learned my addr 172.21.15.80:0/3514241909 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:47.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d8d99640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d415e550 0x7f96d413d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d3fff640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96d4141580 0x7f96d4146fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 -- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d415e550 msgr2=0x7f96d413d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d415e550 0x7f96d413d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 -- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96d4141580 msgr2=0x7f96d4146fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96d4141580 0x7f96d4146fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 -- 172.21.15.80:0/3514241909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96d4145f40 con 0x7f96d41456f0 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96cb7fe640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96d41456f0 0x7f96d415e010 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f96bc001ec0 tx=0x7f96bc00e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc019070 con 0x7f96d41456f0 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96bc005320 con 0x7f96d41456f0 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc007290 con 0x7f96d41456f0 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96d4146220 con 0x7f96d41456f0 2024-04-25T13:46:47.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96d41467e0 con 0x7f96d41456f0 2024-04-25T13:46:47.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96d406a3a0 con 0x7f96d41456f0 2024-04-25T13:46:47.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f96bc01c070 con 0x7f96d41456f0 2024-04-25T13:46:47.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d17fa640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96b406f400 0x7f96b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:47.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 5709+0+0 (secure 0 0 0) 0x7f96bc08e820 con 0x7f96d41456f0 2024-04-25T13:46:47.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.639+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=125+}) -- 0x7f96b4076fe0 con 0x7f96d41456f0 2024-04-25T13:46:47.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.643+0000 7f96d3fff640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96b406f400 0x7f96b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:47.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.643+0000 7f96d3fff640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96b406f400 0x7f96b4071910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f96c40af000 tx=0x7f96c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:47.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.647+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f96bc05b830 con 0x7f96d41456f0 2024-04-25T13:46:47.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:47.815+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) -- 0x7f96d406f880 con 0x7f96d41456f0 2024-04-25T13:46:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.463+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v125) ==== 97+0+0 (secure 0 0 0) 0x7f96bc05b130 con 0x7f96d41456f0 2024-04-25T13:46:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:pauserd,pausewr is unset 2024-04-25T13:46:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.467+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 <== mon.0 v2:172.21.15.80:3300/0 8 ==== osd_map(125..125 src has 1..125) ==== 288+0+0 (secure 0 0 0) 0x7f96bc08ce60 con 0x7f96d41456f0 2024-04-25T13:46:48.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.467+0000 7f96d17fa640 1 -- 172.21.15.80:0/3514241909 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=126}) -- 0x7f96b4077940 con 0x7f96d41456f0 2024-04-25T13:46:48.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96b406f400 msgr2=0x7f96b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:48.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96b406f400 0x7f96b4071910 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f96c40af000 tx=0x7f96c4008040 comp rx=0 tx=0).stop 2024-04-25T13:46:48.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96d41456f0 msgr2=0x7f96d415e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:48.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96d41456f0 0x7f96d415e010 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f96bc001ec0 tx=0x7f96bc00e8a0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 shutdown_connections 2024-04-25T13:46:48.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96b406f400 0x7f96b4071910 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f96d415e550 0x7f96d413d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96d41456f0 0x7f96d415e010 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 --2- 172.21.15.80:0/3514241909 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96d4141580 0x7f96d4146fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 >> 172.21.15.80:0/3514241909 conn(0x7f96d406d080 msgr2=0x7f96d406dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:48.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 shutdown_connections 2024-04-25T13:46:48.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.471+0000 7f96da7fc640 1 -- 172.21.15.80:0/3514241909 wait complete. 2024-04-25T13:46:48.493 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:48.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.643+0000 7f960ec95640 1 Processor -- start 2024-04-25T13:46:48.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.643+0000 7f960ec95640 1 -- start start 2024-04-25T13:46:48.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f960ec95640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9608161f90 0x7f96080cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:48.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f960ec95640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f96080d0510 0x7f960815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f960ec95640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f960815ea70 0x7f960815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f960ec95640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f96080d0d70 con 0x7f96080d0510 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f960ec95640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f96080d1050 con 0x7f9608161f90 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f960ec95640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f960806dca0 con 0x7f960815ea70 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f9607fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f96080d0510 0x7f960815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f9607fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f96080d0510 0x7f960815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57676/0 (socket says 172.21.15.80:57676) 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f9607fff640 1 -- 172.21.15.80:0/2884351875 learned_addr learned my addr 172.21.15.80:0/2884351875 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f960d232640 1 --2- 172.21.15.80:0/2884351875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f960815ea70 0x7f960815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f960ca31640 1 --2- 172.21.15.80:0/2884351875 >> v2:172.21.15.87:3300/0 conn(0x7f9608161f90 0x7f96080cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f9607fff640 1 -- 172.21.15.80:0/2884351875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f960815ea70 msgr2=0x7f960815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f9607fff640 1 --2- 172.21.15.80:0/2884351875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f960815ea70 0x7f960815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f9607fff640 1 -- 172.21.15.80:0/2884351875 >> v2:172.21.15.87:3300/0 conn(0x7f9608161f90 msgr2=0x7f96080cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f9607fff640 1 --2- 172.21.15.80:0/2884351875 >> v2:172.21.15.87:3300/0 conn(0x7f9608161f90 0x7f96080cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.646 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.647+0000 7f9607fff640 1 -- 172.21.15.80:0/2884351875 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f960806df80 con 0x7f96080d0510 2024-04-25T13:46:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f9607fff640 1 --2- 172.21.15.80:0/2884351875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96080d0510 0x7f960815e530 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f96080d17b0 tx=0x7f95fc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5168da8ab02565a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f96077fe640 1 -- 172.21.15.80:0/2884351875 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b1070 con 0x7f96080d0510 2024-04-25T13:46:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f96077fe640 1 -- 172.21.15.80:0/2884351875 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95fc0a8050 con 0x7f96080d0510 2024-04-25T13:46:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f96077fe640 1 -- 172.21.15.80:0/2884351875 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0ac860 con 0x7f96080d0510 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 -- 172.21.15.80:0/2884351875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96080d0510 msgr2=0x7f960815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 --2- 172.21.15.80:0/2884351875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96080d0510 0x7f960815e530 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f96080d17b0 tx=0x7f95fc01c460 comp rx=0 tx=0).stop 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 -- 172.21.15.80:0/2884351875 shutdown_connections 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 --2- 172.21.15.80:0/2884351875 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f960815ea70 0x7f960815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 --2- 172.21.15.80:0/2884351875 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f96080d0510 0x7f960815e530 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 --2- 172.21.15.80:0/2884351875 >> v2:172.21.15.87:3300/0 conn(0x7f9608161f90 0x7f96080cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 -- 172.21.15.80:0/2884351875 >> 172.21.15.80:0/2884351875 conn(0x7f960806d080 msgr2=0x7f96080b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 -- 172.21.15.80:0/2884351875 shutdown_connections 2024-04-25T13:46:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 -- 172.21.15.80:0/2884351875 wait complete. 2024-04-25T13:46:48.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 Processor -- start 2024-04-25T13:46:48.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.651+0000 7f960ec95640 1 -- start start 2024-04-25T13:46:48.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96080d0510 0x7f960813f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:48.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f960815ea70 0x7f960813d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:48.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9608161f90 0x7f9608167280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:48.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f96081447a0 con 0x7f960815ea70 2024-04-25T13:46:48.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9608144620 con 0x7f96080d0510 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f96080d0920 con 0x7f9608161f90 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f960815ea70 0x7f960813d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f960815ea70 0x7f960813d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57678/0 (socket says 172.21.15.80:57678) 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 -- 172.21.15.80:0/3219385877 learned_addr learned my addr 172.21.15.80:0/3219385877 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960d232640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9608161f90 0x7f9608167280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ca31640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96080d0510 0x7f960813f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 -- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9608161f90 msgr2=0x7f9608167280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9608161f90 0x7f9608167280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 -- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96080d0510 msgr2=0x7f960813f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96080d0510 0x7f960813f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 -- 172.21.15.80:0/3219385877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9608073270 con 0x7f960815ea70 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960d232640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9608161f90 0x7f9608167280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f9607fff640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f960815ea70 0x7f960813d580 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f95fc01ab10 tx=0x7f95fc01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:48.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f96057fa640 1 -- 172.21.15.80:0/3219385877 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0bc070 con 0x7f960815ea70 2024-04-25T13:46:48.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f96057fa640 1 -- 172.21.15.80:0/3219385877 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95fc0b7650 con 0x7f960815ea70 2024-04-25T13:46:48.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f96057fa640 1 -- 172.21.15.80:0/3219385877 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b1040 con 0x7f960815ea70 2024-04-25T13:46:48.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f960813e860 con 0x7f960815ea70 2024-04-25T13:46:48.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96081460c0 con 0x7f960815ea70 2024-04-25T13:46:48.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.655+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f960806a3a0 con 0x7f960815ea70 2024-04-25T13:46:48.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.659+0000 7f96057fa640 1 -- 172.21.15.80:0/3219385877 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95fc0bf070 con 0x7f960815ea70 2024-04-25T13:46:48.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.659+0000 7f96057fa640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95f406f430 0x7f95f4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:48.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.659+0000 7f96057fa640 1 -- 172.21.15.80:0/3219385877 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f95fc128d90 con 0x7f960815ea70 2024-04-25T13:46:48.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.659+0000 7f960ca31640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95f406f430 0x7f95f4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:48.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.659+0000 7f960ca31640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95f406f430 0x7f95f4071940 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f960806fa20 tx=0x7f95f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:48.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.663+0000 7f96057fa640 1 -- 172.21.15.80:0/3219385877 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f95fc016110 con 0x7f960815ea70 2024-04-25T13:46:48.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.823+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9608072fc0 con 0x7f960815ea70 2024-04-25T13:46:48.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.827+0000 7f96057fa640 1 -- 172.21.15.80:0/3219385877 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16555 (secure 0 0 0) 0x7f95fc0f5d00 con 0x7f960815ea70 2024-04-25T13:46:48.826 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:48.826 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":125,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:48.458377+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:48.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95f406f430 msgr2=0x7f95f4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:48.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95f406f430 0x7f95f4071940 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f960806fa20 tx=0x7f95f8008040 comp rx=0 tx=0).stop 2024-04-25T13:46:48.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f960815ea70 msgr2=0x7f960813d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:48.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f960815ea70 0x7f960813d580 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f95fc01ab10 tx=0x7f95fc01aca0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 shutdown_connections 2024-04-25T13:46:48.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f95f406f430 0x7f95f4071940 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9608161f90 0x7f9608167280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f960815ea70 0x7f960813d580 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 --2- 172.21.15.80:0/3219385877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f96080d0510 0x7f960813f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:48.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 >> 172.21.15.80:0/3219385877 conn(0x7f960806d080 msgr2=0x7f9608141180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:48.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 shutdown_connections 2024-04-25T13:46:48.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48.831+0000 7f960ec95640 1 -- 172.21.15.80:0/3219385877 wait complete. 2024-04-25T13:46:48.848 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:46:48.848 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:46:48.849 DEBUG:teuthology.orchestra.run.smithi080:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-25T13:46:48.849 DEBUG:teuthology.orchestra.run.smithi080:> os.fchmod(fd, 0O666) 2024-04-25T13:46:48.849 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(fd) 2024-04-25T13:46:48.849 DEBUG:teuthology.orchestra.run.smithi080:> os.close(fd) 2024-04-25T13:46:48.849 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:46:48.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:46:48.917 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:46:48.918 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:46:48.919 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:48.919 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:46:48.919 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:46:49.030 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:49.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:49.042 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:49.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:49.042 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36988.asok status 2024-04-25T13:46:49.212 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:46:49.212 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "36988", 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 1, 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7698, 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:46:49.213 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 7698 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.7698 192.168.0.1:0/1083223254", 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1083223254", 2024-04-25T13:46:49.214 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 2, 2024-04-25T13:46:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 59, 2024-04-25T13:46:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 122, 2024-04-25T13:46:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 122, 2024-04-25T13:46:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:46:49.215 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:46:49.215 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:49.228 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:49.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.379+0000 7f666d903640 1 Processor -- start 2024-04-25T13:46:49.378 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.379+0000 7f666d903640 1 -- start start 2024-04-25T13:46:49.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f666d903640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6668161f90 0x7f66680cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:49.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f666d903640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f666815ec40 0x7f666815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:49.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f666d903640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66680d0d70 con 0x7f66680d0510 2024-04-25T13:46:49.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f666d903640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f66680d1050 con 0x7f666815ec40 2024-04-25T13:46:49.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f666d903640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f666806dca0 con 0x7f6668161f90 2024-04-25T13:46:49.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f66677fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f666815ec40 0x7f666815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:49.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f6666ffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6668161f90 0x7f66680cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:49.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f665ffff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f66680d0510 0x7f666815e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:51986/0 (socket says 172.21.15.80:51986) 2024-04-25T13:46:49.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f66677fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f666815ec40 0x7f666815f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:48468/0 (socket says 172.21.15.80:48468) 2024-04-25T13:46:49.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f6666ffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6668161f90 0x7f66680cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:45332/0 (socket says 172.21.15.80:45332) 2024-04-25T13:46:49.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f665ffff640 1 -- 172.21.15.80:0/3235298813 learned_addr learned my addr 172.21.15.80:0/3235298813 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:49.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f6666ffd640 1 -- 172.21.15.80:0/3235298813 >> v2:172.21.15.87:3300/0 conn(0x7f666815ec40 msgr2=0x7f666815f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:49.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f6666ffd640 1 --2- 172.21.15.80:0/3235298813 >> v2:172.21.15.87:3300/0 conn(0x7f666815ec40 0x7f666815f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f6666ffd640 1 -- 172.21.15.80:0/3235298813 >> v1:172.21.15.87:6789/0 conn(0x7f66680d0510 legacy=0x7f666815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:49.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.383+0000 7f6666ffd640 1 -- 172.21.15.80:0/3235298813 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f666806df80 con 0x7f6668161f90 2024-04-25T13:46:49.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f6666ffd640 1 --2- 172.21.15.80:0/3235298813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6668161f90 0x7f66680cffd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f66580097c0 tx=0x7f665801c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c78752fb6c980c5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:49.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f66667fc640 1 -- 172.21.15.80:0/3235298813 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580b0070 con 0x7f6668161f90 2024-04-25T13:46:49.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f66667fc640 1 -- 172.21.15.80:0/3235298813 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66580ab600 con 0x7f6668161f90 2024-04-25T13:46:49.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f66667fc640 1 -- 172.21.15.80:0/3235298813 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66580ab9d0 con 0x7f6668161f90 2024-04-25T13:46:49.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 -- 172.21.15.80:0/3235298813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6668161f90 msgr2=0x7f66680cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:49.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 --2- 172.21.15.80:0/3235298813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6668161f90 0x7f66680cffd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f66580097c0 tx=0x7f665801c3b0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 -- 172.21.15.80:0/3235298813 shutdown_connections 2024-04-25T13:46:49.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 --2- 172.21.15.80:0/3235298813 >> v2:172.21.15.87:3300/0 conn(0x7f666815ec40 0x7f666815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 --2- 172.21.15.80:0/3235298813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6668161f90 0x7f66680cffd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 -- 172.21.15.80:0/3235298813 >> 172.21.15.80:0/3235298813 conn(0x7f666806d080 msgr2=0x7f66680b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:49.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 -- 172.21.15.80:0/3235298813 shutdown_connections 2024-04-25T13:46:49.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 -- 172.21.15.80:0/3235298813 wait complete. 2024-04-25T13:46:49.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 Processor -- start 2024-04-25T13:46:49.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.387+0000 7f666d903640 1 -- start start 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66680d0510 0x7f666813fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666815ec40 0x7f6668140fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6668161f90 0x7f66681faee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f666806f630 con 0x7f666815ec40 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f666806f4b0 con 0x7f6668161f90 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f66680d0920 con 0x7f66680d0510 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666815ec40 0x7f6668140fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666815ec40 0x7f6668140fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57684/0 (socket says 172.21.15.80:57684) 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 -- 172.21.15.80:0/988790854 learned_addr learned my addr 172.21.15.80:0/988790854 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f6666ffd640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66680d0510 0x7f666813fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f66677fe640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6668161f90 0x7f66681faee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:49.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 -- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66680d0510 msgr2=0x7f666813fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66680d0510 0x7f666813fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 -- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6668161f90 msgr2=0x7f66681faee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6668161f90 0x7f66681faee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 -- 172.21.15.80:0/988790854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66681403d0 con 0x7f666815ec40 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f66677fe640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6668161f90 0x7f66681faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f665ffff640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666815ec40 0x7f6668140fd0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f665000c3c0 tx=0x7f665000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f6664ff9640 1 -- 172.21.15.80:0/988790854 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6650017070 con 0x7f666815ec40 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f6664ff9640 1 -- 172.21.15.80:0/988790854 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6650005cf0 con 0x7f666815ec40 2024-04-25T13:46:49.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f6664ff9640 1 -- 172.21.15.80:0/988790854 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66500072f0 con 0x7f666815ec40 2024-04-25T13:46:49.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6668159c20 con 0x7f666815ec40 2024-04-25T13:46:49.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f666815a1e0 con 0x7f666815ec40 2024-04-25T13:46:49.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.391+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f666806a3a0 con 0x7f666815ec40 2024-04-25T13:46:49.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.395+0000 7f6664ff9640 1 -- 172.21.15.80:0/988790854 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f665001a070 con 0x7f666815ec40 2024-04-25T13:46:49.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.395+0000 7f6664ff9640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664806f400 0x7f6648071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:49.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.395+0000 7f6664ff9640 1 -- 172.21.15.80:0/988790854 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f665008e430 con 0x7f666815ec40 2024-04-25T13:46:49.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.395+0000 7f6666ffd640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664806f400 0x7f6648071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:49.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.395+0000 7f6666ffd640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664806f400 0x7f6648071910 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f66580ad000 tx=0x7f66580a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:49.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.399+0000 7f6664ff9640 1 -- 172.21.15.80:0/988790854 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f665005ad60 con 0x7f666815ec40 2024-04-25T13:46:49.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.595+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66680b3880 con 0x7f666815ec40 2024-04-25T13:46:49.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f6664ff9640 1 -- 172.21.15.80:0/988790854 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5174 (secure 0 0 0) 0x7f66500633a0 con 0x7f666815ec40 2024-04-25T13:46:49.596 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:49.596 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:46:28.718395+0000","modified":"2024-04-25T13:46:31.773079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/2773338889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2773338889},{"type":"v1","addr":"172.21.15.80:6835","nonce":2773338889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":7}]} 2024-04-25T13:46:49.596 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 59 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664806f400 msgr2=0x7f6648071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664806f400 0x7f6648071910 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f66580ad000 tx=0x7f66580a6000 comp rx=0 tx=0).stop 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666815ec40 msgr2=0x7f6668140fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666815ec40 0x7f6668140fd0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f665000c3c0 tx=0x7f665000c880 comp rx=0 tx=0).stop 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 shutdown_connections 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f664806f400 0x7f6648071910 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6668161f90 0x7f66681faee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f666815ec40 0x7f6668140fd0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 --2- 172.21.15.80:0/988790854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f66680d0510 0x7f666813fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 >> 172.21.15.80:0/988790854 conn(0x7f666806d080 msgr2=0x7f666815be20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 shutdown_connections 2024-04-25T13:46:49.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49.599+0000 7f666d903640 1 -- 172.21.15.80:0/988790854 wait complete. 2024-04-25T13:46:49.618 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 125 2024-04-25T13:46:49.787 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-04-25T13:46:49.787 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:46:49.787 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:46:49.787 DEBUG:teuthology.orchestra.run.smithi080:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2024-04-25T13:46:49.787 DEBUG:teuthology.orchestra.run.smithi080:> os.fchmod(fd, 0O666) 2024-04-25T13:46:49.787 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(fd) 2024-04-25T13:46:49.787 DEBUG:teuthology.orchestra.run.smithi080:> os.close(fd) 2024-04-25T13:46:49.787 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:46:49.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:49.860 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:46:49.861 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:46:49.981 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:49.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:49.992 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:49.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:49.992 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36988.asok status 2024-04-25T13:46:50.173 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:50.173 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:46:50.173 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "36988", 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 2, 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7698, 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:46:50.174 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 7698 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.7698 192.168.0.1:0/1083223254", 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1083223254", 2024-04-25T13:46:50.175 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 3, 2024-04-25T13:46:50.176 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 59, 2024-04-25T13:46:50.176 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 125, 2024-04-25T13:46:50.176 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 125, 2024-04-25T13:46:50.176 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:46:50.176 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:46:50.176 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:46:50.191 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:46:50.192 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:46:50.193 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:46:50.193 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:46:50.268 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:50.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:46:50.279 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.36988.asok 2024-04-25T13:46:50.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:50.280 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.36988.asok status 2024-04-25T13:46:50.462 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "36988", 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 2, 2024-04-25T13:46:50.463 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7698, 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 7698 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:46:50.464 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1083223254 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.7698 192.168.0.1:0/1083223254", 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/1083223254", 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 3, 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 59, 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 125, 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 125, 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:46:50.465 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:46:50.480 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:46:50.480 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:46:50.480 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:46:50.529 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:46:50.587 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:46:50.587 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:46:50.587 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:50.588 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:50.595 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:50.595 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:50.645 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:50.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae5127f640 1 Processor -- start 2024-04-25T13:46:50.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae5127f640 1 -- start start 2024-04-25T13:46:50.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae5127f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c0d0270 0x7fae4c06d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae5127f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae4c168320 con 0x7fae4c14ae90 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae5127f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fae4c06db50 con 0x7fae4c168de0 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae5127f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fae4c06de30 con 0x7fae4c0d0270 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae4affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fae4c14ae90 0x7fae4c0cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41992/0 (socket says 172.21.15.80:41992) 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae4affd640 1 -- 172.21.15.80:0/3110552402 learned_addr learned my addr 172.21.15.80:0/3110552402 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2328679049 0 0) 0x7fae4c168320 con 0x7fae4c14ae90 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fae38003530 con 0x7fae4c14ae90 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae4a7fc640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c0d0270 0x7fae4c06d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.843+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1537052217 0 0) 0x7fae38003530 con 0x7fae4c14ae90 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c0d0270 msgr2=0x7fae4c06d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c0d0270 0x7fae4c06d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 >> v1:172.21.15.87:6789/0 conn(0x7fae4c168de0 legacy=0x7fae4c0d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:50.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c1685e0 con 0x7fae4c14ae90 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fae44001a00 con 0x7fae4c14ae90 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 >> v1:172.21.15.80:6789/0 conn(0x7fae4c14ae90 legacy=0x7fae4c0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae380039f0 0x7fae38003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae38004810 0x7fae38004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae38005160 0x7fae3800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c06de30 con 0x7fae38005160 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fae3800c010 con 0x7fae380039f0 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fae3800c210 con 0x7fae38004810 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4b7fe640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae38005160 0x7fae3800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4affd640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae380039f0 0x7fae38003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4a7fc640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae38004810 0x7fae38004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4b7fe640 1 -- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae38004810 msgr2=0x7fae38004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4b7fe640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae38004810 0x7fae38004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:50.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4b7fe640 1 -- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae380039f0 msgr2=0x7fae38003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:50.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4b7fe640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae380039f0 0x7fae38003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:50.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4b7fe640 1 -- 172.21.15.80:0/3110552402 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae38003530 con 0x7fae38005160 2024-04-25T13:46:50.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4affd640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae380039f0 0x7fae38003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:50.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4b7fe640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae38005160 0x7fae3800b9d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fae3c001780 tx=0x7fae3c01c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f19c50fcd4be0d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:50.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0013a0 con 0x7fae38005160 2024-04-25T13:46:50.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3c0a9070 con 0x7fae38005160 2024-04-25T13:46:50.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae49ffb640 1 -- 172.21.15.80:0/3110552402 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c003600 con 0x7fae38005160 2024-04-25T13:46:50.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae5127f640 1 -- 172.21.15.80:0/3110552402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae38005160 msgr2=0x7fae3800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:50.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae5127f640 1 --2- 172.21.15.80:0/3110552402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae38005160 0x7fae3800b9d0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fae3c001780 tx=0x7fae3c01c420 comp rx=0 tx=0).stop 2024-04-25T13:46:50.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae4b7fe640 1 -- 172.21.15.80:0/3110552402 reap_dead start 2024-04-25T13:46:50.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae5127f640 1 -- 172.21.15.80:0/3110552402 shutdown_connections 2024-04-25T13:46:50.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae5127f640 1 -- 172.21.15.80:0/3110552402 >> 172.21.15.80:0/3110552402 conn(0x7fae4c0d33a0 msgr2=0x7fae4c0cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:50.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae5127f640 1 -- 172.21.15.80:0/3110552402 shutdown_connections 2024-04-25T13:46:50.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.847+0000 7fae5127f640 1 -- 172.21.15.80:0/3110552402 wait complete. 2024-04-25T13:46:50.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 Processor -- start 2024-04-25T13:46:50.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 -- start start 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae38005160 0x7fae4c0709b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c155b90 0x7fae4c146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae4c146e40 0x7fae4c06f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c0d0270 con 0x7fae4c146e40 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fae4c1691f0 con 0x7fae38005160 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fae4c0d03f0 con 0x7fae4c155b90 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae4c146e40 0x7fae4c06f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae4c146e40 0x7fae4c06f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57700/0 (socket says 172.21.15.80:57700) 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 -- 172.21.15.80:0/2876060448 learned_addr learned my addr 172.21.15.80:0/2876060448 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4affd640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae38005160 0x7fae4c0709b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:50.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4a7fc640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c155b90 0x7fae4c146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 -- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c155b90 msgr2=0x7fae4c146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c155b90 0x7fae4c146900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 -- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae38005160 msgr2=0x7fae4c0709b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae38005160 0x7fae4c0709b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 -- 172.21.15.80:0/2876060448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae4c147250 con 0x7fae4c146e40 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4a7fc640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c155b90 0x7fae4c146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4affd640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae38005160 0x7fae4c0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae4b7fe640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae4c146e40 0x7fae4c06f1b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fae3c001780 tx=0x7fae38005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae437fe640 1 -- 172.21.15.80:0/2876060448 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0ba020 con 0x7fae4c146e40 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae437fe640 1 -- 172.21.15.80:0/2876060448 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fae3c007240 con 0x7fae4c146e40 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae437fe640 1 -- 172.21.15.80:0/2876060448 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3c007a40 con 0x7fae4c146e40 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae4c164da0 con 0x7fae4c146e40 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.851+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae4c1fe4f0 con 0x7fae4c146e40 2024-04-25T13:46:50.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.855+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae4c06a3a0 con 0x7fae4c146e40 2024-04-25T13:46:50.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.855+0000 7fae437fe640 1 -- 172.21.15.80:0/2876060448 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fae3c0bd070 con 0x7fae4c146e40 2024-04-25T13:46:50.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.855+0000 7fae437fe640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae2c06f400 0x7fae2c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:50.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.855+0000 7fae437fe640 1 -- 172.21.15.80:0/2876060448 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fae3c1284c0 con 0x7fae4c146e40 2024-04-25T13:46:50.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.855+0000 7fae4affd640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae2c06f400 0x7fae2c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:50.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.855+0000 7fae4affd640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae2c06f400 0x7fae2c071910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fae44002240 tx=0x7fae44001520 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:50.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:50.859+0000 7fae437fe640 1 -- 172.21.15.80:0/2876060448 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fae4c147250 con 0x7fae4c146e40 2024-04-25T13:46:51.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.023+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fae4c0b5bc0 con 0x7fae4c146e40 2024-04-25T13:46:51.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.023+0000 7fae437fe640 1 -- 172.21.15.80:0/2876060448 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16555 (secure 0 0 0) 0x7fae3c016130 con 0x7fae4c146e40 2024-04-25T13:46:51.023 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:51.023 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":125,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:48.458377+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:51.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae2c06f400 msgr2=0x7fae2c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae2c06f400 0x7fae2c071910 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fae44002240 tx=0x7fae44001520 comp rx=0 tx=0).stop 2024-04-25T13:46:51.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae4c146e40 msgr2=0x7fae4c06f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae4c146e40 0x7fae4c06f1b0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fae3c001780 tx=0x7fae38005840 comp rx=0 tx=0).stop 2024-04-25T13:46:51.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 shutdown_connections 2024-04-25T13:46:51.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fae2c06f400 0x7fae2c071910 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fae4c146e40 0x7fae4c06f1b0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fae4c155b90 0x7fae4c146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 --2- 172.21.15.80:0/2876060448 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fae38005160 0x7fae4c0709b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 >> 172.21.15.80:0/2876060448 conn(0x7fae4c0d33a0 msgr2=0x7fae4c164070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:51.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 shutdown_connections 2024-04-25T13:46:51.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.027+0000 7fae5127f640 1 -- 172.21.15.80:0/2876060448 wait complete. 2024-04-25T13:46:51.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:46:27.136651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:46:27.696536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:46:51.046 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:46:51.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad7fbc640 1 Processor -- start 2024-04-25T13:46:51.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad7fbc640 1 -- start start 2024-04-25T13:46:51.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad7fbc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ad00b5330 0x7f2ad00d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad7fbc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2ad00d0ff0 0x7f2ad006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad7fbc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad006f2f0 0x7f2ad0161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad7fbc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad006f700 con 0x7f2ad00b5330 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad7fbc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2ad01629b0 con 0x7f2ad00d0ff0 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad7fbc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad00b5740 con 0x7f2ad006f2f0 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5d58640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ad00b5330 0x7f2ad00d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5d58640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ad00b5330 0x7f2ad00d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38260/0 (socket says 172.21.15.80:38260) 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5d58640 1 -- 172.21.15.80:0/2568116184 learned_addr learned my addr 172.21.15.80:0/2568116184 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5557640 1 --2- 172.21.15.80:0/2568116184 >> v2:172.21.15.87:3300/0 conn(0x7f2ad00d0ff0 0x7f2ad006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad6559640 1 --2- 172.21.15.80:0/2568116184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad006f2f0 0x7f2ad0161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5d58640 1 -- 172.21.15.80:0/2568116184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad006f2f0 msgr2=0x7f2ad0161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5d58640 1 --2- 172.21.15.80:0/2568116184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad006f2f0 0x7f2ad0161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5d58640 1 -- 172.21.15.80:0/2568116184 >> v2:172.21.15.87:3300/0 conn(0x7f2ad00d0ff0 msgr2=0x7f2ad006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5d58640 1 --2- 172.21.15.80:0/2568116184 >> v2:172.21.15.87:3300/0 conn(0x7f2ad00d0ff0 0x7f2ad006edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.202+0000 7f2ad5d58640 1 -- 172.21.15.80:0/2568116184 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad006dd10 con 0x7f2ad00b5330 2024-04-25T13:46:51.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad5d58640 1 --2- 172.21.15.80:0/2568116184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad00b5330 0x7f2ad00d0ab0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0074290 tx=0x7f2acc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8529c680e27f1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad4d56640 1 -- 172.21.15.80:0/2568116184 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0b0070 con 0x7f2ad00b5330 2024-04-25T13:46:51.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad4d56640 1 -- 172.21.15.80:0/2568116184 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc0ab600 con 0x7f2ad00b5330 2024-04-25T13:46:51.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad4d56640 1 -- 172.21.15.80:0/2568116184 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0ab9d0 con 0x7f2ad00b5330 2024-04-25T13:46:51.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/2568116184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad00b5330 msgr2=0x7f2ad00d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/2568116184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad00b5330 0x7f2ad00d0ab0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0074290 tx=0x7f2acc01c490 comp rx=0 tx=0).stop 2024-04-25T13:46:51.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/2568116184 shutdown_connections 2024-04-25T13:46:51.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/2568116184 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad006f2f0 0x7f2ad0161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/2568116184 >> v2:172.21.15.87:3300/0 conn(0x7f2ad00d0ff0 0x7f2ad006edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/2568116184 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad00b5330 0x7f2ad00d0ab0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/2568116184 >> 172.21.15.80:0/2568116184 conn(0x7f2ad00b4160 msgr2=0x7f2ad00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:51.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/2568116184 shutdown_connections 2024-04-25T13:46:51.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.206+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/2568116184 wait complete. 2024-04-25T13:46:51.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad7fbc640 1 Processor -- start 2024-04-25T13:46:51.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad7fbc640 1 -- start start 2024-04-25T13:46:51.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad7fbc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad006f2f0 0x7f2ad013d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad7fbc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad00b5330 0x7f2ad013d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad7fbc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad00d0ff0 0x7f2ad01403d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad7fbc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad00b0cd0 con 0x7f2ad006f2f0 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad7fbc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad006e890 con 0x7f2ad00d0ff0 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad7fbc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad00b0e50 con 0x7f2ad00b5330 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad5d58640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad006f2f0 0x7f2ad013d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad5d58640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad006f2f0 0x7f2ad013d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38274/0 (socket says 172.21.15.80:38274) 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad5d58640 1 -- 172.21.15.80:0/806357953 learned_addr learned my addr 172.21.15.80:0/806357953 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad6559640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad00d0ff0 0x7f2ad01403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.210+0000 7f2ad5557640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad00b5330 0x7f2ad013d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad5d58640 1 -- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad00b5330 msgr2=0x7f2ad013d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad5d58640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad00b5330 0x7f2ad013d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad5d58640 1 -- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad00d0ff0 msgr2=0x7f2ad01403d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad5d58640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad00d0ff0 0x7f2ad01403d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad5d58640 1 -- 172.21.15.80:0/806357953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad013f9c0 con 0x7f2ad006f2f0 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad5d58640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad006f2f0 0x7f2ad013d440 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2acc000c10 tx=0x7f2ad015f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ac67fc640 1 -- 172.21.15.80:0/806357953 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0b0040 con 0x7f2ad006f2f0 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ac67fc640 1 -- 172.21.15.80:0/806357953 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2acc0b9070 con 0x7f2ad006f2f0 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad0141b10 con 0x7f2ad006f2f0 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ac67fc640 1 -- 172.21.15.80:0/806357953 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc0b4610 con 0x7f2ad006f2f0 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad013df20 con 0x7f2ad006f2f0 2024-04-25T13:46:51.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad013f4e0 con 0x7f2ad006f2f0 2024-04-25T13:46:51.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ac67fc640 1 -- 172.21.15.80:0/806357953 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2acc0b47b0 con 0x7f2ad006f2f0 2024-04-25T13:46:51.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ac67fc640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ab406f400 0x7f2ab4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ac67fc640 1 -- 172.21.15.80:0/806357953 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f2acc129850 con 0x7f2ad006f2f0 2024-04-25T13:46:51.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.214+0000 7f2ad5557640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ab406f400 0x7f2ab4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.218+0000 7f2ad5557640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ab406f400 0x7f2ab4071910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00d1a30 tx=0x7f2ac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.218+0000 7f2ac67fc640 1 -- 172.21.15.80:0/806357953 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2ad013f4e0 con 0x7f2ad006f2f0 2024-04-25T13:46:51.382 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.382+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2ad0141920 con 0x7f2ad006f2f0 2024-04-25T13:46:51.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.382+0000 7f2ac67fc640 1 -- 172.21.15.80:0/806357953 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) ==== 93+0+10 (secure 0 0 0) 0x7f2acc016110 con 0x7f2ad006f2f0 2024-04-25T13:46:51.383 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:46:51.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ab406f400 msgr2=0x7f2ab4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ab406f400 0x7f2ab4071910 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2ad00d1a30 tx=0x7f2ac0008040 comp rx=0 tx=0).stop 2024-04-25T13:46:51.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad006f2f0 msgr2=0x7f2ad013d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad006f2f0 0x7f2ad013d440 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2acc000c10 tx=0x7f2ad015f580 comp rx=0 tx=0).stop 2024-04-25T13:46:51.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 shutdown_connections 2024-04-25T13:46:51.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2ab406f400 0x7f2ab4071910 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad00d0ff0 0x7f2ad01403d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ad00b5330 0x7f2ad013d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 --2- 172.21.15.80:0/806357953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ad006f2f0 0x7f2ad013d440 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 >> 172.21.15.80:0/806357953 conn(0x7f2ad00b4160 msgr2=0x7f2ad00b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:51.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 shutdown_connections 2024-04-25T13:46:51.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.386+0000 7f2ad7fbc640 1 -- 172.21.15.80:0/806357953 wait complete. 2024-04-25T13:46:51.406 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:46:51.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd579222640 1 Processor -- start 2024-04-25T13:46:51.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd579222640 1 -- start start 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd579222640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740b5330 0x7fd5740d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd579222640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd574161b60 0x7fd574161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd579222640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5741624b0 con 0x7fd5740d0ff0 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd579222640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd5741629b0 con 0x7fd574161b60 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd579222640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5740b5740 con 0x7fd5740b5330 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd57359e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd574161b60 0x7fd574161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd57359e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd574161b60 0x7fd574161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38284/0 (socket says 172.21.15.80:38284) 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd57359e640 1 -- 172.21.15.80:0/4058405837 learned_addr learned my addr 172.21.15.80:0/4058405837 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:51.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd572d9d640 1 --2- 172.21.15.80:0/4058405837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740b5330 0x7fd5740d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd57359e640 1 -- 172.21.15.80:0/4058405837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740b5330 msgr2=0x7fd5740d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:51.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd57359e640 1 --2- 172.21.15.80:0/4058405837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740b5330 0x7fd5740d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd57359e640 1 -- 172.21.15.80:0/4058405837 >> v1:172.21.15.87:6789/0 conn(0x7fd5740d0ff0 legacy=0x7fd57406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:51.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.558+0000 7fd57359e640 1 -- 172.21.15.80:0/4058405837 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57406dd10 con 0x7fd574161b60 2024-04-25T13:46:51.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd57359e640 1 --2- 172.21.15.80:0/4058405837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd574161b60 0x7fd574161f70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009e50 tx=0x7fd56c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d510b72d3029e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd571d9b640 1 -- 172.21.15.80:0/4058405837 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0a5750 con 0x7fd574161b60 2024-04-25T13:46:51.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd571d9b640 1 -- 172.21.15.80:0/4058405837 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c0a58f0 con 0x7fd574161b60 2024-04-25T13:46:51.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd571d9b640 1 -- 172.21.15.80:0/4058405837 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c0a5cc0 con 0x7fd574161b60 2024-04-25T13:46:51.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd579222640 1 -- 172.21.15.80:0/4058405837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd574161b60 msgr2=0x7fd574161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd579222640 1 --2- 172.21.15.80:0/4058405837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd574161b60 0x7fd574161f70 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fd56c009e50 tx=0x7fd56c01c9a0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd579222640 1 -- 172.21.15.80:0/4058405837 shutdown_connections 2024-04-25T13:46:51.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd579222640 1 --2- 172.21.15.80:0/4058405837 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd574161b60 0x7fd574161f70 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd579222640 1 --2- 172.21.15.80:0/4058405837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740b5330 0x7fd5740d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd579222640 1 -- 172.21.15.80:0/4058405837 >> 172.21.15.80:0/4058405837 conn(0x7fd5740b4160 msgr2=0x7fd5740d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:51.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd579222640 1 -- 172.21.15.80:0/4058405837 shutdown_connections 2024-04-25T13:46:51.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.562+0000 7fd579222640 1 -- 172.21.15.80:0/4058405837 wait complete. 2024-04-25T13:46:51.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 Processor -- start 2024-04-25T13:46:51.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 -- start start 2024-04-25T13:46:51.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5740b5330 0x7fd57415f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740d0ff0 0x7fd574146de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd574161b60 0x7fd574148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd5740b0cd0 con 0x7fd5740b5330 2024-04-25T13:46:51.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd57406e890 con 0x7fd574161b60 2024-04-25T13:46:51.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5740b0e50 con 0x7fd5740d0ff0 2024-04-25T13:46:51.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5740b5330 0x7fd57415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5740b5330 0x7fd57415f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38286/0 (socket says 172.21.15.80:38286) 2024-04-25T13:46:51.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 -- 172.21.15.80:0/1665054124 learned_addr learned my addr 172.21.15.80:0/1665054124 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:51.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd57259c640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740d0ff0 0x7fd574146de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd57359e640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd574161b60 0x7fd574148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 -- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740d0ff0 msgr2=0x7fd574146de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740d0ff0 0x7fd574146de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 -- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd574161b60 msgr2=0x7fd574148f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd574161b60 0x7fd574148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 -- 172.21.15.80:0/1665054124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57415b390 con 0x7fd5740b5330 2024-04-25T13:46:51.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd572d9d640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5740b5330 0x7fd57415f080 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd56400d5f0 tx=0x7fd56400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd55b7fe640 1 -- 172.21.15.80:0/1665054124 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640125f0 con 0x7fd5740b5330 2024-04-25T13:46:51.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd55b7fe640 1 -- 172.21.15.80:0/1665054124 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd564019070 con 0x7fd5740b5330 2024-04-25T13:46:51.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd574150900 con 0x7fd5740b5330 2024-04-25T13:46:51.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd55b7fe640 1 -- 172.21.15.80:0/1665054124 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5640129b0 con 0x7fd5740b5330 2024-04-25T13:46:51.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd574150ba0 con 0x7fd5740b5330 2024-04-25T13:46:51.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.566+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57406a3a0 con 0x7fd5740b5330 2024-04-25T13:46:51.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.570+0000 7fd55b7fe640 1 -- 172.21.15.80:0/1665054124 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd564012b50 con 0x7fd5740b5330 2024-04-25T13:46:51.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.570+0000 7fd55b7fe640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd56006f400 0x7fd560071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.570+0000 7fd55b7fe640 1 -- 172.21.15.80:0/1665054124 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fd56408f750 con 0x7fd5740b5330 2024-04-25T13:46:51.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.570+0000 7fd57259c640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd56006f400 0x7fd560071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.570+0000 7fd57259c640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd56006f400 0x7fd560071910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd57414b2b0 tx=0x7fd568009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.574+0000 7fd55b7fe640 1 -- 172.21.15.80:0/1665054124 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd56405c640 con 0x7fd5740b5330 2024-04-25T13:46:51.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.734+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd57406fd00 con 0x7fd5740b5330 2024-04-25T13:46:51.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.734+0000 7fd55b7fe640 1 -- 172.21.15.80:0/1665054124 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) ==== 104+0+11 (secure 0 0 0) 0x7fd56405bf40 con 0x7fd5740b5330 2024-04-25T13:46:51.735 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:46:51.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd56006f400 msgr2=0x7fd560071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd56006f400 0x7fd560071910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd57414b2b0 tx=0x7fd568009040 comp rx=0 tx=0).stop 2024-04-25T13:46:51.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5740b5330 msgr2=0x7fd57415f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5740b5330 0x7fd57415f080 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7fd56400d5f0 tx=0x7fd56400dab0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 shutdown_connections 2024-04-25T13:46:51.737 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd56006f400 0x7fd560071910 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd574161b60 0x7fd574148f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5740d0ff0 0x7fd574146de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 --2- 172.21.15.80:0/1665054124 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5740b5330 0x7fd57415f080 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 >> 172.21.15.80:0/1665054124 conn(0x7fd5740b4160 msgr2=0x7fd574160920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:51.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 shutdown_connections 2024-04-25T13:46:51.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.738+0000 7fd579222640 1 -- 172.21.15.80:0/1665054124 wait complete. 2024-04-25T13:46:51.757 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:46:51.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0738738640 1 Processor -- start 2024-04-25T13:46:51.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0738738640 1 -- start start 2024-04-25T13:46:51.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0738738640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07300b5330 0x7f07300d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0738738640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300d0ff0 0x7f073006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0738738640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07300b5740 con 0x7f073006f2f0 2024-04-25T13:46:51.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0738738640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f073006f700 con 0x7f07300b5330 2024-04-25T13:46:51.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0738738640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07301629b0 con 0x7f07300d0ff0 2024-04-25T13:46:51.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0735cd3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300d0ff0 0x7f073006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f07364d4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f07300b5330 0x7f07300d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0735cd3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300d0ff0 0x7f073006edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:35242/0 (socket says 172.21.15.80:35242) 2024-04-25T13:46:51.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0735cd3640 1 -- 172.21.15.80:0/987682926 learned_addr learned my addr 172.21.15.80:0/987682926 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:51.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f07354d2640 1 -- 172.21.15.80:0/987682926 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662084521 0 0) 0x7f07300b5740 con 0x7f073006f2f0 2024-04-25T13:46:51.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f07354d2640 1 -- 172.21.15.80:0/987682926 --> v1:172.21.15.87:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f071c003530 con 0x7f073006f2f0 2024-04-25T13:46:51.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0735cd3640 1 -- 172.21.15.80:0/987682926 >> v2:172.21.15.87:3300/0 conn(0x7f07300b5330 msgr2=0x7f07300d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:51.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0735cd3640 1 --2- 172.21.15.80:0/987682926 >> v2:172.21.15.87:3300/0 conn(0x7f07300b5330 0x7f07300d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0735cd3640 1 -- 172.21.15.80:0/987682926 >> v1:172.21.15.87:6789/0 conn(0x7f073006f2f0 legacy=0x7f0730161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.914+0000 7f0735cd3640 1 -- 172.21.15.80:0/987682926 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073006dd10 con 0x7f07300d0ff0 2024-04-25T13:46:51.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0735cd3640 1 --2- 172.21.15.80:0/987682926 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300d0ff0 0x7f073006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f072c009870 tx=0x7f072c01c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f29d5bf2a7da66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f07354d2640 1 -- 172.21.15.80:0/987682926 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c0b1070 con 0x7f07300d0ff0 2024-04-25T13:46:51.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f07354d2640 1 -- 172.21.15.80:0/987682926 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072c0ac620 con 0x7f07300d0ff0 2024-04-25T13:46:51.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f07354d2640 1 -- 172.21.15.80:0/987682926 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c0ac9d0 con 0x7f07300d0ff0 2024-04-25T13:46:51.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0738738640 1 -- 172.21.15.80:0/987682926 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300d0ff0 msgr2=0x7f073006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0738738640 1 --2- 172.21.15.80:0/987682926 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300d0ff0 0x7f073006edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f072c009870 tx=0x7f072c01c4d0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0738738640 1 -- 172.21.15.80:0/987682926 shutdown_connections 2024-04-25T13:46:51.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0738738640 1 --2- 172.21.15.80:0/987682926 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300d0ff0 0x7f073006edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0738738640 1 --2- 172.21.15.80:0/987682926 >> v2:172.21.15.87:3300/0 conn(0x7f07300b5330 0x7f07300d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0738738640 1 -- 172.21.15.80:0/987682926 >> 172.21.15.80:0/987682926 conn(0x7f07300b4160 msgr2=0x7f07300d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:51.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0738738640 1 -- 172.21.15.80:0/987682926 shutdown_connections 2024-04-25T13:46:51.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.918+0000 7f0738738640 1 -- 172.21.15.80:0/987682926 wait complete. 2024-04-25T13:46:51.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 Processor -- start 2024-04-25T13:46:51.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 -- start start 2024-04-25T13:46:51.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f073006f2f0 0x7f073015c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300b5330 0x7f073014dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07300d0ff0 0x7f0730072420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f07300b0cd0 con 0x7f07300d0ff0 2024-04-25T13:46:51.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f073006e890 con 0x7f073006f2f0 2024-04-25T13:46:51.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07300b0e50 con 0x7f07300b5330 2024-04-25T13:46:51.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07300d0ff0 0x7f0730072420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07300d0ff0 0x7f0730072420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38302/0 (socket says 172.21.15.80:38302) 2024-04-25T13:46:51.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 -- 172.21.15.80:0/2180882428 learned_addr learned my addr 172.21.15.80:0/2180882428 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:51.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0735cd3640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300b5330 0x7f073014dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f07364d4640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f073006f2f0 0x7f073015c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 -- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300b5330 msgr2=0x7f073014dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300b5330 0x7f073014dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 -- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f073006f2f0 msgr2=0x7f073015c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:51.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f073006f2f0 0x7f073015c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:51.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 -- 172.21.15.80:0/2180882428 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0730073e20 con 0x7f07300d0ff0 2024-04-25T13:46:51.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0735cd3640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300b5330 0x7f073014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:51.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0736cd5640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07300d0ff0 0x7f0730072420 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f072800c6b0 tx=0x7f072800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0726ffd640 1 -- 172.21.15.80:0/2180882428 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728003b80 con 0x7f07300d0ff0 2024-04-25T13:46:51.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0726ffd640 1 -- 172.21.15.80:0/2180882428 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0728003d40 con 0x7f07300d0ff0 2024-04-25T13:46:51.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0730073580 con 0x7f07300d0ff0 2024-04-25T13:46:51.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0726ffd640 1 -- 172.21.15.80:0/2180882428 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0728005310 con 0x7f07300d0ff0 2024-04-25T13:46:51.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.922+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0730073b40 con 0x7f07300d0ff0 2024-04-25T13:46:51.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.926+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07300732f0 con 0x7f07300d0ff0 2024-04-25T13:46:51.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.926+0000 7f0726ffd640 1 -- 172.21.15.80:0/2180882428 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f07280054b0 con 0x7f07300d0ff0 2024-04-25T13:46:51.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.926+0000 7f0726ffd640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f071406f400 0x7f0714071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:51.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.926+0000 7f0726ffd640 1 -- 172.21.15.80:0/2180882428 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f072808f190 con 0x7f07300d0ff0 2024-04-25T13:46:51.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.926+0000 7f07364d4640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f071406f400 0x7f0714071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:51.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.926+0000 7f07364d4640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f071406f400 0x7f0714071910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0720009a50 tx=0x7f0720008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:51.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:51.930+0000 7f0726ffd640 1 -- 172.21.15.80:0/2180882428 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f072805c080 con 0x7f07300d0ff0 2024-04-25T13:46:52.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.078+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f073006a3a0 con 0x7f07300d0ff0 2024-04-25T13:46:52.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0726ffd640 1 -- 172.21.15.80:0/2180882428 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) ==== 100+0+11 (secure 0 0 0) 0x7f072805b980 con 0x7f07300d0ff0 2024-04-25T13:46:52.080 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f071406f400 msgr2=0x7f0714071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f071406f400 0x7f0714071910 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f0720009a50 tx=0x7f0720008040 comp rx=0 tx=0).stop 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07300d0ff0 msgr2=0x7f0730072420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07300d0ff0 0x7f0730072420 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f072800c6b0 tx=0x7f072800cb70 comp rx=0 tx=0).stop 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 shutdown_connections 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f071406f400 0x7f0714071910 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07300d0ff0 0x7f0730072420 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07300b5330 0x7f073014dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 --2- 172.21.15.80:0/2180882428 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f073006f2f0 0x7f073015c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 >> 172.21.15.80:0/2180882428 conn(0x7f07300b4160 msgr2=0x7f07300b6720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:52.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 shutdown_connections 2024-04-25T13:46:52.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.082+0000 7f0738738640 1 -- 172.21.15.80:0/2180882428 wait complete. 2024-04-25T13:46:52.102 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:46:52.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd4365dc640 1 Processor -- start 2024-04-25T13:46:52.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd4365dc640 1 -- start start 2024-04-25T13:46:52.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd4365dc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd43006f5a0 0x7fd430160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd4365dc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd430161070 0x7fd430161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd4365dc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4300b4bd0 0x7fd4300b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd4365dc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd4301619c0 con 0x7fd43006f5a0 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd4365dc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd430073f10 con 0x7fd430161070 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd4365dc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd43006fcf0 con 0x7fd4300b4bd0 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd42ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd43006f5a0 0x7fd430160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd42ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd43006f5a0 0x7fd430160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38316/0 (socket says 172.21.15.80:38316) 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd42ffff640 1 -- 172.21.15.80:0/56195877 learned_addr learned my addr 172.21.15.80:0/56195877 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd434b79640 1 --2- 172.21.15.80:0/56195877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4300b4bd0 0x7fd4300b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd42ffff640 1 -- 172.21.15.80:0/56195877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4300b4bd0 msgr2=0x7fd4300b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd427fff640 1 --2- 172.21.15.80:0/56195877 >> v2:172.21.15.87:3300/0 conn(0x7fd430161070 0x7fd430161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd42ffff640 1 --2- 172.21.15.80:0/56195877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4300b4bd0 0x7fd4300b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd42ffff640 1 -- 172.21.15.80:0/56195877 >> v2:172.21.15.87:3300/0 conn(0x7fd430161070 msgr2=0x7fd430161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd42ffff640 1 --2- 172.21.15.80:0/56195877 >> v2:172.21.15.87:3300/0 conn(0x7fd430161070 0x7fd430161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd42ffff640 1 -- 172.21.15.80:0/56195877 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4300b0c60 con 0x7fd43006f5a0 2024-04-25T13:46:52.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.246+0000 7fd434b79640 1 --2- 172.21.15.80:0/56195877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4300b4bd0 0x7fd4300b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:52.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd42ffff640 1 --2- 172.21.15.80:0/56195877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd430160b30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd43006ff70 tx=0x7fd42001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10e0962fcdc64cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:52.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd42f7fe640 1 -- 172.21.15.80:0/56195877 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4200b0070 con 0x7fd43006f5a0 2024-04-25T13:46:52.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd42f7fe640 1 -- 172.21.15.80:0/56195877 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4200ab600 con 0x7fd43006f5a0 2024-04-25T13:46:52.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd42f7fe640 1 -- 172.21.15.80:0/56195877 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4200ab9d0 con 0x7fd43006f5a0 2024-04-25T13:46:52.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 -- 172.21.15.80:0/56195877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 msgr2=0x7fd430160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 --2- 172.21.15.80:0/56195877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd430160b30 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fd43006ff70 tx=0x7fd42001c490 comp rx=0 tx=0).stop 2024-04-25T13:46:52.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 -- 172.21.15.80:0/56195877 shutdown_connections 2024-04-25T13:46:52.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 --2- 172.21.15.80:0/56195877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd4300b4bd0 0x7fd4300b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 --2- 172.21.15.80:0/56195877 >> v2:172.21.15.87:3300/0 conn(0x7fd430161070 0x7fd430161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 --2- 172.21.15.80:0/56195877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd430160b30 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 -- 172.21.15.80:0/56195877 >> 172.21.15.80:0/56195877 conn(0x7fd43006d080 msgr2=0x7fd4300b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:52.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 -- 172.21.15.80:0/56195877 shutdown_connections 2024-04-25T13:46:52.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.250+0000 7fd4365dc640 1 -- 172.21.15.80:0/56195877 wait complete. 2024-04-25T13:46:52.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 Processor -- start 2024-04-25T13:46:52.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 -- start start 2024-04-25T13:46:52.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd43014f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4300b4bd0 0x7fd43014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd430161070 0x7fd43015ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd430072bb0 con 0x7fd43006f5a0 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd4300b5930 con 0x7fd4300b4bd0 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd430072d30 con 0x7fd430161070 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd43014f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd43014f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38324/0 (socket says 172.21.15.80:38324) 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 -- 172.21.15.80:0/4275666004 learned_addr learned my addr 172.21.15.80:0/4275666004 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd434b79640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd430161070 0x7fd43015ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd427fff640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4300b4bd0 0x7fd43014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 -- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd430161070 msgr2=0x7fd43015ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd430161070 0x7fd43015ac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 -- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4300b4bd0 msgr2=0x7fd43014f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4300b4bd0 0x7fd43014f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 -- 172.21.15.80:0/4275666004 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43015d5b0 con 0x7fd43006f5a0 2024-04-25T13:46:52.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42ffff640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd43014f200 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd42001ab40 tx=0x7fd420019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:52.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42d7fa640 1 -- 172.21.15.80:0/4275666004 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4200b9070 con 0x7fd43006f5a0 2024-04-25T13:46:52.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42d7fa640 1 -- 172.21.15.80:0/4275666004 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4200b4640 con 0x7fd43006f5a0 2024-04-25T13:46:52.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd430150100 con 0x7fd43006f5a0 2024-04-25T13:46:52.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd42d7fa640 1 -- 172.21.15.80:0/4275666004 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4200b0040 con 0x7fd43006f5a0 2024-04-25T13:46:52.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd430155650 con 0x7fd43006f5a0 2024-04-25T13:46:52.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.254+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43006a3a0 con 0x7fd43006f5a0 2024-04-25T13:46:52.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.258+0000 7fd42d7fa640 1 -- 172.21.15.80:0/4275666004 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd4200bc070 con 0x7fd43006f5a0 2024-04-25T13:46:52.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.258+0000 7fd42d7fa640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd41006f430 0x7fd410071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.258+0000 7fd42d7fa640 1 -- 172.21.15.80:0/4275666004 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fd4201283b0 con 0x7fd43006f5a0 2024-04-25T13:46:52.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.258+0000 7fd427fff640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd41006f430 0x7fd410071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.258+0000 7fd427fff640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd41006f430 0x7fd410071940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd43014e2e0 tx=0x7fd418008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:52.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.262+0000 7fd42d7fa640 1 -- 172.21.15.80:0/4275666004 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd420016110 con 0x7fd43006f5a0 2024-04-25T13:46:52.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.438+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd4300739c0 con 0x7fd43006f5a0 2024-04-25T13:46:52.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.438+0000 7fd42d7fa640 1 -- 172.21.15.80:0/4275666004 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) ==== 80+0+0 (secure 0 0 0) 0x7fd4200f5330 con 0x7fd43006f5a0 2024-04-25T13:46:52.438 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:46:52.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd41006f430 msgr2=0x7fd410071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd41006f430 0x7fd410071940 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd43014e2e0 tx=0x7fd418008040 comp rx=0 tx=0).stop 2024-04-25T13:46:52.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 msgr2=0x7fd43014f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd43014f200 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fd42001ab40 tx=0x7fd420019230 comp rx=0 tx=0).stop 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 shutdown_connections 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd41006f430 0x7fd410071940 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd430161070 0x7fd43015ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd4300b4bd0 0x7fd43014f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 --2- 172.21.15.80:0/4275666004 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd43006f5a0 0x7fd43014f200 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 >> 172.21.15.80:0/4275666004 conn(0x7fd43006d080 msgr2=0x7fd4300b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 shutdown_connections 2024-04-25T13:46:52.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.442+0000 7fd4365dc640 1 -- 172.21.15.80:0/4275666004 wait complete. 2024-04-25T13:46:52.460 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:46:52.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:52.460 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:46:52.470 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[36988]: fuse finished with error 0 and tester_r 0 2024-04-25T13:46:52.472 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:46:52.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:52.472 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:46:52.566 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:46:52.566 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:46:52.566 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:52.572 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:46:52.572 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:52.621 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:52.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.766+0000 7fb3585a3640 1 Processor -- start 2024-04-25T13:46:52.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb3585a3640 1 -- start start 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb3585a3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb350161f90 0x7fb3500cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb3585a3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3500d0510 0x7fb35015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb3585a3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3500d0d70 con 0x7fb35015ea70 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb3585a3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb3500d1050 con 0x7fb350161f90 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb3585a3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb35006dca0 con 0x7fb3500d0510 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb356b40640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb35015ea70 0x7fb35015ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59648/0 (socket says 172.21.15.80:59648) 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb356b40640 1 -- 172.21.15.80:0/5158777 learned_addr learned my addr 172.21.15.80:0/5158777 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35633f640 1 --2- 172.21.15.80:0/5158777 >> v2:172.21.15.87:3300/0 conn(0x7fb350161f90 0x7fb3500cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb355b3e640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3500d0510 0x7fb35015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 876497290 0 0) 0x7fb3500d0d70 con 0x7fb35015ea70 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb33c003530 con 0x7fb35015ea70 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2089558617 0 0) 0x7fb33c003530 con 0x7fb35015ea70 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3500d0510 msgr2=0x7fb35015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3500d0510 0x7fb35015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 >> v2:172.21.15.87:3300/0 conn(0x7fb350161f90 msgr2=0x7fb3500cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 --2- 172.21.15.80:0/5158777 >> v2:172.21.15.87:3300/0 conn(0x7fb350161f90 0x7fb3500cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb35006df80 con 0x7fb35015ea70 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb348001a00 con 0x7fb35015ea70 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 >> v1:172.21.15.80:6789/0 conn(0x7fb35015ea70 legacy=0x7fb35015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb33c0039c0 0x7fb33c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3500d0510 0x7fb33c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb33c004d50 0x7fb33c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb3500d1050 con 0x7fb33c0039c0 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb35006dca0 con 0x7fb33c004d50 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb33c00ba50 con 0x7fb3500d0510 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35633f640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb33c0039c0 0x7fb33c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35633f640 1 -- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3500d0510 msgr2=0x7fb33c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb356b40640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb33c004d50 0x7fb33c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35633f640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3500d0510 0x7fb33c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35633f640 1 -- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb33c004d50 msgr2=0x7fb33c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35633f640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb33c004d50 0x7fb33c005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.770+0000 7fb35633f640 1 -- 172.21.15.80:0/5158777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c003530 con 0x7fb33c0039c0 2024-04-25T13:46:52.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb35633f640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb33c0039c0 0x7fb33c003e30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb350147e20 tx=0x7fb34c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=965192d531ba17f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0096b0 con 0x7fb33c0039c0 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34c0a64e0 con 0x7fb33c0039c0 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb35533d640 1 -- 172.21.15.80:0/5158777 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c0a6890 con 0x7fb33c0039c0 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb3585a3640 1 -- 172.21.15.80:0/5158777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb33c0039c0 msgr2=0x7fb33c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb3585a3640 1 --2- 172.21.15.80:0/5158777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb33c0039c0 0x7fb33c003e30 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fb350147e20 tx=0x7fb34c01c9f0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb356b40640 1 -- 172.21.15.80:0/5158777 reap_dead start 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb3585a3640 1 -- 172.21.15.80:0/5158777 shutdown_connections 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb3585a3640 1 -- 172.21.15.80:0/5158777 >> 172.21.15.80:0/5158777 conn(0x7fb35006d080 msgr2=0x7fb3500b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb3585a3640 1 -- 172.21.15.80:0/5158777 shutdown_connections 2024-04-25T13:46:52.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.774+0000 7fb3585a3640 1 -- 172.21.15.80:0/5158777 wait complete. 2024-04-25T13:46:52.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 Processor -- start 2024-04-25T13:46:52.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 -- start start 2024-04-25T13:46:52.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb33c0039c0 0x7fb350140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3501426e0 0x7fb350142af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350163e20 0x7fb350164230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb35006f630 con 0x7fb350163e20 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb35006f4b0 con 0x7fb3501426e0 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3500d0920 con 0x7fb33c0039c0 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350163e20 0x7fb350164230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350163e20 0x7fb350164230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38346/0 (socket says 172.21.15.80:38346) 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 -- 172.21.15.80:0/963958249 learned_addr learned my addr 172.21.15.80:0/963958249 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb35633f640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb33c0039c0 0x7fb350140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb355b3e640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3501426e0 0x7fb350142af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 -- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb33c0039c0 msgr2=0x7fb350140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb33c0039c0 0x7fb350140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 -- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3501426e0 msgr2=0x7fb350142af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3501426e0 0x7fb350142af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 -- 172.21.15.80:0/963958249 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb350164770 con 0x7fb350163e20 2024-04-25T13:46:52.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb355b3e640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3501426e0 0x7fb350142af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:52.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb35633f640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb33c0039c0 0x7fb350140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:52.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb356b40640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350163e20 0x7fb350164230 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb3500ce270 tx=0x7fb34800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:52.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb346ffd640 1 -- 172.21.15.80:0/963958249 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34800bb80 con 0x7fb350163e20 2024-04-25T13:46:52.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb346ffd640 1 -- 172.21.15.80:0/963958249 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb34800bd20 con 0x7fb350163e20 2024-04-25T13:46:52.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb350160020 con 0x7fb350163e20 2024-04-25T13:46:52.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb346ffd640 1 -- 172.21.15.80:0/963958249 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3480072b0 con 0x7fb350163e20 2024-04-25T13:46:52.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3501605e0 con 0x7fb350163e20 2024-04-25T13:46:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.778+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb35006a3a0 con 0x7fb350163e20 2024-04-25T13:46:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.782+0000 7fb346ffd640 1 -- 172.21.15.80:0/963958249 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb34801b060 con 0x7fb350163e20 2024-04-25T13:46:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.782+0000 7fb346ffd640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb33406f400 0x7fb334071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.782+0000 7fb346ffd640 1 -- 172.21.15.80:0/963958249 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fb34808f260 con 0x7fb350163e20 2024-04-25T13:46:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.782+0000 7fb35633f640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb33406f400 0x7fb334071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:52.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.782+0000 7fb35633f640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb33406f400 0x7fb334071910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0af000 tx=0x7fb34c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:52.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.786+0000 7fb346ffd640 1 -- 172.21.15.80:0/963958249 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb350164770 con 0x7fb350163e20 2024-04-25T13:46:52.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.946+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb350159fb0 con 0x7fb350163e20 2024-04-25T13:46:52.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.950+0000 7fb346ffd640 1 -- 172.21.15.80:0/963958249 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16555 (secure 0 0 0) 0x7fb34805c150 con 0x7fb350163e20 2024-04-25T13:46:52.950 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:52.950 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":125,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:48.458377+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:52.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb33406f400 msgr2=0x7fb334071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb33406f400 0x7fb334071910 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0af000 tx=0x7fb34c0046d0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350163e20 msgr2=0x7fb350164230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:52.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350163e20 0x7fb350164230 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fb3500ce270 tx=0x7fb34800b4a0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 shutdown_connections 2024-04-25T13:46:52.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb33406f400 0x7fb334071910 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb350163e20 0x7fb350164230 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3501426e0 0x7fb350142af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 --2- 172.21.15.80:0/963958249 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb33c0039c0 0x7fb350140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:52.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 >> 172.21.15.80:0/963958249 conn(0x7fb35006d080 msgr2=0x7fb35015ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:52.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 shutdown_connections 2024-04-25T13:46:52.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:52.954+0000 7fb3585a3640 1 -- 172.21.15.80:0/963958249 wait complete. 2024-04-25T13:46:52.973 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:46:27.136651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:46:27.696536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:46:52.973 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:46:53.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.118+0000 7fdb062ef640 1 Processor -- start 2024-04-25T13:46:53.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb062ef640 1 -- start start 2024-04-25T13:46:53.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb062ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00161170 0x7fdb001574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb062ef640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdb00073c40 0x7fdb000b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb062ef640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdb0006fbf0 con 0x7fdb00071e60 2024-04-25T13:46:53.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb062ef640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdb000b1350 con 0x7fdb00073c40 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb062ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0006f9a0 con 0x7fdb00161170 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdafffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fdb00071e60 0x7fdb00160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59654/0 (socket says 172.21.15.80:59654) 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdafffff640 1 -- 172.21.15.80:0/3492616128 learned_addr learned my addr 172.21.15.80:0/3492616128 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb0488c640 1 --2- 172.21.15.80:0/3492616128 >> v2:172.21.15.80:3300/0 conn(0x7fdb00073c40 0x7fdb000b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdaff7fe640 1 --2- 172.21.15.80:0/3492616128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00161170 0x7fdb001574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdafeffd640 1 -- 172.21.15.80:0/3492616128 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 590346450 0 0) 0x7fdb0006fbf0 con 0x7fdb00071e60 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdafeffd640 1 -- 172.21.15.80:0/3492616128 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdae8005740 con 0x7fdb00071e60 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb0488c640 1 -- 172.21.15.80:0/3492616128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00161170 msgr2=0x7fdb001574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb0488c640 1 --2- 172.21.15.80:0/3492616128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00161170 0x7fdb001574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb0488c640 1 -- 172.21.15.80:0/3492616128 >> v1:172.21.15.80:6789/0 conn(0x7fdb00071e60 legacy=0x7fdb00160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdafeffd640 1 -- 172.21.15.80:0/3492616128 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 307630395 0 0) 0x7fdae8005740 con 0x7fdb00071e60 2024-04-25T13:46:53.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdb0488c640 1 -- 172.21.15.80:0/3492616128 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb000b0a20 con 0x7fdb00073c40 2024-04-25T13:46:53.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.122+0000 7fdaff7fe640 1 --2- 172.21.15.80:0/3492616128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00161170 0x7fdb001574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:53.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb0488c640 1 --2- 172.21.15.80:0/3492616128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00073c40 0x7fdb000b4ba0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009d90 tx=0x7fdaf801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=589c107af98d160b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdafeffd640 1 -- 172.21.15.80:0/3492616128 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf80b2070 con 0x7fdb00073c40 2024-04-25T13:46:53.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdafeffd640 1 -- 172.21.15.80:0/3492616128 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf80a5580 con 0x7fdb00073c40 2024-04-25T13:46:53.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdafeffd640 1 -- 172.21.15.80:0/3492616128 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0001680 con 0x7fdb00073c40 2024-04-25T13:46:53.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 -- 172.21.15.80:0/3492616128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00073c40 msgr2=0x7fdb000b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 --2- 172.21.15.80:0/3492616128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00073c40 0x7fdb000b4ba0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009d90 tx=0x7fdaf801c890 comp rx=0 tx=0).stop 2024-04-25T13:46:53.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 -- 172.21.15.80:0/3492616128 shutdown_connections 2024-04-25T13:46:53.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 --2- 172.21.15.80:0/3492616128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00073c40 0x7fdb000b4ba0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 --2- 172.21.15.80:0/3492616128 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00161170 0x7fdb001574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 -- 172.21.15.80:0/3492616128 >> 172.21.15.80:0/3492616128 conn(0x7fdb000b4160 msgr2=0x7fdb000d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 -- 172.21.15.80:0/3492616128 shutdown_connections 2024-04-25T13:46:53.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 -- 172.21.15.80:0/3492616128 wait complete. 2024-04-25T13:46:53.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 Processor -- start 2024-04-25T13:46:53.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.126+0000 7fdb062ef640 1 -- start start 2024-04-25T13:46:53.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00071e60 0x7fdb00149e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb00073c40 0x7fdb00152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00155330 0x7fdb00165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdb0006e470 con 0x7fdb00071e60 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00074050 con 0x7fdb00073c40 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdb0006e5f0 con 0x7fdb00155330 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00071e60 0x7fdb00149e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00071e60 0x7fdb00149e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38368/0 (socket says 172.21.15.80:38368) 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 -- 172.21.15.80:0/1409107558 learned_addr learned my addr 172.21.15.80:0/1409107558 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb0488c640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00155330 0x7fdb00165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 -- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00155330 msgr2=0x7fdb00165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdaff7fe640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb00073c40 0x7fdb00152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00155330 0x7fdb00165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 -- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb00073c40 msgr2=0x7fdb00152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb00073c40 0x7fdb00152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 -- 172.21.15.80:0/1409107558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0014a8f0 con 0x7fdb00071e60 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb0488c640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00155330 0x7fdb00165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafffff640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00071e60 0x7fdb00149e80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00017f0 tx=0x7fdaf000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafcff9640 1 -- 172.21.15.80:0/1409107558 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0005ae0 con 0x7fdb00071e60 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafcff9640 1 -- 172.21.15.80:0/1409107558 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf0005ca0 con 0x7fdb00071e60 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafcff9640 1 -- 172.21.15.80:0/1409107558 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf00072b0 con 0x7fdb00071e60 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0014a3c0 con 0x7fdb00071e60 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb0014ee30 con 0x7fdb00071e60 2024-04-25T13:46:53.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb0006a3a0 con 0x7fdb00071e60 2024-04-25T13:46:53.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.130+0000 7fdafcff9640 1 -- 172.21.15.80:0/1409107558 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdaf0007450 con 0x7fdb00071e60 2024-04-25T13:46:53.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.134+0000 7fdafcff9640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdaec06f430 0x7fdaec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.134+0000 7fdafcff9640 1 -- 172.21.15.80:0/1409107558 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fdaf008f080 con 0x7fdb00071e60 2024-04-25T13:46:53.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.134+0000 7fdaff7fe640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdaec06f430 0x7fdaec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.134+0000 7fdaff7fe640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdaec06f430 0x7fdaec071940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdb0014abd0 tx=0x7fdaf4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.138+0000 7fdafcff9640 1 -- 172.21.15.80:0/1409107558 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdaf005c000 con 0x7fdb00071e60 2024-04-25T13:46:53.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdb000008e0 con 0x7fdb00071e60 2024-04-25T13:46:53.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdafcff9640 1 -- 172.21.15.80:0/1409107558 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) ==== 93+0+10 (secure 0 0 0) 0x7fdaf005b900 con 0x7fdb00071e60 2024-04-25T13:46:53.297 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:46:53.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdaec06f430 msgr2=0x7fdaec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdaec06f430 0x7fdaec071940 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fdb0014abd0 tx=0x7fdaf4008040 comp rx=0 tx=0).stop 2024-04-25T13:46:53.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00071e60 msgr2=0x7fdb00149e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00071e60 0x7fdb00149e80 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fdaf00017f0 tx=0x7fdaf000cbe0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 shutdown_connections 2024-04-25T13:46:53.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdaec06f430 0x7fdaec071940 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdb00155330 0x7fdb00165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdb00073c40 0x7fdb00152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 --2- 172.21.15.80:0/1409107558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdb00071e60 0x7fdb00149e80 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 >> 172.21.15.80:0/1409107558 conn(0x7fdb000b4160 msgr2=0x7fdb00072dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 shutdown_connections 2024-04-25T13:46:53.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.298+0000 7fdb062ef640 1 -- 172.21.15.80:0/1409107558 wait complete. 2024-04-25T13:46:53.319 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:46:53.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f26726640 1 Processor -- start 2024-04-25T13:46:53.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f26726640 1 -- start start 2024-04-25T13:46:53.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f26726640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2f2006edb0 0x7f2f2006f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f26726640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 0x7f2f20161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f26726640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f201624b0 con 0x7f2f200b5330 2024-04-25T13:46:53.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f26726640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2f2006f700 con 0x7f2f2006edb0 2024-04-25T13:46:53.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f26726640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f201629b0 con 0x7f2f20161b60 2024-04-25T13:46:53.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2f200b5330 0x7f2f200d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59666/0 (socket says 172.21.15.80:59666) 2024-04-25T13:46:53.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1ffff640 1 -- 172.21.15.80:0/3073328654 learned_addr learned my addr 172.21.15.80:0/3073328654 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:53.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3073328654 >> v2:172.21.15.87:3300/0 conn(0x7f2f2006edb0 0x7f2f2006f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f24cc3640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 0x7f2f20161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 956413436 0 0) 0x7f2f10001120 con 0x7f2f200b5330 2024-04-25T13:46:53.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f08003530 con 0x7f2f200b5330 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 911528615 0 0) 0x7f2f08003530 con 0x7f2f200b5330 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 msgr2=0x7f2f20161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 0x7f2f20161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 >> v2:172.21.15.87:3300/0 conn(0x7f2f2006edb0 msgr2=0x7f2f2006f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 --2- 172.21.15.80:0/3073328654 >> v2:172.21.15.87:3300/0 conn(0x7f2f2006edb0 0x7f2f2006f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f200b5740 con 0x7f2f200b5330 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2f10001850 con 0x7f2f200b5330 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 >> v1:172.21.15.80:6789/0 conn(0x7f2f200b5330 legacy=0x7f2f200d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f080039c0 0x7f2f08003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f08004810 0x7f2f08004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 0x7f2f080052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2006f700 con 0x7f2f08004810 2024-04-25T13:46:53.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201629b0 con 0x7f2f080039c0 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f0800bbb0 con 0x7f2f20161b60 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3073328654 >> v2:172.21.15.87:3300/0 conn(0x7f2f2006edb0 0x7f2f2006f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f08004810 0x7f2f08004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f24cc3640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 0x7f2f080052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1ffff640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f080039c0 0x7f2f08003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1f7fe640 1 -- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 msgr2=0x7f2f080052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 0x7f2f080052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1f7fe640 1 -- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f080039c0 msgr2=0x7f2f08003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f080039c0 0x7f2f08003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f1f7fe640 1 -- 172.21.15.80:0/3073328654 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f08003530 con 0x7f2f08004810 2024-04-25T13:46:53.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.462+0000 7f2f24cc3640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f20161b60 0x7f2f080052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f08004810 0x7f2f08004c20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2f140098b0 tx=0x7f2f1401c600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3f9bcd6b1d6157c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400f040 con 0x7f2f08004810 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f14004020 con 0x7f2f08004810 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f1effd640 1 -- 172.21.15.80:0/3073328654 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f140043f0 con 0x7f2f08004810 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f26726640 1 -- 172.21.15.80:0/3073328654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f08004810 msgr2=0x7f2f08004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f26726640 1 --2- 172.21.15.80:0/3073328654 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f08004810 0x7f2f08004c20 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f2f140098b0 tx=0x7f2f1401c600 comp rx=0 tx=0).stop 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f24cc3640 1 -- 172.21.15.80:0/3073328654 reap_dead start 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f26726640 1 -- 172.21.15.80:0/3073328654 shutdown_connections 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f26726640 1 -- 172.21.15.80:0/3073328654 >> 172.21.15.80:0/3073328654 conn(0x7f2f200b4160 msgr2=0x7f2f200d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f26726640 1 -- 172.21.15.80:0/3073328654 shutdown_connections 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.466+0000 7f2f26726640 1 -- 172.21.15.80:0/3073328654 wait complete. 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 Processor -- start 2024-04-25T13:46:53.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 -- start start 2024-04-25T13:46:53.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f08004810 0x7f2f20163eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f201faed0 0x7f2f201fb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f201fb820 0x7f2f20165c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2006e890 con 0x7f2f201faed0 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2006e5b0 con 0x7f2f08004810 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f200b0cd0 con 0x7f2f201fb820 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f201faed0 0x7f2f201fb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f201faed0 0x7f2f201fb2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38398/0 (socket says 172.21.15.80:38398) 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 -- 172.21.15.80:0/3805459609 learned_addr learned my addr 172.21.15.80:0/3805459609 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f24cc3640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f201fb820 0x7f2f20165c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1ffff640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f08004810 0x7f2f20163eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 -- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f201fb820 msgr2=0x7f2f20165c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f201fb820 0x7f2f20165c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 -- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f08004810 msgr2=0x7f2f20163eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f08004810 0x7f2f20163eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 -- 172.21.15.80:0/3805459609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f200aedd0 con 0x7f2f201faed0 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1ffff640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f08004810 0x7f2f20163eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f24cc3640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f201fb820 0x7f2f20165c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1f7fe640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f201faed0 0x7f2f201fb2e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f2f1401cae0 tx=0x7f2f08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1cff9640 1 -- 172.21.15.80:0/3805459609 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f14003960 con 0x7f2f201faed0 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1cff9640 1 -- 172.21.15.80:0/3805459609 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f14003b00 con 0x7f2f201faed0 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f200af2b0 con 0x7f2f201faed0 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f1cff9640 1 -- 172.21.15.80:0/3805459609 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f140bc440 con 0x7f2f201faed0 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f200af870 con 0x7f2f201faed0 2024-04-25T13:46:53.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.470+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f200af020 con 0x7f2f201faed0 2024-04-25T13:46:53.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.474+0000 7f2f1cff9640 1 -- 172.21.15.80:0/3805459609 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f14003ed0 con 0x7f2f201faed0 2024-04-25T13:46:53.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.474+0000 7f2f1cff9640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0c06f400 0x7f2f0c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.474+0000 7f2f1cff9640 1 -- 172.21.15.80:0/3805459609 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f2f14129870 con 0x7f2f201faed0 2024-04-25T13:46:53.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.474+0000 7f2f1ffff640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0c06f400 0x7f2f0c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.474+0000 7f2f1ffff640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0c06f400 0x7f2f0c071910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f10001ec0 tx=0x7f2f10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.478+0000 7f2f1cff9640 1 -- 172.21.15.80:0/3805459609 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f140c2060 con 0x7f2f201faed0 2024-04-25T13:46:53.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.626+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2f2006a3a0 con 0x7f2f201faed0 2024-04-25T13:46:53.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.626+0000 7f2f1cff9640 1 -- 172.21.15.80:0/3805459609 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) ==== 104+0+11 (secure 0 0 0) 0x7f2f200aedd0 con 0x7f2f201faed0 2024-04-25T13:46:53.625 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0c06f400 msgr2=0x7f2f0c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0c06f400 0x7f2f0c071910 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f2f10001ec0 tx=0x7f2f10009040 comp rx=0 tx=0).stop 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f201faed0 msgr2=0x7f2f201fb2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f201faed0 0x7f2f201fb2e0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f2f1401cae0 tx=0x7f2f08005840 comp rx=0 tx=0).stop 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 shutdown_connections 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0c06f400 0x7f2f0c071910 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f201fb820 0x7f2f20165c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f201faed0 0x7f2f201fb2e0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 --2- 172.21.15.80:0/3805459609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f08004810 0x7f2f20163eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 >> 172.21.15.80:0/3805459609 conn(0x7f2f200b4160 msgr2=0x7f2f20142a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 shutdown_connections 2024-04-25T13:46:53.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.630+0000 7f2f26726640 1 -- 172.21.15.80:0/3805459609 wait complete. 2024-04-25T13:46:53.647 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:46:53.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c34c1640 1 Processor -- start 2024-04-25T13:46:53.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c34c1640 1 -- start start 2024-04-25T13:46:53.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c34c1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22bc0b5330 0x7f22bc0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c34c1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22bc0b5740 con 0x7f22bc0d0ff0 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c34c1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22bc06dd10 con 0x7f22bc161b60 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c34c1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f22bc1629b0 con 0x7f22bc0b5330 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c0a5c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f22bc0d0ff0 0x7f22bc06edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59680/0 (socket says 172.21.15.80:59680) 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c0a5c640 1 -- 172.21.15.80:0/1980074903 learned_addr learned my addr 172.21.15.80:0/1980074903 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 924007046 0 0) 0x7f22bc0b5740 con 0x7f22bc0d0ff0 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c125d640 1 --2- 172.21.15.80:0/1980074903 >> v2:172.21.15.87:3300/0 conn(0x7f22bc0b5330 0x7f22bc0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f22a8003530 con 0x7f22bc0d0ff0 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2165832710 0 0) 0x7f22a8003530 con 0x7f22bc0d0ff0 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 >> v2:172.21.15.87:3300/0 conn(0x7f22bc0b5330 msgr2=0x7f22bc0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 --2- 172.21.15.80:0/1980074903 >> v2:172.21.15.87:3300/0 conn(0x7f22bc0b5330 0x7f22bc0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 >> v1:172.21.15.87:6789/0 conn(0x7f22bc161b60 legacy=0x7f22bc161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc06dff0 con 0x7f22bc0d0ff0 2024-04-25T13:46:53.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f22ac001680 con 0x7f22bc0d0ff0 2024-04-25T13:46:53.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 >> v1:172.21.15.80:6789/0 conn(0x7f22bc0d0ff0 legacy=0x7f22bc06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a80039c0 0x7f22a8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a8004810 0x7f22a8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a8005310 0x7f22a800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc1629b0 con 0x7f22a80039c0 2024-04-25T13:46:53.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22a800c2b0 con 0x7f22a8005310 2024-04-25T13:46:53.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f22a800c4b0 con 0x7f22a8004810 2024-04-25T13:46:53.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c125d640 1 --2- 172.21.15.80:0/1980074903 >> v2:172.21.15.87:3300/0 conn(0x7f22bc0b5330 0x7f22bc0d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:53.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c125d640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a80039c0 0x7f22a8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c0a5c640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a8004810 0x7f22a8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c1a5e640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a8005310 0x7f22a800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c125d640 1 -- 172.21.15.80:0/1980074903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a8004810 msgr2=0x7f22a8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c125d640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a8004810 0x7f22a8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c125d640 1 -- 172.21.15.80:0/1980074903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a8005310 msgr2=0x7f22a800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:53.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c125d640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a8005310 0x7f22a800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c125d640 1 -- 172.21.15.80:0/1980074903 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a8003530 con 0x7f22a80039c0 2024-04-25T13:46:53.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.786+0000 7f22c1a5e640 1 -- 172.21.15.80:0/1980074903 reap_dead start 2024-04-25T13:46:53.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22c125d640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a80039c0 0x7f22a8003e30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f22b80098b0 tx=0x7f22b801c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddea4551073a182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b800f040 con 0x7f22a80039c0 2024-04-25T13:46:53.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b80a8050 con 0x7f22a80039c0 2024-04-25T13:46:53.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22b3fff640 1 -- 172.21.15.80:0/1980074903 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22b8004440 con 0x7f22a80039c0 2024-04-25T13:46:53.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22c34c1640 1 -- 172.21.15.80:0/1980074903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a80039c0 msgr2=0x7f22a8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22c34c1640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a80039c0 0x7f22a8003e30 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f22b80098b0 tx=0x7f22b801c5b0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22c34c1640 1 -- 172.21.15.80:0/1980074903 shutdown_connections 2024-04-25T13:46:53.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22c34c1640 1 --2- 172.21.15.80:0/1980074903 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a80039c0 0x7f22a8003e30 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22c34c1640 1 -- 172.21.15.80:0/1980074903 >> 172.21.15.80:0/1980074903 conn(0x7f22bc0b4160 msgr2=0x7f22bc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22c34c1640 1 -- 172.21.15.80:0/1980074903 shutdown_connections 2024-04-25T13:46:53.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.790+0000 7f22c34c1640 1 -- 172.21.15.80:0/1980074903 wait complete. 2024-04-25T13:46:53.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 Processor -- start 2024-04-25T13:46:53.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 -- start start 2024-04-25T13:46:53.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a80039c0 0x7f22bc145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22bc0c6b40 0x7f22bc0ac440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22bc0c7190 0x7f22bc0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc162380 con 0x7f22bc0c6b40 2024-04-25T13:46:53.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22bc06f5c0 con 0x7f22a80039c0 2024-04-25T13:46:53.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f22bc0b0220 con 0x7f22bc0c7190 2024-04-25T13:46:53.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22bc0c6b40 0x7f22bc0ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22bc0c6b40 0x7f22bc0ac440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38414/0 (socket says 172.21.15.80:38414) 2024-04-25T13:46:53.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 -- 172.21.15.80:0/1637191947 learned_addr learned my addr 172.21.15.80:0/1637191947 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:53.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c1a5e640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22bc0c7190 0x7f22bc0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c125d640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a80039c0 0x7f22bc145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 -- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22bc0c7190 msgr2=0x7f22bc0c75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22bc0c7190 0x7f22bc0c75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 -- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a80039c0 msgr2=0x7f22bc145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a80039c0 0x7f22bc145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 -- 172.21.15.80:0/1637191947 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22bc0c7ae0 con 0x7f22bc0c6b40 2024-04-25T13:46:53.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c125d640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a80039c0 0x7f22bc145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:53.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c1a5e640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22bc0c7190 0x7f22bc0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:53.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c0a5c640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22bc0c6b40 0x7f22bc0ac440 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00e800 tx=0x7f22ac00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22b1ffb640 1 -- 172.21.15.80:0/1637191947 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00c870 con 0x7f22bc0c6b40 2024-04-25T13:46:53.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22b1ffb640 1 -- 172.21.15.80:0/1637191947 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22ac018070 con 0x7f22bc0c6b40 2024-04-25T13:46:53.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22bc0acc90 con 0x7f22bc0c6b40 2024-04-25T13:46:53.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22b1ffb640 1 -- 172.21.15.80:0/1637191947 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ac00cc70 con 0x7f22bc0c6b40 2024-04-25T13:46:53.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22bc0ad250 con 0x7f22bc0c6b40 2024-04-25T13:46:53.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.794+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22bc0ad450 con 0x7f22bc0c6b40 2024-04-25T13:46:53.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.798+0000 7f22b1ffb640 1 -- 172.21.15.80:0/1637191947 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f22ac01d460 con 0x7f22bc0c6b40 2024-04-25T13:46:53.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.798+0000 7f22b1ffb640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22a006f430 0x7f22a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:53.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.798+0000 7f22b1ffb640 1 -- 172.21.15.80:0/1637191947 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f22ac08fca0 con 0x7f22bc0c6b40 2024-04-25T13:46:53.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.798+0000 7f22c125d640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22a006f430 0x7f22a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:53.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.798+0000 7f22c125d640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22a006f430 0x7f22a0071940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f22b80ad000 tx=0x7f22b80046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:53.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.802+0000 7f22b1ffb640 1 -- 172.21.15.80:0/1637191947 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22bc0ad450 con 0x7f22bc0c6b40 2024-04-25T13:46:53.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f22bc06a3a0 con 0x7f22bc0c6b40 2024-04-25T13:46:53.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22b1ffb640 1 -- 172.21.15.80:0/1637191947 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) ==== 100+0+11 (secure 0 0 0) 0x7f22bc0c7ae0 con 0x7f22bc0c6b40 2024-04-25T13:46:53.948 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:46:53.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22a006f430 msgr2=0x7f22a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22a006f430 0x7f22a0071940 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f22b80ad000 tx=0x7f22b80046d0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22bc0c6b40 msgr2=0x7f22bc0ac440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:53.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22bc0c6b40 0x7f22bc0ac440 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f22ac00e800 tx=0x7f22ac00ecc0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 shutdown_connections 2024-04-25T13:46:53.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22a006f430 0x7f22a0071940 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22bc0c7190 0x7f22bc0c75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22bc0c6b40 0x7f22bc0ac440 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 --2- 172.21.15.80:0/1637191947 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a80039c0 0x7f22bc145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:53.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 >> 172.21.15.80:0/1637191947 conn(0x7f22bc0b4160 msgr2=0x7f22bc15f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:53.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 shutdown_connections 2024-04-25T13:46:53.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:53.950+0000 7f22c34c1640 1 -- 172.21.15.80:0/1637191947 wait complete. 2024-04-25T13:46:53.970 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:46:54.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.118+0000 7f241fc26640 1 Processor -- start 2024-04-25T13:46:54.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.118+0000 7f241fc26640 1 -- start start 2024-04-25T13:46:54.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241fc26640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f241806f5a0 0x7f2418160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241fc26640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f241806fcf0 con 0x7f24180733b0 2024-04-25T13:46:54.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241fc26640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24180b59f0 con 0x7f2418161070 2024-04-25T13:46:54.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241fc26640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2418073f10 con 0x7f241806f5a0 2024-04-25T13:46:54.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241e1c3640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f24180733b0 0x7f24180b52e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59696/0 (socket says 172.21.15.80:59696) 2024-04-25T13:46:54.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241e1c3640 1 -- 172.21.15.80:0/369891537 learned_addr learned my addr 172.21.15.80:0/369891537 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:54.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4278128760 0 0) 0x7f241806fcf0 con 0x7f24180733b0 2024-04-25T13:46:54.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2404003530 con 0x7f24180733b0 2024-04-25T13:46:54.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d9c2640 1 --2- 172.21.15.80:0/369891537 >> v2:172.21.15.87:3300/0 conn(0x7f241806f5a0 0x7f2418160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2516114669 0 0) 0x7f2404003530 con 0x7f24180733b0 2024-04-25T13:46:54.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 >> v2:172.21.15.87:3300/0 conn(0x7f241806f5a0 msgr2=0x7f2418160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:54.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 --2- 172.21.15.80:0/369891537 >> v2:172.21.15.87:3300/0 conn(0x7f241806f5a0 0x7f2418160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 >> v1:172.21.15.87:6789/0 conn(0x7f2418161070 legacy=0x7f24180b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24180b0c60 con 0x7f24180733b0 2024-04-25T13:46:54.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2410001a00 con 0x7f24180733b0 2024-04-25T13:46:54.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 >> v1:172.21.15.80:6789/0 conn(0x7f24180733b0 legacy=0x7f24180b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f2404003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2404004810 0x7f2404004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2404005260 0x7f240400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2418073f10 con 0x7f24040039c0 2024-04-25T13:46:54.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f240400c1f0 con 0x7f2404004810 2024-04-25T13:46:54.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f240400e600 con 0x7f2404005260 2024-04-25T13:46:54.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d9c2640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f2404003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d1c1640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2404004810 0x7f2404004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d9c2640 1 -- 172.21.15.80:0/369891537 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2404005260 msgr2=0x7f240400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:54.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d9c2640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2404005260 0x7f240400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d9c2640 1 -- 172.21.15.80:0/369891537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2404004810 msgr2=0x7f2404004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:54.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d9c2640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2404004810 0x7f2404004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d9c2640 1 -- 172.21.15.80:0/369891537 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2404003530 con 0x7f24040039c0 2024-04-25T13:46:54.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241e1c3640 1 -- 172.21.15.80:0/369891537 reap_dead start 2024-04-25T13:46:54.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.122+0000 7f241d1c1640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2404004810 0x7f2404004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:54.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241d9c2640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f2404003e30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2414009ab0 tx=0x7f241401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd34d256f8eac58a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:54.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140096b0 con 0x7f24040039c0 2024-04-25T13:46:54.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24140a6430 con 0x7f24040039c0 2024-04-25T13:46:54.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241c9c0640 1 -- 172.21.15.80:0/369891537 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140a6820 con 0x7f24040039c0 2024-04-25T13:46:54.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 -- 172.21.15.80:0/369891537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 msgr2=0x7f2404003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f2404003e30 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f2414009ab0 tx=0x7f241401ca50 comp rx=0 tx=0).stop 2024-04-25T13:46:54.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 -- 172.21.15.80:0/369891537 shutdown_connections 2024-04-25T13:46:54.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 --2- 172.21.15.80:0/369891537 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f2404003e30 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 -- 172.21.15.80:0/369891537 >> 172.21.15.80:0/369891537 conn(0x7f241806d080 msgr2=0x7f24180b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:54.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 -- 172.21.15.80:0/369891537 shutdown_connections 2024-04-25T13:46:54.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 -- 172.21.15.80:0/369891537 wait complete. 2024-04-25T13:46:54.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 Processor -- start 2024-04-25T13:46:54.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.126+0000 7f241fc26640 1 -- start start 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f24181f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24180ab4a0 0x7f24181f9210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24181f3410 0x7f24181f9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f24180b53e0 con 0x7f24040039c0 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2418161890 con 0x7f24181f3410 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f24180b5560 con 0x7f24180ab4a0 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f24181f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f24181f6730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38436/0 (socket says 172.21.15.80:38436) 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 -- 172.21.15.80:0/1032704305 learned_addr learned my addr 172.21.15.80:0/1032704305 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241e1c3640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24181f3410 0x7f24181f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d1c1640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24180ab4a0 0x7f24181f9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 -- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24180ab4a0 msgr2=0x7f24181f9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24180ab4a0 0x7f24181f9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 -- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24181f3410 msgr2=0x7f24181f9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24181f3410 0x7f24181f9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 -- 172.21.15.80:0/1032704305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24180abbd0 con 0x7f24040039c0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241e1c3640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24181f3410 0x7f24181f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241d9c2640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f24181f6730 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f2414009ab0 tx=0x7f24140197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f240e7fc640 1 -- 172.21.15.80:0/1032704305 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140a6ce0 con 0x7f24040039c0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f240e7fc640 1 -- 172.21.15.80:0/1032704305 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24140b9440 con 0x7f24040039c0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24180abeb0 con 0x7f24040039c0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f240e7fc640 1 -- 172.21.15.80:0/1032704305 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140b97f0 con 0x7f24040039c0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24180ac470 con 0x7f24040039c0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f241806a3a0 con 0x7f24040039c0 2024-04-25T13:46:54.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f240e7fc640 1 -- 172.21.15.80:0/1032704305 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f24140a7410 con 0x7f24040039c0 2024-04-25T13:46:54.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f240e7fc640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23fc06f400 0x7f23fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.130+0000 7f240e7fc640 1 -- 172.21.15.80:0/1032704305 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f24141291b0 con 0x7f24040039c0 2024-04-25T13:46:54.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.134+0000 7f241d1c1640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23fc06f400 0x7f23fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.134+0000 7f241d1c1640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23fc06f400 0x7f23fc071910 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2408000c10 tx=0x7f2408009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:54.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.138+0000 7f240e7fc640 1 -- 172.21.15.80:0/1032704305 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f24140ac020 con 0x7f24040039c0 2024-04-25T13:46:54.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.310+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f24180739c0 con 0x7f24040039c0 2024-04-25T13:46:54.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.310+0000 7f240e7fc640 1 -- 172.21.15.80:0/1032704305 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v125) ==== 80+0+0 (secure 0 0 0) 0x7f2414016760 con 0x7f24040039c0 2024-04-25T13:46:54.310 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:46:54.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23fc06f400 msgr2=0x7f23fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23fc06f400 0x7f23fc071910 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2408000c10 tx=0x7f2408009040 comp rx=0 tx=0).stop 2024-04-25T13:46:54.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 msgr2=0x7f24181f6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f24181f6730 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f2414009ab0 tx=0x7f24140197f0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 shutdown_connections 2024-04-25T13:46:54.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f23fc06f400 0x7f23fc071910 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24181f3410 0x7f24181f9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f24180ab4a0 0x7f24181f9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 --2- 172.21.15.80:0/1032704305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f24040039c0 0x7f24181f6730 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 >> 172.21.15.80:0/1032704305 conn(0x7f241806d080 msgr2=0x7f241815c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:54.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 shutdown_connections 2024-04-25T13:46:54.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.314+0000 7f241fc26640 1 -- 172.21.15.80:0/1032704305 wait complete. 2024-04-25T13:46:54.332 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:46:54.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:54.332 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:46:54.345 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25098]: fuse finished with error 0 and tester_r 0 2024-04-25T13:46:54.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:46:54.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:46:54.347 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:46:54.512 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:46:54.513 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:46:54.513 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:54.519 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:46:54.519 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:46:54.519 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:46:54.519 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:46:54.561 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:46:54.562 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:54.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.718+0000 7f5d93947640 1 Processor -- start 2024-04-25T13:46:54.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.718+0000 7f5d93947640 1 -- start start 2024-04-25T13:46:54.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d93947640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d8c06f5a0 0x7f5d8c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d93947640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d8c06fcf0 con 0x7f5d8c161070 2024-04-25T13:46:54.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d93947640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d8c0b59f0 con 0x7f5d8c0733b0 2024-04-25T13:46:54.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d93947640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5d8c073f10 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d916e3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5d8c06f5a0 0x7f5d8c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d90ee2640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5d8c161070 0x7f5d8c0b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59708/0 (socket says 172.21.15.80:59708) 2024-04-25T13:46:54.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d90ee2640 1 -- 172.21.15.80:0/3615268809 learned_addr learned my addr 172.21.15.80:0/3615268809 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:54.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d916e3640 1 -- 172.21.15.80:0/3615268809 >> v1:172.21.15.87:6789/0 conn(0x7f5d8c0733b0 legacy=0x7f5d8c0b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:54.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d916e3640 1 -- 172.21.15.80:0/3615268809 >> v1:172.21.15.80:6789/0 conn(0x7f5d8c161070 legacy=0x7f5d8c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d83fff640 1 -- 172.21.15.80:0/3615268809 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2081947718 0 0) 0x7f5d8c06fcf0 con 0x7f5d8c161070 2024-04-25T13:46:54.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.722+0000 7f5d916e3640 1 -- 172.21.15.80:0/3615268809 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c0b0c60 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d916e3640 1 --2- 172.21.15.80:0/3615268809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c160b30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0097c0 tx=0x7f5d7c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e604e630ff4faab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d83fff640 1 -- 172.21.15.80:0/3615268809 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b0070 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d83fff640 1 -- 172.21.15.80:0/3615268809 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0ab600 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d83fff640 1 -- 172.21.15.80:0/3615268809 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0ab9d0 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 -- 172.21.15.80:0/3615268809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 msgr2=0x7f5d8c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 --2- 172.21.15.80:0/3615268809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c160b30 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0097c0 tx=0x7f5d7c01c3d0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 -- 172.21.15.80:0/3615268809 shutdown_connections 2024-04-25T13:46:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 --2- 172.21.15.80:0/3615268809 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c160b30 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 -- 172.21.15.80:0/3615268809 >> 172.21.15.80:0/3615268809 conn(0x7f5d8c06d080 msgr2=0x7f5d8c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 -- 172.21.15.80:0/3615268809 shutdown_connections 2024-04-25T13:46:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 -- 172.21.15.80:0/3615268809 wait complete. 2024-04-25T13:46:54.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 Processor -- start 2024-04-25T13:46:54.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.726+0000 7f5d93947640 1 -- start start 2024-04-25T13:46:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c15ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d8c0733b0 0x7f5d8c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c161070 0x7f5d8c15dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c16b6f0 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c161890 con 0x7f5d8c161070 2024-04-25T13:46:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5d8c16b870 con 0x7f5d8c0733b0 2024-04-25T13:46:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c15ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c15ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38458/0 (socket says 172.21.15.80:38458) 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 -- 172.21.15.80:0/1958546350 learned_addr learned my addr 172.21.15.80:0/1958546350 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d90ee2640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d8c0733b0 0x7f5d8c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 -- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d8c0733b0 msgr2=0x7f5d8c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d8c0733b0 0x7f5d8c15d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 -- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c161070 msgr2=0x7f5d8c15dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d91ee4640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c161070 0x7f5d8c15dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c161070 0x7f5d8c15dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 -- 172.21.15.80:0/1958546350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c159a30 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d916e3640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c15ff60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0ae040 tx=0x7f5d7c01ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d81ffb640 1 -- 172.21.15.80:0/1958546350 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b0050 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d81ffb640 1 -- 172.21.15.80:0/1958546350 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d7c0bc070 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8c159d10 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d81ffb640 1 -- 172.21.15.80:0/1958546350 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b5870 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8c15a2d0 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.730+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d8c06a3a0 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.734+0000 7f5d81ffb640 1 -- 172.21.15.80:0/1958546350 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5d7c0b5a10 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.734+0000 7f5d81ffb640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d7006f400 0x7f5d70071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.734+0000 7f5d81ffb640 1 -- 172.21.15.80:0/1958546350 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f5d7c12a610 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.734+0000 7f5d90ee2640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d7006f400 0x7f5d70071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.734+0000 7f5d90ee2640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d7006f400 0x7f5d70071910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0d0cf0 tx=0x7f5d88007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:54.736 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.738+0000 7f5d81ffb640 1 -- 172.21.15.80:0/1958546350 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5d7c016050 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.938+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5d8c1608b0 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.939 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d81ffb640 1 -- 172.21.15.80:0/1958546350 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5174 (secure 0 0 0) 0x7f5d7c0f7670 con 0x7f5d8c06f5a0 2024-04-25T13:46:54.940 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:54.940 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:46:28.718395+0000","modified":"2024-04-25T13:46:31.773079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/2773338889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2773338889},{"type":"v1","addr":"172.21.15.80:6835","nonce":2773338889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":7}]} 2024-04-25T13:46:54.940 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 59 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d7006f400 msgr2=0x7f5d70071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d7006f400 0x7f5d70071910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c0d0cf0 tx=0x7f5d88007c10 comp rx=0 tx=0).stop 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 msgr2=0x7f5d8c15ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c15ff60 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0ae040 tx=0x7f5d7c01ac10 comp rx=0 tx=0).stop 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 shutdown_connections 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5d7006f400 0x7f5d70071910 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d8c161070 0x7f5d8c15dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5d8c0733b0 0x7f5d8c15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 --2- 172.21.15.80:0/1958546350 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d8c06f5a0 0x7f5d8c15ff60 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 >> 172.21.15.80:0/1958546350 conn(0x7f5d8c06d080 msgr2=0x7f5d8c0b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 shutdown_connections 2024-04-25T13:46:54.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:54.942+0000 7f5d93947640 1 -- 172.21.15.80:0/1958546350 wait complete. 2024-04-25T13:46:54.963 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:55.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a7621640 1 Processor -- start 2024-04-25T13:46:55.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a7621640 1 -- start start 2024-04-25T13:46:55.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a7621640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18a0161f90 0x7f18a00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a7621640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f18a00d0510 0x7f18a015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a7621640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f18a00d0d70 con 0x7f18a015ea70 2024-04-25T13:46:55.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a7621640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f18a00d1050 con 0x7f18a00d0510 2024-04-25T13:46:55.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a7621640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18a006dca0 con 0x7f18a0161f90 2024-04-25T13:46:55.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a5bbe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f18a015ea70 0x7f18a015ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59712/0 (socket says 172.21.15.80:59712) 2024-04-25T13:46:55.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a5bbe640 1 -- 172.21.15.80:0/1579489033 learned_addr learned my addr 172.21.15.80:0/1579489033 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:55.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a53bd640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18a0161f90 0x7f18a00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a4bbc640 1 --2- 172.21.15.80:0/1579489033 >> v2:172.21.15.87:3300/0 conn(0x7f18a00d0510 0x7f18a015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905478040 0 0) 0x7f18a00d0d70 con 0x7f18a015ea70 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f188c003530 con 0x7f18a015ea70 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3341868665 0 0) 0x7f188c003530 con 0x7f18a015ea70 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18a0161f90 msgr2=0x7f18a00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18a0161f90 0x7f18a00cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 >> v2:172.21.15.87:3300/0 conn(0x7f18a00d0510 msgr2=0x7f18a015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 --2- 172.21.15.80:0/1579489033 >> v2:172.21.15.87:3300/0 conn(0x7f18a00d0510 0x7f18a015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a006df80 con 0x7f18a015ea70 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1898001a00 con 0x7f18a015ea70 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 >> v1:172.21.15.80:6789/0 conn(0x7f18a015ea70 legacy=0x7f18a015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f188c0039c0 0x7f188c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18a0161f90 0x7f188c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c004d50 0x7f188c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18a00d1050 con 0x7f188c0039c0 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18a006dca0 con 0x7f188c004d50 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f188c00ba50 con 0x7f18a0161f90 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.106+0000 7f18a53bd640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f188c0039c0 0x7f188c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a5bbe640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c004d50 0x7f188c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a53bd640 1 -- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18a0161f90 msgr2=0x7f188c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a53bd640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f18a0161f90 0x7f188c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a53bd640 1 -- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c004d50 msgr2=0x7f188c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a53bd640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c004d50 0x7f188c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a53bd640 1 -- 172.21.15.80:0/1579489033 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188c003530 con 0x7f188c0039c0 2024-04-25T13:46:55.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a5bbe640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f188c004d50 0x7f188c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:55.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a53bd640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f188c0039c0 0x7f188c003e30 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f18a0147e20 tx=0x7f189c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a125fc6705265d43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0096b0 con 0x7f188c0039c0 2024-04-25T13:46:55.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189c0a64e0 con 0x7f188c0039c0 2024-04-25T13:46:55.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f1897fff640 1 -- 172.21.15.80:0/1579489033 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189c0a6890 con 0x7f188c0039c0 2024-04-25T13:46:55.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a7621640 1 -- 172.21.15.80:0/1579489033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f188c0039c0 msgr2=0x7f188c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a7621640 1 --2- 172.21.15.80:0/1579489033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f188c0039c0 0x7f188c003e30 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f18a0147e20 tx=0x7f189c01c9f0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a5bbe640 1 -- 172.21.15.80:0/1579489033 reap_dead start 2024-04-25T13:46:55.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a7621640 1 -- 172.21.15.80:0/1579489033 shutdown_connections 2024-04-25T13:46:55.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a7621640 1 -- 172.21.15.80:0/1579489033 >> 172.21.15.80:0/1579489033 conn(0x7f18a006d080 msgr2=0x7f18a00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:55.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a7621640 1 -- 172.21.15.80:0/1579489033 shutdown_connections 2024-04-25T13:46:55.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.110+0000 7f18a7621640 1 -- 172.21.15.80:0/1579489033 wait complete. 2024-04-25T13:46:55.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 Processor -- start 2024-04-25T13:46:55.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 -- start start 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0039c0 0x7f18a015bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a0141410 0x7f18a01f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a0140e10 0x7f18a01f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f18a006f630 con 0x7f18a0141410 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f18a006f4b0 con 0x7f18a0140e10 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f18a00d0920 con 0x7f188c0039c0 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a0141410 0x7f18a01f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a0141410 0x7f18a01f89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38476/0 (socket says 172.21.15.80:38476) 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 -- 172.21.15.80:0/2524339677 learned_addr learned my addr 172.21.15.80:0/2524339677 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 -- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0039c0 msgr2=0x7f18a015bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0039c0 0x7f18a015bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a5bbe640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a0140e10 0x7f18a01f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 -- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a0140e10 msgr2=0x7f18a01f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a0140e10 0x7f18a01f8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 -- 172.21.15.80:0/2524339677 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18a01f7910 con 0x7f18a0141410 2024-04-25T13:46:55.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a4bbc640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a0141410 0x7f18a01f89b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f189000e7f0 tx=0x7f189000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f1895ffb640 1 -- 172.21.15.80:0/2524339677 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1890017070 con 0x7f18a0141410 2024-04-25T13:46:55.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f1895ffb640 1 -- 172.21.15.80:0/2524339677 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f189000c8e0 con 0x7f18a0141410 2024-04-25T13:46:55.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18a01f7bf0 con 0x7f18a0141410 2024-04-25T13:46:55.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f1895ffb640 1 -- 172.21.15.80:0/2524339677 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f189000eea0 con 0x7f18a0141410 2024-04-25T13:46:55.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.114+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18a01f81b0 con 0x7f18a0141410 2024-04-25T13:46:55.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.118+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18a006a3a0 con 0x7f18a0141410 2024-04-25T13:46:55.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.118+0000 7f1895ffb640 1 -- 172.21.15.80:0/2524339677 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f189001a070 con 0x7f18a0141410 2024-04-25T13:46:55.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.118+0000 7f1895ffb640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f188406f430 0x7f1884071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.118+0000 7f1895ffb640 1 -- 172.21.15.80:0/2524339677 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f189008ef10 con 0x7f18a0141410 2024-04-25T13:46:55.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.118+0000 7f18a53bd640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f188406f430 0x7f1884071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.118+0000 7f18a53bd640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f188406f430 0x7f1884071940 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f189c0af000 tx=0x7f189c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.122+0000 7f1895ffb640 1 -- 172.21.15.80:0/2524339677 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f189005bf20 con 0x7f18a0141410 2024-04-25T13:46:55.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.282+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f18a01f7660 con 0x7f18a0141410 2024-04-25T13:46:55.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.282+0000 7f1895ffb640 1 -- 172.21.15.80:0/2524339677 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16555 (secure 0 0 0) 0x7f189005b820 con 0x7f18a0141410 2024-04-25T13:46:55.282 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:55.282 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":125,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:48.458377+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:55.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f188406f430 msgr2=0x7f1884071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f188406f430 0x7f1884071940 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f189c0af000 tx=0x7f189c004740 comp rx=0 tx=0).stop 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a0141410 msgr2=0x7f18a01f89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a0141410 0x7f18a01f89b0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f189000e7f0 tx=0x7f189000ecb0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 shutdown_connections 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f188406f430 0x7f1884071940 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a0140e10 0x7f18a01f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f18a0141410 0x7f18a01f89b0 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 --2- 172.21.15.80:0/2524339677 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f188c0039c0 0x7f18a015bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 >> 172.21.15.80:0/2524339677 conn(0x7f18a006d080 msgr2=0x7f18a00b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 shutdown_connections 2024-04-25T13:46:55.285 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.286+0000 7f18a7621640 1 -- 172.21.15.80:0/2524339677 wait complete. 2024-04-25T13:46:55.303 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:46:27.136651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '121', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:46:27.696536+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:46:55.304 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:46:55.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d7566640 1 Processor -- start 2024-04-25T13:46:55.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d7566640 1 -- start start 2024-04-25T13:46:55.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d7566640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f77d0071e60 0x7f77d01574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d7566640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 0x7f77d0160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d7566640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77d01613b0 con 0x7f77d0160fa0 2024-04-25T13:46:55.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d7566640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f77d00b1350 con 0x7f77d0071e60 2024-04-25T13:46:55.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d7566640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f77d006f9a0 con 0x7f77d0073c40 2024-04-25T13:46:55.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d5302640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f77d0071e60 0x7f77d01574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d5b03640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f77d0160fa0 0x7f77d00b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59720/0 (socket says 172.21.15.80:59720) 2024-04-25T13:46:55.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d5b03640 1 -- 172.21.15.80:0/3141803636 learned_addr learned my addr 172.21.15.80:0/3141803636 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:55.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d4b01640 1 --2- 172.21.15.80:0/3141803636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 0x7f77d0160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d5302640 1 -- 172.21.15.80:0/3141803636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 msgr2=0x7f77d0160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:55.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d5302640 1 --2- 172.21.15.80:0/3141803636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 0x7f77d0160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d5302640 1 -- 172.21.15.80:0/3141803636 >> v1:172.21.15.80:6789/0 conn(0x7f77d0160fa0 legacy=0x7f77d00b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77d5302640 1 -- 172.21.15.80:0/3141803636 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d006fbf0 con 0x7f77d0071e60 2024-04-25T13:46:55.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.446+0000 7f77c3fff640 1 -- 172.21.15.80:0/3141803636 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 898741347 0 0) 0x7f77c80014a0 con 0x7f77d0160fa0 2024-04-25T13:46:55.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d5302640 1 --2- 172.21.15.80:0/3141803636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d01574c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0097c0 tx=0x7f77cc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1843ffd3d214af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77c3fff640 1 -- 172.21.15.80:0/3141803636 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0b2070 con 0x7f77d0071e60 2024-04-25T13:46:55.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77c3fff640 1 -- 172.21.15.80:0/3141803636 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc0ad640 con 0x7f77d0071e60 2024-04-25T13:46:55.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77c3fff640 1 -- 172.21.15.80:0/3141803636 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0ada10 con 0x7f77d0071e60 2024-04-25T13:46:55.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d7566640 1 -- 172.21.15.80:0/3141803636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 msgr2=0x7f77d01574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d7566640 1 --2- 172.21.15.80:0/3141803636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d01574c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0097c0 tx=0x7f77cc01c4a0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d7566640 1 -- 172.21.15.80:0/3141803636 shutdown_connections 2024-04-25T13:46:55.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d7566640 1 --2- 172.21.15.80:0/3141803636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 0x7f77d0160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d7566640 1 --2- 172.21.15.80:0/3141803636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d01574c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d7566640 1 -- 172.21.15.80:0/3141803636 >> 172.21.15.80:0/3141803636 conn(0x7f77d00b4160 msgr2=0x7f77d00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:55.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d7566640 1 -- 172.21.15.80:0/3141803636 shutdown_connections 2024-04-25T13:46:55.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.450+0000 7f77d7566640 1 -- 172.21.15.80:0/3141803636 wait complete. 2024-04-25T13:46:55.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 Processor -- start 2024-04-25T13:46:55.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 -- start start 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d00b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 0x7f77d01fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77d0160fa0 0x7f77d0166440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0074050 con 0x7f77d0071e60 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0072270 con 0x7f77d0160fa0 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f77d00b56c0 con 0x7f77d0073c40 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d00b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d00b2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38496/0 (socket says 172.21.15.80:38496) 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 -- 172.21.15.80:0/3502300175 learned_addr learned my addr 172.21.15.80:0/3502300175 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 -- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 msgr2=0x7f77d01fdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 0x7f77d01fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 -- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77d0160fa0 msgr2=0x7f77d0166440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77d0160fa0 0x7f77d0166440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 -- 172.21.15.80:0/3502300175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d00b2ec0 con 0x7f77d0071e60 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d5302640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d00b2670 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0097c0 tx=0x7f77cc01ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77c1ffb640 1 -- 172.21.15.80:0/3502300175 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0b2050 con 0x7f77d0071e60 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77c1ffb640 1 -- 172.21.15.80:0/3502300175 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77cc0b3040 con 0x7f77d0071e60 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77c1ffb640 1 -- 172.21.15.80:0/3502300175 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc004240 con 0x7f77d0071e60 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d00b31a0 con 0x7f77d0071e60 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01fd360 con 0x7f77d0071e60 2024-04-25T13:46:55.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.454+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d0156920 con 0x7f77d0071e60 2024-04-25T13:46:55.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.458+0000 7f77c1ffb640 1 -- 172.21.15.80:0/3502300175 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77cc0c0060 con 0x7f77d0071e60 2024-04-25T13:46:55.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.458+0000 7f77c1ffb640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77b406f430 0x7f77b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.458+0000 7f77c1ffb640 1 -- 172.21.15.80:0/3502300175 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f77cc128fb0 con 0x7f77d0071e60 2024-04-25T13:46:55.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.458+0000 7f77d4b01640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77b406f430 0x7f77b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.458+0000 7f77d4b01640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77b406f430 0x7f77b4071940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f77d014abf0 tx=0x7f77c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.462+0000 7f77c1ffb640 1 -- 172.21.15.80:0/3502300175 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f77cc016090 con 0x7f77d0071e60 2024-04-25T13:46:55.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.614+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77d006a3a0 con 0x7f77d0071e60 2024-04-25T13:46:55.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.614+0000 7f77c1ffb640 1 -- 172.21.15.80:0/3502300175 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v125) ==== 93+0+10 (secure 0 0 0) 0x7f77cc0f5ea0 con 0x7f77d0071e60 2024-04-25T13:46:55.614 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:46:55.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77b406f430 msgr2=0x7f77b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77b406f430 0x7f77b4071940 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f77d014abf0 tx=0x7f77c4008040 comp rx=0 tx=0).stop 2024-04-25T13:46:55.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 msgr2=0x7f77d00b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d00b2670 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f77cc0097c0 tx=0x7f77cc01ace0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 shutdown_connections 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77b406f430 0x7f77b4071940 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77d0160fa0 0x7f77d0166440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77d0073c40 0x7f77d01fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 --2- 172.21.15.80:0/3502300175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77d0071e60 0x7f77d00b2670 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 >> 172.21.15.80:0/3502300175 conn(0x7f77d00b4160 msgr2=0x7f77d015db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 shutdown_connections 2024-04-25T13:46:55.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.618+0000 7f77d7566640 1 -- 172.21.15.80:0/3502300175 wait complete. 2024-04-25T13:46:55.635 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:46:55.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.770+0000 7f231d472640 1 Processor -- start 2024-04-25T13:46:55.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f231d472640 1 -- start start 2024-04-25T13:46:55.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f231d472640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f231806edb0 0x7f231806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f231d472640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23180b5740 con 0x7f23180b5330 2024-04-25T13:46:55.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f231d472640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f231806dd10 con 0x7f2318161b60 2024-04-25T13:46:55.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f231d472640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f231806f700 con 0x7f231806edb0 2024-04-25T13:46:55.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f230ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f231806edb0 0x7f231806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f2316ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f23180b5330 0x7f23180d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59722/0 (socket says 172.21.15.80:59722) 2024-04-25T13:46:55.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f2316ffd640 1 -- 172.21.15.80:0/543464705 learned_addr learned my addr 172.21.15.80:0/543464705 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:55.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f230ffff640 1 -- 172.21.15.80:0/543464705 >> v1:172.21.15.87:6789/0 conn(0x7f2318161b60 legacy=0x7f2318161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:55.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f230ffff640 1 -- 172.21.15.80:0/543464705 >> v1:172.21.15.80:6789/0 conn(0x7f23180b5330 legacy=0x7f23180d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.774+0000 7f230ffff640 1 -- 172.21.15.80:0/543464705 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231806dff0 con 0x7f231806edb0 2024-04-25T13:46:55.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f230ffff640 1 --2- 172.21.15.80:0/543464705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f231806edb0 0x7f231806f1c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f23000097c0 tx=0x7f230001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466128498fb49089 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f23167fc640 1 -- 172.21.15.80:0/543464705 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000b0070 con 0x7f231806edb0 2024-04-25T13:46:55.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f23167fc640 1 -- 172.21.15.80:0/543464705 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23000ab600 con 0x7f231806edb0 2024-04-25T13:46:55.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f23167fc640 1 -- 172.21.15.80:0/543464705 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000ab9d0 con 0x7f231806edb0 2024-04-25T13:46:55.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 -- 172.21.15.80:0/543464705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f231806edb0 msgr2=0x7f231806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 --2- 172.21.15.80:0/543464705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f231806edb0 0x7f231806f1c0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f23000097c0 tx=0x7f230001c3d0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 -- 172.21.15.80:0/543464705 shutdown_connections 2024-04-25T13:46:55.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 --2- 172.21.15.80:0/543464705 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f231806edb0 0x7f231806f1c0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 -- 172.21.15.80:0/543464705 >> 172.21.15.80:0/543464705 conn(0x7f23180b4160 msgr2=0x7f23180d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:55.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 -- 172.21.15.80:0/543464705 shutdown_connections 2024-04-25T13:46:55.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 -- 172.21.15.80:0/543464705 wait complete. 2024-04-25T13:46:55.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 Processor -- start 2024-04-25T13:46:55.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.778+0000 7f231d472640 1 -- start start 2024-04-25T13:46:55.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f231806edb0 0x7f2318154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23180b5330 0x7f231814f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2318161b60 0x7f231814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f23180b0e50 con 0x7f23180b5330 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f23180b0cd0 con 0x7f2318161b60 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f23180d11c0 con 0x7f231806edb0 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23180b5330 0x7f231814f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23180b5330 0x7f231814f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38520/0 (socket says 172.21.15.80:38520) 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 -- 172.21.15.80:0/1551998858 learned_addr learned my addr 172.21.15.80:0/1551998858 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f2316ffd640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f231806edb0 0x7f2318154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f23177fe640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2318161b60 0x7f231814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 -- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f231806edb0 msgr2=0x7f2318154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f231806edb0 0x7f2318154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 -- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2318161b60 msgr2=0x7f231814cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2318161b60 0x7f231814cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 -- 172.21.15.80:0/1551998858 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2318159340 con 0x7f23180b5330 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f23177fe640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2318161b60 0x7f231814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f2316ffd640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f231806edb0 0x7f2318154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230ffff640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23180b5330 0x7f231814f330 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f2300018fe0 tx=0x7f2300019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230f7fe640 1 -- 172.21.15.80:0/1551998858 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000ba070 con 0x7f23180b5330 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230f7fe640 1 -- 172.21.15.80:0/1551998858 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23000b43f0 con 0x7f23180b5330 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2318159620 con 0x7f23180b5330 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230f7fe640 1 -- 172.21.15.80:0/1551998858 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23000b0040 con 0x7f23180b5330 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2318159be0 con 0x7f23180b5330 2024-04-25T13:46:55.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231806a3a0 con 0x7f23180b5330 2024-04-25T13:46:55.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230f7fe640 1 -- 172.21.15.80:0/1551998858 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f23000bd070 con 0x7f23180b5330 2024-04-25T13:46:55.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230f7fe640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22f806f430 0x7f22f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:55.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f230f7fe640 1 -- 172.21.15.80:0/1551998858 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f2300128240 con 0x7f23180b5330 2024-04-25T13:46:55.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.782+0000 7f2316ffd640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22f806f430 0x7f22f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:55.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.786+0000 7f2316ffd640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22f806f430 0x7f22f8071940 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2308007fd0 tx=0x7f2308009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:55.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.786+0000 7f230f7fe640 1 -- 172.21.15.80:0/1551998858 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2300016050 con 0x7f23180b5330 2024-04-25T13:46:55.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.946+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f23180cf020 con 0x7f23180b5330 2024-04-25T13:46:55.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.946+0000 7f230f7fe640 1 -- 172.21.15.80:0/1551998858 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v125) ==== 104+0+11 (secure 0 0 0) 0x7f23000f5130 con 0x7f23180b5330 2024-04-25T13:46:55.947 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:46:55.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22f806f430 msgr2=0x7f22f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22f806f430 0x7f22f8071940 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2308007fd0 tx=0x7f2308009040 comp rx=0 tx=0).stop 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23180b5330 msgr2=0x7f231814f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23180b5330 0x7f231814f330 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f2300018fe0 tx=0x7f2300019170 comp rx=0 tx=0).stop 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 shutdown_connections 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f22f806f430 0x7f22f8071940 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2318161b60 0x7f231814cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f23180b5330 0x7f231814f330 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 --2- 172.21.15.80:0/1551998858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f231806edb0 0x7f2318154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 >> 172.21.15.80:0/1551998858 conn(0x7f23180b4160 msgr2=0x7f2318160f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:55.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 shutdown_connections 2024-04-25T13:46:55.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:55.950+0000 7f231d472640 1 -- 172.21.15.80:0/1551998858 wait complete. 2024-04-25T13:46:55.969 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:46:56.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.106+0000 7ff9e67d9640 1 Processor -- start 2024-04-25T13:46:56.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.106+0000 7ff9e67d9640 1 -- start start 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.106+0000 7ff9e67d9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 0x7ff9e00b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.106+0000 7ff9e67d9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e00b92a0 con 0x7ff9e00b9ca0 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.106+0000 7ff9e67d9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9e00d4c10 con 0x7ff9e00d3ed0 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.106+0000 7ff9e67d9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e00d4ef0 con 0x7ff9e00d2ca0 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9e4d76640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff9e00b9ca0 0x7ff9e00ba0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59738/0 (socket says 172.21.15.80:59738) 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9e4d76640 1 -- 172.21.15.80:0/33926131 learned_addr learned my addr 172.21.15.80:0/33926131 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9df7fe640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 0x7ff9e00b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3305605600 0 0) 0x7ff9d00014a0 con 0x7ff9e00b9ca0 2024-04-25T13:46:56.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9cc003530 con 0x7ff9e00b9ca0 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2882411199 0 0) 0x7ff9cc003530 con 0x7ff9e00b9ca0 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 msgr2=0x7ff9e00b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 0x7ff9e00b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 >> v1:172.21.15.87:6789/0 conn(0x7ff9e00d3ed0 legacy=0x7ff9e00d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00d51d0 con 0x7ff9e00b9ca0 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff9d0001bd0 con 0x7ff9e00b9ca0 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 >> v1:172.21.15.80:6789/0 conn(0x7ff9e00b9ca0 legacy=0x7ff9e00ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9cc0039f0 0x7ff9cc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 0x7ff9cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc004d50 0x7ff9cc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e00d4ef0 con 0x7ff9cc0039f0 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9cc00bbc0 con 0x7ff9cc004d50 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9cc00bdc0 con 0x7ff9e00d2ca0 2024-04-25T13:46:56.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9dffff640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9cc0039f0 0x7ff9cc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9df7fe640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 0x7ff9cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9e4d76640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc004d50 0x7ff9cc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9dffff640 1 -- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 msgr2=0x7ff9cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:56.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9dffff640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 0x7ff9cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9dffff640 1 -- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc004d50 msgr2=0x7ff9cc005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:56.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9dffff640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc004d50 0x7ff9cc005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9dffff640 1 -- 172.21.15.80:0/33926131 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9cc003530 con 0x7ff9cc0039f0 2024-04-25T13:46:56.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9df7fe640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00d2ca0 0x7ff9cc004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:56.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9dffff640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9cc0039f0 0x7ff9cc003e60 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8001780 tx=0x7ff9d801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8adb499c5a10b6b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d80013a0 con 0x7ff9cc0039f0 2024-04-25T13:46:56.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d80a9070 con 0x7ff9cc0039f0 2024-04-25T13:46:56.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.110+0000 7ff9deffd640 1 -- 172.21.15.80:0/33926131 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8003600 con 0x7ff9cc0039f0 2024-04-25T13:46:56.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- 172.21.15.80:0/33926131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9cc0039f0 msgr2=0x7ff9cc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 --2- 172.21.15.80:0/33926131 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9cc0039f0 0x7ff9cc003e60 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8001780 tx=0x7ff9d801c420 comp rx=0 tx=0).stop 2024-04-25T13:46:56.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 -- 172.21.15.80:0/33926131 reap_dead start 2024-04-25T13:46:56.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- 172.21.15.80:0/33926131 shutdown_connections 2024-04-25T13:46:56.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- 172.21.15.80:0/33926131 >> 172.21.15.80:0/33926131 conn(0x7ff9e00737b0 msgr2=0x7ff9e006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:56.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- 172.21.15.80:0/33926131 shutdown_connections 2024-04-25T13:46:56.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- 172.21.15.80:0/33926131 wait complete. 2024-04-25T13:46:56.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 Processor -- start 2024-04-25T13:46:56.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- start start 2024-04-25T13:46:56.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc0039f0 0x7ff9e0153340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00ce810 0x7ff9e00ca7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9e00cec20 0x7ff9e00cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e016c290 con 0x7ff9e00cec20 2024-04-25T13:46:56.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e016c410 con 0x7ff9cc0039f0 2024-04-25T13:46:56.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e67d9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e016c590 con 0x7ff9e00ce810 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9e00cec20 0x7ff9e00cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9e00cec20 0x7ff9e00cad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38546/0 (socket says 172.21.15.80:38546) 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 -- 172.21.15.80:0/740104494 learned_addr learned my addr 172.21.15.80:0/740104494 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9df7fe640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00ce810 0x7ff9e00ca7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9dffff640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc0039f0 0x7ff9e0153340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 -- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00ce810 msgr2=0x7ff9e00ca7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00ce810 0x7ff9e00ca7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 -- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc0039f0 msgr2=0x7ff9e0153340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc0039f0 0x7ff9e0153340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 -- 172.21.15.80:0/740104494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e00ce510 con 0x7ff9e00cec20 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.114+0000 7ff9e4d76640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9e00cec20 0x7ff9e00cad10 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7ff9e008ebf0 tx=0x7ff9d000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9dcff9640 1 -- 172.21.15.80:0/740104494 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0004d70 con 0x7ff9e00cec20 2024-04-25T13:46:56.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9dcff9640 1 -- 172.21.15.80:0/740104494 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d000bb80 con 0x7ff9e00cec20 2024-04-25T13:46:56.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e00cf3f0 con 0x7ff9e00cec20 2024-04-25T13:46:56.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9dcff9640 1 -- 172.21.15.80:0/740104494 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0007070 con 0x7ff9e00cec20 2024-04-25T13:46:56.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e00cf9b0 con 0x7ff9e00cec20 2024-04-25T13:46:56.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e00cf160 con 0x7ff9e00cec20 2024-04-25T13:46:56.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9dcff9640 1 -- 172.21.15.80:0/740104494 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff9d0007210 con 0x7ff9e00cec20 2024-04-25T13:46:56.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9dcff9640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c006f400 0x7ff9c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9dcff9640 1 -- 172.21.15.80:0/740104494 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7ff9d008fa60 con 0x7ff9e00cec20 2024-04-25T13:46:56.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9dffff640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c006f400 0x7ff9c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.118+0000 7ff9dffff640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c006f400 0x7ff9c0071910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8001750 tx=0x7ff9d80011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.122+0000 7ff9dcff9640 1 -- 172.21.15.80:0/740104494 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9e00ce510 con 0x7ff9e00cec20 2024-04-25T13:46:56.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9e00cfbb0 con 0x7ff9e00cec20 2024-04-25T13:46:56.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9dcff9640 1 -- 172.21.15.80:0/740104494 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) ==== 100+0+11 (secure 0 0 0) 0x7ff9e00b92a0 con 0x7ff9e00cec20 2024-04-25T13:46:56.273 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:46:56.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c006f400 msgr2=0x7ff9c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c006f400 0x7ff9c0071910 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8001750 tx=0x7ff9d80011a0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9e00cec20 msgr2=0x7ff9e00cad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9e00cec20 0x7ff9e00cad10 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7ff9e008ebf0 tx=0x7ff9d000b4a0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 shutdown_connections 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c006f400 0x7ff9c0071910 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9e00cec20 0x7ff9e00cad10 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9e00ce810 0x7ff9e00ca7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 --2- 172.21.15.80:0/740104494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9cc0039f0 0x7ff9e0153340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.274+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 >> 172.21.15.80:0/740104494 conn(0x7ff9e00737b0 msgr2=0x7ff9e0157060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.278+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 shutdown_connections 2024-04-25T13:46:56.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.278+0000 7ff9e67d9640 1 -- 172.21.15.80:0/740104494 wait complete. 2024-04-25T13:46:56.294 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:56.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.442+0000 7fbe91ed4640 1 Processor -- start 2024-04-25T13:46:56.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.442+0000 7fbe91ed4640 1 -- start start 2024-04-25T13:46:56.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe91ed4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbe8c0d06e0 0x7fbe8c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe91ed4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe8c06e9e0 0x7fbe8c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe91ed4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe8c0d0af0 con 0x7fbe8c161f90 2024-04-25T13:46:56.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe91ed4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8c0d0d70 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe91ed4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8c0d1050 con 0x7fbe8c0d06e0 2024-04-25T13:46:56.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbe8c06e9e0 0x7fbe8c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8b7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fbe8c161f90 0x7fbe8c0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59742/0 (socket says 172.21.15.80:59742) 2024-04-25T13:46:56.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8b7fe640 1 -- 172.21.15.80:0/3775703582 learned_addr learned my addr 172.21.15.80:0/3775703582 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:56.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8affd640 1 --2- 172.21.15.80:0/3775703582 >> v2:172.21.15.87:3300/0 conn(0x7fbe8c0d06e0 0x7fbe8c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8bfff640 1 -- 172.21.15.80:0/3775703582 >> v2:172.21.15.87:3300/0 conn(0x7fbe8c0d06e0 msgr2=0x7fbe8c06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:56.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8bfff640 1 --2- 172.21.15.80:0/3775703582 >> v2:172.21.15.87:3300/0 conn(0x7fbe8c0d06e0 0x7fbe8c06e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8bfff640 1 -- 172.21.15.80:0/3775703582 >> v1:172.21.15.80:6789/0 conn(0x7fbe8c161f90 legacy=0x7fbe8c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8bfff640 1 -- 172.21.15.80:0/3775703582 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c06dca0 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.446+0000 7fbe8a7fc640 1 -- 172.21.15.80:0/3775703582 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3723082925 0 0) 0x7fbe7c0014a0 con 0x7fbe8c161f90 2024-04-25T13:46:56.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe8bfff640 1 --2- 172.21.15.80:0/3775703582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c06edf0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c074290 tx=0x7fbe8401c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466dad4b7b63092a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe8a7fc640 1 -- 172.21.15.80:0/3775703582 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840b0070 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe8a7fc640 1 -- 172.21.15.80:0/3775703582 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe840ab600 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe8a7fc640 1 -- 172.21.15.80:0/3775703582 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe840ab9d0 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 -- 172.21.15.80:0/3775703582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 msgr2=0x7fbe8c06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/3775703582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c06edf0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c074290 tx=0x7fbe8401c330 comp rx=0 tx=0).stop 2024-04-25T13:46:56.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 -- 172.21.15.80:0/3775703582 shutdown_connections 2024-04-25T13:46:56.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/3775703582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c06edf0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/3775703582 >> v2:172.21.15.87:3300/0 conn(0x7fbe8c0d06e0 0x7fbe8c06e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 -- 172.21.15.80:0/3775703582 >> 172.21.15.80:0/3775703582 conn(0x7fbe8c06d080 msgr2=0x7fbe8c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:56.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 -- 172.21.15.80:0/3775703582 shutdown_connections 2024-04-25T13:46:56.449 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 -- 172.21.15.80:0/3775703582 wait complete. 2024-04-25T13:46:56.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 Processor -- start 2024-04-25T13:46:56.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.450+0000 7fbe91ed4640 1 -- start start 2024-04-25T13:46:56.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8c0d06e0 0x7fbe8c165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe8c161f90 0x7fbe8c1656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c1447a0 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c144620 con 0x7fbe8c161f90 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8c16b510 con 0x7fbe8c0d06e0 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c141640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38556/0 (socket says 172.21.15.80:38556) 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 -- 172.21.15.80:0/733793803 learned_addr learned my addr 172.21.15.80:0/733793803 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8affd640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8c0d06e0 0x7fbe8c165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8bfff640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe8c161f90 0x7fbe8c1656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 -- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8c0d06e0 msgr2=0x7fbe8c165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8c0d06e0 0x7fbe8c165160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 -- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe8c161f90 msgr2=0x7fbe8c1656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe8c161f90 0x7fbe8c1656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 -- 172.21.15.80:0/733793803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c0b2670 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8affd640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8c0d06e0 0x7fbe8c165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:56.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8b7fe640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c141640 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c780 tx=0x7fbe7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe73fff640 1 -- 172.21.15.80:0/733793803 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c018070 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe73fff640 1 -- 172.21.15.80:0/733793803 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c00a640 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c1639d0 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe73fff640 1 -- 172.21.15.80:0/733793803 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c00aa30 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c163f90 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8c06a3a0 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe73fff640 1 -- 172.21.15.80:0/733793803 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbe7c01b070 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe73fff640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7806f400 0x7fbe78071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe73fff640 1 -- 172.21.15.80:0/733793803 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7fbe7c08eac0 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.454+0000 7fbe8affd640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7806f400 0x7fbe78071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.458+0000 7fbe8affd640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7806f400 0x7fbe78071910 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c1f8d20 tx=0x7fbe80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.458+0000 7fbe73fff640 1 -- 172.21.15.80:0/733793803 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbe8c0d0af0 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe8c168710 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe73fff640 1 -- 172.21.15.80:0/733793803 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5174 (secure 0 0 0) 0x7fbe7c05bb20 con 0x7fbe8c06e9e0 2024-04-25T13:46:56.657 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:56.657 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:46:28.718395+0000","modified":"2024-04-25T13:46:31.773079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/2773338889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2773338889},{"type":"v1","addr":"172.21.15.80:6835","nonce":2773338889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":7}]} 2024-04-25T13:46:56.658 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 59 2024-04-25T13:46:56.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7806f400 msgr2=0x7fbe78071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7806f400 0x7fbe78071910 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c1f8d20 tx=0x7fbe80008040 comp rx=0 tx=0).stop 2024-04-25T13:46:56.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 msgr2=0x7fbe8c141640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c141640 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00c780 tx=0x7fbe7c00cc40 comp rx=0 tx=0).stop 2024-04-25T13:46:56.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 shutdown_connections 2024-04-25T13:46:56.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbe7806f400 0x7fbe78071910 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbe8c161f90 0x7fbe8c1656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbe8c0d06e0 0x7fbe8c165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 --2- 172.21.15.80:0/733793803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbe8c06e9e0 0x7fbe8c141640 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 >> 172.21.15.80:0/733793803 conn(0x7fbe8c06d080 msgr2=0x7fbe8c0b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:56.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 shutdown_connections 2024-04-25T13:46:56.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.658+0000 7fbe91ed4640 1 -- 172.21.15.80:0/733793803 wait complete. 2024-04-25T13:46:56.679 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:56.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cf83a640 1 Processor -- start 2024-04-25T13:46:56.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cf83a640 1 -- start start 2024-04-25T13:46:56.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cf83a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c80b1e80 0x7f99c8068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cf83a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f99c8068ad0 0x7f99c8163f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cf83a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f99c81644c0 0x7f99c806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cf83a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f99c81611f0 con 0x7f99c81644c0 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cf83a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f99c80b2290 con 0x7f99c8068ad0 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cf83a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f99c8068ee0 con 0x7f99c80b1e80 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cddd7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f99c81644c0 0x7f99c806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cddd7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f99c81644c0 0x7f99c806f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38564/0 (socket says 172.21.15.80:38564) 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cddd7640 1 -- 172.21.15.80:0/2347659041 learned_addr learned my addr 172.21.15.80:0/2347659041 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99ccdd5640 1 --2- 172.21.15.80:0/2347659041 >> v2:172.21.15.87:3300/0 conn(0x7f99c8068ad0 0x7f99c8163f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cd5d6640 1 --2- 172.21.15.80:0/2347659041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c80b1e80 0x7f99c8068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cddd7640 1 -- 172.21.15.80:0/2347659041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c80b1e80 msgr2=0x7f99c8068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:56.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cddd7640 1 --2- 172.21.15.80:0/2347659041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c80b1e80 0x7f99c8068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cddd7640 1 -- 172.21.15.80:0/2347659041 >> v2:172.21.15.87:3300/0 conn(0x7f99c8068ad0 msgr2=0x7f99c8163f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:56.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cddd7640 1 --2- 172.21.15.80:0/2347659041 >> v2:172.21.15.87:3300/0 conn(0x7f99c8068ad0 0x7f99c8163f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cddd7640 1 -- 172.21.15.80:0/2347659041 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c8164bd0 con 0x7f99c81644c0 2024-04-25T13:46:56.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99ccdd5640 1 --2- 172.21.15.80:0/2347659041 >> v2:172.21.15.87:3300/0 conn(0x7f99c8068ad0 0x7f99c8163f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:56.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.826+0000 7f99cd5d6640 1 --2- 172.21.15.80:0/2347659041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c80b1e80 0x7f99c8068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:56.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cddd7640 1 --2- 172.21.15.80:0/2347659041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c806f1c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f99c8069090 tx=0x7f99c001c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a55c0663e3fd6dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99bffff640 1 -- 172.21.15.80:0/2347659041 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00ac070 con 0x7f99c81644c0 2024-04-25T13:46:56.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99bffff640 1 -- 172.21.15.80:0/2347659041 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c00a8420 con 0x7f99c81644c0 2024-04-25T13:46:56.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99bffff640 1 -- 172.21.15.80:0/2347659041 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00a87f0 con 0x7f99c81644c0 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 -- 172.21.15.80:0/2347659041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 msgr2=0x7f99c806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 --2- 172.21.15.80:0/2347659041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c806f1c0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f99c8069090 tx=0x7f99c001c730 comp rx=0 tx=0).stop 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 -- 172.21.15.80:0/2347659041 shutdown_connections 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 --2- 172.21.15.80:0/2347659041 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c806f1c0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 --2- 172.21.15.80:0/2347659041 >> v2:172.21.15.87:3300/0 conn(0x7f99c8068ad0 0x7f99c8163f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 --2- 172.21.15.80:0/2347659041 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c80b1e80 0x7f99c8068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 -- 172.21.15.80:0/2347659041 >> 172.21.15.80:0/2347659041 conn(0x7f99c80d2ad0 msgr2=0x7f99c80d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 -- 172.21.15.80:0/2347659041 shutdown_connections 2024-04-25T13:46:56.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.830+0000 7f99cf83a640 1 -- 172.21.15.80:0/2347659041 wait complete. 2024-04-25T13:46:56.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 Processor -- start 2024-04-25T13:46:56.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 -- start start 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c8068ad0 0x7f99c816a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99c80b1e80 0x7f99c815ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f99c806c770 con 0x7f99c81644c0 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f99c806c5f0 con 0x7f99c80b1e80 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f99c806c8f0 con 0x7f99c8068ad0 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c815bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38574/0 (socket says 172.21.15.80:38574) 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 -- 172.21.15.80:0/3843245113 learned_addr learned my addr 172.21.15.80:0/3843245113 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99ccdd5640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99c80b1e80 0x7f99c815ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cd5d6640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c8068ad0 0x7f99c816a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 -- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c8068ad0 msgr2=0x7f99c816a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c8068ad0 0x7f99c816a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 -- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99c80b1e80 msgr2=0x7f99c815ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99c80b1e80 0x7f99c815ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 -- 172.21.15.80:0/3843245113 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99c8162e40 con 0x7f99c81644c0 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99ccdd5640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99c80b1e80 0x7f99c815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cddd7640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c815bfd0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f99c001cfd0 tx=0x7f99c00194d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99bdffb640 1 -- 172.21.15.80:0/3843245113 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00ac040 con 0x7f99c81644c0 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99bdffb640 1 -- 172.21.15.80:0/3843245113 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99c00bc650 con 0x7f99c81644c0 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99bdffb640 1 -- 172.21.15.80:0/3843245113 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0009510 con 0x7f99c81644c0 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99c8163120 con 0x7f99c81644c0 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99c815b290 con 0x7f99c81644c0 2024-04-25T13:46:56.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.834+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99c806a3a0 con 0x7f99c81644c0 2024-04-25T13:46:56.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.838+0000 7f99bdffb640 1 -- 172.21.15.80:0/3843245113 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f99c00af070 con 0x7f99c81644c0 2024-04-25T13:46:56.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.838+0000 7f99bdffb640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99ac06f400 0x7f99ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:56.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.838+0000 7f99bdffb640 1 -- 172.21.15.80:0/3843245113 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f99c0129d00 con 0x7f99c81644c0 2024-04-25T13:46:56.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.838+0000 7f99cd5d6640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99ac06f400 0x7f99ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:56.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.838+0000 7f99cd5d6640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99ac06f400 0x7f99ac071910 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f99c8072300 tx=0x7f99c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:56.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.842+0000 7f99bdffb640 1 -- 172.21.15.80:0/3843245113 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f99c00165e0 con 0x7f99c81644c0 2024-04-25T13:46:56.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.990+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f99c81601f0 con 0x7f99c81644c0 2024-04-25T13:46:56.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.990+0000 7f99bdffb640 1 -- 172.21.15.80:0/3843245113 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16555 (secure 0 0 0) 0x7f99c00f6b50 con 0x7f99c81644c0 2024-04-25T13:46:56.990 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:56.990 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":125,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:48.458377+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99ac06f400 msgr2=0x7f99ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99ac06f400 0x7f99ac071910 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f99c8072300 tx=0x7f99c4008040 comp rx=0 tx=0).stop 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 msgr2=0x7f99c815bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c815bfd0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f99c001cfd0 tx=0x7f99c00194d0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 shutdown_connections 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f99ac06f400 0x7f99ac071910 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f99c81644c0 0x7f99c815bfd0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f99c80b1e80 0x7f99c815ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 --2- 172.21.15.80:0/3843245113 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f99c8068ad0 0x7f99c816a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 >> 172.21.15.80:0/3843245113 conn(0x7f99c80d2ad0 msgr2=0x7f99c80b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 shutdown_connections 2024-04-25T13:46:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:56.994+0000 7f99cf83a640 1 -- 172.21.15.80:0/3843245113 wait complete. 2024-04-25T13:46:57.011 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:46:57.011 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:57.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f9354072640 1 Processor -- start 2024-04-25T13:46:57.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f9354072640 1 -- start start 2024-04-25T13:46:57.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f9354072640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f934c161240 0x7f934c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f9354072640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f934c161650 0x7f934c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f9354072640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f934c06fcf0 con 0x7f934c06f5a0 2024-04-25T13:46:57.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f9354072640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f934c073f10 con 0x7f934c161240 2024-04-25T13:46:57.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f9354072640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f934c0b0c60 con 0x7f934c161650 2024-04-25T13:46:57.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f935160d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f934c161240 0x7f934c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f935160d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f934c161240 0x7f934c0723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38580/0 (socket says 172.21.15.80:38580) 2024-04-25T13:46:57.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f935160d640 1 -- 172.21.15.80:0/217390169 learned_addr learned my addr 172.21.15.80:0/217390169 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f935260f640 1 --2- 172.21.15.80:0/217390169 >> v2:172.21.15.87:3300/0 conn(0x7f934c161650 0x7f934c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f935160d640 1 -- 172.21.15.80:0/217390169 >> v2:172.21.15.87:3300/0 conn(0x7f934c161650 msgr2=0x7f934c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f935160d640 1 --2- 172.21.15.80:0/217390169 >> v2:172.21.15.87:3300/0 conn(0x7f934c161650 0x7f934c0728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.162+0000 7f935160d640 1 -- 172.21.15.80:0/217390169 >> v1:172.21.15.87:6789/0 conn(0x7f934c06f5a0 legacy=0x7f934c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f935160d640 1 -- 172.21.15.80:0/217390169 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c0b0f40 con 0x7f934c161240 2024-04-25T13:46:57.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f935260f640 1 --2- 172.21.15.80:0/217390169 >> v2:172.21.15.87:3300/0 conn(0x7f934c161650 0x7f934c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:57.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f935160d640 1 --2- 172.21.15.80:0/217390169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c161240 0x7f934c0723a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f934c070270 tx=0x7f934001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=844037ca171ed197 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9350e0c640 1 -- 172.21.15.80:0/217390169 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400b1070 con 0x7f934c161240 2024-04-25T13:46:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9350e0c640 1 -- 172.21.15.80:0/217390169 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93400ac640 con 0x7f934c161240 2024-04-25T13:46:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9350e0c640 1 -- 172.21.15.80:0/217390169 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93400aca10 con 0x7f934c161240 2024-04-25T13:46:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9354072640 1 -- 172.21.15.80:0/217390169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c161240 msgr2=0x7f934c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9354072640 1 --2- 172.21.15.80:0/217390169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c161240 0x7f934c0723a0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f934c070270 tx=0x7f934001c490 comp rx=0 tx=0).stop 2024-04-25T13:46:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9354072640 1 -- 172.21.15.80:0/217390169 shutdown_connections 2024-04-25T13:46:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9354072640 1 --2- 172.21.15.80:0/217390169 >> v2:172.21.15.87:3300/0 conn(0x7f934c161650 0x7f934c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9354072640 1 --2- 172.21.15.80:0/217390169 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c161240 0x7f934c0723a0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9354072640 1 -- 172.21.15.80:0/217390169 >> 172.21.15.80:0/217390169 conn(0x7f934c06d080 msgr2=0x7f934c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9354072640 1 -- 172.21.15.80:0/217390169 shutdown_connections 2024-04-25T13:46:57.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.166+0000 7f9354072640 1 -- 172.21.15.80:0/217390169 wait complete. 2024-04-25T13:46:57.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 Processor -- start 2024-04-25T13:46:57.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 -- start start 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c06f5a0 0x7f934c14c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f934c161240 0x7f934c1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f934c161650 0x7f934c151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f934c06e820 con 0x7f934c06f5a0 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f934c06e6a0 con 0x7f934c161240 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f934c06e9a0 con 0x7f934c161650 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c06f5a0 0x7f934c14c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c06f5a0 0x7f934c14c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38596/0 (socket says 172.21.15.80:38596) 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 -- 172.21.15.80:0/2385431739 learned_addr learned my addr 172.21.15.80:0/2385431739 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:57.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f935260f640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f934c161650 0x7f934c151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f935160d640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f934c161240 0x7f934c1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 -- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f934c161650 msgr2=0x7f934c151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f934c161650 0x7f934c151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 -- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f934c161240 msgr2=0x7f934c1518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f934c161240 0x7f934c1518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 -- 172.21.15.80:0/2385431739 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934c14d210 con 0x7f934c06f5a0 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9351e0e640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c06f5a0 0x7f934c14c590 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f934400c370 tx=0x7f934400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f934a7fc640 1 -- 172.21.15.80:0/2385431739 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93440052d0 con 0x7f934c06f5a0 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f934a7fc640 1 -- 172.21.15.80:0/2385431739 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9344005470 con 0x7f934c06f5a0 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f934a7fc640 1 -- 172.21.15.80:0/2385431739 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9344007260 con 0x7f934c06f5a0 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f934c14d4f0 con 0x7f934c06f5a0 2024-04-25T13:46:57.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.170+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f934c147f60 con 0x7f934c06f5a0 2024-04-25T13:46:57.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.174+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f934c06a3a0 con 0x7f934c06f5a0 2024-04-25T13:46:57.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.174+0000 7f934a7fc640 1 -- 172.21.15.80:0/2385431739 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9344007400 con 0x7f934c06f5a0 2024-04-25T13:46:57.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.174+0000 7f934a7fc640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f933806f430 0x7f9338071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.174+0000 7f934a7fc640 1 -- 172.21.15.80:0/2385431739 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f934408e7a0 con 0x7f934c06f5a0 2024-04-25T13:46:57.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.174+0000 7f935160d640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f933806f430 0x7f9338071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.174+0000 7f935160d640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f933806f430 0x7f9338071940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f93400ae000 tx=0x7f9340008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.178+0000 7f934a7fc640 1 -- 172.21.15.80:0/2385431739 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f934405b690 con 0x7f934c06f5a0 2024-04-25T13:46:57.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.358+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f934c0739c0 con 0x7f934c06f5a0 2024-04-25T13:46:57.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.358+0000 7f934a7fc640 1 -- 172.21.15.80:0/2385431739 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5174 (secure 0 0 0) 0x7f934405af90 con 0x7f934c06f5a0 2024-04-25T13:46:57.358 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:57.358 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:46:28.718395+0000","modified":"2024-04-25T13:46:31.773079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/2773338889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2773338889},{"type":"v1","addr":"172.21.15.80:6835","nonce":2773338889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":7}]} 2024-04-25T13:46:57.359 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 59 2024-04-25T13:46:57.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f933806f430 msgr2=0x7f9338071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f933806f430 0x7f9338071940 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f93400ae000 tx=0x7f9340008040 comp rx=0 tx=0).stop 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c06f5a0 msgr2=0x7f934c14c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c06f5a0 0x7f934c14c590 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f934400c370 tx=0x7f934400c830 comp rx=0 tx=0).stop 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 shutdown_connections 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f933806f430 0x7f9338071940 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f934c161650 0x7f934c151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f934c161240 0x7f934c1518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 --2- 172.21.15.80:0/2385431739 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f934c06f5a0 0x7f934c14c590 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 >> 172.21.15.80:0/2385431739 conn(0x7f934c06d080 msgr2=0x7f934c0722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 shutdown_connections 2024-04-25T13:46:57.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.362+0000 7f9354072640 1 -- 172.21.15.80:0/2385431739 wait complete. 2024-04-25T13:46:57.379 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:46:57.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efd055dc640 1 Processor -- start 2024-04-25T13:46:57.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efd055dc640 1 -- start start 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efd055dc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efd000d06e0 0x7efd0006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efd055dc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd0006e9e0 0x7efd0006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efd055dc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd000d0af0 con 0x7efd00161f90 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efd055dc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efd000d0d70 con 0x7efd0006e9e0 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efd055dc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efd000d1050 con 0x7efd000d06e0 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcff7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd0006e9e0 0x7efd0006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcff7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efd0006e9e0 0x7efd0006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38606/0 (socket says 172.21.15.80:38606) 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcff7fe640 1 -- 172.21.15.80:0/3298042707 learned_addr learned my addr 172.21.15.80:0/3298042707 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:57.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcfe7fc640 1 --2- 172.21.15.80:0/3298042707 >> v2:172.21.15.87:3300/0 conn(0x7efd000d06e0 0x7efd0006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcff7fe640 1 -- 172.21.15.80:0/3298042707 >> v2:172.21.15.87:3300/0 conn(0x7efd000d06e0 msgr2=0x7efd0006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcff7fe640 1 --2- 172.21.15.80:0/3298042707 >> v2:172.21.15.87:3300/0 conn(0x7efd000d06e0 0x7efd0006e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcff7fe640 1 -- 172.21.15.80:0/3298042707 >> v1:172.21.15.87:6789/0 conn(0x7efd00161f90 legacy=0x7efd000cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcff7fe640 1 -- 172.21.15.80:0/3298042707 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0006dca0 con 0x7efd0006e9e0 2024-04-25T13:46:57.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.522+0000 7efcfe7fc640 1 --2- 172.21.15.80:0/3298042707 >> v2:172.21.15.87:3300/0 conn(0x7efd000d06e0 0x7efd0006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efcff7fe640 1 --2- 172.21.15.80:0/3298042707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd0006e9e0 0x7efd0006edf0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7efd00074290 tx=0x7efcf001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f6d7c12bc57943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efcfdffb640 1 -- 172.21.15.80:0/3298042707 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00a5670 con 0x7efd0006e9e0 2024-04-25T13:46:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efcfdffb640 1 -- 172.21.15.80:0/3298042707 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf00a5810 con 0x7efd0006e9e0 2024-04-25T13:46:57.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efcfdffb640 1 -- 172.21.15.80:0/3298042707 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00a5be0 con 0x7efd0006e9e0 2024-04-25T13:46:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efd055dc640 1 -- 172.21.15.80:0/3298042707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd0006e9e0 msgr2=0x7efd0006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efd055dc640 1 --2- 172.21.15.80:0/3298042707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd0006e9e0 0x7efd0006edf0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7efd00074290 tx=0x7efcf001c8a0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efd055dc640 1 -- 172.21.15.80:0/3298042707 shutdown_connections 2024-04-25T13:46:57.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efd055dc640 1 --2- 172.21.15.80:0/3298042707 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd0006e9e0 0x7efd0006edf0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efd055dc640 1 --2- 172.21.15.80:0/3298042707 >> v2:172.21.15.87:3300/0 conn(0x7efd000d06e0 0x7efd0006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efd055dc640 1 -- 172.21.15.80:0/3298042707 >> 172.21.15.80:0/3298042707 conn(0x7efd0006d080 msgr2=0x7efd000b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:57.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efd055dc640 1 -- 172.21.15.80:0/3298042707 shutdown_connections 2024-04-25T13:46:57.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.526+0000 7efd055dc640 1 -- 172.21.15.80:0/3298042707 wait complete. 2024-04-25T13:46:57.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 Processor -- start 2024-04-25T13:46:57.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 -- start start 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd0006e9e0 0x7efd00163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd000d06e0 0x7efd000b5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd00161f90 0x7efd0015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efd001447a0 con 0x7efd00161f90 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efd00144620 con 0x7efd0006e9e0 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efd0016b510 con 0x7efd000d06e0 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd00161f90 0x7efd0015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd00161f90 0x7efd0015d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38614/0 (socket says 172.21.15.80:38614) 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 -- 172.21.15.80:0/3887391612 learned_addr learned my addr 172.21.15.80:0/3887391612 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcfeffd640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd0006e9e0 0x7efd00163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcfe7fc640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd000d06e0 0x7efd000b5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 -- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd000d06e0 msgr2=0x7efd000b5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd000d06e0 0x7efd000b5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 -- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd0006e9e0 msgr2=0x7efd00163ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd0006e9e0 0x7efd00163ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 -- 172.21.15.80:0/3887391612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd00163c90 con 0x7efd00161f90 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcfeffd640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd0006e9e0 0x7efd00163ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcff7fe640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd00161f90 0x7efd0015d7b0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7efcf00194b0 tx=0x7efcf0019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcf77fe640 1 -- 172.21.15.80:0/3887391612 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf00bc020 con 0x7efd00161f90 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcf77fe640 1 -- 172.21.15.80:0/3887391612 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efcf00b8670 con 0x7efd00161f90 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd00165e20 con 0x7efd00161f90 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efcf77fe640 1 -- 172.21.15.80:0/3887391612 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf0009510 con 0x7efd00161f90 2024-04-25T13:46:57.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.530+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd00166100 con 0x7efd00161f90 2024-04-25T13:46:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.534+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd0006a3a0 con 0x7efd00161f90 2024-04-25T13:46:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.534+0000 7efcf77fe640 1 -- 172.21.15.80:0/3887391612 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efcf00bf020 con 0x7efd00161f90 2024-04-25T13:46:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.534+0000 7efcf77fe640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce006f400 0x7efce0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.534+0000 7efcf77fe640 1 -- 172.21.15.80:0/3887391612 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7efcf012a2f0 con 0x7efd00161f90 2024-04-25T13:46:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.534+0000 7efcfeffd640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce006f400 0x7efce0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.534+0000 7efcfeffd640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce006f400 0x7efce0071910 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efd0006fa20 tx=0x7efcf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.538+0000 7efcf77fe640 1 -- 172.21.15.80:0/3887391612 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efcf00c6030 con 0x7efd00161f90 2024-04-25T13:46:57.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.718+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd00168770 con 0x7efd00161f90 2024-04-25T13:46:57.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.718+0000 7efcf77fe640 1 -- 172.21.15.80:0/3887391612 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) ==== 94+0+5174 (secure 0 0 0) 0x7efcf00165e0 con 0x7efd00161f90 2024-04-25T13:46:57.718 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:57.718 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":59,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52}],"filesystems":[{"mdsmap":{"epoch":59,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:46:28.718395+0000","modified":"2024-04-25T13:46:31.773079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"a","rank":0,"incarnation":55,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/2773338889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":2773338889},{"type":"v1","addr":"172.21.15.80:6835","nonce":2773338889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":7}]} 2024-04-25T13:46:57.719 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 59 2024-04-25T13:46:57.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce006f400 msgr2=0x7efce0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce006f400 0x7efce0071910 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7efd0006fa20 tx=0x7efcf8009040 comp rx=0 tx=0).stop 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd00161f90 msgr2=0x7efd0015d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd00161f90 0x7efd0015d7b0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7efcf00194b0 tx=0x7efcf0019640 comp rx=0 tx=0).stop 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 shutdown_connections 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce006f400 0x7efce0071910 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efd00161f90 0x7efd0015d7b0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efd000d06e0 0x7efd000b5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 --2- 172.21.15.80:0/3887391612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efd0006e9e0 0x7efd00163ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 >> 172.21.15.80:0/3887391612 conn(0x7efd0006d080 msgr2=0x7efd000d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 shutdown_connections 2024-04-25T13:46:57.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.722+0000 7efd055dc640 1 -- 172.21.15.80:0/3887391612 wait complete. 2024-04-25T13:46:57.739 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:46:57.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.874+0000 7f2b660bc640 1 Processor -- start 2024-04-25T13:46:57.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.874+0000 7f2b660bc640 1 -- start start 2024-04-25T13:46:57.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b660bc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b600d06e0 0x7f2b6006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b660bc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2b6006e9e0 0x7f2b6006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b660bc640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b600d0af0 con 0x7f2b60161f90 2024-04-25T13:46:57.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b660bc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2b600d0d70 con 0x7f2b6006e9e0 2024-04-25T13:46:57.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b660bc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2b600d1050 con 0x7f2b600d06e0 2024-04-25T13:46:57.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2b6006e9e0 0x7f2b6006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5f7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2b60161f90 0x7f2b600cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:59752/0 (socket says 172.21.15.80:59752) 2024-04-25T13:46:57.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5f7fe640 1 -- 172.21.15.80:0/267504768 learned_addr learned my addr 172.21.15.80:0/267504768 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:46:57.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5effd640 1 --2- 172.21.15.80:0/267504768 >> v2:172.21.15.87:3300/0 conn(0x7f2b600d06e0 0x7f2b6006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5ffff640 1 -- 172.21.15.80:0/267504768 >> v2:172.21.15.87:3300/0 conn(0x7f2b600d06e0 msgr2=0x7f2b6006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:57.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5ffff640 1 --2- 172.21.15.80:0/267504768 >> v2:172.21.15.87:3300/0 conn(0x7f2b600d06e0 0x7f2b6006e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5ffff640 1 -- 172.21.15.80:0/267504768 >> v1:172.21.15.80:6789/0 conn(0x7f2b60161f90 legacy=0x7f2b600cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5e7fc640 1 -- 172.21.15.80:0/267504768 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3645318394 0 0) 0x7f2b50001120 con 0x7f2b60161f90 2024-04-25T13:46:57.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.878+0000 7f2b5ffff640 1 -- 172.21.15.80:0/267504768 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b6006dca0 con 0x7f2b6006e9e0 2024-04-25T13:46:57.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b5ffff640 1 --2- 172.21.15.80:0/267504768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b6006e9e0 0x7f2b6006edf0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2b60074290 tx=0x7f2b5801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1f6f84772ade8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b5e7fc640 1 -- 172.21.15.80:0/267504768 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b580b2070 con 0x7f2b6006e9e0 2024-04-25T13:46:57.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b5e7fc640 1 -- 172.21.15.80:0/267504768 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b580a57b0 con 0x7f2b6006e9e0 2024-04-25T13:46:57.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b5e7fc640 1 -- 172.21.15.80:0/267504768 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b580a5b80 con 0x7f2b6006e9e0 2024-04-25T13:46:57.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 -- 172.21.15.80:0/267504768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b6006e9e0 msgr2=0x7f2b6006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 --2- 172.21.15.80:0/267504768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b6006e9e0 0x7f2b6006edf0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f2b60074290 tx=0x7f2b5801c7e0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 -- 172.21.15.80:0/267504768 shutdown_connections 2024-04-25T13:46:57.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 --2- 172.21.15.80:0/267504768 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b6006e9e0 0x7f2b6006edf0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 --2- 172.21.15.80:0/267504768 >> v2:172.21.15.87:3300/0 conn(0x7f2b600d06e0 0x7f2b6006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 -- 172.21.15.80:0/267504768 >> 172.21.15.80:0/267504768 conn(0x7f2b6006d080 msgr2=0x7f2b600b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:57.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 -- 172.21.15.80:0/267504768 shutdown_connections 2024-04-25T13:46:57.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 -- 172.21.15.80:0/267504768 wait complete. 2024-04-25T13:46:57.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 Processor -- start 2024-04-25T13:46:57.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.882+0000 7f2b660bc640 1 -- start start 2024-04-25T13:46:57.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b6006e9e0 0x7f2b60141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b600d06e0 0x7f2b60165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b60161f90 0x7f2b601656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b601447a0 con 0x7f2b60161f90 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b60144620 con 0x7f2b6006e9e0 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b6016b510 con 0x7f2b600d06e0 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b60161f90 0x7f2b601656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b60161f90 0x7f2b601656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38632/0 (socket says 172.21.15.80:38632) 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 -- 172.21.15.80:0/2317709926 learned_addr learned my addr 172.21.15.80:0/2317709926 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5f7fe640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b6006e9e0 0x7f2b60141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5effd640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b600d06e0 0x7f2b60165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 -- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b600d06e0 msgr2=0x7f2b60165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b600d06e0 0x7f2b60165160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 -- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b6006e9e0 msgr2=0x7f2b60141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:57.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b6006e9e0 0x7f2b60141640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 -- 172.21.15.80:0/2317709926 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b600b2670 con 0x7f2b60161f90 2024-04-25T13:46:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b5ffff640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b60161f90 0x7f2b601656a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2b580193f0 tx=0x7f2b58019580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b47fff640 1 -- 172.21.15.80:0/2317709926 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b580bd020 con 0x7f2b60161f90 2024-04-25T13:46:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b47fff640 1 -- 172.21.15.80:0/2317709926 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b580b8470 con 0x7f2b60161f90 2024-04-25T13:46:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b601639d0 con 0x7f2b60161f90 2024-04-25T13:46:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b47fff640 1 -- 172.21.15.80:0/2317709926 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b580b2040 con 0x7f2b60161f90 2024-04-25T13:46:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b60163f90 con 0x7f2b60161f90 2024-04-25T13:46:57.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b6006a3a0 con 0x7f2b60161f90 2024-04-25T13:46:57.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b47fff640 1 -- 172.21.15.80:0/2317709926 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2b580c0020 con 0x7f2b60161f90 2024-04-25T13:46:57.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b47fff640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4c06f400 0x7f2b4c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:57.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.886+0000 7f2b47fff640 1 -- 172.21.15.80:0/2317709926 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f2b5812a8a0 con 0x7f2b60161f90 2024-04-25T13:46:57.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.890+0000 7f2b5f7fe640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4c06f400 0x7f2b4c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:57.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.890+0000 7f2b5f7fe640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4c06f400 0x7f2b4c071910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000f70 tx=0x7f2b50007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:57.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:57.890+0000 7f2b47fff640 1 -- 172.21.15.80:0/2317709926 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b580c7070 con 0x7f2b60161f90 2024-04-25T13:46:58.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.046+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2b60163740 con 0x7f2b60161f90 2024-04-25T13:46:58.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b47fff640 1 -- 172.21.15.80:0/2317709926 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) ==== 74+0+16555 (secure 0 0 0) 0x7f2b58016520 con 0x7f2b60161f90 2024-04-25T13:46:58.048 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:46:58.049 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":125,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:46:48.458377+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:46:27.136651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T13:46:27.696536+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:46:58.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4c06f400 msgr2=0x7f2b4c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4c06f400 0x7f2b4c071910 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2b50000f70 tx=0x7f2b50007c10 comp rx=0 tx=0).stop 2024-04-25T13:46:58.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b60161f90 msgr2=0x7f2b601656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b60161f90 0x7f2b601656a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f2b580193f0 tx=0x7f2b58019580 comp rx=0 tx=0).stop 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 shutdown_connections 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4c06f400 0x7f2b4c071910 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b60161f90 0x7f2b601656a0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b600d06e0 0x7f2b60165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 --2- 172.21.15.80:0/2317709926 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b6006e9e0 0x7f2b60141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 >> 172.21.15.80:0/2317709926 conn(0x7f2b6006d080 msgr2=0x7f2b600b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 shutdown_connections 2024-04-25T13:46:58.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.050+0000 7f2b660bc640 1 -- 172.21.15.80:0/2317709926 wait complete. 2024-04-25T13:46:58.069 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:46:58.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae9888640 1 Processor -- start 2024-04-25T13:46:58.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae9888640 1 -- start start 2024-04-25T13:46:58.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae9888640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ae40d5490 0x7f4ae40d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae9888640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4ae40d2560 0x7f4ae40d2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae9888640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ae40d43f0 con 0x7f4ae40b76d0 2024-04-25T13:46:58.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae9888640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4ae40d46d0 con 0x7f4ae40d2560 2024-04-25T13:46:58.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae9888640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4ae40d49b0 con 0x7f4ae40d5490 2024-04-25T13:46:58.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae27fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4ae40d2560 0x7f4ae40d2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae27fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4ae40d2560 0x7f4ae40d2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38634/0 (socket says 172.21.15.80:38634) 2024-04-25T13:46:58.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae27fc640 1 -- 172.21.15.80:0/4151674710 learned_addr learned my addr 172.21.15.80:0/4151674710 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:58.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.218+0000 7f4ae2ffd640 1 --2- 172.21.15.80:0/4151674710 >> v2:172.21.15.87:3300/0 conn(0x7f4ae40d5490 0x7f4ae40d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae27fc640 1 -- 172.21.15.80:0/4151674710 >> v2:172.21.15.87:3300/0 conn(0x7f4ae40d5490 msgr2=0x7f4ae40d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:58.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae27fc640 1 --2- 172.21.15.80:0/4151674710 >> v2:172.21.15.87:3300/0 conn(0x7f4ae40d5490 0x7f4ae40d14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae27fc640 1 -- 172.21.15.80:0/4151674710 >> v1:172.21.15.87:6789/0 conn(0x7f4ae40b76d0 legacy=0x7f4ae40b7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae27fc640 1 -- 172.21.15.80:0/4151674710 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae40d4c90 con 0x7f4ae40d2560 2024-04-25T13:46:58.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae27fc640 1 --2- 172.21.15.80:0/4151674710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d2560 0x7f4ae40d2970 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4ae406d5d0 tx=0x7f4ad801cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=788c88df42abb301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:58.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae1ffb640 1 -- 172.21.15.80:0/4151674710 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80ab430 con 0x7f4ae40d2560 2024-04-25T13:46:58.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae1ffb640 1 -- 172.21.15.80:0/4151674710 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ad80b0070 con 0x7f4ae40d2560 2024-04-25T13:46:58.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae1ffb640 1 -- 172.21.15.80:0/4151674710 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ad80ab7f0 con 0x7f4ae40d2560 2024-04-25T13:46:58.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae9888640 1 -- 172.21.15.80:0/4151674710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d2560 msgr2=0x7f4ae40d2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae9888640 1 --2- 172.21.15.80:0/4151674710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d2560 0x7f4ae40d2970 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4ae406d5d0 tx=0x7f4ad801cf80 comp rx=0 tx=0).stop 2024-04-25T13:46:58.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae9888640 1 -- 172.21.15.80:0/4151674710 shutdown_connections 2024-04-25T13:46:58.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae9888640 1 --2- 172.21.15.80:0/4151674710 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d2560 0x7f4ae40d2970 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae9888640 1 --2- 172.21.15.80:0/4151674710 >> v2:172.21.15.87:3300/0 conn(0x7f4ae40d5490 0x7f4ae40d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae9888640 1 -- 172.21.15.80:0/4151674710 >> 172.21.15.80:0/4151674710 conn(0x7f4ae4073740 msgr2=0x7f4ae406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:58.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae9888640 1 -- 172.21.15.80:0/4151674710 shutdown_connections 2024-04-25T13:46:58.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.222+0000 7f4ae9888640 1 -- 172.21.15.80:0/4151674710 wait complete. 2024-04-25T13:46:58.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 Processor -- start 2024-04-25T13:46:58.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 -- start start 2024-04-25T13:46:58.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ae40b76d0 0x7f4ae4157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ae40d2560 0x7f4ae41608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d5490 0x7f4ae4160e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae40b50b0 con 0x7f4ae40d5490 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4ae40b4f30 con 0x7f4ae40d2560 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ae416d650 con 0x7f4ae40b76d0 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d5490 0x7f4ae4160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d5490 0x7f4ae4160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38636/0 (socket says 172.21.15.80:38636) 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 -- 172.21.15.80:0/2753862364 learned_addr learned my addr 172.21.15.80:0/2753862364 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae2ffd640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ae40b76d0 0x7f4ae4157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae27fc640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ae40d2560 0x7f4ae41608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 -- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ae40b76d0 msgr2=0x7f4ae4157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ae40b76d0 0x7f4ae4157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 -- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ae40d2560 msgr2=0x7f4ae41608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ae40d2560 0x7f4ae41608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 -- 172.21.15.80:0/2753862364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ae4162d60 con 0x7f4ae40d5490 2024-04-25T13:46:58.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae37fe640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d5490 0x7f4ae4160e30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00c760 tx=0x7f4adc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:58.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4acbfff640 1 -- 172.21.15.80:0/2753862364 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc003b80 con 0x7f4ae40d5490 2024-04-25T13:46:58.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4acbfff640 1 -- 172.21.15.80:0/2753862364 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4adc003d20 con 0x7f4ae40d5490 2024-04-25T13:46:58.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ae41588e0 con 0x7f4ae40d5490 2024-04-25T13:46:58.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4acbfff640 1 -- 172.21.15.80:0/2753862364 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4adc00aa90 con 0x7f4ae40d5490 2024-04-25T13:46:58.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.226+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ae4158ea0 con 0x7f4ae40d5490 2024-04-25T13:46:58.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.230+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ae4163860 con 0x7f4ae40d5490 2024-04-25T13:46:58.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.230+0000 7f4acbfff640 1 -- 172.21.15.80:0/2753862364 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4adc00ac30 con 0x7f4ae40d5490 2024-04-25T13:46:58.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.230+0000 7f4acbfff640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ad006f400 0x7f4ad0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.230+0000 7f4acbfff640 1 -- 172.21.15.80:0/2753862364 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 5709+0+0 (secure 0 0 0) 0x7f4adc08f0b0 con 0x7f4ae40d5490 2024-04-25T13:46:58.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.230+0000 7f4ae2ffd640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ad006f400 0x7f4ad0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.230+0000 7f4ae2ffd640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ad006f400 0x7f4ad0071910 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006fd0 tx=0x7f4ad4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:58.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.234+0000 7f4acbfff640 1 -- 172.21.15.80:0/2753862364 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4ae4163860 con 0x7f4ae40d5490 2024-04-25T13:46:58.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.430+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4ae40bb850 con 0x7f4ae40d5490 2024-04-25T13:46:58.431 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:46:58.430+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:46:58.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.434+0000 7f4acbfff640 1 -- 172.21.15.80:0/2753862364 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v60) ==== 161+0+0 (secure 0 0 0) 0x7f4adc05c030 con 0x7f4ae40d5490 2024-04-25T13:46:58.433 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:46:58.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ad006f400 msgr2=0x7f4ad0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ad006f400 0x7f4ad0071910 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4ad4006fd0 tx=0x7f4ad4008040 comp rx=0 tx=0).stop 2024-04-25T13:46:58.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d5490 msgr2=0x7f4ae4160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d5490 0x7f4ae4160e30 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f4adc00c760 tx=0x7f4adc00cc20 comp rx=0 tx=0).stop 2024-04-25T13:46:58.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 shutdown_connections 2024-04-25T13:46:58.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ad006f400 0x7f4ad0071910 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ae40d5490 0x7f4ae4160e30 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ae40d2560 0x7f4ae41608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 --2- 172.21.15.80:0/2753862364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ae40b76d0 0x7f4ae4157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 >> 172.21.15.80:0/2753862364 conn(0x7f4ae4073740 msgr2=0x7f4ae40b7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:58.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 shutdown_connections 2024-04-25T13:46:58.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.442+0000 7f4ae9888640 1 -- 172.21.15.80:0/2753862364 wait complete. 2024-04-25T13:46:58.444 INFO:tasks.ceph.mds.a.smithi080.stderr: -3610> 2024-04-25T13:46:34.307+0000 7f83b3cbb640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:46:58.462 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:46:58.511 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:46:58.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.598+0000 7f7c4d6d0640 1 Processor -- start 2024-04-25T13:46:58.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.598+0000 7f7c4d6d0640 1 -- start start 2024-04-25T13:46:58.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c4d6d0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c4815d240 0x7f7c480d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c4d6d0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7c480d16f0 0x7f7c4806fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c4d6d0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c480d1b00 con 0x7f7c480702d0 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c4d6d0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7c4815e130 con 0x7f7c4815d240 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c4d6d0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7c4815e5c0 con 0x7f7c480d16f0 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c46ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c4815d240 0x7f7c480d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c46ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7c4815d240 0x7f7c480d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38662/0 (socket says 172.21.15.80:38662) 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c46ffd640 1 -- 172.21.15.80:0/193757861 learned_addr learned my addr 172.21.15.80:0/193757861 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c46ffd640 1 -- 172.21.15.80:0/193757861 >> v2:172.21.15.87:3300/0 conn(0x7f7c480d16f0 msgr2=0x7f7c4806fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c3ffff640 1 --2- 172.21.15.80:0/193757861 >> v2:172.21.15.87:3300/0 conn(0x7f7c480d16f0 0x7f7c4806fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c46ffd640 1 --2- 172.21.15.80:0/193757861 >> v2:172.21.15.87:3300/0 conn(0x7f7c480d16f0 0x7f7c4806fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c46ffd640 1 -- 172.21.15.80:0/193757861 >> v1:172.21.15.87:6789/0 conn(0x7f7c480702d0 legacy=0x7f7c4806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:58.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.602+0000 7f7c46ffd640 1 -- 172.21.15.80:0/193757861 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4815d800 con 0x7f7c4815d240 2024-04-25T13:46:58.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c46ffd640 1 --2- 172.21.15.80:0/193757861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c4815d240 0x7f7c480d11b0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7c48074500 tx=0x7f7c3801c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491f9b264f889c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:58.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c467fc640 1 -- 172.21.15.80:0/193757861 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c380b0070 con 0x7f7c4815d240 2024-04-25T13:46:58.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c467fc640 1 -- 172.21.15.80:0/193757861 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c380ab600 con 0x7f7c4815d240 2024-04-25T13:46:58.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c467fc640 1 -- 172.21.15.80:0/193757861 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c380ab9d0 con 0x7f7c4815d240 2024-04-25T13:46:58.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/193757861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c4815d240 msgr2=0x7f7c480d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/193757861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c4815d240 0x7f7c480d11b0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f7c48074500 tx=0x7f7c3801c1e0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/193757861 shutdown_connections 2024-04-25T13:46:58.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/193757861 >> v2:172.21.15.87:3300/0 conn(0x7f7c480d16f0 0x7f7c4806fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/193757861 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c4815d240 0x7f7c480d11b0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/193757861 >> 172.21.15.80:0/193757861 conn(0x7f7c480cf450 msgr2=0x7f7c480cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:58.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/193757861 shutdown_connections 2024-04-25T13:46:58.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/193757861 wait complete. 2024-04-25T13:46:58.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 Processor -- start 2024-04-25T13:46:58.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.606+0000 7f7c4d6d0640 1 -- start start 2024-04-25T13:46:58.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c480702d0 0x7f7c4814c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c480d16f0 0x7f7c4815f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c4815d240 0x7f7c481490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48160c00 con 0x7f7c480d16f0 2024-04-25T13:46:58.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c48070af0 con 0x7f7c480702d0 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c48160d80 con 0x7f7c4815d240 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c480d16f0 0x7f7c4815f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c480d16f0 0x7f7c4815f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38670/0 (socket says 172.21.15.80:38670) 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 -- 172.21.15.80:0/3028490864 learned_addr learned my addr 172.21.15.80:0/3028490864 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c477fe640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c4815d240 0x7f7c481490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c46ffd640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c480702d0 0x7f7c4814c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 -- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c4815d240 msgr2=0x7f7c481490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c4815d240 0x7f7c481490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 -- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c480702d0 msgr2=0x7f7c4814c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:58.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c480702d0 0x7f7c4814c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 -- 172.21.15.80:0/3028490864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c48155110 con 0x7f7c480d16f0 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c477fe640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c4815d240 0x7f7c481490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c46ffd640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c480702d0 0x7f7c4814c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3ffff640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c480d16f0 0x7f7c4815f9f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c610 tx=0x7f7c3000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3f7fe640 1 -- 172.21.15.80:0/3028490864 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30015070 con 0x7f7c480d16f0 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3f7fe640 1 -- 172.21.15.80:0/3028490864 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c30004220 con 0x7f7c480d16f0 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c481553f0 con 0x7f7c480d16f0 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3f7fe640 1 -- 172.21.15.80:0/3028490864 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30003620 con 0x7f7c480d16f0 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c481559b0 con 0x7f7c480d16f0 2024-04-25T13:46:58.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c4806a3a0 con 0x7f7c480d16f0 2024-04-25T13:46:58.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.610+0000 7f7c3f7fe640 1 -- 172.21.15.80:0/3028490864 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7c300037c0 con 0x7f7c480d16f0 2024-04-25T13:46:58.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.614+0000 7f7c3f7fe640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2806f400 0x7f7c28071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:58.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.614+0000 7f7c3f7fe640 1 -- 172.21.15.80:0/3028490864 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5795+0+0 (secure 0 0 0) 0x7f7c3008f0e0 con 0x7f7c480d16f0 2024-04-25T13:46:58.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.614+0000 7f7c46ffd640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2806f400 0x7f7c28071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:58.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.614+0000 7f7c46ffd640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2806f400 0x7f7c28071910 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ad000 tx=0x7f7c38008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:58.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.618+0000 7f7c3f7fe640 1 -- 172.21.15.80:0/3028490864 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c3005bf70 con 0x7f7c480d16f0 2024-04-25T13:46:58.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:58.798+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7c48154ea0 con 0x7f7c480d16f0 2024-04-25T13:46:59.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c3f7fe640 1 -- 172.21.15.80:0/3028490864 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v61) ==== 104+0+0 (secure 0 0 0) 0x7f7c3005b870 con 0x7f7c480d16f0 2024-04-25T13:46:59.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2806f400 msgr2=0x7f7c28071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:59.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2806f400 0x7f7c28071910 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7c380ad000 tx=0x7f7c38008040 comp rx=0 tx=0).stop 2024-04-25T13:46:59.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c480d16f0 msgr2=0x7f7c4815f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:59.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c480d16f0 0x7f7c4815f9f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000c610 tx=0x7f7c3000cad0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 shutdown_connections 2024-04-25T13:46:59.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2806f400 0x7f7c28071910 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c4815d240 0x7f7c481490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c480d16f0 0x7f7c4815f9f0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 --2- 172.21.15.80:0/3028490864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c480702d0 0x7f7c4814c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.438+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 >> 172.21.15.80:0/3028490864 conn(0x7f7c480cf450 msgr2=0x7f7c4815c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:59.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.442+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 shutdown_connections 2024-04-25T13:46:59.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.442+0000 7f7c4d6d0640 1 -- 172.21.15.80:0/3028490864 wait complete. 2024-04-25T13:46:59.461 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:46:59.464 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:46:59.471 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:46:59.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd253034640 1 Processor -- start 2024-04-25T13:46:59.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd253034640 1 -- start start 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd253034640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd24c0d0ed0 0x7fd24c0d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd253034640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd24c0d4820 0x7fd24c0d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd253034640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd24c0d1820 con 0x7fd24c0d24a0 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd253034640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd24c164040 con 0x7fd24c0d4820 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd253034640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd24c164bc0 con 0x7fd24c0d0ed0 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd2515d1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd24c0d4820 0x7fd24c0d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd2515d1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd24c0d4820 0x7fd24c0d4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38682/0 (socket says 172.21.15.80:38682) 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd2515d1640 1 -- 172.21.15.80:0/3504662922 learned_addr learned my addr 172.21.15.80:0/3504662922 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd250dd0640 1 --2- 172.21.15.80:0/3504662922 >> v2:172.21.15.87:3300/0 conn(0x7fd24c0d0ed0 0x7fd24c0d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:59.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd2515d1640 1 -- 172.21.15.80:0/3504662922 >> v2:172.21.15.87:3300/0 conn(0x7fd24c0d0ed0 msgr2=0x7fd24c0d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:59.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd2515d1640 1 --2- 172.21.15.80:0/3504662922 >> v2:172.21.15.87:3300/0 conn(0x7fd24c0d0ed0 0x7fd24c0d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd2515d1640 1 -- 172.21.15.80:0/3504662922 >> v1:172.21.15.87:6789/0 conn(0x7fd24c0d24a0 legacy=0x7fd24c0d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:46:59.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.674+0000 7fd2515d1640 1 -- 172.21.15.80:0/3504662922 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c0d2fc0 con 0x7fd24c0d4820 2024-04-25T13:46:59.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd2515d1640 1 --2- 172.21.15.80:0/3504662922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d4820 0x7fd24c0d4c30 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0d33d0 tx=0x7fd24401c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a7a62b0e816196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:59.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd24b7fe640 1 -- 172.21.15.80:0/3504662922 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2440a5660 con 0x7fd24c0d4820 2024-04-25T13:46:59.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd24b7fe640 1 -- 172.21.15.80:0/3504662922 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd2440a5800 con 0x7fd24c0d4820 2024-04-25T13:46:59.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd24b7fe640 1 -- 172.21.15.80:0/3504662922 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2440a5bd0 con 0x7fd24c0d4820 2024-04-25T13:46:59.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd253034640 1 -- 172.21.15.80:0/3504662922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d4820 msgr2=0x7fd24c0d4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:59.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd253034640 1 --2- 172.21.15.80:0/3504662922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d4820 0x7fd24c0d4c30 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fd24c0d33d0 tx=0x7fd24401c7a0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd253034640 1 -- 172.21.15.80:0/3504662922 shutdown_connections 2024-04-25T13:46:59.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd253034640 1 --2- 172.21.15.80:0/3504662922 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d4820 0x7fd24c0d4c30 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd253034640 1 --2- 172.21.15.80:0/3504662922 >> v2:172.21.15.87:3300/0 conn(0x7fd24c0d0ed0 0x7fd24c0d12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd253034640 1 -- 172.21.15.80:0/3504662922 >> 172.21.15.80:0/3504662922 conn(0x7fd24c06c5f0 msgr2=0x7fd24c06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:46:59.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd253034640 1 -- 172.21.15.80:0/3504662922 shutdown_connections 2024-04-25T13:46:59.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.678+0000 7fd253034640 1 -- 172.21.15.80:0/3504662922 wait complete. 2024-04-25T13:46:59.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 Processor -- start 2024-04-25T13:46:59.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 -- start start 2024-04-25T13:46:59.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c0d0ed0 0x7fd24c14c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:59.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d24a0 0x7fd24c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:59.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c0d4820 0x7fd24c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:59.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c16d3e0 con 0x7fd24c0d24a0 2024-04-25T13:46:59.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd24c157820 con 0x7fd24c0d0ed0 2024-04-25T13:46:59.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd24c16d560 con 0x7fd24c0d4820 2024-04-25T13:46:59.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d24a0 0x7fd24c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:59.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d24a0 0x7fd24c15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38684/0 (socket says 172.21.15.80:38684) 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 -- 172.21.15.80:0/2353600499 learned_addr learned my addr 172.21.15.80:0/2353600499 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd2515d1640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c0d4820 0x7fd24c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd250dd0640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c0d0ed0 0x7fd24c14c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 -- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c0d4820 msgr2=0x7fd24c14f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c0d4820 0x7fd24c14f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 -- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c0d0ed0 msgr2=0x7fd24c14c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c0d0ed0 0x7fd24c14c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 -- 172.21.15.80:0/2353600499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd24c158f00 con 0x7fd24c0d24a0 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd2515d1640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c0d4820 0x7fd24c14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd250dd0640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c0d0ed0 0x7fd24c14c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd24bfff640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d24a0 0x7fd24c15b080 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd24000b440 tx=0x7fd24000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:59.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd2497fa640 1 -- 172.21.15.80:0/2353600499 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240015020 con 0x7fd24c0d24a0 2024-04-25T13:46:59.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd2497fa640 1 -- 172.21.15.80:0/2353600499 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd240011890 con 0x7fd24c0d24a0 2024-04-25T13:46:59.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd24c15dd10 con 0x7fd24c0d24a0 2024-04-25T13:46:59.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd2497fa640 1 -- 172.21.15.80:0/2353600499 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240011c80 con 0x7fd24c0d24a0 2024-04-25T13:46:59.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd24c15bbc0 con 0x7fd24c0d24a0 2024-04-25T13:46:59.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.682+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd24c06a440 con 0x7fd24c0d24a0 2024-04-25T13:46:59.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.686+0000 7fd2497fa640 1 -- 172.21.15.80:0/2353600499 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd240018020 con 0x7fd24c0d24a0 2024-04-25T13:46:59.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.686+0000 7fd2497fa640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd23806f430 0x7fd238071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:46:59.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.686+0000 7fd2497fa640 1 -- 172.21.15.80:0/2353600499 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5795+0+0 (secure 0 0 0) 0x7fd24008f4a0 con 0x7fd24c0d24a0 2024-04-25T13:46:59.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.686+0000 7fd250dd0640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd23806f430 0x7fd238071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:46:59.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.686+0000 7fd250dd0640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd23806f430 0x7fd238071940 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0097a0 tx=0x7fd23c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:46:59.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.690+0000 7fd2497fa640 1 -- 172.21.15.80:0/2353600499 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd24001f030 con 0x7fd24c0d24a0 2024-04-25T13:46:59.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:46:59.854+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd24c160400 con 0x7fd24c0d24a0 2024-04-25T13:47:00.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.586+0000 7fd2497fa640 1 -- 172.21.15.80:0/2353600499 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v127) ==== 181+0+0 (secure 0 0 0) 0x7fd24000d040 con 0x7fd24c0d24a0 2024-04-25T13:47:00.586 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd23806f430 msgr2=0x7fd238071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd23806f430 0x7fd238071940 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fd23c0097a0 tx=0x7fd23c008040 comp rx=0 tx=0).stop 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d24a0 msgr2=0x7fd24c15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d24a0 0x7fd24c15b080 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd24000b440 tx=0x7fd24000b900 comp rx=0 tx=0).stop 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 shutdown_connections 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd23806f430 0x7fd238071940 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd24c0d4820 0x7fd24c14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd24c0d24a0 0x7fd24c15b080 secure :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fd24000b440 tx=0x7fd24000b900 comp rx=0 tx=0).stop 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 --2- 172.21.15.80:0/2353600499 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd24c0d0ed0 0x7fd24c14c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:00.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.590+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 >> 172.21.15.80:0/2353600499 conn(0x7fd24c06c5f0 msgr2=0x7fd24c0bb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:00.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.594+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 shutdown_connections 2024-04-25T13:47:00.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.594+0000 7fd253034640 1 -- 172.21.15.80:0/2353600499 wait complete. 2024-04-25T13:47:00.617 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:47:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.770+0000 7f22aa4be640 1 Processor -- start 2024-04-25T13:47:00.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.770+0000 7f22aa4be640 1 -- start start 2024-04-25T13:47:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22aa4be640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22a406cf70 0x7f22a40cfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22aa4be640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 0x7f22a40ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22aa4be640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22a406d560 con 0x7f22a406cb60 2024-04-25T13:47:00.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22aa4be640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f22a40d28d0 con 0x7f22a406cf70 2024-04-25T13:47:00.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22aa4be640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f22a40d2bb0 con 0x7f22a40ce0c0 2024-04-25T13:47:00.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a8a5b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 0x7f22a40ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a37fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22a406cf70 0x7f22a40cfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a37fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f22a406cf70 0x7f22a40cfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:44554/0 (socket says 172.21.15.80:44554) 2024-04-25T13:47:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a37fe640 1 -- 172.21.15.80:0/3786416483 learned_addr learned my addr 172.21.15.80:0/3786416483 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a3fff640 1 --1- 172.21.15.80:0/3786416483 >> v1:172.21.15.87:6789/0 conn(0x7f22a406cb60 0x7f22a40cf410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:34928/0 (socket says 172.21.15.80:34928) 2024-04-25T13:47:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a37fe640 1 -- 172.21.15.80:0/3786416483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 msgr2=0x7f22a40ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a37fe640 1 --2- 172.21.15.80:0/3786416483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 0x7f22a40ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a37fe640 1 -- 172.21.15.80:0/3786416483 >> v1:172.21.15.87:6789/0 conn(0x7f22a406cb60 legacy=0x7f22a40cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:00.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a37fe640 1 -- 172.21.15.80:0/3786416483 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a40b6240 con 0x7f22a406cf70 2024-04-25T13:47:00.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.774+0000 7f22a37fe640 1 --2- 172.21.15.80:0/3786416483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 0x7f22a40cfb20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2298009870 tx=0x7f229801c560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f39a93c4044fceab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:00.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22a2ffd640 1 -- 172.21.15.80:0/3786416483 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980b2070 con 0x7f22a406cf70 2024-04-25T13:47:00.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22a2ffd640 1 -- 172.21.15.80:0/3786416483 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22980a9070 con 0x7f22a406cf70 2024-04-25T13:47:00.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22a2ffd640 1 -- 172.21.15.80:0/3786416483 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980ad880 con 0x7f22a406cf70 2024-04-25T13:47:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 -- 172.21.15.80:0/3786416483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 msgr2=0x7f22a40cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3786416483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 0x7f22a40cfb20 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2298009870 tx=0x7f229801c560 comp rx=0 tx=0).stop 2024-04-25T13:47:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 -- 172.21.15.80:0/3786416483 shutdown_connections 2024-04-25T13:47:00.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3786416483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 0x7f22a40ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:00.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3786416483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 0x7f22a40cfb20 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:00.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 -- 172.21.15.80:0/3786416483 >> 172.21.15.80:0/3786416483 conn(0x7f22a40b5580 msgr2=0x7f22a40b5990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:00.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 -- 172.21.15.80:0/3786416483 shutdown_connections 2024-04-25T13:47:00.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 -- 172.21.15.80:0/3786416483 wait complete. 2024-04-25T13:47:00.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 Processor -- start 2024-04-25T13:47:00.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.778+0000 7f22aa4be640 1 -- start start 2024-04-25T13:47:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a406cb60 0x7f22a4154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 0x7f22a40d3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 0x7f22a40d3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:00.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40cf030 con 0x7f22a406cb60 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40b4df0 con 0x7f22a406cf70 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f22a40d32e0 con 0x7f22a40ce0c0 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a406cb60 0x7f22a4154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a406cb60 0x7f22a4154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38696/0 (socket says 172.21.15.80:38696) 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 -- 172.21.15.80:0/3295390298 learned_addr learned my addr 172.21.15.80:0/3295390298 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a8a5b640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 0x7f22a40d3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a37fe640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 0x7f22a40d3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 -- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 msgr2=0x7f22a40d3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 0x7f22a40d3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 -- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 msgr2=0x7f22a40d3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 0x7f22a40d3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:00.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 -- 172.21.15.80:0/3295390298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a40d4470 con 0x7f22a406cb60 2024-04-25T13:47:00.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a3fff640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a406cb60 0x7f22a4154350 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f229400c3c0 tx=0x7f229400c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:00.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a0ff9640 1 -- 172.21.15.80:0/3295390298 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294017070 con 0x7f22a406cb60 2024-04-25T13:47:00.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a0ff9640 1 -- 172.21.15.80:0/3295390298 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2294005c80 con 0x7f22a406cb60 2024-04-25T13:47:00.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22a0ff9640 1 -- 172.21.15.80:0/3295390298 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22940072d0 con 0x7f22a406cb60 2024-04-25T13:47:00.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a41555f0 con 0x7f22a406cb60 2024-04-25T13:47:00.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a4155bb0 con 0x7f22a406cb60 2024-04-25T13:47:00.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.782+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22a406a440 con 0x7f22a406cb60 2024-04-25T13:47:00.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.786+0000 7f22a0ff9640 1 -- 172.21.15.80:0/3295390298 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f229401a070 con 0x7f22a406cb60 2024-04-25T13:47:00.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.786+0000 7f22a0ff9640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f229006f430 0x7f2290071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:00.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.786+0000 7f22a0ff9640 1 -- 172.21.15.80:0/3295390298 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5357+0+0 (secure 0 0 0) 0x7f229408e9b0 con 0x7f22a406cb60 2024-04-25T13:47:00.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.786+0000 7f22a37fe640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f229006f430 0x7f2290071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:00.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.786+0000 7f22a37fe640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f229006f430 0x7f2290071940 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f22980af000 tx=0x7f22980a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:00.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.790+0000 7f22a0ff9640 1 -- 172.21.15.80:0/3295390298 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f22a40d4470 con 0x7f22a406cb60 2024-04-25T13:47:00.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:00.950+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f22a415d300 con 0x7f22a406cb60 2024-04-25T13:47:01.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.590+0000 7f22a0ff9640 1 -- 172.21.15.80:0/3295390298 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v128) ==== 169+0+0 (secure 0 0 0) 0x7f229405b520 con 0x7f22a406cb60 2024-04-25T13:47:01.589 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f229006f430 msgr2=0x7f2290071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f229006f430 0x7f2290071940 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f22980af000 tx=0x7f22980a7000 comp rx=0 tx=0).stop 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a406cb60 msgr2=0x7f22a4154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a406cb60 0x7f22a4154350 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f229400c3c0 tx=0x7f229400c7c0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 shutdown_connections 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f229006f430 0x7f2290071940 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f22a40ce0c0 0x7f22a40d3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f22a406cf70 0x7f22a40d3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 --2- 172.21.15.80:0/3295390298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f22a406cb60 0x7f22a4154350 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 >> 172.21.15.80:0/3295390298 conn(0x7f22a40b5580 msgr2=0x7f22a40cf1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:01.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 shutdown_connections 2024-04-25T13:47:01.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.602+0000 7f22aa4be640 1 -- 172.21.15.80:0/3295390298 wait complete. 2024-04-25T13:47:01.627 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:47:01.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.774+0000 7fc19206a640 1 Processor -- start 2024-04-25T13:47:01.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.774+0000 7fc19206a640 1 -- start start 2024-04-25T13:47:01.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc19206a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc18c068590 0x7fc18c0689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:01.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc19206a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc18c068ee0 con 0x7fc18c06f060 2024-04-25T13:47:01.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc19206a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc18c06cb60 con 0x7fc18c0b6190 2024-04-25T13:47:01.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc19206a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc18c06ebd0 con 0x7fc18c068590 2024-04-25T13:47:01.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc18c06f060 0x7fc18c06f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52620/0 (socket says 172.21.15.80:52620) 2024-04-25T13:47:01.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18bfff640 1 -- 172.21.15.80:0/2586042139 learned_addr learned my addr 172.21.15.80:0/2586042139 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:01.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3403614826 0 0) 0x7fc1840014a0 con 0x7fc18c06f060 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18affd640 1 --2- 172.21.15.80:0/2586042139 >> v2:172.21.15.87:3300/0 conn(0x7fc18c068590 0x7fc18c0689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc174003530 con 0x7fc18c06f060 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1164060877 0 0) 0x7fc174003530 con 0x7fc18c06f060 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 >> v2:172.21.15.87:3300/0 conn(0x7fc18c068590 msgr2=0x7fc18c0689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 --2- 172.21.15.80:0/2586042139 >> v2:172.21.15.87:3300/0 conn(0x7fc18c068590 0x7fc18c0689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 >> v1:172.21.15.87:6789/0 conn(0x7fc18c0b6190 legacy=0x7fc18c149600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c06ce40 con 0x7fc18c06f060 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc184001bd0 con 0x7fc18c06f060 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 >> v1:172.21.15.80:6789/0 conn(0x7fc18c06f060 legacy=0x7fc18c06f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1740039c0 0x7fc174003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc174004810 0x7fc174004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc174005280 0x7fc17400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c06ebd0 con 0x7fc174005280 2024-04-25T13:47:01.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc17400c210 con 0x7fc1740039c0 2024-04-25T13:47:01.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc17400c410 con 0x7fc174004810 2024-04-25T13:47:01.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18bfff640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc174005280 0x7fc17400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:01.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18affd640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc174004810 0x7fc174004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:01.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18bfff640 1 -- 172.21.15.80:0/2586042139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc174004810 msgr2=0x7fc174004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:01.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18bfff640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc174004810 0x7fc174004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18bfff640 1 -- 172.21.15.80:0/2586042139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1740039c0 msgr2=0x7fc174003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:01.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18bfff640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1740039c0 0x7fc174003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.778+0000 7fc18bfff640 1 -- 172.21.15.80:0/2586042139 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc174003530 con 0x7fc174005280 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc18bfff640 1 -- 172.21.15.80:0/2586042139 reap_dead start 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc18bfff640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc174005280 0x7fc17400b9d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc18400a9f0 tx=0x7fc18401d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6bb684338c0577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18400a640 con 0x7fc174005280 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1840ac640 con 0x7fc174005280 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc18a7fc640 1 -- 172.21.15.80:0/2586042139 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1840aca30 con 0x7fc174005280 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 -- 172.21.15.80:0/2586042139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc174005280 msgr2=0x7fc17400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc174005280 0x7fc17400b9d0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fc18400a9f0 tx=0x7fc18401d340 comp rx=0 tx=0).stop 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 -- 172.21.15.80:0/2586042139 shutdown_connections 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 --2- 172.21.15.80:0/2586042139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc174005280 0x7fc17400b9d0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 -- 172.21.15.80:0/2586042139 >> 172.21.15.80:0/2586042139 conn(0x7fc18c0d2b20 msgr2=0x7fc18c0d2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 -- 172.21.15.80:0/2586042139 shutdown_connections 2024-04-25T13:47:01.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 -- 172.21.15.80:0/2586042139 wait complete. 2024-04-25T13:47:01.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 Processor -- start 2024-04-25T13:47:01.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.782+0000 7fc19206a640 1 -- start start 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc174005280 0x7fc18c1fa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc18c13b470 0x7fc18c0d1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc18c0d1a70 0x7fc18c160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c0b65a0 con 0x7fc18c0d1a70 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c06d400 con 0x7fc174005280 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc18c0689a0 con 0x7fc18c13b470 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc18c0d1a70 0x7fc18c160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc18c0d1a70 0x7fc18c160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38010/0 (socket says 172.21.15.80:38010) 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 -- 172.21.15.80:0/2968277539 learned_addr learned my addr 172.21.15.80:0/2968277539 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18affd640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc18c13b470 0x7fc18c0d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18b7fe640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc174005280 0x7fc18c1fa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 -- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc18c13b470 msgr2=0x7fc18c0d1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc18c13b470 0x7fc18c0d1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 -- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc174005280 msgr2=0x7fc18c1fa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc174005280 0x7fc18c1fa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 -- 172.21.15.80:0/2968277539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c1fa9a0 con 0x7fc18c0d1a70 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc18bfff640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc18c0d1a70 0x7fc18c160550 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc18401d170 tx=0x7fc174005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc173fff640 1 -- 172.21.15.80:0/2968277539 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18400a030 con 0x7fc18c0d1a70 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc173fff640 1 -- 172.21.15.80:0/2968277539 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1840014a0 con 0x7fc18c0d1a70 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18c0d1e80 con 0x7fc18c0d1a70 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc173fff640 1 -- 172.21.15.80:0/2968277539 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc184001bd0 con 0x7fc18c0d1a70 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18c162310 con 0x7fc18c0d1a70 2024-04-25T13:47:01.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18c06a3a0 con 0x7fc18c0d1a70 2024-04-25T13:47:01.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc173fff640 1 -- 172.21.15.80:0/2968277539 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc1840b5070 con 0x7fc18c0d1a70 2024-04-25T13:47:01.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.786+0000 7fc173fff640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc17806f5d0 0x7fc178071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:01.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.790+0000 7fc173fff640 1 -- 172.21.15.80:0/2968277539 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4943+0+0 (secure 0 0 0) 0x7fc184128040 con 0x7fc18c0d1a70 2024-04-25T13:47:01.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.790+0000 7fc18b7fe640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc17806f5d0 0x7fc178071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:01.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.790+0000 7fc18b7fe640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc17806f5d0 0x7fc178071ae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc17c001670 tx=0x7fc17c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:01.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.794+0000 7fc173fff640 1 -- 172.21.15.80:0/2968277539 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc1840bd020 con 0x7fc18c0d1a70 2024-04-25T13:47:01.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.954+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fc18c0d2f30 con 0x7fc18c0d1a70 2024-04-25T13:47:01.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc173fff640 1 -- 172.21.15.80:0/2968277539 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fc18c0d1e80 con 0x7fc18c0d1a70 2024-04-25T13:47:01.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc17806f5d0 msgr2=0x7fc178071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc17806f5d0 0x7fc178071ae0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc17c001670 tx=0x7fc17c009040 comp rx=0 tx=0).stop 2024-04-25T13:47:01.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc18c0d1a70 msgr2=0x7fc18c160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:01.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc18c0d1a70 0x7fc18c160550 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fc18401d170 tx=0x7fc174005840 comp rx=0 tx=0).stop 2024-04-25T13:47:01.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 shutdown_connections 2024-04-25T13:47:01.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc17806f5d0 0x7fc178071ae0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc18c0d1a70 0x7fc18c160550 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc18c13b470 0x7fc18c0d1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 --2- 172.21.15.80:0/2968277539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc174005280 0x7fc18c1fa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:01.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 >> 172.21.15.80:0/2968277539 conn(0x7fc18c0d2b20 msgr2=0x7fc18c070300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:01.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 shutdown_connections 2024-04-25T13:47:01.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:01.962+0000 7fc19206a640 1 -- 172.21.15.80:0/2968277539 wait complete. 2024-04-25T13:47:01.983 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2024-04-25T13:47:02.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86c2906640 1 Processor -- start 2024-04-25T13:47:02.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86c2906640 1 -- start start 2024-04-25T13:47:02.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86c2906640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f86bc0d06e0 0x7f86bc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86c2906640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86bc0d0af0 con 0x7f86bc161f90 2024-04-25T13:47:02.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86c2906640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f86bc0d0d70 con 0x7f86bc06e9e0 2024-04-25T13:47:02.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86c2906640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f86bc0d1050 con 0x7f86bc0d06e0 2024-04-25T13:47:02.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86bb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f86bc0d06e0 0x7f86bc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86bb7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f86bc0d06e0 0x7f86bc06e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38018/0 (socket says 172.21.15.80:38018) 2024-04-25T13:47:02.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86bbfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f86bc161f90 0x7f86bc0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52634/0 (socket says 172.21.15.80:52634) 2024-04-25T13:47:02.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86bb7fe640 1 -- 172.21.15.80:0/1915900173 learned_addr learned my addr 172.21.15.80:0/1915900173 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:02.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86bb7fe640 1 -- 172.21.15.80:0/1915900173 >> v1:172.21.15.87:6789/0 conn(0x7f86bc06e9e0 legacy=0x7f86bc06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86bb7fe640 1 -- 172.21.15.80:0/1915900173 >> v1:172.21.15.80:6789/0 conn(0x7f86bc161f90 legacy=0x7f86bc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86bb7fe640 1 -- 172.21.15.80:0/1915900173 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc06dca0 con 0x7f86bc0d06e0 2024-04-25T13:47:02.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.138+0000 7f86baffd640 1 -- 172.21.15.80:0/1915900173 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 736002142 0 0) 0x7f86ac001120 con 0x7f86bc161f90 2024-04-25T13:47:02.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86bb7fe640 1 --2- 172.21.15.80:0/1915900173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc0d06e0 0x7f86bc06e4a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009870 tx=0x7f86b001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736929092c67363a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:02.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86baffd640 1 -- 172.21.15.80:0/1915900173 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b00b1070 con 0x7f86bc0d06e0 2024-04-25T13:47:02.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86baffd640 1 -- 172.21.15.80:0/1915900173 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86b00ac630 con 0x7f86bc0d06e0 2024-04-25T13:47:02.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86baffd640 1 -- 172.21.15.80:0/1915900173 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b00a8050 con 0x7f86bc0d06e0 2024-04-25T13:47:02.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86c2906640 1 -- 172.21.15.80:0/1915900173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc0d06e0 msgr2=0x7f86bc06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86c2906640 1 --2- 172.21.15.80:0/1915900173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc0d06e0 0x7f86bc06e4a0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009870 tx=0x7f86b001c410 comp rx=0 tx=0).stop 2024-04-25T13:47:02.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86c2906640 1 -- 172.21.15.80:0/1915900173 shutdown_connections 2024-04-25T13:47:02.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86c2906640 1 --2- 172.21.15.80:0/1915900173 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc0d06e0 0x7f86bc06e4a0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86c2906640 1 -- 172.21.15.80:0/1915900173 >> 172.21.15.80:0/1915900173 conn(0x7f86bc06d080 msgr2=0x7f86bc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:02.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86c2906640 1 -- 172.21.15.80:0/1915900173 shutdown_connections 2024-04-25T13:47:02.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.142+0000 7f86c2906640 1 -- 172.21.15.80:0/1915900173 wait complete. 2024-04-25T13:47:02.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 Processor -- start 2024-04-25T13:47:02.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 -- start start 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc06e9e0 0x7f86bc142d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f86bc0d06e0 0x7f86bc143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86bc161f90 0x7f86bc140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc06f680 con 0x7f86bc06e9e0 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc06f500 con 0x7f86bc161f90 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f86bc144620 con 0x7f86bc0d06e0 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc06e9e0 0x7f86bc142d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc06e9e0 0x7f86bc142d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38034/0 (socket says 172.21.15.80:38034) 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 -- 172.21.15.80:0/384625203 learned_addr learned my addr 172.21.15.80:0/384625203 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c0ea3640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86bc161f90 0x7f86bc140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bb7fe640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f86bc0d06e0 0x7f86bc143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 -- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f86bc0d06e0 msgr2=0x7f86bc143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f86bc0d06e0 0x7f86bc143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 -- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86bc161f90 msgr2=0x7f86bc140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86bc161f90 0x7f86bc140010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 -- 172.21.15.80:0/384625203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc1449c0 con 0x7f86bc06e9e0 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86bbfff640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc06e9e0 0x7f86bc142d00 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c410 tx=0x7f86ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86b8ff9640 1 -- 172.21.15.80:0/384625203 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0074e0 con 0x7f86bc06e9e0 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86b8ff9640 1 -- 172.21.15.80:0/384625203 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f86ac0047f0 con 0x7f86bc06e9e0 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc144c60 con 0x7f86bc06e9e0 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86b8ff9640 1 -- 172.21.15.80:0/384625203 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac007680 con 0x7f86bc06e9e0 2024-04-25T13:47:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.146+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc145220 con 0x7f86bc06e9e0 2024-04-25T13:47:02.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.150+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86bc06a3a0 con 0x7f86bc06e9e0 2024-04-25T13:47:02.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.150+0000 7f86b8ff9640 1 -- 172.21.15.80:0/384625203 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f86ac011070 con 0x7f86bc06e9e0 2024-04-25T13:47:02.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.150+0000 7f86b8ff9640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f86a806f400 0x7f86a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.150+0000 7f86b8ff9640 1 -- 172.21.15.80:0/384625203 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4943+0+0 (secure 0 0 0) 0x7f86ac08eb80 con 0x7f86bc06e9e0 2024-04-25T13:47:02.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.150+0000 7f86bb7fe640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f86a806f400 0x7f86a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.150+0000 7f86bb7fe640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f86a806f400 0x7f86a8071910 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f86b00ae000 tx=0x7f86b00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:02.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.154+0000 7f86b8ff9640 1 -- 172.21.15.80:0/384625203 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f86bc0d0af0 con 0x7f86bc06e9e0 2024-04-25T13:47:02.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.366+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) -- 0x7f86bc0b42d0 con 0x7f86bc06e9e0 2024-04-25T13:47:02.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.602+0000 7f86b8ff9640 1 -- 172.21.15.80:0/384625203 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v351) ==== 128+0+0 (secure 0 0 0) 0x7f86ac05bd70 con 0x7f86bc06e9e0 2024-04-25T13:47:02.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f86a806f400 msgr2=0x7f86a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f86a806f400 0x7f86a8071910 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f86b00ae000 tx=0x7f86b00a7000 comp rx=0 tx=0).stop 2024-04-25T13:47:02.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc06e9e0 msgr2=0x7f86bc142d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc06e9e0 0x7f86bc142d00 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c410 tx=0x7f86ac00c8d0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 shutdown_connections 2024-04-25T13:47:02.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f86a806f400 0x7f86a8071910 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f86bc161f90 0x7f86bc140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f86bc0d06e0 0x7f86bc143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 --2- 172.21.15.80:0/384625203 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f86bc06e9e0 0x7f86bc142d00 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 >> 172.21.15.80:0/384625203 conn(0x7f86bc06d080 msgr2=0x7f86bc0b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:02.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 shutdown_connections 2024-04-25T13:47:02.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.606+0000 7f86c2906640 1 -- 172.21.15.80:0/384625203 wait complete. 2024-04-25T13:47:02.627 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2024-04-25T13:47:02.628 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2024-04-25T13:47:02.628 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:02.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.778+0000 7fb5d6d1b640 1 Processor -- start 2024-04-25T13:47:02.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.778+0000 7fb5d6d1b640 1 -- start start 2024-04-25T13:47:02.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d6d1b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb5d006f5a0 0x7fb5d0160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d6d1b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d006fcf0 con 0x7fb5d00733b0 2024-04-25T13:47:02.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d6d1b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5d00b59f0 con 0x7fb5d0161070 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d6d1b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb5d0073f10 con 0x7fb5d006f5a0 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d4ab7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb5d006f5a0 0x7fb5d0160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d52b8640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb5d00733b0 0x7fb5d00b52e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52644/0 (socket says 172.21.15.80:52644) 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d4ab7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb5d006f5a0 0x7fb5d0160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38038/0 (socket says 172.21.15.80:38038) 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d52b8640 1 -- 172.21.15.80:0/2710769781 learned_addr learned my addr 172.21.15.80:0/2710769781 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d4ab7640 1 -- 172.21.15.80:0/2710769781 >> v1:172.21.15.87:6789/0 conn(0x7fb5d0161070 legacy=0x7fb5d00b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d4ab7640 1 -- 172.21.15.80:0/2710769781 >> v1:172.21.15.80:6789/0 conn(0x7fb5d00733b0 legacy=0x7fb5d00b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5d4ab7640 1 -- 172.21.15.80:0/2710769781 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d00b0c60 con 0x7fb5d006f5a0 2024-04-25T13:47:02.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.782+0000 7fb5cf7fe640 1 -- 172.21.15.80:0/2710769781 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2010707914 0 0) 0x7fb5d006fcf0 con 0x7fb5d00733b0 2024-04-25T13:47:02.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d4ab7640 1 --2- 172.21.15.80:0/2710769781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d006f5a0 0x7fb5d0160b30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009d90 tx=0x7fb5c001c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877428e598839953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:02.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5cf7fe640 1 -- 172.21.15.80:0/2710769781 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00b2070 con 0x7fb5d006f5a0 2024-04-25T13:47:02.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5cf7fe640 1 -- 172.21.15.80:0/2710769781 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5c00a5850 con 0x7fb5d006f5a0 2024-04-25T13:47:02.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5cf7fe640 1 -- 172.21.15.80:0/2710769781 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c00a5c20 con 0x7fb5d006f5a0 2024-04-25T13:47:02.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/2710769781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d006f5a0 msgr2=0x7fb5d0160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 --2- 172.21.15.80:0/2710769781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d006f5a0 0x7fb5d0160b30 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fb5c0009d90 tx=0x7fb5c001c8a0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/2710769781 shutdown_connections 2024-04-25T13:47:02.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 --2- 172.21.15.80:0/2710769781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d006f5a0 0x7fb5d0160b30 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/2710769781 >> 172.21.15.80:0/2710769781 conn(0x7fb5d006d080 msgr2=0x7fb5d00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:02.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/2710769781 shutdown_connections 2024-04-25T13:47:02.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/2710769781 wait complete. 2024-04-25T13:47:02.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 Processor -- start 2024-04-25T13:47:02.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.786+0000 7fb5d6d1b640 1 -- start start 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5d006f5a0 0x7fb5d0144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d00733b0 0x7fb5d01466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d0161070 0x7fb5d006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d016b6f0 con 0x7fb5d00733b0 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d0161890 con 0x7fb5d0161070 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d016b870 con 0x7fb5d006f5a0 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d00733b0 0x7fb5d01466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d00733b0 0x7fb5d01466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38052/0 (socket says 172.21.15.80:38052) 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 -- 172.21.15.80:0/409289239 learned_addr learned my addr 172.21.15.80:0/409289239 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d52b8640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d0161070 0x7fb5d006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d4ab7640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5d006f5a0 0x7fb5d0144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 -- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5d006f5a0 msgr2=0x7fb5d0144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5d006f5a0 0x7fb5d0144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 -- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d0161070 msgr2=0x7fb5d006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d0161070 0x7fb5d006e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 -- 172.21.15.80:0/409289239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d014a360 con 0x7fb5d00733b0 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d4ab7640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5d006f5a0 0x7fb5d0144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cffff640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d00733b0 0x7fb5d01466e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b440 tx=0x7fb5b800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cd7fa640 1 -- 172.21.15.80:0/409289239 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8015020 con 0x7fb5d00733b0 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cd7fa640 1 -- 172.21.15.80:0/409289239 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5b8011890 con 0x7fb5d00733b0 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5d014a640 con 0x7fb5d00733b0 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cd7fa640 1 -- 172.21.15.80:0/409289239 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5b8011c80 con 0x7fb5d00733b0 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5d014ac00 con 0x7fb5d00733b0 2024-04-25T13:47:02.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5d006a3a0 con 0x7fb5d00733b0 2024-04-25T13:47:02.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.790+0000 7fb5cd7fa640 1 -- 172.21.15.80:0/409289239 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5b8018020 con 0x7fb5d00733b0 2024-04-25T13:47:02.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.794+0000 7fb5cd7fa640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5b006f430 0x7fb5b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:02.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.794+0000 7fb5cd7fa640 1 -- 172.21.15.80:0/409289239 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4943+0+0 (secure 0 0 0) 0x7fb5b808f1e0 con 0x7fb5d00733b0 2024-04-25T13:47:02.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.794+0000 7fb5d4ab7640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5b006f430 0x7fb5b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:02.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.794+0000 7fb5d4ab7640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5b006f430 0x7fb5b0071940 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00af000 tx=0x7fb5c00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:02.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.798+0000 7fb5cd7fa640 1 -- 172.21.15.80:0/409289239 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb5b801f030 con 0x7fb5d00733b0 2024-04-25T13:47:02.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.958+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb5d00739c0 con 0x7fb5d00733b0 2024-04-25T13:47:02.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.958+0000 7fb5cd7fa640 1 -- 172.21.15.80:0/409289239 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+12959 (secure 0 0 0) 0x7fb5b800d040 con 0x7fb5d00733b0 2024-04-25T13:47:02.959 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:02.959 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":128,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:01.590778+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/2773338889":"2024-04-26T13:46:58.435603+0000","172.21.15.80:6834/2773338889":"2024-04-26T13:46:58.435603+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:02.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5b006f430 msgr2=0x7fb5b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5b006f430 0x7fb5b0071940 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fb5c00af000 tx=0x7fb5c00a7070 comp rx=0 tx=0).stop 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d00733b0 msgr2=0x7fb5d01466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d00733b0 0x7fb5d01466e0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb5b800b440 tx=0x7fb5b800b900 comp rx=0 tx=0).stop 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 shutdown_connections 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5b006f430 0x7fb5b0071940 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5d0161070 0x7fb5d006e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5d00733b0 0x7fb5d01466e0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 --2- 172.21.15.80:0/409289239 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5d006f5a0 0x7fb5d0144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 >> 172.21.15.80:0/409289239 conn(0x7fb5d006d080 msgr2=0x7fb5d0154e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 shutdown_connections 2024-04-25T13:47:02.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:02.962+0000 7fb5d6d1b640 1 -- 172.21.15.80:0/409289239 wait complete. 2024-04-25T13:47:02.983 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:47:02.983 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:47:03.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.130+0000 7f9711285640 1 Processor -- start 2024-04-25T13:47:03.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f9711285640 1 -- start start 2024-04-25T13:47:03.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f9711285640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f970c164ad0 0x7f970c0b77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f9711285640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c0b7d00 0x7f970c0d2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f9711285640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f970c0d4380 con 0x7f970c0d2990 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f9711285640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f970c0b8110 con 0x7f970c164ad0 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f9711285640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f970c164640 con 0x7f970c0b7d00 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f970affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f970c164ad0 0x7f970c0b77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f970affd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f970c164ad0 0x7f970c0b77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38066/0 (socket says 172.21.15.80:38066) 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f970affd640 1 -- 172.21.15.80:0/3938803867 learned_addr learned my addr 172.21.15.80:0/3938803867 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f970a7fc640 1 --2- 172.21.15.80:0/3938803867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c0b7d00 0x7f970c0d2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f970affd640 1 -- 172.21.15.80:0/3938803867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c0b7d00 msgr2=0x7f970c0d2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f970affd640 1 --2- 172.21.15.80:0/3938803867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c0b7d00 0x7f970c0d2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f970affd640 1 -- 172.21.15.80:0/3938803867 >> v1:172.21.15.87:6789/0 conn(0x7f970c0d2990 legacy=0x7f970c160960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:03.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.134+0000 7f970affd640 1 -- 172.21.15.80:0/3938803867 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c0b8610 con 0x7f970c164ad0 2024-04-25T13:47:03.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f970affd640 1 --2- 172.21.15.80:0/3938803867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c164ad0 0x7f970c0b77c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211ce981ddd96932 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:03.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9709ffb640 1 -- 172.21.15.80:0/3938803867 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f40b0070 con 0x7f970c164ad0 2024-04-25T13:47:03.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9709ffb640 1 -- 172.21.15.80:0/3938803867 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96f40ab600 con 0x7f970c164ad0 2024-04-25T13:47:03.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9709ffb640 1 -- 172.21.15.80:0/3938803867 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f40ab9d0 con 0x7f970c164ad0 2024-04-25T13:47:03.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 -- 172.21.15.80:0/3938803867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c164ad0 msgr2=0x7f970c0b77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:03.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 --2- 172.21.15.80:0/3938803867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c164ad0 0x7f970c0b77c0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f96f4009870 tx=0x7f96f401c490 comp rx=0 tx=0).stop 2024-04-25T13:47:03.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 -- 172.21.15.80:0/3938803867 shutdown_connections 2024-04-25T13:47:03.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 --2- 172.21.15.80:0/3938803867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c0b7d00 0x7f970c0d2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 --2- 172.21.15.80:0/3938803867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c164ad0 0x7f970c0b77c0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 -- 172.21.15.80:0/3938803867 >> 172.21.15.80:0/3938803867 conn(0x7f970c073570 msgr2=0x7f970c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:03.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 -- 172.21.15.80:0/3938803867 shutdown_connections 2024-04-25T13:47:03.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 -- 172.21.15.80:0/3938803867 wait complete. 2024-04-25T13:47:03.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 Processor -- start 2024-04-25T13:47:03.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.138+0000 7f9711285640 1 -- start start 2024-04-25T13:47:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f970c0b7d00 0x7f970c0ccdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c0d2990 0x7f970c0cb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c14b730 0x7f970c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f970c16d0e0 con 0x7f970c0d2990 2024-04-25T13:47:03.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f970c0d2da0 con 0x7f970c0b7d00 2024-04-25T13:47:03.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f970c16d260 con 0x7f970c14b730 2024-04-25T13:47:03.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c0d2990 0x7f970c0cb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c0d2990 0x7f970c0cb580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38068/0 (socket says 172.21.15.80:38068) 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 -- 172.21.15.80:0/1928209785 learned_addr learned my addr 172.21.15.80:0/1928209785 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970b7fe640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c14b730 0x7f970c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970affd640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f970c0b7d00 0x7f970c0ccdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 -- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c14b730 msgr2=0x7f970c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c14b730 0x7f970c15a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 -- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f970c0b7d00 msgr2=0x7f970c0ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f970c0b7d00 0x7f970c0ccdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 -- 172.21.15.80:0/1928209785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f970c0cbac0 con 0x7f970c0d2990 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f970a7fc640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c0d2990 0x7f970c0cb580 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9704007e70 tx=0x7f970400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f97037fe640 1 -- 172.21.15.80:0/1928209785 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704014620 con 0x7f970c0d2990 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f97037fe640 1 -- 172.21.15.80:0/1928209785 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9704019070 con 0x7f970c0d2990 2024-04-25T13:47:03.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f970c161cc0 con 0x7f970c0d2990 2024-04-25T13:47:03.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f97037fe640 1 -- 172.21.15.80:0/1928209785 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9704014a00 con 0x7f970c0d2990 2024-04-25T13:47:03.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f970c162280 con 0x7f970c0d2990 2024-04-25T13:47:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f970c06a3a0 con 0x7f970c0d2990 2024-04-25T13:47:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.142+0000 7f97037fe640 1 -- 172.21.15.80:0/1928209785 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9704014ba0 con 0x7f970c0d2990 2024-04-25T13:47:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.146+0000 7f97037fe640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96ec06f400 0x7f96ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.146+0000 7f97037fe640 1 -- 172.21.15.80:0/1928209785 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4943+0+0 (secure 0 0 0) 0x7f970408f6d0 con 0x7f970c0d2990 2024-04-25T13:47:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.146+0000 7f970affd640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96ec06f400 0x7f96ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.146+0000 7f970affd640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96ec06f400 0x7f96ec071910 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f970c0d4990 tx=0x7f96f40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:03.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.150+0000 7f97037fe640 1 -- 172.21.15.80:0/1928209785 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f970405c300 con 0x7f970c0d2990 2024-04-25T13:47:03.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.310+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f970c0d0c10 con 0x7f970c0d2990 2024-04-25T13:47:03.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.310+0000 7f97037fe640 1 -- 172.21.15.80:0/1928209785 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v128) ==== 93+0+10 (secure 0 0 0) 0x7f9704060600 con 0x7f970c0d2990 2024-04-25T13:47:03.310 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:47:03.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96ec06f400 msgr2=0x7f96ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:03.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96ec06f400 0x7f96ec071910 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f970c0d4990 tx=0x7f96f40a6000 comp rx=0 tx=0).stop 2024-04-25T13:47:03.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c0d2990 msgr2=0x7f970c0cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:03.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c0d2990 0x7f970c0cb580 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f9704007e70 tx=0x7f970400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 shutdown_connections 2024-04-25T13:47:03.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f96ec06f400 0x7f96ec071910 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f970c14b730 0x7f970c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f970c0d2990 0x7f970c0cb580 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 --2- 172.21.15.80:0/1928209785 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f970c0b7d00 0x7f970c0ccdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 >> 172.21.15.80:0/1928209785 conn(0x7f970c073570 msgr2=0x7f970c15d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:03.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 shutdown_connections 2024-04-25T13:47:03.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.314+0000 7f9711285640 1 -- 172.21.15.80:0/1928209785 wait complete. 2024-04-25T13:47:03.333 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-04-25T13:47:03.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda3c87640 1 Processor -- start 2024-04-25T13:47:03.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda3c87640 1 -- start start 2024-04-25T13:47:03.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda3c87640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbd9c0d2e80 0x7fbd9c1607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda3c87640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbd9c0d59b0 0x7fbd9c0d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda3c87640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 0x7fbd9c0d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda3c87640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c0d6300 con 0x7fbd9c0d2e80 2024-04-25T13:47:03.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda3c87640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9c161060 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda3c87640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c161340 con 0x7fbd9c0d4330 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1a23640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbd9c0d2e80 0x7fbd9c1607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1a23640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbd9c0d2e80 0x7fbd9c1607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38080/0 (socket says 172.21.15.80:38080) 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1a23640 1 -- 172.21.15.80:0/1261499568 learned_addr learned my addr 172.21.15.80:0/1261499568 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda2224640 1 --2- 172.21.15.80:0/1261499568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 0x7fbd9c0d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1222640 1 --2- 172.21.15.80:0/1261499568 >> v2:172.21.15.87:3300/0 conn(0x7fbd9c0d59b0 0x7fbd9c0d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1a23640 1 -- 172.21.15.80:0/1261499568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 msgr2=0x7fbd9c0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1a23640 1 --2- 172.21.15.80:0/1261499568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 0x7fbd9c0d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1a23640 1 -- 172.21.15.80:0/1261499568 >> v2:172.21.15.87:3300/0 conn(0x7fbd9c0d59b0 msgr2=0x7fbd9c0d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1a23640 1 --2- 172.21.15.80:0/1261499568 >> v2:172.21.15.87:3300/0 conn(0x7fbd9c0d59b0 0x7fbd9c0d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.486+0000 7fbda1a23640 1 -- 172.21.15.80:0/1261499568 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c0d2680 con 0x7fbd9c0d2e80 2024-04-25T13:47:03.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda1a23640 1 --2- 172.21.15.80:0/1261499568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d2e80 0x7fbd9c1607a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0c9670 tx=0x7fbd9801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df14b08c125bc61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:03.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda0a21640 1 -- 172.21.15.80:0/1261499568 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd980ab640 con 0x7fbd9c0d2e80 2024-04-25T13:47:03.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda0a21640 1 -- 172.21.15.80:0/1261499568 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd980b0070 con 0x7fbd9c0d2e80 2024-04-25T13:47:03.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda0a21640 1 -- 172.21.15.80:0/1261499568 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd980aba00 con 0x7fbd9c0d2e80 2024-04-25T13:47:03.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 -- 172.21.15.80:0/1261499568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d2e80 msgr2=0x7fbd9c1607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:03.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 --2- 172.21.15.80:0/1261499568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d2e80 0x7fbd9c1607a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c0c9670 tx=0x7fbd9801c460 comp rx=0 tx=0).stop 2024-04-25T13:47:03.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 -- 172.21.15.80:0/1261499568 shutdown_connections 2024-04-25T13:47:03.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 --2- 172.21.15.80:0/1261499568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 0x7fbd9c0d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 --2- 172.21.15.80:0/1261499568 >> v2:172.21.15.87:3300/0 conn(0x7fbd9c0d59b0 0x7fbd9c0d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 --2- 172.21.15.80:0/1261499568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d2e80 0x7fbd9c1607a0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 -- 172.21.15.80:0/1261499568 >> 172.21.15.80:0/1261499568 conn(0x7fbd9c0737a0 msgr2=0x7fbd9c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:03.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 -- 172.21.15.80:0/1261499568 shutdown_connections 2024-04-25T13:47:03.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.490+0000 7fbda3c87640 1 -- 172.21.15.80:0/1261499568 wait complete. 2024-04-25T13:47:03.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 Processor -- start 2024-04-25T13:47:03.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 -- start start 2024-04-25T13:47:03.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d2e80 0x7fbd9c156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 0x7fbd9c15c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d59b0 0x7fbd9c15c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c0d4e00 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9c0d4c80 con 0x7fbd9c0d2e80 2024-04-25T13:47:03.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbd9c0d3290 con 0x7fbd9c0d4330 2024-04-25T13:47:03.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d59b0 0x7fbd9c15c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d59b0 0x7fbd9c15c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38094/0 (socket says 172.21.15.80:38094) 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 -- 172.21.15.80:0/2562925946 learned_addr learned my addr 172.21.15.80:0/2562925946 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda1a23640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d2e80 0x7fbd9c156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda1222640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 0x7fbd9c15c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 -- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 msgr2=0x7fbd9c15c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 0x7fbd9c15c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 -- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d2e80 msgr2=0x7fbd9c156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d2e80 0x7fbd9c156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 -- 172.21.15.80:0/2562925946 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9c150c50 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda1a23640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d2e80 0x7fbd9c156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda2224640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d59b0 0x7fbd9c15c9c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009fd0 tx=0x7fbd9400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbd927fc640 1 -- 172.21.15.80:0/2562925946 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94014420 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbd927fc640 1 -- 172.21.15.80:0/2562925946 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd94019070 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd9c151a00 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbd927fc640 1 -- 172.21.15.80:0/2562925946 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94014800 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd9c157430 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.494+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd9c157690 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.498+0000 7fbd927fc640 1 -- 172.21.15.80:0/2562925946 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbd940149a0 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.498+0000 7fbd927fc640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbd8006f400 0x7fbd80071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:03.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.498+0000 7fbd927fc640 1 -- 172.21.15.80:0/2562925946 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4943+0+0 (secure 0 0 0) 0x7fbd94090290 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.498+0000 7fbda1a23640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbd8006f400 0x7fbd80071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:03.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.498+0000 7fbda1a23640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbd8006f400 0x7fbd80071910 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbd980af000 tx=0x7fbd98008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:03.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.502+0000 7fbd927fc640 1 -- 172.21.15.80:0/2562925946 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbd9405d560 con 0x7fbd9c0d59b0 2024-04-25T13:47:03.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:03.718+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) -- 0x7fbd9c15bde0 con 0x7fbd9c0d59b0 2024-04-25T13:47:04.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.614+0000 7fbd927fc640 1 -- 172.21.15.80:0/2562925946 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v353) ==== 143+0+0 (secure 0 0 0) 0x7fbd9405ce60 con 0x7fbd9c0d59b0 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbd8006f400 msgr2=0x7fbd80071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbd8006f400 0x7fbd80071910 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fbd980af000 tx=0x7fbd98008040 comp rx=0 tx=0).stop 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d59b0 msgr2=0x7fbd9c15c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d59b0 0x7fbd9c15c9c0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fbd94009fd0 tx=0x7fbd9400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 shutdown_connections 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbd8006f400 0x7fbd80071910 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbd9c0d59b0 0x7fbd9c15c9c0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbd9c0d4330 0x7fbd9c15c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 --2- 172.21.15.80:0/2562925946 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd9c0d2e80 0x7fbd9c156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 >> 172.21.15.80:0/2562925946 conn(0x7fbd9c0737a0 msgr2=0x7fbd9c0d1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:04.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 shutdown_connections 2024-04-25T13:47:04.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.618+0000 7fbda3c87640 1 -- 172.21.15.80:0/2562925946 wait complete. 2024-04-25T13:47:04.641 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:47:04.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1bd95640 1 Processor -- start 2024-04-25T13:47:04.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1bd95640 1 -- start start 2024-04-25T13:47:04.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1bd95640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14071e60 0x7f4c141574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1bd95640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4c14161170 0x7f4c140b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1bd95640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c1406fbf0 con 0x7f4c14073c40 2024-04-25T13:47:04.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1bd95640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4c140b1350 con 0x7f4c14161170 2024-04-25T13:47:04.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1bd95640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4c1406f9a0 con 0x7f4c14071e60 2024-04-25T13:47:04.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c19330640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4c14073c40 0x7f4c14160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52656/0 (socket says 172.21.15.80:52656) 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c19330640 1 -- 172.21.15.80:0/1435806842 learned_addr learned my addr 172.21.15.80:0/1435806842 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1a332640 1 --2- 172.21.15.80:0/1435806842 >> v2:172.21.15.87:3300/0 conn(0x7f4c14161170 0x7f4c140b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c19b31640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14071e60 0x7f4c141574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2879393112 0 0) 0x7f4c1406fbf0 con 0x7f4c14073c40 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4bfc003530 con 0x7f4c14073c40 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1473125416 0 0) 0x7f4bfc003530 con 0x7f4c14073c40 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14071e60 msgr2=0x7f4c141574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14071e60 0x7f4c141574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 >> v2:172.21.15.87:3300/0 conn(0x7f4c14161170 msgr2=0x7f4c140b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 --2- 172.21.15.80:0/1435806842 >> v2:172.21.15.87:3300/0 conn(0x7f4c14161170 0x7f4c140b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c140b0a20 con 0x7f4c14073c40 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4c08001680 con 0x7f4c14073c40 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 >> v1:172.21.15.80:6789/0 conn(0x7f4c14073c40 legacy=0x7f4c14160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:04.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bfc0039c0 0x7f4bfc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14071e60 0x7f4bfc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bfc004d50 0x7f4bfc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4c140b1350 con 0x7f4bfc004d50 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4c1406f9a0 con 0x7f4bfc0039c0 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4bfc00b9d0 con 0x7f4c14071e60 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1a332640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bfc004d50 0x7f4bfc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c19b31640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bfc0039c0 0x7f4bfc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1a332640 1 -- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14071e60 msgr2=0x7f4bfc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1a332640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14071e60 0x7f4bfc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1a332640 1 -- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bfc0039c0 msgr2=0x7f4bfc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1a332640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4bfc0039c0 0x7f4bfc003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.802+0000 7f4c1a332640 1 -- 172.21.15.80:0/1435806842 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bfc003530 con 0x7f4bfc004d50 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c1a332640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bfc004d50 0x7f4bfc005160 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009ae0 tx=0x7f4c0c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb6c2841c617bb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0b2070 con 0x7f4bfc004d50 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c0c0a5660 con 0x7f4bfc004d50 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c18b2f640 1 -- 172.21.15.80:0/1435806842 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c0096b0 con 0x7f4bfc004d50 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1435806842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bfc004d50 msgr2=0x7f4bfc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c1bd95640 1 --2- 172.21.15.80:0/1435806842 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4bfc004d50 0x7f4bfc005160 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c009ae0 tx=0x7f4c0c01ca70 comp rx=0 tx=0).stop 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c1a332640 1 -- 172.21.15.80:0/1435806842 reap_dead start 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1435806842 shutdown_connections 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1435806842 >> 172.21.15.80:0/1435806842 conn(0x7f4c140b4160 msgr2=0x7f4c140d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1435806842 shutdown_connections 2024-04-25T13:47:04.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.806+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1435806842 wait complete. 2024-04-25T13:47:04.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 Processor -- start 2024-04-25T13:47:04.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 -- start start 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c141545d0 0x7f4c1414af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c141f7780 0x7f4c141557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14155d10 0x7f4c1406e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4c14072270 con 0x7f4c141f7780 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4c140b0fe0 con 0x7f4c141545d0 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4c14074050 con 0x7f4c14155d10 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c141f7780 0x7f4c141557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c141f7780 0x7f4c141557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38100/0 (socket says 172.21.15.80:38100) 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 -- 172.21.15.80:0/1511050777 learned_addr learned my addr 172.21.15.80:0/1511050777 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19b31640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c141545d0 0x7f4c1414af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:04.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1a332640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14155d10 0x7f4c1406e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 -- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14155d10 msgr2=0x7f4c1406e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14155d10 0x7f4c1406e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 -- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c141545d0 msgr2=0x7f4c1414af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c141545d0 0x7f4c1414af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 -- 172.21.15.80:0/1511050777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c14156120 con 0x7f4c141f7780 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19b31640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c141545d0 0x7f4c1414af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c19330640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c141f7780 0x7f4c141557d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800caa0 tx=0x7f4c0800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c127fc640 1 -- 172.21.15.80:0/1511050777 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08018070 con 0x7f4c141f7780 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c127fc640 1 -- 172.21.15.80:0/1511050777 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c08004ce0 con 0x7f4c141f7780 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c141f7f60 con 0x7f4c141f7780 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c127fc640 1 -- 172.21.15.80:0/1511050777 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c08007310 con 0x7f4c141f7780 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c141f8520 con 0x7f4c141f7780 2024-04-25T13:47:04.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.810+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c1406a3a0 con 0x7f4c141f7780 2024-04-25T13:47:04.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.814+0000 7f4c127fc640 1 -- 172.21.15.80:0/1511050777 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4c0801b070 con 0x7f4c141f7780 2024-04-25T13:47:04.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.814+0000 7f4c127fc640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4c0006f400 0x7f4c00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:04.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.814+0000 7f4c127fc640 1 -- 172.21.15.80:0/1511050777 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4943+0+0 (secure 0 0 0) 0x7f4c0808e7f0 con 0x7f4c141f7780 2024-04-25T13:47:04.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.814+0000 7f4c19b31640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4c0006f400 0x7f4c00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:04.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.814+0000 7f4c19b31640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4c0006f400 0x7f4c00071910 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4c14072860 tx=0x7f4c0400d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:04.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.818+0000 7f4c127fc640 1 -- 172.21.15.80:0/1511050777 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4c14156120 con 0x7f4c141f7780 2024-04-25T13:47:04.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.978+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4c14073130 con 0x7f4c141f7780 2024-04-25T13:47:04.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c127fc640 1 -- 172.21.15.80:0/1511050777 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) ==== 122+0+0 (secure 0 0 0) 0x7f4c0805bb50 con 0x7f4c141f7780 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4c0006f400 msgr2=0x7f4c00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4c0006f400 0x7f4c00071910 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4c14072860 tx=0x7f4c0400d040 comp rx=0 tx=0).stop 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c141f7780 msgr2=0x7f4c141557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c141f7780 0x7f4c141557d0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f4c0800caa0 tx=0x7f4c0800cf60 comp rx=0 tx=0).stop 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 shutdown_connections 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4c0006f400 0x7f4c00071910 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4c14155d10 0x7f4c1406e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4c141f7780 0x7f4c141557d0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 --2- 172.21.15.80:0/1511050777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4c141545d0 0x7f4c1414af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:04.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 >> 172.21.15.80:0/1511050777 conn(0x7f4c140b4160 msgr2=0x7f4c1415d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:04.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 shutdown_connections 2024-04-25T13:47:04.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:04.982+0000 7f4c1bd95640 1 -- 172.21.15.80:0/1511050777 wait complete. 2024-04-25T13:47:05.003 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:47:05.003 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:47:05.053 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:47:05.106 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:47:05.106 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:05.106 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:05.111 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:47:05.111 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:47:05.111 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:05.117 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:47:05.117 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:05.117 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:05.160 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:47:05.160 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:47:05.160 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:05.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:47:05.166 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:47:05.166 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:05.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f9305d8f640 1 Processor -- start 2024-04-25T13:47:05.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f9305d8f640 1 -- start start 2024-04-25T13:47:05.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f9305d8f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9300161650 0x7f9300072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f9305d8f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f930006fcf0 con 0x7f930006f5a0 2024-04-25T13:47:05.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f9305d8f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93000b0c60 con 0x7f9300161240 2024-04-25T13:47:05.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f9305d8f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9300073f10 con 0x7f9300161650 2024-04-25T13:47:05.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f92ff7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f930006f5a0 0x7f9300160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52658/0 (socket says 172.21.15.80:52658) 2024-04-25T13:47:05.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f92fffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9300161650 0x7f9300072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f92ff7fe640 1 -- 172.21.15.80:0/3836102655 learned_addr learned my addr 172.21.15.80:0/3836102655 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:05.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f92fffff640 1 -- 172.21.15.80:0/3836102655 >> v1:172.21.15.87:6789/0 conn(0x7f9300161240 legacy=0x7f93000723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:05.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f92fffff640 1 -- 172.21.15.80:0/3836102655 >> v1:172.21.15.80:6789/0 conn(0x7f930006f5a0 legacy=0x7f9300160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f92fffff640 1 -- 172.21.15.80:0/3836102655 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93000b0f40 con 0x7f9300161650 2024-04-25T13:47:05.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.362+0000 7f92fe7fc640 1 -- 172.21.15.80:0/3836102655 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4032896053 0 0) 0x7f930006fcf0 con 0x7f930006f5a0 2024-04-25T13:47:05.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f92fffff640 1 --2- 172.21.15.80:0/3836102655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161650 0x7f9300072ab0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009d90 tx=0x7f92f801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76266b06cfe14676 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:05.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f92fe7fc640 1 -- 172.21.15.80:0/3836102655 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f80b2070 con 0x7f9300161650 2024-04-25T13:47:05.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f92fe7fc640 1 -- 172.21.15.80:0/3836102655 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f80a57e0 con 0x7f9300161650 2024-04-25T13:47:05.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f92fe7fc640 1 -- 172.21.15.80:0/3836102655 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f80a5bb0 con 0x7f9300161650 2024-04-25T13:47:05.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 -- 172.21.15.80:0/3836102655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161650 msgr2=0x7f9300072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 --2- 172.21.15.80:0/3836102655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161650 0x7f9300072ab0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009d90 tx=0x7f92f801c8a0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 -- 172.21.15.80:0/3836102655 shutdown_connections 2024-04-25T13:47:05.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 --2- 172.21.15.80:0/3836102655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161650 0x7f9300072ab0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 -- 172.21.15.80:0/3836102655 >> 172.21.15.80:0/3836102655 conn(0x7f930006d080 msgr2=0x7f93000b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:05.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 -- 172.21.15.80:0/3836102655 shutdown_connections 2024-04-25T13:47:05.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 -- 172.21.15.80:0/3836102655 wait complete. 2024-04-25T13:47:05.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 Processor -- start 2024-04-25T13:47:05.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.366+0000 7f9305d8f640 1 -- start start 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f930006f5a0 0x7f9300159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161240 0x7f930013e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9300161650 0x7f930015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f930016b270 con 0x7f9300161240 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f930016b0f0 con 0x7f930006f5a0 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f930016b3f0 con 0x7f9300161650 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161240 0x7f930013e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161240 0x7f930013e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38112/0 (socket says 172.21.15.80:38112) 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 -- 172.21.15.80:0/1201849561 learned_addr learned my addr 172.21.15.80:0/1201849561 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92ff7fe640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f930006f5a0 0x7f9300159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92fffff640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9300161650 0x7f930015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 -- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9300161650 msgr2=0x7f930015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9300161650 0x7f930015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 -- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f930006f5a0 msgr2=0x7f9300159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f930006f5a0 0x7f9300159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 -- 172.21.15.80:0/1201849561 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9300159cb0 con 0x7f9300161240 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92ff7fe640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f930006f5a0 0x7f9300159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92feffd640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161240 0x7f930013e500 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f92f400b410 tx=0x7f92f400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92e7fff640 1 -- 172.21.15.80:0/1201849561 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4015030 con 0x7f9300161240 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92e7fff640 1 -- 172.21.15.80:0/1201849561 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f4004020 con 0x7f9300161240 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930015e980 con 0x7f9300161240 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92e7fff640 1 -- 172.21.15.80:0/1201849561 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4004410 con 0x7f9300161240 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f930015ef40 con 0x7f9300161240 2024-04-25T13:47:05.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f930006a3a0 con 0x7f9300161240 2024-04-25T13:47:05.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.370+0000 7f92e7fff640 1 -- 172.21.15.80:0/1201849561 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f92f4018070 con 0x7f9300161240 2024-04-25T13:47:05.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.374+0000 7f92e7fff640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92ec06f400 0x7f92ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.374+0000 7f92e7fff640 1 -- 172.21.15.80:0/1201849561 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4943+0+0 (secure 0 0 0) 0x7f92f408ea80 con 0x7f9300161240 2024-04-25T13:47:05.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.374+0000 7f92ff7fe640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92ec06f400 0x7f92ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.374+0000 7f92ff7fe640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92ec06f400 0x7f92ec071910 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f92f0007fd0 tx=0x7f92f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:05.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.378+0000 7f92e7fff640 1 -- 172.21.15.80:0/1201849561 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92f405bd00 con 0x7f9300161240 2024-04-25T13:47:05.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.562+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f930015e6d0 con 0x7f9300161240 2024-04-25T13:47:05.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.562+0000 7f92e7fff640 1 -- 172.21.15.80:0/1201849561 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+3860 (secure 0 0 0) 0x7f92f405b600 con 0x7f9300161240 2024-04-25T13:47:05.562 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:05.562 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":61,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/341798465","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":341798465},{"type":"v1","addr":"172.21.15.87:6836","nonce":341798465}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":6157,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/731432476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":731432476},{"type":"v1","addr":"172.21.15.87:6837","nonce":731432476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":53},{"gid":7488,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/120798181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":120798181},{"type":"v1","addr":"172.21.15.80:6833","nonce":120798181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":52},{"gid":7893,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1799112517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1799112517},{"type":"v1","addr":"172.21.15.80:6835","nonce":1799112517}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[]} 2024-04-25T13:47:05.563 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 61 2024-04-25T13:47:05.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92ec06f400 msgr2=0x7f92ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92ec06f400 0x7f92ec071910 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f92f0007fd0 tx=0x7f92f0009040 comp rx=0 tx=0).stop 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161240 msgr2=0x7f930013e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161240 0x7f930013e500 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f92f400b410 tx=0x7f92f400b8d0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 shutdown_connections 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92ec06f400 0x7f92ec071910 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9300161650 0x7f930015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9300161240 0x7f930013e500 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 --2- 172.21.15.80:0/1201849561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f930006f5a0 0x7f9300159770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 >> 172.21.15.80:0/1201849561 conn(0x7f930006d080 msgr2=0x7f93000757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 shutdown_connections 2024-04-25T13:47:05.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.566+0000 7f9305d8f640 1 -- 172.21.15.80:0/1201849561 wait complete. 2024-04-25T13:47:05.584 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:47:05.584 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:47:05.584 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:47:05.584 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:47:05.585 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:47:05.585 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:47:05.585 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:47:05.585 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:47:05.585 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:47:05.585 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:47:05.585 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:47:05.585 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:47:05.586 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:47:05.586 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:47:05.586 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:47:05.586 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:47:05.599 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:47:05.599 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:47:05.601 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:47:05.601 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:47:05.603 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:47:05.604 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:47:05.609 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:47:05.609 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:47:05.644 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:47:05.645 DEBUG:teuthology.parallel:result is None 2024-04-25T13:47:05.645 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:47:05.645 DEBUG:teuthology.parallel:result is None 2024-04-25T13:47:05.657 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:47:05.657 DEBUG:teuthology.parallel:result is None 2024-04-25T13:47:05.659 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:47:05.659 DEBUG:teuthology.parallel:result is None 2024-04-25T13:47:05.659 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:47:05.732 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:47:05.733 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:47:05.758 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:47:05.758 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:47:05.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb7277640 1 Processor -- start 2024-04-25T13:47:05.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb7277640 1 -- start start 2024-04-25T13:47:05.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb7277640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcbb0161f90 0x7fcbb00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb7277640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb00d0510 0x7fcbb015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb7277640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcbb015ea70 0x7fcbb015ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb7277640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb00d0d70 con 0x7fcbb015ea70 2024-04-25T13:47:05.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb7277640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcbb00d1050 con 0x7fcbb0161f90 2024-04-25T13:47:05.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb7277640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb006dca0 con 0x7fcbb00d0510 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb5814640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcbb015ea70 0x7fcbb015ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb5814640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcbb015ea70 0x7fcbb015ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38152/0 (socket says 172.21.15.80:38152) 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb5814640 1 -- 172.21.15.80:0/1568964928 learned_addr learned my addr 172.21.15.80:0/1568964928 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb5013640 1 --2- 172.21.15.80:0/1568964928 >> v2:172.21.15.87:3300/0 conn(0x7fcbb0161f90 0x7fcbb00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.818+0000 7fcbb4812640 1 --2- 172.21.15.80:0/1568964928 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb00d0510 0x7fcbb015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb5814640 1 -- 172.21.15.80:0/1568964928 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb00d0510 msgr2=0x7fcbb015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb5814640 1 --2- 172.21.15.80:0/1568964928 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb00d0510 0x7fcbb015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb5814640 1 -- 172.21.15.80:0/1568964928 >> v2:172.21.15.87:3300/0 conn(0x7fcbb0161f90 msgr2=0x7fcbb00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb5814640 1 --2- 172.21.15.80:0/1568964928 >> v2:172.21.15.87:3300/0 conn(0x7fcbb0161f90 0x7fcbb00cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb5814640 1 -- 172.21.15.80:0/1568964928 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb006df80 con 0x7fcbb015ea70 2024-04-25T13:47:05.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb5013640 1 --2- 172.21.15.80:0/1568964928 >> v2:172.21.15.87:3300/0 conn(0x7fcbb0161f90 0x7fcbb00cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:05.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb5814640 1 --2- 172.21.15.80:0/1568964928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb015ea70 0x7fcbb015ee80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00d17b0 tx=0x7fcba801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a8cba38302e300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:05.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcba7fff640 1 -- 172.21.15.80:0/1568964928 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80b1070 con 0x7fcbb015ea70 2024-04-25T13:47:05.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcba7fff640 1 -- 172.21.15.80:0/1568964928 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba80a8050 con 0x7fcbb015ea70 2024-04-25T13:47:05.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcba7fff640 1 -- 172.21.15.80:0/1568964928 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80ac860 con 0x7fcbb015ea70 2024-04-25T13:47:05.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 -- 172.21.15.80:0/1568964928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb015ea70 msgr2=0x7fcbb015ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 --2- 172.21.15.80:0/1568964928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb015ea70 0x7fcbb015ee80 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00d17b0 tx=0x7fcba801c460 comp rx=0 tx=0).stop 2024-04-25T13:47:05.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 -- 172.21.15.80:0/1568964928 shutdown_connections 2024-04-25T13:47:05.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 --2- 172.21.15.80:0/1568964928 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb015ea70 0x7fcbb015ee80 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 --2- 172.21.15.80:0/1568964928 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb00d0510 0x7fcbb015e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 --2- 172.21.15.80:0/1568964928 >> v2:172.21.15.87:3300/0 conn(0x7fcbb0161f90 0x7fcbb00cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 -- 172.21.15.80:0/1568964928 >> 172.21.15.80:0/1568964928 conn(0x7fcbb006d080 msgr2=0x7fcbb00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:05.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 -- 172.21.15.80:0/1568964928 shutdown_connections 2024-04-25T13:47:05.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.822+0000 7fcbb7277640 1 -- 172.21.15.80:0/1568964928 wait complete. 2024-04-25T13:47:05.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 Processor -- start 2024-04-25T13:47:05.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 -- start start 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbb00d0510 0x7fcbb015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb015ea70 0x7fcbb0165990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb0161f90 0x7fcbb0165ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb01447a0 con 0x7fcbb0161f90 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb0144620 con 0x7fcbb00d0510 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcbb00d0920 con 0x7fcbb015ea70 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb0161f90 0x7fcbb0165ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb0161f90 0x7fcbb0165ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38164/0 (socket says 172.21.15.80:38164) 2024-04-25T13:47:05.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 -- 172.21.15.80:0/3484267663 learned_addr learned my addr 172.21.15.80:0/3484267663 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb4812640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb015ea70 0x7fcbb0165990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5013640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbb00d0510 0x7fcbb015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 -- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb015ea70 msgr2=0x7fcbb0165990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb015ea70 0x7fcbb0165990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 -- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbb00d0510 msgr2=0x7fcbb015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbb00d0510 0x7fcbb015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 -- 172.21.15.80:0/3484267663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb0168070 con 0x7fcbb0161f90 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5013640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbb00d0510 0x7fcbb015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb5814640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb0161f90 0x7fcbb0165ed0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcba801ab10 tx=0x7fcba801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcba5ffb640 1 -- 172.21.15.80:0/3484267663 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80b1040 con 0x7fcbb0161f90 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcba5ffb640 1 -- 172.21.15.80:0/3484267663 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcba80bc070 con 0x7fcbb0161f90 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcba5ffb640 1 -- 172.21.15.80:0/3484267663 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcba80ace60 con 0x7fcbb0161f90 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb0168350 con 0x7fcbb0161f90 2024-04-25T13:47:05.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.826+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb0163880 con 0x7fcbb0161f90 2024-04-25T13:47:05.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.830+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb0167de0 con 0x7fcbb0161f90 2024-04-25T13:47:05.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.830+0000 7fcba5ffb640 1 -- 172.21.15.80:0/3484267663 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcba80b7690 con 0x7fcbb0161f90 2024-04-25T13:47:05.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.830+0000 7fcba5ffb640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb9406f400 0x7fcb94071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:05.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.830+0000 7fcba5ffb640 1 -- 172.21.15.80:0/3484267663 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4943+0+0 (secure 0 0 0) 0x7fcba8129200 con 0x7fcbb0161f90 2024-04-25T13:47:05.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.830+0000 7fcbb5013640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb9406f400 0x7fcb94071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:05.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.830+0000 7fcbb5013640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb9406f400 0x7fcb94071910 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcbb006fa20 tx=0x7fcba0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:05.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:05.834+0000 7fcba5ffb640 1 -- 172.21.15.80:0/3484267663 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcba8016110 con 0x7fcbb0161f90 2024-04-25T13:47:06.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.010+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcbb006a3a0 con 0x7fcbb0161f90 2024-04-25T13:47:06.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.638+0000 7fcba5ffb640 1 -- 172.21.15.80:0/3484267663 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v130) ==== 97+0+0 (secure 0 0 0) 0x7fcba80f6510 con 0x7fcbb0161f90 2024-04-25T13:47:06.636 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:47:06.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb9406f400 msgr2=0x7fcb94071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb9406f400 0x7fcb94071910 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fcbb006fa20 tx=0x7fcba0008040 comp rx=0 tx=0).stop 2024-04-25T13:47:06.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb0161f90 msgr2=0x7fcbb0165ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb0161f90 0x7fcbb0165ed0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcba801ab10 tx=0x7fcba801aca0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 shutdown_connections 2024-04-25T13:47:06.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb9406f400 0x7fcb94071910 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbb0161f90 0x7fcbb0165ed0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbb015ea70 0x7fcbb0165990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 --2- 172.21.15.80:0/3484267663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbb00d0510 0x7fcbb015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 >> 172.21.15.80:0/3484267663 conn(0x7fcbb006d080 msgr2=0x7fcbb01685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:06.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 shutdown_connections 2024-04-25T13:47:06.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.642+0000 7fcbb7277640 1 -- 172.21.15.80:0/3484267663 wait complete. 2024-04-25T13:47:06.669 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:47:06.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc378003640 1 Processor -- start 2024-04-25T13:47:06.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc378003640 1 -- start start 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc378003640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3700b2020 0x7fc370068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc378003640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc3700702b0 0x7fc370068ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc378003640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc37006caf0 con 0x7fc370149410 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc378003640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc37006cdd0 con 0x7fc3700702b0 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc378003640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc37006d0b0 con 0x7fc3700b2020 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc3765a0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc3700702b0 0x7fc370068ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc3765a0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc3700702b0 0x7fc370068ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38176/0 (socket says 172.21.15.80:38176) 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc3765a0640 1 -- 172.21.15.80:0/292559698 learned_addr learned my addr 172.21.15.80:0/292559698 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:06.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc375d9f640 1 --2- 172.21.15.80:0/292559698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3700b2020 0x7fc370068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc3765a0640 1 -- 172.21.15.80:0/292559698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3700b2020 msgr2=0x7fc370068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc3765a0640 1 --2- 172.21.15.80:0/292559698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3700b2020 0x7fc370068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc3765a0640 1 -- 172.21.15.80:0/292559698 >> v1:172.21.15.87:6789/0 conn(0x7fc370149410 legacy=0x7fc370163de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:06.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.822+0000 7fc3765a0640 1 -- 172.21.15.80:0/292559698 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37006d390 con 0x7fc3700702b0 2024-04-25T13:47:06.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc3765a0640 1 --2- 172.21.15.80:0/292559698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700702b0 0x7fc370068ad0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009e50 tx=0x7fc36c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa08b94d1d5ab2dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc374d9d640 1 -- 172.21.15.80:0/292559698 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0a57b0 con 0x7fc3700702b0 2024-04-25T13:47:06.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc374d9d640 1 -- 172.21.15.80:0/292559698 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc36c0a5950 con 0x7fc3700702b0 2024-04-25T13:47:06.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc374d9d640 1 -- 172.21.15.80:0/292559698 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36c0a5d20 con 0x7fc3700702b0 2024-04-25T13:47:06.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc378003640 1 -- 172.21.15.80:0/292559698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700702b0 msgr2=0x7fc370068ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc378003640 1 --2- 172.21.15.80:0/292559698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700702b0 0x7fc370068ad0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fc36c009e50 tx=0x7fc36c01c9f0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc378003640 1 -- 172.21.15.80:0/292559698 shutdown_connections 2024-04-25T13:47:06.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc378003640 1 --2- 172.21.15.80:0/292559698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700702b0 0x7fc370068ad0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc378003640 1 --2- 172.21.15.80:0/292559698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc3700b2020 0x7fc370068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc378003640 1 -- 172.21.15.80:0/292559698 >> 172.21.15.80:0/292559698 conn(0x7fc3700d28f0 msgr2=0x7fc3700d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc378003640 1 -- 172.21.15.80:0/292559698 shutdown_connections 2024-04-25T13:47:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.826+0000 7fc378003640 1 -- 172.21.15.80:0/292559698 wait complete. 2024-04-25T13:47:06.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 Processor -- start 2024-04-25T13:47:06.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 -- start start 2024-04-25T13:47:06.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3700702b0 0x7fc3700d1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700b2020 0x7fc37015e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc370149410 0x7fc37015eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc37016dd20 con 0x7fc3700b2020 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc37016dba0 con 0x7fc3700702b0 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc37016dea0 con 0x7fc370149410 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700b2020 0x7fc37015e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700b2020 0x7fc37015e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38190/0 (socket says 172.21.15.80:38190) 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 -- 172.21.15.80:0/1619044091 learned_addr learned my addr 172.21.15.80:0/1619044091 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc375d9f640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3700702b0 0x7fc3700d1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc3765a0640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc370149410 0x7fc37015eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 -- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc370149410 msgr2=0x7fc37015eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc370149410 0x7fc37015eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 -- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3700702b0 msgr2=0x7fc3700d1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3700702b0 0x7fc3700d1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 -- 172.21.15.80:0/1619044091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc370143550 con 0x7fc3700b2020 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc37559e640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700b2020 0x7fc37015e630 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc36000b440 tx=0x7fc36000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc3667fc640 1 -- 172.21.15.80:0/1619044091 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360015020 con 0x7fc3700b2020 2024-04-25T13:47:06.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc3667fc640 1 -- 172.21.15.80:0/1619044091 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc360011830 con 0x7fc3700b2020 2024-04-25T13:47:06.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc370143270 con 0x7fc3700b2020 2024-04-25T13:47:06.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc3667fc640 1 -- 172.21.15.80:0/1619044091 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360011c20 con 0x7fc3700b2020 2024-04-25T13:47:06.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.830+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3700cf490 con 0x7fc3700b2020 2024-04-25T13:47:06.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.834+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc370142fe0 con 0x7fc3700b2020 2024-04-25T13:47:06.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.834+0000 7fc3667fc640 1 -- 172.21.15.80:0/1619044091 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc360018020 con 0x7fc3700b2020 2024-04-25T13:47:06.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.834+0000 7fc3667fc640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc35406f430 0x7fc354071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:06.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.834+0000 7fc3667fc640 1 -- 172.21.15.80:0/1619044091 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4857+0+0 (secure 0 0 0) 0x7fc36008f1a0 con 0x7fc3700b2020 2024-04-25T13:47:06.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.834+0000 7fc375d9f640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc35406f430 0x7fc354071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:06.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.834+0000 7fc375d9f640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc35406f430 0x7fc354071940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc3680079c0 tx=0x7fc368008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:06.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:06.838+0000 7fc3667fc640 1 -- 172.21.15.80:0/1619044091 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc36001f020 con 0x7fc3700b2020 2024-04-25T13:47:07.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc37006a3a0 con 0x7fc3700b2020 2024-04-25T13:47:07.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc3667fc640 1 -- 172.21.15.80:0/1619044091 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7fc36005be70 con 0x7fc3700b2020 2024-04-25T13:47:07.056 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.057 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:47:07.058 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:47:07.060 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:47:07.061 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.062 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.063 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:47:07.064 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:47:07.065 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:47:07.066 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:47:07.067 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:47:07.068 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:47:07.069 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:47:07.070 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:47:07.070 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:47:07.070 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:47:07.070 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.070 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:07.070 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:07.070 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:07.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc35406f430 msgr2=0x7fc354071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc35406f430 0x7fc354071940 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc3680079c0 tx=0x7fc368008040 comp rx=0 tx=0).stop 2024-04-25T13:47:07.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700b2020 msgr2=0x7fc37015e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700b2020 0x7fc37015e630 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fc36000b440 tx=0x7fc36000b900 comp rx=0 tx=0).stop 2024-04-25T13:47:07.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 shutdown_connections 2024-04-25T13:47:07.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc35406f430 0x7fc354071940 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc370149410 0x7fc37015eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc3700b2020 0x7fc37015e630 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 --2- 172.21.15.80:0/1619044091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3700702b0 0x7fc3700d1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 >> 172.21.15.80:0/1619044091 conn(0x7fc3700d28f0 msgr2=0x7fc37006d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:07.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 shutdown_connections 2024-04-25T13:47:07.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.058+0000 7fc378003640 1 -- 172.21.15.80:0/1619044091 wait complete. 2024-04-25T13:47:07.080 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:07.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.226+0000 7f1fc4846640 1 Processor -- start 2024-04-25T13:47:07.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.226+0000 7f1fc4846640 1 -- start start 2024-04-25T13:47:07.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fc4846640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1fc0161240 0x7f1fc00723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fc4846640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1fc0161650 0x7f1fc00728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fc4846640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fc006fcf0 con 0x7f1fc006f5a0 2024-04-25T13:47:07.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fc4846640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc0073f10 con 0x7f1fc0161650 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fc4846640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1fc00b0c60 con 0x7f1fc0161240 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1fc0161650 0x7f1fc00728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbe59c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1fc006f5a0 0x7f1fc0160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52668/0 (socket says 172.21.15.80:52668) 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbe59c640 1 -- 172.21.15.80:0/2599012023 learned_addr learned my addr 172.21.15.80:0/2599012023 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbdd9b640 1 --2- 172.21.15.80:0/2599012023 >> v2:172.21.15.87:3300/0 conn(0x7f1fc0161240 0x7f1fc00723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbed9d640 1 -- 172.21.15.80:0/2599012023 >> v2:172.21.15.87:3300/0 conn(0x7f1fc0161240 msgr2=0x7f1fc00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbed9d640 1 --2- 172.21.15.80:0/2599012023 >> v2:172.21.15.87:3300/0 conn(0x7f1fc0161240 0x7f1fc00723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbed9d640 1 -- 172.21.15.80:0/2599012023 >> v1:172.21.15.80:6789/0 conn(0x7f1fc006f5a0 legacy=0x7f1fc0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbed9d640 1 -- 172.21.15.80:0/2599012023 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc00b0f40 con 0x7f1fc0161650 2024-04-25T13:47:07.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.230+0000 7f1fbd59a640 1 -- 172.21.15.80:0/2599012023 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2043517468 0 0) 0x7f1fc006fcf0 con 0x7f1fc006f5a0 2024-04-25T13:47:07.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fbed9d640 1 --2- 172.21.15.80:0/2599012023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc0161650 0x7f1fc00728e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1fc006ff70 tx=0x7f1fb801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0de1cb648e97976 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fbd59a640 1 -- 172.21.15.80:0/2599012023 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80b2070 con 0x7f1fc0161650 2024-04-25T13:47:07.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fbd59a640 1 -- 172.21.15.80:0/2599012023 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb80a57e0 con 0x7f1fc0161650 2024-04-25T13:47:07.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fbd59a640 1 -- 172.21.15.80:0/2599012023 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb80a5bb0 con 0x7f1fc0161650 2024-04-25T13:47:07.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 -- 172.21.15.80:0/2599012023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc0161650 msgr2=0x7f1fc00728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 --2- 172.21.15.80:0/2599012023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc0161650 0x7f1fc00728e0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f1fc006ff70 tx=0x7f1fb801c8a0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 -- 172.21.15.80:0/2599012023 shutdown_connections 2024-04-25T13:47:07.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 --2- 172.21.15.80:0/2599012023 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc0161650 0x7f1fc00728e0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 --2- 172.21.15.80:0/2599012023 >> v2:172.21.15.87:3300/0 conn(0x7f1fc0161240 0x7f1fc00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 -- 172.21.15.80:0/2599012023 >> 172.21.15.80:0/2599012023 conn(0x7f1fc006d080 msgr2=0x7f1fc00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:07.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 -- 172.21.15.80:0/2599012023 shutdown_connections 2024-04-25T13:47:07.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 -- 172.21.15.80:0/2599012023 wait complete. 2024-04-25T13:47:07.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 Processor -- start 2024-04-25T13:47:07.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.234+0000 7f1fc4846640 1 -- start start 2024-04-25T13:47:07.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc006f5a0 0x7f1fc015d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1fc0161240 0x7f1fc01411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc0161650 0x7f1fc0150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc016a260 con 0x7f1fc006f5a0 2024-04-25T13:47:07.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc016a0e0 con 0x7f1fc0161650 2024-04-25T13:47:07.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1fc016a3e0 con 0x7f1fc0161240 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc006f5a0 0x7f1fc015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc006f5a0 0x7f1fc015d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38216/0 (socket says 172.21.15.80:38216) 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 -- 172.21.15.80:0/3733277600 learned_addr learned my addr 172.21.15.80:0/3733277600 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 -- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1fc0161240 msgr2=0x7f1fc01411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbed9d640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc0161650 0x7f1fc0150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbdd9b640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1fc0161240 0x7f1fc01411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1fc0161240 0x7f1fc01411d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 -- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc0161650 msgr2=0x7f1fc0150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc0161650 0x7f1fc0150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 -- 172.21.15.80:0/3733277600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc015dee0 con 0x7f1fc006f5a0 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fbe59c640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc006f5a0 0x7f1fc015d590 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000c480 tx=0x7f1fb000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1faeffd640 1 -- 172.21.15.80:0/3733277600 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0018070 con 0x7f1fc006f5a0 2024-04-25T13:47:07.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1faeffd640 1 -- 172.21.15.80:0/3733277600 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fb00074e0 con 0x7f1fc006f5a0 2024-04-25T13:47:07.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1faeffd640 1 -- 172.21.15.80:0/3733277600 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb00049f0 con 0x7f1fc006f5a0 2024-04-25T13:47:07.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc015e1c0 con 0x7f1fc006f5a0 2024-04-25T13:47:07.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc015e780 con 0x7f1fc006f5a0 2024-04-25T13:47:07.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.238+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc006a3a0 con 0x7f1fc006f5a0 2024-04-25T13:47:07.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.242+0000 7f1faeffd640 1 -- 172.21.15.80:0/3733277600 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1fb001b070 con 0x7f1fc006f5a0 2024-04-25T13:47:07.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.242+0000 7f1faeffd640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fa006f400 0x7f1fa0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.242+0000 7f1faeffd640 1 -- 172.21.15.80:0/3733277600 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4857+0+0 (secure 0 0 0) 0x7f1fb008e650 con 0x7f1fc006f5a0 2024-04-25T13:47:07.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.242+0000 7f1fbdd9b640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fa006f400 0x7f1fa0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.242+0000 7f1fbdd9b640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fa006f400 0x7f1fa0071910 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0159850 tx=0x7f1fb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.246+0000 7f1faeffd640 1 -- 172.21.15.80:0/3733277600 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1fb0022080 con 0x7f1fc006f5a0 2024-04-25T13:47:07.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fc00739c0 con 0x7f1fc006f5a0 2024-04-25T13:47:07.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1faeffd640 1 -- 172.21.15.80:0/3733277600 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+12830 (secure 0 0 0) 0x7f1fb005b900 con 0x7f1fc006f5a0 2024-04-25T13:47:07.393 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:07.393 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":132,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:06.653184+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":120,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:07.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fa006f400 msgr2=0x7f1fa0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fa006f400 0x7f1fa0071910 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0159850 tx=0x7f1fb4008040 comp rx=0 tx=0).stop 2024-04-25T13:47:07.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc006f5a0 msgr2=0x7f1fc015d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc006f5a0 0x7f1fc015d590 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f1fb000c480 tx=0x7f1fb000c940 comp rx=0 tx=0).stop 2024-04-25T13:47:07.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 shutdown_connections 2024-04-25T13:47:07.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fa006f400 0x7f1fa0071910 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1fc0161650 0x7f1fc0150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1fc0161240 0x7f1fc01411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 --2- 172.21.15.80:0/3733277600 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1fc006f5a0 0x7f1fc015d590 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.394+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 >> 172.21.15.80:0/3733277600 conn(0x7f1fc006d080 msgr2=0x7f1fc00cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:07.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.398+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 shutdown_connections 2024-04-25T13:47:07.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.398+0000 7f1fc4846640 1 -- 172.21.15.80:0/3733277600 wait complete. 2024-04-25T13:47:07.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:47:07.414 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:47:07.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faad2000640 1 Processor -- start 2024-04-25T13:47:07.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faad2000640 1 -- start start 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faad2000640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faacc162270 0x7faacc06f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faad2000640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faacc0b5740 con 0x7faacc06edb0 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faad2000640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faacc06dd10 con 0x7faacc0b5330 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faad2000640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7faacc06f700 con 0x7faacc162270 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faacbfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faacc162270 0x7faacc06f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faacaffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7faacc06edb0 0x7faacc161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52672/0 (socket says 172.21.15.80:52672) 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faacaffd640 1 -- 172.21.15.80:0/2983302923 learned_addr learned my addr 172.21.15.80:0/2983302923 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faacbfff640 1 -- 172.21.15.80:0/2983302923 >> v1:172.21.15.87:6789/0 conn(0x7faacc0b5330 legacy=0x7faacc0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faacbfff640 1 -- 172.21.15.80:0/2983302923 >> v1:172.21.15.80:6789/0 conn(0x7faacc06edb0 legacy=0x7faacc161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faaca7fc640 1 -- 172.21.15.80:0/2983302923 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2504555506 0 0) 0x7faacc0b5740 con 0x7faacc06edb0 2024-04-25T13:47:07.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.562+0000 7faacbfff640 1 -- 172.21.15.80:0/2983302923 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc06dff0 con 0x7faacc162270 2024-04-25T13:47:07.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faacbfff640 1 --2- 172.21.15.80:0/2983302923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc162270 0x7faacc06f1c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7faac40097c0 tx=0x7faac401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deac86b326d03f49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faaca7fc640 1 -- 172.21.15.80:0/2983302923 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40b0070 con 0x7faacc162270 2024-04-25T13:47:07.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faaca7fc640 1 -- 172.21.15.80:0/2983302923 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faac40ab600 con 0x7faacc162270 2024-04-25T13:47:07.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faaca7fc640 1 -- 172.21.15.80:0/2983302923 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac40ab9d0 con 0x7faacc162270 2024-04-25T13:47:07.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 -- 172.21.15.80:0/2983302923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc162270 msgr2=0x7faacc06f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 --2- 172.21.15.80:0/2983302923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc162270 0x7faacc06f1c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7faac40097c0 tx=0x7faac401c410 comp rx=0 tx=0).stop 2024-04-25T13:47:07.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 -- 172.21.15.80:0/2983302923 shutdown_connections 2024-04-25T13:47:07.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 --2- 172.21.15.80:0/2983302923 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc162270 0x7faacc06f1c0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 -- 172.21.15.80:0/2983302923 >> 172.21.15.80:0/2983302923 conn(0x7faacc0b4160 msgr2=0x7faacc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:07.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 -- 172.21.15.80:0/2983302923 shutdown_connections 2024-04-25T13:47:07.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 -- 172.21.15.80:0/2983302923 wait complete. 2024-04-25T13:47:07.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 Processor -- start 2024-04-25T13:47:07.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.566+0000 7faad2000640 1 -- start start 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc06edb0 0x7faacc153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faacc0b5330 0x7faacc15b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faacc151ad0 0x7faacc151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faacc0b0e50 con 0x7faacc06edb0 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faacc0b0cd0 con 0x7faacc151ad0 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faacc0d11c0 con 0x7faacc0b5330 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc06edb0 0x7faacc153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc06edb0 0x7faacc153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38220/0 (socket says 172.21.15.80:38220) 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 -- 172.21.15.80:0/3924716976 learned_addr learned my addr 172.21.15.80:0/3924716976 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacbfff640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faacc151ad0 0x7faacc151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 -- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faacc0b5330 msgr2=0x7faacc15b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacaffd640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faacc0b5330 0x7faacc15b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faacc0b5330 0x7faacc15b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 -- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faacc151ad0 msgr2=0x7faacc151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faacc151ad0 0x7faacc151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 -- 172.21.15.80:0/3924716976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faacc0d0dc0 con 0x7faacc06edb0 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacbfff640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faacc151ad0 0x7faacc151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacaffd640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faacc0b5330 0x7faacc15b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faacb7fe640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc06edb0 0x7faacc153760 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c3f0 tx=0x7faabc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faab3fff640 1 -- 172.21.15.80:0/3924716976 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc017070 con 0x7faacc06edb0 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faab3fff640 1 -- 172.21.15.80:0/3924716976 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faabc0052d0 con 0x7faacc06edb0 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faacc159f30 con 0x7faacc06edb0 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faab3fff640 1 -- 172.21.15.80:0/3924716976 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc003840 con 0x7faacc06edb0 2024-04-25T13:47:07.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faacc15a4f0 con 0x7faacc06edb0 2024-04-25T13:47:07.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faacc06a3a0 con 0x7faacc06edb0 2024-04-25T13:47:07.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.570+0000 7faab3fff640 1 -- 172.21.15.80:0/3924716976 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faabc01a070 con 0x7faacc06edb0 2024-04-25T13:47:07.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.574+0000 7faab3fff640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faab806f400 0x7faab8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.574+0000 7faab3fff640 1 -- 172.21.15.80:0/3924716976 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4857+0+0 (secure 0 0 0) 0x7faabc08e1c0 con 0x7faacc06edb0 2024-04-25T13:47:07.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.574+0000 7faacaffd640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faab806f400 0x7faab8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.574+0000 7faacaffd640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faab806f400 0x7faab8071910 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faac0007fd0 tx=0x7faac0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.578+0000 7faab3fff640 1 -- 172.21.15.80:0/3924716976 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faabc005480 con 0x7faacc06edb0 2024-04-25T13:47:07.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.742+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faacc06fd00 con 0x7faacc06edb0 2024-04-25T13:47:07.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.742+0000 7faab3fff640 1 -- 172.21.15.80:0/3924716976 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) ==== 93+0+10 (secure 0 0 0) 0x7faabc05b520 con 0x7faacc06edb0 2024-04-25T13:47:07.744 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:47:07.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faab806f400 msgr2=0x7faab8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faab806f400 0x7faab8071910 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7faac0007fd0 tx=0x7faac0009040 comp rx=0 tx=0).stop 2024-04-25T13:47:07.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc06edb0 msgr2=0x7faacc153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc06edb0 0x7faacc153760 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7faabc00c3f0 tx=0x7faabc00c8b0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 shutdown_connections 2024-04-25T13:47:07.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faab806f400 0x7faab8071910 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faacc151ad0 0x7faacc151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faacc0b5330 0x7faacc15b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 --2- 172.21.15.80:0/3924716976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faacc06edb0 0x7faacc153760 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 >> 172.21.15.80:0/3924716976 conn(0x7faacc0b4160 msgr2=0x7faacc0b1eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:07.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 shutdown_connections 2024-04-25T13:47:07.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.746+0000 7faad2000640 1 -- 172.21.15.80:0/3924716976 wait complete. 2024-04-25T13:47:07.766 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:47:07.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6e5cf4640 1 Processor -- start 2024-04-25T13:47:07.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6e5cf4640 1 -- start start 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6e5cf4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc6e0068ca0 0x7fc6e0161490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6e5cf4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 0x7fc6e006ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6e5cf4640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc6e0164600 con 0x7fc6e0161080 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6e5cf4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc6e0160bf0 con 0x7fc6e0068ca0 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6e5cf4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e01648e0 con 0x7fc6e006c5f0 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6df7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc6e0161080 0x7fc6e0068590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52674/0 (socket says 172.21.15.80:52674) 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.910+0000 7fc6df7fe640 1 -- 172.21.15.80:0/2717557325 learned_addr learned my addr 172.21.15.80:0/2717557325 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6deffd640 1 --2- 172.21.15.80:0/2717557325 >> v2:172.21.15.87:3300/0 conn(0x7fc6e0068ca0 0x7fc6e0161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6dffff640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 0x7fc6e006ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2830895213 0 0) 0x7fc6e0164600 con 0x7fc6e0161080 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6c8003530 con 0x7fc6e0161080 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 120272816 0 0) 0x7fc6c8003530 con 0x7fc6e0161080 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 msgr2=0x7fc6e006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 0x7fc6e006ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 >> v2:172.21.15.87:3300/0 conn(0x7fc6e0068ca0 msgr2=0x7fc6e0161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 --2- 172.21.15.80:0/2717557325 >> v2:172.21.15.87:3300/0 conn(0x7fc6e0068ca0 0x7fc6e0161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e0164bc0 con 0x7fc6e0161080 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc6d0001680 con 0x7fc6e0161080 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 >> v1:172.21.15.80:6789/0 conn(0x7fc6e0161080 legacy=0x7fc6e0068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c80039c0 0x7fc6c8003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c8004810 0x7fc6c8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 0x7fc6c8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0160bf0 con 0x7fc6c80039c0 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e01648e0 con 0x7fc6c8004810 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc6c800bab0 con 0x7fc6e006c5f0 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6df7fe640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c80039c0 0x7fc6c8003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6deffd640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c8004810 0x7fc6c8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6dffff640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 0x7fc6c8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6df7fe640 1 -- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 msgr2=0x7fc6c8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6df7fe640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 0x7fc6c8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6df7fe640 1 -- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c8004810 msgr2=0x7fc6c8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6df7fe640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c8004810 0x7fc6c8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6df7fe640 1 -- 172.21.15.80:0/2717557325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6c8003530 con 0x7fc6c80039c0 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6dffff640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6e006c5f0 0x7fc6c8005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:07.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6deffd640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6c8004810 0x7fc6c8004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:07.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6df7fe640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c80039c0 0x7fc6c8003e50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c240 tx=0x7fc6d001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=472a322f29e5755 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0007d40 con 0x7fc6c80039c0 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d00a9020 con 0x7fc6c80039c0 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.914+0000 7fc6de7fc640 1 -- 172.21.15.80:0/2717557325 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d00a5b60 con 0x7fc6c80039c0 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/2717557325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c80039c0 msgr2=0x7fc6c8003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6e5cf4640 1 --2- 172.21.15.80:0/2717557325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6c80039c0 0x7fc6c8003e50 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c240 tx=0x7fc6d001cc40 comp rx=0 tx=0).stop 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6dffff640 1 -- 172.21.15.80:0/2717557325 reap_dead start 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/2717557325 shutdown_connections 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/2717557325 >> 172.21.15.80:0/2717557325 conn(0x7fc6e00d2900 msgr2=0x7fc6e00d2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/2717557325 shutdown_connections 2024-04-25T13:47:07.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/2717557325 wait complete. 2024-04-25T13:47:07.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6e5cf4640 1 Processor -- start 2024-04-25T13:47:07.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.918+0000 7fc6e5cf4640 1 -- start start 2024-04-25T13:47:07.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c80039c0 0x7fc6e0149e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6e0150120 0x7fc6e00b0300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6e0150530 0x7fc6e00b0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0156d00 con 0x7fc6e0150530 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0156b80 con 0x7fc6e0150120 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc6e0156e80 con 0x7fc6c80039c0 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6e0150530 0x7fc6e00b0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6e0150530 0x7fc6e00b0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38226/0 (socket says 172.21.15.80:38226) 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 -- 172.21.15.80:0/3926543037 learned_addr learned my addr 172.21.15.80:0/3926543037 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 -- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c80039c0 msgr2=0x7fc6e0149e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6deffd640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6e0150120 0x7fc6e00b0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6df7fe640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c80039c0 0x7fc6e0149e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c80039c0 0x7fc6e0149e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 -- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6e0150120 msgr2=0x7fc6e00b0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6e0150120 0x7fc6e00b0300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 -- 172.21.15.80:0/3926543037 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e014efa0 con 0x7fc6e0150530 2024-04-25T13:47:07.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6dffff640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6e0150530 0x7fc6e00b0840 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00d38f0 tx=0x7fc6d800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6c7fff640 1 -- 172.21.15.80:0/3926543037 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d801c070 con 0x7fc6e0150530 2024-04-25T13:47:07.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6c7fff640 1 -- 172.21.15.80:0/3926543037 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6d8017400 con 0x7fc6e0150530 2024-04-25T13:47:07.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e014f280 con 0x7fc6e0150530 2024-04-25T13:47:07.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6c7fff640 1 -- 172.21.15.80:0/3926543037 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d80177f0 con 0x7fc6e0150530 2024-04-25T13:47:07.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e014f840 con 0x7fc6e0150530 2024-04-25T13:47:07.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e006a3a0 con 0x7fc6e0150530 2024-04-25T13:47:07.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6c7fff640 1 -- 172.21.15.80:0/3926543037 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc6d801f070 con 0x7fc6e0150530 2024-04-25T13:47:07.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6c7fff640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6cc06f400 0x7fc6cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:07.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.922+0000 7fc6c7fff640 1 -- 172.21.15.80:0/3926543037 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4857+0+0 (secure 0 0 0) 0x7fc6d808f080 con 0x7fc6e0150530 2024-04-25T13:47:07.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.926+0000 7fc6df7fe640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6cc06f400 0x7fc6cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:07.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.926+0000 7fc6df7fe640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6cc06f400 0x7fc6cc071910 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00aa000 tx=0x7fc6e0160b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:07.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:07.930+0000 7fc6c7fff640 1 -- 172.21.15.80:0/3926543037 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc6d8012070 con 0x7fc6e0150530 2024-04-25T13:47:08.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.086+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fc6e006de50 con 0x7fc6e0150530 2024-04-25T13:47:08.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.086+0000 7fc6c7fff640 1 -- 172.21.15.80:0/3926543037 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v132) ==== 84+0+65 (secure 0 0 0) 0x7fc6d805c350 con 0x7fc6e0150530 2024-04-25T13:47:08.086 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:08.086 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:47:08.086 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:08.086 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:47:08.086 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:47:08.086 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:08.086 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6cc06f400 msgr2=0x7fc6cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6cc06f400 0x7fc6cc071910 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fc6d00aa000 tx=0x7fc6e0160b60 comp rx=0 tx=0).stop 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6e0150530 msgr2=0x7fc6e00b0840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6e0150530 0x7fc6e00b0840 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fc6e00d38f0 tx=0x7fc6d800f4a0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 shutdown_connections 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc6cc06f400 0x7fc6cc071910 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc6e0150530 0x7fc6e00b0840 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc6e0150120 0x7fc6e00b0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 --2- 172.21.15.80:0/3926543037 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc6c80039c0 0x7fc6e0149e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 >> 172.21.15.80:0/3926543037 conn(0x7fc6e00d2900 msgr2=0x7fc6e014e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 shutdown_connections 2024-04-25T13:47:08.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.090+0000 7fc6e5cf4640 1 -- 172.21.15.80:0/3926543037 wait complete. 2024-04-25T13:47:08.107 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:47:08.107 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:47:08.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26fcfb7640 1 Processor -- start 2024-04-25T13:47:08.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26fcfb7640 1 -- start start 2024-04-25T13:47:08.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26fcfb7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80b0f10 0x7f26f80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26fcfb7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26f80ce2a0 0x7f26f80ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26fcfb7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26f80b1320 con 0x7f26f80cf730 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26fcfb7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f26f80b5820 con 0x7f26f80ce2a0 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26fcfb7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26f80d13f0 con 0x7f26f80b0f10 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26effff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26f80ce2a0 0x7f26f80ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26effff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26f80ce2a0 0x7f26f80ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38240/0 (socket says 172.21.15.80:38240) 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.246+0000 7f26effff640 1 -- 172.21.15.80:0/493153847 learned_addr learned my addr 172.21.15.80:0/493153847 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26f659c640 1 --2- 172.21.15.80:0/493153847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80b0f10 0x7f26f80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26effff640 1 -- 172.21.15.80:0/493153847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80b0f10 msgr2=0x7f26f80d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26effff640 1 --2- 172.21.15.80:0/493153847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80b0f10 0x7f26f80d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26effff640 1 -- 172.21.15.80:0/493153847 >> v1:172.21.15.87:6789/0 conn(0x7f26f80cf730 legacy=0x7f26f80cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:08.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26effff640 1 -- 172.21.15.80:0/493153847 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f80cebf0 con 0x7f26f80ce2a0 2024-04-25T13:47:08.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26f659c640 1 --2- 172.21.15.80:0/493153847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80b0f10 0x7f26f80d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:08.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26effff640 1 --2- 172.21.15.80:0/493153847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80ce2a0 0x7f26f80ce6b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009870 tx=0x7f26e001c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdeae3e03884e275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:08.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26f5d9b640 1 -- 172.21.15.80:0/493153847 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00b0070 con 0x7f26f80ce2a0 2024-04-25T13:47:08.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26f5d9b640 1 -- 172.21.15.80:0/493153847 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e00ab600 con 0x7f26f80ce2a0 2024-04-25T13:47:08.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26f5d9b640 1 -- 172.21.15.80:0/493153847 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e00ab9d0 con 0x7f26f80ce2a0 2024-04-25T13:47:08.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26fcfb7640 1 -- 172.21.15.80:0/493153847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80ce2a0 msgr2=0x7f26f80ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/493153847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80ce2a0 0x7f26f80ce6b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f26e0009870 tx=0x7f26e001c340 comp rx=0 tx=0).stop 2024-04-25T13:47:08.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26fcfb7640 1 -- 172.21.15.80:0/493153847 shutdown_connections 2024-04-25T13:47:08.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/493153847 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80ce2a0 0x7f26f80ce6b0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/493153847 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80b0f10 0x7f26f80d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26fcfb7640 1 -- 172.21.15.80:0/493153847 >> 172.21.15.80:0/493153847 conn(0x7f26f80b35c0 msgr2=0x7f26f80b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:08.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26fcfb7640 1 -- 172.21.15.80:0/493153847 shutdown_connections 2024-04-25T13:47:08.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.250+0000 7f26fcfb7640 1 -- 172.21.15.80:0/493153847 wait complete. 2024-04-25T13:47:08.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 Processor -- start 2024-04-25T13:47:08.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 -- start start 2024-04-25T13:47:08.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b0f10 0x7f26f81fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80ce2a0 0x7f26f80b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26f80cf730 0x7f26f814f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80b5300 con 0x7f26f80b0f10 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f26f80d1150 con 0x7f26f80cf730 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f26f815e720 con 0x7f26f80ce2a0 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b0f10 0x7f26f81fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b0f10 0x7f26f81fbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38254/0 (socket says 172.21.15.80:38254) 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 -- 172.21.15.80:0/4076142802 learned_addr learned my addr 172.21.15.80:0/4076142802 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26effff640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80ce2a0 0x7f26f80b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f6d9d640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26f80cf730 0x7f26f814f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 -- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80ce2a0 msgr2=0x7f26f80b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80ce2a0 0x7f26f80b2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 -- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26f80cf730 msgr2=0x7f26f814f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26f80cf730 0x7f26f814f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 -- 172.21.15.80:0/4076142802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f81fc160 con 0x7f26f80b0f10 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f6d9d640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26f80cf730 0x7f26f814f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26f659c640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b0f10 0x7f26f81fbc20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c820 tx=0x7f26e800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26eeffd640 1 -- 172.21.15.80:0/4076142802 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8015070 con 0x7f26f80b0f10 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26eeffd640 1 -- 172.21.15.80:0/4076142802 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26e8007a30 con 0x7f26f80b0f10 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f8159960 con 0x7f26f80b0f10 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26eeffd640 1 -- 172.21.15.80:0/4076142802 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e8007e20 con 0x7f26f80b0f10 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.254+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f8156c60 con 0x7f26f80b0f10 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.258+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f8159b30 con 0x7f26f80b0f10 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.258+0000 7f26eeffd640 1 -- 172.21.15.80:0/4076142802 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f26e8018070 con 0x7f26f80b0f10 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.258+0000 7f26eeffd640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f26d806f430 0x7f26d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.258+0000 7f26eeffd640 1 -- 172.21.15.80:0/4076142802 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4857+0+0 (secure 0 0 0) 0x7f26e808ed90 con 0x7f26f80b0f10 2024-04-25T13:47:08.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.258+0000 7f26effff640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f26d806f430 0x7f26d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.258+0000 7f26effff640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f26d806f430 0x7f26d8071940 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26e00ad000 tx=0x7f26e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:08.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.262+0000 7f26eeffd640 1 -- 172.21.15.80:0/4076142802 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f26f81fc160 con 0x7f26f80b0f10 2024-04-25T13:47:08.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.414+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f26f806a440 con 0x7f26f80b0f10 2024-04-25T13:47:08.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.646+0000 7f26eeffd640 1 -- 172.21.15.80:0/4076142802 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v133) ==== 138+0+0 (secure 0 0 0) 0x7f26e805c060 con 0x7f26f80b0f10 2024-04-25T13:47:08.645 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f26d806f430 msgr2=0x7f26d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f26d806f430 0x7f26d8071940 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f26e00ad000 tx=0x7f26e0008040 comp rx=0 tx=0).stop 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b0f10 msgr2=0x7f26f81fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b0f10 0x7f26f81fbc20 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c820 tx=0x7f26e800cce0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 shutdown_connections 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f26d806f430 0x7f26d8071940 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26f80cf730 0x7f26f814f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f26f80ce2a0 0x7f26f80b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 --2- 172.21.15.80:0/4076142802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26f80b0f10 0x7f26f81fbc20 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 >> 172.21.15.80:0/4076142802 conn(0x7f26f80b35c0 msgr2=0x7f26f8161370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:08.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 shutdown_connections 2024-04-25T13:47:08.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.650+0000 7f26fcfb7640 1 -- 172.21.15.80:0/4076142802 wait complete. 2024-04-25T13:47:08.674 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:47:08.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.826+0000 7fbcc890b640 1 Processor -- start 2024-04-25T13:47:08.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.826+0000 7fbcc890b640 1 -- start start 2024-04-25T13:47:08.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.826+0000 7fbcc890b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbcc4161f90 0x7fbcc40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 0x7fbcc415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbcc415ea70 0x7fbcc415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc40d0d70 con 0x7fbcc415ea70 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbcc40d1050 con 0x7fbcc4161f90 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc406dca0 con 0x7fbcc40d0510 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbcc415ea70 0x7fbcc415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbcc415ea70 0x7fbcc415ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38266/0 (socket says 172.21.15.80:38266) 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 -- 172.21.15.80:0/2647577932 learned_addr learned my addr 172.21.15.80:0/2647577932 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc1d9b640 1 --2- 172.21.15.80:0/2647577932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 0x7fbcc415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc259c640 1 --2- 172.21.15.80:0/2647577932 >> v2:172.21.15.87:3300/0 conn(0x7fbcc4161f90 0x7fbcc40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 -- 172.21.15.80:0/2647577932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 msgr2=0x7fbcc415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:08.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 --2- 172.21.15.80:0/2647577932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 0x7fbcc415e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 -- 172.21.15.80:0/2647577932 >> v2:172.21.15.87:3300/0 conn(0x7fbcc4161f90 msgr2=0x7fbcc40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:08.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 --2- 172.21.15.80:0/2647577932 >> v2:172.21.15.87:3300/0 conn(0x7fbcc4161f90 0x7fbcc40cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 -- 172.21.15.80:0/2647577932 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc406df80 con 0x7fbcc415ea70 2024-04-25T13:47:08.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc259c640 1 --2- 172.21.15.80:0/2647577932 >> v2:172.21.15.87:3300/0 conn(0x7fbcc4161f90 0x7fbcc40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:08.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc2d9d640 1 --2- 172.21.15.80:0/2647577932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc415ee80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40d17b0 tx=0x7fbcbc01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c3e223bf5ae001 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:08.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc159a640 1 -- 172.21.15.80:0/2647577932 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0a5770 con 0x7fbcc415ea70 2024-04-25T13:47:08.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc159a640 1 -- 172.21.15.80:0/2647577932 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcbc0a5910 con 0x7fbcc415ea70 2024-04-25T13:47:08.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc159a640 1 -- 172.21.15.80:0/2647577932 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0a5ce0 con 0x7fbcc415ea70 2024-04-25T13:47:08.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 -- 172.21.15.80:0/2647577932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 msgr2=0x7fbcc415ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 --2- 172.21.15.80:0/2647577932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc415ee80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40d17b0 tx=0x7fbcbc01c930 comp rx=0 tx=0).stop 2024-04-25T13:47:08.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 -- 172.21.15.80:0/2647577932 shutdown_connections 2024-04-25T13:47:08.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 --2- 172.21.15.80:0/2647577932 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc415ee80 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 --2- 172.21.15.80:0/2647577932 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 0x7fbcc415e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 --2- 172.21.15.80:0/2647577932 >> v2:172.21.15.87:3300/0 conn(0x7fbcc4161f90 0x7fbcc40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 -- 172.21.15.80:0/2647577932 >> 172.21.15.80:0/2647577932 conn(0x7fbcc406d080 msgr2=0x7fbcc40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:08.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 -- 172.21.15.80:0/2647577932 shutdown_connections 2024-04-25T13:47:08.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.830+0000 7fbcc890b640 1 -- 172.21.15.80:0/2647577932 wait complete. 2024-04-25T13:47:08.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 Processor -- start 2024-04-25T13:47:08.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 -- start start 2024-04-25T13:47:08.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 0x7fbcc4167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc41fa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc4161f90 0x7fbcc41fa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc41447a0 con 0x7fbcc415ea70 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbcc4144620 con 0x7fbcc4161f90 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbcc40d0920 con 0x7fbcc40d0510 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc41fa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc41fa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38278/0 (socket says 172.21.15.80:38278) 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 -- 172.21.15.80:0/3842420669 learned_addr learned my addr 172.21.15.80:0/3842420669 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc259c640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 0x7fbcc4167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc2d9d640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc4161f90 0x7fbcc41fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 -- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 msgr2=0x7fbcc4167ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 0x7fbcc4167ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 -- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc4161f90 msgr2=0x7fbcc41fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:08.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc4161f90 0x7fbcc41fa940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:08.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 -- 172.21.15.80:0/3842420669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcc41fb7a0 con 0x7fbcc415ea70 2024-04-25T13:47:08.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc1d9b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc41fa1e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800d830 tx=0x7fbcb800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:08.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcb2ffd640 1 -- 172.21.15.80:0/3842420669 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8018030 con 0x7fbcc415ea70 2024-04-25T13:47:08.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcb2ffd640 1 -- 172.21.15.80:0/3842420669 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbcb800be10 con 0x7fbcc415ea70 2024-04-25T13:47:08.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcc4145970 con 0x7fbcc415ea70 2024-04-25T13:47:08.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcb2ffd640 1 -- 172.21.15.80:0/3842420669 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcb80098b0 con 0x7fbcc415ea70 2024-04-25T13:47:08.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.834+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcc4166180 con 0x7fbcc415ea70 2024-04-25T13:47:08.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.838+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcc4165ef0 con 0x7fbcc415ea70 2024-04-25T13:47:08.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.838+0000 7fbcb2ffd640 1 -- 172.21.15.80:0/3842420669 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbcb801b070 con 0x7fbcc415ea70 2024-04-25T13:47:08.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.838+0000 7fbcb2ffd640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbca406f400 0x7fbca4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:08.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.838+0000 7fbcb2ffd640 1 -- 172.21.15.80:0/3842420669 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5227+0+0 (secure 0 0 0) 0x7fbcb808f560 con 0x7fbcc415ea70 2024-04-25T13:47:08.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.838+0000 7fbcc259c640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbca406f400 0x7fbca4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:08.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.838+0000 7fbcc259c640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbca406f400 0x7fbca4071910 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4006fd0 tx=0x7fbcb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:08.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.842+0000 7fbcb2ffd640 1 -- 172.21.15.80:0/3842420669 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbcc41fb7a0 con 0x7fbcc415ea70 2024-04-25T13:47:08.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:08.990+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbcc406a3a0 con 0x7fbcc415ea70 2024-04-25T13:47:09.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.650+0000 7fbcb2ffd640 1 -- 172.21.15.80:0/3842420669 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v134) ==== 170+0+0 (secure 0 0 0) 0x7fbcb805c7a0 con 0x7fbcc415ea70 2024-04-25T13:47:09.650 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbca406f400 msgr2=0x7fbca4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbca406f400 0x7fbca4071910 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4006fd0 tx=0x7fbcb4008040 comp rx=0 tx=0).stop 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 msgr2=0x7fbcc41fa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc41fa1e0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fbcb800d830 tx=0x7fbcb800dcf0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 shutdown_connections 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbca406f400 0x7fbca4071910 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbcc4161f90 0x7fbcc41fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbcc415ea70 0x7fbcc41fa1e0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 --2- 172.21.15.80:0/3842420669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbcc40d0510 0x7fbcc4167ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 >> 172.21.15.80:0/3842420669 conn(0x7fbcc406d080 msgr2=0x7fbcc4168120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 shutdown_connections 2024-04-25T13:47:09.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.654+0000 7fbcc890b640 1 -- 172.21.15.80:0/3842420669 wait complete. 2024-04-25T13:47:09.686 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:47:09.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fceee13d640 1 Processor -- start 2024-04-25T13:47:09.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fceee13d640 1 -- start start 2024-04-25T13:47:09.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fceee13d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80b5330 0x7fcee80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:09.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fceee13d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcee80d0ff0 0x7fcee806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:09.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fceee13d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcee80b5740 con 0x7fcee806f2f0 2024-04-25T13:47:09.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fceee13d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fcee806f700 con 0x7fcee80d0ff0 2024-04-25T13:47:09.838 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fceee13d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcee81629b0 con 0x7fcee80b5330 2024-04-25T13:47:09.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fcee6ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcee80d0ff0 0x7fcee806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:09.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fcee6ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fcee80d0ff0 0x7fcee806edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38280/0 (socket says 172.21.15.80:38280) 2024-04-25T13:47:09.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fcee6ffd640 1 -- 172.21.15.80:0/2818968709 learned_addr learned my addr 172.21.15.80:0/2818968709 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:09.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fcee77fe640 1 --2- 172.21.15.80:0/2818968709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80b5330 0x7fcee80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:09.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fcee6ffd640 1 -- 172.21.15.80:0/2818968709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80b5330 msgr2=0x7fcee80d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:09.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fcee6ffd640 1 --2- 172.21.15.80:0/2818968709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80b5330 0x7fcee80d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fcee6ffd640 1 -- 172.21.15.80:0/2818968709 >> v1:172.21.15.87:6789/0 conn(0x7fcee806f2f0 legacy=0x7fcee8161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:09.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.838+0000 7fcee6ffd640 1 -- 172.21.15.80:0/2818968709 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee806dd10 con 0x7fcee80d0ff0 2024-04-25T13:47:09.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fcee6ffd640 1 --2- 172.21.15.80:0/2818968709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee80d0ff0 0x7fcee806edb0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fced0009e50 tx=0x7fced001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6718610cc6b7251b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:09.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fcee67fc640 1 -- 172.21.15.80:0/2818968709 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00a5750 con 0x7fcee80d0ff0 2024-04-25T13:47:09.840 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fcee67fc640 1 -- 172.21.15.80:0/2818968709 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fced00a58f0 con 0x7fcee80d0ff0 2024-04-25T13:47:09.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fcee67fc640 1 -- 172.21.15.80:0/2818968709 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fced00a5cc0 con 0x7fcee80d0ff0 2024-04-25T13:47:09.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fceee13d640 1 -- 172.21.15.80:0/2818968709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee80d0ff0 msgr2=0x7fcee806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:09.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fceee13d640 1 --2- 172.21.15.80:0/2818968709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee80d0ff0 0x7fcee806edb0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fced0009e50 tx=0x7fced001c9a0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fceee13d640 1 -- 172.21.15.80:0/2818968709 shutdown_connections 2024-04-25T13:47:09.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fceee13d640 1 --2- 172.21.15.80:0/2818968709 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee80d0ff0 0x7fcee806edb0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fceee13d640 1 --2- 172.21.15.80:0/2818968709 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80b5330 0x7fcee80d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fceee13d640 1 -- 172.21.15.80:0/2818968709 >> 172.21.15.80:0/2818968709 conn(0x7fcee80b4160 msgr2=0x7fcee80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:09.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fceee13d640 1 -- 172.21.15.80:0/2818968709 shutdown_connections 2024-04-25T13:47:09.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.842+0000 7fceee13d640 1 -- 172.21.15.80:0/2818968709 wait complete. 2024-04-25T13:47:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 Processor -- start 2024-04-25T13:47:09.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 -- start start 2024-04-25T13:47:09.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee806f2f0 0x7fcee8143eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:09.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcee80b5330 0x7fcee815d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:09.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80d0ff0 0x7fcee815a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:09.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcee80b0cd0 con 0x7fcee806f2f0 2024-04-25T13:47:09.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcee806e890 con 0x7fcee80b5330 2024-04-25T13:47:09.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcee80b0e50 con 0x7fcee80d0ff0 2024-04-25T13:47:09.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee806f2f0 0x7fcee8143eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee806f2f0 0x7fcee8143eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38290/0 (socket says 172.21.15.80:38290) 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 -- 172.21.15.80:0/2336526491 learned_addr learned my addr 172.21.15.80:0/2336526491 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee6ffd640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcee80b5330 0x7fcee815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee7fff640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80d0ff0 0x7fcee815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 -- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80d0ff0 msgr2=0x7fcee815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80d0ff0 0x7fcee815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 -- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcee80b5330 msgr2=0x7fcee815d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcee80b5330 0x7fcee815d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 -- 172.21.15.80:0/2336526491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcee815e310 con 0x7fcee806f2f0 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee7fff640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80d0ff0 0x7fcee815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee6ffd640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcee80b5330 0x7fcee815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcee77fe640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee806f2f0 0x7fcee8143eb0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fcee000d650 tx=0x7fcee000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcedffff640 1 -- 172.21.15.80:0/2336526491 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee000b6c0 con 0x7fcee806f2f0 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcedffff640 1 -- 172.21.15.80:0/2336526491 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcee0019070 con 0x7fcee806f2f0 2024-04-25T13:47:09.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fcedffff640 1 -- 172.21.15.80:0/2336526491 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee000ba80 con 0x7fcee806f2f0 2024-04-25T13:47:09.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcee815e5f0 con 0x7fcee806f2f0 2024-04-25T13:47:09.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcee815ebb0 con 0x7fcee806f2f0 2024-04-25T13:47:09.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.846+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcee806a3a0 con 0x7fcee806f2f0 2024-04-25T13:47:09.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.850+0000 7fcedffff640 1 -- 172.21.15.80:0/2336526491 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcee000bc20 con 0x7fcee806f2f0 2024-04-25T13:47:09.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.850+0000 7fcedffff640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcec806f430 0x7fcec8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:09.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.850+0000 7fcedffff640 1 -- 172.21.15.80:0/2336526491 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5609+0+0 (secure 0 0 0) 0x7fcee008ffa0 con 0x7fcee806f2f0 2024-04-25T13:47:09.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.850+0000 7fcee6ffd640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcec806f430 0x7fcec8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:09.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.850+0000 7fcee6ffd640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcec806f430 0x7fcec8071940 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fced00ae000 tx=0x7fced00a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:09.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:09.854+0000 7fcedffff640 1 -- 172.21.15.80:0/2336526491 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcee005d060 con 0x7fcee806f2f0 2024-04-25T13:47:10.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.050+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fcee806fd00 con 0x7fcee806f2f0 2024-04-25T13:47:10.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.718+0000 7fcedffff640 1 -- 172.21.15.80:0/2336526491 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v64) ==== 174+0+0 (secure 0 0 0) 0x7fcee005c960 con 0x7fcee806f2f0 2024-04-25T13:47:10.718 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T13:47:10.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcec806f430 msgr2=0x7fcec8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:10.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcec806f430 0x7fcec8071940 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fced00ae000 tx=0x7fced00a7070 comp rx=0 tx=0).stop 2024-04-25T13:47:10.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee806f2f0 msgr2=0x7fcee8143eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee806f2f0 0x7fcee8143eb0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fcee000d650 tx=0x7fcee000db10 comp rx=0 tx=0).stop 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 shutdown_connections 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcec806f430 0x7fcec8071940 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcee80d0ff0 0x7fcee815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcee80b5330 0x7fcee815d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 --2- 172.21.15.80:0/2336526491 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcee806f2f0 0x7fcee8143eb0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 >> 172.21.15.80:0/2336526491 conn(0x7fcee80b4160 msgr2=0x7fcee80b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.722+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 shutdown_connections 2024-04-25T13:47:10.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.726+0000 7fceee13d640 1 -- 172.21.15.80:0/2336526491 wait complete. 2024-04-25T13:47:10.755 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:10.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.898+0000 7fb3af470640 1 Processor -- start 2024-04-25T13:47:10.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3af470640 1 -- start start 2024-04-25T13:47:10.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3af470640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3a81622d0 0x7fb3a80725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:10.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3af470640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806eb40 0x7fb3a806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:10.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3af470640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3a8072b00 con 0x7fb3a806f490 2024-04-25T13:47:10.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3af470640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb3a8072120 con 0x7fb3a81622d0 2024-04-25T13:47:10.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3af470640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a806dd80 con 0x7fb3a806eb40 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3aca0b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806eb40 0x7fb3a806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3ad20c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb3a81622d0 0x7fb3a80725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3aca0b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806eb40 0x7fb3a806ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:51130/0 (socket says 172.21.15.80:51130) 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3ada0d640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb3a806f490 0x7fb3a8164c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:54266/0 (socket says 172.21.15.80:54266) 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3aca0b640 1 -- 172.21.15.80:0/2906788266 learned_addr learned my addr 172.21.15.80:0/2906788266 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3aca0b640 1 -- 172.21.15.80:0/2906788266 >> v2:172.21.15.87:3300/0 conn(0x7fb3a81622d0 msgr2=0x7fb3a80725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3aca0b640 1 --2- 172.21.15.80:0/2906788266 >> v2:172.21.15.87:3300/0 conn(0x7fb3a81622d0 0x7fb3a80725c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3aca0b640 1 -- 172.21.15.80:0/2906788266 >> v1:172.21.15.87:6789/0 conn(0x7fb3a806f490 legacy=0x7fb3a8164c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:10.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.902+0000 7fb3aca0b640 1 -- 172.21.15.80:0/2906788266 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a806e060 con 0x7fb3a806eb40 2024-04-25T13:47:10.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3aca0b640 1 --2- 172.21.15.80:0/2906788266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806eb40 0x7fb3a806ef50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009870 tx=0x7fb3a401c440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=192b51a0714b155d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:10.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb39bfff640 1 -- 172.21.15.80:0/2906788266 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40ab660 con 0x7fb3a806eb40 2024-04-25T13:47:10.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb39bfff640 1 -- 172.21.15.80:0/2906788266 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a40b0070 con 0x7fb3a806eb40 2024-04-25T13:47:10.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb39bfff640 1 -- 172.21.15.80:0/2906788266 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40aba20 con 0x7fb3a806eb40 2024-04-25T13:47:10.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3af470640 1 -- 172.21.15.80:0/2906788266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806eb40 msgr2=0x7fb3a806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:10.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3af470640 1 --2- 172.21.15.80:0/2906788266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806eb40 0x7fb3a806ef50 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb3a4009870 tx=0x7fb3a401c440 comp rx=0 tx=0).stop 2024-04-25T13:47:10.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3af470640 1 -- 172.21.15.80:0/2906788266 shutdown_connections 2024-04-25T13:47:10.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3af470640 1 --2- 172.21.15.80:0/2906788266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806eb40 0x7fb3a806ef50 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3af470640 1 --2- 172.21.15.80:0/2906788266 >> v2:172.21.15.87:3300/0 conn(0x7fb3a81622d0 0x7fb3a80725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3af470640 1 -- 172.21.15.80:0/2906788266 >> 172.21.15.80:0/2906788266 conn(0x7fb3a806d080 msgr2=0x7fb3a80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:10.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3af470640 1 -- 172.21.15.80:0/2906788266 shutdown_connections 2024-04-25T13:47:10.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.906+0000 7fb3af470640 1 -- 172.21.15.80:0/2906788266 wait complete. 2024-04-25T13:47:10.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 Processor -- start 2024-04-25T13:47:10.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 -- start start 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a806eb40 0x7fb3a80ac300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806f490 0x7fb3a8156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3a81622d0 0x7fb3a815df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a8171740 con 0x7fb3a81622d0 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a81715c0 con 0x7fb3a806eb40 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3a81718c0 con 0x7fb3a806f490 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3a81622d0 0x7fb3a815df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3a81622d0 0x7fb3a815df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38292/0 (socket says 172.21.15.80:38292) 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 -- 172.21.15.80:0/3806331672 learned_addr learned my addr 172.21.15.80:0/3806331672 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3aca0b640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806f490 0x7fb3a8156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ad20c640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a806eb40 0x7fb3a80ac300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:10.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 -- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806f490 msgr2=0x7fb3a8156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806f490 0x7fb3a8156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 -- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a806eb40 msgr2=0x7fb3a80ac300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a806eb40 0x7fb3a80ac300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 -- 172.21.15.80:0/3806331672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a8157d20 con 0x7fb3a81622d0 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3aca0b640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806f490 0x7fb3a8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3ada0d640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3a81622d0 0x7fb3a815df90 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000c3f0 tx=0x7fb3a000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb399ffb640 1 -- 172.21.15.80:0/3806331672 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a00074e0 con 0x7fb3a81622d0 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb399ffb640 1 -- 172.21.15.80:0/3806331672 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3a0007680 con 0x7fb3a81622d0 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a8157fc0 con 0x7fb3a81622d0 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb399ffb640 1 -- 172.21.15.80:0/3806331672 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a0004a30 con 0x7fb3a81622d0 2024-04-25T13:47:10.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.910+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a80ac970 con 0x7fb3a81622d0 2024-04-25T13:47:10.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.914+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a806a3a0 con 0x7fb3a81622d0 2024-04-25T13:47:10.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.914+0000 7fb399ffb640 1 -- 172.21.15.80:0/3806331672 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb3a001a020 con 0x7fb3a81622d0 2024-04-25T13:47:10.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.914+0000 7fb399ffb640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb38c06f430 0x7fb38c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:10.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.914+0000 7fb399ffb640 1 -- 172.21.15.80:0/3806331672 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5709+0+0 (secure 0 0 0) 0x7fb3a008e8f0 con 0x7fb3a81622d0 2024-04-25T13:47:10.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.914+0000 7fb3ad20c640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb38c06f430 0x7fb38c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:10.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.914+0000 7fb3ad20c640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb38c06f430 0x7fb38c071940 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0073f0 tx=0x7fb39c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:10.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:10.918+0000 7fb399ffb640 1 -- 172.21.15.80:0/3806331672 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb3a005b900 con 0x7fb3a81622d0 2024-04-25T13:47:11.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.078+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3a815ebf0 con 0x7fb3a81622d0 2024-04-25T13:47:11.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb399ffb640 1 -- 172.21.15.80:0/3806331672 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+16569 (secure 0 0 0) 0x7fb3a005b200 con 0x7fb3a81622d0 2024-04-25T13:47:11.080 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:11.080 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":135,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:10.055050+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:47:08.431048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T13:47:09.003237+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb38c06f430 msgr2=0x7fb38c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb38c06f430 0x7fb38c071940 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0073f0 tx=0x7fb39c00b040 comp rx=0 tx=0).stop 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3a81622d0 msgr2=0x7fb3a815df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3a81622d0 0x7fb3a815df90 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fb3a000c3f0 tx=0x7fb3a000c8b0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 shutdown_connections 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb38c06f430 0x7fb38c071940 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3a81622d0 0x7fb3a815df90 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb3a806f490 0x7fb3a8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 --2- 172.21.15.80:0/3806331672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb3a806eb40 0x7fb3a80ac300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 >> 172.21.15.80:0/3806331672 conn(0x7fb3a806d080 msgr2=0x7fb3a80b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:11.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 shutdown_connections 2024-04-25T13:47:11.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.082+0000 7fb3af470640 1 -- 172.21.15.80:0/3806331672 wait complete. 2024-04-25T13:47:11.103 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:11.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4ea14a9640 1 Processor -- start 2024-04-25T13:47:11.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4ea14a9640 1 -- start start 2024-04-25T13:47:11.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4ea14a9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4e9c076cb0 0x7f4e9c0d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4ea14a9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c0d15a0 0x7f4e9c06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4ea14a9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4e9c06eb20 0x7f4e9c0b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4ea14a9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4e9c06ef30 con 0x7f4e9c06eb20 2024-04-25T13:47:11.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4ea14a9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4e9c0cca70 con 0x7f4e9c076cb0 2024-04-25T13:47:11.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4ea14a9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9c0ce4b0 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4e9b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4e9c06eb20 0x7f4e9c0b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4e9b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4e9c06eb20 0x7f4e9c0b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52906/0 (socket says 172.21.15.80:52906) 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4e9b7fe640 1 -- 172.21.15.80:0/314921258 learned_addr learned my addr 172.21.15.80:0/314921258 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.250+0000 7f4e9affd640 1 --2- 172.21.15.80:0/314921258 >> v2:172.21.15.87:3300/0 conn(0x7f4e9c076cb0 0x7f4e9c0d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e9a7fc640 1 --2- 172.21.15.80:0/314921258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c0d15a0 0x7f4e9c06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e9b7fe640 1 -- 172.21.15.80:0/314921258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c0d15a0 msgr2=0x7f4e9c06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e9b7fe640 1 --2- 172.21.15.80:0/314921258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c0d15a0 0x7f4e9c06e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e9b7fe640 1 -- 172.21.15.80:0/314921258 >> v2:172.21.15.87:3300/0 conn(0x7f4e9c076cb0 msgr2=0x7f4e9c0d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e9b7fe640 1 --2- 172.21.15.80:0/314921258 >> v2:172.21.15.87:3300/0 conn(0x7f4e9c076cb0 0x7f4e9c0d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e9b7fe640 1 -- 172.21.15.80:0/314921258 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c0ce790 con 0x7f4e9c06eb20 2024-04-25T13:47:11.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e9a7fc640 1 --2- 172.21.15.80:0/314921258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c0d15a0 0x7f4e9c06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:11.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e9b7fe640 1 --2- 172.21.15.80:0/314921258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c06eb20 0x7f4e9c0b0ee0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ceab0 tx=0x7f4e9401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ad0996b7f2253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:11.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e99ffb640 1 -- 172.21.15.80:0/314921258 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940a55e0 con 0x7f4e9c06eb20 2024-04-25T13:47:11.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e99ffb640 1 -- 172.21.15.80:0/314921258 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e940a5780 con 0x7f4e9c06eb20 2024-04-25T13:47:11.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4e99ffb640 1 -- 172.21.15.80:0/314921258 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940a5b50 con 0x7f4e9c06eb20 2024-04-25T13:47:11.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 -- 172.21.15.80:0/314921258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c06eb20 msgr2=0x7f4e9c0b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/314921258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c06eb20 0x7f4e9c0b0ee0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0ceab0 tx=0x7f4e9401c960 comp rx=0 tx=0).stop 2024-04-25T13:47:11.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 -- 172.21.15.80:0/314921258 shutdown_connections 2024-04-25T13:47:11.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/314921258 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c06eb20 0x7f4e9c0b0ee0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/314921258 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c0d15a0 0x7f4e9c06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/314921258 >> v2:172.21.15.87:3300/0 conn(0x7f4e9c076cb0 0x7f4e9c0d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 -- 172.21.15.80:0/314921258 >> 172.21.15.80:0/314921258 conn(0x7f4e9c0d20d0 msgr2=0x7f4e9c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:11.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 -- 172.21.15.80:0/314921258 shutdown_connections 2024-04-25T13:47:11.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.254+0000 7f4ea14a9640 1 -- 172.21.15.80:0/314921258 wait complete. 2024-04-25T13:47:11.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 Processor -- start 2024-04-25T13:47:11.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 -- start start 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c06eb20 0x7f4e9c0c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e9c076cb0 0x7f4e9c14e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c0d15a0 0x7f4e9c155a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c161250 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4e9c1610d0 con 0x7f4e9c076cb0 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4e9c1613d0 con 0x7f4e9c06eb20 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c0d15a0 0x7f4e9c155a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c0d15a0 0x7f4e9c155a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52920/0 (socket says 172.21.15.80:52920) 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 -- 172.21.15.80:0/131215472 learned_addr learned my addr 172.21.15.80:0/131215472 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9a7fc640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e9c076cb0 0x7f4e9c14e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9affd640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c06eb20 0x7f4e9c0c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 -- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c06eb20 msgr2=0x7f4e9c0c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c06eb20 0x7f4e9c0c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 -- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e9c076cb0 msgr2=0x7f4e9c14e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e9c076cb0 0x7f4e9c14e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 -- 172.21.15.80:0/131215472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e9c150010 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e9b7fe640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c0d15a0 0x7f4e9c155a70 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4e940af040 tx=0x7f4e9c1618d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e837fe640 1 -- 172.21.15.80:0/131215472 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94009510 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e837fe640 1 -- 172.21.15.80:0/131215472 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4e940b8470 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e9c14ecd0 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4e837fe640 1 -- 172.21.15.80:0/131215472 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e940b8860 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.258+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e9c152b40 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.262+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e9c06a3a0 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.262+0000 7f4e837fe640 1 -- 172.21.15.80:0/131215472 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4e940c0030 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.262+0000 7f4e837fe640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8806f430 0x7f4e88071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.262+0000 7f4e837fe640 1 -- 172.21.15.80:0/131215472 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5709+0+0 (secure 0 0 0) 0x7f4e94129640 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.262+0000 7f4e9affd640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8806f430 0x7f4e88071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.262+0000 7f4e9affd640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8806f430 0x7f4e88071940 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c051890 tx=0x7f4e8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:11.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.266+0000 7f4e837fe640 1 -- 172.21.15.80:0/131215472 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4e940165e0 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.430+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e9c0b1200 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.430+0000 7f4e837fe640 1 -- 172.21.15.80:0/131215472 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) ==== 74+0+16569 (secure 0 0 0) 0x7f4e940f65c0 con 0x7f4e9c0d15a0 2024-04-25T13:47:11.430 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:11.430 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":135,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:10.055050+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:47:08.431048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T13:47:09.003237+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8806f430 msgr2=0x7f4e88071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8806f430 0x7f4e88071940 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c051890 tx=0x7f4e8c008040 comp rx=0 tx=0).stop 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c0d15a0 msgr2=0x7f4e9c155a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c0d15a0 0x7f4e9c155a70 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f4e940af040 tx=0x7f4e9c1618d0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 shutdown_connections 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4e8806f430 0x7f4e88071940 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4e9c0d15a0 0x7f4e9c155a70 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4e9c076cb0 0x7f4e9c14e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 --2- 172.21.15.80:0/131215472 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4e9c06eb20 0x7f4e9c0c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 >> 172.21.15.80:0/131215472 conn(0x7f4e9c0d20d0 msgr2=0x7f4e9c0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 shutdown_connections 2024-04-25T13:47:11.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.434+0000 7f4ea14a9640 1 -- 172.21.15.80:0/131215472 wait complete. 2024-04-25T13:47:11.453 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:47:11.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa813b91640 1 Processor -- start 2024-04-25T13:47:11.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa813b91640 1 -- start start 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa813b91640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa80c0b5330 0x7fa80c0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa813b91640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa80c0d0ff0 0x7fa80c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa813b91640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c06f2f0 0x7fa80c161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa813b91640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa80c06f700 con 0x7fa80c0b5330 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa813b91640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa80c1629b0 con 0x7fa80c0d0ff0 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa813b91640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa80c0b5740 con 0x7fa80c06f2f0 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81192d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa80c0b5330 0x7fa80c0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81192d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa80c0b5330 0x7fa80c0d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52934/0 (socket says 172.21.15.80:52934) 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81192d640 1 -- 172.21.15.80:0/179057662 learned_addr learned my addr 172.21.15.80:0/179057662 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81112c640 1 --2- 172.21.15.80:0/179057662 >> v2:172.21.15.87:3300/0 conn(0x7fa80c0d0ff0 0x7fa80c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81212e640 1 --2- 172.21.15.80:0/179057662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c06f2f0 0x7fa80c161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81192d640 1 -- 172.21.15.80:0/179057662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c06f2f0 msgr2=0x7fa80c161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:11.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81192d640 1 --2- 172.21.15.80:0/179057662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c06f2f0 0x7fa80c161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81192d640 1 -- 172.21.15.80:0/179057662 >> v2:172.21.15.87:3300/0 conn(0x7fa80c0d0ff0 msgr2=0x7fa80c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:11.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81192d640 1 --2- 172.21.15.80:0/179057662 >> v2:172.21.15.87:3300/0 conn(0x7fa80c0d0ff0 0x7fa80c06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81192d640 1 -- 172.21.15.80:0/179057662 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c06dd10 con 0x7fa80c0b5330 2024-04-25T13:47:11.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.610+0000 7fa81212e640 1 --2- 172.21.15.80:0/179057662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c06f2f0 0x7fa80c161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:11.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa81192d640 1 --2- 172.21.15.80:0/179057662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0d0ab0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa80c074290 tx=0x7fa7fc01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb1d466a9198388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:11.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa81092b640 1 -- 172.21.15.80:0/179057662 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0a55e0 con 0x7fa80c0b5330 2024-04-25T13:47:11.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa81092b640 1 -- 172.21.15.80:0/179057662 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7fc0a5780 con 0x7fa80c0b5330 2024-04-25T13:47:11.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa81092b640 1 -- 172.21.15.80:0/179057662 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7fc0a5b50 con 0x7fa80c0b5330 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 -- 172.21.15.80:0/179057662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 msgr2=0x7fa80c0d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 --2- 172.21.15.80:0/179057662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0d0ab0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fa80c074290 tx=0x7fa7fc01c960 comp rx=0 tx=0).stop 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 -- 172.21.15.80:0/179057662 shutdown_connections 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 --2- 172.21.15.80:0/179057662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c06f2f0 0x7fa80c161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 --2- 172.21.15.80:0/179057662 >> v2:172.21.15.87:3300/0 conn(0x7fa80c0d0ff0 0x7fa80c06edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 --2- 172.21.15.80:0/179057662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0d0ab0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 -- 172.21.15.80:0/179057662 >> 172.21.15.80:0/179057662 conn(0x7fa80c0b4160 msgr2=0x7fa80c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 -- 172.21.15.80:0/179057662 shutdown_connections 2024-04-25T13:47:11.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.614+0000 7fa813b91640 1 -- 172.21.15.80:0/179057662 wait complete. 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 Processor -- start 2024-04-25T13:47:11.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 -- start start 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80c06f2f0 0x7fa80c158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c0d0ff0 0x7fa80c157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c0b0cd0 con 0x7fa80c0b5330 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa80c06e890 con 0x7fa80c06f2f0 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa80c0b0e50 con 0x7fa80c0d0ff0 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0b02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52946/0 (socket says 172.21.15.80:52946) 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 -- 172.21.15.80:0/3407093325 learned_addr learned my addr 172.21.15.80:0/3407093325 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81212e640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c0d0ff0 0x7fa80c157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81192d640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80c06f2f0 0x7fa80c158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 -- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c0d0ff0 msgr2=0x7fa80c157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c0d0ff0 0x7fa80c157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 -- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80c06f2f0 msgr2=0x7fa80c158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80c06f2f0 0x7fa80c158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 -- 172.21.15.80:0/3407093325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80c072340 con 0x7fa80c0b5330 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81192d640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80c06f2f0 0x7fa80c158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa81112c640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0b02f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c7f0 tx=0x7fa80400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa80a7fc640 1 -- 172.21.15.80:0/3407093325 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804015070 con 0x7fa80c0b5330 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa80a7fc640 1 -- 172.21.15.80:0/3407093325 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa804007820 con 0x7fa80c0b5330 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa80c072620 con 0x7fa80c0b5330 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa80a7fc640 1 -- 172.21.15.80:0/3407093325 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa804007c10 con 0x7fa80c0b5330 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa80c072be0 con 0x7fa80c0b5330 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.618+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80c06a3a0 con 0x7fa80c0b5330 2024-04-25T13:47:11.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.622+0000 7fa80a7fc640 1 -- 172.21.15.80:0/3407093325 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa804018070 con 0x7fa80c0b5330 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.622+0000 7fa80a7fc640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa7ec06f430 0x7fa7ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.622+0000 7fa80a7fc640 1 -- 172.21.15.80:0/3407093325 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5709+0+0 (secure 0 0 0) 0x7fa80408f3c0 con 0x7fa80c0b5330 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.622+0000 7fa81192d640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa7ec06f430 0x7fa7ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:11.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.622+0000 7fa81192d640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa7ec06f430 0x7fa7ec071940 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0ae000 tx=0x7fa7fc004ac0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:11.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.626+0000 7fa80a7fc640 1 -- 172.21.15.80:0/3407093325 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa80405c2a0 con 0x7fa80c0b5330 2024-04-25T13:47:11.642 INFO:tasks.ceph.mds.a.smithi080.stderr:2024-04-25T13:47:11.642+0000 7f3275a83640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:47:11.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:11.826+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa80c0ce9d0 con 0x7fa80c0b5330 2024-04-25T13:47:12.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.702+0000 7fa80a7fc640 1 -- 172.21.15.80:0/3407093325 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v66) ==== 118+0+0 (secure 0 0 0) 0x7fa80405bba0 con 0x7fa80c0b5330 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa7ec06f430 msgr2=0x7fa7ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa7ec06f430 0x7fa7ec071940 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc0ae000 tx=0x7fa7fc004ac0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 msgr2=0x7fa80c0b02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0b02f0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fa80400c7f0 tx=0x7fa80400ccb0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 shutdown_connections 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa7ec06f430 0x7fa7ec071940 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa80c0d0ff0 0x7fa80c157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa80c0b5330 0x7fa80c0b02f0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 --2- 172.21.15.80:0/3407093325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa80c06f2f0 0x7fa80c158c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 >> 172.21.15.80:0/3407093325 conn(0x7fa80c0b4160 msgr2=0x7fa80c1451a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:12.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 shutdown_connections 2024-04-25T13:47:12.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.706+0000 7fa813b91640 1 -- 172.21.15.80:0/3407093325 wait complete. 2024-04-25T13:47:12.732 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:47:12.732 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:47:12.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe7488e4640 1 Processor -- start 2024-04-25T13:47:12.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe7488e4640 1 -- start start 2024-04-25T13:47:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe7488e4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe744071e60 0x7fe7441574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe7488e4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe744073c40 0x7fe744160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe7488e4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7441613b0 con 0x7fe744160fa0 2024-04-25T13:47:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe7488e4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe7440b1350 con 0x7fe744073c40 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe7488e4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe74406f9a0 con 0x7fe744071e60 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe741d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe744073c40 0x7fe744160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe741d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe744073c40 0x7fe744160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52950/0 (socket says 172.21.15.80:52950) 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe741d9b640 1 -- 172.21.15.80:0/356126936 learned_addr learned my addr 172.21.15.80:0/356126936 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.890+0000 7fe74259c640 1 --2- 172.21.15.80:0/356126936 >> v2:172.21.15.87:3300/0 conn(0x7fe744071e60 0x7fe7441574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe741d9b640 1 -- 172.21.15.80:0/356126936 >> v2:172.21.15.87:3300/0 conn(0x7fe744071e60 msgr2=0x7fe7441574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe741d9b640 1 --2- 172.21.15.80:0/356126936 >> v2:172.21.15.87:3300/0 conn(0x7fe744071e60 0x7fe7441574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe741d9b640 1 -- 172.21.15.80:0/356126936 >> v1:172.21.15.87:6789/0 conn(0x7fe744160fa0 legacy=0x7fe7440b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe741d9b640 1 -- 172.21.15.80:0/356126936 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74406fbf0 con 0x7fe744073c40 2024-04-25T13:47:12.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe741d9b640 1 --2- 172.21.15.80:0/356126936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744073c40 0x7fe744160a60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe744071630 tx=0x7fe72c01c300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb20a5602c13b111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:12.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe74159a640 1 -- 172.21.15.80:0/356126936 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0b0070 con 0x7fe744073c40 2024-04-25T13:47:12.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe74159a640 1 -- 172.21.15.80:0/356126936 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe72c0ab600 con 0x7fe744073c40 2024-04-25T13:47:12.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe74159a640 1 -- 172.21.15.80:0/356126936 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe72c0ab9d0 con 0x7fe744073c40 2024-04-25T13:47:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe7488e4640 1 -- 172.21.15.80:0/356126936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744073c40 msgr2=0x7fe744160a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe7488e4640 1 --2- 172.21.15.80:0/356126936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744073c40 0x7fe744160a60 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fe744071630 tx=0x7fe72c01c300 comp rx=0 tx=0).stop 2024-04-25T13:47:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe7488e4640 1 -- 172.21.15.80:0/356126936 shutdown_connections 2024-04-25T13:47:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe7488e4640 1 --2- 172.21.15.80:0/356126936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744073c40 0x7fe744160a60 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe7488e4640 1 --2- 172.21.15.80:0/356126936 >> v2:172.21.15.87:3300/0 conn(0x7fe744071e60 0x7fe7441574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe7488e4640 1 -- 172.21.15.80:0/356126936 >> 172.21.15.80:0/356126936 conn(0x7fe7440b4160 msgr2=0x7fe7440d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe7488e4640 1 -- 172.21.15.80:0/356126936 shutdown_connections 2024-04-25T13:47:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.894+0000 7fe7488e4640 1 -- 172.21.15.80:0/356126936 wait complete. 2024-04-25T13:47:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 Processor -- start 2024-04-25T13:47:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 -- start start 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe744071e60 0x7fe74415eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe744073c40 0x7fe74415f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744160fa0 0x7fe744161b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe744074050 con 0x7fe744160fa0 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe744072270 con 0x7fe744071e60 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe7440b56c0 con 0x7fe744073c40 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744160fa0 0x7fe744161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744160fa0 0x7fe744161b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52954/0 (socket says 172.21.15.80:52954) 2024-04-25T13:47:12.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 -- 172.21.15.80:0/1422829807 learned_addr learned my addr 172.21.15.80:0/1422829807 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe74259c640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe744071e60 0x7fe74415eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe741d9b640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe744073c40 0x7fe74415f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 -- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe744073c40 msgr2=0x7fe74415f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe744073c40 0x7fe74415f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 -- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe744071e60 msgr2=0x7fe74415eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe744071e60 0x7fe74415eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 -- 172.21.15.80:0/1422829807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe744151520 con 0x7fe744160fa0 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe742d9d640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744160fa0 0x7fe744161b70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00b440 tx=0x7fe73c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe73affd640 1 -- 172.21.15.80:0/1422829807 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c015020 con 0x7fe744160fa0 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe744166190 con 0x7fe744160fa0 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe73affd640 1 -- 172.21.15.80:0/1422829807 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe73c011850 con 0x7fe744160fa0 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe73affd640 1 -- 172.21.15.80:0/1422829807 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73c011c40 con 0x7fe744160fa0 2024-04-25T13:47:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.898+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7441517e0 con 0x7fe744160fa0 2024-04-25T13:47:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.902+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe74406a3a0 con 0x7fe744160fa0 2024-04-25T13:47:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.902+0000 7fe73affd640 1 -- 172.21.15.80:0/1422829807 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe73c018020 con 0x7fe744160fa0 2024-04-25T13:47:12.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.902+0000 7fe73affd640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe72406f400 0x7fe724071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.902+0000 7fe73affd640 1 -- 172.21.15.80:0/1422829807 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7fe73c08f2f0 con 0x7fe744160fa0 2024-04-25T13:47:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.902+0000 7fe74259c640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe72406f400 0x7fe724071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:12.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.902+0000 7fe74259c640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe72406f400 0x7fe724071910 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe734003fd0 tx=0x7fe7340039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:12.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:12.906+0000 7fe73affd640 1 -- 172.21.15.80:0/1422829807 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe73c01f030 con 0x7fe744160fa0 2024-04-25T13:47:13.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.102+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe74415cc80 con 0x7fe744160fa0 2024-04-25T13:47:13.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.714+0000 7fe73affd640 1 -- 172.21.15.80:0/1422829807 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v67) ==== 122+0+0 (secure 0 0 0) 0x7fe73c05c1e0 con 0x7fe744160fa0 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe72406f400 msgr2=0x7fe724071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe72406f400 0x7fe724071910 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fe734003fd0 tx=0x7fe7340039a0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744160fa0 msgr2=0x7fe744161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744160fa0 0x7fe744161b70 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fe73c00b440 tx=0x7fe73c00b900 comp rx=0 tx=0).stop 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 shutdown_connections 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe72406f400 0x7fe724071910 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe744160fa0 0x7fe744161b70 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe744073c40 0x7fe74415f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 --2- 172.21.15.80:0/1422829807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe744071e60 0x7fe74415eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 >> 172.21.15.80:0/1422829807 conn(0x7fe7440b4160 msgr2=0x7fe744156600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 shutdown_connections 2024-04-25T13:47:13.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.718+0000 7fe7488e4640 1 -- 172.21.15.80:0/1422829807 wait complete. 2024-04-25T13:47:13.743 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:47:13.744 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:13.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.886+0000 7f5135b1e640 1 Processor -- start 2024-04-25T13:47:13.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.886+0000 7f5135b1e640 1 -- start start 2024-04-25T13:47:13.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5135b1e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f51300b4eb0 0x7f51301603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:13.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5135b1e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300b7780 0x7f51300b7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:13.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5135b1e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51300b80d0 con 0x7f51300d0ab0 2024-04-25T13:47:13.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5135b1e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f51300ba400 con 0x7f51300b4eb0 2024-04-25T13:47:13.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5135b1e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51300d48a0 con 0x7f51300b7780 2024-04-25T13:47:13.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5126ffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300b7780 0x7f51300b7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:13.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f512f7fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f51300b4eb0 0x7f51301603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:13.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5126ffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300b7780 0x7f51300b7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:43730/0 (socket says 172.21.15.80:43730) 2024-04-25T13:47:13.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5126ffd640 1 -- 172.21.15.80:0/2843222914 learned_addr learned my addr 172.21.15.80:0/2843222914 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:13.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5126ffd640 1 -- 172.21.15.80:0/2843222914 >> v2:172.21.15.87:3300/0 conn(0x7f51300b4eb0 msgr2=0x7f51301603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:13.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5126ffd640 1 --2- 172.21.15.80:0/2843222914 >> v2:172.21.15.87:3300/0 conn(0x7f51300b4eb0 0x7f51301603c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5126ffd640 1 -- 172.21.15.80:0/2843222914 >> v1:172.21.15.87:6789/0 conn(0x7f51300d0ab0 legacy=0x7f51300d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:13.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5126ffd640 1 -- 172.21.15.80:0/2843222914 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51300d4b80 con 0x7f51300b7780 2024-04-25T13:47:13.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.890+0000 7f5126ffd640 1 --2- 172.21.15.80:0/2843222914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300b7780 0x7f51300b7b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5118009870 tx=0x7f511801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a09b902e07c4fd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:13.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f512effd640 1 -- 172.21.15.80:0/2843222914 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180b2070 con 0x7f51300b7780 2024-04-25T13:47:13.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f512effd640 1 -- 172.21.15.80:0/2843222914 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51180a9070 con 0x7f51300b7780 2024-04-25T13:47:13.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f512effd640 1 -- 172.21.15.80:0/2843222914 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180ad880 con 0x7f51300b7780 2024-04-25T13:47:13.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 -- 172.21.15.80:0/2843222914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300b7780 msgr2=0x7f51300b7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:13.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 --2- 172.21.15.80:0/2843222914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300b7780 0x7f51300b7b90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5118009870 tx=0x7f511801c560 comp rx=0 tx=0).stop 2024-04-25T13:47:13.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 -- 172.21.15.80:0/2843222914 shutdown_connections 2024-04-25T13:47:13.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 --2- 172.21.15.80:0/2843222914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300b7780 0x7f51300b7b90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 --2- 172.21.15.80:0/2843222914 >> v2:172.21.15.87:3300/0 conn(0x7f51300b4eb0 0x7f51301603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 -- 172.21.15.80:0/2843222914 >> 172.21.15.80:0/2843222914 conn(0x7f513006c9f0 msgr2=0x7f513006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:13.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 -- 172.21.15.80:0/2843222914 shutdown_connections 2024-04-25T13:47:13.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 -- 172.21.15.80:0/2843222914 wait complete. 2024-04-25T13:47:13.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 Processor -- start 2024-04-25T13:47:13.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.894+0000 7f5135b1e640 1 -- start start 2024-04-25T13:47:13.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51300b4eb0 0x7f51300c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:13.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51300b7780 0x7f51300cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:13.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300d0ab0 0x7f51300cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:13.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f51300d5940 con 0x7f51300b7780 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5130160900 con 0x7f51300b4eb0 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5130162c60 con 0x7f51300d0ab0 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51300b7780 0x7f51300cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51300b7780 0x7f51300cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52960/0 (socket says 172.21.15.80:52960) 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 -- 172.21.15.80:0/3733304301 learned_addr learned my addr 172.21.15.80:0/3733304301 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f512f7fe640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51300b4eb0 0x7f51300c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f512ffff640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300d0ab0 0x7f51300cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 -- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300d0ab0 msgr2=0x7f51300cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300d0ab0 0x7f51300cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 -- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51300b4eb0 msgr2=0x7f51300c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51300b4eb0 0x7f51300c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 -- 172.21.15.80:0/3733304301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51300cf930 con 0x7f51300b7780 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5126ffd640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51300b7780 0x7f51300cba90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f51180b0040 tx=0x7f5118019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:13.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f512cff9640 1 -- 172.21.15.80:0/3733304301 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180bb070 con 0x7f51300b7780 2024-04-25T13:47:13.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f512cff9640 1 -- 172.21.15.80:0/3733304301 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51180add80 con 0x7f51300b7780 2024-04-25T13:47:13.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51300cfc10 con 0x7f51300b7780 2024-04-25T13:47:13.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f512cff9640 1 -- 172.21.15.80:0/3733304301 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51180b2040 con 0x7f51300b7780 2024-04-25T13:47:13.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5130151280 con 0x7f51300b7780 2024-04-25T13:47:13.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5130150d90 con 0x7f51300b7780 2024-04-25T13:47:13.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.898+0000 7f512cff9640 1 -- 172.21.15.80:0/3733304301 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f51180be070 con 0x7f51300b7780 2024-04-25T13:47:13.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.902+0000 7f512cff9640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f430 0x7f5110071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:13.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.902+0000 7f512cff9640 1 -- 172.21.15.80:0/3733304301 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f51181281b0 con 0x7f51300b7780 2024-04-25T13:47:13.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.902+0000 7f512f7fe640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f430 0x7f5110071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:13.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.902+0000 7f512f7fe640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f430 0x7f5110071940 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5120009a50 tx=0x7f5120008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:13.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:13.906+0000 7f512cff9640 1 -- 172.21.15.80:0/3733304301 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5118016150 con 0x7f51300b7780 2024-04-25T13:47:14.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.090+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f513006a3a0 con 0x7f51300b7780 2024-04-25T13:47:14.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f512cff9640 1 -- 172.21.15.80:0/3733304301 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7f51180f4ac0 con 0x7f51300b7780 2024-04-25T13:47:14.092 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:14.092 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:14.092 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:14.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f430 msgr2=0x7f5110071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f430 0x7f5110071940 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5120009a50 tx=0x7f5120008040 comp rx=0 tx=0).stop 2024-04-25T13:47:14.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51300b7780 msgr2=0x7f51300cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51300b7780 0x7f51300cba90 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f51180b0040 tx=0x7f5118019300 comp rx=0 tx=0).stop 2024-04-25T13:47:14.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 shutdown_connections 2024-04-25T13:47:14.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f430 0x7f5110071940 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f51300d0ab0 0x7f51300cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f51300b7780 0x7f51300cba90 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 --2- 172.21.15.80:0/3733304301 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51300b4eb0 0x7f51300c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 >> 172.21.15.80:0/3733304301 conn(0x7f513006c9f0 msgr2=0x7f51300b93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:14.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 shutdown_connections 2024-04-25T13:47:14.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.094+0000 7f5135b1e640 1 -- 172.21.15.80:0/3733304301 wait complete. 2024-04-25T13:47:14.113 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:14.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.254+0000 7f9556ebc640 1 Processor -- start 2024-04-25T13:47:14.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.254+0000 7f9556ebc640 1 -- start start 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9556ebc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f955006f5a0 0x7f9550160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9556ebc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9550161070 0x7f9550161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9556ebc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f955006fcf0 con 0x7f95500b4bd0 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9556ebc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f95501619c0 con 0x7f9550161070 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9556ebc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9550073f10 con 0x7f955006f5a0 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9554c58640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f955006f5a0 0x7f9550160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9554c58640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f955006f5a0 0x7f9550160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:43742/0 (socket says 172.21.15.80:43742) 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9554c58640 1 -- 172.21.15.80:0/11668821 learned_addr learned my addr 172.21.15.80:0/11668821 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:14.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f954ffff640 1 --2- 172.21.15.80:0/11668821 >> v2:172.21.15.87:3300/0 conn(0x7f9550161070 0x7f9550161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9554c58640 1 -- 172.21.15.80:0/11668821 >> v2:172.21.15.87:3300/0 conn(0x7f9550161070 msgr2=0x7f9550161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:14.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9554c58640 1 --2- 172.21.15.80:0/11668821 >> v2:172.21.15.87:3300/0 conn(0x7f9550161070 0x7f9550161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9554c58640 1 -- 172.21.15.80:0/11668821 >> v1:172.21.15.87:6789/0 conn(0x7f95500b4bd0 legacy=0x7f95500b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f9554c58640 1 -- 172.21.15.80:0/11668821 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95500b0c60 con 0x7f955006f5a0 2024-04-25T13:47:14.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.258+0000 7f954ffff640 1 --2- 172.21.15.80:0/11668821 >> v2:172.21.15.87:3300/0 conn(0x7f9550161070 0x7f9550161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:14.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9554c58640 1 --2- 172.21.15.80:0/11668821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f955006f5a0 0x7f9550160b30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9548009870 tx=0x7f954801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab50cb6f5d7799a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:14.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f954f7fe640 1 -- 172.21.15.80:0/11668821 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95480b1070 con 0x7f955006f5a0 2024-04-25T13:47:14.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f954f7fe640 1 -- 172.21.15.80:0/11668821 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95480ac620 con 0x7f955006f5a0 2024-04-25T13:47:14.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f954f7fe640 1 -- 172.21.15.80:0/11668821 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95480ac9f0 con 0x7f955006f5a0 2024-04-25T13:47:14.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 -- 172.21.15.80:0/11668821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f955006f5a0 msgr2=0x7f9550160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 --2- 172.21.15.80:0/11668821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f955006f5a0 0x7f9550160b30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9548009870 tx=0x7f954801c4d0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 -- 172.21.15.80:0/11668821 shutdown_connections 2024-04-25T13:47:14.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 --2- 172.21.15.80:0/11668821 >> v2:172.21.15.87:3300/0 conn(0x7f9550161070 0x7f9550161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 --2- 172.21.15.80:0/11668821 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f955006f5a0 0x7f9550160b30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 -- 172.21.15.80:0/11668821 >> 172.21.15.80:0/11668821 conn(0x7f955006d080 msgr2=0x7f95500b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:14.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 -- 172.21.15.80:0/11668821 shutdown_connections 2024-04-25T13:47:14.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 -- 172.21.15.80:0/11668821 wait complete. 2024-04-25T13:47:14.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 Processor -- start 2024-04-25T13:47:14.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.262+0000 7f9556ebc640 1 -- start start 2024-04-25T13:47:14.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f955006f5a0 0x7f955015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95500b4bd0 0x7f955013b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9550161070 0x7f95500cf770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f955006e610 con 0x7f9550161070 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f955006e490 con 0x7f955006f5a0 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f955006e790 con 0x7f95500b4bd0 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9550161070 0x7f95500cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9550161070 0x7f95500cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52968/0 (socket says 172.21.15.80:52968) 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 -- 172.21.15.80:0/3067620223 learned_addr learned my addr 172.21.15.80:0/3067620223 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9554c58640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f955006f5a0 0x7f955015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f954ffff640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95500b4bd0 0x7f955013b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 -- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95500b4bd0 msgr2=0x7f955013b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95500b4bd0 0x7f955013b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 -- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f955006f5a0 msgr2=0x7f955015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f955006f5a0 0x7f955015d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 -- 172.21.15.80:0/3067620223 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f955014db70 con 0x7f9550161070 2024-04-25T13:47:14.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9555459640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9550161070 0x7f95500cf770 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f954000c730 tx=0x7f954000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:14.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f954d7fa640 1 -- 172.21.15.80:0/3067620223 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540003b80 con 0x7f9550161070 2024-04-25T13:47:14.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f954d7fa640 1 -- 172.21.15.80:0/3067620223 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9540003d20 con 0x7f9550161070 2024-04-25T13:47:14.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f955014cad0 con 0x7f9550161070 2024-04-25T13:47:14.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f954d7fa640 1 -- 172.21.15.80:0/3067620223 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9540005310 con 0x7f9550161070 2024-04-25T13:47:14.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955014d090 con 0x7f9550161070 2024-04-25T13:47:14.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f955006a3a0 con 0x7f9550161070 2024-04-25T13:47:14.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.266+0000 7f954d7fa640 1 -- 172.21.15.80:0/3067620223 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f95400056c0 con 0x7f9550161070 2024-04-25T13:47:14.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.270+0000 7f954d7fa640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f953006f400 0x7f9530071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.270+0000 7f954d7fa640 1 -- 172.21.15.80:0/3067620223 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f954008fe50 con 0x7f9550161070 2024-04-25T13:47:14.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.270+0000 7f9554c58640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f953006f400 0x7f9530071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.270+0000 7f9554c58640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f953006f400 0x7f9530071910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f95500b6e70 tx=0x7f95480a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:14.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.274+0000 7f954d7fa640 1 -- 172.21.15.80:0/3067620223 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f954005cd40 con 0x7f9550161070 2024-04-25T13:47:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.418+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95500b0a40 con 0x7f9550161070 2024-04-25T13:47:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f954d7fa640 1 -- 172.21.15.80:0/3067620223 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) ==== 74+0+16556 (secure 0 0 0) 0x7f954005c640 con 0x7f9550161070 2024-04-25T13:47:14.419 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:14.419 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":136,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:11.683283+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:47:08.431048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T13:47:09.003237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f953006f400 msgr2=0x7f9530071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f953006f400 0x7f9530071910 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f95500b6e70 tx=0x7f95480a7000 comp rx=0 tx=0).stop 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9550161070 msgr2=0x7f95500cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9550161070 0x7f95500cf770 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f954000c730 tx=0x7f954000cbf0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 shutdown_connections 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f953006f400 0x7f9530071910 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9550161070 0x7f95500cf770 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f95500b4bd0 0x7f955013b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 --2- 172.21.15.80:0/3067620223 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f955006f5a0 0x7f955015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 >> 172.21.15.80:0/3067620223 conn(0x7f955006d080 msgr2=0x7f9550154d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 shutdown_connections 2024-04-25T13:47:14.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.422+0000 7f9556ebc640 1 -- 172.21.15.80:0/3067620223 wait complete. 2024-04-25T13:47:14.440 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:47:14.441 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:47:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c10a35640 1 Processor -- start 2024-04-25T13:47:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c10a35640 1 -- start start 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c10a35640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c0cfb60 0x7f1c0c0cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c10a35640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c0c15d630 0x7f1c0c15da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c10a35640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c0c0d04b0 con 0x7f1c0c161630 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c10a35640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1c0c143920 con 0x7f1c0c15d630 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c10a35640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0c06f350 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c0ad9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c0c15d630 0x7f1c0c15da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c0ad9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c0c15d630 0x7f1c0c15da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52976/0 (socket says 172.21.15.80:52976) 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c0ad9d640 1 -- 172.21.15.80:0/1026631700 learned_addr learned my addr 172.21.15.80:0/1026631700 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c09d9b640 1 --2- 172.21.15.80:0/1026631700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c0cfb60 0x7f1c0c0cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c0ad9d640 1 -- 172.21.15.80:0/1026631700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c0cfb60 msgr2=0x7f1c0c0cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c0ad9d640 1 --2- 172.21.15.80:0/1026631700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c0cfb60 0x7f1c0c0cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c0ad9d640 1 -- 172.21.15.80:0/1026631700 >> v1:172.21.15.87:6789/0 conn(0x7f1c0c161630 legacy=0x7f1c0c06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c0ad9d640 1 -- 172.21.15.80:0/1026631700 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c06cd00 con 0x7f1c0c15d630 2024-04-25T13:47:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.590+0000 7f1c09d9b640 1 --2- 172.21.15.80:0/1026631700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c0cfb60 0x7f1c0c0cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c0ad9d640 1 --2- 172.21.15.80:0/1026631700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c15d630 0x7f1c0c15da40 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009870 tx=0x7f1bf401eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28459468bf0f0c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c0959a640 1 -- 172.21.15.80:0/1026631700 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf40b5070 con 0x7f1c0c15d630 2024-04-25T13:47:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c0959a640 1 -- 172.21.15.80:0/1026631700 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1bf40a87f0 con 0x7f1c0c15d630 2024-04-25T13:47:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c0959a640 1 -- 172.21.15.80:0/1026631700 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bf40a8bc0 con 0x7f1c0c15d630 2024-04-25T13:47:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c10a35640 1 -- 172.21.15.80:0/1026631700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c15d630 msgr2=0x7f1c0c15da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c10a35640 1 --2- 172.21.15.80:0/1026631700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c15d630 0x7f1c0c15da40 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f1bf4009870 tx=0x7f1bf401eb30 comp rx=0 tx=0).stop 2024-04-25T13:47:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c10a35640 1 -- 172.21.15.80:0/1026631700 shutdown_connections 2024-04-25T13:47:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c10a35640 1 --2- 172.21.15.80:0/1026631700 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c15d630 0x7f1c0c15da40 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c10a35640 1 --2- 172.21.15.80:0/1026631700 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c0cfb60 0x7f1c0c0cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c10a35640 1 -- 172.21.15.80:0/1026631700 >> 172.21.15.80:0/1026631700 conn(0x7f1c0c0d33a0 msgr2=0x7f1c0c0cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c10a35640 1 -- 172.21.15.80:0/1026631700 shutdown_connections 2024-04-25T13:47:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.594+0000 7f1c10a35640 1 -- 172.21.15.80:0/1026631700 wait complete. 2024-04-25T13:47:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 Processor -- start 2024-04-25T13:47:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 -- start start 2024-04-25T13:47:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c0cfb60 0x7f1c0c13fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c15d630 0x7f1c0c0c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c0c161630 0x7f1c0c142290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c160e30 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c0c06d270 con 0x7f1c0c161630 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c0c161110 con 0x7f1c0c15d630 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c0cfb60 0x7f1c0c13fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c0cfb60 0x7f1c0c13fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52990/0 (socket says 172.21.15.80:52990) 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 -- 172.21.15.80:0/4124248280 learned_addr learned my addr 172.21.15.80:0/4124248280 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c09d9b640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c15d630 0x7f1c0c0c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0ad9d640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c0c161630 0x7f1c0c142290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 -- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c15d630 msgr2=0x7f1c0c0c7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c15d630 0x7f1c0c0c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 -- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c0c161630 msgr2=0x7f1c0c142290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c0c161630 0x7f1c0c142290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 -- 172.21.15.80:0/4124248280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c0c140430 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c0a59c640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c0cfb60 0x7f1c0c13fef0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400ba50 tx=0x7f1c0400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c02ffd640 1 -- 172.21.15.80:0/4124248280 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04019070 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c02ffd640 1 -- 172.21.15.80:0/4124248280 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c0400cc20 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c0c0c8410 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c02ffd640 1 -- 172.21.15.80:0/4124248280 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04014440 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c0c0c89d0 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.598+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c0c06a3a0 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.602+0000 7f1c02ffd640 1 -- 172.21.15.80:0/4124248280 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1c0401c070 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.602+0000 7f1c02ffd640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1bec06f430 0x7f1bec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.602+0000 7f1c02ffd640 1 -- 172.21.15.80:0/4124248280 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f1c0408eeb0 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.602+0000 7f1c09d9b640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1bec06f430 0x7f1bec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.602+0000 7f1c09d9b640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1bec06f430 0x7f1bec071940 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc006fd0 tx=0x7f1bfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:14.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.606+0000 7f1c02ffd640 1 -- 172.21.15.80:0/4124248280 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c0405b7e0 con 0x7f1c0c0cfb60 2024-04-25T13:47:14.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.750+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1c0c0d0c40 con 0x7f1bec06f430 2024-04-25T13:47:14.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.770+0000 7f1c02ffd640 1 -- 172.21.15.80:0/4124248280 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247980 (secure 0 0 0) 0x7f1c0c0d0c40 con 0x7f1bec06f430 2024-04-25T13:47:14.770 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:14.772 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:47:14.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1bec06f430 msgr2=0x7f1bec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1bec06f430 0x7f1bec071940 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc006fd0 tx=0x7f1bfc008040 comp rx=0 tx=0).stop 2024-04-25T13:47:14.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c0cfb60 msgr2=0x7f1c0c13fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c0cfb60 0x7f1c0c13fef0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f1c0400ba50 tx=0x7f1c0400bf10 comp rx=0 tx=0).stop 2024-04-25T13:47:14.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 shutdown_connections 2024-04-25T13:47:14.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1bec06f430 0x7f1bec071940 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c0c161630 0x7f1c0c142290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c0c15d630 0x7f1c0c0c7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 --2- 172.21.15.80:0/4124248280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c0c0cfb60 0x7f1c0c13fef0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 >> 172.21.15.80:0/4124248280 conn(0x7f1c0c0d33a0 msgr2=0x7f1c0c1418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:14.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 shutdown_connections 2024-04-25T13:47:14.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.774+0000 7f1c10a35640 1 -- 172.21.15.80:0/4124248280 wait complete. 2024-04-25T13:47:14.791 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-04-25T13:47:14.245819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7058,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555441,"statfs":{"total":1600000000,"available":1592772348,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.593746"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733602+0000","last_change":"2024-04-25T13:47:10.728271+0000","last_active":"2024-04-25T13:47:11.733602+0000","last_peered":"2024-04-25T13:47:11.733602+0000","last_clean":"2024-04-25T13:47:11.733602+0000","last_became_active":"2024-04-25T13:47:10.727955+0000","last_became_peered":"2024-04-25T13:47:10.727955+0000","last_unstale":"2024-04-25T13:47:11.733602+0000","last_undegraded":"2024-04-25T13:47:11.733602+0000","last_fullsized":"2024-04-25T13:47:11.733602+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:11.617316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732822+0000","last_change":"2024-04-25T13:47:10.717710+0000","last_active":"2024-04-25T13:47:11.732822+0000","last_peered":"2024-04-25T13:47:11.732822+0000","last_clean":"2024-04-25T13:47:11.732822+0000","last_became_active":"2024-04-25T13:47:10.716906+0000","last_became_peered":"2024-04-25T13:47:10.716906+0000","last_unstale":"2024-04-25T13:47:11.732822+0000","last_undegraded":"2024-04-25T13:47:11.732822+0000","last_fullsized":"2024-04-25T13:47:11.732822+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:39.184374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706170+0000","last_change":"2024-04-25T13:47:10.724405+0000","last_active":"2024-04-25T13:47:11.706170+0000","last_peered":"2024-04-25T13:47:11.706170+0000","last_clean":"2024-04-25T13:47:11.706170+0000","last_became_active":"2024-04-25T13:47:10.711069+0000","last_became_peered":"2024-04-25T13:47:10.711069+0000","last_unstale":"2024-04-25T13:47:11.706170+0000","last_undegraded":"2024-04-25T13:47:11.706170+0000","last_fullsized":"2024-04-25T13:47:11.706170+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:33.192175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734747+0000","last_change":"2024-04-25T13:47:10.710070+0000","last_active":"2024-04-25T13:47:11.734747+0000","last_peered":"2024-04-25T13:47:11.734747+0000","last_clean":"2024-04-25T13:47:11.734747+0000","last_became_active":"2024-04-25T13:47:10.709317+0000","last_became_peered":"2024-04-25T13:47:10.709317+0000","last_unstale":"2024-04-25T13:47:11.734747+0000","last_undegraded":"2024-04-25T13:47:11.734747+0000","last_fullsized":"2024-04-25T13:47:11.734747+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:32.641501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699162+0000","last_change":"2024-04-25T13:47:10.719111+0000","last_active":"2024-04-25T13:47:11.699162+0000","last_peered":"2024-04-25T13:47:11.699162+0000","last_clean":"2024-04-25T13:47:11.699162+0000","last_became_active":"2024-04-25T13:47:10.718303+0000","last_became_peered":"2024-04-25T13:47:10.718303+0000","last_unstale":"2024-04-25T13:47:11.699162+0000","last_undegraded":"2024-04-25T13:47:11.699162+0000","last_fullsized":"2024-04-25T13:47:11.699162+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:15.574220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697663+0000","last_change":"2024-04-25T13:47:10.710951+0000","last_active":"2024-04-25T13:47:11.697663+0000","last_peered":"2024-04-25T13:47:11.697663+0000","last_clean":"2024-04-25T13:47:11.697663+0000","last_became_active":"2024-04-25T13:47:10.709647+0000","last_became_peered":"2024-04-25T13:47:10.709647+0000","last_unstale":"2024-04-25T13:47:11.697663+0000","last_undegraded":"2024-04-25T13:47:11.697663+0000","last_fullsized":"2024-04-25T13:47:11.697663+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:57.857048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729343+0000","last_change":"2024-04-25T13:47:10.704182+0000","last_active":"2024-04-25T13:47:11.729343+0000","last_peered":"2024-04-25T13:47:11.729343+0000","last_clean":"2024-04-25T13:47:11.729343+0000","last_became_active":"2024-04-25T13:47:10.703397+0000","last_became_peered":"2024-04-25T13:47:10.703397+0000","last_unstale":"2024-04-25T13:47:11.729343+0000","last_undegraded":"2024-04-25T13:47:11.729343+0000","last_fullsized":"2024-04-25T13:47:11.729343+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:03.708984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706824+0000","last_change":"2024-04-25T13:47:10.728334+0000","last_active":"2024-04-25T13:47:11.706824+0000","last_peered":"2024-04-25T13:47:11.706824+0000","last_clean":"2024-04-25T13:47:11.706824+0000","last_became_active":"2024-04-25T13:47:10.727134+0000","last_became_peered":"2024-04-25T13:47:10.727134+0000","last_unstale":"2024-04-25T13:47:11.706824+0000","last_undegraded":"2024-04-25T13:47:11.706824+0000","last_fullsized":"2024-04-25T13:47:11.706824+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:34.368179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735174+0000","last_change":"2024-04-25T13:47:10.724323+0000","last_active":"2024-04-25T13:47:11.735174+0000","last_peered":"2024-04-25T13:47:11.735174+0000","last_clean":"2024-04-25T13:47:11.735174+0000","last_became_active":"2024-04-25T13:47:10.723900+0000","last_became_peered":"2024-04-25T13:47:10.723900+0000","last_unstale":"2024-04-25T13:47:11.735174+0000","last_undegraded":"2024-04-25T13:47:11.735174+0000","last_fullsized":"2024-04-25T13:47:11.735174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:11.217761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734090+0000","last_change":"2024-04-25T13:47:10.723569+0000","last_active":"2024-04-25T13:47:11.734090+0000","last_peered":"2024-04-25T13:47:11.734090+0000","last_clean":"2024-04-25T13:47:11.734090+0000","last_became_active":"2024-04-25T13:47:10.718951+0000","last_became_peered":"2024-04-25T13:47:10.718951+0000","last_unstale":"2024-04-25T13:47:11.734090+0000","last_undegraded":"2024-04-25T13:47:11.734090+0000","last_fullsized":"2024-04-25T13:47:11.734090+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:58.480301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732421+0000","last_change":"2024-04-25T13:47:10.722831+0000","last_active":"2024-04-25T13:47:11.732421+0000","last_peered":"2024-04-25T13:47:11.732421+0000","last_clean":"2024-04-25T13:47:11.732421+0000","last_became_active":"2024-04-25T13:47:10.721976+0000","last_became_peered":"2024-04-25T13:47:10.721976+0000","last_unstale":"2024-04-25T13:47:11.732421+0000","last_undegraded":"2024-04-25T13:47:11.732421+0000","last_fullsized":"2024-04-25T13:47:11.732421+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:39.819802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733290+0000","last_change":"2024-04-25T13:47:10.728501+0000","last_active":"2024-04-25T13:47:11.733290+0000","last_peered":"2024-04-25T13:47:11.733290+0000","last_clean":"2024-04-25T13:47:11.733290+0000","last_became_active":"2024-04-25T13:47:10.727668+0000","last_became_peered":"2024-04-25T13:47:10.727668+0000","last_unstale":"2024-04-25T13:47:11.733290+0000","last_undegraded":"2024-04-25T13:47:11.733290+0000","last_fullsized":"2024-04-25T13:47:11.733290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:07.748747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707272+0000","last_change":"2024-04-25T13:47:10.711962+0000","last_active":"2024-04-25T13:47:11.707272+0000","last_peered":"2024-04-25T13:47:11.707272+0000","last_clean":"2024-04-25T13:47:11.707272+0000","last_became_active":"2024-04-25T13:47:10.706626+0000","last_became_peered":"2024-04-25T13:47:10.706626+0000","last_unstale":"2024-04-25T13:47:11.707272+0000","last_undegraded":"2024-04-25T13:47:11.707272+0000","last_fullsized":"2024-04-25T13:47:11.707272+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:40.190225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.705702+0000","last_change":"2024-04-25T13:47:10.730491+0000","last_active":"2024-04-25T13:47:11.705702+0000","last_peered":"2024-04-25T13:47:11.705702+0000","last_clean":"2024-04-25T13:47:11.705702+0000","last_became_active":"2024-04-25T13:47:10.726371+0000","last_became_peered":"2024-04-25T13:47:10.726371+0000","last_unstale":"2024-04-25T13:47:11.705702+0000","last_undegraded":"2024-04-25T13:47:11.705702+0000","last_fullsized":"2024-04-25T13:47:11.705702+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697806+0000","last_change":"2024-04-25T13:47:10.718349+0000","last_active":"2024-04-25T13:47:11.697806+0000","last_peered":"2024-04-25T13:47:11.697806+0000","last_clean":"2024-04-25T13:47:11.697806+0000","last_became_active":"2024-04-25T13:47:10.717611+0000","last_became_peered":"2024-04-25T13:47:10.717611+0000","last_unstale":"2024-04-25T13:47:11.697806+0000","last_undegraded":"2024-04-25T13:47:11.697806+0000","last_fullsized":"2024-04-25T13:47:11.697806+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:37.197698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733850+0000","last_change":"2024-04-25T13:47:10.727444+0000","last_active":"2024-04-25T13:47:11.733850+0000","last_peered":"2024-04-25T13:47:11.733850+0000","last_clean":"2024-04-25T13:47:11.733850+0000","last_became_active":"2024-04-25T13:47:10.726498+0000","last_became_peered":"2024-04-25T13:47:10.726498+0000","last_unstale":"2024-04-25T13:47:11.733850+0000","last_undegraded":"2024-04-25T13:47:11.733850+0000","last_fullsized":"2024-04-25T13:47:11.733850+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:36.707854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733324+0000","last_change":"2024-04-25T13:47:10.719546+0000","last_active":"2024-04-25T13:47:11.733324+0000","last_peered":"2024-04-25T13:47:11.733324+0000","last_clean":"2024-04-25T13:47:11.733324+0000","last_became_active":"2024-04-25T13:47:10.718643+0000","last_became_peered":"2024-04-25T13:47:10.718643+0000","last_unstale":"2024-04-25T13:47:11.733324+0000","last_undegraded":"2024-04-25T13:47:11.733324+0000","last_fullsized":"2024-04-25T13:47:11.733324+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:29.510337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706664+0000","last_change":"2024-04-25T13:47:10.721030+0000","last_active":"2024-04-25T13:47:11.706664+0000","last_peered":"2024-04-25T13:47:11.706664+0000","last_clean":"2024-04-25T13:47:11.706664+0000","last_became_active":"2024-04-25T13:47:10.716697+0000","last_became_peered":"2024-04-25T13:47:10.716697+0000","last_unstale":"2024-04-25T13:47:11.706664+0000","last_undegraded":"2024-04-25T13:47:11.706664+0000","last_fullsized":"2024-04-25T13:47:11.706664+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:20.849844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706182+0000","last_change":"2024-04-25T13:47:10.724526+0000","last_active":"2024-04-25T13:47:11.706182+0000","last_peered":"2024-04-25T13:47:11.706182+0000","last_clean":"2024-04-25T13:47:11.706182+0000","last_became_active":"2024-04-25T13:47:10.722860+0000","last_became_peered":"2024-04-25T13:47:10.722860+0000","last_unstale":"2024-04-25T13:47:11.706182+0000","last_undegraded":"2024-04-25T13:47:11.706182+0000","last_fullsized":"2024-04-25T13:47:11.706182+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:57.828999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733262+0000","last_change":"2024-04-25T13:47:10.714623+0000","last_active":"2024-04-25T13:47:11.733262+0000","last_peered":"2024-04-25T13:47:11.733262+0000","last_clean":"2024-04-25T13:47:11.733262+0000","last_became_active":"2024-04-25T13:47:10.713811+0000","last_became_peered":"2024-04-25T13:47:10.713811+0000","last_unstale":"2024-04-25T13:47:11.733262+0000","last_undegraded":"2024-04-25T13:47:11.733262+0000","last_fullsized":"2024-04-25T13:47:11.733262+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:52.492699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733478+0000","last_change":"2024-04-25T13:47:10.714266+0000","last_active":"2024-04-25T13:47:11.733478+0000","last_peered":"2024-04-25T13:47:11.733478+0000","last_clean":"2024-04-25T13:47:11.733478+0000","last_became_active":"2024-04-25T13:47:10.711127+0000","last_became_peered":"2024-04-25T13:47:10.711127+0000","last_unstale":"2024-04-25T13:47:11.733478+0000","last_undegraded":"2024-04-25T13:47:11.733478+0000","last_fullsized":"2024-04-25T13:47:11.733478+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:11.234058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735827+0000","last_change":"2024-04-25T13:47:10.716561+0000","last_active":"2024-04-25T13:47:11.735827+0000","last_peered":"2024-04-25T13:47:11.735827+0000","last_clean":"2024-04-25T13:47:11.735827+0000","last_became_active":"2024-04-25T13:47:10.714810+0000","last_became_peered":"2024-04-25T13:47:10.714810+0000","last_unstale":"2024-04-25T13:47:11.735827+0000","last_undegraded":"2024-04-25T13:47:11.735827+0000","last_fullsized":"2024-04-25T13:47:11.735827+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:33.278707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732319+0000","last_change":"2024-04-25T13:47:10.727851+0000","last_active":"2024-04-25T13:47:11.732319+0000","last_peered":"2024-04-25T13:47:11.732319+0000","last_clean":"2024-04-25T13:47:11.732319+0000","last_became_active":"2024-04-25T13:47:10.727092+0000","last_became_peered":"2024-04-25T13:47:10.727092+0000","last_unstale":"2024-04-25T13:47:11.732319+0000","last_undegraded":"2024-04-25T13:47:11.732319+0000","last_fullsized":"2024-04-25T13:47:11.732319+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:34.245305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735203+0000","last_change":"2024-04-25T13:47:10.726514+0000","last_active":"2024-04-25T13:47:11.735203+0000","last_peered":"2024-04-25T13:47:11.735203+0000","last_clean":"2024-04-25T13:47:11.735203+0000","last_became_active":"2024-04-25T13:47:10.726237+0000","last_became_peered":"2024-04-25T13:47:10.726237+0000","last_unstale":"2024-04-25T13:47:11.735203+0000","last_undegraded":"2024-04-25T13:47:11.735203+0000","last_fullsized":"2024-04-25T13:47:11.735203+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:24.979487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732784+0000","last_change":"2024-04-25T13:47:10.719674+0000","last_active":"2024-04-25T13:47:11.732784+0000","last_peered":"2024-04-25T13:47:11.732784+0000","last_clean":"2024-04-25T13:47:11.732784+0000","last_became_active":"2024-04-25T13:47:10.718962+0000","last_became_peered":"2024-04-25T13:47:10.718962+0000","last_unstale":"2024-04-25T13:47:11.732784+0000","last_undegraded":"2024-04-25T13:47:11.732784+0000","last_fullsized":"2024-04-25T13:47:11.732784+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:45.367276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734080+0000","last_change":"2024-04-25T13:47:10.727152+0000","last_active":"2024-04-25T13:47:11.734080+0000","last_peered":"2024-04-25T13:47:11.734080+0000","last_clean":"2024-04-25T13:47:11.734080+0000","last_became_active":"2024-04-25T13:47:10.718669+0000","last_became_peered":"2024-04-25T13:47:10.718669+0000","last_unstale":"2024-04-25T13:47:11.734080+0000","last_undegraded":"2024-04-25T13:47:11.734080+0000","last_fullsized":"2024-04-25T13:47:11.734080+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:36.849024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697848+0000","last_change":"2024-04-25T13:47:10.723298+0000","last_active":"2024-04-25T13:47:11.697848+0000","last_peered":"2024-04-25T13:47:11.697848+0000","last_clean":"2024-04-25T13:47:11.697848+0000","last_became_active":"2024-04-25T13:47:10.720107+0000","last_became_peered":"2024-04-25T13:47:10.720107+0000","last_unstale":"2024-04-25T13:47:11.697848+0000","last_undegraded":"2024-04-25T13:47:11.697848+0000","last_fullsized":"2024-04-25T13:47:11.697848+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:27.487106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734926+0000","last_change":"2024-04-25T13:47:10.722026+0000","last_active":"2024-04-25T13:47:11.734926+0000","last_peered":"2024-04-25T13:47:11.734926+0000","last_clean":"2024-04-25T13:47:11.734926+0000","last_became_active":"2024-04-25T13:47:10.719261+0000","last_became_peered":"2024-04-25T13:47:10.719261+0000","last_unstale":"2024-04-25T13:47:11.734926+0000","last_undegraded":"2024-04-25T13:47:11.734926+0000","last_fullsized":"2024-04-25T13:47:11.734926+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:00.808019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732550+0000","last_change":"2024-04-25T13:47:10.725208+0000","last_active":"2024-04-25T13:47:11.732550+0000","last_peered":"2024-04-25T13:47:11.732550+0000","last_clean":"2024-04-25T13:47:11.732550+0000","last_became_active":"2024-04-25T13:47:10.724377+0000","last_became_peered":"2024-04-25T13:47:10.724377+0000","last_unstale":"2024-04-25T13:47:11.732550+0000","last_undegraded":"2024-04-25T13:47:11.732550+0000","last_fullsized":"2024-04-25T13:47:11.732550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:49.785000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728834+0000","last_change":"2024-04-25T13:47:10.717443+0000","last_active":"2024-04-25T13:47:11.728834+0000","last_peered":"2024-04-25T13:47:11.728834+0000","last_clean":"2024-04-25T13:47:11.728834+0000","last_became_active":"2024-04-25T13:47:10.716734+0000","last_became_peered":"2024-04-25T13:47:10.716734+0000","last_unstale":"2024-04-25T13:47:11.728834+0000","last_undegraded":"2024-04-25T13:47:11.728834+0000","last_fullsized":"2024-04-25T13:47:11.728834+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:19.058456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734304+0000","last_change":"2024-04-25T13:47:10.726875+0000","last_active":"2024-04-25T13:47:11.734304+0000","last_peered":"2024-04-25T13:47:11.734304+0000","last_clean":"2024-04-25T13:47:11.734304+0000","last_became_active":"2024-04-25T13:47:10.717310+0000","last_became_peered":"2024-04-25T13:47:10.717310+0000","last_unstale":"2024-04-25T13:47:11.734304+0000","last_undegraded":"2024-04-25T13:47:11.734304+0000","last_fullsized":"2024-04-25T13:47:11.734304+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:35.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698486+0000","last_change":"2024-04-25T13:47:10.723988+0000","last_active":"2024-04-25T13:47:11.698486+0000","last_peered":"2024-04-25T13:47:11.698486+0000","last_clean":"2024-04-25T13:47:11.698486+0000","last_became_active":"2024-04-25T13:47:10.721475+0000","last_became_peered":"2024-04-25T13:47:10.721475+0000","last_unstale":"2024-04-25T13:47:11.698486+0000","last_undegraded":"2024-04-25T13:47:11.698486+0000","last_fullsized":"2024-04-25T13:47:11.698486+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:32.493732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697310+0000","last_change":"2024-04-25T13:47:10.704315+0000","last_active":"2024-04-25T13:47:11.697310+0000","last_peered":"2024-04-25T13:47:11.697310+0000","last_clean":"2024-04-25T13:47:11.697310+0000","last_became_active":"2024-04-25T13:47:10.703571+0000","last_became_peered":"2024-04-25T13:47:10.703571+0000","last_unstale":"2024-04-25T13:47:11.697310+0000","last_undegraded":"2024-04-25T13:47:11.697310+0000","last_fullsized":"2024-04-25T13:47:11.697310+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:20.121342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"135'4","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728996+0000","last_change":"2024-04-25T13:47:09.682009+0000","last_active":"2024-04-25T13:47:11.728996+0000","last_peered":"2024-04-25T13:47:11.728996+0000","last_clean":"2024-04-25T13:47:11.728996+0000","last_became_active":"2024-04-25T13:47:09.681182+0000","last_became_peered":"2024-04-25T13:47:09.681182+0000","last_unstale":"2024-04-25T13:47:11.728996+0000","last_undegraded":"2024-04-25T13:47:11.728996+0000","last_fullsized":"2024-04-25T13:47:11.728996+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:54.568585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707124+0000","last_change":"2024-04-25T13:47:10.711064+0000","last_active":"2024-04-25T13:47:11.707124+0000","last_peered":"2024-04-25T13:47:11.707124+0000","last_clean":"2024-04-25T13:47:11.707124+0000","last_became_active":"2024-04-25T13:47:10.710331+0000","last_became_peered":"2024-04-25T13:47:10.710331+0000","last_unstale":"2024-04-25T13:47:11.707124+0000","last_undegraded":"2024-04-25T13:47:11.707124+0000","last_fullsized":"2024-04-25T13:47:11.707124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:39.478214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707473+0000","last_change":"2024-04-25T13:47:09.674718+0000","last_active":"2024-04-25T13:47:11.707473+0000","last_peered":"2024-04-25T13:47:11.707473+0000","last_clean":"2024-04-25T13:47:11.707473+0000","last_became_active":"2024-04-25T13:47:09.673707+0000","last_became_peered":"2024-04-25T13:47:09.673707+0000","last_unstale":"2024-04-25T13:47:11.707473+0000","last_undegraded":"2024-04-25T13:47:11.707473+0000","last_fullsized":"2024-04-25T13:47:11.707473+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:29.296176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698141+0000","last_change":"2024-04-25T13:47:10.710337+0000","last_active":"2024-04-25T13:47:11.698141+0000","last_peered":"2024-04-25T13:47:11.698141+0000","last_clean":"2024-04-25T13:47:11.698141+0000","last_became_active":"2024-04-25T13:47:10.708714+0000","last_became_peered":"2024-04-25T13:47:10.708714+0000","last_unstale":"2024-04-25T13:47:11.698141+0000","last_undegraded":"2024-04-25T13:47:11.698141+0000","last_fullsized":"2024-04-25T13:47:11.698141+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:17.366901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699553+0000","last_change":"2024-04-25T13:47:09.681907+0000","last_active":"2024-04-25T13:47:11.699553+0000","last_peered":"2024-04-25T13:47:11.699553+0000","last_clean":"2024-04-25T13:47:11.699553+0000","last_became_active":"2024-04-25T13:47:09.680918+0000","last_became_peered":"2024-04-25T13:47:09.680918+0000","last_unstale":"2024-04-25T13:47:11.699553+0000","last_undegraded":"2024-04-25T13:47:11.699553+0000","last_fullsized":"2024-04-25T13:47:11.699553+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:50.073408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706237+0000","last_change":"2024-04-25T13:47:10.711635+0000","last_active":"2024-04-25T13:47:11.706237+0000","last_peered":"2024-04-25T13:47:11.706237+0000","last_clean":"2024-04-25T13:47:11.706237+0000","last_became_active":"2024-04-25T13:47:10.709693+0000","last_became_peered":"2024-04-25T13:47:10.709693+0000","last_unstale":"2024-04-25T13:47:11.706237+0000","last_undegraded":"2024-04-25T13:47:11.706237+0000","last_fullsized":"2024-04-25T13:47:11.706237+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:04.196466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.700209+0000","last_change":"2024-04-25T13:47:09.674753+0000","last_active":"2024-04-25T13:47:11.700209+0000","last_peered":"2024-04-25T13:47:11.700209+0000","last_clean":"2024-04-25T13:47:11.700209+0000","last_became_active":"2024-04-25T13:47:09.674140+0000","last_became_peered":"2024-04-25T13:47:09.674140+0000","last_unstale":"2024-04-25T13:47:11.700209+0000","last_undegraded":"2024-04-25T13:47:11.700209+0000","last_fullsized":"2024-04-25T13:47:11.700209+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:31.121015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733543+0000","last_change":"2024-04-25T13:47:10.719036+0000","last_active":"2024-04-25T13:47:11.733543+0000","last_peered":"2024-04-25T13:47:11.733543+0000","last_clean":"2024-04-25T13:47:11.733543+0000","last_became_active":"2024-04-25T13:47:10.713900+0000","last_became_peered":"2024-04-25T13:47:10.713900+0000","last_unstale":"2024-04-25T13:47:11.733543+0000","last_undegraded":"2024-04-25T13:47:11.733543+0000","last_fullsized":"2024-04-25T13:47:11.733543+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:57.714122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729098+0000","last_change":"2024-04-25T13:47:09.682797+0000","last_active":"2024-04-25T13:47:11.729098+0000","last_peered":"2024-04-25T13:47:11.729098+0000","last_clean":"2024-04-25T13:47:11.729098+0000","last_became_active":"2024-04-25T13:47:09.679375+0000","last_became_peered":"2024-04-25T13:47:09.679375+0000","last_unstale":"2024-04-25T13:47:11.729098+0000","last_undegraded":"2024-04-25T13:47:11.729098+0000","last_fullsized":"2024-04-25T13:47:11.729098+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:21.675830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733799+0000","last_change":"2024-04-25T13:47:10.715936+0000","last_active":"2024-04-25T13:47:11.733799+0000","last_peered":"2024-04-25T13:47:11.733799+0000","last_clean":"2024-04-25T13:47:11.733799+0000","last_became_active":"2024-04-25T13:47:10.714659+0000","last_became_peered":"2024-04-25T13:47:10.714659+0000","last_unstale":"2024-04-25T13:47:11.733799+0000","last_undegraded":"2024-04-25T13:47:11.733799+0000","last_fullsized":"2024-04-25T13:47:11.733799+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:39.174347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735223+0000","last_change":"2024-04-25T13:47:09.686782+0000","last_active":"2024-04-25T13:47:11.735223+0000","last_peered":"2024-04-25T13:47:11.735223+0000","last_clean":"2024-04-25T13:47:11.735223+0000","last_became_active":"2024-04-25T13:47:09.686553+0000","last_became_peered":"2024-04-25T13:47:09.686553+0000","last_unstale":"2024-04-25T13:47:11.735223+0000","last_undegraded":"2024-04-25T13:47:11.735223+0000","last_fullsized":"2024-04-25T13:47:11.735223+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:40.729092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734561+0000","last_change":"2024-04-25T13:47:10.723184+0000","last_active":"2024-04-25T13:47:11.734561+0000","last_peered":"2024-04-25T13:47:11.734561+0000","last_clean":"2024-04-25T13:47:11.734561+0000","last_became_active":"2024-04-25T13:47:10.714016+0000","last_became_peered":"2024-04-25T13:47:10.714016+0000","last_unstale":"2024-04-25T13:47:11.734561+0000","last_undegraded":"2024-04-25T13:47:11.734561+0000","last_fullsized":"2024-04-25T13:47:11.734561+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:37.800132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732744+0000","last_change":"2024-04-25T13:47:09.673427+0000","last_active":"2024-04-25T13:47:11.732744+0000","last_peered":"2024-04-25T13:47:11.732744+0000","last_clean":"2024-04-25T13:47:11.732744+0000","last_became_active":"2024-04-25T13:47:09.672383+0000","last_became_peered":"2024-04-25T13:47:09.672383+0000","last_unstale":"2024-04-25T13:47:11.732744+0000","last_undegraded":"2024-04-25T13:47:11.732744+0000","last_fullsized":"2024-04-25T13:47:11.732744+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:36.201666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728407+0000","last_change":"2024-04-25T13:47:10.704385+0000","last_active":"2024-04-25T13:47:11.728407+0000","last_peered":"2024-04-25T13:47:11.728407+0000","last_clean":"2024-04-25T13:47:11.728407+0000","last_became_active":"2024-04-25T13:47:10.703678+0000","last_became_peered":"2024-04-25T13:47:10.703678+0000","last_unstale":"2024-04-25T13:47:11.728407+0000","last_undegraded":"2024-04-25T13:47:11.728407+0000","last_fullsized":"2024-04-25T13:47:11.728407+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.503945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735255+0000","last_change":"2024-04-25T13:47:09.673539+0000","last_active":"2024-04-25T13:47:11.735255+0000","last_peered":"2024-04-25T13:47:11.735255+0000","last_clean":"2024-04-25T13:47:11.735255+0000","last_became_active":"2024-04-25T13:47:09.672528+0000","last_became_peered":"2024-04-25T13:47:09.672528+0000","last_unstale":"2024-04-25T13:47:11.735255+0000","last_undegraded":"2024-04-25T13:47:11.735255+0000","last_fullsized":"2024-04-25T13:47:11.735255+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:21.673022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728832+0000","last_change":"2024-04-25T13:47:10.709949+0000","last_active":"2024-04-25T13:47:11.728832+0000","last_peered":"2024-04-25T13:47:11.728832+0000","last_clean":"2024-04-25T13:47:11.728832+0000","last_became_active":"2024-04-25T13:47:10.709077+0000","last_became_peered":"2024-04-25T13:47:10.709077+0000","last_unstale":"2024-04-25T13:47:11.728832+0000","last_undegraded":"2024-04-25T13:47:11.728832+0000","last_fullsized":"2024-04-25T13:47:11.728832+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:23.249112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.700397+0000","last_change":"2024-04-25T13:47:09.674999+0000","last_active":"2024-04-25T13:47:11.700397+0000","last_peered":"2024-04-25T13:47:11.700397+0000","last_clean":"2024-04-25T13:47:11.700397+0000","last_became_active":"2024-04-25T13:47:09.674495+0000","last_became_peered":"2024-04-25T13:47:09.674495+0000","last_unstale":"2024-04-25T13:47:11.700397+0000","last_undegraded":"2024-04-25T13:47:11.700397+0000","last_fullsized":"2024-04-25T13:47:11.700397+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.383571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.708299+0000","last_change":"2024-04-25T13:47:10.728205+0000","last_active":"2024-04-25T13:47:11.708299+0000","last_peered":"2024-04-25T13:47:11.708299+0000","last_clean":"2024-04-25T13:47:11.708299+0000","last_became_active":"2024-04-25T13:47:10.727269+0000","last_became_peered":"2024-04-25T13:47:10.727269+0000","last_unstale":"2024-04-25T13:47:11.708299+0000","last_undegraded":"2024-04-25T13:47:11.708299+0000","last_fullsized":"2024-04-25T13:47:11.708299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:38.834861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736535+0000","last_change":"2024-04-25T13:47:09.685948+0000","last_active":"2024-04-25T13:47:11.736535+0000","last_peered":"2024-04-25T13:47:11.736535+0000","last_clean":"2024-04-25T13:47:11.736535+0000","last_became_active":"2024-04-25T13:47:09.685716+0000","last_became_peered":"2024-04-25T13:47:09.685716+0000","last_unstale":"2024-04-25T13:47:11.736535+0000","last_undegraded":"2024-04-25T13:47:11.736535+0000","last_fullsized":"2024-04-25T13:47:11.736535+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:57.872201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735655+0000","last_change":"2024-04-25T13:47:10.711143+0000","last_active":"2024-04-25T13:47:11.735655+0000","last_peered":"2024-04-25T13:47:11.735655+0000","last_clean":"2024-04-25T13:47:11.735655+0000","last_became_active":"2024-04-25T13:47:10.710457+0000","last_became_peered":"2024-04-25T13:47:10.710457+0000","last_unstale":"2024-04-25T13:47:11.735655+0000","last_undegraded":"2024-04-25T13:47:11.735655+0000","last_fullsized":"2024-04-25T13:47:11.735655+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:56.109270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"135'3","reported_seq":17,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734300+0000","last_change":"2024-04-25T13:47:09.674010+0000","last_active":"2024-04-25T13:47:11.734300+0000","last_peered":"2024-04-25T13:47:11.734300+0000","last_clean":"2024-04-25T13:47:11.734300+0000","last_became_active":"2024-04-25T13:47:09.673459+0000","last_became_peered":"2024-04-25T13:47:09.673459+0000","last_unstale":"2024-04-25T13:47:11.734300+0000","last_undegraded":"2024-04-25T13:47:11.734300+0000","last_fullsized":"2024-04-25T13:47:11.734300+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:31.446118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728726+0000","last_change":"2024-04-25T13:47:10.701426+0000","last_active":"2024-04-25T13:47:11.728726+0000","last_peered":"2024-04-25T13:47:11.728726+0000","last_clean":"2024-04-25T13:47:11.728726+0000","last_became_active":"2024-04-25T13:47:10.700653+0000","last_became_peered":"2024-04-25T13:47:10.700653+0000","last_unstale":"2024-04-25T13:47:11.728726+0000","last_undegraded":"2024-04-25T13:47:11.728726+0000","last_fullsized":"2024-04-25T13:47:11.728726+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:25.584164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"135'6","reported_seq":20,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735681+0000","last_change":"2024-04-25T13:47:09.673006+0000","last_active":"2024-04-25T13:47:11.735681+0000","last_peered":"2024-04-25T13:47:11.735681+0000","last_clean":"2024-04-25T13:47:11.735681+0000","last_became_active":"2024-04-25T13:47:09.672656+0000","last_became_peered":"2024-04-25T13:47:09.672656+0000","last_unstale":"2024-04-25T13:47:11.735681+0000","last_undegraded":"2024-04-25T13:47:11.735681+0000","last_fullsized":"2024-04-25T13:47:11.735681+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:09.587533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736051+0000","last_change":"2024-04-25T13:47:10.710716+0000","last_active":"2024-04-25T13:47:11.736051+0000","last_peered":"2024-04-25T13:47:11.736051+0000","last_clean":"2024-04-25T13:47:11.736051+0000","last_became_active":"2024-04-25T13:47:10.710023+0000","last_became_peered":"2024-04-25T13:47:10.710023+0000","last_unstale":"2024-04-25T13:47:11.736051+0000","last_undegraded":"2024-04-25T13:47:11.736051+0000","last_fullsized":"2024-04-25T13:47:11.736051+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:53.029454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698056+0000","last_change":"2024-04-25T13:47:09.677556+0000","last_active":"2024-04-25T13:47:11.698056+0000","last_peered":"2024-04-25T13:47:11.698056+0000","last_clean":"2024-04-25T13:47:11.698056+0000","last_became_active":"2024-04-25T13:47:09.676926+0000","last_became_peered":"2024-04-25T13:47:09.676926+0000","last_unstale":"2024-04-25T13:47:11.698056+0000","last_undegraded":"2024-04-25T13:47:11.698056+0000","last_fullsized":"2024-04-25T13:47:11.698056+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:24.384131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735798+0000","last_change":"2024-04-25T13:47:10.718863+0000","last_active":"2024-04-25T13:47:11.735798+0000","last_peered":"2024-04-25T13:47:11.735798+0000","last_clean":"2024-04-25T13:47:11.735798+0000","last_became_active":"2024-04-25T13:47:10.717846+0000","last_became_peered":"2024-04-25T13:47:10.717846+0000","last_unstale":"2024-04-25T13:47:11.735798+0000","last_undegraded":"2024-04-25T13:47:11.735798+0000","last_fullsized":"2024-04-25T13:47:11.735798+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:06.641310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.708588+0000","last_change":"2024-04-25T13:47:09.687086+0000","last_active":"2024-04-25T13:47:11.708588+0000","last_peered":"2024-04-25T13:47:11.708588+0000","last_clean":"2024-04-25T13:47:11.708588+0000","last_became_active":"2024-04-25T13:47:09.686017+0000","last_became_peered":"2024-04-25T13:47:09.686017+0000","last_unstale":"2024-04-25T13:47:11.708588+0000","last_undegraded":"2024-04-25T13:47:11.708588+0000","last_fullsized":"2024-04-25T13:47:11.708588+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:22.637768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736264+0000","last_change":"2024-04-25T13:47:10.717412+0000","last_active":"2024-04-25T13:47:11.736264+0000","last_peered":"2024-04-25T13:47:11.736264+0000","last_clean":"2024-04-25T13:47:11.736264+0000","last_became_active":"2024-04-25T13:47:10.715835+0000","last_became_peered":"2024-04-25T13:47:10.715835+0000","last_unstale":"2024-04-25T13:47:11.736264+0000","last_undegraded":"2024-04-25T13:47:11.736264+0000","last_fullsized":"2024-04-25T13:47:11.736264+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:41.116680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"135'8","reported_seq":22,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698674+0000","last_change":"2024-04-25T13:47:09.678846+0000","last_active":"2024-04-25T13:47:11.698674+0000","last_peered":"2024-04-25T13:47:11.698674+0000","last_clean":"2024-04-25T13:47:11.698674+0000","last_became_active":"2024-04-25T13:47:09.677706+0000","last_became_peered":"2024-04-25T13:47:09.677706+0000","last_unstale":"2024-04-25T13:47:11.698674+0000","last_undegraded":"2024-04-25T13:47:11.698674+0000","last_fullsized":"2024-04-25T13:47:11.698674+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:55.968598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699339+0000","last_change":"2024-04-25T13:47:10.721946+0000","last_active":"2024-04-25T13:47:11.699339+0000","last_peered":"2024-04-25T13:47:11.699339+0000","last_clean":"2024-04-25T13:47:11.699339+0000","last_became_active":"2024-04-25T13:47:10.720879+0000","last_became_peered":"2024-04-25T13:47:10.720879+0000","last_unstale":"2024-04-25T13:47:11.699339+0000","last_undegraded":"2024-04-25T13:47:11.699339+0000","last_fullsized":"2024-04-25T13:47:11.699339+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:51.842983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735980+0000","last_change":"2024-04-25T13:47:09.675954+0000","last_active":"2024-04-25T13:47:11.735980+0000","last_peered":"2024-04-25T13:47:11.735980+0000","last_clean":"2024-04-25T13:47:11.735980+0000","last_became_active":"2024-04-25T13:47:09.675437+0000","last_became_peered":"2024-04-25T13:47:09.675437+0000","last_unstale":"2024-04-25T13:47:11.735980+0000","last_undegraded":"2024-04-25T13:47:11.735980+0000","last_fullsized":"2024-04-25T13:47:11.735980+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.735437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697757+0000","last_change":"2024-04-25T13:47:10.712195+0000","last_active":"2024-04-25T13:47:11.697757+0000","last_peered":"2024-04-25T13:47:11.697757+0000","last_clean":"2024-04-25T13:47:11.697757+0000","last_became_active":"2024-04-25T13:47:10.711487+0000","last_became_peered":"2024-04-25T13:47:10.711487+0000","last_unstale":"2024-04-25T13:47:11.697757+0000","last_undegraded":"2024-04-25T13:47:11.697757+0000","last_fullsized":"2024-04-25T13:47:11.697757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:24.893923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699681+0000","last_change":"2024-04-25T13:47:09.682498+0000","last_active":"2024-04-25T13:47:11.699681+0000","last_peered":"2024-04-25T13:47:11.699681+0000","last_clean":"2024-04-25T13:47:11.699681+0000","last_became_active":"2024-04-25T13:47:09.681492+0000","last_became_peered":"2024-04-25T13:47:09.681492+0000","last_unstale":"2024-04-25T13:47:11.699681+0000","last_undegraded":"2024-04-25T13:47:11.699681+0000","last_fullsized":"2024-04-25T13:47:11.699681+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:32.184310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.705433+0000","last_change":"2024-04-25T13:47:10.713277+0000","last_active":"2024-04-25T13:47:11.705433+0000","last_peered":"2024-04-25T13:47:11.705433+0000","last_clean":"2024-04-25T13:47:11.705433+0000","last_became_active":"2024-04-25T13:47:10.712479+0000","last_became_peered":"2024-04-25T13:47:10.712479+0000","last_unstale":"2024-04-25T13:47:11.705433+0000","last_undegraded":"2024-04-25T13:47:11.705433+0000","last_fullsized":"2024-04-25T13:47:11.705433+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:52.455886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.708925+0000","last_change":"2024-04-25T13:47:09.686695+0000","last_active":"2024-04-25T13:47:11.708925+0000","last_peered":"2024-04-25T13:47:11.708925+0000","last_clean":"2024-04-25T13:47:11.708925+0000","last_became_active":"2024-04-25T13:47:09.685056+0000","last_became_peered":"2024-04-25T13:47:09.685056+0000","last_unstale":"2024-04-25T13:47:11.708925+0000","last_undegraded":"2024-04-25T13:47:11.708925+0000","last_fullsized":"2024-04-25T13:47:11.708925+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:20.181056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.705727+0000","last_change":"2024-04-25T13:47:10.703818+0000","last_active":"2024-04-25T13:47:11.705727+0000","last_peered":"2024-04-25T13:47:11.705727+0000","last_clean":"2024-04-25T13:47:11.705727+0000","last_became_active":"2024-04-25T13:47:10.703079+0000","last_became_peered":"2024-04-25T13:47:10.703079+0000","last_unstale":"2024-04-25T13:47:11.705727+0000","last_undegraded":"2024-04-25T13:47:11.705727+0000","last_fullsized":"2024-04-25T13:47:11.705727+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:34.831413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729874+0000","last_change":"2024-04-25T13:47:09.683617+0000","last_active":"2024-04-25T13:47:11.729874+0000","last_peered":"2024-04-25T13:47:11.729874+0000","last_clean":"2024-04-25T13:47:11.729874+0000","last_became_active":"2024-04-25T13:47:09.680631+0000","last_became_peered":"2024-04-25T13:47:09.680631+0000","last_unstale":"2024-04-25T13:47:11.729874+0000","last_undegraded":"2024-04-25T13:47:11.729874+0000","last_fullsized":"2024-04-25T13:47:11.729874+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:29.636035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707088+0000","last_change":"2024-04-25T13:47:10.729244+0000","last_active":"2024-04-25T13:47:11.707088+0000","last_peered":"2024-04-25T13:47:11.707088+0000","last_clean":"2024-04-25T13:47:11.707088+0000","last_became_active":"2024-04-25T13:47:10.724005+0000","last_became_peered":"2024-04-25T13:47:10.724005+0000","last_unstale":"2024-04-25T13:47:11.707088+0000","last_undegraded":"2024-04-25T13:47:11.707088+0000","last_fullsized":"2024-04-25T13:47:11.707088+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:55.933401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697159+0000","last_change":"2024-04-25T13:47:09.680220+0000","last_active":"2024-04-25T13:47:11.697159+0000","last_peered":"2024-04-25T13:47:11.697159+0000","last_clean":"2024-04-25T13:47:11.697159+0000","last_became_active":"2024-04-25T13:47:09.679844+0000","last_became_peered":"2024-04-25T13:47:09.679844+0000","last_unstale":"2024-04-25T13:47:11.697159+0000","last_undegraded":"2024-04-25T13:47:11.697159+0000","last_fullsized":"2024-04-25T13:47:11.697159+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:27.888201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733910+0000","last_change":"2024-04-25T13:47:10.715674+0000","last_active":"2024-04-25T13:47:11.733910+0000","last_peered":"2024-04-25T13:47:11.733910+0000","last_clean":"2024-04-25T13:47:11.733910+0000","last_became_active":"2024-04-25T13:47:10.712719+0000","last_became_peered":"2024-04-25T13:47:10.712719+0000","last_unstale":"2024-04-25T13:47:11.733910+0000","last_undegraded":"2024-04-25T13:47:11.733910+0000","last_fullsized":"2024-04-25T13:47:11.733910+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:17.481231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699020+0000","last_change":"2024-04-25T13:47:09.675253+0000","last_active":"2024-04-25T13:47:11.699020+0000","last_peered":"2024-04-25T13:47:11.699020+0000","last_clean":"2024-04-25T13:47:11.699020+0000","last_became_active":"2024-04-25T13:47:09.674726+0000","last_became_peered":"2024-04-25T13:47:09.674726+0000","last_unstale":"2024-04-25T13:47:11.699020+0000","last_undegraded":"2024-04-25T13:47:11.699020+0000","last_fullsized":"2024-04-25T13:47:11.699020+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:50.769092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706201+0000","last_change":"2024-04-25T13:47:10.708913+0000","last_active":"2024-04-25T13:47:11.706201+0000","last_peered":"2024-04-25T13:47:11.706201+0000","last_clean":"2024-04-25T13:47:11.706201+0000","last_became_active":"2024-04-25T13:47:10.707779+0000","last_became_peered":"2024-04-25T13:47:10.707779+0000","last_unstale":"2024-04-25T13:47:11.706201+0000","last_undegraded":"2024-04-25T13:47:11.706201+0000","last_fullsized":"2024-04-25T13:47:11.706201+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:54.431922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698169+0000","last_change":"2024-04-25T13:47:09.682911+0000","last_active":"2024-04-25T13:47:11.698169+0000","last_peered":"2024-04-25T13:47:11.698169+0000","last_clean":"2024-04-25T13:47:11.698169+0000","last_became_active":"2024-04-25T13:47:09.682630+0000","last_became_peered":"2024-04-25T13:47:09.682630+0000","last_unstale":"2024-04-25T13:47:11.698169+0000","last_undegraded":"2024-04-25T13:47:11.698169+0000","last_fullsized":"2024-04-25T13:47:11.698169+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:58.767649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735978+0000","last_change":"2024-04-25T13:47:10.724260+0000","last_active":"2024-04-25T13:47:11.735978+0000","last_peered":"2024-04-25T13:47:11.735978+0000","last_clean":"2024-04-25T13:47:11.735978+0000","last_became_active":"2024-04-25T13:47:10.722596+0000","last_became_peered":"2024-04-25T13:47:10.722596+0000","last_unstale":"2024-04-25T13:47:11.735978+0000","last_undegraded":"2024-04-25T13:47:11.735978+0000","last_fullsized":"2024-04-25T13:47:11.735978+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:13.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734579+0000","last_change":"2024-04-25T13:47:09.687931+0000","last_active":"2024-04-25T13:47:11.734579+0000","last_peered":"2024-04-25T13:47:11.734579+0000","last_clean":"2024-04-25T13:47:11.734579+0000","last_became_active":"2024-04-25T13:47:09.687516+0000","last_became_peered":"2024-04-25T13:47:09.687516+0000","last_unstale":"2024-04-25T13:47:11.734579+0000","last_undegraded":"2024-04-25T13:47:11.734579+0000","last_fullsized":"2024-04-25T13:47:11.734579+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:59.666307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707583+0000","last_change":"2024-04-25T13:47:10.720440+0000","last_active":"2024-04-25T13:47:11.707583+0000","last_peered":"2024-04-25T13:47:11.707583+0000","last_clean":"2024-04-25T13:47:11.707583+0000","last_became_active":"2024-04-25T13:47:10.715651+0000","last_became_peered":"2024-04-25T13:47:10.715651+0000","last_unstale":"2024-04-25T13:47:11.707583+0000","last_undegraded":"2024-04-25T13:47:11.707583+0000","last_fullsized":"2024-04-25T13:47:11.707583+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:37.217172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.737125+0000","last_change":"2024-04-25T13:47:09.674420+0000","last_active":"2024-04-25T13:47:11.737125+0000","last_peered":"2024-04-25T13:47:11.737125+0000","last_clean":"2024-04-25T13:47:11.737125+0000","last_became_active":"2024-04-25T13:47:09.674150+0000","last_became_peered":"2024-04-25T13:47:09.674150+0000","last_unstale":"2024-04-25T13:47:11.737125+0000","last_undegraded":"2024-04-25T13:47:11.737125+0000","last_fullsized":"2024-04-25T13:47:11.737125+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:46.065746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733481+0000","last_change":"2024-04-25T13:47:10.712327+0000","last_active":"2024-04-25T13:47:11.733481+0000","last_peered":"2024-04-25T13:47:11.733481+0000","last_clean":"2024-04-25T13:47:11.733481+0000","last_became_active":"2024-04-25T13:47:10.708854+0000","last_became_peered":"2024-04-25T13:47:10.708854+0000","last_unstale":"2024-04-25T13:47:11.733481+0000","last_undegraded":"2024-04-25T13:47:11.733481+0000","last_fullsized":"2024-04-25T13:47:11.733481+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:43.868098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"136'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729044+0000","last_change":"2024-04-25T13:47:09.685680+0000","last_active":"2024-04-25T13:47:11.729044+0000","last_peered":"2024-04-25T13:47:11.729044+0000","last_clean":"2024-04-25T13:47:11.729044+0000","last_became_active":"2024-04-25T13:47:09.684307+0000","last_became_peered":"2024-04-25T13:47:09.684307+0000","last_unstale":"2024-04-25T13:47:11.729044+0000","last_undegraded":"2024-04-25T13:47:11.729044+0000","last_fullsized":"2024-04-25T13:47:11.729044+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:52.716976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734979+0000","last_change":"2024-04-25T13:47:10.722891+0000","last_active":"2024-04-25T13:47:11.734979+0000","last_peered":"2024-04-25T13:47:11.734979+0000","last_clean":"2024-04-25T13:47:11.734979+0000","last_became_active":"2024-04-25T13:47:10.713062+0000","last_became_peered":"2024-04-25T13:47:10.713062+0000","last_unstale":"2024-04-25T13:47:11.734979+0000","last_undegraded":"2024-04-25T13:47:11.734979+0000","last_fullsized":"2024-04-25T13:47:11.734979+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:16.779445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736247+0000","last_change":"2024-04-25T13:47:09.689101+0000","last_active":"2024-04-25T13:47:11.736247+0000","last_peered":"2024-04-25T13:47:11.736247+0000","last_clean":"2024-04-25T13:47:11.736247+0000","last_became_active":"2024-04-25T13:47:09.688456+0000","last_became_peered":"2024-04-25T13:47:09.688456+0000","last_unstale":"2024-04-25T13:47:11.736247+0000","last_undegraded":"2024-04-25T13:47:11.736247+0000","last_fullsized":"2024-04-25T13:47:11.736247+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:49.842860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706987+0000","last_change":"2024-04-25T13:47:10.726548+0000","last_active":"2024-04-25T13:47:11.706987+0000","last_peered":"2024-04-25T13:47:11.706987+0000","last_clean":"2024-04-25T13:47:11.706987+0000","last_became_active":"2024-04-25T13:47:10.725467+0000","last_became_peered":"2024-04-25T13:47:10.725467+0000","last_unstale":"2024-04-25T13:47:11.706987+0000","last_undegraded":"2024-04-25T13:47:11.706987+0000","last_fullsized":"2024-04-25T13:47:11.706987+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:22.314753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736107+0000","last_change":"2024-04-25T13:47:09.667421+0000","last_active":"2024-04-25T13:47:11.736107+0000","last_peered":"2024-04-25T13:47:11.736107+0000","last_clean":"2024-04-25T13:47:11.736107+0000","last_became_active":"2024-04-25T13:47:09.667125+0000","last_became_peered":"2024-04-25T13:47:09.667125+0000","last_unstale":"2024-04-25T13:47:11.736107+0000","last_undegraded":"2024-04-25T13:47:11.736107+0000","last_fullsized":"2024-04-25T13:47:11.736107+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:08.137395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734269+0000","last_change":"2024-04-25T13:47:10.705223+0000","last_active":"2024-04-25T13:47:11.734269+0000","last_peered":"2024-04-25T13:47:11.734269+0000","last_clean":"2024-04-25T13:47:11.734269+0000","last_became_active":"2024-04-25T13:47:10.700343+0000","last_became_peered":"2024-04-25T13:47:10.700343+0000","last_unstale":"2024-04-25T13:47:11.734269+0000","last_undegraded":"2024-04-25T13:47:11.734269+0000","last_fullsized":"2024-04-25T13:47:11.734269+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:30.822799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729392+0000","last_change":"2024-04-25T13:47:09.681287+0000","last_active":"2024-04-25T13:47:11.729392+0000","last_peered":"2024-04-25T13:47:11.729392+0000","last_clean":"2024-04-25T13:47:11.729392+0000","last_became_active":"2024-04-25T13:47:09.680381+0000","last_became_peered":"2024-04-25T13:47:09.680381+0000","last_unstale":"2024-04-25T13:47:11.729392+0000","last_undegraded":"2024-04-25T13:47:11.729392+0000","last_fullsized":"2024-04-25T13:47:11.729392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:51.766362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732920+0000","last_change":"2024-04-25T13:47:10.713257+0000","last_active":"2024-04-25T13:47:11.732920+0000","last_peered":"2024-04-25T13:47:11.732920+0000","last_clean":"2024-04-25T13:47:11.732920+0000","last_became_active":"2024-04-25T13:47:10.709365+0000","last_became_peered":"2024-04-25T13:47:10.709365+0000","last_unstale":"2024-04-25T13:47:11.732920+0000","last_undegraded":"2024-04-25T13:47:11.732920+0000","last_fullsized":"2024-04-25T13:47:11.732920+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:28.292174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734582+0000","last_change":"2024-04-25T13:47:09.685419+0000","last_active":"2024-04-25T13:47:11.734582+0000","last_peered":"2024-04-25T13:47:11.734582+0000","last_clean":"2024-04-25T13:47:11.734582+0000","last_became_active":"2024-04-25T13:47:09.683323+0000","last_became_peered":"2024-04-25T13:47:09.683323+0000","last_unstale":"2024-04-25T13:47:11.734582+0000","last_undegraded":"2024-04-25T13:47:11.734582+0000","last_fullsized":"2024-04-25T13:47:11.734582+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:31.462591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706638+0000","last_change":"2024-04-25T13:47:10.731341+0000","last_active":"2024-04-25T13:47:11.706638+0000","last_peered":"2024-04-25T13:47:11.706638+0000","last_clean":"2024-04-25T13:47:11.706638+0000","last_became_active":"2024-04-25T13:47:10.728523+0000","last_became_peered":"2024-04-25T13:47:10.728523+0000","last_unstale":"2024-04-25T13:47:11.706638+0000","last_undegraded":"2024-04-25T13:47:11.706638+0000","last_fullsized":"2024-04-25T13:47:11.706638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:13.373575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698792+0000","last_change":"2024-04-25T13:47:09.678277+0000","last_active":"2024-04-25T13:47:11.698792+0000","last_peered":"2024-04-25T13:47:11.698792+0000","last_clean":"2024-04-25T13:47:11.698792+0000","last_became_active":"2024-04-25T13:47:09.677007+0000","last_became_peered":"2024-04-25T13:47:09.677007+0000","last_unstale":"2024-04-25T13:47:11.698792+0000","last_undegraded":"2024-04-25T13:47:11.698792+0000","last_fullsized":"2024-04-25T13:47:11.698792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:10.815071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734133+0000","last_change":"2024-04-25T13:47:10.714638+0000","last_active":"2024-04-25T13:47:11.734133+0000","last_peered":"2024-04-25T13:47:11.734133+0000","last_clean":"2024-04-25T13:47:11.734133+0000","last_became_active":"2024-04-25T13:47:10.709924+0000","last_became_peered":"2024-04-25T13:47:10.709924+0000","last_unstale":"2024-04-25T13:47:11.734133+0000","last_undegraded":"2024-04-25T13:47:11.734133+0000","last_fullsized":"2024-04-25T13:47:11.734133+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:10.542469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736499+0000","last_change":"2024-04-25T13:47:09.688832+0000","last_active":"2024-04-25T13:47:11.736499+0000","last_peered":"2024-04-25T13:47:11.736499+0000","last_clean":"2024-04-25T13:47:11.736499+0000","last_became_active":"2024-04-25T13:47:09.688091+0000","last_became_peered":"2024-04-25T13:47:09.688091+0000","last_unstale":"2024-04-25T13:47:11.736499+0000","last_undegraded":"2024-04-25T13:47:11.736499+0000","last_fullsized":"2024-04-25T13:47:11.736499+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:42.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735102+0000","last_change":"2024-04-25T13:47:10.715441+0000","last_active":"2024-04-25T13:47:11.735102+0000","last_peered":"2024-04-25T13:47:11.735102+0000","last_clean":"2024-04-25T13:47:11.735102+0000","last_became_active":"2024-04-25T13:47:10.714643+0000","last_became_peered":"2024-04-25T13:47:10.714643+0000","last_unstale":"2024-04-25T13:47:11.735102+0000","last_undegraded":"2024-04-25T13:47:11.735102+0000","last_fullsized":"2024-04-25T13:47:11.735102+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:42.100752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729600+0000","last_change":"2024-04-25T13:47:09.684332+0000","last_active":"2024-04-25T13:47:11.729600+0000","last_peered":"2024-04-25T13:47:11.729600+0000","last_clean":"2024-04-25T13:47:11.729600+0000","last_became_active":"2024-04-25T13:47:09.681584+0000","last_became_peered":"2024-04-25T13:47:09.681584+0000","last_unstale":"2024-04-25T13:47:11.729600+0000","last_undegraded":"2024-04-25T13:47:11.729600+0000","last_fullsized":"2024-04-25T13:47:11.729600+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:50.455303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":339,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707676+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:47:11.707676+0000","last_peered":"2024-04-25T13:47:11.707676+0000","last_clean":"2024-04-25T13:47:11.707676+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:47:11.707676+0000","last_undegraded":"2024-04-25T13:47:11.707676+0000","last_fullsized":"2024-04-25T13:47:11.707676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1314,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193997,"statfs":{"total":200000000,"available":198653754,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":737,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":738,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244134,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":737,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244738,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":739,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194573,"statfs":{"total":200000000,"available":199243056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":737,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244738,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":737,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193996,"statfs":{"total":200000000,"available":198652646,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:47:14.795 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:47:14.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.986+0000 7fb0affa9640 1 Processor -- start 2024-04-25T13:47:14.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.986+0000 7fb0affa9640 1 -- start start 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0affa9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0a8161f90 0x7fb0a80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0affa9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a80d0510 0x7fb0a815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0affa9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb0a815ea70 0x7fb0a815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0affa9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a80d0d70 con 0x7fb0a8161f90 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0affa9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb0a80d1050 con 0x7fb0a815ea70 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0affa9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a806dca0 con 0x7fb0a80d0510 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0a8161f90 0x7fb0a80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0a8161f90 0x7fb0a80cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52992/0 (socket says 172.21.15.80:52992) 2024-04-25T13:47:14.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 -- 172.21.15.80:0/2422123738 learned_addr learned my addr 172.21.15.80:0/2422123738 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0ad544640 1 --2- 172.21.15.80:0/2422123738 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a80d0510 0x7fb0a815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0ae546640 1 --2- 172.21.15.80:0/2422123738 >> v2:172.21.15.87:3300/0 conn(0x7fb0a815ea70 0x7fb0a815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 -- 172.21.15.80:0/2422123738 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a80d0510 msgr2=0x7fb0a815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 --2- 172.21.15.80:0/2422123738 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a80d0510 0x7fb0a815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 -- 172.21.15.80:0/2422123738 >> v2:172.21.15.87:3300/0 conn(0x7fb0a815ea70 msgr2=0x7fb0a815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 --2- 172.21.15.80:0/2422123738 >> v2:172.21.15.87:3300/0 conn(0x7fb0a815ea70 0x7fb0a815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 -- 172.21.15.80:0/2422123738 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a806df80 con 0x7fb0a8161f90 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0ae546640 1 --2- 172.21.15.80:0/2422123738 >> v2:172.21.15.87:3300/0 conn(0x7fb0a815ea70 0x7fb0a815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:14.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.990+0000 7fb0add45640 1 --2- 172.21.15.80:0/2422123738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a8161f90 0x7fb0a80cffd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8071090 tx=0x7fb0a401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d2b2bc7e5f81b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:14.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0acd43640 1 -- 172.21.15.80:0/2422123738 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40b1070 con 0x7fb0a8161f90 2024-04-25T13:47:14.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0acd43640 1 -- 172.21.15.80:0/2422123738 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a40a8050 con 0x7fb0a8161f90 2024-04-25T13:47:14.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0acd43640 1 -- 172.21.15.80:0/2422123738 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40ac860 con 0x7fb0a8161f90 2024-04-25T13:47:14.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 -- 172.21.15.80:0/2422123738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a8161f90 msgr2=0x7fb0a80cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2422123738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a8161f90 0x7fb0a80cffd0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8071090 tx=0x7fb0a401c460 comp rx=0 tx=0).stop 2024-04-25T13:47:14.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 -- 172.21.15.80:0/2422123738 shutdown_connections 2024-04-25T13:47:14.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2422123738 >> v2:172.21.15.87:3300/0 conn(0x7fb0a815ea70 0x7fb0a815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2422123738 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a80d0510 0x7fb0a815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2422123738 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a8161f90 0x7fb0a80cffd0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 -- 172.21.15.80:0/2422123738 >> 172.21.15.80:0/2422123738 conn(0x7fb0a806d080 msgr2=0x7fb0a80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:14.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 -- 172.21.15.80:0/2422123738 shutdown_connections 2024-04-25T13:47:14.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 -- 172.21.15.80:0/2422123738 wait complete. 2024-04-25T13:47:14.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 Processor -- start 2024-04-25T13:47:14.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.994+0000 7fb0affa9640 1 -- start start 2024-04-25T13:47:14.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a80d0510 0x7fb0a815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a815ea70 0x7fb0a815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a8161f90 0x7fb0a815b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81447a0 con 0x7fb0a80d0510 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8144620 con 0x7fb0a815ea70 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb0a80d0920 con 0x7fb0a8161f90 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a80d0510 0x7fb0a815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a80d0510 0x7fb0a815cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52996/0 (socket says 172.21.15.80:52996) 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 -- 172.21.15.80:0/2656489644 learned_addr learned my addr 172.21.15.80:0/2656489644 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0ae546640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a8161f90 0x7fb0a815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0ad544640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a815ea70 0x7fb0a815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 -- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a8161f90 msgr2=0x7fb0a815b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a8161f90 0x7fb0a815b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 -- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a815ea70 msgr2=0x7fb0a815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a815ea70 0x7fb0a815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 -- 172.21.15.80:0/2656489644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a8072480 con 0x7fb0a80d0510 2024-04-25T13:47:14.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0add45640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a80d0510 0x7fb0a815cad0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb0a401cd00 tx=0x7fb0a401aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:14.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb09e7fc640 1 -- 172.21.15.80:0/2656489644 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40b1040 con 0x7fb0a80d0510 2024-04-25T13:47:14.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb09e7fc640 1 -- 172.21.15.80:0/2656489644 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0a40bd070 con 0x7fb0a80d0510 2024-04-25T13:47:14.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a8072760 con 0x7fb0a80d0510 2024-04-25T13:47:14.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb09e7fc640 1 -- 172.21.15.80:0/2656489644 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40ace60 con 0x7fb0a80d0510 2024-04-25T13:47:14.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a8072d20 con 0x7fb0a80d0510 2024-04-25T13:47:14.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a806a3a0 con 0x7fb0a80d0510 2024-04-25T13:47:14.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb09e7fc640 1 -- 172.21.15.80:0/2656489644 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb0a40b7860 con 0x7fb0a80d0510 2024-04-25T13:47:14.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb09e7fc640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb08c06f430 0x7fb08c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:14.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb09e7fc640 1 -- 172.21.15.80:0/2656489644 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7fb0a4129780 con 0x7fb0a80d0510 2024-04-25T13:47:14.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:14.998+0000 7fb0ad544640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb08c06f430 0x7fb08c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:14.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.002+0000 7fb0ad544640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb08c06f430 0x7fb08c071940 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80b0bd0 tx=0x7fb098008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.006+0000 7fb09e7fc640 1 -- 172.21.15.80:0/2656489644 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0a4016110 con 0x7fb0a80d0510 2024-04-25T13:47:15.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.158+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb0a8161090 con 0x7fb08c06f430 2024-04-25T13:47:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.178+0000 7fb09e7fc640 1 -- 172.21.15.80:0/2656489644 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247980 (secure 0 0 0) 0x7fb0a8161090 con 0x7fb08c06f430 2024-04-25T13:47:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:15.181 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb08c06f430 msgr2=0x7fb08c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb08c06f430 0x7fb08c071940 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb0a80b0bd0 tx=0x7fb098008040 comp rx=0 tx=0).stop 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a80d0510 msgr2=0x7fb0a815cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a80d0510 0x7fb0a815cad0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fb0a401cd00 tx=0x7fb0a401aca0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 shutdown_connections 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb08c06f430 0x7fb08c071940 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0a8161f90 0x7fb0a815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0a815ea70 0x7fb0a815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 --2- 172.21.15.80:0/2656489644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0a80d0510 0x7fb0a815cad0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 >> 172.21.15.80:0/2656489644 conn(0x7fb0a806d080 msgr2=0x7fb0a815d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:15.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 shutdown_connections 2024-04-25T13:47:15.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.182+0000 7fb0affa9640 1 -- 172.21.15.80:0/2656489644 wait complete. 2024-04-25T13:47:15.201 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-04-25T13:47:14.245819+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":133,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7058,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555441,"statfs":{"total":1600000000,"available":1592772348,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.593746"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733602+0000","last_change":"2024-04-25T13:47:10.728271+0000","last_active":"2024-04-25T13:47:11.733602+0000","last_peered":"2024-04-25T13:47:11.733602+0000","last_clean":"2024-04-25T13:47:11.733602+0000","last_became_active":"2024-04-25T13:47:10.727955+0000","last_became_peered":"2024-04-25T13:47:10.727955+0000","last_unstale":"2024-04-25T13:47:11.733602+0000","last_undegraded":"2024-04-25T13:47:11.733602+0000","last_fullsized":"2024-04-25T13:47:11.733602+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:11.617316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732822+0000","last_change":"2024-04-25T13:47:10.717710+0000","last_active":"2024-04-25T13:47:11.732822+0000","last_peered":"2024-04-25T13:47:11.732822+0000","last_clean":"2024-04-25T13:47:11.732822+0000","last_became_active":"2024-04-25T13:47:10.716906+0000","last_became_peered":"2024-04-25T13:47:10.716906+0000","last_unstale":"2024-04-25T13:47:11.732822+0000","last_undegraded":"2024-04-25T13:47:11.732822+0000","last_fullsized":"2024-04-25T13:47:11.732822+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:39.184374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706170+0000","last_change":"2024-04-25T13:47:10.724405+0000","last_active":"2024-04-25T13:47:11.706170+0000","last_peered":"2024-04-25T13:47:11.706170+0000","last_clean":"2024-04-25T13:47:11.706170+0000","last_became_active":"2024-04-25T13:47:10.711069+0000","last_became_peered":"2024-04-25T13:47:10.711069+0000","last_unstale":"2024-04-25T13:47:11.706170+0000","last_undegraded":"2024-04-25T13:47:11.706170+0000","last_fullsized":"2024-04-25T13:47:11.706170+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:31:33.192175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734747+0000","last_change":"2024-04-25T13:47:10.710070+0000","last_active":"2024-04-25T13:47:11.734747+0000","last_peered":"2024-04-25T13:47:11.734747+0000","last_clean":"2024-04-25T13:47:11.734747+0000","last_became_active":"2024-04-25T13:47:10.709317+0000","last_became_peered":"2024-04-25T13:47:10.709317+0000","last_unstale":"2024-04-25T13:47:11.734747+0000","last_undegraded":"2024-04-25T13:47:11.734747+0000","last_fullsized":"2024-04-25T13:47:11.734747+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:32.641501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699162+0000","last_change":"2024-04-25T13:47:10.719111+0000","last_active":"2024-04-25T13:47:11.699162+0000","last_peered":"2024-04-25T13:47:11.699162+0000","last_clean":"2024-04-25T13:47:11.699162+0000","last_became_active":"2024-04-25T13:47:10.718303+0000","last_became_peered":"2024-04-25T13:47:10.718303+0000","last_unstale":"2024-04-25T13:47:11.699162+0000","last_undegraded":"2024-04-25T13:47:11.699162+0000","last_fullsized":"2024-04-25T13:47:11.699162+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:15.574220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697663+0000","last_change":"2024-04-25T13:47:10.710951+0000","last_active":"2024-04-25T13:47:11.697663+0000","last_peered":"2024-04-25T13:47:11.697663+0000","last_clean":"2024-04-25T13:47:11.697663+0000","last_became_active":"2024-04-25T13:47:10.709647+0000","last_became_peered":"2024-04-25T13:47:10.709647+0000","last_unstale":"2024-04-25T13:47:11.697663+0000","last_undegraded":"2024-04-25T13:47:11.697663+0000","last_fullsized":"2024-04-25T13:47:11.697663+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:57.857048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729343+0000","last_change":"2024-04-25T13:47:10.704182+0000","last_active":"2024-04-25T13:47:11.729343+0000","last_peered":"2024-04-25T13:47:11.729343+0000","last_clean":"2024-04-25T13:47:11.729343+0000","last_became_active":"2024-04-25T13:47:10.703397+0000","last_became_peered":"2024-04-25T13:47:10.703397+0000","last_unstale":"2024-04-25T13:47:11.729343+0000","last_undegraded":"2024-04-25T13:47:11.729343+0000","last_fullsized":"2024-04-25T13:47:11.729343+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:03.708984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706824+0000","last_change":"2024-04-25T13:47:10.728334+0000","last_active":"2024-04-25T13:47:11.706824+0000","last_peered":"2024-04-25T13:47:11.706824+0000","last_clean":"2024-04-25T13:47:11.706824+0000","last_became_active":"2024-04-25T13:47:10.727134+0000","last_became_peered":"2024-04-25T13:47:10.727134+0000","last_unstale":"2024-04-25T13:47:11.706824+0000","last_undegraded":"2024-04-25T13:47:11.706824+0000","last_fullsized":"2024-04-25T13:47:11.706824+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:24:34.368179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735174+0000","last_change":"2024-04-25T13:47:10.724323+0000","last_active":"2024-04-25T13:47:11.735174+0000","last_peered":"2024-04-25T13:47:11.735174+0000","last_clean":"2024-04-25T13:47:11.735174+0000","last_became_active":"2024-04-25T13:47:10.723900+0000","last_became_peered":"2024-04-25T13:47:10.723900+0000","last_unstale":"2024-04-25T13:47:11.735174+0000","last_undegraded":"2024-04-25T13:47:11.735174+0000","last_fullsized":"2024-04-25T13:47:11.735174+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:11:11.217761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734090+0000","last_change":"2024-04-25T13:47:10.723569+0000","last_active":"2024-04-25T13:47:11.734090+0000","last_peered":"2024-04-25T13:47:11.734090+0000","last_clean":"2024-04-25T13:47:11.734090+0000","last_became_active":"2024-04-25T13:47:10.718951+0000","last_became_peered":"2024-04-25T13:47:10.718951+0000","last_unstale":"2024-04-25T13:47:11.734090+0000","last_undegraded":"2024-04-25T13:47:11.734090+0000","last_fullsized":"2024-04-25T13:47:11.734090+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:58.480301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732421+0000","last_change":"2024-04-25T13:47:10.722831+0000","last_active":"2024-04-25T13:47:11.732421+0000","last_peered":"2024-04-25T13:47:11.732421+0000","last_clean":"2024-04-25T13:47:11.732421+0000","last_became_active":"2024-04-25T13:47:10.721976+0000","last_became_peered":"2024-04-25T13:47:10.721976+0000","last_unstale":"2024-04-25T13:47:11.732421+0000","last_undegraded":"2024-04-25T13:47:11.732421+0000","last_fullsized":"2024-04-25T13:47:11.732421+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:39.819802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733290+0000","last_change":"2024-04-25T13:47:10.728501+0000","last_active":"2024-04-25T13:47:11.733290+0000","last_peered":"2024-04-25T13:47:11.733290+0000","last_clean":"2024-04-25T13:47:11.733290+0000","last_became_active":"2024-04-25T13:47:10.727668+0000","last_became_peered":"2024-04-25T13:47:10.727668+0000","last_unstale":"2024-04-25T13:47:11.733290+0000","last_undegraded":"2024-04-25T13:47:11.733290+0000","last_fullsized":"2024-04-25T13:47:11.733290+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:50:07.748747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707272+0000","last_change":"2024-04-25T13:47:10.711962+0000","last_active":"2024-04-25T13:47:11.707272+0000","last_peered":"2024-04-25T13:47:11.707272+0000","last_clean":"2024-04-25T13:47:11.707272+0000","last_became_active":"2024-04-25T13:47:10.706626+0000","last_became_peered":"2024-04-25T13:47:10.706626+0000","last_unstale":"2024-04-25T13:47:11.707272+0000","last_undegraded":"2024-04-25T13:47:11.707272+0000","last_fullsized":"2024-04-25T13:47:11.707272+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:40.190225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.705702+0000","last_change":"2024-04-25T13:47:10.730491+0000","last_active":"2024-04-25T13:47:11.705702+0000","last_peered":"2024-04-25T13:47:11.705702+0000","last_clean":"2024-04-25T13:47:11.705702+0000","last_became_active":"2024-04-25T13:47:10.726371+0000","last_became_peered":"2024-04-25T13:47:10.726371+0000","last_unstale":"2024-04-25T13:47:11.705702+0000","last_undegraded":"2024-04-25T13:47:11.705702+0000","last_fullsized":"2024-04-25T13:47:11.705702+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:35.265192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697806+0000","last_change":"2024-04-25T13:47:10.718349+0000","last_active":"2024-04-25T13:47:11.697806+0000","last_peered":"2024-04-25T13:47:11.697806+0000","last_clean":"2024-04-25T13:47:11.697806+0000","last_became_active":"2024-04-25T13:47:10.717611+0000","last_became_peered":"2024-04-25T13:47:10.717611+0000","last_unstale":"2024-04-25T13:47:11.697806+0000","last_undegraded":"2024-04-25T13:47:11.697806+0000","last_fullsized":"2024-04-25T13:47:11.697806+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:37.197698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733850+0000","last_change":"2024-04-25T13:47:10.727444+0000","last_active":"2024-04-25T13:47:11.733850+0000","last_peered":"2024-04-25T13:47:11.733850+0000","last_clean":"2024-04-25T13:47:11.733850+0000","last_became_active":"2024-04-25T13:47:10.726498+0000","last_became_peered":"2024-04-25T13:47:10.726498+0000","last_unstale":"2024-04-25T13:47:11.733850+0000","last_undegraded":"2024-04-25T13:47:11.733850+0000","last_fullsized":"2024-04-25T13:47:11.733850+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:36.707854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733324+0000","last_change":"2024-04-25T13:47:10.719546+0000","last_active":"2024-04-25T13:47:11.733324+0000","last_peered":"2024-04-25T13:47:11.733324+0000","last_clean":"2024-04-25T13:47:11.733324+0000","last_became_active":"2024-04-25T13:47:10.718643+0000","last_became_peered":"2024-04-25T13:47:10.718643+0000","last_unstale":"2024-04-25T13:47:11.733324+0000","last_undegraded":"2024-04-25T13:47:11.733324+0000","last_fullsized":"2024-04-25T13:47:11.733324+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:29.510337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706664+0000","last_change":"2024-04-25T13:47:10.721030+0000","last_active":"2024-04-25T13:47:11.706664+0000","last_peered":"2024-04-25T13:47:11.706664+0000","last_clean":"2024-04-25T13:47:11.706664+0000","last_became_active":"2024-04-25T13:47:10.716697+0000","last_became_peered":"2024-04-25T13:47:10.716697+0000","last_unstale":"2024-04-25T13:47:11.706664+0000","last_undegraded":"2024-04-25T13:47:11.706664+0000","last_fullsized":"2024-04-25T13:47:11.706664+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:20.849844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706182+0000","last_change":"2024-04-25T13:47:10.724526+0000","last_active":"2024-04-25T13:47:11.706182+0000","last_peered":"2024-04-25T13:47:11.706182+0000","last_clean":"2024-04-25T13:47:11.706182+0000","last_became_active":"2024-04-25T13:47:10.722860+0000","last_became_peered":"2024-04-25T13:47:10.722860+0000","last_unstale":"2024-04-25T13:47:11.706182+0000","last_undegraded":"2024-04-25T13:47:11.706182+0000","last_fullsized":"2024-04-25T13:47:11.706182+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:31:57.828999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733262+0000","last_change":"2024-04-25T13:47:10.714623+0000","last_active":"2024-04-25T13:47:11.733262+0000","last_peered":"2024-04-25T13:47:11.733262+0000","last_clean":"2024-04-25T13:47:11.733262+0000","last_became_active":"2024-04-25T13:47:10.713811+0000","last_became_peered":"2024-04-25T13:47:10.713811+0000","last_unstale":"2024-04-25T13:47:11.733262+0000","last_undegraded":"2024-04-25T13:47:11.733262+0000","last_fullsized":"2024-04-25T13:47:11.733262+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:52.492699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733478+0000","last_change":"2024-04-25T13:47:10.714266+0000","last_active":"2024-04-25T13:47:11.733478+0000","last_peered":"2024-04-25T13:47:11.733478+0000","last_clean":"2024-04-25T13:47:11.733478+0000","last_became_active":"2024-04-25T13:47:10.711127+0000","last_became_peered":"2024-04-25T13:47:10.711127+0000","last_unstale":"2024-04-25T13:47:11.733478+0000","last_undegraded":"2024-04-25T13:47:11.733478+0000","last_fullsized":"2024-04-25T13:47:11.733478+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:11.234058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735827+0000","last_change":"2024-04-25T13:47:10.716561+0000","last_active":"2024-04-25T13:47:11.735827+0000","last_peered":"2024-04-25T13:47:11.735827+0000","last_clean":"2024-04-25T13:47:11.735827+0000","last_became_active":"2024-04-25T13:47:10.714810+0000","last_became_peered":"2024-04-25T13:47:10.714810+0000","last_unstale":"2024-04-25T13:47:11.735827+0000","last_undegraded":"2024-04-25T13:47:11.735827+0000","last_fullsized":"2024-04-25T13:47:11.735827+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:33.278707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732319+0000","last_change":"2024-04-25T13:47:10.727851+0000","last_active":"2024-04-25T13:47:11.732319+0000","last_peered":"2024-04-25T13:47:11.732319+0000","last_clean":"2024-04-25T13:47:11.732319+0000","last_became_active":"2024-04-25T13:47:10.727092+0000","last_became_peered":"2024-04-25T13:47:10.727092+0000","last_unstale":"2024-04-25T13:47:11.732319+0000","last_undegraded":"2024-04-25T13:47:11.732319+0000","last_fullsized":"2024-04-25T13:47:11.732319+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:34.245305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735203+0000","last_change":"2024-04-25T13:47:10.726514+0000","last_active":"2024-04-25T13:47:11.735203+0000","last_peered":"2024-04-25T13:47:11.735203+0000","last_clean":"2024-04-25T13:47:11.735203+0000","last_became_active":"2024-04-25T13:47:10.726237+0000","last_became_peered":"2024-04-25T13:47:10.726237+0000","last_unstale":"2024-04-25T13:47:11.735203+0000","last_undegraded":"2024-04-25T13:47:11.735203+0000","last_fullsized":"2024-04-25T13:47:11.735203+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:24.979487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732784+0000","last_change":"2024-04-25T13:47:10.719674+0000","last_active":"2024-04-25T13:47:11.732784+0000","last_peered":"2024-04-25T13:47:11.732784+0000","last_clean":"2024-04-25T13:47:11.732784+0000","last_became_active":"2024-04-25T13:47:10.718962+0000","last_became_peered":"2024-04-25T13:47:10.718962+0000","last_unstale":"2024-04-25T13:47:11.732784+0000","last_undegraded":"2024-04-25T13:47:11.732784+0000","last_fullsized":"2024-04-25T13:47:11.732784+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:45.367276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734080+0000","last_change":"2024-04-25T13:47:10.727152+0000","last_active":"2024-04-25T13:47:11.734080+0000","last_peered":"2024-04-25T13:47:11.734080+0000","last_clean":"2024-04-25T13:47:11.734080+0000","last_became_active":"2024-04-25T13:47:10.718669+0000","last_became_peered":"2024-04-25T13:47:10.718669+0000","last_unstale":"2024-04-25T13:47:11.734080+0000","last_undegraded":"2024-04-25T13:47:11.734080+0000","last_fullsized":"2024-04-25T13:47:11.734080+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:12:36.849024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697848+0000","last_change":"2024-04-25T13:47:10.723298+0000","last_active":"2024-04-25T13:47:11.697848+0000","last_peered":"2024-04-25T13:47:11.697848+0000","last_clean":"2024-04-25T13:47:11.697848+0000","last_became_active":"2024-04-25T13:47:10.720107+0000","last_became_peered":"2024-04-25T13:47:10.720107+0000","last_unstale":"2024-04-25T13:47:11.697848+0000","last_undegraded":"2024-04-25T13:47:11.697848+0000","last_fullsized":"2024-04-25T13:47:11.697848+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:27.487106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734926+0000","last_change":"2024-04-25T13:47:10.722026+0000","last_active":"2024-04-25T13:47:11.734926+0000","last_peered":"2024-04-25T13:47:11.734926+0000","last_clean":"2024-04-25T13:47:11.734926+0000","last_became_active":"2024-04-25T13:47:10.719261+0000","last_became_peered":"2024-04-25T13:47:10.719261+0000","last_unstale":"2024-04-25T13:47:11.734926+0000","last_undegraded":"2024-04-25T13:47:11.734926+0000","last_fullsized":"2024-04-25T13:47:11.734926+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:00.808019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732550+0000","last_change":"2024-04-25T13:47:10.725208+0000","last_active":"2024-04-25T13:47:11.732550+0000","last_peered":"2024-04-25T13:47:11.732550+0000","last_clean":"2024-04-25T13:47:11.732550+0000","last_became_active":"2024-04-25T13:47:10.724377+0000","last_became_peered":"2024-04-25T13:47:10.724377+0000","last_unstale":"2024-04-25T13:47:11.732550+0000","last_undegraded":"2024-04-25T13:47:11.732550+0000","last_fullsized":"2024-04-25T13:47:11.732550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:49.785000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728834+0000","last_change":"2024-04-25T13:47:10.717443+0000","last_active":"2024-04-25T13:47:11.728834+0000","last_peered":"2024-04-25T13:47:11.728834+0000","last_clean":"2024-04-25T13:47:11.728834+0000","last_became_active":"2024-04-25T13:47:10.716734+0000","last_became_peered":"2024-04-25T13:47:10.716734+0000","last_unstale":"2024-04-25T13:47:11.728834+0000","last_undegraded":"2024-04-25T13:47:11.728834+0000","last_fullsized":"2024-04-25T13:47:11.728834+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:19.058456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734304+0000","last_change":"2024-04-25T13:47:10.726875+0000","last_active":"2024-04-25T13:47:11.734304+0000","last_peered":"2024-04-25T13:47:11.734304+0000","last_clean":"2024-04-25T13:47:11.734304+0000","last_became_active":"2024-04-25T13:47:10.717310+0000","last_became_peered":"2024-04-25T13:47:10.717310+0000","last_unstale":"2024-04-25T13:47:11.734304+0000","last_undegraded":"2024-04-25T13:47:11.734304+0000","last_fullsized":"2024-04-25T13:47:11.734304+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:35.573010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698486+0000","last_change":"2024-04-25T13:47:10.723988+0000","last_active":"2024-04-25T13:47:11.698486+0000","last_peered":"2024-04-25T13:47:11.698486+0000","last_clean":"2024-04-25T13:47:11.698486+0000","last_became_active":"2024-04-25T13:47:10.721475+0000","last_became_peered":"2024-04-25T13:47:10.721475+0000","last_unstale":"2024-04-25T13:47:11.698486+0000","last_undegraded":"2024-04-25T13:47:11.698486+0000","last_fullsized":"2024-04-25T13:47:11.698486+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:32.493732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697310+0000","last_change":"2024-04-25T13:47:10.704315+0000","last_active":"2024-04-25T13:47:11.697310+0000","last_peered":"2024-04-25T13:47:11.697310+0000","last_clean":"2024-04-25T13:47:11.697310+0000","last_became_active":"2024-04-25T13:47:10.703571+0000","last_became_peered":"2024-04-25T13:47:10.703571+0000","last_unstale":"2024-04-25T13:47:11.697310+0000","last_undegraded":"2024-04-25T13:47:11.697310+0000","last_fullsized":"2024-04-25T13:47:11.697310+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:20.121342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"135'4","reported_seq":18,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728996+0000","last_change":"2024-04-25T13:47:09.682009+0000","last_active":"2024-04-25T13:47:11.728996+0000","last_peered":"2024-04-25T13:47:11.728996+0000","last_clean":"2024-04-25T13:47:11.728996+0000","last_became_active":"2024-04-25T13:47:09.681182+0000","last_became_peered":"2024-04-25T13:47:09.681182+0000","last_unstale":"2024-04-25T13:47:11.728996+0000","last_undegraded":"2024-04-25T13:47:11.728996+0000","last_fullsized":"2024-04-25T13:47:11.728996+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:54.568585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707124+0000","last_change":"2024-04-25T13:47:10.711064+0000","last_active":"2024-04-25T13:47:11.707124+0000","last_peered":"2024-04-25T13:47:11.707124+0000","last_clean":"2024-04-25T13:47:11.707124+0000","last_became_active":"2024-04-25T13:47:10.710331+0000","last_became_peered":"2024-04-25T13:47:10.710331+0000","last_unstale":"2024-04-25T13:47:11.707124+0000","last_undegraded":"2024-04-25T13:47:11.707124+0000","last_fullsized":"2024-04-25T13:47:11.707124+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:19:39.478214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707473+0000","last_change":"2024-04-25T13:47:09.674718+0000","last_active":"2024-04-25T13:47:11.707473+0000","last_peered":"2024-04-25T13:47:11.707473+0000","last_clean":"2024-04-25T13:47:11.707473+0000","last_became_active":"2024-04-25T13:47:09.673707+0000","last_became_peered":"2024-04-25T13:47:09.673707+0000","last_unstale":"2024-04-25T13:47:11.707473+0000","last_undegraded":"2024-04-25T13:47:11.707473+0000","last_fullsized":"2024-04-25T13:47:11.707473+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:29.296176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698141+0000","last_change":"2024-04-25T13:47:10.710337+0000","last_active":"2024-04-25T13:47:11.698141+0000","last_peered":"2024-04-25T13:47:11.698141+0000","last_clean":"2024-04-25T13:47:11.698141+0000","last_became_active":"2024-04-25T13:47:10.708714+0000","last_became_peered":"2024-04-25T13:47:10.708714+0000","last_unstale":"2024-04-25T13:47:11.698141+0000","last_undegraded":"2024-04-25T13:47:11.698141+0000","last_fullsized":"2024-04-25T13:47:11.698141+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:17.366901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699553+0000","last_change":"2024-04-25T13:47:09.681907+0000","last_active":"2024-04-25T13:47:11.699553+0000","last_peered":"2024-04-25T13:47:11.699553+0000","last_clean":"2024-04-25T13:47:11.699553+0000","last_became_active":"2024-04-25T13:47:09.680918+0000","last_became_peered":"2024-04-25T13:47:09.680918+0000","last_unstale":"2024-04-25T13:47:11.699553+0000","last_undegraded":"2024-04-25T13:47:11.699553+0000","last_fullsized":"2024-04-25T13:47:11.699553+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:50.073408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706237+0000","last_change":"2024-04-25T13:47:10.711635+0000","last_active":"2024-04-25T13:47:11.706237+0000","last_peered":"2024-04-25T13:47:11.706237+0000","last_clean":"2024-04-25T13:47:11.706237+0000","last_became_active":"2024-04-25T13:47:10.709693+0000","last_became_peered":"2024-04-25T13:47:10.709693+0000","last_unstale":"2024-04-25T13:47:11.706237+0000","last_undegraded":"2024-04-25T13:47:11.706237+0000","last_fullsized":"2024-04-25T13:47:11.706237+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:04.196466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.700209+0000","last_change":"2024-04-25T13:47:09.674753+0000","last_active":"2024-04-25T13:47:11.700209+0000","last_peered":"2024-04-25T13:47:11.700209+0000","last_clean":"2024-04-25T13:47:11.700209+0000","last_became_active":"2024-04-25T13:47:09.674140+0000","last_became_peered":"2024-04-25T13:47:09.674140+0000","last_unstale":"2024-04-25T13:47:11.700209+0000","last_undegraded":"2024-04-25T13:47:11.700209+0000","last_fullsized":"2024-04-25T13:47:11.700209+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:59:31.121015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733543+0000","last_change":"2024-04-25T13:47:10.719036+0000","last_active":"2024-04-25T13:47:11.733543+0000","last_peered":"2024-04-25T13:47:11.733543+0000","last_clean":"2024-04-25T13:47:11.733543+0000","last_became_active":"2024-04-25T13:47:10.713900+0000","last_became_peered":"2024-04-25T13:47:10.713900+0000","last_unstale":"2024-04-25T13:47:11.733543+0000","last_undegraded":"2024-04-25T13:47:11.733543+0000","last_fullsized":"2024-04-25T13:47:11.733543+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:57.714122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729098+0000","last_change":"2024-04-25T13:47:09.682797+0000","last_active":"2024-04-25T13:47:11.729098+0000","last_peered":"2024-04-25T13:47:11.729098+0000","last_clean":"2024-04-25T13:47:11.729098+0000","last_became_active":"2024-04-25T13:47:09.679375+0000","last_became_peered":"2024-04-25T13:47:09.679375+0000","last_unstale":"2024-04-25T13:47:11.729098+0000","last_undegraded":"2024-04-25T13:47:11.729098+0000","last_fullsized":"2024-04-25T13:47:11.729098+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:21.675830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733799+0000","last_change":"2024-04-25T13:47:10.715936+0000","last_active":"2024-04-25T13:47:11.733799+0000","last_peered":"2024-04-25T13:47:11.733799+0000","last_clean":"2024-04-25T13:47:11.733799+0000","last_became_active":"2024-04-25T13:47:10.714659+0000","last_became_peered":"2024-04-25T13:47:10.714659+0000","last_unstale":"2024-04-25T13:47:11.733799+0000","last_undegraded":"2024-04-25T13:47:11.733799+0000","last_fullsized":"2024-04-25T13:47:11.733799+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:39.174347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735223+0000","last_change":"2024-04-25T13:47:09.686782+0000","last_active":"2024-04-25T13:47:11.735223+0000","last_peered":"2024-04-25T13:47:11.735223+0000","last_clean":"2024-04-25T13:47:11.735223+0000","last_became_active":"2024-04-25T13:47:09.686553+0000","last_became_peered":"2024-04-25T13:47:09.686553+0000","last_unstale":"2024-04-25T13:47:11.735223+0000","last_undegraded":"2024-04-25T13:47:11.735223+0000","last_fullsized":"2024-04-25T13:47:11.735223+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:25:40.729092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734561+0000","last_change":"2024-04-25T13:47:10.723184+0000","last_active":"2024-04-25T13:47:11.734561+0000","last_peered":"2024-04-25T13:47:11.734561+0000","last_clean":"2024-04-25T13:47:11.734561+0000","last_became_active":"2024-04-25T13:47:10.714016+0000","last_became_peered":"2024-04-25T13:47:10.714016+0000","last_unstale":"2024-04-25T13:47:11.734561+0000","last_undegraded":"2024-04-25T13:47:11.734561+0000","last_fullsized":"2024-04-25T13:47:11.734561+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:37.800132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732744+0000","last_change":"2024-04-25T13:47:09.673427+0000","last_active":"2024-04-25T13:47:11.732744+0000","last_peered":"2024-04-25T13:47:11.732744+0000","last_clean":"2024-04-25T13:47:11.732744+0000","last_became_active":"2024-04-25T13:47:09.672383+0000","last_became_peered":"2024-04-25T13:47:09.672383+0000","last_unstale":"2024-04-25T13:47:11.732744+0000","last_undegraded":"2024-04-25T13:47:11.732744+0000","last_fullsized":"2024-04-25T13:47:11.732744+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:36.201666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728407+0000","last_change":"2024-04-25T13:47:10.704385+0000","last_active":"2024-04-25T13:47:11.728407+0000","last_peered":"2024-04-25T13:47:11.728407+0000","last_clean":"2024-04-25T13:47:11.728407+0000","last_became_active":"2024-04-25T13:47:10.703678+0000","last_became_peered":"2024-04-25T13:47:10.703678+0000","last_unstale":"2024-04-25T13:47:11.728407+0000","last_undegraded":"2024-04-25T13:47:11.728407+0000","last_fullsized":"2024-04-25T13:47:11.728407+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:54.503945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735255+0000","last_change":"2024-04-25T13:47:09.673539+0000","last_active":"2024-04-25T13:47:11.735255+0000","last_peered":"2024-04-25T13:47:11.735255+0000","last_clean":"2024-04-25T13:47:11.735255+0000","last_became_active":"2024-04-25T13:47:09.672528+0000","last_became_peered":"2024-04-25T13:47:09.672528+0000","last_unstale":"2024-04-25T13:47:11.735255+0000","last_undegraded":"2024-04-25T13:47:11.735255+0000","last_fullsized":"2024-04-25T13:47:11.735255+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:21.673022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728832+0000","last_change":"2024-04-25T13:47:10.709949+0000","last_active":"2024-04-25T13:47:11.728832+0000","last_peered":"2024-04-25T13:47:11.728832+0000","last_clean":"2024-04-25T13:47:11.728832+0000","last_became_active":"2024-04-25T13:47:10.709077+0000","last_became_peered":"2024-04-25T13:47:10.709077+0000","last_unstale":"2024-04-25T13:47:11.728832+0000","last_undegraded":"2024-04-25T13:47:11.728832+0000","last_fullsized":"2024-04-25T13:47:11.728832+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:23.249112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.700397+0000","last_change":"2024-04-25T13:47:09.674999+0000","last_active":"2024-04-25T13:47:11.700397+0000","last_peered":"2024-04-25T13:47:11.700397+0000","last_clean":"2024-04-25T13:47:11.700397+0000","last_became_active":"2024-04-25T13:47:09.674495+0000","last_became_peered":"2024-04-25T13:47:09.674495+0000","last_unstale":"2024-04-25T13:47:11.700397+0000","last_undegraded":"2024-04-25T13:47:11.700397+0000","last_fullsized":"2024-04-25T13:47:11.700397+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:47.383571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.708299+0000","last_change":"2024-04-25T13:47:10.728205+0000","last_active":"2024-04-25T13:47:11.708299+0000","last_peered":"2024-04-25T13:47:11.708299+0000","last_clean":"2024-04-25T13:47:11.708299+0000","last_became_active":"2024-04-25T13:47:10.727269+0000","last_became_peered":"2024-04-25T13:47:10.727269+0000","last_unstale":"2024-04-25T13:47:11.708299+0000","last_undegraded":"2024-04-25T13:47:11.708299+0000","last_fullsized":"2024-04-25T13:47:11.708299+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:38.834861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736535+0000","last_change":"2024-04-25T13:47:09.685948+0000","last_active":"2024-04-25T13:47:11.736535+0000","last_peered":"2024-04-25T13:47:11.736535+0000","last_clean":"2024-04-25T13:47:11.736535+0000","last_became_active":"2024-04-25T13:47:09.685716+0000","last_became_peered":"2024-04-25T13:47:09.685716+0000","last_unstale":"2024-04-25T13:47:11.736535+0000","last_undegraded":"2024-04-25T13:47:11.736535+0000","last_fullsized":"2024-04-25T13:47:11.736535+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:57.872201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735655+0000","last_change":"2024-04-25T13:47:10.711143+0000","last_active":"2024-04-25T13:47:11.735655+0000","last_peered":"2024-04-25T13:47:11.735655+0000","last_clean":"2024-04-25T13:47:11.735655+0000","last_became_active":"2024-04-25T13:47:10.710457+0000","last_became_peered":"2024-04-25T13:47:10.710457+0000","last_unstale":"2024-04-25T13:47:11.735655+0000","last_undegraded":"2024-04-25T13:47:11.735655+0000","last_fullsized":"2024-04-25T13:47:11.735655+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:56.109270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"135'3","reported_seq":17,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734300+0000","last_change":"2024-04-25T13:47:09.674010+0000","last_active":"2024-04-25T13:47:11.734300+0000","last_peered":"2024-04-25T13:47:11.734300+0000","last_clean":"2024-04-25T13:47:11.734300+0000","last_became_active":"2024-04-25T13:47:09.673459+0000","last_became_peered":"2024-04-25T13:47:09.673459+0000","last_unstale":"2024-04-25T13:47:11.734300+0000","last_undegraded":"2024-04-25T13:47:11.734300+0000","last_fullsized":"2024-04-25T13:47:11.734300+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:31.446118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.728726+0000","last_change":"2024-04-25T13:47:10.701426+0000","last_active":"2024-04-25T13:47:11.728726+0000","last_peered":"2024-04-25T13:47:11.728726+0000","last_clean":"2024-04-25T13:47:11.728726+0000","last_became_active":"2024-04-25T13:47:10.700653+0000","last_became_peered":"2024-04-25T13:47:10.700653+0000","last_unstale":"2024-04-25T13:47:11.728726+0000","last_undegraded":"2024-04-25T13:47:11.728726+0000","last_fullsized":"2024-04-25T13:47:11.728726+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:25.584164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"135'6","reported_seq":20,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735681+0000","last_change":"2024-04-25T13:47:09.673006+0000","last_active":"2024-04-25T13:47:11.735681+0000","last_peered":"2024-04-25T13:47:11.735681+0000","last_clean":"2024-04-25T13:47:11.735681+0000","last_became_active":"2024-04-25T13:47:09.672656+0000","last_became_peered":"2024-04-25T13:47:09.672656+0000","last_unstale":"2024-04-25T13:47:11.735681+0000","last_undegraded":"2024-04-25T13:47:11.735681+0000","last_fullsized":"2024-04-25T13:47:11.735681+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:09.587533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736051+0000","last_change":"2024-04-25T13:47:10.710716+0000","last_active":"2024-04-25T13:47:11.736051+0000","last_peered":"2024-04-25T13:47:11.736051+0000","last_clean":"2024-04-25T13:47:11.736051+0000","last_became_active":"2024-04-25T13:47:10.710023+0000","last_became_peered":"2024-04-25T13:47:10.710023+0000","last_unstale":"2024-04-25T13:47:11.736051+0000","last_undegraded":"2024-04-25T13:47:11.736051+0000","last_fullsized":"2024-04-25T13:47:11.736051+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:53.029454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698056+0000","last_change":"2024-04-25T13:47:09.677556+0000","last_active":"2024-04-25T13:47:11.698056+0000","last_peered":"2024-04-25T13:47:11.698056+0000","last_clean":"2024-04-25T13:47:11.698056+0000","last_became_active":"2024-04-25T13:47:09.676926+0000","last_became_peered":"2024-04-25T13:47:09.676926+0000","last_unstale":"2024-04-25T13:47:11.698056+0000","last_undegraded":"2024-04-25T13:47:11.698056+0000","last_fullsized":"2024-04-25T13:47:11.698056+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:24.384131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735798+0000","last_change":"2024-04-25T13:47:10.718863+0000","last_active":"2024-04-25T13:47:11.735798+0000","last_peered":"2024-04-25T13:47:11.735798+0000","last_clean":"2024-04-25T13:47:11.735798+0000","last_became_active":"2024-04-25T13:47:10.717846+0000","last_became_peered":"2024-04-25T13:47:10.717846+0000","last_unstale":"2024-04-25T13:47:11.735798+0000","last_undegraded":"2024-04-25T13:47:11.735798+0000","last_fullsized":"2024-04-25T13:47:11.735798+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:06.641310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.708588+0000","last_change":"2024-04-25T13:47:09.687086+0000","last_active":"2024-04-25T13:47:11.708588+0000","last_peered":"2024-04-25T13:47:11.708588+0000","last_clean":"2024-04-25T13:47:11.708588+0000","last_became_active":"2024-04-25T13:47:09.686017+0000","last_became_peered":"2024-04-25T13:47:09.686017+0000","last_unstale":"2024-04-25T13:47:11.708588+0000","last_undegraded":"2024-04-25T13:47:11.708588+0000","last_fullsized":"2024-04-25T13:47:11.708588+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:27:22.637768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736264+0000","last_change":"2024-04-25T13:47:10.717412+0000","last_active":"2024-04-25T13:47:11.736264+0000","last_peered":"2024-04-25T13:47:11.736264+0000","last_clean":"2024-04-25T13:47:11.736264+0000","last_became_active":"2024-04-25T13:47:10.715835+0000","last_became_peered":"2024-04-25T13:47:10.715835+0000","last_unstale":"2024-04-25T13:47:11.736264+0000","last_undegraded":"2024-04-25T13:47:11.736264+0000","last_fullsized":"2024-04-25T13:47:11.736264+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:41.116680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"135'8","reported_seq":22,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698674+0000","last_change":"2024-04-25T13:47:09.678846+0000","last_active":"2024-04-25T13:47:11.698674+0000","last_peered":"2024-04-25T13:47:11.698674+0000","last_clean":"2024-04-25T13:47:11.698674+0000","last_became_active":"2024-04-25T13:47:09.677706+0000","last_became_peered":"2024-04-25T13:47:09.677706+0000","last_unstale":"2024-04-25T13:47:11.698674+0000","last_undegraded":"2024-04-25T13:47:11.698674+0000","last_fullsized":"2024-04-25T13:47:11.698674+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:55.968598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699339+0000","last_change":"2024-04-25T13:47:10.721946+0000","last_active":"2024-04-25T13:47:11.699339+0000","last_peered":"2024-04-25T13:47:11.699339+0000","last_clean":"2024-04-25T13:47:11.699339+0000","last_became_active":"2024-04-25T13:47:10.720879+0000","last_became_peered":"2024-04-25T13:47:10.720879+0000","last_unstale":"2024-04-25T13:47:11.699339+0000","last_undegraded":"2024-04-25T13:47:11.699339+0000","last_fullsized":"2024-04-25T13:47:11.699339+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:27:51.842983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735980+0000","last_change":"2024-04-25T13:47:09.675954+0000","last_active":"2024-04-25T13:47:11.735980+0000","last_peered":"2024-04-25T13:47:11.735980+0000","last_clean":"2024-04-25T13:47:11.735980+0000","last_became_active":"2024-04-25T13:47:09.675437+0000","last_became_peered":"2024-04-25T13:47:09.675437+0000","last_unstale":"2024-04-25T13:47:11.735980+0000","last_undegraded":"2024-04-25T13:47:11.735980+0000","last_fullsized":"2024-04-25T13:47:11.735980+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:30.735437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697757+0000","last_change":"2024-04-25T13:47:10.712195+0000","last_active":"2024-04-25T13:47:11.697757+0000","last_peered":"2024-04-25T13:47:11.697757+0000","last_clean":"2024-04-25T13:47:11.697757+0000","last_became_active":"2024-04-25T13:47:10.711487+0000","last_became_peered":"2024-04-25T13:47:10.711487+0000","last_unstale":"2024-04-25T13:47:11.697757+0000","last_undegraded":"2024-04-25T13:47:11.697757+0000","last_fullsized":"2024-04-25T13:47:11.697757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:24.893923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699681+0000","last_change":"2024-04-25T13:47:09.682498+0000","last_active":"2024-04-25T13:47:11.699681+0000","last_peered":"2024-04-25T13:47:11.699681+0000","last_clean":"2024-04-25T13:47:11.699681+0000","last_became_active":"2024-04-25T13:47:09.681492+0000","last_became_peered":"2024-04-25T13:47:09.681492+0000","last_unstale":"2024-04-25T13:47:11.699681+0000","last_undegraded":"2024-04-25T13:47:11.699681+0000","last_fullsized":"2024-04-25T13:47:11.699681+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:32.184310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.705433+0000","last_change":"2024-04-25T13:47:10.713277+0000","last_active":"2024-04-25T13:47:11.705433+0000","last_peered":"2024-04-25T13:47:11.705433+0000","last_clean":"2024-04-25T13:47:11.705433+0000","last_became_active":"2024-04-25T13:47:10.712479+0000","last_became_peered":"2024-04-25T13:47:10.712479+0000","last_unstale":"2024-04-25T13:47:11.705433+0000","last_undegraded":"2024-04-25T13:47:11.705433+0000","last_fullsized":"2024-04-25T13:47:11.705433+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:48:52.455886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"135'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.708925+0000","last_change":"2024-04-25T13:47:09.686695+0000","last_active":"2024-04-25T13:47:11.708925+0000","last_peered":"2024-04-25T13:47:11.708925+0000","last_clean":"2024-04-25T13:47:11.708925+0000","last_became_active":"2024-04-25T13:47:09.685056+0000","last_became_peered":"2024-04-25T13:47:09.685056+0000","last_unstale":"2024-04-25T13:47:11.708925+0000","last_undegraded":"2024-04-25T13:47:11.708925+0000","last_fullsized":"2024-04-25T13:47:11.708925+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:20.181056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.705727+0000","last_change":"2024-04-25T13:47:10.703818+0000","last_active":"2024-04-25T13:47:11.705727+0000","last_peered":"2024-04-25T13:47:11.705727+0000","last_clean":"2024-04-25T13:47:11.705727+0000","last_became_active":"2024-04-25T13:47:10.703079+0000","last_became_peered":"2024-04-25T13:47:10.703079+0000","last_unstale":"2024-04-25T13:47:11.705727+0000","last_undegraded":"2024-04-25T13:47:11.705727+0000","last_fullsized":"2024-04-25T13:47:11.705727+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:20:34.831413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729874+0000","last_change":"2024-04-25T13:47:09.683617+0000","last_active":"2024-04-25T13:47:11.729874+0000","last_peered":"2024-04-25T13:47:11.729874+0000","last_clean":"2024-04-25T13:47:11.729874+0000","last_became_active":"2024-04-25T13:47:09.680631+0000","last_became_peered":"2024-04-25T13:47:09.680631+0000","last_unstale":"2024-04-25T13:47:11.729874+0000","last_undegraded":"2024-04-25T13:47:11.729874+0000","last_fullsized":"2024-04-25T13:47:11.729874+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:29.636035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707088+0000","last_change":"2024-04-25T13:47:10.729244+0000","last_active":"2024-04-25T13:47:11.707088+0000","last_peered":"2024-04-25T13:47:11.707088+0000","last_clean":"2024-04-25T13:47:11.707088+0000","last_became_active":"2024-04-25T13:47:10.724005+0000","last_became_peered":"2024-04-25T13:47:10.724005+0000","last_unstale":"2024-04-25T13:47:11.707088+0000","last_undegraded":"2024-04-25T13:47:11.707088+0000","last_fullsized":"2024-04-25T13:47:11.707088+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:18:55.933401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.697159+0000","last_change":"2024-04-25T13:47:09.680220+0000","last_active":"2024-04-25T13:47:11.697159+0000","last_peered":"2024-04-25T13:47:11.697159+0000","last_clean":"2024-04-25T13:47:11.697159+0000","last_became_active":"2024-04-25T13:47:09.679844+0000","last_became_peered":"2024-04-25T13:47:09.679844+0000","last_unstale":"2024-04-25T13:47:11.697159+0000","last_undegraded":"2024-04-25T13:47:11.697159+0000","last_fullsized":"2024-04-25T13:47:11.697159+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:27.888201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733910+0000","last_change":"2024-04-25T13:47:10.715674+0000","last_active":"2024-04-25T13:47:11.733910+0000","last_peered":"2024-04-25T13:47:11.733910+0000","last_clean":"2024-04-25T13:47:11.733910+0000","last_became_active":"2024-04-25T13:47:10.712719+0000","last_became_peered":"2024-04-25T13:47:10.712719+0000","last_unstale":"2024-04-25T13:47:11.733910+0000","last_undegraded":"2024-04-25T13:47:11.733910+0000","last_fullsized":"2024-04-25T13:47:11.733910+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:17.481231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.699020+0000","last_change":"2024-04-25T13:47:09.675253+0000","last_active":"2024-04-25T13:47:11.699020+0000","last_peered":"2024-04-25T13:47:11.699020+0000","last_clean":"2024-04-25T13:47:11.699020+0000","last_became_active":"2024-04-25T13:47:09.674726+0000","last_became_peered":"2024-04-25T13:47:09.674726+0000","last_unstale":"2024-04-25T13:47:11.699020+0000","last_undegraded":"2024-04-25T13:47:11.699020+0000","last_fullsized":"2024-04-25T13:47:11.699020+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:50.769092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706201+0000","last_change":"2024-04-25T13:47:10.708913+0000","last_active":"2024-04-25T13:47:11.706201+0000","last_peered":"2024-04-25T13:47:11.706201+0000","last_clean":"2024-04-25T13:47:11.706201+0000","last_became_active":"2024-04-25T13:47:10.707779+0000","last_became_peered":"2024-04-25T13:47:10.707779+0000","last_unstale":"2024-04-25T13:47:11.706201+0000","last_undegraded":"2024-04-25T13:47:11.706201+0000","last_fullsized":"2024-04-25T13:47:11.706201+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:20:54.431922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698169+0000","last_change":"2024-04-25T13:47:09.682911+0000","last_active":"2024-04-25T13:47:11.698169+0000","last_peered":"2024-04-25T13:47:11.698169+0000","last_clean":"2024-04-25T13:47:11.698169+0000","last_became_active":"2024-04-25T13:47:09.682630+0000","last_became_peered":"2024-04-25T13:47:09.682630+0000","last_unstale":"2024-04-25T13:47:11.698169+0000","last_undegraded":"2024-04-25T13:47:11.698169+0000","last_fullsized":"2024-04-25T13:47:11.698169+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:58.767649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735978+0000","last_change":"2024-04-25T13:47:10.724260+0000","last_active":"2024-04-25T13:47:11.735978+0000","last_peered":"2024-04-25T13:47:11.735978+0000","last_clean":"2024-04-25T13:47:11.735978+0000","last_became_active":"2024-04-25T13:47:10.722596+0000","last_became_peered":"2024-04-25T13:47:10.722596+0000","last_unstale":"2024-04-25T13:47:11.735978+0000","last_undegraded":"2024-04-25T13:47:11.735978+0000","last_fullsized":"2024-04-25T13:47:11.735978+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:13.545122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734579+0000","last_change":"2024-04-25T13:47:09.687931+0000","last_active":"2024-04-25T13:47:11.734579+0000","last_peered":"2024-04-25T13:47:11.734579+0000","last_clean":"2024-04-25T13:47:11.734579+0000","last_became_active":"2024-04-25T13:47:09.687516+0000","last_became_peered":"2024-04-25T13:47:09.687516+0000","last_unstale":"2024-04-25T13:47:11.734579+0000","last_undegraded":"2024-04-25T13:47:11.734579+0000","last_fullsized":"2024-04-25T13:47:11.734579+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:59.666307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707583+0000","last_change":"2024-04-25T13:47:10.720440+0000","last_active":"2024-04-25T13:47:11.707583+0000","last_peered":"2024-04-25T13:47:11.707583+0000","last_clean":"2024-04-25T13:47:11.707583+0000","last_became_active":"2024-04-25T13:47:10.715651+0000","last_became_peered":"2024-04-25T13:47:10.715651+0000","last_unstale":"2024-04-25T13:47:11.707583+0000","last_undegraded":"2024-04-25T13:47:11.707583+0000","last_fullsized":"2024-04-25T13:47:11.707583+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:37.217172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"135'2","reported_seq":16,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.737125+0000","last_change":"2024-04-25T13:47:09.674420+0000","last_active":"2024-04-25T13:47:11.737125+0000","last_peered":"2024-04-25T13:47:11.737125+0000","last_clean":"2024-04-25T13:47:11.737125+0000","last_became_active":"2024-04-25T13:47:09.674150+0000","last_became_peered":"2024-04-25T13:47:09.674150+0000","last_unstale":"2024-04-25T13:47:11.737125+0000","last_undegraded":"2024-04-25T13:47:11.737125+0000","last_fullsized":"2024-04-25T13:47:11.737125+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:46.065746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.733481+0000","last_change":"2024-04-25T13:47:10.712327+0000","last_active":"2024-04-25T13:47:11.733481+0000","last_peered":"2024-04-25T13:47:11.733481+0000","last_clean":"2024-04-25T13:47:11.733481+0000","last_became_active":"2024-04-25T13:47:10.708854+0000","last_became_peered":"2024-04-25T13:47:10.708854+0000","last_unstale":"2024-04-25T13:47:11.733481+0000","last_undegraded":"2024-04-25T13:47:11.733481+0000","last_fullsized":"2024-04-25T13:47:11.733481+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:43.868098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"136'1","reported_seq":15,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729044+0000","last_change":"2024-04-25T13:47:09.685680+0000","last_active":"2024-04-25T13:47:11.729044+0000","last_peered":"2024-04-25T13:47:11.729044+0000","last_clean":"2024-04-25T13:47:11.729044+0000","last_became_active":"2024-04-25T13:47:09.684307+0000","last_became_peered":"2024-04-25T13:47:09.684307+0000","last_unstale":"2024-04-25T13:47:11.729044+0000","last_undegraded":"2024-04-25T13:47:11.729044+0000","last_fullsized":"2024-04-25T13:47:11.729044+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:58:52.716976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734979+0000","last_change":"2024-04-25T13:47:10.722891+0000","last_active":"2024-04-25T13:47:11.734979+0000","last_peered":"2024-04-25T13:47:11.734979+0000","last_clean":"2024-04-25T13:47:11.734979+0000","last_became_active":"2024-04-25T13:47:10.713062+0000","last_became_peered":"2024-04-25T13:47:10.713062+0000","last_unstale":"2024-04-25T13:47:11.734979+0000","last_undegraded":"2024-04-25T13:47:11.734979+0000","last_fullsized":"2024-04-25T13:47:11.734979+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:07:16.779445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736247+0000","last_change":"2024-04-25T13:47:09.689101+0000","last_active":"2024-04-25T13:47:11.736247+0000","last_peered":"2024-04-25T13:47:11.736247+0000","last_clean":"2024-04-25T13:47:11.736247+0000","last_became_active":"2024-04-25T13:47:09.688456+0000","last_became_peered":"2024-04-25T13:47:09.688456+0000","last_unstale":"2024-04-25T13:47:11.736247+0000","last_undegraded":"2024-04-25T13:47:11.736247+0000","last_fullsized":"2024-04-25T13:47:11.736247+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:49.842860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706987+0000","last_change":"2024-04-25T13:47:10.726548+0000","last_active":"2024-04-25T13:47:11.706987+0000","last_peered":"2024-04-25T13:47:11.706987+0000","last_clean":"2024-04-25T13:47:11.706987+0000","last_became_active":"2024-04-25T13:47:10.725467+0000","last_became_peered":"2024-04-25T13:47:10.725467+0000","last_unstale":"2024-04-25T13:47:11.706987+0000","last_undegraded":"2024-04-25T13:47:11.706987+0000","last_fullsized":"2024-04-25T13:47:11.706987+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:22.314753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736107+0000","last_change":"2024-04-25T13:47:09.667421+0000","last_active":"2024-04-25T13:47:11.736107+0000","last_peered":"2024-04-25T13:47:11.736107+0000","last_clean":"2024-04-25T13:47:11.736107+0000","last_became_active":"2024-04-25T13:47:09.667125+0000","last_became_peered":"2024-04-25T13:47:09.667125+0000","last_unstale":"2024-04-25T13:47:11.736107+0000","last_undegraded":"2024-04-25T13:47:11.736107+0000","last_fullsized":"2024-04-25T13:47:11.736107+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:08.137395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734269+0000","last_change":"2024-04-25T13:47:10.705223+0000","last_active":"2024-04-25T13:47:11.734269+0000","last_peered":"2024-04-25T13:47:11.734269+0000","last_clean":"2024-04-25T13:47:11.734269+0000","last_became_active":"2024-04-25T13:47:10.700343+0000","last_became_peered":"2024-04-25T13:47:10.700343+0000","last_unstale":"2024-04-25T13:47:11.734269+0000","last_undegraded":"2024-04-25T13:47:11.734269+0000","last_fullsized":"2024-04-25T13:47:11.734269+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:30.822799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729392+0000","last_change":"2024-04-25T13:47:09.681287+0000","last_active":"2024-04-25T13:47:11.729392+0000","last_peered":"2024-04-25T13:47:11.729392+0000","last_clean":"2024-04-25T13:47:11.729392+0000","last_became_active":"2024-04-25T13:47:09.680381+0000","last_became_peered":"2024-04-25T13:47:09.680381+0000","last_unstale":"2024-04-25T13:47:11.729392+0000","last_undegraded":"2024-04-25T13:47:11.729392+0000","last_fullsized":"2024-04-25T13:47:11.729392+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:51.766362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.732920+0000","last_change":"2024-04-25T13:47:10.713257+0000","last_active":"2024-04-25T13:47:11.732920+0000","last_peered":"2024-04-25T13:47:11.732920+0000","last_clean":"2024-04-25T13:47:11.732920+0000","last_became_active":"2024-04-25T13:47:10.709365+0000","last_became_peered":"2024-04-25T13:47:10.709365+0000","last_unstale":"2024-04-25T13:47:11.732920+0000","last_undegraded":"2024-04-25T13:47:11.732920+0000","last_fullsized":"2024-04-25T13:47:11.732920+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:28.292174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734582+0000","last_change":"2024-04-25T13:47:09.685419+0000","last_active":"2024-04-25T13:47:11.734582+0000","last_peered":"2024-04-25T13:47:11.734582+0000","last_clean":"2024-04-25T13:47:11.734582+0000","last_became_active":"2024-04-25T13:47:09.683323+0000","last_became_peered":"2024-04-25T13:47:09.683323+0000","last_unstale":"2024-04-25T13:47:11.734582+0000","last_undegraded":"2024-04-25T13:47:11.734582+0000","last_fullsized":"2024-04-25T13:47:11.734582+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:31.462591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.706638+0000","last_change":"2024-04-25T13:47:10.731341+0000","last_active":"2024-04-25T13:47:11.706638+0000","last_peered":"2024-04-25T13:47:11.706638+0000","last_clean":"2024-04-25T13:47:11.706638+0000","last_became_active":"2024-04-25T13:47:10.728523+0000","last_became_peered":"2024-04-25T13:47:10.728523+0000","last_unstale":"2024-04-25T13:47:11.706638+0000","last_undegraded":"2024-04-25T13:47:11.706638+0000","last_fullsized":"2024-04-25T13:47:11.706638+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:13.373575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.698792+0000","last_change":"2024-04-25T13:47:09.678277+0000","last_active":"2024-04-25T13:47:11.698792+0000","last_peered":"2024-04-25T13:47:11.698792+0000","last_clean":"2024-04-25T13:47:11.698792+0000","last_became_active":"2024-04-25T13:47:09.677007+0000","last_became_peered":"2024-04-25T13:47:09.677007+0000","last_unstale":"2024-04-25T13:47:11.698792+0000","last_undegraded":"2024-04-25T13:47:11.698792+0000","last_fullsized":"2024-04-25T13:47:11.698792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:15:10.815071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.734133+0000","last_change":"2024-04-25T13:47:10.714638+0000","last_active":"2024-04-25T13:47:11.734133+0000","last_peered":"2024-04-25T13:47:11.734133+0000","last_clean":"2024-04-25T13:47:11.734133+0000","last_became_active":"2024-04-25T13:47:10.709924+0000","last_became_peered":"2024-04-25T13:47:10.709924+0000","last_unstale":"2024-04-25T13:47:11.734133+0000","last_undegraded":"2024-04-25T13:47:11.734133+0000","last_fullsized":"2024-04-25T13:47:11.734133+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:10.542469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.736499+0000","last_change":"2024-04-25T13:47:09.688832+0000","last_active":"2024-04-25T13:47:11.736499+0000","last_peered":"2024-04-25T13:47:11.736499+0000","last_clean":"2024-04-25T13:47:11.736499+0000","last_became_active":"2024-04-25T13:47:09.688091+0000","last_became_peered":"2024-04-25T13:47:09.688091+0000","last_unstale":"2024-04-25T13:47:11.736499+0000","last_undegraded":"2024-04-25T13:47:11.736499+0000","last_fullsized":"2024-04-25T13:47:11.736499+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:42.054034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.735102+0000","last_change":"2024-04-25T13:47:10.715441+0000","last_active":"2024-04-25T13:47:11.735102+0000","last_peered":"2024-04-25T13:47:11.735102+0000","last_clean":"2024-04-25T13:47:11.735102+0000","last_became_active":"2024-04-25T13:47:10.714643+0000","last_became_peered":"2024-04-25T13:47:10.714643+0000","last_unstale":"2024-04-25T13:47:11.735102+0000","last_undegraded":"2024-04-25T13:47:11.735102+0000","last_fullsized":"2024-04-25T13:47:11.735102+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:09.649285+0000","last_clean_scrub_stamp":"2024-04-25T13:47:09.649285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:42.100752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.729600+0000","last_change":"2024-04-25T13:47:09.684332+0000","last_active":"2024-04-25T13:47:11.729600+0000","last_peered":"2024-04-25T13:47:11.729600+0000","last_clean":"2024-04-25T13:47:11.729600+0000","last_became_active":"2024-04-25T13:47:09.681584+0000","last_became_peered":"2024-04-25T13:47:09.681584+0000","last_unstale":"2024-04-25T13:47:11.729600+0000","last_undegraded":"2024-04-25T13:47:11.729600+0000","last_fullsized":"2024-04-25T13:47:11.729600+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:47:08.636405+0000","last_clean_scrub_stamp":"2024-04-25T13:47:08.636405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:50.455303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":339,"reported_epoch":136,"state":"active+clean","last_fresh":"2024-04-25T13:47:11.707676+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:47:11.707676+0000","last_peered":"2024-04-25T13:47:11.707676+0000","last_clean":"2024-04-25T13:47:11.707676+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:47:11.707676+0000","last_undegraded":"2024-04-25T13:47:11.707676+0000","last_fullsized":"2024-04-25T13:47:11.707676+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574933,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1314,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193997,"statfs":{"total":200000000,"available":198653754,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574933,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":737,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244668,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574933,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":738,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244134,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574933,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":737,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244738,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574933,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":739,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194573,"statfs":{"total":200000000,"available":199243056,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574933,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":737,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244738,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":737,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194574,"statfs":{"total":200000000,"available":199244614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1315,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193996,"statfs":{"total":200000000,"available":198652646,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:47:15.205 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:47:15.205 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:47:15.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.398+0000 7f7f1114e640 1 Processor -- start 2024-04-25T13:47:15.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.398+0000 7f7f1114e640 1 -- start start 2024-04-25T13:47:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.398+0000 7f7f1114e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f0c161f90 0x7f7f0c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.398+0000 7f7f1114e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.398+0000 7f7f1114e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f0c0d0d70 con 0x7f7f0c15ea70 2024-04-25T13:47:15.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.398+0000 7f7f1114e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7f0c0d1050 con 0x7f7f0c161f90 2024-04-25T13:47:15.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.398+0000 7f7f1114e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c06dca0 con 0x7f7f0c0d0510 2024-04-25T13:47:15.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0a59c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0ad9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f0c161f90 0x7f7f0c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0ad9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7f0c161f90 0x7f7f0c0cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:51472/0 (socket says 172.21.15.80:51472) 2024-04-25T13:47:15.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0b59e640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f7f0c15ea70 0x7f7f0c15ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:55986/0 (socket says 172.21.15.80:55986) 2024-04-25T13:47:15.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0a59c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:43784/0 (socket says 172.21.15.80:43784) 2024-04-25T13:47:15.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0ad9d640 1 -- 172.21.15.80:0/723013863 learned_addr learned my addr 172.21.15.80:0/723013863 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:15.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0ad9d640 1 -- 172.21.15.80:0/723013863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 msgr2=0x7f7f0c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:15.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0ad9d640 1 --2- 172.21.15.80:0/723013863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0ad9d640 1 -- 172.21.15.80:0/723013863 >> v1:172.21.15.87:6789/0 conn(0x7f7f0c15ea70 legacy=0x7f7f0c15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0ad9d640 1 -- 172.21.15.80:0/723013863 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c06df80 con 0x7f7f0c161f90 2024-04-25T13:47:15.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0a59c640 1 --2- 172.21.15.80:0/723013863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:15.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f0ad9d640 1 --2- 172.21.15.80:0/723013863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 0x7f7f0c0cffd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009870 tx=0x7f7efc01c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3ae0a615fc3b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:15.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f09d9b640 1 -- 172.21.15.80:0/723013863 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0b1070 con 0x7f7f0c161f90 2024-04-25T13:47:15.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f09d9b640 1 -- 172.21.15.80:0/723013863 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7efc0ac600 con 0x7f7f0c161f90 2024-04-25T13:47:15.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.402+0000 7f7f09d9b640 1 -- 172.21.15.80:0/723013863 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc0ac9d0 con 0x7f7f0c161f90 2024-04-25T13:47:15.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 -- 172.21.15.80:0/723013863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 msgr2=0x7f7f0c0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 --2- 172.21.15.80:0/723013863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 0x7f7f0c0cffd0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009870 tx=0x7f7efc01c470 comp rx=0 tx=0).stop 2024-04-25T13:47:15.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 -- 172.21.15.80:0/723013863 shutdown_connections 2024-04-25T13:47:15.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 --2- 172.21.15.80:0/723013863 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 --2- 172.21.15.80:0/723013863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 0x7f7f0c0cffd0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 -- 172.21.15.80:0/723013863 >> 172.21.15.80:0/723013863 conn(0x7f7f0c06d080 msgr2=0x7f7f0c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:15.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 -- 172.21.15.80:0/723013863 shutdown_connections 2024-04-25T13:47:15.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 -- 172.21.15.80:0/723013863 wait complete. 2024-04-25T13:47:15.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 Processor -- start 2024-04-25T13:47:15.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.406+0000 7f7f1114e640 1 -- start start 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f0c15ea70 0x7f7f0c15a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 0x7f7f0c15aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c06f4b0 con 0x7f7f0c15ea70 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c06f630 con 0x7f7f0c161f90 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c0d0920 con 0x7f7f0c0d0510 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f0c15ea70 0x7f7f0c15a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f0c15ea70 0x7f7f0c15a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:52998/0 (socket says 172.21.15.80:52998) 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 -- 172.21.15.80:0/439390546 learned_addr learned my addr 172.21.15.80:0/439390546 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0b59e640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 0x7f7f0c15aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0ad9d640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 -- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 msgr2=0x7f7f0c140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c140b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 -- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 msgr2=0x7f7f0c15aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 0x7f7f0c15aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 -- 172.21.15.80:0/439390546 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c15f190 con 0x7f7f0c15ea70 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0b59e640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 0x7f7f0c15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f0a59c640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f0c15ea70 0x7f7f0c15a530 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000e9b0 tx=0x7f7f0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7ef37fe640 1 -- 172.21.15.80:0/439390546 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f00017070 con 0x7f7f0c15ea70 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7ef37fe640 1 -- 172.21.15.80:0/439390546 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f0000ca00 con 0x7f7f0c15ea70 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0c163c80 con 0x7f7f0c15ea70 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7ef37fe640 1 -- 172.21.15.80:0/439390546 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0000cdd0 con 0x7f7f0c15ea70 2024-04-25T13:47:15.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0c164240 con 0x7f7f0c15ea70 2024-04-25T13:47:15.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0c06a3a0 con 0x7f7f0c15ea70 2024-04-25T13:47:15.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.410+0000 7f7ef37fe640 1 -- 172.21.15.80:0/439390546 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7f0001a070 con 0x7f7f0c15ea70 2024-04-25T13:47:15.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.414+0000 7f7ef37fe640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ef806f430 0x7f7ef8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.414+0000 7f7ef37fe640 1 -- 172.21.15.80:0/439390546 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f7f0008f030 con 0x7f7f0c15ea70 2024-04-25T13:47:15.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.414+0000 7f7f0ad9d640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ef806f430 0x7f7ef8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.414+0000 7f7f0ad9d640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ef806f430 0x7f7ef8071940 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0ae000 tx=0x7f7efc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:15.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.418+0000 7f7ef37fe640 1 -- 172.21.15.80:0/439390546 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f0005c040 con 0x7f7f0c15ea70 2024-04-25T13:47:15.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.622+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7f0c143cd0 con 0x7f7f0c15ea70 2024-04-25T13:47:15.623 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7ef37fe640 1 -- 172.21.15.80:0/439390546 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7f7f0005b940 con 0x7f7f0c15ea70 2024-04-25T13:47:15.623 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:47:15.623 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:47:15.624 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:47:15.624 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:47:15.624 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:47:15.624 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:47:15.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ef806f430 msgr2=0x7f7ef8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ef806f430 0x7f7ef8071940 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f7efc0ae000 tx=0x7f7efc0a7000 comp rx=0 tx=0).stop 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f0c15ea70 msgr2=0x7f7f0c15a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f0c15ea70 0x7f7f0c15a530 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f7f0000e9b0 tx=0x7f7f0000ee70 comp rx=0 tx=0).stop 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 shutdown_connections 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ef806f430 0x7f7ef8071940 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f0c161f90 0x7f7f0c15aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f0c15ea70 0x7f7f0c15a530 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 --2- 172.21.15.80:0/439390546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f0c0d0510 0x7f7f0c140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 >> 172.21.15.80:0/439390546 conn(0x7f7f0c06d080 msgr2=0x7f7f0c0b59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 shutdown_connections 2024-04-25T13:47:15.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.626+0000 7f7f1114e640 1 -- 172.21.15.80:0/439390546 wait complete. 2024-04-25T13:47:15.644 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:15.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.790+0000 7f75a1553640 1 Processor -- start 2024-04-25T13:47:15.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f75a1553640 1 -- start start 2024-04-25T13:47:15.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f75a1553640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f759c161240 0x7f759c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f75a1553640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f759c161650 0x7f759c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f75a1553640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f759c06fcf0 con 0x7f759c06f5a0 2024-04-25T13:47:15.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f75a1553640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f759c073f10 con 0x7f759c161240 2024-04-25T13:47:15.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f75a1553640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f759c0b0c60 con 0x7f759c161650 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759a7fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f759c161240 0x7f759c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759affd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f759c06f5a0 0x7f759c160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42592/0 (socket says 172.21.15.80:42592) 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759affd640 1 -- 172.21.15.80:0/1152682595 learned_addr learned my addr 172.21.15.80:0/1152682595 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759a7fc640 1 -- 172.21.15.80:0/1152682595 >> v2:172.21.15.87:3300/0 conn(0x7f759c161650 msgr2=0x7f759c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759b7fe640 1 --2- 172.21.15.80:0/1152682595 >> v2:172.21.15.87:3300/0 conn(0x7f759c161650 0x7f759c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f7599ffb640 1 -- 172.21.15.80:0/1152682595 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3553454383 0 0) 0x7f759c06fcf0 con 0x7f759c06f5a0 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759a7fc640 1 --2- 172.21.15.80:0/1152682595 >> v2:172.21.15.87:3300/0 conn(0x7f759c161650 0x7f759c0728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759a7fc640 1 -- 172.21.15.80:0/1152682595 >> v1:172.21.15.80:6789/0 conn(0x7f759c06f5a0 legacy=0x7f759c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759a7fc640 1 -- 172.21.15.80:0/1152682595 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c0b0f40 con 0x7f759c161240 2024-04-25T13:47:15.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.794+0000 7f759b7fe640 1 --2- 172.21.15.80:0/1152682595 >> v2:172.21.15.87:3300/0 conn(0x7f759c161650 0x7f759c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:15.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f759a7fc640 1 --2- 172.21.15.80:0/1152682595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161240 0x7f759c0723a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f759c070270 tx=0x7f758401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fced5e4e93e5510 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:15.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f7599ffb640 1 -- 172.21.15.80:0/1152682595 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840b0070 con 0x7f759c161240 2024-04-25T13:47:15.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f7599ffb640 1 -- 172.21.15.80:0/1152682595 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75840ab600 con 0x7f759c161240 2024-04-25T13:47:15.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f7599ffb640 1 -- 172.21.15.80:0/1152682595 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840ab9d0 con 0x7f759c161240 2024-04-25T13:47:15.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 -- 172.21.15.80:0/1152682595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161240 msgr2=0x7f759c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 --2- 172.21.15.80:0/1152682595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161240 0x7f759c0723a0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f759c070270 tx=0x7f758401c3f0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 -- 172.21.15.80:0/1152682595 shutdown_connections 2024-04-25T13:47:15.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 --2- 172.21.15.80:0/1152682595 >> v2:172.21.15.87:3300/0 conn(0x7f759c161650 0x7f759c0728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 --2- 172.21.15.80:0/1152682595 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161240 0x7f759c0723a0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 -- 172.21.15.80:0/1152682595 >> 172.21.15.80:0/1152682595 conn(0x7f759c06d080 msgr2=0x7f759c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:15.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 -- 172.21.15.80:0/1152682595 shutdown_connections 2024-04-25T13:47:15.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 -- 172.21.15.80:0/1152682595 wait complete. 2024-04-25T13:47:15.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 Processor -- start 2024-04-25T13:47:15.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.798+0000 7f75a1553640 1 -- start start 2024-04-25T13:47:15.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f759c06f5a0 0x7f759c159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f759c161240 0x7f759c13f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161650 0x7f759c153b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f759c06e820 con 0x7f759c161650 2024-04-25T13:47:15.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f759c06e6a0 con 0x7f759c06f5a0 2024-04-25T13:47:15.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f759c06e9a0 con 0x7f759c161240 2024-04-25T13:47:15.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161650 0x7f759c153b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161650 0x7f759c153b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53004/0 (socket says 172.21.15.80:53004) 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 -- 172.21.15.80:0/59117598 learned_addr learned my addr 172.21.15.80:0/59117598 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 -- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f759c161240 msgr2=0x7f759c13f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759a7fc640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f759c161240 0x7f759c13f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759affd640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f759c06f5a0 0x7f759c159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f759c161240 0x7f759c13f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 -- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f759c06f5a0 msgr2=0x7f759c159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f759c06f5a0 0x7f759c159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 -- 172.21.15.80:0/59117598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759c159350 con 0x7f759c161650 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f759b7fe640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161650 0x7f759c153b80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f758c00cd10 tx=0x7f758c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:15.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75937fe640 1 -- 172.21.15.80:0/59117598 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c003ba0 con 0x7f759c161650 2024-04-25T13:47:15.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75937fe640 1 -- 172.21.15.80:0/59117598 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f758c003d60 con 0x7f759c161650 2024-04-25T13:47:15.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759c15e860 con 0x7f759c161650 2024-04-25T13:47:15.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75937fe640 1 -- 172.21.15.80:0/59117598 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758c004260 con 0x7f759c161650 2024-04-25T13:47:15.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759c159610 con 0x7f759c161650 2024-04-25T13:47:15.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f759c15c670 con 0x7f759c161650 2024-04-25T13:47:15.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75937fe640 1 -- 172.21.15.80:0/59117598 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f758c004400 con 0x7f759c161650 2024-04-25T13:47:15.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.802+0000 7f75937fe640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f757c06f430 0x7f757c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:15.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.806+0000 7f75937fe640 1 -- 172.21.15.80:0/59117598 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f758c08ff70 con 0x7f759c161650 2024-04-25T13:47:15.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.806+0000 7f759affd640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f757c06f430 0x7f757c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:15.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.806+0000 7f759affd640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f757c06f430 0x7f757c071940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7594000f70 tx=0x7f7594009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:15.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.806+0000 7f75937fe640 1 -- 172.21.15.80:0/59117598 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f758c014030 con 0x7f759c161650 2024-04-25T13:47:15.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.986+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f759c15ea30 con 0x7f759c161650 2024-04-25T13:47:15.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75937fe640 1 -- 172.21.15.80:0/59117598 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7f758c05cfd0 con 0x7f759c161650 2024-04-25T13:47:15.987 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:15.988 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:15.988 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:15.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f757c06f430 msgr2=0x7f757c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f757c06f430 0x7f757c071940 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f7594000f70 tx=0x7f7594009040 comp rx=0 tx=0).stop 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161650 msgr2=0x7f759c153b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161650 0x7f759c153b80 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f758c00cd10 tx=0x7f758c00a650 comp rx=0 tx=0).stop 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 shutdown_connections 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f757c06f430 0x7f757c071940 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f759c161650 0x7f759c153b80 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f759c161240 0x7f759c13f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 --2- 172.21.15.80:0/59117598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f759c06f5a0 0x7f759c159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 >> 172.21.15.80:0/59117598 conn(0x7f759c06d080 msgr2=0x7f759c0b1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 shutdown_connections 2024-04-25T13:47:15.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:15.990+0000 7f75a1553640 1 -- 172.21.15.80:0/59117598 wait complete. 2024-04-25T13:47:16.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 67, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:47:10.054954+0000', 'modified': '2024-04-25T13:47:13.107291+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7968}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7968': {'gid': 7968, 'name': 'a', 'rank': 0, 'incarnation': 64, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6835/1563317639', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6833', 'nonce': 1563317639}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 1563317639}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7968, 'qdb_cluster': [7968]} 2024-04-25T13:47:16.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:47:16.411 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T13:47:16.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f8c66640 1 Processor -- start 2024-04-25T13:47:16.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f8c66640 1 -- start start 2024-04-25T13:47:16.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f8c66640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f40ce730 0x7f57f40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f8c66640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f4073710 con 0x7f57f40b0ed0 2024-04-25T13:47:16.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f8c66640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f40739f0 con 0x7f57f406e490 2024-04-25T13:47:16.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f8c66640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57f4073cd0 con 0x7f57f40ce730 2024-04-25T13:47:16.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f259c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f57f40b0ed0 0x7f57f415c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42606/0 (socket says 172.21.15.80:42606) 2024-04-25T13:47:16.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f259c640 1 -- 172.21.15.80:0/1276647256 learned_addr learned my addr 172.21.15.80:0/1276647256 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:16.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1499090137 0 0) 0x7f57f4073710 con 0x7f57f40b0ed0 2024-04-25T13:47:16.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57ebfff640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f40ce730 0x7f57f40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.566+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57e0003450 con 0x7f57f40b0ed0 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1438620231 0 0) 0x7f57e0003450 con 0x7f57f40b0ed0 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f40ce730 msgr2=0x7f57f40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f40ce730 0x7f57f40ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 >> v1:172.21.15.87:6789/0 conn(0x7f57f406e490 legacy=0x7f57f406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f4073fb0 con 0x7f57f40b0ed0 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f57e4001680 con 0x7f57f40b0ed0 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 >> v1:172.21.15.80:6789/0 conn(0x7f57f40b0ed0 legacy=0x7f57f415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f40ce730 0x7f57e0003ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57e000a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e000b0f0 0x7f57e000d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57f4073cd0 con 0x7f57e0004110 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57e000dea0 con 0x7f57e000b0f0 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57e000e0a0 con 0x7f57f40ce730 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57ebfff640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57e000a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f259c640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f40ce730 0x7f57e0003ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f2d9d640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e000b0f0 0x7f57e000d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57ebfff640 1 -- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f40ce730 msgr2=0x7f57e0003ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57ebfff640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f40ce730 0x7f57e0003ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57ebfff640 1 -- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e000b0f0 msgr2=0x7f57e000d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57ebfff640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57e000b0f0 0x7f57e000d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57ebfff640 1 -- 172.21.15.80:0/1276647256 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57e0003450 con 0x7f57e0004110 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57ebfff640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57e000a8f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00ba30 tx=0x7f57dc01c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71ffa782c6c874e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00b600 con 0x7f57e0004110 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc0aa630 con 0x7f57e0004110 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.570+0000 7f57f1d9b640 1 -- 172.21.15.80:0/1276647256 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0aaa20 con 0x7f57e0004110 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- 172.21.15.80:0/1276647256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 msgr2=0x7f57e000a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 --2- 172.21.15.80:0/1276647256 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57e000a8f0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00ba30 tx=0x7f57dc01c6c0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f2d9d640 1 -- 172.21.15.80:0/1276647256 reap_dead start 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- 172.21.15.80:0/1276647256 shutdown_connections 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- 172.21.15.80:0/1276647256 >> 172.21.15.80:0/1276647256 conn(0x7f57f406d080 msgr2=0x7f57f40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- 172.21.15.80:0/1276647256 shutdown_connections 2024-04-25T13:47:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- 172.21.15.80:0/1276647256 wait complete. 2024-04-25T13:47:16.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 Processor -- start 2024-04-25T13:47:16.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- start start 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57f41fbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f4157610 0x7f57f40abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f415b070 0x7f57f4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40ceb40 con 0x7f57e0004110 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40733a0 con 0x7f57f4157610 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f8c66640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57f40cecc0 con 0x7f57f415b070 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57f41fbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57f41fbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53016/0 (socket says 172.21.15.80:53016) 2024-04-25T13:47:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f259c640 1 -- 172.21.15.80:0/4101796443 learned_addr learned my addr 172.21.15.80:0/4101796443 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f259c640 1 -- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f415b070 msgr2=0x7f57f4156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f2d9d640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f415b070 0x7f57f4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f259c640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f415b070 0x7f57f4156bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f259c640 1 -- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f4157610 msgr2=0x7f57f40abe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57ebfff640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f4157610 0x7f57f40abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f259c640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f4157610 0x7f57f40abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.574+0000 7f57f259c640 1 -- 172.21.15.80:0/4101796443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f4159fd0 con 0x7f57e0004110 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57f259c640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57f41fbcf0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e7e0 tx=0x7f57e400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57eaffd640 1 -- 172.21.15.80:0/4101796443 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e400c8d0 con 0x7f57e0004110 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57eaffd640 1 -- 172.21.15.80:0/4101796443 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e4018070 con 0x7f57e0004110 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57f8c66640 1 -- 172.21.15.80:0/4101796443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f415a2b0 con 0x7f57e0004110 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57eaffd640 1 -- 172.21.15.80:0/4101796443 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e400ccd0 con 0x7f57e0004110 2024-04-25T13:47:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57f8c66640 1 -- 172.21.15.80:0/4101796443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f415a870 con 0x7f57e0004110 2024-04-25T13:47:16.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57eaffd640 1 -- 172.21.15.80:0/4101796443 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57e401d460 con 0x7f57e0004110 2024-04-25T13:47:16.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57eaffd640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d406f430 0x7f57d4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57eaffd640 1 -- 172.21.15.80:0/4101796443 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f57e408f1d0 con 0x7f57e0004110 2024-04-25T13:47:16.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57ebfff640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d406f430 0x7f57d4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57fffc3000 1 Processor -- start 2024-04-25T13:47:16.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57fffc3000 1 -- start start 2024-04-25T13:47:16.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.578+0000 7f57ebfff640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d406f430 0x7f57d4071940 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0af000 tx=0x7f57dc0ac040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c452c20 0x55e39c5b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c43eea0 0x55e39c5b1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 0x55e39c5b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e39c420160 con 0x55e39c43eea0 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e39c3b5e20 con 0x55e39c452c20 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55e39c36ea20 con 0x55e39c037280 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c43eea0 0x55e39c5b1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c43eea0 0x55e39c5b1420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53020/0 (socket says 172.21.15.80:53020) 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 -- 172.21.15.80:0/1891095557 learned_addr learned my addr 172.21.15.80:0/1891095557 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57f259c640 1 --2- 172.21.15.80:0/1891095557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c452c20 0x55e39c5b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 -- 172.21.15.80:0/1891095557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 msgr2=0x55e39c5b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 --2- 172.21.15.80:0/1891095557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 0x55e39c5b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 -- 172.21.15.80:0/1891095557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c452c20 msgr2=0x55e39c5b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 --2- 172.21.15.80:0/1891095557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c452c20 0x55e39c5b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 -- 172.21.15.80:0/1891095557 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39c373100 con 0x55e39c43eea0 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57ebfff640 1 --2- 172.21.15.80:0/1891095557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c43eea0 0x55e39c5b1420 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f57dc01c4f0 tx=0x7f57dc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be96b938386f803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57d3fff640 1 -- 172.21.15.80:0/1891095557 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0b8050 con 0x55e39c43eea0 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57d3fff640 1 -- 172.21.15.80:0/1891095557 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc0c2070 con 0x55e39c43eea0 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57d3fff640 1 -- 172.21.15.80:0/1891095557 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57dc0bd800 con 0x55e39c43eea0 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- 172.21.15.80:0/1891095557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c43eea0 msgr2=0x55e39c5b1420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 --2- 172.21.15.80:0/1891095557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c43eea0 0x55e39c5b1420 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f57dc01c4f0 tx=0x7f57dc004740 comp rx=0 tx=0).stop 2024-04-25T13:47:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- 172.21.15.80:0/1891095557 shutdown_connections 2024-04-25T13:47:16.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 --2- 172.21.15.80:0/1891095557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 0x55e39c5b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 --2- 172.21.15.80:0/1891095557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c43eea0 0x55e39c5b1420 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 --2- 172.21.15.80:0/1891095557 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c452c20 0x55e39c5b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- 172.21.15.80:0/1891095557 >> 172.21.15.80:0/1891095557 conn(0x55e39c076ff0 msgr2=0x55e39c5664c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- 172.21.15.80:0/1891095557 shutdown_connections 2024-04-25T13:47:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- 172.21.15.80:0/1891095557 wait complete. 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 -- start start 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.582+0000 7f57fffc3000 1 Processor -- start 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 0x55e39c5b7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 0x55e39c5b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5730c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e39c3c3220 con 0x55e39c452c20 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e39c359ea0 con 0x55e39c43eea0 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55e39c3b0b00 con 0x55e39c037280 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 10 client.0 _ll_register_callbacks cb 0x55e39c3c2dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:16.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5730c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5730c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53032/0 (socket says 172.21.15.80:53032) 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 -- 172.21.15.80:0/553288469 learned_addr learned my addr 172.21.15.80:0/553288469 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f259c640 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 0x55e39c5b7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57ebfff640 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 0x55e39c5b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 -- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 msgr2=0x55e39c5b7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 0x55e39c5b7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 -- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 msgr2=0x55e39c5b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 0x55e39c5b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 -- 172.21.15.80:0/553288469 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39c355500 con 0x55e39c452c20 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57ebfff640 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 0x55e39c5b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f2d9d640 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5730c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f57f40b4970 tx=0x7f57ec00f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39a43d17b4c1468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 1 -- 172.21.15.80:0/553288469 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec003980 con 0x55e39c452c20 2024-04-25T13:47:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 1 -- 172.21.15.80:0/553288469 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f57ec003b20 con 0x55e39c452c20 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- 172.21.15.80:0/553288469 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57dc0c2070 con 0x55e39c452c20 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 1 -- 172.21.15.80:0/553288469 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f57ec020050 con 0x55e39c452c20 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 10 client.8097 fetch_fsmap learned FSMap version 67 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- 172.21.15.80:0/553288469 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55e39c33bea0 con 0x55e39c452c20 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 1 -- 172.21.15.80:0/553288469 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57dc0c2070 con 0x55e39c452c20 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 1 -- 172.21.15.80:0/553288469 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f57ec023020 con 0x55e39c452c20 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 10 client.8097 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 10 client.8097 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 20 client.8097 populate_metadata read hostname 'smithi080' 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] conn(0x55e39c05b7d0 0x55e39c573e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 4 client.8097 mds_command: new command op to 7968 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- 172.21.15.80:0/553288469 --> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e39c34ebe0 con 0x55e39c05b7d0 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f259c640 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] conn(0x55e39c05b7d0 0x55e39c573e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 10 client.8097 ms_handle_connect on v2:172.21.15.80:6833/1563317639 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57f259c640 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] conn(0x55e39c05b7d0 0x55e39c573e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e115a37e22ee7570 server_cookie=a8de1e2d79a8bc0f in_seq=0 out_seq=0 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 1 -- 172.21.15.80:0/553288469 <== mds.0 v2:172.21.15.80:6833/1563317639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e39c34ebe0 con 0x55e39c05b7d0 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 10 client.8097 handle_command_reply: tid=0 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 client.8097 shutdown 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- 172.21.15.80:0/553288469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 msgr2=0x55e39c5730c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5730c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f57f40b4970 tx=0x7f57ec00f660 comp rx=0 tx=0).stop 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- 172.21.15.80:0/553288469 shutdown_connections 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] conn(0x55e39c05b7d0 0x55e39c573e00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5730c0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 0x55e39c5b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57d17fa640 0 client.8097 ms_handle_reset on v2:172.21.15.80:6833/1563317639 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 --2- 172.21.15.80:0/553288469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c037280 0x55e39c5b7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- 172.21.15.80:0/553288469 >> 172.21.15.80:0/553288469 conn(0x55e39c076ff0 msgr2=0x55e39c566c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- 172.21.15.80:0/553288469 shutdown_connections 2024-04-25T13:47:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 1 -- 172.21.15.80:0/553288469 wait complete. 2024-04-25T13:47:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.586+0000 7f57fffc3000 20 client.8097 trim_cache size 0 max 16384 2024-04-25T13:47:16.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57fffc3000 1 Processor -- start 2024-04-25T13:47:16.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57fffc3000 1 -- start start 2024-04-25T13:47:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c037280 0x55e39c5bd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 0x55e39c5c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c452c20 0x55e39c5c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57fffc3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57f4206c30 con 0x55e39c037280 2024-04-25T13:47:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57fffc3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57f41fec60 con 0x55e39c43eea0 2024-04-25T13:47:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57fffc3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57f41ff2b0 con 0x55e39c452c20 2024-04-25T13:47:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c037280 0x55e39c5bd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c037280 0x55e39c5bd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53036/0 (socket says 172.21.15.80:53036) 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 -- 172.21.15.80:0/551053020 learned_addr learned my addr 172.21.15.80:0/551053020 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 -- 172.21.15.80:0/551053020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c452c20 msgr2=0x55e39c5c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 --2- 172.21.15.80:0/551053020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c452c20 0x55e39c5c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 -- 172.21.15.80:0/551053020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 msgr2=0x55e39c5c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 --2- 172.21.15.80:0/551053020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 0x55e39c5c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 -- 172.21.15.80:0/551053020 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39c573430 con 0x55e39c037280 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57f259c640 1 --2- 172.21.15.80:0/551053020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c037280 0x55e39c5bd0b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f57e400c4b0 tx=0x7f57e408d5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd30c5d72559460c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.606+0000 7f57d37fe640 1 -- 172.21.15.80:0/551053020 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e405b130 con 0x55e39c037280 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57d37fe640 1 -- 172.21.15.80:0/551053020 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e4064160 con 0x55e39c037280 2024-04-25T13:47:16.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57d37fe640 1 -- 172.21.15.80:0/551053020 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e405b4d0 con 0x55e39c037280 2024-04-25T13:47:16.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- 172.21.15.80:0/551053020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c037280 msgr2=0x55e39c5bd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 --2- 172.21.15.80:0/551053020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c037280 0x55e39c5bd0b0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f57e400c4b0 tx=0x7f57e408d5d0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- 172.21.15.80:0/551053020 shutdown_connections 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 --2- 172.21.15.80:0/551053020 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c452c20 0x55e39c5c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 --2- 172.21.15.80:0/551053020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c43eea0 0x55e39c5c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 --2- 172.21.15.80:0/551053020 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c037280 0x55e39c5bd0b0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- 172.21.15.80:0/551053020 >> 172.21.15.80:0/551053020 conn(0x55e39c076ff0 msgr2=0x55e39c56e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- 172.21.15.80:0/551053020 shutdown_connections 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- 172.21.15.80:0/551053020 wait complete. 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- start start 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 Processor -- start 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c037280 0x55e39c623590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c43eea0 0x55e39c623ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5bb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e39c3b5130 con 0x55e39c452c20 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e39c34b7d0 con 0x55e39c037280 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55e39c3ba450 con 0x55e39c43eea0 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57fffc3000 10 client.0 _ll_register_callbacks cb 0x7f57ec001620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5bb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5bb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53050/0 (socket says 172.21.15.80:53050) 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 -- 172.21.15.80:0/2078851731 learned_addr learned my addr 172.21.15.80:0/2078851731 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57ebfff640 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c43eea0 0x55e39c623ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f259c640 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c037280 0x55e39c623590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 -- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c43eea0 msgr2=0x55e39c623ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c43eea0 0x55e39c623ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 -- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c037280 msgr2=0x55e39c623590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c037280 0x55e39c623590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 -- 172.21.15.80:0/2078851731 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39c36ec70 con 0x55e39c452c20 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f259c640 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c037280 0x55e39c623590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57f2d9d640 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5bb890 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00f6a0 tx=0x7f57ec001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfbc043be2b2fb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.610+0000 7f57d17fa640 1 -- 172.21.15.80:0/2078851731 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec031060 con 0x55e39c452c20 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 -- 172.21.15.80:0/2078851731 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e39c3c3a30 con 0x55e39c452c20 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57d17fa640 1 -- 172.21.15.80:0/2078851731 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f57ec02c630 con 0x55e39c452c20 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57d17fa640 1 -- 172.21.15.80:0/2078851731 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f57ec037050 con 0x55e39c452c20 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 10 client.8103 fetch_fsmap learned FSMap version 67 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 -- 172.21.15.80:0/2078851731 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=2+}) -- 0x55e39c438180 con 0x55e39c452c20 2024-04-25T13:47:16.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57d17fa640 1 -- 172.21.15.80:0/2078851731 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55e39c3c3a30 con 0x55e39c452c20 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57d17fa640 1 -- 172.21.15.80:0/2078851731 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 67) ==== 3494+0+0 (secure 0 0 0) 0x7f57ec03a020 con 0x55e39c452c20 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 10 client.8103 fetch_fsmap finished waiting for FSMap version 67 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 10 client.8103 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 20 client.8103 populate_metadata read hostname 'smithi080' 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] conn(0x55e39c049830 0x55e39c5bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 4 client.8103 mds_command: new command op to 7968 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 -- 172.21.15.80:0/2078851731 --> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] -- command(tid 0: {"prefix": "status"}) -- 0x55e39c3bd8a0 con 0x55e39c049830 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f259c640 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] conn(0x55e39c049830 0x55e39c5bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57d17fa640 10 client.8103 ms_handle_connect on v2:172.21.15.80:6833/1563317639 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f259c640 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] conn(0x55e39c049830 0x55e39c5bc430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b223c0a4c9ece29d server_cookie=78122d4b82707dbb in_seq=0 out_seq=0 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57d17fa640 1 -- 172.21.15.80:0/2078851731 <== mds.0 v2:172.21.15.80:6833/1563317639 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e39c3bd8a0 con 0x55e39c049830 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57d17fa640 10 client.8103 handle_command_reply: tid=0 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 client.8103 shutdown 2024-04-25T13:47:16.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 -- 172.21.15.80:0/2078851731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 msgr2=0x55e39c5bb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5bb890 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00f6a0 tx=0x7f57ec001520 comp rx=0 tx=0).stop 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 -- 172.21.15.80:0/2078851731 shutdown_connections 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.80:6833/1563317639,v1:172.21.15.80:6835/1563317639] conn(0x55e39c049830 0x55e39c5bc430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e39c452c20 0x55e39c5bb890 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e39c43eea0 0x55e39c623ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 --2- 172.21.15.80:0/2078851731 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e39c037280 0x55e39c623590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57d17fa640 0 client.8103 ms_handle_reset on v2:172.21.15.80:6833/1563317639 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 -- 172.21.15.80:0/2078851731 >> 172.21.15.80:0/2078851731 conn(0x55e39c076ff0 msgr2=0x55e39c5b6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 -- 172.21.15.80:0/2078851731 shutdown_connections 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 1 -- 172.21.15.80:0/2078851731 wait complete. 2024-04-25T13:47:16.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57fffc3000 20 client.8103 trim_cache size 0 max 16384 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 7968, 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 5.8952301890000003, 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 67, 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 136, 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 136, 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 10.878036495 2024-04-25T13:47:16.619 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 -- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d406f430 msgr2=0x7f57d4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d406f430 0x7f57d4071940 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0af000 tx=0x7f57dc0ac040 comp rx=0 tx=0).stop 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 -- 172.21.15.80:0/4101796443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 msgr2=0x7f57f41fbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57f41fbcf0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f57e400e7e0 tx=0x7f57e400eca0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 -- 172.21.15.80:0/4101796443 shutdown_connections 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57d406f430 0x7f57d4071940 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57f415b070 0x7f57f4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57f4157610 0x7f57f40abe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 --2- 172.21.15.80:0/4101796443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57e0004110 0x7f57f41fbcf0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 -- 172.21.15.80:0/4101796443 >> 172.21.15.80:0/4101796443 conn(0x7f57f406d080 msgr2=0x7f57f415e870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 -- 172.21.15.80:0/4101796443 shutdown_connections 2024-04-25T13:47:16.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.614+0000 7f57f8c66640 1 -- 172.21.15.80:0/4101796443 wait complete. 2024-04-25T13:47:16.632 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:47:16.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7806ca4640 1 Processor -- start 2024-04-25T13:47:16.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7806ca4640 1 -- start start 2024-04-25T13:47:16.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7806ca4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 0x7f7800068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7806ca4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7800068ad0 0x7f7800161bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7806ca4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78001620f0 0x7f7800162500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7806ca4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7800162a40 con 0x7f78001620f0 2024-04-25T13:47:16.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7806ca4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f78000d2000 con 0x7f7800068ad0 2024-04-25T13:47:16.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7806ca4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f78000d2490 con 0x7f780006d000 2024-04-25T13:47:16.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7805241640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78001620f0 0x7f7800162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7805241640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f78001620f0 0x7f7800162500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53054/0 (socket says 172.21.15.80:53054) 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7805241640 1 -- 172.21.15.80:0/2623859603 learned_addr learned my addr 172.21.15.80:0/2623859603 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f77fffff640 1 --2- 172.21.15.80:0/2623859603 >> v2:172.21.15.87:3300/0 conn(0x7f7800068ad0 0x7f7800161bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7804a40640 1 --2- 172.21.15.80:0/2623859603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 0x7f7800068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7805241640 1 -- 172.21.15.80:0/2623859603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 msgr2=0x7f7800068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7805241640 1 --2- 172.21.15.80:0/2623859603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 0x7f7800068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7805241640 1 -- 172.21.15.80:0/2623859603 >> v2:172.21.15.87:3300/0 conn(0x7f7800068ad0 msgr2=0x7f7800161bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7805241640 1 --2- 172.21.15.80:0/2623859603 >> v2:172.21.15.87:3300/0 conn(0x7f7800068ad0 0x7f7800161bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.766+0000 7f7805241640 1 -- 172.21.15.80:0/2623859603 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7800068ee0 con 0x7f78001620f0 2024-04-25T13:47:16.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7805241640 1 --2- 172.21.15.80:0/2623859603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f7800162500 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f78000d2640 tx=0x7f77f801c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd2d762c32dc21d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f77ff7fe640 1 -- 172.21.15.80:0/2623859603 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80a5840 con 0x7f78001620f0 2024-04-25T13:47:16.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f77ff7fe640 1 -- 172.21.15.80:0/2623859603 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f80a59e0 con 0x7f78001620f0 2024-04-25T13:47:16.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f77ff7fe640 1 -- 172.21.15.80:0/2623859603 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80a5db0 con 0x7f78001620f0 2024-04-25T13:47:16.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 -- 172.21.15.80:0/2623859603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 msgr2=0x7f7800162500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 --2- 172.21.15.80:0/2623859603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f7800162500 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f78000d2640 tx=0x7f77f801c9c0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 -- 172.21.15.80:0/2623859603 shutdown_connections 2024-04-25T13:47:16.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 --2- 172.21.15.80:0/2623859603 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f7800162500 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 --2- 172.21.15.80:0/2623859603 >> v2:172.21.15.87:3300/0 conn(0x7f7800068ad0 0x7f7800161bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 --2- 172.21.15.80:0/2623859603 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 0x7f7800068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 -- 172.21.15.80:0/2623859603 >> 172.21.15.80:0/2623859603 conn(0x7f78000d28f0 msgr2=0x7f78000d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 -- 172.21.15.80:0/2623859603 shutdown_connections 2024-04-25T13:47:16.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.770+0000 7f7806ca4640 1 -- 172.21.15.80:0/2623859603 wait complete. 2024-04-25T13:47:16.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 Processor -- start 2024-04-25T13:47:16.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 -- start start 2024-04-25T13:47:16.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7800068ad0 0x7f780014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 0x7f7800159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f78000b25c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f78000d10a0 con 0x7f78001620f0 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f78000d1c90 con 0x7f7800068ad0 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f780006d410 con 0x7f780006d000 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f78000b25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f78000b25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53066/0 (socket says 172.21.15.80:53066) 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 -- 172.21.15.80:0/3112561443 learned_addr learned my addr 172.21.15.80:0/3112561443 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 -- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 msgr2=0x7f7800159ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 0x7f7800159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 -- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7800068ad0 msgr2=0x7f780014f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7800068ad0 0x7f780014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 -- 172.21.15.80:0/3112561443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f780014f960 con 0x7f78001620f0 2024-04-25T13:47:16.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7805241640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f78000b25c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f77f80af040 tx=0x7f77f8019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f77fd7fa640 1 -- 172.21.15.80:0/3112561443 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80b8880 con 0x7f78001620f0 2024-04-25T13:47:16.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f77fd7fa640 1 -- 172.21.15.80:0/3112561443 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f80be070 con 0x7f78001620f0 2024-04-25T13:47:16.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f780015bb40 con 0x7f78001620f0 2024-04-25T13:47:16.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f77fd7fa640 1 -- 172.21.15.80:0/3112561443 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f80b8c60 con 0x7f78001620f0 2024-04-25T13:47:16.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f780015c100 con 0x7f78001620f0 2024-04-25T13:47:16.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.774+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f780006a3a0 con 0x7f78001620f0 2024-04-25T13:47:16.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.778+0000 7f77fd7fa640 1 -- 172.21.15.80:0/3112561443 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f77f80c3460 con 0x7f78001620f0 2024-04-25T13:47:16.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.778+0000 7f77fd7fa640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77ec06f400 0x7f77ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:16.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.778+0000 7f77fd7fa640 1 -- 172.21.15.80:0/3112561443 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5709+0+0 (secure 0 0 0) 0x7f77f812b0e0 con 0x7f78001620f0 2024-04-25T13:47:16.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.778+0000 7f7804a40640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77ec06f400 0x7f77ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:16.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.778+0000 7f7804a40640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77ec06f400 0x7f77ec071910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f78000b55f0 tx=0x7f77f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:16.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.782+0000 7f77fd7fa640 1 -- 172.21.15.80:0/3112561443 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7800068ee0 con 0x7f78001620f0 2024-04-25T13:47:16.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.978+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f780015e590 con 0x7f78001620f0 2024-04-25T13:47:16.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.978+0000 7f77fd7fa640 1 -- 172.21.15.80:0/3112561443 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f77f80165e0 con 0x7f78001620f0 2024-04-25T13:47:16.977 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:47:16.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77ec06f400 msgr2=0x7f77ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77ec06f400 0x7f77ec071910 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f78000b55f0 tx=0x7f77f0008040 comp rx=0 tx=0).stop 2024-04-25T13:47:16.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 msgr2=0x7f78000b25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f78000b25c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f77f80af040 tx=0x7f77f8019760 comp rx=0 tx=0).stop 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 shutdown_connections 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f77ec06f400 0x7f77ec071910 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78001620f0 0x7f78000b25c0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f780006d000 0x7f7800159ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 --2- 172.21.15.80:0/3112561443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7800068ad0 0x7f780014f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 >> 172.21.15.80:0/3112561443 conn(0x7f78000d28f0 msgr2=0x7f78000b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 shutdown_connections 2024-04-25T13:47:16.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:16.982+0000 7f7806ca4640 1 -- 172.21.15.80:0/3112561443 wait complete. 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:47:16.999 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:47:16.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:17.000 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:17.006 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42131sec preferred_lft 42131sec 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:47:17.007 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:47:17.008 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:47:17.009 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:47:17.009 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:17.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:17.009 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:47:17.055 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:47:17.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:17.056 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:47:17.106 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:47:17.106 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:47:17.107 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:47:17.107 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:47:17.108 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:47:17.108 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:47:17.108 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:47:17.108 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:47:17.108 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:47:17.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:47:17.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:47:17.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:17.228 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:47:17.275 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:47:17.276 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:47:17.276 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:47:17.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:17.276 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:47:17.332 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:47:17.333 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:47:17.333 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:47:17.379 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:47:17.380 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:47:17.380 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:47:17.427 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:47:17.484 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:47:17.533 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:47:17.534 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:47:17.534 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:47:17.534 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:47:17.535 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:47:17.535 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:47:17.535 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:47:17.535 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:47:17.535 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:47:17.536 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:47:17.537 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:47:17.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:17.538 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:47:17.580 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:47:17.580 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:47:17.625 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:47:17.639 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:47:17.687 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:47:17.687 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:47:17.687 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:47:17.687 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:47:17.687 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:47:17.688 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:47:17.689 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:47:17.690 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:47:17.690 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:47:17.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:17.693 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:47:17.734 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:47:17.734+0000 7fda22e58580 -1 init, newargv = 0x7fda1001b580 newargc=15 2024-04-25T13:47:17.734 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[38760]: starting ceph client 2024-04-25T13:47:17.738 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[38760]: starting fuse 2024-04-25T13:47:18.841 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:47:18.858 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:47:18.905 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:47:18.906 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:47:18.907 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:47:18.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:18.908 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:47:18.955 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:47:18.955 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:47:18.956 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:47:18.957 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:47:19.081 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.38760.asok 2024-04-25T13:47:19.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:47:19.093 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38760.asok 2024-04-25T13:47:19.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:19.093 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38760.asok status 2024-04-25T13:47:19.274 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:19.274 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:47:19.274 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "38760", 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 8115, 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:47:19.275 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 8115 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 518559282 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 518559282 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.8115 192.168.0.1:0/518559282", 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/518559282", 2024-04-25T13:47:19.276 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:47:19.277 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 67, 2024-04-25T13:47:19.277 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 136, 2024-04-25T13:47:19.277 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:47:19.277 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:47:19.277 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:47:19.277 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:19.291 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:19.291 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:47:19.291 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:47:19.297 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:47:19.298 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:47:19.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:19.298 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:47:19.352 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:47:19.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.554+0000 7f6ee7d35640 1 Processor -- start 2024-04-25T13:47:19.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.554+0000 7f6ee7d35640 1 -- start start 2024-04-25T13:47:19.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee7d35640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f6ee00d06e0 0x7f6ee006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee7d35640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6ee006e9e0 0x7f6ee006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee7d35640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ee00d0af0 con 0x7f6ee0161f90 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee7d35640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee00d0d70 con 0x7f6ee006e9e0 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee7d35640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f6ee00d1050 con 0x7f6ee00d06e0 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee62d2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6ee006e9e0 0x7f6ee006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee5ad1640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6ee0161f90 0x7f6ee00cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42610/0 (socket says 172.21.15.80:42610) 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee5ad1640 1 -- 172.21.15.80:0/2775258250 learned_addr learned my addr 172.21.15.80:0/2775258250 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee52d0640 1 --2- 172.21.15.80:0/2775258250 >> v2:172.21.15.87:3300/0 conn(0x7f6ee00d06e0 0x7f6ee006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee62d2640 1 -- 172.21.15.80:0/2775258250 >> v2:172.21.15.87:3300/0 conn(0x7f6ee00d06e0 msgr2=0x7f6ee006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee62d2640 1 --2- 172.21.15.80:0/2775258250 >> v2:172.21.15.87:3300/0 conn(0x7f6ee00d06e0 0x7f6ee006e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee62d2640 1 -- 172.21.15.80:0/2775258250 >> v1:172.21.15.80:6789/0 conn(0x7f6ee0161f90 legacy=0x7f6ee00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee62d2640 1 -- 172.21.15.80:0/2775258250 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee006dca0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee4acf640 1 -- 172.21.15.80:0/2775258250 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1795707897 0 0) 0x7f6edc001120 con 0x7f6ee0161f90 2024-04-25T13:47:19.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.558+0000 7f6ee52d0640 1 --2- 172.21.15.80:0/2775258250 >> v2:172.21.15.87:3300/0 conn(0x7f6ee00d06e0 0x7f6ee006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:19.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee62d2640 1 --2- 172.21.15.80:0/2775258250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee006edf0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0074290 tx=0x7f6ed801c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9557a6dd05e7918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee4acf640 1 -- 172.21.15.80:0/2775258250 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80b2070 con 0x7f6ee006e9e0 2024-04-25T13:47:19.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee4acf640 1 -- 172.21.15.80:0/2775258250 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ed80a57b0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee4acf640 1 -- 172.21.15.80:0/2775258250 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80a5b80 con 0x7f6ee006e9e0 2024-04-25T13:47:19.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 -- 172.21.15.80:0/2775258250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 msgr2=0x7f6ee006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/2775258250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee006edf0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0074290 tx=0x7f6ed801c7e0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 -- 172.21.15.80:0/2775258250 shutdown_connections 2024-04-25T13:47:19.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/2775258250 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee006edf0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/2775258250 >> v2:172.21.15.87:3300/0 conn(0x7f6ee00d06e0 0x7f6ee006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 -- 172.21.15.80:0/2775258250 >> 172.21.15.80:0/2775258250 conn(0x7f6ee006d080 msgr2=0x7f6ee00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 -- 172.21.15.80:0/2775258250 shutdown_connections 2024-04-25T13:47:19.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 -- 172.21.15.80:0/2775258250 wait complete. 2024-04-25T13:47:19.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 Processor -- start 2024-04-25T13:47:19.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.562+0000 7f6ee7d35640 1 -- start start 2024-04-25T13:47:19.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee015db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6ee00d06e0 0x7f6ee01455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee0161f90 0x7f6ee0145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee01447a0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee0144620 con 0x7f6ee0161f90 2024-04-25T13:47:19.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee016b510 con 0x7f6ee00d06e0 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee015db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee015db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53074/0 (socket says 172.21.15.80:53074) 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 -- 172.21.15.80:0/4029617395 learned_addr learned my addr 172.21.15.80:0/4029617395 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee52d0640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6ee00d06e0 0x7f6ee01455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee62d2640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee0161f90 0x7f6ee0145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 -- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6ee00d06e0 msgr2=0x7f6ee01455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6ee00d06e0 0x7f6ee01455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 -- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee0161f90 msgr2=0x7f6ee0145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee0161f90 0x7f6ee0145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 -- 172.21.15.80:0/4029617395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee0141ac0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee5ad1640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee015db10 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c410 tx=0x7f6edc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ed67fc640 1 -- 172.21.15.80:0/4029617395 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc0074e0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ed67fc640 1 -- 172.21.15.80:0/4029617395 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6edc0047f0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee01417e0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ed67fc640 1 -- 172.21.15.80:0/4029617395 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc007680 con 0x7f6ee006e9e0 2024-04-25T13:47:19.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee0144db0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.566+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee006a3a0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.570+0000 7f6ed67fc640 1 -- 172.21.15.80:0/4029617395 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6edc011070 con 0x7f6ee006e9e0 2024-04-25T13:47:19.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.570+0000 7f6ed67fc640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6ec406f400 0x7f6ec4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:19.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.570+0000 7f6ed67fc640 1 -- 172.21.15.80:0/4029617395 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5767+0+0 (secure 0 0 0) 0x7f6edc08f2c0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.570+0000 7f6ee52d0640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6ec406f400 0x7f6ec4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:19.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.570+0000 7f6ee52d0640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6ec406f400 0x7f6ec4071910 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0076430 tx=0x7f6ed0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:19.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.574+0000 7f6ed67fc640 1 -- 172.21.15.80:0/4029617395 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6ee00d0af0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.774+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6ee006e4a0 con 0x7f6ee006e9e0 2024-04-25T13:47:19.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.774+0000 7f6ed67fc640 1 -- 172.21.15.80:0/4029617395 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v67) ==== 53+0+73 (secure 0 0 0) 0x7f6edc05bc90 con 0x7f6ee006e9e0 2024-04-25T13:47:19.774 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:47:19.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6ec406f400 msgr2=0x7f6ec4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6ec406f400 0x7f6ec4071910 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0076430 tx=0x7f6ed0008040 comp rx=0 tx=0).stop 2024-04-25T13:47:19.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 msgr2=0x7f6ee015db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:19.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee015db10 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00c410 tx=0x7f6edc00c8d0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 shutdown_connections 2024-04-25T13:47:19.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6ec406f400 0x7f6ec4071910 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6ee0161f90 0x7f6ee0145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6ee00d06e0 0x7f6ee01455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 --2- 172.21.15.80:0/4029617395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6ee006e9e0 0x7f6ee015db10 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:19.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 >> 172.21.15.80:0/4029617395 conn(0x7f6ee006d080 msgr2=0x7f6ee00b2510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:19.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 shutdown_connections 2024-04-25T13:47:19.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:19.778+0000 7f6ee7d35640 1 -- 172.21.15.80:0/4029617395 wait complete. 2024-04-25T13:47:19.797 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:47:19.797 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:47:19.797 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:47:19.797 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:47:19.797 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:47:19.797 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:47:19.797 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:47:19.798 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:47:19.798 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:47:19.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:19.798 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:47:19.805 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:47:19.806 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42129sec preferred_lft 42129sec 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:47:19.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:47:19.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:19.809 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:47:19.861 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:47:19.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:19.863 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:47:19.914 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:47:19.915 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:47:19.915 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:47:19.915 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:47:19.915 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:47:19.915 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:47:19.915 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:47:20.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:47:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:47:20.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:47:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:47:20.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:20.038 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:47:20.043 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:47:20.043 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:47:20.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:47:20.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:20.043 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:47:20.096 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:47:20.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:47:20.096 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:47:20.145 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:47:20.145 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:47:20.145 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:47:20.193 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:47:20.250 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:47:20.304 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:47:20.304 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:47:20.304 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:47:20.304 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:47:20.304 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:47:20.304 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:47:20.304 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:47:20.304 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:47:20.305 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:47:20.306 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:47:20.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:20.307 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:47:20.353 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:47:20.353 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:47:20.398 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:47:20.412 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:47:20.461 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:47:20.462 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:47:20.463 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:47:20.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:20.464 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:47:20.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:47:20.507+0000 7fa8c260e580 -1 init, newargv = 0x7fa8b401b230 newargc=15 2024-04-25T13:47:20.508 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25251]: starting ceph client 2024-04-25T13:47:20.515 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25251]: starting fuse 2024-04-25T13:47:21.614 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:47:21.630 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:47:21.678 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:47:21.679 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:47:21.680 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:47:21.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:21.681 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:47:21.729 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:47:21.729 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:47:21.729 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:47:21.730 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:47:21.731 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:47:21.732 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:47:21.732 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:47:21.854 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.25251.asok 2024-04-25T13:47:21.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:47:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:47:21.866 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.25251.asok 2024-04-25T13:47:21.866 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:21.866 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.25251.asok status 2024-04-25T13:47:22.045 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "25251", 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:47:22.046 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 5960, 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 5960 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2817715503 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:47:22.047 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2817715503 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.5960 192.168.0.1:0/2817715503", 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/2817715503", 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 67, 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 138, 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:47:22.048 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:47:22.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:22.063 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:47:22.063 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:47:22.069 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:47:22.069 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:47:22.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:22.070 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:47:22.123 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.282+0000 7fc7f124b640 1 Processor -- start 2024-04-25T13:47:22.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.282+0000 7fc7f124b640 1 -- start start 2024-04-25T13:47:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7f124b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7ec0d06e0 0x7fc7ec06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7f124b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec06e9e0 0x7fc7ec06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7f124b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7ec0d0af0 con 0x7fc7ec161f90 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7f124b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc7ec0d0d70 con 0x7fc7ec0d06e0 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7f124b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ec0d1050 con 0x7fc7ec06e9e0 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ead9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc7ec161f90 0x7fc7ec0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45822/0 (socket says 172.21.15.80:45822) 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ead9d640 1 -- 172.21.15.80:0/1785949244 learned_addr learned my addr 172.21.15.80:0/1785949244 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7eb59e640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec06e9e0 0x7fc7ec06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ea59c640 1 --2- 172.21.15.80:0/1785949244 >> v2:172.21.15.87:3300/0 conn(0x7fc7ec0d06e0 0x7fc7ec06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1286985040 0 0) 0x7fc7dc001120 con 0x7fc7ec161f90 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc7d4003530 con 0x7fc7ec161f90 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1935479125 0 0) 0x7fc7d4003530 con 0x7fc7ec161f90 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec06e9e0 msgr2=0x7fc7ec06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec06e9e0 0x7fc7ec06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 >> v2:172.21.15.87:3300/0 conn(0x7fc7ec0d06e0 msgr2=0x7fc7ec06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 --2- 172.21.15.80:0/1785949244 >> v2:172.21.15.87:3300/0 conn(0x7fc7ec0d06e0 0x7fc7ec06e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.287 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec06dca0 con 0x7fc7ec161f90 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc7dc001850 con 0x7fc7ec161f90 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 >> v1:172.21.15.80:6789/0 conn(0x7fc7ec161f90 legacy=0x7fc7ec0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d40039c0 0x7fc7d4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d4004810 0x7fc7d4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec06e9e0 0x7fc7d4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec0d0d70 con 0x7fc7d40039c0 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec0d1050 con 0x7fc7d4004810 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d400ba80 con 0x7fc7ec06e9e0 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ead9d640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d40039c0 0x7fc7d4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ea59c640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d4004810 0x7fc7d4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ead9d640 1 -- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec06e9e0 msgr2=0x7fc7d4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7eb59e640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec06e9e0 0x7fc7d4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ead9d640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec06e9e0 0x7fc7d4005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ead9d640 1 -- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d4004810 msgr2=0x7fc7d4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:22.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ead9d640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d4004810 0x7fc7d4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ead9d640 1 -- 172.21.15.80:0/1785949244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d4003530 con 0x7fc7d40039c0 2024-04-25T13:47:22.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.286+0000 7fc7ea59c640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d4004810 0x7fc7d4004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7ead9d640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d40039c0 0x7fc7d4003e30 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec147e20 tx=0x7fc7dc01cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef5a367459fe5e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc00c0e0 con 0x7fc7d40039c0 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc0a9020 con 0x7fc7d40039c0 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7e9d9b640 1 -- 172.21.15.80:0/1785949244 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc005b00 con 0x7fc7d40039c0 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7f124b640 1 -- 172.21.15.80:0/1785949244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d40039c0 msgr2=0x7fc7d4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7f124b640 1 --2- 172.21.15.80:0/1785949244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d40039c0 0x7fc7d4003e30 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec147e20 tx=0x7fc7dc01cc30 comp rx=0 tx=0).stop 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7eb59e640 1 -- 172.21.15.80:0/1785949244 reap_dead start 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7f124b640 1 -- 172.21.15.80:0/1785949244 shutdown_connections 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7f124b640 1 -- 172.21.15.80:0/1785949244 >> 172.21.15.80:0/1785949244 conn(0x7fc7ec06d080 msgr2=0x7fc7ec0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7f124b640 1 -- 172.21.15.80:0/1785949244 shutdown_connections 2024-04-25T13:47:22.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7f124b640 1 -- 172.21.15.80:0/1785949244 wait complete. 2024-04-25T13:47:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.290+0000 7fc7f124b640 1 Processor -- start 2024-04-25T13:47:22.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 -- start start 2024-04-25T13:47:22.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d40039c0 0x7fc7ec1412d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec15ff00 0x7fc7ec0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7ec160310 0x7fc7ec0afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec1447a0 con 0x7fc7ec160310 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec144620 con 0x7fc7d40039c0 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7ec168cf0 con 0x7fc7ec15ff00 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7ec160310 0x7fc7ec0afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7ec160310 0x7fc7ec0afe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37138/0 (socket says 172.21.15.80:37138) 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 -- 172.21.15.80:0/388674554 learned_addr learned my addr 172.21.15.80:0/388674554 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7ead9d640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d40039c0 0x7fc7ec1412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7ea59c640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec15ff00 0x7fc7ec0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 -- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec15ff00 msgr2=0x7fc7ec0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec15ff00 0x7fc7ec0af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 -- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d40039c0 msgr2=0x7fc7ec1412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d40039c0 0x7fc7ec1412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 -- 172.21.15.80:0/388674554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec160b10 con 0x7fc7ec160310 2024-04-25T13:47:22.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7eb59e640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7ec160310 0x7fc7ec0afe20 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0b4ba0 tx=0x7fc7e400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:22.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7d37fe640 1 -- 172.21.15.80:0/388674554 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e401c070 con 0x7fc7ec160310 2024-04-25T13:47:22.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7d37fe640 1 -- 172.21.15.80:0/388674554 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7e4017400 con 0x7fc7ec160310 2024-04-25T13:47:22.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec160df0 con 0x7fc7ec160310 2024-04-25T13:47:22.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7d37fe640 1 -- 172.21.15.80:0/388674554 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40177f0 con 0x7fc7ec160310 2024-04-25T13:47:22.295 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec142200 con 0x7fc7ec160310 2024-04-25T13:47:22.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.294+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec142460 con 0x7fc7ec160310 2024-04-25T13:47:22.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.298+0000 7fc7d37fe640 1 -- 172.21.15.80:0/388674554 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc7e401f070 con 0x7fc7ec160310 2024-04-25T13:47:22.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.298+0000 7fc7d37fe640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7d806f400 0x7fc7d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.298+0000 7fc7d37fe640 1 -- 172.21.15.80:0/388674554 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5767+0+0 (secure 0 0 0) 0x7fc7e408f900 con 0x7fc7ec160310 2024-04-25T13:47:22.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.298+0000 7fc7ead9d640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7d806f400 0x7fc7d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.298+0000 7fc7ead9d640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7d806f400 0x7fc7d8071910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0aa000 tx=0x7fc7dc001c20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:22.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.302+0000 7fc7d37fe640 1 -- 172.21.15.80:0/388674554 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7e4012070 con 0x7fc7ec160310 2024-04-25T13:47:22.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7ec160880 con 0x7fc7ec160310 2024-04-25T13:47:22.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7d37fe640 1 -- 172.21.15.80:0/388674554 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7fc7e405c890 con 0x7fc7ec160310 2024-04-25T13:47:22.497 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:22.497 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:22.497 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7d806f400 msgr2=0x7fc7d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7d806f400 0x7fc7d8071910 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0aa000 tx=0x7fc7dc001c20 comp rx=0 tx=0).stop 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7ec160310 msgr2=0x7fc7ec0afe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7ec160310 0x7fc7ec0afe20 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0b4ba0 tx=0x7fc7e400f4a0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 shutdown_connections 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7d806f400 0x7fc7d8071910 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7ec160310 0x7fc7ec0afe20 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7ec15ff00 0x7fc7ec0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 --2- 172.21.15.80:0/388674554 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d40039c0 0x7fc7ec1412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 >> 172.21.15.80:0/388674554 conn(0x7fc7ec06d080 msgr2=0x7fc7ec159630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:22.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 shutdown_connections 2024-04-25T13:47:22.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.498+0000 7fc7f124b640 1 -- 172.21.15.80:0/388674554 wait complete. 2024-04-25T13:47:22.519 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:47:22.686 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":7968,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.949126925,"mdsmap_epoch":67,"osdmap_epoch":138,"osdmap_epoch_barrier":136,"uptime":16.931947616999999} 2024-04-25T13:47:22.686 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 7968, "mdsmap_epoch": 67, "osdmap_epoch": 138, "osdmap_epoch_barrier": 136, "rank_uptime": 11.949126925, "state": "up:active", "uptime": 16.931947617, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:47:22.687 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-25T13:47:22.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e748a640 1 Processor -- start 2024-04-25T13:47:22.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e748a640 1 -- start start 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e748a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f14e00cf5a0 0x7f14e00cf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e748a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f14e00d23d0 0x7f14e00d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e748a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14e00cfef0 con 0x7f14e00d04b0 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e748a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f14e00b4680 con 0x7f14e00d23d0 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e748a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f14e00d1f40 con 0x7f14e00cf5a0 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5a27640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f14e00d23d0 0x7f14e00d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5a27640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f14e00d23d0 0x7f14e00d27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37146/0 (socket says 172.21.15.80:37146) 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5a27640 1 -- 172.21.15.80:0/1761989190 learned_addr learned my addr 172.21.15.80:0/1761989190 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:22.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5226640 1 --2- 172.21.15.80:0/1761989190 >> v2:172.21.15.87:3300/0 conn(0x7f14e00cf5a0 0x7f14e00cf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5a27640 1 -- 172.21.15.80:0/1761989190 >> v2:172.21.15.87:3300/0 conn(0x7f14e00cf5a0 msgr2=0x7f14e00cf9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:22.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5a27640 1 --2- 172.21.15.80:0/1761989190 >> v2:172.21.15.87:3300/0 conn(0x7f14e00cf5a0 0x7f14e00cf9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5a27640 1 -- 172.21.15.80:0/1761989190 >> v1:172.21.15.87:6789/0 conn(0x7f14e00d04b0 legacy=0x7f14e00d08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:22.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5a27640 1 -- 172.21.15.80:0/1761989190 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e00d1c10 con 0x7f14e00d23d0 2024-04-25T13:47:22.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.846+0000 7f14e5226640 1 --2- 172.21.15.80:0/1761989190 >> v2:172.21.15.87:3300/0 conn(0x7f14e00cf5a0 0x7f14e00cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:22.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e5a27640 1 --2- 172.21.15.80:0/1761989190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e00d27e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f14e00b4800 tx=0x7f14d801c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ca879fd7f3d96c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:22.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14d7fff640 1 -- 172.21.15.80:0/1761989190 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80ac020 con 0x7f14e00d23d0 2024-04-25T13:47:22.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14d7fff640 1 -- 172.21.15.80:0/1761989190 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d80a64a0 con 0x7f14e00d23d0 2024-04-25T13:47:22.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14d7fff640 1 -- 172.21.15.80:0/1761989190 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80a6870 con 0x7f14e00d23d0 2024-04-25T13:47:22.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e748a640 1 -- 172.21.15.80:0/1761989190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 msgr2=0x7f14e00d27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e748a640 1 --2- 172.21.15.80:0/1761989190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e00d27e0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f14e00b4800 tx=0x7f14d801c7d0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e748a640 1 -- 172.21.15.80:0/1761989190 shutdown_connections 2024-04-25T13:47:22.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e748a640 1 --2- 172.21.15.80:0/1761989190 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e00d27e0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e748a640 1 --2- 172.21.15.80:0/1761989190 >> v2:172.21.15.87:3300/0 conn(0x7f14e00cf5a0 0x7f14e00cf9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e748a640 1 -- 172.21.15.80:0/1761989190 >> 172.21.15.80:0/1761989190 conn(0x7f14e00d3a60 msgr2=0x7f14e00d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:22.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e748a640 1 -- 172.21.15.80:0/1761989190 shutdown_connections 2024-04-25T13:47:22.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.850+0000 7f14e748a640 1 -- 172.21.15.80:0/1761989190 wait complete. 2024-04-25T13:47:22.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 Processor -- start 2024-04-25T13:47:22.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 -- start start 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14e00cf5a0 0x7f14e015d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e00d04b0 0x7f14e015dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e015e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f14e00b4280 con 0x7f14e00d23d0 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f14e00d0fd0 con 0x7f14e00d04b0 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f14e00b4400 con 0x7f14e00cf5a0 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e015e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e015e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37158/0 (socket says 172.21.15.80:37158) 2024-04-25T13:47:22.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 -- 172.21.15.80:0/1059743522 learned_addr learned my addr 172.21.15.80:0/1059743522 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e4a25640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e00d04b0 0x7f14e015dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5226640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14e00cf5a0 0x7f14e015d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 -- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14e00cf5a0 msgr2=0x7f14e015d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14e00cf5a0 0x7f14e015d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 -- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e00d04b0 msgr2=0x7f14e015dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e00d04b0 0x7f14e015dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 -- 172.21.15.80:0/1059743522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14e016a530 con 0x7f14e00d23d0 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5226640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14e00cf5a0 0x7f14e015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e5a27640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e015e520 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f14d80ab040 tx=0x7f14d8019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14d5ffb640 1 -- 172.21.15.80:0/1059743522 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80ac040 con 0x7f14e00d23d0 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14d5ffb640 1 -- 172.21.15.80:0/1059743522 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14d80a7680 con 0x7f14e00d23d0 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14e0166980 con 0x7f14e00d23d0 2024-04-25T13:47:22.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14d5ffb640 1 -- 172.21.15.80:0/1059743522 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14d80a7a70 con 0x7f14e00d23d0 2024-04-25T13:47:22.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.854+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14e0166f40 con 0x7f14e00d23d0 2024-04-25T13:47:22.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.858+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14e016a2a0 con 0x7f14e00d23d0 2024-04-25T13:47:22.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.858+0000 7f14d5ffb640 1 -- 172.21.15.80:0/1059743522 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f14d80af020 con 0x7f14e00d23d0 2024-04-25T13:47:22.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.858+0000 7f14d5ffb640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14c406f400 0x7f14c4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:22.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.858+0000 7f14d5ffb640 1 -- 172.21.15.80:0/1059743522 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5767+0+0 (secure 0 0 0) 0x7f14d81296e0 con 0x7f14e00d23d0 2024-04-25T13:47:22.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.858+0000 7f14e5226640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14c406f400 0x7f14c4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:22.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.858+0000 7f14e5226640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14c406f400 0x7f14c4071910 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006fd0 tx=0x7f14dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:22.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:22.862+0000 7f14d5ffb640 1 -- 172.21.15.80:0/1059743522 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f14d80c1060 con 0x7f14e00d23d0 2024-04-25T13:47:23.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.022+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f14e006a440 con 0x7f14e00d23d0 2024-04-25T13:47:23.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14d5ffb640 1 -- 172.21.15.80:0/1059743522 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v139) ==== 181+0+0 (secure 0 0 0) 0x7f14e00cfef0 con 0x7f14e00d23d0 2024-04-25T13:47:23.316 INFO:teuthology.orchestra.run.smithi080.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14c406f400 msgr2=0x7f14c4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14c406f400 0x7f14c4071910 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006fd0 tx=0x7f14dc008040 comp rx=0 tx=0).stop 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 msgr2=0x7f14e015e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e015e520 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f14d80ab040 tx=0x7f14d8019570 comp rx=0 tx=0).stop 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 shutdown_connections 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f14c406f400 0x7f14c4071910 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f14e00d23d0 0x7f14e015e520 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14e00d04b0 0x7f14e015dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:23.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 --2- 172.21.15.80:0/1059743522 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f14e00cf5a0 0x7f14e015d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:23.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.318+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 >> 172.21.15.80:0/1059743522 conn(0x7f14e00d3a60 msgr2=0x7f14e00d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:23.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.322+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 shutdown_connections 2024-04-25T13:47:23.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:23.322+0000 7f14e748a640 1 -- 172.21.15.80:0/1059743522 wait complete. 2024-04-25T13:47:23.346 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-25T13:47:23.510 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_mon_report_interval":"5"} 2024-04-25T13:47:23.510 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-25T13:47:23.510 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-04-25T13:47:23.510 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:23.510 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:23.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:23.516 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-04-25T13:47:23.694 INFO:teuthology.orchestra.run.smithi080.stderr:16+0 records in 2024-04-25T13:47:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:16+0 records out 2024-04-25T13:47:23.695 INFO:teuthology.orchestra.run.smithi080.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.124534 s, 135 MB/s 2024-04-25T13:47:23.696 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:23.696 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:23.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:23.702 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-04-25T13:47:23.854 INFO:teuthology.orchestra.run.smithi080.stderr:17+0 records in 2024-04-25T13:47:23.854 INFO:teuthology.orchestra.run.smithi080.stderr:17+0 records out 2024-04-25T13:47:23.855 INFO:teuthology.orchestra.run.smithi080.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.102101 s, 175 MB/s 2024-04-25T13:47:23.855 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-04-25T13:47:23.856 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:47:24.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb60d287640 1 Processor -- start 2024-04-25T13:47:24.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb60d287640 1 -- start start 2024-04-25T13:47:24.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb60d287640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb60806f5a0 0x7fb608160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:24.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb60d287640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb608161070 0x7fb608161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:24.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb60d287640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6080b4bd0 0x7fb6080b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb60d287640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb6081619c0 con 0x7fb608161070 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb60d287640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb608073f10 con 0x7fb60806f5a0 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb60d287640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb60806fcf0 con 0x7fb6080b4bd0 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6067fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb608161070 0x7fb608161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6067fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb608161070 0x7fb608161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37170/0 (socket says 172.21.15.80:37170) 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6067fc640 1 -- 172.21.15.80:0/1842645542 learned_addr learned my addr 172.21.15.80:0/1842645542 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6077fe640 1 --2- 172.21.15.80:0/1842645542 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6080b4bd0 0x7fb6080b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb606ffd640 1 --2- 172.21.15.80:0/1842645542 >> v2:172.21.15.87:3300/0 conn(0x7fb60806f5a0 0x7fb608160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6067fc640 1 -- 172.21.15.80:0/1842645542 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6080b4bd0 msgr2=0x7fb6080b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6067fc640 1 --2- 172.21.15.80:0/1842645542 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6080b4bd0 0x7fb6080b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6067fc640 1 -- 172.21.15.80:0/1842645542 >> v2:172.21.15.87:3300/0 conn(0x7fb60806f5a0 msgr2=0x7fb608160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6067fc640 1 --2- 172.21.15.80:0/1842645542 >> v2:172.21.15.87:3300/0 conn(0x7fb60806f5a0 0x7fb608160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.006+0000 7fb6067fc640 1 -- 172.21.15.80:0/1842645542 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6080b0c60 con 0x7fb608161070 2024-04-25T13:47:24.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb6067fc640 1 --2- 172.21.15.80:0/1842645542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb608161070 0x7fb608161480 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb608071e30 tx=0x7fb5fc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcbb812e8182f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:24.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb605ffb640 1 -- 172.21.15.80:0/1842645542 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0b0070 con 0x7fb608161070 2024-04-25T13:47:24.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb605ffb640 1 -- 172.21.15.80:0/1842645542 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5fc0ab600 con 0x7fb608161070 2024-04-25T13:47:24.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb605ffb640 1 -- 172.21.15.80:0/1842645542 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5fc0ab9d0 con 0x7fb608161070 2024-04-25T13:47:24.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 -- 172.21.15.80:0/1842645542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb608161070 msgr2=0x7fb608161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:24.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 --2- 172.21.15.80:0/1842645542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb608161070 0x7fb608161480 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb608071e30 tx=0x7fb5fc01c490 comp rx=0 tx=0).stop 2024-04-25T13:47:24.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 -- 172.21.15.80:0/1842645542 shutdown_connections 2024-04-25T13:47:24.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 --2- 172.21.15.80:0/1842645542 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6080b4bd0 0x7fb6080b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 --2- 172.21.15.80:0/1842645542 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb608161070 0x7fb608161480 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 --2- 172.21.15.80:0/1842645542 >> v2:172.21.15.87:3300/0 conn(0x7fb60806f5a0 0x7fb608160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 -- 172.21.15.80:0/1842645542 >> 172.21.15.80:0/1842645542 conn(0x7fb60806d080 msgr2=0x7fb6080b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:24.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 -- 172.21.15.80:0/1842645542 shutdown_connections 2024-04-25T13:47:24.011 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.010+0000 7fb60d287640 1 -- 172.21.15.80:0/1842645542 wait complete. 2024-04-25T13:47:24.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb60d287640 1 Processor -- start 2024-04-25T13:47:24.012 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb60d287640 1 -- start start 2024-04-25T13:47:24.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb60d287640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60806f5a0 0x7fb6080af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:24.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb60d287640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6080b4bd0 0x7fb6080afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:24.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb60d287640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb608161070 0x7fb60815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:24.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb60d287640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb608072bb0 con 0x7fb60806f5a0 2024-04-25T13:47:24.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb60d287640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb6080b5930 con 0x7fb6080b4bd0 2024-04-25T13:47:24.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb60d287640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb608072d30 con 0x7fb608161070 2024-04-25T13:47:24.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60806f5a0 0x7fb6080af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:24.015 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60806f5a0 0x7fb6080af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37184/0 (socket says 172.21.15.80:37184) 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 -- 172.21.15.80:0/2549783938 learned_addr learned my addr 172.21.15.80:0/2549783938 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb6067fc640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6080b4bd0 0x7fb6080afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb6077fe640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb608161070 0x7fb60815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 -- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb608161070 msgr2=0x7fb60815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb608161070 0x7fb60815e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 -- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6080b4bd0 msgr2=0x7fb6080afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6080b4bd0 0x7fb6080afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 -- 172.21.15.80:0/2549783938 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb608154a90 con 0x7fb60806f5a0 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb6077fe640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb608161070 0x7fb60815e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.014+0000 7fb606ffd640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60806f5a0 0x7fb6080af8b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800d620 tx=0x7fb5f800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb5ef7fe640 1 -- 172.21.15.80:0/2549783938 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8004240 con 0x7fb60806f5a0 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb5ef7fe640 1 -- 172.21.15.80:0/2549783938 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f80043e0 con 0x7fb60806f5a0 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb5ef7fe640 1 -- 172.21.15.80:0/2549783938 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f8003620 con 0x7fb60806f5a0 2024-04-25T13:47:24.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb608155040 con 0x7fb60806f5a0 2024-04-25T13:47:24.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb608154570 con 0x7fb60806f5a0 2024-04-25T13:47:24.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb608155760 con 0x7fb60806f5a0 2024-04-25T13:47:24.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb5ef7fe640 1 -- 172.21.15.80:0/2549783938 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5f801b020 con 0x7fb60806f5a0 2024-04-25T13:47:24.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb5ef7fe640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f406f400 0x7fb5f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:24.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb5ef7fe640 1 -- 172.21.15.80:0/2549783938 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5767+0+0 (secure 0 0 0) 0x7fb5f808fa40 con 0x7fb60806f5a0 2024-04-25T13:47:24.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.018+0000 7fb6067fc640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f406f400 0x7fb5f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:24.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.022+0000 7fb6067fc640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f406f400 0x7fb5f4071910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb60806ff70 tx=0x7fb5fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:24.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.022+0000 7fb5ef7fe640 1 -- 172.21.15.80:0/2549783938 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb5f805ca60 con 0x7fb60806f5a0 2024-04-25T13:47:24.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.182+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fb60806a3a0 con 0x7fb60806f5a0 2024-04-25T13:47:24.185 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.186+0000 7fb5ef7fe640 1 -- 172.21.15.80:0/2549783938 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v139) ==== 81+0+34484 (secure 0 0 0) 0x7fb5f805c360 con 0x7fb60806f5a0 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 139, 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:47:23.307344+0000", 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:47:24.186 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 17, 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:47:24.187 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "luminous", 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:24.188 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:24.189 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:24.190 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:24.191 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.192 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.193 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 16, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:08.431048+0000", 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:24.194 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.195 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "135", 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:24.196 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:24.197 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.198 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:24.199 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 17, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:09.003237+0000", 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:24.200 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:24.201 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "139", 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:47:24.202 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:24.203 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:24.204 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.6299999952316284, 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.6299999952316284, 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.6299999952316284, 2024-04-25T13:47:24.205 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.6299999952316284, 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:24.206 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.207 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.208 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.209 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.211 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.215 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:24.216 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:24.217 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.218 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.221 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:24.222 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:24.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:24.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:24.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:24.227 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:24.228 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.231 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.233 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.233 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:24.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.233 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:47:24.234 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:24.235 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:47:24.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:47:24.237 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:47:24.238 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:47:24.239 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:24.240 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.242 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.243 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.244 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:24.245 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:24.246 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:24.247 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:24.248 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.249 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.250 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:24.251 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:24.252 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:24.253 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:24.254 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.255 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:24.256 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [ 2024-04-25T13:47:24.257 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.c", 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 5 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.30", 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:24.258 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 2 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:47:24.259 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:47:24.260 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:47:24.261 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:24.261 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:24.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f406f400 msgr2=0x7fb5f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f406f400 0x7fb5f4071910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb60806ff70 tx=0x7fb5fc008040 comp rx=0 tx=0).stop 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60806f5a0 msgr2=0x7fb6080af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60806f5a0 0x7fb6080af8b0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb5f800d620 tx=0x7fb5f800dae0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 shutdown_connections 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5f406f400 0x7fb5f4071910 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb608161070 0x7fb60815e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6080b4bd0 0x7fb6080afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 --2- 172.21.15.80:0/2549783938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb60806f5a0 0x7fb6080af8b0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 >> 172.21.15.80:0/2549783938 conn(0x7fb60806d080 msgr2=0x7fb60815aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 shutdown_connections 2024-04-25T13:47:24.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:24.190+0000 7fb60d287640 1 -- 172.21.15.80:0/2549783938 wait complete. 2024-04-25T13:47:24.264 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:47:29.265 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:47:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e60a15640 1 Processor -- start 2024-04-25T13:47:29.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e60a15640 1 -- start start 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e60a15640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8e5c161f90 0x7f8e5c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e60a15640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5c0d0d70 con 0x7f8e5c15ec40 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e60a15640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e5c0d1050 con 0x7f8e5c0d0510 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e60a15640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8e5c06dca0 con 0x7f8e5c161f90 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e5ad9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8e5c15ec40 0x7f8e5c168cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45838/0 (socket says 172.21.15.80:45838) 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e5ad9d640 1 -- 172.21.15.80:0/528747268 learned_addr learned my addr 172.21.15.80:0/528747268 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e5a59c640 1 --2- 172.21.15.80:0/528747268 >> v2:172.21.15.87:3300/0 conn(0x7f8e5c161f90 0x7f8e5c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.426+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2188611615 0 0) 0x7f8e5c0d0d70 con 0x7f8e5c15ec40 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8e44003530 con 0x7f8e5c15ec40 2024-04-25T13:47:29.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 741849104 0 0) 0x7f8e44003530 con 0x7f8e5c15ec40 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 >> v2:172.21.15.87:3300/0 conn(0x7f8e5c161f90 msgr2=0x7f8e5c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 --2- 172.21.15.80:0/528747268 >> v2:172.21.15.87:3300/0 conn(0x7f8e5c161f90 0x7f8e5c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 >> v1:172.21.15.87:6789/0 conn(0x7f8e5c0d0510 legacy=0x7f8e5c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c06df80 con 0x7f8e5c15ec40 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f8e54001a00 con 0x7f8e5c15ec40 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 >> v1:172.21.15.80:6789/0 conn(0x7f8e5c15ec40 legacy=0x7f8e5c168cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039c0 0x7f8e44003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e44004810 0x7f8e44004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e44005360 0x7f8e4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c06dca0 con 0x7f8e440039c0 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4400c290 con 0x7f8e44005360 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8e4400c490 con 0x7f8e44004810 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5a59c640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039c0 0x7f8e44003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5ad9d640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e44005360 0x7f8e4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e59d9b640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e44004810 0x7f8e44004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5a59c640 1 -- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e44004810 msgr2=0x7f8e44004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5a59c640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e44004810 0x7f8e44004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5a59c640 1 -- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e44005360 msgr2=0x7f8e4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5a59c640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e44005360 0x7f8e4400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5a59c640 1 -- 172.21.15.80:0/528747268 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e44003530 con 0x7f8e440039c0 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5ad9d640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e44005360 0x7f8e4400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e59d9b640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e44004810 0x7f8e44004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:29.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5ad9d640 1 -- 172.21.15.80:0/528747268 reap_dead start 2024-04-25T13:47:29.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5a59c640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039c0 0x7f8e44003e30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c147e20 tx=0x7f8e4c01c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a7b0dba7fabef8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:29.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0096b0 con 0x7f8e440039c0 2024-04-25T13:47:29.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e4c0a64a0 con 0x7f8e440039c0 2024-04-25T13:47:29.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.430+0000 7f8e5959a640 1 -- 172.21.15.80:0/528747268 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0a6890 con 0x7f8e440039c0 2024-04-25T13:47:29.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 -- 172.21.15.80:0/528747268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039c0 msgr2=0x7f8e44003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:29.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039c0 0x7f8e44003e30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c147e20 tx=0x7f8e4c01c9b0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 -- 172.21.15.80:0/528747268 shutdown_connections 2024-04-25T13:47:29.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 --2- 172.21.15.80:0/528747268 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e440039c0 0x7f8e44003e30 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 -- 172.21.15.80:0/528747268 >> 172.21.15.80:0/528747268 conn(0x7f8e5c06d080 msgr2=0x7f8e5c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:29.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 -- 172.21.15.80:0/528747268 shutdown_connections 2024-04-25T13:47:29.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 -- 172.21.15.80:0/528747268 wait complete. 2024-04-25T13:47:29.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 Processor -- start 2024-04-25T13:47:29.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.434+0000 7f8e60a15640 1 -- start start 2024-04-25T13:47:29.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e440039c0 0x7f8e5c0aff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:29.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e5c15ba20 0x7f8e5c140250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:29.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e5c1576c0 0x7f8e5c157ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:29.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c15f460 con 0x7f8e5c1576c0 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5c0d0920 con 0x7f8e440039c0 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5c0d0aa0 con 0x7f8e5c15ba20 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e5c1576c0 0x7f8e5c157ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e5c1576c0 0x7f8e5c157ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37190/0 (socket says 172.21.15.80:37190) 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 -- 172.21.15.80:0/1090162191 learned_addr learned my addr 172.21.15.80:0/1090162191 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5a59c640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e440039c0 0x7f8e5c0aff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e59d9b640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e5c15ba20 0x7f8e5c140250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 -- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e5c15ba20 msgr2=0x7f8e5c140250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e5c15ba20 0x7f8e5c140250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 -- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e440039c0 msgr2=0x7f8e5c0aff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e440039c0 0x7f8e5c0aff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 -- 172.21.15.80:0/1090162191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5c158010 con 0x7f8e5c1576c0 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e5ad9d640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e5c1576c0 0x7f8e5c157ad0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8e54001470 tx=0x7f8e5400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:29.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e4affd640 1 -- 172.21.15.80:0/1090162191 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5400de70 con 0x7f8e5c1576c0 2024-04-25T13:47:29.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e4affd640 1 -- 172.21.15.80:0/1090162191 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e5400bb20 con 0x7f8e5c1576c0 2024-04-25T13:47:29.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5c140790 con 0x7f8e5c1576c0 2024-04-25T13:47:29.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e4affd640 1 -- 172.21.15.80:0/1090162191 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e54007070 con 0x7f8e5c1576c0 2024-04-25T13:47:29.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5c159730 con 0x7f8e5c1576c0 2024-04-25T13:47:29.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.438+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e5c06a3a0 con 0x7f8e5c1576c0 2024-04-25T13:47:29.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.442+0000 7f8e4affd640 1 -- 172.21.15.80:0/1090162191 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8e5401b020 con 0x7f8e5c1576c0 2024-04-25T13:47:29.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.442+0000 7f8e4affd640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8e3c06f400 0x7f8e3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:29.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.442+0000 7f8e4affd640 1 -- 172.21.15.80:0/1090162191 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5767+0+0 (secure 0 0 0) 0x7f8e5408fa80 con 0x7f8e5c1576c0 2024-04-25T13:47:29.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.442+0000 7f8e5a59c640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8e3c06f400 0x7f8e3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:29.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.442+0000 7f8e5a59c640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8e3c06f400 0x7f8e3c071910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0ad000 tx=0x7f8e4c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:29.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.446+0000 7f8e4affd640 1 -- 172.21.15.80:0/1090162191 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8e54022070 con 0x7f8e5c1576c0 2024-04-25T13:47:29.585 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:47:29.588 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:47:29.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.610+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f8e5c143ee0 con 0x7f8e5c1576c0 2024-04-25T13:47:29.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.610+0000 7f8e4affd640 1 -- 172.21.15.80:0/1090162191 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v139) ==== 81+0+34484 (secure 0 0 0) 0x7f8e5c158010 con 0x7f8e5c1576c0 2024-04-25T13:47:29.610 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:29.610 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:29.610 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 139, 2024-04-25T13:47:29.610 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:29.610 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:47:29.610 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:47:23.307344+0000", 2024-04-25T13:47:29.610 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:47:29.610 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 17, 2024-04-25T13:47:29.611 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "luminous", 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:29.612 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:29.613 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:29.614 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:29.615 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:29.616 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:47:29.617 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 16, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:08.431048+0000", 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:29.618 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:29.619 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "135", 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:29.620 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:29.621 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:29.622 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:29.623 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 17, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:09.003237+0000", 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:29.625 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:29.626 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "139", 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:29.627 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.628 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.629 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.6299999952316284, 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.6299999952316284, 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.6299999952316284, 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.6299999952316284, 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:29.630 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:29.631 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.634 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:47:29.635 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:29.636 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:47:29.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:47:29.639 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:29.640 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:29.641 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:47:29.642 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:47:29.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:47:29.644 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:29.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:29.646 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:29.647 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.648 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.651 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:29.652 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.653 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.656 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.657 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.658 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:29.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.661 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.662 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:29.663 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.664 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.665 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.666 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:47:29.667 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:29.668 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.669 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.670 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:47:29.671 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:29.672 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:29.673 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.674 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:29.675 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:29.676 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:29.677 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:29.678 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [ 2024-04-25T13:47:29.679 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.c", 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 5 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.30", 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 2 2024-04-25T13:47:29.680 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:47:29.681 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:29.682 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:29.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8e3c06f400 msgr2=0x7f8e3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:29.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8e3c06f400 0x7f8e3c071910 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0ad000 tx=0x7f8e4c0046d0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e5c1576c0 msgr2=0x7f8e5c157ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:29.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e5c1576c0 0x7f8e5c157ad0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f8e54001470 tx=0x7f8e5400b4a0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 shutdown_connections 2024-04-25T13:47:29.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8e3c06f400 0x7f8e3c071910 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8e5c1576c0 0x7f8e5c157ad0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8e5c15ba20 0x7f8e5c140250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 --2- 172.21.15.80:0/1090162191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e440039c0 0x7f8e5c0aff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:29.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 >> 172.21.15.80:0/1090162191 conn(0x7f8e5c06d080 msgr2=0x7f8e5c15e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:29.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 shutdown_connections 2024-04-25T13:47:29.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:29.614+0000 7f8e60a15640 1 -- 172.21.15.80:0/1090162191 wait complete. 2024-04-25T13:47:29.687 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:47:34.688 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:47:34.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f396a640 1 Processor -- start 2024-04-25T13:47:34.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f396a640 1 -- start start 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f396a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe7ec0ba500 0x7fe7ec16b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f396a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ec0d4a40 con 0x7fe7ec0d0560 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f396a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7ec0d4d20 con 0x7fe7ec0d2ba0 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f396a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe7ec0ba910 con 0x7fe7ec0ba500 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f0f05640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe7ec0d0560 0x7fe7ec0d0970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43866/0 (socket says 172.21.15.80:43866) 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f0f05640 1 -- 172.21.15.80:0/83345402 learned_addr learned my addr 172.21.15.80:0/83345402 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.850+0000 7fe7f1f07640 1 --2- 172.21.15.80:0/83345402 >> v2:172.21.15.80:3300/0 conn(0x7fe7ec0ba500 0x7fe7ec16b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f1f07640 1 -- 172.21.15.80:0/83345402 >> v1:172.21.15.87:6789/0 conn(0x7fe7ec0d2ba0 legacy=0x7fe7ec0d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f1f07640 1 -- 172.21.15.80:0/83345402 >> v1:172.21.15.80:6789/0 conn(0x7fe7ec0d0560 legacy=0x7fe7ec0d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f1f07640 1 -- 172.21.15.80:0/83345402 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec0d5000 con 0x7fe7ec0ba500 2024-04-25T13:47:34.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7ebfff640 1 -- 172.21.15.80:0/83345402 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 462404391 0 0) 0x7fe7ec0d4a40 con 0x7fe7ec0d0560 2024-04-25T13:47:34.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f1f07640 1 --2- 172.21.15.80:0/83345402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec16b200 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40097c0 tx=0x7fe7e401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1b121370160140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:34.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7ebfff640 1 -- 172.21.15.80:0/83345402 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40b0070 con 0x7fe7ec0ba500 2024-04-25T13:47:34.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7ebfff640 1 -- 172.21.15.80:0/83345402 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7e40ab600 con 0x7fe7ec0ba500 2024-04-25T13:47:34.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7ebfff640 1 -- 172.21.15.80:0/83345402 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e40ab9d0 con 0x7fe7ec0ba500 2024-04-25T13:47:34.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f396a640 1 -- 172.21.15.80:0/83345402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 msgr2=0x7fe7ec16b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:34.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f396a640 1 --2- 172.21.15.80:0/83345402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec16b200 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fe7e40097c0 tx=0x7fe7e401c3d0 comp rx=0 tx=0).stop 2024-04-25T13:47:34.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f396a640 1 -- 172.21.15.80:0/83345402 shutdown_connections 2024-04-25T13:47:34.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f396a640 1 --2- 172.21.15.80:0/83345402 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec16b200 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:34.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f396a640 1 -- 172.21.15.80:0/83345402 >> 172.21.15.80:0/83345402 conn(0x7fe7ec06c9f0 msgr2=0x7fe7ec06ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f396a640 1 -- 172.21.15.80:0/83345402 shutdown_connections 2024-04-25T13:47:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.854+0000 7fe7f396a640 1 -- 172.21.15.80:0/83345402 wait complete. 2024-04-25T13:47:34.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 Processor -- start 2024-04-25T13:47:34.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 -- start start 2024-04-25T13:47:34.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec1fc730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:34.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7ec0d0560 0x7fe7ec15d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7ec0d2ba0 0x7fe7ec15dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec16e0e0 con 0x7fe7ec0ba500 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec16df60 con 0x7fe7ec0d0560 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ec16e260 con 0x7fe7ec0d2ba0 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec1fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec1fc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45776/0 (socket says 172.21.15.80:45776) 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 -- 172.21.15.80:0/3240579048 learned_addr learned my addr 172.21.15.80:0/3240579048 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1f07640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7ec0d2ba0 0x7fe7ec15dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f0f05640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7ec0d0560 0x7fe7ec15d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 -- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7ec0d2ba0 msgr2=0x7fe7ec15dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7ec0d2ba0 0x7fe7ec15dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 -- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7ec0d0560 msgr2=0x7fe7ec15d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:34.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7ec0d0560 0x7fe7ec15d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:34.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 -- 172.21.15.80:0/3240579048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec15c730 con 0x7fe7ec0ba500 2024-04-25T13:47:34.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f1706640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec1fc730 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00b3e0 tx=0x7fe7dc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:34.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7e9ffb640 1 -- 172.21.15.80:0/3240579048 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc015020 con 0x7fe7ec0ba500 2024-04-25T13:47:34.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7e9ffb640 1 -- 172.21.15.80:0/3240579048 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7dc011830 con 0x7fe7ec0ba500 2024-04-25T13:47:34.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec15c9f0 con 0x7fe7ec0ba500 2024-04-25T13:47:34.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7e9ffb640 1 -- 172.21.15.80:0/3240579048 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc011c60 con 0x7fe7ec0ba500 2024-04-25T13:47:34.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.858+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec15cfb0 con 0x7fe7ec0ba500 2024-04-25T13:47:34.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.862+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec06a3a0 con 0x7fe7ec0ba500 2024-04-25T13:47:34.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.862+0000 7fe7e9ffb640 1 -- 172.21.15.80:0/3240579048 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7dc018020 con 0x7fe7ec0ba500 2024-04-25T13:47:34.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.862+0000 7fe7e9ffb640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7d806f400 0x7fe7d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:34.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.862+0000 7fe7e9ffb640 1 -- 172.21.15.80:0/3240579048 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5767+0+0 (secure 0 0 0) 0x7fe7dc08ef70 con 0x7fe7ec0ba500 2024-04-25T13:47:34.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.862+0000 7fe7e9ffb640 1 -- 172.21.15.80:0/3240579048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7fe7d80770a0 con 0x7fe7ec0ba500 2024-04-25T13:47:34.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.862+0000 7fe7f0f05640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7d806f400 0x7fe7d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:34.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.862+0000 7fe7f0f05640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7d806f400 0x7fe7d8071910 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0007fd0 tx=0x7fe7e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:34.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:34.866+0000 7fe7e9ffb640 1 -- 172.21.15.80:0/3240579048 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe7dc01f030 con 0x7fe7ec0ba500 2024-04-25T13:47:35.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.026+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fe7ec0b8a80 con 0x7fe7ec0ba500 2024-04-25T13:47:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7e9ffb640 1 -- 172.21.15.80:0/3240579048 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) ==== 81+0+34503 (secure 0 0 0) 0x7fe7dc05be20 con 0x7fe7ec0ba500 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 140, 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:47:31.219108+0000", 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:47:35.029 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 17, 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:47:35.030 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "luminous", 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:35.031 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:47:35.032 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:47:35.033 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:35.034 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:35.034 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:35.034 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:35.034 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:35.034 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:35.034 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:35.034 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:35.035 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:35.035 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:35.035 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:35.035 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:35.035 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:35.035 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:35.035 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:35.036 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:35.036 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:35.036 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:35.036 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:35.036 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:35.036 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:35.037 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:35.037 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:35.037 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:35.037 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:35.037 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:35.037 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.037 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:35.038 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:35.038 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:35.038 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:35.038 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:35.038 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:35.038 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:35.039 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:35.039 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:35.039 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:35.039 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:35.039 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:35.039 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:47:35.039 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:47:35.039 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:47:35.040 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:47:35.041 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:47:35.041 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:35.041 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:35.041 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:35.041 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.041 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.042 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.042 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 16, 2024-04-25T13:47:35.042 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:47:35.042 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:08.431048+0000", 2024-04-25T13:47:35.042 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:35.042 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:35.043 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:35.043 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:35.043 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:35.043 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:35.043 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:35.043 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:35.043 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:35.043 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:35.044 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "135", 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:35.045 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.046 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:47:35.047 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:35.048 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 17, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:09.003237+0000", 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1027, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:35.049 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:35.050 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "140", 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:35.051 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:35.052 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:47:35.053 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.6299999952316284, 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.6299999952316284, 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.6299999952316284, 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.6299999952316284, 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:35.054 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.055 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.056 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.057 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.058 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.059 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.060 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.061 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.062 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.063 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.064 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.065 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.066 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.067 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.068 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.069 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.070 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.071 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.072 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.073 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.074 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.075 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.076 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.077 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.078 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.079 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.080 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.081 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.082 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.083 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.084 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.085 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.086 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:47:35.087 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.089 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.089 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.089 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.089 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.090 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.091 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.092 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.093 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.094 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:47:35.095 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.096 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.097 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.098 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.099 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.100 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.101 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.102 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:47:35.102 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:47:35.102 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:47:35.102 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:47:35.102 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.102 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.103 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:35.104 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.105 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:47:35.106 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:47:35.107 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:35.108 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [ 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.c", 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.109 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:35.110 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 5 2024-04-25T13:47:35.110 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.110 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.110 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.110 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.110 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.30", 2024-04-25T13:47:35.110 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:35.110 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 2 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:47:35.111 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:47:35.112 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:47:35.113 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:47:35.113 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.113 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:35.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7d806f400 msgr2=0x7fe7d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7d806f400 0x7fe7d8071910 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe7e0007fd0 tx=0x7fe7e0009040 comp rx=0 tx=0).stop 2024-04-25T13:47:35.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 msgr2=0x7fe7ec1fc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec1fc730 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc00b3e0 tx=0x7fe7dc00b8a0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 shutdown_connections 2024-04-25T13:47:35.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7d806f400 0x7fe7d8071910 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe7ec0d2ba0 0x7fe7ec15dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7ec0d0560 0x7fe7ec15d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 --2- 172.21.15.80:0/3240579048 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7ec0ba500 0x7fe7ec1fc730 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 >> 172.21.15.80:0/3240579048 conn(0x7fe7ec06c9f0 msgr2=0x7fe7ec0bac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:35.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 shutdown_connections 2024-04-25T13:47:35.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.030+0000 7fe7f396a640 1 -- 172.21.15.80:0/3240579048 wait complete. 2024-04-25T13:47:35.116 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-25T13:47:35.116 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:35.116 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:35.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:35.123 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-04-25T13:47:35.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:47:35.259 INFO:teuthology.orchestra.run.smithi080.stderr:dd: error writing 'large_file_b': No space left on device 2024-04-25T13:47:35.260 INFO:teuthology.orchestra.run.smithi080.stderr:dd: closing output file 'large_file_b': No space left on device 2024-04-25T13:47:35.260 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:47:35.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.414+0000 7f13db35a640 1 Processor -- start 2024-04-25T13:47:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.414+0000 7f13db35a640 1 -- start start 2024-04-25T13:47:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13db35a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f13d4161f90 0x7f13d40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13db35a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d415ec40 0x7f13d415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13db35a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13d40d0d70 con 0x7f13d40d0510 2024-04-25T13:47:35.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13db35a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f13d40d1050 con 0x7f13d4161f90 2024-04-25T13:47:35.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13db35a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13d406dca0 con 0x7f13d415ec40 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d98f7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d415ec40 0x7f13d415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d88f5640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f13d40d0510 0x7f13d415e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:40538/0 (socket says 172.21.15.80:40538) 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d88f5640 1 -- 172.21.15.80:0/279686112 learned_addr learned my addr 172.21.15.80:0/279686112 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d90f6640 1 --2- 172.21.15.80:0/279686112 >> v2:172.21.15.87:3300/0 conn(0x7f13d4161f90 0x7f13d40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d98f7640 1 -- 172.21.15.80:0/279686112 >> v2:172.21.15.87:3300/0 conn(0x7f13d4161f90 msgr2=0x7f13d40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d98f7640 1 --2- 172.21.15.80:0/279686112 >> v2:172.21.15.87:3300/0 conn(0x7f13d4161f90 0x7f13d40cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d98f7640 1 -- 172.21.15.80:0/279686112 >> v1:172.21.15.87:6789/0 conn(0x7f13d40d0510 legacy=0x7f13d415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d98f7640 1 -- 172.21.15.80:0/279686112 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d406df80 con 0x7f13d415ec40 2024-04-25T13:47:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.418+0000 7f13d90f6640 1 --2- 172.21.15.80:0/279686112 >> v2:172.21.15.87:3300/0 conn(0x7f13d4161f90 0x7f13d40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:35.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13d98f7640 1 --2- 172.21.15.80:0/279686112 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d415ec40 0x7f13d415f050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009d90 tx=0x7f13cc01c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14dc1b5ea9c919c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:35.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13cbfff640 1 -- 172.21.15.80:0/279686112 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0b2070 con 0x7f13d415ec40 2024-04-25T13:47:35.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13cbfff640 1 -- 172.21.15.80:0/279686112 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc0a5870 con 0x7f13d415ec40 2024-04-25T13:47:35.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13cbfff640 1 -- 172.21.15.80:0/279686112 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0a5c40 con 0x7f13d415ec40 2024-04-25T13:47:35.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13db35a640 1 -- 172.21.15.80:0/279686112 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d415ec40 msgr2=0x7f13d415f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13db35a640 1 --2- 172.21.15.80:0/279686112 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d415ec40 0x7f13d415f050 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009d90 tx=0x7f13cc01c880 comp rx=0 tx=0).stop 2024-04-25T13:47:35.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13db35a640 1 -- 172.21.15.80:0/279686112 shutdown_connections 2024-04-25T13:47:35.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13db35a640 1 --2- 172.21.15.80:0/279686112 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d415ec40 0x7f13d415f050 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13db35a640 1 --2- 172.21.15.80:0/279686112 >> v2:172.21.15.87:3300/0 conn(0x7f13d4161f90 0x7f13d40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13db35a640 1 -- 172.21.15.80:0/279686112 >> 172.21.15.80:0/279686112 conn(0x7f13d406d080 msgr2=0x7f13d40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:35.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13db35a640 1 -- 172.21.15.80:0/279686112 shutdown_connections 2024-04-25T13:47:35.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.422+0000 7f13db35a640 1 -- 172.21.15.80:0/279686112 wait complete. 2024-04-25T13:47:35.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13db35a640 1 Processor -- start 2024-04-25T13:47:35.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13db35a640 1 -- start start 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13db35a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d40d0510 0x7f13d4141720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13db35a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d415ec40 0x7f13d40b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13db35a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4161f90 0x7f13d40b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13db35a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13d406f4b0 con 0x7f13d4161f90 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13db35a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13d406f630 con 0x7f13d415ec40 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13db35a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13d40d0920 con 0x7f13d40d0510 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4161f90 0x7f13d40b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4161f90 0x7f13d40b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45784/0 (socket says 172.21.15.80:45784) 2024-04-25T13:47:35.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 -- 172.21.15.80:0/2506663345 learned_addr learned my addr 172.21.15.80:0/2506663345 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d90f6640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d40d0510 0x7f13d4141720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d88f5640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d415ec40 0x7f13d40b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 -- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d40d0510 msgr2=0x7f13d4141720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d40d0510 0x7f13d4141720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 -- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d415ec40 msgr2=0x7f13d40b0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d415ec40 0x7f13d40b0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 -- 172.21.15.80:0/2506663345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d40b09c0 con 0x7f13d4161f90 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d88f5640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d415ec40 0x7f13d40b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13d98f7640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4161f90 0x7f13d40b14f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f13cc01af90 tx=0x7f13cc019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13c9ffb640 1 -- 172.21.15.80:0/2506663345 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0bd020 con 0x7f13d4161f90 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13c9ffb640 1 -- 172.21.15.80:0/2506663345 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc0b8470 con 0x7f13d4161f90 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.426+0000 7f13c9ffb640 1 -- 172.21.15.80:0/2506663345 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0b2040 con 0x7f13d4161f90 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d4142960 con 0x7f13d4161f90 2024-04-25T13:47:35.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d4142f20 con 0x7f13d4161f90 2024-04-25T13:47:35.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d406a3a0 con 0x7f13d4161f90 2024-04-25T13:47:35.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13c9ffb640 1 -- 172.21.15.80:0/2506663345 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13cc0c0020 con 0x7f13d4161f90 2024-04-25T13:47:35.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13c9ffb640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b806f400 0x7f13b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13c9ffb640 1 -- 172.21.15.80:0/2506663345 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5767+0+0 (secure 0 0 0) 0x7f13cc12a220 con 0x7f13d4161f90 2024-04-25T13:47:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13c9ffb640 1 -- 172.21.15.80:0/2506663345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7f13b8077040 con 0x7f13d4161f90 2024-04-25T13:47:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13d90f6640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b806f400 0x7f13b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.430+0000 7f13d90f6640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b806f400 0x7f13b8071910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f13d406fa20 tx=0x7f13d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:35.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.434+0000 7f13c9ffb640 1 -- 172.21.15.80:0/2506663345 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13cc0c7030 con 0x7f13d4161f90 2024-04-25T13:47:35.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.582+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f13d4143160 con 0x7f13d4161f90 2024-04-25T13:47:35.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13c9ffb640 1 -- 172.21.15.80:0/2506663345 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) ==== 81+0+34503 (secure 0 0 0) 0x7f13cc016560 con 0x7f13d4161f90 2024-04-25T13:47:35.585 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:35.585 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:35.585 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 140, 2024-04-25T13:47:35.585 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:35.585 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:47:35.585 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:47:31.219108+0000", 2024-04-25T13:47:35.585 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:47:35.586 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 17, 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "luminous", 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:47:35.587 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:47:35.588 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:35.589 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:35.590 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:35.591 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:35.591 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:35.591 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:35.591 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:35.591 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:35.591 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:35.591 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:35.591 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:35.592 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:35.593 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:35.593 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:35.593 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:35.593 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:35.593 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:35.593 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:35.593 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:47:35.594 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:47:35.594 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.594 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:35.594 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:47:35.594 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.594 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.595 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 16, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:08.431048+0000", 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:35.596 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:35.597 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "135", 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:35.598 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:35.599 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:47:35.600 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:47:35.601 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 17, 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:09.003237+0000", 2024-04-25T13:47:35.602 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1027, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:47:35.603 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "140", 2024-04-25T13:47:35.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:35.605 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:35.606 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:35.607 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.608 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.6299999952316284, 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.6299999952316284, 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.6299999952316284, 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.6299999952316284, 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.609 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.610 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.611 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.611 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.611 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.611 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.611 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.611 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:47:35.611 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.611 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.612 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.613 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:47:35.613 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.613 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.613 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.613 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.614 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.615 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:47:35.616 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.617 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.618 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.619 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.620 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.621 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.622 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.623 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.624 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.625 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.626 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:47:35.627 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.628 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.629 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.630 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.631 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:47:35.633 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.634 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.635 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.636 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.639 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:47:35.639 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:47:35.640 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:47:35.641 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:47:35.641 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.641 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.641 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.641 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.641 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.641 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.641 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:35.642 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.642 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.642 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.642 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.642 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.642 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.642 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.642 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:47:35.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.644 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:47:35.644 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.644 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.644 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.646 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.647 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:47:35.647 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.647 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.647 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.647 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.647 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:47:35.647 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:47:35.648 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:47:35.649 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.650 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.651 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:47:35.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.655 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:35.656 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.658 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:47:35.660 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:35.662 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.664 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:47:35.665 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.666 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [ 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.c", 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 5 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.667 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.30", 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 2 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:47:35.668 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:47:35.669 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:47:35.670 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:47:35.670 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:47:35.670 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:47:35.670 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:35.670 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b806f400 msgr2=0x7f13b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b806f400 0x7f13b8071910 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f13d406fa20 tx=0x7f13d0008040 comp rx=0 tx=0).stop 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4161f90 msgr2=0x7f13d40b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4161f90 0x7f13d40b14f0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f13cc01af90 tx=0x7f13cc019680 comp rx=0 tx=0).stop 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 shutdown_connections 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13b806f400 0x7f13b8071910 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4161f90 0x7f13d40b14f0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d415ec40 0x7f13d40b0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 --2- 172.21.15.80:0/2506663345 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13d40d0510 0x7f13d4141720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.586+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 >> 172.21.15.80:0/2506663345 conn(0x7f13d406d080 msgr2=0x7f13d4160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:35.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.590+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 shutdown_connections 2024-04-25T13:47:35.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.590+0000 7f13db35a640 1 -- 172.21.15.80:0/2506663345 wait complete. 2024-04-25T13:47:35.673 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09c57e3640 1 Processor -- start 2024-04-25T13:47:35.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09c57e3640 1 -- start start 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09c57e3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09c00d2e80 0x7f09c01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09c57e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d4330 0x7f09c00d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09c57e3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09c0161060 con 0x7f09c00d59b0 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09c57e3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f09c0161340 con 0x7f09c00d2e80 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09c57e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09c00d2680 con 0x7f09c00d4330 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09beffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09c00d2e80 0x7f09c01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09beffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f09c00d2e80 0x7f09c01607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45792/0 (socket says 172.21.15.80:45792) 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09beffd640 1 -- 172.21.15.80:0/1198624781 learned_addr learned my addr 172.21.15.80:0/1198624781 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09bf7fe640 1 --2- 172.21.15.80:0/1198624781 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d4330 0x7f09c00d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09beffd640 1 -- 172.21.15.80:0/1198624781 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d4330 msgr2=0x7f09c00d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:35.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09beffd640 1 --2- 172.21.15.80:0/1198624781 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d4330 0x7f09c00d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09beffd640 1 -- 172.21.15.80:0/1198624781 >> v1:172.21.15.87:6789/0 conn(0x7f09c00d59b0 legacy=0x7f09c00d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09beffd640 1 -- 172.21.15.80:0/1198624781 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c00d2960 con 0x7f09c00d2e80 2024-04-25T13:47:35.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.810+0000 7f09bf7fe640 1 --2- 172.21.15.80:0/1198624781 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d4330 0x7f09c00d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:35.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09beffd640 1 --2- 172.21.15.80:0/1198624781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01607a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009870 tx=0x7f09b001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f1441e5d9942004 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:35.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09bdffb640 1 -- 172.21.15.80:0/1198624781 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b00a7070 con 0x7f09c00d2e80 2024-04-25T13:47:35.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09bdffb640 1 -- 172.21.15.80:0/1198624781 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b00b0070 con 0x7f09c00d2e80 2024-04-25T13:47:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09bdffb640 1 -- 172.21.15.80:0/1198624781 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b00ab890 con 0x7f09c00d2e80 2024-04-25T13:47:35.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09c57e3640 1 -- 172.21.15.80:0/1198624781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 msgr2=0x7f09c01607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09c57e3640 1 --2- 172.21.15.80:0/1198624781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01607a0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009870 tx=0x7f09b001c490 comp rx=0 tx=0).stop 2024-04-25T13:47:35.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09c57e3640 1 -- 172.21.15.80:0/1198624781 shutdown_connections 2024-04-25T13:47:35.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09c57e3640 1 --2- 172.21.15.80:0/1198624781 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d4330 0x7f09c00d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09c57e3640 1 --2- 172.21.15.80:0/1198624781 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01607a0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09c57e3640 1 -- 172.21.15.80:0/1198624781 >> 172.21.15.80:0/1198624781 conn(0x7f09c00737a0 msgr2=0x7f09c006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:35.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09c57e3640 1 -- 172.21.15.80:0/1198624781 shutdown_connections 2024-04-25T13:47:35.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.814+0000 7f09c57e3640 1 -- 172.21.15.80:0/1198624781 wait complete. 2024-04-25T13:47:35.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 Processor -- start 2024-04-25T13:47:35.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 -- start start 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01fdde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09c00d4330 0x7f09c01557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d59b0 0x7f09c01599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09c00d3410 con 0x7f09c00d2e80 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f09c00d3290 con 0x7f09c00d4330 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09c015df60 con 0x7f09c00d59b0 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01fdde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01fdde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45800/0 (socket says 172.21.15.80:45800) 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 -- 172.21.15.80:0/81622091 learned_addr learned my addr 172.21.15.80:0/81622091 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09bf7fe640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d59b0 0x7f09c01599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09be7fc640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09c00d4330 0x7f09c01557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 -- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d59b0 msgr2=0x7f09c01599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d59b0 0x7f09c01599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 -- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09c00d4330 msgr2=0x7f09c01557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09c00d4330 0x7f09c01557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 -- 172.21.15.80:0/81622091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c01fe320 con 0x7f09c00d2e80 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09beffd640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01fdde0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f09b00ad040 tx=0x7f09b001acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09a77fe640 1 -- 172.21.15.80:0/81622091 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b00b4800 con 0x7f09c00d2e80 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09a77fe640 1 -- 172.21.15.80:0/81622091 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09b00b49c0 con 0x7f09c00d2e80 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c01f9d90 con 0x7f09c00d2e80 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09a77fe640 1 -- 172.21.15.80:0/81622091 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b00b4db0 con 0x7f09c00d2e80 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.818+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c01fa580 con 0x7f09c00d2e80 2024-04-25T13:47:35.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.822+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09c006a3a0 con 0x7f09c00d2e80 2024-04-25T13:47:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.822+0000 7f09a77fe640 1 -- 172.21.15.80:0/81622091 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f09b00b3380 con 0x7f09c00d2e80 2024-04-25T13:47:35.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.822+0000 7f09a77fe640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09ac06f430 0x7f09ac071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:35.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.822+0000 7f09a77fe640 1 -- 172.21.15.80:0/81622091 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5767+0+0 (secure 0 0 0) 0x7f09b01295e0 con 0x7f09c00d2e80 2024-04-25T13:47:35.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.822+0000 7f09a77fe640 1 -- 172.21.15.80:0/81622091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7f09ac0770d0 con 0x7f09c00d2e80 2024-04-25T13:47:35.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.822+0000 7f09be7fc640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09ac06f430 0x7f09ac071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:35.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.822+0000 7f09be7fc640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09ac06f430 0x7f09ac071940 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f09c015ab20 tx=0x7f09b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:35.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:35.826+0000 7f09a77fe640 1 -- 172.21.15.80:0/81622091 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09b00c4060 con 0x7f09c00d2e80 2024-04-25T13:47:36.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09c015adf0 con 0x7f09c00d2e80 2024-04-25T13:47:36.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09a77fe640 1 -- 172.21.15.80:0/81622091 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7f09c01fe320 con 0x7f09c00d2e80 2024-04-25T13:47:36.024 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:36.025 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:36.025 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09ac06f430 msgr2=0x7f09ac071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09ac06f430 0x7f09ac071940 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f09c015ab20 tx=0x7f09b4008040 comp rx=0 tx=0).stop 2024-04-25T13:47:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 msgr2=0x7f09c01fdde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01fdde0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f09b00ad040 tx=0x7f09b001acd0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 shutdown_connections 2024-04-25T13:47:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09ac06f430 0x7f09ac071940 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09c00d59b0 0x7f09c01599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09c00d4330 0x7f09c01557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 --2- 172.21.15.80:0/81622091 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09c00d2e80 0x7f09c01fdde0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 >> 172.21.15.80:0/81622091 conn(0x7f09c00737a0 msgr2=0x7f09c0161a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 shutdown_connections 2024-04-25T13:47:36.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.026+0000 7f09c57e3640 1 -- 172.21.15.80:0/81622091 wait complete. 2024-04-25T13:47:36.045 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:47:36.213 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":7968,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":25.479133075,"mdsmap_epoch":67,"osdmap_epoch":140,"osdmap_epoch_barrier":136,"uptime":30.461976849999999} 2024-04-25T13:47:36.213 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 7968, "mdsmap_epoch": 67, "osdmap_epoch": 140, "osdmap_epoch_barrier": 136, "rank_uptime": 25.479133075, "state": "up:active", "uptime": 30.46197685, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:47:36.214 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:47:36.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:36.214 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-04-25T13:47:36.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:36.224 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-04-25T13:47:36.278 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:47:36.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.474+0000 7fe1d583c640 1 Processor -- start 2024-04-25T13:47:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.474+0000 7fe1d583c640 1 -- start start 2024-04-25T13:47:36.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1d583c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161240 0x7fe1d00723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:36.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1d583c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1d0161650 0x7fe1d00728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:36.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1d583c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1d006fcf0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1d583c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe1d0073f10 con 0x7fe1d0161650 2024-04-25T13:47:36.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1d583c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d00b0c60 con 0x7fe1d0161240 2024-04-25T13:47:36.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1cf7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1d0161650 0x7fe1d00728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:36.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1cf7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1d0161650 0x7fe1d00728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45814/0 (socket says 172.21.15.80:45814) 2024-04-25T13:47:36.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1ceffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe1d006f5a0 0x7fe1d0160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:43882/0 (socket says 172.21.15.80:43882) 2024-04-25T13:47:36.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1cf7fe640 1 -- 172.21.15.80:0/1932446935 learned_addr learned my addr 172.21.15.80:0/1932446935 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:36.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1ce7fc640 1 --2- 172.21.15.80:0/1932446935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161240 0x7fe1d00723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:36.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1cf7fe640 1 -- 172.21.15.80:0/1932446935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161240 msgr2=0x7fe1d00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:36.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1cf7fe640 1 --2- 172.21.15.80:0/1932446935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161240 0x7fe1d00723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1cf7fe640 1 -- 172.21.15.80:0/1932446935 >> v1:172.21.15.80:6789/0 conn(0x7fe1d006f5a0 legacy=0x7fe1d0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:36.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1cdffb640 1 -- 172.21.15.80:0/1932446935 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1408808740 0 0) 0x7fe1d006fcf0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.478+0000 7fe1cf7fe640 1 -- 172.21.15.80:0/1932446935 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d00b0f40 con 0x7fe1d0161650 2024-04-25T13:47:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1cf7fe640 1 --2- 172.21.15.80:0/1932446935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d0161650 0x7fe1d00728e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009e50 tx=0x7fe1c801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7698d112fb2aea63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1cdffb640 1 -- 172.21.15.80:0/1932446935 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80a56e0 con 0x7fe1d0161650 2024-04-25T13:47:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1cdffb640 1 -- 172.21.15.80:0/1932446935 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c80a5880 con 0x7fe1d0161650 2024-04-25T13:47:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1cdffb640 1 -- 172.21.15.80:0/1932446935 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80a5c50 con 0x7fe1d0161650 2024-04-25T13:47:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 -- 172.21.15.80:0/1932446935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d0161650 msgr2=0x7fe1d00728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1932446935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d0161650 0x7fe1d00728e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8009e50 tx=0x7fe1c801c9a0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 -- 172.21.15.80:0/1932446935 shutdown_connections 2024-04-25T13:47:36.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1932446935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d0161650 0x7fe1d00728e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1932446935 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161240 0x7fe1d00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 -- 172.21.15.80:0/1932446935 >> 172.21.15.80:0/1932446935 conn(0x7fe1d006d080 msgr2=0x7fe1d00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:36.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 -- 172.21.15.80:0/1932446935 shutdown_connections 2024-04-25T13:47:36.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 -- 172.21.15.80:0/1932446935 wait complete. 2024-04-25T13:47:36.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 Processor -- start 2024-04-25T13:47:36.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.482+0000 7fe1d583c640 1 -- start start 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d006f5a0 0x7fe1d0152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d0161240 0x7fe1d01599f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161650 0x7fe1d0153c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d006e820 con 0x7fe1d006f5a0 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe1d006e6a0 con 0x7fe1d0161240 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe1d006e9a0 con 0x7fe1d0161650 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d006f5a0 0x7fe1d0152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d006f5a0 0x7fe1d0152500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:45824/0 (socket says 172.21.15.80:45824) 2024-04-25T13:47:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 -- 172.21.15.80:0/1455760641 learned_addr learned my addr 172.21.15.80:0/1455760641 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1cf7fe640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161650 0x7fe1d0153c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ce7fc640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d0161240 0x7fe1d01599f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 -- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161650 msgr2=0x7fe1d0153c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161650 0x7fe1d0153c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 -- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d0161240 msgr2=0x7fe1d01599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d0161240 0x7fe1d01599f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 -- 172.21.15.80:0/1455760641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1d015bae0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1ceffd640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d006f5a0 0x7fe1d0152500 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c450 tx=0x7fe1c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1b7fff640 1 -- 172.21.15.80:0/1455760641 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0018070 con 0x7fe1d006f5a0 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1b7fff640 1 -- 172.21.15.80:0/1455760641 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1c00074e0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1b7fff640 1 -- 172.21.15.80:0/1455760641 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c00049f0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1d015bdc0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1d015c380 con 0x7fe1d006f5a0 2024-04-25T13:47:36.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.486+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1d006a3a0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.490+0000 7fe1b7fff640 1 -- 172.21.15.80:0/1455760641 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe1c0012400 con 0x7fe1d006f5a0 2024-04-25T13:47:36.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.490+0000 7fe1b7fff640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1bc06f430 0x7fe1bc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:36.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.490+0000 7fe1b7fff640 1 -- 172.21.15.80:0/1455760641 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5767+0+0 (secure 0 0 0) 0x7fe1c008e590 con 0x7fe1d006f5a0 2024-04-25T13:47:36.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.490+0000 7fe1b7fff640 1 -- 172.21.15.80:0/1455760641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7fe1bc077070 con 0x7fe1d006f5a0 2024-04-25T13:47:36.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.490+0000 7fe1ce7fc640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1bc06f430 0x7fe1bc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:36.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.490+0000 7fe1ce7fc640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1bc06f430 0x7fe1bc071940 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00b0960 tx=0x7fe1c4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:36.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.494+0000 7fe1b7fff640 1 -- 172.21.15.80:0/1455760641 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe1c005b5b0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.654+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fe1d00739c0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1b7fff640 1 -- 172.21.15.80:0/1455760641 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) ==== 81+0+34503 (secure 0 0 0) 0x7fe1c005aeb0 con 0x7fe1d006f5a0 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 140, 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:47:31.219108+0000", 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:47:36.657 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:47:36.658 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 17, 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "luminous", 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:36.659 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:36.660 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:36.660 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:36.660 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:36.660 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:36.660 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:36.660 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:36.660 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:36.662 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:36.663 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:36.663 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:36.663 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:36.663 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:36.663 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:36.663 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:36.663 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.664 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.665 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:36.666 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 16, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:08.431048+0000", 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:36.667 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:36.668 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "135", 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:36.669 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:36.670 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:36.671 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:36.672 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:47:36.673 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:47:36.674 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:36.674 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:36.674 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:36.674 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.674 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.674 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.674 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 17, 2024-04-25T13:47:36.674 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:09.003237+0000", 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1027, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:36.675 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "140", 2024-04-25T13:47:36.676 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:36.677 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:36.678 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:36.679 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:36.680 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:36.681 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.6299999952316284, 2024-04-25T13:47:36.681 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.6299999952316284, 2024-04-25T13:47:36.681 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:36.681 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.6299999952316284, 2024-04-25T13:47:36.681 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.6299999952316284, 2024-04-25T13:47:36.681 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:36.681 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:36.681 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:36.682 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.683 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.684 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:47:36.685 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:47:36.686 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:36.687 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:36.688 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:47:36.689 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.690 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:36.691 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.692 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.693 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:47:36.694 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:36.694 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.694 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.694 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.694 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:47:36.694 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:47:36.694 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:47:36.694 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:36.695 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:36.696 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:36.697 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:36.698 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:36.699 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:36.700 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:36.701 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.702 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.703 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.704 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:47:36.705 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:36.705 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.705 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.705 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.705 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:47:36.705 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.706 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:36.707 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:36.708 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.709 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.710 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.711 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:36.712 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:36.713 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:47:36.714 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:36.715 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:47:36.716 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:47:36.717 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.718 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.719 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.722 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.723 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:36.723 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.723 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.723 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.723 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:47:36.724 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:36.725 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.727 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:47:36.727 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:36.727 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.731 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.731 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.731 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:36.731 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.731 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.733 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:36.735 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:36.736 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:36.737 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:36.738 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:36.739 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:36.740 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [ 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.c", 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 5 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.741 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.30", 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 2 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:47:36.742 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:47:36.743 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:36.744 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:36.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1bc06f430 msgr2=0x7fe1bc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:36.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1bc06f430 0x7fe1bc071940 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00b0960 tx=0x7fe1c4008040 comp rx=0 tx=0).stop 2024-04-25T13:47:36.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d006f5a0 msgr2=0x7fe1d0152500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:36.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d006f5a0 0x7fe1d0152500 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000c450 tx=0x7fe1c000c910 comp rx=0 tx=0).stop 2024-04-25T13:47:36.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 shutdown_connections 2024-04-25T13:47:36.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe1bc06f430 0x7fe1bc071940 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1d0161650 0x7fe1d0153c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe1d0161240 0x7fe1d01599f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 --2- 172.21.15.80:0/1455760641 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1d006f5a0 0x7fe1d0152500 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:36.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.658+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 >> 172.21.15.80:0/1455760641 conn(0x7fe1d006d080 msgr2=0x7fe1d00cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:36.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.662+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 shutdown_connections 2024-04-25T13:47:36.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:36.662+0000 7fe1d583c640 1 -- 172.21.15.80:0/1455760641 wait complete. 2024-04-25T13:47:36.747 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:47:41.749 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:47:41.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef197cc640 1 Processor -- start 2024-04-25T13:47:41.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef197cc640 1 -- start start 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef197cc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fef14161240 0x7fef140723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef197cc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fef14161650 0x7fef140728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef197cc640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef1406fcf0 con 0x7fef1406f5a0 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef197cc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fef14073f10 con 0x7fef14161240 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef197cc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fef140b0c60 con 0x7fef14161650 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef12ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fef1406f5a0 0x7fef14160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45464/0 (socket says 172.21.15.80:45464) 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef12ffd640 1 -- 172.21.15.80:0/2332631604 learned_addr learned my addr 172.21.15.80:0/2332631604 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef0bfff640 1 --2- 172.21.15.80:0/2332631604 >> v2:172.21.15.80:3300/0 conn(0x7fef14161240 0x7fef140723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef137fe640 1 --2- 172.21.15.80:0/2332631604 >> v2:172.21.15.87:3300/0 conn(0x7fef14161650 0x7fef140728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef127fc640 1 -- 172.21.15.80:0/2332631604 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2874238932 0 0) 0x7fef1406fcf0 con 0x7fef1406f5a0 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef127fc640 1 -- 172.21.15.80:0/2332631604 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef00003530 con 0x7fef1406f5a0 2024-04-25T13:47:41.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef0bfff640 1 -- 172.21.15.80:0/2332631604 >> v2:172.21.15.87:3300/0 conn(0x7fef14161650 msgr2=0x7fef140728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:41.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef0bfff640 1 --2- 172.21.15.80:0/2332631604 >> v2:172.21.15.87:3300/0 conn(0x7fef14161650 0x7fef140728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:41.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef0bfff640 1 -- 172.21.15.80:0/2332631604 >> v1:172.21.15.80:6789/0 conn(0x7fef1406f5a0 legacy=0x7fef14160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:41.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.910+0000 7fef0bfff640 1 -- 172.21.15.80:0/2332631604 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef140b0f40 con 0x7fef14161240 2024-04-25T13:47:41.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef0bfff640 1 --2- 172.21.15.80:0/2332631604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161240 0x7fef140723a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fef14070270 tx=0x7feefc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fddc5e1125eae8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:41.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef127fc640 1 -- 172.21.15.80:0/2332631604 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0b0070 con 0x7fef14161240 2024-04-25T13:47:41.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef127fc640 1 -- 172.21.15.80:0/2332631604 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feefc0ab600 con 0x7fef14161240 2024-04-25T13:47:41.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef127fc640 1 -- 172.21.15.80:0/2332631604 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0ab9d0 con 0x7fef14161240 2024-04-25T13:47:41.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef197cc640 1 -- 172.21.15.80:0/2332631604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161240 msgr2=0x7fef140723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:41.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef197cc640 1 --2- 172.21.15.80:0/2332631604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161240 0x7fef140723a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fef14070270 tx=0x7feefc01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:47:41.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef197cc640 1 -- 172.21.15.80:0/2332631604 shutdown_connections 2024-04-25T13:47:41.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef197cc640 1 --2- 172.21.15.80:0/2332631604 >> v2:172.21.15.87:3300/0 conn(0x7fef14161650 0x7fef140728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:41.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef197cc640 1 --2- 172.21.15.80:0/2332631604 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161240 0x7fef140723a0 secure :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fef14070270 tx=0x7feefc01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:47:41.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef197cc640 1 -- 172.21.15.80:0/2332631604 >> 172.21.15.80:0/2332631604 conn(0x7fef1406d080 msgr2=0x7fef140b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:41.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef197cc640 1 -- 172.21.15.80:0/2332631604 shutdown_connections 2024-04-25T13:47:41.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.914+0000 7fef197cc640 1 -- 172.21.15.80:0/2332631604 wait complete. 2024-04-25T13:47:41.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef197cc640 1 Processor -- start 2024-04-25T13:47:41.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef197cc640 1 -- start start 2024-04-25T13:47:41.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef197cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef1406f5a0 0x7fef14157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:41.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef197cc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161650 0x7fef14141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:41.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef197cc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef141595f0 0x7fef140b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:41.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef197cc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fef1416a260 con 0x7fef14161650 2024-04-25T13:47:41.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef197cc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fef1416a0e0 con 0x7fef1406f5a0 2024-04-25T13:47:41.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef197cc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fef1416a3e0 con 0x7fef141595f0 2024-04-25T13:47:41.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161650 0x7fef14141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161650 0x7fef14141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43762/0 (socket says 172.21.15.80:43762) 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 -- 172.21.15.80:0/3375604936 learned_addr learned my addr 172.21.15.80:0/3375604936 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef137fe640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef141595f0 0x7fef140b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef12ffd640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef1406f5a0 0x7fef14157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 -- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef141595f0 msgr2=0x7fef140b1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef141595f0 0x7fef140b1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 -- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef1406f5a0 msgr2=0x7fef14157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef1406f5a0 0x7fef14157990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 -- 172.21.15.80:0/3375604936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef14158300 con 0x7fef14161650 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef12ffd640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef1406f5a0 0x7fef14157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0bfff640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161650 0x7fef14141480 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7feefc01c8d0 tx=0x7feefc019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0b7fe640 1 -- 172.21.15.80:0/3375604936 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0b0040 con 0x7fef14161650 2024-04-25T13:47:41.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0b7fe640 1 -- 172.21.15.80:0/3375604936 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feefc0b9070 con 0x7fef14161650 2024-04-25T13:47:41.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.918+0000 7fef0b7fe640 1 -- 172.21.15.80:0/3375604936 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc0b4670 con 0x7fef14161650 2024-04-25T13:47:41.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef141585e0 con 0x7fef14161650 2024-04-25T13:47:41.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef14158ba0 con 0x7fef14161650 2024-04-25T13:47:41.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1414f1c0 con 0x7fef14161650 2024-04-25T13:47:41.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef0b7fe640 1 -- 172.21.15.80:0/3375604936 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7feefc0b4810 con 0x7fef14161650 2024-04-25T13:47:41.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef0b7fe640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feef406f400 0x7feef4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:41.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef0b7fe640 1 -- 172.21.15.80:0/3375604936 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5767+0+0 (secure 0 0 0) 0x7feefc1290a0 con 0x7fef14161650 2024-04-25T13:47:41.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef0b7fe640 1 -- 172.21.15.80:0/3375604936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=141}) -- 0x7feef40770a0 con 0x7fef14161650 2024-04-25T13:47:41.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef12ffd640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feef406f400 0x7feef4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:41.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.922+0000 7fef12ffd640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feef406f400 0x7feef4071910 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fef04005890 tx=0x7fef04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:41.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:41.926+0000 7fef0b7fe640 1 -- 172.21.15.80:0/3375604936 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7feefc016070 con 0x7fef14161650 2024-04-25T13:47:42.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.090+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fef1406a3a0 con 0x7fef14161650 2024-04-25T13:47:42.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.090+0000 7fef0b7fe640 1 -- 172.21.15.80:0/3375604936 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v140) ==== 81+0+34503 (secure 0 0 0) 0x7feefc0f5f50 con 0x7fef14161650 2024-04-25T13:47:42.091 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:42.091 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:42.091 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 140, 2024-04-25T13:47:42.091 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:42.091 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:47:42.091 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:47:31.219108+0000", 2024-04-25T13:47:42.091 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:47:42.091 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:47:42.092 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 17, 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "luminous", 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:42.093 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:47:42.094 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:42.095 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:42.096 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:42.097 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:47:42.098 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:42.099 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 16, 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:08.431048+0000", 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:42.100 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:47:42.101 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "135", 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:42.102 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:42.103 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:42.104 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:42.105 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.106 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 17, 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:09.003237+0000", 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1027, 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:42.107 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:42.108 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "140", 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:47:42.109 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:42.110 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:42.111 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.6299999952316284, 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.6299999952316284, 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:42.112 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.6299999952316284, 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.6299999952316284, 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:42.113 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:47:42.114 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:42.115 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.116 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.117 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.118 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:42.119 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.120 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.121 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.122 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.123 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:42.124 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.125 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:42.126 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.127 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.128 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:42.129 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:42.130 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.131 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.132 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.133 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.134 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:42.135 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.136 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:42.137 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:42.138 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:42.139 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:42.140 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:42.141 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.142 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.143 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:42.144 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:42.145 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:42.146 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.147 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.148 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:47:42.149 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:47:42.150 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:42.151 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:42.152 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.153 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.154 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.155 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.156 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:42.157 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:47:42.158 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:42.159 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:42.160 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:42.161 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:42.162 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [ 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.c", 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 5 2024-04-25T13:47:42.163 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.30", 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 2 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:47:42.164 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:47:42.165 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:42.166 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feef406f400 msgr2=0x7feef4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feef406f400 0x7feef4071910 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fef04005890 tx=0x7fef04009040 comp rx=0 tx=0).stop 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161650 msgr2=0x7fef14141480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161650 0x7fef14141480 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7feefc01c8d0 tx=0x7feefc019190 comp rx=0 tx=0).stop 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 shutdown_connections 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feef406f400 0x7feef4071910 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fef141595f0 0x7fef140b1f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fef14161650 0x7fef14141480 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:42.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 --2- 172.21.15.80:0/3375604936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fef1406f5a0 0x7fef14157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:42.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 >> 172.21.15.80:0/3375604936 conn(0x7fef1406d080 msgr2=0x7fef1406e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:42.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 shutdown_connections 2024-04-25T13:47:42.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:42.094+0000 7fef197cc640 1 -- 172.21.15.80:0/3375604936 wait complete. 2024-04-25T13:47:42.169 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:47:47.171 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:47:47.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffacb814640 1 Processor -- start 2024-04-25T13:47:47.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffacb814640 1 -- start start 2024-04-25T13:47:47.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffacb814640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffac406f5a0 0x7ffac4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffacb814640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac4161070 0x7ffac4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffacb814640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffac406fcf0 con 0x7ffac40b4bd0 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffacb814640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ffac41619c0 con 0x7ffac406f5a0 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffacb814640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ffac4073f10 con 0x7ffac4161070 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffac9db1640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ffac40b4bd0 0x7ffac40b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45478/0 (socket says 172.21.15.80:45478) 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffac95b0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ffac406f5a0 0x7ffac4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffac9db1640 1 -- 172.21.15.80:0/956132780 learned_addr learned my addr 172.21.15.80:0/956132780 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffac8daf640 1 --2- 172.21.15.80:0/956132780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac4161070 0x7ffac4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffab7fff640 1 -- 172.21.15.80:0/956132780 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 883794666 0 0) 0x7ffac406fcf0 con 0x7ffac40b4bd0 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffab7fff640 1 -- 172.21.15.80:0/956132780 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffab0003530 con 0x7ffac40b4bd0 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffac95b0640 1 -- 172.21.15.80:0/956132780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac4161070 msgr2=0x7ffac4161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffac95b0640 1 --2- 172.21.15.80:0/956132780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac4161070 0x7ffac4161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffac95b0640 1 -- 172.21.15.80:0/956132780 >> v1:172.21.15.80:6789/0 conn(0x7ffac40b4bd0 legacy=0x7ffac40b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.334+0000 7ffac95b0640 1 -- 172.21.15.80:0/956132780 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac40b0c60 con 0x7ffac406f5a0 2024-04-25T13:47:47.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffac95b0640 1 --2- 172.21.15.80:0/956132780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4160b30 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7ffab80097f0 tx=0x7ffab801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afcc76d771ed624f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:47.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffab7fff640 1 -- 172.21.15.80:0/956132780 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab80b0070 con 0x7ffac406f5a0 2024-04-25T13:47:47.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffab7fff640 1 -- 172.21.15.80:0/956132780 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab80ab600 con 0x7ffac406f5a0 2024-04-25T13:47:47.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffab7fff640 1 -- 172.21.15.80:0/956132780 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab80ab9d0 con 0x7ffac406f5a0 2024-04-25T13:47:47.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffacb814640 1 -- 172.21.15.80:0/956132780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 msgr2=0x7ffac4160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffacb814640 1 --2- 172.21.15.80:0/956132780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4160b30 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7ffab80097f0 tx=0x7ffab801c410 comp rx=0 tx=0).stop 2024-04-25T13:47:47.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffacb814640 1 -- 172.21.15.80:0/956132780 shutdown_connections 2024-04-25T13:47:47.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffacb814640 1 --2- 172.21.15.80:0/956132780 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac4161070 0x7ffac4161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffacb814640 1 --2- 172.21.15.80:0/956132780 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4160b30 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffacb814640 1 -- 172.21.15.80:0/956132780 >> 172.21.15.80:0/956132780 conn(0x7ffac406d080 msgr2=0x7ffac40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:47.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffacb814640 1 -- 172.21.15.80:0/956132780 shutdown_connections 2024-04-25T13:47:47.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.338+0000 7ffacb814640 1 -- 172.21.15.80:0/956132780 wait complete. 2024-04-25T13:47:47.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 Processor -- start 2024-04-25T13:47:47.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 -- start start 2024-04-25T13:47:47.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac40b4bd0 0x7ffac415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4161070 0x7ffac414fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ffac416a860 con 0x7ffac406f5a0 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ffac416a6e0 con 0x7ffac4161070 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ffac416a9e0 con 0x7ffac40b4bd0 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43786/0 (socket says 172.21.15.80:43786) 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 -- 172.21.15.80:0/3826648740 learned_addr learned my addr 172.21.15.80:0/3826648740 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac9db1640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4161070 0x7ffac414fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac8daf640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac40b4bd0 0x7ffac415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 -- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac40b4bd0 msgr2=0x7ffac415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac40b4bd0 0x7ffac415c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 -- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4161070 msgr2=0x7ffac414fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4161070 0x7ffac414fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 -- 172.21.15.80:0/3826648740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac4150420 con 0x7ffac406f5a0 2024-04-25T13:47:47.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffac95b0640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4154760 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ffab80097f0 tx=0x7ffab801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:47.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffab5ffb640 1 -- 172.21.15.80:0/3826648740 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab80b0050 con 0x7ffac406f5a0 2024-04-25T13:47:47.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffab5ffb640 1 -- 172.21.15.80:0/3826648740 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab80bc070 con 0x7ffac406f5a0 2024-04-25T13:47:47.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffab5ffb640 1 -- 172.21.15.80:0/3826648740 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab80b5870 con 0x7ffac406f5a0 2024-04-25T13:47:47.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac4154fb0 con 0x7ffac406f5a0 2024-04-25T13:47:47.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac415daa0 con 0x7ffac406f5a0 2024-04-25T13:47:47.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.342+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac415d5b0 con 0x7ffac406f5a0 2024-04-25T13:47:47.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.346+0000 7ffab5ffb640 1 -- 172.21.15.80:0/3826648740 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ffab80b5a10 con 0x7ffac406f5a0 2024-04-25T13:47:47.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.346+0000 7ffab5ffb640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffaa806f400 0x7ffaa8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.346+0000 7ffab5ffb640 1 -- 172.21.15.80:0/3826648740 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7ffab8129980 con 0x7ffac406f5a0 2024-04-25T13:47:47.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.346+0000 7ffac8daf640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffaa806f400 0x7ffaa8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.346+0000 7ffac8daf640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffaa806f400 0x7ffaa8071910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffac40d2870 tx=0x7ffac0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.350+0000 7ffab5ffb640 1 -- 172.21.15.80:0/3826648740 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ffab80c6020 con 0x7ffac406f5a0 2024-04-25T13:47:47.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.510+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7ffac406a3a0 con 0x7ffac406f5a0 2024-04-25T13:47:47.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.514+0000 7ffab5ffb640 1 -- 172.21.15.80:0/3826648740 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) ==== 81+0+34484 (secure 0 0 0) 0x7ffab8016090 con 0x7ffac406f5a0 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 141, 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:47:46.223394+0000", 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:47:47.515 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 17, 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "luminous", 2024-04-25T13:47:47.516 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:47.517 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:47.518 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:47.519 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:47.520 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:47.521 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:47:47.522 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 16, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:08.431048+0000", 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:47.523 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:47.524 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "135", 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:47.525 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:47.526 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:47.527 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:47.528 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.529 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 17, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:09.003237+0000", 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:47.530 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:47.531 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "141", 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:47.532 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:47.533 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:47.534 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.6299999952316284, 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.6299999952316284, 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.6299999952316284, 2024-04-25T13:47:47.535 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.6299999952316284, 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.536 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.537 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:47:47.538 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:47:47.539 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.540 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:47:47.541 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.542 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:47:47.543 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.545 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.546 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.547 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:47:47.548 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:47:47.549 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.550 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:47:47.551 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.552 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.553 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.554 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.555 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.556 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.557 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.558 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:47:47.559 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.560 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.561 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.562 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.563 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.564 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.565 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.566 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.567 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:47:47.568 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.569 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.570 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.571 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.572 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:47:47.573 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.574 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.575 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.576 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.577 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.578 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.579 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.580 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.581 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.582 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:47.583 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.584 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.585 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [ 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.c", 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 5 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.30", 2024-04-25T13:47:47.586 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 2 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:47:47.587 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:47:47.588 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:47:47.589 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:47:47.589 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:47:47.589 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:47:47.589 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.589 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffaa806f400 msgr2=0x7ffaa8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffaa806f400 0x7ffaa8071910 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffac40d2870 tx=0x7ffac0008040 comp rx=0 tx=0).stop 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 msgr2=0x7ffac4154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4154760 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7ffab80097f0 tx=0x7ffab801ac50 comp rx=0 tx=0).stop 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 shutdown_connections 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ffaa806f400 0x7ffaa8071910 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffac4161070 0x7ffac414fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ffac40b4bd0 0x7ffac415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 --2- 172.21.15.80:0/3826648740 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ffac406f5a0 0x7ffac4154760 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 >> 172.21.15.80:0/3826648740 conn(0x7ffac406d080 msgr2=0x7ffac40cfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:47.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 shutdown_connections 2024-04-25T13:47:47.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.518+0000 7ffacb814640 1 -- 172.21.15.80:0/3826648740 wait complete. 2024-04-25T13:47:47.592 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-25T13:47:47.592 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:47:47.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff3776ed640 1 Processor -- start 2024-04-25T13:47:47.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff3776ed640 1 -- start start 2024-04-25T13:47:47.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff3776ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff370072ac0 0x7ff37006f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff3776ed640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff3700d1570 0x7ff37006fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff3776ed640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff370070070 con 0x7ff3700726b0 2024-04-25T13:47:47.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff3776ed640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff3700ce0d0 con 0x7ff3700d1570 2024-04-25T13:47:47.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff3776ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff3700ce930 con 0x7ff370072ac0 2024-04-25T13:47:47.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff375c8a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff3700d1570 0x7ff37006fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff375c8a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff3700d1570 0x7ff37006fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43802/0 (socket says 172.21.15.80:43802) 2024-04-25T13:47:47.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff375c8a640 1 -- 172.21.15.80:0/4038252458 learned_addr learned my addr 172.21.15.80:0/4038252458 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:47.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff374c88640 1 --2- 172.21.15.80:0/4038252458 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff370072ac0 0x7ff37006f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff375c8a640 1 -- 172.21.15.80:0/4038252458 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff370072ac0 msgr2=0x7ff37006f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:47.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff375c8a640 1 --2- 172.21.15.80:0/4038252458 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff370072ac0 0x7ff37006f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff375c8a640 1 -- 172.21.15.80:0/4038252458 >> v1:172.21.15.87:6789/0 conn(0x7ff3700726b0 legacy=0x7ff3700d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:47.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.750+0000 7ff375c8a640 1 -- 172.21.15.80:0/4038252458 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3700cc930 con 0x7ff3700d1570 2024-04-25T13:47:47.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff375c8a640 1 --2- 172.21.15.80:0/4038252458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3700d1570 0x7ff37006fb30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff368009870 tx=0x7ff36801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597ecb6ec0c1fd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:47.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff36ffff640 1 -- 172.21.15.80:0/4038252458 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3680b1070 con 0x7ff3700d1570 2024-04-25T13:47:47.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff36ffff640 1 -- 172.21.15.80:0/4038252458 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3680ac630 con 0x7ff3700d1570 2024-04-25T13:47:47.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff36ffff640 1 -- 172.21.15.80:0/4038252458 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3680a8050 con 0x7ff3700d1570 2024-04-25T13:47:47.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 -- 172.21.15.80:0/4038252458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3700d1570 msgr2=0x7ff37006fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 --2- 172.21.15.80:0/4038252458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3700d1570 0x7ff37006fb30 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7ff368009870 tx=0x7ff36801c520 comp rx=0 tx=0).stop 2024-04-25T13:47:47.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 -- 172.21.15.80:0/4038252458 shutdown_connections 2024-04-25T13:47:47.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 --2- 172.21.15.80:0/4038252458 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff3700d1570 0x7ff37006fb30 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 --2- 172.21.15.80:0/4038252458 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff370072ac0 0x7ff37006f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 -- 172.21.15.80:0/4038252458 >> 172.21.15.80:0/4038252458 conn(0x7ff3700d20d0 msgr2=0x7ff3700d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:47.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 -- 172.21.15.80:0/4038252458 shutdown_connections 2024-04-25T13:47:47.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 -- 172.21.15.80:0/4038252458 wait complete. 2024-04-25T13:47:47.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 Processor -- start 2024-04-25T13:47:47.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.754+0000 7ff3776ed640 1 -- start start 2024-04-25T13:47:47.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3700726b0 0x7ff3700b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff370072ac0 0x7ff3700c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3700d1570 0x7ff3700c71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff37015e2c0 con 0x7ff370072ac0 2024-04-25T13:47:47.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff37015e140 con 0x7ff3700d1570 2024-04-25T13:47:47.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff370164640 con 0x7ff3700726b0 2024-04-25T13:47:47.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff370072ac0 0x7ff3700c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff370072ac0 0x7ff3700c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43806/0 (socket says 172.21.15.80:43806) 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 -- 172.21.15.80:0/1912319086 learned_addr learned my addr 172.21.15.80:0/1912319086 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff375c8a640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3700d1570 0x7ff3700c71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff375489640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3700726b0 0x7ff3700b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 -- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3700726b0 msgr2=0x7ff3700b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3700726b0 0x7ff3700b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 -- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3700d1570 msgr2=0x7ff3700c71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3700d1570 0x7ff3700c71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 -- 172.21.15.80:0/1912319086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3700b4e20 con 0x7ff370072ac0 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff374c88640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff370072ac0 0x7ff3700c6c70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c610 tx=0x7ff36400cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff36dffb640 1 -- 172.21.15.80:0/1912319086 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364015070 con 0x7ff370072ac0 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff36dffb640 1 -- 172.21.15.80:0/1912319086 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff364004220 con 0x7ff370072ac0 2024-04-25T13:47:47.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3700b5100 con 0x7ff370072ac0 2024-04-25T13:47:47.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff36dffb640 1 -- 172.21.15.80:0/1912319086 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364003620 con 0x7ff370072ac0 2024-04-25T13:47:47.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3700b56c0 con 0x7ff370072ac0 2024-04-25T13:47:47.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.758+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff37006a3a0 con 0x7ff370072ac0 2024-04-25T13:47:47.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.762+0000 7ff36dffb640 1 -- 172.21.15.80:0/1912319086 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff3640037c0 con 0x7ff370072ac0 2024-04-25T13:47:47.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.762+0000 7ff36dffb640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff35c06f430 0x7ff35c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:47.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.762+0000 7ff36dffb640 1 -- 172.21.15.80:0/1912319086 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7ff36408f570 con 0x7ff370072ac0 2024-04-25T13:47:47.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.762+0000 7ff375489640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff35c06f430 0x7ff35c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:47.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.762+0000 7ff375489640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff35c06f430 0x7ff35c071940 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff3700511a0 tx=0x7ff3600094f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:47.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.766+0000 7ff36dffb640 1 -- 172.21.15.80:0/1912319086 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff36405c420 con 0x7ff370072ac0 2024-04-25T13:47:47.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.922+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7ff37015a810 con 0x7ff370072ac0 2024-04-25T13:47:47.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.926+0000 7ff36dffb640 1 -- 172.21.15.80:0/1912319086 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v141) ==== 81+0+34484 (secure 0 0 0) 0x7ff36405bd20 con 0x7ff370072ac0 2024-04-25T13:47:47.926 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 141, 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:47:46.223394+0000", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:47:47.927 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 17, 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "luminous", 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:47:47.928 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:47.929 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:47:47.930 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:47.931 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.932 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:47:47.933 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.934 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 16, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:08.431048+0000", 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:47.935 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.936 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "135", 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:47.937 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:47.938 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.939 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.75, 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.75, 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.75, 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.75, 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:47.940 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 17, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:47:09.003237+0000", 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:47:47.941 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:47:47.942 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "141", 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:47:47.943 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:47:47.944 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:47:47.945 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.6299999952316284, 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.6299999952316284, 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.6299999952316284, 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.6299999952316284, 2024-04-25T13:47:47.946 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.947 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.948 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:47:47.949 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.950 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:47:47.951 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.952 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.953 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.954 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.955 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.956 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.957 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.958 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.959 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.960 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.961 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:47:47.962 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.963 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.964 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.965 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.966 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.967 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.968 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.969 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.970 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.971 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.972 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:47:47.973 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.974 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.975 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.976 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.977 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.978 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.979 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 134, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.980 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.981 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.983 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.984 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.985 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:47:47.986 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:47:47.987 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 137, 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:47:47.989 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.990 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:47:47.991 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.992 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:47:47.993 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.994 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.995 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.996 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.997 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:47:47.998 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:47:47.999 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [ 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.c", 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 5 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: "pgid": "17.30", 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: "mappings": [ 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:47:48.000 INFO:teuthology.orchestra.run.smithi080.stdout: "from": 0, 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "to": 2 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:47:48.001 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:47:48.002 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:47:48.003 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:47:48.003 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:47:48.003 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:47:48.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff35c06f430 msgr2=0x7ff35c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:48.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff35c06f430 0x7ff35c071940 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ff3700511a0 tx=0x7ff3600094f0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff370072ac0 msgr2=0x7ff3700c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff370072ac0 0x7ff3700c6c70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c610 tx=0x7ff36400cad0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 shutdown_connections 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff35c06f430 0x7ff35c071940 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3700d1570 0x7ff3700c71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff370072ac0 0x7ff3700c6c70 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 --2- 172.21.15.80:0/1912319086 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff3700726b0 0x7ff3700b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 >> 172.21.15.80:0/1912319086 conn(0x7ff3700d20d0 msgr2=0x7ff3700b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 shutdown_connections 2024-04-25T13:47:48.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:47.930+0000 7ff3776ed640 1 -- 172.21.15.80:0/1912319086 wait complete. 2024-04-25T13:47:48.006 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:48.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b1371f640 1 Processor -- start 2024-04-25T13:47:48.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b1371f640 1 -- start start 2024-04-25T13:47:48.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b1371f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3b0c1622d0 0x7f3b0c0725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:48.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b1371f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3b0c168220 0x7f3b0c168630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:48.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b1371f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b0c072b00 con 0x7f3b0c06eb40 2024-04-25T13:47:48.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b1371f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0c072120 con 0x7f3b0c1622d0 2024-04-25T13:47:48.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b1371f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3b0c06dd80 con 0x7f3b0c168220 2024-04-25T13:47:48.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b114bb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3b0c1622d0 0x7f3b0c0725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b114bb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3b0c1622d0 0x7f3b0c0725c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43808/0 (socket says 172.21.15.80:43808) 2024-04-25T13:47:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b114bb640 1 -- 172.21.15.80:0/1887002820 learned_addr learned my addr 172.21.15.80:0/1887002820 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b11cbc640 1 --2- 172.21.15.80:0/1887002820 >> v2:172.21.15.87:3300/0 conn(0x7f3b0c168220 0x7f3b0c168630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b114bb640 1 -- 172.21.15.80:0/1887002820 >> v2:172.21.15.87:3300/0 conn(0x7f3b0c168220 msgr2=0x7f3b0c168630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b114bb640 1 --2- 172.21.15.80:0/1887002820 >> v2:172.21.15.87:3300/0 conn(0x7f3b0c168220 0x7f3b0c168630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b114bb640 1 -- 172.21.15.80:0/1887002820 >> v1:172.21.15.87:6789/0 conn(0x7f3b0c06eb40 legacy=0x7f3b0c06ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:48.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.158+0000 7f3b114bb640 1 -- 172.21.15.80:0/1887002820 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c06e060 con 0x7f3b0c1622d0 2024-04-25T13:47:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b114bb640 1 --2- 172.21.15.80:0/1887002820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c1622d0 0x7f3b0c0725c0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c070db0 tx=0x7f3b0401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801079c973175e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b03fff640 1 -- 172.21.15.80:0/1887002820 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b040a5770 con 0x7f3b0c1622d0 2024-04-25T13:47:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b03fff640 1 -- 172.21.15.80:0/1887002820 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b040a5910 con 0x7f3b0c1622d0 2024-04-25T13:47:48.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b03fff640 1 -- 172.21.15.80:0/1887002820 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b040a5ce0 con 0x7f3b0c1622d0 2024-04-25T13:47:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 -- 172.21.15.80:0/1887002820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c1622d0 msgr2=0x7f3b0c0725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 --2- 172.21.15.80:0/1887002820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c1622d0 0x7f3b0c0725c0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c070db0 tx=0x7f3b0401c890 comp rx=0 tx=0).stop 2024-04-25T13:47:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 -- 172.21.15.80:0/1887002820 shutdown_connections 2024-04-25T13:47:48.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 --2- 172.21.15.80:0/1887002820 >> v2:172.21.15.87:3300/0 conn(0x7f3b0c168220 0x7f3b0c168630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 --2- 172.21.15.80:0/1887002820 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c1622d0 0x7f3b0c0725c0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 -- 172.21.15.80:0/1887002820 >> 172.21.15.80:0/1887002820 conn(0x7f3b0c06d080 msgr2=0x7f3b0c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 -- 172.21.15.80:0/1887002820 shutdown_connections 2024-04-25T13:47:48.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 -- 172.21.15.80:0/1887002820 wait complete. 2024-04-25T13:47:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 Processor -- start 2024-04-25T13:47:48.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.162+0000 7f3b1371f640 1 -- start start 2024-04-25T13:47:48.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c06eb40 0x7f3b0c163b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:48.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b0c1622d0 0x7f3b0c165cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:48.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0c168220 0x7f3b0c140820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:48.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c169100 con 0x7f3b0c06eb40 2024-04-25T13:47:48.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0c168f80 con 0x7f3b0c168220 2024-04-25T13:47:48.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b0c171bc0 con 0x7f3b0c1622d0 2024-04-25T13:47:48.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c06eb40 0x7f3b0c163b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c06eb40 0x7f3b0c163b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43818/0 (socket says 172.21.15.80:43818) 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 -- 172.21.15.80:0/548476684 learned_addr learned my addr 172.21.15.80:0/548476684 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b10cba640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b0c1622d0 0x7f3b0c165cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b11cbc640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0c168220 0x7f3b0c140820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 -- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b0c1622d0 msgr2=0x7f3b0c165cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b0c1622d0 0x7f3b0c165cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 -- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0c168220 msgr2=0x7f3b0c140820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0c168220 0x7f3b0c140820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 -- 172.21.15.80:0/548476684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b0c162ac0 con 0x7f3b0c06eb40 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b10cba640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b0c1622d0 0x7f3b0c165cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:48.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b114bb640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c06eb40 0x7f3b0c163b60 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3b040af040 tx=0x7f3b04019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:48.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b01ffb640 1 -- 172.21.15.80:0/548476684 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b040bc020 con 0x7f3b0c06eb40 2024-04-25T13:47:48.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b01ffb640 1 -- 172.21.15.80:0/548476684 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b040b86b0 con 0x7f3b0c06eb40 2024-04-25T13:47:48.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b0c162da0 con 0x7f3b0c06eb40 2024-04-25T13:47:48.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b01ffb640 1 -- 172.21.15.80:0/548476684 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b040b8aa0 con 0x7f3b0c06eb40 2024-04-25T13:47:48.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b0c163360 con 0x7f3b0c06eb40 2024-04-25T13:47:48.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b0c06a3a0 con 0x7f3b0c06eb40 2024-04-25T13:47:48.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.166+0000 7f3b01ffb640 1 -- 172.21.15.80:0/548476684 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b040bf020 con 0x7f3b0c06eb40 2024-04-25T13:47:48.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.170+0000 7f3b01ffb640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3af006f400 0x7f3af0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:48.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.170+0000 7f3b01ffb640 1 -- 172.21.15.80:0/548476684 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f3b04129e90 con 0x7f3b0c06eb40 2024-04-25T13:47:48.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.170+0000 7f3b10cba640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3af006f400 0x7f3af0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:48.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.170+0000 7f3b10cba640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3af006f400 0x7f3af0071910 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c201770 tx=0x7f3afc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:48.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.174+0000 7f3b01ffb640 1 -- 172.21.15.80:0/548476684 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b040c6020 con 0x7f3b0c06eb40 2024-04-25T13:47:48.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.362+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b0c076140 con 0x7f3b0c06eb40 2024-04-25T13:47:48.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b01ffb640 1 -- 172.21.15.80:0/548476684 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7f3b040165a0 con 0x7f3b0c06eb40 2024-04-25T13:47:48.363 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:48.364 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:48.364 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:48.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3af006f400 msgr2=0x7f3af0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:48.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3af006f400 0x7f3af0071910 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c201770 tx=0x7f3afc008040 comp rx=0 tx=0).stop 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c06eb40 msgr2=0x7f3b0c163b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c06eb40 0x7f3b0c163b60 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3b040af040 tx=0x7f3b04019630 comp rx=0 tx=0).stop 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 shutdown_connections 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3af006f400 0x7f3af0071910 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b0c168220 0x7f3b0c140820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b0c1622d0 0x7f3b0c165cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 --2- 172.21.15.80:0/548476684 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b0c06eb40 0x7f3b0c163b60 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 >> 172.21.15.80:0/548476684 conn(0x7f3b0c06d080 msgr2=0x7f3b0c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 shutdown_connections 2024-04-25T13:47:48.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:48.366+0000 7f3b1371f640 1 -- 172.21.15.80:0/548476684 wait complete. 2024-04-25T13:47:48.384 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:47:48.551 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":7968,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":37.816350251999999,"mdsmap_epoch":67,"osdmap_epoch":141,"osdmap_epoch_barrier":140,"uptime":42.799180776999997} 2024-04-25T13:47:48.551 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 7968, "mdsmap_epoch": 67, "osdmap_epoch": 141, "osdmap_epoch_barrier": 140, "rank_uptime": 37.816350252, "state": "up:active", "uptime": 42.799180777, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:47:48.551 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:47:48.551 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:48.552 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:48.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:48.558 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-04-25T13:47:49.005 INFO:teuthology.orchestra.run.smithi080.stderr:50+0 records in 2024-04-25T13:47:49.005 INFO:teuthology.orchestra.run.smithi080.stderr:50+0 records out 2024-04-25T13:47:49.006 INFO:teuthology.orchestra.run.smithi080.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.399693 s, 131 MB/s 2024-04-25T13:47:49.010 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:47:49.010 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:47:49.053 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:47:49.106 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:47:49.106 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:47:49.106 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:49.106 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:49.112 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:49.112 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:49.161 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:49.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facf4c30640 1 Processor -- start 2024-04-25T13:47:49.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facf4c30640 1 -- start start 2024-04-25T13:47:49.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facf4c30640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7facf00b7470 0x7facf00710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facf4c30640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf0161210 0x7facf00b5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facf4c30640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7facf0070030 con 0x7facf00d2200 2024-04-25T13:47:49.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facf4c30640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7facf0070310 con 0x7facf00b7470 2024-04-25T13:47:49.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facf4c30640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7facf00705f0 con 0x7facf0161210 2024-04-25T13:47:49.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facee59c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7facf00b7470 0x7facf00710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7faceed9d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf0161210 0x7facf00b5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facedd9b640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7facf00d2200 0x7facf0160b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:46858/0 (socket says 172.21.15.80:46858) 2024-04-25T13:47:49.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.358+0000 7facedd9b640 1 -- 172.21.15.80:0/1091461918 learned_addr learned my addr 172.21.15.80:0/1091461918 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:49.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7facee59c640 1 -- 172.21.15.80:0/1091461918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf0161210 msgr2=0x7facf00b5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7facee59c640 1 --2- 172.21.15.80:0/1091461918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf0161210 0x7facf00b5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7facee59c640 1 -- 172.21.15.80:0/1091461918 >> v1:172.21.15.87:6789/0 conn(0x7facf00d2200 legacy=0x7facf0160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7facee59c640 1 -- 172.21.15.80:0/1091461918 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf00708d0 con 0x7facf00b7470 2024-04-25T13:47:49.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7faceed9d640 1 --2- 172.21.15.80:0/1091461918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf0161210 0x7facf00b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7facee59c640 1 --2- 172.21.15.80:0/1091461918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf00b7470 0x7facf00710d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7face8009dc0 tx=0x7face801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4b5e3ded032c89c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7faced59a640 1 -- 172.21.15.80:0/1091461918 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80b2070 con 0x7facf00b7470 2024-04-25T13:47:49.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7faced59a640 1 -- 172.21.15.80:0/1091461918 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face80a5920 con 0x7facf00b7470 2024-04-25T13:47:49.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.362+0000 7faced59a640 1 -- 172.21.15.80:0/1091461918 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80a5cf0 con 0x7facf00b7470 2024-04-25T13:47:49.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 -- 172.21.15.80:0/1091461918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf00b7470 msgr2=0x7facf00710d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 --2- 172.21.15.80:0/1091461918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf00b7470 0x7facf00710d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7face8009dc0 tx=0x7face801c950 comp rx=0 tx=0).stop 2024-04-25T13:47:49.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 -- 172.21.15.80:0/1091461918 shutdown_connections 2024-04-25T13:47:49.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 --2- 172.21.15.80:0/1091461918 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf0161210 0x7facf00b5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 --2- 172.21.15.80:0/1091461918 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf00b7470 0x7facf00710d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 -- 172.21.15.80:0/1091461918 >> 172.21.15.80:0/1091461918 conn(0x7facf0073800 msgr2=0x7facf0073c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 -- 172.21.15.80:0/1091461918 shutdown_connections 2024-04-25T13:47:49.364 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 -- 172.21.15.80:0/1091461918 wait complete. 2024-04-25T13:47:49.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 Processor -- start 2024-04-25T13:47:49.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.366+0000 7facf4c30640 1 -- start start 2024-04-25T13:47:49.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facf00b7470 0x7facf014e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf00d2200 0x7facf015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf015ef30 0x7facf015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7facf0168220 con 0x7facf00b7470 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7facf01680a0 con 0x7facf015ef30 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7facf01683a0 con 0x7facf00d2200 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facf00b7470 0x7facf014e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facf00b7470 0x7facf014e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43826/0 (socket says 172.21.15.80:43826) 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 -- 172.21.15.80:0/3560411312 learned_addr learned my addr 172.21.15.80:0/3560411312 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7faceed9d640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf015ef30 0x7facf015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facedd9b640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf00d2200 0x7facf015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 -- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf00d2200 msgr2=0x7facf015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf00d2200 0x7facf015d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 -- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf015ef30 msgr2=0x7facf015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf015ef30 0x7facf015f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 -- 172.21.15.80:0/3560411312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf0160040 con 0x7facf00b7470 2024-04-25T13:47:49.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facee59c640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facf00b7470 0x7facf014e750 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7face80b0040 tx=0x7face80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7face6ffd640 1 -- 172.21.15.80:0/3560411312 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80b2050 con 0x7facf00b7470 2024-04-25T13:47:49.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7face6ffd640 1 -- 172.21.15.80:0/3560411312 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face80be070 con 0x7facf00b7470 2024-04-25T13:47:49.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf0160320 con 0x7facf00b7470 2024-04-25T13:47:49.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7face6ffd640 1 -- 172.21.15.80:0/3560411312 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face80b9850 con 0x7facf00b7470 2024-04-25T13:47:49.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf013ed30 con 0x7facf00b7470 2024-04-25T13:47:49.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf006a3a0 con 0x7facf00b7470 2024-04-25T13:47:49.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.370+0000 7face6ffd640 1 -- 172.21.15.80:0/3560411312 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7face80b99f0 con 0x7facf00b7470 2024-04-25T13:47:49.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.374+0000 7face6ffd640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7facd006f400 0x7facd0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.374+0000 7face6ffd640 1 -- 172.21.15.80:0/3560411312 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7face812a270 con 0x7facf00b7470 2024-04-25T13:47:49.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.374+0000 7facedd9b640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7facd006f400 0x7facd0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.374+0000 7facedd9b640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7facd006f400 0x7facd0071910 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7facd8007fd0 tx=0x7facd8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.378+0000 7face6ffd640 1 -- 172.21.15.80:0/3560411312 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7face8016540 con 0x7facf00b7470 2024-04-25T13:47:49.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.538+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7facf014f3f0 con 0x7facf00b7470 2024-04-25T13:47:49.541 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.542+0000 7face6ffd640 1 -- 172.21.15.80:0/3560411312 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16716 (secure 0 0 0) 0x7face80f7240 con 0x7facf00b7470 2024-04-25T13:47:49.542 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:49.542 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":141,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:46.223394+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:47:08.431048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T13:47:09.003237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:49.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7facd006f400 msgr2=0x7facd0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7facd006f400 0x7facd0071910 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7facd8007fd0 tx=0x7facd8009040 comp rx=0 tx=0).stop 2024-04-25T13:47:49.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facf00b7470 msgr2=0x7facf014e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facf00b7470 0x7facf014e750 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7face80b0040 tx=0x7face80196f0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 shutdown_connections 2024-04-25T13:47:49.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7facd006f400 0x7facd0071910 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7facf015ef30 0x7facf015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7facf00d2200 0x7facf015d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 --2- 172.21.15.80:0/3560411312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7facf00b7470 0x7facf014e750 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 >> 172.21.15.80:0/3560411312 conn(0x7facf0073800 msgr2=0x7facf0161100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 shutdown_connections 2024-04-25T13:47:49.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.546+0000 7facf4c30640 1 -- 172.21.15.80:0/3560411312 wait complete. 2024-04-25T13:47:49.563 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:47:08.431048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:47:09.003237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:47:49.564 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:47:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca2b77d640 1 Processor -- start 2024-04-25T13:47:49.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca2b77d640 1 -- start start 2024-04-25T13:47:49.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca2b77d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24161170 0x7fca241574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca2b77d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca24073c40 0x7fca240b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca2b77d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca2406fbf0 con 0x7fca24071e60 2024-04-25T13:47:49.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca2b77d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fca240b1350 con 0x7fca24073c40 2024-04-25T13:47:49.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca2b77d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fca2406f9a0 con 0x7fca24161170 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca29d1a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fca24073c40 0x7fca240b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca28d18640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24161170 0x7fca241574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca28d18640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24161170 0x7fca241574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:51724/0 (socket says 172.21.15.80:51724) 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca29519640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fca24071e60 0x7fca24160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:46864/0 (socket says 172.21.15.80:46864) 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.722+0000 7fca28d18640 1 -- 172.21.15.80:0/1435366002 learned_addr learned my addr 172.21.15.80:0/1435366002 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca28d18640 1 -- 172.21.15.80:0/1435366002 >> v2:172.21.15.87:3300/0 conn(0x7fca24073c40 msgr2=0x7fca240b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca28d18640 1 --2- 172.21.15.80:0/1435366002 >> v2:172.21.15.87:3300/0 conn(0x7fca24073c40 0x7fca240b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca28d18640 1 -- 172.21.15.80:0/1435366002 >> v1:172.21.15.87:6789/0 conn(0x7fca24071e60 legacy=0x7fca24160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca28d18640 1 -- 172.21.15.80:0/1435366002 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca240b0a20 con 0x7fca24161170 2024-04-25T13:47:49.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca29d1a640 1 --2- 172.21.15.80:0/1435366002 >> v2:172.21.15.87:3300/0 conn(0x7fca24073c40 0x7fca240b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:49.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca28d18640 1 --2- 172.21.15.80:0/1435366002 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24161170 0x7fca241574c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fca1400bb10 tx=0x7fca1401ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e42e3c479e3c4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca1bfff640 1 -- 172.21.15.80:0/1435366002 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca140b4070 con 0x7fca24161170 2024-04-25T13:47:49.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca1bfff640 1 -- 172.21.15.80:0/1435366002 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca140a7bf0 con 0x7fca24161170 2024-04-25T13:47:49.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca1bfff640 1 -- 172.21.15.80:0/1435366002 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca140af440 con 0x7fca24161170 2024-04-25T13:47:49.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca2b77d640 1 -- 172.21.15.80:0/1435366002 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24161170 msgr2=0x7fca241574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca2b77d640 1 --2- 172.21.15.80:0/1435366002 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24161170 0x7fca241574c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fca1400bb10 tx=0x7fca1401ec30 comp rx=0 tx=0).stop 2024-04-25T13:47:49.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca2b77d640 1 -- 172.21.15.80:0/1435366002 shutdown_connections 2024-04-25T13:47:49.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca2b77d640 1 --2- 172.21.15.80:0/1435366002 >> v2:172.21.15.87:3300/0 conn(0x7fca24073c40 0x7fca240b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca2b77d640 1 --2- 172.21.15.80:0/1435366002 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24161170 0x7fca241574c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca2b77d640 1 -- 172.21.15.80:0/1435366002 >> 172.21.15.80:0/1435366002 conn(0x7fca240b4160 msgr2=0x7fca240d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca2b77d640 1 -- 172.21.15.80:0/1435366002 shutdown_connections 2024-04-25T13:47:49.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.726+0000 7fca2b77d640 1 -- 172.21.15.80:0/1435366002 wait complete. 2024-04-25T13:47:49.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.730+0000 7fca2b77d640 1 Processor -- start 2024-04-25T13:47:49.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.730+0000 7fca2b77d640 1 -- start start 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.730+0000 7fca2b77d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca24071e60 0x7fca241552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.730+0000 7fca2b77d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24073c40 0x7fca24150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.730+0000 7fca2b77d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca240affb0 0x7fca240ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.730+0000 7fca2b77d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fca2406e470 con 0x7fca24071e60 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.730+0000 7fca2b77d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fca24074050 con 0x7fca240affb0 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.730+0000 7fca2b77d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fca2406e5f0 con 0x7fca24073c40 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca24071e60 0x7fca241552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca24071e60 0x7fca241552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43828/0 (socket says 172.21.15.80:43828) 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 -- 172.21.15.80:0/160904244 learned_addr learned my addr 172.21.15.80:0/160904244 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:49.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca28d18640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24073c40 0x7fca24150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29d1a640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca240affb0 0x7fca240ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 -- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24073c40 msgr2=0x7fca24150330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24073c40 0x7fca24150330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 -- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca240affb0 msgr2=0x7fca240ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca240affb0 0x7fca240ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 -- 172.21.15.80:0/160904244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca240734e0 con 0x7fca24071e60 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca29519640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca24071e60 0x7fca241552c0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c370 tx=0x7fca2000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca19ffb640 1 -- 172.21.15.80:0/160904244 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca200074e0 con 0x7fca24071e60 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca19ffb640 1 -- 172.21.15.80:0/160904244 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca20007680 con 0x7fca24071e60 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca24156bb0 con 0x7fca24071e60 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca19ffb640 1 -- 172.21.15.80:0/160904244 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca20004a30 con 0x7fca24071e60 2024-04-25T13:47:49.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca24155df0 con 0x7fca24071e60 2024-04-25T13:47:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca2406a3a0 con 0x7fca24071e60 2024-04-25T13:47:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca19ffb640 1 -- 172.21.15.80:0/160904244 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fca2001a020 con 0x7fca24071e60 2024-04-25T13:47:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca19ffb640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca0806f430 0x7fca08071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:49.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca19ffb640 1 -- 172.21.15.80:0/160904244 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7fca2008ea50 con 0x7fca24071e60 2024-04-25T13:47:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.734+0000 7fca28d18640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca0806f430 0x7fca08071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:49.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.738+0000 7fca28d18640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca0806f430 0x7fca08071940 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fca140b1000 tx=0x7fca140a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:49.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.738+0000 7fca19ffb640 1 -- 172.21.15.80:0/160904244 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fca2005b990 con 0x7fca24071e60 2024-04-25T13:47:49.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.902+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fca240b4ba0 con 0x7fca24071e60 2024-04-25T13:47:49.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.902+0000 7fca19ffb640 1 -- 172.21.15.80:0/160904244 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) ==== 93+0+10 (secure 0 0 0) 0x7fca2005b290 con 0x7fca24071e60 2024-04-25T13:47:49.901 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:47:49.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca0806f430 msgr2=0x7fca08071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca0806f430 0x7fca08071940 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fca140b1000 tx=0x7fca140a9070 comp rx=0 tx=0).stop 2024-04-25T13:47:49.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca24071e60 msgr2=0x7fca241552c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:49.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca24071e60 0x7fca241552c0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fca2000c370 tx=0x7fca2000c830 comp rx=0 tx=0).stop 2024-04-25T13:47:49.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 shutdown_connections 2024-04-25T13:47:49.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fca0806f430 0x7fca08071940 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fca240affb0 0x7fca240ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fca24073c40 0x7fca24150330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 --2- 172.21.15.80:0/160904244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fca24071e60 0x7fca241552c0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:49.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 >> 172.21.15.80:0/160904244 conn(0x7fca240b4160 msgr2=0x7fca24072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:49.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 shutdown_connections 2024-04-25T13:47:49.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:49.906+0000 7fca2b77d640 1 -- 172.21.15.80:0/160904244 wait complete. 2024-04-25T13:47:49.922 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:47:50.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f5cd3d640 1 Processor -- start 2024-04-25T13:47:50.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f5cd3d640 1 -- start start 2024-04-25T13:47:50.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f5cd3d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9f58161170 0x7f9f581574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f5cd3d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58073c40 0x7f9f580b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f5cd3d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f5806fbf0 con 0x7f9f58071e60 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f5cd3d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9f580b1350 con 0x7f9f58161170 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f5cd3d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9f5806f9a0 con 0x7f9f58073c40 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f4ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9f58161170 0x7f9f581574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f4ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9f58161170 0x7f9f581574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43830/0 (socket says 172.21.15.80:43830) 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f4ffff640 1 -- 172.21.15.80:0/1987921482 learned_addr learned my addr 172.21.15.80:0/1987921482 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f56d9d640 1 --2- 172.21.15.80:0/1987921482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58073c40 0x7f9f580b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f4ffff640 1 -- 172.21.15.80:0/1987921482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58073c40 msgr2=0x7f9f580b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f4ffff640 1 --2- 172.21.15.80:0/1987921482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58073c40 0x7f9f580b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f4ffff640 1 -- 172.21.15.80:0/1987921482 >> v1:172.21.15.87:6789/0 conn(0x7f9f58071e60 legacy=0x7f9f58160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.074+0000 7f9f4ffff640 1 -- 172.21.15.80:0/1987921482 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f580b0a20 con 0x7f9f58161170 2024-04-25T13:47:50.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f4ffff640 1 --2- 172.21.15.80:0/1987921482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f58161170 0x7f9f581574c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009870 tx=0x7f9f4001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8cd9e1b436b4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f55d9b640 1 -- 172.21.15.80:0/1987921482 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400ab650 con 0x7f9f58161170 2024-04-25T13:47:50.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f55d9b640 1 -- 172.21.15.80:0/1987921482 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f400b0070 con 0x7f9f58161170 2024-04-25T13:47:50.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f55d9b640 1 -- 172.21.15.80:0/1987921482 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f400aba10 con 0x7f9f58161170 2024-04-25T13:47:50.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/1987921482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f58161170 msgr2=0x7f9f581574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/1987921482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f58161170 0x7f9f581574c0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009870 tx=0x7f9f4001c560 comp rx=0 tx=0).stop 2024-04-25T13:47:50.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/1987921482 shutdown_connections 2024-04-25T13:47:50.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/1987921482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58073c40 0x7f9f580b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/1987921482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f58161170 0x7f9f581574c0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/1987921482 >> 172.21.15.80:0/1987921482 conn(0x7f9f580b4160 msgr2=0x7f9f580d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/1987921482 shutdown_connections 2024-04-25T13:47:50.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.078+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/1987921482 wait complete. 2024-04-25T13:47:50.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 Processor -- start 2024-04-25T13:47:50.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 -- start start 2024-04-25T13:47:50.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58071e60 0x7f9f581f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f58073c40 0x7f9f580abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f580ae560 0x7f9f580ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58169bb0 con 0x7f9f580ae560 2024-04-25T13:47:50.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9f58074050 con 0x7f9f58073c40 2024-04-25T13:47:50.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9f58169d30 con 0x7f9f58071e60 2024-04-25T13:47:50.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f580ae560 0x7f9f580ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f580ae560 0x7f9f580ae970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43844/0 (socket says 172.21.15.80:43844) 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 -- 172.21.15.80:0/2896922638 learned_addr learned my addr 172.21.15.80:0/2896922638 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5659c640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58071e60 0x7f9f581f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f4ffff640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f58073c40 0x7f9f580abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 -- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58071e60 msgr2=0x7f9f581f76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58071e60 0x7f9f581f76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 -- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f58073c40 msgr2=0x7f9f580abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f58073c40 0x7f9f580abb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 -- 172.21.15.80:0/2896922638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f581446f0 con 0x7f9f580ae560 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f4ffff640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f58073c40 0x7f9f580abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f56d9d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f580ae560 0x7f9f580ae970 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f9f50007d40 tx=0x7f9f5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f4effd640 1 -- 172.21.15.80:0/2896922638 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50014420 con 0x7f9f580ae560 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f4effd640 1 -- 172.21.15.80:0/2896922638 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f50019070 con 0x7f9f580ae560 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f580aeeb0 con 0x7f9f580ae560 2024-04-25T13:47:50.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f4effd640 1 -- 172.21.15.80:0/2896922638 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50014820 con 0x7f9f580ae560 2024-04-25T13:47:50.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.082+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f580aca30 con 0x7f9f580ae560 2024-04-25T13:47:50.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.086+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f580ac4a0 con 0x7f9f580ae560 2024-04-25T13:47:50.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.086+0000 7f9f4effd640 1 -- 172.21.15.80:0/2896922638 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9f500149c0 con 0x7f9f580ae560 2024-04-25T13:47:50.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.086+0000 7f9f4effd640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f3806f400 0x7f9f38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.086+0000 7f9f4effd640 1 -- 172.21.15.80:0/2896922638 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f9f5008fa90 con 0x7f9f580ae560 2024-04-25T13:47:50.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.086+0000 7f9f5659c640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f3806f400 0x7f9f38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.086+0000 7f9f5659c640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f3806f400 0x7f9f38071910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f480015b0 tx=0x7f9f48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.090+0000 7f9f4effd640 1 -- 172.21.15.80:0/2896922638 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9f5005c380 con 0x7f9f580ae560 2024-04-25T13:47:50.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.254+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f5806a3a0 con 0x7f9f580ae560 2024-04-25T13:47:50.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.254+0000 7f9f4effd640 1 -- 172.21.15.80:0/2896922638 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) ==== 104+0+11 (secure 0 0 0) 0x7f9f50060680 con 0x7f9f580ae560 2024-04-25T13:47:50.254 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:47:50.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f3806f400 msgr2=0x7f9f38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f3806f400 0x7f9f38071910 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9f480015b0 tx=0x7f9f48009040 comp rx=0 tx=0).stop 2024-04-25T13:47:50.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f580ae560 msgr2=0x7f9f580ae970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f580ae560 0x7f9f580ae970 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f9f50007d40 tx=0x7f9f5000e4b0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 shutdown_connections 2024-04-25T13:47:50.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9f3806f400 0x7f9f38071910 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9f580ae560 0x7f9f580ae970 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f58073c40 0x7f9f580abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 --2- 172.21.15.80:0/2896922638 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9f58071e60 0x7f9f581f76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 >> 172.21.15.80:0/2896922638 conn(0x7f9f580b4160 msgr2=0x7f9f5815d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 shutdown_connections 2024-04-25T13:47:50.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.258+0000 7f9f5cd3d640 1 -- 172.21.15.80:0/2896922638 wait complete. 2024-04-25T13:47:50.276 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:47:50.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.426+0000 7f2f2ab80640 1 Processor -- start 2024-04-25T13:47:50.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.426+0000 7f2f2ab80640 1 -- start start 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.426+0000 7f2f2ab80640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f24068590 0x7f2f240689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.426+0000 7f2f2ab80640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f24068ee0 con 0x7f2f240b0500 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.426+0000 7f2f2ab80640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2f240b5c10 con 0x7f2f2415a7a0 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.426+0000 7f2f2ab80640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f240b5ef0 con 0x7f2f24068590 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.426+0000 7f2f2911d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2f240b0500 0x7f2f240b0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45486/0 (socket says 172.21.15.80:45486) 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f2911d640 1 -- 172.21.15.80:0/1191834170 learned_addr learned my addr 172.21.15.80:0/1191834170 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f23fff640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f24068590 0x7f2f240689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3400207640 0 0) 0x7f2f140014a0 con 0x7f2f240b0500 2024-04-25T13:47:50.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2f10003530 con 0x7f2f240b0500 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3005716118 0 0) 0x7f2f10003530 con 0x7f2f240b0500 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f24068590 msgr2=0x7f2f240689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f24068590 0x7f2f240689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 >> v1:172.21.15.87:6789/0 conn(0x7f2f2415a7a0 legacy=0x7f2f240b7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f240b61d0 con 0x7f2f240b0500 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2f14001bd0 con 0x7f2f240b0500 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 >> v1:172.21.15.80:6789/0 conn(0x7f2f240b0500 legacy=0x7f2f240b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f24068590 0x7f2f100039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f10003f70 0x7f2f10004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f10004d50 0x7f2f10005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f240b5ef0 con 0x7f2f10004d50 2024-04-25T13:47:50.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1000bc10 con 0x7f2f10003f70 2024-04-25T13:47:50.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1000be10 con 0x7f2f24068590 2024-04-25T13:47:50.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f2911d640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f10004d50 0x7f2f10005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f23fff640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f10003f70 0x7f2f10004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f2911d640 1 -- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f24068590 msgr2=0x7f2f100039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f2911d640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f24068590 0x7f2f100039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f2911d640 1 -- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f10003f70 msgr2=0x7f2f10004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f2911d640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f10003f70 0x7f2f10004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f2911d640 1 -- 172.21.15.80:0/1191834170 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f10003530 con 0x7f2f10004d50 2024-04-25T13:47:50.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f2911d640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f10004d50 0x7f2f10005160 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400a9f0 tx=0x7f2f1401faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=571de0457867c48c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1400a640 con 0x7f2f10004d50 2024-04-25T13:47:50.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f140aa6f0 con 0x7f2f10004d50 2024-04-25T13:47:50.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.430+0000 7f2f237fe640 1 -- 172.21.15.80:0/1191834170 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f140aab20 con 0x7f2f10004d50 2024-04-25T13:47:50.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1191834170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f10004d50 msgr2=0x7f2f10005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 --2- 172.21.15.80:0/1191834170 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f10004d50 0x7f2f10005160 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400a9f0 tx=0x7f2f1401faa0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2911d640 1 -- 172.21.15.80:0/1191834170 reap_dead start 2024-04-25T13:47:50.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1191834170 shutdown_connections 2024-04-25T13:47:50.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1191834170 >> 172.21.15.80:0/1191834170 conn(0x7f2f24073790 msgr2=0x7f2f24073ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1191834170 shutdown_connections 2024-04-25T13:47:50.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1191834170 wait complete. 2024-04-25T13:47:50.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 Processor -- start 2024-04-25T13:47:50.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- start start 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f2415fe70 0x7f2f24163250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f2415c2a0 0x7f2f24150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f240705c0 0x7f2f240709d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2f2415abb0 con 0x7f2f2415c2a0 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2f240b69d0 con 0x7f2f240705c0 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f2ab80640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2f240689a0 con 0x7f2f2415fe70 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f23fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f2415c2a0 0x7f2f24150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f23fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f2415c2a0 0x7f2f24150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43868/0 (socket says 172.21.15.80:43868) 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.434+0000 7f2f23fff640 1 -- 172.21.15.80:0/1057137636 learned_addr learned my addr 172.21.15.80:0/1057137636 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:50.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f2911d640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f240705c0 0x7f2f240709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f2891c640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f2415fe70 0x7f2f24163250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f23fff640 1 -- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f2415fe70 msgr2=0x7f2f24163250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f23fff640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f2415fe70 0x7f2f24163250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f23fff640 1 -- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f240705c0 msgr2=0x7f2f240709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f23fff640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f240705c0 0x7f2f240709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f23fff640 1 -- 172.21.15.80:0/1057137636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f24070f10 con 0x7f2f2415c2a0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f23fff640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f2415c2a0 0x7f2f24150470 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c010890 tx=0x7f2f0c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f217fa640 1 -- 172.21.15.80:0/1057137636 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c00f040 con 0x7f2f2415c2a0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f217fa640 1 -- 172.21.15.80:0/1057137636 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f0c00eb60 con 0x7f2f2415c2a0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f241fa0d0 con 0x7f2f2415c2a0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f217fa640 1 -- 172.21.15.80:0/1057137636 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c01c400 con 0x7f2f2415c2a0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f241fa690 con 0x7f2f2415c2a0 2024-04-25T13:47:50.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f2406a3a0 con 0x7f2f2415c2a0 2024-04-25T13:47:50.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f217fa640 1 -- 172.21.15.80:0/1057137636 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2f0c01c5a0 con 0x7f2f2415c2a0 2024-04-25T13:47:50.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f217fa640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0406f400 0x7f2f04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f217fa640 1 -- 172.21.15.80:0/1057137636 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f2f0c08f7c0 con 0x7f2f2415c2a0 2024-04-25T13:47:50.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.438+0000 7f2f2891c640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0406f400 0x7f2f04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.442+0000 7f2f2891c640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0406f400 0x7f2f04071910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2f24162b00 tx=0x7f2f1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.442+0000 7f2f217fa640 1 -- 172.21.15.80:0/1057137636 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2f24070f10 con 0x7f2f2415c2a0 2024-04-25T13:47:50.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.606+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2f240b73d0 con 0x7f2f2415c2a0 2024-04-25T13:47:50.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.606+0000 7f2f217fa640 1 -- 172.21.15.80:0/1057137636 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) ==== 100+0+11 (secure 0 0 0) 0x7f2f0c018030 con 0x7f2f2415c2a0 2024-04-25T13:47:50.605 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:47:50.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0406f400 msgr2=0x7f2f04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0406f400 0x7f2f04071910 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2f24162b00 tx=0x7f2f1c009040 comp rx=0 tx=0).stop 2024-04-25T13:47:50.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f2415c2a0 msgr2=0x7f2f24150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f2415c2a0 0x7f2f24150470 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c010890 tx=0x7f2f0c010d50 comp rx=0 tx=0).stop 2024-04-25T13:47:50.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 shutdown_connections 2024-04-25T13:47:50.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2f0406f400 0x7f2f04071910 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2f240705c0 0x7f2f240709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2f2415c2a0 0x7f2f24150470 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 --2- 172.21.15.80:0/1057137636 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2f2415fe70 0x7f2f24163250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 >> 172.21.15.80:0/1057137636 conn(0x7f2f24073790 msgr2=0x7f2f24152770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 shutdown_connections 2024-04-25T13:47:50.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.610+0000 7f2f2ab80640 1 -- 172.21.15.80:0/1057137636 wait complete. 2024-04-25T13:47:50.626 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:47:50.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.778+0000 7f5915c12640 1 Processor -- start 2024-04-25T13:47:50.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.778+0000 7f5915c12640 1 -- start start 2024-04-25T13:47:50.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f5915c12640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f591006f5a0 0x7f5910160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f5915c12640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f591006fcf0 con 0x7f59100733b0 2024-04-25T13:47:50.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f5915c12640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59100b59f0 con 0x7f5910161070 2024-04-25T13:47:50.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f5915c12640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5910073f10 con 0x7f591006f5a0 2024-04-25T13:47:50.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f59100733b0 0x7f59100b52e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45488/0 (socket says 172.21.15.80:45488) 2024-04-25T13:47:50.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590ffff640 1 -- 172.21.15.80:0/4087182054 learned_addr learned my addr 172.21.15.80:0/4087182054 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590f7fe640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f591006f5a0 0x7f5910160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1531457740 0 0) 0x7f591006fcf0 con 0x7f59100733b0 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f58fc003530 con 0x7f59100733b0 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2795025394 0 0) 0x7f58fc003530 con 0x7f59100733b0 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f591006f5a0 msgr2=0x7f5910160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f591006f5a0 0x7f5910160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 >> v1:172.21.15.87:6789/0 conn(0x7f5910161070 legacy=0x7f59100b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59100b0c60 con 0x7f59100733b0 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f5908001a00 con 0x7f59100733b0 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 >> v1:172.21.15.80:6789/0 conn(0x7f59100733b0 legacy=0x7f59100b52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f591006f5a0 0x7f58fc0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58fc003f70 0x7f58fc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58fc004d50 0x7f58fc005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5910073f10 con 0x7f58fc004d50 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc00bc10 con 0x7f58fc003f70 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f58fc00be10 con 0x7f591006f5a0 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590ffff640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58fc004d50 0x7f58fc005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590f7fe640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f591006f5a0 0x7f58fc0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f5907fff640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58fc003f70 0x7f58fc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590ffff640 1 -- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f591006f5a0 msgr2=0x7f58fc0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590ffff640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f591006f5a0 0x7f58fc0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590ffff640 1 -- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58fc003f70 msgr2=0x7f58fc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590ffff640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f58fc003f70 0x7f58fc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.782+0000 7f590ffff640 1 -- 172.21.15.80:0/4087182054 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc003530 con 0x7f58fc004d50 2024-04-25T13:47:50.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f590ffff640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58fc004d50 0x7f58fc005160 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f59100701f0 tx=0x7f590801fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813311d77ae67d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590800a4c0 con 0x7f58fc004d50 2024-04-25T13:47:50.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59080aa700 con 0x7f58fc004d50 2024-04-25T13:47:50.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f590effd640 1 -- 172.21.15.80:0/4087182054 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59080aab30 con 0x7f58fc004d50 2024-04-25T13:47:50.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f5915c12640 1 -- 172.21.15.80:0/4087182054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58fc004d50 msgr2=0x7f58fc005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f5915c12640 1 --2- 172.21.15.80:0/4087182054 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58fc004d50 0x7f58fc005160 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f59100701f0 tx=0x7f590801fb40 comp rx=0 tx=0).stop 2024-04-25T13:47:50.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f590ffff640 1 -- 172.21.15.80:0/4087182054 reap_dead start 2024-04-25T13:47:50.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f5915c12640 1 -- 172.21.15.80:0/4087182054 shutdown_connections 2024-04-25T13:47:50.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f5915c12640 1 -- 172.21.15.80:0/4087182054 >> 172.21.15.80:0/4087182054 conn(0x7f591006d080 msgr2=0x7f59100b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f5915c12640 1 -- 172.21.15.80:0/4087182054 shutdown_connections 2024-04-25T13:47:50.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f5915c12640 1 -- 172.21.15.80:0/4087182054 wait complete. 2024-04-25T13:47:50.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f5915c12640 1 Processor -- start 2024-04-25T13:47:50.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.786+0000 7f5915c12640 1 -- start start 2024-04-25T13:47:50.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59101e7220 0x7f5910154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5910144170 0x7f5910153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5910153560 0x7f591006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f591006f9b0 con 0x7f59101e7220 2024-04-25T13:47:50.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f59100b1220 con 0x7f5910144170 2024-04-25T13:47:50.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5910160b30 con 0x7f5910153560 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59101e7220 0x7f5910154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59101e7220 0x7f5910154ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:43884/0 (socket says 172.21.15.80:43884) 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 -- 172.21.15.80:0/3779584978 learned_addr learned my addr 172.21.15.80:0/3779584978 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5907fff640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5910144170 0x7f5910153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590ffff640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5910153560 0x7f591006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 -- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5910153560 msgr2=0x7f591006e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5910153560 0x7f591006e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 -- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5910144170 msgr2=0x7f5910153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5910144170 0x7f5910153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 -- 172.21.15.80:0/3779584978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5910153a20 con 0x7f59101e7220 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590f7fe640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59101e7220 0x7f5910154ac0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5900011930 tx=0x7f5900011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590cff9640 1 -- 172.21.15.80:0/3779584978 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f590000faf0 con 0x7f59101e7220 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590cff9640 1 -- 172.21.15.80:0/3779584978 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f590000fc90 con 0x7f59101e7220 2024-04-25T13:47:50.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5910153d00 con 0x7f59101e7220 2024-04-25T13:47:50.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590cff9640 1 -- 172.21.15.80:0/3779584978 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5900014440 con 0x7f59101e7220 2024-04-25T13:47:50.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59101542c0 con 0x7f59101e7220 2024-04-25T13:47:50.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f591006a3a0 con 0x7f59101e7220 2024-04-25T13:47:50.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590cff9640 1 -- 172.21.15.80:0/3779584978 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5900012040 con 0x7f59101e7220 2024-04-25T13:47:50.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590cff9640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58f006f400 0x7f58f0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:50.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.790+0000 7f590cff9640 1 -- 172.21.15.80:0/3779584978 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f590008fbc0 con 0x7f59101e7220 2024-04-25T13:47:50.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.794+0000 7f5907fff640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58f006f400 0x7f58f0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:50.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.794+0000 7f5907fff640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58f006f400 0x7f58f0071910 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5910074270 tx=0x7f58f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:50.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.794+0000 7f590cff9640 1 -- 172.21.15.80:0/3779584978 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5900019030 con 0x7f59101e7220 2024-04-25T13:47:50.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.966+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f59100739c0 con 0x7f59101e7220 2024-04-25T13:47:50.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f590cff9640 1 -- 172.21.15.80:0/3779584978 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v141) ==== 80+0+0 (secure 0 0 0) 0x7f590005cbe0 con 0x7f59101e7220 2024-04-25T13:47:50.967 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:47:50.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58f006f400 msgr2=0x7f58f0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58f006f400 0x7f58f0071910 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5910074270 tx=0x7f58f8009040 comp rx=0 tx=0).stop 2024-04-25T13:47:50.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59101e7220 msgr2=0x7f5910154ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:50.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59101e7220 0x7f5910154ac0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f5900011930 tx=0x7f5900011df0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 shutdown_connections 2024-04-25T13:47:50.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f58f006f400 0x7f58f0071910 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5910153560 0x7f591006e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5910144170 0x7f5910153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 --2- 172.21.15.80:0/3779584978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59101e7220 0x7f5910154ac0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:50.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 >> 172.21.15.80:0/3779584978 conn(0x7f591006d080 msgr2=0x7f59100cfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:50.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 shutdown_connections 2024-04-25T13:47:50.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:50.970+0000 7f5915c12640 1 -- 172.21.15.80:0/3779584978 wait complete. 2024-04-25T13:47:50.988 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:47:50.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:50.988 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:47:50.999 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[38760]: fuse finished with error 0 and tester_r 0 2024-04-25T13:47:51.006 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:47:51.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:51.006 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:47:51.136 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:47:51.136 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:47:51.136 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:51.142 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:47:51.143 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:51.189 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:51.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.338+0000 7f0babab5640 1 Processor -- start 2024-04-25T13:47:51.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.338+0000 7f0babab5640 1 -- start start 2024-04-25T13:47:51.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0babab5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ba406f5a0 0x7f0ba4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0babab5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0ba4161070 0x7f0ba4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0babab5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ba406fcf0 con 0x7f0ba40b4bd0 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0babab5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba41619c0 con 0x7f0ba406f5a0 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0babab5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0ba4073f10 con 0x7f0ba4161070 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0ba9851640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ba406f5a0 0x7f0ba4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0ba9851640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ba406f5a0 0x7f0ba4160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59948/0 (socket says 172.21.15.80:59948) 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0ba9851640 1 -- 172.21.15.80:0/517509086 learned_addr learned my addr 172.21.15.80:0/517509086 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0ba9050640 1 --2- 172.21.15.80:0/517509086 >> v2:172.21.15.87:3300/0 conn(0x7f0ba4161070 0x7f0ba4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0ba9851640 1 -- 172.21.15.80:0/517509086 >> v2:172.21.15.87:3300/0 conn(0x7f0ba4161070 msgr2=0x7f0ba4161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0ba9851640 1 --2- 172.21.15.80:0/517509086 >> v2:172.21.15.87:3300/0 conn(0x7f0ba4161070 0x7f0ba4161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0ba9851640 1 -- 172.21.15.80:0/517509086 >> v1:172.21.15.87:6789/0 conn(0x7f0ba40b4bd0 legacy=0x7f0ba40b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.342+0000 7f0ba9851640 1 -- 172.21.15.80:0/517509086 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba40b0c60 con 0x7f0ba406f5a0 2024-04-25T13:47:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0ba9851640 1 --2- 172.21.15.80:0/517509086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba406f5a0 0x7f0ba4160b30 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f0ba406ff70 tx=0x7f0ba001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854b01eae9abe18d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0ba884f640 1 -- 172.21.15.80:0/517509086 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00b2070 con 0x7f0ba406f5a0 2024-04-25T13:47:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0ba884f640 1 -- 172.21.15.80:0/517509086 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ba00ad680 con 0x7f0ba406f5a0 2024-04-25T13:47:51.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0ba884f640 1 -- 172.21.15.80:0/517509086 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba00ada50 con 0x7f0ba406f5a0 2024-04-25T13:47:51.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 -- 172.21.15.80:0/517509086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba406f5a0 msgr2=0x7f0ba4160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 --2- 172.21.15.80:0/517509086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba406f5a0 0x7f0ba4160b30 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f0ba406ff70 tx=0x7f0ba001c490 comp rx=0 tx=0).stop 2024-04-25T13:47:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 -- 172.21.15.80:0/517509086 shutdown_connections 2024-04-25T13:47:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 --2- 172.21.15.80:0/517509086 >> v2:172.21.15.87:3300/0 conn(0x7f0ba4161070 0x7f0ba4161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 --2- 172.21.15.80:0/517509086 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba406f5a0 0x7f0ba4160b30 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 -- 172.21.15.80:0/517509086 >> 172.21.15.80:0/517509086 conn(0x7f0ba406d080 msgr2=0x7f0ba40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 -- 172.21.15.80:0/517509086 shutdown_connections 2024-04-25T13:47:51.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 -- 172.21.15.80:0/517509086 wait complete. 2024-04-25T13:47:51.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 Processor -- start 2024-04-25T13:47:51.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.346+0000 7f0babab5640 1 -- start start 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ba406f5a0 0x7f0ba4159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ba40b4bd0 0x7f0ba4140090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba4161070 0x7f0ba415b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba4168850 con 0x7f0ba4161070 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0ba41686d0 con 0x7f0ba40b4bd0 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0ba41689d0 con 0x7f0ba406f5a0 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba4161070 0x7f0ba415b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba4161070 0x7f0ba415b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59950/0 (socket says 172.21.15.80:59950) 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 -- 172.21.15.80:0/1858516672 learned_addr learned my addr 172.21.15.80:0/1858516672 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0ba9851640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ba406f5a0 0x7f0ba4159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0ba9050640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ba40b4bd0 0x7f0ba4140090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 -- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ba406f5a0 msgr2=0x7f0ba4159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ba406f5a0 0x7f0ba4159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 -- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ba40b4bd0 msgr2=0x7f0ba4140090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ba40b4bd0 0x7f0ba4140090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 -- 172.21.15.80:0/1858516672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ba415a200 con 0x7f0ba4161070 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0baa052640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba4161070 0x7f0ba415b5e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00c730 tx=0x7f0b9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0b9a7fc640 1 -- 172.21.15.80:0/1858516672 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c016020 con 0x7f0ba4161070 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0b9a7fc640 1 -- 172.21.15.80:0/1858516672 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b9c006d60 con 0x7f0ba4161070 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0b9a7fc640 1 -- 172.21.15.80:0/1858516672 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c003bc0 con 0x7f0ba4161070 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ba415a4e0 con 0x7f0ba4161070 2024-04-25T13:47:51.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ba415aaa0 con 0x7f0ba4161070 2024-04-25T13:47:51.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ba406a3a0 con 0x7f0ba4161070 2024-04-25T13:47:51.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.350+0000 7f0b9a7fc640 1 -- 172.21.15.80:0/1858516672 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0b9c0050d0 con 0x7f0ba4161070 2024-04-25T13:47:51.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.354+0000 7f0b9a7fc640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0b8806f400 0x7f0b88071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.354+0000 7f0b9a7fc640 1 -- 172.21.15.80:0/1858516672 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f0b9c08eeb0 con 0x7f0ba4161070 2024-04-25T13:47:51.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.354+0000 7f0ba9851640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0b8806f400 0x7f0b88071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.354+0000 7f0ba9851640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0b8806f400 0x7f0b88071910 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00af000 tx=0x7f0ba00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.358+0000 7f0b9a7fc640 1 -- 172.21.15.80:0/1858516672 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0b9c020020 con 0x7f0ba4161070 2024-04-25T13:47:51.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.518+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0ba40739c0 con 0x7f0ba4161070 2024-04-25T13:47:51.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.522+0000 7f0b9a7fc640 1 -- 172.21.15.80:0/1858516672 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16716 (secure 0 0 0) 0x7f0b9c05bdf0 con 0x7f0ba4161070 2024-04-25T13:47:51.521 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:51.521 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":141,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:46.223394+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:47:08.431048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T13:47:09.003237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:51.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0b8806f400 msgr2=0x7f0b88071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0b8806f400 0x7f0b88071910 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f0ba00af000 tx=0x7f0ba00a7000 comp rx=0 tx=0).stop 2024-04-25T13:47:51.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba4161070 msgr2=0x7f0ba415b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba4161070 0x7f0ba415b5e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c00c730 tx=0x7f0b9c00cbf0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 shutdown_connections 2024-04-25T13:47:51.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0b8806f400 0x7f0b88071910 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ba4161070 0x7f0ba415b5e0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ba40b4bd0 0x7f0ba4140090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 --2- 172.21.15.80:0/1858516672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ba406f5a0 0x7f0ba4159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 >> 172.21.15.80:0/1858516672 conn(0x7f0ba406d080 msgr2=0x7f0ba40b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 shutdown_connections 2024-04-25T13:47:51.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.526+0000 7f0babab5640 1 -- 172.21.15.80:0/1858516672 wait complete. 2024-04-25T13:47:51.545 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:47:08.431048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:47:09.003237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:47:51.545 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:47:51.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f307d637640 1 Processor -- start 2024-04-25T13:47:51.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f307d637640 1 -- start start 2024-04-25T13:47:51.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f307d637640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780d2430 0x7f30780cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f307d637640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f30780d2b60 con 0x7f30780b42f0 2024-04-25T13:47:51.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f307d637640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f307806cf90 con 0x7f30780d2020 2024-04-25T13:47:51.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f307d637640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f307806e230 con 0x7f30780d2430 2024-04-25T13:47:51.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f30767fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f30780b42f0 0x7f30780ce4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54052/0 (socket says 172.21.15.80:54052) 2024-04-25T13:47:51.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f30767fc640 1 -- 172.21.15.80:0/657626230 learned_addr learned my addr 172.21.15.80:0/657626230 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:51.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f30777fe640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780d2430 0x7f30780cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2465957677 0 0) 0x7f306c001120 con 0x7f30780b42f0 2024-04-25T13:47:51.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3060003530 con 0x7f30780b42f0 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 526402491 0 0) 0x7f3060003530 con 0x7f30780b42f0 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780d2430 msgr2=0x7f30780cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780d2430 0x7f30780cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 >> v1:172.21.15.87:6789/0 conn(0x7f30780d2020 legacy=0x7f30780b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307806dc70 con 0x7f30780b42f0 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f306c001850 con 0x7f30780b42f0 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 >> v1:172.21.15.80:6789/0 conn(0x7f30780b42f0 legacy=0x7f30780ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780d2430 0x7f30600039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f3060004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060004d50 0x7f3060005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f307806e230 con 0x7f3060003f50 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f306000bae0 con 0x7f3060004d50 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f306000bce0 con 0x7f30780d2430 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.698+0000 7f30767fc640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f3060004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f3076ffd640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780d2430 0x7f30600039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f30777fe640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060004d50 0x7f3060005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f30767fc640 1 -- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780d2430 msgr2=0x7f30600039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f30767fc640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780d2430 0x7f30600039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f30767fc640 1 -- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060004d50 msgr2=0x7f3060005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f30767fc640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3060004d50 0x7f3060005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f30767fc640 1 -- 172.21.15.80:0/657626230 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3060003530 con 0x7f3060003f50 2024-04-25T13:47:51.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f30767fc640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f3060004810 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f306c00c4c0 tx=0x7f306c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91bb0108d19848a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c00c0e0 con 0x7f3060003f50 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f306c0a9070 con 0x7f3060003f50 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f3075ffb640 1 -- 172.21.15.80:0/657626230 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306c0a5e00 con 0x7f3060003f50 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f307d637640 1 -- 172.21.15.80:0/657626230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 msgr2=0x7f3060004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f307d637640 1 --2- 172.21.15.80:0/657626230 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f3060004810 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f306c00c4c0 tx=0x7f306c01cc80 comp rx=0 tx=0).stop 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f30777fe640 1 -- 172.21.15.80:0/657626230 reap_dead start 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f307d637640 1 -- 172.21.15.80:0/657626230 shutdown_connections 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f307d637640 1 -- 172.21.15.80:0/657626230 >> 172.21.15.80:0/657626230 conn(0x7f30780d3ab0 msgr2=0x7f30780d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f307d637640 1 -- 172.21.15.80:0/657626230 shutdown_connections 2024-04-25T13:47:51.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.702+0000 7f307d637640 1 -- 172.21.15.80:0/657626230 wait complete. 2024-04-25T13:47:51.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 Processor -- start 2024-04-25T13:47:51.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 -- start start 2024-04-25T13:47:51.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f30781e5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780ca1c0 0x7f30781f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30781e5c90 0x7f30780cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f307806cd30 con 0x7f3060003f50 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f307806cbb0 con 0x7f30781e5c90 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f30780b4d90 con 0x7f30780ca1c0 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f30781e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f30781e5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59974/0 (socket says 172.21.15.80:59974) 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 -- 172.21.15.80:0/1390387653 learned_addr learned my addr 172.21.15.80:0/1390387653 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f30767fc640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780ca1c0 0x7f30781f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f30777fe640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30781e5c90 0x7f30780cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 -- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780ca1c0 msgr2=0x7f30781f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780ca1c0 0x7f30781f6980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 -- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30781e5c90 msgr2=0x7f30780cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30781e5c90 0x7f30780cd310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 -- 172.21.15.80:0/1390387653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30780cc270 con 0x7f3060003f50 2024-04-25T13:47:51.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f3076ffd640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f30781e5580 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3068010f70 tx=0x7f306800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f305f7fe640 1 -- 172.21.15.80:0/1390387653 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068018020 con 0x7f3060003f50 2024-04-25T13:47:51.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f305f7fe640 1 -- 172.21.15.80:0/1390387653 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3068005290 con 0x7f3060003f50 2024-04-25T13:47:51.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f305f7fe640 1 -- 172.21.15.80:0/1390387653 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3068003840 con 0x7f3060003f50 2024-04-25T13:47:51.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30780cc550 con 0x7f3060003f50 2024-04-25T13:47:51.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.706+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30780ccb10 con 0x7f3060003f50 2024-04-25T13:47:51.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.710+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f307806a3a0 con 0x7f3060003f50 2024-04-25T13:47:51.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.710+0000 7f305f7fe640 1 -- 172.21.15.80:0/1390387653 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f306801b020 con 0x7f3060003f50 2024-04-25T13:47:51.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.710+0000 7f305f7fe640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f306406f400 0x7f3064071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:51.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.710+0000 7f305f7fe640 1 -- 172.21.15.80:0/1390387653 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f3068092bb0 con 0x7f3060003f50 2024-04-25T13:47:51.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.710+0000 7f30767fc640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f306406f400 0x7f3064071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:51.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.710+0000 7f30767fc640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f306406f400 0x7f3064071910 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f306c0ab000 tx=0x7f306c0a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:51.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.714+0000 7f305f7fe640 1 -- 172.21.15.80:0/1390387653 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3068022020 con 0x7f3060003f50 2024-04-25T13:47:51.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30780b95e0 con 0x7f3060003f50 2024-04-25T13:47:51.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f305f7fe640 1 -- 172.21.15.80:0/1390387653 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) ==== 93+0+10 (secure 0 0 0) 0x7f306805faf0 con 0x7f3060003f50 2024-04-25T13:47:51.864 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f306406f400 msgr2=0x7f3064071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f306406f400 0x7f3064071910 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f306c0ab000 tx=0x7f306c0a7070 comp rx=0 tx=0).stop 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 msgr2=0x7f30781e5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f30781e5580 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3068010f70 tx=0x7f306800e530 comp rx=0 tx=0).stop 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 shutdown_connections 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f306406f400 0x7f3064071910 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30781e5c90 0x7f30780cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f30780ca1c0 0x7f30781f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 --2- 172.21.15.80:0/1390387653 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3060003f50 0x7f30781e5580 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:51.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 >> 172.21.15.80:0/1390387653 conn(0x7f30780d3ab0 msgr2=0x7f30780d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:51.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 shutdown_connections 2024-04-25T13:47:51.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:51.866+0000 7f307d637640 1 -- 172.21.15.80:0/1390387653 wait complete. 2024-04-25T13:47:51.885 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:47:52.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.038+0000 7f28c35fb640 1 Processor -- start 2024-04-25T13:47:52.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.038+0000 7f28c35fb640 1 -- start start 2024-04-25T13:47:52.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c35fb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f28bc073080 0x7f28bc073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c35fb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc06eb80 0x7f28bc0ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c35fb640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28bc0737f0 con 0x7f28bc15c710 2024-04-25T13:47:52.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c35fb640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f28bc0b53d0 con 0x7f28bc073080 2024-04-25T13:47:52.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c35fb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f28bc160b20 con 0x7f28bc06eb80 2024-04-25T13:47:52.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1397640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f28bc073080 0x7f28bc073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1397640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f28bc073080 0x7f28bc073ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59982/0 (socket says 172.21.15.80:59982) 2024-04-25T13:47:52.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1397640 1 -- 172.21.15.80:0/1452000559 learned_addr learned my addr 172.21.15.80:0/1452000559 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:52.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1b98640 1 --2- 172.21.15.80:0/1452000559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc06eb80 0x7f28bc0ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1397640 1 -- 172.21.15.80:0/1452000559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc06eb80 msgr2=0x7f28bc0ce630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:52.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1397640 1 --2- 172.21.15.80:0/1452000559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc06eb80 0x7f28bc0ce630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1397640 1 -- 172.21.15.80:0/1452000559 >> v1:172.21.15.87:6789/0 conn(0x7f28bc15c710 legacy=0x7f28bc06e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:52.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1397640 1 -- 172.21.15.80:0/1452000559 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc1604d0 con 0x7f28bc073080 2024-04-25T13:47:52.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.042+0000 7f28c1b98640 1 --2- 172.21.15.80:0/1452000559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc06eb80 0x7f28bc0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c1397640 1 --2- 172.21.15.80:0/1452000559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc073080 0x7f28bc073ad0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006fd0 tx=0x7f28b801e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1a370273badfc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28affff640 1 -- 172.21.15.80:0/1452000559 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b80ad070 con 0x7f28bc073080 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28affff640 1 -- 172.21.15.80:0/1452000559 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8004230 con 0x7f28bc073080 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28affff640 1 -- 172.21.15.80:0/1452000559 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b80a7640 con 0x7f28bc073080 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 -- 172.21.15.80:0/1452000559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc073080 msgr2=0x7f28bc073ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1452000559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc073080 0x7f28bc073ad0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f28b8006fd0 tx=0x7f28b801e920 comp rx=0 tx=0).stop 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 -- 172.21.15.80:0/1452000559 shutdown_connections 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1452000559 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc06eb80 0x7f28bc0ce630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1452000559 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc073080 0x7f28bc073ad0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 -- 172.21.15.80:0/1452000559 >> 172.21.15.80:0/1452000559 conn(0x7f28bc0b4160 msgr2=0x7f28bc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 -- 172.21.15.80:0/1452000559 shutdown_connections 2024-04-25T13:47:52.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 -- 172.21.15.80:0/1452000559 wait complete. 2024-04-25T13:47:52.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 Processor -- start 2024-04-25T13:47:52.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.046+0000 7f28c35fb640 1 -- start start 2024-04-25T13:47:52.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc073080 0x7f28bc1402c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc15c710 0x7f28bc15edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28bc15f2f0 0x7f28bc15cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc0cecf0 con 0x7f28bc15c710 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f28bc0ceb70 con 0x7f28bc15f2f0 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f28bc0cee70 con 0x7f28bc073080 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc15c710 0x7f28bc15edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc15c710 0x7f28bc15edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59988/0 (socket says 172.21.15.80:59988) 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 -- 172.21.15.80:0/1453699156 learned_addr learned my addr 172.21.15.80:0/1453699156 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c1397640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc073080 0x7f28bc1402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c1b98640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28bc15f2f0 0x7f28bc15cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 -- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc073080 msgr2=0x7f28bc1402c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc073080 0x7f28bc1402c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 -- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28bc15f2f0 msgr2=0x7f28bc15cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28bc15f2f0 0x7f28bc15cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 -- 172.21.15.80:0/1453699156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28bc150800 con 0x7f28bc15c710 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c0b96640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc15c710 0x7f28bc15edb0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f28b000b440 tx=0x7f28b000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28adffb640 1 -- 172.21.15.80:0/1453699156 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0015020 con 0x7f28bc15c710 2024-04-25T13:47:52.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28adffb640 1 -- 172.21.15.80:0/1453699156 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b0011890 con 0x7f28bc15c710 2024-04-25T13:47:52.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28adffb640 1 -- 172.21.15.80:0/1453699156 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0011c80 con 0x7f28bc15c710 2024-04-25T13:47:52.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28bc15fae0 con 0x7f28bc15c710 2024-04-25T13:47:52.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28bc150d80 con 0x7f28bc15c710 2024-04-25T13:47:52.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28bc15f850 con 0x7f28bc15c710 2024-04-25T13:47:52.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28adffb640 1 -- 172.21.15.80:0/1453699156 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f28b0018020 con 0x7f28bc15c710 2024-04-25T13:47:52.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28adffb640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f28a006f400 0x7f28a0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28adffb640 1 -- 172.21.15.80:0/1453699156 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f28b008f130 con 0x7f28bc15c710 2024-04-25T13:47:52.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.050+0000 7f28c1397640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f28a006f400 0x7f28a0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.054+0000 7f28c1397640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f28a006f400 0x7f28a0071910 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f28b80b1000 tx=0x7f28b80a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.058+0000 7f28adffb640 1 -- 172.21.15.80:0/1453699156 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f28b001f030 con 0x7f28bc15c710 2024-04-25T13:47:52.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.214+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f28bc06a3a0 con 0x7f28bc15c710 2024-04-25T13:47:52.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28adffb640 1 -- 172.21.15.80:0/1453699156 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) ==== 104+0+11 (secure 0 0 0) 0x7f28b000d040 con 0x7f28bc15c710 2024-04-25T13:47:52.215 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f28a006f400 msgr2=0x7f28a0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f28a006f400 0x7f28a0071910 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f28b80b1000 tx=0x7f28b80a9070 comp rx=0 tx=0).stop 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc15c710 msgr2=0x7f28bc15edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc15c710 0x7f28bc15edb0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f28b000b440 tx=0x7f28b000b900 comp rx=0 tx=0).stop 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 shutdown_connections 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f28a006f400 0x7f28a0071910 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28bc15f2f0 0x7f28bc15cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f28bc15c710 0x7f28bc15edb0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 --2- 172.21.15.80:0/1453699156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f28bc073080 0x7f28bc1402c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 >> 172.21.15.80:0/1453699156 conn(0x7f28bc0b4160 msgr2=0x7f28bc0d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:52.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 shutdown_connections 2024-04-25T13:47:52.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.218+0000 7f28c35fb640 1 -- 172.21.15.80:0/1453699156 wait complete. 2024-04-25T13:47:52.238 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:47:52.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e2d6d640 1 Processor -- start 2024-04-25T13:47:52.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e2d6d640 1 -- start start 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e2d6d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff9dc161170 0x7ff9dc1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e2d6d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc073c40 0x7ff9dc0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e2d6d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff9dc06fbf0 con 0x7ff9dc071e60 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e2d6d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff9dc0b1350 con 0x7ff9dc161170 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e2d6d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc06f9a0 con 0x7ff9dc073c40 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e0b09640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff9dc071e60 0x7ff9dc160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54062/0 (socket says 172.21.15.80:54062) 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e0b09640 1 -- 172.21.15.80:0/569523098 learned_addr learned my addr 172.21.15.80:0/569523098 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3019783175 0 0) 0x7ff9dc06fbf0 con 0x7ff9dc071e60 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9e130a640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc073c40 0x7ff9dc0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9dbfff640 1 --2- 172.21.15.80:0/569523098 >> v2:172.21.15.87:3300/0 conn(0x7ff9dc161170 0x7ff9dc1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.386+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff9c4005740 con 0x7ff9dc071e60 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2421217097 0 0) 0x7ff9c4005740 con 0x7ff9dc071e60 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc073c40 msgr2=0x7ff9dc0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc073c40 0x7ff9dc0b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 >> v2:172.21.15.87:3300/0 conn(0x7ff9dc161170 msgr2=0x7ff9dc1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 --2- 172.21.15.80:0/569523098 >> v2:172.21.15.87:3300/0 conn(0x7ff9dc161170 0x7ff9dc1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0b0a20 con 0x7ff9dc071e60 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff9cc001680 con 0x7ff9dc071e60 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 >> v1:172.21.15.80:6789/0 conn(0x7ff9dc071e60 legacy=0x7ff9dc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c4005bd0 0x7ff9c4006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc073c40 0x7ff9c4006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c4006f60 0x7ff9c4007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0b1350 con 0x7ff9c4006f60 2024-04-25T13:47:52.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc06f9a0 con 0x7ff9c4005bd0 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c400dd40 con 0x7ff9dc073c40 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e130a640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c4006f60 0x7ff9c4007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e0b09640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c4005bd0 0x7ff9c4006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9dbfff640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc073c40 0x7ff9c4006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e130a640 1 -- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc073c40 msgr2=0x7ff9c4006a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e130a640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc073c40 0x7ff9c4006a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e130a640 1 -- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c4005bd0 msgr2=0x7ff9c4006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e130a640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9c4005bd0 0x7ff9c4006040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e130a640 1 -- 172.21.15.80:0/569523098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c4005740 con 0x7ff9c4006f60 2024-04-25T13:47:52.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e130a640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c4006f60 0x7ff9c4007370 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0d21b0 tx=0x7ff9d401ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e44522e5605e57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40b2070 con 0x7ff9c4006f60 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40a5d00 con 0x7ff9c4006f60 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9db7fe640 1 -- 172.21.15.80:0/569523098 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4009be0 con 0x7ff9c4006f60 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/569523098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c4006f60 msgr2=0x7ff9c4007370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e2d6d640 1 --2- 172.21.15.80:0/569523098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9c4006f60 0x7ff9c4007370 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0d21b0 tx=0x7ff9d401ce00 comp rx=0 tx=0).stop 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e130a640 1 -- 172.21.15.80:0/569523098 reap_dead start 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/569523098 shutdown_connections 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/569523098 >> 172.21.15.80:0/569523098 conn(0x7ff9dc0b4160 msgr2=0x7ff9dc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/569523098 shutdown_connections 2024-04-25T13:47:52.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.390+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/569523098 wait complete. 2024-04-25T13:47:52.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 Processor -- start 2024-04-25T13:47:52.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 -- start start 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9dc148720 0x7ff9dc147dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc0cffd0 0x7ff9dc0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9dc1f3470 0x7ff9dc1f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc072270 con 0x7ff9dc1f3470 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0b0fe0 con 0x7ff9dc148720 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc074050 con 0x7ff9dc0cffd0 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9dc1f3470 0x7ff9dc1f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9dc1f3470 0x7ff9dc1f3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:59994/0 (socket says 172.21.15.80:59994) 2024-04-25T13:47:52.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 -- 172.21.15.80:0/2445842786 learned_addr learned my addr 172.21.15.80:0/2445842786 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9dbfff640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc0cffd0 0x7ff9dc0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e0b09640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9dc148720 0x7ff9dc147dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 -- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc0cffd0 msgr2=0x7ff9dc0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc0cffd0 0x7ff9dc0d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 -- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9dc148720 msgr2=0x7ff9dc147dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9dc148720 0x7ff9dc147dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 -- 172.21.15.80:0/2445842786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0cef70 con 0x7ff9dc1f3470 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e130a640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9dc1f3470 0x7ff9dc1f3880 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40b0040 tx=0x7ff9c4007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9d97fa640 1 -- 172.21.15.80:0/2445842786 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40bb070 con 0x7ff9dc1f3470 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9d97fa640 1 -- 172.21.15.80:0/2445842786 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9d40a5b00 con 0x7ff9dc1f3470 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc0cf210 con 0x7ff9dc1f3470 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9d97fa640 1 -- 172.21.15.80:0/2445842786 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40b2040 con 0x7ff9dc1f3470 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.394+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc0cf7d0 con 0x7ff9dc1f3470 2024-04-25T13:47:52.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.398+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc06a3a0 con 0x7ff9dc1f3470 2024-04-25T13:47:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.398+0000 7ff9d97fa640 1 -- 172.21.15.80:0/2445842786 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff9d40be070 con 0x7ff9dc1f3470 2024-04-25T13:47:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.398+0000 7ff9d97fa640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c806f400 0x7ff9c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.398+0000 7ff9d97fa640 1 -- 172.21.15.80:0/2445842786 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7ff9d4129600 con 0x7ff9dc1f3470 2024-04-25T13:47:52.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.398+0000 7ff9e0b09640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c806f400 0x7ff9c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.398+0000 7ff9e0b09640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c806f400 0x7ff9c8071910 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc154a30 tx=0x7ff9cc004810 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.402+0000 7ff9d97fa640 1 -- 172.21.15.80:0/2445842786 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff9d40c5020 con 0x7ff9dc1f3470 2024-04-25T13:47:52.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.558+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff9dc06f240 con 0x7ff9dc1f3470 2024-04-25T13:47:52.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.558+0000 7ff9d97fa640 1 -- 172.21.15.80:0/2445842786 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) ==== 100+0+11 (secure 0 0 0) 0x7ff9d4016b10 con 0x7ff9dc1f3470 2024-04-25T13:47:52.557 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c806f400 msgr2=0x7ff9c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c806f400 0x7ff9c8071910 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc154a30 tx=0x7ff9cc004810 comp rx=0 tx=0).stop 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9dc1f3470 msgr2=0x7ff9dc1f3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9dc1f3470 0x7ff9dc1f3880 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40b0040 tx=0x7ff9c4007a50 comp rx=0 tx=0).stop 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 shutdown_connections 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff9c806f400 0x7ff9c8071910 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff9dc1f3470 0x7ff9dc1f3880 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff9dc0cffd0 0x7ff9dc0d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 --2- 172.21.15.80:0/2445842786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9dc148720 0x7ff9dc147dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 >> 172.21.15.80:0/2445842786 conn(0x7ff9dc0b4160 msgr2=0x7ff9dc156320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 shutdown_connections 2024-04-25T13:47:52.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.562+0000 7ff9e2d6d640 1 -- 172.21.15.80:0/2445842786 wait complete. 2024-04-25T13:47:52.579 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:47:52.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe05e8d4640 1 Processor -- start 2024-04-25T13:47:52.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe05e8d4640 1 -- start start 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe05e8d4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 0x7fe0581603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe05e8d4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0580b93e0 0x7fe0580d2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe05e8d4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe058164270 con 0x7fe0580d2940 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe05e8d4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe058164e80 con 0x7fe0580b93e0 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe05e8d4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe058164550 con 0x7fe0580b4950 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe0577fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0580b93e0 0x7fe0580d2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe0577fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe0580b93e0 0x7fe0580d2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60000/0 (socket says 172.21.15.80:60000) 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe0577fe640 1 -- 172.21.15.80:0/1931825080 learned_addr learned my addr 172.21.15.80:0/1931825080 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:52.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe0577fe640 1 -- 172.21.15.80:0/1931825080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 msgr2=0x7fe0581603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:52.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe0577fe640 1 --2- 172.21.15.80:0/1931825080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 0x7fe0581603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe0577fe640 1 -- 172.21.15.80:0/1931825080 >> v1:172.21.15.87:6789/0 conn(0x7fe0580d2940 legacy=0x7fe0580d4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:52.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.718+0000 7fe0577fe640 1 -- 172.21.15.80:0/1931825080 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe058164830 con 0x7fe0580b93e0 2024-04-25T13:47:52.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe0577fe640 1 --2- 172.21.15.80:0/1931825080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580b93e0 0x7fe0580d2400 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009870 tx=0x7fe04c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e85f1d13d7fe8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe056ffd640 1 -- 172.21.15.80:0/1931825080 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0b1070 con 0x7fe0580b93e0 2024-04-25T13:47:52.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe056ffd640 1 -- 172.21.15.80:0/1931825080 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe04c0ac410 con 0x7fe0580b93e0 2024-04-25T13:47:52.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe056ffd640 1 -- 172.21.15.80:0/1931825080 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c0ac7e0 con 0x7fe0580b93e0 2024-04-25T13:47:52.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe05e8d4640 1 -- 172.21.15.80:0/1931825080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580b93e0 msgr2=0x7fe0580d2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/1931825080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580b93e0 0x7fe0580d2400 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fe04c009870 tx=0x7fe04c01c520 comp rx=0 tx=0).stop 2024-04-25T13:47:52.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe05e8d4640 1 -- 172.21.15.80:0/1931825080 shutdown_connections 2024-04-25T13:47:52.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/1931825080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580b93e0 0x7fe0580d2400 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/1931825080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 0x7fe0581603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe05e8d4640 1 -- 172.21.15.80:0/1931825080 >> 172.21.15.80:0/1931825080 conn(0x7fe05806c5f0 msgr2=0x7fe05806ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:52.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe05e8d4640 1 -- 172.21.15.80:0/1931825080 shutdown_connections 2024-04-25T13:47:52.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.722+0000 7fe05e8d4640 1 -- 172.21.15.80:0/1931825080 wait complete. 2024-04-25T13:47:52.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 Processor -- start 2024-04-25T13:47:52.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 -- start start 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 0x7fe058153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580b93e0 0x7fe05815f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d2940 0x7fe0580cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe05816d2a0 con 0x7fe0580d2940 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe058163f00 con 0x7fe0580b93e0 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe05816d420 con 0x7fe0580b4950 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d2940 0x7fe0580cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d2940 0x7fe0580cf560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60012/0 (socket says 172.21.15.80:60012) 2024-04-25T13:47:52.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 -- 172.21.15.80:0/3096226073 learned_addr learned my addr 172.21.15.80:0/3096226073 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe0577fe640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580b93e0 0x7fe05815f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe057fff640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 0x7fe058153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 -- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 msgr2=0x7fe058153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 0x7fe058153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 -- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580b93e0 msgr2=0x7fe05815f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580b93e0 0x7fe05815f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 -- 172.21.15.80:0/3096226073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe058153d30 con 0x7fe0580d2940 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe0577fe640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580b93e0 0x7fe05815f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05ce71640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d2940 0x7fe0580cf560 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe05000b9e0 tx=0x7fe05000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe054ff9640 1 -- 172.21.15.80:0/3096226073 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050019070 con 0x7fe0580d2940 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe054ff9640 1 -- 172.21.15.80:0/3096226073 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe05000cb20 con 0x7fe0580d2940 2024-04-25T13:47:52.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe054ff9640 1 -- 172.21.15.80:0/3096226073 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe050014440 con 0x7fe0580d2940 2024-04-25T13:47:52.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe058154010 con 0x7fe0580d2940 2024-04-25T13:47:52.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.726+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0581545d0 con 0x7fe0580d2940 2024-04-25T13:47:52.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.730+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05806a3a0 con 0x7fe0580d2940 2024-04-25T13:47:52.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.730+0000 7fe054ff9640 1 -- 172.21.15.80:0/3096226073 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe05001c070 con 0x7fe0580d2940 2024-04-25T13:47:52.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.730+0000 7fe054ff9640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f400 0x7fe044071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:52.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.730+0000 7fe054ff9640 1 -- 172.21.15.80:0/3096226073 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7fe05008f2b0 con 0x7fe0580d2940 2024-04-25T13:47:52.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.730+0000 7fe057fff640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f400 0x7fe044071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:52.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.730+0000 7fe057fff640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f400 0x7fe044071910 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe04800a790 tx=0x7fe048008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:52.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.734+0000 7fe054ff9640 1 -- 172.21.15.80:0/3096226073 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe050023020 con 0x7fe0580d2940 2024-04-25T13:47:52.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.890+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe0580d42c0 con 0x7fe0580d2940 2024-04-25T13:47:52.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe054ff9640 1 -- 172.21.15.80:0/3096226073 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v141) ==== 80+0+0 (secure 0 0 0) 0x7fe05005c360 con 0x7fe0580d2940 2024-04-25T13:47:52.892 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:47:52.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f400 msgr2=0x7fe044071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f400 0x7fe044071910 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe04800a790 tx=0x7fe048008040 comp rx=0 tx=0).stop 2024-04-25T13:47:52.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d2940 msgr2=0x7fe0580cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:52.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d2940 0x7fe0580cf560 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fe05000b9e0 tx=0x7fe05000bea0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 shutdown_connections 2024-04-25T13:47:52.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe04406f400 0x7fe044071910 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe0580d2940 0x7fe0580cf560 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe0580b93e0 0x7fe05815f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 --2- 172.21.15.80:0/3096226073 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe0580b4950 0x7fe058153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:52.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 >> 172.21.15.80:0/3096226073 conn(0x7fe05806c5f0 msgr2=0x7fe0580d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:52.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 shutdown_connections 2024-04-25T13:47:52.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:52.894+0000 7fe05e8d4640 1 -- 172.21.15.80:0/3096226073 wait complete. 2024-04-25T13:47:52.912 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:47:52.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:52.912 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:47:52.925 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25251]: fuse finished with error 0 and tester_r 0 2024-04-25T13:47:52.927 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:47:52.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:47:52.927 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:47:53.100 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:47:53.101 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:47:53.101 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:53.107 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:47:53.107 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:47:53.107 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:47:53.107 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:47:53.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:47:53.149 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:53.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3877789640 1 Processor -- start 2024-04-25T13:47:53.303 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3877789640 1 -- start start 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3877789640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38700ce730 0x7f38700ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3877789640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3877789640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f387006ede0 con 0x7f38700b0ed0 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3877789640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3870073710 con 0x7f38700ce730 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3877789640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f38700739f0 con 0x7f387006e490 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3875525640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f38700b0ed0 0x7f387015c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54074/0 (socket says 172.21.15.80:54074) 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3875525640 1 -- 172.21.15.80:0/858176102 learned_addr learned my addr 172.21.15.80:0/858176102 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3874d24640 1 --2- 172.21.15.80:0/858176102 >> v2:172.21.15.80:3300/0 conn(0x7f38700ce730 0x7f38700ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3875d26640 1 --2- 172.21.15.80:0/858176102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3867fff640 1 -- 172.21.15.80:0/858176102 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3984550537 0 0) 0x7f386c001120 con 0x7f38700b0ed0 2024-04-25T13:47:53.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3867fff640 1 -- 172.21.15.80:0/858176102 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f385c003530 con 0x7f38700b0ed0 2024-04-25T13:47:53.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3874d24640 1 -- 172.21.15.80:0/858176102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 msgr2=0x7f387006e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:53.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3874d24640 1 --2- 172.21.15.80:0/858176102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387006e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3874d24640 1 -- 172.21.15.80:0/858176102 >> v1:172.21.15.80:6789/0 conn(0x7f38700b0ed0 legacy=0x7f387015c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.306+0000 7f3874d24640 1 -- 172.21.15.80:0/858176102 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3870073cd0 con 0x7f38700ce730 2024-04-25T13:47:53.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3874d24640 1 --2- 172.21.15.80:0/858176102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700ce730 0x7f38700ceb40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f38600097c0 tx=0x7f386001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e288bed475315fc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:53.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3867fff640 1 -- 172.21.15.80:0/858176102 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600b5070 con 0x7f38700ce730 2024-04-25T13:47:53.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3867fff640 1 -- 172.21.15.80:0/858176102 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38600a8990 con 0x7f38700ce730 2024-04-25T13:47:53.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3867fff640 1 -- 172.21.15.80:0/858176102 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600a8d60 con 0x7f38700ce730 2024-04-25T13:47:53.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3877789640 1 -- 172.21.15.80:0/858176102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700ce730 msgr2=0x7f38700ceb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3877789640 1 --2- 172.21.15.80:0/858176102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700ce730 0x7f38700ceb40 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f38600097c0 tx=0x7f386001ec90 comp rx=0 tx=0).stop 2024-04-25T13:47:53.309 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3877789640 1 -- 172.21.15.80:0/858176102 shutdown_connections 2024-04-25T13:47:53.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3877789640 1 --2- 172.21.15.80:0/858176102 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387006e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3877789640 1 --2- 172.21.15.80:0/858176102 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700ce730 0x7f38700ceb40 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3877789640 1 -- 172.21.15.80:0/858176102 >> 172.21.15.80:0/858176102 conn(0x7f387006d080 msgr2=0x7f38700b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:53.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3877789640 1 -- 172.21.15.80:0/858176102 shutdown_connections 2024-04-25T13:47:53.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.310+0000 7f3877789640 1 -- 172.21.15.80:0/858176102 wait complete. 2024-04-25T13:47:53.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 Processor -- start 2024-04-25T13:47:53.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 -- start start 2024-04-25T13:47:53.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387013d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700b0ed0 0x7f387015da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38700ce730 0x7f38701677d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f38700733a0 con 0x7f38700b0ed0 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f38700b12e0 con 0x7f38700ce730 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f387006dc00 con 0x7f387006e490 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700b0ed0 0x7f387015da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700b0ed0 0x7f387015da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60030/0 (socket says 172.21.15.80:60030) 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 -- 172.21.15.80:0/2891185775 learned_addr learned my addr 172.21.15.80:0/2891185775 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 -- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 msgr2=0x7f387013d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3875525640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387013d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3875d26640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38700ce730 0x7f38701677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387013d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 -- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38700ce730 msgr2=0x7f38701677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38700ce730 0x7f38701677d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 -- 172.21.15.80:0/2891185775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3870154970 con 0x7f38700b0ed0 2024-04-25T13:47:53.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3875525640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387013d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:53.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3874d24640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700b0ed0 0x7f387015da40 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f38600b3040 tx=0x7f386001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:53.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3865ffb640 1 -- 172.21.15.80:0/2891185775 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600bf070 con 0x7f38700b0ed0 2024-04-25T13:47:53.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3865ffb640 1 -- 172.21.15.80:0/2891185775 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38600b0660 con 0x7f38700b0ed0 2024-04-25T13:47:53.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3865ffb640 1 -- 172.21.15.80:0/2891185775 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600b5040 con 0x7f38700b0ed0 2024-04-25T13:47:53.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3870154c50 con 0x7f38700b0ed0 2024-04-25T13:47:53.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.314+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3870155210 con 0x7f38700b0ed0 2024-04-25T13:47:53.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.318+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f387006a3a0 con 0x7f38700b0ed0 2024-04-25T13:47:53.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.318+0000 7f3865ffb640 1 -- 172.21.15.80:0/2891185775 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f38600b0a50 con 0x7f38700b0ed0 2024-04-25T13:47:53.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.318+0000 7f3865ffb640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f385406f430 0x7f3854071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.318+0000 7f3865ffb640 1 -- 172.21.15.80:0/2891185775 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f386012ba20 con 0x7f38700b0ed0 2024-04-25T13:47:53.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.318+0000 7f3875525640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f385406f430 0x7f3854071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.318+0000 7f3875525640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f385406f430 0x7f3854071940 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f386c0012c0 tx=0x7f386c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:53.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.322+0000 7f3865ffb640 1 -- 172.21.15.80:0/2891185775 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f38600187f0 con 0x7f38700b0ed0 2024-04-25T13:47:53.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.502+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f387006f880 con 0x7f38700b0ed0 2024-04-25T13:47:53.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3865ffb640 1 -- 172.21.15.80:0/2891185775 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7f38600f8960 con 0x7f38700b0ed0 2024-04-25T13:47:53.503 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:53.503 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:53.503 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:53.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f385406f430 msgr2=0x7f3854071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f385406f430 0x7f3854071940 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f386c0012c0 tx=0x7f386c009040 comp rx=0 tx=0).stop 2024-04-25T13:47:53.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700b0ed0 msgr2=0x7f387015da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700b0ed0 0x7f387015da40 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f38600b3040 tx=0x7f386001ba30 comp rx=0 tx=0).stop 2024-04-25T13:47:53.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 shutdown_connections 2024-04-25T13:47:53.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f385406f430 0x7f3854071940 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f38700ce730 0x7f38701677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38700b0ed0 0x7f387015da40 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 --2- 172.21.15.80:0/2891185775 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f387006e490 0x7f387013d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 >> 172.21.15.80:0/2891185775 conn(0x7f387006d080 msgr2=0x7f38700d0dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:53.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 shutdown_connections 2024-04-25T13:47:53.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.506+0000 7f3877789640 1 -- 172.21.15.80:0/2891185775 wait complete. 2024-04-25T13:47:53.524 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:53.670 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.670+0000 7f79d89d7640 1 Processor -- start 2024-04-25T13:47:53.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.670+0000 7f79d89d7640 1 -- start start 2024-04-25T13:47:53.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d89d7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d406eba0 0x7f79d40ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d89d7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79d4073710 con 0x7f79d40ce730 2024-04-25T13:47:53.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d89d7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f79d40739f0 con 0x7f79d40b0ed0 2024-04-25T13:47:53.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d89d7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f79d4073cd0 con 0x7f79d406eba0 2024-04-25T13:47:53.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d1d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f79d40ce730 0x7f79d406e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54080/0 (socket says 172.21.15.80:54080) 2024-04-25T13:47:53.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d1d9b640 1 -- 172.21.15.80:0/1948489575 learned_addr learned my addr 172.21.15.80:0/1948489575 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:53.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388791658 0 0) 0x7f79d4073710 con 0x7f79d40ce730 2024-04-25T13:47:53.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d2d9d640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d406eba0 0x7f79d40ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f79c0003450 con 0x7f79d40ce730 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 50445027 0 0) 0x7f79c0003450 con 0x7f79d40ce730 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d406eba0 msgr2=0x7f79d40ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d406eba0 0x7f79d40ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 >> v1:172.21.15.87:6789/0 conn(0x7f79d40b0ed0 legacy=0x7f79d415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d4073fb0 con 0x7f79d40ce730 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f79bc001680 con 0x7f79d40ce730 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 >> v1:172.21.15.80:6789/0 conn(0x7f79d40ce730 legacy=0x7f79d406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79c0003ae0 0x7f79c0003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79c0008a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79c0009250 0x7f79c000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f79d4073cd0 con 0x7f79c0006650 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f79c000c000 con 0x7f79c0009250 2024-04-25T13:47:53.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f79c000c200 con 0x7f79c0003ae0 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d1d9b640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79c0008a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d2d9d640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79c0009250 0x7f79c000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d259c640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79c0003ae0 0x7f79c0003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d1d9b640 1 -- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79c0003ae0 msgr2=0x7f79c0003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d1d9b640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79c0003ae0 0x7f79c0003ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d1d9b640 1 -- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79c0009250 msgr2=0x7f79c000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d1d9b640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79c0009250 0x7f79c000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d1d9b640 1 -- 172.21.15.80:0/1948489575 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79c0003450 con 0x7f79c0006650 2024-04-25T13:47:53.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.674+0000 7f79d2d9d640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79c0009250 0x7f79c000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:53.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d1d9b640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79c0008a70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c210 tx=0x7f79bc01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355a722a349179d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:53.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc007d40 con 0x7f79c0006650 2024-04-25T13:47:53.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79bc0a9070 con 0x7f79c0006650 2024-04-25T13:47:53.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d159a640 1 -- 172.21.15.80:0/1948489575 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79bc0a5e40 con 0x7f79c0006650 2024-04-25T13:47:53.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d89d7640 1 -- 172.21.15.80:0/1948489575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 msgr2=0x7f79c0008a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d89d7640 1 --2- 172.21.15.80:0/1948489575 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79c0008a70 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00c210 tx=0x7f79bc01cca0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d2d9d640 1 -- 172.21.15.80:0/1948489575 reap_dead start 2024-04-25T13:47:53.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d89d7640 1 -- 172.21.15.80:0/1948489575 shutdown_connections 2024-04-25T13:47:53.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d89d7640 1 -- 172.21.15.80:0/1948489575 >> 172.21.15.80:0/1948489575 conn(0x7f79d406d080 msgr2=0x7f79d40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:53.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d89d7640 1 -- 172.21.15.80:0/1948489575 shutdown_connections 2024-04-25T13:47:53.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d89d7640 1 -- 172.21.15.80:0/1948489575 wait complete. 2024-04-25T13:47:53.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.678+0000 7f79d89d7640 1 Processor -- start 2024-04-25T13:47:53.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 -- start start 2024-04-25T13:47:53.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79d41f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d415da40 0x7f79d415de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79d40af8b0 0x7f79d40afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f79d406eba0 con 0x7f79c0006650 2024-04-25T13:47:53.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f79d40733a0 con 0x7f79d40af8b0 2024-04-25T13:47:53.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f79d406ed20 con 0x7f79d415da40 2024-04-25T13:47:53.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79d41f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79d41f6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60054/0 (socket says 172.21.15.80:60054) 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 -- 172.21.15.80:0/3961511181 learned_addr learned my addr 172.21.15.80:0/3961511181 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d1d9b640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d415da40 0x7f79d415de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d2d9d640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79d40af8b0 0x7f79d40afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 -- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d415da40 msgr2=0x7f79d415de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d415da40 0x7f79d415de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 -- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79d40af8b0 msgr2=0x7f79d40afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79d40af8b0 0x7f79d40afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 -- 172.21.15.80:0/3961511181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d41f6580 con 0x7f79c0006650 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d259c640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79d41f6040 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f79c400d910 tx=0x7f79c400ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79caffd640 1 -- 172.21.15.80:0/3961511181 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c4003db0 con 0x7f79c0006650 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79caffd640 1 -- 172.21.15.80:0/3961511181 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f79c4017070 con 0x7f79c0006650 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79caffd640 1 -- 172.21.15.80:0/3961511181 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c40052b0 con 0x7f79c0006650 2024-04-25T13:47:53.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d40b0200 con 0x7f79c0006650 2024-04-25T13:47:53.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d414a000 con 0x7f79c0006650 2024-04-25T13:47:53.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.682+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d406a3a0 con 0x7f79c0006650 2024-04-25T13:47:53.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.686+0000 7f79caffd640 1 -- 172.21.15.80:0/3961511181 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f79c4005450 con 0x7f79c0006650 2024-04-25T13:47:53.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.686+0000 7f79caffd640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79b406f430 0x7f79b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:53.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.686+0000 7f79caffd640 1 -- 172.21.15.80:0/3961511181 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f79c408f490 con 0x7f79c0006650 2024-04-25T13:47:53.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.686+0000 7f79d1d9b640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79b406f430 0x7f79b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:53.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.686+0000 7f79d1d9b640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79b406f430 0x7f79b4071940 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0ab000 tx=0x7f79bc0b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:53.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.690+0000 7f79caffd640 1 -- 172.21.15.80:0/3961511181 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f79d40b0200 con 0x7f79c0006650 2024-04-25T13:47:53.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.850+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79d415c1e0 con 0x7f79c0006650 2024-04-25T13:47:53.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79caffd640 1 -- 172.21.15.80:0/3961511181 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16716 (secure 0 0 0) 0x7f79d41f6580 con 0x7f79c0006650 2024-04-25T13:47:53.852 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:53.852 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":141,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:46.223394+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:47:08.431048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T13:47:09.003237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79b406f430 msgr2=0x7f79b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79b406f430 0x7f79b4071940 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0ab000 tx=0x7f79bc0b1040 comp rx=0 tx=0).stop 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 msgr2=0x7f79d41f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79d41f6040 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f79c400d910 tx=0x7f79c400ddd0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 shutdown_connections 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f79b406f430 0x7f79b4071940 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79d40af8b0 0x7f79d40afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f79d415da40 0x7f79d415de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 --2- 172.21.15.80:0/3961511181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f79c0006650 0x7f79d41f6040 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:53.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 >> 172.21.15.80:0/3961511181 conn(0x7f79d406d080 msgr2=0x7f79d40d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:53.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 shutdown_connections 2024-04-25T13:47:53.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:53.854+0000 7f79d89d7640 1 -- 172.21.15.80:0/3961511181 wait complete. 2024-04-25T13:47:53.875 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:47:08.431048+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:47:09.003237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '141', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:47:53.875 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:47:54.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4d5c48640 1 Processor -- start 2024-04-25T13:47:54.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4d5c48640 1 -- start start 2024-04-25T13:47:54.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4d5c48640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc4d00d1a80 0x7fc4d00d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4d5c48640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc4d00cf4b0 0x7fc4d00cf8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4d5c48640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc4d00d4a30 con 0x7fc4d00b6920 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4d5c48640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d00d2b40 con 0x7fc4d00d1a80 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4d5c48640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc4d00b4490 con 0x7fc4d00cf4b0 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4cffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc4d00b6920 0x7fc4d0162770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54086/0 (socket says 172.21.15.80:54086) 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4cffff640 1 -- 172.21.15.80:0/757809244 learned_addr learned my addr 172.21.15.80:0/757809244 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4cf7fe640 1 --2- 172.21.15.80:0/757809244 >> v2:172.21.15.80:3300/0 conn(0x7fc4d00d1a80 0x7fc4d00d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4c7fff640 1 --2- 172.21.15.80:0/757809244 >> v2:172.21.15.87:3300/0 conn(0x7fc4d00cf4b0 0x7fc4d00cf8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4ceffd640 1 -- 172.21.15.80:0/757809244 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2226473927 0 0) 0x7fc4d00d4a30 con 0x7fc4d00b6920 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4ceffd640 1 -- 172.21.15.80:0/757809244 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc4bc003530 con 0x7fc4d00b6920 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4cf7fe640 1 -- 172.21.15.80:0/757809244 >> v2:172.21.15.87:3300/0 conn(0x7fc4d00cf4b0 msgr2=0x7fc4d00cf8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4cf7fe640 1 --2- 172.21.15.80:0/757809244 >> v2:172.21.15.87:3300/0 conn(0x7fc4d00cf4b0 0x7fc4d00cf8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4cf7fe640 1 -- 172.21.15.80:0/757809244 >> v1:172.21.15.80:6789/0 conn(0x7fc4d00b6920 legacy=0x7fc4d0162770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.034+0000 7fc4cf7fe640 1 -- 172.21.15.80:0/757809244 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d00b4cd0 con 0x7fc4d00d1a80 2024-04-25T13:47:54.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4cf7fe640 1 --2- 172.21.15.80:0/757809244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d00d1e90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0069730 tx=0x7fc4c001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04297c866eeeffb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4ceffd640 1 -- 172.21.15.80:0/757809244 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00b0070 con 0x7fc4d00d1a80 2024-04-25T13:47:54.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4ceffd640 1 -- 172.21.15.80:0/757809244 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c00ab600 con 0x7fc4d00d1a80 2024-04-25T13:47:54.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4ceffd640 1 -- 172.21.15.80:0/757809244 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00ab9b0 con 0x7fc4d00d1a80 2024-04-25T13:47:54.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4d5c48640 1 -- 172.21.15.80:0/757809244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 msgr2=0x7fc4d00d1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/757809244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d00d1e90 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0069730 tx=0x7fc4c001c3f0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4d5c48640 1 -- 172.21.15.80:0/757809244 shutdown_connections 2024-04-25T13:47:54.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/757809244 >> v2:172.21.15.87:3300/0 conn(0x7fc4d00cf4b0 0x7fc4d00cf8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/757809244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d00d1e90 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4d5c48640 1 -- 172.21.15.80:0/757809244 >> 172.21.15.80:0/757809244 conn(0x7fc4d00d3ad0 msgr2=0x7fc4d00d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:54.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4d5c48640 1 -- 172.21.15.80:0/757809244 shutdown_connections 2024-04-25T13:47:54.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.038+0000 7fc4d5c48640 1 -- 172.21.15.80:0/757809244 wait complete. 2024-04-25T13:47:54.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 Processor -- start 2024-04-25T13:47:54.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 -- start start 2024-04-25T13:47:54.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4d00b6920 0x7fc4d01fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d00cf4b0 0x7fc4d0151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d015a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d00b7140 con 0x7fc4d00d1a80 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d00d2760 con 0x7fc4d00cf4b0 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d016b9d0 con 0x7fc4d00b6920 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d015a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d015a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60084/0 (socket says 172.21.15.80:60084) 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 -- 172.21.15.80:0/995968364 learned_addr learned my addr 172.21.15.80:0/995968364 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cf7fe640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4d00b6920 0x7fc4d01fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4c7fff640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d00cf4b0 0x7fc4d0151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 -- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4d00b6920 msgr2=0x7fc4d01fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4d00b6920 0x7fc4d01fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 -- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d00cf4b0 msgr2=0x7fc4d0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d00cf4b0 0x7fc4d0151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 -- 172.21.15.80:0/995968364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d0162ad0 con 0x7fc4d00d1a80 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4c7fff640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d00cf4b0 0x7fc4d0151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4cffff640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d015a730 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800ca70 tx=0x7fc4c800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4ccff9640 1 -- 172.21.15.80:0/995968364 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8017070 con 0x7fc4d00d1a80 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4ccff9640 1 -- 172.21.15.80:0/995968364 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4c800abd0 con 0x7fc4d00d1a80 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d00c7810 con 0x7fc4d00d1a80 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4ccff9640 1 -- 172.21.15.80:0/995968364 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8012400 con 0x7fc4d00d1a80 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d00c9370 con 0x7fc4d00d1a80 2024-04-25T13:47:54.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.042+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4d0162840 con 0x7fc4d00d1a80 2024-04-25T13:47:54.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.046+0000 7fc4ccff9640 1 -- 172.21.15.80:0/995968364 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc4c801a070 con 0x7fc4d00d1a80 2024-04-25T13:47:54.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.046+0000 7fc4ccff9640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc4b006f400 0x7fc4b0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.046+0000 7fc4ccff9640 1 -- 172.21.15.80:0/995968364 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7fc4c808f730 con 0x7fc4d00d1a80 2024-04-25T13:47:54.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.046+0000 7fc4cf7fe640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc4b006f400 0x7fc4b0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.046+0000 7fc4cf7fe640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc4b006f400 0x7fc4b0071910 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ad000 tx=0x7fc4c00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.050+0000 7fc4ccff9640 1 -- 172.21.15.80:0/995968364 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc4c805c070 con 0x7fc4d00d1a80 2024-04-25T13:47:54.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc4d00c95d0 con 0x7fc4d00d1a80 2024-04-25T13:47:54.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4ccff9640 1 -- 172.21.15.80:0/995968364 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v141) ==== 93+0+10 (secure 0 0 0) 0x7fc4c8060370 con 0x7fc4d00d1a80 2024-04-25T13:47:54.212 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:47:54.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc4b006f400 msgr2=0x7fc4b0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc4b006f400 0x7fc4b0071910 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00ad000 tx=0x7fc4c00a6000 comp rx=0 tx=0).stop 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 msgr2=0x7fc4d015a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d015a730 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc4c800ca70 tx=0x7fc4c800cf30 comp rx=0 tx=0).stop 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 shutdown_connections 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc4b006f400 0x7fc4b0071910 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc4d00d1a80 0x7fc4d015a730 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc4d00cf4b0 0x7fc4d0151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 --2- 172.21.15.80:0/995968364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc4d00b6920 0x7fc4d01fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 >> 172.21.15.80:0/995968364 conn(0x7fc4d00d3ad0 msgr2=0x7fc4d0070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 shutdown_connections 2024-04-25T13:47:54.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.214+0000 7fc4d5c48640 1 -- 172.21.15.80:0/995968364 wait complete. 2024-04-25T13:47:54.235 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:47:54.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac9262f640 1 Processor -- start 2024-04-25T13:47:54.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac9262f640 1 -- start start 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac9262f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fac8c0b5330 0x7fac8c0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac9262f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fac8c161b60 0x7fac8c161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac9262f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8c1624b0 con 0x7fac8c0d0ff0 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac9262f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fac8c1629b0 con 0x7fac8c0b5330 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac9262f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fac8c0b5740 con 0x7fac8c161b60 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac8bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fac8c0b5330 0x7fac8c0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac83fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fac8c0d0ff0 0x7fac8c06edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54088/0 (socket says 172.21.15.80:54088) 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac83fff640 1 -- 172.21.15.80:0/2160608456 learned_addr learned my addr 172.21.15.80:0/2160608456 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:54.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac90bcc640 1 --2- 172.21.15.80:0/2160608456 >> v2:172.21.15.87:3300/0 conn(0x7fac8c161b60 0x7fac8c161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac8bfff640 1 -- 172.21.15.80:0/2160608456 >> v2:172.21.15.87:3300/0 conn(0x7fac8c161b60 msgr2=0x7fac8c161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:54.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac8bfff640 1 --2- 172.21.15.80:0/2160608456 >> v2:172.21.15.87:3300/0 conn(0x7fac8c161b60 0x7fac8c161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac8bfff640 1 -- 172.21.15.80:0/2160608456 >> v1:172.21.15.80:6789/0 conn(0x7fac8c0d0ff0 legacy=0x7fac8c06edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac8b7fe640 1 -- 172.21.15.80:0/2160608456 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3962991347 0 0) 0x7fac74001120 con 0x7fac8c0d0ff0 2024-04-25T13:47:54.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac8bfff640 1 -- 172.21.15.80:0/2160608456 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c06dd10 con 0x7fac8c0b5330 2024-04-25T13:47:54.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.394+0000 7fac90bcc640 1 --2- 172.21.15.80:0/2160608456 >> v2:172.21.15.87:3300/0 conn(0x7fac8c161b60 0x7fac8c161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:54.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac8bfff640 1 --2- 172.21.15.80:0/2160608456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c0b5330 0x7fac8c0d0ab0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fac8c147cb0 tx=0x7fac7c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f0a48f7dc64b3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac8b7fe640 1 -- 172.21.15.80:0/2160608456 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0b0070 con 0x7fac8c0b5330 2024-04-25T13:47:54.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac8b7fe640 1 -- 172.21.15.80:0/2160608456 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c0ab600 con 0x7fac8c0b5330 2024-04-25T13:47:54.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac8b7fe640 1 -- 172.21.15.80:0/2160608456 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0ab9d0 con 0x7fac8c0b5330 2024-04-25T13:47:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac9262f640 1 -- 172.21.15.80:0/2160608456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c0b5330 msgr2=0x7fac8c0d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac9262f640 1 --2- 172.21.15.80:0/2160608456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c0b5330 0x7fac8c0d0ab0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fac8c147cb0 tx=0x7fac7c01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac9262f640 1 -- 172.21.15.80:0/2160608456 shutdown_connections 2024-04-25T13:47:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac9262f640 1 --2- 172.21.15.80:0/2160608456 >> v2:172.21.15.87:3300/0 conn(0x7fac8c161b60 0x7fac8c161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac9262f640 1 --2- 172.21.15.80:0/2160608456 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c0b5330 0x7fac8c0d0ab0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac9262f640 1 -- 172.21.15.80:0/2160608456 >> 172.21.15.80:0/2160608456 conn(0x7fac8c0b4160 msgr2=0x7fac8c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac9262f640 1 -- 172.21.15.80:0/2160608456 shutdown_connections 2024-04-25T13:47:54.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.398+0000 7fac9262f640 1 -- 172.21.15.80:0/2160608456 wait complete. 2024-04-25T13:47:54.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 Processor -- start 2024-04-25T13:47:54.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 -- start start 2024-04-25T13:47:54.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8c0b5330 0x7fac8c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac8c0d0ff0 0x7fac8c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c161b60 0x7fac8c154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c0b0cd0 con 0x7fac8c161b60 2024-04-25T13:47:54.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fac8c06e890 con 0x7fac8c0d0ff0 2024-04-25T13:47:54.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fac8c0b0e50 con 0x7fac8c0b5330 2024-04-25T13:47:54.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c161b60 0x7fac8c154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c161b60 0x7fac8c154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60098/0 (socket says 172.21.15.80:60098) 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 -- 172.21.15.80:0/2194690994 learned_addr learned my addr 172.21.15.80:0/2194690994 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac83fff640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac8c0d0ff0 0x7fac8c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac8bfff640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8c0b5330 0x7fac8c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 -- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8c0b5330 msgr2=0x7fac8c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8c0b5330 0x7fac8c159480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 -- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac8c0d0ff0 msgr2=0x7fac8c15e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac8c0d0ff0 0x7fac8c15e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 -- 172.21.15.80:0/2194690994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac8c159dd0 con 0x7fac8c161b60 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac83fff640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac8c0d0ff0 0x7fac8c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac8bfff640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8c0b5330 0x7fac8c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac90bcc640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c161b60 0x7fac8c154b60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fac84007e20 tx=0x7fac8400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac897fa640 1 -- 172.21.15.80:0/2194690994 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac8401a070 con 0x7fac8c161b60 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac897fa640 1 -- 172.21.15.80:0/2194690994 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac840039a0 con 0x7fac8c161b60 2024-04-25T13:47:54.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac8c15a0b0 con 0x7fac8c161b60 2024-04-25T13:47:54.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac897fa640 1 -- 172.21.15.80:0/2194690994 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84003d70 con 0x7fac8c161b60 2024-04-25T13:47:54.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.402+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac8c156d30 con 0x7fac8c161b60 2024-04-25T13:47:54.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.406+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8c156f90 con 0x7fac8c161b60 2024-04-25T13:47:54.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.406+0000 7fac897fa640 1 -- 172.21.15.80:0/2194690994 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fac8401d070 con 0x7fac8c161b60 2024-04-25T13:47:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.406+0000 7fac897fa640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac6c06f400 0x7fac6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.406+0000 7fac897fa640 1 -- 172.21.15.80:0/2194690994 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7fac8408f5a0 con 0x7fac8c161b60 2024-04-25T13:47:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.406+0000 7fac8bfff640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac6c06f400 0x7fac6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.406+0000 7fac8bfff640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac6c06f400 0x7fac6c071910 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0ad000 tx=0x7fac7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.410+0000 7fac897fa640 1 -- 172.21.15.80:0/2194690994 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac84010070 con 0x7fac8c161b60 2024-04-25T13:47:54.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.566+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fac8c06a3a0 con 0x7fac8c161b60 2024-04-25T13:47:54.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.566+0000 7fac897fa640 1 -- 172.21.15.80:0/2194690994 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v141) ==== 104+0+11 (secure 0 0 0) 0x7fac8405c450 con 0x7fac8c161b60 2024-04-25T13:47:54.565 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:47:54.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac6c06f400 msgr2=0x7fac6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac6c06f400 0x7fac6c071910 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fac7c0ad000 tx=0x7fac7c008040 comp rx=0 tx=0).stop 2024-04-25T13:47:54.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c161b60 msgr2=0x7fac8c154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c161b60 0x7fac8c154b60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fac84007e20 tx=0x7fac8400d4a0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 shutdown_connections 2024-04-25T13:47:54.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac6c06f400 0x7fac6c071910 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac8c161b60 0x7fac8c154b60 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac8c0d0ff0 0x7fac8c15e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 --2- 172.21.15.80:0/2194690994 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8c0b5330 0x7fac8c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 >> 172.21.15.80:0/2194690994 conn(0x7fac8c0b4160 msgr2=0x7fac8c160790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:54.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 shutdown_connections 2024-04-25T13:47:54.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.570+0000 7fac9262f640 1 -- 172.21.15.80:0/2194690994 wait complete. 2024-04-25T13:47:54.586 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:47:54.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.722+0000 7f837d11c640 1 Processor -- start 2024-04-25T13:47:54.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.722+0000 7f837d11c640 1 -- start start 2024-04-25T13:47:54.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837d11c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f83780d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837d11c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f83780d0ff0 0x7f837806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837d11c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f837806f2f0 0x7f8378161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837d11c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f837806f700 con 0x7f83780d0ff0 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837d11c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f83781629b0 con 0x7f837806f2f0 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837d11c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f83780b5740 con 0x7f83780b5330 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f83780d0ff0 0x7f837806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f83780d0ff0 0x7f837806edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60104/0 (socket says 172.21.15.80:60104) 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 -- 172.21.15.80:0/3964236914 learned_addr learned my addr 172.21.15.80:0/3964236914 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 -- 172.21.15.80:0/3964236914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 msgr2=0x7f83780d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837759e640 1 --2- 172.21.15.80:0/3964236914 >> v2:172.21.15.87:3300/0 conn(0x7f837806f2f0 0x7f8378161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f8376d9d640 1 --2- 172.21.15.80:0/3964236914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f83780d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 --2- 172.21.15.80:0/3964236914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f83780d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 -- 172.21.15.80:0/3964236914 >> v2:172.21.15.87:3300/0 conn(0x7f837806f2f0 msgr2=0x7f8378161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 --2- 172.21.15.80:0/3964236914 >> v2:172.21.15.87:3300/0 conn(0x7f837806f2f0 0x7f8378161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 -- 172.21.15.80:0/3964236914 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837806dd10 con 0x7f83780d0ff0 2024-04-25T13:47:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837759e640 1 --2- 172.21.15.80:0/3964236914 >> v2:172.21.15.87:3300/0 conn(0x7f837806f2f0 0x7f8378161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:54.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f8376d9d640 1 --2- 172.21.15.80:0/3964236914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f83780d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f837659c640 1 --2- 172.21.15.80:0/3964236914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83780d0ff0 0x7f837806edb0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f837815f680 tx=0x7f836c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3896e26591781f2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f8375d9b640 1 -- 172.21.15.80:0/3964236914 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0b0070 con 0x7f83780d0ff0 2024-04-25T13:47:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.726+0000 7f8375d9b640 1 -- 172.21.15.80:0/3964236914 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f836c0ab600 con 0x7f83780d0ff0 2024-04-25T13:47:54.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f8375d9b640 1 -- 172.21.15.80:0/3964236914 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f836c0ab9d0 con 0x7f83780d0ff0 2024-04-25T13:47:54.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 -- 172.21.15.80:0/3964236914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83780d0ff0 msgr2=0x7f837806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 --2- 172.21.15.80:0/3964236914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83780d0ff0 0x7f837806edb0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f837815f680 tx=0x7f836c01c490 comp rx=0 tx=0).stop 2024-04-25T13:47:54.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 -- 172.21.15.80:0/3964236914 shutdown_connections 2024-04-25T13:47:54.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 --2- 172.21.15.80:0/3964236914 >> v2:172.21.15.87:3300/0 conn(0x7f837806f2f0 0x7f8378161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 --2- 172.21.15.80:0/3964236914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83780d0ff0 0x7f837806edb0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 --2- 172.21.15.80:0/3964236914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f83780d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 -- 172.21.15.80:0/3964236914 >> 172.21.15.80:0/3964236914 conn(0x7f83780b4160 msgr2=0x7f83780d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:54.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 -- 172.21.15.80:0/3964236914 shutdown_connections 2024-04-25T13:47:54.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 -- 172.21.15.80:0/3964236914 wait complete. 2024-04-25T13:47:54.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 Processor -- start 2024-04-25T13:47:54.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.730+0000 7f837d11c640 1 -- start start 2024-04-25T13:47:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f837806f2f0 0x7f837815c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f8378072a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83780d0ff0 0x7f83780af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f83780b0cd0 con 0x7f837806f2f0 2024-04-25T13:47:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f837806e890 con 0x7f83780d0ff0 2024-04-25T13:47:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f83780b0e50 con 0x7f83780b5330 2024-04-25T13:47:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f837806f2f0 0x7f837815c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f837806f2f0 0x7f837815c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60112/0 (socket says 172.21.15.80:60112) 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 -- 172.21.15.80:0/2896430786 learned_addr learned my addr 172.21.15.80:0/2896430786 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837759e640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83780d0ff0 0x7f83780af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837659c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f8378072a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 -- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 msgr2=0x7f8378072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f8378072a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 -- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83780d0ff0 msgr2=0x7f83780af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83780d0ff0 0x7f83780af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 -- 172.21.15.80:0/2896430786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f837815c6d0 con 0x7f837806f2f0 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837759e640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83780d0ff0 0x7f83780af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f8376d9d640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f837806f2f0 0x7f837815c190 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f836800c820 tx=0x7f836800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f835f7fe640 1 -- 172.21.15.80:0/2896430786 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368015070 con 0x7f837806f2f0 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f835f7fe640 1 -- 172.21.15.80:0/2896430786 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8368007a30 con 0x7f837806f2f0 2024-04-25T13:47:54.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83781fa710 con 0x7f837806f2f0 2024-04-25T13:47:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f835f7fe640 1 -- 172.21.15.80:0/2896430786 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8368007e40 con 0x7f837806f2f0 2024-04-25T13:47:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8378072270 con 0x7f837806f2f0 2024-04-25T13:47:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f837806a3a0 con 0x7f837806f2f0 2024-04-25T13:47:54.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f835f7fe640 1 -- 172.21.15.80:0/2896430786 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8368018070 con 0x7f837806f2f0 2024-04-25T13:47:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f835f7fe640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f836406f400 0x7f8364071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f835f7fe640 1 -- 172.21.15.80:0/2896430786 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f836808f210 con 0x7f837806f2f0 2024-04-25T13:47:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.734+0000 7f837659c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f836406f400 0x7f8364071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:54.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.738+0000 7f837659c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f836406f400 0x7f8364071910 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f836c0ad000 tx=0x7f836c004740 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:54.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.738+0000 7f835f7fe640 1 -- 172.21.15.80:0/2896430786 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f836805c150 con 0x7f837806f2f0 2024-04-25T13:47:54.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f83781615c0 con 0x7f837806f2f0 2024-04-25T13:47:54.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f835f7fe640 1 -- 172.21.15.80:0/2896430786 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v141) ==== 100+0+11 (secure 0 0 0) 0x7f836805ba50 con 0x7f837806f2f0 2024-04-25T13:47:54.892 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:47:54.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f836406f400 msgr2=0x7f8364071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f836406f400 0x7f8364071910 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f836c0ad000 tx=0x7f836c004740 comp rx=0 tx=0).stop 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f837806f2f0 msgr2=0x7f837815c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f837806f2f0 0x7f837815c190 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f836800c820 tx=0x7f836800cce0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 shutdown_connections 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f836406f400 0x7f8364071910 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83780d0ff0 0x7f83780af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83780b5330 0x7f8378072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 --2- 172.21.15.80:0/2896430786 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f837806f2f0 0x7f837815c190 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 >> 172.21.15.80:0/2896430786 conn(0x7f83780b4160 msgr2=0x7f83780d0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 shutdown_connections 2024-04-25T13:47:54.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:54.894+0000 7f837d11c640 1 -- 172.21.15.80:0/2896430786 wait complete. 2024-04-25T13:47:54.913 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:55.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.054+0000 7f532a9ed640 1 Processor -- start 2024-04-25T13:47:55.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.054+0000 7f532a9ed640 1 -- start start 2024-04-25T13:47:55.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f532a9ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f532406f5a0 0x7f5324160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f532a9ed640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5324161070 0x7f5324161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f532a9ed640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f53240b4bd0 0x7f53240b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f532a9ed640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f53241619c0 con 0x7f5324161070 2024-04-25T13:47:55.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f532a9ed640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5324073f10 con 0x7f53240b4bd0 2024-04-25T13:47:55.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f532a9ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f532406fcf0 con 0x7f532406f5a0 2024-04-25T13:47:55.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f53237fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5324161070 0x7f5324161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f53237fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5324161070 0x7f5324161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60126/0 (socket says 172.21.15.80:60126) 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f53237fe640 1 -- 172.21.15.80:0/3362500669 learned_addr learned my addr 172.21.15.80:0/3362500669 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f5328f8a640 1 --2- 172.21.15.80:0/3362500669 >> v2:172.21.15.87:3300/0 conn(0x7f53240b4bd0 0x7f53240b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f5323fff640 1 --2- 172.21.15.80:0/3362500669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f532406f5a0 0x7f5324160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f53237fe640 1 -- 172.21.15.80:0/3362500669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f532406f5a0 msgr2=0x7f5324160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f53237fe640 1 --2- 172.21.15.80:0/3362500669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f532406f5a0 0x7f5324160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f53237fe640 1 -- 172.21.15.80:0/3362500669 >> v2:172.21.15.87:3300/0 conn(0x7f53240b4bd0 msgr2=0x7f53240b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f53237fe640 1 --2- 172.21.15.80:0/3362500669 >> v2:172.21.15.87:3300/0 conn(0x7f53240b4bd0 0x7f53240b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f53237fe640 1 -- 172.21.15.80:0/3362500669 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53240b0c60 con 0x7f5324161070 2024-04-25T13:47:55.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.058+0000 7f5323fff640 1 --2- 172.21.15.80:0/3362500669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f532406f5a0 0x7f5324160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:55.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f53237fe640 1 --2- 172.21.15.80:0/3362500669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324161480 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f5324070270 tx=0x7f531801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94accc5966bb0f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f5322ffd640 1 -- 172.21.15.80:0/3362500669 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180b0070 con 0x7f5324161070 2024-04-25T13:47:55.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f5322ffd640 1 -- 172.21.15.80:0/3362500669 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53180ab600 con 0x7f5324161070 2024-04-25T13:47:55.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f5322ffd640 1 -- 172.21.15.80:0/3362500669 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53180ab9d0 con 0x7f5324161070 2024-04-25T13:47:55.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 -- 172.21.15.80:0/3362500669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 msgr2=0x7f5324161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 --2- 172.21.15.80:0/3362500669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324161480 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f5324070270 tx=0x7f531801c490 comp rx=0 tx=0).stop 2024-04-25T13:47:55.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 -- 172.21.15.80:0/3362500669 shutdown_connections 2024-04-25T13:47:55.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 --2- 172.21.15.80:0/3362500669 >> v2:172.21.15.87:3300/0 conn(0x7f53240b4bd0 0x7f53240b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 --2- 172.21.15.80:0/3362500669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324161480 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 --2- 172.21.15.80:0/3362500669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f532406f5a0 0x7f5324160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 -- 172.21.15.80:0/3362500669 >> 172.21.15.80:0/3362500669 conn(0x7f532406d080 msgr2=0x7f53240b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:55.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 -- 172.21.15.80:0/3362500669 shutdown_connections 2024-04-25T13:47:55.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 -- 172.21.15.80:0/3362500669 wait complete. 2024-04-25T13:47:55.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 Processor -- start 2024-04-25T13:47:55.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.062+0000 7f532a9ed640 1 -- start start 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532406f5a0 0x7f5324151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f53240b4bd0 0x7f532415e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5324072bb0 con 0x7f5324161070 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f53240b5930 con 0x7f532406f5a0 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5324072d30 con 0x7f53240b4bd0 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60140/0 (socket says 172.21.15.80:60140) 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 -- 172.21.15.80:0/2392176127 learned_addr learned my addr 172.21.15.80:0/2392176127 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f53237fe640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f53240b4bd0 0x7f532415e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5323fff640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532406f5a0 0x7f5324151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 -- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f53240b4bd0 msgr2=0x7f532415e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f53240b4bd0 0x7f532415e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 -- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532406f5a0 msgr2=0x7f5324151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532406f5a0 0x7f5324151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 -- 172.21.15.80:0/2392176127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f532415eb70 con 0x7f5324161070 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f53237fe640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f53240b4bd0 0x7f532415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5328f8a640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324159470 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f531c00d500 tx=0x7f531c00d9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5320ff9640 1 -- 172.21.15.80:0/2392176127 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c0125f0 con 0x7f5324161070 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5320ff9640 1 -- 172.21.15.80:0/2392176127 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f531c019070 con 0x7f5324161070 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f532415ee50 con 0x7f5324161070 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5320ff9640 1 -- 172.21.15.80:0/2392176127 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531c0129d0 con 0x7f5324161070 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f532415f410 con 0x7f5324161070 2024-04-25T13:47:55.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f532406a3a0 con 0x7f5324161070 2024-04-25T13:47:55.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5320ff9640 1 -- 172.21.15.80:0/2392176127 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f531c012b70 con 0x7f5324161070 2024-04-25T13:47:55.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5320ff9640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f531006f400 0x7f5310071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.066+0000 7f5320ff9640 1 -- 172.21.15.80:0/2392176127 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f531c08f6c0 con 0x7f5324161070 2024-04-25T13:47:55.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.070+0000 7f5323fff640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f531006f400 0x7f5310071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.070+0000 7f5323fff640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f531006f400 0x7f5310071910 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f53140070b0 tx=0x7f5314007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.074+0000 7f5320ff9640 1 -- 172.21.15.80:0/2392176127 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f531c05c580 con 0x7f5324161070 2024-04-25T13:47:55.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.250+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53241581e0 con 0x7f5324161070 2024-04-25T13:47:55.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f5320ff9640 1 -- 172.21.15.80:0/2392176127 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7f531c05be80 con 0x7f5324161070 2024-04-25T13:47:55.251 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:55.251 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:55.251 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:55.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f531006f400 msgr2=0x7f5310071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f531006f400 0x7f5310071910 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f53140070b0 tx=0x7f5314007040 comp rx=0 tx=0).stop 2024-04-25T13:47:55.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 msgr2=0x7f5324159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324159470 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f531c00d500 tx=0x7f531c00d9c0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 shutdown_connections 2024-04-25T13:47:55.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f531006f400 0x7f5310071910 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5324161070 0x7f5324159470 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f53240b4bd0 0x7f532415e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 --2- 172.21.15.80:0/2392176127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f532406f5a0 0x7f5324151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 >> 172.21.15.80:0/2392176127 conn(0x7f532406d080 msgr2=0x7f5324160260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:55.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 shutdown_connections 2024-04-25T13:47:55.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.254+0000 7f532a9ed640 1 -- 172.21.15.80:0/2392176127 wait complete. 2024-04-25T13:47:55.271 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.406+0000 7f27a077a640 1 Processor -- start 2024-04-25T13:47:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.406+0000 7f27a077a640 1 -- start start 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f27a077a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f279806f5a0 0x7f2798160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f27a077a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f27980733b0 0x7f2798161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f27a077a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f279806fcf0 con 0x7f2798161070 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f27a077a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f27981619c0 con 0x7f27980733b0 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f27a077a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2798073f10 con 0x7f279806f5a0 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f279ed17640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f27980733b0 0x7f2798161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f279ed17640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f27980733b0 0x7f2798161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60148/0 (socket says 172.21.15.80:60148) 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f279ed17640 1 -- 172.21.15.80:0/2368295453 learned_addr learned my addr 172.21.15.80:0/2368295453 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f279e516640 1 --2- 172.21.15.80:0/2368295453 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f279806f5a0 0x7f2798160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f279ed17640 1 -- 172.21.15.80:0/2368295453 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f279806f5a0 msgr2=0x7f2798160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:55.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f279ed17640 1 --2- 172.21.15.80:0/2368295453 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f279806f5a0 0x7f2798160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f279ed17640 1 -- 172.21.15.80:0/2368295453 >> v1:172.21.15.87:6789/0 conn(0x7f2798161070 legacy=0x7f27980b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:55.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.410+0000 7f279ed17640 1 -- 172.21.15.80:0/2368295453 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27980b0c60 con 0x7f27980733b0 2024-04-25T13:47:55.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f279ed17640 1 --2- 172.21.15.80:0/2368295453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27980733b0 0x7f2798161480 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2790009e80 tx=0x7f279001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e977c1e644da8019 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f279d514640 1 -- 172.21.15.80:0/2368295453 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27900a55d0 con 0x7f27980733b0 2024-04-25T13:47:55.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f279d514640 1 -- 172.21.15.80:0/2368295453 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f27900a5770 con 0x7f27980733b0 2024-04-25T13:47:55.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f279d514640 1 -- 172.21.15.80:0/2368295453 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27900a5b40 con 0x7f27980733b0 2024-04-25T13:47:55.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 -- 172.21.15.80:0/2368295453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27980733b0 msgr2=0x7f2798161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 --2- 172.21.15.80:0/2368295453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27980733b0 0x7f2798161480 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f2790009e80 tx=0x7f279001c940 comp rx=0 tx=0).stop 2024-04-25T13:47:55.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 -- 172.21.15.80:0/2368295453 shutdown_connections 2024-04-25T13:47:55.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 --2- 172.21.15.80:0/2368295453 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f27980733b0 0x7f2798161480 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 --2- 172.21.15.80:0/2368295453 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f279806f5a0 0x7f2798160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 -- 172.21.15.80:0/2368295453 >> 172.21.15.80:0/2368295453 conn(0x7f279806d080 msgr2=0x7f27980b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:55.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 -- 172.21.15.80:0/2368295453 shutdown_connections 2024-04-25T13:47:55.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 -- 172.21.15.80:0/2368295453 wait complete. 2024-04-25T13:47:55.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 Processor -- start 2024-04-25T13:47:55.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.414+0000 7f27a077a640 1 -- start start 2024-04-25T13:47:55.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f279806f5a0 0x7f279814c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27980733b0 0x7f279815baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2798161070 0x7f2798154740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f279816a050 con 0x7f279806f5a0 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2798169ed0 con 0x7f2798161070 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f279816a1d0 con 0x7f27980733b0 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f279806f5a0 0x7f279814c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f279806f5a0 0x7f279814c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60154/0 (socket says 172.21.15.80:60154) 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 -- 172.21.15.80:0/3281054191 learned_addr learned my addr 172.21.15.80:0/3281054191 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279dd15640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27980733b0 0x7f279815baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 -- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27980733b0 msgr2=0x7f279815baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27980733b0 0x7f279815baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279ed17640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2798161070 0x7f2798154740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 -- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2798161070 msgr2=0x7f2798154740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2798161070 0x7f2798154740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 -- 172.21.15.80:0/3281054191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f279815a4d0 con 0x7f279806f5a0 2024-04-25T13:47:55.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f279e516640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f279806f5a0 0x7f279814c0c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f279400fa40 tx=0x7f279400ff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f278effd640 1 -- 172.21.15.80:0/3281054191 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794004440 con 0x7f279806f5a0 2024-04-25T13:47:55.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f278effd640 1 -- 172.21.15.80:0/3281054191 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2794017070 con 0x7f279806f5a0 2024-04-25T13:47:55.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f279815a7b0 con 0x7f279806f5a0 2024-04-25T13:47:55.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f278effd640 1 -- 172.21.15.80:0/3281054191 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794007a40 con 0x7f279806f5a0 2024-04-25T13:47:55.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f279815ad70 con 0x7f279806f5a0 2024-04-25T13:47:55.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f279806a3a0 con 0x7f279806f5a0 2024-04-25T13:47:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f278effd640 1 -- 172.21.15.80:0/3281054191 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2794007be0 con 0x7f279806f5a0 2024-04-25T13:47:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f278effd640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f277c06f400 0x7f277c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.418+0000 7f278effd640 1 -- 172.21.15.80:0/3281054191 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7f2794091980 con 0x7f279806f5a0 2024-04-25T13:47:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.422+0000 7f279dd15640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f277c06f400 0x7f277c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.422+0000 7f279dd15640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f277c06f400 0x7f277c071910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2798160a30 tx=0x7f2788008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.422 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.422+0000 7f278effd640 1 -- 172.21.15.80:0/3281054191 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f279405e350 con 0x7f279806f5a0 2024-04-25T13:47:55.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.574+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27980739c0 con 0x7f279806f5a0 2024-04-25T13:47:55.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f278effd640 1 -- 172.21.15.80:0/3281054191 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v141) ==== 74+0+16716 (secure 0 0 0) 0x7f2794062650 con 0x7f279806f5a0 2024-04-25T13:47:55.575 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:55.575 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":141,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:46.223394+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:47:08.431048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T13:47:09.003237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"141","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:55.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f277c06f400 msgr2=0x7f277c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f277c06f400 0x7f277c071910 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2798160a30 tx=0x7f2788008040 comp rx=0 tx=0).stop 2024-04-25T13:47:55.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f279806f5a0 msgr2=0x7f279814c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f279806f5a0 0x7f279814c0c0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f279400fa40 tx=0x7f279400ff00 comp rx=0 tx=0).stop 2024-04-25T13:47:55.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 shutdown_connections 2024-04-25T13:47:55.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f277c06f400 0x7f277c071910 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2798161070 0x7f2798154740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f27980733b0 0x7f279815baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 --2- 172.21.15.80:0/3281054191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f279806f5a0 0x7f279814c0c0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 >> 172.21.15.80:0/3281054191 conn(0x7f279806d080 msgr2=0x7f27980d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:55.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 shutdown_connections 2024-04-25T13:47:55.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.578+0000 7f27a077a640 1 -- 172.21.15.80:0/3281054191 wait complete. 2024-04-25T13:47:55.597 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:47:55.597 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:55.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.746+0000 7fb8039b1640 1 Processor -- start 2024-04-25T13:47:55.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.746+0000 7fb8039b1640 1 -- start start 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb8039b1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb7fc06f5a0 0x7fb7fc160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb8039b1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb7fc161070 0x7fb7fc161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb8039b1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7fc06fcf0 con 0x7fb7fc0b4bd0 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb8039b1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb7fc1619c0 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb8039b1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb7fc073f10 con 0x7fb7fc161070 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb80174d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb7fc06f5a0 0x7fb7fc160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb80174d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb7fc06f5a0 0x7fb7fc160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60158/0 (socket says 172.21.15.80:60158) 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb80174d640 1 -- 172.21.15.80:0/293190645 learned_addr learned my addr 172.21.15.80:0/293190645 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb800f4c640 1 --2- 172.21.15.80:0/293190645 >> v2:172.21.15.87:3300/0 conn(0x7fb7fc161070 0x7fb7fc161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb80174d640 1 -- 172.21.15.80:0/293190645 >> v2:172.21.15.87:3300/0 conn(0x7fb7fc161070 msgr2=0x7fb7fc161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb80174d640 1 --2- 172.21.15.80:0/293190645 >> v2:172.21.15.87:3300/0 conn(0x7fb7fc161070 0x7fb7fc161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb80174d640 1 -- 172.21.15.80:0/293190645 >> v1:172.21.15.87:6789/0 conn(0x7fb7fc0b4bd0 legacy=0x7fb7fc0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.750+0000 7fb80174d640 1 -- 172.21.15.80:0/293190645 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc0b0c60 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb80174d640 1 --2- 172.21.15.80:0/293190645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc160b30 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc06ff70 tx=0x7fb7f801eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed11c4699ea8884e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb7f3fff640 1 -- 172.21.15.80:0/293190645 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80b4070 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb7f3fff640 1 -- 172.21.15.80:0/293190645 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f80a7a20 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb7f3fff640 1 -- 172.21.15.80:0/293190645 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80a7df0 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 -- 172.21.15.80:0/293190645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 msgr2=0x7fb7fc160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 --2- 172.21.15.80:0/293190645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc160b30 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc06ff70 tx=0x7fb7f801eb60 comp rx=0 tx=0).stop 2024-04-25T13:47:55.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 -- 172.21.15.80:0/293190645 shutdown_connections 2024-04-25T13:47:55.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 --2- 172.21.15.80:0/293190645 >> v2:172.21.15.87:3300/0 conn(0x7fb7fc161070 0x7fb7fc161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 --2- 172.21.15.80:0/293190645 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc160b30 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 -- 172.21.15.80:0/293190645 >> 172.21.15.80:0/293190645 conn(0x7fb7fc06d080 msgr2=0x7fb7fc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:55.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 -- 172.21.15.80:0/293190645 shutdown_connections 2024-04-25T13:47:55.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 -- 172.21.15.80:0/293190645 wait complete. 2024-04-25T13:47:55.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 Processor -- start 2024-04-25T13:47:55.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.754+0000 7fb8039b1640 1 -- start start 2024-04-25T13:47:55.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc0b4bd0 0x7fb7fc1506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7fc161070 0x7fb7fc150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc06ee20 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc06eca0 con 0x7fb7fc0b4bd0 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb7fc06efa0 con 0x7fb7fc161070 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60160/0 (socket says 172.21.15.80:60160) 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 -- 172.21.15.80:0/2579815265 learned_addr learned my addr 172.21.15.80:0/2579815265 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb800f4c640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc0b4bd0 0x7fb7fc1506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 -- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7fc161070 msgr2=0x7fb7fc150be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb801f4e640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7fc161070 0x7fb7fc150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7fc161070 0x7fb7fc150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 -- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc0b4bd0 msgr2=0x7fb7fc1506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc0b4bd0 0x7fb7fc1506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 -- 172.21.15.80:0/2579815265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc1512a0 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb800f4c640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc0b4bd0 0x7fb7fc1506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:55.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb80174d640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc152a70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fb7f801b770 tx=0x7fb7f801b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb7f1ffb640 1 -- 172.21.15.80:0/2579815265 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80be070 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb7f1ffb640 1 -- 172.21.15.80:0/2579815265 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f80af800 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb7f1ffb640 1 -- 172.21.15.80:0/2579815265 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f80b4040 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc15b2c0 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc153f30 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.758+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7fc15b0b0 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.762+0000 7fb7f1ffb640 1 -- 172.21.15.80:0/2579815265 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb7f80c1070 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.762+0000 7fb7f1ffb640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7e006f430 0x7fb7e0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:55.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.762+0000 7fb7f1ffb640 1 -- 172.21.15.80:0/2579815265 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7fb7f812c790 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.762+0000 7fb800f4c640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7e006f430 0x7fb7e0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:55.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.762+0000 7fb800f4c640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7e006f430 0x7fb7e0071940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc154d70 tx=0x7fb7ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:55.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.766+0000 7fb7f1ffb640 1 -- 172.21.15.80:0/2579815265 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb7f80187e0 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.950+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7fc06a3a0 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.950+0000 7fb7f1ffb640 1 -- 172.21.15.80:0/2579815265 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7fb7f80f9880 con 0x7fb7fc06f5a0 2024-04-25T13:47:55.949 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:55.949 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:55.949 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:55.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7e006f430 msgr2=0x7fb7e0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7e006f430 0x7fb7e0071940 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc154d70 tx=0x7fb7ec008040 comp rx=0 tx=0).stop 2024-04-25T13:47:55.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 msgr2=0x7fb7fc152a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:55.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc152a70 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fb7f801b770 tx=0x7fb7f801b900 comp rx=0 tx=0).stop 2024-04-25T13:47:55.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 shutdown_connections 2024-04-25T13:47:55.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7e006f430 0x7fb7e0071940 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7fc161070 0x7fb7fc150be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc0b4bd0 0x7fb7fc1506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 --2- 172.21.15.80:0/2579815265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7fc06f5a0 0x7fb7fc152a70 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:55.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 >> 172.21.15.80:0/2579815265 conn(0x7fb7fc06d080 msgr2=0x7fb7fc0d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:55.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 shutdown_connections 2024-04-25T13:47:55.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:55.954+0000 7fb8039b1640 1 -- 172.21.15.80:0/2579815265 wait complete. 2024-04-25T13:47:55.969 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:47:56.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.118+0000 7fb36d1c3640 1 Processor -- start 2024-04-25T13:47:56.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36d1c3640 1 -- start start 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36d1c3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb368161f90 0x7fb3680cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36d1c3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb36815ec40 0x7fb36815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36d1c3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3680d0d70 con 0x7fb3680d0510 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36d1c3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb3680d1050 con 0x7fb368161f90 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36d1c3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb36806dca0 con 0x7fb36815ec40 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36759e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb36815ec40 0x7fb36815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb366d9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb368161f90 0x7fb3680cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36659c640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb3680d0510 0x7fb36815e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:35216/0 (socket says 172.21.15.80:35216) 2024-04-25T13:47:56.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36659c640 1 -- 172.21.15.80:0/471077950 learned_addr learned my addr 172.21.15.80:0/471077950 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:56.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36759e640 1 -- 172.21.15.80:0/471077950 >> v2:172.21.15.87:3300/0 conn(0x7fb368161f90 msgr2=0x7fb3680cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:56.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36759e640 1 --2- 172.21.15.80:0/471077950 >> v2:172.21.15.87:3300/0 conn(0x7fb368161f90 0x7fb3680cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36759e640 1 -- 172.21.15.80:0/471077950 >> v1:172.21.15.87:6789/0 conn(0x7fb3680d0510 legacy=0x7fb36815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.122+0000 7fb36759e640 1 -- 172.21.15.80:0/471077950 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb36806df80 con 0x7fb36815ec40 2024-04-25T13:47:56.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36759e640 1 --2- 172.21.15.80:0/471077950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb36815ec40 0x7fb36815f050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb358009d90 tx=0x7fb35801c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a39a15ef23c65917 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb365d9b640 1 -- 172.21.15.80:0/471077950 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3580b2070 con 0x7fb36815ec40 2024-04-25T13:47:56.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb365d9b640 1 -- 172.21.15.80:0/471077950 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3580a5870 con 0x7fb36815ec40 2024-04-25T13:47:56.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb365d9b640 1 -- 172.21.15.80:0/471077950 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3580a5c40 con 0x7fb36815ec40 2024-04-25T13:47:56.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36d1c3640 1 -- 172.21.15.80:0/471077950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb36815ec40 msgr2=0x7fb36815f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/471077950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb36815ec40 0x7fb36815f050 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb358009d90 tx=0x7fb35801c880 comp rx=0 tx=0).stop 2024-04-25T13:47:56.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36d1c3640 1 -- 172.21.15.80:0/471077950 shutdown_connections 2024-04-25T13:47:56.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/471077950 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb36815ec40 0x7fb36815f050 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/471077950 >> v2:172.21.15.87:3300/0 conn(0x7fb368161f90 0x7fb3680cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36d1c3640 1 -- 172.21.15.80:0/471077950 >> 172.21.15.80:0/471077950 conn(0x7fb36806d080 msgr2=0x7fb3680b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:56.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36d1c3640 1 -- 172.21.15.80:0/471077950 shutdown_connections 2024-04-25T13:47:56.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.126+0000 7fb36d1c3640 1 -- 172.21.15.80:0/471077950 wait complete. 2024-04-25T13:47:56.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 Processor -- start 2024-04-25T13:47:56.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 -- start start 2024-04-25T13:47:56.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3680d0510 0x7fb368157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb36815ec40 0x7fb368160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb368161f90 0x7fb368161250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb36806f4b0 con 0x7fb3680d0510 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb36806f630 con 0x7fb36815ec40 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb3680d0920 con 0x7fb368161f90 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3680d0510 0x7fb368157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3680d0510 0x7fb368157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60170/0 (socket says 172.21.15.80:60170) 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 -- 172.21.15.80:0/2933154647 learned_addr learned my addr 172.21.15.80:0/2933154647 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36759e640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb368161f90 0x7fb368161250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36659c640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb36815ec40 0x7fb368160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 -- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb368161f90 msgr2=0x7fb368161250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb368161f90 0x7fb368161250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 -- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb36815ec40 msgr2=0x7fb368160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb36815ec40 0x7fb368160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 -- 172.21.15.80:0/2933154647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3681422b0 con 0x7fb3680d0510 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36759e640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb368161f90 0x7fb368161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36659c640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb36815ec40 0x7fb368160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb366d9d640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3680d0510 0x7fb368157a20 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb36000e870 tx=0x7fb36000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb34f7fe640 1 -- 172.21.15.80:0/2933154647 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360017070 con 0x7fb3680d0510 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb34f7fe640 1 -- 172.21.15.80:0/2933154647 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb36000ee60 con 0x7fb3680d0510 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb34f7fe640 1 -- 172.21.15.80:0/2933154647 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36000cae0 con 0x7fb3680d0510 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3681427b0 con 0x7fb3680d0510 2024-04-25T13:47:56.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.130+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb368142d70 con 0x7fb3680d0510 2024-04-25T13:47:56.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.134+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb36806a3a0 con 0x7fb3680d0510 2024-04-25T13:47:56.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.134+0000 7fb34f7fe640 1 -- 172.21.15.80:0/2933154647 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb36001a070 con 0x7fb3680d0510 2024-04-25T13:47:56.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.134+0000 7fb34f7fe640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb35406f400 0x7fb354071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.134+0000 7fb34f7fe640 1 -- 172.21.15.80:0/2933154647 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5767+0+0 (secure 0 0 0) 0x7fb36008f420 con 0x7fb3680d0510 2024-04-25T13:47:56.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.134+0000 7fb36659c640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb35406f400 0x7fb354071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.134+0000 7fb36659c640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb35406f400 0x7fb354071910 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb368145380 tx=0x7fb35c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.138+0000 7fb34f7fe640 1 -- 172.21.15.80:0/2933154647 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb36005c360 con 0x7fb3680d0510 2024-04-25T13:47:56.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb36815a7b0 con 0x7fb3680d0510 2024-04-25T13:47:56.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb34f7fe640 1 -- 172.21.15.80:0/2933154647 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) ==== 94+0+5183 (secure 0 0 0) 0x7fb36005bc60 con 0x7fb3680d0510 2024-04-25T13:47:56.336 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:56.336 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":67,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[{"mdsmap":{"epoch":67,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:47:10.054954+0000","modified":"2024-04-25T13:47:13.107291+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7968},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7968":{"gid":7968,"name":"a","rank":0,"incarnation":64,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1563317639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1563317639},{"type":"v1","addr":"172.21.15.80:6835","nonce":1563317639}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7968,"qdb_cluster":[7968]},"id":8}]} 2024-04-25T13:47:56.337 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 67 2024-04-25T13:47:56.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb35406f400 msgr2=0x7fb354071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb35406f400 0x7fb354071910 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb368145380 tx=0x7fb35c009040 comp rx=0 tx=0).stop 2024-04-25T13:47:56.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3680d0510 msgr2=0x7fb368157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3680d0510 0x7fb368157a20 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb36000e870 tx=0x7fb36000ed30 comp rx=0 tx=0).stop 2024-04-25T13:47:56.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 shutdown_connections 2024-04-25T13:47:56.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb35406f400 0x7fb354071910 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb368161f90 0x7fb368161250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb36815ec40 0x7fb368160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 --2- 172.21.15.80:0/2933154647 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb3680d0510 0x7fb368157a20 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 >> 172.21.15.80:0/2933154647 conn(0x7fb36806d080 msgr2=0x7fb3680b5010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:56.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 shutdown_connections 2024-04-25T13:47:56.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.338+0000 7fb36d1c3640 1 -- 172.21.15.80:0/2933154647 wait complete. 2024-04-25T13:47:56.358 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:47:56.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f7607c5c640 1 Processor -- start 2024-04-25T13:47:56.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f7607c5c640 1 -- start start 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f7607c5c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76000d45b0 0x7f76000b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f7607c5c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f76000d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f7607c5c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f76000d1870 0x7f76000d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f7607c5c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f76000d49c0 con 0x7f76000d1870 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f7607c5c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f76000cf980 con 0x7f76000d45b0 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f7607c5c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76000b6d80 con 0x7f76000d0160 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f76061f9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f76000d1870 0x7f76000d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f76061f9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f76000d1870 0x7f76000d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60186/0 (socket says 172.21.15.80:60186) 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f76061f9640 1 -- 172.21.15.80:0/2382707633 learned_addr learned my addr 172.21.15.80:0/2382707633 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f76059f8640 1 --2- 172.21.15.80:0/2382707633 >> v2:172.21.15.87:3300/0 conn(0x7f76000d45b0 0x7f76000b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.502+0000 7f76051f7640 1 --2- 172.21.15.80:0/2382707633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f76000d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76061f9640 1 -- 172.21.15.80:0/2382707633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 msgr2=0x7f76000d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:56.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76061f9640 1 --2- 172.21.15.80:0/2382707633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f76000d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76061f9640 1 -- 172.21.15.80:0/2382707633 >> v2:172.21.15.87:3300/0 conn(0x7f76000d45b0 msgr2=0x7f76000b4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:56.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76061f9640 1 --2- 172.21.15.80:0/2382707633 >> v2:172.21.15.87:3300/0 conn(0x7f76000d45b0 0x7f76000b4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76061f9640 1 -- 172.21.15.80:0/2382707633 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76000d0ab0 con 0x7f76000d1870 2024-04-25T13:47:56.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76051f7640 1 --2- 172.21.15.80:0/2382707633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f76000d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:56.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76059f8640 1 --2- 172.21.15.80:0/2382707633 >> v2:172.21.15.87:3300/0 conn(0x7f76000d45b0 0x7f76000b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:56.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76061f9640 1 --2- 172.21.15.80:0/2382707633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000d1c80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f76000770d0 tx=0x7f75f801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd1ebe37664ff0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76049f6640 1 -- 172.21.15.80:0/2382707633 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f80a56d0 con 0x7f76000d1870 2024-04-25T13:47:56.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76049f6640 1 -- 172.21.15.80:0/2382707633 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f80a5870 con 0x7f76000d1870 2024-04-25T13:47:56.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f76049f6640 1 -- 172.21.15.80:0/2382707633 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f80a5c40 con 0x7f76000d1870 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 -- 172.21.15.80:0/2382707633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 msgr2=0x7f76000d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 --2- 172.21.15.80:0/2382707633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000d1c80 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f76000770d0 tx=0x7f75f801c960 comp rx=0 tx=0).stop 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 -- 172.21.15.80:0/2382707633 shutdown_connections 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 --2- 172.21.15.80:0/2382707633 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000d1c80 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 --2- 172.21.15.80:0/2382707633 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f76000d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 --2- 172.21.15.80:0/2382707633 >> v2:172.21.15.87:3300/0 conn(0x7f76000d45b0 0x7f76000b4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 -- 172.21.15.80:0/2382707633 >> 172.21.15.80:0/2382707633 conn(0x7f76000d3a90 msgr2=0x7f76000d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 -- 172.21.15.80:0/2382707633 shutdown_connections 2024-04-25T13:47:56.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.506+0000 7f7607c5c640 1 -- 172.21.15.80:0/2382707633 wait complete. 2024-04-25T13:47:56.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.510+0000 7f7607c5c640 1 Processor -- start 2024-04-25T13:47:56.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.510+0000 7f7607c5c640 1 -- start start 2024-04-25T13:47:56.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.510+0000 7f7607c5c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f7600150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.510+0000 7f7607c5c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.510+0000 7f7607c5c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000d45b0 0x7f76001601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.510+0000 7f7607c5c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f76000d25d0 con 0x7f76000d1870 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.510+0000 7f7607c5c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f76000cf070 con 0x7f76000d45b0 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.510+0000 7f7607c5c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f76000b7a20 con 0x7f76000d0160 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000cd3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60192/0 (socket says 172.21.15.80:60192) 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 -- 172.21.15.80:0/3093005098 learned_addr learned my addr 172.21.15.80:0/3093005098 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76059f8640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f7600150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 -- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 msgr2=0x7f7600150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76061f9640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000d45b0 0x7f76001601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f7600150c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 -- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000d45b0 msgr2=0x7f76001601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000d45b0 0x7f76001601a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 -- 172.21.15.80:0/3093005098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76000cd900 con 0x7f76000d1870 2024-04-25T13:47:56.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76051f7640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000cd3c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c7f0 tx=0x7f75f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f75f67fc640 1 -- 172.21.15.80:0/3093005098 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0015070 con 0x7f76000d1870 2024-04-25T13:47:56.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f75f67fc640 1 -- 172.21.15.80:0/3093005098 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f00077c0 con 0x7f76000d1870 2024-04-25T13:47:56.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76001557d0 con 0x7f76000d1870 2024-04-25T13:47:56.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f75f67fc640 1 -- 172.21.15.80:0/3093005098 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0007bd0 con 0x7f76000d1870 2024-04-25T13:47:56.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76000cba80 con 0x7f76000d1870 2024-04-25T13:47:56.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760006a3a0 con 0x7f76000d1870 2024-04-25T13:47:56.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f75f67fc640 1 -- 172.21.15.80:0/3093005098 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f75f0018070 con 0x7f76000d1870 2024-04-25T13:47:56.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f75f67fc640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f75e406f400 0x7f75e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f75f67fc640 1 -- 172.21.15.80:0/3093005098 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5767+0+0 (secure 0 0 0) 0x7f75f008f310 con 0x7f76000d1870 2024-04-25T13:47:56.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f75f67fc640 1 -- 172.21.15.80:0/3093005098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=143}) -- 0x7f75e4077040 con 0x7f76000d1870 2024-04-25T13:47:56.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.514+0000 7f76059f8640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f75e406f400 0x7f75e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.518+0000 7f76059f8640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f75e406f400 0x7f75e4071910 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00b7e0 tx=0x7f75fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.518+0000 7f75f67fc640 1 -- 172.21.15.80:0/3093005098 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f75f005c250 con 0x7f76000d1870 2024-04-25T13:47:56.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.682+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76000b61a0 con 0x7f76000d1870 2024-04-25T13:47:56.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.682+0000 7f75f67fc640 1 -- 172.21.15.80:0/3093005098 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) ==== 74+0+16735 (secure 0 0 0) 0x7f75f005bb50 con 0x7f76000d1870 2024-04-25T13:47:56.681 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:47:56.681 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":142,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:47:56.226280+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:47:08.431048+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T13:47:09.003237+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.c","mappings":[{"from":0,"to":5}]},{"pgid":"17.30","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f75e406f400 msgr2=0x7f75e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f75e406f400 0x7f75e4071910 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00b7e0 tx=0x7f75fc008040 comp rx=0 tx=0).stop 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 msgr2=0x7f76000cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000cd3c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f75f000c7f0 tx=0x7f75f000ccb0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 shutdown_connections 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f75e406f400 0x7f75e4071910 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f76000d45b0 0x7f76001601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f76000d1870 0x7f76000cd3c0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 --2- 172.21.15.80:0/3093005098 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76000d0160 0x7f7600150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 >> 172.21.15.80:0/3093005098 conn(0x7f76000d3a90 msgr2=0x7f7600070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:56.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 shutdown_connections 2024-04-25T13:47:56.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.686+0000 7f7607c5c640 1 -- 172.21.15.80:0/3093005098 wait complete. 2024-04-25T13:47:56.705 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:47:56.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6db5f15640 1 Processor -- start 2024-04-25T13:47:56.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6db5f15640 1 -- start start 2024-04-25T13:47:56.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6db5f15640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6db00b7e70 0x7f6db00d1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6db5f15640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 0x7f6db00b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6db5f15640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6db00d4e00 con 0x7f6db01639e0 2024-04-25T13:47:56.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6db5f15640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6db00d1a70 con 0x7f6db00b7e70 2024-04-25T13:47:56.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6db5f15640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6db00d5a10 con 0x7f6db00b9770 2024-04-25T13:47:56.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6daeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6db00b7e70 0x7f6db00d1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6daeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6db00b7e70 0x7f6db00d1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60202/0 (socket says 172.21.15.80:60202) 2024-04-25T13:47:56.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6daeffd640 1 -- 172.21.15.80:0/3248720736 learned_addr learned my addr 172.21.15.80:0/3248720736 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:56.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6daffff640 1 --2- 172.21.15.80:0/3248720736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 0x7f6db00b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6daeffd640 1 -- 172.21.15.80:0/3248720736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 msgr2=0x7f6db00b9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:56.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6daeffd640 1 --2- 172.21.15.80:0/3248720736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 0x7f6db00b9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6daeffd640 1 -- 172.21.15.80:0/3248720736 >> v1:172.21.15.87:6789/0 conn(0x7f6db01639e0 legacy=0x7f6db00b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.854+0000 7f6daeffd640 1 -- 172.21.15.80:0/3248720736 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db00d50e0 con 0x7f6db00b7e70 2024-04-25T13:47:56.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6daeffd640 1 --2- 172.21.15.80:0/3248720736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db00b7e70 0x7f6db00d1530 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009870 tx=0x7f6da401c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66856eedea578c34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6dae7fc640 1 -- 172.21.15.80:0/3248720736 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da40b2070 con 0x7f6db00b7e70 2024-04-25T13:47:56.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6dae7fc640 1 -- 172.21.15.80:0/3248720736 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da40a9070 con 0x7f6db00b7e70 2024-04-25T13:47:56.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6dae7fc640 1 -- 172.21.15.80:0/3248720736 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da40ad880 con 0x7f6db00b7e70 2024-04-25T13:47:56.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6db5f15640 1 -- 172.21.15.80:0/3248720736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db00b7e70 msgr2=0x7f6db00d1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6db5f15640 1 --2- 172.21.15.80:0/3248720736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db00b7e70 0x7f6db00d1530 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f6da4009870 tx=0x7f6da401c560 comp rx=0 tx=0).stop 2024-04-25T13:47:56.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6db5f15640 1 -- 172.21.15.80:0/3248720736 shutdown_connections 2024-04-25T13:47:56.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6db5f15640 1 --2- 172.21.15.80:0/3248720736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 0x7f6db00b9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6db5f15640 1 --2- 172.21.15.80:0/3248720736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db00b7e70 0x7f6db00d1530 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6db5f15640 1 -- 172.21.15.80:0/3248720736 >> 172.21.15.80:0/3248720736 conn(0x7f6db006c9f0 msgr2=0x7f6db006ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:56.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6db5f15640 1 -- 172.21.15.80:0/3248720736 shutdown_connections 2024-04-25T13:47:56.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.858+0000 7f6db5f15640 1 -- 172.21.15.80:0/3248720736 wait complete. 2024-04-25T13:47:56.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 Processor -- start 2024-04-25T13:47:56.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 -- start start 2024-04-25T13:47:56.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 0x7f6db00b6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db01639e0 0x7f6db0153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6db00b6780 0x7f6db0154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6db016d1f0 con 0x7f6db01639e0 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6db016d070 con 0x7f6db00b6780 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6db016d370 con 0x7f6db00b9770 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db01639e0 0x7f6db0153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db01639e0 0x7f6db0153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60204/0 (socket says 172.21.15.80:60204) 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 -- 172.21.15.80:0/386868142 learned_addr learned my addr 172.21.15.80:0/386868142 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 -- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 msgr2=0x7f6db00b6240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 0x7f6db00b6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 -- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6db00b6780 msgr2=0x7f6db0154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6db00b6780 0x7f6db0154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 -- 172.21.15.80:0/386868142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db00d5b90 con 0x7f6db01639e0 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6daeffd640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db01639e0 0x7f6db0153f40 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6da401a190 tx=0x7f6da4019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da40bb070 con 0x7f6db01639e0 2024-04-25T13:47:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6da40adce0 con 0x7f6db01639e0 2024-04-25T13:47:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db00b8090 con 0x7f6db01639e0 2024-04-25T13:47:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da40b2040 con 0x7f6db01639e0 2024-04-25T13:47:56.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db00ceff0 con 0x7f6db01639e0 2024-04-25T13:47:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.862+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db006a3a0 con 0x7f6db01639e0 2024-04-25T13:47:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.866+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6da40be070 con 0x7f6db01639e0 2024-04-25T13:47:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.866+0000 7f6d97fff640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d9c06f400 0x7f6d9c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.866+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5767+0+0 (secure 0 0 0) 0x7f6da4128270 con 0x7f6db01639e0 2024-04-25T13:47:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.866+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=143}) -- 0x7f6d9c0770a0 con 0x7f6db01639e0 2024-04-25T13:47:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.866+0000 7f6daf7fe640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d9c06f400 0x7f6d9c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:56.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.866+0000 7f6daf7fe640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d9c06f400 0x7f6d9c071910 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6da0007fd0 tx=0x7f6da0007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:56.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:56.870+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6da4016150 con 0x7f6db01639e0 2024-04-25T13:47:57.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.062+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f6db0159360 con 0x7f6db01639e0 2024-04-25T13:47:57.060 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:47:57.062+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:47:57.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.062+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v68) ==== 161+0+0 (secure 0 0 0) 0x7f6da40f5080 con 0x7f6db01639e0 2024-04-25T13:47:57.062 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:47:57.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.066+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 <== mon.0 v2:172.21.15.80:3300/0 8 ==== osd_map(143..143 src has 1..143) ==== 374+0+0 (secure 0 0 0) 0x7f6da41267d0 con 0x7f6db01639e0 2024-04-25T13:47:57.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.066+0000 7f6d97fff640 1 -- 172.21.15.80:0/386868142 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=144}) -- 0x7f6d9c077a70 con 0x7f6db01639e0 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d9c06f400 msgr2=0x7f6d9c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d9c06f400 0x7f6d9c071910 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6da0007fd0 tx=0x7f6da0007d60 comp rx=0 tx=0).stop 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db01639e0 msgr2=0x7f6db0153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db01639e0 0x7f6db0153f40 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f6da401a190 tx=0x7f6da4019300 comp rx=0 tx=0).stop 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 shutdown_connections 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d9c06f400 0x7f6d9c071910 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6db00b6780 0x7f6db0154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6db01639e0 0x7f6db0153f40 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 --2- 172.21.15.80:0/386868142 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6db00b9770 0x7f6db00b6240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 >> 172.21.15.80:0/386868142 conn(0x7f6db006c9f0 msgr2=0x7f6db0161700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 shutdown_connections 2024-04-25T13:47:57.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.070+0000 7f6db5f15640 1 -- 172.21.15.80:0/386868142 wait complete. 2024-04-25T13:47:57.076 INFO:tasks.ceph.mds.a.smithi080.stderr: -8550> 2024-04-25T13:47:11.642+0000 7f3275a83640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:47:57.090 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:47:57.162 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:47:57.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.246+0000 7ff425df1640 1 Processor -- start 2024-04-25T13:47:57.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.246+0000 7ff425df1640 1 -- start start 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.246+0000 7ff425df1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff420071e60 0x7ff4201574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.246+0000 7ff425df1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff420161170 0x7ff4200b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.246+0000 7ff425df1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff42006fbf0 con 0x7ff420073c40 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.246+0000 7ff425df1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff4200b1350 con 0x7ff420161170 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.246+0000 7ff425df1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff42006f9a0 con 0x7ff420071e60 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff420071e60 0x7ff4201574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41ffff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff420161170 0x7ff4200b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41effd640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7ff420073c40 0x7ff420160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:35252/0 (socket says 172.21.15.80:35252) 2024-04-25T13:47:57.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41effd640 1 -- 172.21.15.80:0/368564329 learned_addr learned my addr 172.21.15.80:0/368564329 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:57.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41ffff640 1 -- 172.21.15.80:0/368564329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff420071e60 msgr2=0x7ff4201574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:57.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41ffff640 1 --2- 172.21.15.80:0/368564329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff420071e60 0x7ff4201574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41ffff640 1 -- 172.21.15.80:0/368564329 >> v1:172.21.15.87:6789/0 conn(0x7ff420073c40 legacy=0x7ff420160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:57.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41ffff640 1 -- 172.21.15.80:0/368564329 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4200b0a20 con 0x7ff420161170 2024-04-25T13:47:57.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41f7fe640 1 --2- 172.21.15.80:0/368564329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff420071e60 0x7ff4201574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:57.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41ffff640 1 --2- 172.21.15.80:0/368564329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420161170 0x7ff4200b4ba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff418009d90 tx=0x7ff41801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cfbc0ac146011e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:57.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41e7fc640 1 -- 172.21.15.80:0/368564329 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180b2070 con 0x7ff420161170 2024-04-25T13:47:57.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41e7fc640 1 -- 172.21.15.80:0/368564329 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4180a5920 con 0x7ff420161170 2024-04-25T13:47:57.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.250+0000 7ff41e7fc640 1 -- 172.21.15.80:0/368564329 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4180a5cf0 con 0x7ff420161170 2024-04-25T13:47:57.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- 172.21.15.80:0/368564329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420161170 msgr2=0x7ff4200b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:57.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 --2- 172.21.15.80:0/368564329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420161170 0x7ff4200b4ba0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7ff418009d90 tx=0x7ff41801c970 comp rx=0 tx=0).stop 2024-04-25T13:47:57.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- 172.21.15.80:0/368564329 shutdown_connections 2024-04-25T13:47:57.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 --2- 172.21.15.80:0/368564329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420161170 0x7ff4200b4ba0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 --2- 172.21.15.80:0/368564329 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff420071e60 0x7ff4201574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- 172.21.15.80:0/368564329 >> 172.21.15.80:0/368564329 conn(0x7ff4200b4160 msgr2=0x7ff4200d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:57.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- 172.21.15.80:0/368564329 shutdown_connections 2024-04-25T13:47:57.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- 172.21.15.80:0/368564329 wait complete. 2024-04-25T13:47:57.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 Processor -- start 2024-04-25T13:47:57.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- start start 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff420071e60 0x7ff42014b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420073c40 0x7ff4201502f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42014cf90 0x7ff42014d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff420169bb0 con 0x7ff420071e60 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff420074050 con 0x7ff420073c40 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff425df1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff420169d30 con 0x7ff42014cf90 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff420071e60 0x7ff42014b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff41f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff420071e60 0x7ff42014b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60224/0 (socket says 172.21.15.80:60224) 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.254+0000 7ff41f7fe640 1 -- 172.21.15.80:0/1516474914 learned_addr learned my addr 172.21.15.80:0/1516474914 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41ffff640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42014cf90 0x7ff42014d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41effd640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420073c40 0x7ff4201502f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41f7fe640 1 -- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42014cf90 msgr2=0x7ff42014d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:57.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41f7fe640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42014cf90 0x7ff42014d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41f7fe640 1 -- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420073c40 msgr2=0x7ff4201502f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41f7fe640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420073c40 0x7ff4201502f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41f7fe640 1 -- 172.21.15.80:0/1516474914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff42014d8e0 con 0x7ff420071e60 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41ffff640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42014cf90 0x7ff42014d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41effd640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420073c40 0x7ff4201502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41f7fe640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff420071e60 0x7ff42014b9e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e9b0 tx=0x7ff40800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff417fff640 1 -- 172.21.15.80:0/1516474914 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff408017070 con 0x7ff420071e60 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff417fff640 1 -- 172.21.15.80:0/1516474914 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff40800ca00 con 0x7ff420071e60 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff42014bf20 con 0x7ff420071e60 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff417fff640 1 -- 172.21.15.80:0/1516474914 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40800cdd0 con 0x7ff420071e60 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff42014f500 con 0x7ff420071e60 2024-04-25T13:47:57.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42006a3a0 con 0x7ff420071e60 2024-04-25T13:47:57.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff417fff640 1 -- 172.21.15.80:0/1516474914 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff40801a070 con 0x7ff420071e60 2024-04-25T13:47:57.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff417fff640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff40006f400 0x7ff400071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:57.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff417fff640 1 -- 172.21.15.80:0/1516474914 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5853+0+0 (secure 0 0 0) 0x7ff40808f860 con 0x7ff420071e60 2024-04-25T13:47:57.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff417fff640 1 -- 172.21.15.80:0/1516474914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=144}) -- 0x7ff400077150 con 0x7ff420071e60 2024-04-25T13:47:57.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.258+0000 7ff41effd640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff40006f400 0x7ff400071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:57.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.262+0000 7ff41effd640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff40006f400 0x7ff400071910 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff420072580 tx=0x7ff410009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:57.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.262+0000 7ff417fff640 1 -- 172.21.15.80:0/1516474914 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff42014d8e0 con 0x7ff420071e60 2024-04-25T13:47:57.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:57.458+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7ff420153770 con 0x7ff420071e60 2024-04-25T13:47:58.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.066+0000 7ff417fff640 1 -- 172.21.15.80:0/1516474914 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v69) ==== 104+0+0 (secure 0 0 0) 0x7ff40805c830 con 0x7ff420071e60 2024-04-25T13:47:58.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff40006f400 msgr2=0x7ff400071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff40006f400 0x7ff400071910 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff420072580 tx=0x7ff410009040 comp rx=0 tx=0).stop 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff420071e60 msgr2=0x7ff42014b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff420071e60 0x7ff42014b9e0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e9b0 tx=0x7ff40800ee70 comp rx=0 tx=0).stop 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 shutdown_connections 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff40006f400 0x7ff400071910 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff42014cf90 0x7ff42014d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff420073c40 0x7ff4201502f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 --2- 172.21.15.80:0/1516474914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff420071e60 0x7ff42014b9e0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 >> 172.21.15.80:0/1516474914 conn(0x7ff4200b4160 msgr2=0x7ff4200b47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 shutdown_connections 2024-04-25T13:47:58.068 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.070+0000 7ff425df1640 1 -- 172.21.15.80:0/1516474914 wait complete. 2024-04-25T13:47:58.095 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:47:58.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.254+0000 7f92ce37b640 1 Processor -- start 2024-04-25T13:47:58.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.254+0000 7f92ce37b640 1 -- start start 2024-04-25T13:47:58.256 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92ce37b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92c80d0ed0 0x7f92c80d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92ce37b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f92c80d4820 0x7f92c80d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92ce37b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92c80d1820 con 0x7f92c80d24a0 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92ce37b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f92c8164040 con 0x7f92c80d0ed0 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92ce37b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f92c8164bc0 con 0x7f92c80d4820 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92c7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92c80d0ed0 0x7f92c80d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92c77fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f92c80d24a0 0x7f92c80d28b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54102/0 (socket says 172.21.15.80:54102) 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92c7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92c80d0ed0 0x7f92c80d12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60234/0 (socket says 172.21.15.80:60234) 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92c77fe640 1 -- 172.21.15.80:0/2747252741 learned_addr learned my addr 172.21.15.80:0/2747252741 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92cc918640 1 --2- 172.21.15.80:0/2747252741 >> v2:172.21.15.87:3300/0 conn(0x7f92c80d4820 0x7f92c80d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92c7fff640 1 -- 172.21.15.80:0/2747252741 >> v2:172.21.15.87:3300/0 conn(0x7f92c80d4820 msgr2=0x7f92c80d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92c7fff640 1 --2- 172.21.15.80:0/2747252741 >> v2:172.21.15.87:3300/0 conn(0x7f92c80d4820 0x7f92c80d4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92c7fff640 1 -- 172.21.15.80:0/2747252741 >> v1:172.21.15.80:6789/0 conn(0x7f92c80d24a0 legacy=0x7f92c80d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:58.257 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.258+0000 7f92c7fff640 1 -- 172.21.15.80:0/2747252741 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c80d2fc0 con 0x7f92c80d0ed0 2024-04-25T13:47:58.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92c7fff640 1 --2- 172.21.15.80:0/2747252741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c80d12e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f92c80d3e80 tx=0x7f92b801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a966989277c3b2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:58.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92c6ffd640 1 -- 172.21.15.80:0/2747252741 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b80ab650 con 0x7f92c80d0ed0 2024-04-25T13:47:58.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92c6ffd640 1 -- 172.21.15.80:0/2747252741 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b80b0070 con 0x7f92c80d0ed0 2024-04-25T13:47:58.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92c6ffd640 1 -- 172.21.15.80:0/2747252741 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b80aba10 con 0x7f92c80d0ed0 2024-04-25T13:47:58.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 -- 172.21.15.80:0/2747252741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 msgr2=0x7f92c80d12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:58.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 --2- 172.21.15.80:0/2747252741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c80d12e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f92c80d3e80 tx=0x7f92b801c450 comp rx=0 tx=0).stop 2024-04-25T13:47:58.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 -- 172.21.15.80:0/2747252741 shutdown_connections 2024-04-25T13:47:58.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 --2- 172.21.15.80:0/2747252741 >> v2:172.21.15.87:3300/0 conn(0x7f92c80d4820 0x7f92c80d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 --2- 172.21.15.80:0/2747252741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c80d12e0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 -- 172.21.15.80:0/2747252741 >> 172.21.15.80:0/2747252741 conn(0x7f92c806c5f0 msgr2=0x7f92c806ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:58.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 -- 172.21.15.80:0/2747252741 shutdown_connections 2024-04-25T13:47:58.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 -- 172.21.15.80:0/2747252741 wait complete. 2024-04-25T13:47:58.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 Processor -- start 2024-04-25T13:47:58.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.262+0000 7f92ce37b640 1 -- start start 2024-04-25T13:47:58.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c8162090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:58.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92c80d24a0 0x7f92c814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:58.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92c80d4820 0x7f92c80b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f92c816d3e0 con 0x7f92c80d0ed0 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f92c8157820 con 0x7f92c80d4820 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92c816d560 con 0x7f92c80d24a0 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c8162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c8162090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60250/0 (socket says 172.21.15.80:60250) 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 -- 172.21.15.80:0/7049180 learned_addr learned my addr 172.21.15.80:0/7049180 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92cc918640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92c80d4820 0x7f92c80b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 -- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92c80d24a0 msgr2=0x7f92c814a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c77fe640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92c80d24a0 0x7f92c814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92c80d24a0 0x7f92c814a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 -- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92c80d4820 msgr2=0x7f92c80b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:58.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92c80d4820 0x7f92c80b9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 -- 172.21.15.80:0/7049180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c815ac70 con 0x7f92c80d0ed0 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92cc918640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92c80d4820 0x7f92c80b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c7fff640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c8162090 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f92b80ad040 tx=0x7f92b801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b80bf610 con 0x7f92c80d0ed0 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b80bf7b0 con 0x7f92c80d0ed0 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c8162a70 con 0x7f92c80d0ed0 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b80bfb80 con 0x7f92c80d0ed0 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c8157eb0 con 0x7f92c80d0ed0 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92c806a440 con 0x7f92c80d0ed0 2024-04-25T13:47:58.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f92b80bd020 con 0x7f92c80d0ed0 2024-04-25T13:47:58.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.266+0000 7f92c4ff9640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92b406f400 0x7f92b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:58.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.270+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5853+0+0 (secure 0 0 0) 0x7f92b8129f70 con 0x7f92c80d0ed0 2024-04-25T13:47:58.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.270+0000 7f92c77fe640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92b406f400 0x7f92b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:58.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.270+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=144}) -- 0x7f92b4077140 con 0x7f92c80d0ed0 2024-04-25T13:47:58.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.270+0000 7f92c77fe640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92b406f400 0x7f92b4071910 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f92c8076180 tx=0x7f92bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:58.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.274+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92b8016070 con 0x7f92c80d0ed0 2024-04-25T13:47:58.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:58.430+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f92c816b1b0 con 0x7f92c80d0ed0 2024-04-25T13:47:59.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.078+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v144) ==== 181+0+0 (secure 0 0 0) 0x7f92b80f6e60 con 0x7f92c80d0ed0 2024-04-25T13:47:59.077 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:47:59.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.078+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 <== mon.0 v2:172.21.15.80:3300/0 8 ==== osd_map(144..144 src has 1..144) ==== 296+0+0 (secure 0 0 0) 0x7f92b80fde00 con 0x7f92c80d0ed0 2024-04-25T13:47:59.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.078+0000 7f92c4ff9640 1 -- 172.21.15.80:0/7049180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=145}) -- 0x7f92b4075570 con 0x7f92c80d0ed0 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92b406f400 msgr2=0x7f92b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92b406f400 0x7f92b4071910 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f92c8076180 tx=0x7f92bc009040 comp rx=0 tx=0).stop 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 msgr2=0x7f92c8162090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c8162090 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f92b80ad040 tx=0x7f92b801ac90 comp rx=0 tx=0).stop 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 shutdown_connections 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92b406f400 0x7f92b4071910 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92c80d4820 0x7f92c80b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92c80d24a0 0x7f92c814a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 --2- 172.21.15.80:0/7049180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92c80d0ed0 0x7f92c8162090 secure :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f92b80ad040 tx=0x7f92b801ac90 comp rx=0 tx=0).stop 2024-04-25T13:47:59.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 >> 172.21.15.80:0/7049180 conn(0x7f92c806c5f0 msgr2=0x7f92c815e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:59.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.086+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 shutdown_connections 2024-04-25T13:47:59.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.090+0000 7f92ce37b640 1 -- 172.21.15.80:0/7049180 wait complete. 2024-04-25T13:47:59.111 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:47:59.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.262+0000 7f98c6cf2640 1 Processor -- start 2024-04-25T13:47:59.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.262+0000 7f98c6cf2640 1 -- start start 2024-04-25T13:47:59.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.262+0000 7f98c6cf2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f98c00b0f10 0x7f98c00d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.262+0000 7f98c6cf2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00ce2a0 0x7f98c00ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.262+0000 7f98c6cf2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f98c00b1320 con 0x7f98c00cf730 2024-04-25T13:47:59.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.262+0000 7f98c6cf2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f98c00b5820 con 0x7f98c00b0f10 2024-04-25T13:47:59.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.262+0000 7f98c6cf2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f98c00d13f0 con 0x7f98c00ce2a0 2024-04-25T13:47:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98c4a8e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f98c00b0f10 0x7f98c00d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bffff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00ce2a0 0x7f98c00ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98c528f640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f98c00cf730 0x7f98c00cfb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:35260/0 (socket says 172.21.15.80:35260) 2024-04-25T13:47:59.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98c528f640 1 -- 172.21.15.80:0/1408519490 learned_addr learned my addr 172.21.15.80:0/1408519490 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:47:59.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bffff640 1 -- 172.21.15.80:0/1408519490 >> v2:172.21.15.87:3300/0 conn(0x7f98c00b0f10 msgr2=0x7f98c00d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:47:59.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bffff640 1 --2- 172.21.15.80:0/1408519490 >> v2:172.21.15.87:3300/0 conn(0x7f98c00b0f10 0x7f98c00d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bffff640 1 -- 172.21.15.80:0/1408519490 >> v1:172.21.15.87:6789/0 conn(0x7f98c00cf730 legacy=0x7f98c00cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bffff640 1 -- 172.21.15.80:0/1408519490 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c00cebf0 con 0x7f98c00ce2a0 2024-04-25T13:47:59.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98c4a8e640 1 --2- 172.21.15.80:0/1408519490 >> v2:172.21.15.87:3300/0 conn(0x7f98c00b0f10 0x7f98c00d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:47:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bffff640 1 --2- 172.21.15.80:0/1408519490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00ce2a0 0x7f98c00ce6b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98b40097c0 tx=0x7f98b401c4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec5023e151bc481 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bf7fe640 1 -- 172.21.15.80:0/1408519490 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40b1070 con 0x7f98c00ce2a0 2024-04-25T13:47:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bf7fe640 1 -- 172.21.15.80:0/1408519490 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b40ac410 con 0x7f98c00ce2a0 2024-04-25T13:47:59.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.266+0000 7f98bf7fe640 1 -- 172.21.15.80:0/1408519490 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40ac7e0 con 0x7f98c00ce2a0 2024-04-25T13:47:59.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- 172.21.15.80:0/1408519490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00ce2a0 msgr2=0x7f98c00ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/1408519490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00ce2a0 0x7f98c00ce6b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f98b40097c0 tx=0x7f98b401c4a0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- 172.21.15.80:0/1408519490 shutdown_connections 2024-04-25T13:47:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/1408519490 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00ce2a0 0x7f98c00ce6b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/1408519490 >> v2:172.21.15.87:3300/0 conn(0x7f98c00b0f10 0x7f98c00d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- 172.21.15.80:0/1408519490 >> 172.21.15.80:0/1408519490 conn(0x7f98c00b35c0 msgr2=0x7f98c00b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:47:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- 172.21.15.80:0/1408519490 shutdown_connections 2024-04-25T13:47:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- 172.21.15.80:0/1408519490 wait complete. 2024-04-25T13:47:59.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 Processor -- start 2024-04-25T13:47:59.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- start start 2024-04-25T13:47:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00b0f10 0x7f98c014c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c00ce2a0 0x7f98c0161820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00cf730 0x7f98c0161d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f98c00b5300 con 0x7f98c00ce2a0 2024-04-25T13:47:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f98c00d1150 con 0x7f98c00b0f10 2024-04-25T13:47:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c6cf2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f98c015e720 con 0x7f98c00cf730 2024-04-25T13:47:59.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98bffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c00ce2a0 0x7f98c0161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98bffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c00ce2a0 0x7f98c0161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60256/0 (socket says 172.21.15.80:60256) 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98bffff640 1 -- 172.21.15.80:0/3849617665 learned_addr learned my addr 172.21.15.80:0/3849617665 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c528f640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00cf730 0x7f98c0161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.270+0000 7f98c4a8e640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00b0f10 0x7f98c014c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bffff640 1 -- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00cf730 msgr2=0x7f98c0161d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bffff640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00cf730 0x7f98c0161d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bffff640 1 -- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00b0f10 msgr2=0x7f98c014c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bffff640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00b0f10 0x7f98c014c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bffff640 1 -- 172.21.15.80:0/3849617665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c00b18c0 con 0x7f98c00ce2a0 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bffff640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c00ce2a0 0x7f98c0161820 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f98b401a0d0 tx=0x7f98b4019240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b40b1040 con 0x7f98c00ce2a0 2024-04-25T13:47:59.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f98b40ba070 con 0x7f98c00ce2a0 2024-04-25T13:47:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c00b1ba0 con 0x7f98c00ce2a0 2024-04-25T13:47:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b4006e20 con 0x7f98c00ce2a0 2024-04-25T13:47:59.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c00b2160 con 0x7f98c00ce2a0 2024-04-25T13:47:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c006a440 con 0x7f98c00ce2a0 2024-04-25T13:47:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f98b40b5650 con 0x7f98c00ce2a0 2024-04-25T13:47:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bd7fa640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98ac06f400 0x7f98ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:47:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5415+0+0 (secure 0 0 0) 0x7f98b4129840 con 0x7f98c00ce2a0 2024-04-25T13:47:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=145}) -- 0x7f98ac0762e0 con 0x7f98c00ce2a0 2024-04-25T13:47:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.274+0000 7f98c4a8e640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98ac06f400 0x7f98ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:47:59.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.278+0000 7f98c4a8e640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98ac06f400 0x7f98ac071910 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f98c00d28d0 tx=0x7f98b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:47:59.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.278+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f98c00cebf0 con 0x7f98c00ce2a0 2024-04-25T13:47:59.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:47:59.442+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f98c00b0cf0 con 0x7f98c00ce2a0 2024-04-25T13:47:59.687 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:47:59.690 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:48:00.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.082+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v145) ==== 169+0+0 (secure 0 0 0) 0x7f98b4016090 con 0x7f98c00ce2a0 2024-04-25T13:48:00.079 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:48:00.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.082+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 <== mon.0 v2:172.21.15.80:3300/0 8 ==== osd_map(145..145 src has 1..145) ==== 330+0+0 (secure 0 0 0) 0x7f98b40f2550 con 0x7f98c00ce2a0 2024-04-25T13:48:00.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.082+0000 7f98bd7fa640 1 -- 172.21.15.80:0/3849617665 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=146}) -- 0x7f98ac075070 con 0x7f98c00ce2a0 2024-04-25T13:48:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.090+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98ac06f400 msgr2=0x7f98ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.090+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98ac06f400 0x7f98ac071910 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f98c00d28d0 tx=0x7f98b0008040 comp rx=0 tx=0).stop 2024-04-25T13:48:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.090+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c00ce2a0 msgr2=0x7f98c0161820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.090+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c00ce2a0 0x7f98c0161820 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f98b401a0d0 tx=0x7f98b4019240 comp rx=0 tx=0).stop 2024-04-25T13:48:00.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.094+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 shutdown_connections 2024-04-25T13:48:00.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.094+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f98ac06f400 0x7f98ac071910 secure :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f98c00d28d0 tx=0x7f98b0008040 comp rx=0 tx=0).stop 2024-04-25T13:48:00.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.094+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f98c00cf730 0x7f98c0161d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.094+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f98c00ce2a0 0x7f98c0161820 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.094+0000 7f98c6cf2640 1 --2- 172.21.15.80:0/3849617665 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98c00b0f10 0x7f98c014c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.094+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 >> 172.21.15.80:0/3849617665 conn(0x7f98c00b35c0 msgr2=0x7f98c0072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:00.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.098+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 shutdown_connections 2024-04-25T13:48:00.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.098+0000 7f98c6cf2640 1 -- 172.21.15.80:0/3849617665 wait complete. 2024-04-25T13:48:00.123 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:48:00.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.314+0000 7f5d0bcc0640 1 Processor -- start 2024-04-25T13:48:00.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.314+0000 7f5d0bcc0640 1 -- start start 2024-04-25T13:48:00.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0bcc0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5d0406edb0 0x7f5d0406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0bcc0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d040b5740 con 0x7f5d040b5330 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0bcc0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5d0406dd10 con 0x7f5d04161b60 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0bcc0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5d0406f700 con 0x7f5d0406edb0 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d09a5c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5d040b5330 0x7f5d040d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54110/0 (socket says 172.21.15.80:54110) 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d09a5c640 1 -- 172.21.15.80:0/96485843 learned_addr learned my addr 172.21.15.80:0/96485843 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0925b640 1 --2- 172.21.15.80:0/96485843 >> v2:172.21.15.87:3300/0 conn(0x7f5d0406edb0 0x7f5d0406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1260243407 0 0) 0x7f5d040b5740 con 0x7f5d040b5330 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5cf0003530 con 0x7f5d040b5330 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 370710353 0 0) 0x7f5cf0003530 con 0x7f5d040b5330 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 >> v2:172.21.15.87:3300/0 conn(0x7f5d0406edb0 msgr2=0x7f5d0406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 --2- 172.21.15.80:0/96485843 >> v2:172.21.15.87:3300/0 conn(0x7f5d0406edb0 0x7f5d0406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 >> v1:172.21.15.87:6789/0 conn(0x7f5d04161b60 legacy=0x7f5d04161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d0406dff0 con 0x7f5d040b5330 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f5d00001680 con 0x7f5d040b5330 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 >> v1:172.21.15.80:6789/0 conn(0x7f5d040b5330 legacy=0x7f5d040d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf00039c0 0x7f5cf0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0004810 0x7f5cf0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf0005260 0x7f5cf000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0406f700 con 0x7f5cf0005260 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf000c1f0 con 0x7f5cf00039c0 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5cf000c3f0 con 0x7f5cf0004810 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0a25d640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf0005260 0x7f5cf000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d09a5c640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf00039c0 0x7f5cf0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0925b640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0004810 0x7f5cf0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0a25d640 1 -- 172.21.15.80:0/96485843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0004810 msgr2=0x7f5cf0004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:00.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0a25d640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0004810 0x7f5cf0004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0a25d640 1 -- 172.21.15.80:0/96485843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf00039c0 msgr2=0x7f5cf0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:00.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0a25d640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5cf00039c0 0x7f5cf0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.318+0000 7f5d0a25d640 1 -- 172.21.15.80:0/96485843 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf0003530 con 0x7f5cf0005260 2024-04-25T13:48:00.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0a25d640 1 -- 172.21.15.80:0/96485843 reap_dead start 2024-04-25T13:48:00.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0a25d640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf0005260 0x7f5cf000b9d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc001e70 tx=0x7f5cfc01c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24b8a7939b21941a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:00.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc001a90 con 0x7f5cf0005260 2024-04-25T13:48:00.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cfc003ba0 con 0x7f5cf0005260 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d08a5a640 1 -- 172.21.15.80:0/96485843 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc0050d0 con 0x7f5cf0005260 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/96485843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf0005260 msgr2=0x7f5cf000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf0005260 0x7f5cf000b9d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc001e70 tx=0x7f5cfc01c780 comp rx=0 tx=0).stop 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/96485843 shutdown_connections 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 --2- 172.21.15.80:0/96485843 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5cf0005260 0x7f5cf000b9d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/96485843 >> 172.21.15.80:0/96485843 conn(0x7f5d040b4160 msgr2=0x7f5d040d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/96485843 shutdown_connections 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/96485843 wait complete. 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 Processor -- start 2024-04-25T13:48:00.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.322+0000 7f5d0bcc0640 1 -- start start 2024-04-25T13:48:00.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0005260 0x7f5d04134050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d04135bb0 0x7f5d041362b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d041367f0 0x7f5d041e7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5d0406e5b0 con 0x7f5d041367f0 2024-04-25T13:48:00.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041629b0 con 0x7f5d04135bb0 2024-04-25T13:48:00.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5d0406e890 con 0x7f5cf0005260 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d041367f0 0x7f5d041e7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d041367f0 0x7f5d041e7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60262/0 (socket says 172.21.15.80:60262) 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 -- 172.21.15.80:0/1474948666 learned_addr learned my addr 172.21.15.80:0/1474948666 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0925b640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d04135bb0 0x7f5d041362b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d09a5c640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0005260 0x7f5d04134050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 -- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0005260 msgr2=0x7f5d04134050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0005260 0x7f5d04134050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 -- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d04135bb0 msgr2=0x7f5d041362b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d04135bb0 0x7f5d041362b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 -- 172.21.15.80:0/1474948666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d04134b10 con 0x7f5d041367f0 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0a25d640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d041367f0 0x7f5d041e7df0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc01d020 tx=0x7f5cf0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5cfa7fc640 1 -- 172.21.15.80:0/1474948666 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc001a90 con 0x7f5d041367f0 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5cfa7fc640 1 -- 172.21.15.80:0/1474948666 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cfc0af070 con 0x7f5d041367f0 2024-04-25T13:48:00.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5cfa7fc640 1 -- 172.21.15.80:0/1474948666 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc007dc0 con 0x7f5d041367f0 2024-04-25T13:48:00.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d04134df0 con 0x7f5d041367f0 2024-04-25T13:48:00.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d041353b0 con 0x7f5d041367f0 2024-04-25T13:48:00.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d04134880 con 0x7f5d041367f0 2024-04-25T13:48:00.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.326+0000 7f5cfa7fc640 1 -- 172.21.15.80:0/1474948666 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5cfc0b1020 con 0x7f5d041367f0 2024-04-25T13:48:00.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.330+0000 7f5cfa7fc640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ce806f400 0x7f5ce8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.330+0000 7f5cfa7fc640 1 -- 172.21.15.80:0/1474948666 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f5cfc128880 con 0x7f5d041367f0 2024-04-25T13:48:00.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.330+0000 7f5d09a5c640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ce806f400 0x7f5ce8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.330+0000 7f5d09a5c640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ce806f400 0x7f5ce8071910 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5d00000c10 tx=0x7f5d00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:00.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.334+0000 7f5cfa7fc640 1 -- 172.21.15.80:0/1474948666 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5cfc016490 con 0x7f5d041367f0 2024-04-25T13:48:00.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.494+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f5d0406a3a0 con 0x7f5d041367f0 2024-04-25T13:48:00.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5cfa7fc640 1 -- 172.21.15.80:0/1474948666 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) ==== 104+0+0 (secure 0 0 0) 0x7f5cfc0f5a70 con 0x7f5d041367f0 2024-04-25T13:48:00.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ce806f400 msgr2=0x7f5ce8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ce806f400 0x7f5ce8071910 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f5d00000c10 tx=0x7f5d00009040 comp rx=0 tx=0).stop 2024-04-25T13:48:00.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d041367f0 msgr2=0x7f5d041e7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d041367f0 0x7f5d041e7df0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc01d020 tx=0x7f5cf0005840 comp rx=0 tx=0).stop 2024-04-25T13:48:00.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 shutdown_connections 2024-04-25T13:48:00.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ce806f400 0x7f5ce8071910 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5d041367f0 0x7f5d041e7df0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5d04135bb0 0x7f5d041362b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 --2- 172.21.15.80:0/1474948666 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5cf0005260 0x7f5d04134050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 >> 172.21.15.80:0/1474948666 conn(0x7f5d040b4160 msgr2=0x7f5d04161f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:00.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 shutdown_connections 2024-04-25T13:48:00.503 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.502+0000 7f5d0bcc0640 1 -- 172.21.15.80:0/1474948666 wait complete. 2024-04-25T13:48:00.523 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2024-04-25T13:48:00.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f043acef640 1 Processor -- start 2024-04-25T13:48:00.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f043acef640 1 -- start start 2024-04-25T13:48:00.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f043acef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04340745a0 0x7f043406ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f043acef640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04340cd6c0 con 0x7f04340cc720 2024-04-25T13:48:00.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f043acef640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f04340729e0 con 0x7f043406f170 2024-04-25T13:48:00.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f043acef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f043415ea50 con 0x7f04340745a0 2024-04-25T13:48:00.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f0438a8b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f04340cc720 0x7f0434073e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54114/0 (socket says 172.21.15.80:54114) 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f0438a8b640 1 -- 172.21.15.80:0/1046721468 learned_addr learned my addr 172.21.15.80:0/1046721468 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f0433fff640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04340745a0 0x7f043406ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2679782626 0 0) 0x7f04340cd6c0 con 0x7f04340cc720 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f041c003530 con 0x7f04340cc720 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2926741307 0 0) 0x7f041c003530 con 0x7f04340cc720 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04340745a0 msgr2=0x7f043406ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04340745a0 0x7f043406ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 >> v1:172.21.15.87:6789/0 conn(0x7f043406f170 legacy=0x7f043406f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04340cce20 con 0x7f04340cc720 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0424001680 con 0x7f04340cc720 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 >> v1:172.21.15.80:6789/0 conn(0x7f04340cc720 legacy=0x7f0434073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f041c0039f0 0x7f041c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04340745a0 0x7f041c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f041c004d50 0x7f041c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f043415ea50 con 0x7f041c004d50 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f041c00bb60 con 0x7f041c0039f0 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.678+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f041c00bd30 con 0x7f04340745a0 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043928c640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f041c004d50 0x7f041c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f0433fff640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04340745a0 0x7f041c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f0438a8b640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f041c0039f0 0x7f041c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043928c640 1 -- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04340745a0 msgr2=0x7f041c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043928c640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f04340745a0 0x7f041c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043928c640 1 -- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f041c0039f0 msgr2=0x7f041c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043928c640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f041c0039f0 0x7f041c003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043928c640 1 -- 172.21.15.80:0/1046721468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c003530 con 0x7f041c004d50 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043928c640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f041c004d50 0x7f041c005160 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f04340d31d0 tx=0x7f042c01c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790e3c78c7112a65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c0ac070 con 0x7f041c004d50 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042c0a6610 con 0x7f041c004d50 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f04337fe640 1 -- 172.21.15.80:0/1046721468 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c001b60 con 0x7f041c004d50 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043acef640 1 -- 172.21.15.80:0/1046721468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f041c004d50 msgr2=0x7f041c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043acef640 1 --2- 172.21.15.80:0/1046721468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f041c004d50 0x7f041c005160 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f04340d31d0 tx=0x7f042c01c840 comp rx=0 tx=0).stop 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043928c640 1 -- 172.21.15.80:0/1046721468 reap_dead start 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043acef640 1 -- 172.21.15.80:0/1046721468 shutdown_connections 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043acef640 1 -- 172.21.15.80:0/1046721468 >> 172.21.15.80:0/1046721468 conn(0x7f04340d20d0 msgr2=0x7f04340d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043acef640 1 -- 172.21.15.80:0/1046721468 shutdown_connections 2024-04-25T13:48:00.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.682+0000 7f043acef640 1 -- 172.21.15.80:0/1046721468 wait complete. 2024-04-25T13:48:00.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 Processor -- start 2024-04-25T13:48:00.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 -- start start 2024-04-25T13:48:00.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0434153090 0x7f0434167ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0434163c00 0x7f0434164010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04341595b0 0x7f04341599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f043415f490 con 0x7f04341595b0 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f04340ccb30 con 0x7f0434163c00 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f043415f610 con 0x7f0434153090 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04341595b0 0x7f04341599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04341595b0 0x7f04341599c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:60284/0 (socket says 172.21.15.80:60284) 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 -- 172.21.15.80:0/1594545982 learned_addr learned my addr 172.21.15.80:0/1594545982 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f0433fff640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0434163c00 0x7f0434164010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f0438a8b640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0434153090 0x7f0434167ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 -- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0434153090 msgr2=0x7f0434167ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0434153090 0x7f0434167ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 -- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0434163c00 msgr2=0x7f0434164010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:00.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0434163c00 0x7f0434164010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:00.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 -- 172.21.15.80:0/1594545982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0434164550 con 0x7f04341595b0 2024-04-25T13:48:00.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043928c640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04341595b0 0x7f04341599c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f042c01cd20 tx=0x7f042c0195e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:00.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f04317fa640 1 -- 172.21.15.80:0/1594545982 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c0ac040 con 0x7f04341595b0 2024-04-25T13:48:00.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f04317fa640 1 -- 172.21.15.80:0/1594545982 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f042c0b5020 con 0x7f04341595b0 2024-04-25T13:48:00.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04340d0c60 con 0x7f04341595b0 2024-04-25T13:48:00.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.686+0000 7f04317fa640 1 -- 172.21.15.80:0/1594545982 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f042c001b40 con 0x7f04341595b0 2024-04-25T13:48:00.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.690+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04340d1220 con 0x7f04341595b0 2024-04-25T13:48:00.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.690+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04341547b0 con 0x7f04341595b0 2024-04-25T13:48:00.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.690+0000 7f04317fa640 1 -- 172.21.15.80:0/1594545982 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f042c0af070 con 0x7f04341595b0 2024-04-25T13:48:00.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.690+0000 7f04317fa640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f042006f400 0x7f0420071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:00.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.690+0000 7f04317fa640 1 -- 172.21.15.80:0/1594545982 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f042c128520 con 0x7f04341595b0 2024-04-25T13:48:00.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.690+0000 7f0438a8b640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f042006f400 0x7f0420071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:00.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.690+0000 7f0438a8b640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f042006f400 0x7f0420071910 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0424001ec0 tx=0x7f0424009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:00.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.694+0000 7f04317fa640 1 -- 172.21.15.80:0/1594545982 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f042c0bf030 con 0x7f04341595b0 2024-04-25T13:48:00.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:00.918+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) -- 0x7f043406a3a0 con 0x7f04341595b0 2024-04-25T13:48:01.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.466+0000 7f04317fa640 1 -- 172.21.15.80:0/1594545982 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v401) ==== 140+0+0 (secure 0 0 0) 0x7f0434164550 con 0x7f04341595b0 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f042006f400 msgr2=0x7f0420071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f042006f400 0x7f0420071910 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0424001ec0 tx=0x7f0424009040 comp rx=0 tx=0).stop 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04341595b0 msgr2=0x7f04341599c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04341595b0 0x7f04341599c0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f042c01cd20 tx=0x7f042c0195e0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 shutdown_connections 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f042006f400 0x7f0420071910 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f04341595b0 0x7f04341599c0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0434163c00 0x7f0434164010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 --2- 172.21.15.80:0/1594545982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0434153090 0x7f0434167ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 >> 172.21.15.80:0/1594545982 conn(0x7f04340d20d0 msgr2=0x7f04340cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:01.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 shutdown_connections 2024-04-25T13:48:01.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.470+0000 7f043acef640 1 -- 172.21.15.80:0/1594545982 wait complete. 2024-04-25T13:48:01.491 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-04-25T13:48:01.491 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2024-04-25T13:48:01.492 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:48:01.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcce5f7640 1 Processor -- start 2024-04-25T13:48:01.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcce5f7640 1 -- start start 2024-04-25T13:48:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcce5f7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdcc8161f90 0x7fdcc80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcce5f7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdcc80d0510 0x7fdcc815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcce5f7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdcc80d0d70 con 0x7fdcc815ea70 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcce5f7640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc80d1050 con 0x7fdcc8161f90 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcce5f7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdcc806dca0 con 0x7fdcc80d0510 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcc7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdcc8161f90 0x7fdcc80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcc7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdcc8161f90 0x7fdcc80cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53508/0 (socket says 172.21.15.80:53508) 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcc7fff640 1 -- 172.21.15.80:0/2765403386 learned_addr learned my addr 172.21.15.80:0/2765403386 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcc77fe640 1 --2- 172.21.15.80:0/2765403386 >> v2:172.21.15.87:3300/0 conn(0x7fdcc80d0510 0x7fdcc815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcc7fff640 1 -- 172.21.15.80:0/2765403386 >> v2:172.21.15.87:3300/0 conn(0x7fdcc80d0510 msgr2=0x7fdcc815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcc7fff640 1 --2- 172.21.15.80:0/2765403386 >> v2:172.21.15.87:3300/0 conn(0x7fdcc80d0510 0x7fdcc815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcc7fff640 1 -- 172.21.15.80:0/2765403386 >> v1:172.21.15.87:6789/0 conn(0x7fdcc815ea70 legacy=0x7fdcc815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:01.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.650+0000 7fdcc7fff640 1 -- 172.21.15.80:0/2765403386 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc806df80 con 0x7fdcc8161f90 2024-04-25T13:48:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcc7fff640 1 --2- 172.21.15.80:0/2765403386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc8161f90 0x7fdcc80cffd0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8074290 tx=0x7fdcb801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984d676aa24e0db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcc6ffd640 1 -- 172.21.15.80:0/2765403386 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80b0070 con 0x7fdcc8161f90 2024-04-25T13:48:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcc6ffd640 1 -- 172.21.15.80:0/2765403386 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcb80ab600 con 0x7fdcc8161f90 2024-04-25T13:48:01.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcc6ffd640 1 -- 172.21.15.80:0/2765403386 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb80ab9d0 con 0x7fdcc8161f90 2024-04-25T13:48:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcce5f7640 1 -- 172.21.15.80:0/2765403386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc8161f90 msgr2=0x7fdcc80cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/2765403386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc8161f90 0x7fdcc80cffd0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8074290 tx=0x7fdcb801c3f0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcce5f7640 1 -- 172.21.15.80:0/2765403386 shutdown_connections 2024-04-25T13:48:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/2765403386 >> v2:172.21.15.87:3300/0 conn(0x7fdcc80d0510 0x7fdcc815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/2765403386 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc8161f90 0x7fdcc80cffd0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcce5f7640 1 -- 172.21.15.80:0/2765403386 >> 172.21.15.80:0/2765403386 conn(0x7fdcc806d080 msgr2=0x7fdcc80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcce5f7640 1 -- 172.21.15.80:0/2765403386 shutdown_connections 2024-04-25T13:48:01.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.654+0000 7fdcce5f7640 1 -- 172.21.15.80:0/2765403386 wait complete. 2024-04-25T13:48:01.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 Processor -- start 2024-04-25T13:48:01.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 -- start start 2024-04-25T13:48:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcc80d0510 0x7fdcc81411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc815ea70 0x7fdcc815a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc8161f90 0x7fdcc80b1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc806f4b0 con 0x7fdcc815ea70 2024-04-25T13:48:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc806f630 con 0x7fdcc8161f90 2024-04-25T13:48:01.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc80d0920 con 0x7fdcc80d0510 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc815ea70 0x7fdcc815a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc815ea70 0x7fdcc815a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53524/0 (socket says 172.21.15.80:53524) 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 -- 172.21.15.80:0/3068837952 learned_addr learned my addr 172.21.15.80:0/3068837952 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc7fff640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcc80d0510 0x7fdcc81411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcccb94640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc8161f90 0x7fdcc80b1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 -- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcc80d0510 msgr2=0x7fdcc81411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcc80d0510 0x7fdcc81411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 -- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc8161f90 msgr2=0x7fdcc80b1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc8161f90 0x7fdcc80b1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 -- 172.21.15.80:0/3068837952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc815b160 con 0x7fdcc815ea70 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc77fe640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc815ea70 0x7fdcc815a430 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00d5f0 tx=0x7fdcbc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc4ff9640 1 -- 172.21.15.80:0/3068837952 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0125f0 con 0x7fdcc815ea70 2024-04-25T13:48:01.658 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc4ff9640 1 -- 172.21.15.80:0/3068837952 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdcbc019070 con 0x7fdcc815ea70 2024-04-25T13:48:01.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc815b440 con 0x7fdcc815ea70 2024-04-25T13:48:01.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcc4ff9640 1 -- 172.21.15.80:0/3068837952 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcbc0129b0 con 0x7fdcc815ea70 2024-04-25T13:48:01.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc8141710 con 0x7fdcc815ea70 2024-04-25T13:48:01.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.658+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc815aed0 con 0x7fdcc815ea70 2024-04-25T13:48:01.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.662+0000 7fdcc4ff9640 1 -- 172.21.15.80:0/3068837952 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdcbc012b50 con 0x7fdcc815ea70 2024-04-25T13:48:01.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.662+0000 7fdcc4ff9640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdcb406f400 0x7fdcb4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.662+0000 7fdcc4ff9640 1 -- 172.21.15.80:0/3068837952 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7fdcbc08f370 con 0x7fdcc815ea70 2024-04-25T13:48:01.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.662+0000 7fdcc7fff640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdcb406f400 0x7fdcb4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.662+0000 7fdcc7fff640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdcb406f400 0x7fdcb4071910 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80ad000 tx=0x7fdcb80a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:01.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.666+0000 7fdcc4ff9640 1 -- 172.21.15.80:0/3068837952 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdcbc05c5f0 con 0x7fdcc815ea70 2024-04-25T13:48:01.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.814+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdcc806a3a0 con 0x7fdcc815ea70 2024-04-25T13:48:01.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.814+0000 7fdcc4ff9640 1 -- 172.21.15.80:0/3068837952 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+12959 (secure 0 0 0) 0x7fdcbc05bef0 con 0x7fdcc815ea70 2024-04-25T13:48:01.814 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:01.814 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":145,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:48:00.081731+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/1563317639":"2024-04-26T13:47:57.065482+0000","172.21.15.80:6833/1563317639":"2024-04-26T13:47:57.065482+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdcb406f400 msgr2=0x7fdcb4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdcb406f400 0x7fdcb4071910 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdcb80ad000 tx=0x7fdcb80a6000 comp rx=0 tx=0).stop 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc815ea70 msgr2=0x7fdcc815a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc815ea70 0x7fdcc815a430 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc00d5f0 tx=0x7fdcbc00dab0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 shutdown_connections 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdcb406f400 0x7fdcb4071910 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdcc8161f90 0x7fdcc80b1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdcc815ea70 0x7fdcc815a430 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 --2- 172.21.15.80:0/3068837952 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdcc80d0510 0x7fdcc81411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 >> 172.21.15.80:0/3068837952 conn(0x7fdcc806d080 msgr2=0x7fdcc80b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:01.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 shutdown_connections 2024-04-25T13:48:01.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.818+0000 7fdcce5f7640 1 -- 172.21.15.80:0/3068837952 wait complete. 2024-04-25T13:48:01.835 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:48:01.835 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:48:01.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9cbbb5640 1 Processor -- start 2024-04-25T13:48:01.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9cbbb5640 1 -- start start 2024-04-25T13:48:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9cbbb5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40d1b00 0x7fd9c4072700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9cbbb5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9c40d04e0 0x7fd9c40d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9cbbb5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9c4072c40 con 0x7fd9c406f490 2024-04-25T13:48:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9cbbb5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd9c406e760 con 0x7fd9c40d04e0 2024-04-25T13:48:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9cbbb5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c40d0e30 con 0x7fd9c40d1b00 2024-04-25T13:48:01.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9ca152640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd9c406f490 0x7fd9c406f8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51350/0 (socket says 172.21.15.80:51350) 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9ca152640 1 -- 172.21.15.80:0/3618812340 learned_addr learned my addr 172.21.15.80:0/3618812340 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c9951640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40d1b00 0x7fd9c4072700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c9150640 1 --2- 172.21.15.80:0/3618812340 >> v2:172.21.15.87:3300/0 conn(0x7fd9c40d04e0 0x7fd9c40d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2777679360 0 0) 0x7fd9c4072c40 con 0x7fd9c406f490 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd9b0003530 con 0x7fd9c406f490 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3145649778 0 0) 0x7fd9b0003530 con 0x7fd9c406f490 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40d1b00 msgr2=0x7fd9c4072700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c894f640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40d1b00 0x7fd9c4072700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 >> v2:172.21.15.87:3300/0 conn(0x7fd9c40d04e0 msgr2=0x7fd9c40d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:01.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.974+0000 7fd9c894f640 1 --2- 172.21.15.80:0/3618812340 >> v2:172.21.15.87:3300/0 conn(0x7fd9c40d04e0 0x7fd9c40d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c40d1f10 con 0x7fd9c406f490 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fd9bc001680 con 0x7fd9c406f490 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 >> v1:172.21.15.80:6789/0 conn(0x7fd9c406f490 legacy=0x7fd9c406f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b00039c0 0x7fd9b0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9b0004810 0x7fd9b0004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40d1b00 0x7fd9b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c406e760 con 0x7fd9b0004810 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c40d0e30 con 0x7fd9b00039c0 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9b000bb00 con 0x7fd9c40d1b00 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c9150640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9b0004810 0x7fd9b0004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9ca152640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40d1b00 0x7fd9b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c9951640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b00039c0 0x7fd9b0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c9150640 1 -- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40d1b00 msgr2=0x7fd9b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:01.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c9150640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40d1b00 0x7fd9b0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c9150640 1 -- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b00039c0 msgr2=0x7fd9b0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:01.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c9150640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b00039c0 0x7fd9b0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c9150640 1 -- 172.21.15.80:0/3618812340 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b0003530 con 0x7fd9b0004810 2024-04-25T13:48:01.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c9150640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9b0004810 0x7fd9b0004c20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40726b0 tx=0x7fd9b401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c13be0c3833c28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:01.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40096b0 con 0x7fd9b0004810 2024-04-25T13:48:01.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9b40a6450 con 0x7fd9b0004810 2024-04-25T13:48:01.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9c894f640 1 -- 172.21.15.80:0/3618812340 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40a6800 con 0x7fd9b0004810 2024-04-25T13:48:01.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/3618812340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9b0004810 msgr2=0x7fd9b0004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9cbbb5640 1 --2- 172.21.15.80:0/3618812340 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9b0004810 0x7fd9b0004c20 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fd9c40726b0 tx=0x7fd9b401c960 comp rx=0 tx=0).stop 2024-04-25T13:48:01.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9ca152640 1 -- 172.21.15.80:0/3618812340 reap_dead start 2024-04-25T13:48:01.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/3618812340 shutdown_connections 2024-04-25T13:48:01.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/3618812340 >> 172.21.15.80:0/3618812340 conn(0x7fd9c40b3cc0 msgr2=0x7fd9c40b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:01.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/3618812340 shutdown_connections 2024-04-25T13:48:01.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.978+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/3618812340 wait complete. 2024-04-25T13:48:01.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 Processor -- start 2024-04-25T13:48:01.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 -- start start 2024-04-25T13:48:01.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b0004810 0x7fd9c4157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40afc00 0x7fd9c41fd9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9c40d2110 0x7fd9c40d2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c40ce910 con 0x7fd9c40d2110 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c406ef70 con 0x7fd9b0004810 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9c40d1240 con 0x7fd9c40afc00 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9c40d2110 0x7fd9c40d2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9c40d2110 0x7fd9c40d2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53540/0 (socket says 172.21.15.80:53540) 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 -- 172.21.15.80:0/2210476145 learned_addr learned my addr 172.21.15.80:0/2210476145 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9c9951640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b0004810 0x7fd9c4157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9c9150640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40afc00 0x7fd9c41fd9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 -- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40afc00 msgr2=0x7fd9c41fd9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40afc00 0x7fd9c41fd9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 -- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b0004810 msgr2=0x7fd9c4157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:01.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b0004810 0x7fd9c4157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 -- 172.21.15.80:0/2210476145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c40d2a60 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ca152640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9c40d2110 0x7fd9c40d2520 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc000c10 tx=0x7fd9bc00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ba7fc640 1 -- 172.21.15.80:0/2210476145 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00c690 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ba7fc640 1 -- 172.21.15.80:0/2210476145 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9bc018070 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9ba7fc640 1 -- 172.21.15.80:0/2210476145 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00ca50 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c40b0530 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.982+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c40af120 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.986+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9c40b0cb0 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.986+0000 7fd9ba7fc640 1 -- 172.21.15.80:0/2210476145 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd9bc00cbf0 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.986+0000 7fd9ba7fc640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9a806f400 0x7fd9a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.986+0000 7fd9ba7fc640 1 -- 172.21.15.80:0/2210476145 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7fd9bc08eb20 con 0x7fd9c40d2110 2024-04-25T13:48:01.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.986+0000 7fd9c9951640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9a806f400 0x7fd9a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:01.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.986+0000 7fd9c9951640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9a806f400 0x7fd9a8071910 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00073f0 tx=0x7fd9c000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:01.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:01.990+0000 7fd9ba7fc640 1 -- 172.21.15.80:0/2210476145 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd9c40d2a60 con 0x7fd9c40d2110 2024-04-25T13:48:02.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.150+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd9c40b0760 con 0x7fd9c40d2110 2024-04-25T13:48:02.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.150+0000 7fd9ba7fc640 1 -- 172.21.15.80:0/2210476145 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) ==== 93+0+10 (secure 0 0 0) 0x7fd9bc05bd10 con 0x7fd9c40d2110 2024-04-25T13:48:02.148 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.150+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9a806f400 msgr2=0x7fd9a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.150+0000 7fd9cbbb5640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9a806f400 0x7fd9a8071910 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00073f0 tx=0x7fd9c000b040 comp rx=0 tx=0).stop 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.150+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9c40d2110 msgr2=0x7fd9c40d2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.150+0000 7fd9cbbb5640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9c40d2110 0x7fd9c40d2520 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc000c10 tx=0x7fd9bc00eb90 comp rx=0 tx=0).stop 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.154+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 shutdown_connections 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.154+0000 7fd9cbbb5640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9a806f400 0x7fd9a8071910 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.154+0000 7fd9cbbb5640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9c40d2110 0x7fd9c40d2520 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.154+0000 7fd9cbbb5640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9c40afc00 0x7fd9c41fd9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.154+0000 7fd9cbbb5640 1 --2- 172.21.15.80:0/2210476145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9b0004810 0x7fd9c4157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.154+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 >> 172.21.15.80:0/2210476145 conn(0x7fd9c40b3cc0 msgr2=0x7fd9c4167470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:02.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.154+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 shutdown_connections 2024-04-25T13:48:02.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.154+0000 7fd9cbbb5640 1 -- 172.21.15.80:0/2210476145 wait complete. 2024-04-25T13:48:02.170 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-04-25T13:48:02.307 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67d258c640 1 Processor -- start 2024-04-25T13:48:02.308 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67d258c640 1 -- start start 2024-04-25T13:48:02.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67d258c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67cc161f90 0x7f67cc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:02.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67d258c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f67cc15ec40 0x7f67cc15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:02.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67d258c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f67cc0d0d70 con 0x7f67cc0d0510 2024-04-25T13:48:02.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67d258c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f67cc0d1050 con 0x7f67cc161f90 2024-04-25T13:48:02.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67d258c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f67cc06dca0 con 0x7f67cc15ec40 2024-04-25T13:48:02.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67cbfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67cc161f90 0x7f67cc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:02.310 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67cbfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f67cc161f90 0x7f67cc0cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53546/0 (socket says 172.21.15.80:53546) 2024-04-25T13:48:02.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67cbfff640 1 -- 172.21.15.80:0/1894413193 learned_addr learned my addr 172.21.15.80:0/1894413193 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:02.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67d0b29640 1 --2- 172.21.15.80:0/1894413193 >> v2:172.21.15.87:3300/0 conn(0x7f67cc15ec40 0x7f67cc15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:02.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67cbfff640 1 -- 172.21.15.80:0/1894413193 >> v2:172.21.15.87:3300/0 conn(0x7f67cc15ec40 msgr2=0x7f67cc15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:02.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67cbfff640 1 --2- 172.21.15.80:0/1894413193 >> v2:172.21.15.87:3300/0 conn(0x7f67cc15ec40 0x7f67cc15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67cbfff640 1 -- 172.21.15.80:0/1894413193 >> v1:172.21.15.87:6789/0 conn(0x7f67cc0d0510 legacy=0x7f67cc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:02.311 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.310+0000 7f67cbfff640 1 -- 172.21.15.80:0/1894413193 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc06df80 con 0x7f67cc161f90 2024-04-25T13:48:02.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67cbfff640 1 --2- 172.21.15.80:0/1894413193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc161f90 0x7f67cc0cffd0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0b4b80 tx=0x7f67bc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a316db0bda68f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:02.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67caffd640 1 -- 172.21.15.80:0/1894413193 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0b0070 con 0x7f67cc161f90 2024-04-25T13:48:02.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67caffd640 1 -- 172.21.15.80:0/1894413193 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67bc0ab600 con 0x7f67cc161f90 2024-04-25T13:48:02.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67caffd640 1 -- 172.21.15.80:0/1894413193 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0ab9d0 con 0x7f67cc161f90 2024-04-25T13:48:02.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67d258c640 1 -- 172.21.15.80:0/1894413193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc161f90 msgr2=0x7f67cc0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:02.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67d258c640 1 --2- 172.21.15.80:0/1894413193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc161f90 0x7f67cc0cffd0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f67cc0b4b80 tx=0x7f67bc01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67d258c640 1 -- 172.21.15.80:0/1894413193 shutdown_connections 2024-04-25T13:48:02.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67d258c640 1 --2- 172.21.15.80:0/1894413193 >> v2:172.21.15.87:3300/0 conn(0x7f67cc15ec40 0x7f67cc15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67d258c640 1 --2- 172.21.15.80:0/1894413193 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc161f90 0x7f67cc0cffd0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67d258c640 1 -- 172.21.15.80:0/1894413193 >> 172.21.15.80:0/1894413193 conn(0x7f67cc06d080 msgr2=0x7f67cc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:02.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67d258c640 1 -- 172.21.15.80:0/1894413193 shutdown_connections 2024-04-25T13:48:02.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.314+0000 7f67d258c640 1 -- 172.21.15.80:0/1894413193 wait complete. 2024-04-25T13:48:02.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 Processor -- start 2024-04-25T13:48:02.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 -- start start 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc0d0510 0x7f67cc140d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67cc15ec40 0x7f67cc157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67cc161f90 0x7f67cc15a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc06f630 con 0x7f67cc0d0510 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f67cc06f4b0 con 0x7f67cc161f90 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f67cc0d0920 con 0x7f67cc15ec40 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc0d0510 0x7f67cc140d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc0d0510 0x7f67cc140d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53554/0 (socket says 172.21.15.80:53554) 2024-04-25T13:48:02.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 -- 172.21.15.80:0/3722033331 learned_addr learned my addr 172.21.15.80:0/3722033331 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cb7fe640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67cc15ec40 0x7f67cc157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 -- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67cc15ec40 msgr2=0x7f67cc157cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d0b29640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67cc161f90 0x7f67cc15a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67cc15ec40 0x7f67cc157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 -- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67cc161f90 msgr2=0x7f67cc15a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67cc161f90 0x7f67cc15a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 -- 172.21.15.80:0/3722033331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67cc158620 con 0x7f67cc0d0510 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67cbfff640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc0d0510 0x7f67cc140d60 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0ae040 tx=0x7f67bc0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67c8ff9640 1 -- 172.21.15.80:0/3722033331 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0b0040 con 0x7f67cc0d0510 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67c8ff9640 1 -- 172.21.15.80:0/3722033331 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f67bc0b9070 con 0x7f67cc0d0510 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67cc158900 con 0x7f67cc0d0510 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67c8ff9640 1 -- 172.21.15.80:0/3722033331 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67bc0b4680 con 0x7f67cc0d0510 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67cc160e40 con 0x7f67cc0d0510 2024-04-25T13:48:02.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.318+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67cc15f720 con 0x7f67cc0d0510 2024-04-25T13:48:02.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.322+0000 7f67c8ff9640 1 -- 172.21.15.80:0/3722033331 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f67bc0b4820 con 0x7f67cc0d0510 2024-04-25T13:48:02.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.322+0000 7f67c8ff9640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f67ac06f400 0x7f67ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:02.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.322+0000 7f67c8ff9640 1 -- 172.21.15.80:0/3722033331 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f67bc128ce0 con 0x7f67cc0d0510 2024-04-25T13:48:02.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.322+0000 7f67cb7fe640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f67ac06f400 0x7f67ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:02.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.322+0000 7f67cb7fe640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f67ac06f400 0x7f67ac071910 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f67cc159070 tx=0x7f67b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:02.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.326+0000 7f67c8ff9640 1 -- 172.21.15.80:0/3722033331 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f67bc0160d0 con 0x7f67cc0d0510 2024-04-25T13:48:02.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:02.526+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) -- 0x7f67cc06a3a0 con 0x7f67cc0d0510 2024-04-25T13:48:03.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.478+0000 7f67c8ff9640 1 -- 172.21.15.80:0/3722033331 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v403) ==== 135+0+0 (secure 0 0 0) 0x7f67bc0f5ed0 con 0x7f67cc0d0510 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f67ac06f400 msgr2=0x7f67ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f67ac06f400 0x7f67ac071910 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f67cc159070 tx=0x7f67b4008040 comp rx=0 tx=0).stop 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc0d0510 msgr2=0x7f67cc140d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc0d0510 0x7f67cc140d60 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f67bc0ae040 tx=0x7f67bc0191f0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 shutdown_connections 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f67ac06f400 0x7f67ac071910 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67cc161f90 0x7f67cc15a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f67cc15ec40 0x7f67cc157cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 --2- 172.21.15.80:0/3722033331 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f67cc0d0510 0x7f67cc140d60 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 >> 172.21.15.80:0/3722033331 conn(0x7f67cc06d080 msgr2=0x7f67cc0b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:03.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 shutdown_connections 2024-04-25T13:48:03.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.482+0000 7f67d258c640 1 -- 172.21.15.80:0/3722033331 wait complete. 2024-04-25T13:48:03.506 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:48:03.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc86a9ce640 1 Processor -- start 2024-04-25T13:48:03.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc86a9ce640 1 -- start start 2024-04-25T13:48:03.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc86a9ce640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc864161170 0x7fc8641574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:03.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc86a9ce640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc86406fbf0 con 0x7fc864073c40 2024-04-25T13:48:03.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc86a9ce640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8640b0a20 con 0x7fc864071e60 2024-04-25T13:48:03.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc86a9ce640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc8640b1350 con 0x7fc864161170 2024-04-25T13:48:03.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc868f6b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc864073c40 0x7fc8640b4ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51366/0 (socket says 172.21.15.80:51366) 2024-04-25T13:48:03.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc868f6b640 1 -- 172.21.15.80:0/2298351617 learned_addr learned my addr 172.21.15.80:0/2298351617 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:03.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc8637fe640 1 --2- 172.21.15.80:0/2298351617 >> v2:172.21.15.87:3300/0 conn(0x7fc864161170 0x7fc8641574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:03.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3484538326 0 0) 0x7fc86406fbf0 con 0x7fc864073c40 2024-04-25T13:48:03.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc84c003530 con 0x7fc864073c40 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4093551744 0 0) 0x7fc84c003530 con 0x7fc864073c40 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 >> v2:172.21.15.87:3300/0 conn(0x7fc864161170 msgr2=0x7fc8641574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 --2- 172.21.15.80:0/2298351617 >> v2:172.21.15.87:3300/0 conn(0x7fc864161170 0x7fc8641574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 >> v1:172.21.15.87:6789/0 conn(0x7fc864071e60 legacy=0x7fc864160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8640b0d00 con 0x7fc864073c40 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc85c001a00 con 0x7fc864073c40 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 >> v1:172.21.15.80:6789/0 conn(0x7fc864073c40 legacy=0x7fc8640b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84c0039c0 0x7fc84c003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc84c004810 0x7fc84c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c005280 0x7fc84c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc8640b1350 con 0x7fc84c004810 2024-04-25T13:48:03.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc84c00c1a0 con 0x7fc84c005280 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc84c00e5b0 con 0x7fc84c0039c0 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc8637fe640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc84c004810 0x7fc84c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc863fff640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84c0039c0 0x7fc84c003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.670+0000 7fc868f6b640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c005280 0x7fc84c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc8637fe640 1 -- 172.21.15.80:0/2298351617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84c0039c0 msgr2=0x7fc84c003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc8637fe640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc84c0039c0 0x7fc84c003e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc8637fe640 1 -- 172.21.15.80:0/2298351617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c005280 msgr2=0x7fc84c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc8637fe640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c005280 0x7fc84c00b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc868f6b640 1 -- 172.21.15.80:0/2298351617 reap_dead start 2024-04-25T13:48:03.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc8637fe640 1 -- 172.21.15.80:0/2298351617 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84c003530 con 0x7fc84c004810 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc8637fe640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc84c004810 0x7fc84c004c20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc858009ae0 tx=0x7fc85801ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62fa0fe9926ec3c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8580b2070 con 0x7fc84c004810 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8580a5660 con 0x7fc84c004810 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc862ffd640 1 -- 172.21.15.80:0/2298351617 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8580096b0 con 0x7fc84c004810 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc86a9ce640 1 -- 172.21.15.80:0/2298351617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc84c004810 msgr2=0x7fc84c004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc86a9ce640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc84c004810 0x7fc84c004c20 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc858009ae0 tx=0x7fc85801ca70 comp rx=0 tx=0).stop 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc86a9ce640 1 -- 172.21.15.80:0/2298351617 shutdown_connections 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc86a9ce640 1 --2- 172.21.15.80:0/2298351617 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc84c004810 0x7fc84c004c20 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc86a9ce640 1 -- 172.21.15.80:0/2298351617 >> 172.21.15.80:0/2298351617 conn(0x7fc8640b4160 msgr2=0x7fc8640d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc86a9ce640 1 -- 172.21.15.80:0/2298351617 shutdown_connections 2024-04-25T13:48:03.675 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.674+0000 7fc86a9ce640 1 -- 172.21.15.80:0/2298351617 wait complete. 2024-04-25T13:48:03.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 Processor -- start 2024-04-25T13:48:03.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 -- start start 2024-04-25T13:48:03.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c004810 0x7fc8641513d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:03.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc864152650 0x7fc864152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:03.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc864155980 0x7fc864155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:03.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc864072270 con 0x7fc864152650 2024-04-25T13:48:03.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc86406f9a0 con 0x7fc84c004810 2024-04-25T13:48:03.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc864074050 con 0x7fc864155980 2024-04-25T13:48:03.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc864152650 0x7fc864152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc864152650 0x7fc864152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53584/0 (socket says 172.21.15.80:53584) 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 -- 172.21.15.80:0/1126800539 learned_addr learned my addr 172.21.15.80:0/1126800539 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc868f6b640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc864155980 0x7fc864155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc863fff640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c004810 0x7fc8641513d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 -- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc864155980 msgr2=0x7fc864155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc864155980 0x7fc864155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 -- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c004810 msgr2=0x7fc8641513d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c004810 0x7fc8641513d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 -- 172.21.15.80:0/1126800539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8641562d0 con 0x7fc864152650 2024-04-25T13:48:03.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc8637fe640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc864152650 0x7fc864152a60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc8580b0040 tx=0x7fc858019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:03.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc860ff9640 1 -- 172.21.15.80:0/1126800539 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8580b2040 con 0x7fc864152650 2024-04-25T13:48:03.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc860ff9640 1 -- 172.21.15.80:0/1126800539 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc858004020 con 0x7fc864152650 2024-04-25T13:48:03.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc864153060 con 0x7fc864152650 2024-04-25T13:48:03.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc860ff9640 1 -- 172.21.15.80:0/1126800539 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8580096b0 con 0x7fc864152650 2024-04-25T13:48:03.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.678+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8640cf3e0 con 0x7fc864152650 2024-04-25T13:48:03.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.682+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc86406a3a0 con 0x7fc864152650 2024-04-25T13:48:03.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.682+0000 7fc860ff9640 1 -- 172.21.15.80:0/1126800539 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc8580bf060 con 0x7fc864152650 2024-04-25T13:48:03.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.682+0000 7fc860ff9640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc85006f430 0x7fc850071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:03.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.682+0000 7fc860ff9640 1 -- 172.21.15.80:0/1126800539 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7fc858128bf0 con 0x7fc864152650 2024-04-25T13:48:03.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.682+0000 7fc863fff640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc85006f430 0x7fc850071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:03.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.682+0000 7fc863fff640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc85006f430 0x7fc850071940 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc854000c10 tx=0x7fc854009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:03.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.686+0000 7fc860ff9640 1 -- 172.21.15.80:0/1126800539 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc8641562d0 con 0x7fc864152650 2024-04-25T13:48:03.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.842+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc864156ce0 con 0x7fc864152650 2024-04-25T13:48:03.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.850+0000 7fc860ff9640 1 -- 172.21.15.80:0/1126800539 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fc858016780 con 0x7fc864152650 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.850+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc85006f430 msgr2=0x7fc850071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.850+0000 7fc86a9ce640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc85006f430 0x7fc850071940 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fc854000c10 tx=0x7fc854009040 comp rx=0 tx=0).stop 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.850+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc864152650 msgr2=0x7fc864152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.850+0000 7fc86a9ce640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc864152650 0x7fc864152a60 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc8580b0040 tx=0x7fc858019810 comp rx=0 tx=0).stop 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.850+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 shutdown_connections 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.850+0000 7fc86a9ce640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc85006f430 0x7fc850071940 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.854+0000 7fc86a9ce640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc864155980 0x7fc864155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.854+0000 7fc86a9ce640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc864152650 0x7fc864152a60 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.854+0000 7fc86a9ce640 1 --2- 172.21.15.80:0/1126800539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c004810 0x7fc8641513d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.854+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 >> 172.21.15.80:0/1126800539 conn(0x7fc8640b4160 msgr2=0x7fc86406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:03.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.854+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 shutdown_connections 2024-04-25T13:48:03.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:03.854+0000 7fc86a9ce640 1 -- 172.21.15.80:0/1126800539 wait complete. 2024-04-25T13:48:03.870 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:48:03.870 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:48:03.916 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:48:03.974 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:48:03.974 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:48:03.974 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:48:03.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:48:03.979 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:48:03.979 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:48:03.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:48:03.985 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:48:03.985 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:48:04.027 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:48:04.027 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:48:04.027 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:48:04.033 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:48:04.033 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:48:04.033 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:48:04.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.206+0000 7f0e14776640 1 Processor -- start 2024-04-25T13:48:04.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.206+0000 7f0e14776640 1 -- start start 2024-04-25T13:48:04.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e14776640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c161f90 0x7f0e0c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e14776640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0e0c0d0510 0x7f0e0c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e14776640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e0c0d0d70 con 0x7f0e0c15ea70 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e14776640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0e0c0d1050 con 0x7f0e0c0d0510 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e14776640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c06dca0 con 0x7f0e0c161f90 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e11d11640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0e0c0d0510 0x7f0e0c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e11d11640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0e0c0d0510 0x7f0e0c15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53592/0 (socket says 172.21.15.80:53592) 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e11d11640 1 -- 172.21.15.80:0/3639316530 learned_addr learned my addr 172.21.15.80:0/3639316530 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e12512640 1 --2- 172.21.15.80:0/3639316530 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c161f90 0x7f0e0c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e11d11640 1 -- 172.21.15.80:0/3639316530 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c161f90 msgr2=0x7f0e0c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e11d11640 1 --2- 172.21.15.80:0/3639316530 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c161f90 0x7f0e0c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e11d11640 1 -- 172.21.15.80:0/3639316530 >> v1:172.21.15.87:6789/0 conn(0x7f0e0c15ea70 legacy=0x7f0e0c15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e11d11640 1 -- 172.21.15.80:0/3639316530 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c06df80 con 0x7f0e0c0d0510 2024-04-25T13:48:04.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.210+0000 7f0e11d11640 1 --2- 172.21.15.80:0/3639316530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c0d0510 0x7f0e0c15e530 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009870 tx=0x7f0e0801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f9b81c63699c2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:04.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e11510640 1 -- 172.21.15.80:0/3639316530 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e080b1070 con 0x7f0e0c0d0510 2024-04-25T13:48:04.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e11510640 1 -- 172.21.15.80:0/3639316530 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e080ac600 con 0x7f0e0c0d0510 2024-04-25T13:48:04.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e11510640 1 -- 172.21.15.80:0/3639316530 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e080ac9d0 con 0x7f0e0c0d0510 2024-04-25T13:48:04.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 -- 172.21.15.80:0/3639316530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c0d0510 msgr2=0x7f0e0c15e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 --2- 172.21.15.80:0/3639316530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c0d0510 0x7f0e0c15e530 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009870 tx=0x7f0e0801c470 comp rx=0 tx=0).stop 2024-04-25T13:48:04.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 -- 172.21.15.80:0/3639316530 shutdown_connections 2024-04-25T13:48:04.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 --2- 172.21.15.80:0/3639316530 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c0d0510 0x7f0e0c15e530 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 --2- 172.21.15.80:0/3639316530 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c161f90 0x7f0e0c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 -- 172.21.15.80:0/3639316530 >> 172.21.15.80:0/3639316530 conn(0x7f0e0c06d080 msgr2=0x7f0e0c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:04.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 -- 172.21.15.80:0/3639316530 shutdown_connections 2024-04-25T13:48:04.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 -- 172.21.15.80:0/3639316530 wait complete. 2024-04-25T13:48:04.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 Processor -- start 2024-04-25T13:48:04.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.214+0000 7f0e14776640 1 -- start start 2024-04-25T13:48:04.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c0d0510 0x7f0e0c1fa480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c15ea70 0x7f0e0c140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e0c161f90 0x7f0e0c071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c06f630 con 0x7f0e0c15ea70 2024-04-25T13:48:04.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0e0c06f4b0 con 0x7f0e0c161f90 2024-04-25T13:48:04.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e0c0d0920 con 0x7f0e0c0d0510 2024-04-25T13:48:04.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c15ea70 0x7f0e0c140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c15ea70 0x7f0e0c140ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53598/0 (socket says 172.21.15.80:53598) 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 -- 172.21.15.80:0/4050756488 learned_addr learned my addr 172.21.15.80:0/4050756488 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e12512640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c0d0510 0x7f0e0c1fa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e12d13640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e0c161f90 0x7f0e0c071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 -- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c0d0510 msgr2=0x7f0e0c1fa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c0d0510 0x7f0e0c1fa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 -- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e0c161f90 msgr2=0x7f0e0c071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e0c161f90 0x7f0e0c071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 -- 172.21.15.80:0/4050756488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e0c1f9a00 con 0x7f0e0c15ea70 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e12d13640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e0c161f90 0x7f0e0c071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e12512640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c0d0510 0x7f0e0c1fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e11d11640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c15ea70 0x7f0e0c140ea0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0e0801a100 tx=0x7f0e0801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e02ffd640 1 -- 172.21.15.80:0/4050756488 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e080bb020 con 0x7f0e0c15ea70 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e02ffd640 1 -- 172.21.15.80:0/4050756488 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e080b7420 con 0x7f0e0c15ea70 2024-04-25T13:48:04.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e0c141640 con 0x7f0e0c15ea70 2024-04-25T13:48:04.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e02ffd640 1 -- 172.21.15.80:0/4050756488 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e080b1040 con 0x7f0e0c15ea70 2024-04-25T13:48:04.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e0c1fdce0 con 0x7f0e0c15ea70 2024-04-25T13:48:04.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e0c06a3a0 con 0x7f0e0c15ea70 2024-04-25T13:48:04.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e02ffd640 1 -- 172.21.15.80:0/4050756488 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0e080be020 con 0x7f0e0c15ea70 2024-04-25T13:48:04.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e02ffd640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0df006f430 0x7f0df0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e02ffd640 1 -- 172.21.15.80:0/4050756488 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f0e081291b0 con 0x7f0e0c15ea70 2024-04-25T13:48:04.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.218+0000 7f0e12512640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0df006f430 0x7f0df0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.222+0000 7f0e12512640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0df006f430 0x7f0df0071940 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c06fa20 tx=0x7f0dfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:04.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.222+0000 7f0e02ffd640 1 -- 172.21.15.80:0/4050756488 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e080c5020 con 0x7f0e0c15ea70 2024-04-25T13:48:04.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.410+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e0c0b64a0 con 0x7f0e0c15ea70 2024-04-25T13:48:04.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e02ffd640 1 -- 172.21.15.80:0/4050756488 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+3869 (secure 0 0 0) 0x7f0e0c1f9a00 con 0x7f0e0c15ea70 2024-04-25T13:48:04.412 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:04.412 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":69,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5885,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/1632687727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":1632687727},{"type":"v1","addr":"172.21.15.87:6837","nonce":1632687727}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6400,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/3510017031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":3510017031},{"type":"v1","addr":"172.21.15.80:6834","nonce":3510017031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":6412,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/1126594501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":1126594501},{"type":"v1","addr":"172.21.15.87:6835","nonce":1126594501}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63},{"gid":8340,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/2637946854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":2637946854},{"type":"v1","addr":"172.21.15.80:6835","nonce":2637946854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":69}],"filesystems":[]} 2024-04-25T13:48:04.412 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 69 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0df006f430 msgr2=0x7f0df0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0df006f430 0x7f0df0071940 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c06fa20 tx=0x7f0dfc008040 comp rx=0 tx=0).stop 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c15ea70 msgr2=0x7f0e0c140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c15ea70 0x7f0e0c140ea0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f0e0801a100 tx=0x7f0e0801ad10 comp rx=0 tx=0).stop 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 shutdown_connections 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0df006f430 0x7f0df0071940 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e0c161f90 0x7f0e0c071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e0c15ea70 0x7f0e0c140ea0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 --2- 172.21.15.80:0/4050756488 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e0c0d0510 0x7f0e0c1fa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 >> 172.21.15.80:0/4050756488 conn(0x7f0e0c06d080 msgr2=0x7f0e0c0b5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:04.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 shutdown_connections 2024-04-25T13:48:04.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.414+0000 7f0e14776640 1 -- 172.21.15.80:0/4050756488 wait complete. 2024-04-25T13:48:04.432 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:48:04.432 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:48:04.433 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:48:04.433 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:48:04.433 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:48:04.433 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:48:04.433 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:48:04.433 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:48:04.434 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:48:04.434 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:48:04.434 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:48:04.434 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:48:04.434 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:48:04.434 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:48:04.434 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:48:04.435 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:48:04.449 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:48:04.450 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:48:04.452 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:48:04.452 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:48:04.462 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:48:04.462 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:48:04.467 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:48:04.468 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:48:04.495 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:48:04.496 DEBUG:teuthology.parallel:result is None 2024-04-25T13:48:04.496 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:48:04.497 DEBUG:teuthology.parallel:result is None 2024-04-25T13:48:04.513 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:48:04.513 DEBUG:teuthology.parallel:result is None 2024-04-25T13:48:04.514 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:48:04.514 DEBUG:teuthology.parallel:result is None 2024-04-25T13:48:04.514 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:48:04.586 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:48:04.587 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:48:04.610 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:48:04.613 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:48:04.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.662+0000 7f560ab54640 1 Processor -- start 2024-04-25T13:48:04.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.662+0000 7f560ab54640 1 -- start start 2024-04-25T13:48:04.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f560ab54640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56040d5490 0x7f56040d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f560ab54640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f56040b92a0 0x7f56040b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f560ab54640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56040d43f0 con 0x7f56040d2560 2024-04-25T13:48:04.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f560ab54640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f56040d46d0 con 0x7f56040d5490 2024-04-25T13:48:04.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f560ab54640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f56040d49b0 con 0x7f56040b92a0 2024-04-25T13:48:04.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56088f0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56040d5490 0x7f56040d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.664 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56088f0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56040d5490 0x7f56040d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53638/0 (socket says 172.21.15.80:53638) 2024-04-25T13:48:04.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56088f0640 1 -- 172.21.15.80:0/1317499015 learned_addr learned my addr 172.21.15.80:0/1317499015 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:04.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56090f1640 1 --2- 172.21.15.80:0/1317499015 >> v2:172.21.15.87:3300/0 conn(0x7f56040b92a0 0x7f56040b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56088f0640 1 -- 172.21.15.80:0/1317499015 >> v2:172.21.15.87:3300/0 conn(0x7f56040b92a0 msgr2=0x7f56040b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:04.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56088f0640 1 --2- 172.21.15.80:0/1317499015 >> v2:172.21.15.87:3300/0 conn(0x7f56040b92a0 0x7f56040b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56088f0640 1 -- 172.21.15.80:0/1317499015 >> v1:172.21.15.87:6789/0 conn(0x7f56040d2560 legacy=0x7f56040d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56088f0640 1 -- 172.21.15.80:0/1317499015 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56040d4c90 con 0x7f56040d5490 2024-04-25T13:48:04.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56088f0640 1 --2- 172.21.15.80:0/1317499015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040d14a0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f560408ef40 tx=0x7f55f801c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08626f28cb80b72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:04.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56037fe640 1 -- 172.21.15.80:0/1317499015 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f80aa670 con 0x7f56040d5490 2024-04-25T13:48:04.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56037fe640 1 -- 172.21.15.80:0/1317499015 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55f80af070 con 0x7f56040d5490 2024-04-25T13:48:04.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.666+0000 7f56037fe640 1 -- 172.21.15.80:0/1317499015 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f80aaa30 con 0x7f56040d5490 2024-04-25T13:48:04.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- 172.21.15.80:0/1317499015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 msgr2=0x7f56040d14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 --2- 172.21.15.80:0/1317499015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040d14a0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f560408ef40 tx=0x7f55f801c4f0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- 172.21.15.80:0/1317499015 shutdown_connections 2024-04-25T13:48:04.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 --2- 172.21.15.80:0/1317499015 >> v2:172.21.15.87:3300/0 conn(0x7f56040b92a0 0x7f56040b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 --2- 172.21.15.80:0/1317499015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040d14a0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- 172.21.15.80:0/1317499015 >> 172.21.15.80:0/1317499015 conn(0x7f5604073740 msgr2=0x7f560406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:04.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- 172.21.15.80:0/1317499015 shutdown_connections 2024-04-25T13:48:04.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- 172.21.15.80:0/1317499015 wait complete. 2024-04-25T13:48:04.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 Processor -- start 2024-04-25T13:48:04.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- start start 2024-04-25T13:48:04.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56040b92a0 0x7f56040cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56040d2560 0x7f56040b58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040b5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f56040ba000 con 0x7f56040d5490 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f56040b5140 con 0x7f56040d2560 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f560ab54640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f56040ba180 con 0x7f56040b92a0 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f56090f1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040b5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.670+0000 7f56090f1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040b5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53646/0 (socket says 172.21.15.80:53646) 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56090f1640 1 -- 172.21.15.80:0/2549239154 learned_addr learned my addr 172.21.15.80:0/2549239154 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f5603fff640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56040d2560 0x7f56040b58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56088f0640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56040b92a0 0x7f56040cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56090f1640 1 -- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56040b92a0 msgr2=0x7f56040cc170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56090f1640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56040b92a0 0x7f56040cc170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56090f1640 1 -- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56040d2560 msgr2=0x7f56040b58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56090f1640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56040d2560 0x7f56040b58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:04.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56090f1640 1 -- 172.21.15.80:0/2549239154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5604153f00 con 0x7f56040d5490 2024-04-25T13:48:04.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56090f1640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040b5de0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009cb0 tx=0x7f55fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:04.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56017fa640 1 -- 172.21.15.80:0/2549239154 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc018020 con 0x7f56040d5490 2024-04-25T13:48:04.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56017fa640 1 -- 172.21.15.80:0/2549239154 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f55fc014480 con 0x7f56040d5490 2024-04-25T13:48:04.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56017fa640 1 -- 172.21.15.80:0/2549239154 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc014870 con 0x7f56040d5490 2024-04-25T13:48:04.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56040b6320 con 0x7f56040d5490 2024-04-25T13:48:04.673 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f560415ddc0 con 0x7f56040d5490 2024-04-25T13:48:04.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f560415b6f0 con 0x7f56040d5490 2024-04-25T13:48:04.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56017fa640 1 -- 172.21.15.80:0/2549239154 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f55fc01b020 con 0x7f56040d5490 2024-04-25T13:48:04.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56017fa640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f55e806f400 0x7f55e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:04.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56017fa640 1 -- 172.21.15.80:0/2549239154 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 4943+0+0 (secure 0 0 0) 0x7f55fc08fb20 con 0x7f56040d5490 2024-04-25T13:48:04.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56088f0640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f55e806f400 0x7f55e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:04.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.674+0000 7f56088f0640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f55e806f400 0x7f55e8071910 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f55f80ae000 tx=0x7f55f80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:04.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.678+0000 7f56017fa640 1 -- 172.21.15.80:0/2549239154 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f55fc022030 con 0x7f56040d5490 2024-04-25T13:48:04.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:04.854+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f560406a3a0 con 0x7f56040d5490 2024-04-25T13:48:05.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.882+0000 7f56017fa640 1 -- 172.21.15.80:0/2549239154 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v150) ==== 97+0+0 (secure 0 0 0) 0x7f55fc010040 con 0x7f56040d5490 2024-04-25T13:48:05.882 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f55e806f400 msgr2=0x7f55e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f55e806f400 0x7f55e8071910 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f55f80ae000 tx=0x7f55f80a7000 comp rx=0 tx=0).stop 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 msgr2=0x7f56040b5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040b5de0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f55fc009cb0 tx=0x7f55fc00c490 comp rx=0 tx=0).stop 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 shutdown_connections 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f55e806f400 0x7f55e8071910 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56040d5490 0x7f56040b5de0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56040d2560 0x7f56040b58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 --2- 172.21.15.80:0/2549239154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56040b92a0 0x7f56040cc170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 >> 172.21.15.80:0/2549239154 conn(0x7f5604073740 msgr2=0x7f56041628f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:05.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 shutdown_connections 2024-04-25T13:48:05.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:05.886+0000 7f560ab54640 1 -- 172.21.15.80:0/2549239154 wait complete. 2024-04-25T13:48:05.908 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:48:06.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.058+0000 7fe811aac640 1 Processor -- start 2024-04-25T13:48:06.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.058+0000 7fe811aac640 1 -- start start 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe811aac640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c161240 0x7fe80c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe811aac640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe80c161650 0x7fe80c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe811aac640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe80c06fcf0 con 0x7fe80c06f5a0 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe811aac640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe80c073f10 con 0x7fe80c161650 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe811aac640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c0b0c60 con 0x7fe80c161240 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80b7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe80c06f5a0 0x7fe80c160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51376/0 (socket says 172.21.15.80:51376) 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80b7fe640 1 -- 172.21.15.80:0/3978909503 learned_addr learned my addr 172.21.15.80:0/3978909503 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80affd640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c161240 0x7fe80c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80bfff640 1 --2- 172.21.15.80:0/3978909503 >> v2:172.21.15.87:3300/0 conn(0x7fe80c161650 0x7fe80c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3880666417 0 0) 0x7fe80c06fcf0 con 0x7fe80c06f5a0 2024-04-25T13:48:06.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7f4003530 con 0x7fe80c06f5a0 2024-04-25T13:48:06.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3071203156 0 0) 0x7fe7f4003530 con 0x7fe80c06f5a0 2024-04-25T13:48:06.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c161240 msgr2=0x7fe80c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c161240 0x7fe80c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 >> v2:172.21.15.87:3300/0 conn(0x7fe80c161650 msgr2=0x7fe80c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 --2- 172.21.15.80:0/3978909503 >> v2:172.21.15.87:3300/0 conn(0x7fe80c161650 0x7fe80c0728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c0b0f40 con 0x7fe80c06f5a0 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fe7fc001680 con 0x7fe80c06f5a0 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 >> v1:172.21.15.80:6789/0 conn(0x7fe80c06f5a0 legacy=0x7fe80c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7f40039c0 0x7fe7f4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f4004810 0x7fe7f4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c161240 0x7fe7f4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c073f10 con 0x7fe7f40039c0 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c0b0c60 con 0x7fe7f4004810 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe7f400ba80 con 0x7fe80c161240 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80b7fe640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7f40039c0 0x7fe7f4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80affd640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f4004810 0x7fe7f4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80bfff640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c161240 0x7fe7f4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80b7fe640 1 -- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c161240 msgr2=0x7fe7f4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80b7fe640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c161240 0x7fe7f4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80b7fe640 1 -- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f4004810 msgr2=0x7fe7f4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80b7fe640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe7f4004810 0x7fe7f4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.062+0000 7fe80b7fe640 1 -- 172.21.15.80:0/3978909503 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f4003530 con 0x7fe7f40039c0 2024-04-25T13:48:06.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe80b7fe640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7f40039c0 0x7fe7f4003e30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc001060 tx=0x7fe7fc01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac01e0eaaecd807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc007d80 con 0x7fe7f40039c0 2024-04-25T13:48:06.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7fc0a9020 con 0x7fe7f40039c0 2024-04-25T13:48:06.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe80a7fc640 1 -- 172.21.15.80:0/3978909503 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc005b00 con 0x7fe7f40039c0 2024-04-25T13:48:06.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe811aac640 1 -- 172.21.15.80:0/3978909503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7f40039c0 msgr2=0x7fe7f4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe811aac640 1 --2- 172.21.15.80:0/3978909503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe7f40039c0 0x7fe7f4003e30 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc001060 tx=0x7fe7fc01cc80 comp rx=0 tx=0).stop 2024-04-25T13:48:06.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe80bfff640 1 -- 172.21.15.80:0/3978909503 reap_dead start 2024-04-25T13:48:06.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe811aac640 1 -- 172.21.15.80:0/3978909503 shutdown_connections 2024-04-25T13:48:06.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe811aac640 1 -- 172.21.15.80:0/3978909503 >> 172.21.15.80:0/3978909503 conn(0x7fe80c06d080 msgr2=0x7fe80c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:06.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe811aac640 1 -- 172.21.15.80:0/3978909503 shutdown_connections 2024-04-25T13:48:06.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe811aac640 1 -- 172.21.15.80:0/3978909503 wait complete. 2024-04-25T13:48:06.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe811aac640 1 Processor -- start 2024-04-25T13:48:06.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.066+0000 7fe811aac640 1 -- start start 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe80c151240 0x7fe80c15d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c0afdb0 0x7fe80c0cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe80c0b01c0 0x7fe80c0cfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c168250 con 0x7fe80c151240 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe80c1680d0 con 0x7fe80c0b01c0 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe80c1683d0 con 0x7fe80c0afdb0 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe80c151240 0x7fe80c15d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe80c151240 0x7fe80c15d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53664/0 (socket says 172.21.15.80:53664) 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 -- 172.21.15.80:0/1428322936 learned_addr learned my addr 172.21.15.80:0/1428322936 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80affd640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c0afdb0 0x7fe80c0cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80bfff640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe80c0b01c0 0x7fe80c0cfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 -- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c0afdb0 msgr2=0x7fe80c0cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c0afdb0 0x7fe80c0cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 -- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe80c0b01c0 msgr2=0x7fe80c0cfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe80c0b01c0 0x7fe80c0cfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 -- 172.21.15.80:0/1428322936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80c15daf0 con 0x7fe80c151240 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80affd640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c0afdb0 0x7fe80c0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80bfff640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe80c0b01c0 0x7fe80c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe80b7fe640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe80c151240 0x7fe80c15d5b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc01c6f0 tx=0x7fe7f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe7f3fff640 1 -- 172.21.15.80:0/1428322936 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc004ab0 con 0x7fe80c151240 2024-04-25T13:48:06.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe7f3fff640 1 -- 172.21.15.80:0/1428322936 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7fc0b2070 con 0x7fe80c151240 2024-04-25T13:48:06.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe7f3fff640 1 -- 172.21.15.80:0/1428322936 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc004e50 con 0x7fe80c151240 2024-04-25T13:48:06.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80c15a500 con 0x7fe80c151240 2024-04-25T13:48:06.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe80c15aac0 con 0x7fe80c151240 2024-04-25T13:48:06.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80c06a3a0 con 0x7fe80c151240 2024-04-25T13:48:06.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.070+0000 7fe7f3fff640 1 -- 172.21.15.80:0/1428322936 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe7fc0a5760 con 0x7fe80c151240 2024-04-25T13:48:06.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.074+0000 7fe7f3fff640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7f806f5a0 0x7fe7f8071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.074+0000 7fe7f3fff640 1 -- 172.21.15.80:0/1428322936 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7fe7fc128300 con 0x7fe80c151240 2024-04-25T13:48:06.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.074+0000 7fe80affd640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7f806f5a0 0x7fe7f8071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.074+0000 7fe80affd640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7f806f5a0 0x7fe7f8071ab0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe800009a70 tx=0x7fe800008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.078+0000 7fe7f3fff640 1 -- 172.21.15.80:0/1428322936 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe7fc016990 con 0x7fe80c151240 2024-04-25T13:48:06.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.278+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe80c0739c0 con 0x7fe80c151240 2024-04-25T13:48:06.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.278+0000 7fe7f3fff640 1 -- 172.21.15.80:0/1428322936 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fe7fc0f5540 con 0x7fe80c151240 2024-04-25T13:48:06.278 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:06.278 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:48:06.278 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:48:06.278 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.278 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:48:06.279 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.280 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.281 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:48:06.282 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.283 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:48:06.284 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.285 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.286 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:48:06.287 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:48:06.288 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.289 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:48:06.290 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:48:06.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7f806f5a0 msgr2=0x7fe7f8071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7f806f5a0 0x7fe7f8071ab0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe800009a70 tx=0x7fe800008040 comp rx=0 tx=0).stop 2024-04-25T13:48:06.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe80c151240 msgr2=0x7fe80c15d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.291 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe80c151240 0x7fe80c15d5b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc01c6f0 tx=0x7fe7f4005840 comp rx=0 tx=0).stop 2024-04-25T13:48:06.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 shutdown_connections 2024-04-25T13:48:06.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe7f806f5a0 0x7fe7f8071ab0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe80c0b01c0 0x7fe80c0cfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe80c0afdb0 0x7fe80c0cf6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 --2- 172.21.15.80:0/1428322936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe80c151240 0x7fe80c15d5b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 >> 172.21.15.80:0/1428322936 conn(0x7fe80c06d080 msgr2=0x7fe80c0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:06.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 shutdown_connections 2024-04-25T13:48:06.292 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.282+0000 7fe811aac640 1 -- 172.21.15.80:0/1428322936 wait complete. 2024-04-25T13:48:06.300 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:48:06.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.434+0000 7f9cbedad640 1 Processor -- start 2024-04-25T13:48:06.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.434+0000 7f9cbedad640 1 -- start start 2024-04-25T13:48:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbedad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 0x7f9cb80ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbedad640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9cb80cec60 0x7f9cb80cfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbedad640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9cb80cf070 con 0x7f9cb80d0150 2024-04-25T13:48:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbedad640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9cb8070020 con 0x7f9cb80cec60 2024-04-25T13:48:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbedad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb815e7a0 con 0x7f9cb806fc10 2024-04-25T13:48:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbd34a640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9cb80d0150 0x7f9cb8161db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51380/0 (socket says 172.21.15.80:51380) 2024-04-25T13:48:06.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbd34a640 1 -- 172.21.15.80:0/1867452482 learned_addr learned my addr 172.21.15.80:0/1867452482 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb7fff640 1 --2- 172.21.15.80:0/1867452482 >> v2:172.21.15.87:3300/0 conn(0x7f9cb80cec60 0x7f9cb80cfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbcb49640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 0x7f9cb80ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2500108482 0 0) 0x7f9cb00014a0 con 0x7f9cb80d0150 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ca8003530 con 0x7f9cb80d0150 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2958967487 0 0) 0x7f9ca8003530 con 0x7f9cb80d0150 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 msgr2=0x7f9cb80ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 0x7f9cb80ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 >> v2:172.21.15.87:3300/0 conn(0x7f9cb80cec60 msgr2=0x7f9cb80cfc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 --2- 172.21.15.80:0/1867452482 >> v2:172.21.15.87:3300/0 conn(0x7f9cb80cec60 0x7f9cb80cfc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb815ea60 con 0x7f9cb80d0150 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f9cb0001bd0 con 0x7f9cb80d0150 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 >> v1:172.21.15.80:6789/0 conn(0x7f9cb80d0150 legacy=0x7f9cb8161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca80039c0 0x7f9ca8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.439 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ca8004810 0x7f9ca8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 0x7f9ca8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb8070020 con 0x7f9ca8004810 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb815e7a0 con 0x7f9ca80039c0 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ca800b9d0 con 0x7f9cb806fc10 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb7fff640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ca8004810 0x7f9ca8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbd34a640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 0x7f9ca8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbcb49640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca80039c0 0x7f9ca8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb7fff640 1 -- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 msgr2=0x7f9ca8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb7fff640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 0x7f9ca8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb7fff640 1 -- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca80039c0 msgr2=0x7f9ca8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb7fff640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca80039c0 0x7f9ca8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cb7fff640 1 -- 172.21.15.80:0/1867452482 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ca8003530 con 0x7f9ca8004810 2024-04-25T13:48:06.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbd34a640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9cb806fc10 0x7f9ca8005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:06.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.438+0000 7f9cbcb49640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ca80039c0 0x7f9ca8003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cb7fff640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ca8004810 0x7f9ca8004c20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f9cb806f5a0 tx=0x7f9ca401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b577d7dacb41fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40096b0 con 0x7f9ca8004810 2024-04-25T13:48:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca40a6480 con 0x7f9ca8004810 2024-04-25T13:48:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cb77fe640 1 -- 172.21.15.80:0/1867452482 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40a6830 con 0x7f9ca8004810 2024-04-25T13:48:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbedad640 1 -- 172.21.15.80:0/1867452482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ca8004810 msgr2=0x7f9ca8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbedad640 1 --2- 172.21.15.80:0/1867452482 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ca8004810 0x7f9ca8004c20 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f9cb806f5a0 tx=0x7f9ca401c990 comp rx=0 tx=0).stop 2024-04-25T13:48:06.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbd34a640 1 -- 172.21.15.80:0/1867452482 reap_dead start 2024-04-25T13:48:06.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbedad640 1 -- 172.21.15.80:0/1867452482 shutdown_connections 2024-04-25T13:48:06.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbedad640 1 -- 172.21.15.80:0/1867452482 >> 172.21.15.80:0/1867452482 conn(0x7f9cb80d1d10 msgr2=0x7f9cb80d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:06.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbedad640 1 -- 172.21.15.80:0/1867452482 shutdown_connections 2024-04-25T13:48:06.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbedad640 1 -- 172.21.15.80:0/1867452482 wait complete. 2024-04-25T13:48:06.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbedad640 1 Processor -- start 2024-04-25T13:48:06.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.442+0000 7f9cbedad640 1 -- start start 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca8004810 0x7f9cb81409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cb815af90 0x7f9cb8073b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb815b3a0 0x7f9cb80740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb815f300 con 0x7f9cb815af90 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9cb815f020 con 0x7f9cb815b3a0 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9cb806e310 con 0x7f9ca8004810 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cb815af90 0x7f9cb8073b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cb815af90 0x7f9cb8073b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53676/0 (socket says 172.21.15.80:53676) 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 -- 172.21.15.80:0/1711520382 learned_addr learned my addr 172.21.15.80:0/1711520382 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbd34a640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb815b3a0 0x7f9cb80740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbcb49640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca8004810 0x7f9cb81409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 -- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca8004810 msgr2=0x7f9cb81409e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca8004810 0x7f9cb81409e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 -- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb815b3a0 msgr2=0x7f9cb80740a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb815b3a0 0x7f9cb80740a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 -- 172.21.15.80:0/1711520382 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cb81437a0 con 0x7f9cb815af90 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb7fff640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cb815af90 0x7f9cb8073b60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4000c10 tx=0x7f9ca8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb57fa640 1 -- 172.21.15.80:0/1711520382 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40be070 con 0x7f9cb815af90 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb57fa640 1 -- 172.21.15.80:0/1711520382 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9ca40a7460 con 0x7f9cb815af90 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cb81447f0 con 0x7f9cb815af90 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb57fa640 1 -- 172.21.15.80:0/1711520382 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ca40096b0 con 0x7f9cb815af90 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cb8143f50 con 0x7f9cb815af90 2024-04-25T13:48:06.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cb806a3a0 con 0x7f9cb815af90 2024-04-25T13:48:06.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.446+0000 7f9cb57fa640 1 -- 172.21.15.80:0/1711520382 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9ca40a7850 con 0x7f9cb815af90 2024-04-25T13:48:06.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.450+0000 7f9cb57fa640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9c06f430 0x7f9c9c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.450+0000 7f9cb57fa640 1 -- 172.21.15.80:0/1711520382 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7f9ca4128d00 con 0x7f9cb815af90 2024-04-25T13:48:06.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.450+0000 7f9cbcb49640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9c06f430 0x7f9c9c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.450+0000 7f9cbcb49640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9c06f430 0x7f9c9c071940 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0090e0 tx=0x7f9cac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.454+0000 7f9cb57fa640 1 -- 172.21.15.80:0/1711520382 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9ca40ac020 con 0x7f9cb815af90 2024-04-25T13:48:06.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.614+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9cb8143a00 con 0x7f9cb815af90 2024-04-25T13:48:06.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cb57fa640 1 -- 172.21.15.80:0/1711520382 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+12830 (secure 0 0 0) 0x7f9ca40166a0 con 0x7f9cb815af90 2024-04-25T13:48:06.615 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:06.615 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":150,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:48:05.884316+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:48:06.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9c06f430 msgr2=0x7f9c9c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9c06f430 0x7f9c9c071940 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9cac0090e0 tx=0x7f9cac008040 comp rx=0 tx=0).stop 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cb815af90 msgr2=0x7f9cb8073b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cb815af90 0x7f9cb8073b60 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f9ca4000c10 tx=0x7f9ca8005840 comp rx=0 tx=0).stop 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 shutdown_connections 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9c9c06f430 0x7f9c9c071940 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9cb815b3a0 0x7f9cb80740a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9cb815af90 0x7f9cb8073b60 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 --2- 172.21.15.80:0/1711520382 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ca8004810 0x7f9cb81409e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 >> 172.21.15.80:0/1711520382 conn(0x7f9cb80d1d10 msgr2=0x7f9cb80b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 shutdown_connections 2024-04-25T13:48:06.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.618+0000 7f9cbedad640 1 -- 172.21.15.80:0/1711520382 wait complete. 2024-04-25T13:48:06.637 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:48:06.638 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:48:06.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f735d765640 1 Processor -- start 2024-04-25T13:48:06.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f735d765640 1 -- start start 2024-04-25T13:48:06.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f735d765640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73581602a0 0x7f73580ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f735d765640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f73580d0ae0 0x7f73580d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f735d765640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73580d26c0 con 0x7f7358068590 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f735d765640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f735806f270 con 0x7f73580d0ae0 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f735d765640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f735806f550 con 0x7f73581602a0 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f73577fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7358068590 0x7f73580689a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51384/0 (socket says 172.21.15.80:51384) 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f73577fe640 1 -- 172.21.15.80:0/2195684637 learned_addr learned my addr 172.21.15.80:0/2195684637 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f73567fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f73580d0ae0 0x7f73580d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f7356ffd640 1 --2- 172.21.15.80:0/2195684637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73581602a0 0x7f73580ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f7355ffb640 1 -- 172.21.15.80:0/2195684637 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1820427010 0 0) 0x7f73580d26c0 con 0x7f7358068590 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f7355ffb640 1 -- 172.21.15.80:0/2195684637 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7338003530 con 0x7f7358068590 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f73567fc640 1 -- 172.21.15.80:0/2195684637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73581602a0 msgr2=0x7f73580ceb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f73567fc640 1 --2- 172.21.15.80:0/2195684637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73581602a0 0x7f73580ceb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f73567fc640 1 -- 172.21.15.80:0/2195684637 >> v1:172.21.15.80:6789/0 conn(0x7f7358068590 legacy=0x7f73580689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.794+0000 7f73567fc640 1 -- 172.21.15.80:0/2195684637 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73580d21a0 con 0x7f73580d0ae0 2024-04-25T13:48:06.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f73567fc640 1 --2- 172.21.15.80:0/2195684637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73580d0ae0 0x7f73580d0ef0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f73400097c0 tx=0x7f734001c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db012b3c6779e61f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f7355ffb640 1 -- 172.21.15.80:0/2195684637 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73400b0070 con 0x7f73580d0ae0 2024-04-25T13:48:06.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f7355ffb640 1 -- 172.21.15.80:0/2195684637 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73400ab600 con 0x7f73580d0ae0 2024-04-25T13:48:06.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f7355ffb640 1 -- 172.21.15.80:0/2195684637 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73400ab9d0 con 0x7f73580d0ae0 2024-04-25T13:48:06.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f735d765640 1 -- 172.21.15.80:0/2195684637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73580d0ae0 msgr2=0x7f73580d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f735d765640 1 --2- 172.21.15.80:0/2195684637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73580d0ae0 0x7f73580d0ef0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f73400097c0 tx=0x7f734001c380 comp rx=0 tx=0).stop 2024-04-25T13:48:06.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f735d765640 1 -- 172.21.15.80:0/2195684637 shutdown_connections 2024-04-25T13:48:06.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f735d765640 1 --2- 172.21.15.80:0/2195684637 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73580d0ae0 0x7f73580d0ef0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f735d765640 1 --2- 172.21.15.80:0/2195684637 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73581602a0 0x7f73580ceb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f735d765640 1 -- 172.21.15.80:0/2195684637 >> 172.21.15.80:0/2195684637 conn(0x7f73580cfa50 msgr2=0x7f73580cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:06.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f735d765640 1 -- 172.21.15.80:0/2195684637 shutdown_connections 2024-04-25T13:48:06.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.798+0000 7f735d765640 1 -- 172.21.15.80:0/2195684637 wait complete. 2024-04-25T13:48:06.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 Processor -- start 2024-04-25T13:48:06.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 -- start start 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7358068590 0x7f735814ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73580d0ae0 0x7f735814f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73581602a0 0x7f735815e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7358160830 con 0x7f73581602a0 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f73581606b0 con 0x7f7358068590 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7358071730 con 0x7f73580d0ae0 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73581602a0 0x7f735815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73581602a0 0x7f735815e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53680/0 (socket says 172.21.15.80:53680) 2024-04-25T13:48:06.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 -- 172.21.15.80:0/49427247 learned_addr learned my addr 172.21.15.80:0/49427247 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73567fc640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73580d0ae0 0x7f735814f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f7356ffd640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7358068590 0x7f735814ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 -- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73580d0ae0 msgr2=0x7f735814f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73580d0ae0 0x7f735814f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 -- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7358068590 msgr2=0x7f735814ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7358068590 0x7f735814ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 -- 172.21.15.80:0/49427247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7358163620 con 0x7f73581602a0 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f73577fe640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73581602a0 0x7f735815e320 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f735000caa0 tx=0x7f735000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f734f7fe640 1 -- 172.21.15.80:0/49427247 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350017070 con 0x7f73581602a0 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f734f7fe640 1 -- 172.21.15.80:0/49427247 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f735000ac00 con 0x7f73581602a0 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f734f7fe640 1 -- 172.21.15.80:0/49427247 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7350012400 con 0x7f73581602a0 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7358163900 con 0x7f73581602a0 2024-04-25T13:48:06.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73580b64e0 con 0x7f73581602a0 2024-04-25T13:48:06.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.802+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7358163390 con 0x7f73581602a0 2024-04-25T13:48:06.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.806+0000 7f734f7fe640 1 -- 172.21.15.80:0/49427247 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f735001a070 con 0x7f73581602a0 2024-04-25T13:48:06.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.806+0000 7f734f7fe640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f733c06f400 0x7f733c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:06.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.806+0000 7f734f7fe640 1 -- 172.21.15.80:0/49427247 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7f735008ec00 con 0x7f73581602a0 2024-04-25T13:48:06.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.806+0000 7f7356ffd640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f733c06f400 0x7f733c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:06.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.806+0000 7f7356ffd640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f733c06f400 0x7f733c071910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f735806e080 tx=0x7f7348008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:06.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.810+0000 7f734f7fe640 1 -- 172.21.15.80:0/49427247 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7350095080 con 0x7f73581602a0 2024-04-25T13:48:06.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.970+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f73580b2180 con 0x7f73581602a0 2024-04-25T13:48:06.969 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.970+0000 7f734f7fe640 1 -- 172.21.15.80:0/49427247 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v150) ==== 93+0+10 (secure 0 0 0) 0x7f735005b7f0 con 0x7f73581602a0 2024-04-25T13:48:06.969 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:48:06.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f733c06f400 msgr2=0x7f733c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f733c06f400 0x7f733c071910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f735806e080 tx=0x7f7348008040 comp rx=0 tx=0).stop 2024-04-25T13:48:06.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73581602a0 msgr2=0x7f735815e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:06.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73581602a0 0x7f735815e320 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f735000caa0 tx=0x7f735000cf60 comp rx=0 tx=0).stop 2024-04-25T13:48:06.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 shutdown_connections 2024-04-25T13:48:06.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f733c06f400 0x7f733c071910 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f73581602a0 0x7f735815e320 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f73580d0ae0 0x7f735814f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 --2- 172.21.15.80:0/49427247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7358068590 0x7f735814ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:06.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 >> 172.21.15.80:0/49427247 conn(0x7f73580cfa50 msgr2=0x7f73580b2830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:06.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 shutdown_connections 2024-04-25T13:48:06.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:06.974+0000 7f735d765640 1 -- 172.21.15.80:0/49427247 wait complete. 2024-04-25T13:48:06.992 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:48:07.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.126+0000 7f83da173640 1 Processor -- start 2024-04-25T13:48:07.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.126+0000 7f83da173640 1 -- start start 2024-04-25T13:48:07.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83da173640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f83d4161240 0x7f83d40723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83da173640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d406fcf0 con 0x7f83d406f5a0 2024-04-25T13:48:07.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83da173640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f83d40b0c60 con 0x7f83d4161650 2024-04-25T13:48:07.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83da173640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f83d4073f10 con 0x7f83d4161240 2024-04-25T13:48:07.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d37fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f83d406f5a0 0x7f83d4160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51394/0 (socket says 172.21.15.80:51394) 2024-04-25T13:48:07.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d37fe640 1 -- 172.21.15.80:0/2855808609 learned_addr learned my addr 172.21.15.80:0/2855808609 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d2ffd640 1 --2- 172.21.15.80:0/2855808609 >> v2:172.21.15.87:3300/0 conn(0x7f83d4161240 0x7f83d40723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 612531375 0 0) 0x7f83d406fcf0 con 0x7f83d406f5a0 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f83bc003530 con 0x7f83d406f5a0 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3260378070 0 0) 0x7f83bc003530 con 0x7f83d406f5a0 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 >> v2:172.21.15.87:3300/0 conn(0x7f83d4161240 msgr2=0x7f83d40723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 --2- 172.21.15.80:0/2855808609 >> v2:172.21.15.87:3300/0 conn(0x7f83d4161240 0x7f83d40723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 >> v1:172.21.15.87:6789/0 conn(0x7f83d4161650 legacy=0x7f83d40728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d40b0f40 con 0x7f83d406f5a0 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f83c4001680 con 0x7f83d406f5a0 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 >> v1:172.21.15.80:6789/0 conn(0x7f83d406f5a0 legacy=0x7f83d4160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0039c0 0x7f83bc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc004810 0x7f83bc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83bc005260 0x7f83bc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f83d4073f10 con 0x7f83bc004810 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f83bc00c1f0 con 0x7f83bc005260 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f83bc00c3f0 con 0x7f83bc0039c0 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d2ffd640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc004810 0x7f83bc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d37fe640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0039c0 0x7f83bc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d2ffd640 1 -- 172.21.15.80:0/2855808609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0039c0 msgr2=0x7f83bc003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d2ffd640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc0039c0 0x7f83bc003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d2ffd640 1 -- 172.21.15.80:0/2855808609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83bc005260 msgr2=0x7f83bc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d2ffd640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83bc005260 0x7f83bc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d3fff640 1 -- 172.21.15.80:0/2855808609 reap_dead start 2024-04-25T13:48:07.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.130+0000 7f83d2ffd640 1 -- 172.21.15.80:0/2855808609 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83bc003530 con 0x7f83bc004810 2024-04-25T13:48:07.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83d2ffd640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc004810 0x7f83bc004c20 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009ae0 tx=0x7f83c801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6563637f2adb01f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:07.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c80096b0 con 0x7f83bc004810 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c80a64f0 con 0x7f83bc004810 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83d27fc640 1 -- 172.21.15.80:0/2855808609 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83c80a68e0 con 0x7f83bc004810 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83da173640 1 -- 172.21.15.80:0/2855808609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc004810 msgr2=0x7f83bc004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83da173640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc004810 0x7f83bc004c20 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f83c8009ae0 tx=0x7f83c801ca90 comp rx=0 tx=0).stop 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83da173640 1 -- 172.21.15.80:0/2855808609 shutdown_connections 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83da173640 1 --2- 172.21.15.80:0/2855808609 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83bc004810 0x7f83bc004c20 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83da173640 1 -- 172.21.15.80:0/2855808609 >> 172.21.15.80:0/2855808609 conn(0x7f83d406d080 msgr2=0x7f83d40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83da173640 1 -- 172.21.15.80:0/2855808609 shutdown_connections 2024-04-25T13:48:07.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83da173640 1 -- 172.21.15.80:0/2855808609 wait complete. 2024-04-25T13:48:07.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.134+0000 7f83da173640 1 Processor -- start 2024-04-25T13:48:07.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 -- start start 2024-04-25T13:48:07.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc004810 0x7f83d4147470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83d415d560 0x7f83d4157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4157a90 0x7f83d41e8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40b5560 con 0x7f83d4157a90 2024-04-25T13:48:07.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f83d40b53e0 con 0x7f83d415d560 2024-04-25T13:48:07.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f83d40b56e0 con 0x7f83bc004810 2024-04-25T13:48:07.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4157a90 0x7f83d41e8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4157a90 0x7f83d41e8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53684/0 (socket says 172.21.15.80:53684) 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 -- 172.21.15.80:0/3478030211 learned_addr learned my addr 172.21.15.80:0/3478030211 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d37fe640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc004810 0x7f83d4147470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d2ffd640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83d415d560 0x7f83d4157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 -- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc004810 msgr2=0x7f83d4147470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc004810 0x7f83d4147470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 -- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83d415d560 msgr2=0x7f83d4157550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83d415d560 0x7f83d4157550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 -- 172.21.15.80:0/3478030211 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83d4157ea0 con 0x7f83d4157a90 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83d3fff640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4157a90 0x7f83d41e8bb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f83d40b4970 tx=0x7f83cc00ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83bbfff640 1 -- 172.21.15.80:0/3478030211 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc019070 con 0x7f83d4157a90 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83bbfff640 1 -- 172.21.15.80:0/3478030211 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83cc00cbd0 con 0x7f83d4157a90 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83bbfff640 1 -- 172.21.15.80:0/3478030211 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83cc014400 con 0x7f83d4157a90 2024-04-25T13:48:07.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83d4160610 con 0x7f83d4157a90 2024-04-25T13:48:07.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83d4144da0 con 0x7f83d4157a90 2024-04-25T13:48:07.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83d406a3a0 con 0x7f83d4157a90 2024-04-25T13:48:07.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.138+0000 7f83bbfff640 1 -- 172.21.15.80:0/3478030211 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f83cc01c070 con 0x7f83d4157a90 2024-04-25T13:48:07.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.142+0000 7f83bbfff640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f83c006f400 0x7f83c0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.142+0000 7f83bbfff640 1 -- 172.21.15.80:0/3478030211 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7f83cc0909d0 con 0x7f83d4157a90 2024-04-25T13:48:07.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.142+0000 7f83d37fe640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f83c006f400 0x7f83c0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.142+0000 7f83d37fe640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f83c006f400 0x7f83c0071910 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f83c4001c20 tx=0x7f83c4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:07.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.146+0000 7f83bbfff640 1 -- 172.21.15.80:0/3478030211 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f83d4157ea0 con 0x7f83d4157a90 2024-04-25T13:48:07.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.294+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f83d40739c0 con 0x7f83d4157a90 2024-04-25T13:48:07.293 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.294+0000 7f83bbfff640 1 -- 172.21.15.80:0/3478030211 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v150) ==== 84+0+65 (secure 0 0 0) 0x7f83cc05dc10 con 0x7f83d4157a90 2024-04-25T13:48:07.293 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:07.293 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:48:07.294 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:48:07.294 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:48:07.294 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:48:07.294 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:07.294 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f83c006f400 msgr2=0x7f83c0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f83c006f400 0x7f83c0071910 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f83c4001c20 tx=0x7f83c4009040 comp rx=0 tx=0).stop 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4157a90 msgr2=0x7f83d41e8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4157a90 0x7f83d41e8bb0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f83d40b4970 tx=0x7f83cc00ef00 comp rx=0 tx=0).stop 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 shutdown_connections 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f83c006f400 0x7f83c0071910 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f83d4157a90 0x7f83d41e8bb0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f83d415d560 0x7f83d4157550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 --2- 172.21.15.80:0/3478030211 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f83bc004810 0x7f83d4147470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 >> 172.21.15.80:0/3478030211 conn(0x7f83d406d080 msgr2=0x7f83d40b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 shutdown_connections 2024-04-25T13:48:07.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.298+0000 7f83da173640 1 -- 172.21.15.80:0/3478030211 wait complete. 2024-04-25T13:48:07.314 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:48:07.314 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:48:07.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc2a2865640 1 Processor -- start 2024-04-25T13:48:07.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc2a2865640 1 -- start start 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc2a2865640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c0d24a0 0x7fc29c0d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc2a2865640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc29c0d4820 0x7fc29c0d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc2a2865640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc29c164290 con 0x7fc29c0d0ed0 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc2a2865640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc29c164040 con 0x7fc29c0d4820 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc2a2865640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c164bc0 con 0x7fc29c0d24a0 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc29bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc29c0d0ed0 0x7fc29c0d12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51398/0 (socket says 172.21.15.80:51398) 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc29bfff640 1 -- 172.21.15.80:0/2546221806 learned_addr learned my addr 172.21.15.80:0/2546221806 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc2a0e02640 1 --2- 172.21.15.80:0/2546221806 >> v2:172.21.15.87:3300/0 conn(0x7fc29c0d4820 0x7fc29c0d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.470+0000 7fc29b7fe640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c0d24a0 0x7fc29c0d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3101411424 0 0) 0x7fc29c164290 con 0x7fc29c0d0ed0 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc284003530 con 0x7fc29c0d0ed0 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3200351380 0 0) 0x7fc284003530 con 0x7fc29c0d0ed0 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c0d24a0 msgr2=0x7fc29c0d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c0d24a0 0x7fc29c0d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 >> v2:172.21.15.87:3300/0 conn(0x7fc29c0d4820 msgr2=0x7fc29c0d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 --2- 172.21.15.80:0/2546221806 >> v2:172.21.15.87:3300/0 conn(0x7fc29c0d4820 0x7fc29c0d4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c164570 con 0x7fc29c0d0ed0 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc28c001680 con 0x7fc29c0d0ed0 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 >> v1:172.21.15.80:6789/0 conn(0x7fc29c0d0ed0 legacy=0x7fc29c0d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2840039c0 0x7fc284003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc284004810 0x7fc284004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c0d24a0 0x7fc284005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c164040 con 0x7fc2840039c0 2024-04-25T13:48:07.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c164bc0 con 0x7fc284004810 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc28400b9d0 con 0x7fc29c0d24a0 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29bfff640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2840039c0 0x7fc284003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc2a0e02640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c0d24a0 0x7fc284005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29b7fe640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc284004810 0x7fc284004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29bfff640 1 -- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c0d24a0 msgr2=0x7fc284005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29bfff640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c0d24a0 0x7fc284005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29bfff640 1 -- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc284004810 msgr2=0x7fc284004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29bfff640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc284004810 0x7fc284004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29bfff640 1 -- 172.21.15.80:0/2546221806 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc284003530 con 0x7fc2840039c0 2024-04-25T13:48:07.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29b7fe640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc284004810 0x7fc284004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:07.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29bfff640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2840039c0 0x7fc284003e30 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00c210 tx=0x7fc28c01cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b83bea2dea2e1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:07.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c007d60 con 0x7fc2840039c0 2024-04-25T13:48:07.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc28c0a9020 con 0x7fc2840039c0 2024-04-25T13:48:07.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.474+0000 7fc29affd640 1 -- 172.21.15.80:0/2546221806 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c0a5bb0 con 0x7fc2840039c0 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- 172.21.15.80:0/2546221806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2840039c0 msgr2=0x7fc284003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 --2- 172.21.15.80:0/2546221806 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc2840039c0 0x7fc284003e30 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fc28c00c210 tx=0x7fc28c01cc70 comp rx=0 tx=0).stop 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a0e02640 1 -- 172.21.15.80:0/2546221806 reap_dead start 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- 172.21.15.80:0/2546221806 shutdown_connections 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- 172.21.15.80:0/2546221806 >> 172.21.15.80:0/2546221806 conn(0x7fc29c06c5f0 msgr2=0x7fc29c06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- 172.21.15.80:0/2546221806 shutdown_connections 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- 172.21.15.80:0/2546221806 wait complete. 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 Processor -- start 2024-04-25T13:48:07.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- start start 2024-04-25T13:48:07.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c1614d0 0x7fc29c14c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc29c14a470 0x7fc29c1566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc29c1fd2a0 0x7fc29c1fd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c0d5500 con 0x7fc29c14a470 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c0d5380 con 0x7fc29c1fd2a0 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc2a2865640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc29c0d5680 con 0x7fc29c1614d0 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc29b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc29c14a470 0x7fc29c1566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc29b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc29c14a470 0x7fc29c1566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53700/0 (socket says 172.21.15.80:53700) 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.478+0000 7fc29b7fe640 1 -- 172.21.15.80:0/1125656982 learned_addr learned my addr 172.21.15.80:0/1125656982 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc2a0e02640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc29c1fd2a0 0x7fc29c1fd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29bfff640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c1614d0 0x7fc29c14c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29b7fe640 1 -- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c1614d0 msgr2=0x7fc29c14c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29b7fe640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c1614d0 0x7fc29c14c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29b7fe640 1 -- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc29c1fd2a0 msgr2=0x7fc29c1fd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29b7fe640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc29c1fd2a0 0x7fc29c1fd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29b7fe640 1 -- 172.21.15.80:0/1125656982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c1fe100 con 0x7fc29c14a470 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29b7fe640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc29c14a470 0x7fc29c1566a0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fc2900108d0 tx=0x7fc290010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:07.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc298ff9640 1 -- 172.21.15.80:0/1125656982 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29000f040 con 0x7fc29c14a470 2024-04-25T13:48:07.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc298ff9640 1 -- 172.21.15.80:0/1125656982 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc290004220 con 0x7fc29c14a470 2024-04-25T13:48:07.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c1fdbf0 con 0x7fc29c14a470 2024-04-25T13:48:07.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc298ff9640 1 -- 172.21.15.80:0/1125656982 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc29000e960 con 0x7fc29c14a470 2024-04-25T13:48:07.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c156be0 con 0x7fc29c14a470 2024-04-25T13:48:07.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c06a440 con 0x7fc29c14a470 2024-04-25T13:48:07.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc298ff9640 1 -- 172.21.15.80:0/1125656982 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc29000eb00 con 0x7fc29c14a470 2024-04-25T13:48:07.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc298ff9640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc28806f400 0x7fc288071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:07.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc298ff9640 1 -- 172.21.15.80:0/1125656982 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 4857+0+0 (secure 0 0 0) 0x7fc29008ef70 con 0x7fc29c14a470 2024-04-25T13:48:07.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29bfff640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc28806f400 0x7fc288071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:07.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.482+0000 7fc29bfff640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc28806f400 0x7fc288071910 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0aa000 tx=0x7fc28c000ff0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:07.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.486+0000 7fc298ff9640 1 -- 172.21.15.80:0/1125656982 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc29c1fdbf0 con 0x7fc29c14a470 2024-04-25T13:48:07.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:07.650+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc29c160400 con 0x7fc29c14a470 2024-04-25T13:48:08.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.502+0000 7fc298ff9640 1 -- 172.21.15.80:0/1125656982 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v151) ==== 138+0+0 (secure 0 0 0) 0x7fc29c1fe100 con 0x7fc29c14a470 2024-04-25T13:48:08.502 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:48:08.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc28806f400 msgr2=0x7fc288071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:08.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc28806f400 0x7fc288071910 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc28c0aa000 tx=0x7fc28c000ff0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc29c14a470 msgr2=0x7fc29c1566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:08.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc29c14a470 0x7fc29c1566a0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fc2900108d0 tx=0x7fc290010d90 comp rx=0 tx=0).stop 2024-04-25T13:48:08.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 shutdown_connections 2024-04-25T13:48:08.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc28806f400 0x7fc288071910 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc29c1fd2a0 0x7fc29c1fd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc29c14a470 0x7fc29c1566a0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 --2- 172.21.15.80:0/1125656982 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc29c1614d0 0x7fc29c14c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 >> 172.21.15.80:0/1125656982 conn(0x7fc29c06c5f0 msgr2=0x7fc29c06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:08.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 shutdown_connections 2024-04-25T13:48:08.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.506+0000 7fc2a2865640 1 -- 172.21.15.80:0/1125656982 wait complete. 2024-04-25T13:48:08.532 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:48:08.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.694+0000 7f32f0cc5640 1 Processor -- start 2024-04-25T13:48:08.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.694+0000 7f32f0cc5640 1 -- start start 2024-04-25T13:48:08.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32f0cc5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32ec0d06e0 0x7f32ec06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:08.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32f0cc5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f32ec06e9e0 0x7f32ec06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:08.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32f0cc5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32ec0d0af0 con 0x7f32ec161f90 2024-04-25T13:48:08.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32f0cc5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f32ec0d0d70 con 0x7f32ec0d06e0 2024-04-25T13:48:08.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32f0cc5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f32ec0d1050 con 0x7f32ec06e9e0 2024-04-25T13:48:08.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32e3fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32ec0d06e0 0x7f32ec06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:08.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32e3fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32ec0d06e0 0x7f32ec06e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53704/0 (socket says 172.21.15.80:53704) 2024-04-25T13:48:08.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32e3fff640 1 -- 172.21.15.80:0/803910648 learned_addr learned my addr 172.21.15.80:0/803910648 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:08.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32ead9d640 1 --2- 172.21.15.80:0/803910648 >> v2:172.21.15.87:3300/0 conn(0x7f32ec06e9e0 0x7f32ec06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:08.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32e3fff640 1 -- 172.21.15.80:0/803910648 >> v2:172.21.15.87:3300/0 conn(0x7f32ec06e9e0 msgr2=0x7f32ec06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:08.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32e3fff640 1 --2- 172.21.15.80:0/803910648 >> v2:172.21.15.87:3300/0 conn(0x7f32ec06e9e0 0x7f32ec06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32e3fff640 1 -- 172.21.15.80:0/803910648 >> v1:172.21.15.87:6789/0 conn(0x7f32ec161f90 legacy=0x7f32ec0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:08.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32e3fff640 1 -- 172.21.15.80:0/803910648 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec06dca0 con 0x7f32ec0d06e0 2024-04-25T13:48:08.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32ead9d640 1 --2- 172.21.15.80:0/803910648 >> v2:172.21.15.87:3300/0 conn(0x7f32ec06e9e0 0x7f32ec06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:08.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.698+0000 7f32e3fff640 1 --2- 172.21.15.80:0/803910648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec06e4a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0b4b80 tx=0x7f32d401c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2b8403fd2295c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:08.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32e9d9b640 1 -- 172.21.15.80:0/803910648 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40b0070 con 0x7f32ec0d06e0 2024-04-25T13:48:08.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32e9d9b640 1 -- 172.21.15.80:0/803910648 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d40ab600 con 0x7f32ec0d06e0 2024-04-25T13:48:08.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32e9d9b640 1 -- 172.21.15.80:0/803910648 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40ab9d0 con 0x7f32ec0d06e0 2024-04-25T13:48:08.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 -- 172.21.15.80:0/803910648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 msgr2=0x7f32ec06e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:08.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/803910648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec06e4a0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f32ec0b4b80 tx=0x7f32d401c1b0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 -- 172.21.15.80:0/803910648 shutdown_connections 2024-04-25T13:48:08.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/803910648 >> v2:172.21.15.87:3300/0 conn(0x7f32ec06e9e0 0x7f32ec06edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/803910648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec06e4a0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 -- 172.21.15.80:0/803910648 >> 172.21.15.80:0/803910648 conn(0x7f32ec06d080 msgr2=0x7f32ec0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:08.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 -- 172.21.15.80:0/803910648 shutdown_connections 2024-04-25T13:48:08.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 -- 172.21.15.80:0/803910648 wait complete. 2024-04-25T13:48:08.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 Processor -- start 2024-04-25T13:48:08.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.702+0000 7f32f0cc5640 1 -- start start 2024-04-25T13:48:08.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32ec06e9e0 0x7f32ec162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:08.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec1640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:08.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32ec161f90 0x7f32ec146cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec1447a0 con 0x7f32ec0d06e0 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f32ec144620 con 0x7f32ec161f90 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f32ec16b510 con 0x7f32ec06e9e0 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec1640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec1640e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53712/0 (socket says 172.21.15.80:53712) 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 -- 172.21.15.80:0/1426843996 learned_addr learned my addr 172.21.15.80:0/1426843996 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32ead9d640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32ec161f90 0x7f32ec146cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32ea59c640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32ec06e9e0 0x7f32ec162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 -- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32ec06e9e0 msgr2=0x7f32ec162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32ec06e9e0 0x7f32ec162ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 -- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32ec161f90 msgr2=0x7f32ec146cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32ec161f90 0x7f32ec146cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:08.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 -- 172.21.15.80:0/1426843996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ec145c20 con 0x7f32ec0d06e0 2024-04-25T13:48:08.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e3fff640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec1640e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f32d40ae040 tx=0x7f32d401a9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:08.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e2ffd640 1 -- 172.21.15.80:0/1426843996 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40ba020 con 0x7f32ec0d06e0 2024-04-25T13:48:08.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e2ffd640 1 -- 172.21.15.80:0/1426843996 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f32d40b6640 con 0x7f32ec0d06e0 2024-04-25T13:48:08.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ec145f00 con 0x7f32ec0d06e0 2024-04-25T13:48:08.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e2ffd640 1 -- 172.21.15.80:0/1426843996 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d40b0040 con 0x7f32ec0d06e0 2024-04-25T13:48:08.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ec1464c0 con 0x7f32ec0d06e0 2024-04-25T13:48:08.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32ec06a3a0 con 0x7f32ec0d06e0 2024-04-25T13:48:08.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e2ffd640 1 -- 172.21.15.80:0/1426843996 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f32d40bd020 con 0x7f32ec0d06e0 2024-04-25T13:48:08.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e2ffd640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32cc06f400 0x7f32cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:08.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.706+0000 7f32e2ffd640 1 -- 172.21.15.80:0/1426843996 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5227+0+0 (secure 0 0 0) 0x7f32d4129450 con 0x7f32ec0d06e0 2024-04-25T13:48:08.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.710+0000 7f32ea59c640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32cc06f400 0x7f32cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:08.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.710+0000 7f32ea59c640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32cc06f400 0x7f32cc071910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f32ec06fa20 tx=0x7f32dc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:08.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.714+0000 7f32e2ffd640 1 -- 172.21.15.80:0/1426843996 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f32d40c4070 con 0x7f32ec0d06e0 2024-04-25T13:48:08.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:08.858+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f32ec0769e0 con 0x7f32ec0d06e0 2024-04-25T13:48:09.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.510+0000 7f32e2ffd640 1 -- 172.21.15.80:0/1426843996 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v152) ==== 170+0+0 (secure 0 0 0) 0x7f32ec0d0af0 con 0x7f32ec0d06e0 2024-04-25T13:48:09.507 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:48:09.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32cc06f400 msgr2=0x7f32cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:09.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32cc06f400 0x7f32cc071910 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f32ec06fa20 tx=0x7f32dc008040 comp rx=0 tx=0).stop 2024-04-25T13:48:09.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 msgr2=0x7f32ec1640e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:09.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec1640e0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f32d40ae040 tx=0x7f32d401a9f0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 shutdown_connections 2024-04-25T13:48:09.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f32cc06f400 0x7f32cc071910 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32ec161f90 0x7f32ec146cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32ec0d06e0 0x7f32ec1640e0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 --2- 172.21.15.80:0/1426843996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32ec06e9e0 0x7f32ec162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 >> 172.21.15.80:0/1426843996 conn(0x7f32ec06d080 msgr2=0x7f32ec0b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:09.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 shutdown_connections 2024-04-25T13:48:09.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.514+0000 7f32f0cc5640 1 -- 172.21.15.80:0/1426843996 wait complete. 2024-04-25T13:48:09.544 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:48:09.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ec0ba640 1 Processor -- start 2024-04-25T13:48:09.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ec0ba640 1 -- start start 2024-04-25T13:48:09.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ec0ba640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff5e40b5330 0x7ff5e40d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ec0ba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e40d0ff0 0x7ff5e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ec0ba640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5e40b5740 con 0x7ff5e406f2f0 2024-04-25T13:48:09.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ec0ba640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff5e406f700 con 0x7ff5e40b5330 2024-04-25T13:48:09.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ec0ba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e41629b0 con 0x7ff5e40d0ff0 2024-04-25T13:48:09.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ea657640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff5e406f2f0 0x7ff5e4161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51414/0 (socket says 172.21.15.80:51414) 2024-04-25T13:48:09.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.690+0000 7ff5ea657640 1 -- 172.21.15.80:0/2707984532 learned_addr learned my addr 172.21.15.80:0/2707984532 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9e56640 1 --2- 172.21.15.80:0/2707984532 >> v2:172.21.15.87:3300/0 conn(0x7ff5e40b5330 0x7ff5e40d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9655640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e40d0ff0 0x7ff5e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3689685569 0 0) 0x7ff5e40b5740 con 0x7ff5e406f2f0 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff5d0003530 con 0x7ff5e406f2f0 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 626766579 0 0) 0x7ff5d0003530 con 0x7ff5e406f2f0 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e40d0ff0 msgr2=0x7ff5e406edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e40d0ff0 0x7ff5e406edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 >> v2:172.21.15.87:3300/0 conn(0x7ff5e40b5330 msgr2=0x7ff5e40d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 --2- 172.21.15.80:0/2707984532 >> v2:172.21.15.87:3300/0 conn(0x7ff5e40b5330 0x7ff5e40d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e406dd10 con 0x7ff5e406f2f0 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff5dc001a00 con 0x7ff5e406f2f0 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 >> v1:172.21.15.80:6789/0 conn(0x7ff5e406f2f0 legacy=0x7ff5e4161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5d0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e40d0ff0 0x7ff5d0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5d0004d50 0x7ff5d0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e406f700 con 0x7ff5d00039c0 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e41629b0 con 0x7ff5d0004d50 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d000ba50 con 0x7ff5e40d0ff0 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9e56640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5d0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9655640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e40d0ff0 0x7ff5d0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5ea657640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5d0004d50 0x7ff5d0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9e56640 1 -- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e40d0ff0 msgr2=0x7ff5d0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9e56640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e40d0ff0 0x7ff5d0004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9e56640 1 -- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5d0004d50 msgr2=0x7ff5d0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:09.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9e56640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5d0004d50 0x7ff5d0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9e56640 1 -- 172.21.15.80:0/2707984532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d0003530 con 0x7ff5d00039c0 2024-04-25T13:48:09.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5ea657640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5d0004d50 0x7ff5d0005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:09.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e9e56640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5d0003e30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009ae0 tx=0x7ff5e001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e6ca538640f3b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:09.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00b2070 con 0x7ff5d00039c0 2024-04-25T13:48:09.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e00a5570 con 0x7ff5d00039c0 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.694+0000 7ff5e8e54640 1 -- 172.21.15.80:0/2707984532 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00096b0 con 0x7ff5d00039c0 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/2707984532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 msgr2=0x7ff5d0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 --2- 172.21.15.80:0/2707984532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5d0003e30 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009ae0 tx=0x7ff5e001cab0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ea657640 1 -- 172.21.15.80:0/2707984532 reap_dead start 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/2707984532 shutdown_connections 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/2707984532 >> 172.21.15.80:0/2707984532 conn(0x7ff5e40b4160 msgr2=0x7ff5e40d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/2707984532 shutdown_connections 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/2707984532 wait complete. 2024-04-25T13:48:09.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 Processor -- start 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- start start 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5e414cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5e4164790 0x7ff5e414a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e414d6c0 0x7ff5e414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e40b0cd0 con 0x7ff5d00039c0 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff5e406e890 con 0x7ff5e4164790 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ec0ba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5e40b0e50 con 0x7ff5e414d6c0 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5e414cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5e414cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53734/0 (socket says 172.21.15.80:53734) 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 -- 172.21.15.80:0/1050909195 learned_addr learned my addr 172.21.15.80:0/1050909195 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9655640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5e4164790 0x7ff5e414a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ea657640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e414d6c0 0x7ff5e414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 -- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e414d6c0 msgr2=0x7ff5e414dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e414d6c0 0x7ff5e414dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 -- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5e4164790 msgr2=0x7ff5e414a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5e4164790 0x7ff5e414a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 -- 172.21.15.80:0/1050909195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5e414e010 con 0x7ff5d00039c0 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5ea657640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e414d6c0 0x7ff5e414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9655640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5e4164790 0x7ff5e414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.698+0000 7ff5e9e56640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5e414cb70 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009ae0 tx=0x7ff5d0005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5da7fc640 1 -- 172.21.15.80:0/1050909195 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00bd020 con 0x7ff5d00039c0 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5da7fc640 1 -- 172.21.15.80:0/1050909195 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5e00a5d20 con 0x7ff5d00039c0 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5e414eb70 con 0x7ff5d00039c0 2024-04-25T13:48:09.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5da7fc640 1 -- 172.21.15.80:0/1050909195 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00b2040 con 0x7ff5d00039c0 2024-04-25T13:48:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5e414b930 con 0x7ff5d00039c0 2024-04-25T13:48:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5e406a3a0 con 0x7ff5d00039c0 2024-04-25T13:48:09.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5da7fc640 1 -- 172.21.15.80:0/1050909195 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff5e0004260 con 0x7ff5d00039c0 2024-04-25T13:48:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5da7fc640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5c806f400 0x7ff5c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5da7fc640 1 -- 172.21.15.80:0/1050909195 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5609+0+0 (secure 0 0 0) 0x7ff5e0128f90 con 0x7ff5d00039c0 2024-04-25T13:48:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5e9655640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5c806f400 0x7ff5c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:09.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.702+0000 7ff5e9655640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5c806f400 0x7ff5c8071910 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff5e41645f0 tx=0x7ff5d4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:09.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.706+0000 7ff5da7fc640 1 -- 172.21.15.80:0/1050909195 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff5e414e010 con 0x7ff5d00039c0 2024-04-25T13:48:09.900 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:09.902+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff5e406fd00 con 0x7ff5d00039c0 2024-04-25T13:48:10.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.550+0000 7ff5da7fc640 1 -- 172.21.15.80:0/1050909195 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v71) ==== 174+0+0 (secure 0 0 0) 0x7ff5e00167c0 con 0x7ff5d00039c0 2024-04-25T13:48:10.548 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 18 and data pool 19 2024-04-25T13:48:10.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5c806f400 msgr2=0x7ff5c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5c806f400 0x7ff5c8071910 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff5e41645f0 tx=0x7ff5d4008040 comp rx=0 tx=0).stop 2024-04-25T13:48:10.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 msgr2=0x7ff5e414cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5e414cb70 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009ae0 tx=0x7ff5d0005840 comp rx=0 tx=0).stop 2024-04-25T13:48:10.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 shutdown_connections 2024-04-25T13:48:10.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff5c806f400 0x7ff5c8071910 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5e414d6c0 0x7ff5e414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff5e4164790 0x7ff5e414a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 --2- 172.21.15.80:0/1050909195 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5d00039c0 0x7ff5e414cb70 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.554+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 >> 172.21.15.80:0/1050909195 conn(0x7ff5e40b4160 msgr2=0x7ff5e41507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:10.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.558+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 shutdown_connections 2024-04-25T13:48:10.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.558+0000 7ff5ec0ba640 1 -- 172.21.15.80:0/1050909195 wait complete. 2024-04-25T13:48:10.593 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:48:10.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8448b66640 1 Processor -- start 2024-04-25T13:48:10.739 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8448b66640 1 -- start start 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8448b66640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 0x7f84440d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8448b66640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84440d0e30 con 0x7f84440b9240 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8448b66640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84440cfb20 con 0x7f84440b7650 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8448b66640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f84440b6920 con 0x7f84440d04e0 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844259c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f84440b9240 0x7f84440cf1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51418/0 (socket says 172.21.15.80:51418) 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844259c640 1 -- 172.21.15.80:0/3447594683 learned_addr learned my addr 172.21.15.80:0/3447594683 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8441d9b640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 0x7f84440d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2755216964 0 0) 0x7f84380012a0 con 0x7f84440b9240 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f842c003530 con 0x7f84440b9240 2024-04-25T13:48:10.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2881873097 0 0) 0x7f842c003530 con 0x7f84440b9240 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 msgr2=0x7f84440d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 0x7f84440d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 >> v1:172.21.15.87:6789/0 conn(0x7f84440b7650 legacy=0x7f84440b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84440b6b70 con 0x7f84440b9240 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f84380019d0 con 0x7f84440b9240 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 >> v1:172.21.15.80:6789/0 conn(0x7f84440b9240 legacy=0x7f84440cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c0039f0 0x7f842c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 0x7f842c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f842c004d50 0x7f842c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f84440b6920 con 0x7f842c004d50 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f842c00bb90 con 0x7f842c0039f0 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f842c00bd90 con 0x7f84440d04e0 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8442d9d640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f842c004d50 0x7f842c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:10.744 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f844259c640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c0039f0 0x7f842c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:10.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.742+0000 7f8441d9b640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 0x7f842c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:10.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8442d9d640 1 -- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 msgr2=0x7f842c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:10.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8442d9d640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 0x7f842c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8442d9d640 1 -- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c0039f0 msgr2=0x7f842c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:10.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8442d9d640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c0039f0 0x7f842c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8442d9d640 1 -- 172.21.15.80:0/3447594683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842c003530 con 0x7f842c004d50 2024-04-25T13:48:10.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f844259c640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f842c0039f0 0x7f842c003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:10.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8441d9b640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f84440d04e0 0x7f842c004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:10.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8442d9d640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f842c004d50 0x7f842c005160 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f84440d0100 tx=0x7f843c01c720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=580aa9ce67fb2219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:10.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c001a90 con 0x7f842c004d50 2024-04-25T13:48:10.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f843c003b80 con 0x7f842c004d50 2024-04-25T13:48:10.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f844159a640 1 -- 172.21.15.80:0/3447594683 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843c005080 con 0x7f842c004d50 2024-04-25T13:48:10.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8448b66640 1 -- 172.21.15.80:0/3447594683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f842c004d50 msgr2=0x7f842c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8448b66640 1 --2- 172.21.15.80:0/3447594683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f842c004d50 0x7f842c005160 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f84440d0100 tx=0x7f843c01c720 comp rx=0 tx=0).stop 2024-04-25T13:48:10.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8442d9d640 1 -- 172.21.15.80:0/3447594683 reap_dead start 2024-04-25T13:48:10.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8448b66640 1 -- 172.21.15.80:0/3447594683 shutdown_connections 2024-04-25T13:48:10.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8448b66640 1 -- 172.21.15.80:0/3447594683 >> 172.21.15.80:0/3447594683 conn(0x7f84440d3ad0 msgr2=0x7f84440d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:10.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8448b66640 1 -- 172.21.15.80:0/3447594683 shutdown_connections 2024-04-25T13:48:10.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.746+0000 7f8448b66640 1 -- 172.21.15.80:0/3447594683 wait complete. 2024-04-25T13:48:10.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 Processor -- start 2024-04-25T13:48:10.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 -- start start 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844406d930 0x7f84440ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844409ee20 0x7f844409f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844409f770 0x7f84440c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f84440b7130 con 0x7f844409ee20 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f84440cf8c0 con 0x7f844406d930 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f84440d08f0 con 0x7f844409f770 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844409ee20 0x7f844409f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844409ee20 0x7f844409f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53762/0 (socket says 172.21.15.80:53762) 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 -- 172.21.15.80:0/2006694408 learned_addr learned my addr 172.21.15.80:0/2006694408 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:10.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f844259c640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844406d930 0x7f84440ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8442d9d640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844409f770 0x7f84440c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 -- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844409f770 msgr2=0x7f84440c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844409f770 0x7f84440c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 -- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844406d930 msgr2=0x7f84440ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844406d930 0x7f84440ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 -- 172.21.15.80:0/2006694408 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84440c9cf0 con 0x7f844409ee20 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f844259c640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844406d930 0x7f84440ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8441d9b640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844409ee20 0x7f844409f230 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f84340108d0 tx=0x7f8434010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8432ffd640 1 -- 172.21.15.80:0/2006694408 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843400f040 con 0x7f844409ee20 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8432ffd640 1 -- 172.21.15.80:0/2006694408 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8434004220 con 0x7f844409ee20 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84440c8b10 con 0x7f844409ee20 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8432ffd640 1 -- 172.21.15.80:0/2006694408 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f843400e960 con 0x7f844409ee20 2024-04-25T13:48:10.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.750+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844406d130 con 0x7f844409ee20 2024-04-25T13:48:10.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.754+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844406a3a0 con 0x7f844409ee20 2024-04-25T13:48:10.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.754+0000 7f8432ffd640 1 -- 172.21.15.80:0/2006694408 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f843400eb00 con 0x7f844409ee20 2024-04-25T13:48:10.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.754+0000 7f8432ffd640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842406f400 0x7f8424071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:10.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.754+0000 7f8432ffd640 1 -- 172.21.15.80:0/2006694408 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f843408f2b0 con 0x7f844409ee20 2024-04-25T13:48:10.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.754+0000 7f844259c640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842406f400 0x7f8424071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:10.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.754+0000 7f844259c640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842406f400 0x7f8424071910 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8438001560 tx=0x7f84380014b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:10.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.758+0000 7f8432ffd640 1 -- 172.21.15.80:0/2006694408 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8434018030 con 0x7f844409ee20 2024-04-25T13:48:10.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.918+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8444074c30 con 0x7f844409ee20 2024-04-25T13:48:10.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.922+0000 7f8432ffd640 1 -- 172.21.15.80:0/2006694408 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16621 (secure 0 0 0) 0x7f843405c2c0 con 0x7f844409ee20 2024-04-25T13:48:10.921 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:10.921 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":153,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:48:09.907145+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:48:07.666863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T13:48:08.875350+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842406f400 msgr2=0x7f8424071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842406f400 0x7f8424071910 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8438001560 tx=0x7f84380014b0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844409ee20 msgr2=0x7f844409f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844409ee20 0x7f844409f230 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f84340108d0 tx=0x7f8434010d90 comp rx=0 tx=0).stop 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 shutdown_connections 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842406f400 0x7f8424071910 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844409f770 0x7f84440c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844409ee20 0x7f844409f230 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 --2- 172.21.15.80:0/2006694408 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844406d930 0x7f84440ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 >> 172.21.15.80:0/2006694408 conn(0x7f84440d3ad0 msgr2=0x7f84441609d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 shutdown_connections 2024-04-25T13:48:10.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:10.926+0000 7f8448b66640 1 -- 172.21.15.80:0/2006694408 wait complete. 2024-04-25T13:48:10.945 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:48:11.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe12de4b640 1 Processor -- start 2024-04-25T13:48:11.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe12de4b640 1 -- start start 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe12de4b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1281622d0 0x7fe1280725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe12de4b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe128072b00 con 0x7fe12806eb40 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe12de4b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe128072120 con 0x7fe128168220 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe12de4b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe12806dd80 con 0x7fe1281622d0 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe1277fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe1281622d0 0x7fe1280725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe126ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe12806eb40 0x7fe12806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53474/0 (socket says 172.21.15.80:53474) 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe126ffd640 1 -- 172.21.15.80:0/2159492067 learned_addr learned my addr 172.21.15.80:0/2159492067 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe1277fe640 1 -- 172.21.15.80:0/2159492067 >> v1:172.21.15.87:6789/0 conn(0x7fe128168220 legacy=0x7fe128168630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe1277fe640 1 -- 172.21.15.80:0/2159492067 >> v1:172.21.15.80:6789/0 conn(0x7fe12806eb40 legacy=0x7fe12806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.098+0000 7fe1277fe640 1 -- 172.21.15.80:0/2159492067 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12806e060 con 0x7fe1281622d0 2024-04-25T13:48:11.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe1277fe640 1 --2- 172.21.15.80:0/2159492067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1281622d0 0x7fe1280725c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe1180097c0 tx=0x7fe11801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711d36eea58fa8b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:11.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe1267fc640 1 -- 172.21.15.80:0/2159492067 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1180b2070 con 0x7fe1281622d0 2024-04-25T13:48:11.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe1267fc640 1 -- 172.21.15.80:0/2159492067 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe1180a9070 con 0x7fe1281622d0 2024-04-25T13:48:11.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe1267fc640 1 -- 172.21.15.80:0/2159492067 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1180ad880 con 0x7fe1281622d0 2024-04-25T13:48:11.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe12de4b640 1 -- 172.21.15.80:0/2159492067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1281622d0 msgr2=0x7fe1280725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe12de4b640 1 --2- 172.21.15.80:0/2159492067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1281622d0 0x7fe1280725c0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fe1180097c0 tx=0x7fe11801c340 comp rx=0 tx=0).stop 2024-04-25T13:48:11.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe12de4b640 1 -- 172.21.15.80:0/2159492067 shutdown_connections 2024-04-25T13:48:11.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe12de4b640 1 --2- 172.21.15.80:0/2159492067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe1281622d0 0x7fe1280725c0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe12de4b640 1 -- 172.21.15.80:0/2159492067 >> 172.21.15.80:0/2159492067 conn(0x7fe12806d080 msgr2=0x7fe1280b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:11.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe12de4b640 1 -- 172.21.15.80:0/2159492067 shutdown_connections 2024-04-25T13:48:11.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.102+0000 7fe12de4b640 1 -- 172.21.15.80:0/2159492067 wait complete. 2024-04-25T13:48:11.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 Processor -- start 2024-04-25T13:48:11.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 -- start start 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe12806eb40 0x7fe1281558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1281622d0 0x7fe12815d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe128168220 0x7fe12815b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe128172bd0 con 0x7fe128168220 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe128169150 con 0x7fe12806eb40 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe128172d50 con 0x7fe1281622d0 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe128168220 0x7fe12815b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe128168220 0x7fe12815b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50470/0 (socket says 172.21.15.80:50470) 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 -- 172.21.15.80:0/1710121370 learned_addr learned my addr 172.21.15.80:0/1710121370 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe1277fe640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe12806eb40 0x7fe1281558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe126ffd640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1281622d0 0x7fe12815d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 -- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1281622d0 msgr2=0x7fe12815d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1281622d0 0x7fe12815d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 -- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe12806eb40 msgr2=0x7fe1281558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe12806eb40 0x7fe1281558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 -- 172.21.15.80:0/1710121370 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe128159210 con 0x7fe128168220 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe127fff640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe128168220 0x7fe12815b6b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe12000b4c0 tx=0x7fe12000b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe10ffff640 1 -- 172.21.15.80:0/1710121370 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120015020 con 0x7fe128168220 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe10ffff640 1 -- 172.21.15.80:0/1710121370 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe120011830 con 0x7fe128168220 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe128165e50 con 0x7fe128168220 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe10ffff640 1 -- 172.21.15.80:0/1710121370 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120011c20 con 0x7fe128168220 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe128166410 con 0x7fe128168220 2024-04-25T13:48:11.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.106+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12806a3a0 con 0x7fe128168220 2024-04-25T13:48:11.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.110+0000 7fe10ffff640 1 -- 172.21.15.80:0/1710121370 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe120018020 con 0x7fe128168220 2024-04-25T13:48:11.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.110+0000 7fe10ffff640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe11406f400 0x7fe114071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.110+0000 7fe10ffff640 1 -- 172.21.15.80:0/1710121370 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7fe12008eeb0 con 0x7fe128168220 2024-04-25T13:48:11.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.110+0000 7fe1277fe640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe11406f400 0x7fe114071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.110+0000 7fe1277fe640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe11406f400 0x7fe114071910 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe1180af000 tx=0x7fe1180a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:11.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.114+0000 7fe10ffff640 1 -- 172.21.15.80:0/1710121370 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe12001f030 con 0x7fe128168220 2024-04-25T13:48:11.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.286+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe12815d630 con 0x7fe128168220 2024-04-25T13:48:11.286 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe10ffff640 1 -- 172.21.15.80:0/1710121370 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v153) ==== 74+0+16621 (secure 0 0 0) 0x7fe12005bda0 con 0x7fe128168220 2024-04-25T13:48:11.287 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:11.287 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":153,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:48:09.907145+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:48:07.666863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T13:48:08.875350+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:48:11.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe11406f400 msgr2=0x7fe114071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe11406f400 0x7fe114071910 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fe1180af000 tx=0x7fe1180a7000 comp rx=0 tx=0).stop 2024-04-25T13:48:11.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe128168220 msgr2=0x7fe12815b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.289 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe128168220 0x7fe12815b6b0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe12000b4c0 tx=0x7fe12000b980 comp rx=0 tx=0).stop 2024-04-25T13:48:11.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 shutdown_connections 2024-04-25T13:48:11.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe11406f400 0x7fe114071910 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe128168220 0x7fe12815b6b0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe1281622d0 0x7fe12815d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 --2- 172.21.15.80:0/1710121370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe12806eb40 0x7fe1281558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 >> 172.21.15.80:0/1710121370 conn(0x7fe12806d080 msgr2=0x7fe1280b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:11.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 shutdown_connections 2024-04-25T13:48:11.290 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.290+0000 7fe12de4b640 1 -- 172.21.15.80:0/1710121370 wait complete. 2024-04-25T13:48:11.309 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:48:11.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7859a33640 1 Processor -- start 2024-04-25T13:48:11.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7859a33640 1 -- start start 2024-04-25T13:48:11.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7859a33640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f78540d2020 0x7f78540d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7859a33640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 0x7f78540ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7859a33640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78540d2b60 con 0x7f78540b3be0 2024-04-25T13:48:11.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7859a33640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f785406e230 con 0x7f78540d2020 2024-04-25T13:48:11.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7859a33640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f785406cf90 con 0x7f78540ce4a0 2024-04-25T13:48:11.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f78527fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f78540b3be0 0x7f78540b3ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53480/0 (socket says 172.21.15.80:53480) 2024-04-25T13:48:11.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f78527fc640 1 -- 172.21.15.80:0/1028652882 learned_addr learned my addr 172.21.15.80:0/1028652882 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:11.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7852ffd640 1 --2- 172.21.15.80:0/1028652882 >> v2:172.21.15.87:3300/0 conn(0x7f78540d2020 0x7f78540d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f78537fe640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 0x7f78540ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1315583402 0 0) 0x7f7848001120 con 0x7f78540b3be0 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f783c003530 con 0x7f78540b3be0 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1574187640 0 0) 0x7f783c003530 con 0x7f78540b3be0 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 msgr2=0x7f78540ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 0x7f78540ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 >> v2:172.21.15.87:3300/0 conn(0x7f78540d2020 msgr2=0x7f78540d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 --2- 172.21.15.80:0/1028652882 >> v2:172.21.15.87:3300/0 conn(0x7f78540d2020 0x7f78540d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f785406dc70 con 0x7f78540b3be0 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7848001850 con 0x7f78540b3be0 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 >> v1:172.21.15.80:6789/0 conn(0x7f78540b3be0 legacy=0x7f78540b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f783c0039c0 0x7f783c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c004810 0x7f783c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 0x7f783c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f785406e230 con 0x7f783c0039c0 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f785406cf90 con 0x7f783c004810 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f783c00b9d0 con 0x7f78540ce4a0 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.466+0000 7f7852ffd640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f783c0039c0 0x7f783c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f78527fc640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c004810 0x7f783c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f78537fe640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 0x7f783c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7852ffd640 1 -- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 msgr2=0x7f783c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7852ffd640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 0x7f783c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7852ffd640 1 -- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c004810 msgr2=0x7f783c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7852ffd640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c004810 0x7f783c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7852ffd640 1 -- 172.21.15.80:0/1028652882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f783c003530 con 0x7f783c0039c0 2024-04-25T13:48:11.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f78537fe640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f78540ce4a0 0x7f783c005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7852ffd640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f783c0039c0 0x7f783c003e30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7844009ae0 tx=0x7f784401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cf6285655d2f6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440b2070 con 0x7f783c0039c0 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f78440a56a0 con 0x7f783c0039c0 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7851ffb640 1 -- 172.21.15.80:0/1028652882 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78440096b0 con 0x7f783c0039c0 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7859a33640 1 -- 172.21.15.80:0/1028652882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f783c0039c0 msgr2=0x7f783c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7859a33640 1 --2- 172.21.15.80:0/1028652882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f783c0039c0 0x7f783c003e30 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f7844009ae0 tx=0x7f784401cab0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f78537fe640 1 -- 172.21.15.80:0/1028652882 reap_dead start 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7859a33640 1 -- 172.21.15.80:0/1028652882 shutdown_connections 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7859a33640 1 -- 172.21.15.80:0/1028652882 >> 172.21.15.80:0/1028652882 conn(0x7f78540d3ab0 msgr2=0x7f78540d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7859a33640 1 -- 172.21.15.80:0/1028652882 shutdown_connections 2024-04-25T13:48:11.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.470+0000 7f7859a33640 1 -- 172.21.15.80:0/1028652882 wait complete. 2024-04-25T13:48:11.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 Processor -- start 2024-04-25T13:48:11.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 -- start start 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c0039c0 0x7f7854163ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78540cc920 0x7f78540ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7854167bd0 0x7f7854167fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f78540d45d0 con 0x7f78540cc920 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f78540b4fa0 con 0x7f783c0039c0 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f78540d4750 con 0x7f7854167bd0 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78540cc920 0x7f78540ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78540cc920 0x7f78540ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50494/0 (socket says 172.21.15.80:50494) 2024-04-25T13:48:11.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 -- 172.21.15.80:0/3858657818 learned_addr learned my addr 172.21.15.80:0/3858657818 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7852ffd640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c0039c0 0x7f7854163ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78537fe640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7854167bd0 0x7f7854167fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 -- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7854167bd0 msgr2=0x7f7854167fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7854167bd0 0x7f7854167fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 -- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c0039c0 msgr2=0x7f7854163ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c0039c0 0x7f7854163ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 -- 172.21.15.80:0/3858657818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7854168520 con 0x7f78540cc920 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78537fe640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7854167bd0 0x7f7854167fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f78527fc640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78540cc920 0x7f78540ccd30 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f784800e7b0 tx=0x7f784800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f783bfff640 1 -- 172.21.15.80:0/3858657818 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784800ee60 con 0x7f78540cc920 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f783bfff640 1 -- 172.21.15.80:0/3858657818 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7848018070 con 0x7f78540cc920 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78540cadb0 con 0x7f78540cc920 2024-04-25T13:48:11.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f783bfff640 1 -- 172.21.15.80:0/3858657818 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784800caf0 con 0x7f78540cc920 2024-04-25T13:48:11.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.474+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78540cb370 con 0x7f78540cc920 2024-04-25T13:48:11.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.478+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f785406a3a0 con 0x7f78540cc920 2024-04-25T13:48:11.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.478+0000 7f783bfff640 1 -- 172.21.15.80:0/3858657818 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f784800cc90 con 0x7f78540cc920 2024-04-25T13:48:11.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.478+0000 7f783bfff640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f784006f400 0x7f7840071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:11.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.478+0000 7f783bfff640 1 -- 172.21.15.80:0/3858657818 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5709+0+0 (secure 0 0 0) 0x7f784808fbb0 con 0x7f78540cc920 2024-04-25T13:48:11.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.478+0000 7f7852ffd640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f784006f400 0x7f7840071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:11.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.478+0000 7f7852ffd640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f784006f400 0x7f7840071910 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f78440af000 tx=0x7f78440046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:11.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.482+0000 7f783bfff640 1 -- 172.21.15.80:0/3858657818 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f78540d2b60 con 0x7f78540cc920 2024-04-25T13:48:11.671 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:11.674+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f78540b95e0 con 0x7f78540cc920 2024-04-25T13:48:12.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.590+0000 7f783bfff640 1 -- 172.21.15.80:0/3858657818 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) ==== 118+0+0 (secure 0 0 0) 0x7f784805cb80 con 0x7f78540cc920 2024-04-25T13:48:12.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f784006f400 msgr2=0x7f7840071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:12.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f784006f400 0x7f7840071910 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f78440af000 tx=0x7f78440046d0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78540cc920 msgr2=0x7f78540ccd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:12.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78540cc920 0x7f78540ccd30 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f784800e7b0 tx=0x7f784800ec70 comp rx=0 tx=0).stop 2024-04-25T13:48:12.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 shutdown_connections 2024-04-25T13:48:12.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f784006f400 0x7f7840071910 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7854167bd0 0x7f7854167fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f78540cc920 0x7f78540ccd30 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 --2- 172.21.15.80:0/3858657818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f783c0039c0 0x7f7854163ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 >> 172.21.15.80:0/3858657818 conn(0x7f78540d3ab0 msgr2=0x7f785406d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:12.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 shutdown_connections 2024-04-25T13:48:12.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.594+0000 7f7859a33640 1 -- 172.21.15.80:0/3858657818 wait complete. 2024-04-25T13:48:12.618 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:48:12.618 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:48:12.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4618c84640 1 Processor -- start 2024-04-25T13:48:12.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4618c84640 1 -- start start 2024-04-25T13:48:12.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4618c84640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614161170 0x7f46141574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:12.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4618c84640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4614073c40 0x7f46140b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:12.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4618c84640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f461406fbf0 con 0x7f4614071e60 2024-04-25T13:48:12.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4618c84640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f46140b1350 con 0x7f4614073c40 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4618c84640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f461406f9a0 con 0x7f4614161170 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4612d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4614073c40 0x7f46140b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4612d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4614073c40 0x7f46140b4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50504/0 (socket says 172.21.15.80:50504) 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4612d9d640 1 -- 172.21.15.80:0/3759123800 learned_addr learned my addr 172.21.15.80:0/3759123800 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4611d9b640 1 --2- 172.21.15.80:0/3759123800 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614161170 0x7f46141574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4612d9d640 1 -- 172.21.15.80:0/3759123800 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614161170 msgr2=0x7f46141574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4612d9d640 1 --2- 172.21.15.80:0/3759123800 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614161170 0x7f46141574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4612d9d640 1 -- 172.21.15.80:0/3759123800 >> v1:172.21.15.87:6789/0 conn(0x7f4614071e60 legacy=0x7f4614160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:12.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.774+0000 7f4612d9d640 1 -- 172.21.15.80:0/3759123800 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46140b0a20 con 0x7f4614073c40 2024-04-25T13:48:12.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4612d9d640 1 --2- 172.21.15.80:0/3759123800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4614073c40 0x7f46140b4ba0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f460c009870 tx=0x7f460c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4545f7b8d3fe527c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:12.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f461159a640 1 -- 172.21.15.80:0/3759123800 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c0b1070 con 0x7f4614073c40 2024-04-25T13:48:12.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f461159a640 1 -- 172.21.15.80:0/3759123800 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c0ac630 con 0x7f4614073c40 2024-04-25T13:48:12.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f461159a640 1 -- 172.21.15.80:0/3759123800 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c0a8050 con 0x7f4614073c40 2024-04-25T13:48:12.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4618c84640 1 -- 172.21.15.80:0/3759123800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4614073c40 msgr2=0x7f46140b4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:12.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4618c84640 1 --2- 172.21.15.80:0/3759123800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4614073c40 0x7f46140b4ba0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f460c009870 tx=0x7f460c01c520 comp rx=0 tx=0).stop 2024-04-25T13:48:12.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4618c84640 1 -- 172.21.15.80:0/3759123800 shutdown_connections 2024-04-25T13:48:12.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4618c84640 1 --2- 172.21.15.80:0/3759123800 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4614073c40 0x7f46140b4ba0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4618c84640 1 --2- 172.21.15.80:0/3759123800 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614161170 0x7f46141574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4618c84640 1 -- 172.21.15.80:0/3759123800 >> 172.21.15.80:0/3759123800 conn(0x7f46140b4160 msgr2=0x7f46140d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:12.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4618c84640 1 -- 172.21.15.80:0/3759123800 shutdown_connections 2024-04-25T13:48:12.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.778+0000 7f4618c84640 1 -- 172.21.15.80:0/3759123800 wait complete. 2024-04-25T13:48:12.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 Processor -- start 2024-04-25T13:48:12.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 -- start start 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614071e60 0x7f461413d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4614073c40 0x7f4614156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f461414b490 0x7f461414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4614169bb0 con 0x7f461414b490 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4614074050 con 0x7f4614073c40 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4614169d30 con 0x7f4614071e60 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f461414b490 0x7f461414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f461414b490 0x7f461414f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50516/0 (socket says 172.21.15.80:50516) 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 -- 172.21.15.80:0/1724862483 learned_addr learned my addr 172.21.15.80:0/1724862483 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4611d9b640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4614073c40 0x7f4614156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f461259c640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614071e60 0x7f461413d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:12.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 -- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614071e60 msgr2=0x7f461413d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614071e60 0x7f461413d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 -- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4614073c40 msgr2=0x7f4614156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4614073c40 0x7f4614156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 -- 172.21.15.80:0/1724862483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f461414f8b0 con 0x7f461414b490 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f461259c640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614071e60 0x7f461413d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4612d9d640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f461414b490 0x7f461414f370 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f460c01abd0 tx=0x7f460c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f460affd640 1 -- 172.21.15.80:0/1724862483 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c0bc070 con 0x7f461414b490 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f460affd640 1 -- 172.21.15.80:0/1724862483 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f460c0b7430 con 0x7f461414b490 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f461414d180 con 0x7f461414b490 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f460affd640 1 -- 172.21.15.80:0/1724862483 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460c0b1040 con 0x7f461414b490 2024-04-25T13:48:12.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f461414d740 con 0x7f461414b490 2024-04-25T13:48:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f461406a3a0 con 0x7f461414b490 2024-04-25T13:48:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.782+0000 7f460affd640 1 -- 172.21.15.80:0/1724862483 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f460c0bf070 con 0x7f461414b490 2024-04-25T13:48:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.786+0000 7f460affd640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45f406f400 0x7f45f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.786+0000 7f460affd640 1 -- 172.21.15.80:0/1724862483 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f460c128b60 con 0x7f461414b490 2024-04-25T13:48:12.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.786+0000 7f461259c640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45f406f400 0x7f45f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:12.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.786+0000 7f461259c640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45f406f400 0x7f45f4071910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4604007e90 tx=0x7f46040073a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:12.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.790+0000 7f460affd640 1 -- 172.21.15.80:0/1724862483 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f460c016110 con 0x7f461414b490 2024-04-25T13:48:12.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:12.974+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f46141609f0 con 0x7f461414b490 2024-04-25T13:48:13.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f460affd640 1 -- 172.21.15.80:0/1724862483 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) ==== 122+0+0 (secure 0 0 0) 0x7f460c0f5a50 con 0x7f461414b490 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45f406f400 msgr2=0x7f45f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45f406f400 0x7f45f4071910 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f4604007e90 tx=0x7f46040073a0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f461414b490 msgr2=0x7f461414f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f461414b490 0x7f461414f370 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f460c01abd0 tx=0x7f460c0192c0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 shutdown_connections 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f45f406f400 0x7f45f4071910 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f461414b490 0x7f461414f370 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4614073c40 0x7f4614156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 --2- 172.21.15.80:0/1724862483 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4614071e60 0x7f461413d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 >> 172.21.15.80:0/1724862483 conn(0x7f46140b4160 msgr2=0x7f461415cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 shutdown_connections 2024-04-25T13:48:13.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.594+0000 7f4618c84640 1 -- 172.21.15.80:0/1724862483 wait complete. 2024-04-25T13:48:13.617 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:48:13.617 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:48:13.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a9eb7640 1 Processor -- start 2024-04-25T13:48:13.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a9eb7640 1 -- start start 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a9eb7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a4068ca0 0x7f05a406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a9eb7640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a406c690 con 0x7f05a40c88e0 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a9eb7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f05a4160ba0 con 0x7f05a4160790 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a9eb7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f05a40c7ee0 con 0x7f05a4068ca0 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a37fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f05a40c88e0 0x7f05a40c77d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53484/0 (socket says 172.21.15.80:53484) 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a37fe640 1 -- 172.21.15.80:0/158019944 learned_addr learned my addr 172.21.15.80:0/158019944 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a3fff640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a4068ca0 0x7f05a406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3112386745 0 0) 0x7f05a406c690 con 0x7f05a40c88e0 2024-04-25T13:48:13.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0584003530 con 0x7f05a40c88e0 2024-04-25T13:48:13.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.770+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 45050125 0 0) 0x7f0584003530 con 0x7f05a40c88e0 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a4068ca0 msgr2=0x7f05a406cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a4068ca0 0x7f05a406cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 >> v1:172.21.15.87:6789/0 conn(0x7f05a4160790 legacy=0x7f05a4068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a4160e40 con 0x7f05a40c88e0 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f0594001680 con 0x7f05a40c88e0 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 >> v1:172.21.15.80:6789/0 conn(0x7f05a40c88e0 legacy=0x7f05a40c77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05840039f0 0x7f0584003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0584004810 0x7f0584004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0584005160 0x7f058400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f05a40c7ee0 con 0x7f0584005160 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f058400bf10 con 0x7f05840039f0 2024-04-25T13:48:13.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f058400c090 con 0x7f0584004810 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a3fff640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0584005160 0x7f058400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a2ffd640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0584004810 0x7f0584004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a37fe640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05840039f0 0x7f0584003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a3fff640 1 -- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0584004810 msgr2=0x7f0584004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a3fff640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0584004810 0x7f0584004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a3fff640 1 -- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05840039f0 msgr2=0x7f0584003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a3fff640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05840039f0 0x7f0584003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a3fff640 1 -- 172.21.15.80:0/158019944 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0584003530 con 0x7f0584005160 2024-04-25T13:48:13.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a2ffd640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0584004810 0x7f0584004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a3fff640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0584005160 0x7f058400b9d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f059c009ab0 tx=0x7f059c01ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624942966641ae6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0b2070 con 0x7f0584005160 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059c0a5550 con 0x7f0584005160 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a27fc640 1 -- 172.21.15.80:0/158019944 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0096b0 con 0x7f0584005160 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a9eb7640 1 -- 172.21.15.80:0/158019944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0584005160 msgr2=0x7f058400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a9eb7640 1 --2- 172.21.15.80:0/158019944 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0584005160 0x7f058400b9d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f059c009ab0 tx=0x7f059c01ca70 comp rx=0 tx=0).stop 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a3fff640 1 -- 172.21.15.80:0/158019944 reap_dead start 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a9eb7640 1 -- 172.21.15.80:0/158019944 shutdown_connections 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a9eb7640 1 -- 172.21.15.80:0/158019944 >> 172.21.15.80:0/158019944 conn(0x7f05a40cf390 msgr2=0x7f05a40cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a9eb7640 1 -- 172.21.15.80:0/158019944 shutdown_connections 2024-04-25T13:48:13.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.774+0000 7f05a9eb7640 1 -- 172.21.15.80:0/158019944 wait complete. 2024-04-25T13:48:13.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a9eb7640 1 Processor -- start 2024-04-25T13:48:13.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a9eb7640 1 -- start start 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a9eb7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a41f9660 0x7f05a40cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a9eb7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a40cbfe0 0x7f05a4164280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a9eb7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05a41f9a70 0x7f05a4149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a9eb7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41616e0 con 0x7f05a41f9a70 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a9eb7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4161400 con 0x7f05a40cbfe0 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a9eb7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f05a4070b90 con 0x7f05a41f9660 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05a41f9a70 0x7f05a4149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05a41f9a70 0x7f05a4149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50528/0 (socket says 172.21.15.80:50528) 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 -- 172.21.15.80:0/4174183770 learned_addr learned my addr 172.21.15.80:0/4174183770 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:13.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a2ffd640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a40cbfe0 0x7f05a4164280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a37fe640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a41f9660 0x7f05a40cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 -- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a41f9660 msgr2=0x7f05a40cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a41f9660 0x7f05a40cbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 -- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a40cbfe0 msgr2=0x7f05a4164280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a40cbfe0 0x7f05a4164280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 -- 172.21.15.80:0/4174183770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a40caa00 con 0x7f05a41f9a70 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a2ffd640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a40cbfe0 0x7f05a4164280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f05a3fff640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05a41f9a70 0x7f05a4149570 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f059c009ab0 tx=0x7f059c019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.778+0000 7f059bfff640 1 -- 172.21.15.80:0/4174183770 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0b2040 con 0x7f05a41f9a70 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f059bfff640 1 -- 172.21.15.80:0/4174183770 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f059c0a5e10 con 0x7f05a41f9a70 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f059bfff640 1 -- 172.21.15.80:0/4174183770 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c0096b0 con 0x7f05a41f9a70 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a40cace0 con 0x7f05a41f9a70 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a40cb2a0 con 0x7f05a41f9a70 2024-04-25T13:48:13.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a406a3a0 con 0x7f05a41f9a70 2024-04-25T13:48:13.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f059bfff640 1 -- 172.21.15.80:0/4174183770 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f059c0bf060 con 0x7f05a41f9a70 2024-04-25T13:48:13.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f059bfff640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f058806f400 0x7f0588071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:13.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f059bfff640 1 -- 172.21.15.80:0/4174183770 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f059c128ab0 con 0x7f05a41f9a70 2024-04-25T13:48:13.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f05a37fe640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f058806f400 0x7f0588071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:13.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.782+0000 7f05a37fe640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f058806f400 0x7f0588071910 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f05940017f0 tx=0x7f0594001500 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:13.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.786+0000 7f059bfff640 1 -- 172.21.15.80:0/4174183770 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f059c016780 con 0x7f05a41f9a70 2024-04-25T13:48:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.982+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05a406db80 con 0x7f05a41f9a70 2024-04-25T13:48:13.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.982+0000 7f059bfff640 1 -- 172.21.15.80:0/4174183770 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7f059c0f59a0 con 0x7f05a41f9a70 2024-04-25T13:48:13.981 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:13.981 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/521602421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":521602421},{"type":"v1","addr":"172.21.15.87:6835","nonce":521602421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4170043750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4170043750},{"type":"v1","addr":"172.21.15.87:6837","nonce":4170043750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/1423926990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1423926990},{"type":"v1","addr":"172.21.15.80:6834","nonce":1423926990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:48:09.907104+0000","modified":"2024-04-25T13:48:12.977708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1134081162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1134081162},{"type":"v1","addr":"172.21.15.80:6835","nonce":1134081162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-25T13:48:13.981 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 74 2024-04-25T13:48:13.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f058806f400 msgr2=0x7f0588071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f058806f400 0x7f0588071910 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f05940017f0 tx=0x7f0594001500 comp rx=0 tx=0).stop 2024-04-25T13:48:13.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05a41f9a70 msgr2=0x7f05a4149570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:13.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05a41f9a70 0x7f05a4149570 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f059c009ab0 tx=0x7f059c019810 comp rx=0 tx=0).stop 2024-04-25T13:48:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 shutdown_connections 2024-04-25T13:48:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f058806f400 0x7f0588071910 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f05a41f9a70 0x7f05a4149570 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f05a40cbfe0 0x7f05a4164280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 --2- 172.21.15.80:0/4174183770 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f05a41f9660 0x7f05a40cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 >> 172.21.15.80:0/4174183770 conn(0x7f05a40cf390 msgr2=0x7f05a40c77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 shutdown_connections 2024-04-25T13:48:13.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:13.986+0000 7f05a9eb7640 1 -- 172.21.15.80:0/4174183770 wait complete. 2024-04-25T13:48:14.003 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:48:14.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.150+0000 7fce4e280640 1 Processor -- start 2024-04-25T13:48:14.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.150+0000 7fce4e280640 1 -- start start 2024-04-25T13:48:14.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4e280640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fce480d59b0 0x7fce480d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4e280640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4e280640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce480d6300 con 0x7fce480d2e80 2024-04-25T13:48:14.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4e280640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fce48161060 con 0x7fce480d59b0 2024-04-25T13:48:14.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4e280640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce48161340 con 0x7fce480d4330 2024-04-25T13:48:14.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce477fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fce480d59b0 0x7fce480d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4c81d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce47fff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fce480d2e80 0x7fce481607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:38468/0 (socket says 172.21.15.80:38468) 2024-04-25T13:48:14.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce47fff640 1 -- 172.21.15.80:0/3797595934 learned_addr learned my addr 172.21.15.80:0/3797595934 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:14.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4c81d640 1 -- 172.21.15.80:0/3797595934 >> v2:172.21.15.87:3300/0 conn(0x7fce480d59b0 msgr2=0x7fce480d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:14.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4c81d640 1 --2- 172.21.15.80:0/3797595934 >> v2:172.21.15.87:3300/0 conn(0x7fce480d59b0 0x7fce480d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4c81d640 1 -- 172.21.15.80:0/3797595934 >> v1:172.21.15.87:6789/0 conn(0x7fce480d2e80 legacy=0x7fce481607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.154+0000 7fce4c81d640 1 -- 172.21.15.80:0/3797595934 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce480d2680 con 0x7fce480d4330 2024-04-25T13:48:14.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.158+0000 7fce4c81d640 1 --2- 172.21.15.80:0/3797595934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480d4740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fce40009d90 tx=0x7fce4001c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=320fe36999f76fa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.158+0000 7fce46ffd640 1 -- 172.21.15.80:0/3797595934 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400b2070 con 0x7fce480d4330 2024-04-25T13:48:14.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.158+0000 7fce46ffd640 1 -- 172.21.15.80:0/3797595934 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce400a58b0 con 0x7fce480d4330 2024-04-25T13:48:14.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.158+0000 7fce46ffd640 1 -- 172.21.15.80:0/3797595934 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400a5c80 con 0x7fce480d4330 2024-04-25T13:48:14.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 -- 172.21.15.80:0/3797595934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 msgr2=0x7fce480d4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 --2- 172.21.15.80:0/3797595934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480d4740 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fce40009d90 tx=0x7fce4001c8e0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 -- 172.21.15.80:0/3797595934 shutdown_connections 2024-04-25T13:48:14.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 --2- 172.21.15.80:0/3797595934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480d4740 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 --2- 172.21.15.80:0/3797595934 >> v2:172.21.15.87:3300/0 conn(0x7fce480d59b0 0x7fce480d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 -- 172.21.15.80:0/3797595934 >> 172.21.15.80:0/3797595934 conn(0x7fce480737a0 msgr2=0x7fce4806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:14.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 -- 172.21.15.80:0/3797595934 shutdown_connections 2024-04-25T13:48:14.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 -- 172.21.15.80:0/3797595934 wait complete. 2024-04-25T13:48:14.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 Processor -- start 2024-04-25T13:48:14.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.162+0000 7fce4e280640 1 -- start start 2024-04-25T13:48:14.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce480d2e80 0x7fce480ba840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce480d59b0 0x7fce480cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce480d3410 con 0x7fce480d59b0 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce480d3290 con 0x7fce480d2e80 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce4815df60 con 0x7fce480d4330 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce480d59b0 0x7fce480cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce480d59b0 0x7fce480cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50544/0 (socket says 172.21.15.80:50544) 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 -- 172.21.15.80:0/2286091957 learned_addr learned my addr 172.21.15.80:0/2286091957 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce47fff640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce480d2e80 0x7fce480ba840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce477fe640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 -- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 msgr2=0x7fce480cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480cc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 -- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce480d2e80 msgr2=0x7fce480ba840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce480d2e80 0x7fce480ba840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 -- 172.21.15.80:0/2286091957 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce480cfad0 con 0x7fce480d59b0 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4c81d640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce480d59b0 0x7fce480cabf0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fce400b0040 tx=0x7fce40019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce44ff9640 1 -- 172.21.15.80:0/2286091957 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400b2050 con 0x7fce480d59b0 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce44ff9640 1 -- 172.21.15.80:0/2286091957 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce400be070 con 0x7fce480d59b0 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce480cf7f0 con 0x7fce480d59b0 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce44ff9640 1 -- 172.21.15.80:0/2286091957 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce400ada50 con 0x7fce480d59b0 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce480ba040 con 0x7fce480d59b0 2024-04-25T13:48:14.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce4806a3a0 con 0x7fce480d59b0 2024-04-25T13:48:14.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce44ff9640 1 -- 172.21.15.80:0/2286091957 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fce400a5730 con 0x7fce480d59b0 2024-04-25T13:48:14.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce44ff9640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3406f430 0x7fce34071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce44ff9640 1 -- 172.21.15.80:0/2286091957 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fce40129d20 con 0x7fce480d59b0 2024-04-25T13:48:14.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.166+0000 7fce47fff640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3406f430 0x7fce34071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.170+0000 7fce47fff640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3406f430 0x7fce34071940 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fce38000f70 tx=0x7fce38009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.174+0000 7fce44ff9640 1 -- 172.21.15.80:0/2286091957 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce400c7020 con 0x7fce480d59b0 2024-04-25T13:48:14.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.334+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce480bbb90 con 0x7fce480d59b0 2024-04-25T13:48:14.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.334+0000 7fce44ff9640 1 -- 172.21.15.80:0/2286091957 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+16608 (secure 0 0 0) 0x7fce40016560 con 0x7fce480d59b0 2024-04-25T13:48:14.334 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:14.334 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":154,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:48:11.553226+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:48:07.666863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T13:48:08.875350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:48:14.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3406f430 msgr2=0x7fce34071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3406f430 0x7fce34071940 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fce38000f70 tx=0x7fce38009040 comp rx=0 tx=0).stop 2024-04-25T13:48:14.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce480d59b0 msgr2=0x7fce480cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce480d59b0 0x7fce480cabf0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fce400b0040 tx=0x7fce40019680 comp rx=0 tx=0).stop 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 shutdown_connections 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3406f430 0x7fce34071940 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce480d59b0 0x7fce480cabf0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce480d4330 0x7fce480cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 --2- 172.21.15.80:0/2286091957 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce480d2e80 0x7fce480ba840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 >> 172.21.15.80:0/2286091957 conn(0x7fce480737a0 msgr2=0x7fce4806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 shutdown_connections 2024-04-25T13:48:14.337 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.338+0000 7fce4e280640 1 -- 172.21.15.80:0/2286091957 wait complete. 2024-04-25T13:48:14.357 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:48:14.357 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:48:14.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.502+0000 7f8fc7ac9640 1 Processor -- start 2024-04-25T13:48:14.502 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.502+0000 7f8fc7ac9640 1 -- start start 2024-04-25T13:48:14.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc7ac9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8fc00d06e0 0x7f8fc006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc7ac9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8fc006e9e0 0x7f8fc006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.504 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc7ac9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fc00d0af0 con 0x7f8fc0161f90 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc7ac9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc00d0d70 con 0x7f8fc006e9e0 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc7ac9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8fc00d1050 con 0x7f8fc00d06e0 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc6066640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8fc006e9e0 0x7f8fc006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc6066640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8fc006e9e0 0x7f8fc006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50558/0 (socket says 172.21.15.80:50558) 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc6066640 1 -- 172.21.15.80:0/3072885841 learned_addr learned my addr 172.21.15.80:0/3072885841 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc5064640 1 --2- 172.21.15.80:0/3072885841 >> v2:172.21.15.87:3300/0 conn(0x7f8fc00d06e0 0x7f8fc006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc6066640 1 -- 172.21.15.80:0/3072885841 >> v2:172.21.15.87:3300/0 conn(0x7f8fc00d06e0 msgr2=0x7f8fc006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc6066640 1 --2- 172.21.15.80:0/3072885841 >> v2:172.21.15.87:3300/0 conn(0x7f8fc00d06e0 0x7f8fc006e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc6066640 1 -- 172.21.15.80:0/3072885841 >> v1:172.21.15.87:6789/0 conn(0x7f8fc0161f90 legacy=0x7f8fc00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:14.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.506+0000 7f8fc6066640 1 -- 172.21.15.80:0/3072885841 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc006dca0 con 0x7f8fc006e9e0 2024-04-25T13:48:14.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc6066640 1 --2- 172.21.15.80:0/3072885841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc006edf0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0074290 tx=0x7f8fb801c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d582ff66c092f446 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc4863640 1 -- 172.21.15.80:0/3072885841 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80ac020 con 0x7f8fc006e9e0 2024-04-25T13:48:14.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc4863640 1 -- 172.21.15.80:0/3072885841 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb80a6460 con 0x7f8fc006e9e0 2024-04-25T13:48:14.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc4863640 1 -- 172.21.15.80:0/3072885841 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb80a6830 con 0x7f8fc006e9e0 2024-04-25T13:48:14.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/3072885841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 msgr2=0x7f8fc006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/3072885841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc006edf0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0074290 tx=0x7f8fb801c680 comp rx=0 tx=0).stop 2024-04-25T13:48:14.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/3072885841 shutdown_connections 2024-04-25T13:48:14.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/3072885841 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc006edf0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/3072885841 >> v2:172.21.15.87:3300/0 conn(0x7f8fc00d06e0 0x7f8fc006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/3072885841 >> 172.21.15.80:0/3072885841 conn(0x7f8fc006d080 msgr2=0x7f8fc00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:14.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/3072885841 shutdown_connections 2024-04-25T13:48:14.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/3072885841 wait complete. 2024-04-25T13:48:14.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.510+0000 7f8fc7ac9640 1 Processor -- start 2024-04-25T13:48:14.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 -- start start 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc0133ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fc00d06e0 0x7f8fc0073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8fc0161f90 0x7f8fc00b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc01447a0 con 0x7f8fc006e9e0 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc0144620 con 0x7f8fc00d06e0 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8fc016b510 con 0x7f8fc0161f90 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc0133ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc0133ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50560/0 (socket says 172.21.15.80:50560) 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 -- 172.21.15.80:0/1482933418 learned_addr learned my addr 172.21.15.80:0/1482933418 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc6066640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8fc0161f90 0x7f8fc00b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5064640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fc00d06e0 0x7f8fc0073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 -- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8fc0161f90 msgr2=0x7f8fc00b4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8fc0161f90 0x7f8fc00b4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 -- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fc00d06e0 msgr2=0x7f8fc0073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fc00d06e0 0x7f8fc0073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 -- 172.21.15.80:0/1482933418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc0072800 con 0x7f8fc006e9e0 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc6066640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8fc0161f90 0x7f8fc00b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc5865640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc0133ec0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00b3f0 tx=0x7f8fbc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fb67fc640 1 -- 172.21.15.80:0/1482933418 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc015020 con 0x7f8fc006e9e0 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fb67fc640 1 -- 172.21.15.80:0/1482933418 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fbc011830 con 0x7f8fc006e9e0 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc015b9d0 con 0x7f8fc006e9e0 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fb67fc640 1 -- 172.21.15.80:0/1482933418 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc011c00 con 0x7f8fc006e9e0 2024-04-25T13:48:14.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc0072d80 con 0x7f8fc006e9e0 2024-04-25T13:48:14.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc006a3a0 con 0x7f8fc006e9e0 2024-04-25T13:48:14.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.514+0000 7f8fb67fc640 1 -- 172.21.15.80:0/1482933418 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8fbc018020 con 0x7f8fc006e9e0 2024-04-25T13:48:14.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.518+0000 7f8fb67fc640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fa406f400 0x7f8fa4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.518+0000 7f8fb67fc640 1 -- 172.21.15.80:0/1482933418 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f8fbc08eec0 con 0x7f8fc006e9e0 2024-04-25T13:48:14.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.518+0000 7f8fc5064640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fa406f400 0x7f8fa4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.518+0000 7f8fc5064640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fa406f400 0x7f8fa4071910 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00b3b30 tx=0x7f8fb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.522+0000 7f8fb67fc640 1 -- 172.21.15.80:0/1482933418 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8fbc01f030 con 0x7f8fc006e9e0 2024-04-25T13:48:14.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.670+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8fc0134400 con 0x7f8fa406f400 2024-04-25T13:48:14.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.686+0000 7f8fb67fc640 1 -- 172.21.15.80:0/1482933418 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247981 (secure 0 0 0) 0x7f8fb000f830 con 0x7f8fa406f400 2024-04-25T13:48:14.689 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:14.690 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:48:14.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fa406f400 msgr2=0x7f8fa4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.691 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fa406f400 0x7f8fa4071910 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8fc00b3b30 tx=0x7f8fb0008040 comp rx=0 tx=0).stop 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 msgr2=0x7f8fc0133ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc0133ec0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00b3f0 tx=0x7f8fbc00b8b0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 shutdown_connections 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8fa406f400 0x7f8fa4071910 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8fc0161f90 0x7f8fc00b4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8fc00d06e0 0x7f8fc0073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 --2- 172.21.15.80:0/1482933418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8fc006e9e0 0x7f8fc0133ec0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 >> 172.21.15.80:0/1482933418 conn(0x7f8fc006d080 msgr2=0x7f8fc015d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 shutdown_connections 2024-04-25T13:48:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.690+0000 7f8fc7ac9640 1 -- 172.21.15.80:0/1482933418 wait complete. 2024-04-25T13:48:14.709 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-25T13:48:14.270265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7887,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554612,"statfs":{"total":1600000000,"available":1591923508,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.758739"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635201+0000","last_change":"2024-04-25T13:48:10.592925+0000","last_active":"2024-04-25T13:48:11.635201+0000","last_peered":"2024-04-25T13:48:11.635201+0000","last_clean":"2024-04-25T13:48:11.635201+0000","last_became_active":"2024-04-25T13:48:10.592749+0000","last_became_peered":"2024-04-25T13:48:10.592749+0000","last_unstale":"2024-04-25T13:48:11.635201+0000","last_undegraded":"2024-04-25T13:48:11.635201+0000","last_fullsized":"2024-04-25T13:48:11.635201+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:31.807837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637960+0000","last_change":"2024-04-25T13:48:10.600062+0000","last_active":"2024-04-25T13:48:11.637960+0000","last_peered":"2024-04-25T13:48:11.637960+0000","last_clean":"2024-04-25T13:48:11.637960+0000","last_became_active":"2024-04-25T13:48:10.599776+0000","last_became_peered":"2024-04-25T13:48:10.599776+0000","last_unstale":"2024-04-25T13:48:11.637960+0000","last_undegraded":"2024-04-25T13:48:11.637960+0000","last_fullsized":"2024-04-25T13:48:11.637960+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:33.717096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634698+0000","last_change":"2024-04-25T13:48:10.608299+0000","last_active":"2024-04-25T13:48:11.634698+0000","last_peered":"2024-04-25T13:48:11.634698+0000","last_clean":"2024-04-25T13:48:11.634698+0000","last_became_active":"2024-04-25T13:48:10.604112+0000","last_became_peered":"2024-04-25T13:48:10.604112+0000","last_unstale":"2024-04-25T13:48:11.634698+0000","last_undegraded":"2024-04-25T13:48:11.634698+0000","last_fullsized":"2024-04-25T13:48:11.634698+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:49.048283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634136+0000","last_change":"2024-04-25T13:48:10.592632+0000","last_active":"2024-04-25T13:48:11.634136+0000","last_peered":"2024-04-25T13:48:11.634136+0000","last_clean":"2024-04-25T13:48:11.634136+0000","last_became_active":"2024-04-25T13:48:10.592343+0000","last_became_peered":"2024-04-25T13:48:10.592343+0000","last_unstale":"2024-04-25T13:48:11.634136+0000","last_undegraded":"2024-04-25T13:48:11.634136+0000","last_fullsized":"2024-04-25T13:48:11.634136+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:32.059799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567395+0000","last_change":"2024-04-25T13:48:10.578533+0000","last_active":"2024-04-25T13:48:11.567395+0000","last_peered":"2024-04-25T13:48:11.567395+0000","last_clean":"2024-04-25T13:48:11.567395+0000","last_became_active":"2024-04-25T13:48:10.576452+0000","last_became_peered":"2024-04-25T13:48:10.576452+0000","last_unstale":"2024-04-25T13:48:11.567395+0000","last_undegraded":"2024-04-25T13:48:11.567395+0000","last_fullsized":"2024-04-25T13:48:11.567395+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:58.914815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.560929+0000","last_change":"2024-04-25T13:48:10.581808+0000","last_active":"2024-04-25T13:48:11.560929+0000","last_peered":"2024-04-25T13:48:11.560929+0000","last_clean":"2024-04-25T13:48:11.560929+0000","last_became_active":"2024-04-25T13:48:10.580999+0000","last_became_peered":"2024-04-25T13:48:10.580999+0000","last_unstale":"2024-04-25T13:48:11.560929+0000","last_undegraded":"2024-04-25T13:48:11.560929+0000","last_fullsized":"2024-04-25T13:48:11.560929+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:38.170247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.560950+0000","last_change":"2024-04-25T13:48:10.605992+0000","last_active":"2024-04-25T13:48:11.560950+0000","last_peered":"2024-04-25T13:48:11.560950+0000","last_clean":"2024-04-25T13:48:11.560950+0000","last_became_active":"2024-04-25T13:48:10.603624+0000","last_became_peered":"2024-04-25T13:48:10.603624+0000","last_unstale":"2024-04-25T13:48:11.560950+0000","last_undegraded":"2024-04-25T13:48:11.560950+0000","last_fullsized":"2024-04-25T13:48:11.560950+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:15.179659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635166+0000","last_change":"2024-04-25T13:48:10.606153+0000","last_active":"2024-04-25T13:48:11.635166+0000","last_peered":"2024-04-25T13:48:11.635166+0000","last_clean":"2024-04-25T13:48:11.635166+0000","last_became_active":"2024-04-25T13:48:10.595457+0000","last_became_peered":"2024-04-25T13:48:10.595457+0000","last_unstale":"2024-04-25T13:48:11.635166+0000","last_undegraded":"2024-04-25T13:48:11.635166+0000","last_fullsized":"2024-04-25T13:48:11.635166+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:17.284240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633522+0000","last_change":"2024-04-25T13:48:10.609502+0000","last_active":"2024-04-25T13:48:11.633522+0000","last_peered":"2024-04-25T13:48:11.633522+0000","last_clean":"2024-04-25T13:48:11.633522+0000","last_became_active":"2024-04-25T13:48:10.607502+0000","last_became_peered":"2024-04-25T13:48:10.607502+0000","last_unstale":"2024-04-25T13:48:11.633522+0000","last_undegraded":"2024-04-25T13:48:11.633522+0000","last_fullsized":"2024-04-25T13:48:11.633522+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:51.404585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638158+0000","last_change":"2024-04-25T13:48:10.581304+0000","last_active":"2024-04-25T13:48:11.638158+0000","last_peered":"2024-04-25T13:48:11.638158+0000","last_clean":"2024-04-25T13:48:11.638158+0000","last_became_active":"2024-04-25T13:48:10.581043+0000","last_became_peered":"2024-04-25T13:48:10.581043+0000","last_unstale":"2024-04-25T13:48:11.638158+0000","last_undegraded":"2024-04-25T13:48:11.638158+0000","last_fullsized":"2024-04-25T13:48:11.638158+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.187343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558177+0000","last_change":"2024-04-25T13:48:10.568217+0000","last_active":"2024-04-25T13:48:11.558177+0000","last_peered":"2024-04-25T13:48:11.558177+0000","last_clean":"2024-04-25T13:48:11.558177+0000","last_became_active":"2024-04-25T13:48:10.567891+0000","last_became_peered":"2024-04-25T13:48:10.567891+0000","last_unstale":"2024-04-25T13:48:11.558177+0000","last_undegraded":"2024-04-25T13:48:11.558177+0000","last_fullsized":"2024-04-25T13:48:11.558177+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:38.896254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634083+0000","last_change":"2024-04-25T13:48:10.596081+0000","last_active":"2024-04-25T13:48:11.634083+0000","last_peered":"2024-04-25T13:48:11.634083+0000","last_clean":"2024-04-25T13:48:11.634083+0000","last_became_active":"2024-04-25T13:48:10.594744+0000","last_became_peered":"2024-04-25T13:48:10.594744+0000","last_unstale":"2024-04-25T13:48:11.634083+0000","last_undegraded":"2024-04-25T13:48:11.634083+0000","last_fullsized":"2024-04-25T13:48:11.634083+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:49.910198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634336+0000","last_change":"2024-04-25T13:48:10.581211+0000","last_active":"2024-04-25T13:48:11.634336+0000","last_peered":"2024-04-25T13:48:11.634336+0000","last_clean":"2024-04-25T13:48:11.634336+0000","last_became_active":"2024-04-25T13:48:10.580915+0000","last_became_peered":"2024-04-25T13:48:10.580915+0000","last_unstale":"2024-04-25T13:48:11.634336+0000","last_undegraded":"2024-04-25T13:48:11.634336+0000","last_fullsized":"2024-04-25T13:48:11.634336+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:43.025919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635311+0000","last_change":"2024-04-25T13:48:10.574585+0000","last_active":"2024-04-25T13:48:11.635311+0000","last_peered":"2024-04-25T13:48:11.635311+0000","last_clean":"2024-04-25T13:48:11.635311+0000","last_became_active":"2024-04-25T13:48:10.574286+0000","last_became_peered":"2024-04-25T13:48:10.574286+0000","last_unstale":"2024-04-25T13:48:11.635311+0000","last_undegraded":"2024-04-25T13:48:11.635311+0000","last_fullsized":"2024-04-25T13:48:11.635311+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:43.345750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636289+0000","last_change":"2024-04-25T13:48:10.608035+0000","last_active":"2024-04-25T13:48:11.636289+0000","last_peered":"2024-04-25T13:48:11.636289+0000","last_clean":"2024-04-25T13:48:11.636289+0000","last_became_active":"2024-04-25T13:48:10.607799+0000","last_became_peered":"2024-04-25T13:48:10.607799+0000","last_unstale":"2024-04-25T13:48:11.636289+0000","last_undegraded":"2024-04-25T13:48:11.636289+0000","last_fullsized":"2024-04-25T13:48:11.636289+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:13.009054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635771+0000","last_change":"2024-04-25T13:48:10.574696+0000","last_active":"2024-04-25T13:48:11.635771+0000","last_peered":"2024-04-25T13:48:11.635771+0000","last_clean":"2024-04-25T13:48:11.635771+0000","last_became_active":"2024-04-25T13:48:10.574100+0000","last_became_peered":"2024-04-25T13:48:10.574100+0000","last_unstale":"2024-04-25T13:48:11.635771+0000","last_undegraded":"2024-04-25T13:48:11.635771+0000","last_fullsized":"2024-04-25T13:48:11.635771+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:56.604752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561528+0000","last_change":"2024-04-25T13:48:10.604598+0000","last_active":"2024-04-25T13:48:11.561528+0000","last_peered":"2024-04-25T13:48:11.561528+0000","last_clean":"2024-04-25T13:48:11.561528+0000","last_became_active":"2024-04-25T13:48:10.595277+0000","last_became_peered":"2024-04-25T13:48:10.595277+0000","last_unstale":"2024-04-25T13:48:11.561528+0000","last_undegraded":"2024-04-25T13:48:11.561528+0000","last_fullsized":"2024-04-25T13:48:11.561528+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:23.351222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636319+0000","last_change":"2024-04-25T13:48:10.577753+0000","last_active":"2024-04-25T13:48:11.636319+0000","last_peered":"2024-04-25T13:48:11.636319+0000","last_clean":"2024-04-25T13:48:11.636319+0000","last_became_active":"2024-04-25T13:48:10.577130+0000","last_became_peered":"2024-04-25T13:48:10.577130+0000","last_unstale":"2024-04-25T13:48:11.636319+0000","last_undegraded":"2024-04-25T13:48:11.636319+0000","last_fullsized":"2024-04-25T13:48:11.636319+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:08.529838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558273+0000","last_change":"2024-04-25T13:48:10.584718+0000","last_active":"2024-04-25T13:48:11.558273+0000","last_peered":"2024-04-25T13:48:11.558273+0000","last_clean":"2024-04-25T13:48:11.558273+0000","last_became_active":"2024-04-25T13:48:10.584502+0000","last_became_peered":"2024-04-25T13:48:10.584502+0000","last_unstale":"2024-04-25T13:48:11.558273+0000","last_undegraded":"2024-04-25T13:48:11.558273+0000","last_fullsized":"2024-04-25T13:48:11.558273+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:23.345882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637310+0000","last_change":"2024-04-25T13:48:10.568056+0000","last_active":"2024-04-25T13:48:11.637310+0000","last_peered":"2024-04-25T13:48:11.637310+0000","last_clean":"2024-04-25T13:48:11.637310+0000","last_became_active":"2024-04-25T13:48:10.567784+0000","last_became_peered":"2024-04-25T13:48:10.567784+0000","last_unstale":"2024-04-25T13:48:11.637310+0000","last_undegraded":"2024-04-25T13:48:11.637310+0000","last_fullsized":"2024-04-25T13:48:11.637310+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.031661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.568040+0000","last_change":"2024-04-25T13:48:10.601480+0000","last_active":"2024-04-25T13:48:11.568040+0000","last_peered":"2024-04-25T13:48:11.568040+0000","last_clean":"2024-04-25T13:48:11.568040+0000","last_became_active":"2024-04-25T13:48:10.599829+0000","last_became_peered":"2024-04-25T13:48:10.599829+0000","last_unstale":"2024-04-25T13:48:11.568040+0000","last_undegraded":"2024-04-25T13:48:11.568040+0000","last_fullsized":"2024-04-25T13:48:11.568040+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:53.241219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.559103+0000","last_change":"2024-04-25T13:48:10.564202+0000","last_active":"2024-04-25T13:48:11.559103+0000","last_peered":"2024-04-25T13:48:11.559103+0000","last_clean":"2024-04-25T13:48:11.559103+0000","last_became_active":"2024-04-25T13:48:10.563907+0000","last_became_peered":"2024-04-25T13:48:10.563907+0000","last_unstale":"2024-04-25T13:48:11.559103+0000","last_undegraded":"2024-04-25T13:48:11.559103+0000","last_fullsized":"2024-04-25T13:48:11.559103+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:17.613120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636377+0000","last_change":"2024-04-25T13:48:10.609109+0000","last_active":"2024-04-25T13:48:11.636377+0000","last_peered":"2024-04-25T13:48:11.636377+0000","last_clean":"2024-04-25T13:48:11.636377+0000","last_became_active":"2024-04-25T13:48:10.605064+0000","last_became_peered":"2024-04-25T13:48:10.605064+0000","last_unstale":"2024-04-25T13:48:11.636377+0000","last_undegraded":"2024-04-25T13:48:11.636377+0000","last_fullsized":"2024-04-25T13:48:11.636377+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:18.015278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558748+0000","last_change":"2024-04-25T13:48:10.590782+0000","last_active":"2024-04-25T13:48:11.558748+0000","last_peered":"2024-04-25T13:48:11.558748+0000","last_clean":"2024-04-25T13:48:11.558748+0000","last_became_active":"2024-04-25T13:48:10.590523+0000","last_became_peered":"2024-04-25T13:48:10.590523+0000","last_unstale":"2024-04-25T13:48:11.558748+0000","last_undegraded":"2024-04-25T13:48:11.558748+0000","last_fullsized":"2024-04-25T13:48:11.558748+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:27.850740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638561+0000","last_change":"2024-04-25T13:48:10.572798+0000","last_active":"2024-04-25T13:48:11.638561+0000","last_peered":"2024-04-25T13:48:11.638561+0000","last_clean":"2024-04-25T13:48:11.638561+0000","last_became_active":"2024-04-25T13:48:10.572557+0000","last_became_peered":"2024-04-25T13:48:10.572557+0000","last_unstale":"2024-04-25T13:48:11.638561+0000","last_undegraded":"2024-04-25T13:48:11.638561+0000","last_fullsized":"2024-04-25T13:48:11.638561+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:04.512135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636444+0000","last_change":"2024-04-25T13:48:10.574409+0000","last_active":"2024-04-25T13:48:11.636444+0000","last_peered":"2024-04-25T13:48:11.636444+0000","last_clean":"2024-04-25T13:48:11.636444+0000","last_became_active":"2024-04-25T13:48:10.573716+0000","last_became_peered":"2024-04-25T13:48:10.573716+0000","last_unstale":"2024-04-25T13:48:11.636444+0000","last_undegraded":"2024-04-25T13:48:11.636444+0000","last_fullsized":"2024-04-25T13:48:11.636444+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:24.911925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633805+0000","last_change":"2024-04-25T13:48:10.600233+0000","last_active":"2024-04-25T13:48:11.633805+0000","last_peered":"2024-04-25T13:48:11.633805+0000","last_clean":"2024-04-25T13:48:11.633805+0000","last_became_active":"2024-04-25T13:48:10.600016+0000","last_became_peered":"2024-04-25T13:48:10.600016+0000","last_unstale":"2024-04-25T13:48:11.633805+0000","last_undegraded":"2024-04-25T13:48:11.633805+0000","last_fullsized":"2024-04-25T13:48:11.633805+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:17.409340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633626+0000","last_change":"2024-04-25T13:48:10.581556+0000","last_active":"2024-04-25T13:48:11.633626+0000","last_peered":"2024-04-25T13:48:11.633626+0000","last_clean":"2024-04-25T13:48:11.633626+0000","last_became_active":"2024-04-25T13:48:10.580516+0000","last_became_peered":"2024-04-25T13:48:10.580516+0000","last_unstale":"2024-04-25T13:48:11.633626+0000","last_undegraded":"2024-04-25T13:48:11.633626+0000","last_fullsized":"2024-04-25T13:48:11.633626+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:35.153374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634815+0000","last_change":"2024-04-25T13:48:10.610753+0000","last_active":"2024-04-25T13:48:11.634815+0000","last_peered":"2024-04-25T13:48:11.634815+0000","last_clean":"2024-04-25T13:48:11.634815+0000","last_became_active":"2024-04-25T13:48:10.610511+0000","last_became_peered":"2024-04-25T13:48:10.610511+0000","last_unstale":"2024-04-25T13:48:11.634815+0000","last_undegraded":"2024-04-25T13:48:11.634815+0000","last_fullsized":"2024-04-25T13:48:11.634815+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:56.209730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636720+0000","last_change":"2024-04-25T13:48:10.606471+0000","last_active":"2024-04-25T13:48:11.636720+0000","last_peered":"2024-04-25T13:48:11.636720+0000","last_clean":"2024-04-25T13:48:11.636720+0000","last_became_active":"2024-04-25T13:48:10.606304+0000","last_became_peered":"2024-04-25T13:48:10.606304+0000","last_unstale":"2024-04-25T13:48:11.636720+0000","last_undegraded":"2024-04-25T13:48:11.636720+0000","last_fullsized":"2024-04-25T13:48:11.636720+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:52.210208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566449+0000","last_change":"2024-04-25T13:48:10.599094+0000","last_active":"2024-04-25T13:48:11.566449+0000","last_peered":"2024-04-25T13:48:11.566449+0000","last_clean":"2024-04-25T13:48:11.566449+0000","last_became_active":"2024-04-25T13:48:10.587767+0000","last_became_peered":"2024-04-25T13:48:10.587767+0000","last_unstale":"2024-04-25T13:48:11.566449+0000","last_undegraded":"2024-04-25T13:48:11.566449+0000","last_fullsized":"2024-04-25T13:48:11.566449+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:24.565204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561252+0000","last_change":"2024-04-25T13:48:10.598425+0000","last_active":"2024-04-25T13:48:11.561252+0000","last_peered":"2024-04-25T13:48:11.561252+0000","last_clean":"2024-04-25T13:48:11.561252+0000","last_became_active":"2024-04-25T13:48:10.591615+0000","last_became_peered":"2024-04-25T13:48:10.591615+0000","last_unstale":"2024-04-25T13:48:11.561252+0000","last_undegraded":"2024-04-25T13:48:11.561252+0000","last_fullsized":"2024-04-25T13:48:11.561252+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:08.551456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634573+0000","last_change":"2024-04-25T13:48:10.591056+0000","last_active":"2024-04-25T13:48:11.634573+0000","last_peered":"2024-04-25T13:48:11.634573+0000","last_clean":"2024-04-25T13:48:11.634573+0000","last_became_active":"2024-04-25T13:48:10.590774+0000","last_became_peered":"2024-04-25T13:48:10.590774+0000","last_unstale":"2024-04-25T13:48:11.634573+0000","last_undegraded":"2024-04-25T13:48:11.634573+0000","last_fullsized":"2024-04-25T13:48:11.634573+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:34.007994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"153'4","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566927+0000","last_change":"2024-04-25T13:48:09.549395+0000","last_active":"2024-04-25T13:48:11.566927+0000","last_peered":"2024-04-25T13:48:11.566927+0000","last_clean":"2024-04-25T13:48:11.566927+0000","last_became_active":"2024-04-25T13:48:09.547446+0000","last_became_peered":"2024-04-25T13:48:09.547446+0000","last_unstale":"2024-04-25T13:48:11.566927+0000","last_undegraded":"2024-04-25T13:48:11.566927+0000","last_fullsized":"2024-04-25T13:48:11.566927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:31.331278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635581+0000","last_change":"2024-04-25T13:48:10.607392+0000","last_active":"2024-04-25T13:48:11.635581+0000","last_peered":"2024-04-25T13:48:11.635581+0000","last_clean":"2024-04-25T13:48:11.635581+0000","last_became_active":"2024-04-25T13:48:10.596232+0000","last_became_peered":"2024-04-25T13:48:10.596232+0000","last_unstale":"2024-04-25T13:48:11.635581+0000","last_undegraded":"2024-04-25T13:48:11.635581+0000","last_fullsized":"2024-04-25T13:48:11.635581+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:55.747274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634026+0000","last_change":"2024-04-25T13:48:09.543046+0000","last_active":"2024-04-25T13:48:11.634026+0000","last_peered":"2024-04-25T13:48:11.634026+0000","last_clean":"2024-04-25T13:48:11.634026+0000","last_became_active":"2024-04-25T13:48:09.540531+0000","last_became_peered":"2024-04-25T13:48:09.540531+0000","last_unstale":"2024-04-25T13:48:11.634026+0000","last_undegraded":"2024-04-25T13:48:11.634026+0000","last_fullsized":"2024-04-25T13:48:11.634026+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:12.036299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566963+0000","last_change":"2024-04-25T13:48:10.602093+0000","last_active":"2024-04-25T13:48:11.566963+0000","last_peered":"2024-04-25T13:48:11.566963+0000","last_clean":"2024-04-25T13:48:11.566963+0000","last_became_active":"2024-04-25T13:48:10.600734+0000","last_became_peered":"2024-04-25T13:48:10.600734+0000","last_unstale":"2024-04-25T13:48:11.566963+0000","last_undegraded":"2024-04-25T13:48:11.566963+0000","last_fullsized":"2024-04-25T13:48:11.566963+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:34.960163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558376+0000","last_change":"2024-04-25T13:48:09.524381+0000","last_active":"2024-04-25T13:48:11.558376+0000","last_peered":"2024-04-25T13:48:11.558376+0000","last_clean":"2024-04-25T13:48:11.558376+0000","last_became_active":"2024-04-25T13:48:09.524087+0000","last_became_peered":"2024-04-25T13:48:09.524087+0000","last_unstale":"2024-04-25T13:48:11.558376+0000","last_undegraded":"2024-04-25T13:48:11.558376+0000","last_fullsized":"2024-04-25T13:48:11.558376+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:16.986135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567649+0000","last_change":"2024-04-25T13:48:10.573373+0000","last_active":"2024-04-25T13:48:11.567649+0000","last_peered":"2024-04-25T13:48:11.567649+0000","last_clean":"2024-04-25T13:48:11.567649+0000","last_became_active":"2024-04-25T13:48:10.572448+0000","last_became_peered":"2024-04-25T13:48:10.572448+0000","last_unstale":"2024-04-25T13:48:11.567649+0000","last_undegraded":"2024-04-25T13:48:11.567649+0000","last_fullsized":"2024-04-25T13:48:11.567649+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:51.528130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636532+0000","last_change":"2024-04-25T13:48:09.535052+0000","last_active":"2024-04-25T13:48:11.636532+0000","last_peered":"2024-04-25T13:48:11.636532+0000","last_clean":"2024-04-25T13:48:11.636532+0000","last_became_active":"2024-04-25T13:48:09.534432+0000","last_became_peered":"2024-04-25T13:48:09.534432+0000","last_unstale":"2024-04-25T13:48:11.636532+0000","last_undegraded":"2024-04-25T13:48:11.636532+0000","last_fullsized":"2024-04-25T13:48:11.636532+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:18.233215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636774+0000","last_change":"2024-04-25T13:48:10.577497+0000","last_active":"2024-04-25T13:48:11.636774+0000","last_peered":"2024-04-25T13:48:11.636774+0000","last_clean":"2024-04-25T13:48:11.636774+0000","last_became_active":"2024-04-25T13:48:10.576743+0000","last_became_peered":"2024-04-25T13:48:10.576743+0000","last_unstale":"2024-04-25T13:48:11.636774+0000","last_undegraded":"2024-04-25T13:48:11.636774+0000","last_fullsized":"2024-04-25T13:48:11.636774+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:21.517098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638256+0000","last_change":"2024-04-25T13:48:09.532353+0000","last_active":"2024-04-25T13:48:11.638256+0000","last_peered":"2024-04-25T13:48:11.638256+0000","last_clean":"2024-04-25T13:48:11.638256+0000","last_became_active":"2024-04-25T13:48:09.532088+0000","last_became_peered":"2024-04-25T13:48:09.532088+0000","last_unstale":"2024-04-25T13:48:11.638256+0000","last_undegraded":"2024-04-25T13:48:11.638256+0000","last_fullsized":"2024-04-25T13:48:11.638256+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:07.095781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561798+0000","last_change":"2024-04-25T13:48:10.600214+0000","last_active":"2024-04-25T13:48:11.561798+0000","last_peered":"2024-04-25T13:48:11.561798+0000","last_clean":"2024-04-25T13:48:11.561798+0000","last_became_active":"2024-04-25T13:48:10.592710+0000","last_became_peered":"2024-04-25T13:48:10.592710+0000","last_unstale":"2024-04-25T13:48:11.561798+0000","last_undegraded":"2024-04-25T13:48:11.561798+0000","last_fullsized":"2024-04-25T13:48:11.561798+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:09.910090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637082+0000","last_change":"2024-04-25T13:48:09.537201+0000","last_active":"2024-04-25T13:48:11.637082+0000","last_peered":"2024-04-25T13:48:11.637082+0000","last_clean":"2024-04-25T13:48:11.637082+0000","last_became_active":"2024-04-25T13:48:09.535209+0000","last_became_peered":"2024-04-25T13:48:09.535209+0000","last_unstale":"2024-04-25T13:48:11.637082+0000","last_undegraded":"2024-04-25T13:48:11.637082+0000","last_fullsized":"2024-04-25T13:48:11.637082+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:18.479251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633724+0000","last_change":"2024-04-25T13:48:10.605251+0000","last_active":"2024-04-25T13:48:11.633724+0000","last_peered":"2024-04-25T13:48:11.633724+0000","last_clean":"2024-04-25T13:48:11.633724+0000","last_became_active":"2024-04-25T13:48:10.603716+0000","last_became_peered":"2024-04-25T13:48:10.603716+0000","last_unstale":"2024-04-25T13:48:11.633724+0000","last_undegraded":"2024-04-25T13:48:11.633724+0000","last_fullsized":"2024-04-25T13:48:11.633724+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:38.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561304+0000","last_change":"2024-04-25T13:48:09.545812+0000","last_active":"2024-04-25T13:48:11.561304+0000","last_peered":"2024-04-25T13:48:11.561304+0000","last_clean":"2024-04-25T13:48:11.561304+0000","last_became_active":"2024-04-25T13:48:09.544969+0000","last_became_peered":"2024-04-25T13:48:09.544969+0000","last_unstale":"2024-04-25T13:48:11.561304+0000","last_undegraded":"2024-04-25T13:48:11.561304+0000","last_fullsized":"2024-04-25T13:48:11.561304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:27.155907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635792+0000","last_change":"2024-04-25T13:48:10.564110+0000","last_active":"2024-04-25T13:48:11.635792+0000","last_peered":"2024-04-25T13:48:11.635792+0000","last_clean":"2024-04-25T13:48:11.635792+0000","last_became_active":"2024-04-25T13:48:10.563794+0000","last_became_peered":"2024-04-25T13:48:10.563794+0000","last_unstale":"2024-04-25T13:48:11.635792+0000","last_undegraded":"2024-04-25T13:48:11.635792+0000","last_fullsized":"2024-04-25T13:48:11.635792+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:32.439930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.639063+0000","last_change":"2024-04-25T13:48:09.532274+0000","last_active":"2024-04-25T13:48:11.639063+0000","last_peered":"2024-04-25T13:48:11.639063+0000","last_clean":"2024-04-25T13:48:11.639063+0000","last_became_active":"2024-04-25T13:48:09.532033+0000","last_became_peered":"2024-04-25T13:48:09.532033+0000","last_unstale":"2024-04-25T13:48:11.639063+0000","last_undegraded":"2024-04-25T13:48:11.639063+0000","last_fullsized":"2024-04-25T13:48:11.639063+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:56.919040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634666+0000","last_change":"2024-04-25T13:48:10.577914+0000","last_active":"2024-04-25T13:48:11.634666+0000","last_peered":"2024-04-25T13:48:11.634666+0000","last_clean":"2024-04-25T13:48:11.634666+0000","last_became_active":"2024-04-25T13:48:10.577481+0000","last_became_peered":"2024-04-25T13:48:10.577481+0000","last_unstale":"2024-04-25T13:48:11.634666+0000","last_undegraded":"2024-04-25T13:48:11.634666+0000","last_fullsized":"2024-04-25T13:48:11.634666+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:33.890097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634707+0000","last_change":"2024-04-25T13:48:09.547682+0000","last_active":"2024-04-25T13:48:11.634707+0000","last_peered":"2024-04-25T13:48:11.634707+0000","last_clean":"2024-04-25T13:48:11.634707+0000","last_became_active":"2024-04-25T13:48:09.538057+0000","last_became_peered":"2024-04-25T13:48:09.538057+0000","last_unstale":"2024-04-25T13:48:11.634707+0000","last_undegraded":"2024-04-25T13:48:11.634707+0000","last_fullsized":"2024-04-25T13:48:11.634707+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:09.226507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634000+0000","last_change":"2024-04-25T13:48:10.594293+0000","last_active":"2024-04-25T13:48:11.634000+0000","last_peered":"2024-04-25T13:48:11.634000+0000","last_clean":"2024-04-25T13:48:11.634000+0000","last_became_active":"2024-04-25T13:48:10.586999+0000","last_became_peered":"2024-04-25T13:48:10.586999+0000","last_unstale":"2024-04-25T13:48:11.634000+0000","last_undegraded":"2024-04-25T13:48:11.634000+0000","last_fullsized":"2024-04-25T13:48:11.634000+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:42.566295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635956+0000","last_change":"2024-04-25T13:48:09.545932+0000","last_active":"2024-04-25T13:48:11.635956+0000","last_peered":"2024-04-25T13:48:11.635956+0000","last_clean":"2024-04-25T13:48:11.635956+0000","last_became_active":"2024-04-25T13:48:09.545690+0000","last_became_peered":"2024-04-25T13:48:09.545690+0000","last_unstale":"2024-04-25T13:48:11.635956+0000","last_undegraded":"2024-04-25T13:48:11.635956+0000","last_fullsized":"2024-04-25T13:48:11.635956+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:08.797820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566972+0000","last_change":"2024-04-25T13:48:10.563040+0000","last_active":"2024-04-25T13:48:11.566972+0000","last_peered":"2024-04-25T13:48:11.566972+0000","last_clean":"2024-04-25T13:48:11.566972+0000","last_became_active":"2024-04-25T13:48:10.562138+0000","last_became_peered":"2024-04-25T13:48:10.562138+0000","last_unstale":"2024-04-25T13:48:11.566972+0000","last_undegraded":"2024-04-25T13:48:11.566972+0000","last_fullsized":"2024-04-25T13:48:11.566972+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:34.063336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"153'3","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633929+0000","last_change":"2024-04-25T13:48:09.545942+0000","last_active":"2024-04-25T13:48:11.633929+0000","last_peered":"2024-04-25T13:48:11.633929+0000","last_clean":"2024-04-25T13:48:11.633929+0000","last_became_active":"2024-04-25T13:48:09.542727+0000","last_became_peered":"2024-04-25T13:48:09.542727+0000","last_unstale":"2024-04-25T13:48:11.633929+0000","last_undegraded":"2024-04-25T13:48:11.633929+0000","last_fullsized":"2024-04-25T13:48:11.633929+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:42.229513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566625+0000","last_change":"2024-04-25T13:48:10.602665+0000","last_active":"2024-04-25T13:48:11.566625+0000","last_peered":"2024-04-25T13:48:11.566625+0000","last_clean":"2024-04-25T13:48:11.566625+0000","last_became_active":"2024-04-25T13:48:10.602062+0000","last_became_peered":"2024-04-25T13:48:10.602062+0000","last_unstale":"2024-04-25T13:48:11.566625+0000","last_undegraded":"2024-04-25T13:48:11.566625+0000","last_fullsized":"2024-04-25T13:48:11.566625+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:43.381559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"153'6","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633828+0000","last_change":"2024-04-25T13:48:09.545049+0000","last_active":"2024-04-25T13:48:11.633828+0000","last_peered":"2024-04-25T13:48:11.633828+0000","last_clean":"2024-04-25T13:48:11.633828+0000","last_became_active":"2024-04-25T13:48:09.544302+0000","last_became_peered":"2024-04-25T13:48:09.544302+0000","last_unstale":"2024-04-25T13:48:11.633828+0000","last_undegraded":"2024-04-25T13:48:11.633828+0000","last_fullsized":"2024-04-25T13:48:11.633828+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:56.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637349+0000","last_change":"2024-04-25T13:48:10.576425+0000","last_active":"2024-04-25T13:48:11.637349+0000","last_peered":"2024-04-25T13:48:11.637349+0000","last_clean":"2024-04-25T13:48:11.637349+0000","last_became_active":"2024-04-25T13:48:10.575950+0000","last_became_peered":"2024-04-25T13:48:10.575950+0000","last_unstale":"2024-04-25T13:48:11.637349+0000","last_undegraded":"2024-04-25T13:48:11.637349+0000","last_fullsized":"2024-04-25T13:48:11.637349+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:10.352567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635457+0000","last_change":"2024-04-25T13:48:09.548416+0000","last_active":"2024-04-25T13:48:11.635457+0000","last_peered":"2024-04-25T13:48:11.635457+0000","last_clean":"2024-04-25T13:48:11.635457+0000","last_became_active":"2024-04-25T13:48:09.548017+0000","last_became_peered":"2024-04-25T13:48:09.548017+0000","last_unstale":"2024-04-25T13:48:11.635457+0000","last_undegraded":"2024-04-25T13:48:11.635457+0000","last_fullsized":"2024-04-25T13:48:11.635457+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:47.964220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635916+0000","last_change":"2024-04-25T13:48:10.608112+0000","last_active":"2024-04-25T13:48:11.635916+0000","last_peered":"2024-04-25T13:48:11.635916+0000","last_clean":"2024-04-25T13:48:11.635916+0000","last_became_active":"2024-04-25T13:48:10.607893+0000","last_became_peered":"2024-04-25T13:48:10.607893+0000","last_unstale":"2024-04-25T13:48:11.635916+0000","last_undegraded":"2024-04-25T13:48:11.635916+0000","last_fullsized":"2024-04-25T13:48:11.635916+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:15.268203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638531+0000","last_change":"2024-04-25T13:48:09.538976+0000","last_active":"2024-04-25T13:48:11.638531+0000","last_peered":"2024-04-25T13:48:11.638531+0000","last_clean":"2024-04-25T13:48:11.638531+0000","last_became_active":"2024-04-25T13:48:09.538683+0000","last_became_peered":"2024-04-25T13:48:09.538683+0000","last_unstale":"2024-04-25T13:48:11.638531+0000","last_undegraded":"2024-04-25T13:48:11.638531+0000","last_fullsized":"2024-04-25T13:48:11.638531+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:58.327116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634519+0000","last_change":"2024-04-25T13:48:10.595502+0000","last_active":"2024-04-25T13:48:11.634519+0000","last_peered":"2024-04-25T13:48:11.634519+0000","last_clean":"2024-04-25T13:48:11.634519+0000","last_became_active":"2024-04-25T13:48:10.593965+0000","last_became_peered":"2024-04-25T13:48:10.593965+0000","last_unstale":"2024-04-25T13:48:11.634519+0000","last_undegraded":"2024-04-25T13:48:11.634519+0000","last_fullsized":"2024-04-25T13:48:11.634519+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:40.830996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"153'8","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636683+0000","last_change":"2024-04-25T13:48:09.541558+0000","last_active":"2024-04-25T13:48:11.636683+0000","last_peered":"2024-04-25T13:48:11.636683+0000","last_clean":"2024-04-25T13:48:11.636683+0000","last_became_active":"2024-04-25T13:48:09.540279+0000","last_became_peered":"2024-04-25T13:48:09.540279+0000","last_unstale":"2024-04-25T13:48:11.636683+0000","last_undegraded":"2024-04-25T13:48:11.636683+0000","last_fullsized":"2024-04-25T13:48:11.636683+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:08.749125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634170+0000","last_change":"2024-04-25T13:48:10.604585+0000","last_active":"2024-04-25T13:48:11.634170+0000","last_peered":"2024-04-25T13:48:11.634170+0000","last_clean":"2024-04-25T13:48:11.634170+0000","last_became_active":"2024-04-25T13:48:10.592540+0000","last_became_peered":"2024-04-25T13:48:10.592540+0000","last_unstale":"2024-04-25T13:48:11.634170+0000","last_undegraded":"2024-04-25T13:48:11.634170+0000","last_fullsized":"2024-04-25T13:48:11.634170+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:58.744133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634362+0000","last_change":"2024-04-25T13:48:09.543768+0000","last_active":"2024-04-25T13:48:11.634362+0000","last_peered":"2024-04-25T13:48:11.634362+0000","last_clean":"2024-04-25T13:48:11.634362+0000","last_became_active":"2024-04-25T13:48:09.541684+0000","last_became_peered":"2024-04-25T13:48:09.541684+0000","last_unstale":"2024-04-25T13:48:11.634362+0000","last_undegraded":"2024-04-25T13:48:11.634362+0000","last_fullsized":"2024-04-25T13:48:11.634362+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:18.000963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633681+0000","last_change":"2024-04-25T13:48:10.593401+0000","last_active":"2024-04-25T13:48:11.633681+0000","last_peered":"2024-04-25T13:48:11.633681+0000","last_clean":"2024-04-25T13:48:11.633681+0000","last_became_active":"2024-04-25T13:48:10.588064+0000","last_became_peered":"2024-04-25T13:48:10.588064+0000","last_unstale":"2024-04-25T13:48:11.633681+0000","last_undegraded":"2024-04-25T13:48:11.633681+0000","last_fullsized":"2024-04-25T13:48:11.633681+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:05.633467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558722+0000","last_change":"2024-04-25T13:48:09.537807+0000","last_active":"2024-04-25T13:48:11.558722+0000","last_peered":"2024-04-25T13:48:11.558722+0000","last_clean":"2024-04-25T13:48:11.558722+0000","last_became_active":"2024-04-25T13:48:09.534705+0000","last_became_peered":"2024-04-25T13:48:09.534705+0000","last_unstale":"2024-04-25T13:48:11.558722+0000","last_undegraded":"2024-04-25T13:48:11.558722+0000","last_fullsized":"2024-04-25T13:48:11.558722+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:25.489483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635849+0000","last_change":"2024-04-25T13:48:10.574406+0000","last_active":"2024-04-25T13:48:11.635849+0000","last_peered":"2024-04-25T13:48:11.635849+0000","last_clean":"2024-04-25T13:48:11.635849+0000","last_became_active":"2024-04-25T13:48:10.574094+0000","last_became_peered":"2024-04-25T13:48:10.574094+0000","last_unstale":"2024-04-25T13:48:11.635849+0000","last_undegraded":"2024-04-25T13:48:11.635849+0000","last_fullsized":"2024-04-25T13:48:11.635849+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:11.426615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561747+0000","last_change":"2024-04-25T13:48:09.545837+0000","last_active":"2024-04-25T13:48:11.561747+0000","last_peered":"2024-04-25T13:48:11.561747+0000","last_clean":"2024-04-25T13:48:11.561747+0000","last_became_active":"2024-04-25T13:48:09.543847+0000","last_became_peered":"2024-04-25T13:48:09.543847+0000","last_unstale":"2024-04-25T13:48:11.561747+0000","last_undegraded":"2024-04-25T13:48:11.561747+0000","last_fullsized":"2024-04-25T13:48:11.561747+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:56.932162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636998+0000","last_change":"2024-04-25T13:48:10.568663+0000","last_active":"2024-04-25T13:48:11.636998+0000","last_peered":"2024-04-25T13:48:11.636998+0000","last_clean":"2024-04-25T13:48:11.636998+0000","last_became_active":"2024-04-25T13:48:10.568148+0000","last_became_peered":"2024-04-25T13:48:10.568148+0000","last_unstale":"2024-04-25T13:48:11.636998+0000","last_undegraded":"2024-04-25T13:48:11.636998+0000","last_fullsized":"2024-04-25T13:48:11.636998+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:26.118303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635192+0000","last_change":"2024-04-25T13:48:09.535686+0000","last_active":"2024-04-25T13:48:11.635192+0000","last_peered":"2024-04-25T13:48:11.635192+0000","last_clean":"2024-04-25T13:48:11.635192+0000","last_became_active":"2024-04-25T13:48:09.534896+0000","last_became_peered":"2024-04-25T13:48:09.534896+0000","last_unstale":"2024-04-25T13:48:11.635192+0000","last_undegraded":"2024-04-25T13:48:11.635192+0000","last_fullsized":"2024-04-25T13:48:11.635192+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:05.690492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635259+0000","last_change":"2024-04-25T13:48:10.598218+0000","last_active":"2024-04-25T13:48:11.635259+0000","last_peered":"2024-04-25T13:48:11.635259+0000","last_clean":"2024-04-25T13:48:11.635259+0000","last_became_active":"2024-04-25T13:48:10.598011+0000","last_became_peered":"2024-04-25T13:48:10.598011+0000","last_unstale":"2024-04-25T13:48:11.635259+0000","last_undegraded":"2024-04-25T13:48:11.635259+0000","last_fullsized":"2024-04-25T13:48:11.635259+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:43.907565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634259+0000","last_change":"2024-04-25T13:48:09.545084+0000","last_active":"2024-04-25T13:48:11.634259+0000","last_peered":"2024-04-25T13:48:11.634259+0000","last_clean":"2024-04-25T13:48:11.634259+0000","last_became_active":"2024-04-25T13:48:09.541104+0000","last_became_peered":"2024-04-25T13:48:09.541104+0000","last_unstale":"2024-04-25T13:48:11.634259+0000","last_undegraded":"2024-04-25T13:48:11.634259+0000","last_fullsized":"2024-04-25T13:48:11.634259+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:36.917263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561184+0000","last_change":"2024-04-25T13:48:10.607422+0000","last_active":"2024-04-25T13:48:11.561184+0000","last_peered":"2024-04-25T13:48:11.561184+0000","last_clean":"2024-04-25T13:48:11.561184+0000","last_became_active":"2024-04-25T13:48:10.606733+0000","last_became_peered":"2024-04-25T13:48:10.606733+0000","last_unstale":"2024-04-25T13:48:11.561184+0000","last_undegraded":"2024-04-25T13:48:11.561184+0000","last_fullsized":"2024-04-25T13:48:11.561184+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:44.705944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636023+0000","last_change":"2024-04-25T13:48:09.546240+0000","last_active":"2024-04-25T13:48:11.636023+0000","last_peered":"2024-04-25T13:48:11.636023+0000","last_clean":"2024-04-25T13:48:11.636023+0000","last_became_active":"2024-04-25T13:48:09.545915+0000","last_became_peered":"2024-04-25T13:48:09.545915+0000","last_unstale":"2024-04-25T13:48:11.636023+0000","last_undegraded":"2024-04-25T13:48:11.636023+0000","last_fullsized":"2024-04-25T13:48:11.636023+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.648505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636215+0000","last_change":"2024-04-25T13:48:10.568452+0000","last_active":"2024-04-25T13:48:11.636215+0000","last_peered":"2024-04-25T13:48:11.636215+0000","last_clean":"2024-04-25T13:48:11.636215+0000","last_became_active":"2024-04-25T13:48:10.567753+0000","last_became_peered":"2024-04-25T13:48:10.567753+0000","last_unstale":"2024-04-25T13:48:11.636215+0000","last_undegraded":"2024-04-25T13:48:11.636215+0000","last_fullsized":"2024-04-25T13:48:11.636215+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:38.717680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637583+0000","last_change":"2024-04-25T13:48:09.536206+0000","last_active":"2024-04-25T13:48:11.637583+0000","last_peered":"2024-04-25T13:48:11.637583+0000","last_clean":"2024-04-25T13:48:11.637583+0000","last_became_active":"2024-04-25T13:48:09.536013+0000","last_became_peered":"2024-04-25T13:48:09.536013+0000","last_unstale":"2024-04-25T13:48:11.637583+0000","last_undegraded":"2024-04-25T13:48:11.637583+0000","last_fullsized":"2024-04-25T13:48:11.637583+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:56.362942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566946+0000","last_change":"2024-04-25T13:48:10.603426+0000","last_active":"2024-04-25T13:48:11.566946+0000","last_peered":"2024-04-25T13:48:11.566946+0000","last_clean":"2024-04-25T13:48:11.566946+0000","last_became_active":"2024-04-25T13:48:10.585576+0000","last_became_peered":"2024-04-25T13:48:10.585576+0000","last_unstale":"2024-04-25T13:48:11.566946+0000","last_undegraded":"2024-04-25T13:48:11.566946+0000","last_fullsized":"2024-04-25T13:48:11.566946+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:03.126219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558942+0000","last_change":"2024-04-25T13:48:09.532911+0000","last_active":"2024-04-25T13:48:11.558942+0000","last_peered":"2024-04-25T13:48:11.558942+0000","last_clean":"2024-04-25T13:48:11.558942+0000","last_became_active":"2024-04-25T13:48:09.532026+0000","last_became_peered":"2024-04-25T13:48:09.532026+0000","last_unstale":"2024-04-25T13:48:11.558942+0000","last_undegraded":"2024-04-25T13:48:11.558942+0000","last_fullsized":"2024-04-25T13:48:11.558942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:42.439991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561652+0000","last_change":"2024-04-25T13:48:10.592701+0000","last_active":"2024-04-25T13:48:11.561652+0000","last_peered":"2024-04-25T13:48:11.561652+0000","last_clean":"2024-04-25T13:48:11.561652+0000","last_became_active":"2024-04-25T13:48:10.591813+0000","last_became_peered":"2024-04-25T13:48:10.591813+0000","last_unstale":"2024-04-25T13:48:11.561652+0000","last_undegraded":"2024-04-25T13:48:11.561652+0000","last_fullsized":"2024-04-25T13:48:11.561652+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:02.788870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635370+0000","last_change":"2024-04-25T13:48:09.548276+0000","last_active":"2024-04-25T13:48:11.635370+0000","last_peered":"2024-04-25T13:48:11.635370+0000","last_clean":"2024-04-25T13:48:11.635370+0000","last_became_active":"2024-04-25T13:48:09.548096+0000","last_became_peered":"2024-04-25T13:48:09.548096+0000","last_unstale":"2024-04-25T13:48:11.635370+0000","last_undegraded":"2024-04-25T13:48:11.635370+0000","last_fullsized":"2024-04-25T13:48:11.635370+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:03.563156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635073+0000","last_change":"2024-04-25T13:48:10.577700+0000","last_active":"2024-04-25T13:48:11.635073+0000","last_peered":"2024-04-25T13:48:11.635073+0000","last_clean":"2024-04-25T13:48:11.635073+0000","last_became_active":"2024-04-25T13:48:10.577164+0000","last_became_peered":"2024-04-25T13:48:10.577164+0000","last_unstale":"2024-04-25T13:48:11.635073+0000","last_undegraded":"2024-04-25T13:48:11.635073+0000","last_fullsized":"2024-04-25T13:48:11.635073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:36.951133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"154'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.568908+0000","last_change":"2024-04-25T13:48:09.550250+0000","last_active":"2024-04-25T13:48:11.568908+0000","last_peered":"2024-04-25T13:48:11.568908+0000","last_clean":"2024-04-25T13:48:11.568908+0000","last_became_active":"2024-04-25T13:48:09.548228+0000","last_became_peered":"2024-04-25T13:48:09.548228+0000","last_unstale":"2024-04-25T13:48:11.568908+0000","last_undegraded":"2024-04-25T13:48:11.568908+0000","last_fullsized":"2024-04-25T13:48:11.568908+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:57.684938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638915+0000","last_change":"2024-04-25T13:48:10.592335+0000","last_active":"2024-04-25T13:48:11.638915+0000","last_peered":"2024-04-25T13:48:11.638915+0000","last_clean":"2024-04-25T13:48:11.638915+0000","last_became_active":"2024-04-25T13:48:10.591980+0000","last_became_peered":"2024-04-25T13:48:10.591980+0000","last_unstale":"2024-04-25T13:48:11.638915+0000","last_undegraded":"2024-04-25T13:48:11.638915+0000","last_fullsized":"2024-04-25T13:48:11.638915+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:44.091724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636092+0000","last_change":"2024-04-25T13:48:09.532976+0000","last_active":"2024-04-25T13:48:11.636092+0000","last_peered":"2024-04-25T13:48:11.636092+0000","last_clean":"2024-04-25T13:48:11.636092+0000","last_became_active":"2024-04-25T13:48:09.532729+0000","last_became_peered":"2024-04-25T13:48:09.532729+0000","last_unstale":"2024-04-25T13:48:11.636092+0000","last_undegraded":"2024-04-25T13:48:11.636092+0000","last_fullsized":"2024-04-25T13:48:11.636092+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.266519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.560749+0000","last_change":"2024-04-25T13:48:10.596857+0000","last_active":"2024-04-25T13:48:11.560749+0000","last_peered":"2024-04-25T13:48:11.560749+0000","last_clean":"2024-04-25T13:48:11.560749+0000","last_became_active":"2024-04-25T13:48:10.582230+0000","last_became_peered":"2024-04-25T13:48:10.582230+0000","last_unstale":"2024-04-25T13:48:11.560749+0000","last_undegraded":"2024-04-25T13:48:11.560749+0000","last_fullsized":"2024-04-25T13:48:11.560749+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:54.679392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.559006+0000","last_change":"2024-04-25T13:48:09.534257+0000","last_active":"2024-04-25T13:48:11.559006+0000","last_peered":"2024-04-25T13:48:11.559006+0000","last_clean":"2024-04-25T13:48:11.559006+0000","last_became_active":"2024-04-25T13:48:09.530230+0000","last_became_peered":"2024-04-25T13:48:09.530230+0000","last_unstale":"2024-04-25T13:48:11.559006+0000","last_undegraded":"2024-04-25T13:48:11.559006+0000","last_fullsized":"2024-04-25T13:48:11.559006+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:14.040233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566389+0000","last_change":"2024-04-25T13:48:10.598975+0000","last_active":"2024-04-25T13:48:11.566389+0000","last_peered":"2024-04-25T13:48:11.566389+0000","last_clean":"2024-04-25T13:48:11.566389+0000","last_became_active":"2024-04-25T13:48:10.598012+0000","last_became_peered":"2024-04-25T13:48:10.598012+0000","last_unstale":"2024-04-25T13:48:11.566389+0000","last_undegraded":"2024-04-25T13:48:11.566389+0000","last_fullsized":"2024-04-25T13:48:11.566389+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:38.253502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637272+0000","last_change":"2024-04-25T13:48:09.539642+0000","last_active":"2024-04-25T13:48:11.637272+0000","last_peered":"2024-04-25T13:48:11.637272+0000","last_clean":"2024-04-25T13:48:11.637272+0000","last_became_active":"2024-04-25T13:48:09.539095+0000","last_became_peered":"2024-04-25T13:48:09.539095+0000","last_unstale":"2024-04-25T13:48:11.637272+0000","last_undegraded":"2024-04-25T13:48:11.637272+0000","last_fullsized":"2024-04-25T13:48:11.637272+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:57.268234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634407+0000","last_change":"2024-04-25T13:48:10.587919+0000","last_active":"2024-04-25T13:48:11.634407+0000","last_peered":"2024-04-25T13:48:11.634407+0000","last_clean":"2024-04-25T13:48:11.634407+0000","last_became_active":"2024-04-25T13:48:10.586169+0000","last_became_peered":"2024-04-25T13:48:10.586169+0000","last_unstale":"2024-04-25T13:48:11.634407+0000","last_undegraded":"2024-04-25T13:48:11.634407+0000","last_fullsized":"2024-04-25T13:48:11.634407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:48.401823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566845+0000","last_change":"2024-04-25T13:48:09.535810+0000","last_active":"2024-04-25T13:48:11.566845+0000","last_peered":"2024-04-25T13:48:11.566845+0000","last_clean":"2024-04-25T13:48:11.566845+0000","last_became_active":"2024-04-25T13:48:09.534927+0000","last_became_peered":"2024-04-25T13:48:09.534927+0000","last_unstale":"2024-04-25T13:48:11.566845+0000","last_undegraded":"2024-04-25T13:48:11.566845+0000","last_fullsized":"2024-04-25T13:48:11.566845+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:09.121478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635729+0000","last_change":"2024-04-25T13:48:10.572221+0000","last_active":"2024-04-25T13:48:11.635729+0000","last_peered":"2024-04-25T13:48:11.635729+0000","last_clean":"2024-04-25T13:48:11.635729+0000","last_became_active":"2024-04-25T13:48:10.571890+0000","last_became_peered":"2024-04-25T13:48:10.571890+0000","last_unstale":"2024-04-25T13:48:11.635729+0000","last_undegraded":"2024-04-25T13:48:11.635729+0000","last_fullsized":"2024-04-25T13:48:11.635729+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:36.359597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637363+0000","last_change":"2024-04-25T13:48:09.547060+0000","last_active":"2024-04-25T13:48:11.637363+0000","last_peered":"2024-04-25T13:48:11.637363+0000","last_clean":"2024-04-25T13:48:11.637363+0000","last_became_active":"2024-04-25T13:48:09.546321+0000","last_became_peered":"2024-04-25T13:48:09.546321+0000","last_unstale":"2024-04-25T13:48:11.637363+0000","last_undegraded":"2024-04-25T13:48:11.637363+0000","last_fullsized":"2024-04-25T13:48:11.637363+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:14.502478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636346+0000","last_change":"2024-04-25T13:48:10.574015+0000","last_active":"2024-04-25T13:48:11.636346+0000","last_peered":"2024-04-25T13:48:11.636346+0000","last_clean":"2024-04-25T13:48:11.636346+0000","last_became_active":"2024-04-25T13:48:10.573609+0000","last_became_peered":"2024-04-25T13:48:10.573609+0000","last_unstale":"2024-04-25T13:48:11.636346+0000","last_undegraded":"2024-04-25T13:48:11.636346+0000","last_fullsized":"2024-04-25T13:48:11.636346+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:38.142413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.559024+0000","last_change":"2024-04-25T13:48:09.539563+0000","last_active":"2024-04-25T13:48:11.559024+0000","last_peered":"2024-04-25T13:48:11.559024+0000","last_clean":"2024-04-25T13:48:11.559024+0000","last_became_active":"2024-04-25T13:48:09.539303+0000","last_became_peered":"2024-04-25T13:48:09.539303+0000","last_unstale":"2024-04-25T13:48:11.559024+0000","last_undegraded":"2024-04-25T13:48:11.559024+0000","last_fullsized":"2024-04-25T13:48:11.559024+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:35.043621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567417+0000","last_change":"2024-04-25T13:48:10.597719+0000","last_active":"2024-04-25T13:48:11.567417+0000","last_peered":"2024-04-25T13:48:11.567417+0000","last_clean":"2024-04-25T13:48:11.567417+0000","last_became_active":"2024-04-25T13:48:10.586892+0000","last_became_peered":"2024-04-25T13:48:10.586892+0000","last_unstale":"2024-04-25T13:48:11.567417+0000","last_undegraded":"2024-04-25T13:48:11.567417+0000","last_fullsized":"2024-04-25T13:48:11.567417+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:13.759569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567811+0000","last_change":"2024-04-25T13:48:09.542646+0000","last_active":"2024-04-25T13:48:11.567811+0000","last_peered":"2024-04-25T13:48:11.567811+0000","last_clean":"2024-04-25T13:48:11.567811+0000","last_became_active":"2024-04-25T13:48:09.541898+0000","last_became_peered":"2024-04-25T13:48:09.541898+0000","last_unstale":"2024-04-25T13:48:11.567811+0000","last_undegraded":"2024-04-25T13:48:11.567811+0000","last_fullsized":"2024-04-25T13:48:11.567811+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:14.687224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":374,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567133+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:48:11.567133+0000","last_peered":"2024-04-25T13:48:11.567133+0000","last_clean":"2024-04-25T13:48:11.567133+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:48:11.567133+0000","last_undegraded":"2024-04-25T13:48:11.567133+0000","last_fullsized":"2024-04-25T13:48:11.567133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1419,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193892,"statfs":{"total":200000000,"available":198546015,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":842,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194469,"statfs":{"total":200000000,"available":199137021,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194471,"statfs":{"total":200000000,"available":199138521,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194471,"statfs":{"total":200000000,"available":199138563,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194471,"statfs":{"total":200000000,"available":199138473,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194470,"statfs":{"total":200000000,"available":199138017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194471,"statfs":{"total":200000000,"available":199138633,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1417,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193894,"statfs":{"total":200000000,"available":198548265,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:48:14.754 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:48:14.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.906+0000 7fe4df321640 1 Processor -- start 2024-04-25T13:48:14.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.906+0000 7fe4df321640 1 -- start start 2024-04-25T13:48:14.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4df321640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe4d8161f90 0x7fe4d80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4df321640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4df321640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4d80d0d70 con 0x7fe4d80d0510 2024-04-25T13:48:14.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4df321640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe4d80d1050 con 0x7fe4d8161f90 2024-04-25T13:48:14.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4df321640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d806dca0 con 0x7fe4d815ec40 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd0bd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe4d8161f90 0x7fe4d80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd0bd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe4d8161f90 0x7fe4d80cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50576/0 (socket says 172.21.15.80:50576) 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd0bd640 1 -- 172.21.15.80:0/72453352 learned_addr learned my addr 172.21.15.80:0/72453352 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd8be640 1 --2- 172.21.15.80:0/72453352 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd0bd640 1 -- 172.21.15.80:0/72453352 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 msgr2=0x7fe4d815f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd0bd640 1 --2- 172.21.15.80:0/72453352 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d815f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd0bd640 1 -- 172.21.15.80:0/72453352 >> v1:172.21.15.87:6789/0 conn(0x7fe4d80d0510 legacy=0x7fe4d815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd0bd640 1 -- 172.21.15.80:0/72453352 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d806df80 con 0x7fe4d8161f90 2024-04-25T13:48:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.910+0000 7fe4dd8be640 1 --2- 172.21.15.80:0/72453352 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:14.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4dd0bd640 1 --2- 172.21.15.80:0/72453352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d8161f90 0x7fe4d80cffd0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009870 tx=0x7fe4d401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4b25f9ce29adf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4cffff640 1 -- 172.21.15.80:0/72453352 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40b1070 con 0x7fe4d8161f90 2024-04-25T13:48:14.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4cffff640 1 -- 172.21.15.80:0/72453352 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d40ac600 con 0x7fe4d8161f90 2024-04-25T13:48:14.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4cffff640 1 -- 172.21.15.80:0/72453352 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40ac9d0 con 0x7fe4d8161f90 2024-04-25T13:48:14.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 -- 172.21.15.80:0/72453352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d8161f90 msgr2=0x7fe4d80cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 --2- 172.21.15.80:0/72453352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d8161f90 0x7fe4d80cffd0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4009870 tx=0x7fe4d401c470 comp rx=0 tx=0).stop 2024-04-25T13:48:14.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 -- 172.21.15.80:0/72453352 shutdown_connections 2024-04-25T13:48:14.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 --2- 172.21.15.80:0/72453352 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d815f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 --2- 172.21.15.80:0/72453352 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d8161f90 0x7fe4d80cffd0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 -- 172.21.15.80:0/72453352 >> 172.21.15.80:0/72453352 conn(0x7fe4d806d080 msgr2=0x7fe4d80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:14.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 -- 172.21.15.80:0/72453352 shutdown_connections 2024-04-25T13:48:14.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 -- 172.21.15.80:0/72453352 wait complete. 2024-04-25T13:48:14.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 Processor -- start 2024-04-25T13:48:14.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.914+0000 7fe4df321640 1 -- start start 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d80d0510 0x7fe4d80ab670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d8147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4d8161f90 0x7fe4d81482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d806f4b0 con 0x7fe4d80d0510 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d806f630 con 0x7fe4d8161f90 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe4d80d0920 con 0x7fe4d815ec40 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d80d0510 0x7fe4d80ab670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d80d0510 0x7fe4d80ab670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50586/0 (socket says 172.21.15.80:50586) 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 -- 172.21.15.80:0/1297173635 learned_addr learned my addr 172.21.15.80:0/1297173635 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd8be640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4d8161f90 0x7fe4d81482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dc8bc640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d8147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 -- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 msgr2=0x7fe4d8147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d8147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 -- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4d8161f90 msgr2=0x7fe4d81482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4d8161f90 0x7fe4d81482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 -- 172.21.15.80:0/1297173635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d80abf50 con 0x7fe4d80d0510 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dd0bd640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d80d0510 0x7fe4d80ab670 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fe4d401a100 tx=0x7fe4d401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4cdffb640 1 -- 172.21.15.80:0/1297173635 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40b1040 con 0x7fe4d80d0510 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4cdffb640 1 -- 172.21.15.80:0/1297173635 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4d40bd070 con 0x7fe4d80d0510 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d80ac230 con 0x7fe4d80d0510 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4cdffb640 1 -- 172.21.15.80:0/1297173635 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40b7860 con 0x7fe4d80d0510 2024-04-25T13:48:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d80ac7f0 con 0x7fe4d80d0510 2024-04-25T13:48:14.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4d815d190 con 0x7fe4d80d0510 2024-04-25T13:48:14.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4cdffb640 1 -- 172.21.15.80:0/1297173635 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe4d40b7a00 con 0x7fe4d80d0510 2024-04-25T13:48:14.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4cdffb640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe4bc06f400 0x7fe4bc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:14.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4cdffb640 1 -- 172.21.15.80:0/1297173635 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fe4d412a0c0 con 0x7fe4d80d0510 2024-04-25T13:48:14.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.918+0000 7fe4dc8bc640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe4bc06f400 0x7fe4bc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:14.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.922+0000 7fe4dc8bc640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe4bc06f400 0x7fe4bc071910 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80b4760 tx=0x7fe4c8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:14.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:14.922+0000 7fe4cdffb640 1 -- 172.21.15.80:0/1297173635 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe4d40c8060 con 0x7fe4d80d0510 2024-04-25T13:48:15.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.078+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe4d806a3a0 con 0x7fe4bc06f400 2024-04-25T13:48:15.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.094+0000 7fe4cdffb640 1 -- 172.21.15.80:0/1297173635 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247981 (secure 0 0 0) 0x7fe4d806a3a0 con 0x7fe4bc06f400 2024-04-25T13:48:15.095 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:15.096 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:48:15.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe4bc06f400 msgr2=0x7fe4bc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe4bc06f400 0x7fe4bc071910 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fe4d80b4760 tx=0x7fe4c8009040 comp rx=0 tx=0).stop 2024-04-25T13:48:15.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d80d0510 msgr2=0x7fe4d80ab670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d80d0510 0x7fe4d80ab670 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7fe4d401a100 tx=0x7fe4d401ad10 comp rx=0 tx=0).stop 2024-04-25T13:48:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 shutdown_connections 2024-04-25T13:48:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe4bc06f400 0x7fe4bc071910 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4d8161f90 0x7fe4d81482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe4d815ec40 0x7fe4d8147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 --2- 172.21.15.80:0/1297173635 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe4d80d0510 0x7fe4d80ab670 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 >> 172.21.15.80:0/1297173635 conn(0x7fe4d806d080 msgr2=0x7fe4d815dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 shutdown_connections 2024-04-25T13:48:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.098+0000 7fe4df321640 1 -- 172.21.15.80:0/1297173635 wait complete. 2024-04-25T13:48:15.114 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-25T13:48:14.270265+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7887,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554612,"statfs":{"total":1600000000,"available":1591923508,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.758739"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635201+0000","last_change":"2024-04-25T13:48:10.592925+0000","last_active":"2024-04-25T13:48:11.635201+0000","last_peered":"2024-04-25T13:48:11.635201+0000","last_clean":"2024-04-25T13:48:11.635201+0000","last_became_active":"2024-04-25T13:48:10.592749+0000","last_became_peered":"2024-04-25T13:48:10.592749+0000","last_unstale":"2024-04-25T13:48:11.635201+0000","last_undegraded":"2024-04-25T13:48:11.635201+0000","last_fullsized":"2024-04-25T13:48:11.635201+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:31.807837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637960+0000","last_change":"2024-04-25T13:48:10.600062+0000","last_active":"2024-04-25T13:48:11.637960+0000","last_peered":"2024-04-25T13:48:11.637960+0000","last_clean":"2024-04-25T13:48:11.637960+0000","last_became_active":"2024-04-25T13:48:10.599776+0000","last_became_peered":"2024-04-25T13:48:10.599776+0000","last_unstale":"2024-04-25T13:48:11.637960+0000","last_undegraded":"2024-04-25T13:48:11.637960+0000","last_fullsized":"2024-04-25T13:48:11.637960+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:33.717096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634698+0000","last_change":"2024-04-25T13:48:10.608299+0000","last_active":"2024-04-25T13:48:11.634698+0000","last_peered":"2024-04-25T13:48:11.634698+0000","last_clean":"2024-04-25T13:48:11.634698+0000","last_became_active":"2024-04-25T13:48:10.604112+0000","last_became_peered":"2024-04-25T13:48:10.604112+0000","last_unstale":"2024-04-25T13:48:11.634698+0000","last_undegraded":"2024-04-25T13:48:11.634698+0000","last_fullsized":"2024-04-25T13:48:11.634698+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:49.048283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634136+0000","last_change":"2024-04-25T13:48:10.592632+0000","last_active":"2024-04-25T13:48:11.634136+0000","last_peered":"2024-04-25T13:48:11.634136+0000","last_clean":"2024-04-25T13:48:11.634136+0000","last_became_active":"2024-04-25T13:48:10.592343+0000","last_became_peered":"2024-04-25T13:48:10.592343+0000","last_unstale":"2024-04-25T13:48:11.634136+0000","last_undegraded":"2024-04-25T13:48:11.634136+0000","last_fullsized":"2024-04-25T13:48:11.634136+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:32.059799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567395+0000","last_change":"2024-04-25T13:48:10.578533+0000","last_active":"2024-04-25T13:48:11.567395+0000","last_peered":"2024-04-25T13:48:11.567395+0000","last_clean":"2024-04-25T13:48:11.567395+0000","last_became_active":"2024-04-25T13:48:10.576452+0000","last_became_peered":"2024-04-25T13:48:10.576452+0000","last_unstale":"2024-04-25T13:48:11.567395+0000","last_undegraded":"2024-04-25T13:48:11.567395+0000","last_fullsized":"2024-04-25T13:48:11.567395+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:53:58.914815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.560929+0000","last_change":"2024-04-25T13:48:10.581808+0000","last_active":"2024-04-25T13:48:11.560929+0000","last_peered":"2024-04-25T13:48:11.560929+0000","last_clean":"2024-04-25T13:48:11.560929+0000","last_became_active":"2024-04-25T13:48:10.580999+0000","last_became_peered":"2024-04-25T13:48:10.580999+0000","last_unstale":"2024-04-25T13:48:11.560929+0000","last_undegraded":"2024-04-25T13:48:11.560929+0000","last_fullsized":"2024-04-25T13:48:11.560929+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:38.170247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.560950+0000","last_change":"2024-04-25T13:48:10.605992+0000","last_active":"2024-04-25T13:48:11.560950+0000","last_peered":"2024-04-25T13:48:11.560950+0000","last_clean":"2024-04-25T13:48:11.560950+0000","last_became_active":"2024-04-25T13:48:10.603624+0000","last_became_peered":"2024-04-25T13:48:10.603624+0000","last_unstale":"2024-04-25T13:48:11.560950+0000","last_undegraded":"2024-04-25T13:48:11.560950+0000","last_fullsized":"2024-04-25T13:48:11.560950+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:15.179659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635166+0000","last_change":"2024-04-25T13:48:10.606153+0000","last_active":"2024-04-25T13:48:11.635166+0000","last_peered":"2024-04-25T13:48:11.635166+0000","last_clean":"2024-04-25T13:48:11.635166+0000","last_became_active":"2024-04-25T13:48:10.595457+0000","last_became_peered":"2024-04-25T13:48:10.595457+0000","last_unstale":"2024-04-25T13:48:11.635166+0000","last_undegraded":"2024-04-25T13:48:11.635166+0000","last_fullsized":"2024-04-25T13:48:11.635166+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:17.284240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633522+0000","last_change":"2024-04-25T13:48:10.609502+0000","last_active":"2024-04-25T13:48:11.633522+0000","last_peered":"2024-04-25T13:48:11.633522+0000","last_clean":"2024-04-25T13:48:11.633522+0000","last_became_active":"2024-04-25T13:48:10.607502+0000","last_became_peered":"2024-04-25T13:48:10.607502+0000","last_unstale":"2024-04-25T13:48:11.633522+0000","last_undegraded":"2024-04-25T13:48:11.633522+0000","last_fullsized":"2024-04-25T13:48:11.633522+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:51.404585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638158+0000","last_change":"2024-04-25T13:48:10.581304+0000","last_active":"2024-04-25T13:48:11.638158+0000","last_peered":"2024-04-25T13:48:11.638158+0000","last_clean":"2024-04-25T13:48:11.638158+0000","last_became_active":"2024-04-25T13:48:10.581043+0000","last_became_peered":"2024-04-25T13:48:10.581043+0000","last_unstale":"2024-04-25T13:48:11.638158+0000","last_undegraded":"2024-04-25T13:48:11.638158+0000","last_fullsized":"2024-04-25T13:48:11.638158+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:27.187343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558177+0000","last_change":"2024-04-25T13:48:10.568217+0000","last_active":"2024-04-25T13:48:11.558177+0000","last_peered":"2024-04-25T13:48:11.558177+0000","last_clean":"2024-04-25T13:48:11.558177+0000","last_became_active":"2024-04-25T13:48:10.567891+0000","last_became_peered":"2024-04-25T13:48:10.567891+0000","last_unstale":"2024-04-25T13:48:11.558177+0000","last_undegraded":"2024-04-25T13:48:11.558177+0000","last_fullsized":"2024-04-25T13:48:11.558177+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:43:38.896254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634083+0000","last_change":"2024-04-25T13:48:10.596081+0000","last_active":"2024-04-25T13:48:11.634083+0000","last_peered":"2024-04-25T13:48:11.634083+0000","last_clean":"2024-04-25T13:48:11.634083+0000","last_became_active":"2024-04-25T13:48:10.594744+0000","last_became_peered":"2024-04-25T13:48:10.594744+0000","last_unstale":"2024-04-25T13:48:11.634083+0000","last_undegraded":"2024-04-25T13:48:11.634083+0000","last_fullsized":"2024-04-25T13:48:11.634083+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:49.910198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634336+0000","last_change":"2024-04-25T13:48:10.581211+0000","last_active":"2024-04-25T13:48:11.634336+0000","last_peered":"2024-04-25T13:48:11.634336+0000","last_clean":"2024-04-25T13:48:11.634336+0000","last_became_active":"2024-04-25T13:48:10.580915+0000","last_became_peered":"2024-04-25T13:48:10.580915+0000","last_unstale":"2024-04-25T13:48:11.634336+0000","last_undegraded":"2024-04-25T13:48:11.634336+0000","last_fullsized":"2024-04-25T13:48:11.634336+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:43.025919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635311+0000","last_change":"2024-04-25T13:48:10.574585+0000","last_active":"2024-04-25T13:48:11.635311+0000","last_peered":"2024-04-25T13:48:11.635311+0000","last_clean":"2024-04-25T13:48:11.635311+0000","last_became_active":"2024-04-25T13:48:10.574286+0000","last_became_peered":"2024-04-25T13:48:10.574286+0000","last_unstale":"2024-04-25T13:48:11.635311+0000","last_undegraded":"2024-04-25T13:48:11.635311+0000","last_fullsized":"2024-04-25T13:48:11.635311+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:43.345750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636289+0000","last_change":"2024-04-25T13:48:10.608035+0000","last_active":"2024-04-25T13:48:11.636289+0000","last_peered":"2024-04-25T13:48:11.636289+0000","last_clean":"2024-04-25T13:48:11.636289+0000","last_became_active":"2024-04-25T13:48:10.607799+0000","last_became_peered":"2024-04-25T13:48:10.607799+0000","last_unstale":"2024-04-25T13:48:11.636289+0000","last_undegraded":"2024-04-25T13:48:11.636289+0000","last_fullsized":"2024-04-25T13:48:11.636289+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:03:13.009054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635771+0000","last_change":"2024-04-25T13:48:10.574696+0000","last_active":"2024-04-25T13:48:11.635771+0000","last_peered":"2024-04-25T13:48:11.635771+0000","last_clean":"2024-04-25T13:48:11.635771+0000","last_became_active":"2024-04-25T13:48:10.574100+0000","last_became_peered":"2024-04-25T13:48:10.574100+0000","last_unstale":"2024-04-25T13:48:11.635771+0000","last_undegraded":"2024-04-25T13:48:11.635771+0000","last_fullsized":"2024-04-25T13:48:11.635771+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:54:56.604752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561528+0000","last_change":"2024-04-25T13:48:10.604598+0000","last_active":"2024-04-25T13:48:11.561528+0000","last_peered":"2024-04-25T13:48:11.561528+0000","last_clean":"2024-04-25T13:48:11.561528+0000","last_became_active":"2024-04-25T13:48:10.595277+0000","last_became_peered":"2024-04-25T13:48:10.595277+0000","last_unstale":"2024-04-25T13:48:11.561528+0000","last_undegraded":"2024-04-25T13:48:11.561528+0000","last_fullsized":"2024-04-25T13:48:11.561528+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:23.351222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636319+0000","last_change":"2024-04-25T13:48:10.577753+0000","last_active":"2024-04-25T13:48:11.636319+0000","last_peered":"2024-04-25T13:48:11.636319+0000","last_clean":"2024-04-25T13:48:11.636319+0000","last_became_active":"2024-04-25T13:48:10.577130+0000","last_became_peered":"2024-04-25T13:48:10.577130+0000","last_unstale":"2024-04-25T13:48:11.636319+0000","last_undegraded":"2024-04-25T13:48:11.636319+0000","last_fullsized":"2024-04-25T13:48:11.636319+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:08.529838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558273+0000","last_change":"2024-04-25T13:48:10.584718+0000","last_active":"2024-04-25T13:48:11.558273+0000","last_peered":"2024-04-25T13:48:11.558273+0000","last_clean":"2024-04-25T13:48:11.558273+0000","last_became_active":"2024-04-25T13:48:10.584502+0000","last_became_peered":"2024-04-25T13:48:10.584502+0000","last_unstale":"2024-04-25T13:48:11.558273+0000","last_undegraded":"2024-04-25T13:48:11.558273+0000","last_fullsized":"2024-04-25T13:48:11.558273+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:23.345882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637310+0000","last_change":"2024-04-25T13:48:10.568056+0000","last_active":"2024-04-25T13:48:11.637310+0000","last_peered":"2024-04-25T13:48:11.637310+0000","last_clean":"2024-04-25T13:48:11.637310+0000","last_became_active":"2024-04-25T13:48:10.567784+0000","last_became_peered":"2024-04-25T13:48:10.567784+0000","last_unstale":"2024-04-25T13:48:11.637310+0000","last_undegraded":"2024-04-25T13:48:11.637310+0000","last_fullsized":"2024-04-25T13:48:11.637310+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:30:10.031661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.568040+0000","last_change":"2024-04-25T13:48:10.601480+0000","last_active":"2024-04-25T13:48:11.568040+0000","last_peered":"2024-04-25T13:48:11.568040+0000","last_clean":"2024-04-25T13:48:11.568040+0000","last_became_active":"2024-04-25T13:48:10.599829+0000","last_became_peered":"2024-04-25T13:48:10.599829+0000","last_unstale":"2024-04-25T13:48:11.568040+0000","last_undegraded":"2024-04-25T13:48:11.568040+0000","last_fullsized":"2024-04-25T13:48:11.568040+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:53.241219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.559103+0000","last_change":"2024-04-25T13:48:10.564202+0000","last_active":"2024-04-25T13:48:11.559103+0000","last_peered":"2024-04-25T13:48:11.559103+0000","last_clean":"2024-04-25T13:48:11.559103+0000","last_became_active":"2024-04-25T13:48:10.563907+0000","last_became_peered":"2024-04-25T13:48:10.563907+0000","last_unstale":"2024-04-25T13:48:11.559103+0000","last_undegraded":"2024-04-25T13:48:11.559103+0000","last_fullsized":"2024-04-25T13:48:11.559103+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:17.613120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636377+0000","last_change":"2024-04-25T13:48:10.609109+0000","last_active":"2024-04-25T13:48:11.636377+0000","last_peered":"2024-04-25T13:48:11.636377+0000","last_clean":"2024-04-25T13:48:11.636377+0000","last_became_active":"2024-04-25T13:48:10.605064+0000","last_became_peered":"2024-04-25T13:48:10.605064+0000","last_unstale":"2024-04-25T13:48:11.636377+0000","last_undegraded":"2024-04-25T13:48:11.636377+0000","last_fullsized":"2024-04-25T13:48:11.636377+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:18.015278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558748+0000","last_change":"2024-04-25T13:48:10.590782+0000","last_active":"2024-04-25T13:48:11.558748+0000","last_peered":"2024-04-25T13:48:11.558748+0000","last_clean":"2024-04-25T13:48:11.558748+0000","last_became_active":"2024-04-25T13:48:10.590523+0000","last_became_peered":"2024-04-25T13:48:10.590523+0000","last_unstale":"2024-04-25T13:48:11.558748+0000","last_undegraded":"2024-04-25T13:48:11.558748+0000","last_fullsized":"2024-04-25T13:48:11.558748+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:27.850740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638561+0000","last_change":"2024-04-25T13:48:10.572798+0000","last_active":"2024-04-25T13:48:11.638561+0000","last_peered":"2024-04-25T13:48:11.638561+0000","last_clean":"2024-04-25T13:48:11.638561+0000","last_became_active":"2024-04-25T13:48:10.572557+0000","last_became_peered":"2024-04-25T13:48:10.572557+0000","last_unstale":"2024-04-25T13:48:11.638561+0000","last_undegraded":"2024-04-25T13:48:11.638561+0000","last_fullsized":"2024-04-25T13:48:11.638561+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:04.512135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636444+0000","last_change":"2024-04-25T13:48:10.574409+0000","last_active":"2024-04-25T13:48:11.636444+0000","last_peered":"2024-04-25T13:48:11.636444+0000","last_clean":"2024-04-25T13:48:11.636444+0000","last_became_active":"2024-04-25T13:48:10.573716+0000","last_became_peered":"2024-04-25T13:48:10.573716+0000","last_unstale":"2024-04-25T13:48:11.636444+0000","last_undegraded":"2024-04-25T13:48:11.636444+0000","last_fullsized":"2024-04-25T13:48:11.636444+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:24.911925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633805+0000","last_change":"2024-04-25T13:48:10.600233+0000","last_active":"2024-04-25T13:48:11.633805+0000","last_peered":"2024-04-25T13:48:11.633805+0000","last_clean":"2024-04-25T13:48:11.633805+0000","last_became_active":"2024-04-25T13:48:10.600016+0000","last_became_peered":"2024-04-25T13:48:10.600016+0000","last_unstale":"2024-04-25T13:48:11.633805+0000","last_undegraded":"2024-04-25T13:48:11.633805+0000","last_fullsized":"2024-04-25T13:48:11.633805+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:13:17.409340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633626+0000","last_change":"2024-04-25T13:48:10.581556+0000","last_active":"2024-04-25T13:48:11.633626+0000","last_peered":"2024-04-25T13:48:11.633626+0000","last_clean":"2024-04-25T13:48:11.633626+0000","last_became_active":"2024-04-25T13:48:10.580516+0000","last_became_peered":"2024-04-25T13:48:10.580516+0000","last_unstale":"2024-04-25T13:48:11.633626+0000","last_undegraded":"2024-04-25T13:48:11.633626+0000","last_fullsized":"2024-04-25T13:48:11.633626+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:35.153374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634815+0000","last_change":"2024-04-25T13:48:10.610753+0000","last_active":"2024-04-25T13:48:11.634815+0000","last_peered":"2024-04-25T13:48:11.634815+0000","last_clean":"2024-04-25T13:48:11.634815+0000","last_became_active":"2024-04-25T13:48:10.610511+0000","last_became_peered":"2024-04-25T13:48:10.610511+0000","last_unstale":"2024-04-25T13:48:11.634815+0000","last_undegraded":"2024-04-25T13:48:11.634815+0000","last_fullsized":"2024-04-25T13:48:11.634815+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:56.209730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636720+0000","last_change":"2024-04-25T13:48:10.606471+0000","last_active":"2024-04-25T13:48:11.636720+0000","last_peered":"2024-04-25T13:48:11.636720+0000","last_clean":"2024-04-25T13:48:11.636720+0000","last_became_active":"2024-04-25T13:48:10.606304+0000","last_became_peered":"2024-04-25T13:48:10.606304+0000","last_unstale":"2024-04-25T13:48:11.636720+0000","last_undegraded":"2024-04-25T13:48:11.636720+0000","last_fullsized":"2024-04-25T13:48:11.636720+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:43:52.210208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566449+0000","last_change":"2024-04-25T13:48:10.599094+0000","last_active":"2024-04-25T13:48:11.566449+0000","last_peered":"2024-04-25T13:48:11.566449+0000","last_clean":"2024-04-25T13:48:11.566449+0000","last_became_active":"2024-04-25T13:48:10.587767+0000","last_became_peered":"2024-04-25T13:48:10.587767+0000","last_unstale":"2024-04-25T13:48:11.566449+0000","last_undegraded":"2024-04-25T13:48:11.566449+0000","last_fullsized":"2024-04-25T13:48:11.566449+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:24.565204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561252+0000","last_change":"2024-04-25T13:48:10.598425+0000","last_active":"2024-04-25T13:48:11.561252+0000","last_peered":"2024-04-25T13:48:11.561252+0000","last_clean":"2024-04-25T13:48:11.561252+0000","last_became_active":"2024-04-25T13:48:10.591615+0000","last_became_peered":"2024-04-25T13:48:10.591615+0000","last_unstale":"2024-04-25T13:48:11.561252+0000","last_undegraded":"2024-04-25T13:48:11.561252+0000","last_fullsized":"2024-04-25T13:48:11.561252+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:08.551456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634573+0000","last_change":"2024-04-25T13:48:10.591056+0000","last_active":"2024-04-25T13:48:11.634573+0000","last_peered":"2024-04-25T13:48:11.634573+0000","last_clean":"2024-04-25T13:48:11.634573+0000","last_became_active":"2024-04-25T13:48:10.590774+0000","last_became_peered":"2024-04-25T13:48:10.590774+0000","last_unstale":"2024-04-25T13:48:11.634573+0000","last_undegraded":"2024-04-25T13:48:11.634573+0000","last_fullsized":"2024-04-25T13:48:11.634573+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:34.007994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"153'4","reported_seq":18,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566927+0000","last_change":"2024-04-25T13:48:09.549395+0000","last_active":"2024-04-25T13:48:11.566927+0000","last_peered":"2024-04-25T13:48:11.566927+0000","last_clean":"2024-04-25T13:48:11.566927+0000","last_became_active":"2024-04-25T13:48:09.547446+0000","last_became_peered":"2024-04-25T13:48:09.547446+0000","last_unstale":"2024-04-25T13:48:11.566927+0000","last_undegraded":"2024-04-25T13:48:11.566927+0000","last_fullsized":"2024-04-25T13:48:11.566927+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:31.331278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635581+0000","last_change":"2024-04-25T13:48:10.607392+0000","last_active":"2024-04-25T13:48:11.635581+0000","last_peered":"2024-04-25T13:48:11.635581+0000","last_clean":"2024-04-25T13:48:11.635581+0000","last_became_active":"2024-04-25T13:48:10.596232+0000","last_became_peered":"2024-04-25T13:48:10.596232+0000","last_unstale":"2024-04-25T13:48:11.635581+0000","last_undegraded":"2024-04-25T13:48:11.635581+0000","last_fullsized":"2024-04-25T13:48:11.635581+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:08:55.747274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634026+0000","last_change":"2024-04-25T13:48:09.543046+0000","last_active":"2024-04-25T13:48:11.634026+0000","last_peered":"2024-04-25T13:48:11.634026+0000","last_clean":"2024-04-25T13:48:11.634026+0000","last_became_active":"2024-04-25T13:48:09.540531+0000","last_became_peered":"2024-04-25T13:48:09.540531+0000","last_unstale":"2024-04-25T13:48:11.634026+0000","last_undegraded":"2024-04-25T13:48:11.634026+0000","last_fullsized":"2024-04-25T13:48:11.634026+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:12.036299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566963+0000","last_change":"2024-04-25T13:48:10.602093+0000","last_active":"2024-04-25T13:48:11.566963+0000","last_peered":"2024-04-25T13:48:11.566963+0000","last_clean":"2024-04-25T13:48:11.566963+0000","last_became_active":"2024-04-25T13:48:10.600734+0000","last_became_peered":"2024-04-25T13:48:10.600734+0000","last_unstale":"2024-04-25T13:48:11.566963+0000","last_undegraded":"2024-04-25T13:48:11.566963+0000","last_fullsized":"2024-04-25T13:48:11.566963+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:34.960163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558376+0000","last_change":"2024-04-25T13:48:09.524381+0000","last_active":"2024-04-25T13:48:11.558376+0000","last_peered":"2024-04-25T13:48:11.558376+0000","last_clean":"2024-04-25T13:48:11.558376+0000","last_became_active":"2024-04-25T13:48:09.524087+0000","last_became_peered":"2024-04-25T13:48:09.524087+0000","last_unstale":"2024-04-25T13:48:11.558376+0000","last_undegraded":"2024-04-25T13:48:11.558376+0000","last_fullsized":"2024-04-25T13:48:11.558376+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:16.986135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567649+0000","last_change":"2024-04-25T13:48:10.573373+0000","last_active":"2024-04-25T13:48:11.567649+0000","last_peered":"2024-04-25T13:48:11.567649+0000","last_clean":"2024-04-25T13:48:11.567649+0000","last_became_active":"2024-04-25T13:48:10.572448+0000","last_became_peered":"2024-04-25T13:48:10.572448+0000","last_unstale":"2024-04-25T13:48:11.567649+0000","last_undegraded":"2024-04-25T13:48:11.567649+0000","last_fullsized":"2024-04-25T13:48:11.567649+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:51.528130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636532+0000","last_change":"2024-04-25T13:48:09.535052+0000","last_active":"2024-04-25T13:48:11.636532+0000","last_peered":"2024-04-25T13:48:11.636532+0000","last_clean":"2024-04-25T13:48:11.636532+0000","last_became_active":"2024-04-25T13:48:09.534432+0000","last_became_peered":"2024-04-25T13:48:09.534432+0000","last_unstale":"2024-04-25T13:48:11.636532+0000","last_undegraded":"2024-04-25T13:48:11.636532+0000","last_fullsized":"2024-04-25T13:48:11.636532+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:42:18.233215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636774+0000","last_change":"2024-04-25T13:48:10.577497+0000","last_active":"2024-04-25T13:48:11.636774+0000","last_peered":"2024-04-25T13:48:11.636774+0000","last_clean":"2024-04-25T13:48:11.636774+0000","last_became_active":"2024-04-25T13:48:10.576743+0000","last_became_peered":"2024-04-25T13:48:10.576743+0000","last_unstale":"2024-04-25T13:48:11.636774+0000","last_undegraded":"2024-04-25T13:48:11.636774+0000","last_fullsized":"2024-04-25T13:48:11.636774+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:27:21.517098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638256+0000","last_change":"2024-04-25T13:48:09.532353+0000","last_active":"2024-04-25T13:48:11.638256+0000","last_peered":"2024-04-25T13:48:11.638256+0000","last_clean":"2024-04-25T13:48:11.638256+0000","last_became_active":"2024-04-25T13:48:09.532088+0000","last_became_peered":"2024-04-25T13:48:09.532088+0000","last_unstale":"2024-04-25T13:48:11.638256+0000","last_undegraded":"2024-04-25T13:48:11.638256+0000","last_fullsized":"2024-04-25T13:48:11.638256+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:16:07.095781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561798+0000","last_change":"2024-04-25T13:48:10.600214+0000","last_active":"2024-04-25T13:48:11.561798+0000","last_peered":"2024-04-25T13:48:11.561798+0000","last_clean":"2024-04-25T13:48:11.561798+0000","last_became_active":"2024-04-25T13:48:10.592710+0000","last_became_peered":"2024-04-25T13:48:10.592710+0000","last_unstale":"2024-04-25T13:48:11.561798+0000","last_undegraded":"2024-04-25T13:48:11.561798+0000","last_fullsized":"2024-04-25T13:48:11.561798+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:09.910090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637082+0000","last_change":"2024-04-25T13:48:09.537201+0000","last_active":"2024-04-25T13:48:11.637082+0000","last_peered":"2024-04-25T13:48:11.637082+0000","last_clean":"2024-04-25T13:48:11.637082+0000","last_became_active":"2024-04-25T13:48:09.535209+0000","last_became_peered":"2024-04-25T13:48:09.535209+0000","last_unstale":"2024-04-25T13:48:11.637082+0000","last_undegraded":"2024-04-25T13:48:11.637082+0000","last_fullsized":"2024-04-25T13:48:11.637082+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:18.479251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633724+0000","last_change":"2024-04-25T13:48:10.605251+0000","last_active":"2024-04-25T13:48:11.633724+0000","last_peered":"2024-04-25T13:48:11.633724+0000","last_clean":"2024-04-25T13:48:11.633724+0000","last_became_active":"2024-04-25T13:48:10.603716+0000","last_became_peered":"2024-04-25T13:48:10.603716+0000","last_unstale":"2024-04-25T13:48:11.633724+0000","last_undegraded":"2024-04-25T13:48:11.633724+0000","last_fullsized":"2024-04-25T13:48:11.633724+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:38.580146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561304+0000","last_change":"2024-04-25T13:48:09.545812+0000","last_active":"2024-04-25T13:48:11.561304+0000","last_peered":"2024-04-25T13:48:11.561304+0000","last_clean":"2024-04-25T13:48:11.561304+0000","last_became_active":"2024-04-25T13:48:09.544969+0000","last_became_peered":"2024-04-25T13:48:09.544969+0000","last_unstale":"2024-04-25T13:48:11.561304+0000","last_undegraded":"2024-04-25T13:48:11.561304+0000","last_fullsized":"2024-04-25T13:48:11.561304+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:27.155907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635792+0000","last_change":"2024-04-25T13:48:10.564110+0000","last_active":"2024-04-25T13:48:11.635792+0000","last_peered":"2024-04-25T13:48:11.635792+0000","last_clean":"2024-04-25T13:48:11.635792+0000","last_became_active":"2024-04-25T13:48:10.563794+0000","last_became_peered":"2024-04-25T13:48:10.563794+0000","last_unstale":"2024-04-25T13:48:11.635792+0000","last_undegraded":"2024-04-25T13:48:11.635792+0000","last_fullsized":"2024-04-25T13:48:11.635792+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:32.439930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.639063+0000","last_change":"2024-04-25T13:48:09.532274+0000","last_active":"2024-04-25T13:48:11.639063+0000","last_peered":"2024-04-25T13:48:11.639063+0000","last_clean":"2024-04-25T13:48:11.639063+0000","last_became_active":"2024-04-25T13:48:09.532033+0000","last_became_peered":"2024-04-25T13:48:09.532033+0000","last_unstale":"2024-04-25T13:48:11.639063+0000","last_undegraded":"2024-04-25T13:48:11.639063+0000","last_fullsized":"2024-04-25T13:48:11.639063+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:56.919040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634666+0000","last_change":"2024-04-25T13:48:10.577914+0000","last_active":"2024-04-25T13:48:11.634666+0000","last_peered":"2024-04-25T13:48:11.634666+0000","last_clean":"2024-04-25T13:48:11.634666+0000","last_became_active":"2024-04-25T13:48:10.577481+0000","last_became_peered":"2024-04-25T13:48:10.577481+0000","last_unstale":"2024-04-25T13:48:11.634666+0000","last_undegraded":"2024-04-25T13:48:11.634666+0000","last_fullsized":"2024-04-25T13:48:11.634666+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:33:33.890097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634707+0000","last_change":"2024-04-25T13:48:09.547682+0000","last_active":"2024-04-25T13:48:11.634707+0000","last_peered":"2024-04-25T13:48:11.634707+0000","last_clean":"2024-04-25T13:48:11.634707+0000","last_became_active":"2024-04-25T13:48:09.538057+0000","last_became_peered":"2024-04-25T13:48:09.538057+0000","last_unstale":"2024-04-25T13:48:11.634707+0000","last_undegraded":"2024-04-25T13:48:11.634707+0000","last_fullsized":"2024-04-25T13:48:11.634707+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:09.226507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634000+0000","last_change":"2024-04-25T13:48:10.594293+0000","last_active":"2024-04-25T13:48:11.634000+0000","last_peered":"2024-04-25T13:48:11.634000+0000","last_clean":"2024-04-25T13:48:11.634000+0000","last_became_active":"2024-04-25T13:48:10.586999+0000","last_became_peered":"2024-04-25T13:48:10.586999+0000","last_unstale":"2024-04-25T13:48:11.634000+0000","last_undegraded":"2024-04-25T13:48:11.634000+0000","last_fullsized":"2024-04-25T13:48:11.634000+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:42.566295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635956+0000","last_change":"2024-04-25T13:48:09.545932+0000","last_active":"2024-04-25T13:48:11.635956+0000","last_peered":"2024-04-25T13:48:11.635956+0000","last_clean":"2024-04-25T13:48:11.635956+0000","last_became_active":"2024-04-25T13:48:09.545690+0000","last_became_peered":"2024-04-25T13:48:09.545690+0000","last_unstale":"2024-04-25T13:48:11.635956+0000","last_undegraded":"2024-04-25T13:48:11.635956+0000","last_fullsized":"2024-04-25T13:48:11.635956+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:52:08.797820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566972+0000","last_change":"2024-04-25T13:48:10.563040+0000","last_active":"2024-04-25T13:48:11.566972+0000","last_peered":"2024-04-25T13:48:11.566972+0000","last_clean":"2024-04-25T13:48:11.566972+0000","last_became_active":"2024-04-25T13:48:10.562138+0000","last_became_peered":"2024-04-25T13:48:10.562138+0000","last_unstale":"2024-04-25T13:48:11.566972+0000","last_undegraded":"2024-04-25T13:48:11.566972+0000","last_fullsized":"2024-04-25T13:48:11.566972+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:41:34.063336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"153'3","reported_seq":17,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633929+0000","last_change":"2024-04-25T13:48:09.545942+0000","last_active":"2024-04-25T13:48:11.633929+0000","last_peered":"2024-04-25T13:48:11.633929+0000","last_clean":"2024-04-25T13:48:11.633929+0000","last_became_active":"2024-04-25T13:48:09.542727+0000","last_became_peered":"2024-04-25T13:48:09.542727+0000","last_unstale":"2024-04-25T13:48:11.633929+0000","last_undegraded":"2024-04-25T13:48:11.633929+0000","last_fullsized":"2024-04-25T13:48:11.633929+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:42.229513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566625+0000","last_change":"2024-04-25T13:48:10.602665+0000","last_active":"2024-04-25T13:48:11.566625+0000","last_peered":"2024-04-25T13:48:11.566625+0000","last_clean":"2024-04-25T13:48:11.566625+0000","last_became_active":"2024-04-25T13:48:10.602062+0000","last_became_peered":"2024-04-25T13:48:10.602062+0000","last_unstale":"2024-04-25T13:48:11.566625+0000","last_undegraded":"2024-04-25T13:48:11.566625+0000","last_fullsized":"2024-04-25T13:48:11.566625+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:43.381559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"153'6","reported_seq":20,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633828+0000","last_change":"2024-04-25T13:48:09.545049+0000","last_active":"2024-04-25T13:48:11.633828+0000","last_peered":"2024-04-25T13:48:11.633828+0000","last_clean":"2024-04-25T13:48:11.633828+0000","last_became_active":"2024-04-25T13:48:09.544302+0000","last_became_peered":"2024-04-25T13:48:09.544302+0000","last_unstale":"2024-04-25T13:48:11.633828+0000","last_undegraded":"2024-04-25T13:48:11.633828+0000","last_fullsized":"2024-04-25T13:48:11.633828+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:23:56.653254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637349+0000","last_change":"2024-04-25T13:48:10.576425+0000","last_active":"2024-04-25T13:48:11.637349+0000","last_peered":"2024-04-25T13:48:11.637349+0000","last_clean":"2024-04-25T13:48:11.637349+0000","last_became_active":"2024-04-25T13:48:10.575950+0000","last_became_peered":"2024-04-25T13:48:10.575950+0000","last_unstale":"2024-04-25T13:48:11.637349+0000","last_undegraded":"2024-04-25T13:48:11.637349+0000","last_fullsized":"2024-04-25T13:48:11.637349+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:10.352567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635457+0000","last_change":"2024-04-25T13:48:09.548416+0000","last_active":"2024-04-25T13:48:11.635457+0000","last_peered":"2024-04-25T13:48:11.635457+0000","last_clean":"2024-04-25T13:48:11.635457+0000","last_became_active":"2024-04-25T13:48:09.548017+0000","last_became_peered":"2024-04-25T13:48:09.548017+0000","last_unstale":"2024-04-25T13:48:11.635457+0000","last_undegraded":"2024-04-25T13:48:11.635457+0000","last_fullsized":"2024-04-25T13:48:11.635457+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:45:47.964220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635916+0000","last_change":"2024-04-25T13:48:10.608112+0000","last_active":"2024-04-25T13:48:11.635916+0000","last_peered":"2024-04-25T13:48:11.635916+0000","last_clean":"2024-04-25T13:48:11.635916+0000","last_became_active":"2024-04-25T13:48:10.607893+0000","last_became_peered":"2024-04-25T13:48:10.607893+0000","last_unstale":"2024-04-25T13:48:11.635916+0000","last_undegraded":"2024-04-25T13:48:11.635916+0000","last_fullsized":"2024-04-25T13:48:11.635916+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:27:15.268203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638531+0000","last_change":"2024-04-25T13:48:09.538976+0000","last_active":"2024-04-25T13:48:11.638531+0000","last_peered":"2024-04-25T13:48:11.638531+0000","last_clean":"2024-04-25T13:48:11.638531+0000","last_became_active":"2024-04-25T13:48:09.538683+0000","last_became_peered":"2024-04-25T13:48:09.538683+0000","last_unstale":"2024-04-25T13:48:11.638531+0000","last_undegraded":"2024-04-25T13:48:11.638531+0000","last_fullsized":"2024-04-25T13:48:11.638531+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:45:58.327116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634519+0000","last_change":"2024-04-25T13:48:10.595502+0000","last_active":"2024-04-25T13:48:11.634519+0000","last_peered":"2024-04-25T13:48:11.634519+0000","last_clean":"2024-04-25T13:48:11.634519+0000","last_became_active":"2024-04-25T13:48:10.593965+0000","last_became_peered":"2024-04-25T13:48:10.593965+0000","last_unstale":"2024-04-25T13:48:11.634519+0000","last_undegraded":"2024-04-25T13:48:11.634519+0000","last_fullsized":"2024-04-25T13:48:11.634519+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:40.830996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"153'8","reported_seq":22,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636683+0000","last_change":"2024-04-25T13:48:09.541558+0000","last_active":"2024-04-25T13:48:11.636683+0000","last_peered":"2024-04-25T13:48:11.636683+0000","last_clean":"2024-04-25T13:48:11.636683+0000","last_became_active":"2024-04-25T13:48:09.540279+0000","last_became_peered":"2024-04-25T13:48:09.540279+0000","last_unstale":"2024-04-25T13:48:11.636683+0000","last_undegraded":"2024-04-25T13:48:11.636683+0000","last_fullsized":"2024-04-25T13:48:11.636683+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:08.749125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634170+0000","last_change":"2024-04-25T13:48:10.604585+0000","last_active":"2024-04-25T13:48:11.634170+0000","last_peered":"2024-04-25T13:48:11.634170+0000","last_clean":"2024-04-25T13:48:11.634170+0000","last_became_active":"2024-04-25T13:48:10.592540+0000","last_became_peered":"2024-04-25T13:48:10.592540+0000","last_unstale":"2024-04-25T13:48:11.634170+0000","last_undegraded":"2024-04-25T13:48:11.634170+0000","last_fullsized":"2024-04-25T13:48:11.634170+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:58.744133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634362+0000","last_change":"2024-04-25T13:48:09.543768+0000","last_active":"2024-04-25T13:48:11.634362+0000","last_peered":"2024-04-25T13:48:11.634362+0000","last_clean":"2024-04-25T13:48:11.634362+0000","last_became_active":"2024-04-25T13:48:09.541684+0000","last_became_peered":"2024-04-25T13:48:09.541684+0000","last_unstale":"2024-04-25T13:48:11.634362+0000","last_undegraded":"2024-04-25T13:48:11.634362+0000","last_fullsized":"2024-04-25T13:48:11.634362+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:18:18.000963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.633681+0000","last_change":"2024-04-25T13:48:10.593401+0000","last_active":"2024-04-25T13:48:11.633681+0000","last_peered":"2024-04-25T13:48:11.633681+0000","last_clean":"2024-04-25T13:48:11.633681+0000","last_became_active":"2024-04-25T13:48:10.588064+0000","last_became_peered":"2024-04-25T13:48:10.588064+0000","last_unstale":"2024-04-25T13:48:11.633681+0000","last_undegraded":"2024-04-25T13:48:11.633681+0000","last_fullsized":"2024-04-25T13:48:11.633681+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:18:05.633467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558722+0000","last_change":"2024-04-25T13:48:09.537807+0000","last_active":"2024-04-25T13:48:11.558722+0000","last_peered":"2024-04-25T13:48:11.558722+0000","last_clean":"2024-04-25T13:48:11.558722+0000","last_became_active":"2024-04-25T13:48:09.534705+0000","last_became_peered":"2024-04-25T13:48:09.534705+0000","last_unstale":"2024-04-25T13:48:11.558722+0000","last_undegraded":"2024-04-25T13:48:11.558722+0000","last_fullsized":"2024-04-25T13:48:11.558722+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:25.489483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635849+0000","last_change":"2024-04-25T13:48:10.574406+0000","last_active":"2024-04-25T13:48:11.635849+0000","last_peered":"2024-04-25T13:48:11.635849+0000","last_clean":"2024-04-25T13:48:11.635849+0000","last_became_active":"2024-04-25T13:48:10.574094+0000","last_became_peered":"2024-04-25T13:48:10.574094+0000","last_unstale":"2024-04-25T13:48:11.635849+0000","last_undegraded":"2024-04-25T13:48:11.635849+0000","last_fullsized":"2024-04-25T13:48:11.635849+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:43:11.426615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"153'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561747+0000","last_change":"2024-04-25T13:48:09.545837+0000","last_active":"2024-04-25T13:48:11.561747+0000","last_peered":"2024-04-25T13:48:11.561747+0000","last_clean":"2024-04-25T13:48:11.561747+0000","last_became_active":"2024-04-25T13:48:09.543847+0000","last_became_peered":"2024-04-25T13:48:09.543847+0000","last_unstale":"2024-04-25T13:48:11.561747+0000","last_undegraded":"2024-04-25T13:48:11.561747+0000","last_fullsized":"2024-04-25T13:48:11.561747+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:56.932162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636998+0000","last_change":"2024-04-25T13:48:10.568663+0000","last_active":"2024-04-25T13:48:11.636998+0000","last_peered":"2024-04-25T13:48:11.636998+0000","last_clean":"2024-04-25T13:48:11.636998+0000","last_became_active":"2024-04-25T13:48:10.568148+0000","last_became_peered":"2024-04-25T13:48:10.568148+0000","last_unstale":"2024-04-25T13:48:11.636998+0000","last_undegraded":"2024-04-25T13:48:11.636998+0000","last_fullsized":"2024-04-25T13:48:11.636998+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:26.118303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635192+0000","last_change":"2024-04-25T13:48:09.535686+0000","last_active":"2024-04-25T13:48:11.635192+0000","last_peered":"2024-04-25T13:48:11.635192+0000","last_clean":"2024-04-25T13:48:11.635192+0000","last_became_active":"2024-04-25T13:48:09.534896+0000","last_became_peered":"2024-04-25T13:48:09.534896+0000","last_unstale":"2024-04-25T13:48:11.635192+0000","last_undegraded":"2024-04-25T13:48:11.635192+0000","last_fullsized":"2024-04-25T13:48:11.635192+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:05.690492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635259+0000","last_change":"2024-04-25T13:48:10.598218+0000","last_active":"2024-04-25T13:48:11.635259+0000","last_peered":"2024-04-25T13:48:11.635259+0000","last_clean":"2024-04-25T13:48:11.635259+0000","last_became_active":"2024-04-25T13:48:10.598011+0000","last_became_peered":"2024-04-25T13:48:10.598011+0000","last_unstale":"2024-04-25T13:48:11.635259+0000","last_undegraded":"2024-04-25T13:48:11.635259+0000","last_fullsized":"2024-04-25T13:48:11.635259+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:43.907565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634259+0000","last_change":"2024-04-25T13:48:09.545084+0000","last_active":"2024-04-25T13:48:11.634259+0000","last_peered":"2024-04-25T13:48:11.634259+0000","last_clean":"2024-04-25T13:48:11.634259+0000","last_became_active":"2024-04-25T13:48:09.541104+0000","last_became_peered":"2024-04-25T13:48:09.541104+0000","last_unstale":"2024-04-25T13:48:11.634259+0000","last_undegraded":"2024-04-25T13:48:11.634259+0000","last_fullsized":"2024-04-25T13:48:11.634259+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:36.917263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561184+0000","last_change":"2024-04-25T13:48:10.607422+0000","last_active":"2024-04-25T13:48:11.561184+0000","last_peered":"2024-04-25T13:48:11.561184+0000","last_clean":"2024-04-25T13:48:11.561184+0000","last_became_active":"2024-04-25T13:48:10.606733+0000","last_became_peered":"2024-04-25T13:48:10.606733+0000","last_unstale":"2024-04-25T13:48:11.561184+0000","last_undegraded":"2024-04-25T13:48:11.561184+0000","last_fullsized":"2024-04-25T13:48:11.561184+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:53:44.705944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636023+0000","last_change":"2024-04-25T13:48:09.546240+0000","last_active":"2024-04-25T13:48:11.636023+0000","last_peered":"2024-04-25T13:48:11.636023+0000","last_clean":"2024-04-25T13:48:11.636023+0000","last_became_active":"2024-04-25T13:48:09.545915+0000","last_became_peered":"2024-04-25T13:48:09.545915+0000","last_unstale":"2024-04-25T13:48:11.636023+0000","last_undegraded":"2024-04-25T13:48:11.636023+0000","last_fullsized":"2024-04-25T13:48:11.636023+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:23.648505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636215+0000","last_change":"2024-04-25T13:48:10.568452+0000","last_active":"2024-04-25T13:48:11.636215+0000","last_peered":"2024-04-25T13:48:11.636215+0000","last_clean":"2024-04-25T13:48:11.636215+0000","last_became_active":"2024-04-25T13:48:10.567753+0000","last_became_peered":"2024-04-25T13:48:10.567753+0000","last_unstale":"2024-04-25T13:48:11.636215+0000","last_undegraded":"2024-04-25T13:48:11.636215+0000","last_fullsized":"2024-04-25T13:48:11.636215+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:38.717680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637583+0000","last_change":"2024-04-25T13:48:09.536206+0000","last_active":"2024-04-25T13:48:11.637583+0000","last_peered":"2024-04-25T13:48:11.637583+0000","last_clean":"2024-04-25T13:48:11.637583+0000","last_became_active":"2024-04-25T13:48:09.536013+0000","last_became_peered":"2024-04-25T13:48:09.536013+0000","last_unstale":"2024-04-25T13:48:11.637583+0000","last_undegraded":"2024-04-25T13:48:11.637583+0000","last_fullsized":"2024-04-25T13:48:11.637583+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:56.362942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566946+0000","last_change":"2024-04-25T13:48:10.603426+0000","last_active":"2024-04-25T13:48:11.566946+0000","last_peered":"2024-04-25T13:48:11.566946+0000","last_clean":"2024-04-25T13:48:11.566946+0000","last_became_active":"2024-04-25T13:48:10.585576+0000","last_became_peered":"2024-04-25T13:48:10.585576+0000","last_unstale":"2024-04-25T13:48:11.566946+0000","last_undegraded":"2024-04-25T13:48:11.566946+0000","last_fullsized":"2024-04-25T13:48:11.566946+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:03.126219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.558942+0000","last_change":"2024-04-25T13:48:09.532911+0000","last_active":"2024-04-25T13:48:11.558942+0000","last_peered":"2024-04-25T13:48:11.558942+0000","last_clean":"2024-04-25T13:48:11.558942+0000","last_became_active":"2024-04-25T13:48:09.532026+0000","last_became_peered":"2024-04-25T13:48:09.532026+0000","last_unstale":"2024-04-25T13:48:11.558942+0000","last_undegraded":"2024-04-25T13:48:11.558942+0000","last_fullsized":"2024-04-25T13:48:11.558942+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:42.439991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.561652+0000","last_change":"2024-04-25T13:48:10.592701+0000","last_active":"2024-04-25T13:48:11.561652+0000","last_peered":"2024-04-25T13:48:11.561652+0000","last_clean":"2024-04-25T13:48:11.561652+0000","last_became_active":"2024-04-25T13:48:10.591813+0000","last_became_peered":"2024-04-25T13:48:10.591813+0000","last_unstale":"2024-04-25T13:48:11.561652+0000","last_undegraded":"2024-04-25T13:48:11.561652+0000","last_fullsized":"2024-04-25T13:48:11.561652+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:02.788870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635370+0000","last_change":"2024-04-25T13:48:09.548276+0000","last_active":"2024-04-25T13:48:11.635370+0000","last_peered":"2024-04-25T13:48:11.635370+0000","last_clean":"2024-04-25T13:48:11.635370+0000","last_became_active":"2024-04-25T13:48:09.548096+0000","last_became_peered":"2024-04-25T13:48:09.548096+0000","last_unstale":"2024-04-25T13:48:11.635370+0000","last_undegraded":"2024-04-25T13:48:11.635370+0000","last_fullsized":"2024-04-25T13:48:11.635370+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:03.563156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635073+0000","last_change":"2024-04-25T13:48:10.577700+0000","last_active":"2024-04-25T13:48:11.635073+0000","last_peered":"2024-04-25T13:48:11.635073+0000","last_clean":"2024-04-25T13:48:11.635073+0000","last_became_active":"2024-04-25T13:48:10.577164+0000","last_became_peered":"2024-04-25T13:48:10.577164+0000","last_unstale":"2024-04-25T13:48:11.635073+0000","last_undegraded":"2024-04-25T13:48:11.635073+0000","last_fullsized":"2024-04-25T13:48:11.635073+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:36.951133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"154'1","reported_seq":15,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.568908+0000","last_change":"2024-04-25T13:48:09.550250+0000","last_active":"2024-04-25T13:48:11.568908+0000","last_peered":"2024-04-25T13:48:11.568908+0000","last_clean":"2024-04-25T13:48:11.568908+0000","last_became_active":"2024-04-25T13:48:09.548228+0000","last_became_peered":"2024-04-25T13:48:09.548228+0000","last_unstale":"2024-04-25T13:48:11.568908+0000","last_undegraded":"2024-04-25T13:48:11.568908+0000","last_fullsized":"2024-04-25T13:48:11.568908+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:57.684938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.638915+0000","last_change":"2024-04-25T13:48:10.592335+0000","last_active":"2024-04-25T13:48:11.638915+0000","last_peered":"2024-04-25T13:48:11.638915+0000","last_clean":"2024-04-25T13:48:11.638915+0000","last_became_active":"2024-04-25T13:48:10.591980+0000","last_became_peered":"2024-04-25T13:48:10.591980+0000","last_unstale":"2024-04-25T13:48:11.638915+0000","last_undegraded":"2024-04-25T13:48:11.638915+0000","last_fullsized":"2024-04-25T13:48:11.638915+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:44.091724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636092+0000","last_change":"2024-04-25T13:48:09.532976+0000","last_active":"2024-04-25T13:48:11.636092+0000","last_peered":"2024-04-25T13:48:11.636092+0000","last_clean":"2024-04-25T13:48:11.636092+0000","last_became_active":"2024-04-25T13:48:09.532729+0000","last_became_peered":"2024-04-25T13:48:09.532729+0000","last_unstale":"2024-04-25T13:48:11.636092+0000","last_undegraded":"2024-04-25T13:48:11.636092+0000","last_fullsized":"2024-04-25T13:48:11.636092+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.266519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.560749+0000","last_change":"2024-04-25T13:48:10.596857+0000","last_active":"2024-04-25T13:48:11.560749+0000","last_peered":"2024-04-25T13:48:11.560749+0000","last_clean":"2024-04-25T13:48:11.560749+0000","last_became_active":"2024-04-25T13:48:10.582230+0000","last_became_peered":"2024-04-25T13:48:10.582230+0000","last_unstale":"2024-04-25T13:48:11.560749+0000","last_undegraded":"2024-04-25T13:48:11.560749+0000","last_fullsized":"2024-04-25T13:48:11.560749+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:54.679392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.559006+0000","last_change":"2024-04-25T13:48:09.534257+0000","last_active":"2024-04-25T13:48:11.559006+0000","last_peered":"2024-04-25T13:48:11.559006+0000","last_clean":"2024-04-25T13:48:11.559006+0000","last_became_active":"2024-04-25T13:48:09.530230+0000","last_became_peered":"2024-04-25T13:48:09.530230+0000","last_unstale":"2024-04-25T13:48:11.559006+0000","last_undegraded":"2024-04-25T13:48:11.559006+0000","last_fullsized":"2024-04-25T13:48:11.559006+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:14.040233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566389+0000","last_change":"2024-04-25T13:48:10.598975+0000","last_active":"2024-04-25T13:48:11.566389+0000","last_peered":"2024-04-25T13:48:11.566389+0000","last_clean":"2024-04-25T13:48:11.566389+0000","last_became_active":"2024-04-25T13:48:10.598012+0000","last_became_peered":"2024-04-25T13:48:10.598012+0000","last_unstale":"2024-04-25T13:48:11.566389+0000","last_undegraded":"2024-04-25T13:48:11.566389+0000","last_fullsized":"2024-04-25T13:48:11.566389+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:04:38.253502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637272+0000","last_change":"2024-04-25T13:48:09.539642+0000","last_active":"2024-04-25T13:48:11.637272+0000","last_peered":"2024-04-25T13:48:11.637272+0000","last_clean":"2024-04-25T13:48:11.637272+0000","last_became_active":"2024-04-25T13:48:09.539095+0000","last_became_peered":"2024-04-25T13:48:09.539095+0000","last_unstale":"2024-04-25T13:48:11.637272+0000","last_undegraded":"2024-04-25T13:48:11.637272+0000","last_fullsized":"2024-04-25T13:48:11.637272+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:16:57.268234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.634407+0000","last_change":"2024-04-25T13:48:10.587919+0000","last_active":"2024-04-25T13:48:11.634407+0000","last_peered":"2024-04-25T13:48:11.634407+0000","last_clean":"2024-04-25T13:48:11.634407+0000","last_became_active":"2024-04-25T13:48:10.586169+0000","last_became_peered":"2024-04-25T13:48:10.586169+0000","last_unstale":"2024-04-25T13:48:11.634407+0000","last_undegraded":"2024-04-25T13:48:11.634407+0000","last_fullsized":"2024-04-25T13:48:11.634407+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:48.401823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.566845+0000","last_change":"2024-04-25T13:48:09.535810+0000","last_active":"2024-04-25T13:48:11.566845+0000","last_peered":"2024-04-25T13:48:11.566845+0000","last_clean":"2024-04-25T13:48:11.566845+0000","last_became_active":"2024-04-25T13:48:09.534927+0000","last_became_peered":"2024-04-25T13:48:09.534927+0000","last_unstale":"2024-04-25T13:48:11.566845+0000","last_undegraded":"2024-04-25T13:48:11.566845+0000","last_fullsized":"2024-04-25T13:48:11.566845+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:57:09.121478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.635729+0000","last_change":"2024-04-25T13:48:10.572221+0000","last_active":"2024-04-25T13:48:11.635729+0000","last_peered":"2024-04-25T13:48:11.635729+0000","last_clean":"2024-04-25T13:48:11.635729+0000","last_became_active":"2024-04-25T13:48:10.571890+0000","last_became_peered":"2024-04-25T13:48:10.571890+0000","last_unstale":"2024-04-25T13:48:11.635729+0000","last_undegraded":"2024-04-25T13:48:11.635729+0000","last_fullsized":"2024-04-25T13:48:11.635729+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:36.359597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.637363+0000","last_change":"2024-04-25T13:48:09.547060+0000","last_active":"2024-04-25T13:48:11.637363+0000","last_peered":"2024-04-25T13:48:11.637363+0000","last_clean":"2024-04-25T13:48:11.637363+0000","last_became_active":"2024-04-25T13:48:09.546321+0000","last_became_peered":"2024-04-25T13:48:09.546321+0000","last_unstale":"2024-04-25T13:48:11.637363+0000","last_undegraded":"2024-04-25T13:48:11.637363+0000","last_fullsized":"2024-04-25T13:48:11.637363+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:23:14.502478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.636346+0000","last_change":"2024-04-25T13:48:10.574015+0000","last_active":"2024-04-25T13:48:11.636346+0000","last_peered":"2024-04-25T13:48:11.636346+0000","last_clean":"2024-04-25T13:48:11.636346+0000","last_became_active":"2024-04-25T13:48:10.573609+0000","last_became_peered":"2024-04-25T13:48:10.573609+0000","last_unstale":"2024-04-25T13:48:11.636346+0000","last_undegraded":"2024-04-25T13:48:11.636346+0000","last_fullsized":"2024-04-25T13:48:11.636346+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:38.142413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.559024+0000","last_change":"2024-04-25T13:48:09.539563+0000","last_active":"2024-04-25T13:48:11.559024+0000","last_peered":"2024-04-25T13:48:11.559024+0000","last_clean":"2024-04-25T13:48:11.559024+0000","last_became_active":"2024-04-25T13:48:09.539303+0000","last_became_peered":"2024-04-25T13:48:09.539303+0000","last_unstale":"2024-04-25T13:48:11.559024+0000","last_undegraded":"2024-04-25T13:48:11.559024+0000","last_fullsized":"2024-04-25T13:48:11.559024+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:35.043621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567417+0000","last_change":"2024-04-25T13:48:10.597719+0000","last_active":"2024-04-25T13:48:11.567417+0000","last_peered":"2024-04-25T13:48:11.567417+0000","last_clean":"2024-04-25T13:48:11.567417+0000","last_became_active":"2024-04-25T13:48:10.586892+0000","last_became_peered":"2024-04-25T13:48:10.586892+0000","last_unstale":"2024-04-25T13:48:11.567417+0000","last_undegraded":"2024-04-25T13:48:11.567417+0000","last_fullsized":"2024-04-25T13:48:11.567417+0000","mapping_epoch":152,"log_start":"0'0","ondisk_log_start":"0'0","created":152,"last_epoch_clean":153,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:09.507827+0000","last_clean_scrub_stamp":"2024-04-25T13:48:09.507827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:13.759569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"153'2","reported_seq":16,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567811+0000","last_change":"2024-04-25T13:48:09.542646+0000","last_active":"2024-04-25T13:48:11.567811+0000","last_peered":"2024-04-25T13:48:11.567811+0000","last_clean":"2024-04-25T13:48:11.567811+0000","last_became_active":"2024-04-25T13:48:09.541898+0000","last_became_peered":"2024-04-25T13:48:09.541898+0000","last_unstale":"2024-04-25T13:48:11.567811+0000","last_undegraded":"2024-04-25T13:48:11.567811+0000","last_fullsized":"2024-04-25T13:48:11.567811+0000","mapping_epoch":151,"log_start":"0'0","ondisk_log_start":"0'0","created":151,"last_epoch_clean":152,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:48:08.497197+0000","last_clean_scrub_stamp":"2024-04-25T13:48:08.497197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:14.687224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":374,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-25T13:48:11.567133+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:48:11.567133+0000","last_peered":"2024-04-25T13:48:11.567133+0000","last_clean":"2024-04-25T13:48:11.567133+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:48:11.567133+0000","last_undegraded":"2024-04-25T13:48:11.567133+0000","last_fullsized":"2024-04-25T13:48:11.567133+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1419,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193892,"statfs":{"total":200000000,"available":198546015,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574945,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":842,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194469,"statfs":{"total":200000000,"available":199137021,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194471,"statfs":{"total":200000000,"available":199138521,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574945,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194471,"statfs":{"total":200000000,"available":199138563,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574945,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194471,"statfs":{"total":200000000,"available":199138473,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574945,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194470,"statfs":{"total":200000000,"available":199138017,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574945,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":841,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194471,"statfs":{"total":200000000,"available":199138633,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574945,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1417,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193894,"statfs":{"total":200000000,"available":198548265,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:48:15.118 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:48:15.118 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:48:15.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f269e12a640 1 Processor -- start 2024-04-25T13:48:15.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f269e12a640 1 -- start start 2024-04-25T13:48:15.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f269e12a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f26980d06e0 0x7f269806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f269e12a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26980d0af0 con 0x7f269806e9e0 2024-04-25T13:48:15.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f269e12a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f26980d0d70 con 0x7f2698161f90 2024-04-25T13:48:15.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f269e12a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f26980d1050 con 0x7f26980d06e0 2024-04-25T13:48:15.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f2697fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f269806e9e0 0x7f269806edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53494/0 (socket says 172.21.15.80:53494) 2024-04-25T13:48:15.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f2697fff640 1 -- 172.21.15.80:0/4114402109 learned_addr learned my addr 172.21.15.80:0/4114402109 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:15.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f2696ffd640 1 --2- 172.21.15.80:0/4114402109 >> v2:172.21.15.80:3300/0 conn(0x7f26980d06e0 0x7f269806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.322+0000 7f26967fc640 1 -- 172.21.15.80:0/4114402109 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1487285811 0 0) 0x7f2690001120 con 0x7f269806e9e0 2024-04-25T13:48:15.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f26967fc640 1 -- 172.21.15.80:0/4114402109 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2680003530 con 0x7f269806e9e0 2024-04-25T13:48:15.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f2696ffd640 1 -- 172.21.15.80:0/4114402109 >> v1:172.21.15.87:6789/0 conn(0x7f2698161f90 legacy=0x7f26980cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f2696ffd640 1 -- 172.21.15.80:0/4114402109 >> v1:172.21.15.80:6789/0 conn(0x7f269806e9e0 legacy=0x7f269806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.323 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f2696ffd640 1 -- 172.21.15.80:0/4114402109 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269806dca0 con 0x7f26980d06e0 2024-04-25T13:48:15.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f2696ffd640 1 --2- 172.21.15.80:0/4114402109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26980d06e0 0x7f269806e4a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f268c0097c0 tx=0x7f268c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc9aaf28ec999b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:15.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f26967fc640 1 -- 172.21.15.80:0/4114402109 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0b1070 con 0x7f26980d06e0 2024-04-25T13:48:15.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f26967fc640 1 -- 172.21.15.80:0/4114402109 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f268c0ac640 con 0x7f26980d06e0 2024-04-25T13:48:15.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f26967fc640 1 -- 172.21.15.80:0/4114402109 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268c0aca10 con 0x7f26980d06e0 2024-04-25T13:48:15.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f269e12a640 1 -- 172.21.15.80:0/4114402109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26980d06e0 msgr2=0x7f269806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f269e12a640 1 --2- 172.21.15.80:0/4114402109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26980d06e0 0x7f269806e4a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f268c0097c0 tx=0x7f268c01c350 comp rx=0 tx=0).stop 2024-04-25T13:48:15.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f269e12a640 1 -- 172.21.15.80:0/4114402109 shutdown_connections 2024-04-25T13:48:15.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f269e12a640 1 --2- 172.21.15.80:0/4114402109 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f26980d06e0 0x7f269806e4a0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f269e12a640 1 -- 172.21.15.80:0/4114402109 >> 172.21.15.80:0/4114402109 conn(0x7f269806d080 msgr2=0x7f26980b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:15.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f269e12a640 1 -- 172.21.15.80:0/4114402109 shutdown_connections 2024-04-25T13:48:15.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.326+0000 7f269e12a640 1 -- 172.21.15.80:0/4114402109 wait complete. 2024-04-25T13:48:15.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 Processor -- start 2024-04-25T13:48:15.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 -- start start 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f269806e9e0 0x7f26980ac120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26980d06e0 0x7f269815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2698161f90 0x7f2698140e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f269806f680 con 0x7f269806e9e0 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f269806f500 con 0x7f26980d06e0 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2698144620 con 0x7f2698161f90 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f269806e9e0 0x7f26980ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f269806e9e0 0x7f26980ac120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50604/0 (socket says 172.21.15.80:50604) 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 -- 172.21.15.80:0/4030374519 learned_addr learned my addr 172.21.15.80:0/4030374519 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f2696ffd640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26980d06e0 0x7f269815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f2697fff640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2698161f90 0x7f2698140e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 -- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2698161f90 msgr2=0x7f2698140e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2698161f90 0x7f2698140e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 -- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26980d06e0 msgr2=0x7f269815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26980d06e0 0x7f269815daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 -- 172.21.15.80:0/4030374519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f269813fd70 con 0x7f269806e9e0 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f26977fe640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f269806e9e0 0x7f26980ac120 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f268800c760 tx=0x7f268800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f267ffff640 1 -- 172.21.15.80:0/4030374519 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2688003b80 con 0x7f269806e9e0 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f267ffff640 1 -- 172.21.15.80:0/4030374519 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2688003d20 con 0x7f269806e9e0 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2698140050 con 0x7f269806e9e0 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f267ffff640 1 -- 172.21.15.80:0/4030374519 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f268800aa90 con 0x7f269806e9e0 2024-04-25T13:48:15.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.330+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2698140610 con 0x7f269806e9e0 2024-04-25T13:48:15.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.334+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f269806a3a0 con 0x7f269806e9e0 2024-04-25T13:48:15.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.334+0000 7f267ffff640 1 -- 172.21.15.80:0/4030374519 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f268800ac30 con 0x7f269806e9e0 2024-04-25T13:48:15.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.334+0000 7f267ffff640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f268406f400 0x7f2684071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.334+0000 7f267ffff640 1 -- 172.21.15.80:0/4030374519 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f268808f120 con 0x7f269806e9e0 2024-04-25T13:48:15.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.334+0000 7f2696ffd640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f268406f400 0x7f2684071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.334+0000 7f2696ffd640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f268406f400 0x7f2684071910 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f268c0ae000 tx=0x7f268c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:15.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.338+0000 7f267ffff640 1 -- 172.21.15.80:0/4030374519 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f268805c010 con 0x7f269806e9e0 2024-04-25T13:48:15.492 INFO:tasks.ceph.mds.a.smithi080.stderr:2024-04-25T13:48:15.494+0000 7fa6ebed5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:48:15.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.558+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f26980ce720 con 0x7f269806e9e0 2024-04-25T13:48:15.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.562+0000 7f267ffff640 1 -- 172.21.15.80:0/4030374519 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f2688064090 con 0x7f269806e9e0 2024-04-25T13:48:15.562 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:48:15.562 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:48:15.562 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:48:15.562 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:48:15.562 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:48:15.562 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f268406f400 msgr2=0x7f2684071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f268406f400 0x7f2684071910 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f268c0ae000 tx=0x7f268c008040 comp rx=0 tx=0).stop 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f269806e9e0 msgr2=0x7f26980ac120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f269806e9e0 0x7f26980ac120 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f268800c760 tx=0x7f268800cc20 comp rx=0 tx=0).stop 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 shutdown_connections 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f268406f400 0x7f2684071910 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2698161f90 0x7f2698140e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f26980d06e0 0x7f269815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 --2- 172.21.15.80:0/4030374519 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f269806e9e0 0x7f26980ac120 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 >> 172.21.15.80:0/4030374519 conn(0x7f269806d080 msgr2=0x7f26980757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:15.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 shutdown_connections 2024-04-25T13:48:15.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.566+0000 7f269e12a640 1 -- 172.21.15.80:0/4030374519 wait complete. 2024-04-25T13:48:15.585 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:48:15.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.722+0000 7fb02fdaa640 1 Processor -- start 2024-04-25T13:48:15.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.722+0000 7fb02fdaa640 1 -- start start 2024-04-25T13:48:15.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02fdaa640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb02806f5a0 0x7fb028160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02fdaa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0280733b0 0x7fb028161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02fdaa640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb02806fcf0 con 0x7fb028161070 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02fdaa640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb0281619c0 con 0x7fb02806f5a0 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02fdaa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb028073f10 con 0x7fb0280733b0 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02db46640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb02806f5a0 0x7fb028160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02db46640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb02806f5a0 0x7fb028160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50612/0 (socket says 172.21.15.80:50612) 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02db46640 1 -- 172.21.15.80:0/3776937222 learned_addr learned my addr 172.21.15.80:0/3776937222 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02e347640 1 --2- 172.21.15.80:0/3776937222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0280733b0 0x7fb028161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02db46640 1 -- 172.21.15.80:0/3776937222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0280733b0 msgr2=0x7fb028161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02db46640 1 --2- 172.21.15.80:0/3776937222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0280733b0 0x7fb028161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02db46640 1 -- 172.21.15.80:0/3776937222 >> v1:172.21.15.87:6789/0 conn(0x7fb028161070 legacy=0x7fb0280b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.726+0000 7fb02db46640 1 -- 172.21.15.80:0/3776937222 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0280b0c60 con 0x7fb02806f5a0 2024-04-25T13:48:15.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02db46640 1 --2- 172.21.15.80:0/3776937222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02806f5a0 0x7fb028160b30 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb0240098a0 tx=0x7fb02401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66f2eeb8129d114 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:15.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02cb44640 1 -- 172.21.15.80:0/3776937222 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240b0070 con 0x7fb02806f5a0 2024-04-25T13:48:15.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02cb44640 1 -- 172.21.15.80:0/3776937222 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0240ab600 con 0x7fb02806f5a0 2024-04-25T13:48:15.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02cb44640 1 -- 172.21.15.80:0/3776937222 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240ab9d0 con 0x7fb02806f5a0 2024-04-25T13:48:15.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3776937222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02806f5a0 msgr2=0x7fb028160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3776937222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02806f5a0 0x7fb028160b30 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb0240098a0 tx=0x7fb02401c4b0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3776937222 shutdown_connections 2024-04-25T13:48:15.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3776937222 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0280733b0 0x7fb028161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3776937222 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb02806f5a0 0x7fb028160b30 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3776937222 >> 172.21.15.80:0/3776937222 conn(0x7fb02806d080 msgr2=0x7fb0280b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:15.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3776937222 shutdown_connections 2024-04-25T13:48:15.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3776937222 wait complete. 2024-04-25T13:48:15.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 Processor -- start 2024-04-25T13:48:15.729 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.730+0000 7fb02fdaa640 1 -- start start 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02806f5a0 0x7fb0281fc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0280733b0 0x7fb02815b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028161070 0x7fb028156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb02816a050 con 0x7fb0280733b0 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb028169ed0 con 0x7fb028161070 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb02816a1d0 con 0x7fb02806f5a0 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0280733b0 0x7fb02815b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0280733b0 0x7fb02815b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50624/0 (socket says 172.21.15.80:50624) 2024-04-25T13:48:15.732 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 -- 172.21.15.80:0/3210232859 learned_addr learned my addr 172.21.15.80:0/3210232859 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02db46640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02806f5a0 0x7fb0281fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02e347640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028161070 0x7fb028156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 -- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02806f5a0 msgr2=0x7fb0281fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02806f5a0 0x7fb0281fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 -- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028161070 msgr2=0x7fb028156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028161070 0x7fb028156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 -- 172.21.15.80:0/3210232859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02815bc50 con 0x7fb0280733b0 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02d345640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0280733b0 0x7fb02815b320 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fb01800c370 tx=0x7fb01800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb01e7fc640 1 -- 172.21.15.80:0/3210232859 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0180052d0 con 0x7fb0280733b0 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb01e7fc640 1 -- 172.21.15.80:0/3210232859 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb018005470 con 0x7fb0280733b0 2024-04-25T13:48:15.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02815bf30 con 0x7fb0280733b0 2024-04-25T13:48:15.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb01e7fc640 1 -- 172.21.15.80:0/3210232859 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018007260 con 0x7fb0280733b0 2024-04-25T13:48:15.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02815c4f0 con 0x7fb0280733b0 2024-04-25T13:48:15.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb02806a3a0 con 0x7fb0280733b0 2024-04-25T13:48:15.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb01e7fc640 1 -- 172.21.15.80:0/3210232859 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb018007400 con 0x7fb0280733b0 2024-04-25T13:48:15.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb01e7fc640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:15.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb01e7fc640 1 -- 172.21.15.80:0/3210232859 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fb01808e6a0 con 0x7fb0280733b0 2024-04-25T13:48:15.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.734+0000 7fb02db46640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:15.735 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.738+0000 7fb02db46640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb0240ad000 tx=0x7fb0240a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:15.738 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.742+0000 7fb01e7fc640 1 -- 172.21.15.80:0/3210232859 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb01805b620 con 0x7fb0280733b0 2024-04-25T13:48:15.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.922+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb02806f1f0 con 0x7fb0280733b0 2024-04-25T13:48:15.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.922+0000 7fb01e7fc640 1 -- 172.21.15.80:0/3210232859 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7fb01805af20 con 0x7fb0280733b0 2024-04-25T13:48:15.920 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:15.921 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/521602421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":521602421},{"type":"v1","addr":"172.21.15.87:6835","nonce":521602421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4170043750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4170043750},{"type":"v1","addr":"172.21.15.87:6837","nonce":4170043750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/1423926990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1423926990},{"type":"v1","addr":"172.21.15.80:6834","nonce":1423926990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:48:09.907104+0000","modified":"2024-04-25T13:48:12.977708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1134081162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1134081162},{"type":"v1","addr":"172.21.15.80:6835","nonce":1134081162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-25T13:48:15.921 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 74 2024-04-25T13:48:15.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.922+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 msgr2=0x7fb00c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.922+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fb0240ad000 tx=0x7fb0240a6000 comp rx=0 tx=0).stop 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.922+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0280733b0 msgr2=0x7fb02815b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.922+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0280733b0 0x7fb02815b320 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fb01800c370 tx=0x7fb01800c830 comp rx=0 tx=0).stop 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.922+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 shutdown_connections 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.926+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb00c06f400 0x7fb00c071910 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.926+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb028161070 0x7fb028156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.926+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0280733b0 0x7fb02815b320 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.926+0000 7fb02fdaa640 1 --2- 172.21.15.80:0/3210232859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb02806f5a0 0x7fb0281fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.926+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 >> 172.21.15.80:0/3210232859 conn(0x7fb02806d080 msgr2=0x7fb0280b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.926+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 shutdown_connections 2024-04-25T13:48:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:15.926+0000 7fb02fdaa640 1 -- 172.21.15.80:0/3210232859 wait complete. 2024-04-25T13:48:16.344 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:48:09.907104+0000', 'modified': '2024-04-25T13:48:12.977708+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8412}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8412': {'gid': 8412, 'name': 'a', 'rank': 0, 'incarnation': 71, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6835/1134081162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6833', 'nonce': 1134081162}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 1134081162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8412, 'qdb_cluster': [8412]} 2024-04-25T13:48:16.344 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:48:16.345 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T13:48:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.506+0000 7f612fdeb640 1 Processor -- start 2024-04-25T13:48:16.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.506+0000 7f612fdeb640 1 -- start start 2024-04-25T13:48:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612fdeb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f61280d06e0 0x7f612806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612fdeb640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61280d0af0 con 0x7f6128161f90 2024-04-25T13:48:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612fdeb640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61280d0d70 con 0x7f612806e9e0 2024-04-25T13:48:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612fdeb640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f61280d1050 con 0x7f61280d06e0 2024-04-25T13:48:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612db87640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6128161f90 0x7f61280cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53510/0 (socket says 172.21.15.80:53510) 2024-04-25T13:48:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612d386640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f61280d06e0 0x7f612806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612db87640 1 -- 172.21.15.80:0/3976771583 learned_addr learned my addr 172.21.15.80:0/3976771583 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612d386640 1 -- 172.21.15.80:0/3976771583 >> v1:172.21.15.87:6789/0 conn(0x7f612806e9e0 legacy=0x7f612806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612d386640 1 -- 172.21.15.80:0/3976771583 >> v1:172.21.15.80:6789/0 conn(0x7f6128161f90 legacy=0x7f61280cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612d386640 1 -- 172.21.15.80:0/3976771583 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612806dca0 con 0x7f61280d06e0 2024-04-25T13:48:16.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612cb85640 1 -- 172.21.15.80:0/3976771583 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4027023425 0 0) 0x7f6124001120 con 0x7f6128161f90 2024-04-25T13:48:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612d386640 1 --2- 172.21.15.80:0/3976771583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61280d06e0 0x7f612806e4a0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f61180097c0 tx=0x7f611801c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff3ae90aec56dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612cb85640 1 -- 172.21.15.80:0/3976771583 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61180b1070 con 0x7f61280d06e0 2024-04-25T13:48:16.509 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612cb85640 1 -- 172.21.15.80:0/3976771583 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61180ac640 con 0x7f61280d06e0 2024-04-25T13:48:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.510+0000 7f612cb85640 1 -- 172.21.15.80:0/3976771583 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61180aca10 con 0x7f61280d06e0 2024-04-25T13:48:16.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- 172.21.15.80:0/3976771583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61280d06e0 msgr2=0x7f612806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 --2- 172.21.15.80:0/3976771583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61280d06e0 0x7f612806e4a0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f61180097c0 tx=0x7f611801c350 comp rx=0 tx=0).stop 2024-04-25T13:48:16.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- 172.21.15.80:0/3976771583 shutdown_connections 2024-04-25T13:48:16.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 --2- 172.21.15.80:0/3976771583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f61280d06e0 0x7f612806e4a0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- 172.21.15.80:0/3976771583 >> 172.21.15.80:0/3976771583 conn(0x7f612806d080 msgr2=0x7f61280b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:16.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- 172.21.15.80:0/3976771583 shutdown_connections 2024-04-25T13:48:16.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- 172.21.15.80:0/3976771583 wait complete. 2024-04-25T13:48:16.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 Processor -- start 2024-04-25T13:48:16.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- start start 2024-04-25T13:48:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f612806e9e0 0x7f6128140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61280d06e0 0x7f61280abd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6128161f90 0x7f61281693b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f612806f680 con 0x7f612806e9e0 2024-04-25T13:48:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f612806f500 con 0x7f61280d06e0 2024-04-25T13:48:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612fdeb640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6128144620 con 0x7f6128161f90 2024-04-25T13:48:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612db87640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f612806e9e0 0x7f6128140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612db87640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f612806e9e0 0x7f6128140e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50650/0 (socket says 172.21.15.80:50650) 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612db87640 1 -- 172.21.15.80:0/432725389 learned_addr learned my addr 172.21.15.80:0/432725389 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612d386640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61280d06e0 0x7f61280abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612e388640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6128161f90 0x7f61281693b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612db87640 1 -- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6128161f90 msgr2=0x7f61281693b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612db87640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6128161f90 0x7f61281693b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612db87640 1 -- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61280d06e0 msgr2=0x7f61280abd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612db87640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61280d06e0 0x7f61280abd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612db87640 1 -- 172.21.15.80:0/432725389 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6128144920 con 0x7f612806e9e0 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.514+0000 7f612d386640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61280d06e0 0x7f61280abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f612db87640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f612806e9e0 0x7f6128140e60 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f612400c410 tx=0x7f612400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f611e7fc640 1 -- 172.21.15.80:0/432725389 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61240074e0 con 0x7f612806e9e0 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f611e7fc640 1 -- 172.21.15.80:0/432725389 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61240047f0 con 0x7f612806e9e0 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f612fdeb640 1 -- 172.21.15.80:0/432725389 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6128144c00 con 0x7f612806e9e0 2024-04-25T13:48:16.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f611e7fc640 1 -- 172.21.15.80:0/432725389 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124007680 con 0x7f612806e9e0 2024-04-25T13:48:16.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f612fdeb640 1 -- 172.21.15.80:0/432725389 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61281451c0 con 0x7f612806e9e0 2024-04-25T13:48:16.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f611e7fc640 1 -- 172.21.15.80:0/432725389 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6124011070 con 0x7f612806e9e0 2024-04-25T13:48:16.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f611e7fc640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f610c06f400 0x7f610c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f611e7fc640 1 -- 172.21.15.80:0/432725389 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f612408ee80 con 0x7f612806e9e0 2024-04-25T13:48:16.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f612d386640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f610c06f400 0x7f610c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f6137148000 1 Processor -- start 2024-04-25T13:48:16.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f6137148000 1 -- start start 2024-04-25T13:48:16.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.518+0000 7f612d386640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f610c06f400 0x7f610c071910 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f61180ae000 tx=0x7f61180a6070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a723767ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a723768420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a723768960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a7235d7160 con 0x55a723609c20 2024-04-25T13:48:16.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a72356ce20 con 0x55a7235f5ea0 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55a723525a20 con 0x55a7231ee280 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a723767ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a723767ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50658/0 (socket says 172.21.15.80:50658) 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 -- 172.21.15.80:0/913727611 learned_addr learned my addr 172.21.15.80:0/913727611 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612e388640 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a723768960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612d386640 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a723768420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 -- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 msgr2=0x55a723768960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a723768960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 -- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 msgr2=0x55a723768420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a723768420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 -- 172.21.15.80:0/913727611 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a72352a100 con 0x55a723609c20 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612e388640 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a723768960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612d386640 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a723768420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:16.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f612db87640 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a723767ee0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f612408d3d0 tx=0x7f612405b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b16502c9f7a2ef8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f610b7fe640 1 -- 172.21.15.80:0/913727611 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612405b670 con 0x55a723609c20 2024-04-25T13:48:16.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f610b7fe640 1 -- 172.21.15.80:0/913727611 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612405fa20 con 0x55a723609c20 2024-04-25T13:48:16.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f610b7fe640 1 -- 172.21.15.80:0/913727611 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6124004de0 con 0x55a723609c20 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- 172.21.15.80:0/913727611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 msgr2=0x55a723767ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a723767ee0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f612408d3d0 tx=0x7f612405b2c0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- 172.21.15.80:0/913727611 shutdown_connections 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a723768960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a723768420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 --2- 172.21.15.80:0/913727611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a723767ee0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- 172.21.15.80:0/913727611 >> 172.21.15.80:0/913727611 conn(0x55a72322dff0 msgr2=0x55a72371d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- 172.21.15.80:0/913727611 shutdown_connections 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- 172.21.15.80:0/913727611 wait complete. 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 -- start start 2024-04-25T13:48:16.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.522+0000 7f6137148000 1 Processor -- start 2024-04-25T13:48:16.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 0x55a72376ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7235f5ea0 0x55a72376f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72372a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a72357a220 con 0x55a723609c20 2024-04-25T13:48:16.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a723510ea0 con 0x55a7231ee280 2024-04-25T13:48:16.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55a723567b00 con 0x55a7235f5ea0 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 10 client.0 _ll_register_callbacks cb 0x55a723579dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72372a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72372a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50672/0 (socket says 172.21.15.80:50672) 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 -- 172.21.15.80:0/707979145 learned_addr learned my addr 172.21.15.80:0/707979145 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612db87640 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 0x55a72376ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612d386640 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7235f5ea0 0x55a72376f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 -- 172.21.15.80:0/707979145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7235f5ea0 msgr2=0x55a72376f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7235f5ea0 0x55a72376f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 -- 172.21.15.80:0/707979145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 msgr2=0x55a72376ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 0x55a72376ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 -- 172.21.15.80:0/707979145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a72350c500 con 0x55a723609c20 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612e388640 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72372a0c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f612000f4b0 tx=0x7f612000f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54759137134c7c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 1 -- 172.21.15.80:0/707979145 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120003b90 con 0x55a723609c20 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 1 -- 172.21.15.80:0/707979145 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f6120003d30 con 0x55a723609c20 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 -- 172.21.15.80:0/707979145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f612405fa20 con 0x55a723609c20 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 1 -- 172.21.15.80:0/707979145 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f6120020050 con 0x55a723609c20 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 10 client.8553 fetch_fsmap learned FSMap version 74 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 -- 172.21.15.80:0/707979145 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a7234f2ea0 con 0x55a723609c20 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 1 -- 172.21.15.80:0/707979145 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f612405fa20 con 0x55a723609c20 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 1 -- 172.21.15.80:0/707979145 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f6120023020 con 0x55a723609c20 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 10 client.8553 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T13:48:16.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 10 client.8553 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 20 client.8553 populate_metadata read hostname 'smithi080' 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] conn(0x55a7232127d0 0x55a72372ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 4 client.8553 mds_command: new command op to 8412 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 -- 172.21.15.80:0/707979145 --> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a723505be0 con 0x55a7232127d0 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612db87640 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] conn(0x55a7232127d0 0x55a72372ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 10 client.8553 ms_handle_connect on v2:172.21.15.80:6833/1134081162 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f612db87640 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] conn(0x55a7232127d0 0x55a72372ae00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ce18a9903536d5 server_cookie=bf6cc1207c464670 in_seq=0 out_seq=0 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 1 -- 172.21.15.80:0/707979145 <== mds.0 v2:172.21.15.80:6833/1134081162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a723505be0 con 0x55a7232127d0 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6108ff9640 10 client.8553 handle_command_reply: tid=0 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 client.8553 shutdown 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 -- 172.21.15.80:0/707979145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 msgr2=0x55a72372a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.526+0000 7f6137148000 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72372a0c0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f612000f4b0 tx=0x7f612000f970 comp rx=0 tx=0).stop 2024-04-25T13:48:16.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 1 -- 172.21.15.80:0/707979145 shutdown_connections 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72372a0c0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7235f5ea0 0x55a72376f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] conn(0x55a7232127d0 0x55a72372ae00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 1 --2- 172.21.15.80:0/707979145 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 0x55a72376ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 1 -- 172.21.15.80:0/707979145 >> 172.21.15.80:0/707979145 conn(0x55a72322dff0 msgr2=0x55a72371dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6108ff9640 0 client.8553 ms_handle_reset on v2:172.21.15.80:6833/1134081162 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 1 -- 172.21.15.80:0/707979145 shutdown_connections 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 1 -- 172.21.15.80:0/707979145 wait complete. 2024-04-25T13:48:16.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.530+0000 7f6137148000 20 client.8553 trim_cache size 0 max 16384 2024-04-25T13:48:16.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f6137148000 1 Processor -- start 2024-04-25T13:48:16.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f6137148000 1 -- start start 2024-04-25T13:48:16.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f6137148000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a72377f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f6137148000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a72377dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f6137148000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72377e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f6137148000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f612815ac90 con 0x55a723609c20 2024-04-25T13:48:16.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f6137148000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6128147670 con 0x55a7235f5ea0 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f6137148000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f612815a640 con 0x55a7231ee280 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72377e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72377e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50680/0 (socket says 172.21.15.80:50680) 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 -- 172.21.15.80:0/2501048040 learned_addr learned my addr 172.21.15.80:0/2501048040 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612d386640 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a72377dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 -- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 msgr2=0x55a72377f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612db87640 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a72377f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a72377f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 -- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 msgr2=0x55a72377dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a72377dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 -- 172.21.15.80:0/2501048040 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a72372a430 con 0x55a723609c20 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612d386640 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a72377dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.546+0000 7f612e388640 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72377e530 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6120010080 tx=0x7f612000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f583c2b164bb6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f610affd640 1 -- 172.21.15.80:0/2501048040 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120003b90 con 0x55a723609c20 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f610affd640 1 -- 172.21.15.80:0/2501048040 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6120018070 con 0x55a723609c20 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f610affd640 1 -- 172.21.15.80:0/2501048040 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120005690 con 0x55a723609c20 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- 172.21.15.80:0/2501048040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 msgr2=0x55a72377e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72377e530 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f6120010080 tx=0x7f612000a000 comp rx=0 tx=0).stop 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- 172.21.15.80:0/2501048040 shutdown_connections 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a723609c20 0x55a72377e530 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7235f5ea0 0x55a72377dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 --2- 172.21.15.80:0/2501048040 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a7231ee280 0x55a72377f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- 172.21.15.80:0/2501048040 >> 172.21.15.80:0/2501048040 conn(0x55a72322dff0 msgr2=0x55a723725040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- 172.21.15.80:0/2501048040 shutdown_connections 2024-04-25T13:48:16.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- 172.21.15.80:0/2501048040 wait complete. 2024-04-25T13:48:16.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- start start 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 Processor -- start 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 0x55a72377f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a7235f5ea0 0x55a72379a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a723609c20 0x55a72379af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55a72356c130 con 0x55a7235f5ea0 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55a7235027d0 con 0x55a7231ee280 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55a723571450 con 0x55a723609c20 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6137148000 10 client.0 _ll_register_callbacks cb 0x7f6120001570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a7235f5ea0 0x55a72379a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a7235f5ea0 0x55a72379a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50682/0 (socket says 172.21.15.80:50682) 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 -- 172.21.15.80:0/1523246824 learned_addr learned my addr 172.21.15.80:0/1523246824 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612db87640 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 0x55a72377f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612e388640 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a723609c20 0x55a72379af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 -- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a723609c20 msgr2=0x55a72379af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a723609c20 0x55a72379af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 -- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 msgr2=0x55a72377f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 0x55a72377f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 -- 172.21.15.80:0/1523246824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a723525c70 con 0x55a7235f5ea0 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612e388640 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a723609c20 0x55a72379af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f612d386640 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a7235f5ea0 0x55a72379a9c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f61180af040 tx=0x7f611801ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba54488e7d80b102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.550+0000 7f6108ff9640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 1 -- 172.21.15.80:0/1523246824 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61180c8070 con 0x55a7235f5ea0 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 1 -- 172.21.15.80:0/1523246824 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7f61180b79c0 con 0x55a7235f5ea0 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 -- 172.21.15.80:0/1523246824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6120018070 con 0x55a7235f5ea0 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 1 -- 172.21.15.80:0/1523246824 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f61180d3050 con 0x55a7235f5ea0 2024-04-25T13:48:16.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 10 client.8559 fetch_fsmap learned FSMap version 74 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 -- 172.21.15.80:0/1523246824 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) -- 0x55a7235ef180 con 0x55a7235f5ea0 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 1 -- 172.21.15.80:0/1523246824 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6120018070 con 0x55a7235f5ea0 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 1 -- 172.21.15.80:0/1523246824 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 74) ==== 3494+0+0 (secure 0 0 0) 0x7f61180d6020 con 0x55a7235f5ea0 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 10 client.8559 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 10 client.8559 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 20 client.8559 populate_metadata read hostname 'smithi080' 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] conn(0x55a723200830 0x55a7237de420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 4 client.8559 mds_command: new command op to 8412 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 -- 172.21.15.80:0/1523246824 --> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] -- command(tid 0: {"prefix": "status"}) -- 0x55a72357aa30 con 0x55a723200830 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612db87640 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] conn(0x55a723200830 0x55a7237de420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 10 client.8559 ms_handle_connect on v2:172.21.15.80:6833/1134081162 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612db87640 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] conn(0x55a723200830 0x55a7237de420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daf93577d2694dca server_cookie=e6affec0be3fe3e8 in_seq=0 out_seq=0 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 1 -- 172.21.15.80:0/1523246824 <== mds.0 v2:172.21.15.80:6833/1134081162 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a72357aa30 con 0x55a723200830 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 10 client.8559 handle_command_reply: tid=0 2024-04-25T13:48:16.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 client.8559 shutdown 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 -- 172.21.15.80:0/1523246824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a7235f5ea0 msgr2=0x55a72379a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a7235f5ea0 0x55a72379a9c0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f61180af040 tx=0x7f611801ab90 comp rx=0 tx=0).stop 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 -- 172.21.15.80:0/1523246824 shutdown_connections 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55a723609c20 0x55a72379af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55a7235f5ea0 0x55a72379a9c0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.80:6833/1134081162,v1:172.21.15.80:6835/1134081162] conn(0x55a723200830 0x55a7237de420 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 --2- 172.21.15.80:0/1523246824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55a7231ee280 0x55a72377f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6108ff9640 0 client.8559 ms_handle_reset on v2:172.21.15.80:6833/1134081162 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 -- 172.21.15.80:0/1523246824 >> 172.21.15.80:0/1523246824 conn(0x55a72322dff0 msgr2=0x55a723724610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 -- 172.21.15.80:0/1523246824 shutdown_connections 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 1 -- 172.21.15.80:0/1523246824 wait complete. 2024-04-25T13:48:16.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f6137148000 20 client.8559 trim_cache size 0 max 16384 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 8412, 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 6.0030732200000001, 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 74, 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 154, 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 154, 2024-04-25T13:48:16.558 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 11.963587555 2024-04-25T13:48:16.559 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:48:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 -- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f610c06f400 msgr2=0x7f610c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f610c06f400 0x7f610c071910 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f61180ae000 tx=0x7f61180a6070 comp rx=0 tx=0).stop 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 -- 172.21.15.80:0/432725389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f612806e9e0 msgr2=0x7f6128140e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f612806e9e0 0x7f6128140e60 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f612400c410 tx=0x7f612400c8d0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 -- 172.21.15.80:0/432725389 shutdown_connections 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f610c06f400 0x7f610c071910 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6128161f90 0x7f61281693b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f61280d06e0 0x7f61280abd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 --2- 172.21.15.80:0/432725389 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f612806e9e0 0x7f6128140e60 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 -- 172.21.15.80:0/432725389 >> 172.21.15.80:0/432725389 conn(0x7f612806d080 msgr2=0x7f61280b2330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 -- 172.21.15.80:0/432725389 shutdown_connections 2024-04-25T13:48:16.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.554+0000 7f612fdeb640 1 -- 172.21.15.80:0/432725389 wait complete. 2024-04-25T13:48:16.572 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:48:16.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fda01723640 1 Processor -- start 2024-04-25T13:48:16.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fda01723640 1 -- start start 2024-04-25T13:48:16.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fda01723640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc164870 0x7fd9fc0d04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fda01723640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9fc068ca0 0x7fd9fc06ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.705 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fda01723640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9fc06ece0 con 0x7fd9fc160400 2024-04-25T13:48:16.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fda01723640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd9fc164de0 con 0x7fd9fc068ca0 2024-04-25T13:48:16.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fda01723640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9fc06c5f0 con 0x7fd9fc164870 2024-04-25T13:48:16.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fd9faffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc164870 0x7fd9fc0d04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fd9fb7fe640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9fc068ca0 0x7fd9fc06ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fd9faffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc164870 0x7fd9fc0d04e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:33492/0 (socket says 172.21.15.80:33492) 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fd9faffd640 1 -- 172.21.15.80:0/935017465 learned_addr learned my addr 172.21.15.80:0/935017465 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.706+0000 7fd9fa7fc640 1 --1- 172.21.15.80:0/935017465 >> v1:172.21.15.87:6789/0 conn(0x7fd9fc160400 0x7fd9fc068590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:38512/0 (socket says 172.21.15.80:38512) 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9faffd640 1 -- 172.21.15.80:0/935017465 >> v2:172.21.15.87:3300/0 conn(0x7fd9fc068ca0 msgr2=0x7fd9fc06ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9faffd640 1 --2- 172.21.15.80:0/935017465 >> v2:172.21.15.87:3300/0 conn(0x7fd9fc068ca0 0x7fd9fc06ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9faffd640 1 -- 172.21.15.80:0/935017465 >> v1:172.21.15.87:6789/0 conn(0x7fd9fc160400 legacy=0x7fd9fc068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9faffd640 1 -- 172.21.15.80:0/935017465 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc06efc0 con 0x7fd9fc164870 2024-04-25T13:48:16.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9fb7fe640 1 --2- 172.21.15.80:0/935017465 >> v2:172.21.15.87:3300/0 conn(0x7fd9fc068ca0 0x7fd9fc06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:48:16.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9faffd640 1 --2- 172.21.15.80:0/935017465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc164870 0x7fd9fc0d04e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009870 tx=0x7fd9ec01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8eab0376c071798 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9f9ffb640 1 -- 172.21.15.80:0/935017465 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0ab430 con 0x7fd9fc164870 2024-04-25T13:48:16.709 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9f9ffb640 1 -- 172.21.15.80:0/935017465 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9ec0b0070 con 0x7fd9fc164870 2024-04-25T13:48:16.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.710+0000 7fd9f9ffb640 1 -- 172.21.15.80:0/935017465 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0ab7f0 con 0x7fd9fc164870 2024-04-25T13:48:16.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- 172.21.15.80:0/935017465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc164870 msgr2=0x7fd9fc0d04e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 --2- 172.21.15.80:0/935017465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc164870 0x7fd9fc0d04e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec009870 tx=0x7fd9ec01c410 comp rx=0 tx=0).stop 2024-04-25T13:48:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- 172.21.15.80:0/935017465 shutdown_connections 2024-04-25T13:48:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 --2- 172.21.15.80:0/935017465 >> v2:172.21.15.87:3300/0 conn(0x7fd9fc068ca0 0x7fd9fc06ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 --2- 172.21.15.80:0/935017465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc164870 0x7fd9fc0d04e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- 172.21.15.80:0/935017465 >> 172.21.15.80:0/935017465 conn(0x7fd9fc0cf390 msgr2=0x7fd9fc0cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- 172.21.15.80:0/935017465 shutdown_connections 2024-04-25T13:48:16.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- 172.21.15.80:0/935017465 wait complete. 2024-04-25T13:48:16.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 Processor -- start 2024-04-25T13:48:16.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- start start 2024-04-25T13:48:16.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc160400 0x7fd9fc13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9fc164870 0x7fd9fc15df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9fc15e4a0 0x7fd9fc15e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc070c80 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd9fc070b00 con 0x7fd9fc164870 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.714+0000 7fda01723640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9fc16da10 con 0x7fd9fc160400 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9fc15e4a0 0x7fd9fc15e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9fc15e4a0 0x7fd9fc15e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50684/0 (socket says 172.21.15.80:50684) 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 -- 172.21.15.80:0/2932767474 learned_addr learned my addr 172.21.15.80:0/2932767474 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9faffd640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc160400 0x7fd9fc13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fa7fc640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9fc164870 0x7fd9fc15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 -- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc160400 msgr2=0x7fd9fc13fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc160400 0x7fd9fc13fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 -- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9fc164870 msgr2=0x7fd9fc15df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9fc164870 0x7fd9fc15df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 -- 172.21.15.80:0/2932767474 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9fc163090 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9faffd640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc160400 0x7fd9fc13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9fb7fe640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9fc15e4a0 0x7fd9fc15e8b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009e00 tx=0x7fd9f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9e37fe640 1 -- 172.21.15.80:0/2932767474 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4018020 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9e37fe640 1 -- 172.21.15.80:0/2932767474 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9f4014480 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9fc163370 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9e37fe640 1 -- 172.21.15.80:0/2932767474 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4014870 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9fc13ff60 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9fc06a3a0 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9e37fe640 1 -- 172.21.15.80:0/2932767474 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd9f401b020 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9e37fe640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9e806f430 0x7fd9e8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:16.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9e37fe640 1 -- 172.21.15.80:0/2932767474 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5709+0+0 (secure 0 0 0) 0x7fd9f408ff30 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.718+0000 7fd9faffd640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9e806f430 0x7fd9e8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:16.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.722+0000 7fd9faffd640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9e806f430 0x7fd9e8071940 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0af000 tx=0x7fd9ec0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:16.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.726+0000 7fd9e37fe640 1 -- 172.21.15.80:0/2932767474 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd9f4022020 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.918+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd9fc160c80 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.918+0000 7fd9e37fe640 1 -- 172.21.15.80:0/2932767474 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7fd9f4010040 con 0x7fd9fc15e4a0 2024-04-25T13:48:16.917 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:48:16.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9e806f430 msgr2=0x7fd9e8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9e806f430 0x7fd9e8071940 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0af000 tx=0x7fd9ec0a7000 comp rx=0 tx=0).stop 2024-04-25T13:48:16.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9fc15e4a0 msgr2=0x7fd9fc15e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:16.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9fc15e4a0 0x7fd9fc15e8b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009e00 tx=0x7fd9f400c490 comp rx=0 tx=0).stop 2024-04-25T13:48:16.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 shutdown_connections 2024-04-25T13:48:16.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd9e806f430 0x7fd9e8071940 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9fc15e4a0 0x7fd9fc15e8b0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9fc164870 0x7fd9fc15df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 --2- 172.21.15.80:0/2932767474 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9fc160400 0x7fd9fc13fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:16.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 >> 172.21.15.80:0/2932767474 conn(0x7fd9fc0cf390 msgr2=0x7fd9fc0b8510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:16.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 shutdown_connections 2024-04-25T13:48:16.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:16.922+0000 7fda01723640 1 -- 172.21.15.80:0/2932767474 wait complete. 2024-04-25T13:48:16.938 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:48:16.938 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:48:16.938 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:48:16.938 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:48:16.939 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:48:16.939 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:48:16.939 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:48:16.939 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:48:16.939 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:48:16.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:16.939 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:48:16.944 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:48:16.944 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:48:16.944 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:48:16.944 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:16.944 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:48:16.944 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 42071sec preferred_lft 42071sec 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:16.945 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:48:16.946 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:16.947 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:48:16.947 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:48:16.947 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:48:16.947 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:16.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:16.947 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:48:16.997 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:48:16.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:16.998 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:48:17.051 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:48:17.051 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:48:17.051 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:48:17.051 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:48:17.051 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:48:17.051 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:48:17.052 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:48:17.052 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:48:17.052 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:48:17.052 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:48:17.052 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:48:17.052 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:48:17.052 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:48:17.052 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:48:17.052 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:48:17.052 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:48:17.052 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:48:17.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:48:17.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:48:17.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:17.170 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:48:17.219 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:48:17.219 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:48:17.220 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:48:17.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:17.220 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:48:17.275 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:48:17.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:48:17.275 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:48:17.323 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:48:17.323 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:48:17.323 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:48:17.371 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:48:17.428 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:48:17.477 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:48:17.477 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:48:17.478 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:48:17.479 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:48:17.480 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:48:17.480 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:48:17.480 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:48:17.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:17.481 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:48:17.527 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:48:17.528 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:48:17.572 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:48:17.587 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:48:17.635 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:48:17.636 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:48:17.637 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:48:17.638 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:48:17.638 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:48:17.638 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:48:17.638 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:48:17.638 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:48:17.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:17.638 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:48:17.681 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:48:17.682+0000 7f4edf208580 -1 init, newargv = 0x7f4ed000a100 newargc=15 2024-04-25T13:48:17.681 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[40673]: starting ceph client 2024-04-25T13:48:17.687 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[40673]: starting fuse 2024-04-25T13:48:18.784 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:48:18.801 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:48:18.848 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:48:18.848 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:48:18.848 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:48:18.848 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:48:18.848 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:48:18.848 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:48:18.848 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:48:18.848 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:48:18.849 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:48:18.850 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:48:18.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:18.852 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:48:18.899 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:48:18.899 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:48:18.900 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:48:18.901 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:48:19.023 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.40673.asok 2024-04-25T13:48:19.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:48:19.036 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.40673.asok 2024-04-25T13:48:19.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:19.036 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.40673.asok status 2024-04-25T13:48:19.208 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "40673", 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 8565, 2024-04-25T13:48:19.209 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 8565 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 884097213 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:48:19.210 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 884097213 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.8565 192.168.0.1:0/884097213", 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/884097213", 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 74, 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 154, 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:48:19.211 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:48:19.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:19.228 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:48:19.228 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:48:19.234 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:48:19.234 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:48:19.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:19.234 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:48:19.291 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:48:19.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b9a0a8640 1 Processor -- start 2024-04-25T13:48:19.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b9a0a8640 1 -- start start 2024-04-25T13:48:19.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b9a0a8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b940b5760 0x7f9b94156d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:19.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b9a0a8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b94162170 con 0x7f9b94161d60 2024-04-25T13:48:19.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b9a0a8640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b940af1b0 con 0x7f9b94163980 2024-04-25T13:48:19.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b9a0a8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b941619d0 con 0x7f9b940b5760 2024-04-25T13:48:19.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b8bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9b94161d60 0x7f9b940b5050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53522/0 (socket says 172.21.15.80:53522) 2024-04-25T13:48:19.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b8bfff640 1 -- 172.21.15.80:0/875569558 learned_addr learned my addr 172.21.15.80:0/875569558 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:19.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b93fff640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b940b5760 0x7f9b94156d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:19.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 792036638 0 0) 0x7f9b7c001120 con 0x7f9b94161d60 2024-04-25T13:48:19.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9b80003530 con 0x7f9b94161d60 2024-04-25T13:48:19.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3365732870 0 0) 0x7f9b80003530 con 0x7f9b94161d60 2024-04-25T13:48:19.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b940b5760 msgr2=0x7f9b94156d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:19.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b940b5760 0x7f9b94156d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 >> v1:172.21.15.87:6789/0 conn(0x7f9b94163980 legacy=0x7f9b940b6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:19.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b940af490 con 0x7f9b94161d60 2024-04-25T13:48:19.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f9b7c001850 con 0x7f9b94161d60 2024-04-25T13:48:19.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 >> v1:172.21.15.80:6789/0 conn(0x7f9b94161d60 legacy=0x7f9b940b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:19.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b800039f0 0x7f9b80003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b80004810 0x7f9b80004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b800051b0 0x7f9b8000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9b941619d0 con 0x7f9b800039f0 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8000bf10 con 0x7f9b80004810 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8000c110 con 0x7f9b800051b0 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b937fe640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b800039f0 0x7f9b80003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b8bfff640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b80004810 0x7f9b80004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b93fff640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b800051b0 0x7f9b8000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b937fe640 1 -- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b800051b0 msgr2=0x7f9b8000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b937fe640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b800051b0 0x7f9b8000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b937fe640 1 -- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b80004810 msgr2=0x7f9b80004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:19.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b937fe640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b80004810 0x7f9b80004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.490+0000 7f9b937fe640 1 -- 172.21.15.80:0/875569558 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b80003530 con 0x7f9b800039f0 2024-04-25T13:48:19.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b937fe640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b800039f0 0x7f9b80003e60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9b84001750 tx=0x7f9b8401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=345c1f4246a318a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:19.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b840013a0 con 0x7f9b800039f0 2024-04-25T13:48:19.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b840052f0 con 0x7f9b800039f0 2024-04-25T13:48:19.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b92ffd640 1 -- 172.21.15.80:0/875569558 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b84003840 con 0x7f9b800039f0 2024-04-25T13:48:19.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/875569558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b800039f0 msgr2=0x7f9b80003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:19.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b9a0a8640 1 --2- 172.21.15.80:0/875569558 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b800039f0 0x7f9b80003e60 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f9b84001750 tx=0x7f9b8401c3e0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b93fff640 1 -- 172.21.15.80:0/875569558 reap_dead start 2024-04-25T13:48:19.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/875569558 shutdown_connections 2024-04-25T13:48:19.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/875569558 >> 172.21.15.80:0/875569558 conn(0x7f9b94072950 msgr2=0x7f9b94073b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:19.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/875569558 shutdown_connections 2024-04-25T13:48:19.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.494+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/875569558 wait complete. 2024-04-25T13:48:19.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 Processor -- start 2024-04-25T13:48:19.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 -- start start 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b800039f0 0x7f9b9414c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b94154f50 0x7f9b9406ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b94154540 0x7f9b9406ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9b940b5970 con 0x7f9b94154f50 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9b94163d90 con 0x7f9b800039f0 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9b94157510 con 0x7f9b94154540 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b94154f50 0x7f9b9406ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b94154f50 0x7f9b9406ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:50700/0 (socket says 172.21.15.80:50700) 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 -- 172.21.15.80:0/2403276325 learned_addr learned my addr 172.21.15.80:0/2403276325 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:19.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b93fff640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b94154540 0x7f9b9406ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b937fe640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b800039f0 0x7f9b9414c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 -- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b94154540 msgr2=0x7f9b9406ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b94154540 0x7f9b9406ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 -- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b800039f0 msgr2=0x7f9b9414c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b800039f0 0x7f9b9414c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 -- 172.21.15.80:0/2403276325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b9414d9e0 con 0x7f9b94154f50 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b93fff640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b94154540 0x7f9b9406ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b937fe640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b800039f0 0x7f9b9414c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b8bfff640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b94154f50 0x7f9b9406ea00 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00e7b0 tx=0x7f9b7c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b90ff9640 1 -- 172.21.15.80:0/2403276325 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00c660 con 0x7f9b94154f50 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b90ff9640 1 -- 172.21.15.80:0/2403276325 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b7c018070 con 0x7f9b94154f50 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b9414dcc0 con 0x7f9b94154f50 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b90ff9640 1 -- 172.21.15.80:0/2403276325 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00ca40 con 0x7f9b94154f50 2024-04-25T13:48:19.499 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b9414e280 con 0x7f9b94154f50 2024-04-25T13:48:19.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.498+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b9406a3a0 con 0x7f9b94154f50 2024-04-25T13:48:19.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.502+0000 7f9b90ff9640 1 -- 172.21.15.80:0/2403276325 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9b7c00cbe0 con 0x7f9b94154f50 2024-04-25T13:48:19.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.502+0000 7f9b90ff9640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b7406f430 0x7f9b74071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:19.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.502+0000 7f9b90ff9640 1 -- 172.21.15.80:0/2403276325 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5767+0+0 (secure 0 0 0) 0x7f9b7c08fa00 con 0x7f9b94154f50 2024-04-25T13:48:19.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.502+0000 7f9b937fe640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b7406f430 0x7f9b74071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:19.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.502+0000 7f9b937fe640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b7406f430 0x7f9b74071940 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9b80004790 tx=0x7f9b840011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:19.506 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.506+0000 7f9b90ff9640 1 -- 172.21.15.80:0/2403276325 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9b94162170 con 0x7f9b94154f50 2024-04-25T13:48:19.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f9b940d2e70 con 0x7f9b94154f50 2024-04-25T13:48:19.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b90ff9640 1 -- 172.21.15.80:0/2403276325 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) ==== 53+0+73 (secure 0 0 0) 0x7f9b7c05ca20 con 0x7f9b94154f50 2024-04-25T13:48:19.707 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:48:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b7406f430 msgr2=0x7f9b74071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b7406f430 0x7f9b74071940 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9b80004790 tx=0x7f9b840011a0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b94154f50 msgr2=0x7f9b9406ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b94154f50 0x7f9b9406ea00 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00e7b0 tx=0x7f9b7c00ec70 comp rx=0 tx=0).stop 2024-04-25T13:48:19.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 shutdown_connections 2024-04-25T13:48:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9b7406f430 0x7f9b74071940 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9b94154540 0x7f9b9406ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9b94154f50 0x7f9b9406ea00 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 --2- 172.21.15.80:0/2403276325 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9b800039f0 0x7f9b9414c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 >> 172.21.15.80:0/2403276325 conn(0x7f9b94072950 msgr2=0x7f9b94074260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 shutdown_connections 2024-04-25T13:48:19.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:19.710+0000 7f9b9a0a8640 1 -- 172.21.15.80:0/2403276325 wait complete. 2024-04-25T13:48:19.730 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:48:19.730 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:48:19.730 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:48:19.730 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:48:19.730 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:48:19.730 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:48:19.730 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:48:19.731 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:48:19.731 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:48:19.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:19.731 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:48:19.738 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:19.739 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42069sec preferred_lft 42069sec 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:19.740 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:48:19.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:19.741 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:48:19.795 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:48:19.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:19.797 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:48:19.849 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:48:19.850 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:48:19.851 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:48:19.851 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:48:19.851 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:48:19.851 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:48:19.851 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:48:19.851 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:48:19.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:48:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:48:19.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:48:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:48:19.972 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:19.972 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:48:19.978 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:48:19.978 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:48:19.979 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:48:19.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:19.979 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:48:20.032 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:48:20.033 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:48:20.033 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:48:20.080 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:48:20.080 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:48:20.080 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:48:20.129 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:48:20.186 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:48:20.234 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:48:20.234 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:48:20.234 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:48:20.235 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:48:20.236 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:48:20.236 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:48:20.236 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:48:20.236 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:48:20.236 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:48:20.236 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:48:20.236 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:48:20.236 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:48:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:48:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:48:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:48:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:48:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:48:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:48:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:48:20.237 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:48:20.238 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:48:20.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:20.239 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:48:20.288 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:48:20.288 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:48:20.334 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:48:20.349 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:48:20.396 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:48:20.396 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:48:20.396 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:48:20.397 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:48:20.398 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:48:20.399 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:48:20.399 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:48:20.399 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:48:20.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:20.400 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:48:20.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:48:20.447+0000 7f30e25da580 -1 init, newargv = 0x7f30c401b4e0 newargc=15 2024-04-25T13:48:20.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25409]: starting ceph client 2024-04-25T13:48:20.453 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25409]: starting fuse 2024-04-25T13:48:21.553 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:48:21.570 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:48:21.617 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:48:21.617 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:48:21.617 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:48:21.618 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:48:21.619 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:48:21.620 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:48:21.620 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:48:21.620 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:48:21.620 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:48:21.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:21.621 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:48:21.668 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:48:21.668 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:48:21.668 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:48:21.669 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:48:21.670 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:48:21.791 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.25409.asok 2024-04-25T13:48:21.795 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:48:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:48:21.803 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.25409.asok 2024-04-25T13:48:21.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:21.803 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.25409.asok status 2024-04-25T13:48:21.980 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:48:21.980 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:48:21.980 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:48:21.981 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:48:21.981 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:48:21.981 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:48:21.981 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:48:21.981 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "25409", 2024-04-25T13:48:21.981 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:48:21.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:48:21.982 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:48:21.982 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:48:21.982 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 6200, 2024-04-25T13:48:21.982 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:48:21.982 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:48:21.982 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 6200 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2205966170 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 2205966170 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.6200 192.168.0.1:0/2205966170", 2024-04-25T13:48:21.983 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/2205966170", 2024-04-25T13:48:21.984 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:48:21.984 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 74, 2024-04-25T13:48:21.984 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 156, 2024-04-25T13:48:21.984 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:48:21.984 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:48:21.984 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:48:21.984 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:48:21.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:21.996 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:48:21.996 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:48:22.002 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:48:22.002 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:48:22.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:48:22.003 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:48:22.055 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:48:22.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.214+0000 7fc78b159640 1 Processor -- start 2024-04-25T13:48:22.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.214+0000 7fc78b159640 1 -- start start 2024-04-25T13:48:22.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc78b159640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc784161240 0x7fc7840723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc78b159640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc784161650 0x7fc7840728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc78b159640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc78406fcf0 con 0x7fc78406f5a0 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc78b159640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc784073f10 con 0x7fc784161240 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc78b159640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7840b0c60 con 0x7fc784161650 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc788ef5640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc78406f5a0 0x7fc784160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45458/0 (socket says 172.21.15.80:45458) 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc788ef5640 1 -- 172.21.15.80:0/3710856970 learned_addr learned my addr 172.21.15.80:0/3710856970 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3447444143 0 0) 0x7fc78406fcf0 con 0x7fc78406f5a0 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc770003530 con 0x7fc78406f5a0 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc783fff640 1 --2- 172.21.15.80:0/3710856970 >> v2:172.21.15.87:3300/0 conn(0x7fc784161240 0x7fc7840723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7896f6640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc784161650 0x7fc7840728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2167247926 0 0) 0x7fc770003530 con 0x7fc78406f5a0 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc784161650 msgr2=0x7fc7840728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc784161650 0x7fc7840728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 >> v2:172.21.15.87:3300/0 conn(0x7fc784161240 msgr2=0x7fc7840723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 --2- 172.21.15.80:0/3710856970 >> v2:172.21.15.87:3300/0 conn(0x7fc784161240 0x7fc7840723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7840b0f40 con 0x7fc78406f5a0 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc76c001680 con 0x7fc78406f5a0 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 >> v1:172.21.15.80:6789/0 conn(0x7fc78406f5a0 legacy=0x7fc784160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039c0 0x7fc770003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc770004810 0x7fc770004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc784161650 0x7fc7700051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc784073f10 con 0x7fc770004810 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7840b0c60 con 0x7fc7700039c0 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc77000bb30 con 0x7fc784161650 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc783fff640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc770004810 0x7fc770004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc788ef5640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039c0 0x7fc770003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc7896f6640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc784161650 0x7fc7700051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc783fff640 1 -- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc784161650 msgr2=0x7fc7700051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc783fff640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc784161650 0x7fc7700051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc783fff640 1 -- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039c0 msgr2=0x7fc770003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc783fff640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7700039c0 0x7fc770003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.218+0000 7fc783fff640 1 -- 172.21.15.80:0/3710856970 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc770003530 con 0x7fc770004810 2024-04-25T13:48:22.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc783fff640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc770004810 0x7fc770004c20 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc774009ae0 tx=0x7fc77401cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e4568b764bc68d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740096b0 con 0x7fc770004810 2024-04-25T13:48:22.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7740a6510 con 0x7fc770004810 2024-04-25T13:48:22.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc7837fe640 1 -- 172.21.15.80:0/3710856970 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7740a6900 con 0x7fc770004810 2024-04-25T13:48:22.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc78b159640 1 -- 172.21.15.80:0/3710856970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc770004810 msgr2=0x7fc770004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc78b159640 1 --2- 172.21.15.80:0/3710856970 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc770004810 0x7fc770004c20 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc774009ae0 tx=0x7fc77401cab0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc7896f6640 1 -- 172.21.15.80:0/3710856970 reap_dead start 2024-04-25T13:48:22.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc78b159640 1 -- 172.21.15.80:0/3710856970 shutdown_connections 2024-04-25T13:48:22.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc78b159640 1 -- 172.21.15.80:0/3710856970 >> 172.21.15.80:0/3710856970 conn(0x7fc78406d080 msgr2=0x7fc7840b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc78b159640 1 -- 172.21.15.80:0/3710856970 shutdown_connections 2024-04-25T13:48:22.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.222+0000 7fc78b159640 1 -- 172.21.15.80:0/3710856970 wait complete. 2024-04-25T13:48:22.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 Processor -- start 2024-04-25T13:48:22.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 -- start start 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc770004810 0x7fc784151560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78415e710 0x7fc78415ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc78415ae30 0x7fc78415b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc784073480 con 0x7fc78415ae30 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc784073300 con 0x7fc78415e710 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc784073600 con 0x7fc770004810 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc78415ae30 0x7fc78415b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc78415ae30 0x7fc78415b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55812/0 (socket says 172.21.15.80:55812) 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 -- 172.21.15.80:0/1589763265 learned_addr learned my addr 172.21.15.80:0/1589763265 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:22.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc783fff640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78415e710 0x7fc78415ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc788ef5640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc770004810 0x7fc784151560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 -- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc770004810 msgr2=0x7fc784151560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc770004810 0x7fc784151560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 -- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78415e710 msgr2=0x7fc78415ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78415e710 0x7fc78415ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 -- 172.21.15.80:0/1589763265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc78415b780 con 0x7fc78415ae30 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc788ef5640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc770004810 0x7fc784151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7896f6640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc78415ae30 0x7fc78415b240 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc7840701f0 tx=0x7fc77c00e510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7817fa640 1 -- 172.21.15.80:0/1589763265 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c00ee20 con 0x7fc78415ae30 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7817fa640 1 -- 172.21.15.80:0/1589763265 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc77c01a070 con 0x7fc78415ae30 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7841fa460 con 0x7fc78415ae30 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc7817fa640 1 -- 172.21.15.80:0/1589763265 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc77c015650 con 0x7fc78415ae30 2024-04-25T13:48:22.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc78415d950 con 0x7fc78415ae30 2024-04-25T13:48:22.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.226+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc78406a3a0 con 0x7fc78415ae30 2024-04-25T13:48:22.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.230+0000 7fc7817fa640 1 -- 172.21.15.80:0/1589763265 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc77c012030 con 0x7fc78415ae30 2024-04-25T13:48:22.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.230+0000 7fc7817fa640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc76406f430 0x7fc764071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.230+0000 7fc7817fa640 1 -- 172.21.15.80:0/1589763265 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7fc77c08f990 con 0x7fc78415ae30 2024-04-25T13:48:22.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.230+0000 7fc788ef5640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc76406f430 0x7fc764071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.230+0000 7fc788ef5640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc76406f430 0x7fc764071940 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc76c001060 tx=0x7fc76c001160 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.234+0000 7fc7817fa640 1 -- 172.21.15.80:0/1589763265 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc78415b780 con 0x7fc78415ae30 2024-04-25T13:48:22.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.430+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7840d1020 con 0x7fc78415ae30 2024-04-25T13:48:22.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.430+0000 7fc7817fa640 1 -- 172.21.15.80:0/1589763265 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7fc77c010040 con 0x7fc78415ae30 2024-04-25T13:48:22.430 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:48:22.430 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/521602421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":521602421},{"type":"v1","addr":"172.21.15.87:6835","nonce":521602421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4170043750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4170043750},{"type":"v1","addr":"172.21.15.87:6837","nonce":4170043750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/1423926990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1423926990},{"type":"v1","addr":"172.21.15.80:6834","nonce":1423926990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:48:09.907104+0000","modified":"2024-04-25T13:48:12.977708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1134081162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1134081162},{"type":"v1","addr":"172.21.15.80:6835","nonce":1134081162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-25T13:48:22.430 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 74 2024-04-25T13:48:22.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc76406f430 msgr2=0x7fc764071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc76406f430 0x7fc764071940 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fc76c001060 tx=0x7fc76c001160 comp rx=0 tx=0).stop 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc78415ae30 msgr2=0x7fc78415b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc78415ae30 0x7fc78415b240 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc7840701f0 tx=0x7fc77c00e510 comp rx=0 tx=0).stop 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 shutdown_connections 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc76406f430 0x7fc764071940 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc78415ae30 0x7fc78415b240 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc78415e710 0x7fc78415ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 --2- 172.21.15.80:0/1589763265 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc770004810 0x7fc784151560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 >> 172.21.15.80:0/1589763265 conn(0x7fc78406d080 msgr2=0x7fc7840b1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 shutdown_connections 2024-04-25T13:48:22.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.434+0000 7fc78b159640 1 -- 172.21.15.80:0/1589763265 wait complete. 2024-04-25T13:48:22.452 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:48:22.618 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":8412,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.052779326,"mdsmap_epoch":74,"osdmap_epoch":156,"osdmap_epoch_barrier":154,"uptime":18.013306493999998} 2024-04-25T13:48:22.618 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 8412, "mdsmap_epoch": 74, "osdmap_epoch": 156, "osdmap_epoch_barrier": 154, "rank_uptime": 12.052779326, "state": "up:active", "uptime": 18.013306494, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:48:22.619 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-25T13:48:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.774+0000 7f39685ed640 1 Processor -- start 2024-04-25T13:48:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.774+0000 7f39685ed640 1 -- start start 2024-04-25T13:48:22.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f39685ed640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f39600b0f10 0x7f39600d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f39685ed640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3960161d10 0x7f3960162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f39685ed640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f39600b1320 con 0x7f39600ce2a0 2024-04-25T13:48:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f39685ed640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f39600b5820 con 0x7f3960161d10 2024-04-25T13:48:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f39685ed640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f39600d13f0 con 0x7f39600b0f10 2024-04-25T13:48:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3966b8a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3960161d10 0x7f3960162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3965b88640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f39600ce2a0 0x7f39600ce6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45466/0 (socket says 172.21.15.80:45466) 2024-04-25T13:48:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3965b88640 1 -- 172.21.15.80:0/1491227743 learned_addr learned my addr 172.21.15.80:0/1491227743 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:48:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3966389640 1 --2- 172.21.15.80:0/1491227743 >> v2:172.21.15.87:3300/0 conn(0x7f39600b0f10 0x7f39600d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3966b8a640 1 -- 172.21.15.80:0/1491227743 >> v2:172.21.15.87:3300/0 conn(0x7f39600b0f10 msgr2=0x7f39600d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3966b8a640 1 --2- 172.21.15.80:0/1491227743 >> v2:172.21.15.87:3300/0 conn(0x7f39600b0f10 0x7f39600d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3966b8a640 1 -- 172.21.15.80:0/1491227743 >> v1:172.21.15.80:6789/0 conn(0x7f39600ce2a0 legacy=0x7f39600ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3965387640 1 -- 172.21.15.80:0/1491227743 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 210448694 0 0) 0x7f3950001120 con 0x7f39600ce2a0 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3966b8a640 1 -- 172.21.15.80:0/1491227743 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39600d1150 con 0x7f3960161d10 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3966b8a640 1 --2- 172.21.15.80:0/1491227743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3960161d10 0x7f3960162120 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f39600713a0 tx=0x7f395801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9791256116520e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3965387640 1 -- 172.21.15.80:0/1491227743 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580b2070 con 0x7f3960161d10 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3965387640 1 -- 172.21.15.80:0/1491227743 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39580a58e0 con 0x7f3960161d10 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.778+0000 7f3965387640 1 -- 172.21.15.80:0/1491227743 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39580a5cb0 con 0x7f3960161d10 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- 172.21.15.80:0/1491227743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3960161d10 msgr2=0x7f3960162120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 --2- 172.21.15.80:0/1491227743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3960161d10 0x7f3960162120 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f39600713a0 tx=0x7f395801c930 comp rx=0 tx=0).stop 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- 172.21.15.80:0/1491227743 shutdown_connections 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 --2- 172.21.15.80:0/1491227743 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3960161d10 0x7f3960162120 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 --2- 172.21.15.80:0/1491227743 >> v2:172.21.15.87:3300/0 conn(0x7f39600b0f10 0x7f39600d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- 172.21.15.80:0/1491227743 >> 172.21.15.80:0/1491227743 conn(0x7f39600b35c0 msgr2=0x7f39600b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- 172.21.15.80:0/1491227743 shutdown_connections 2024-04-25T13:48:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- 172.21.15.80:0/1491227743 wait complete. 2024-04-25T13:48:22.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 Processor -- start 2024-04-25T13:48:22.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- start start 2024-04-25T13:48:22.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39600b0f10 0x7f39601519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39600ce2a0 0x7f3960141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3960161d10 0x7f396015a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f396015e720 con 0x7f39600b0f10 2024-04-25T13:48:22.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f39600b5300 con 0x7f3960161d10 2024-04-25T13:48:22.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f39685ed640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3960162a70 con 0x7f39600ce2a0 2024-04-25T13:48:22.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f3966389640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39600b0f10 0x7f39601519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f3966389640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39600b0f10 0x7f39601519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:55836/0 (socket says 172.21.15.80:55836) 2024-04-25T13:48:22.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f3966389640 1 -- 172.21.15.80:0/1416553197 learned_addr learned my addr 172.21.15.80:0/1416553197 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:48:22.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f3966389640 1 -- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39600ce2a0 msgr2=0x7f3960141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f3966389640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39600ce2a0 0x7f3960141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f3966389640 1 -- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3960161d10 msgr2=0x7f396015a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:22.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.782+0000 7f3966b8a640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3960161d10 0x7f396015a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3966389640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3960161d10 0x7f396015a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:22.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3966389640 1 -- 172.21.15.80:0/1416553197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3960155430 con 0x7f39600b0f10 2024-04-25T13:48:22.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3966389640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39600b0f10 0x7f39601519c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f395c00d5c0 tx=0x7f395c00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3956ffd640 1 -- 172.21.15.80:0/1416553197 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c004240 con 0x7f39600b0f10 2024-04-25T13:48:22.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3956ffd640 1 -- 172.21.15.80:0/1416553197 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f395c0043e0 con 0x7f39600b0f10 2024-04-25T13:48:22.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f396015ac90 con 0x7f39600b0f10 2024-04-25T13:48:22.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3956ffd640 1 -- 172.21.15.80:0/1416553197 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395c003840 con 0x7f39600b0f10 2024-04-25T13:48:22.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39601559e0 con 0x7f39600b0f10 2024-04-25T13:48:22.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f396006a440 con 0x7f39600b0f10 2024-04-25T13:48:22.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3956ffd640 1 -- 172.21.15.80:0/1416553197 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f395c0039e0 con 0x7f39600b0f10 2024-04-25T13:48:22.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3956ffd640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394406f400 0x7f3944071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:48:22.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3956ffd640 1 -- 172.21.15.80:0/1416553197 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5767+0+0 (secure 0 0 0) 0x7f395c090050 con 0x7f39600b0f10 2024-04-25T13:48:22.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3965b88640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394406f400 0x7f3944071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:48:22.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.786+0000 7f3965b88640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394406f400 0x7f3944071910 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f39601fb440 tx=0x7f3950007970 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:48:22.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.790+0000 7f3956ffd640 1 -- 172.21.15.80:0/1416553197 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f395c05cfe0 con 0x7f39600b0f10 2024-04-25T13:48:22.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:22.966+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f3960151fb0 con 0x7f39600b0f10 2024-04-25T13:48:23.334 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.338+0000 7f3956ffd640 1 -- 172.21.15.80:0/1416553197 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v157) ==== 181+0+0 (secure 0 0 0) 0x7f3960151fb0 con 0x7f39600b0f10 2024-04-25T13:48:23.335 INFO:teuthology.orchestra.run.smithi080.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-25T13:48:23.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394406f400 msgr2=0x7f3944071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394406f400 0x7f3944071910 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f39601fb440 tx=0x7f3950007970 comp rx=0 tx=0).stop 2024-04-25T13:48:23.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39600b0f10 msgr2=0x7f39601519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:48:23.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39600b0f10 0x7f39601519c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f395c00d5c0 tx=0x7f395c00da80 comp rx=0 tx=0).stop 2024-04-25T13:48:23.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 shutdown_connections 2024-04-25T13:48:23.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f394406f400 0x7f3944071910 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3960161d10 0x7f396015a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f39600ce2a0 0x7f3960141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 --2- 172.21.15.80:0/1416553197 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f39600b0f10 0x7f39601519c0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:48:23.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 >> 172.21.15.80:0/1416553197 conn(0x7f39600b35c0 msgr2=0x7f3960072da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:48:23.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 shutdown_connections 2024-04-25T13:48:23.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:48:23.342+0000 7f39685ed640 1 -- 172.21.15.80:0/1416553197 wait complete. 2024-04-25T13:48:23.366 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-25T13:48:23.533 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_mon_report_interval":"5"} 2024-04-25T13:48:23.533 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-25T13:48:23.533 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-04-25T13:48:23.690 INFO:teuthology.orchestra.run.smithi080.stdout:{"mon_tick_interval":"5"} 2024-04-25T13:48:23.691 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-04-25T13:48:23.691 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-04-25T13:48:23.691 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:48:23.691 DEBUG:teuthology.orchestra.run.smithi080:> import time 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> import datetime 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> # Write some buffered data through before going full, all should be well 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> print("writing some data through which we expect to succeed") 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> bytes = 0 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(f) 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-04-25T13:48:23.692 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> # buffered IO, the error should always only appear from fsync and not 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> # from write 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> full = False 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> for n in range(0, int(32 * 0.9)): 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> print("done writing {0} bytes".format(bytes)) 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> # OK, now we should sneak in under the full condition 2024-04-25T13:48:23.693 DEBUG:teuthology.orchestra.run.smithi080:> # due to the time it takes the OSDs to report to the 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> # mons, and get a successful fsync on our full-making data 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(f) 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> print("successfully fsync'"'"'ed prior to getting full state reported") 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> # buffered write, add more dirty data to the buffer 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> print("starting buffered write") 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> try: 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> for n in range(0, int(32 * 0.2)): 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> time.sleep(15.0) 2024-04-25T13:48:23.694 DEBUG:teuthology.orchestra.run.smithi080:> except OSError: 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> pass; 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> # Wait long enough for a background flush that should fail 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> time.sleep(30) 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> if True: 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> # ...and check that the failed background flush is reflected in fclose 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> try: 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> os.close(f) 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> except OSError: 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> print("close() returned an error as expected") 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> else: 2024-04-25T13:48:23.695 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("close() failed to raise error") 2024-04-25T13:48:23.696 DEBUG:teuthology.orchestra.run.smithi080:> else: 2024-04-25T13:48:23.696 DEBUG:teuthology.orchestra.run.smithi080:> # The kernel cephfs client does not raise errors on fclose 2024-04-25T13:48:23.696 DEBUG:teuthology.orchestra.run.smithi080:> os.close(f) 2024-04-25T13:48:23.696 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:48:23.696 DEBUG:teuthology.orchestra.run.smithi080:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-04-25T13:48:23.696 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:48:30.095 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:48:30.098 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:49:00.221 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:49:00.224 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:49:30.268 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:49:30.272 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:50:00.403 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:50:00.406 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi080.stdout:writing some data through which we expect to succeed 2024-04-25T13:50:09.897 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 1572864 bytes via buffered write, may repeat 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 2621440 bytes via buffered write, may repeat 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 3670016 bytes via buffered write, may repeat 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 4718592 bytes via buffered write, may repeat 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 5767168 bytes via buffered write, may repeat 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 6815744 bytes via buffered write, may repeat 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 7864320 bytes via buffered write, may repeat 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 8912896 bytes via buffered write, may repeat 2024-04-25T13:50:09.898 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 9961472 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 11010048 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 12058624 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 13107200 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 14155776 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 15204352 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 16252928 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 17301504 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 18350080 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 19398656 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 20447232 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 21495808 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 22544384 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 23592960 bytes via buffered write, may repeat 2024-04-25T13:50:09.899 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 24641536 bytes via buffered write, may repeat 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 25690112 bytes via buffered write, may repeat 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 26738688 bytes via buffered write, may repeat 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 27787264 bytes via buffered write, may repeat 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 28835840 bytes via buffered write, may repeat 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:wrote 29884416 bytes via buffered write, may repeat 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:done writing 29884416 bytes 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:successfully fsync'ed prior to getting full state reported 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:starting buffered write 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:50:09.900 INFO:teuthology.orchestra.run.smithi080.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2024-04-25T13:50:09.901 INFO:teuthology.orchestra.run.smithi080.stdout:close() returned an error as expected 2024-04-25T13:50:09.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:50:09.920 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:50:09.920 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:50:09.967 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:50:10.000 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:50:10.000 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:50:10.000 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:50:10.001 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:10.006 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:50:10.006 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:10.054 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:10.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f5da18640 1 Processor -- start 2024-04-25T13:50:10.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f5da18640 1 -- start start 2024-04-25T13:50:10.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f5da18640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3f581645d0 0x7f3f580b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f5da18640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f581649e0 con 0x7f3f580b1e10 2024-04-25T13:50:10.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f5da18640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f58168220 con 0x7f3f58073c90 2024-04-25T13:50:10.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f5da18640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3f58168500 con 0x7f3f581645d0 2024-04-25T13:50:10.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f56ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3f580b1e10 0x7f3f58073580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:51950/0 (socket says 172.21.15.80:51950) 2024-04-25T13:50:10.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f56ffd640 1 -- 172.21.15.80:0/2800166972 learned_addr learned my addr 172.21.15.80:0/2800166972 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:10.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f577fe640 1 --2- 172.21.15.80:0/2800166972 >> v2:172.21.15.87:3300/0 conn(0x7f3f581645d0 0x7f3f580b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3711003620 0 0) 0x7f3f581649e0 con 0x7f3f580b1e10 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3f44003530 con 0x7f3f580b1e10 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2143084194 0 0) 0x7f3f44003530 con 0x7f3f580b1e10 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 >> v2:172.21.15.87:3300/0 conn(0x7f3f581645d0 msgr2=0x7f3f580b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 --2- 172.21.15.80:0/2800166972 >> v2:172.21.15.87:3300/0 conn(0x7f3f581645d0 0x7f3f580b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 >> v1:172.21.15.87:6789/0 conn(0x7f3f58073c90 legacy=0x7f3f580b61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f581687e0 con 0x7f3f580b1e10 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f3f48001a00 con 0x7f3f580b1e10 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 >> v1:172.21.15.80:6789/0 conn(0x7f3f580b1e10 legacy=0x7f3f58073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f440039c0 0x7f3f44003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f44004810 0x7f3f44004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f44005310 0x7f3f4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58168500 con 0x7f3f44004810 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4400c2b0 con 0x7f3f44005310 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4400c4b0 con 0x7f3f440039c0 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.265+0000 7f3f567fc640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f44004810 0x7f3f44004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f577fe640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f44005310 0x7f3f4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f56ffd640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f440039c0 0x7f3f44003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f567fc640 1 -- 172.21.15.80:0/2800166972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f440039c0 msgr2=0x7f3f44003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f567fc640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f440039c0 0x7f3f44003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f567fc640 1 -- 172.21.15.80:0/2800166972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f44005310 msgr2=0x7f3f4400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f567fc640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f44005310 0x7f3f4400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f577fe640 1 -- 172.21.15.80:0/2800166972 reap_dead start 2024-04-25T13:50:10.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f567fc640 1 -- 172.21.15.80:0/2800166972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f44003530 con 0x7f3f44004810 2024-04-25T13:50:10.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f567fc640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f44004810 0x7f3f44004c20 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3f58072e20 tx=0x7f3f4001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2a2e000dbd574e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f400013a0 con 0x7f3f44004810 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f400052f0 con 0x7f3f44004810 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f55ffb640 1 -- 172.21.15.80:0/2800166972 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40003840 con 0x7f3f44004810 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f5da18640 1 -- 172.21.15.80:0/2800166972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f44004810 msgr2=0x7f3f44004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f5da18640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f44004810 0x7f3f44004c20 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f3f58072e20 tx=0x7f3f4001c3a0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f5da18640 1 -- 172.21.15.80:0/2800166972 shutdown_connections 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f5da18640 1 --2- 172.21.15.80:0/2800166972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f44004810 0x7f3f44004c20 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f5da18640 1 -- 172.21.15.80:0/2800166972 >> 172.21.15.80:0/2800166972 conn(0x7f3f580d3500 msgr2=0x7f3f58072e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f5da18640 1 -- 172.21.15.80:0/2800166972 shutdown_connections 2024-04-25T13:50:10.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.269+0000 7f3f5da18640 1 -- 172.21.15.80:0/2800166972 wait complete. 2024-04-25T13:50:10.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 Processor -- start 2024-04-25T13:50:10.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 -- start start 2024-04-25T13:50:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f44004810 0x7f3f581fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f58148680 0x7f3f58167340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f58167880 0x7f3f58164ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3f58068920 con 0x7f3f58167880 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f580687a0 con 0x7f3f58148680 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f58068aa0 con 0x7f3f44004810 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f58167880 0x7f3f58164ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f58167880 0x7f3f58164ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40344/0 (socket says 172.21.15.80:40344) 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 -- 172.21.15.80:0/4269767512 learned_addr learned my addr 172.21.15.80:0/4269767512 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f56ffd640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f44004810 0x7f3f581fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f567fc640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f58148680 0x7f3f58167340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 -- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f44004810 msgr2=0x7f3f581fd4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f44004810 0x7f3f581fd4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 -- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f58148680 msgr2=0x7f3f58167340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f58148680 0x7f3f58167340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 -- 172.21.15.80:0/4269767512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f58166580 con 0x7f3f58167880 2024-04-25T13:50:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f56ffd640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f44004810 0x7f3f581fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f577fe640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f58167880 0x7f3f58164ea0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3f580702f0 tx=0x7f3f5000f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f4ffff640 1 -- 172.21.15.80:0/4269767512 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50014870 con 0x7f3f58167880 2024-04-25T13:50:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f4ffff640 1 -- 172.21.15.80:0/4269767512 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f5001a070 con 0x7f3f58167880 2024-04-25T13:50:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f4ffff640 1 -- 172.21.15.80:0/4269767512 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f50014c70 con 0x7f3f58167880 2024-04-25T13:50:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f58166860 con 0x7f3f58167880 2024-04-25T13:50:10.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.273+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f581fda00 con 0x7f3f58167880 2024-04-25T13:50:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.277+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f5806a3a0 con 0x7f3f58167880 2024-04-25T13:50:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.277+0000 7f3f4ffff640 1 -- 172.21.15.80:0/4269767512 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3f50012070 con 0x7f3f58167880 2024-04-25T13:50:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.277+0000 7f3f4ffff640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f3806f400 0x7f3f38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.277+0000 7f3f4ffff640 1 -- 172.21.15.80:0/4269767512 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f3f5008f0e0 con 0x7f3f58167880 2024-04-25T13:50:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.277+0000 7f3f4ffff640 1 -- 172.21.15.80:0/4269767512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f3f38077040 con 0x7f3f58167880 2024-04-25T13:50:10.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.277+0000 7f3f56ffd640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f3806f400 0x7f3f38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.277+0000 7f3f56ffd640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f3806f400 0x7f3f38071910 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f58072640 tx=0x7f3f48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.281+0000 7f3f4ffff640 1 -- 172.21.15.80:0/4269767512 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3f5005bf90 con 0x7f3f58167880 2024-04-25T13:50:10.440 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.441+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f58165960 con 0x7f3f58167880 2024-04-25T13:50:10.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.445+0000 7f3f4ffff640 1 -- 172.21.15.80:0/4269767512 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16674 (secure 0 0 0) 0x7f3f5005b890 con 0x7f3f58167880 2024-04-25T13:50:10.443 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:10.443 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":158,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:49:21.252418+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:48:07.666863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T13:48:08.875350+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.3","mappings":[{"from":2,"to":0}]},{"pgid":"19.a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:10.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f3806f400 msgr2=0x7f3f38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f3806f400 0x7f3f38071910 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f3f58072640 tx=0x7f3f48009040 comp rx=0 tx=0).stop 2024-04-25T13:50:10.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f58167880 msgr2=0x7f3f58164ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f58167880 0x7f3f58164ea0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f3f580702f0 tx=0x7f3f5000f8a0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 shutdown_connections 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f3806f400 0x7f3f38071910 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f58167880 0x7f3f58164ea0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f58148680 0x7f3f58167340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 --2- 172.21.15.80:0/4269767512 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f44004810 0x7f3f581fd4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 >> 172.21.15.80:0/4269767512 conn(0x7f3f580d3500 msgr2=0x7f3f580b27b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 shutdown_connections 2024-04-25T13:50:10.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.449+0000 7f3f5da18640 1 -- 172.21.15.80:0/4269767512 wait complete. 2024-04-25T13:50:10.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:48:07.666863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:48:08.875350+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:50:10.466 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:50:10.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe63dd52640 1 Processor -- start 2024-04-25T13:50:10.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe63dd52640 1 -- start start 2024-04-25T13:50:10.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe63dd52640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe638161170 0x7fe6381574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe63dd52640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638073c40 0x7fe6380b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe63dd52640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe63806fbf0 con 0x7fe638071e60 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe63dd52640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe6380b1350 con 0x7fe638161170 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe63dd52640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe63806f9a0 con 0x7fe638073c40 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe62effd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe638161170 0x7fe6381574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe62effd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe638161170 0x7fe6381574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40350/0 (socket says 172.21.15.80:40350) 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe62effd640 1 -- 172.21.15.80:0/3839853114 learned_addr learned my addr 172.21.15.80:0/3839853114 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe637fff640 1 --2- 172.21.15.80:0/3839853114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638073c40 0x7fe6380b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe62effd640 1 -- 172.21.15.80:0/3839853114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638073c40 msgr2=0x7fe6380b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe62effd640 1 --2- 172.21.15.80:0/3839853114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638073c40 0x7fe6380b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe62effd640 1 -- 172.21.15.80:0/3839853114 >> v1:172.21.15.87:6789/0 conn(0x7fe638071e60 legacy=0x7fe638160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe62effd640 1 -- 172.21.15.80:0/3839853114 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6380b0a20 con 0x7fe638161170 2024-04-25T13:50:10.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.629+0000 7fe637fff640 1 --2- 172.21.15.80:0/3839853114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638073c40 0x7fe6380b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:10.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe62effd640 1 --2- 172.21.15.80:0/3839853114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638161170 0x7fe6381574c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fe62000bb10 tx=0x7fe62001ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb1d124bd6e7ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe636ffd640 1 -- 172.21.15.80:0/3839853114 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6200b4070 con 0x7fe638161170 2024-04-25T13:50:10.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe636ffd640 1 -- 172.21.15.80:0/3839853114 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6200a7b70 con 0x7fe638161170 2024-04-25T13:50:10.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe636ffd640 1 -- 172.21.15.80:0/3839853114 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6200af440 con 0x7fe638161170 2024-04-25T13:50:10.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 -- 172.21.15.80:0/3839853114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638161170 msgr2=0x7fe6381574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 --2- 172.21.15.80:0/3839853114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638161170 0x7fe6381574c0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fe62000bb10 tx=0x7fe62001ebf0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 -- 172.21.15.80:0/3839853114 shutdown_connections 2024-04-25T13:50:10.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 --2- 172.21.15.80:0/3839853114 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638073c40 0x7fe6380b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 --2- 172.21.15.80:0/3839853114 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638161170 0x7fe6381574c0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 -- 172.21.15.80:0/3839853114 >> 172.21.15.80:0/3839853114 conn(0x7fe6380b4160 msgr2=0x7fe6380d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:10.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 -- 172.21.15.80:0/3839853114 shutdown_connections 2024-04-25T13:50:10.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 -- 172.21.15.80:0/3839853114 wait complete. 2024-04-25T13:50:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 Processor -- start 2024-04-25T13:50:10.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.633+0000 7fe63dd52640 1 -- start start 2024-04-25T13:50:10.635 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638071e60 0x7fe638150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe638073c40 0x7fe638155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638156160 0x7fe6381526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe63806e470 con 0x7fe638071e60 2024-04-25T13:50:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe638074050 con 0x7fe638073c40 2024-04-25T13:50:10.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe63806e5f0 con 0x7fe638156160 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638071e60 0x7fe638150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638071e60 0x7fe638150450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40354/0 (socket says 172.21.15.80:40354) 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 -- 172.21.15.80:0/2258061503 learned_addr learned my addr 172.21.15.80:0/2258061503 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe62effd640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe638073c40 0x7fe638155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe637fff640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638156160 0x7fe6381526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 -- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638156160 msgr2=0x7fe6381526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638156160 0x7fe6381526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 -- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe638073c40 msgr2=0x7fe638155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe638073c40 0x7fe638155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 -- 172.21.15.80:0/2258061503 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe638150990 con 0x7fe638071e60 2024-04-25T13:50:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe6377fe640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638071e60 0x7fe638150450 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fe62800b440 tx=0x7fe62800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe634ff9640 1 -- 172.21.15.80:0/2258061503 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628015020 con 0x7fe638071e60 2024-04-25T13:50:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe634ff9640 1 -- 172.21.15.80:0/2258061503 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe628011890 con 0x7fe638071e60 2024-04-25T13:50:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe634ff9640 1 -- 172.21.15.80:0/2258061503 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628011c80 con 0x7fe638071e60 2024-04-25T13:50:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe638150ea0 con 0x7fe638071e60 2024-04-25T13:50:10.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe638151460 con 0x7fe638071e60 2024-04-25T13:50:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63806a3a0 con 0x7fe638071e60 2024-04-25T13:50:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.637+0000 7fe634ff9640 1 -- 172.21.15.80:0/2258061503 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe628018020 con 0x7fe638071e60 2024-04-25T13:50:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.641+0000 7fe634ff9640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe61806f430 0x7fe618071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.641+0000 7fe634ff9640 1 -- 172.21.15.80:0/2258061503 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7fe62808f0d0 con 0x7fe638071e60 2024-04-25T13:50:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.641+0000 7fe634ff9640 1 -- 172.21.15.80:0/2258061503 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fe6180770d0 con 0x7fe638071e60 2024-04-25T13:50:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.641+0000 7fe62effd640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe61806f430 0x7fe618071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.641+0000 7fe62effd640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe61806f430 0x7fe618071940 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe6200b1000 tx=0x7fe6200a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.645+0000 7fe634ff9640 1 -- 172.21.15.80:0/2258061503 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe62801f030 con 0x7fe638071e60 2024-04-25T13:50:10.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.805+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe638160c70 con 0x7fe638071e60 2024-04-25T13:50:10.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.805+0000 7fe634ff9640 1 -- 172.21.15.80:0/2258061503 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7fe62800d040 con 0x7fe638071e60 2024-04-25T13:50:10.803 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.805+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe61806f430 msgr2=0x7fe618071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.805+0000 7fe63dd52640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe61806f430 0x7fe618071940 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe6200b1000 tx=0x7fe6200a9070 comp rx=0 tx=0).stop 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.805+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638071e60 msgr2=0x7fe638150450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.805+0000 7fe63dd52640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638071e60 0x7fe638150450 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fe62800b440 tx=0x7fe62800b900 comp rx=0 tx=0).stop 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.809+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 shutdown_connections 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.809+0000 7fe63dd52640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe61806f430 0x7fe618071940 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.809+0000 7fe63dd52640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe638156160 0x7fe6381526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.809+0000 7fe63dd52640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe638073c40 0x7fe638155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.809+0000 7fe63dd52640 1 --2- 172.21.15.80:0/2258061503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe638071e60 0x7fe638150450 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.809+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 >> 172.21.15.80:0/2258061503 conn(0x7fe6380b4160 msgr2=0x7fe63815dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:10.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.809+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 shutdown_connections 2024-04-25T13:50:10.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.809+0000 7fe63dd52640 1 -- 172.21.15.80:0/2258061503 wait complete. 2024-04-25T13:50:10.824 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:50:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c2d58d640 1 Processor -- start 2024-04-25T13:50:10.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c2d58d640 1 -- start start 2024-04-25T13:50:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c2d58d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c281574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c2d58d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c28073c40 0x7f1c28160a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c2d58d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c281613b0 con 0x7f1c28160fa0 2024-04-25T13:50:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c2d58d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1c280b1350 con 0x7f1c28073c40 2024-04-25T13:50:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c2d58d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c2806f9a0 con 0x7f1c28071e60 2024-04-25T13:50:10.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c281574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c267fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c28073c40 0x7f1c28160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c26ffd640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c281574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:42100/0 (socket says 172.21.15.80:42100) 2024-04-25T13:50:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c26ffd640 1 -- 172.21.15.80:0/2209319326 learned_addr learned my addr 172.21.15.80:0/2209319326 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:10.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c26ffd640 1 -- 172.21.15.80:0/2209319326 >> v2:172.21.15.87:3300/0 conn(0x7f1c28073c40 msgr2=0x7f1c28160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:10.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c26ffd640 1 --2- 172.21.15.80:0/2209319326 >> v2:172.21.15.87:3300/0 conn(0x7f1c28073c40 0x7f1c28160a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c26ffd640 1 -- 172.21.15.80:0/2209319326 >> v1:172.21.15.87:6789/0 conn(0x7f1c28160fa0 legacy=0x7f1c280b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c26ffd640 1 -- 172.21.15.80:0/2209319326 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c2806fbf0 con 0x7f1c28071e60 2024-04-25T13:50:10.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.977+0000 7f1c267fc640 1 --2- 172.21.15.80:0/2209319326 >> v2:172.21.15.87:3300/0 conn(0x7f1c28073c40 0x7f1c28160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c26ffd640 1 --2- 172.21.15.80:0/2209319326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c281574c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800bb10 tx=0x7f1c1801ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae5e028c027d2084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c25ffb640 1 -- 172.21.15.80:0/2209319326 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180b4070 con 0x7f1c28071e60 2024-04-25T13:50:10.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c25ffb640 1 -- 172.21.15.80:0/2209319326 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c180a7af0 con 0x7f1c28071e60 2024-04-25T13:50:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c25ffb640 1 -- 172.21.15.80:0/2209319326 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c180af440 con 0x7f1c28071e60 2024-04-25T13:50:10.979 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c2d58d640 1 -- 172.21.15.80:0/2209319326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 msgr2=0x7f1c281574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/2209319326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c281574c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1c1800bb10 tx=0x7f1c1801ec30 comp rx=0 tx=0).stop 2024-04-25T13:50:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c2d58d640 1 -- 172.21.15.80:0/2209319326 shutdown_connections 2024-04-25T13:50:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/2209319326 >> v2:172.21.15.87:3300/0 conn(0x7f1c28073c40 0x7f1c28160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/2209319326 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c281574c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c2d58d640 1 -- 172.21.15.80:0/2209319326 >> 172.21.15.80:0/2209319326 conn(0x7f1c280b4160 msgr2=0x7f1c280d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c2d58d640 1 -- 172.21.15.80:0/2209319326 shutdown_connections 2024-04-25T13:50:10.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.981+0000 7f1c2d58d640 1 -- 172.21.15.80:0/2209319326 wait complete. 2024-04-25T13:50:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c2d58d640 1 Processor -- start 2024-04-25T13:50:10.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c2d58d640 1 -- start start 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c2d58d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c2815a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c2d58d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c28073c40 0x7f1c28155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c2d58d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c28160fa0 0x7f1c28161d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c2d58d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28074050 con 0x7f1c28073c40 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c2d58d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c28072270 con 0x7f1c28160fa0 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c2d58d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c280b56c0 con 0x7f1c28071e60 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c28073c40 0x7f1c28155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c28073c40 0x7f1c28155680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40358/0 (socket says 172.21.15.80:40358) 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 -- 172.21.15.80:0/1856942848 learned_addr learned my addr 172.21.15.80:0/1856942848 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:10.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c26ffd640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c2815a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c277fe640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c28160fa0 0x7f1c28161d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 -- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 msgr2=0x7f1c2815a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c2815a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 -- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c28160fa0 msgr2=0x7f1c28161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c28160fa0 0x7f1c28161d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 -- 172.21.15.80:0/1856942848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c28158c70 con 0x7f1c28073c40 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c26ffd640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c2815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c267fc640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c28073c40 0x7f1c28155680 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00e9e0 tx=0x7f1c1c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.985+0000 7f1c0f7fe640 1 -- 172.21.15.80:0/1856942848 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c017070 con 0x7f1c28073c40 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c0f7fe640 1 -- 172.21.15.80:0/1856942848 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c1c00c9d0 con 0x7f1c28073c40 2024-04-25T13:50:10.986 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c28157bd0 con 0x7f1c28073c40 2024-04-25T13:50:10.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c0f7fe640 1 -- 172.21.15.80:0/1856942848 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c00cdc0 con 0x7f1c28073c40 2024-04-25T13:50:10.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c28158190 con 0x7f1c28073c40 2024-04-25T13:50:10.987 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c2806a3a0 con 0x7f1c28073c40 2024-04-25T13:50:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c0f7fe640 1 -- 172.21.15.80:0/1856942848 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1c1c01a070 con 0x7f1c28073c40 2024-04-25T13:50:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c0f7fe640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c1406f400 0x7f1c14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c0f7fe640 1 -- 172.21.15.80:0/1856942848 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f1c1c08ed50 con 0x7f1c28073c40 2024-04-25T13:50:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c0f7fe640 1 -- 172.21.15.80:0/1856942848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f1c140770a0 con 0x7f1c28073c40 2024-04-25T13:50:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c26ffd640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c1406f400 0x7f1c14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:10.988 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.989+0000 7f1c26ffd640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c1406f400 0x7f1c14071910 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1c180b1000 tx=0x7f1c180a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:10.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:10.997+0000 7f1c0f7fe640 1 -- 172.21.15.80:0/1856942848 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c1c092030 con 0x7f1c28073c40 2024-04-25T13:50:11.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c28156100 con 0x7f1c28073c40 2024-04-25T13:50:11.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c0f7fe640 1 -- 172.21.15.80:0/1856942848 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) ==== 104+0+11 (secure 0 0 0) 0x7f1c1c05b640 con 0x7f1c28073c40 2024-04-25T13:50:11.142 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:50:11.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c1406f400 msgr2=0x7f1c14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c1406f400 0x7f1c14071910 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f1c180b1000 tx=0x7f1c180a9070 comp rx=0 tx=0).stop 2024-04-25T13:50:11.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c28073c40 msgr2=0x7f1c28155680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c28073c40 0x7f1c28155680 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c00e9e0 tx=0x7f1c1c00eea0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 shutdown_connections 2024-04-25T13:50:11.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c1406f400 0x7f1c14071910 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c28160fa0 0x7f1c28161d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c28073c40 0x7f1c28155680 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 --2- 172.21.15.80:0/1856942848 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c28071e60 0x7f1c2815a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 >> 172.21.15.80:0/1856942848 conn(0x7f1c280b4160 msgr2=0x7f1c280b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:11.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 shutdown_connections 2024-04-25T13:50:11.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.145+0000 7f1c2d58d640 1 -- 172.21.15.80:0/1856942848 wait complete. 2024-04-25T13:50:11.162 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:50:11.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc72c5640 1 Processor -- start 2024-04-25T13:50:11.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc72c5640 1 -- start start 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc72c5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015f810 0x7f7bc0068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc72c5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7bc015fc20 0x7f7bc0068ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc72c5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bc015ec20 con 0x7f7bc015e5d0 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc72c5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7bc015e9e0 con 0x7f7bc015fc20 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc72c5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc015ef00 con 0x7f7bc015f810 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc5061640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7bc015e5d0 0x7f7bc0070f00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58374/0 (socket says 172.21.15.80:58374) 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc5061640 1 -- 172.21.15.80:0/1554965221 learned_addr learned my addr 172.21.15.80:0/1554965221 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc5862640 1 --2- 172.21.15.80:0/1554965221 >> v2:172.21.15.80:3300/0 conn(0x7f7bc015fc20 0x7f7bc0068ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bc4860640 1 --2- 172.21.15.80:0/1554965221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015f810 0x7f7bc0068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.317+0000 7f7bb7fff640 1 -- 172.21.15.80:0/1554965221 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1631213266 0 0) 0x7f7bc015ec20 con 0x7f7bc015e5d0 2024-04-25T13:50:11.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bb7fff640 1 -- 172.21.15.80:0/1554965221 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7bac003530 con 0x7f7bc015e5d0 2024-04-25T13:50:11.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc5862640 1 -- 172.21.15.80:0/1554965221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015f810 msgr2=0x7f7bc0068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:11.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc5862640 1 --2- 172.21.15.80:0/1554965221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015f810 0x7f7bc0068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc5862640 1 -- 172.21.15.80:0/1554965221 >> v1:172.21.15.80:6789/0 conn(0x7f7bc015e5d0 legacy=0x7f7bc0070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc5862640 1 -- 172.21.15.80:0/1554965221 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc015f1e0 con 0x7f7bc015fc20 2024-04-25T13:50:11.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc5862640 1 --2- 172.21.15.80:0/1554965221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015fc20 0x7f7bc0068ad0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009d90 tx=0x7f7bb801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526dd6de08b51152 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bb7fff640 1 -- 172.21.15.80:0/1554965221 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80b2070 con 0x7f7bc015fc20 2024-04-25T13:50:11.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bb7fff640 1 -- 172.21.15.80:0/1554965221 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb80a5830 con 0x7f7bc015fc20 2024-04-25T13:50:11.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bb7fff640 1 -- 172.21.15.80:0/1554965221 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb80a5c00 con 0x7f7bc015fc20 2024-04-25T13:50:11.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1554965221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015fc20 msgr2=0x7f7bc0068ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1554965221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015fc20 0x7f7bc0068ad0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8009d90 tx=0x7f7bb801c970 comp rx=0 tx=0).stop 2024-04-25T13:50:11.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1554965221 shutdown_connections 2024-04-25T13:50:11.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1554965221 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015fc20 0x7f7bc0068ad0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1554965221 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015f810 0x7f7bc0068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1554965221 >> 172.21.15.80:0/1554965221 conn(0x7f7bc0073790 msgr2=0x7f7bc0073ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:11.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1554965221 shutdown_connections 2024-04-25T13:50:11.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.321+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1554965221 wait complete. 2024-04-25T13:50:11.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 Processor -- start 2024-04-25T13:50:11.322 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 -- start start 2024-04-25T13:50:11.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015e5d0 0x7f7bc0158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bc015f810 0x7f7bc014ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015fc20 0x7f7bc01568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc016afe0 con 0x7f7bc015e5d0 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc0070050 con 0x7f7bc015f810 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7bc016b160 con 0x7f7bc015fc20 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015e5d0 0x7f7bc0158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015e5d0 0x7f7bc0158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57976/0 (socket says 172.21.15.80:57976) 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 -- 172.21.15.80:0/1034434746 learned_addr learned my addr 172.21.15.80:0/1034434746 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc4860640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bc015f810 0x7f7bc014ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5862640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015fc20 0x7f7bc01568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 -- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015fc20 msgr2=0x7f7bc01568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015fc20 0x7f7bc01568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 -- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bc015f810 msgr2=0x7f7bc014ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bc015f810 0x7f7bc014ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 -- 172.21.15.80:0/1034434746 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc015b790 con 0x7f7bc015e5d0 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc5061640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015e5d0 0x7f7bc0158e60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00c3c0 tx=0x7f7bbc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bb5ffb640 1 -- 172.21.15.80:0/1034434746 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc017070 con 0x7f7bc015e5d0 2024-04-25T13:50:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bb5ffb640 1 -- 172.21.15.80:0/1034434746 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bbc005cf0 con 0x7f7bc015e5d0 2024-04-25T13:50:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bb5ffb640 1 -- 172.21.15.80:0/1034434746 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc0072f0 con 0x7f7bc015e5d0 2024-04-25T13:50:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc015ba70 con 0x7f7bc015e5d0 2024-04-25T13:50:11.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.325+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc015a340 con 0x7f7bc015e5d0 2024-04-25T13:50:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.329+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc01593a0 con 0x7f7bc015e5d0 2024-04-25T13:50:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.329+0000 7f7bb5ffb640 1 -- 172.21.15.80:0/1034434746 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7bbc01a070 con 0x7f7bc015e5d0 2024-04-25T13:50:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.329+0000 7f7bb5ffb640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ba406f400 0x7f7ba4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.329+0000 7f7bb5ffb640 1 -- 172.21.15.80:0/1034434746 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f7bbc08e6f0 con 0x7f7bc015e5d0 2024-04-25T13:50:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.329+0000 7f7bb5ffb640 1 -- 172.21.15.80:0/1034434746 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f7ba4077040 con 0x7f7bc015e5d0 2024-04-25T13:50:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.329+0000 7f7bc4860640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ba406f400 0x7f7ba4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.329+0000 7f7bc4860640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ba406f400 0x7f7ba4071910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0151ee0 tx=0x7f7bb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.331 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.333+0000 7f7bb5ffb640 1 -- 172.21.15.80:0/1034434746 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7bbc05b630 con 0x7f7bc015e5d0 2024-04-25T13:50:11.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.485+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7bc006a3a0 con 0x7f7bc015e5d0 2024-04-25T13:50:11.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bb5ffb640 1 -- 172.21.15.80:0/1034434746 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) ==== 100+0+11 (secure 0 0 0) 0x7f7bbc05af30 con 0x7f7bc015e5d0 2024-04-25T13:50:11.485 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:50:11.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ba406f400 msgr2=0x7f7ba4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ba406f400 0x7f7ba4071910 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0151ee0 tx=0x7f7bb0008040 comp rx=0 tx=0).stop 2024-04-25T13:50:11.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015e5d0 msgr2=0x7f7bc0158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015e5d0 0x7f7bc0158e60 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc00c3c0 tx=0x7f7bbc00c880 comp rx=0 tx=0).stop 2024-04-25T13:50:11.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 shutdown_connections 2024-04-25T13:50:11.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ba406f400 0x7f7ba4071910 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7bc015fc20 0x7f7bc01568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7bc015f810 0x7f7bc014ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 --2- 172.21.15.80:0/1034434746 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7bc015e5d0 0x7f7bc0158e60 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 >> 172.21.15.80:0/1034434746 conn(0x7f7bc0073790 msgr2=0x7f7bc0151080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:11.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 shutdown_connections 2024-04-25T13:50:11.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.489+0000 7f7bc72c5640 1 -- 172.21.15.80:0/1034434746 wait complete. 2024-04-25T13:50:11.505 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:50:11.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb2607d9640 1 Processor -- start 2024-04-25T13:50:11.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb2607d9640 1 -- start start 2024-04-25T13:50:11.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb2607d9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb2580d0890 0x7fb2580b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb2607d9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2580b77a0 0x7fb2580b7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb2607d9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2580d49c0 con 0x7fb2580d45b0 2024-04-25T13:50:11.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb2607d9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb2580b53f0 con 0x7fb2580d0890 2024-04-25T13:50:11.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb2607d9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb2580cf470 con 0x7fb2580b77a0 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb25ed76640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2580b77a0 0x7fb2580b7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb25dd74640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb2580d0890 0x7fb2580b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb25e575640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fb2580d45b0 0x7fb2580d0180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:39320/0 (socket says 172.21.15.80:39320) 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.657+0000 7fb25e575640 1 -- 172.21.15.80:0/3909317661 learned_addr learned my addr 172.21.15.80:0/3909317661 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25dd74640 1 -- 172.21.15.80:0/3909317661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2580b77a0 msgr2=0x7fb2580b7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25dd74640 1 --2- 172.21.15.80:0/3909317661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2580b77a0 0x7fb2580b7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25dd74640 1 -- 172.21.15.80:0/3909317661 >> v1:172.21.15.87:6789/0 conn(0x7fb2580d45b0 legacy=0x7fb2580d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25dd74640 1 -- 172.21.15.80:0/3909317661 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2580cf750 con 0x7fb2580d0890 2024-04-25T13:50:11.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25ed76640 1 --2- 172.21.15.80:0/3909317661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2580b77a0 0x7fb2580b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:11.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25dd74640 1 --2- 172.21.15.80:0/3909317661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580d0890 0x7fb2580b7260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb2480097c0 tx=0x7fb24801c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5414931fd53d4418 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25d573640 1 -- 172.21.15.80:0/3909317661 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2480ab610 con 0x7fb2580d0890 2024-04-25T13:50:11.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25d573640 1 -- 172.21.15.80:0/3909317661 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2480b0070 con 0x7fb2580d0890 2024-04-25T13:50:11.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb25d573640 1 -- 172.21.15.80:0/3909317661 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2480ab9d0 con 0x7fb2580d0890 2024-04-25T13:50:11.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb2607d9640 1 -- 172.21.15.80:0/3909317661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580d0890 msgr2=0x7fb2580b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3909317661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580d0890 0x7fb2580b7260 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fb2480097c0 tx=0x7fb24801c410 comp rx=0 tx=0).stop 2024-04-25T13:50:11.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb2607d9640 1 -- 172.21.15.80:0/3909317661 shutdown_connections 2024-04-25T13:50:11.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3909317661 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2580b77a0 0x7fb2580b7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3909317661 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580d0890 0x7fb2580b7260 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.661+0000 7fb2607d9640 1 -- 172.21.15.80:0/3909317661 >> 172.21.15.80:0/3909317661 conn(0x7fb2580d3a90 msgr2=0x7fb2580d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:11.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 -- 172.21.15.80:0/3909317661 shutdown_connections 2024-04-25T13:50:11.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 -- 172.21.15.80:0/3909317661 wait complete. 2024-04-25T13:50:11.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 Processor -- start 2024-04-25T13:50:11.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 -- start start 2024-04-25T13:50:11.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580b77a0 0x7fb2580b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2580d45b0 0x7fb25809f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb25815af00 0x7fb25809fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb2580cf210 con 0x7fb2580d45b0 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb2580cf090 con 0x7fb2580b77a0 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb2607d9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb2580b9300 con 0x7fb25815af00 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25dd74640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2580d45b0 0x7fb25809f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25dd74640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2580d45b0 0x7fb25809f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:57986/0 (socket says 172.21.15.80:57986) 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25dd74640 1 -- 172.21.15.80:0/3153706127 learned_addr learned my addr 172.21.15.80:0/3153706127 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25e575640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580b77a0 0x7fb2580b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25ed76640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb25815af00 0x7fb25809fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25dd74640 1 -- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb25815af00 msgr2=0x7fb25809fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25dd74640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb25815af00 0x7fb25809fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25dd74640 1 -- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580b77a0 msgr2=0x7fb2580b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25dd74640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580b77a0 0x7fb2580b42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25dd74640 1 -- 172.21.15.80:0/3153706127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2581f7530 con 0x7fb2580d45b0 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.665+0000 7fb25ed76640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb25815af00 0x7fb25809fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb25dd74640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2580d45b0 0x7fb25809f8f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb24801a040 tx=0x7fb24801ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb24effd640 1 -- 172.21.15.80:0/3153706127 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2480bf610 con 0x7fb2580d45b0 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb24effd640 1 -- 172.21.15.80:0/3153706127 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2480bf7d0 con 0x7fb2580d45b0 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb25809e6d0 con 0x7fb2580d45b0 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb24effd640 1 -- 172.21.15.80:0/3153706127 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2480bfbc0 con 0x7fb2580d45b0 2024-04-25T13:50:11.667 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb25809ec90 con 0x7fb2580d45b0 2024-04-25T13:50:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb25806a3a0 con 0x7fb2580d45b0 2024-04-25T13:50:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb24effd640 1 -- 172.21.15.80:0/3153706127 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb2480bd020 con 0x7fb2580d45b0 2024-04-25T13:50:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb24effd640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb23c06f400 0x7fb23c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb24effd640 1 -- 172.21.15.80:0/3153706127 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7fb248129950 con 0x7fb2580d45b0 2024-04-25T13:50:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb24effd640 1 -- 172.21.15.80:0/3153706127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fb23c0770a0 con 0x7fb2580d45b0 2024-04-25T13:50:11.668 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb25e575640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb23c06f400 0x7fb23c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:11.669 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.669+0000 7fb25e575640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb23c06f400 0x7fb23c071910 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb254000f70 tx=0x7fb254007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:11.672 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.673+0000 7fb24effd640 1 -- 172.21.15.80:0/3153706127 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2480a8050 con 0x7fb2580d45b0 2024-04-25T13:50:11.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb258168960 con 0x7fb2580d45b0 2024-04-25T13:50:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb24effd640 1 -- 172.21.15.80:0/3153706127 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v158) ==== 80+0+0 (secure 0 0 0) 0x7fb248016090 con 0x7fb2580d45b0 2024-04-25T13:50:11.830 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb23c06f400 msgr2=0x7fb23c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb23c06f400 0x7fb23c071910 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb254000f70 tx=0x7fb254007e90 comp rx=0 tx=0).stop 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2580d45b0 msgr2=0x7fb25809f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2580d45b0 0x7fb25809f8f0 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fb24801a040 tx=0x7fb24801ac50 comp rx=0 tx=0).stop 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 shutdown_connections 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb23c06f400 0x7fb23c071910 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb25815af00 0x7fb25809fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2580d45b0 0x7fb25809f8f0 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 --2- 172.21.15.80:0/3153706127 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2580b77a0 0x7fb2580b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 >> 172.21.15.80:0/3153706127 conn(0x7fb2580d3a90 msgr2=0x7fb2580715b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 shutdown_connections 2024-04-25T13:50:11.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:11.833+0000 7fb2607d9640 1 -- 172.21.15.80:0/3153706127 wait complete. 2024-04-25T13:50:11.851 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:50:11.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:11.851 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:50:11.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[40673]: fuse finished with error 0 and tester_r 0 2024-04-25T13:50:11.863 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:50:11.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:11.863 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:50:12.001 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:50:12.001 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:50:12.001 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:12.007 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:50:12.008 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:12.055 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:12.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae2670640 1 Processor -- start 2024-04-25T13:50:12.201 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae2670640 1 -- start start 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae2670640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc0d06e0 0x7feadc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae2670640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7feadc06e9e0 0x7feadc06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae2670640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feadc0d0af0 con 0x7feadc161f90 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae2670640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7feadc0d0d70 con 0x7feadc06e9e0 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae2670640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feadc0d1050 con 0x7feadc0d06e0 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadbfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7feadc161f90 0x7feadc0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58380/0 (socket says 172.21.15.80:58380) 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadbfff640 1 -- 172.21.15.80:0/3628085377 learned_addr learned my addr 172.21.15.80:0/3628085377 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae0c0d640 1 --2- 172.21.15.80:0/3628085377 >> v2:172.21.15.87:3300/0 conn(0x7feadc06e9e0 0x7feadc06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadb7fe640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc0d06e0 0x7feadc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3592717927 0 0) 0x7fead0001120 con 0x7feadc161f90 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feac4003530 con 0x7feadc161f90 2024-04-25T13:50:12.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2067754963 0 0) 0x7feac4003530 con 0x7feadc161f90 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc0d06e0 msgr2=0x7feadc06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc0d06e0 0x7feadc06e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 >> v2:172.21.15.87:3300/0 conn(0x7feadc06e9e0 msgr2=0x7feadc06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 --2- 172.21.15.80:0/3628085377 >> v2:172.21.15.87:3300/0 conn(0x7feadc06e9e0 0x7feadc06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc06dca0 con 0x7feadc161f90 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fead0001850 con 0x7feadc161f90 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 >> v1:172.21.15.80:6789/0 conn(0x7feadc161f90 legacy=0x7feadc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feac40039c0 0x7feac4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc0d06e0 0x7feac4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4004d50 0x7feac4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feadc0d0d70 con 0x7feac4004d50 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feadc0d1050 con 0x7feac40039c0 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feac400bb00 con 0x7feadc0d06e0 2024-04-25T13:50:12.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae0c0d640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4004d50 0x7feac4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadbfff640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feac40039c0 0x7feac4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadb7fe640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc0d06e0 0x7feac4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae0c0d640 1 -- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc0d06e0 msgr2=0x7feac4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae0c0d640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc0d06e0 0x7feac4004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae0c0d640 1 -- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feac40039c0 msgr2=0x7feac4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae0c0d640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feac40039c0 0x7feac4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feae0c0d640 1 -- 172.21.15.80:0/3628085377 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feac4003530 con 0x7feac4004d50 2024-04-25T13:50:12.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.205+0000 7feadbfff640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feac40039c0 0x7feac4003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:12.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feae0c0d640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4004d50 0x7feac4005160 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7feadc071e30 tx=0x7fead401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fd839fe3a738f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead40b2070 con 0x7feac4004d50 2024-04-25T13:50:12.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead40a5b60 con 0x7feac4004d50 2024-04-25T13:50:12.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feadaffd640 1 -- 172.21.15.80:0/3628085377 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4009c10 con 0x7feac4004d50 2024-04-25T13:50:12.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feae2670640 1 -- 172.21.15.80:0/3628085377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4004d50 msgr2=0x7feac4005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feae2670640 1 --2- 172.21.15.80:0/3628085377 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feac4004d50 0x7feac4005160 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7feadc071e30 tx=0x7fead401cc60 comp rx=0 tx=0).stop 2024-04-25T13:50:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feae0c0d640 1 -- 172.21.15.80:0/3628085377 reap_dead start 2024-04-25T13:50:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feae2670640 1 -- 172.21.15.80:0/3628085377 shutdown_connections 2024-04-25T13:50:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feae2670640 1 -- 172.21.15.80:0/3628085377 >> 172.21.15.80:0/3628085377 conn(0x7feadc06d080 msgr2=0x7feadc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feae2670640 1 -- 172.21.15.80:0/3628085377 shutdown_connections 2024-04-25T13:50:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.209+0000 7feae2670640 1 -- 172.21.15.80:0/3628085377 wait complete. 2024-04-25T13:50:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 Processor -- start 2024-04-25T13:50:12.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 -- start start 2024-04-25T13:50:12.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc15d7f0 0x7feadc0b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feadc15dc00 0x7feadc0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feadc0b2640 0x7feadc0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7feadc1623a0 con 0x7feadc0b2640 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7feadc0d1330 con 0x7feadc15dc00 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7feadc162520 con 0x7feadc15d7f0 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feadc0b2640 0x7feadc0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feadc0b2640 0x7feadc0afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58014/0 (socket says 172.21.15.80:58014) 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 -- 172.21.15.80:0/3919247630 learned_addr learned my addr 172.21.15.80:0/3919247630 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feadb7fe640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feadc15dc00 0x7feadc0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 -- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc15d7f0 msgr2=0x7feadc0b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc15d7f0 0x7feadc0b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 -- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feadc15dc00 msgr2=0x7feadc0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feadc15dc00 0x7feadc0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 -- 172.21.15.80:0/3919247630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feadc0b2b10 con 0x7feadc0b2640 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae0c0d640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feadc0b2640 0x7feadc0afdf0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fead4002240 tx=0x7fead4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7fead8ff9640 1 -- 172.21.15.80:0/3919247630 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead40b2040 con 0x7feadc0b2640 2024-04-25T13:50:12.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7fead8ff9640 1 -- 172.21.15.80:0/3919247630 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fead40a5960 con 0x7feadc0b2640 2024-04-25T13:50:12.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feadc0b2df0 con 0x7feadc0b2640 2024-04-25T13:50:12.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7fead8ff9640 1 -- 172.21.15.80:0/3919247630 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4009c10 con 0x7feadc0b2640 2024-04-25T13:50:12.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feadc072670 con 0x7feadc0b2640 2024-04-25T13:50:12.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.213+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feadc06a3a0 con 0x7feadc0b2640 2024-04-25T13:50:12.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.217+0000 7fead8ff9640 1 -- 172.21.15.80:0/3919247630 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fead40bf060 con 0x7feadc0b2640 2024-04-25T13:50:12.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.217+0000 7fead8ff9640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feac806f400 0x7feac8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.217+0000 7fead8ff9640 1 -- 172.21.15.80:0/3919247630 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7fead4128e90 con 0x7feadc0b2640 2024-04-25T13:50:12.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.217+0000 7fead8ff9640 1 -- 172.21.15.80:0/3919247630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7feac8077040 con 0x7feadc0b2640 2024-04-25T13:50:12.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.217+0000 7feadbfff640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feac806f400 0x7feac8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.217+0000 7feadbfff640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feac806f400 0x7feac8071910 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fead00019c0 tx=0x7fead0000bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.221+0000 7fead8ff9640 1 -- 172.21.15.80:0/3919247630 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fead4016970 con 0x7feadc0b2640 2024-04-25T13:50:12.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.381+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feadc0d03e0 con 0x7feadc0b2640 2024-04-25T13:50:12.380 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.381+0000 7fead8ff9640 1 -- 172.21.15.80:0/3919247630 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16674 (secure 0 0 0) 0x7fead40f5e60 con 0x7feadc0b2640 2024-04-25T13:50:12.380 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:12.381 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":158,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:49:21.252418+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:48:07.666863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T13:48:08.875350+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.3","mappings":[{"from":2,"to":0}]},{"pgid":"19.a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feac806f400 msgr2=0x7feac8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feac806f400 0x7feac8071910 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fead00019c0 tx=0x7fead0000bd0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feadc0b2640 msgr2=0x7feadc0afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feadc0b2640 0x7feadc0afdf0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fead4002240 tx=0x7fead4009750 comp rx=0 tx=0).stop 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 shutdown_connections 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7feac806f400 0x7feac8071910 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7feadc0b2640 0x7feadc0afdf0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7feadc15dc00 0x7feadc0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 --2- 172.21.15.80:0/3919247630 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7feadc15d7f0 0x7feadc0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 >> 172.21.15.80:0/3919247630 conn(0x7feadc06d080 msgr2=0x7feadc0b6320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 shutdown_connections 2024-04-25T13:50:12.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.385+0000 7feae2670640 1 -- 172.21.15.80:0/3919247630 wait complete. 2024-04-25T13:50:12.402 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:48:07.666863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:48:08.875350+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:50:12.402 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:50:12.538 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ebb4d640 1 Processor -- start 2024-04-25T13:50:12.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ebb4d640 1 -- start start 2024-04-25T13:50:12.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ebb4d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161170 0x7fb8e41574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ebb4d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb8e4073c40 0x7fb8e40b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ebb4d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8e406fbf0 con 0x7fb8e4071e60 2024-04-25T13:50:12.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ebb4d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb8e40b1350 con 0x7fb8e4073c40 2024-04-25T13:50:12.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ebb4d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e406f9a0 con 0x7fb8e4161170 2024-04-25T13:50:12.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e98e9640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb8e4071e60 0x7fb8e4160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58388/0 (socket says 172.21.15.80:58388) 2024-04-25T13:50:12.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e98e9640 1 -- 172.21.15.80:0/1560285737 learned_addr learned my addr 172.21.15.80:0/1560285737 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e90e8640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161170 0x7fb8e41574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ea0ea640 1 --2- 172.21.15.80:0/1560285737 >> v2:172.21.15.87:3300/0 conn(0x7fb8e4073c40 0x7fb8e40b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1985197267 0 0) 0x7fb8e406fbf0 con 0x7fb8e4071e60 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb8cc003530 con 0x7fb8e4071e60 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3308555552 0 0) 0x7fb8cc003530 con 0x7fb8e4071e60 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161170 msgr2=0x7fb8e41574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4161170 0x7fb8e41574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 >> v2:172.21.15.87:3300/0 conn(0x7fb8e4073c40 msgr2=0x7fb8e40b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 --2- 172.21.15.80:0/1560285737 >> v2:172.21.15.87:3300/0 conn(0x7fb8e4073c40 0x7fb8e40b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e40b0a20 con 0x7fb8e4071e60 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb8d4001a00 con 0x7fb8e4071e60 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 >> v1:172.21.15.80:6789/0 conn(0x7fb8e4071e60 legacy=0x7fb8e4160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8cc0039c0 0x7fb8cc003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc004810 0x7fb8cc004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8cc005160 0x7fb8cc00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e40b1350 con 0x7fb8cc0039c0 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e406f9a0 con 0x7fb8cc005160 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8cc00bf10 con 0x7fb8cc004810 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8e98e9640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8cc0039c0 0x7fb8cc003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.541+0000 7fb8ea0ea640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8cc005160 0x7fb8cc00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e90e8640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc004810 0x7fb8cc004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e98e9640 1 -- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc004810 msgr2=0x7fb8cc004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e98e9640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc004810 0x7fb8cc004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e98e9640 1 -- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8cc005160 msgr2=0x7fb8cc00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e98e9640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8cc005160 0x7fb8cc00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e98e9640 1 -- 172.21.15.80:0/1560285737 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8cc003530 con 0x7fb8cc0039c0 2024-04-25T13:50:12.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e90e8640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8cc004810 0x7fb8cc004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e98e9640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8cc0039c0 0x7fb8cc003e30 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400a880 tx=0x7fb8d401d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e08c23069e7b673 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d400a4a0 con 0x7fb8cc0039c0 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8d40ad650 con 0x7fb8cc0039c0 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8e88e7640 1 -- 172.21.15.80:0/1560285737 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8d40ada40 con 0x7fb8cc0039c0 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/1560285737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8cc0039c0 msgr2=0x7fb8cc003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8ebb4d640 1 --2- 172.21.15.80:0/1560285737 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8cc0039c0 0x7fb8cc003e30 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400a880 tx=0x7fb8d401d370 comp rx=0 tx=0).stop 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8ea0ea640 1 -- 172.21.15.80:0/1560285737 reap_dead start 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/1560285737 shutdown_connections 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/1560285737 >> 172.21.15.80:0/1560285737 conn(0x7fb8e40b4160 msgr2=0x7fb8e40d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/1560285737 shutdown_connections 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.545+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/1560285737 wait complete. 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ebb4d640 1 Processor -- start 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ebb4d640 1 -- start start 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ebb4d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4156570 0x7fb8e40d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ebb4d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40d06c0 0x7fb8e414d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ebb4d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e4156980 0x7fb8e414de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ebb4d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4161170 con 0x7fb8e4156980 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ebb4d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8e4074050 con 0x7fb8e40d06c0 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ebb4d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8e41612f0 con 0x7fb8e4156570 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e4156980 0x7fb8e414de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e4156980 0x7fb8e414de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58034/0 (socket says 172.21.15.80:58034) 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 -- 172.21.15.80:0/2298927693 learned_addr learned my addr 172.21.15.80:0/2298927693 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 -- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4156570 msgr2=0x7fb8e40d0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8e90e8640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40d06c0 0x7fb8e414d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4156570 0x7fb8e40d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 -- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40d06c0 msgr2=0x7fb8e414d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40d06c0 0x7fb8e414d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 -- 172.21.15.80:0/2298927693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8e4152c30 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8ea0ea640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e4156980 0x7fb8e414de70 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4070120 tx=0x7fb8dc011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.549+0000 7fb8e27fc640 1 -- 172.21.15.80:0/2298927693 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc003620 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8e27fc640 1 -- 172.21.15.80:0/2298927693 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8dc01b070 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8e4152f10 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8e27fc640 1 -- 172.21.15.80:0/2298927693 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc003a20 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8e414e480 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8e41535d0 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8e27fc640 1 -- 172.21.15.80:0/2298927693 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb8dc003bc0 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8e27fc640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8d006f400 0x7fb8d0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8e27fc640 1 -- 172.21.15.80:0/2298927693 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7fb8dc08eff0 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8e27fc640 1 -- 172.21.15.80:0/2298927693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fb8d00770a0 con 0x7fb8e4156980 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8e98e9640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8d006f400 0x7fb8d0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.553+0000 7fb8e98e9640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8d006f400 0x7fb8d0071910 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40ad000 tx=0x7fb8e40b0bf0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.557+0000 7fb8e27fc640 1 -- 172.21.15.80:0/2298927693 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8e41535d0 con 0x7fb8e4156980 2024-04-25T13:50:12.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb8e406a3a0 con 0x7fb8e4156980 2024-04-25T13:50:12.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8e27fc640 1 -- 172.21.15.80:0/2298927693 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7fb8dc024020 con 0x7fb8e4156980 2024-04-25T13:50:12.714 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:50:12.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8d006f400 msgr2=0x7fb8d0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8d006f400 0x7fb8d0071910 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb8d40ad000 tx=0x7fb8e40b0bf0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e4156980 msgr2=0x7fb8e414de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e4156980 0x7fb8e414de70 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4070120 tx=0x7fb8dc011920 comp rx=0 tx=0).stop 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 shutdown_connections 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8d006f400 0x7fb8d0071910 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb8e4156980 0x7fb8e414de70 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb8e40d06c0 0x7fb8e414d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 --2- 172.21.15.80:0/2298927693 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8e4156570 0x7fb8e40d0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 >> 172.21.15.80:0/2298927693 conn(0x7fb8e40b4160 msgr2=0x7fb8e406e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 shutdown_connections 2024-04-25T13:50:12.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.717+0000 7fb8ebb4d640 1 -- 172.21.15.80:0/2298927693 wait complete. 2024-04-25T13:50:12.736 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.885+0000 7fbf02902640 1 Processor -- start 2024-04-25T13:50:12.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.885+0000 7fbf02902640 1 -- start start 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf02902640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fbefc06fb90 0x7fbefc160c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf02902640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc161160 0x7fbefc0d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf02902640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbefc0d2170 0x7fbefc0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf02902640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc070ea0 con 0x7fbefc0d2170 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf02902640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fbefc161570 con 0x7fbefc06fb90 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf02902640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc070290 con 0x7fbefc161160 2024-04-25T13:50:12.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf00e9f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbefc0d2170 0x7fbefc0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf00e9f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbefc0d2170 0x7fbefc0b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58050/0 (socket says 172.21.15.80:58050) 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf00e9f640 1 -- 172.21.15.80:0/42778568 learned_addr learned my addr 172.21.15.80:0/42778568 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbefbfff640 1 --2- 172.21.15.80:0/42778568 >> v2:172.21.15.87:3300/0 conn(0x7fbefc06fb90 0x7fbefc160c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbefb7fe640 1 --2- 172.21.15.80:0/42778568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc161160 0x7fbefc0d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf00e9f640 1 -- 172.21.15.80:0/42778568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc161160 msgr2=0x7fbefc0d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf00e9f640 1 --2- 172.21.15.80:0/42778568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc161160 0x7fbefc0d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf00e9f640 1 -- 172.21.15.80:0/42778568 >> v2:172.21.15.87:3300/0 conn(0x7fbefc06fb90 msgr2=0x7fbefc160c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf00e9f640 1 --2- 172.21.15.80:0/42778568 >> v2:172.21.15.87:3300/0 conn(0x7fbefc06fb90 0x7fbefc160c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.889+0000 7fbf00e9f640 1 -- 172.21.15.80:0/42778568 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc070570 con 0x7fbefc0d2170 2024-04-25T13:50:12.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf00e9f640 1 --2- 172.21.15.80:0/42778568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc0d2170 0x7fbefc0b5a70 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0b7960 tx=0x7fbeec01c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9fce1becad4a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbefaffd640 1 -- 172.21.15.80:0/42778568 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0a57b0 con 0x7fbefc0d2170 2024-04-25T13:50:12.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbefaffd640 1 -- 172.21.15.80:0/42778568 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbeec0a5950 con 0x7fbefc0d2170 2024-04-25T13:50:12.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbefaffd640 1 -- 172.21.15.80:0/42778568 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbeec0a5d20 con 0x7fbefc0d2170 2024-04-25T13:50:12.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 -- 172.21.15.80:0/42778568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc0d2170 msgr2=0x7fbefc0b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 --2- 172.21.15.80:0/42778568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc0d2170 0x7fbefc0b5a70 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0b7960 tx=0x7fbeec01c930 comp rx=0 tx=0).stop 2024-04-25T13:50:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 -- 172.21.15.80:0/42778568 shutdown_connections 2024-04-25T13:50:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 --2- 172.21.15.80:0/42778568 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc0d2170 0x7fbefc0b5a70 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 --2- 172.21.15.80:0/42778568 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc161160 0x7fbefc0d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 --2- 172.21.15.80:0/42778568 >> v2:172.21.15.87:3300/0 conn(0x7fbefc06fb90 0x7fbefc160c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 -- 172.21.15.80:0/42778568 >> 172.21.15.80:0/42778568 conn(0x7fbefc073790 msgr2=0x7fbefc073ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 -- 172.21.15.80:0/42778568 shutdown_connections 2024-04-25T13:50:12.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 -- 172.21.15.80:0/42778568 wait complete. 2024-04-25T13:50:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 Processor -- start 2024-04-25T13:50:12.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.893+0000 7fbf02902640 1 -- start start 2024-04-25T13:50:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc06fb90 0x7fbefc149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc0d2170 0x7fbefc14abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbefc161160 0x7fbefc14b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc15d7b0 con 0x7fbefc06fb90 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbefc15d630 con 0x7fbefc161160 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbefc15d930 con 0x7fbefc0d2170 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc06fb90 0x7fbefc149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc06fb90 0x7fbefc149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58062/0 (socket says 172.21.15.80:58062) 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 -- 172.21.15.80:0/2315433263 learned_addr learned my addr 172.21.15.80:0/2315433263 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf00e9f640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbefc161160 0x7fbefc14b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 -- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc0d2170 msgr2=0x7fbefc14abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefb7fe640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc0d2170 0x7fbefc14abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc0d2170 0x7fbefc14abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 -- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbefc161160 msgr2=0x7fbefc14b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:12.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbefc161160 0x7fbefc14b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 -- 172.21.15.80:0/2315433263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbefc14a470 con 0x7fbefc06fb90 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf00e9f640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbefc161160 0x7fbefc14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbefbfff640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc06fb90 0x7fbefc149f30 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbef400e870 tx=0x7fbef400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbef8ff9640 1 -- 172.21.15.80:0/2315433263 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4017070 con 0x7fbefc06fb90 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbef8ff9640 1 -- 172.21.15.80:0/2315433263 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbef400ee60 con 0x7fbefc06fb90 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbefc150cb0 con 0x7fbefc06fb90 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbef8ff9640 1 -- 172.21.15.80:0/2315433263 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef400cb80 con 0x7fbefc06fb90 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbefc14c950 con 0x7fbefc06fb90 2024-04-25T13:50:12.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbefc135f20 con 0x7fbefc06fb90 2024-04-25T13:50:12.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.897+0000 7fbef8ff9640 1 -- 172.21.15.80:0/2315433263 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbef401a070 con 0x7fbefc06fb90 2024-04-25T13:50:12.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.901+0000 7fbef8ff9640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbedc06f400 0x7fbedc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:12.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.901+0000 7fbef8ff9640 1 -- 172.21.15.80:0/2315433263 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7fbef408f180 con 0x7fbefc06fb90 2024-04-25T13:50:12.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.901+0000 7fbef8ff9640 1 -- 172.21.15.80:0/2315433263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fbedc077040 con 0x7fbefc06fb90 2024-04-25T13:50:12.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.901+0000 7fbefb7fe640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbedc06f400 0x7fbedc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:12.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.901+0000 7fbefb7fe640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbedc06f400 0x7fbedc071910 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbee40096f0 tx=0x7fbee4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:12.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:12.905+0000 7fbef8ff9640 1 -- 172.21.15.80:0/2315433263 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbef405c030 con 0x7fbefc06fb90 2024-04-25T13:50:13.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fbefc06a3a0 con 0x7fbefc06fb90 2024-04-25T13:50:13.062 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbef8ff9640 1 -- 172.21.15.80:0/2315433263 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) ==== 104+0+11 (secure 0 0 0) 0x7fbef405b930 con 0x7fbefc06fb90 2024-04-25T13:50:13.062 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:50:13.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbedc06f400 msgr2=0x7fbedc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbedc06f400 0x7fbedc071910 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fbee40096f0 tx=0x7fbee4008040 comp rx=0 tx=0).stop 2024-04-25T13:50:13.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc06fb90 msgr2=0x7fbefc149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc06fb90 0x7fbefc149f30 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fbef400e870 tx=0x7fbef400ed30 comp rx=0 tx=0).stop 2024-04-25T13:50:13.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 shutdown_connections 2024-04-25T13:50:13.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbedc06f400 0x7fbedc071910 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbefc161160 0x7fbefc14b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbefc0d2170 0x7fbefc14abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 --2- 172.21.15.80:0/2315433263 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbefc06fb90 0x7fbefc149f30 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 >> 172.21.15.80:0/2315433263 conn(0x7fbefc073790 msgr2=0x7fbefc14d160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:13.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 shutdown_connections 2024-04-25T13:50:13.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.065+0000 7fbf02902640 1 -- 172.21.15.80:0/2315433263 wait complete. 2024-04-25T13:50:13.084 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:50:13.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.233+0000 7f3585d7e640 1 Processor -- start 2024-04-25T13:50:13.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.233+0000 7f3585d7e640 1 -- start start 2024-04-25T13:50:13.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.233+0000 7f3585d7e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35801602a0 0x7f35800ceb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f3585d7e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f35800d0ae0 0x7f35800d0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f3585d7e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35800d26c0 con 0x7f3580068590 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f3585d7e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f358006f270 con 0x7f35801602a0 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f3585d7e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f358006f550 con 0x7f35800d0ae0 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35801602a0 0x7f35800ceb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35801602a0 0x7f35800ceb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58078/0 (socket says 172.21.15.80:58078) 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357f7fe640 1 -- 172.21.15.80:0/3072387935 learned_addr learned my addr 172.21.15.80:0/3072387935 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357effd640 1 --2- 172.21.15.80:0/3072387935 >> v2:172.21.15.87:3300/0 conn(0x7f35800d0ae0 0x7f35800d0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357f7fe640 1 -- 172.21.15.80:0/3072387935 >> v2:172.21.15.87:3300/0 conn(0x7f35800d0ae0 msgr2=0x7f35800d0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357f7fe640 1 --2- 172.21.15.80:0/3072387935 >> v2:172.21.15.87:3300/0 conn(0x7f35800d0ae0 0x7f35800d0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357f7fe640 1 -- 172.21.15.80:0/3072387935 >> v1:172.21.15.87:6789/0 conn(0x7f3580068590 legacy=0x7f35800689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:13.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357f7fe640 1 -- 172.21.15.80:0/3072387935 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35800d21a0 con 0x7f35801602a0 2024-04-25T13:50:13.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357f7fe640 1 --2- 172.21.15.80:0/3072387935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35801602a0 0x7f35800ceb90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f3580074220 tx=0x7f357001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37268ffd2cd513a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:13.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357e7fc640 1 -- 172.21.15.80:0/3072387935 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700b0070 con 0x7f35801602a0 2024-04-25T13:50:13.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357e7fc640 1 -- 172.21.15.80:0/3072387935 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35700ab600 con 0x7f35801602a0 2024-04-25T13:50:13.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.237+0000 7f357e7fc640 1 -- 172.21.15.80:0/3072387935 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700ab9d0 con 0x7f35801602a0 2024-04-25T13:50:13.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- 172.21.15.80:0/3072387935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35801602a0 msgr2=0x7f35800ceb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 --2- 172.21.15.80:0/3072387935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35801602a0 0x7f35800ceb90 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f3580074220 tx=0x7f357001c3c0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- 172.21.15.80:0/3072387935 shutdown_connections 2024-04-25T13:50:13.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 --2- 172.21.15.80:0/3072387935 >> v2:172.21.15.87:3300/0 conn(0x7f35800d0ae0 0x7f35800d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 --2- 172.21.15.80:0/3072387935 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35801602a0 0x7f35800ceb90 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- 172.21.15.80:0/3072387935 >> 172.21.15.80:0/3072387935 conn(0x7f35800cfa50 msgr2=0x7f35800cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:13.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- 172.21.15.80:0/3072387935 shutdown_connections 2024-04-25T13:50:13.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- 172.21.15.80:0/3072387935 wait complete. 2024-04-25T13:50:13.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 Processor -- start 2024-04-25T13:50:13.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- start start 2024-04-25T13:50:13.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3580068590 0x7f358014d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f35800d0ae0 0x7f358015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35801602a0 0x7f358015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3580160830 con 0x7f3580068590 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f35801606b0 con 0x7f35801602a0 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f3585d7e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3580071730 con 0x7f35800d0ae0 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3580068590 0x7f358014d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f357f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3580068590 0x7f358014d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58086/0 (socket says 172.21.15.80:58086) 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.241+0000 7f357f7fe640 1 -- 172.21.15.80:0/2051562640 learned_addr learned my addr 172.21.15.80:0/2051562640 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357ffff640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35801602a0 0x7f358015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357effd640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f35800d0ae0 0x7f358015abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357f7fe640 1 -- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f35800d0ae0 msgr2=0x7f358015abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357f7fe640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f35800d0ae0 0x7f358015abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357f7fe640 1 -- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35801602a0 msgr2=0x7f358015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357f7fe640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35801602a0 0x7f358015b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357f7fe640 1 -- 172.21.15.80:0/2051562640 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3580162090 con 0x7f3580068590 2024-04-25T13:50:13.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357ffff640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35801602a0 0x7f358015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:13.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357f7fe640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3580068590 0x7f358014d9f0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f35700ae040 tx=0x7f3570019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:13.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3567fff640 1 -- 172.21.15.80:0/2051562640 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700ba020 con 0x7f3580068590 2024-04-25T13:50:13.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3567fff640 1 -- 172.21.15.80:0/2051562640 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35700b6420 con 0x7f3580068590 2024-04-25T13:50:13.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3580162370 con 0x7f3580068590 2024-04-25T13:50:13.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3567fff640 1 -- 172.21.15.80:0/2051562640 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35700b0040 con 0x7f3580068590 2024-04-25T13:50:13.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3580162930 con 0x7f3580068590 2024-04-25T13:50:13.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358006a3a0 con 0x7f3580068590 2024-04-25T13:50:13.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3567fff640 1 -- 172.21.15.80:0/2051562640 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f35700bd020 con 0x7f3580068590 2024-04-25T13:50:13.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3567fff640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f356c06f400 0x7f356c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3567fff640 1 -- 172.21.15.80:0/2051562640 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f35701292d0 con 0x7f3580068590 2024-04-25T13:50:13.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f3567fff640 1 -- 172.21.15.80:0/2051562640 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f356c077040 con 0x7f3580068590 2024-04-25T13:50:13.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357effd640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f356c06f400 0x7f356c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.245+0000 7f357effd640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f356c06f400 0x7f356c071910 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f358015ef20 tx=0x7f3574008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:13.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.249+0000 7f3567fff640 1 -- 172.21.15.80:0/2051562640 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f35700fe030 con 0x7f3580068590 2024-04-25T13:50:13.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.401+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f35800d0110 con 0x7f3580068590 2024-04-25T13:50:13.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.401+0000 7f3567fff640 1 -- 172.21.15.80:0/2051562640 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) ==== 100+0+11 (secure 0 0 0) 0x7f35700c4070 con 0x7f3580068590 2024-04-25T13:50:13.400 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:50:13.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f356c06f400 msgr2=0x7f356c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f356c06f400 0x7f356c071910 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f358015ef20 tx=0x7f3574008040 comp rx=0 tx=0).stop 2024-04-25T13:50:13.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3580068590 msgr2=0x7f358014d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3580068590 0x7f358014d9f0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f35700ae040 tx=0x7f3570019160 comp rx=0 tx=0).stop 2024-04-25T13:50:13.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 shutdown_connections 2024-04-25T13:50:13.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f356c06f400 0x7f356c071910 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35801602a0 0x7f358015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f35800d0ae0 0x7f358015abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 --2- 172.21.15.80:0/2051562640 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3580068590 0x7f358014d9f0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 >> 172.21.15.80:0/2051562640 conn(0x7f35800cfa50 msgr2=0x7f35800b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:13.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 shutdown_connections 2024-04-25T13:50:13.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.405+0000 7f3585d7e640 1 -- 172.21.15.80:0/2051562640 wait complete. 2024-04-25T13:50:13.422 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:50:13.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97ebef640 1 Processor -- start 2024-04-25T13:50:13.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97ebef640 1 -- start start 2024-04-25T13:50:13.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97ebef640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa978161f90 0x7fa9780cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97ebef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa97815ec40 0x7fa97815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97ebef640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9780d0d70 con 0x7fa9780d0510 2024-04-25T13:50:13.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97ebef640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa9780d1050 con 0x7fa978161f90 2024-04-25T13:50:13.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97ebef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa97806dca0 con 0x7fa97815ec40 2024-04-25T13:50:13.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97d18c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa97815ec40 0x7fa97815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97d18c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa97815ec40 0x7fa97815f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:50488/0 (socket says 172.21.15.80:50488) 2024-04-25T13:50:13.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97c98b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa978161f90 0x7fa9780cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa977fff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7fa9780d0510 0x7fa97815e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:39334/0 (socket says 172.21.15.80:39334) 2024-04-25T13:50:13.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.565+0000 7fa97d18c640 1 -- 172.21.15.80:0/2403087362 learned_addr learned my addr 172.21.15.80:0/2403087362 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:13.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97d18c640 1 -- 172.21.15.80:0/2403087362 >> v2:172.21.15.87:3300/0 conn(0x7fa978161f90 msgr2=0x7fa9780cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:13.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97d18c640 1 --2- 172.21.15.80:0/2403087362 >> v2:172.21.15.87:3300/0 conn(0x7fa978161f90 0x7fa9780cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97d18c640 1 -- 172.21.15.80:0/2403087362 >> v1:172.21.15.87:6789/0 conn(0x7fa9780d0510 legacy=0x7fa97815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97d18c640 1 -- 172.21.15.80:0/2403087362 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa97806df80 con 0x7fa97815ec40 2024-04-25T13:50:13.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa9777fe640 1 -- 172.21.15.80:0/2403087362 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 436496488 0 0) 0x7fa9780d0d70 con 0x7fa9780d0510 2024-04-25T13:50:13.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97d18c640 1 --2- 172.21.15.80:0/2403087362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa97815ec40 0x7fa97815f050 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa960009870 tx=0x7fa96001c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f579274af4248a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:13.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa9777fe640 1 -- 172.21.15.80:0/2403087362 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600b1070 con 0x7fa97815ec40 2024-04-25T13:50:13.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa9777fe640 1 -- 172.21.15.80:0/2403087362 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9600ac600 con 0x7fa97815ec40 2024-04-25T13:50:13.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa9777fe640 1 -- 172.21.15.80:0/2403087362 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9600ac9d0 con 0x7fa97815ec40 2024-04-25T13:50:13.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97ebef640 1 -- 172.21.15.80:0/2403087362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa97815ec40 msgr2=0x7fa97815f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97ebef640 1 --2- 172.21.15.80:0/2403087362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa97815ec40 0x7fa97815f050 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa960009870 tx=0x7fa96001c470 comp rx=0 tx=0).stop 2024-04-25T13:50:13.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97ebef640 1 -- 172.21.15.80:0/2403087362 shutdown_connections 2024-04-25T13:50:13.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97ebef640 1 --2- 172.21.15.80:0/2403087362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa97815ec40 0x7fa97815f050 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97ebef640 1 --2- 172.21.15.80:0/2403087362 >> v2:172.21.15.87:3300/0 conn(0x7fa978161f90 0x7fa9780cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97ebef640 1 -- 172.21.15.80:0/2403087362 >> 172.21.15.80:0/2403087362 conn(0x7fa97806d080 msgr2=0x7fa9780b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:13.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.569+0000 7fa97ebef640 1 -- 172.21.15.80:0/2403087362 shutdown_connections 2024-04-25T13:50:13.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 -- 172.21.15.80:0/2403087362 wait complete. 2024-04-25T13:50:13.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 Processor -- start 2024-04-25T13:50:13.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 -- start start 2024-04-25T13:50:13.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9780d0510 0x7fa9781589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa97815ec40 0x7fa978071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa978161f90 0x7fa978072d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa97806f4b0 con 0x7fa97815ec40 2024-04-25T13:50:13.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa97806f630 con 0x7fa978161f90 2024-04-25T13:50:13.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97ebef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa9780d0920 con 0x7fa9780d0510 2024-04-25T13:50:13.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa97815ec40 0x7fa978071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa97815ec40 0x7fa978071e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58102/0 (socket says 172.21.15.80:58102) 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 -- 172.21.15.80:0/247312605 learned_addr learned my addr 172.21.15.80:0/247312605 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97d18c640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa978161f90 0x7fa978072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa97c98b640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9780d0510 0x7fa9781589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 -- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9780d0510 msgr2=0x7fa9781589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9780d0510 0x7fa9781589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 -- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa978161f90 msgr2=0x7fa978072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa978161f90 0x7fa978072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 -- 172.21.15.80:0/247312605 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9780b6010 con 0x7fa97815ec40 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.573+0000 7fa977fff640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa97815ec40 0x7fa978071e80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa96800c7f0 tx=0x7fa96800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa9757fa640 1 -- 172.21.15.80:0/247312605 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa968018070 con 0x7fa97815ec40 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa9757fa640 1 -- 172.21.15.80:0/247312605 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96800a5e0 con 0x7fa97815ec40 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9780b62f0 con 0x7fa97815ec40 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa9757fa640 1 -- 172.21.15.80:0/247312605 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96800a9d0 con 0x7fa97815ec40 2024-04-25T13:50:13.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9780b68b0 con 0x7fa97815ec40 2024-04-25T13:50:13.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa97806a3a0 con 0x7fa97815ec40 2024-04-25T13:50:13.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa9757fa640 1 -- 172.21.15.80:0/247312605 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa96801b070 con 0x7fa97815ec40 2024-04-25T13:50:13.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa9757fa640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa95806f400 0x7fa958071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:13.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa9757fa640 1 -- 172.21.15.80:0/247312605 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7fa96808e920 con 0x7fa97815ec40 2024-04-25T13:50:13.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa9757fa640 1 -- 172.21.15.80:0/247312605 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7fa9580770a0 con 0x7fa97815ec40 2024-04-25T13:50:13.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa97c98b640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa95806f400 0x7fa958071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:13.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.577+0000 7fa97c98b640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa95806f400 0x7fa958071910 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa97806fa20 tx=0x7fa970008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:13.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.581+0000 7fa9757fa640 1 -- 172.21.15.80:0/247312605 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa96805b7d0 con 0x7fa97815ec40 2024-04-25T13:50:13.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.741+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa9780b16d0 con 0x7fa97815ec40 2024-04-25T13:50:13.740 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.741+0000 7fa9757fa640 1 -- 172.21.15.80:0/247312605 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v158) ==== 80+0+0 (secure 0 0 0) 0x7fa96805b0d0 con 0x7fa97815ec40 2024-04-25T13:50:13.740 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa95806f400 msgr2=0x7fa958071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa95806f400 0x7fa958071910 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa97806fa20 tx=0x7fa970008040 comp rx=0 tx=0).stop 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa97815ec40 msgr2=0x7fa978071e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa97815ec40 0x7fa978071e80 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fa96800c7f0 tx=0x7fa96800ccb0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 shutdown_connections 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa95806f400 0x7fa958071910 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa978161f90 0x7fa978072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa97815ec40 0x7fa978071e80 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 --2- 172.21.15.80:0/247312605 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa9780d0510 0x7fa9781589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 >> 172.21.15.80:0/247312605 conn(0x7fa97806d080 msgr2=0x7fa9780757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 shutdown_connections 2024-04-25T13:50:13.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:13.745+0000 7fa97ebef640 1 -- 172.21.15.80:0/247312605 wait complete. 2024-04-25T13:50:13.761 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:50:13.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:13.761 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:50:13.774 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25409]: fuse finished with error 0 and tester_r 0 2024-04-25T13:50:13.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:50:13.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:13.777 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:50:13.936 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:50:13.936 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:50:13.936 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:13.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:50:13.942 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:50:13.942 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:50:13.942 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:13.983 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:50:13.984 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:14.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f6592a70640 1 Processor -- start 2024-04-25T13:50:14.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f6592a70640 1 -- start start 2024-04-25T13:50:14.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f6592a70640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f658c161f90 0x7f658c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f6592a70640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c0d0510 0x7f658c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f6592a70640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f658c15ea70 0x7f658c15ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f6592a70640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f658c0d0d70 con 0x7f658c15ea70 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f6592a70640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f658c0d1050 con 0x7f658c161f90 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f6592a70640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f658c06dca0 con 0x7f658c0d0510 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659100d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f658c15ea70 0x7f658c15ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659100d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f658c15ea70 0x7f658c15ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58118/0 (socket says 172.21.15.80:58118) 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659100d640 1 -- 172.21.15.80:0/3103210533 learned_addr learned my addr 172.21.15.80:0/3103210533 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659080c640 1 --2- 172.21.15.80:0/3103210533 >> v2:172.21.15.87:3300/0 conn(0x7f658c161f90 0x7f658c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f658bfff640 1 --2- 172.21.15.80:0/3103210533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c0d0510 0x7f658c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659100d640 1 -- 172.21.15.80:0/3103210533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c0d0510 msgr2=0x7f658c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659100d640 1 --2- 172.21.15.80:0/3103210533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c0d0510 0x7f658c15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659100d640 1 -- 172.21.15.80:0/3103210533 >> v2:172.21.15.87:3300/0 conn(0x7f658c161f90 msgr2=0x7f658c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:14.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659100d640 1 --2- 172.21.15.80:0/3103210533 >> v2:172.21.15.87:3300/0 conn(0x7f658c161f90 0x7f658c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659100d640 1 -- 172.21.15.80:0/3103210533 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c06df80 con 0x7f658c15ea70 2024-04-25T13:50:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f659080c640 1 --2- 172.21.15.80:0/3103210533 >> v2:172.21.15.87:3300/0 conn(0x7f658c161f90 0x7f658c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:14.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.141+0000 7f658bfff640 1 --2- 172.21.15.80:0/3103210533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c0d0510 0x7f658c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:14.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f659100d640 1 --2- 172.21.15.80:0/3103210533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c15ea70 0x7f658c15ee80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f658c0d17b0 tx=0x7f658401c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b450c5fd74c6a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f658b7fe640 1 -- 172.21.15.80:0/3103210533 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840a5770 con 0x7f658c15ea70 2024-04-25T13:50:14.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f658b7fe640 1 -- 172.21.15.80:0/3103210533 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65840a5910 con 0x7f658c15ea70 2024-04-25T13:50:14.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f658b7fe640 1 -- 172.21.15.80:0/3103210533 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65840a5ce0 con 0x7f658c15ea70 2024-04-25T13:50:14.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 -- 172.21.15.80:0/3103210533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c15ea70 msgr2=0x7f658c15ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 --2- 172.21.15.80:0/3103210533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c15ea70 0x7f658c15ee80 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f658c0d17b0 tx=0x7f658401c930 comp rx=0 tx=0).stop 2024-04-25T13:50:14.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 -- 172.21.15.80:0/3103210533 shutdown_connections 2024-04-25T13:50:14.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 --2- 172.21.15.80:0/3103210533 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c15ea70 0x7f658c15ee80 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 --2- 172.21.15.80:0/3103210533 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c0d0510 0x7f658c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 --2- 172.21.15.80:0/3103210533 >> v2:172.21.15.87:3300/0 conn(0x7f658c161f90 0x7f658c0cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 -- 172.21.15.80:0/3103210533 >> 172.21.15.80:0/3103210533 conn(0x7f658c06d080 msgr2=0x7f658c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:14.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 -- 172.21.15.80:0/3103210533 shutdown_connections 2024-04-25T13:50:14.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.145+0000 7f6592a70640 1 -- 172.21.15.80:0/3103210533 wait complete. 2024-04-25T13:50:14.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 Processor -- start 2024-04-25T13:50:14.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 -- start start 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c0d0510 0x7f658c147050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c15ea70 0x7f658c15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c161f90 0x7f658c0b5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f658c1447a0 con 0x7f658c0d0510 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f658c144620 con 0x7f658c161f90 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f658c0d0920 con 0x7f658c15ea70 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c0d0510 0x7f658c147050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c0d0510 0x7f658c147050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58130/0 (socket says 172.21.15.80:58130) 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 -- 172.21.15.80:0/2823570013 learned_addr learned my addr 172.21.15.80:0/2823570013 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659100d640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c161f90 0x7f658c0b5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 -- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c15ea70 msgr2=0x7f658c15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c15ea70 0x7f658c15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 -- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c161f90 msgr2=0x7f658c0b5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c161f90 0x7f658c0b5fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 -- 172.21.15.80:0/2823570013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f658c15cb00 con 0x7f658c0d0510 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f659080c640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c0d0510 0x7f658c147050 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f658000c820 tx=0x7f658000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f65897fa640 1 -- 172.21.15.80:0/2823570013 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580015070 con 0x7f658c0d0510 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f65897fa640 1 -- 172.21.15.80:0/2823570013 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6580007a30 con 0x7f658c0d0510 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f658c15cde0 con 0x7f658c0d0510 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f65897fa640 1 -- 172.21.15.80:0/2823570013 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6580007e20 con 0x7f658c0d0510 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f658c15d3a0 con 0x7f658c0d0510 2024-04-25T13:50:14.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.149+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f658c06a3a0 con 0x7f658c0d0510 2024-04-25T13:50:14.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.153+0000 7f65897fa640 1 -- 172.21.15.80:0/2823570013 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6580018070 con 0x7f658c0d0510 2024-04-25T13:50:14.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.153+0000 7f65897fa640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657006f400 0x7f6570071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.153+0000 7f65897fa640 1 -- 172.21.15.80:0/2823570013 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f658008f1f0 con 0x7f658c0d0510 2024-04-25T13:50:14.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.153+0000 7f65897fa640 1 -- 172.21.15.80:0/2823570013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f65700770a0 con 0x7f658c0d0510 2024-04-25T13:50:14.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.153+0000 7f658bfff640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657006f400 0x7f6570071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.153+0000 7f658bfff640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657006f400 0x7f6570071910 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6578006b20 tx=0x7f6578008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.157+0000 7f65897fa640 1 -- 172.21.15.80:0/2823570013 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f658005c0a0 con 0x7f658c0d0510 2024-04-25T13:50:14.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.349+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f658c0b43f0 con 0x7f658c0d0510 2024-04-25T13:50:14.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.349+0000 7f65897fa640 1 -- 172.21.15.80:0/2823570013 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7f658005b9a0 con 0x7f658c0d0510 2024-04-25T13:50:14.348 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:14.348 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/521602421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":521602421},{"type":"v1","addr":"172.21.15.87:6835","nonce":521602421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4170043750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4170043750},{"type":"v1","addr":"172.21.15.87:6837","nonce":4170043750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/1423926990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1423926990},{"type":"v1","addr":"172.21.15.80:6834","nonce":1423926990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:48:09.907104+0000","modified":"2024-04-25T13:48:12.977708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1134081162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1134081162},{"type":"v1","addr":"172.21.15.80:6835","nonce":1134081162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-25T13:50:14.348 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 74 2024-04-25T13:50:14.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657006f400 msgr2=0x7f6570071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657006f400 0x7f6570071910 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6578006b20 tx=0x7f6578008040 comp rx=0 tx=0).stop 2024-04-25T13:50:14.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c0d0510 msgr2=0x7f658c147050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c0d0510 0x7f658c147050 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f658000c820 tx=0x7f658000cce0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 shutdown_connections 2024-04-25T13:50:14.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f657006f400 0x7f6570071910 secure :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6578006b20 tx=0x7f6578008040 comp rx=0 tx=0).stop 2024-04-25T13:50:14.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c161f90 0x7f658c0b5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f658c15ea70 0x7f658c15c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 --2- 172.21.15.80:0/2823570013 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f658c0d0510 0x7f658c147050 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 >> 172.21.15.80:0/2823570013 conn(0x7f658c06d080 msgr2=0x7f658c1608e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:14.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 shutdown_connections 2024-04-25T13:50:14.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.353+0000 7f6592a70640 1 -- 172.21.15.80:0/2823570013 wait complete. 2024-04-25T13:50:14.369 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:14.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f216b789640 1 Processor -- start 2024-04-25T13:50:14.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f216b789640 1 -- start start 2024-04-25T13:50:14.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f216b789640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21640cf320 0x7f2164161ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f216b789640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f21640cf730 0x7f2164162420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f216b789640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f21640ce5a0 con 0x7f21640d45b0 2024-04-25T13:50:14.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f216b789640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f21640d49c0 con 0x7f21640cf730 2024-04-25T13:50:14.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f216b789640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f21640d0fb0 con 0x7f21640cf320 2024-04-25T13:50:14.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f2169525640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f21640d45b0 0x7f21640b4e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58404/0 (socket says 172.21.15.80:58404) 2024-04-25T13:50:14.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f2169525640 1 -- 172.21.15.80:0/3436525395 learned_addr learned my addr 172.21.15.80:0/3436525395 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f2169d26640 1 --2- 172.21.15.80:0/3436525395 >> v2:172.21.15.87:3300/0 conn(0x7f21640cf730 0x7f2164162420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f2168d24640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21640cf320 0x7f2164161ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1081937334 0 0) 0x7f21640ce5a0 con 0x7f21640d45b0 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2150003530 con 0x7f21640d45b0 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2664462063 0 0) 0x7f2150003530 con 0x7f21640d45b0 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21640cf320 msgr2=0x7f2164161ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21640cf320 0x7f2164161ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 >> v2:172.21.15.87:3300/0 conn(0x7f21640cf730 msgr2=0x7f2164162420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 --2- 172.21.15.80:0/3436525395 >> v2:172.21.15.87:3300/0 conn(0x7f21640cf730 0x7f2164162420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21640ce840 con 0x7f21640d45b0 2024-04-25T13:50:14.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.521+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2160001680 con 0x7f21640d45b0 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 >> v1:172.21.15.80:6789/0 conn(0x7f21640d45b0 legacy=0x7f21640b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21640cf320 0x7f21500039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150003f40 0x7f2150004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2150004d50 0x7f2150005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f21640d49c0 con 0x7f2150004d50 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f21640d0fb0 con 0x7f2150003f40 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f215000bab0 con 0x7f21640cf320 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169d26640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2150004d50 0x7f2150005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169525640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21640cf320 0x7f21500039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2168d24640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150003f40 0x7f2150004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169d26640 1 -- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21640cf320 msgr2=0x7f21500039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169d26640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f21640cf320 0x7f21500039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169d26640 1 -- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150003f40 msgr2=0x7f2150004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169d26640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150003f40 0x7f2150004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169d26640 1 -- 172.21.15.80:0/3436525395 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2150003530 con 0x7f2150004d50 2024-04-25T13:50:14.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2168d24640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150003f40 0x7f2150004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169d26640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2150004d50 0x7f2150005160 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f2164073fb0 tx=0x7f215c01cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9df062bcaa05af2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c0b2070 con 0x7f2150004d50 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215c0a5c70 con 0x7f2150004d50 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f215bfff640 1 -- 172.21.15.80:0/3436525395 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c009c10 con 0x7f2150004d50 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f216b789640 1 -- 172.21.15.80:0/3436525395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2150004d50 msgr2=0x7f2150005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f216b789640 1 --2- 172.21.15.80:0/3436525395 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2150004d50 0x7f2150005160 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f2164073fb0 tx=0x7f215c01cc60 comp rx=0 tx=0).stop 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f2169d26640 1 -- 172.21.15.80:0/3436525395 reap_dead start 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f216b789640 1 -- 172.21.15.80:0/3436525395 shutdown_connections 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f216b789640 1 -- 172.21.15.80:0/3436525395 >> 172.21.15.80:0/3436525395 conn(0x7f21640d3a90 msgr2=0x7f21640d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f216b789640 1 -- 172.21.15.80:0/3436525395 shutdown_connections 2024-04-25T13:50:14.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.525+0000 7f216b789640 1 -- 172.21.15.80:0/3436525395 wait complete. 2024-04-25T13:50:14.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 Processor -- start 2024-04-25T13:50:14.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 -- start start 2024-04-25T13:50:14.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150004d50 0x7f216409e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216415eec0 0x7f216415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f216415a900 0x7f216406d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f21640b9370 con 0x7f216415a900 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f21640cee00 con 0x7f2150004d50 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2164161ee0 con 0x7f216415eec0 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f216415a900 0x7f216406d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f216415a900 0x7f216406d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58148/0 (socket says 172.21.15.80:58148) 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 -- 172.21.15.80:0/66897498 learned_addr learned my addr 172.21.15.80:0/66897498 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169525640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150004d50 0x7f216409e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2168d24640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216415eec0 0x7f216415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 -- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216415eec0 msgr2=0x7f216415f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216415eec0 0x7f216415f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 -- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150004d50 msgr2=0x7f216409e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150004d50 0x7f216409e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 -- 172.21.15.80:0/66897498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f216406c720 con 0x7f216415a900 2024-04-25T13:50:14.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2169d26640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f216415a900 0x7f216406d7c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f215c0b0040 tx=0x7f215c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2159ffb640 1 -- 172.21.15.80:0/66897498 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c0bb070 con 0x7f216415a900 2024-04-25T13:50:14.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2159ffb640 1 -- 172.21.15.80:0/66897498 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215c0adb70 con 0x7f216415a900 2024-04-25T13:50:14.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f216406ca00 con 0x7f216415a900 2024-04-25T13:50:14.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f2159ffb640 1 -- 172.21.15.80:0/66897498 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f215c0b2040 con 0x7f216415a900 2024-04-25T13:50:14.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.529+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f216406cfc0 con 0x7f216415a900 2024-04-25T13:50:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.533+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f216406a3a0 con 0x7f216415a900 2024-04-25T13:50:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.533+0000 7f2159ffb640 1 -- 172.21.15.80:0/66897498 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f215c0be070 con 0x7f216415a900 2024-04-25T13:50:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.533+0000 7f2159ffb640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f214806f400 0x7f2148071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.533+0000 7f2159ffb640 1 -- 172.21.15.80:0/66897498 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f215c1295d0 con 0x7f216415a900 2024-04-25T13:50:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.533+0000 7f2159ffb640 1 -- 172.21.15.80:0/66897498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f21480770a0 con 0x7f216415a900 2024-04-25T13:50:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.533+0000 7f2169525640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f214806f400 0x7f2148071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.533+0000 7f2169525640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f214806f400 0x7f2148071910 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2160000c10 tx=0x7f2160009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.537+0000 7f2159ffb640 1 -- 172.21.15.80:0/66897498 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f215c0a5e20 con 0x7f216415a900 2024-04-25T13:50:14.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.701+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21640704b0 con 0x7f216415a900 2024-04-25T13:50:14.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.701+0000 7f2159ffb640 1 -- 172.21.15.80:0/66897498 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) ==== 74+0+16674 (secure 0 0 0) 0x7f215c016970 con 0x7f216415a900 2024-04-25T13:50:14.701 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:14.701 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":158,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:49:21.252418+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:48:07.666863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T13:48:08.875350+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.3","mappings":[{"from":2,"to":0}]},{"pgid":"19.a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:14.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f214806f400 msgr2=0x7f2148071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f214806f400 0x7f2148071910 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f2160000c10 tx=0x7f2160009040 comp rx=0 tx=0).stop 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f216415a900 msgr2=0x7f216406d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f216415a900 0x7f216406d7c0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f215c0b0040 tx=0x7f215c009750 comp rx=0 tx=0).stop 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 shutdown_connections 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f214806f400 0x7f2148071910 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f216415a900 0x7f216406d7c0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f216415eec0 0x7f216415f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 --2- 172.21.15.80:0/66897498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2150004d50 0x7f216409e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 >> 172.21.15.80:0/66897498 conn(0x7f21640d3a90 msgr2=0x7f21640b6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 shutdown_connections 2024-04-25T13:50:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.705+0000 7f216b789640 1 -- 172.21.15.80:0/66897498 wait complete. 2024-04-25T13:50:14.724 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:48:07.666863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '153', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:48:08.875350+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:50:14.724 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:50:14.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8439d20640 1 Processor -- start 2024-04-25T13:50:14.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8439d20640 1 -- start start 2024-04-25T13:50:14.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8439d20640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8434161170 0x7f84341574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8439d20640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434073c40 0x7f84340b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8439d20640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f843406fbf0 con 0x7f8434071e60 2024-04-25T13:50:14.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8439d20640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f84340b1350 con 0x7f8434161170 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8439d20640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f843406f9a0 con 0x7f8434073c40 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8432ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8434161170 0x7f84341574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f84337fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8434071e60 0x7f8434160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58414/0 (socket says 172.21.15.80:58414) 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f84337fe640 1 -- 172.21.15.80:0/508436503 learned_addr learned my addr 172.21.15.80:0/508436503 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8433fff640 1 --2- 172.21.15.80:0/508436503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434073c40 0x7f84340b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8432ffd640 1 -- 172.21.15.80:0/508436503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434073c40 msgr2=0x7f84340b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8432ffd640 1 --2- 172.21.15.80:0/508436503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434073c40 0x7f84340b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8432ffd640 1 -- 172.21.15.80:0/508436503 >> v1:172.21.15.80:6789/0 conn(0x7f8434071e60 legacy=0x7f8434160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f8432ffd640 1 -- 172.21.15.80:0/508436503 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84340b0a20 con 0x7f8434161170 2024-04-25T13:50:14.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.865+0000 7f84327fc640 1 -- 172.21.15.80:0/508436503 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2252834341 0 0) 0x7f843406fbf0 con 0x7f8434071e60 2024-04-25T13:50:14.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8432ffd640 1 --2- 172.21.15.80:0/508436503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434161170 0x7f84341574c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f84280097c0 tx=0x7f842801c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a5efcb81aee7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f84327fc640 1 -- 172.21.15.80:0/508436503 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280b2070 con 0x7f8434161170 2024-04-25T13:50:14.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f84327fc640 1 -- 172.21.15.80:0/508436503 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84280ad640 con 0x7f8434161170 2024-04-25T13:50:14.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f84327fc640 1 -- 172.21.15.80:0/508436503 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84280ada10 con 0x7f8434161170 2024-04-25T13:50:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8439d20640 1 -- 172.21.15.80:0/508436503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434161170 msgr2=0x7f84341574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8439d20640 1 --2- 172.21.15.80:0/508436503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434161170 0x7f84341574c0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f84280097c0 tx=0x7f842801c4a0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8439d20640 1 -- 172.21.15.80:0/508436503 shutdown_connections 2024-04-25T13:50:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8439d20640 1 --2- 172.21.15.80:0/508436503 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434073c40 0x7f84340b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8439d20640 1 --2- 172.21.15.80:0/508436503 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434161170 0x7f84341574c0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8439d20640 1 -- 172.21.15.80:0/508436503 >> 172.21.15.80:0/508436503 conn(0x7f84340b4160 msgr2=0x7f84340d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8439d20640 1 -- 172.21.15.80:0/508436503 shutdown_connections 2024-04-25T13:50:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.869+0000 7f8439d20640 1 -- 172.21.15.80:0/508436503 wait complete. 2024-04-25T13:50:14.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 Processor -- start 2024-04-25T13:50:14.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 -- start start 2024-04-25T13:50:14.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434071e60 0x7f8434153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8434073c40 0x7f843414ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434151c70 0x7f843414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8434169bb0 con 0x7f8434151c70 2024-04-25T13:50:14.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8434074050 con 0x7f8434073c40 2024-04-25T13:50:14.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8434169d30 con 0x7f8434071e60 2024-04-25T13:50:14.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434151c70 0x7f843414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434151c70 0x7f843414c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58168/0 (socket says 172.21.15.80:58168) 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 -- 172.21.15.80:0/3090181518 learned_addr learned my addr 172.21.15.80:0/3090181518 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f84337fe640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434071e60 0x7f8434153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8432ffd640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8434073c40 0x7f843414ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 -- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434071e60 msgr2=0x7f8434153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434071e60 0x7f8434153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 -- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8434073c40 msgr2=0x7f843414ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8434073c40 0x7f843414ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 -- 172.21.15.80:0/3090181518 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84341523e0 con 0x7f8434151c70 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8432ffd640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8434073c40 0x7f843414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8433fff640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434151c70 0x7f843414c760 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f842c007d70 tx=0x7f842c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f841bfff640 1 -- 172.21.15.80:0/3090181518 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c014420 con 0x7f8434151c70 2024-04-25T13:50:14.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f841bfff640 1 -- 172.21.15.80:0/3090181518 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f842c019070 con 0x7f8434151c70 2024-04-25T13:50:14.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84341526c0 con 0x7f8434151c70 2024-04-25T13:50:14.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f841bfff640 1 -- 172.21.15.80:0/3090181518 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c014820 con 0x7f8434151c70 2024-04-25T13:50:14.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.873+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8434152c80 con 0x7f8434151c70 2024-04-25T13:50:14.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.877+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8434152130 con 0x7f8434151c70 2024-04-25T13:50:14.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.877+0000 7f841bfff640 1 -- 172.21.15.80:0/3090181518 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f842c0149c0 con 0x7f8434151c70 2024-04-25T13:50:14.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.877+0000 7f841bfff640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842006f400 0x7f8420071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:14.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.877+0000 7f841bfff640 1 -- 172.21.15.80:0/3090181518 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f842c08fac0 con 0x7f8434151c70 2024-04-25T13:50:14.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.877+0000 7f841bfff640 1 -- 172.21.15.80:0/3090181518 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f84200770a0 con 0x7f8434151c70 2024-04-25T13:50:14.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.877+0000 7f84337fe640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842006f400 0x7f8420071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:14.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.877+0000 7f84337fe640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842006f400 0x7f8420071910 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8424000f70 tx=0x7f8424007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:14.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:14.881+0000 7f841bfff640 1 -- 172.21.15.80:0/3090181518 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f842c05ca00 con 0x7f8434151c70 2024-04-25T13:50:15.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f843406a3a0 con 0x7f8434151c70 2024-04-25T13:50:15.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f841bfff640 1 -- 172.21.15.80:0/3090181518 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) ==== 93+0+10 (secure 0 0 0) 0x7f842c05c300 con 0x7f8434151c70 2024-04-25T13:50:15.027 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842006f400 msgr2=0x7f8420071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842006f400 0x7f8420071910 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f8424000f70 tx=0x7f8424007c10 comp rx=0 tx=0).stop 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434151c70 msgr2=0x7f843414c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434151c70 0x7f843414c760 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f842c007d70 tx=0x7f842c00e4b0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 shutdown_connections 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f842006f400 0x7f8420071910 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8434151c70 0x7f843414c760 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8434073c40 0x7f843414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 --2- 172.21.15.80:0/3090181518 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8434071e60 0x7f8434153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.029+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 >> 172.21.15.80:0/3090181518 conn(0x7f84340b4160 msgr2=0x7f84340728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:15.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.033+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 shutdown_connections 2024-04-25T13:50:15.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.033+0000 7f8439d20640 1 -- 172.21.15.80:0/3090181518 wait complete. 2024-04-25T13:50:15.047 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:50:15.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b37510640 1 Processor -- start 2024-04-25T13:50:15.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b37510640 1 -- start start 2024-04-25T13:50:15.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b37510640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4b300d3ed0 0x7f4b300d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b37510640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300ba770 0x7f4b300b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b37510640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b300b92a0 con 0x7f4b300d2ad0 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b37510640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4b300d4c10 con 0x7f4b300d3ed0 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b37510640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4b300d4ef0 con 0x7f4b300ba770 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b34aab640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4b300d2ad0 0x7f4b300b76d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58424/0 (socket says 172.21.15.80:58424) 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b34aab640 1 -- 172.21.15.80:0/1964995465 learned_addr learned my addr 172.21.15.80:0/1964995465 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3573788782 0 0) 0x7f4b20001120 con 0x7f4b300d2ad0 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b352ac640 1 --2- 172.21.15.80:0/1964995465 >> v2:172.21.15.87:3300/0 conn(0x7f4b300d3ed0 0x7f4b300d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b1c003530 con 0x7f4b300d2ad0 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b35aad640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300ba770 0x7f4b300b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3236152491 0 0) 0x7f4b1c003530 con 0x7f4b300d2ad0 2024-04-25T13:50:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300ba770 msgr2=0x7f4b300b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b27fff640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300ba770 0x7f4b300b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 >> v2:172.21.15.87:3300/0 conn(0x7f4b300d3ed0 msgr2=0x7f4b300d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b27fff640 1 --2- 172.21.15.80:0/1964995465 >> v2:172.21.15.87:3300/0 conn(0x7f4b300d3ed0 0x7f4b300d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.189+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b300d51d0 con 0x7f4b300d2ad0 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4b20001850 con 0x7f4b300d2ad0 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 >> v1:172.21.15.80:6789/0 conn(0x7f4b300d2ad0 legacy=0x7f4b300b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b1c0039c0 0x7f4b1c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300ba770 0x7f4b1c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b1c004d50 0x7f4b1c0051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4b300d4c10 con 0x7f4b1c004d50 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4b300d4ef0 con 0x7f4b1c0039c0 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1c00bc60 con 0x7f4b300ba770 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b35aad640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b1c004d50 0x7f4b1c0051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b34aab640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300ba770 0x7f4b1c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b352ac640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b1c0039c0 0x7f4b1c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b35aad640 1 -- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300ba770 msgr2=0x7f4b1c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b35aad640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300ba770 0x7f4b1c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b35aad640 1 -- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b1c0039c0 msgr2=0x7f4b1c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b35aad640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b1c0039c0 0x7f4b1c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b35aad640 1 -- 172.21.15.80:0/1964995465 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1c003530 con 0x7f4b1c004d50 2024-04-25T13:50:15.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b35aad640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b1c004d50 0x7f4b1c0051a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f4b300d1d60 tx=0x7f4b2801c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7f23906bf3ea36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800b600 con 0x7f4b1c004d50 2024-04-25T13:50:15.193 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b280b1070 con 0x7f4b1c004d50 2024-04-25T13:50:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b27fff640 1 -- 172.21.15.80:0/1964995465 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280ac840 con 0x7f4b1c004d50 2024-04-25T13:50:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b37510640 1 -- 172.21.15.80:0/1964995465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b1c004d50 msgr2=0x7f4b1c0051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b37510640 1 --2- 172.21.15.80:0/1964995465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b1c004d50 0x7f4b1c0051a0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f4b300d1d60 tx=0x7f4b2801c630 comp rx=0 tx=0).stop 2024-04-25T13:50:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b35aad640 1 -- 172.21.15.80:0/1964995465 reap_dead start 2024-04-25T13:50:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b37510640 1 -- 172.21.15.80:0/1964995465 shutdown_connections 2024-04-25T13:50:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b37510640 1 -- 172.21.15.80:0/1964995465 >> 172.21.15.80:0/1964995465 conn(0x7f4b300737b0 msgr2=0x7f4b3006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b37510640 1 -- 172.21.15.80:0/1964995465 shutdown_connections 2024-04-25T13:50:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.193+0000 7f4b37510640 1 -- 172.21.15.80:0/1964995465 wait complete. 2024-04-25T13:50:15.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 Processor -- start 2024-04-25T13:50:15.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 -- start start 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b30151060 0x7f4b301fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300c5ef0 0x7f4b300c6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b300c6840 0x7f4b301497d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30160620 con 0x7f4b30151060 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4b301607a0 con 0x7f4b300c6840 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4b3016aa70 con 0x7f4b300c5ef0 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b30151060 0x7f4b301fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b30151060 0x7f4b301fcc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58178/0 (socket says 172.21.15.80:58178) 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 -- 172.21.15.80:0/2477910274 learned_addr learned my addr 172.21.15.80:0/2477910274 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b35aad640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b300c6840 0x7f4b301497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b34aab640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300c5ef0 0x7f4b300c6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 -- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300c5ef0 msgr2=0x7f4b300c6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300c5ef0 0x7f4b300c6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 -- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b300c6840 msgr2=0x7f4b301497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b300c6840 0x7f4b301497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 -- 172.21.15.80:0/2477910274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b300ceb00 con 0x7f4b30151060 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b352ac640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b30151060 0x7f4b301fcc30 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009c80 tx=0x7f4b2c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b25ffb640 1 -- 172.21.15.80:0/2477910274 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c014610 con 0x7f4b30151060 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b25ffb640 1 -- 172.21.15.80:0/2477910274 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b2c019070 con 0x7f4b30151060 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b300c7a00 con 0x7f4b30151060 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b25ffb640 1 -- 172.21.15.80:0/2477910274 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2c0149d0 con 0x7f4b30151060 2024-04-25T13:50:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.197+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b300c7fc0 con 0x7f4b30151060 2024-04-25T13:50:15.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.201+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b300d5ae0 con 0x7f4b30151060 2024-04-25T13:50:15.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.201+0000 7f4b25ffb640 1 -- 172.21.15.80:0/2477910274 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b2c014b70 con 0x7f4b30151060 2024-04-25T13:50:15.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.201+0000 7f4b25ffb640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b1406f400 0x7f4b14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.201+0000 7f4b25ffb640 1 -- 172.21.15.80:0/2477910274 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f4b2c08fca0 con 0x7f4b30151060 2024-04-25T13:50:15.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.201+0000 7f4b25ffb640 1 -- 172.21.15.80:0/2477910274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f4b140770a0 con 0x7f4b30151060 2024-04-25T13:50:15.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.201+0000 7f4b34aab640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b1406f400 0x7f4b14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.201+0000 7f4b34aab640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b1406f400 0x7f4b14071910 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4b20001300 tx=0x7f4b20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.205+0000 7f4b25ffb640 1 -- 172.21.15.80:0/2477910274 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b300d5ae0 con 0x7f4b30151060 2024-04-25T13:50:15.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4b3006a3a0 con 0x7f4b30151060 2024-04-25T13:50:15.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b25ffb640 1 -- 172.21.15.80:0/2477910274 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) ==== 104+0+11 (secure 0 0 0) 0x7f4b2c05cbe0 con 0x7f4b30151060 2024-04-25T13:50:15.358 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:50:15.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b1406f400 msgr2=0x7f4b14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b1406f400 0x7f4b14071910 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4b20001300 tx=0x7f4b20009040 comp rx=0 tx=0).stop 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b30151060 msgr2=0x7f4b301fcc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b30151060 0x7f4b301fcc30 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009c80 tx=0x7f4b2c007700 comp rx=0 tx=0).stop 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 shutdown_connections 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b1406f400 0x7f4b14071910 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b300c6840 0x7f4b301497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b300c5ef0 0x7f4b300c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 --2- 172.21.15.80:0/2477910274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b30151060 0x7f4b301fcc30 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 >> 172.21.15.80:0/2477910274 conn(0x7f4b300737b0 msgr2=0x7f4b3015e770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 shutdown_connections 2024-04-25T13:50:15.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.361+0000 7f4b37510640 1 -- 172.21.15.80:0/2477910274 wait complete. 2024-04-25T13:50:15.379 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:50:15.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f85214640 1 Processor -- start 2024-04-25T13:50:15.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f85214640 1 -- start start 2024-04-25T13:50:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f85214640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3f800b5330 0x7f3f800d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f85214640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3f800d0ff0 0x7f3f8006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f85214640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f80161b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f85214640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3f8006f700 con 0x7f3f800d0ff0 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f85214640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3f801629b0 con 0x7f3f800b5330 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f85214640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f800b5740 con 0x7f3f8006f2f0 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7e59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3f800d0ff0 0x7f3f8006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7e59c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3f800d0ff0 0x7f3f8006edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58182/0 (socket says 172.21.15.80:58182) 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7e59c640 1 -- 172.21.15.80:0/3374123989 learned_addr learned my addr 172.21.15.80:0/3374123989 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7f59e640 1 --2- 172.21.15.80:0/3374123989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f80161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7ed9d640 1 --2- 172.21.15.80:0/3374123989 >> v2:172.21.15.87:3300/0 conn(0x7f3f800b5330 0x7f3f800d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7e59c640 1 -- 172.21.15.80:0/3374123989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 msgr2=0x7f3f80161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7e59c640 1 --2- 172.21.15.80:0/3374123989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f80161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7e59c640 1 -- 172.21.15.80:0/3374123989 >> v2:172.21.15.87:3300/0 conn(0x7f3f800b5330 msgr2=0x7f3f800d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7e59c640 1 --2- 172.21.15.80:0/3374123989 >> v2:172.21.15.87:3300/0 conn(0x7f3f800b5330 0x7f3f800d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7e59c640 1 -- 172.21.15.80:0/3374123989 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8006dd10 con 0x7f3f800d0ff0 2024-04-25T13:50:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7f59e640 1 --2- 172.21.15.80:0/3374123989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f80161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.517+0000 7f3f7ed9d640 1 --2- 172.21.15.80:0/3374123989 >> v2:172.21.15.87:3300/0 conn(0x7f3f800b5330 0x7f3f800d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f7e59c640 1 --2- 172.21.15.80:0/3374123989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f8006edb0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f3f800701f0 tx=0x7f3f6801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809e18861441f835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f7dd9b640 1 -- 172.21.15.80:0/3374123989 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680b0070 con 0x7f3f800d0ff0 2024-04-25T13:50:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f7dd9b640 1 -- 172.21.15.80:0/3374123989 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f680ab600 con 0x7f3f800d0ff0 2024-04-25T13:50:15.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f7dd9b640 1 -- 172.21.15.80:0/3374123989 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f680ab9d0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 -- 172.21.15.80:0/3374123989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 msgr2=0x7f3f8006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 --2- 172.21.15.80:0/3374123989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f8006edb0 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f3f800701f0 tx=0x7f3f6801c490 comp rx=0 tx=0).stop 2024-04-25T13:50:15.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 -- 172.21.15.80:0/3374123989 shutdown_connections 2024-04-25T13:50:15.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 --2- 172.21.15.80:0/3374123989 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f80161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 --2- 172.21.15.80:0/3374123989 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f8006edb0 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 --2- 172.21.15.80:0/3374123989 >> v2:172.21.15.87:3300/0 conn(0x7f3f800b5330 0x7f3f800d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 -- 172.21.15.80:0/3374123989 >> 172.21.15.80:0/3374123989 conn(0x7f3f800b4160 msgr2=0x7f3f800d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:15.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 -- 172.21.15.80:0/3374123989 shutdown_connections 2024-04-25T13:50:15.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.521+0000 7f3f85214640 1 -- 172.21.15.80:0/3374123989 wait complete. 2024-04-25T13:50:15.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 Processor -- start 2024-04-25T13:50:15.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 -- start start 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f8015ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f800b5330 0x7f3f801fac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f801fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3f800b0cd0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8006e890 con 0x7f3f800b5330 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3f800b0e50 con 0x7f3f8006f2f0 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f801fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f801fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58190/0 (socket says 172.21.15.80:58190) 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 -- 172.21.15.80:0/1695423689 learned_addr learned my addr 172.21.15.80:0/1695423689 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:15.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7ed9d640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f8015ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7e59c640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f800b5330 0x7f3f801fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 -- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 msgr2=0x7f3f8015ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f8015ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 -- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f800b5330 msgr2=0x7f3f801fac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f800b5330 0x7f3f801fac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 -- 172.21.15.80:0/1695423689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8015b9f0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7e59c640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f800b5330 0x7f3f801fac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f7f59e640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f801fb190 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c590 tx=0x7f3f7800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f777fe640 1 -- 172.21.15.80:0/1695423689 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78015070 con 0x7f3f800d0ff0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f777fe640 1 -- 172.21.15.80:0/1695423689 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f78011800 con 0x7f3f800d0ff0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f80159920 con 0x7f3f800d0ff0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f777fe640 1 -- 172.21.15.80:0/1695423689 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78011bf0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8015d1d0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.525+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8006a3a0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.529+0000 7f3f777fe640 1 -- 172.21.15.80:0/1695423689 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3f78018070 con 0x7f3f800d0ff0 2024-04-25T13:50:15.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.529+0000 7f3f777fe640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f6006f430 0x7f3f60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.529+0000 7f3f777fe640 1 -- 172.21.15.80:0/1695423689 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f3f7808f090 con 0x7f3f800d0ff0 2024-04-25T13:50:15.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.529+0000 7f3f777fe640 1 -- 172.21.15.80:0/1695423689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f3f600770d0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.529+0000 7f3f7ed9d640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f6006f430 0x7f3f60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.529+0000 7f3f7ed9d640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f6006f430 0x7f3f60071940 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3f70007510 tx=0x7f3f700074a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.533+0000 7f3f777fe640 1 -- 172.21.15.80:0/1695423689 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3f7805bfc0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f8015b780 con 0x7f3f800d0ff0 2024-04-25T13:50:15.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f777fe640 1 -- 172.21.15.80:0/1695423689 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) ==== 100+0+11 (secure 0 0 0) 0x7f3f7805b8c0 con 0x7f3f800d0ff0 2024-04-25T13:50:15.682 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:50:15.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f6006f430 msgr2=0x7f3f60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f6006f430 0x7f3f60071940 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3f70007510 tx=0x7f3f700074a0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 msgr2=0x7f3f801fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f801fb190 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c590 tx=0x7f3f7800ca50 comp rx=0 tx=0).stop 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 shutdown_connections 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3f6006f430 0x7f3f60071940 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3f800d0ff0 0x7f3f801fb190 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f800b5330 0x7f3f801fac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 --2- 172.21.15.80:0/1695423689 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3f8006f2f0 0x7f3f8015ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 >> 172.21.15.80:0/1695423689 conn(0x7f3f800b4160 msgr2=0x7f3f800d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 shutdown_connections 2024-04-25T13:50:15.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.685+0000 7f3f85214640 1 -- 172.21.15.80:0/1695423689 wait complete. 2024-04-25T13:50:15.704 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:15.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b9a8e640 1 Processor -- start 2024-04-25T13:50:15.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b9a8e640 1 -- start start 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b9a8e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f01b4161f90 0x7f01b40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b9a8e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b415ec40 0x7f01b415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b9a8e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01b40d0d70 con 0x7f01b40d0510 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b9a8e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f01b40d1050 con 0x7f01b4161f90 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b9a8e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f01b406dca0 con 0x7f01b415ec40 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f01b4161f90 0x7f01b40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b2ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f01b40d0510 0x7f01b415e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58438/0 (socket says 172.21.15.80:58438) 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b2ffd640 1 -- 172.21.15.80:0/1160353648 learned_addr learned my addr 172.21.15.80:0/1160353648 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:15.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b3fff640 1 --2- 172.21.15.80:0/1160353648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b415ec40 0x7f01b415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b37fe640 1 -- 172.21.15.80:0/1160353648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b415ec40 msgr2=0x7f01b415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:15.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b37fe640 1 --2- 172.21.15.80:0/1160353648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b415ec40 0x7f01b415f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b37fe640 1 -- 172.21.15.80:0/1160353648 >> v1:172.21.15.80:6789/0 conn(0x7f01b40d0510 legacy=0x7f01b415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.857+0000 7f01b37fe640 1 -- 172.21.15.80:0/1160353648 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b406df80 con 0x7f01b4161f90 2024-04-25T13:50:15.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b37fe640 1 --2- 172.21.15.80:0/1160353648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b4161f90 0x7f01b40cffd0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097c0 tx=0x7f01a401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f713efca9f9250b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b27fc640 1 -- 172.21.15.80:0/1160353648 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40b0070 con 0x7f01b4161f90 2024-04-25T13:50:15.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b27fc640 1 -- 172.21.15.80:0/1160353648 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a40ab600 con 0x7f01b4161f90 2024-04-25T13:50:15.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b27fc640 1 -- 172.21.15.80:0/1160353648 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40ab9d0 con 0x7f01b4161f90 2024-04-25T13:50:15.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b9a8e640 1 -- 172.21.15.80:0/1160353648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b4161f90 msgr2=0x7f01b40cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/1160353648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b4161f90 0x7f01b40cffd0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097c0 tx=0x7f01a401c3b0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b9a8e640 1 -- 172.21.15.80:0/1160353648 shutdown_connections 2024-04-25T13:50:15.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/1160353648 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b415ec40 0x7f01b415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/1160353648 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b4161f90 0x7f01b40cffd0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b9a8e640 1 -- 172.21.15.80:0/1160353648 >> 172.21.15.80:0/1160353648 conn(0x7f01b406d080 msgr2=0x7f01b40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:15.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b9a8e640 1 -- 172.21.15.80:0/1160353648 shutdown_connections 2024-04-25T13:50:15.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.861+0000 7f01b9a8e640 1 -- 172.21.15.80:0/1160353648 wait complete. 2024-04-25T13:50:15.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 Processor -- start 2024-04-25T13:50:15.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 -- start start 2024-04-25T13:50:15.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b40d0510 0x7f01b4141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01b415ec40 0x7f01b4162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.863 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b4161f90 0x7f01b40af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f01b406f4b0 con 0x7f01b40d0510 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f01b406f630 con 0x7f01b415ec40 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f01b40d0920 con 0x7f01b4161f90 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b40d0510 0x7f01b4141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b40d0510 0x7f01b4141420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58208/0 (socket says 172.21.15.80:58208) 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 -- 172.21.15.80:0/3307379399 learned_addr learned my addr 172.21.15.80:0/3307379399 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b2ffd640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01b415ec40 0x7f01b4162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 -- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b4161f90 msgr2=0x7f01b40af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b4161f90 0x7f01b40af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 -- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01b415ec40 msgr2=0x7f01b4162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:15.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01b415ec40 0x7f01b4162ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:15.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 -- 172.21.15.80:0/3307379399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b4160f40 con 0x7f01b40d0510 2024-04-25T13:50:15.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b37fe640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b40d0510 0x7f01b4141420 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097c0 tx=0x7f01a401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f019bfff640 1 -- 172.21.15.80:0/3307379399 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a40b0050 con 0x7f01b40d0510 2024-04-25T13:50:15.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f019bfff640 1 -- 172.21.15.80:0/3307379399 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01a40bc070 con 0x7f01b40d0510 2024-04-25T13:50:15.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b4072de0 con 0x7f01b40d0510 2024-04-25T13:50:15.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f019bfff640 1 -- 172.21.15.80:0/3307379399 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a4006e20 con 0x7f01b40d0510 2024-04-25T13:50:15.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b4073a90 con 0x7f01b40d0510 2024-04-25T13:50:15.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.865+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b406a3a0 con 0x7f01b40d0510 2024-04-25T13:50:15.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.869+0000 7f019bfff640 1 -- 172.21.15.80:0/3307379399 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01a40b5810 con 0x7f01b40d0510 2024-04-25T13:50:15.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.869+0000 7f019bfff640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01a006f430 0x7f01a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:15.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.869+0000 7f019bfff640 1 -- 172.21.15.80:0/3307379399 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f01a4129940 con 0x7f01b40d0510 2024-04-25T13:50:15.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.869+0000 7f019bfff640 1 -- 172.21.15.80:0/3307379399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f01a0077070 con 0x7f01b40d0510 2024-04-25T13:50:15.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.869+0000 7f01b2ffd640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01a006f430 0x7f01a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:15.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.869+0000 7f01b2ffd640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01a006f430 0x7f01a0071940 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f01b415e580 tx=0x7f01a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:15.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:15.873+0000 7f019bfff640 1 -- 172.21.15.80:0/3307379399 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f01a40c6020 con 0x7f01b40d0510 2024-04-25T13:50:16.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01b41631e0 con 0x7f01b40d0510 2024-04-25T13:50:16.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f019bfff640 1 -- 172.21.15.80:0/3307379399 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7f01b4160f40 con 0x7f01b40d0510 2024-04-25T13:50:16.051 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:16.051 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/521602421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":521602421},{"type":"v1","addr":"172.21.15.87:6835","nonce":521602421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4170043750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4170043750},{"type":"v1","addr":"172.21.15.87:6837","nonce":4170043750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/1423926990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1423926990},{"type":"v1","addr":"172.21.15.80:6834","nonce":1423926990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:48:09.907104+0000","modified":"2024-04-25T13:48:12.977708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1134081162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1134081162},{"type":"v1","addr":"172.21.15.80:6835","nonce":1134081162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-25T13:50:16.051 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 74 2024-04-25T13:50:16.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01a006f430 msgr2=0x7f01a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01a006f430 0x7f01a0071940 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f01b415e580 tx=0x7f01a8009040 comp rx=0 tx=0).stop 2024-04-25T13:50:16.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b40d0510 msgr2=0x7f01b4141420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b40d0510 0x7f01b4141420 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097c0 tx=0x7f01a401ac50 comp rx=0 tx=0).stop 2024-04-25T13:50:16.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 shutdown_connections 2024-04-25T13:50:16.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01a006f430 0x7f01a0071940 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.053+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01b4161f90 0x7f01b40af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.057+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01b415ec40 0x7f01b4162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.057+0000 7f01b9a8e640 1 --2- 172.21.15.80:0/3307379399 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01b40d0510 0x7f01b4141420 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.057+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 >> 172.21.15.80:0/3307379399 conn(0x7f01b406d080 msgr2=0x7f01b40b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:16.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.057+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 shutdown_connections 2024-04-25T13:50:16.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.057+0000 7f01b9a8e640 1 -- 172.21.15.80:0/3307379399 wait complete. 2024-04-25T13:50:16.071 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:16.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1608f4f640 1 Processor -- start 2024-04-25T13:50:16.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1608f4f640 1 -- start start 2024-04-25T13:50:16.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1608f4f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d14a0 0x7f16040b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1608f4f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16040d46a0 0x7f16040d4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1608f4f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16040b9210 con 0x7f16040d26d0 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1608f4f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f16040d18b0 con 0x7f16040d46a0 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1608f4f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f16040b5120 con 0x7f16040d14a0 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f160259c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d14a0 0x7f16040b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1602d9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16040d46a0 0x7f16040d4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1601d9b640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f16040d26d0 0x7f16040d2ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:39336/0 (socket says 172.21.15.80:39336) 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f160259c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d14a0 0x7f16040b4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:50654/0 (socket says 172.21.15.80:50654) 2024-04-25T13:50:16.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.221+0000 7f1601d9b640 1 -- 172.21.15.80:0/3258080506 learned_addr learned my addr 172.21.15.80:0/3258080506 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:16.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160259c640 1 -- 172.21.15.80:0/3258080506 >> v2:172.21.15.87:3300/0 conn(0x7f16040d46a0 msgr2=0x7f16040d4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:16.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160259c640 1 --2- 172.21.15.80:0/3258080506 >> v2:172.21.15.87:3300/0 conn(0x7f16040d46a0 0x7f16040d4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160259c640 1 -- 172.21.15.80:0/3258080506 >> v1:172.21.15.87:6789/0 conn(0x7f16040d26d0 legacy=0x7f16040d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160159a640 1 -- 172.21.15.80:0/3258080506 <== mon.1 v1:172.21.15.87:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3038146424 0 0) 0x7f15f8001170 con 0x7f16040d26d0 2024-04-25T13:50:16.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160259c640 1 -- 172.21.15.80:0/3258080506 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1604163f60 con 0x7f16040d14a0 2024-04-25T13:50:16.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f1602d9d640 1 --2- 172.21.15.80:0/3258080506 >> v2:172.21.15.87:3300/0 conn(0x7f16040d46a0 0x7f16040d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:50:16.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160259c640 1 --2- 172.21.15.80:0/3258080506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d14a0 0x7f16040b4860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15f40097c0 tx=0x7f15f401c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5482936bbf303e81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160159a640 1 -- 172.21.15.80:0/3258080506 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f40b0070 con 0x7f16040d14a0 2024-04-25T13:50:16.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160159a640 1 -- 172.21.15.80:0/3258080506 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15f40ab600 con 0x7f16040d14a0 2024-04-25T13:50:16.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.225+0000 7f160159a640 1 -- 172.21.15.80:0/3258080506 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f40ab9d0 con 0x7f16040d14a0 2024-04-25T13:50:16.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- 172.21.15.80:0/3258080506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d14a0 msgr2=0x7f16040b4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 --2- 172.21.15.80:0/3258080506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d14a0 0x7f16040b4860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f15f40097c0 tx=0x7f15f401c410 comp rx=0 tx=0).stop 2024-04-25T13:50:16.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- 172.21.15.80:0/3258080506 shutdown_connections 2024-04-25T13:50:16.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 --2- 172.21.15.80:0/3258080506 >> v2:172.21.15.87:3300/0 conn(0x7f16040d46a0 0x7f16040d4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 --2- 172.21.15.80:0/3258080506 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d14a0 0x7f16040b4860 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- 172.21.15.80:0/3258080506 >> 172.21.15.80:0/3258080506 conn(0x7f1604073770 msgr2=0x7f160406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:16.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- 172.21.15.80:0/3258080506 shutdown_connections 2024-04-25T13:50:16.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- 172.21.15.80:0/3258080506 wait complete. 2024-04-25T13:50:16.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 Processor -- start 2024-04-25T13:50:16.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- start start 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16040d14a0 0x7f1604150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d26d0 0x7f160414b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16040d46a0 0x7f1604152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f16040b4da0 con 0x7f16040d46a0 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1604164d20 con 0x7f16040d14a0 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1608f4f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f160416d1d0 con 0x7f16040d26d0 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.229+0000 7f1602d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16040d46a0 0x7f1604152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1602d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16040d46a0 0x7f1604152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58218/0 (socket says 172.21.15.80:58218) 2024-04-25T13:50:16.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1602d9d640 1 -- 172.21.15.80:0/443822436 learned_addr learned my addr 172.21.15.80:0/443822436 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f160259c640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16040d14a0 0x7f1604150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1601d9b640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d26d0 0x7f160414b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1602d9d640 1 -- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d26d0 msgr2=0x7f160414b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1602d9d640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d26d0 0x7f160414b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1602d9d640 1 -- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16040d14a0 msgr2=0x7f1604150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1602d9d640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16040d14a0 0x7f1604150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1602d9d640 1 -- 172.21.15.80:0/443822436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16041529c0 con 0x7f16040d46a0 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f160259c640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16040d14a0 0x7f1604150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1602d9d640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16040d46a0 0x7f1604152480 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00efc0 tx=0x7f15fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc018030 con 0x7f16040d46a0 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f15fc00cdf0 con 0x7f16040d46a0 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16041513e0 con 0x7f16040d46a0 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15fc003bc0 con 0x7f16040d46a0 2024-04-25T13:50:16.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16041519a0 con 0x7f16040d46a0 2024-04-25T13:50:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f160406a3a0 con 0x7f16040d46a0 2024-04-25T13:50:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f15fc01b070 con 0x7f16040d46a0 2024-04-25T13:50:16.232 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f15f2ffd640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f15e406f400 0x7f15e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5767+0+0 (secure 0 0 0) 0x7f15fc08fc30 con 0x7f16040d46a0 2024-04-25T13:50:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=159}) -- 0x7f15e40770a0 con 0x7f16040d46a0 2024-04-25T13:50:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.233+0000 7f160259c640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f15e406f400 0x7f15e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.237+0000 7f160259c640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f15e406f400 0x7f15e4071910 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f15f40ad000 tx=0x7f15f40a4f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.237+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f15fc022070 con 0x7f16040d46a0 2024-04-25T13:50:16.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.265+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 <== mon.0 v2:172.21.15.80:3300/0 7 ==== osd_map(159..159 src has 1..159) ==== 679+0+0 (secure 0 0 0) 0x7f15fc08e230 con 0x7f16040d46a0 2024-04-25T13:50:16.262 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.265+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=160}) -- 0x7f15e4077b90 con 0x7f16040d46a0 2024-04-25T13:50:16.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.393+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1604151170 con 0x7f16040d46a0 2024-04-25T13:50:16.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.393+0000 7f15f2ffd640 1 -- 172.21.15.80:0/443822436 <== mon.0 v2:172.21.15.80:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+16655 (secure 0 0 0) 0x7f15fc010040 con 0x7f16040d46a0 2024-04-25T13:50:16.393 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:16.393 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":159,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:50:16.264643+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:48:07.666863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T13:48:08.875350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.3","mappings":[{"from":2,"to":0}]},{"pgid":"19.a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:16.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f15e406f400 msgr2=0x7f15e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f15e406f400 0x7f15e4071910 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f15f40ad000 tx=0x7f15f40a4f90 comp rx=0 tx=0).stop 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16040d46a0 msgr2=0x7f1604152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16040d46a0 0x7f1604152480 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f15fc00efc0 tx=0x7f15fc00c490 comp rx=0 tx=0).stop 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 shutdown_connections 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f15e406f400 0x7f15e4071910 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16040d46a0 0x7f1604152480 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16040d26d0 0x7f160414b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 --2- 172.21.15.80:0/443822436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16040d14a0 0x7f1604150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 >> 172.21.15.80:0/443822436 conn(0x7f1604073770 msgr2=0x7f16040ba0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 shutdown_connections 2024-04-25T13:50:16.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.397+0000 7f1608f4f640 1 -- 172.21.15.80:0/443822436 wait complete. 2024-04-25T13:50:16.413 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:50:16.414 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:16.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.565+0000 7f59a69e4640 1 Processor -- start 2024-04-25T13:50:16.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.565+0000 7f59a69e4640 1 -- start start 2024-04-25T13:50:16.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a69e4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a0161240 0x7f59a00723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a69e4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f59a0161650 0x7f59a00728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a69e4640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59a006fcf0 con 0x7f59a006f5a0 2024-04-25T13:50:16.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a69e4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f59a0073f10 con 0x7f59a0161650 2024-04-25T13:50:16.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a69e4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f59a00b0c60 con 0x7f59a0161240 2024-04-25T13:50:16.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a4f81640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f59a0161650 0x7f59a00728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a4f81640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f59a0161650 0x7f59a00728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58224/0 (socket says 172.21.15.80:58224) 2024-04-25T13:50:16.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a4f81640 1 -- 172.21.15.80:0/1316046502 learned_addr learned my addr 172.21.15.80:0/1316046502 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59977fe640 1 --2- 172.21.15.80:0/1316046502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a0161240 0x7f59a00723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a4f81640 1 -- 172.21.15.80:0/1316046502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a0161240 msgr2=0x7f59a00723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a4f81640 1 --2- 172.21.15.80:0/1316046502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a0161240 0x7f59a00723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a4f81640 1 -- 172.21.15.80:0/1316046502 >> v1:172.21.15.87:6789/0 conn(0x7f59a006f5a0 legacy=0x7f59a0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.569+0000 7f59a4f81640 1 -- 172.21.15.80:0/1316046502 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a00b0f40 con 0x7f59a0161650 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a4f81640 1 --2- 172.21.15.80:0/1316046502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a00728e0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f5998009e80 tx=0x7f599801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2e976f66f99a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f599f7fe640 1 -- 172.21.15.80:0/1316046502 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980a56c0 con 0x7f59a0161650 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f599f7fe640 1 -- 172.21.15.80:0/1316046502 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59980a5860 con 0x7f59a0161650 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f599f7fe640 1 -- 172.21.15.80:0/1316046502 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980a5c30 con 0x7f59a0161650 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 -- 172.21.15.80:0/1316046502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 msgr2=0x7f59a00728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 --2- 172.21.15.80:0/1316046502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a00728e0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f5998009e80 tx=0x7f599801c980 comp rx=0 tx=0).stop 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 -- 172.21.15.80:0/1316046502 shutdown_connections 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 --2- 172.21.15.80:0/1316046502 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a00728e0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 --2- 172.21.15.80:0/1316046502 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a0161240 0x7f59a00723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 -- 172.21.15.80:0/1316046502 >> 172.21.15.80:0/1316046502 conn(0x7f59a006d080 msgr2=0x7f59a00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:16.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 -- 172.21.15.80:0/1316046502 shutdown_connections 2024-04-25T13:50:16.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 -- 172.21.15.80:0/1316046502 wait complete. 2024-04-25T13:50:16.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 Processor -- start 2024-04-25T13:50:16.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.573+0000 7f59a69e4640 1 -- start start 2024-04-25T13:50:16.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a006f5a0 0x7f59a0152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0161240 0x7f59a0155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a01494a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f59a0168290 con 0x7f59a0161650 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f59a0168110 con 0x7f59a0161240 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f59a0168410 con 0x7f59a006f5a0 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a01494a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a01494a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58236/0 (socket says 172.21.15.80:58236) 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 -- 172.21.15.80:0/2544725774 learned_addr learned my addr 172.21.15.80:0/2544725774 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59977fe640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0161240 0x7f59a0155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f599ffff640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a006f5a0 0x7f59a0152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 -- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a006f5a0 msgr2=0x7f59a0152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a006f5a0 0x7f59a0152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 -- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0161240 msgr2=0x7f59a0155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0161240 0x7f59a0155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 -- 172.21.15.80:0/2544725774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a00d25f0 con 0x7f59a0161650 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f599ffff640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a006f5a0 0x7f59a0152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:16.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a4f81640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a01494a0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f599801a5b0 tx=0x7f5998019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f599d7fa640 1 -- 172.21.15.80:0/2544725774 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59980bc020 con 0x7f59a0161650 2024-04-25T13:50:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f599d7fa640 1 -- 172.21.15.80:0/2544725774 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59980b8670 con 0x7f59a0161650 2024-04-25T13:50:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a015f3d0 con 0x7f59a0161650 2024-04-25T13:50:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f599d7fa640 1 -- 172.21.15.80:0/2544725774 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5998009510 con 0x7f59a0161650 2024-04-25T13:50:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a015e610 con 0x7f59a0161650 2024-04-25T13:50:16.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a015e080 con 0x7f59a0161650 2024-04-25T13:50:16.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f599d7fa640 1 -- 172.21.15.80:0/2544725774 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f59980bf020 con 0x7f59a0161650 2024-04-25T13:50:16.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f599d7fa640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f598006f430 0x7f5980071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.577+0000 7f599d7fa640 1 -- 172.21.15.80:0/2544725774 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5767+0+0 (secure 0 0 0) 0x7f5998129ee0 con 0x7f59a0161650 2024-04-25T13:50:16.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.581+0000 7f599ffff640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f598006f430 0x7f5980071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.581+0000 7f599ffff640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f598006f430 0x7f5980071940 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f59a00b6e70 tx=0x7f5990009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.585+0000 7f599d7fa640 1 -- 172.21.15.80:0/2544725774 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f59980c6070 con 0x7f59a0161650 2024-04-25T13:50:16.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.781+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59a006a3a0 con 0x7f59a0161650 2024-04-25T13:50:16.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f599d7fa640 1 -- 172.21.15.80:0/2544725774 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7f5998016600 con 0x7f59a0161650 2024-04-25T13:50:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:16.781 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/521602421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":521602421},{"type":"v1","addr":"172.21.15.87:6835","nonce":521602421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4170043750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4170043750},{"type":"v1","addr":"172.21.15.87:6837","nonce":4170043750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/1423926990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1423926990},{"type":"v1","addr":"172.21.15.80:6834","nonce":1423926990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:48:09.907104+0000","modified":"2024-04-25T13:48:12.977708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1134081162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1134081162},{"type":"v1","addr":"172.21.15.80:6835","nonce":1134081162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-25T13:50:16.781 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 74 2024-04-25T13:50:16.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f598006f430 msgr2=0x7f5980071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f598006f430 0x7f5980071940 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f59a00b6e70 tx=0x7f5990009040 comp rx=0 tx=0).stop 2024-04-25T13:50:16.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 msgr2=0x7f59a01494a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a01494a0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f599801a5b0 tx=0x7f5998019720 comp rx=0 tx=0).stop 2024-04-25T13:50:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 shutdown_connections 2024-04-25T13:50:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f598006f430 0x7f5980071940 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f59a0161650 0x7f59a01494a0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f59a0161240 0x7f59a0155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 --2- 172.21.15.80:0/2544725774 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f59a006f5a0 0x7f59a0152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 >> 172.21.15.80:0/2544725774 conn(0x7f59a006d080 msgr2=0x7f59a00d0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 shutdown_connections 2024-04-25T13:50:16.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.785+0000 7f59a69e4640 1 -- 172.21.15.80:0/2544725774 wait complete. 2024-04-25T13:50:16.804 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:16.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.953+0000 7f451f59e640 1 Processor -- start 2024-04-25T13:50:16.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.953+0000 7f451f59e640 1 -- start start 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451f59e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200c7db0 0x7f452006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451f59e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45200d1220 0x7f452006eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451f59e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4520160bd0 con 0x7f45200d0e10 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451f59e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f45201617e0 con 0x7f45200d1220 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451f59e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4520160eb0 con 0x7f45200c7db0 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45200d1220 0x7f452006eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f45200d1220 0x7f452006eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58238/0 (socket says 172.21.15.80:58238) 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451ed9d640 1 -- 172.21.15.80:0/1161978140 learned_addr learned my addr 172.21.15.80:0/1161978140 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:16.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451dd9b640 1 --2- 172.21.15.80:0/1161978140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200c7db0 0x7f452006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451ed9d640 1 -- 172.21.15.80:0/1161978140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200c7db0 msgr2=0x7f452006eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:16.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451ed9d640 1 --2- 172.21.15.80:0/1161978140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200c7db0 0x7f452006eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451ed9d640 1 -- 172.21.15.80:0/1161978140 >> v1:172.21.15.87:6789/0 conn(0x7f45200d0e10 legacy=0x7f45200c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451ed9d640 1 -- 172.21.15.80:0/1161978140 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4520161190 con 0x7f45200d1220 2024-04-25T13:50:16.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.957+0000 7f451dd9b640 1 --2- 172.21.15.80:0/1161978140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200c7db0 0x7f452006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451ed9d640 1 --2- 172.21.15.80:0/1161978140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d1220 0x7f452006eff0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4518009e50 tx=0x7f451801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5653b0d347aeb6c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451d59a640 1 -- 172.21.15.80:0/1161978140 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45180a5850 con 0x7f45200d1220 2024-04-25T13:50:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451d59a640 1 -- 172.21.15.80:0/1161978140 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45180a59f0 con 0x7f45200d1220 2024-04-25T13:50:16.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451d59a640 1 -- 172.21.15.80:0/1161978140 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45180a5dc0 con 0x7f45200d1220 2024-04-25T13:50:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451f59e640 1 -- 172.21.15.80:0/1161978140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d1220 msgr2=0x7f452006eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451f59e640 1 --2- 172.21.15.80:0/1161978140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d1220 0x7f452006eff0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4518009e50 tx=0x7f451801c970 comp rx=0 tx=0).stop 2024-04-25T13:50:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451f59e640 1 -- 172.21.15.80:0/1161978140 shutdown_connections 2024-04-25T13:50:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451f59e640 1 --2- 172.21.15.80:0/1161978140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d1220 0x7f452006eff0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451f59e640 1 --2- 172.21.15.80:0/1161978140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200c7db0 0x7f452006eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451f59e640 1 -- 172.21.15.80:0/1161978140 >> 172.21.15.80:0/1161978140 conn(0x7f45200b34a0 msgr2=0x7f45200b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451f59e640 1 -- 172.21.15.80:0/1161978140 shutdown_connections 2024-04-25T13:50:16.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.961+0000 7f451f59e640 1 -- 172.21.15.80:0/1161978140 wait complete. 2024-04-25T13:50:16.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 Processor -- start 2024-04-25T13:50:16.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 -- start start 2024-04-25T13:50:16.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d0e10 0x7f4520146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200d1220 0x7f452014b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4520150340 0x7f4520152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f452016a160 con 0x7f45200d0e10 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4520169fe0 con 0x7f4520150340 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f452016a2e0 con 0x7f45200d1220 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d0e10 0x7f4520146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d0e10 0x7f4520146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58248/0 (socket says 172.21.15.80:58248) 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 -- 172.21.15.80:0/2796522374 learned_addr learned my addr 172.21.15.80:0/2796522374 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451ed9d640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4520150340 0x7f4520152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451dd9b640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200d1220 0x7f452014b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 -- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200d1220 msgr2=0x7f452014b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200d1220 0x7f452014b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 -- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4520150340 msgr2=0x7f4520152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4520150340 0x7f4520152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 -- 172.21.15.80:0/2796522374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452014f2a0 con 0x7f45200d0e10 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451dd9b640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200d1220 0x7f452014b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451e59c640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d0e10 0x7f4520146c10 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f451000c3a0 tx=0x7f451000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f450effd640 1 -- 172.21.15.80:0/2796522374 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4510017070 con 0x7f45200d0e10 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f450effd640 1 -- 172.21.15.80:0/2796522374 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f45100052d0 con 0x7f45200d0e10 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452014f580 con 0x7f45200d0e10 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f450effd640 1 -- 172.21.15.80:0/2796522374 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45100054c0 con 0x7f45200d0e10 2024-04-25T13:50:16.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452014fb40 con 0x7f45200d0e10 2024-04-25T13:50:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.965+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452006a2c0 con 0x7f45200d0e10 2024-04-25T13:50:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.969+0000 7f450effd640 1 -- 172.21.15.80:0/2796522374 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f451001a070 con 0x7f45200d0e10 2024-04-25T13:50:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.969+0000 7f450effd640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f450006f430 0x7f4500071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.969+0000 7f450effd640 1 -- 172.21.15.80:0/2796522374 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5767+0+0 (secure 0 0 0) 0x7f451008e5b0 con 0x7f45200d0e10 2024-04-25T13:50:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.969+0000 7f451dd9b640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f450006f430 0x7f4500071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:16.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.969+0000 7f451dd9b640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f450006f430 0x7f4500071940 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f45200b0710 tx=0x7f4514008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:16.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:16.973+0000 7f450effd640 1 -- 172.21.15.80:0/2796522374 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f451005b4f0 con 0x7f45200d0e10 2024-04-25T13:50:17.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.169+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45200d00c0 con 0x7f45200d0e10 2024-04-25T13:50:17.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.169+0000 7f450effd640 1 -- 172.21.15.80:0/2796522374 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+5180 (secure 0 0 0) 0x7f451005adf0 con 0x7f45200d0e10 2024-04-25T13:50:17.167 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:17.167 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":74,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/521602421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":521602421},{"type":"v1","addr":"172.21.15.87:6835","nonce":521602421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4170043750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4170043750},{"type":"v1","addr":"172.21.15.87:6837","nonce":4170043750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/1423926990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1423926990},{"type":"v1","addr":"172.21.15.80:6834","nonce":1423926990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:48:09.907104+0000","modified":"2024-04-25T13:48:12.977708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8412":{"gid":8412,"name":"a","rank":0,"incarnation":71,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/1134081162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":1134081162},{"type":"v1","addr":"172.21.15.80:6835","nonce":1134081162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8412,"qdb_cluster":[8412]},"id":9}]} 2024-04-25T13:50:17.167 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 74 2024-04-25T13:50:17.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f450006f430 msgr2=0x7f4500071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f450006f430 0x7f4500071940 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f45200b0710 tx=0x7f4514008040 comp rx=0 tx=0).stop 2024-04-25T13:50:17.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d0e10 msgr2=0x7f4520146c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d0e10 0x7f4520146c10 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f451000c3a0 tx=0x7f451000c860 comp rx=0 tx=0).stop 2024-04-25T13:50:17.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 shutdown_connections 2024-04-25T13:50:17.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f450006f430 0x7f4500071940 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4520150340 0x7f4520152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f45200d1220 0x7f452014b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 --2- 172.21.15.80:0/2796522374 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f45200d0e10 0x7f4520146c10 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 >> 172.21.15.80:0/2796522374 conn(0x7f45200b34a0 msgr2=0x7f452014de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:17.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 shutdown_connections 2024-04-25T13:50:17.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.173+0000 7f451f59e640 1 -- 172.21.15.80:0/2796522374 wait complete. 2024-04-25T13:50:17.189 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:17.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8f5c1640 1 Processor -- start 2024-04-25T13:50:17.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8f5c1640 1 -- start start 2024-04-25T13:50:17.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8f5c1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6d88161650 0x7f6d88072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8f5c1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d8806fcf0 con 0x7f6d8806f5a0 2024-04-25T13:50:17.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8f5c1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d880b0c60 con 0x7f6d88161240 2024-04-25T13:50:17.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8f5c1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f6d88073f10 con 0x7f6d88161650 2024-04-25T13:50:17.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8db5e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f6d88161650 0x7f6d88072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8d35d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f6d8806f5a0 0x7f6d88160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58448/0 (socket says 172.21.15.80:58448) 2024-04-25T13:50:17.335 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8d35d640 1 -- 172.21.15.80:0/757556512 learned_addr learned my addr 172.21.15.80:0/757556512 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:17.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8db5e640 1 -- 172.21.15.80:0/757556512 >> v1:172.21.15.87:6789/0 conn(0x7f6d88161240 legacy=0x7f6d880723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8db5e640 1 -- 172.21.15.80:0/757556512 >> v1:172.21.15.80:6789/0 conn(0x7f6d8806f5a0 legacy=0x7f6d88160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d7bfff640 1 -- 172.21.15.80:0/757556512 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 121235082 0 0) 0x7f6d8806fcf0 con 0x7f6d8806f5a0 2024-04-25T13:50:17.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.337+0000 7f6d8db5e640 1 -- 172.21.15.80:0/757556512 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d880b0f40 con 0x7f6d88161650 2024-04-25T13:50:17.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8db5e640 1 --2- 172.21.15.80:0/757556512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d88161650 0x7f6d88072ab0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009d90 tx=0x7f6d8001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c91a886ab7e91e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:17.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d7bfff640 1 -- 172.21.15.80:0/757556512 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d800b2070 con 0x7f6d88161650 2024-04-25T13:50:17.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d7bfff640 1 -- 172.21.15.80:0/757556512 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d800a5820 con 0x7f6d88161650 2024-04-25T13:50:17.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d7bfff640 1 -- 172.21.15.80:0/757556512 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d800a5bf0 con 0x7f6d88161650 2024-04-25T13:50:17.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/757556512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d88161650 msgr2=0x7f6d88072ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 --2- 172.21.15.80:0/757556512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d88161650 0x7f6d88072ab0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009d90 tx=0x7f6d8001c8e0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/757556512 shutdown_connections 2024-04-25T13:50:17.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 --2- 172.21.15.80:0/757556512 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d88161650 0x7f6d88072ab0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/757556512 >> 172.21.15.80:0/757556512 conn(0x7f6d8806d080 msgr2=0x7f6d880b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:17.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/757556512 shutdown_connections 2024-04-25T13:50:17.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/757556512 wait complete. 2024-04-25T13:50:17.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 Processor -- start 2024-04-25T13:50:17.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.341+0000 7f6d8f5c1640 1 -- start start 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d8806f5a0 0x7f6d88160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d88161240 0x7f6d881fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d88161650 0x7f6d881fb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8816b270 con 0x7f6d8806f5a0 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8816b0f0 con 0x7f6d88161650 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f6d8816b3f0 con 0x7f6d88161240 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d8806f5a0 0x7f6d88160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d8806f5a0 0x7f6d88160080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58264/0 (socket says 172.21.15.80:58264) 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 -- 172.21.15.80:0/1090570307 learned_addr learned my addr 172.21.15.80:0/1090570307 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 -- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d88161240 msgr2=0x7f6d881fb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d88161240 0x7f6d881fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 -- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d88161650 msgr2=0x7f6d881fb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d88161650 0x7f6d881fb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 -- 172.21.15.80:0/1090570307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d8814f280 con 0x7f6d8806f5a0 2024-04-25T13:50:17.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8d35d640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d8806f5a0 0x7f6d88160080 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c410 tx=0x7f6d8400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:17.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d79ffb640 1 -- 172.21.15.80:0/1090570307 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d840074e0 con 0x7f6d8806f5a0 2024-04-25T13:50:17.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d79ffb640 1 -- 172.21.15.80:0/1090570307 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d840047f0 con 0x7f6d8806f5a0 2024-04-25T13:50:17.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d8814f560 con 0x7f6d8806f5a0 2024-04-25T13:50:17.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d79ffb640 1 -- 172.21.15.80:0/1090570307 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d84007680 con 0x7f6d8806f5a0 2024-04-25T13:50:17.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d8814fb20 con 0x7f6d8806f5a0 2024-04-25T13:50:17.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d8806a3a0 con 0x7f6d8806f5a0 2024-04-25T13:50:17.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d79ffb640 1 -- 172.21.15.80:0/1090570307 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6d84011070 con 0x7f6d8806f5a0 2024-04-25T13:50:17.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.345+0000 7f6d79ffb640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d6c06f400 0x7f6d6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.349+0000 7f6d79ffb640 1 -- 172.21.15.80:0/1090570307 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5767+0+0 (secure 0 0 0) 0x7f6d8408edd0 con 0x7f6d8806f5a0 2024-04-25T13:50:17.345 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.349+0000 7f6d8cb5c640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d6c06f400 0x7f6d6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.349+0000 7f6d8cb5c640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d6c06f400 0x7f6d6c071910 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000f70 tx=0x7f6d7c0079c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:17.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.353+0000 7f6d79ffb640 1 -- 172.21.15.80:0/1090570307 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f6d8405b750 con 0x7f6d8806f5a0 2024-04-25T13:50:17.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.497+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d880739c0 con 0x7f6d8806f5a0 2024-04-25T13:50:17.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d79ffb640 1 -- 172.21.15.80:0/1090570307 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+16655 (secure 0 0 0) 0x7f6d8405fa50 con 0x7f6d8806f5a0 2024-04-25T13:50:17.498 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:17.499 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":159,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:50:16.264643+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:48:07.666863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"153","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T13:48:08.875350+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.3","mappings":[{"from":2,"to":0}]},{"pgid":"19.a","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:17.500 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d6c06f400 msgr2=0x7f6d6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d6c06f400 0x7f6d6c071910 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c000f70 tx=0x7f6d7c0079c0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d8806f5a0 msgr2=0x7f6d88160080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d8806f5a0 0x7f6d88160080 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f6d8400c410 tx=0x7f6d8400c8d0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 shutdown_connections 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f6d6c06f400 0x7f6d6c071910 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d88161650 0x7f6d881fb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f6d88161240 0x7f6d881fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 --2- 172.21.15.80:0/1090570307 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f6d8806f5a0 0x7f6d88160080 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.501+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 >> 172.21.15.80:0/1090570307 conn(0x7f6d8806d080 msgr2=0x7f6d880b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.505+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 shutdown_connections 2024-04-25T13:50:17.501 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.505+0000 7f6d8f5c1640 1 -- 172.21.15.80:0/1090570307 wait complete. 2024-04-25T13:50:17.520 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:50:17.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb22dfb3640 1 Processor -- start 2024-04-25T13:50:17.674 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb22dfb3640 1 -- start start 2024-04-25T13:50:17.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb22dfb3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb228161f90 0x7fb2280cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb22dfb3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb2280d0510 0x7fb22815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb22dfb3640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2280d0d70 con 0x7fb22815ea70 2024-04-25T13:50:17.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb22dfb3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb2280d1050 con 0x7fb2280d0510 2024-04-25T13:50:17.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb22dfb3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb22806dca0 con 0x7fb228161f90 2024-04-25T13:50:17.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb226ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb2280d0510 0x7fb22815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb226ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb2280d0510 0x7fb22815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58274/0 (socket says 172.21.15.80:58274) 2024-04-25T13:50:17.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb226ffd640 1 -- 172.21.15.80:0/4125474808 learned_addr learned my addr 172.21.15.80:0/4125474808 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:17.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb2277fe640 1 --2- 172.21.15.80:0/4125474808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb228161f90 0x7fb2280cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb226ffd640 1 -- 172.21.15.80:0/4125474808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb228161f90 msgr2=0x7fb2280cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:17.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb226ffd640 1 --2- 172.21.15.80:0/4125474808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb228161f90 0x7fb2280cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb226ffd640 1 -- 172.21.15.80:0/4125474808 >> v1:172.21.15.87:6789/0 conn(0x7fb22815ea70 legacy=0x7fb22815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.677+0000 7fb226ffd640 1 -- 172.21.15.80:0/4125474808 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22806df80 con 0x7fb2280d0510 2024-04-25T13:50:17.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb226ffd640 1 --2- 172.21.15.80:0/4125474808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2280d0510 0x7fb22815e530 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009870 tx=0x7fb21c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ba26b4319ff6e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:17.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb2267fc640 1 -- 172.21.15.80:0/4125474808 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0b1070 con 0x7fb2280d0510 2024-04-25T13:50:17.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb2267fc640 1 -- 172.21.15.80:0/4125474808 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21c0ac600 con 0x7fb2280d0510 2024-04-25T13:50:17.678 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb2267fc640 1 -- 172.21.15.80:0/4125474808 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0ac9d0 con 0x7fb2280d0510 2024-04-25T13:50:17.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb22dfb3640 1 -- 172.21.15.80:0/4125474808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2280d0510 msgr2=0x7fb22815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/4125474808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2280d0510 0x7fb22815e530 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009870 tx=0x7fb21c01c470 comp rx=0 tx=0).stop 2024-04-25T13:50:17.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb22dfb3640 1 -- 172.21.15.80:0/4125474808 shutdown_connections 2024-04-25T13:50:17.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/4125474808 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2280d0510 0x7fb22815e530 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/4125474808 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb228161f90 0x7fb2280cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb22dfb3640 1 -- 172.21.15.80:0/4125474808 >> 172.21.15.80:0/4125474808 conn(0x7fb22806d080 msgr2=0x7fb2280b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:17.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb22dfb3640 1 -- 172.21.15.80:0/4125474808 shutdown_connections 2024-04-25T13:50:17.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.681+0000 7fb22dfb3640 1 -- 172.21.15.80:0/4125474808 wait complete. 2024-04-25T13:50:17.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 Processor -- start 2024-04-25T13:50:17.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 -- start start 2024-04-25T13:50:17.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2280d0510 0x7fb22815beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb22815ea70 0x7fb22815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb228161f90 0x7fb22813a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb22806f630 con 0x7fb22815ea70 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb22806f4b0 con 0x7fb228161f90 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb2280d0920 con 0x7fb2280d0510 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb22815ea70 0x7fb22815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb22815ea70 0x7fb22815c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58278/0 (socket says 172.21.15.80:58278) 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 -- 172.21.15.80:0/418059884 learned_addr learned my addr 172.21.15.80:0/418059884 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb227fff640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb228161f90 0x7fb22813a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb2277fe640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2280d0510 0x7fb22815beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 -- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2280d0510 msgr2=0x7fb22815beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2280d0510 0x7fb22815beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 -- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb228161f90 msgr2=0x7fb22813a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb228161f90 0x7fb22813a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 -- 172.21.15.80:0/418059884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb22815cc50 con 0x7fb22815ea70 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb2277fe640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2280d0510 0x7fb22815beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb226ffd640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb22815ea70 0x7fb22815c420 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb21c01ab80 tx=0x7fb21c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb20ffff640 1 -- 172.21.15.80:0/418059884 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0bb020 con 0x7fb22815ea70 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb20ffff640 1 -- 172.21.15.80:0/418059884 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21c0b7620 con 0x7fb22815ea70 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb22815cf30 con 0x7fb22815ea70 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb20ffff640 1 -- 172.21.15.80:0/418059884 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21c0b1040 con 0x7fb22815ea70 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb22815d4f0 con 0x7fb22815ea70 2024-04-25T13:50:17.685 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb22815c990 con 0x7fb22815ea70 2024-04-25T13:50:17.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.685+0000 7fb20ffff640 1 -- 172.21.15.80:0/418059884 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb21c0be020 con 0x7fb22815ea70 2024-04-25T13:50:17.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.689+0000 7fb20ffff640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb21406f430 0x7fb214071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:17.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.689+0000 7fb20ffff640 1 -- 172.21.15.80:0/418059884 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5767+0+0 (secure 0 0 0) 0x7fb21c129530 con 0x7fb22815ea70 2024-04-25T13:50:17.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.689+0000 7fb2277fe640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb21406f430 0x7fb214071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:17.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.689+0000 7fb2277fe640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb21406f430 0x7fb214071940 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb218006fd0 tx=0x7fb218008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:17.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.693+0000 7fb20ffff640 1 -- 172.21.15.80:0/418059884 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb21c0c5030 con 0x7fb22815ea70 2024-04-25T13:50:17.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.873+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fb22806a3a0 con 0x7fb22815ea70 2024-04-25T13:50:17.871 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:50:17.873+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:50:17.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.877+0000 7fb20ffff640 1 -- 172.21.15.80:0/418059884 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) ==== 161+0+0 (secure 0 0 0) 0x7fb21c016150 con 0x7fb22815ea70 2024-04-25T13:50:17.874 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:50:17.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb21406f430 msgr2=0x7fb214071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb21406f430 0x7fb214071940 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb218006fd0 tx=0x7fb218008040 comp rx=0 tx=0).stop 2024-04-25T13:50:17.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb22815ea70 msgr2=0x7fb22815c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:17.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb22815ea70 0x7fb22815c420 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fb21c01ab80 tx=0x7fb21c01ad10 comp rx=0 tx=0).stop 2024-04-25T13:50:17.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 shutdown_connections 2024-04-25T13:50:17.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb21406f430 0x7fb214071940 secure :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb218006fd0 tx=0x7fb218008040 comp rx=0 tx=0).stop 2024-04-25T13:50:17.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb228161f90 0x7fb22813a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb22815ea70 0x7fb22815c420 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 --2- 172.21.15.80:0/418059884 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2280d0510 0x7fb22815beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:17.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 >> 172.21.15.80:0/418059884 conn(0x7fb22806d080 msgr2=0x7fb2280b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:17.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 shutdown_connections 2024-04-25T13:50:17.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:17.881+0000 7fb22dfb3640 1 -- 172.21.15.80:0/418059884 wait complete. 2024-04-25T13:50:17.888 INFO:tasks.ceph.mds.a.smithi080.stderr: -9924> 2024-04-25T13:48:15.494+0000 7fa6ebed5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:50:17.900 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:50:18.001 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:50:18.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa63937b640 1 Processor -- start 2024-04-25T13:50:18.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa63937b640 1 -- start start 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa63937b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa6340b9e70 0x7fa6340ba280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa63937b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6340b92a0 con 0x7fa6340d3ed0 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa63937b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa6340d4c10 con 0x7fa6340d2ca0 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa63937b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa6340d4ef0 con 0x7fa6340b9e70 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa6337fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa6340b9e70 0x7fa6340ba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa632ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa6340d3ed0 0x7fa6340d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58480/0 (socket says 172.21.15.80:58480) 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.041+0000 7fa632ffd640 1 -- 172.21.15.80:0/1087580759 learned_addr learned my addr 172.21.15.80:0/1087580759 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa6337fe640 1 -- 172.21.15.80:0/1087580759 >> v1:172.21.15.87:6789/0 conn(0x7fa6340d2ca0 legacy=0x7fa6340b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa6337fe640 1 -- 172.21.15.80:0/1087580759 >> v1:172.21.15.80:6789/0 conn(0x7fa6340d3ed0 legacy=0x7fa6340d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa631ffb640 1 -- 172.21.15.80:0/1087580759 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3730846285 0 0) 0x7fa628001120 con 0x7fa6340d3ed0 2024-04-25T13:50:18.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa6337fe640 1 -- 172.21.15.80:0/1087580759 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340d51d0 con 0x7fa6340b9e70 2024-04-25T13:50:18.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa6337fe640 1 --2- 172.21.15.80:0/1087580759 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340b9e70 0x7fa6340ba280 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0097c0 tx=0x7fa62c01c350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70e4cfebb8c8e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:18.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa631ffb640 1 -- 172.21.15.80:0/1087580759 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0b0070 con 0x7fa6340b9e70 2024-04-25T13:50:18.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa631ffb640 1 -- 172.21.15.80:0/1087580759 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa62c0ab600 con 0x7fa6340b9e70 2024-04-25T13:50:18.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa631ffb640 1 -- 172.21.15.80:0/1087580759 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0ab9d0 con 0x7fa6340b9e70 2024-04-25T13:50:18.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa63937b640 1 -- 172.21.15.80:0/1087580759 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340b9e70 msgr2=0x7fa6340ba280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:18.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa63937b640 1 --2- 172.21.15.80:0/1087580759 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340b9e70 0x7fa6340ba280 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa62c0097c0 tx=0x7fa62c01c350 comp rx=0 tx=0).stop 2024-04-25T13:50:18.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa63937b640 1 -- 172.21.15.80:0/1087580759 shutdown_connections 2024-04-25T13:50:18.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa63937b640 1 --2- 172.21.15.80:0/1087580759 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340b9e70 0x7fa6340ba280 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:18.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa63937b640 1 -- 172.21.15.80:0/1087580759 >> 172.21.15.80:0/1087580759 conn(0x7fa6340737b0 msgr2=0x7fa63406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:18.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa63937b640 1 -- 172.21.15.80:0/1087580759 shutdown_connections 2024-04-25T13:50:18.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.045+0000 7fa63937b640 1 -- 172.21.15.80:0/1087580759 wait complete. 2024-04-25T13:50:18.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 Processor -- start 2024-04-25T13:50:18.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 -- start start 2024-04-25T13:50:18.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6340b9e70 0x7fa634150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:18.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340d2ca0 0x7fa6340c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:18.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6340d3ed0 0x7fa6341551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:18.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa63416e6b0 con 0x7fa6340d2ca0 2024-04-25T13:50:18.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa63416e830 con 0x7fa6340b9e70 2024-04-25T13:50:18.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa63416e9b0 con 0x7fa6340d3ed0 2024-04-25T13:50:18.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340d2ca0 0x7fa6340c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:18.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340d2ca0 0x7fa6340c6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58306/0 (socket says 172.21.15.80:58306) 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 -- 172.21.15.80:0/4085143033 learned_addr learned my addr 172.21.15.80:0/4085143033 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6337fe640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6340d3ed0 0x7fa6341551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa632ffd640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6340b9e70 0x7fa634150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 -- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6340d3ed0 msgr2=0x7fa6341551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6340d3ed0 0x7fa6341551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 -- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6340b9e70 msgr2=0x7fa634150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6340b9e70 0x7fa634150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 -- 172.21.15.80:0/4085143033 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6340b7e50 con 0x7fa6340d2ca0 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa632ffd640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6340b9e70 0x7fa634150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa6327fc640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340d2ca0 0x7fa6340c6b10 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c7d0 tx=0x7fa62400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa61b7fe640 1 -- 172.21.15.80:0/4085143033 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa624003b80 con 0x7fa6340d2ca0 2024-04-25T13:50:18.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa61b7fe640 1 -- 172.21.15.80:0/4085143033 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa624003d20 con 0x7fa6340d2ca0 2024-04-25T13:50:18.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6340b8130 con 0x7fa6340d2ca0 2024-04-25T13:50:18.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa61b7fe640 1 -- 172.21.15.80:0/4085143033 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62400aa90 con 0x7fa6340d2ca0 2024-04-25T13:50:18.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.049+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6340b78e0 con 0x7fa6340d2ca0 2024-04-25T13:50:18.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.053+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6340b7bc0 con 0x7fa6340d2ca0 2024-04-25T13:50:18.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.053+0000 7fa61b7fe640 1 -- 172.21.15.80:0/4085143033 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa62400ac30 con 0x7fa6340d2ca0 2024-04-25T13:50:18.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.053+0000 7fa61b7fe640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa62006f400 0x7fa620071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:18.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.053+0000 7fa61b7fe640 1 -- 172.21.15.80:0/4085143033 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5853+0+0 (secure 0 0 0) 0x7fa62408f250 con 0x7fa6340d2ca0 2024-04-25T13:50:18.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.053+0000 7fa632ffd640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa62006f400 0x7fa620071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:18.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.053+0000 7fa632ffd640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa62006f400 0x7fa620071910 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa628000f70 tx=0x7fa628009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:18.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.057+0000 7fa61b7fe640 1 -- 172.21.15.80:0/4085143033 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa62405c0a0 con 0x7fa6340d2ca0 2024-04-25T13:50:18.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.245+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa63406a3a0 con 0x7fa6340d2ca0 2024-04-25T13:50:18.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa61b7fe640 1 -- 172.21.15.80:0/4085143033 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) ==== 104+0+0 (secure 0 0 0) 0x7fa62405b9a0 con 0x7fa6340d2ca0 2024-04-25T13:50:18.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa62006f400 msgr2=0x7fa620071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:18.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa62006f400 0x7fa620071910 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa628000f70 tx=0x7fa628009040 comp rx=0 tx=0).stop 2024-04-25T13:50:18.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340d2ca0 msgr2=0x7fa6340c6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340d2ca0 0x7fa6340c6b10 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fa62400c7d0 tx=0x7fa62400cc90 comp rx=0 tx=0).stop 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 shutdown_connections 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa62006f400 0x7fa620071910 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa6340d3ed0 0x7fa6341551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa6340d2ca0 0x7fa6340c6b10 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 --2- 172.21.15.80:0/4085143033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa6340b9e70 0x7fa634150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 >> 172.21.15.80:0/4085143033 conn(0x7fa6340737b0 msgr2=0x7fa63415e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 shutdown_connections 2024-04-25T13:50:18.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:18.881+0000 7fa63937b640 1 -- 172.21.15.80:0/4085143033 wait complete. 2024-04-25T13:50:18.908 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:50:19.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c266a1640 1 Processor -- start 2024-04-25T13:50:19.069 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c266a1640 1 -- start start 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c266a1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5c2015a810 0x7f5c20161270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c266a1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c2015ac20 con 0x7f5c20068ca0 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c266a1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c200b7d70 con 0x7f5c200b6760 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c266a1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5c200b8050 con 0x7f5c2015a810 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1f7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5c20068ca0 0x7f5c200b0570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:58494/0 (socket says 172.21.15.80:58494) 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1f7fe640 1 -- 172.21.15.80:0/3602459284 learned_addr learned my addr 172.21.15.80:0/3602459284 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c24c3e640 1 --2- 172.21.15.80:0/3602459284 >> v2:172.21.15.87:3300/0 conn(0x7f5c2015a810 0x7f5c20161270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4145454254 0 0) 0x7f5c2015ac20 con 0x7f5c20068ca0 2024-04-25T13:50:19.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5c08003530 con 0x7f5c20068ca0 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4046530617 0 0) 0x7f5c08003530 con 0x7f5c20068ca0 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 >> v2:172.21.15.87:3300/0 conn(0x7f5c2015a810 msgr2=0x7f5c20161270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 --2- 172.21.15.80:0/3602459284 >> v2:172.21.15.87:3300/0 conn(0x7f5c2015a810 0x7f5c20161270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 >> v1:172.21.15.87:6789/0 conn(0x7f5c200b6760 legacy=0x7f5c20068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c200b5c80 con 0x7f5c20068ca0 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f5c14001680 con 0x7f5c20068ca0 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 >> v1:172.21.15.80:6789/0 conn(0x7f5c20068ca0 legacy=0x7f5c200b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c080039c0 0x7f5c08003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c08004810 0x7f5c08004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c080051d0 0x7f5c0800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0800c140 con 0x7f5c08004810 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0800c2f0 con 0x7f5c080051d0 2024-04-25T13:50:19.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c0800c4f0 con 0x7f5c080039c0 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.073+0000 7f5c1f7fe640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c08004810 0x7f5c08004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c24c3e640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c080051d0 0x7f5c0800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1ffff640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c080039c0 0x7f5c08003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1f7fe640 1 -- 172.21.15.80:0/3602459284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c080039c0 msgr2=0x7f5c08003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1f7fe640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c080039c0 0x7f5c08003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1f7fe640 1 -- 172.21.15.80:0/3602459284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c080051d0 msgr2=0x7f5c0800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1f7fe640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c080051d0 0x7f5c0800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c24c3e640 1 -- 172.21.15.80:0/3602459284 reap_dead start 2024-04-25T13:50:19.075 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1f7fe640 1 -- 172.21.15.80:0/3602459284 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c08003530 con 0x7f5c08004810 2024-04-25T13:50:19.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1f7fe640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c08004810 0x7f5c08004c20 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c160 tx=0x7f5c1401cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7bcf5f46e7e938c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14007cb0 con 0x7f5c08004810 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c140a9020 con 0x7f5c08004810 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c1effd640 1 -- 172.21.15.80:0/3602459284 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14005b20 con 0x7f5c08004810 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c266a1640 1 -- 172.21.15.80:0/3602459284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c08004810 msgr2=0x7f5c08004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c266a1640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c08004810 0x7f5c08004c20 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f5c1400c160 tx=0x7f5c1401cba0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c266a1640 1 -- 172.21.15.80:0/3602459284 shutdown_connections 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c266a1640 1 --2- 172.21.15.80:0/3602459284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c08004810 0x7f5c08004c20 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c266a1640 1 -- 172.21.15.80:0/3602459284 >> 172.21.15.80:0/3602459284 conn(0x7f5c20073750 msgr2=0x7f5c20073b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c266a1640 1 -- 172.21.15.80:0/3602459284 shutdown_connections 2024-04-25T13:50:19.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.077+0000 7f5c266a1640 1 -- 172.21.15.80:0/3602459284 wait complete. 2024-04-25T13:50:19.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c266a1640 1 Processor -- start 2024-04-25T13:50:19.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c266a1640 1 -- start start 2024-04-25T13:50:19.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c266a1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c08004810 0x7f5c201554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c266a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c2015ee60 0x7f5c2015f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c266a1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c20163cd0 0x7f5c201640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c266a1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5c20161680 con 0x7f5c2015ee60 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c266a1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5c200b21e0 con 0x7f5c08004810 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c266a1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5c200b0cb0 con 0x7f5c20163cd0 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c2015ee60 0x7f5c2015f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c2015ee60 0x7f5c2015f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58326/0 (socket says 172.21.15.80:58326) 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 -- 172.21.15.80:0/1962896365 learned_addr learned my addr 172.21.15.80:0/1962896365 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1ffff640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c08004810 0x7f5c201554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c24c3e640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c20163cd0 0x7f5c201640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 -- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c20163cd0 msgr2=0x7f5c201640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c20163cd0 0x7f5c201640e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 -- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c08004810 msgr2=0x7f5c201554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c08004810 0x7f5c201554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 -- 172.21.15.80:0/1962896365 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c20164620 con 0x7f5c2015ee60 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1f7fe640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c2015ee60 0x7f5c2015f270 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5c140ab040 tx=0x7f5c08005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1cff9640 1 -- 172.21.15.80:0/1962896365 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14007cb0 con 0x7f5c2015ee60 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1cff9640 1 -- 172.21.15.80:0/1962896365 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c140b2070 con 0x7f5c2015ee60 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.081+0000 7f5c1cff9640 1 -- 172.21.15.80:0/1962896365 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c14001be0 con 0x7f5c2015ee60 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.085+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c2015c300 con 0x7f5c2015ee60 2024-04-25T13:50:19.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.085+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c2015c8c0 con 0x7f5c2015ee60 2024-04-25T13:50:19.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.085+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c2015c070 con 0x7f5c2015ee60 2024-04-25T13:50:19.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.085+0000 7f5c1cff9640 1 -- 172.21.15.80:0/1962896365 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5c14004bf0 con 0x7f5c2015ee60 2024-04-25T13:50:19.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.085+0000 7f5c1cff9640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c0c06f430 0x7f5c0c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:19.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.085+0000 7f5c1cff9640 1 -- 172.21.15.80:0/1962896365 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5853+0+0 (secure 0 0 0) 0x7f5c14128a40 con 0x7f5c2015ee60 2024-04-25T13:50:19.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.085+0000 7f5c1ffff640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c0c06f430 0x7f5c0c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:19.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.085+0000 7f5c1ffff640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c0c06f430 0x7f5c0c071940 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5c10000c10 tx=0x7f5c10009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:19.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.089+0000 7f5c1cff9640 1 -- 172.21.15.80:0/1962896365 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5c1412ae20 con 0x7f5c2015ee60 2024-04-25T13:50:19.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.253+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5c201559e0 con 0x7f5c2015ee60 2024-04-25T13:50:19.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.889+0000 7f5c1cff9640 1 -- 172.21.15.80:0/1962896365 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v161) ==== 181+0+0 (secure 0 0 0) 0x7f5c20164620 con 0x7f5c2015ee60 2024-04-25T13:50:19.887 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:50:19.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c0c06f430 msgr2=0x7f5c0c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c0c06f430 0x7f5c0c071940 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5c10000c10 tx=0x7f5c10009040 comp rx=0 tx=0).stop 2024-04-25T13:50:19.891 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c2015ee60 msgr2=0x7f5c2015f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c2015ee60 0x7f5c2015f270 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7f5c140ab040 tx=0x7f5c08005840 comp rx=0 tx=0).stop 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 shutdown_connections 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5c0c06f430 0x7f5c0c071940 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5c20163cd0 0x7f5c201640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5c2015ee60 0x7f5c2015f270 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 --2- 172.21.15.80:0/1962896365 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c08004810 0x7f5c201554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 >> 172.21.15.80:0/1962896365 conn(0x7f5c20073750 msgr2=0x7f5c20070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 shutdown_connections 2024-04-25T13:50:19.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:19.893+0000 7f5c266a1640 1 -- 172.21.15.80:0/1962896365 wait complete. 2024-04-25T13:50:19.918 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:50:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c7772b640 1 Processor -- start 2024-04-25T13:50:20.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c7772b640 1 -- start start 2024-04-25T13:50:20.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c7772b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8c70164900 0x7f8c7006da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c7772b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 0x7f8c700cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c7772b640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c700d04b0 con 0x7f8c70160890 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c7772b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8c700b1d70 con 0x7f8c70164900 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c7772b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8c70163eb0 con 0x7f8c700cfb60 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c754c7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8c70164900 0x7f8c7006da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c754c7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8c70164900 0x7f8c7006da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58334/0 (socket says 172.21.15.80:58334) 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c754c7640 1 -- 172.21.15.80:0/1229384291 learned_addr learned my addr 172.21.15.80:0/1229384291 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c74cc6640 1 --2- 172.21.15.80:0/1229384291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 0x7f8c700cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c754c7640 1 -- 172.21.15.80:0/1229384291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 msgr2=0x7f8c700cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c754c7640 1 --2- 172.21.15.80:0/1229384291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 0x7f8c700cff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c754c7640 1 -- 172.21.15.80:0/1229384291 >> v1:172.21.15.87:6789/0 conn(0x7f8c70160890 legacy=0x7f8c70160ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:20.077 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.077+0000 7f8c754c7640 1 -- 172.21.15.80:0/1229384291 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c70164d10 con 0x7f8c70164900 2024-04-25T13:50:20.078 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c754c7640 1 --2- 172.21.15.80:0/1229384291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70164900 0x7f8c7006da70 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009870 tx=0x7f8c6c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d16cb36329517c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:20.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c67fff640 1 -- 172.21.15.80:0/1229384291 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0ab650 con 0x7f8c70164900 2024-04-25T13:50:20.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c67fff640 1 -- 172.21.15.80:0/1229384291 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c6c0b0070 con 0x7f8c70164900 2024-04-25T13:50:20.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c67fff640 1 -- 172.21.15.80:0/1229384291 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c0aba10 con 0x7f8c70164900 2024-04-25T13:50:20.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c7772b640 1 -- 172.21.15.80:0/1229384291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70164900 msgr2=0x7f8c7006da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:20.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c7772b640 1 --2- 172.21.15.80:0/1229384291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70164900 0x7f8c7006da70 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c009870 tx=0x7f8c6c01c560 comp rx=0 tx=0).stop 2024-04-25T13:50:20.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c7772b640 1 -- 172.21.15.80:0/1229384291 shutdown_connections 2024-04-25T13:50:20.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c7772b640 1 --2- 172.21.15.80:0/1229384291 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 0x7f8c700cff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c7772b640 1 --2- 172.21.15.80:0/1229384291 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70164900 0x7f8c7006da70 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c7772b640 1 -- 172.21.15.80:0/1229384291 >> 172.21.15.80:0/1229384291 conn(0x7f8c700d3310 msgr2=0x7f8c700cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:20.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c7772b640 1 -- 172.21.15.80:0/1229384291 shutdown_connections 2024-04-25T13:50:20.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.081+0000 7f8c7772b640 1 -- 172.21.15.80:0/1229384291 wait complete. 2024-04-25T13:50:20.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 Processor -- start 2024-04-25T13:50:20.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 -- start start 2024-04-25T13:50:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 0x7f8c700b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70160890 0x7f8c701539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c70164900 0x7f8c701fd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8c70157640 con 0x7f8c70160890 2024-04-25T13:50:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7016d850 con 0x7f8c70164900 2024-04-25T13:50:20.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8c700b3e70 con 0x7f8c700cfb60 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70160890 0x7f8c701539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70160890 0x7f8c701539b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58336/0 (socket says 172.21.15.80:58336) 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 -- 172.21.15.80:0/3798760180 learned_addr learned my addr 172.21.15.80:0/3798760180 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c754c7640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 0x7f8c700b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c75cc8640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c70164900 0x7f8c701fd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 -- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 msgr2=0x7f8c700b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 0x7f8c700b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 -- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c70164900 msgr2=0x7f8c701fd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c70164900 0x7f8c701fd5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 -- 172.21.15.80:0/3798760180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c700b1200 con 0x7f8c70160890 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c74cc6640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70160890 0x7f8c701539b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d620 tx=0x7f8c6000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:20.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c65ffb640 1 -- 172.21.15.80:0/3798760180 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000b620 con 0x7f8c70160890 2024-04-25T13:50:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c65ffb640 1 -- 172.21.15.80:0/3798760180 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c60019070 con 0x7f8c70160890 2024-04-25T13:50:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7014aca0 con 0x7f8c70160890 2024-04-25T13:50:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c65ffb640 1 -- 172.21.15.80:0/3798760180 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000b9e0 con 0x7f8c70160890 2024-04-25T13:50:20.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.085+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7014b260 con 0x7f8c70160890 2024-04-25T13:50:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.089+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c7006a440 con 0x7f8c70160890 2024-04-25T13:50:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.089+0000 7f8c65ffb640 1 -- 172.21.15.80:0/3798760180 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8c6000bb80 con 0x7f8c70160890 2024-04-25T13:50:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.089+0000 7f8c65ffb640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8c5406f430 0x7f8c54071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:20.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.089+0000 7f8c65ffb640 1 -- 172.21.15.80:0/3798760180 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5415+0+0 (secure 0 0 0) 0x7f8c6008ffd0 con 0x7f8c70160890 2024-04-25T13:50:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.089+0000 7f8c754c7640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8c5406f430 0x7f8c54071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:20.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.089+0000 7f8c754c7640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8c5406f430 0x7f8c54071940 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0af000 tx=0x7f8c6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:20.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.093+0000 7f8c65ffb640 1 -- 172.21.15.80:0/3798760180 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8c6005d150 con 0x7f8c70160890 2024-04-25T13:50:20.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.253+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f8c7014aa30 con 0x7f8c70160890 2024-04-25T13:50:20.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.901+0000 7f8c65ffb640 1 -- 172.21.15.80:0/3798760180 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v162) ==== 169+0+0 (secure 0 0 0) 0x7f8c6005ca50 con 0x7f8c70160890 2024-04-25T13:50:20.899 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:50:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8c5406f430 msgr2=0x7f8c54071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8c5406f430 0x7f8c54071940 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c0af000 tx=0x7f8c6c008040 comp rx=0 tx=0).stop 2024-04-25T13:50:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70160890 msgr2=0x7f8c701539b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:20.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70160890 0x7f8c701539b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8c6000d620 tx=0x7f8c6000dae0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 shutdown_connections 2024-04-25T13:50:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8c5406f430 0x7f8c54071940 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c70164900 0x7f8c701fd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8c70160890 0x7f8c701539b0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 --2- 172.21.15.80:0/3798760180 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8c700cfb60 0x7f8c700b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 >> 172.21.15.80:0/3798760180 conn(0x7f8c700d3310 msgr2=0x7f8c700d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 shutdown_connections 2024-04-25T13:50:20.903 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:20.905+0000 7f8c7772b640 1 -- 172.21.15.80:0/3798760180 wait complete. 2024-04-25T13:50:20.927 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:50:21.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f43553e9640 1 Processor -- start 2024-04-25T13:50:21.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f43553e9640 1 -- start start 2024-04-25T13:50:21.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f43553e9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f43501574c0 0x7f4350073c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f43553e9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f435006fbf0 con 0x7f4350071e60 2024-04-25T13:50:21.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f43553e9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f43500b0a20 con 0x7f4350161170 2024-04-25T13:50:21.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f43553e9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f43500b1350 con 0x7f43501574c0 2024-04-25T13:50:21.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f434effd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4350071e60 0x7f4350160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45890/0 (socket says 172.21.15.80:45890) 2024-04-25T13:50:21.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f434effd640 1 -- 172.21.15.80:0/1993678782 learned_addr learned my addr 172.21.15.80:0/1993678782 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:21.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f434f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f43501574c0 0x7f4350073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f434dffb640 1 -- 172.21.15.80:0/1993678782 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3325507976 0 0) 0x7f435006fbf0 con 0x7f4350071e60 2024-04-25T13:50:21.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f434dffb640 1 -- 172.21.15.80:0/1993678782 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4338003530 con 0x7f4350071e60 2024-04-25T13:50:21.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f434f7fe640 1 -- 172.21.15.80:0/1993678782 >> v1:172.21.15.87:6789/0 conn(0x7f4350161170 legacy=0x7f43500b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f434f7fe640 1 -- 172.21.15.80:0/1993678782 >> v1:172.21.15.80:6789/0 conn(0x7f4350071e60 legacy=0x7f4350160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.085+0000 7f434f7fe640 1 -- 172.21.15.80:0/1993678782 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43500b0d00 con 0x7f43501574c0 2024-04-25T13:50:21.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f434f7fe640 1 --2- 172.21.15.80:0/1993678782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350073c40 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f43500b14d0 tx=0x7f434801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5ee1b47ef000e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:21.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f434dffb640 1 -- 172.21.15.80:0/1993678782 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480b2070 con 0x7f43501574c0 2024-04-25T13:50:21.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f434dffb640 1 -- 172.21.15.80:0/1993678782 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f43480a5810 con 0x7f43501574c0 2024-04-25T13:50:21.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f434dffb640 1 -- 172.21.15.80:0/1993678782 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43480a5be0 con 0x7f43501574c0 2024-04-25T13:50:21.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f43553e9640 1 -- 172.21.15.80:0/1993678782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 msgr2=0x7f4350073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f43553e9640 1 --2- 172.21.15.80:0/1993678782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350073c40 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f43500b14d0 tx=0x7f434801c950 comp rx=0 tx=0).stop 2024-04-25T13:50:21.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f43553e9640 1 -- 172.21.15.80:0/1993678782 shutdown_connections 2024-04-25T13:50:21.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f43553e9640 1 --2- 172.21.15.80:0/1993678782 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350073c40 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f43553e9640 1 -- 172.21.15.80:0/1993678782 >> 172.21.15.80:0/1993678782 conn(0x7f43500b4160 msgr2=0x7f43500d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:21.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f43553e9640 1 -- 172.21.15.80:0/1993678782 shutdown_connections 2024-04-25T13:50:21.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.089+0000 7f43553e9640 1 -- 172.21.15.80:0/1993678782 wait complete. 2024-04-25T13:50:21.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 Processor -- start 2024-04-25T13:50:21.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 -- start start 2024-04-25T13:50:21.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4350071e60 0x7f435013aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43500af8b0 0x7f43500afcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f43500b56c0 con 0x7f43501574c0 2024-04-25T13:50:21.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f435016b7e0 con 0x7f43500af8b0 2024-04-25T13:50:21.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f43500ceff0 con 0x7f4350071e60 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53380/0 (socket says 172.21.15.80:53380) 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 -- 172.21.15.80:0/4064927732 learned_addr learned my addr 172.21.15.80:0/4064927732 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434f7fe640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43500af8b0 0x7f43500afcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 -- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4350071e60 msgr2=0x7f435013aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4350071e60 0x7f435013aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 -- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43500af8b0 msgr2=0x7f43500afcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43500af8b0 0x7f43500afcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 -- 172.21.15.80:0/4064927732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43500b0200 con 0x7f43501574c0 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434f7fe640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43500af8b0 0x7f43500afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f434e7fc640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350152430 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f434000c370 tx=0x7f434000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:21.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43377fe640 1 -- 172.21.15.80:0/4064927732 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400052d0 con 0x7f43501574c0 2024-04-25T13:50:21.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43377fe640 1 -- 172.21.15.80:0/4064927732 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4340005470 con 0x7f43501574c0 2024-04-25T13:50:21.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435015fd20 con 0x7f43501574c0 2024-04-25T13:50:21.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43377fe640 1 -- 172.21.15.80:0/4064927732 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340007260 con 0x7f43501574c0 2024-04-25T13:50:21.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435013d130 con 0x7f43501574c0 2024-04-25T13:50:21.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.093+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4350154280 con 0x7f43501574c0 2024-04-25T13:50:21.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.097+0000 7f43377fe640 1 -- 172.21.15.80:0/4064927732 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4340007400 con 0x7f43501574c0 2024-04-25T13:50:21.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.097+0000 7f43377fe640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f433c06f400 0x7f433c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.097+0000 7f43377fe640 1 -- 172.21.15.80:0/4064927732 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4943+0+0 (secure 0 0 0) 0x7f434008e3d0 con 0x7f43501574c0 2024-04-25T13:50:21.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.097+0000 7f434effd640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f433c06f400 0x7f433c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.097+0000 7f434effd640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f433c06f400 0x7f433c071910 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4344007fd0 tx=0x7f4344007940 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:21.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.101+0000 7f43377fe640 1 -- 172.21.15.80:0/4064927732 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f43500b0200 con 0x7f43501574c0 2024-04-25T13:50:21.259 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.261+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f435013bd30 con 0x7f43501574c0 2024-04-25T13:50:21.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43377fe640 1 -- 172.21.15.80:0/4064927732 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) ==== 104+0+0 (secure 0 0 0) 0x7f434005b650 con 0x7f43501574c0 2024-04-25T13:50:21.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f433c06f400 msgr2=0x7f433c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f433c06f400 0x7f433c071910 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f4344007fd0 tx=0x7f4344007940 comp rx=0 tx=0).stop 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 msgr2=0x7f4350152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350152430 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f434000c370 tx=0x7f434000c830 comp rx=0 tx=0).stop 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 shutdown_connections 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f433c06f400 0x7f433c071910 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f43500af8b0 0x7f43500afcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f43501574c0 0x7f4350152430 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 --2- 172.21.15.80:0/4064927732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4350071e60 0x7f435013aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 >> 172.21.15.80:0/4064927732 conn(0x7f43500b4160 msgr2=0x7f43500b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 shutdown_connections 2024-04-25T13:50:21.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.269+0000 7f43553e9640 1 -- 172.21.15.80:0/4064927732 wait complete. 2024-04-25T13:50:21.290 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2024-04-25T13:50:21.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f094d5ad640 1 Processor -- start 2024-04-25T13:50:21.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f094d5ad640 1 -- start start 2024-04-25T13:50:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f094d5ad640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f09480b1990 0x7f09480d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f094d5ad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09480c7c60 0x7f09480c8070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f094d5ad640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09480d0900 con 0x7f0948068590 2024-04-25T13:50:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f094d5ad640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f09480c77d0 con 0x7f09480b1990 2024-04-25T13:50:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f094d5ad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09480d2640 con 0x7f09480c7c60 2024-04-25T13:50:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f09467fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0948068590 0x7f09480689a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45904/0 (socket says 172.21.15.80:45904) 2024-04-25T13:50:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f09467fc640 1 -- 172.21.15.80:0/2771344509 learned_addr learned my addr 172.21.15.80:0/2771344509 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:21.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0946ffd640 1 --2- 172.21.15.80:0/2771344509 >> v2:172.21.15.87:3300/0 conn(0x7f09480b1990 0x7f09480d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f09477fe640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09480c7c60 0x7f09480c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3660542414 0 0) 0x7f09480d0900 con 0x7f0948068590 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0930003530 con 0x7f0948068590 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3879617081 0 0) 0x7f0930003530 con 0x7f0948068590 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09480c7c60 msgr2=0x7f09480c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09480c7c60 0x7f09480c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 >> v2:172.21.15.87:3300/0 conn(0x7f09480b1990 msgr2=0x7f09480d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 --2- 172.21.15.80:0/2771344509 >> v2:172.21.15.87:3300/0 conn(0x7f09480b1990 0x7f09480d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09480d0b70 con 0x7f0948068590 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f093c001680 con 0x7f0948068590 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 >> v1:172.21.15.80:6789/0 conn(0x7f0948068590 legacy=0x7f09480689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09300039c0 0x7f0930003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0930004810 0x7f0930004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.442 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09480c7c60 0x7f0930005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09480d2640 con 0x7f09300039c0 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f093000b9d0 con 0x7f0930004810 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f093000bb50 con 0x7f09480c7c60 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0946ffd640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09300039c0 0x7f0930003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f09467fc640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0930004810 0x7f0930004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f09477fe640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09480c7c60 0x7f0930005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0946ffd640 1 -- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09480c7c60 msgr2=0x7f0930005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0946ffd640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09480c7c60 0x7f0930005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0946ffd640 1 -- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0930004810 msgr2=0x7f0930004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0946ffd640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0930004810 0x7f0930004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.443 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.441+0000 7f0946ffd640 1 -- 172.21.15.80:0/2771344509 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0930003530 con 0x7f09300039c0 2024-04-25T13:50:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f0946ffd640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09300039c0 0x7f0930003e50 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f0938009ae0 tx=0x7f093801cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397cb6b06f89dec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380b2070 con 0x7f09300039c0 2024-04-25T13:50:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09380a5570 con 0x7f09300039c0 2024-04-25T13:50:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f0945ffb640 1 -- 172.21.15.80:0/2771344509 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09380096b0 con 0x7f09300039c0 2024-04-25T13:50:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f094d5ad640 1 -- 172.21.15.80:0/2771344509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09300039c0 msgr2=0x7f0930003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f094d5ad640 1 --2- 172.21.15.80:0/2771344509 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09300039c0 0x7f0930003e50 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f0938009ae0 tx=0x7f093801cab0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f09477fe640 1 -- 172.21.15.80:0/2771344509 reap_dead start 2024-04-25T13:50:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f094d5ad640 1 -- 172.21.15.80:0/2771344509 shutdown_connections 2024-04-25T13:50:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f094d5ad640 1 -- 172.21.15.80:0/2771344509 >> 172.21.15.80:0/2771344509 conn(0x7f09480cf390 msgr2=0x7f09480cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f094d5ad640 1 -- 172.21.15.80:0/2771344509 shutdown_connections 2024-04-25T13:50:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.445+0000 7f094d5ad640 1 -- 172.21.15.80:0/2771344509 wait complete. 2024-04-25T13:50:21.446 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.449+0000 7f094d5ad640 1 Processor -- start 2024-04-25T13:50:21.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.449+0000 7f094d5ad640 1 -- start start 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.449+0000 7f094d5ad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09300039c0 0x7f09480b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f094d5ad640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09480cb100 0x7f09480cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f094d5ad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09480cb510 0x7f09480cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f094d5ad640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09480c8880 con 0x7f09480cb100 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f094d5ad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f09480b1da0 con 0x7f09480cb510 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f094d5ad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09480c8a00 con 0x7f09300039c0 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09480cb100 0x7f09480cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09480cb100 0x7f09480cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53406/0 (socket says 172.21.15.80:53406) 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 -- 172.21.15.80:0/250987010 learned_addr learned my addr 172.21.15.80:0/250987010 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:21.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f0946ffd640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09300039c0 0x7f09480b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09477fe640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09480cb510 0x7f09480cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 -- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09300039c0 msgr2=0x7f09480b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09300039c0 0x7f09480b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 -- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09480cb510 msgr2=0x7f09480cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09480cb510 0x7f09480cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 -- 172.21.15.80:0/250987010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09480cedb0 con 0x7f09480cb100 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09477fe640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09480cb510 0x7f09480cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f09467fc640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09480cb100 0x7f09480cba90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f093c00e7b0 tx=0x7f093c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f092f7fe640 1 -- 172.21.15.80:0/250987010 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c00ee60 con 0x7f09480cb100 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f092f7fe640 1 -- 172.21.15.80:0/250987010 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f093c018070 con 0x7f09480cb100 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0948163530 con 0x7f09480cb100 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f092f7fe640 1 -- 172.21.15.80:0/250987010 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f093c00cab0 con 0x7f09480cb100 2024-04-25T13:50:21.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0948162a50 con 0x7f09480cb100 2024-04-25T13:50:21.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f094806a3a0 con 0x7f09480cb100 2024-04-25T13:50:21.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f092f7fe640 1 -- 172.21.15.80:0/250987010 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f093c00cc50 con 0x7f09480cb100 2024-04-25T13:50:21.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f092f7fe640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f093406f400 0x7f0934071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:21.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f092f7fe640 1 -- 172.21.15.80:0/250987010 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4943+0+0 (secure 0 0 0) 0x7f093c08ece0 con 0x7f09480cb100 2024-04-25T13:50:21.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.453+0000 7f0946ffd640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f093406f400 0x7f0934071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:21.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.457+0000 7f0946ffd640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f093406f400 0x7f0934071910 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f09380af000 tx=0x7f09380046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:21.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.461+0000 7f092f7fe640 1 -- 172.21.15.80:0/250987010 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f093c05bed0 con 0x7f09480cb100 2024-04-25T13:50:21.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.685+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) -- 0x7f09481648d0 con 0x7f09480cb100 2024-04-25T13:50:21.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f092f7fe640 1 -- 172.21.15.80:0/250987010 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v488) ==== 132+0+0 (secure 0 0 0) 0x7f093c05b7d0 con 0x7f09480cb100 2024-04-25T13:50:21.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f093406f400 msgr2=0x7f0934071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f093406f400 0x7f0934071910 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f09380af000 tx=0x7f09380046d0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09480cb100 msgr2=0x7f09480cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:21.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09480cb100 0x7f09480cba90 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f093c00e7b0 tx=0x7f093c00ec70 comp rx=0 tx=0).stop 2024-04-25T13:50:21.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 shutdown_connections 2024-04-25T13:50:21.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f093406f400 0x7f0934071910 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09480cb510 0x7f09480cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09480cb100 0x7f09480cba90 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 --2- 172.21.15.80:0/250987010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09300039c0 0x7f09480b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:21.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 >> 172.21.15.80:0/250987010 conn(0x7f09480cf390 msgr2=0x7f09480b6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:21.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 shutdown_connections 2024-04-25T13:50:21.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:21.905+0000 7f094d5ad640 1 -- 172.21.15.80:0/250987010 wait complete. 2024-04-25T13:50:21.929 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-04-25T13:50:21.930 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2024-04-25T13:50:21.930 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2024-04-25T13:50:21.930 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:22.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.085+0000 7fecca6b9640 1 Processor -- start 2024-04-25T13:50:22.084 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.085+0000 7fecca6b9640 1 -- start start 2024-04-25T13:50:22.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecca6b9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fecc406f5a0 0x7fecc4160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecca6b9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fecc4161070 0x7fecc4161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecca6b9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fecc406fcf0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecca6b9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fecc41619c0 con 0x7fecc406f5a0 2024-04-25T13:50:22.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecca6b9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fecc4073f10 con 0x7fecc4161070 2024-04-25T13:50:22.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecc3fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fecc406f5a0 0x7fecc4160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecc8c56640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fecc40b4bd0 0x7fecc40b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45912/0 (socket says 172.21.15.80:45912) 2024-04-25T13:50:22.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecc8c56640 1 -- 172.21.15.80:0/3484153695 learned_addr learned my addr 172.21.15.80:0/3484153695 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:22.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecc37fe640 1 --2- 172.21.15.80:0/3484153695 >> v2:172.21.15.87:3300/0 conn(0x7fecc4161070 0x7fecc4161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecc3fff640 1 -- 172.21.15.80:0/3484153695 >> v2:172.21.15.87:3300/0 conn(0x7fecc4161070 msgr2=0x7fecc4161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:22.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecc3fff640 1 --2- 172.21.15.80:0/3484153695 >> v2:172.21.15.87:3300/0 conn(0x7fecc4161070 0x7fecc4161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecc3fff640 1 -- 172.21.15.80:0/3484153695 >> v1:172.21.15.80:6789/0 conn(0x7fecc40b4bd0 legacy=0x7fecc40b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.089+0000 7fecc3fff640 1 -- 172.21.15.80:0/3484153695 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc40b0c60 con 0x7fecc406f5a0 2024-04-25T13:50:22.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecc3fff640 1 --2- 172.21.15.80:0/3484153695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc406f5a0 0x7fecc4160b30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fecc406ff70 tx=0x7fecb401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e485912667e62386 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecc2ffd640 1 -- 172.21.15.80:0/3484153695 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40b0070 con 0x7fecc406f5a0 2024-04-25T13:50:22.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecc2ffd640 1 -- 172.21.15.80:0/3484153695 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb40ab600 con 0x7fecc406f5a0 2024-04-25T13:50:22.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecc2ffd640 1 -- 172.21.15.80:0/3484153695 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb40ab9d0 con 0x7fecc406f5a0 2024-04-25T13:50:22.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 -- 172.21.15.80:0/3484153695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc406f5a0 msgr2=0x7fecc4160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3484153695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc406f5a0 0x7fecc4160b30 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fecc406ff70 tx=0x7fecb401c3d0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 -- 172.21.15.80:0/3484153695 shutdown_connections 2024-04-25T13:50:22.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3484153695 >> v2:172.21.15.87:3300/0 conn(0x7fecc4161070 0x7fecc4161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3484153695 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc406f5a0 0x7fecc4160b30 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 -- 172.21.15.80:0/3484153695 >> 172.21.15.80:0/3484153695 conn(0x7fecc406d080 msgr2=0x7fecc40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:22.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 -- 172.21.15.80:0/3484153695 shutdown_connections 2024-04-25T13:50:22.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 -- 172.21.15.80:0/3484153695 wait complete. 2024-04-25T13:50:22.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 Processor -- start 2024-04-25T13:50:22.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.093+0000 7fecca6b9640 1 -- start start 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecc406f5a0 0x7fecc41f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc40b4bd0 0x7fecc41576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecc4161070 0x7fecc415b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fecc4168850 con 0x7fecc40b4bd0 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fecc41686d0 con 0x7fecc4161070 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fecc41689d0 con 0x7fecc406f5a0 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc40b4bd0 0x7fecc41576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc40b4bd0 0x7fecc41576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53418/0 (socket says 172.21.15.80:53418) 2024-04-25T13:50:22.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 -- 172.21.15.80:0/3734510902 learned_addr learned my addr 172.21.15.80:0/3734510902 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc8c56640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecc4161070 0x7fecc415b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 -- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecc406f5a0 msgr2=0x7fecc41f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc3fff640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecc406f5a0 0x7fecc41f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecc406f5a0 0x7fecc41f9f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 -- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecc4161070 msgr2=0x7fecc415b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecc4161070 0x7fecc415b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 -- 172.21.15.80:0/3734510902 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc416a2c0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc8c56640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecc4161070 0x7fecc415b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc37fe640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc40b4bd0 0x7fecc41576c0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fecb800d5f0 tx=0x7fecb800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc0ff9640 1 -- 172.21.15.80:0/3734510902 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80125f0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc0ff9640 1 -- 172.21.15.80:0/3734510902 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecb8019070 con 0x7fecc40b4bd0 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc415a940 con 0x7fecc40b4bd0 2024-04-25T13:50:22.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc0ff9640 1 -- 172.21.15.80:0/3734510902 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80129b0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc41fa470 con 0x7fecc40b4bd0 2024-04-25T13:50:22.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc406a3a0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.097+0000 7fecc0ff9640 1 -- 172.21.15.80:0/3734510902 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fecb8012b50 con 0x7fecc40b4bd0 2024-04-25T13:50:22.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.101+0000 7fecc0ff9640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fecb006f430 0x7fecb0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.101+0000 7fecc0ff9640 1 -- 172.21.15.80:0/3734510902 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4943+0+0 (secure 0 0 0) 0x7fecb808f3f0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.101+0000 7fecc3fff640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fecb006f430 0x7fecb0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.101+0000 7fecc3fff640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fecb006f430 0x7fecb0071940 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fecb40ad000 tx=0x7fecb4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.105+0000 7fecc0ff9640 1 -- 172.21.15.80:0/3734510902 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fecb805c5d0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.253+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fecc40739c0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.253+0000 7fecc0ff9640 1 -- 172.21.15.80:0/3734510902 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+12959 (secure 0 0 0) 0x7fecb805bed0 con 0x7fecc40b4bd0 2024-04-25T13:50:22.250 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:22.250 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":162,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:50:20.894924+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/1134081162":"2024-04-26T13:50:17.878926+0000","172.21.15.80:6833/1134081162":"2024-04-26T13:50:17.878926+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:22.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fecb006f430 msgr2=0x7fecb0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fecb006f430 0x7fecb0071940 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fecb40ad000 tx=0x7fecb4008040 comp rx=0 tx=0).stop 2024-04-25T13:50:22.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc40b4bd0 msgr2=0x7fecc41576c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc40b4bd0 0x7fecc41576c0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fecb800d5f0 tx=0x7fecb800dab0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 shutdown_connections 2024-04-25T13:50:22.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fecb006f430 0x7fecb0071940 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fecc4161070 0x7fecc415b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fecc40b4bd0 0x7fecc41576c0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 --2- 172.21.15.80:0/3734510902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fecc406f5a0 0x7fecc41f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 >> 172.21.15.80:0/3734510902 conn(0x7fecc406d080 msgr2=0x7fecc40b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:22.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 shutdown_connections 2024-04-25T13:50:22.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.257+0000 7fecca6b9640 1 -- 172.21.15.80:0/3734510902 wait complete. 2024-04-25T13:50:22.273 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:50:22.273 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:50:22.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e6b7d640 1 Processor -- start 2024-04-25T13:50:22.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e6b7d640 1 -- start start 2024-04-25T13:50:22.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e6b7d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e0073080 0x7f07e0073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e6b7d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f07e006eb80 0x7f07e00ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e6b7d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07e00737f0 con 0x7f07e015c710 2024-04-25T13:50:22.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e6b7d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f07e00b53d0 con 0x7f07e006eb80 2024-04-25T13:50:22.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e6b7d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07e0160b20 con 0x7f07e0073080 2024-04-25T13:50:22.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e511a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f07e006eb80 0x7f07e00ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e511a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f07e006eb80 0x7f07e00ce630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53422/0 (socket says 172.21.15.80:53422) 2024-04-25T13:50:22.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e511a640 1 -- 172.21.15.80:0/3959742066 learned_addr learned my addr 172.21.15.80:0/3959742066 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:22.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e4919640 1 --2- 172.21.15.80:0/3959742066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e0073080 0x7f07e0073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e511a640 1 -- 172.21.15.80:0/3959742066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e0073080 msgr2=0x7f07e0073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:22.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e511a640 1 --2- 172.21.15.80:0/3959742066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e0073080 0x7f07e0073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e511a640 1 -- 172.21.15.80:0/3959742066 >> v1:172.21.15.87:6789/0 conn(0x7f07e015c710 legacy=0x7f07e006e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.429+0000 7f07e511a640 1 -- 172.21.15.80:0/3959742066 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e01604d0 con 0x7f07e006eb80 2024-04-25T13:50:22.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e511a640 1 --2- 172.21.15.80:0/3959742066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e006eb80 0x7f07e00ce630 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f07d800af60 tx=0x7f07d801daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41464f7b5900341b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07df7fe640 1 -- 172.21.15.80:0/3959742066 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d800a2d0 con 0x7f07e006eb80 2024-04-25T13:50:22.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07df7fe640 1 -- 172.21.15.80:0/3959742066 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d80a6810 con 0x7f07e006eb80 2024-04-25T13:50:22.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07df7fe640 1 -- 172.21.15.80:0/3959742066 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d80a6be0 con 0x7f07e006eb80 2024-04-25T13:50:22.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 -- 172.21.15.80:0/3959742066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e006eb80 msgr2=0x7f07e00ce630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/3959742066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e006eb80 0x7f07e00ce630 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f07d800af60 tx=0x7f07d801daa0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 -- 172.21.15.80:0/3959742066 shutdown_connections 2024-04-25T13:50:22.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/3959742066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e006eb80 0x7f07e00ce630 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/3959742066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e0073080 0x7f07e0073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 -- 172.21.15.80:0/3959742066 >> 172.21.15.80:0/3959742066 conn(0x7f07e00b4160 msgr2=0x7f07e00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:22.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 -- 172.21.15.80:0/3959742066 shutdown_connections 2024-04-25T13:50:22.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 -- 172.21.15.80:0/3959742066 wait complete. 2024-04-25T13:50:22.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 Processor -- start 2024-04-25T13:50:22.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.433+0000 7f07e6b7d640 1 -- start start 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e0073080 0x7f07e015ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e015c710 0x7f07e0145d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e0147100 0x7f07e0147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f07e00cecf0 con 0x7f07e0073080 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f07e00ceb70 con 0x7f07e0147100 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f07e00cee70 con 0x7f07e015c710 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e0073080 0x7f07e015ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e0073080 0x7f07e015ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53438/0 (socket says 172.21.15.80:53438) 2024-04-25T13:50:22.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 -- 172.21.15.80:0/1089065914 learned_addr learned my addr 172.21.15.80:0/1089065914 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07dffff640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e015c710 0x7f07e0145d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e511a640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e0147100 0x7f07e0147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 -- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e015c710 msgr2=0x7f07e0145d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e015c710 0x7f07e0145d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 -- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e0147100 msgr2=0x7f07e0147510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e0147100 0x7f07e0147510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 -- 172.21.15.80:0/1089065914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07e015d990 con 0x7f07e0073080 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e511a640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e0147100 0x7f07e0147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e4919640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e0073080 0x7f07e015ea30 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f07d400d600 tx=0x7f07d400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07dd7fa640 1 -- 172.21.15.80:0/1089065914 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4004240 con 0x7f07e0073080 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07dd7fa640 1 -- 172.21.15.80:0/1089065914 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07d40043e0 con 0x7f07e0073080 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07e015dc70 con 0x7f07e0073080 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07dd7fa640 1 -- 172.21.15.80:0/1089065914 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d4003840 con 0x7f07e0073080 2024-04-25T13:50:22.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07e015e230 con 0x7f07e0073080 2024-04-25T13:50:22.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07e006a3a0 con 0x7f07e0073080 2024-04-25T13:50:22.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07dd7fa640 1 -- 172.21.15.80:0/1089065914 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f07d40039e0 con 0x7f07e0073080 2024-04-25T13:50:22.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.437+0000 7f07dd7fa640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07c406f430 0x7f07c4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.441+0000 7f07dd7fa640 1 -- 172.21.15.80:0/1089065914 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4943+0+0 (secure 0 0 0) 0x7f07d408f240 con 0x7f07e0073080 2024-04-25T13:50:22.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.441+0000 7f07dffff640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07c406f430 0x7f07c4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.441+0000 7f07dffff640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07c406f430 0x7f07c4071940 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f07e00b4ee0 tx=0x7f07cc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.441 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.445+0000 7f07dd7fa640 1 -- 172.21.15.80:0/1089065914 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f07d405c4a0 con 0x7f07e0073080 2024-04-25T13:50:22.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.605+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f07e006f840 con 0x7f07e0073080 2024-04-25T13:50:22.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.605+0000 7f07dd7fa640 1 -- 172.21.15.80:0/1089065914 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) ==== 93+0+10 (secure 0 0 0) 0x7f07d405bda0 con 0x7f07e0073080 2024-04-25T13:50:22.603 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:50:22.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07c406f430 msgr2=0x7f07c4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07c406f430 0x7f07c4071940 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f07e00b4ee0 tx=0x7f07cc009040 comp rx=0 tx=0).stop 2024-04-25T13:50:22.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e0073080 msgr2=0x7f07e015ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e0073080 0x7f07e015ea30 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f07d400d600 tx=0x7f07d400dac0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 shutdown_connections 2024-04-25T13:50:22.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07c406f430 0x7f07c4071940 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07e0147100 0x7f07e0147510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.605 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f07e015c710 0x7f07e0145d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 --2- 172.21.15.80:0/1089065914 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07e0073080 0x7f07e015ea30 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 >> 172.21.15.80:0/1089065914 conn(0x7f07e00b4160 msgr2=0x7f07e00d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:22.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 shutdown_connections 2024-04-25T13:50:22.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.609+0000 7f07e6b7d640 1 -- 172.21.15.80:0/1089065914 wait complete. 2024-04-25T13:50:22.626 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-04-25T13:50:22.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff942940640 1 Processor -- start 2024-04-25T13:50:22.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff942940640 1 -- start start 2024-04-25T13:50:22.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff942940640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c0d06e0 0x7ff93c06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff942940640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff93c06e9e0 0x7ff93c06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff942940640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff93c0d0af0 con 0x7ff93c161f90 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff942940640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff93c0d0d70 con 0x7ff93c06e9e0 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff942940640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff93c0d1050 con 0x7ff93c0d06e0 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7ff93c161f90 0x7ff93c0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45924/0 (socket says 172.21.15.80:45924) 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93bfff640 1 -- 172.21.15.80:0/2903757655 learned_addr learned my addr 172.21.15.80:0/2903757655 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff933fff640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c0d06e0 0x7ff93c06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff940edd640 1 --2- 172.21.15.80:0/2903757655 >> v2:172.21.15.87:3300/0 conn(0x7ff93c06e9e0 0x7ff93c06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2871065691 0 0) 0x7ff92c001120 con 0x7ff93c161f90 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff928003530 con 0x7ff93c161f90 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1309586837 0 0) 0x7ff928003530 con 0x7ff93c161f90 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c0d06e0 msgr2=0x7ff93c06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:22.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c0d06e0 0x7ff93c06e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 >> v2:172.21.15.87:3300/0 conn(0x7ff93c06e9e0 msgr2=0x7ff93c06edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 --2- 172.21.15.80:0/2903757655 >> v2:172.21.15.87:3300/0 conn(0x7ff93c06e9e0 0x7ff93c06edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c06dca0 con 0x7ff93c161f90 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7ff92c001850 con 0x7ff93c161f90 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 >> v1:172.21.15.80:6789/0 conn(0x7ff93c161f90 legacy=0x7ff93c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9280039c0 0x7ff928003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff928004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c0d06e0 0x7ff928005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c0d0d70 con 0x7ff928004810 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c0d1050 con 0x7ff9280039c0 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff92800ba80 con 0x7ff93c0d06e0 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff933fff640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff928004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff940edd640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c0d06e0 0x7ff928005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff933fff640 1 -- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c0d06e0 msgr2=0x7ff928005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:22.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff933fff640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c0d06e0 0x7ff928005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff933fff640 1 -- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9280039c0 msgr2=0x7ff928003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:22.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff933fff640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff9280039c0 0x7ff928003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.773+0000 7ff933fff640 1 -- 172.21.15.80:0/2903757655 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff928003530 con 0x7ff928004810 2024-04-25T13:50:22.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff933fff640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff928004c20 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7ff924009ae0 tx=0x7ff92401ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572fe30fb50a32a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240b2070 con 0x7ff928004810 2024-04-25T13:50:22.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff9240a5540 con 0x7ff928004810 2024-04-25T13:50:22.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff93b7fe640 1 -- 172.21.15.80:0/2903757655 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9240096b0 con 0x7ff928004810 2024-04-25T13:50:22.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff942940640 1 -- 172.21.15.80:0/2903757655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 msgr2=0x7ff928004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff942940640 1 --2- 172.21.15.80:0/2903757655 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff928004c20 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7ff924009ae0 tx=0x7ff92401ca60 comp rx=0 tx=0).stop 2024-04-25T13:50:22.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff940edd640 1 -- 172.21.15.80:0/2903757655 reap_dead start 2024-04-25T13:50:22.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff942940640 1 -- 172.21.15.80:0/2903757655 shutdown_connections 2024-04-25T13:50:22.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff942940640 1 -- 172.21.15.80:0/2903757655 >> 172.21.15.80:0/2903757655 conn(0x7ff93c06d080 msgr2=0x7ff93c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:22.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff942940640 1 -- 172.21.15.80:0/2903757655 shutdown_connections 2024-04-25T13:50:22.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff942940640 1 -- 172.21.15.80:0/2903757655 wait complete. 2024-04-25T13:50:22.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.777+0000 7ff942940640 1 Processor -- start 2024-04-25T13:50:22.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 -- start start 2024-04-25T13:50:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff93c1415f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c1575f0 0x7ff93c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff93c157a00 0x7ff93c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c1447a0 con 0x7ff928004810 2024-04-25T13:50:22.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff93c144620 con 0x7ff93c157a00 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff93c169500 con 0x7ff93c1575f0 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff93c1415f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff93c1415f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53458/0 (socket says 172.21.15.80:53458) 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 -- 172.21.15.80:0/2562660715 learned_addr learned my addr 172.21.15.80:0/2562660715 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff940edd640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff93c157a00 0x7ff93c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff933fff640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c1575f0 0x7ff93c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 -- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c1575f0 msgr2=0x7ff93c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c1575f0 0x7ff93c0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 -- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff93c157a00 msgr2=0x7ff93c0afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff93c157a00 0x7ff93c0afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 -- 172.21.15.80:0/2562660715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff93c158160 con 0x7ff928004810 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff940edd640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff93c157a00 0x7ff93c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff93bfff640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff93c1415f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c7d0 tx=0x7ff92c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff9397fa640 1 -- 172.21.15.80:0/2562660715 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c018070 con 0x7ff928004810 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff9397fa640 1 -- 172.21.15.80:0/2562660715 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff92c0072d0 con 0x7ff928004810 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff9397fa640 1 -- 172.21.15.80:0/2562660715 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff92c0047f0 con 0x7ff928004810 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff93c158440 con 0x7ff928004810 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff93c1592a0 con 0x7ff928004810 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff93c06a3a0 con 0x7ff928004810 2024-04-25T13:50:22.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.781+0000 7ff9397fa640 1 -- 172.21.15.80:0/2562660715 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff92c01b070 con 0x7ff928004810 2024-04-25T13:50:22.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.785+0000 7ff9397fa640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91c06f430 0x7ff91c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:22.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.785+0000 7ff9397fa640 1 -- 172.21.15.80:0/2562660715 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4943+0+0 (secure 0 0 0) 0x7ff92c08e3e0 con 0x7ff928004810 2024-04-25T13:50:22.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.785+0000 7ff933fff640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91c06f430 0x7ff91c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:22.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.785+0000 7ff933fff640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91c06f430 0x7ff91c071940 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff924009ae0 tx=0x7ff9240046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:22.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.789+0000 7ff9397fa640 1 -- 172.21.15.80:0/2562660715 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff93c0d0af0 con 0x7ff928004810 2024-04-25T13:50:22.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:22.993+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) -- 0x7ff93c15a160 con 0x7ff928004810 2024-04-25T13:50:23.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.917+0000 7ff9397fa640 1 -- 172.21.15.80:0/2562660715 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v490) ==== 134+0+0 (secure 0 0 0) 0x7ff92c05b660 con 0x7ff928004810 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91c06f430 msgr2=0x7ff91c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91c06f430 0x7ff91c071940 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7ff924009ae0 tx=0x7ff9240046d0 comp rx=0 tx=0).stop 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 msgr2=0x7ff93c1415f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff93c1415f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7ff92c00c7d0 tx=0x7ff92c00cc90 comp rx=0 tx=0).stop 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 shutdown_connections 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff91c06f430 0x7ff91c071940 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff93c157a00 0x7ff93c0afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff93c1575f0 0x7ff93c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:23.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 --2- 172.21.15.80:0/2562660715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff928004810 0x7ff93c1415f0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:23.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 >> 172.21.15.80:0/2562660715 conn(0x7ff93c06d080 msgr2=0x7ff93c1595b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:23.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 shutdown_connections 2024-04-25T13:50:23.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:23.921+0000 7ff942940640 1 -- 172.21.15.80:0/2562660715 wait complete. 2024-04-25T13:50:23.944 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:50:24.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8be5681640 1 Processor -- start 2024-04-25T13:50:24.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8be5681640 1 -- start start 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8be5681640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8be00b4f10 0x7f8be00b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8be5681640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b6f00 0x7f8be00b7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8be5681640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8be00b7850 0x7f8be00ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8be5681640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8be00b7c60 con 0x7f8be00b4f10 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8be5681640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8be00d0390 con 0x7f8be00b7850 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8be5681640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00d0670 con 0x7f8be00b6f00 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8bdeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8be00b4f10 0x7f8be00b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8bdeffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8be00b4f10 0x7f8be00b69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53468/0 (socket says 172.21.15.80:53468) 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8bdeffd640 1 -- 172.21.15.80:0/3361236631 learned_addr learned my addr 172.21.15.80:0/3361236631 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:24.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.101+0000 7f8bde7fc640 1 --2- 172.21.15.80:0/3361236631 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b6f00 0x7f8be00b7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bdf7fe640 1 --2- 172.21.15.80:0/3361236631 >> v2:172.21.15.87:3300/0 conn(0x7f8be00b7850 0x7f8be00ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bdeffd640 1 -- 172.21.15.80:0/3361236631 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b6f00 msgr2=0x7f8be00b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:24.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bdeffd640 1 --2- 172.21.15.80:0/3361236631 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b6f00 0x7f8be00b7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bdeffd640 1 -- 172.21.15.80:0/3361236631 >> v2:172.21.15.87:3300/0 conn(0x7f8be00b7850 msgr2=0x7f8be00ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:24.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bdeffd640 1 --2- 172.21.15.80:0/3361236631 >> v2:172.21.15.87:3300/0 conn(0x7f8be00b7850 0x7f8be00ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bdeffd640 1 -- 172.21.15.80:0/3361236631 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be00d0950 con 0x7f8be00b4f10 2024-04-25T13:50:24.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bdf7fe640 1 --2- 172.21.15.80:0/3361236631 >> v2:172.21.15.87:3300/0 conn(0x7f8be00b7850 0x7f8be00ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:24.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bdeffd640 1 --2- 172.21.15.80:0/3361236631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b4f10 0x7f8be00b69c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8be0074000 tx=0x7f8bd001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b3bc63295ccf35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:24.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bddffb640 1 -- 172.21.15.80:0/3361236631 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00aa610 con 0x7f8be00b4f10 2024-04-25T13:50:24.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bddffb640 1 -- 172.21.15.80:0/3361236631 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd00af070 con 0x7f8be00b4f10 2024-04-25T13:50:24.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8bddffb640 1 -- 172.21.15.80:0/3361236631 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00aa9d0 con 0x7f8be00b4f10 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 -- 172.21.15.80:0/3361236631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b4f10 msgr2=0x7f8be00b69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 --2- 172.21.15.80:0/3361236631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b4f10 0x7f8be00b69c0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8be0074000 tx=0x7f8bd001c4f0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 -- 172.21.15.80:0/3361236631 shutdown_connections 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 --2- 172.21.15.80:0/3361236631 >> v2:172.21.15.87:3300/0 conn(0x7f8be00b7850 0x7f8be00ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 --2- 172.21.15.80:0/3361236631 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b6f00 0x7f8be00b7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 --2- 172.21.15.80:0/3361236631 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b4f10 0x7f8be00b69c0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 -- 172.21.15.80:0/3361236631 >> 172.21.15.80:0/3361236631 conn(0x7f8be00d3a90 msgr2=0x7f8be00d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 -- 172.21.15.80:0/3361236631 shutdown_connections 2024-04-25T13:50:24.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.105+0000 7f8be5681640 1 -- 172.21.15.80:0/3361236631 wait complete. 2024-04-25T13:50:24.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 Processor -- start 2024-04-25T13:50:24.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 -- start start 2024-04-25T13:50:24.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b4f10 0x7f8be015fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b6f00 0x7f8be006c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b7850 0x7f8be00cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00cf6d0 con 0x7f8be00b6f00 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8be00cf550 con 0x7f8be00b7850 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8be00cf850 con 0x7f8be00b4f10 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b6f00 0x7f8be006c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b6f00 0x7f8be006c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53484/0 (socket says 172.21.15.80:53484) 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 -- 172.21.15.80:0/2125694010 learned_addr learned my addr 172.21.15.80:0/2125694010 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bdeffd640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b4f10 0x7f8be015fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bdf7fe640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b7850 0x7f8be00cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 -- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b4f10 msgr2=0x7f8be015fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b4f10 0x7f8be015fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 -- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b7850 msgr2=0x7f8be00cc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b7850 0x7f8be00cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 -- 172.21.15.80:0/2125694010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8be006d4e0 con 0x7f8be00b6f00 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bdeffd640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b4f10 0x7f8be015fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bde7fc640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b6f00 0x7f8be006c780 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400d650 tx=0x7f8bd400db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bc77fe640 1 -- 172.21.15.80:0/2125694010 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400b6c0 con 0x7f8be00b6f00 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bc77fe640 1 -- 172.21.15.80:0/2125694010 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8bd4019070 con 0x7f8be00b6f00 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8be006d7c0 con 0x7f8be00b6f00 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8bc77fe640 1 -- 172.21.15.80:0/2125694010 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd400ba80 con 0x7f8be00b6f00 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.109+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8be006dd80 con 0x7f8be00b6f00 2024-04-25T13:50:24.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.113+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8be006d250 con 0x7f8be00b6f00 2024-04-25T13:50:24.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.113+0000 7f8bc77fe640 1 -- 172.21.15.80:0/2125694010 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8bd400bc20 con 0x7f8be00b6f00 2024-04-25T13:50:24.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.113+0000 7f8bc77fe640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bcc06f430 0x7f8bcc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.113+0000 7f8bc77fe640 1 -- 172.21.15.80:0/2125694010 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4943+0+0 (secure 0 0 0) 0x7f8bd408f420 con 0x7f8be00b6f00 2024-04-25T13:50:24.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.113+0000 7f8bdeffd640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bcc06f430 0x7f8bcc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.113+0000 7f8bdeffd640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bcc06f430 0x7f8bcc071940 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00ae000 tx=0x7f8bd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:24.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.117+0000 7f8bc77fe640 1 -- 172.21.15.80:0/2125694010 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8bd405c570 con 0x7f8be00b6f00 2024-04-25T13:50:24.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.269+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8be006a3a0 con 0x7f8be00b6f00 2024-04-25T13:50:24.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.273+0000 7f8bc77fe640 1 -- 172.21.15.80:0/2125694010 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f8bd405be70 con 0x7f8be00b6f00 2024-04-25T13:50:24.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bcc06f430 msgr2=0x7f8bcc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bcc06f430 0x7f8bcc071940 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8bd00ae000 tx=0x7f8bd0008040 comp rx=0 tx=0).stop 2024-04-25T13:50:24.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b6f00 msgr2=0x7f8be006c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b6f00 0x7f8be006c780 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400d650 tx=0x7f8bd400db10 comp rx=0 tx=0).stop 2024-04-25T13:50:24.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 shutdown_connections 2024-04-25T13:50:24.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8bcc06f430 0x7f8bcc071940 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8be00b7850 0x7f8be00cc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8be00b6f00 0x7f8be006c780 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 --2- 172.21.15.80:0/2125694010 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8be00b4f10 0x7f8be015fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 >> 172.21.15.80:0/2125694010 conn(0x7f8be00d3a90 msgr2=0x7f8be00b7de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:24.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 shutdown_connections 2024-04-25T13:50:24.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.277+0000 7f8be5681640 1 -- 172.21.15.80:0/2125694010 wait complete. 2024-04-25T13:50:24.293 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:50:24.293 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:50:24.338 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:50:24.388 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:50:24.388 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:50:24.388 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:24.394 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:50:24.394 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:50:24.394 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:24.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:50:24.400 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:50:24.400 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:24.441 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:50:24.441 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:50:24.442 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:50:24.447 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:50:24.448 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:50:24.448 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:24.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.636+0000 7fb4894b0640 1 Processor -- start 2024-04-25T13:50:24.636 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.636+0000 7fb4894b0640 1 -- start start 2024-04-25T13:50:24.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4894b0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb484161f90 0x7fb4840cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4894b0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb4840d0510 0x7fb48415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4894b0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 0x7fb48415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4894b0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb4840d0d70 con 0x7fb4840d0510 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4894b0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb4840d1050 con 0x7fb484161f90 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4894b0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb48406dca0 con 0x7fb48415ea70 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4827fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb4840d0510 0x7fb48415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4827fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb4840d0510 0x7fb48415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53498/0 (socket says 172.21.15.80:53498) 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4827fc640 1 -- 172.21.15.80:0/1943856094 learned_addr learned my addr 172.21.15.80:0/1943856094 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb482ffd640 1 --2- 172.21.15.80:0/1943856094 >> v2:172.21.15.87:3300/0 conn(0x7fb484161f90 0x7fb4840cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4837fe640 1 --2- 172.21.15.80:0/1943856094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 0x7fb48415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4827fc640 1 -- 172.21.15.80:0/1943856094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 msgr2=0x7fb48415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4827fc640 1 --2- 172.21.15.80:0/1943856094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 0x7fb48415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4827fc640 1 -- 172.21.15.80:0/1943856094 >> v2:172.21.15.87:3300/0 conn(0x7fb484161f90 msgr2=0x7fb4840cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4827fc640 1 --2- 172.21.15.80:0/1943856094 >> v2:172.21.15.87:3300/0 conn(0x7fb484161f90 0x7fb4840cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.640+0000 7fb4827fc640 1 -- 172.21.15.80:0/1943856094 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48406df80 con 0x7fb4840d0510 2024-04-25T13:50:24.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4827fc640 1 --2- 172.21.15.80:0/1943856094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4840d0510 0x7fb48415e530 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb4840d17b0 tx=0x7fb47801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=903b21fcf37ea99f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:24.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb481ffb640 1 -- 172.21.15.80:0/1943856094 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4780b1070 con 0x7fb4840d0510 2024-04-25T13:50:24.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb481ffb640 1 -- 172.21.15.80:0/1943856094 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4780a8050 con 0x7fb4840d0510 2024-04-25T13:50:24.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb481ffb640 1 -- 172.21.15.80:0/1943856094 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4780ac860 con 0x7fb4840d0510 2024-04-25T13:50:24.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 -- 172.21.15.80:0/1943856094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4840d0510 msgr2=0x7fb48415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 --2- 172.21.15.80:0/1943856094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4840d0510 0x7fb48415e530 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb4840d17b0 tx=0x7fb47801c460 comp rx=0 tx=0).stop 2024-04-25T13:50:24.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 -- 172.21.15.80:0/1943856094 shutdown_connections 2024-04-25T13:50:24.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 --2- 172.21.15.80:0/1943856094 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 0x7fb48415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 --2- 172.21.15.80:0/1943856094 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4840d0510 0x7fb48415e530 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 --2- 172.21.15.80:0/1943856094 >> v2:172.21.15.87:3300/0 conn(0x7fb484161f90 0x7fb4840cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 -- 172.21.15.80:0/1943856094 >> 172.21.15.80:0/1943856094 conn(0x7fb48406d080 msgr2=0x7fb4840b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:24.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 -- 172.21.15.80:0/1943856094 shutdown_connections 2024-04-25T13:50:24.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 -- 172.21.15.80:0/1943856094 wait complete. 2024-04-25T13:50:24.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 Processor -- start 2024-04-25T13:50:24.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.644+0000 7fb4894b0640 1 -- start start 2024-04-25T13:50:24.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4840d0510 0x7fb484163260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 0x7fb48415cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb484161f90 0x7fb48413ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb4841447a0 con 0x7fb484161f90 2024-04-25T13:50:24.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb484144620 con 0x7fb4840d0510 2024-04-25T13:50:24.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb4840d0920 con 0x7fb48415ea70 2024-04-25T13:50:24.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb484161f90 0x7fb48413ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb484161f90 0x7fb48413ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53514/0 (socket says 172.21.15.80:53514) 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 -- 172.21.15.80:0/970755972 learned_addr learned my addr 172.21.15.80:0/970755972 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb482ffd640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4840d0510 0x7fb484163260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4827fc640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 0x7fb48415cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 -- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 msgr2=0x7fb48415cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 0x7fb48415cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 -- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4840d0510 msgr2=0x7fb484163260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4840d0510 0x7fb484163260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 -- 172.21.15.80:0/970755972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48413e6b0 con 0x7fb484161f90 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4837fe640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb484161f90 0x7fb48413ee40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb47c007d40 tx=0x7fb47c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb46b7fe640 1 -- 172.21.15.80:0/970755972 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c014420 con 0x7fb484161f90 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb46b7fe640 1 -- 172.21.15.80:0/970755972 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb47c019070 con 0x7fb484161f90 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4841637a0 con 0x7fb484161f90 2024-04-25T13:50:24.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb46b7fe640 1 -- 172.21.15.80:0/970755972 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c014800 con 0x7fb484161f90 2024-04-25T13:50:24.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48413d0e0 con 0x7fb484161f90 2024-04-25T13:50:24.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb48406a3a0 con 0x7fb484161f90 2024-04-25T13:50:24.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.648+0000 7fb46b7fe640 1 -- 172.21.15.80:0/970755972 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb47c0149a0 con 0x7fb484161f90 2024-04-25T13:50:24.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.652+0000 7fb46b7fe640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb47006f400 0x7fb470071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:24.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.652+0000 7fb46b7fe640 1 -- 172.21.15.80:0/970755972 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4943+0+0 (secure 0 0 0) 0x7fb47c08f6d0 con 0x7fb484161f90 2024-04-25T13:50:24.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.652+0000 7fb482ffd640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb47006f400 0x7fb470071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:24.650 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.652+0000 7fb482ffd640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb47006f400 0x7fb470071910 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb48406fa20 tx=0x7fb474008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:24.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.656+0000 7fb46b7fe640 1 -- 172.21.15.80:0/970755972 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb47c05c8c0 con 0x7fb484161f90 2024-04-25T13:50:24.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.852+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb48415f9e0 con 0x7fb484161f90 2024-04-25T13:50:24.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb46b7fe640 1 -- 172.21.15.80:0/970755972 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+3863 (secure 0 0 0) 0x7fb47c05c1c0 con 0x7fb484161f90 2024-04-25T13:50:24.853 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:24.853 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6107,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/521602421","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":521602421},{"type":"v1","addr":"172.21.15.87:6835","nonce":521602421}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":6688,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4170043750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4170043750},{"type":"v1","addr":"172.21.15.87:6837","nonce":4170043750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8409,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6834/1423926990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1423926990},{"type":"v1","addr":"172.21.15.80:6834","nonce":1423926990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":70},{"gid":8718,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/611653558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6833","nonce":611653558},{"type":"v1","addr":"172.21.15.80:6835","nonce":611653558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-04-25T13:50:24.854 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 76 2024-04-25T13:50:24.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb47006f400 msgr2=0x7fb470071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb47006f400 0x7fb470071910 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb48406fa20 tx=0x7fb474008040 comp rx=0 tx=0).stop 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb484161f90 msgr2=0x7fb48413ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb484161f90 0x7fb48413ee40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fb47c007d40 tx=0x7fb47c00e4b0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 shutdown_connections 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb47006f400 0x7fb470071910 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb484161f90 0x7fb48413ee40 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb48415ea70 0x7fb48415cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 --2- 172.21.15.80:0/970755972 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4840d0510 0x7fb484163260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 >> 172.21.15.80:0/970755972 conn(0x7fb48406d080 msgr2=0x7fb484141180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 shutdown_connections 2024-04-25T13:50:24.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:24.856+0000 7fb4894b0640 1 -- 172.21.15.80:0/970755972 wait complete. 2024-04-25T13:50:24.876 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:50:24.876 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:50:24.877 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:50:24.877 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:50:24.877 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:50:24.877 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:50:24.877 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:50:24.877 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:50:24.878 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:50:24.878 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:50:24.878 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:50:24.878 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:50:24.878 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:50:24.878 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:50:24.878 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:50:24.878 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:50:24.891 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:50:24.891 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:50:24.892 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:50:24.892 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:50:24.895 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:50:24.895 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:50:24.899 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:50:24.899 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:50:24.938 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:50:24.938 DEBUG:teuthology.parallel:result is None 2024-04-25T13:50:24.939 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:50:24.939 DEBUG:teuthology.parallel:result is None 2024-04-25T13:50:24.943 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:50:24.943 DEBUG:teuthology.parallel:result is None 2024-04-25T13:50:24.944 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:50:24.944 DEBUG:teuthology.parallel:result is None 2024-04-25T13:50:24.944 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:50:25.029 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:50:25.030 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:50:25.042 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:50:25.042 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:50:25.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.088+0000 7f57d44e9640 1 Processor -- start 2024-04-25T13:50:25.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.088+0000 7f57d44e9640 1 -- start start 2024-04-25T13:50:25.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.088+0000 7f57d44e9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57cc0d59b0 0x7f57cc0d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:25.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.088+0000 7f57d44e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d4330 0x7f57cc0d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:25.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57cc0d6300 con 0x7f57cc0d2e80 2024-04-25T13:50:25.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f57cc161060 con 0x7f57cc0d59b0 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc161340 con 0x7f57cc0d4330 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1a84640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57cc0d59b0 0x7f57cc0d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1a84640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f57cc0d59b0 0x7f57cc0d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:55870/0 (socket says 172.21.15.80:55870) 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1a84640 1 -- 172.21.15.80:0/1407486812 learned_addr learned my addr 172.21.15.80:0/1407486812 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d2a86640 1 --2- 172.21.15.80:0/1407486812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d4330 0x7f57cc0d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1a84640 1 -- 172.21.15.80:0/1407486812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d4330 msgr2=0x7f57cc0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1a84640 1 --2- 172.21.15.80:0/1407486812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d4330 0x7f57cc0d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1a84640 1 -- 172.21.15.80:0/1407486812 >> v1:172.21.15.87:6789/0 conn(0x7f57cc0d2e80 legacy=0x7f57cc1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:25.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1a84640 1 -- 172.21.15.80:0/1407486812 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc0d2680 con 0x7f57cc0d59b0 2024-04-25T13:50:25.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1a84640 1 --2- 172.21.15.80:0/1407486812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d59b0 0x7f57cc0d5dc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009870 tx=0x7f57bc01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c56727f65947ca15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:25.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1283640 1 -- 172.21.15.80:0/1407486812 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0b1070 con 0x7f57cc0d59b0 2024-04-25T13:50:25.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1283640 1 -- 172.21.15.80:0/1407486812 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57bc0a8050 con 0x7f57cc0d59b0 2024-04-25T13:50:25.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d1283640 1 -- 172.21.15.80:0/1407486812 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc0ac860 con 0x7f57cc0d59b0 2024-04-25T13:50:25.092 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 -- 172.21.15.80:0/1407486812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d59b0 msgr2=0x7f57cc0d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:25.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 --2- 172.21.15.80:0/1407486812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d59b0 0x7f57cc0d5dc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009870 tx=0x7f57bc01c4d0 comp rx=0 tx=0).stop 2024-04-25T13:50:25.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 -- 172.21.15.80:0/1407486812 shutdown_connections 2024-04-25T13:50:25.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 --2- 172.21.15.80:0/1407486812 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d4330 0x7f57cc0d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:25.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 --2- 172.21.15.80:0/1407486812 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d59b0 0x7f57cc0d5dc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:25.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 -- 172.21.15.80:0/1407486812 >> 172.21.15.80:0/1407486812 conn(0x7f57cc0737a0 msgr2=0x7f57cc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:25.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 -- 172.21.15.80:0/1407486812 shutdown_connections 2024-04-25T13:50:25.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.092+0000 7f57d44e9640 1 -- 172.21.15.80:0/1407486812 wait complete. 2024-04-25T13:50:25.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 Processor -- start 2024-04-25T13:50:25.094 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 -- start start 2024-04-25T13:50:25.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57cc0d2e80 0x7f57cc14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d4330 0x7f57cc0baf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d59b0 0x7f57cc0c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0d3410 con 0x7f57cc0d2e80 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f57cc0d3290 con 0x7f57cc0d4330 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f57cc15df60 con 0x7f57cc0d59b0 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57cc0d2e80 0x7f57cc14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57cc0d2e80 0x7f57cc14d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53550/0 (socket says 172.21.15.80:53550) 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 -- 172.21.15.80:0/3720387280 learned_addr learned my addr 172.21.15.80:0/3720387280 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d1a84640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d4330 0x7f57cc0baf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2a86640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d59b0 0x7f57cc0c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 -- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d59b0 msgr2=0x7f57cc0c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d59b0 0x7f57cc0c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 -- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d4330 msgr2=0x7f57cc0baf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:25.097 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d4330 0x7f57cc0baf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:25.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 -- 172.21.15.80:0/3720387280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57cc14e650 con 0x7f57cc0d2e80 2024-04-25T13:50:25.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2a86640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d59b0 0x7f57cc0c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:25.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d2285640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57cc0d2e80 0x7f57cc14d2f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c760 tx=0x7f57c800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:25.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57c2ffd640 1 -- 172.21.15.80:0/3720387280 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8003b80 con 0x7f57cc0d2e80 2024-04-25T13:50:25.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57c2ffd640 1 -- 172.21.15.80:0/3720387280 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57c8003d20 con 0x7f57cc0d2e80 2024-04-25T13:50:25.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57cc14ddb0 con 0x7f57cc0d2e80 2024-04-25T13:50:25.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57c2ffd640 1 -- 172.21.15.80:0/3720387280 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c800aa90 con 0x7f57cc0d2e80 2024-04-25T13:50:25.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.096+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57cc14e370 con 0x7f57cc0d2e80 2024-04-25T13:50:25.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.100+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57cc06a3a0 con 0x7f57cc0d2e80 2024-04-25T13:50:25.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.100+0000 7f57c2ffd640 1 -- 172.21.15.80:0/3720387280 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f57c800ac30 con 0x7f57cc0d2e80 2024-04-25T13:50:25.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.100+0000 7f57c2ffd640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57b006f430 0x7f57b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:25.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.100+0000 7f57c2ffd640 1 -- 172.21.15.80:0/3720387280 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4943+0+0 (secure 0 0 0) 0x7f57c808ed00 con 0x7f57cc0d2e80 2024-04-25T13:50:25.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.100+0000 7f57d1a84640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57b006f430 0x7f57b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:25.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.100+0000 7f57d1a84640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57b006f430 0x7f57b0071940 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0ae000 tx=0x7f57bc0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:25.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.104+0000 7f57c2ffd640 1 -- 172.21.15.80:0/3720387280 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f57cc14e650 con 0x7f57cc0d2e80 2024-04-25T13:50:25.260 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:25.260+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f57cc16af00 con 0x7f57cc0d2e80 2024-04-25T13:50:26.301 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.300+0000 7f57c2ffd640 1 -- 172.21.15.80:0/3720387280 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v167) ==== 97+0+0 (secure 0 0 0) 0x7f57cc0d6300 con 0x7f57cc0d2e80 2024-04-25T13:50:26.301 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57b006f430 msgr2=0x7f57b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57b006f430 0x7f57b0071940 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0ae000 tx=0x7f57bc0a7000 comp rx=0 tx=0).stop 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57cc0d2e80 msgr2=0x7f57cc14d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57cc0d2e80 0x7f57cc14d2f0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c760 tx=0x7f57c800cc20 comp rx=0 tx=0).stop 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 shutdown_connections 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f57b006f430 0x7f57b0071940 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f57cc0d59b0 0x7f57cc0c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f57cc0d4330 0x7f57cc0baf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 --2- 172.21.15.80:0/3720387280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f57cc0d2e80 0x7f57cc14d2f0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 >> 172.21.15.80:0/3720387280 conn(0x7f57cc0737a0 msgr2=0x7f57cc15f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 shutdown_connections 2024-04-25T13:50:26.305 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.304+0000 7f57d44e9640 1 -- 172.21.15.80:0/3720387280 wait complete. 2024-04-25T13:50:26.327 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:50:26.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f20033bc640 1 Processor -- start 2024-04-25T13:50:26.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f20033bc640 1 -- start start 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f20033bc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f20033bc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1ffc0d2930 0x7f1ffc0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f20033bc640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ffc0d0d30 con 0x7f1ffc0d0920 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f20033bc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1ffc0ce350 con 0x7f1ffc0d2930 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f20033bc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1ffc0ce630 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f2000957640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1ffc0d2930 0x7f1ffc0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f2000957640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1ffc0d2930 0x7f1ffc0d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53560/0 (socket says 172.21.15.80:53560) 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f2000957640 1 -- 172.21.15.80:0/1856585802 learned_addr learned my addr 172.21.15.80:0/1856585802 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.480+0000 7f2001959640 1 --1- 172.21.15.80:0/1856585802 >> v1:172.21.15.80:6789/0 conn(0x7f1ffc0d0920 0x7f1ffc0cf110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45950/0 (socket says 172.21.15.80:45950) 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f2001158640 1 --2- 172.21.15.80:0/1856585802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f2000957640 1 -- 172.21.15.80:0/1856585802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0b69b0 msgr2=0x7f1ffc0b5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f2000957640 1 --2- 172.21.15.80:0/1856585802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f2000957640 1 -- 172.21.15.80:0/1856585802 >> v1:172.21.15.80:6789/0 conn(0x7f1ffc0d0920 legacy=0x7f1ffc0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f2000957640 1 -- 172.21.15.80:0/1856585802 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc0ce910 con 0x7f1ffc0d2930 2024-04-25T13:50:26.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f2000957640 1 --2- 172.21.15.80:0/1856585802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0d2930 0x7f1ffc0d03e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009870 tx=0x7f1fec01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3adb44553787216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f1ff3fff640 1 -- 172.21.15.80:0/1856585802 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec0b0070 con 0x7f1ffc0d2930 2024-04-25T13:50:26.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f1ff3fff640 1 -- 172.21.15.80:0/1856585802 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fec004230 con 0x7f1ffc0d2930 2024-04-25T13:50:26.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.484+0000 7f1ff3fff640 1 -- 172.21.15.80:0/1856585802 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fec003620 con 0x7f1ffc0d2930 2024-04-25T13:50:26.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- 172.21.15.80:0/1856585802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0d2930 msgr2=0x7f1ffc0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 --2- 172.21.15.80:0/1856585802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0d2930 0x7f1ffc0d03e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009870 tx=0x7f1fec01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- 172.21.15.80:0/1856585802 shutdown_connections 2024-04-25T13:50:26.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 --2- 172.21.15.80:0/1856585802 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0d2930 0x7f1ffc0d03e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 --2- 172.21.15.80:0/1856585802 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- 172.21.15.80:0/1856585802 >> 172.21.15.80:0/1856585802 conn(0x7f1ffc0d3b60 msgr2=0x7f1ffc0d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- 172.21.15.80:0/1856585802 shutdown_connections 2024-04-25T13:50:26.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- 172.21.15.80:0/1856585802 wait complete. 2024-04-25T13:50:26.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 Processor -- start 2024-04-25T13:50:26.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- start start 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0d0920 0x7f1ffc15acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ffc0d2930 0x7f1ffc0b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc06dd20 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1ffc06dba0 con 0x7f1ffc0d2930 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f20033bc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1ffc0cf820 con 0x7f1ffc0d0920 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f2001158640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f2001158640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53568/0 (socket says 172.21.15.80:53568) 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.488+0000 7f2001158640 1 -- 172.21.15.80:0/3370050882 learned_addr learned my addr 172.21.15.80:0/3370050882 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:26.490 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2000957640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0d0920 0x7f1ffc15acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2001959640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ffc0d2930 0x7f1ffc0b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2001158640 1 -- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0d0920 msgr2=0x7f1ffc15acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2001158640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0d0920 0x7f1ffc15acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2001158640 1 -- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ffc0d2930 msgr2=0x7f1ffc0b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2001158640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ffc0d2930 0x7f1ffc0b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2001158640 1 -- 172.21.15.80:0/3370050882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ffc0b07f0 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2001158640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b8900 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800d5a0 tx=0x7f1ff800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f1ff1ffb640 1 -- 172.21.15.80:0/3370050882 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8004020 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f1ff1ffb640 1 -- 172.21.15.80:0/3370050882 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ff80041e0 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ffc0b0cf0 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f1ff1ffb640 1 -- 172.21.15.80:0/3370050882 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff80043d0 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.491 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ffc0b12b0 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ffc06a3a0 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f1ff1ffb640 1 -- 172.21.15.80:0/3370050882 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1ff801b020 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f1ff1ffb640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fe006f400 0x7f1fe0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f1ff1ffb640 1 -- 172.21.15.80:0/3370050882 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 4857+0+0 (secure 0 0 0) 0x7f1ff808f0d0 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2000957640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fe006f400 0x7f1fe0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.492+0000 7f2000957640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fe006f400 0x7f1fe0071910 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0ad000 tx=0x7f1fec0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.496+0000 7f1ff1ffb640 1 -- 172.21.15.80:0/3370050882 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1ff8022020 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.720+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1ffc0d0580 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.720+0000 7f1ff1ffb640 1 -- 172.21.15.80:0/3370050882 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f1ff805c3f0 con 0x7f1ffc0b69b0 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:50:26.719 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:50:26.721 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:50:26.722 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.723 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:50:26.724 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.725 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:50:26.726 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.727 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:50:26.728 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:50:26.729 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.730 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:50:26.731 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:50:26.732 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.733 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:26.733 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:50:26.733 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:50:26.733 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fe006f400 msgr2=0x7f1fe0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fe006f400 0x7f1fe0071910 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1fec0ad000 tx=0x7f1fec0a6000 comp rx=0 tx=0).stop 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0b69b0 msgr2=0x7f1ffc0b8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b8900 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800d5a0 tx=0x7f1ff800da60 comp rx=0 tx=0).stop 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 shutdown_connections 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1fe006f400 0x7f1fe0071910 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1ffc0d2930 0x7f1ffc0b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1ffc0d0920 0x7f1ffc15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 --2- 172.21.15.80:0/3370050882 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1ffc0b69b0 0x7f1ffc0b8900 secure :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800d5a0 tx=0x7f1ff800da60 comp rx=0 tx=0).stop 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 >> 172.21.15.80:0/3370050882 conn(0x7f1ffc0d3b60 msgr2=0x7f1ffc0700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 shutdown_connections 2024-04-25T13:50:26.734 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.724+0000 7f20033bc640 1 -- 172.21.15.80:0/3370050882 wait complete. 2024-04-25T13:50:26.742 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:26.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb8966b3640 1 Processor -- start 2024-04-25T13:50:26.890 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb8966b3640 1 -- start start 2024-04-25T13:50:26.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb8966b3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 0x7fb890068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb8966b3640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb890068ad0 0x7fb890160d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb8966b3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb890161280 0x7fb890163e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb8966b3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb89006f3e0 con 0x7fb890161280 2024-04-25T13:50:26.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb8966b3640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb89006d130 con 0x7fb890068ad0 2024-04-25T13:50:26.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb8966b3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb890068ee0 con 0x7fb89006efd0 2024-04-25T13:50:26.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb894c50640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb890161280 0x7fb890163e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb894c50640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb890161280 0x7fb890163e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53584/0 (socket says 172.21.15.80:53584) 2024-04-25T13:50:26.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb894c50640 1 -- 172.21.15.80:0/335362613 learned_addr learned my addr 172.21.15.80:0/335362613 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:26.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb88ffff640 1 --2- 172.21.15.80:0/335362613 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 0x7fb890068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb88f7fe640 1 --2- 172.21.15.80:0/335362613 >> v2:172.21.15.87:3300/0 conn(0x7fb890068ad0 0x7fb890160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb894c50640 1 -- 172.21.15.80:0/335362613 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 msgr2=0x7fb890068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb894c50640 1 --2- 172.21.15.80:0/335362613 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 0x7fb890068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb894c50640 1 -- 172.21.15.80:0/335362613 >> v2:172.21.15.87:3300/0 conn(0x7fb890068ad0 msgr2=0x7fb890160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:26.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb894c50640 1 --2- 172.21.15.80:0/335362613 >> v2:172.21.15.87:3300/0 conn(0x7fb890068ad0 0x7fb890160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.892+0000 7fb894c50640 1 -- 172.21.15.80:0/335362613 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb89006c800 con 0x7fb890161280 2024-04-25T13:50:26.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb894c50640 1 --2- 172.21.15.80:0/335362613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890161280 0x7fb890163e30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb8900741a0 tx=0x7fb88801c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c218f5cae111aed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb88effd640 1 -- 172.21.15.80:0/335362613 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880a64d0 con 0x7fb890161280 2024-04-25T13:50:26.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb88effd640 1 -- 172.21.15.80:0/335362613 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8880a6670 con 0x7fb890161280 2024-04-25T13:50:26.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb88effd640 1 -- 172.21.15.80:0/335362613 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8880a6a40 con 0x7fb890161280 2024-04-25T13:50:26.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 -- 172.21.15.80:0/335362613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890161280 msgr2=0x7fb890163e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 --2- 172.21.15.80:0/335362613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890161280 0x7fb890163e30 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fb8900741a0 tx=0x7fb88801c730 comp rx=0 tx=0).stop 2024-04-25T13:50:26.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 -- 172.21.15.80:0/335362613 shutdown_connections 2024-04-25T13:50:26.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 --2- 172.21.15.80:0/335362613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890161280 0x7fb890163e30 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 --2- 172.21.15.80:0/335362613 >> v2:172.21.15.87:3300/0 conn(0x7fb890068ad0 0x7fb890160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 --2- 172.21.15.80:0/335362613 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 0x7fb890068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 -- 172.21.15.80:0/335362613 >> 172.21.15.80:0/335362613 conn(0x7fb8900cf390 msgr2=0x7fb8900cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:26.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 -- 172.21.15.80:0/335362613 shutdown_connections 2024-04-25T13:50:26.896 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.896+0000 7fb8966b3640 1 -- 172.21.15.80:0/335362613 wait complete. 2024-04-25T13:50:26.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 Processor -- start 2024-04-25T13:50:26.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 -- start start 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890068ad0 0x7fb8900b7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 0x7fb890143230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb890161280 0x7fb890167dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb89016c350 con 0x7fb890068ad0 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb89016c1d0 con 0x7fb890161280 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb89016c4d0 con 0x7fb89006efd0 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890068ad0 0x7fb8900b7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890068ad0 0x7fb8900b7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53586/0 (socket says 172.21.15.80:53586) 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 -- 172.21.15.80:0/3108665853 learned_addr learned my addr 172.21.15.80:0/3108665853 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:26.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88f7fe640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 0x7fb890143230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb894c50640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb890161280 0x7fb890167dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 -- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 msgr2=0x7fb890143230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 0x7fb890143230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 -- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb890161280 msgr2=0x7fb890167dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb890161280 0x7fb890167dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 -- 172.21.15.80:0/3108665853 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb890166a60 con 0x7fb890068ad0 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88ffff640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890068ad0 0x7fb8900b7cd0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb88000d5f0 tx=0x7fb88000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88cff9640 1 -- 172.21.15.80:0/3108665853 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800125f0 con 0x7fb890068ad0 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88cff9640 1 -- 172.21.15.80:0/3108665853 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb880019070 con 0x7fb890068ad0 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb890166d40 con 0x7fb890068ad0 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb88cff9640 1 -- 172.21.15.80:0/3108665853 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8800129b0 con 0x7fb890068ad0 2024-04-25T13:50:26.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.900+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb89015aff0 con 0x7fb890068ad0 2024-04-25T13:50:26.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.904+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb89006a3a0 con 0x7fb890068ad0 2024-04-25T13:50:26.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.904+0000 7fb88cff9640 1 -- 172.21.15.80:0/3108665853 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb880012b50 con 0x7fb890068ad0 2024-04-25T13:50:26.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.904+0000 7fb88cff9640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb87c06f400 0x7fb87c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:26.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.904+0000 7fb88cff9640 1 -- 172.21.15.80:0/3108665853 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 4857+0+0 (secure 0 0 0) 0x7fb88008f3b0 con 0x7fb890068ad0 2024-04-25T13:50:26.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.904+0000 7fb88f7fe640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb87c06f400 0x7fb87c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:26.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.904+0000 7fb88f7fe640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb87c06f400 0x7fb87c071910 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb8900b8320 tx=0x7fb884008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:26.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:26.912+0000 7fb88cff9640 1 -- 172.21.15.80:0/3108665853 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb88005c5f0 con 0x7fb890068ad0 2024-04-25T13:50:27.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.060+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb8900cef60 con 0x7fb890068ad0 2024-04-25T13:50:27.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb88cff9640 1 -- 172.21.15.80:0/3108665853 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+12830 (secure 0 0 0) 0x7fb88005bef0 con 0x7fb890068ad0 2024-04-25T13:50:27.061 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:27.061 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":167,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:50:26.305369+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":155,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":152,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:27.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb87c06f400 msgr2=0x7fb87c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb87c06f400 0x7fb87c071910 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb8900b8320 tx=0x7fb884008040 comp rx=0 tx=0).stop 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890068ad0 msgr2=0x7fb8900b7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890068ad0 0x7fb8900b7cd0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fb88000d5f0 tx=0x7fb88000dab0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 shutdown_connections 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb87c06f400 0x7fb87c071910 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb890161280 0x7fb890167dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb89006efd0 0x7fb890143230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 --2- 172.21.15.80:0/3108665853 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb890068ad0 0x7fb8900b7cd0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 >> 172.21.15.80:0/3108665853 conn(0x7fb8900cf390 msgr2=0x7fb8900af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 shutdown_connections 2024-04-25T13:50:27.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.064+0000 7fb8966b3640 1 -- 172.21.15.80:0/3108665853 wait complete. 2024-04-25T13:50:27.083 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:50:27.083 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:50:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba1e57a640 1 Processor -- start 2024-04-25T13:50:27.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba1e57a640 1 -- start start 2024-04-25T13:50:27.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba1e57a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fba180b42f0 0x7fba180d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba1e57a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 0x7fba180ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba1e57a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba180d2b60 con 0x7fba180d2020 2024-04-25T13:50:27.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba1e57a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fba1806e230 con 0x7fba180b42f0 2024-04-25T13:50:27.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba1e57a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fba1806cf90 con 0x7fba180ce4a0 2024-04-25T13:50:27.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba0ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fba180b42f0 0x7fba180d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba0ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fba180b42f0 0x7fba180d2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53600/0 (socket says 172.21.15.80:53600) 2024-04-25T13:50:27.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba0ffff640 1 -- 172.21.15.80:0/3973480947 learned_addr learned my addr 172.21.15.80:0/3973480947 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:27.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba1cb17640 1 --2- 172.21.15.80:0/3973480947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 0x7fba180ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba0ffff640 1 -- 172.21.15.80:0/3973480947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 msgr2=0x7fba180ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:27.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba0ffff640 1 --2- 172.21.15.80:0/3973480947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 0x7fba180ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba0ffff640 1 -- 172.21.15.80:0/3973480947 >> v1:172.21.15.87:6789/0 conn(0x7fba180d2020 legacy=0x7fba180b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.236+0000 7fba0ffff640 1 -- 172.21.15.80:0/3973480947 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1806dc70 con 0x7fba180b42f0 2024-04-25T13:50:27.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba0ffff640 1 --2- 172.21.15.80:0/3973480947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba180d2430 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fba00009870 tx=0x7fba0001c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512bb3335356813f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba177fe640 1 -- 172.21.15.80:0/3973480947 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000a9640 con 0x7fba180b42f0 2024-04-25T13:50:27.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba177fe640 1 -- 172.21.15.80:0/3973480947 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba000a97e0 con 0x7fba180b42f0 2024-04-25T13:50:27.238 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba177fe640 1 -- 172.21.15.80:0/3973480947 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba000a9bb0 con 0x7fba180b42f0 2024-04-25T13:50:27.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba1e57a640 1 -- 172.21.15.80:0/3973480947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 msgr2=0x7fba180d2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba1e57a640 1 --2- 172.21.15.80:0/3973480947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba180d2430 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fba00009870 tx=0x7fba0001c560 comp rx=0 tx=0).stop 2024-04-25T13:50:27.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba1e57a640 1 -- 172.21.15.80:0/3973480947 shutdown_connections 2024-04-25T13:50:27.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba1e57a640 1 --2- 172.21.15.80:0/3973480947 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 0x7fba180ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba1e57a640 1 --2- 172.21.15.80:0/3973480947 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba180d2430 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba1e57a640 1 -- 172.21.15.80:0/3973480947 >> 172.21.15.80:0/3973480947 conn(0x7fba180d3ab0 msgr2=0x7fba180d3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:27.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba1e57a640 1 -- 172.21.15.80:0/3973480947 shutdown_connections 2024-04-25T13:50:27.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.240+0000 7fba1e57a640 1 -- 172.21.15.80:0/3973480947 wait complete. 2024-04-25T13:50:27.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 Processor -- start 2024-04-25T13:50:27.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 -- start start 2024-04-25T13:50:27.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba1816b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 0x7fba18160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba180d2020 0x7fba18164b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fba180d4750 con 0x7fba180b42f0 2024-04-25T13:50:27.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fba180d45d0 con 0x7fba180d2020 2024-04-25T13:50:27.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fba1816d7c0 con 0x7fba180ce4a0 2024-04-25T13:50:27.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba1816b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba1816b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53606/0 (socket says 172.21.15.80:53606) 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 -- 172.21.15.80:0/367948693 learned_addr learned my addr 172.21.15.80:0/367948693 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1cb17640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba180d2020 0x7fba18164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba0ffff640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 0x7fba18160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 -- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 msgr2=0x7fba18160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 0x7fba18160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 -- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba180d2020 msgr2=0x7fba18164b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba180d2020 0x7fba18164b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 -- 172.21.15.80:0/367948693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1816a1b0 con 0x7fba180b42f0 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba17fff640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba1816b010 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fba0800c3a0 tx=0x7fba0800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba157fa640 1 -- 172.21.15.80:0/367948693 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba080052d0 con 0x7fba180b42f0 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba157fa640 1 -- 172.21.15.80:0/367948693 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba08005470 con 0x7fba180b42f0 2024-04-25T13:50:27.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1816a490 con 0x7fba180b42f0 2024-04-25T13:50:27.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba157fa640 1 -- 172.21.15.80:0/367948693 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba08007260 con 0x7fba180b42f0 2024-04-25T13:50:27.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.244+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba180b0c70 con 0x7fba180b42f0 2024-04-25T13:50:27.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.248+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1806a3a0 con 0x7fba180b42f0 2024-04-25T13:50:27.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.248+0000 7fba157fa640 1 -- 172.21.15.80:0/367948693 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fba08007400 con 0x7fba180b42f0 2024-04-25T13:50:27.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.248+0000 7fba157fa640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb9f806f430 0x7fb9f8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.248+0000 7fba157fa640 1 -- 172.21.15.80:0/367948693 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 4857+0+0 (secure 0 0 0) 0x7fba0808e390 con 0x7fba180b42f0 2024-04-25T13:50:27.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.248+0000 7fba0ffff640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb9f806f430 0x7fb9f8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.248+0000 7fba0ffff640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb9f806f430 0x7fb9f8071940 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fba000ae000 tx=0x7fba00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.252+0000 7fba157fa640 1 -- 172.21.15.80:0/367948693 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fba180d2b60 con 0x7fba180b42f0 2024-04-25T13:50:27.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.412+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fba180b95e0 con 0x7fba180b42f0 2024-04-25T13:50:27.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba157fa640 1 -- 172.21.15.80:0/367948693 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) ==== 93+0+10 (secure 0 0 0) 0x7fba0805b660 con 0x7fba180b42f0 2024-04-25T13:50:27.413 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb9f806f430 msgr2=0x7fb9f8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb9f806f430 0x7fb9f8071940 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fba000ae000 tx=0x7fba00009040 comp rx=0 tx=0).stop 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 msgr2=0x7fba1816b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba1816b010 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fba0800c3a0 tx=0x7fba0800c860 comp rx=0 tx=0).stop 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 shutdown_connections 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb9f806f430 0x7fb9f8071940 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba180d2020 0x7fba18164b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fba180ce4a0 0x7fba18160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 --2- 172.21.15.80:0/367948693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fba180b42f0 0x7fba1816b010 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 >> 172.21.15.80:0/367948693 conn(0x7fba180d3ab0 msgr2=0x7fba180757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 shutdown_connections 2024-04-25T13:50:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.416+0000 7fba1e57a640 1 -- 172.21.15.80:0/367948693 wait complete. 2024-04-25T13:50:27.437 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:50:27.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa97e72640 1 Processor -- start 2024-04-25T13:50:27.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa97e72640 1 -- start start 2024-04-25T13:50:27.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa97e72640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 0x7faa900d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa97e72640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faa900b7650 0x7faa900b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa97e72640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faa900d0e30 con 0x7faa900b9240 2024-04-25T13:50:27.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa97e72640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7faa900b6920 con 0x7faa900b7650 2024-04-25T13:50:27.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa97e72640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faa900cf8c0 con 0x7faa900d04e0 2024-04-25T13:50:27.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa9640f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faa900b7650 0x7faa900b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa9640f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7faa900b7650 0x7faa900b7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53616/0 (socket says 172.21.15.80:53616) 2024-04-25T13:50:27.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa9640f640 1 -- 172.21.15.80:0/132995779 learned_addr learned my addr 172.21.15.80:0/132995779 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:27.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa9540d640 1 --2- 172.21.15.80:0/132995779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 0x7faa900d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa9640f640 1 -- 172.21.15.80:0/132995779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 msgr2=0x7faa900d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:27.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa9640f640 1 --2- 172.21.15.80:0/132995779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 0x7faa900d08f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa9640f640 1 -- 172.21.15.80:0/132995779 >> v1:172.21.15.87:6789/0 conn(0x7faa900b9240 legacy=0x7faa900cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:27.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.588+0000 7faa9640f640 1 -- 172.21.15.80:0/132995779 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa900cfb20 con 0x7faa900b7650 2024-04-25T13:50:27.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa9640f640 1 --2- 172.21.15.80:0/132995779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900b7a60 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7faa88009e50 tx=0x7faa8801c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f095c81c30b5f502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa94c0c640 1 -- 172.21.15.80:0/132995779 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880a55d0 con 0x7faa900b7650 2024-04-25T13:50:27.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa94c0c640 1 -- 172.21.15.80:0/132995779 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa880a5770 con 0x7faa900b7650 2024-04-25T13:50:27.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa94c0c640 1 -- 172.21.15.80:0/132995779 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa880a5b40 con 0x7faa900b7650 2024-04-25T13:50:27.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa97e72640 1 -- 172.21.15.80:0/132995779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 msgr2=0x7faa900b7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa97e72640 1 --2- 172.21.15.80:0/132995779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900b7a60 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7faa88009e50 tx=0x7faa8801c750 comp rx=0 tx=0).stop 2024-04-25T13:50:27.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa97e72640 1 -- 172.21.15.80:0/132995779 shutdown_connections 2024-04-25T13:50:27.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa97e72640 1 --2- 172.21.15.80:0/132995779 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900b7a60 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa97e72640 1 --2- 172.21.15.80:0/132995779 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 0x7faa900d08f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa97e72640 1 -- 172.21.15.80:0/132995779 >> 172.21.15.80:0/132995779 conn(0x7faa900d3ad0 msgr2=0x7faa900d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:27.591 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa97e72640 1 -- 172.21.15.80:0/132995779 shutdown_connections 2024-04-25T13:50:27.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.592+0000 7faa97e72640 1 -- 172.21.15.80:0/132995779 wait complete. 2024-04-25T13:50:27.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 Processor -- start 2024-04-25T13:50:27.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 -- start start 2024-04-25T13:50:27.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa900b9240 0x7faa90154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 0x7faa901512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7faa9016b0f0 con 0x7faa900b7650 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7faa9016af70 con 0x7faa900b9240 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7faa9016b270 con 0x7faa900d04e0 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53626/0 (socket says 172.21.15.80:53626) 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 -- 172.21.15.80:0/1998585279 learned_addr learned my addr 172.21.15.80:0/1998585279 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa9640f640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 0x7faa901512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa9540d640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa900b9240 0x7faa90154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 -- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 msgr2=0x7faa901512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 0x7faa901512a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 -- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa900b9240 msgr2=0x7faa90154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa900b9240 0x7faa90154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 -- 172.21.15.80:0/1998585279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa90153290 con 0x7faa900b7650 2024-04-25T13:50:27.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa9540d640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa900b9240 0x7faa90154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:27.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa95c0e640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900cd650 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b3c0 tx=0x7faa8c00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa867fc640 1 -- 172.21.15.80:0/1998585279 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c011830 con 0x7faa900b7650 2024-04-25T13:50:27.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa867fc640 1 -- 172.21.15.80:0/1998585279 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faa8c016070 con 0x7faa900b7650 2024-04-25T13:50:27.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa90153570 con 0x7faa900b7650 2024-04-25T13:50:27.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa867fc640 1 -- 172.21.15.80:0/1998585279 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa8c011c70 con 0x7faa900b7650 2024-04-25T13:50:27.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.596+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa90153b30 con 0x7faa900b7650 2024-04-25T13:50:27.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.600+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa9015e610 con 0x7faa900b7650 2024-04-25T13:50:27.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.600+0000 7faa867fc640 1 -- 172.21.15.80:0/1998585279 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7faa8c01a410 con 0x7faa900b7650 2024-04-25T13:50:27.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.600+0000 7faa867fc640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa7406f400 0x7faa74071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.600+0000 7faa867fc640 1 -- 172.21.15.80:0/1998585279 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 4857+0+0 (secure 0 0 0) 0x7faa8c08e8b0 con 0x7faa900b7650 2024-04-25T13:50:27.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.600+0000 7faa9540d640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa7406f400 0x7faa74071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.600+0000 7faa9540d640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa7406f400 0x7faa74071910 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7faa80006fd0 tx=0x7faa80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.604+0000 7faa867fc640 1 -- 172.21.15.80:0/1998585279 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7faa8c05bae0 con 0x7faa900b7650 2024-04-25T13:50:27.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.760+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faa90074c30 con 0x7faa900b7650 2024-04-25T13:50:27.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.760+0000 7faa867fc640 1 -- 172.21.15.80:0/1998585279 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v167) ==== 84+0+65 (secure 0 0 0) 0x7faa8c05b3e0 con 0x7faa900b7650 2024-04-25T13:50:27.759 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:27.759 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:50:27.759 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:50:27.759 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:50:27.759 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:50:27.759 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:27.760 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:50:27.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa7406f400 msgr2=0x7faa74071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa7406f400 0x7faa74071910 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7faa80006fd0 tx=0x7faa80008040 comp rx=0 tx=0).stop 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 msgr2=0x7faa900cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900cd650 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7faa8c00b3c0 tx=0x7faa8c00b880 comp rx=0 tx=0).stop 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 shutdown_connections 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7faa7406f400 0x7faa74071910 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7faa900d04e0 0x7faa901512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faa900b9240 0x7faa90154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 --2- 172.21.15.80:0/1998585279 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7faa900b7650 0x7faa900cd650 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 >> 172.21.15.80:0/1998585279 conn(0x7faa900d3ad0 msgr2=0x7faa90158160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 shutdown_connections 2024-04-25T13:50:27.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.764+0000 7faa97e72640 1 -- 172.21.15.80:0/1998585279 wait complete. 2024-04-25T13:50:27.780 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:50:27.780 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:50:27.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.920+0000 7fcb4f081640 1 Processor -- start 2024-04-25T13:50:27.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb4f081640 1 -- start start 2024-04-25T13:50:27.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb4f081640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb480b0f10 0x7fcb480d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb4f081640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb480ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.922 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb4f081640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb480b1320 con 0x7fcb480cf730 2024-04-25T13:50:27.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb4f081640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcb480b5820 con 0x7fcb480b0f10 2024-04-25T13:50:27.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb4f081640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcb480d13f0 con 0x7fcb480ce2a0 2024-04-25T13:50:27.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb3ffff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb480ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb4ce1d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcb480b0f10 0x7fcb480d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb3ffff640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb480ce6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:46866/0 (socket says 172.21.15.80:46866) 2024-04-25T13:50:27.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb3ffff640 1 -- 172.21.15.80:0/3010270425 learned_addr learned my addr 172.21.15.80:0/3010270425 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:27.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb3ffff640 1 -- 172.21.15.80:0/3010270425 >> v2:172.21.15.87:3300/0 conn(0x7fcb480b0f10 msgr2=0x7fcb480d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:27.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb3ffff640 1 --2- 172.21.15.80:0/3010270425 >> v2:172.21.15.87:3300/0 conn(0x7fcb480b0f10 0x7fcb480d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb3ffff640 1 -- 172.21.15.80:0/3010270425 >> v1:172.21.15.87:6789/0 conn(0x7fcb480cf730 legacy=0x7fcb480cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.924+0000 7fcb3ffff640 1 -- 172.21.15.80:0/3010270425 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb480cebf0 con 0x7fcb480ce2a0 2024-04-25T13:50:27.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb3ffff640 1 --2- 172.21.15.80:0/3010270425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb480ce6b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009870 tx=0x7fcb3801c560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa60436d8257421b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb3f7fe640 1 -- 172.21.15.80:0/3010270425 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb380a7070 con 0x7fcb480ce2a0 2024-04-25T13:50:27.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb3f7fe640 1 -- 172.21.15.80:0/3010270425 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb380b0070 con 0x7fcb480ce2a0 2024-04-25T13:50:27.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb3f7fe640 1 -- 172.21.15.80:0/3010270425 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb380ab890 con 0x7fcb480ce2a0 2024-04-25T13:50:27.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb4f081640 1 -- 172.21.15.80:0/3010270425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 msgr2=0x7fcb480ce6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb4f081640 1 --2- 172.21.15.80:0/3010270425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb480ce6b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fcb38009870 tx=0x7fcb3801c560 comp rx=0 tx=0).stop 2024-04-25T13:50:27.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb4f081640 1 -- 172.21.15.80:0/3010270425 shutdown_connections 2024-04-25T13:50:27.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb4f081640 1 --2- 172.21.15.80:0/3010270425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb480ce6b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb4f081640 1 --2- 172.21.15.80:0/3010270425 >> v2:172.21.15.87:3300/0 conn(0x7fcb480b0f10 0x7fcb480d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb4f081640 1 -- 172.21.15.80:0/3010270425 >> 172.21.15.80:0/3010270425 conn(0x7fcb480b35c0 msgr2=0x7fcb480b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:27.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb4f081640 1 -- 172.21.15.80:0/3010270425 shutdown_connections 2024-04-25T13:50:27.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.928+0000 7fcb4f081640 1 -- 172.21.15.80:0/3010270425 wait complete. 2024-04-25T13:50:27.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4f081640 1 Processor -- start 2024-04-25T13:50:27.929 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4f081640 1 -- start start 2024-04-25T13:50:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4f081640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb480b0f10 0x7fcb4815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4f081640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb4813ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4f081640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb480cf730 0x7fcb4813f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4f081640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcb480b5300 con 0x7fcb480b0f10 2024-04-25T13:50:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4f081640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcb480d1150 con 0x7fcb480cf730 2024-04-25T13:50:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4f081640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcb4815e720 con 0x7fcb480ce2a0 2024-04-25T13:50:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb480b0f10 0x7fcb4815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb480b0f10 0x7fcb4815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53642/0 (socket says 172.21.15.80:53642) 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 -- 172.21.15.80:0/2907442298 learned_addr learned my addr 172.21.15.80:0/2907442298 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb3ffff640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb4813ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4d61e640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb480cf730 0x7fcb4813f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 -- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 msgr2=0x7fcb4813ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb4813ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 -- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb480cf730 msgr2=0x7fcb4813f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb480cf730 0x7fcb4813f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 -- 172.21.15.80:0/2907442298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4815af60 con 0x7fcb480b0f10 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4d61e640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb480cf730 0x7fcb4813f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:27.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.932+0000 7fcb4ce1d640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb480b0f10 0x7fcb4815d8e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400e8a0 tx=0x7fcb4400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.936+0000 7fcb3d7fa640 1 -- 172.21.15.80:0/2907442298 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb44017070 con 0x7fcb480b0f10 2024-04-25T13:50:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.936+0000 7fcb3d7fa640 1 -- 172.21.15.80:0/2907442298 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb4400ee90 con 0x7fcb480b0f10 2024-04-25T13:50:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.936+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4815b240 con 0x7fcb480b0f10 2024-04-25T13:50:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.936+0000 7fcb3d7fa640 1 -- 172.21.15.80:0/2907442298 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb4400cb40 con 0x7fcb480b0f10 2024-04-25T13:50:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.936+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb4815b800 con 0x7fcb480b0f10 2024-04-25T13:50:27.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.936+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb480b4520 con 0x7fcb480b0f10 2024-04-25T13:50:27.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.940+0000 7fcb3d7fa640 1 -- 172.21.15.80:0/2907442298 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcb4401a070 con 0x7fcb480b0f10 2024-04-25T13:50:27.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.940+0000 7fcb3d7fa640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb3406f430 0x7fcb34071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:27.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.940+0000 7fcb3d7fa640 1 -- 172.21.15.80:0/2907442298 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 4857+0+0 (secure 0 0 0) 0x7fcb4408f140 con 0x7fcb480b0f10 2024-04-25T13:50:27.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.940+0000 7fcb3ffff640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb3406f430 0x7fcb34071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:27.938 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.940+0000 7fcb3ffff640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb3406f430 0x7fcb34071940 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcb380af000 tx=0x7fcb380a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:27.941 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:27.944+0000 7fcb3d7fa640 1 -- 172.21.15.80:0/2907442298 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcb480b4520 con 0x7fcb480b0f10 2024-04-25T13:50:28.090 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.092+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcb4806a440 con 0x7fcb480b0f10 2024-04-25T13:50:28.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.944+0000 7fcb3d7fa640 1 -- 172.21.15.80:0/2907442298 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v168) ==== 138+0+0 (secure 0 0 0) 0x7fcb4405c380 con 0x7fcb480b0f10 2024-04-25T13:50:28.943 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:50:28.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb3406f430 msgr2=0x7fcb34071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:28.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb3406f430 0x7fcb34071940 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fcb380af000 tx=0x7fcb380a7000 comp rx=0 tx=0).stop 2024-04-25T13:50:28.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb480b0f10 msgr2=0x7fcb4815d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:28.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb480b0f10 0x7fcb4815d8e0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7fcb4400e8a0 tx=0x7fcb4400ed60 comp rx=0 tx=0).stop 2024-04-25T13:50:28.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 shutdown_connections 2024-04-25T13:50:28.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcb3406f430 0x7fcb34071940 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:28.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcb480cf730 0x7fcb4813f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:28.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcb480ce2a0 0x7fcb4813ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:28.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 --2- 172.21.15.80:0/2907442298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcb480b0f10 0x7fcb4815d8e0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:28.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 >> 172.21.15.80:0/2907442298 conn(0x7fcb480b35c0 msgr2=0x7fcb480b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:28.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 shutdown_connections 2024-04-25T13:50:28.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:28.948+0000 7fcb4f081640 1 -- 172.21.15.80:0/2907442298 wait complete. 2024-04-25T13:50:28.974 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:50:29.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389f7ef640 1 Processor -- start 2024-04-25T13:50:29.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389f7ef640 1 -- start start 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389f7ef640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f389806ebb0 0x7f389806efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389f7ef640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38980d0af0 con 0x7f3898161f90 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389f7ef640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f38980d0d70 con 0x7f38980d06e0 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389f7ef640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f38980d1050 con 0x7f389806ebb0 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389d58b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3898161f90 0x7f38980cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45956/0 (socket says 172.21.15.80:45956) 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389d58b640 1 -- 172.21.15.80:0/1547797274 learned_addr learned my addr 172.21.15.80:0/1547797274 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.132+0000 7f389dd8c640 1 --2- 172.21.15.80:0/1547797274 >> v2:172.21.15.80:3300/0 conn(0x7f389806ebb0 0x7f389806efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f388ffff640 1 -- 172.21.15.80:0/1547797274 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 140635412 0 0) 0x7f3894001120 con 0x7f3898161f90 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f388ffff640 1 -- 172.21.15.80:0/1547797274 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3884003530 con 0x7f3898161f90 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389dd8c640 1 -- 172.21.15.80:0/1547797274 >> v1:172.21.15.87:6789/0 conn(0x7f38980d06e0 legacy=0x7f389806e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:29.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389dd8c640 1 -- 172.21.15.80:0/1547797274 >> v1:172.21.15.80:6789/0 conn(0x7f3898161f90 legacy=0x7f38980cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:29.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389dd8c640 1 -- 172.21.15.80:0/1547797274 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389806dca0 con 0x7f389806ebb0 2024-04-25T13:50:29.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389dd8c640 1 --2- 172.21.15.80:0/1547797274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f389806ebb0 0x7f389806efc0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3890009d90 tx=0x7f389001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b661fb175bd26390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:29.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f388ffff640 1 -- 172.21.15.80:0/1547797274 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38900b2070 con 0x7f389806ebb0 2024-04-25T13:50:29.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f388ffff640 1 -- 172.21.15.80:0/1547797274 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f38900a5860 con 0x7f389806ebb0 2024-04-25T13:50:29.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f388ffff640 1 -- 172.21.15.80:0/1547797274 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38900a5c30 con 0x7f389806ebb0 2024-04-25T13:50:29.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389f7ef640 1 -- 172.21.15.80:0/1547797274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f389806ebb0 msgr2=0x7f389806efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:29.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389f7ef640 1 --2- 172.21.15.80:0/1547797274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f389806ebb0 0x7f389806efc0 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f3890009d90 tx=0x7f389001c7e0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389f7ef640 1 -- 172.21.15.80:0/1547797274 shutdown_connections 2024-04-25T13:50:29.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389f7ef640 1 --2- 172.21.15.80:0/1547797274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f389806ebb0 0x7f389806efc0 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389f7ef640 1 -- 172.21.15.80:0/1547797274 >> 172.21.15.80:0/1547797274 conn(0x7f389806d080 msgr2=0x7f38980b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:29.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389f7ef640 1 -- 172.21.15.80:0/1547797274 shutdown_connections 2024-04-25T13:50:29.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.136+0000 7f389f7ef640 1 -- 172.21.15.80:0/1547797274 wait complete. 2024-04-25T13:50:29.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 Processor -- start 2024-04-25T13:50:29.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 -- start start 2024-04-25T13:50:29.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389806ebb0 0x7f3898160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:29.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38980d06e0 0x7f38980af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3898161f90 0x7f38980afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f389806f680 con 0x7f38980d06e0 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f389806f500 con 0x7f389806ebb0 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3898144620 con 0x7f3898161f90 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38980d06e0 0x7f38980af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38980d06e0 0x7f38980af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53648/0 (socket says 172.21.15.80:53648) 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 -- 172.21.15.80:0/719169977 learned_addr learned my addr 172.21.15.80:0/719169977 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389d58b640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389806ebb0 0x7f3898160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389dd8c640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3898161f90 0x7f38980afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 -- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3898161f90 msgr2=0x7f38980afdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3898161f90 0x7f38980afdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 -- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389806ebb0 msgr2=0x7f3898160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389806ebb0 0x7f3898160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 -- 172.21.15.80:0/719169977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38981fabe0 con 0x7f38980d06e0 2024-04-25T13:50:29.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389dd8c640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3898161f90 0x7f38980afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:29.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389cd8a640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38980d06e0 0x7f38980af8b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f388800b410 tx=0x7f388800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:29.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f388dffb640 1 -- 172.21.15.80:0/719169977 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888015030 con 0x7f38980d06e0 2024-04-25T13:50:29.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f388dffb640 1 -- 172.21.15.80:0/719169977 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3888004020 con 0x7f38980d06e0 2024-04-25T13:50:29.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38981faec0 con 0x7f38980d06e0 2024-04-25T13:50:29.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f388dffb640 1 -- 172.21.15.80:0/719169977 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888004450 con 0x7f38980d06e0 2024-04-25T13:50:29.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.140+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38981fb480 con 0x7f38980d06e0 2024-04-25T13:50:29.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.144+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389806a3a0 con 0x7f38980d06e0 2024-04-25T13:50:29.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.144+0000 7f388dffb640 1 -- 172.21.15.80:0/719169977 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3888018070 con 0x7f38980d06e0 2024-04-25T13:50:29.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.144+0000 7f388dffb640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f387c06f400 0x7f387c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:29.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.144+0000 7f388dffb640 1 -- 172.21.15.80:0/719169977 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5227+0+0 (secure 0 0 0) 0x7f388808f3a0 con 0x7f38980d06e0 2024-04-25T13:50:29.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.144+0000 7f389d58b640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f387c06f400 0x7f387c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:29.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.144+0000 7f389d58b640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f387c06f400 0x7f387c071910 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3894005890 tx=0x7f3894009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:29.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.148+0000 7f388dffb640 1 -- 172.21.15.80:0/719169977 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f388805bf90 con 0x7f38980d06e0 2024-04-25T13:50:29.306 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.308+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f38980b6720 con 0x7f38980d06e0 2024-04-25T13:50:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.948+0000 7f388dffb640 1 -- 172.21.15.80:0/719169977 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v169) ==== 170+0+0 (secure 0 0 0) 0x7f3888060290 con 0x7f38980d06e0 2024-04-25T13:50:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:50:29.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f387c06f400 msgr2=0x7f387c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:29.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f387c06f400 0x7f387c071910 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f3894005890 tx=0x7f3894009040 comp rx=0 tx=0).stop 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38980d06e0 msgr2=0x7f38980af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38980d06e0 0x7f38980af8b0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f388800b410 tx=0x7f388800b8d0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 shutdown_connections 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f387c06f400 0x7f387c071910 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3898161f90 0x7f38980afdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38980d06e0 0x7f38980af8b0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 --2- 172.21.15.80:0/719169977 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f389806ebb0 0x7f3898160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 >> 172.21.15.80:0/719169977 conn(0x7f389806d080 msgr2=0x7f38980757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 shutdown_connections 2024-04-25T13:50:29.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:29.956+0000 7f389f7ef640 1 -- 172.21.15.80:0/719169977 wait complete. 2024-04-25T13:50:29.992 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:50:30.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f5433804640 1 Processor -- start 2024-04-25T13:50:30.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f5433804640 1 -- start start 2024-04-25T13:50:30.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f5433804640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f542c0d0af0 0x7f542c0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:30.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f5433804640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f542c162be0 con 0x7f542c0ceaa0 2024-04-25T13:50:30.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f5433804640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f542c0d1910 con 0x7f542c0cffd0 2024-04-25T13:50:30.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f5433804640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f542c162750 con 0x7f542c0d0af0 2024-04-25T13:50:30.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f5431da1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f542c0d0af0 0x7f542c0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:30.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f54315a0640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f542c0ceaa0 0x7f542c0ceeb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:45968/0 (socket says 172.21.15.80:45968) 2024-04-25T13:50:30.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f5431da1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f542c0d0af0 0x7f542c0d0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53664/0 (socket says 172.21.15.80:53664) 2024-04-25T13:50:30.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.148+0000 7f54315a0640 1 -- 172.21.15.80:0/1672786128 learned_addr learned my addr 172.21.15.80:0/1672786128 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:30.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5431da1640 1 -- 172.21.15.80:0/1672786128 >> v1:172.21.15.87:6789/0 conn(0x7f542c0cffd0 legacy=0x7f542c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:30.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5431da1640 1 -- 172.21.15.80:0/1672786128 >> v1:172.21.15.80:6789/0 conn(0x7f542c0ceaa0 legacy=0x7f542c0ceeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:30.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5423fff640 1 -- 172.21.15.80:0/1672786128 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2662969728 0 0) 0x7f542c162be0 con 0x7f542c0ceaa0 2024-04-25T13:50:30.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5431da1640 1 -- 172.21.15.80:0/1672786128 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c15e4e0 con 0x7f542c0d0af0 2024-04-25T13:50:30.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5431da1640 1 --2- 172.21.15.80:0/1672786128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0d0af0 0x7f542c0d0f00 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5424009d90 tx=0x7f542401c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbf1d925961dc3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:30.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5423fff640 1 -- 172.21.15.80:0/1672786128 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54240b2070 con 0x7f542c0d0af0 2024-04-25T13:50:30.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5423fff640 1 -- 172.21.15.80:0/1672786128 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54240a5970 con 0x7f542c0d0af0 2024-04-25T13:50:30.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5423fff640 1 -- 172.21.15.80:0/1672786128 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54240a5d40 con 0x7f542c0d0af0 2024-04-25T13:50:30.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5433804640 1 -- 172.21.15.80:0/1672786128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0d0af0 msgr2=0x7f542c0d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:30.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5433804640 1 --2- 172.21.15.80:0/1672786128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0d0af0 0x7f542c0d0f00 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f5424009d90 tx=0x7f542401c8b0 comp rx=0 tx=0).stop 2024-04-25T13:50:30.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5433804640 1 -- 172.21.15.80:0/1672786128 shutdown_connections 2024-04-25T13:50:30.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5433804640 1 --2- 172.21.15.80:0/1672786128 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0d0af0 0x7f542c0d0f00 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:30.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5433804640 1 -- 172.21.15.80:0/1672786128 >> 172.21.15.80:0/1672786128 conn(0x7f542c0b3cc0 msgr2=0x7f542c0b40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:30.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.152+0000 7f5433804640 1 -- 172.21.15.80:0/1672786128 shutdown_connections 2024-04-25T13:50:30.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 -- 172.21.15.80:0/1672786128 wait complete. 2024-04-25T13:50:30.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 Processor -- start 2024-04-25T13:50:30.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 -- start start 2024-04-25T13:50:30.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0ceaa0 0x7f542c09cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:30.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c0cffd0 0x7f542c1589f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:30.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0d0af0 0x7f542c0b1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:30.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f542c0745f0 con 0x7f542c0ceaa0 2024-04-25T13:50:30.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f542c074470 con 0x7f542c0d0af0 2024-04-25T13:50:30.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.156+0000 7f5433804640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f542c16c840 con 0x7f542c0cffd0 2024-04-25T13:50:30.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0ceaa0 0x7f542c09cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0ceaa0 0x7f542c09cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:53672/0 (socket says 172.21.15.80:53672) 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 -- 172.21.15.80:0/1454254796 learned_addr learned my addr 172.21.15.80:0/1454254796 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5430d9f640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c0cffd0 0x7f542c1589f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5431da1640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0d0af0 0x7f542c0b1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 -- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c0cffd0 msgr2=0x7f542c1589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c0cffd0 0x7f542c1589f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 -- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0d0af0 msgr2=0x7f542c0b1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0d0af0 0x7f542c0b1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 -- 172.21.15.80:0/1454254796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c09d0a0 con 0x7f542c0ceaa0 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5431da1640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0d0af0 0x7f542c0b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f54315a0640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0ceaa0 0x7f542c09cb60 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f542800c470 tx=0x7f542800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:30.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5421ffb640 1 -- 172.21.15.80:0/1454254796 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428018070 con 0x7f542c0ceaa0 2024-04-25T13:50:30.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5421ffb640 1 -- 172.21.15.80:0/1454254796 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54280074e0 con 0x7f542c0ceaa0 2024-04-25T13:50:30.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542c09f720 con 0x7f542c0ceaa0 2024-04-25T13:50:30.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5421ffb640 1 -- 172.21.15.80:0/1454254796 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5428004a30 con 0x7f542c0ceaa0 2024-04-25T13:50:30.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542c0b6b60 con 0x7f542c0ceaa0 2024-04-25T13:50:30.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f542c06a3a0 con 0x7f542c0ceaa0 2024-04-25T13:50:30.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5421ffb640 1 -- 172.21.15.80:0/1454254796 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f542801b070 con 0x7f542c0ceaa0 2024-04-25T13:50:30.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5421ffb640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f541006f430 0x7f5410071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:30.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5421ffb640 1 -- 172.21.15.80:0/1454254796 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5609+0+0 (secure 0 0 0) 0x7f542808e4a0 con 0x7f542c0ceaa0 2024-04-25T13:50:30.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.160+0000 7f5430d9f640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f541006f430 0x7f5410071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:30.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.164+0000 7f5430d9f640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f541006f430 0x7f5410071940 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f542c0ce4b0 tx=0x7f541c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:30.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.168+0000 7f5421ffb640 1 -- 172.21.15.80:0/1454254796 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f542805b3f0 con 0x7f542c0ceaa0 2024-04-25T13:50:30.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:30.364+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f542c0a0100 con 0x7f542c0ceaa0 2024-04-25T13:50:30.532 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:50:30.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:50:31.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.020+0000 7f5421ffb640 1 -- 172.21.15.80:0/1454254796 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v78) ==== 174+0+0 (secure 0 0 0) 0x7f542805acf0 con 0x7f542c0ceaa0 2024-04-25T13:50:31.020 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 20 and data pool 21 2024-04-25T13:50:31.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f541006f430 msgr2=0x7f5410071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f541006f430 0x7f5410071940 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f542c0ce4b0 tx=0x7f541c009040 comp rx=0 tx=0).stop 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0ceaa0 msgr2=0x7f542c09cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0ceaa0 0x7f542c09cb60 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f542800c470 tx=0x7f542800c930 comp rx=0 tx=0).stop 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 shutdown_connections 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f541006f430 0x7f5410071940 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f542c0d0af0 0x7f542c0b1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f542c0cffd0 0x7f542c1589f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 --2- 172.21.15.80:0/1454254796 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f542c0ceaa0 0x7f542c09cb60 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 >> 172.21.15.80:0/1454254796 conn(0x7f542c0b3cc0 msgr2=0x7f542c06f550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 shutdown_connections 2024-04-25T13:50:31.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.024+0000 7f5433804640 1 -- 172.21.15.80:0/1454254796 wait complete. 2024-04-25T13:50:31.054 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138ea7e640 1 Processor -- start 2024-04-25T13:50:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138ea7e640 1 -- start start 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138ea7e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 0x7f1388072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138ea7e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138806fcf0 con 0x7f138806f5a0 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138ea7e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13880b0c60 con 0x7f1388161240 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138ea7e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1388073f10 con 0x7f1388161650 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138c81a640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f138806f5a0 0x7f1388160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52966/0 (socket says 172.21.15.80:52966) 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138c81a640 1 -- 172.21.15.80:0/3885805181 learned_addr learned my addr 172.21.15.80:0/3885805181 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138d01b640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 0x7f1388072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2790247724 0 0) 0x7f138806fcf0 con 0x7f138806f5a0 2024-04-25T13:50:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1370005740 con 0x7f138806f5a0 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2673124681 0 0) 0x7f1370005740 con 0x7f138806f5a0 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 msgr2=0x7f1388072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 0x7f1388072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 >> v1:172.21.15.87:6789/0 conn(0x7f1388161240 legacy=0x7f13880723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13880b0f40 con 0x7f138806f5a0 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f137c001680 con 0x7f138806f5a0 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 >> v1:172.21.15.80:6789/0 conn(0x7f138806f5a0 legacy=0x7f1388160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1370005c00 0x7f1370006030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1370006a20 0x7f1370006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 0x7f1370007420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1388073f10 con 0x7f1370005c00 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f137000dd10 con 0x7f1370006a20 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f137000df10 con 0x7f1388161650 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138c81a640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1370005c00 0x7f1370006030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f1387fff640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1370006a20 0x7f1370006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138d01b640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 0x7f1370007420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138c81a640 1 -- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 msgr2=0x7f1370007420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138c81a640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 0x7f1370007420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138c81a640 1 -- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1370006a20 msgr2=0x7f1370006e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138c81a640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1370006a20 0x7f1370006e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138c81a640 1 -- 172.21.15.80:0/3885805181 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1370005740 con 0x7f1370005c00 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f1387fff640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1370006a20 0x7f1370006e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:31.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.208+0000 7f138d01b640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1388161650 0x7f1370007420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f138c81a640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1370005c00 0x7f1370006030 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f137c00c290 tx=0x7f137c01cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e55d1962f10005e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c007d60 con 0x7f1370005c00 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137c0a9070 con 0x7f1370005c00 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f13877fe640 1 -- 172.21.15.80:0/3885805181 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c005d40 con 0x7f1370005c00 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f138ea7e640 1 -- 172.21.15.80:0/3885805181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1370005c00 msgr2=0x7f1370006030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f138ea7e640 1 --2- 172.21.15.80:0/3885805181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1370005c00 0x7f1370006030 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f137c00c290 tx=0x7f137c01cd20 comp rx=0 tx=0).stop 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f138d01b640 1 -- 172.21.15.80:0/3885805181 reap_dead start 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f138ea7e640 1 -- 172.21.15.80:0/3885805181 shutdown_connections 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f138ea7e640 1 -- 172.21.15.80:0/3885805181 >> 172.21.15.80:0/3885805181 conn(0x7f138806d080 msgr2=0x7f13880b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f138ea7e640 1 -- 172.21.15.80:0/3885805181 shutdown_connections 2024-04-25T13:50:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.212+0000 7f138ea7e640 1 -- 172.21.15.80:0/3885805181 wait complete. 2024-04-25T13:50:31.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 Processor -- start 2024-04-25T13:50:31.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 -- start start 2024-04-25T13:50:31.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1388153d40 0x7f13881453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138815b8b0 0x7f13880d0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13880d0980 0x7f1388156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1388073650 con 0x7f138815b8b0 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13880734d0 con 0x7f1388153d40 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f138806e490 con 0x7f13880d0980 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138815b8b0 0x7f13880d0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138815b8b0 0x7f13880d0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54554/0 (socket says 172.21.15.80:54554) 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 -- 172.21.15.80:0/3242310418 learned_addr learned my addr 172.21.15.80:0/3242310418 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138c81a640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1388153d40 0x7f13881453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138d01b640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13880d0980 0x7f1388156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 -- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13880d0980 msgr2=0x7f1388156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13880d0980 0x7f1388156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 -- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1388153d40 msgr2=0x7f13881453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1388153d40 0x7f13881453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 -- 172.21.15.80:0/3242310418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13881546b0 con 0x7f138815b8b0 2024-04-25T13:50:31.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f1387fff640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138815b8b0 0x7f13880d0440 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f137800db60 tx=0x7f137800df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f13857fa640 1 -- 172.21.15.80:0/3242310418 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378003bb0 con 0x7f138815b8b0 2024-04-25T13:50:31.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f13857fa640 1 -- 172.21.15.80:0/3242310418 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1378003d50 con 0x7f138815b8b0 2024-04-25T13:50:31.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1388154150 con 0x7f138815b8b0 2024-04-25T13:50:31.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f13857fa640 1 -- 172.21.15.80:0/3242310418 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378006d80 con 0x7f138815b8b0 2024-04-25T13:50:31.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.216+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13881543d0 con 0x7f138815b8b0 2024-04-25T13:50:31.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.220+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f138806a3a0 con 0x7f138815b8b0 2024-04-25T13:50:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.220+0000 7f13857fa640 1 -- 172.21.15.80:0/3242310418 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f137801a020 con 0x7f138815b8b0 2024-04-25T13:50:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.220+0000 7f13857fa640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.220+0000 7f13857fa640 1 -- 172.21.15.80:0/3242310418 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f137808f8d0 con 0x7f138815b8b0 2024-04-25T13:50:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.220+0000 7f138c81a640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.220+0000 7f138c81a640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f137c0ab000 tx=0x7f137c0b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.224+0000 7f13857fa640 1 -- 172.21.15.80:0/3242310418 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1388154150 con 0x7f138815b8b0 2024-04-25T13:50:31.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.384+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13881596b0 con 0x7f138815b8b0 2024-04-25T13:50:31.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.388+0000 7f13857fa640 1 -- 172.21.15.80:0/3242310418 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16621 (secure 0 0 0) 0x7f13881546b0 con 0x7f138815b8b0 2024-04-25T13:50:31.387 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:31.387 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":170,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:50:30.371584+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:50:28.108369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T13:50:29.324161+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 msgr2=0x7f1374071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f137c0ab000 tx=0x7f137c0b0040 comp rx=0 tx=0).stop 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138815b8b0 msgr2=0x7f13880d0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138815b8b0 0x7f13880d0440 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f137800db60 tx=0x7f137800df60 comp rx=0 tx=0).stop 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 shutdown_connections 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13880d0980 0x7f1388156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f138815b8b0 0x7f13880d0440 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 --2- 172.21.15.80:0/3242310418 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1388153d40 0x7f13881453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 >> 172.21.15.80:0/3242310418 conn(0x7f138806d080 msgr2=0x7f13880cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:31.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 shutdown_connections 2024-04-25T13:50:31.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.392+0000 7f138ea7e640 1 -- 172.21.15.80:0/3242310418 wait complete. 2024-04-25T13:50:31.410 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f5c04a640 1 Processor -- start 2024-04-25T13:50:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f5c04a640 1 -- start start 2024-04-25T13:50:31.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f5c04a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7f540b0ed0 0x7f7f5415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f5c04a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f5415cda0 con 0x7f7f540ce730 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f5c04a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f54073710 con 0x7f7f5406eba0 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f5c04a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f7f540739f0 con 0x7f7f540b0ed0 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f59de6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f7f540b0ed0 0x7f7f5415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f595e5640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7f540ce730 0x7f7f5406e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52970/0 (socket says 172.21.15.80:52970) 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f595e5640 1 -- 172.21.15.80:0/2049481205 learned_addr learned my addr 172.21.15.80:0/2049481205 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f58de4640 1 -- 172.21.15.80:0/2049481205 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3870251293 0 0) 0x7f7f5415cda0 con 0x7f7f540ce730 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f58de4640 1 -- 172.21.15.80:0/2049481205 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7f40003530 con 0x7f7f540ce730 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f59de6640 1 -- 172.21.15.80:0/2049481205 >> v1:172.21.15.87:6789/0 conn(0x7f7f5406eba0 legacy=0x7f7f541608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f59de6640 1 -- 172.21.15.80:0/2049481205 >> v1:172.21.15.80:6789/0 conn(0x7f7f540ce730 legacy=0x7f7f5406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.564+0000 7f7f59de6640 1 -- 172.21.15.80:0/2049481205 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f54073cd0 con 0x7f7f540b0ed0 2024-04-25T13:50:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f59de6640 1 --2- 172.21.15.80:0/2049481205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5415c860 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f7f500097c0 tx=0x7f7f5001ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdde064d7f6a4ad9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f58de4640 1 -- 172.21.15.80:0/2049481205 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500b5070 con 0x7f7f540b0ed0 2024-04-25T13:50:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f58de4640 1 -- 172.21.15.80:0/2049481205 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f500a8990 con 0x7f7f540b0ed0 2024-04-25T13:50:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f58de4640 1 -- 172.21.15.80:0/2049481205 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500a8d40 con 0x7f7f540b0ed0 2024-04-25T13:50:31.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f5c04a640 1 -- 172.21.15.80:0/2049481205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 msgr2=0x7f7f5415c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f5c04a640 1 --2- 172.21.15.80:0/2049481205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5415c860 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f7f500097c0 tx=0x7f7f5001ec90 comp rx=0 tx=0).stop 2024-04-25T13:50:31.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f5c04a640 1 -- 172.21.15.80:0/2049481205 shutdown_connections 2024-04-25T13:50:31.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f5c04a640 1 --2- 172.21.15.80:0/2049481205 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5415c860 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f5c04a640 1 -- 172.21.15.80:0/2049481205 >> 172.21.15.80:0/2049481205 conn(0x7f7f5406d080 msgr2=0x7f7f540b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:31.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f5c04a640 1 -- 172.21.15.80:0/2049481205 shutdown_connections 2024-04-25T13:50:31.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.568+0000 7f7f5c04a640 1 -- 172.21.15.80:0/2049481205 wait complete. 2024-04-25T13:50:31.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 Processor -- start 2024-04-25T13:50:31.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 -- start start 2024-04-25T13:50:31.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5414d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f540ce730 0x7f7f540cef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f541520a0 0x7f7f540cf450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7f540733a0 con 0x7f7f540b0ed0 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7f540b12e0 con 0x7f7f540ce730 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7f540ceb40 con 0x7f7f541520a0 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5414d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5414d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54570/0 (socket says 172.21.15.80:54570) 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 -- 172.21.15.80:0/1413506888 learned_addr learned my addr 172.21.15.80:0/1413506888 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5a5e7640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f541520a0 0x7f7f540cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f595e5640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f540ce730 0x7f7f540cef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 -- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f541520a0 msgr2=0x7f7f540cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f541520a0 0x7f7f540cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 -- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f540ce730 msgr2=0x7f7f540cef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f540ce730 0x7f7f540cef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 -- 172.21.15.80:0/1413506888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f541fa910 con 0x7f7f540b0ed0 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5a5e7640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f541520a0 0x7f7f540cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f59de6640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5414d6b0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f7f500097c0 tx=0x7f7f5001ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f4a7fc640 1 -- 172.21.15.80:0/1413506888 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500c0070 con 0x7f7f540b0ed0 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f4a7fc640 1 -- 172.21.15.80:0/1413506888 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f50004230 con 0x7f7f540b0ed0 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f541fa070 con 0x7f7f540b0ed0 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f4a7fc640 1 -- 172.21.15.80:0/1413506888 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f500b5040 con 0x7f7f540b0ed0 2024-04-25T13:50:31.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f541fa630 con 0x7f7f540b0ed0 2024-04-25T13:50:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.572+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f5406a3a0 con 0x7f7f540b0ed0 2024-04-25T13:50:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.576+0000 7f7f4a7fc640 1 -- 172.21.15.80:0/1413506888 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7f500c3070 con 0x7f7f540b0ed0 2024-04-25T13:50:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.576+0000 7f7f4a7fc640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f3806f400 0x7f7f38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.576+0000 7f7f4a7fc640 1 -- 172.21.15.80:0/1413506888 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f7f5012b9b0 con 0x7f7f540b0ed0 2024-04-25T13:50:31.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.576+0000 7f7f595e5640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f3806f400 0x7f7f38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.576+0000 7f7f595e5640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f3806f400 0x7f7f38071910 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7f540cdf60 tx=0x7f7f44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.580+0000 7f7f4a7fc640 1 -- 172.21.15.80:0/1413506888 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7f500187f0 con 0x7f7f540b0ed0 2024-04-25T13:50:31.742 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.744+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f5406f880 con 0x7f7f540b0ed0 2024-04-25T13:50:31.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f4a7fc640 1 -- 172.21.15.80:0/1413506888 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16621 (secure 0 0 0) 0x7f7f500f88a0 con 0x7f7f540b0ed0 2024-04-25T13:50:31.745 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:31.745 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":170,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:50:30.371584+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:50:28.108369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T13:50:29.324161+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f3806f400 msgr2=0x7f7f38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f3806f400 0x7f7f38071910 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f7f540cdf60 tx=0x7f7f44009040 comp rx=0 tx=0).stop 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 msgr2=0x7f7f5414d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5414d6b0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f7f500097c0 tx=0x7f7f5001ba30 comp rx=0 tx=0).stop 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 shutdown_connections 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7f3806f400 0x7f7f38071910 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7f541520a0 0x7f7f540cf450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f540ce730 0x7f7f540cef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 --2- 172.21.15.80:0/1413506888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7f540b0ed0 0x7f7f5414d6b0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 >> 172.21.15.80:0/1413506888 conn(0x7f7f5406d080 msgr2=0x7f7f540d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 shutdown_connections 2024-04-25T13:50:31.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.748+0000 7f7f5c04a640 1 -- 172.21.15.80:0/1413506888 wait complete. 2024-04-25T13:50:31.769 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:50:31.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d69601640 1 Processor -- start 2024-04-25T13:50:31.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d69601640 1 -- start start 2024-04-25T13:50:31.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d69601640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3d64071e60 0x7f3d641574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d69601640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3d64161170 0x7f3d640b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d69601640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d6406fbf0 con 0x7f3d64073c40 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d69601640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3d640b1350 con 0x7f3d64071e60 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d69601640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3d6406f9a0 con 0x7f3d64161170 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d62ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3d64071e60 0x7f3d641574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d5bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f3d64073c40 0x7f3d64160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52984/0 (socket says 172.21.15.80:52984) 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d62ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3d64071e60 0x7f3d641574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54584/0 (socket says 172.21.15.80:54584) 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d5bfff640 1 -- 172.21.15.80:0/2907802602 learned_addr learned my addr 172.21.15.80:0/2907802602 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d637fe640 1 --2- 172.21.15.80:0/2907802602 >> v2:172.21.15.87:3300/0 conn(0x7f3d64161170 0x7f3d640b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d627fc640 1 -- 172.21.15.80:0/2907802602 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215817495 0 0) 0x7f3d6406fbf0 con 0x7f3d64073c40 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d627fc640 1 -- 172.21.15.80:0/2907802602 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3d50005740 con 0x7f3d64073c40 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d62ffd640 1 -- 172.21.15.80:0/2907802602 >> v2:172.21.15.87:3300/0 conn(0x7f3d64161170 msgr2=0x7f3d640b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d62ffd640 1 --2- 172.21.15.80:0/2907802602 >> v2:172.21.15.87:3300/0 conn(0x7f3d64161170 0x7f3d640b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d62ffd640 1 -- 172.21.15.80:0/2907802602 >> v1:172.21.15.80:6789/0 conn(0x7f3d64073c40 legacy=0x7f3d64160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.924+0000 7f3d62ffd640 1 -- 172.21.15.80:0/2907802602 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d640b0a20 con 0x7f3d64071e60 2024-04-25T13:50:31.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d62ffd640 1 --2- 172.21.15.80:0/2907802602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64071e60 0x7f3d641574c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f3d64161a50 tx=0x7f3d5401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766a2cf00fb18f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d627fc640 1 -- 172.21.15.80:0/2907802602 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d540b0070 con 0x7f3d64071e60 2024-04-25T13:50:31.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d627fc640 1 -- 172.21.15.80:0/2907802602 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d540ab600 con 0x7f3d64071e60 2024-04-25T13:50:31.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d627fc640 1 -- 172.21.15.80:0/2907802602 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d540ab9d0 con 0x7f3d64071e60 2024-04-25T13:50:31.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d69601640 1 -- 172.21.15.80:0/2907802602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64071e60 msgr2=0x7f3d641574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d69601640 1 --2- 172.21.15.80:0/2907802602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64071e60 0x7f3d641574c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f3d64161a50 tx=0x7f3d5401c480 comp rx=0 tx=0).stop 2024-04-25T13:50:31.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d69601640 1 -- 172.21.15.80:0/2907802602 shutdown_connections 2024-04-25T13:50:31.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d69601640 1 --2- 172.21.15.80:0/2907802602 >> v2:172.21.15.87:3300/0 conn(0x7f3d64161170 0x7f3d640b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d69601640 1 --2- 172.21.15.80:0/2907802602 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64071e60 0x7f3d641574c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d69601640 1 -- 172.21.15.80:0/2907802602 >> 172.21.15.80:0/2907802602 conn(0x7f3d640b4160 msgr2=0x7f3d640d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:31.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d69601640 1 -- 172.21.15.80:0/2907802602 shutdown_connections 2024-04-25T13:50:31.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.928+0000 7f3d69601640 1 -- 172.21.15.80:0/2907802602 wait complete. 2024-04-25T13:50:31.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 Processor -- start 2024-04-25T13:50:31.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 -- start start 2024-04-25T13:50:31.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d64071e60 0x7f3d64147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64073c40 0x7f3d641553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d641558e0 0x7f3d6414eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6406e470 con 0x7f3d64073c40 2024-04-25T13:50:31.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3d64074050 con 0x7f3d64071e60 2024-04-25T13:50:31.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3d6406e5f0 con 0x7f3d641558e0 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64073c40 0x7f3d641553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64073c40 0x7f3d641553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54586/0 (socket says 172.21.15.80:54586) 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 -- 172.21.15.80:0/923705815 learned_addr learned my addr 172.21.15.80:0/923705815 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d62ffd640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d64071e60 0x7f3d64147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d637fe640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d641558e0 0x7f3d6414eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 -- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d641558e0 msgr2=0x7f3d6414eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d641558e0 0x7f3d6414eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 -- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d64071e60 msgr2=0x7f3d64147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d64071e60 0x7f3d64147bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 -- 172.21.15.80:0/923705815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d64148100 con 0x7f3d64073c40 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5bfff640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64073c40 0x7f3d641553a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c420 tx=0x7f3d4c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5b7fe640 1 -- 172.21.15.80:0/923705815 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c017070 con 0x7f3d64073c40 2024-04-25T13:50:31.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5b7fe640 1 -- 172.21.15.80:0/923705815 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d4c005cf0 con 0x7f3d64073c40 2024-04-25T13:50:31.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d5b7fe640 1 -- 172.21.15.80:0/923705815 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d4c0072d0 con 0x7f3d64073c40 2024-04-25T13:50:31.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6414fd30 con 0x7f3d64073c40 2024-04-25T13:50:31.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d640cfc20 con 0x7f3d64073c40 2024-04-25T13:50:31.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.932+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6406a3a0 con 0x7f3d64073c40 2024-04-25T13:50:31.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.936+0000 7f3d5b7fe640 1 -- 172.21.15.80:0/923705815 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3d4c01a070 con 0x7f3d64073c40 2024-04-25T13:50:31.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.936+0000 7f3d5b7fe640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d4406f430 0x7f3d44071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:31.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.936+0000 7f3d5b7fe640 1 -- 172.21.15.80:0/923705815 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5709+0+0 (secure 0 0 0) 0x7f3d4c08e510 con 0x7f3d64073c40 2024-04-25T13:50:31.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.936+0000 7f3d62ffd640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d4406f430 0x7f3d44071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:31.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.936+0000 7f3d62ffd640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d4406f430 0x7f3d44071940 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3d540ad000 tx=0x7f3d54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:31.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:31.940+0000 7f3d5b7fe640 1 -- 172.21.15.80:0/923705815 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3d64148100 con 0x7f3d64073c40 2024-04-25T13:50:32.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:32.140+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3d640d1400 con 0x7f3d64073c40 2024-04-25T13:50:33.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.036+0000 7f3d5b7fe640 1 -- 172.21.15.80:0/923705815 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) ==== 118+0+0 (secure 0 0 0) 0x7f3d4c05adb0 con 0x7f3d64073c40 2024-04-25T13:50:33.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d4406f430 msgr2=0x7f3d44071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d4406f430 0x7f3d44071940 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3d540ad000 tx=0x7f3d54008040 comp rx=0 tx=0).stop 2024-04-25T13:50:33.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64073c40 msgr2=0x7f3d641553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64073c40 0x7f3d641553a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f3d4c00c420 tx=0x7f3d4c00c8e0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 shutdown_connections 2024-04-25T13:50:33.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3d4406f430 0x7f3d44071940 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3d641558e0 0x7f3d6414eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3d64073c40 0x7f3d641553a0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 --2- 172.21.15.80:0/923705815 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3d64071e60 0x7f3d64147bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 >> 172.21.15.80:0/923705815 conn(0x7f3d640b4160 msgr2=0x7f3d640757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 shutdown_connections 2024-04-25T13:50:33.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.040+0000 7f3d69601640 1 -- 172.21.15.80:0/923705815 wait complete. 2024-04-25T13:50:33.080 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:50:33.081 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:50:33.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16d1de2640 1 Processor -- start 2024-04-25T13:50:33.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16d1de2640 1 -- start start 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16d1de2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc0b5330 0x7f16cc0d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16d1de2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16cc0d0ff0 0x7f16cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16d1de2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16cc0b5740 con 0x7f16cc06f2f0 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16d1de2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f16cc06f700 con 0x7f16cc0d0ff0 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16d1de2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f16cc1629b0 con 0x7f16cc0b5330 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16c3fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16cc0d0ff0 0x7f16cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16cb7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc0b5330 0x7f16cc0d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16cbfff640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f16cc06f2f0 0x7f16cc161b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:53510/0 (socket says 172.21.15.80:53510) 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16c3fff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f16cc0d0ff0 0x7f16cc06edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:37962/0 (socket says 172.21.15.80:37962) 2024-04-25T13:50:33.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16cb7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc0b5330 0x7f16cc0d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:56616/0 (socket says 172.21.15.80:56616) 2024-04-25T13:50:33.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16cbfff640 1 -- 172.21.15.80:0/2979319837 learned_addr learned my addr 172.21.15.80:0/2979319837 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:33.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16c3fff640 1 -- 172.21.15.80:0/2979319837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc0b5330 msgr2=0x7f16cc0d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:33.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16c3fff640 1 --2- 172.21.15.80:0/2979319837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc0b5330 0x7f16cc0d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16c3fff640 1 -- 172.21.15.80:0/2979319837 >> v1:172.21.15.87:6789/0 conn(0x7f16cc06f2f0 legacy=0x7f16cc161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.240+0000 7f16c3fff640 1 -- 172.21.15.80:0/2979319837 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc06dd10 con 0x7f16cc0d0ff0 2024-04-25T13:50:33.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16c3fff640 1 --2- 172.21.15.80:0/2979319837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 0x7f16cc06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f16b40097c0 tx=0x7f16b401c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e3b336b286e573e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16caffd640 1 -- 172.21.15.80:0/2979319837 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40b0070 con 0x7f16cc0d0ff0 2024-04-25T13:50:33.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16caffd640 1 -- 172.21.15.80:0/2979319837 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b40ab600 con 0x7f16cc0d0ff0 2024-04-25T13:50:33.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16caffd640 1 -- 172.21.15.80:0/2979319837 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40ab9d0 con 0x7f16cc0d0ff0 2024-04-25T13:50:33.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16d1de2640 1 -- 172.21.15.80:0/2979319837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 msgr2=0x7f16cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16d1de2640 1 --2- 172.21.15.80:0/2979319837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 0x7f16cc06edb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f16b40097c0 tx=0x7f16b401c410 comp rx=0 tx=0).stop 2024-04-25T13:50:33.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16d1de2640 1 -- 172.21.15.80:0/2979319837 shutdown_connections 2024-04-25T13:50:33.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16d1de2640 1 --2- 172.21.15.80:0/2979319837 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 0x7f16cc06edb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16d1de2640 1 --2- 172.21.15.80:0/2979319837 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc0b5330 0x7f16cc0d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16d1de2640 1 -- 172.21.15.80:0/2979319837 >> 172.21.15.80:0/2979319837 conn(0x7f16cc0b4160 msgr2=0x7f16cc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:33.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16d1de2640 1 -- 172.21.15.80:0/2979319837 shutdown_connections 2024-04-25T13:50:33.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.244+0000 7f16d1de2640 1 -- 172.21.15.80:0/2979319837 wait complete. 2024-04-25T13:50:33.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 Processor -- start 2024-04-25T13:50:33.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 -- start start 2024-04-25T13:50:33.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc06f2f0 0x7f16cc1383f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16cc0b5330 0x7f16cc15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 0x7f16cc160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc0b0cd0 con 0x7f16cc0b5330 2024-04-25T13:50:33.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f16cc06e890 con 0x7f16cc0d0ff0 2024-04-25T13:50:33.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f16cc0b0e50 con 0x7f16cc06f2f0 2024-04-25T13:50:33.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16cc0b5330 0x7f16cc15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16cc0b5330 0x7f16cc15fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54594/0 (socket says 172.21.15.80:54594) 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 -- 172.21.15.80:0/148609355 learned_addr learned my addr 172.21.15.80:0/148609355 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16cb7fe640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc06f2f0 0x7f16cc1383f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16cbfff640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 0x7f16cc160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 -- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc06f2f0 msgr2=0x7f16cc1383f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc06f2f0 0x7f16cc1383f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 -- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 msgr2=0x7f16cc160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 0x7f16cc160310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 -- 172.21.15.80:0/148609355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16cc1474b0 con 0x7f16cc0b5330 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16cb7fe640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc06f2f0 0x7f16cc1383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:33.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c3fff640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16cc0b5330 0x7f16cc15fdd0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f16b401aac0 tx=0x7f16b40191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c8ff9640 1 -- 172.21.15.80:0/148609355 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40b0040 con 0x7f16cc0b5330 2024-04-25T13:50:33.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c8ff9640 1 -- 172.21.15.80:0/148609355 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b40bb070 con 0x7f16cc0b5330 2024-04-25T13:50:33.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16cc145810 con 0x7f16cc0b5330 2024-04-25T13:50:33.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16c8ff9640 1 -- 172.21.15.80:0/148609355 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b40b6630 con 0x7f16cc0b5330 2024-04-25T13:50:33.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.248+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16cc13c960 con 0x7f16cc0b5330 2024-04-25T13:50:33.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.252+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16cc06a3a0 con 0x7f16cc0b5330 2024-04-25T13:50:33.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.252+0000 7f16c8ff9640 1 -- 172.21.15.80:0/148609355 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f16b40b3070 con 0x7f16cc0b5330 2024-04-25T13:50:33.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.252+0000 7f16c8ff9640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f16ac06f400 0x7f16ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:33.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.252+0000 7f16c8ff9640 1 -- 172.21.15.80:0/148609355 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f16b4128f10 con 0x7f16cc0b5330 2024-04-25T13:50:33.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.252+0000 7f16cb7fe640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f16ac06f400 0x7f16ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:33.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.252+0000 7f16cb7fe640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f16ac06f400 0x7f16ac071910 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009a40 tx=0x7f16bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:33.255 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.256+0000 7f16c8ff9640 1 -- 172.21.15.80:0/148609355 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f16b4016090 con 0x7f16cc0b5330 2024-04-25T13:50:33.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:33.456+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f16cc145d30 con 0x7f16cc0b5330 2024-04-25T13:50:34.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.040+0000 7f16c8ff9640 1 -- 172.21.15.80:0/148609355 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) ==== 122+0+0 (secure 0 0 0) 0x7f16b40f5e90 con 0x7f16cc0b5330 2024-04-25T13:50:34.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f16ac06f400 msgr2=0x7f16ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f16ac06f400 0x7f16ac071910 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f16bc009a40 tx=0x7f16bc008040 comp rx=0 tx=0).stop 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16cc0b5330 msgr2=0x7f16cc15fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16cc0b5330 0x7f16cc15fdd0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f16b401aac0 tx=0x7f16b40191b0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 shutdown_connections 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f16ac06f400 0x7f16ac071910 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16cc0d0ff0 0x7f16cc160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f16cc0b5330 0x7f16cc15fdd0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 --2- 172.21.15.80:0/148609355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f16cc06f2f0 0x7f16cc1383f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 >> 172.21.15.80:0/148609355 conn(0x7f16cc0b4160 msgr2=0x7f16cc0b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 shutdown_connections 2024-04-25T13:50:34.043 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.044+0000 7f16d1de2640 1 -- 172.21.15.80:0/148609355 wait complete. 2024-04-25T13:50:34.066 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:50:34.067 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:34.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1cc9ef640 1 Processor -- start 2024-04-25T13:50:34.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1cc9ef640 1 -- start start 2024-04-25T13:50:34.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1cc9ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c806f5a0 0x7fb1c8160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1cc9ef640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb1c80733b0 0x7fb1c8161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1cc9ef640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1c806fcf0 con 0x7fb1c8161070 2024-04-25T13:50:34.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1cc9ef640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb1c81619c0 con 0x7fb1c80733b0 2024-04-25T13:50:34.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1cc9ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c8073f10 con 0x7fb1c806f5a0 2024-04-25T13:50:34.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1c659c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c806f5a0 0x7fb1c8160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1c659c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c806f5a0 0x7fb1c8160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:56624/0 (socket says 172.21.15.80:56624) 2024-04-25T13:50:34.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1c659c640 1 -- 172.21.15.80:0/626657763 learned_addr learned my addr 172.21.15.80:0/626657763 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:34.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1c6d9d640 1 --2- 172.21.15.80:0/626657763 >> v2:172.21.15.87:3300/0 conn(0x7fb1c80733b0 0x7fb1c8161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1c659c640 1 -- 172.21.15.80:0/626657763 >> v2:172.21.15.87:3300/0 conn(0x7fb1c80733b0 msgr2=0x7fb1c8161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:34.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1c659c640 1 --2- 172.21.15.80:0/626657763 >> v2:172.21.15.87:3300/0 conn(0x7fb1c80733b0 0x7fb1c8161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1c659c640 1 -- 172.21.15.80:0/626657763 >> v1:172.21.15.87:6789/0 conn(0x7fb1c8161070 legacy=0x7fb1c80b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.220+0000 7fb1c659c640 1 -- 172.21.15.80:0/626657763 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c80b0c60 con 0x7fb1c806f5a0 2024-04-25T13:50:34.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1c659c640 1 --2- 172.21.15.80:0/626657763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c806f5a0 0x7fb1c8160b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009870 tx=0x7fb1b801c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c9dcc852546e51a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1c559a640 1 -- 172.21.15.80:0/626657763 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80b0070 con 0x7fb1c806f5a0 2024-04-25T13:50:34.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1c559a640 1 -- 172.21.15.80:0/626657763 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b80ab600 con 0x7fb1c806f5a0 2024-04-25T13:50:34.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1c559a640 1 -- 172.21.15.80:0/626657763 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80ab9d0 con 0x7fb1c806f5a0 2024-04-25T13:50:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/626657763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c806f5a0 msgr2=0x7fb1c8160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/626657763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c806f5a0 0x7fb1c8160b30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009870 tx=0x7fb1b801c4f0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/626657763 shutdown_connections 2024-04-25T13:50:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/626657763 >> v2:172.21.15.87:3300/0 conn(0x7fb1c80733b0 0x7fb1c8161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/626657763 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c806f5a0 0x7fb1c8160b30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/626657763 >> 172.21.15.80:0/626657763 conn(0x7fb1c806d080 msgr2=0x7fb1c80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/626657763 shutdown_connections 2024-04-25T13:50:34.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.224+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/626657763 wait complete. 2024-04-25T13:50:34.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 Processor -- start 2024-04-25T13:50:34.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 -- start start 2024-04-25T13:50:34.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c806f5a0 0x7fb1c8144630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c80733b0 0x7fb1c8141d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1c8161070 0x7fb1c8156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c816a050 con 0x7fb1c8161070 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8169ed0 con 0x7fb1c806f5a0 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c816a1d0 con 0x7fb1c80733b0 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1c8161070 0x7fb1c8156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1c8161070 0x7fb1c8156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54596/0 (socket says 172.21.15.80:54596) 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 -- 172.21.15.80:0/203505398 learned_addr learned my addr 172.21.15.80:0/203505398 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c659c640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c806f5a0 0x7fb1c8144630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c5d9b640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c80733b0 0x7fb1c8141d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 -- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c80733b0 msgr2=0x7fb1c8141d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c80733b0 0x7fb1c8141d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 -- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c806f5a0 msgr2=0x7fb1c8144630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c806f5a0 0x7fb1c8144630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 -- 172.21.15.80:0/203505398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c81422b0 con 0x7fb1c8161070 2024-04-25T13:50:34.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c5d9b640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c80733b0 0x7fb1c8141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c659c640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c806f5a0 0x7fb1c8144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1c6d9d640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1c8161070 0x7fb1c8156be0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000efc0 tx=0x7fb1c000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1b6ffd640 1 -- 172.21.15.80:0/203505398 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0018020 con 0x7fb1c8161070 2024-04-25T13:50:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1b6ffd640 1 -- 172.21.15.80:0/203505398 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1c0014480 con 0x7fb1c8161070 2024-04-25T13:50:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c8144f30 con 0x7fb1c8161070 2024-04-25T13:50:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1b6ffd640 1 -- 172.21.15.80:0/203505398 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0009e60 con 0x7fb1c8161070 2024-04-25T13:50:34.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.228+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c81454f0 con 0x7fb1c8161070 2024-04-25T13:50:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.232+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c8144ca0 con 0x7fb1c8161070 2024-04-25T13:50:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.232+0000 7fb1b6ffd640 1 -- 172.21.15.80:0/203505398 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb1c001b020 con 0x7fb1c8161070 2024-04-25T13:50:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.232+0000 7fb1b6ffd640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a806f400 0x7fb1a8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.232+0000 7fb1b6ffd640 1 -- 172.21.15.80:0/203505398 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7fb1c00905d0 con 0x7fb1c8161070 2024-04-25T13:50:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.232+0000 7fb1c659c640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a806f400 0x7fb1a8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.232+0000 7fb1c659c640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a806f400 0x7fb1a8071910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80b6e70 tx=0x7fb1b80a4f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.236+0000 7fb1b6ffd640 1 -- 172.21.15.80:0/203505398 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb1c0022030 con 0x7fb1c8161070 2024-04-25T13:50:34.432 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.432+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1c806a3a0 con 0x7fb1c8161070 2024-04-25T13:50:34.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1b6ffd640 1 -- 172.21.15.80:0/203505398 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5176 (secure 0 0 0) 0x7fb1c0010040 con 0x7fb1c8161070 2024-04-25T13:50:34.433 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:34.433 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/29039900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":29039900},{"type":"v1","addr":"172.21.15.87:6836","nonce":29039900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1466426643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1466426643},{"type":"v1","addr":"172.21.15.80:6833","nonce":1466426643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1170719891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1170719891},{"type":"v1","addr":"172.21.15.80:6835","nonce":1170719891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:50:30.371502+0000","modified":"2024-04-25T13:50:33.461986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/986211877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":986211877},{"type":"v1","addr":"172.21.15.87:6837","nonce":986211877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":10}]} 2024-04-25T13:50:34.434 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 81 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a806f400 msgr2=0x7fb1a8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a806f400 0x7fb1a8071910 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb1c80b6e70 tx=0x7fb1b80a4f90 comp rx=0 tx=0).stop 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1c8161070 msgr2=0x7fb1c8156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1c8161070 0x7fb1c8156be0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000efc0 tx=0x7fb1c000c490 comp rx=0 tx=0).stop 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 shutdown_connections 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb1a806f400 0x7fb1a8071910 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb1c8161070 0x7fb1c8156be0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb1c80733b0 0x7fb1c8141d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 --2- 172.21.15.80:0/203505398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb1c806f5a0 0x7fb1c8144630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 >> 172.21.15.80:0/203505398 conn(0x7fb1c806d080 msgr2=0x7fb1c80b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 shutdown_connections 2024-04-25T13:50:34.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.436+0000 7fb1cc9ef640 1 -- 172.21.15.80:0/203505398 wait complete. 2024-04-25T13:50:34.456 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:50:34.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.604+0000 7f63127ec640 1 Processor -- start 2024-04-25T13:50:34.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.604+0000 7f63127ec640 1 -- start start 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f63127ec640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f630c161f90 0x7f630c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f63127ec640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f630c0d0510 0x7f630c15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f63127ec640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f630c0d0d70 con 0x7f630c15ea70 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f63127ec640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f630c0d1050 con 0x7f630c161f90 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f63127ec640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f630c06dca0 con 0x7f630c0d0510 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f630bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f630c161f90 0x7f630c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f630bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f630c161f90 0x7f630c0cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54598/0 (socket says 172.21.15.80:54598) 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f630bfff640 1 -- 172.21.15.80:0/466546015 learned_addr learned my addr 172.21.15.80:0/466546015 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f63037fe640 1 --2- 172.21.15.80:0/466546015 >> v2:172.21.15.87:3300/0 conn(0x7f630c0d0510 0x7f630c15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f630bfff640 1 -- 172.21.15.80:0/466546015 >> v2:172.21.15.87:3300/0 conn(0x7f630c0d0510 msgr2=0x7f630c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f630bfff640 1 --2- 172.21.15.80:0/466546015 >> v2:172.21.15.87:3300/0 conn(0x7f630c0d0510 0x7f630c15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f630bfff640 1 -- 172.21.15.80:0/466546015 >> v1:172.21.15.87:6789/0 conn(0x7f630c15ea70 legacy=0x7f630c15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:34.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f630bfff640 1 -- 172.21.15.80:0/466546015 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c06df80 con 0x7f630c161f90 2024-04-25T13:50:34.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.608+0000 7f63037fe640 1 --2- 172.21.15.80:0/466546015 >> v2:172.21.15.87:3300/0 conn(0x7f630c0d0510 0x7f630c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:34.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f630bfff640 1 --2- 172.21.15.80:0/466546015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c0cffd0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f630c074290 tx=0x7f62fc01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76450bbb04f7d69d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f630b7fe640 1 -- 172.21.15.80:0/466546015 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0b0070 con 0x7f630c161f90 2024-04-25T13:50:34.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f630b7fe640 1 -- 172.21.15.80:0/466546015 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62fc0ab600 con 0x7f630c161f90 2024-04-25T13:50:34.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f630b7fe640 1 -- 172.21.15.80:0/466546015 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62fc0ab9d0 con 0x7f630c161f90 2024-04-25T13:50:34.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 -- 172.21.15.80:0/466546015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 msgr2=0x7f630c0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 --2- 172.21.15.80:0/466546015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c0cffd0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f630c074290 tx=0x7f62fc01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 -- 172.21.15.80:0/466546015 shutdown_connections 2024-04-25T13:50:34.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 --2- 172.21.15.80:0/466546015 >> v2:172.21.15.87:3300/0 conn(0x7f630c0d0510 0x7f630c15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 --2- 172.21.15.80:0/466546015 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c0cffd0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 -- 172.21.15.80:0/466546015 >> 172.21.15.80:0/466546015 conn(0x7f630c06d080 msgr2=0x7f630c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:34.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 -- 172.21.15.80:0/466546015 shutdown_connections 2024-04-25T13:50:34.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 -- 172.21.15.80:0/466546015 wait complete. 2024-04-25T13:50:34.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 Processor -- start 2024-04-25T13:50:34.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.612+0000 7f63127ec640 1 -- start start 2024-04-25T13:50:34.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f630c0d0510 0x7f630c0abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630c15ea70 0x7f630c1411d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c1670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f630c06f4b0 con 0x7f630c161f90 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f630c06f630 con 0x7f630c15ea70 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f630c0d0920 con 0x7f630c0d0510 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c1670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c1670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54612/0 (socket says 172.21.15.80:54612) 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 -- 172.21.15.80:0/3112244698 learned_addr learned my addr 172.21.15.80:0/3112244698 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f630bfff640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f630c0d0510 0x7f630c0abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63037fe640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630c15ea70 0x7f630c1411d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 -- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f630c0d0510 msgr2=0x7f630c0abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f630c0d0510 0x7f630c0abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 -- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630c15ea70 msgr2=0x7f630c1411d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630c15ea70 0x7f630c1411d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 -- 172.21.15.80:0/3112244698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f630c142100 con 0x7f630c161f90 2024-04-25T13:50:34.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f6310d89640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c1670b0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f630400b9e0 tx=0x7f630400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63097fa640 1 -- 172.21.15.80:0/3112244698 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304003b80 con 0x7f630c161f90 2024-04-25T13:50:34.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63097fa640 1 -- 172.21.15.80:0/3112244698 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6304003d20 con 0x7f630c161f90 2024-04-25T13:50:34.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f630c1423e0 con 0x7f630c161f90 2024-04-25T13:50:34.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63097fa640 1 -- 172.21.15.80:0/3112244698 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630400cb60 con 0x7f630c161f90 2024-04-25T13:50:34.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f630c141710 con 0x7f630c161f90 2024-04-25T13:50:34.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f630c06a3a0 con 0x7f630c161f90 2024-04-25T13:50:34.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.616+0000 7f63097fa640 1 -- 172.21.15.80:0/3112244698 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f6304006d10 con 0x7f630c161f90 2024-04-25T13:50:34.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.620+0000 7f63097fa640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62ec06f430 0x7f62ec071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.620+0000 7f63097fa640 1 -- 172.21.15.80:0/3112244698 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f630408ed10 con 0x7f630c161f90 2024-04-25T13:50:34.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.620+0000 7f630bfff640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62ec06f430 0x7f62ec071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.620+0000 7f630bfff640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62ec06f430 0x7f62ec071940 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0ad000 tx=0x7f62fc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.624+0000 7f63097fa640 1 -- 172.21.15.80:0/3112244698 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f630405bc90 con 0x7f630c161f90 2024-04-25T13:50:34.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.784+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f630c0ac200 con 0x7f630c161f90 2024-04-25T13:50:34.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63097fa640 1 -- 172.21.15.80:0/3112244698 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) ==== 74+0+16608 (secure 0 0 0) 0x7f630405b590 con 0x7f630c161f90 2024-04-25T13:50:34.786 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:34.786 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":171,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:50:31.977888+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:50:28.108369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T13:50:29.324161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:50:34.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62ec06f430 msgr2=0x7f62ec071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62ec06f430 0x7f62ec071940 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f62fc0ad000 tx=0x7f62fc0a6000 comp rx=0 tx=0).stop 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 msgr2=0x7f630c1670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c1670b0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f630400b9e0 tx=0x7f630400bea0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 shutdown_connections 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f62ec06f430 0x7f62ec071940 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f630c161f90 0x7f630c1670b0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f630c15ea70 0x7f630c1411d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 --2- 172.21.15.80:0/3112244698 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f630c0d0510 0x7f630c0abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 >> 172.21.15.80:0/3112244698 conn(0x7f630c06d080 msgr2=0x7f630c0b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 shutdown_connections 2024-04-25T13:50:34.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.788+0000 7f63127ec640 1 -- 172.21.15.80:0/3112244698 wait complete. 2024-04-25T13:50:34.809 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:50:34.810 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:50:34.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.944+0000 7f8016b44640 1 Processor -- start 2024-04-25T13:50:34.943 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.944+0000 7f8016b44640 1 -- start start 2024-04-25T13:50:34.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f8016b44640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f801006ebb0 0x7f801006efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f8016b44640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80100d0af0 con 0x7f80100d06e0 2024-04-25T13:50:34.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f8016b44640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80100d0d70 con 0x7f8010161f90 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f8016b44640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f80100d1050 con 0x7f801006ebb0 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f80150e1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f801006ebb0 0x7f801006efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f80150e1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f801006ebb0 0x7f801006efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54624/0 (socket says 172.21.15.80:54624) 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f80150e1640 1 -- 172.21.15.80:0/3724357613 learned_addr learned my addr 172.21.15.80:0/3724357613 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f800ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f80100d06e0 0x7f801006e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:52988/0 (socket says 172.21.15.80:52988) 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f80150e1640 1 -- 172.21.15.80:0/3724357613 >> v1:172.21.15.87:6789/0 conn(0x7f8010161f90 legacy=0x7f80100cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f80150e1640 1 -- 172.21.15.80:0/3724357613 >> v1:172.21.15.80:6789/0 conn(0x7f80100d06e0 legacy=0x7f801006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f800f7fe640 1 -- 172.21.15.80:0/3724357613 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1970937596 0 0) 0x7f7ff8001120 con 0x7f80100d06e0 2024-04-25T13:50:34.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.948+0000 7f80150e1640 1 -- 172.21.15.80:0/3724357613 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f801006dca0 con 0x7f801006ebb0 2024-04-25T13:50:34.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f80150e1640 1 --2- 172.21.15.80:0/3724357613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f801006efc0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8008009e50 tx=0x7f800801c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c114eec3063e23e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f800f7fe640 1 -- 172.21.15.80:0/3724357613 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80080a5720 con 0x7f801006ebb0 2024-04-25T13:50:34.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f800f7fe640 1 -- 172.21.15.80:0/3724357613 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80080a58c0 con 0x7f801006ebb0 2024-04-25T13:50:34.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f800f7fe640 1 -- 172.21.15.80:0/3724357613 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80080a5c90 con 0x7f801006ebb0 2024-04-25T13:50:34.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f8016b44640 1 -- 172.21.15.80:0/3724357613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 msgr2=0x7f801006efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f8016b44640 1 --2- 172.21.15.80:0/3724357613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f801006efc0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f8008009e50 tx=0x7f800801c8a0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f8016b44640 1 -- 172.21.15.80:0/3724357613 shutdown_connections 2024-04-25T13:50:34.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f8016b44640 1 --2- 172.21.15.80:0/3724357613 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f801006efc0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f8016b44640 1 -- 172.21.15.80:0/3724357613 >> 172.21.15.80:0/3724357613 conn(0x7f801006d080 msgr2=0x7f80100b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:34.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f8016b44640 1 -- 172.21.15.80:0/3724357613 shutdown_connections 2024-04-25T13:50:34.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.952+0000 7f8016b44640 1 -- 172.21.15.80:0/3724357613 wait complete. 2024-04-25T13:50:34.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 Processor -- start 2024-04-25T13:50:34.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 -- start start 2024-04-25T13:50:34.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f80100b5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80100d06e0 0x7f80100afa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8010161f90 0x7f8010140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f801006f680 con 0x7f801006ebb0 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f801006f500 con 0x7f80100d06e0 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8010144620 con 0x7f8010161f90 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f80100b5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f80100b5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54626/0 (socket says 172.21.15.80:54626) 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 -- 172.21.15.80:0/1779686978 learned_addr learned my addr 172.21.15.80:0/1779686978 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f800ffff640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80100d06e0 0x7f80100afa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80150e1640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8010161f90 0x7f8010140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 -- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8010161f90 msgr2=0x7f8010140640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8010161f90 0x7f8010140640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 -- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80100d06e0 msgr2=0x7f80100afa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80100d06e0 0x7f80100afa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:34.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 -- 172.21.15.80:0/1779686978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8010071e80 con 0x7f801006ebb0 2024-04-25T13:50:34.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f80148e0640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f80100b5e80 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f800000b3c0 tx=0x7f800000b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f800d7fa640 1 -- 172.21.15.80:0/1779686978 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000011830 con 0x7f801006ebb0 2024-04-25T13:50:34.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f800d7fa640 1 -- 172.21.15.80:0/1779686978 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8000016070 con 0x7f801006ebb0 2024-04-25T13:50:34.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8010146c60 con 0x7f801006ebb0 2024-04-25T13:50:34.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f800d7fa640 1 -- 172.21.15.80:0/1779686978 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000011c30 con 0x7f801006ebb0 2024-04-25T13:50:34.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.956+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8010147220 con 0x7f801006ebb0 2024-04-25T13:50:34.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.960+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f801006a3a0 con 0x7f801006ebb0 2024-04-25T13:50:34.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.960+0000 7f800d7fa640 1 -- 172.21.15.80:0/1779686978 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f800001b460 con 0x7f801006ebb0 2024-04-25T13:50:34.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.960+0000 7f800d7fa640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ff006f430 0x7f7ff0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:34.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.960+0000 7f800d7fa640 1 -- 172.21.15.80:0/1779686978 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f800008edf0 con 0x7f801006ebb0 2024-04-25T13:50:34.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.960+0000 7f800ffff640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ff006f430 0x7f7ff0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:34.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.960+0000 7f800ffff640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ff006f430 0x7f7ff0071940 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8000f70 tx=0x7f7ff8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:34.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:34.964+0000 7f800d7fa640 1 -- 172.21.15.80:0/1779686978 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8000020020 con 0x7f801006ebb0 2024-04-25T13:50:35.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.112+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80100b4ec0 con 0x7f7ff006f430 2024-04-25T13:50:35.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f800d7fa640 1 -- 172.21.15.80:0/1779686978 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247967 (secure 0 0 0) 0x7f80100b4ec0 con 0x7f7ff006f430 2024-04-25T13:50:35.133 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:35.136 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ff006f430 msgr2=0x7f7ff0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ff006f430 0x7f7ff0071940 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8000f70 tx=0x7f7ff8009040 comp rx=0 tx=0).stop 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 msgr2=0x7f80100b5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f80100b5e80 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f800000b3c0 tx=0x7f800000b880 comp rx=0 tx=0).stop 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 shutdown_connections 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7ff006f430 0x7f7ff0071940 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8010161f90 0x7f8010140640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80100d06e0 0x7f80100afa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 --2- 172.21.15.80:0/1779686978 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f801006ebb0 0x7f80100b5e80 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 >> 172.21.15.80:0/1779686978 conn(0x7f801006d080 msgr2=0x7f80100b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 shutdown_connections 2024-04-25T13:50:35.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.136+0000 7f8016b44640 1 -- 172.21.15.80:0/1779686978 wait complete. 2024-04-25T13:50:35.154 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-04-25T13:50:34.324406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553845,"statfs":{"total":1600000000,"available":1591138132,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374096"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041847+0000","last_change":"2024-04-25T13:50:31.012172+0000","last_active":"2024-04-25T13:50:32.041847+0000","last_peered":"2024-04-25T13:50:32.041847+0000","last_clean":"2024-04-25T13:50:32.041847+0000","last_became_active":"2024-04-25T13:50:31.010320+0000","last_became_peered":"2024-04-25T13:50:31.010320+0000","last_unstale":"2024-04-25T13:50:32.041847+0000","last_undegraded":"2024-04-25T13:50:32.041847+0000","last_fullsized":"2024-04-25T13:50:32.041847+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:31.642433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983278+0000","last_change":"2024-04-25T13:50:31.001777+0000","last_active":"2024-04-25T13:50:31.983278+0000","last_peered":"2024-04-25T13:50:31.983278+0000","last_clean":"2024-04-25T13:50:31.983278+0000","last_became_active":"2024-04-25T13:50:31.000850+0000","last_became_peered":"2024-04-25T13:50:31.000850+0000","last_unstale":"2024-04-25T13:50:31.983278+0000","last_undegraded":"2024-04-25T13:50:31.983278+0000","last_fullsized":"2024-04-25T13:50:31.983278+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:14.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882322+0000","last_change":"2024-04-25T13:50:31.027413+0000","last_active":"2024-04-25T13:50:32.882322+0000","last_peered":"2024-04-25T13:50:32.882322+0000","last_clean":"2024-04-25T13:50:32.882322+0000","last_became_active":"2024-04-25T13:50:31.024726+0000","last_became_peered":"2024-04-25T13:50:31.024726+0000","last_unstale":"2024-04-25T13:50:32.882322+0000","last_undegraded":"2024-04-25T13:50:32.882322+0000","last_fullsized":"2024-04-25T13:50:32.882322+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:04.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.982436+0000","last_change":"2024-04-25T13:50:31.024640+0000","last_active":"2024-04-25T13:50:31.982436+0000","last_peered":"2024-04-25T13:50:31.982436+0000","last_clean":"2024-04-25T13:50:31.982436+0000","last_became_active":"2024-04-25T13:50:31.023987+0000","last_became_peered":"2024-04-25T13:50:31.023987+0000","last_unstale":"2024-04-25T13:50:31.982436+0000","last_undegraded":"2024-04-25T13:50:31.982436+0000","last_fullsized":"2024-04-25T13:50:31.982436+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:37.652058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440773+0000","last_change":"2024-04-25T13:50:31.007799+0000","last_active":"2024-04-25T13:50:32.440773+0000","last_peered":"2024-04-25T13:50:32.440773+0000","last_clean":"2024-04-25T13:50:32.440773+0000","last_became_active":"2024-04-25T13:50:31.006933+0000","last_became_peered":"2024-04-25T13:50:31.006933+0000","last_unstale":"2024-04-25T13:50:32.440773+0000","last_undegraded":"2024-04-25T13:50:32.440773+0000","last_fullsized":"2024-04-25T13:50:32.440773+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:12.710289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041497+0000","last_change":"2024-04-25T13:50:31.024267+0000","last_active":"2024-04-25T13:50:32.041497+0000","last_peered":"2024-04-25T13:50:32.041497+0000","last_clean":"2024-04-25T13:50:32.041497+0000","last_became_active":"2024-04-25T13:50:31.023289+0000","last_became_peered":"2024-04-25T13:50:31.023289+0000","last_unstale":"2024-04-25T13:50:32.041497+0000","last_undegraded":"2024-04-25T13:50:32.041497+0000","last_fullsized":"2024-04-25T13:50:32.041497+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:51.248739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.015633+0000","last_change":"2024-04-25T13:50:31.015388+0000","last_active":"2024-04-25T13:50:31.015633+0000","last_peered":"2024-04-25T13:50:31.015633+0000","last_clean":"2024-04-25T13:50:31.015633+0000","last_became_active":"2024-04-25T13:50:31.014989+0000","last_became_peered":"2024-04-25T13:50:31.014989+0000","last_unstale":"2024-04-25T13:50:31.015633+0000","last_undegraded":"2024-04-25T13:50:31.015633+0000","last_fullsized":"2024-04-25T13:50:31.015633+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:33.621773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039597+0000","last_change":"2024-04-25T13:50:31.013881+0000","last_active":"2024-04-25T13:50:32.039597+0000","last_peered":"2024-04-25T13:50:32.039597+0000","last_clean":"2024-04-25T13:50:32.039597+0000","last_became_active":"2024-04-25T13:50:31.013456+0000","last_became_peered":"2024-04-25T13:50:31.013456+0000","last_unstale":"2024-04-25T13:50:32.039597+0000","last_undegraded":"2024-04-25T13:50:32.039597+0000","last_fullsized":"2024-04-25T13:50:32.039597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:59.725655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983237+0000","last_change":"2024-04-25T13:50:30.998855+0000","last_active":"2024-04-25T13:50:31.983237+0000","last_peered":"2024-04-25T13:50:31.983237+0000","last_clean":"2024-04-25T13:50:31.983237+0000","last_became_active":"2024-04-25T13:50:30.998104+0000","last_became_peered":"2024-04-25T13:50:30.998104+0000","last_unstale":"2024-04-25T13:50:31.983237+0000","last_undegraded":"2024-04-25T13:50:31.983237+0000","last_fullsized":"2024-04-25T13:50:31.983237+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:26.713936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041163+0000","last_change":"2024-04-25T13:50:31.020776+0000","last_active":"2024-04-25T13:50:32.041163+0000","last_peered":"2024-04-25T13:50:32.041163+0000","last_clean":"2024-04-25T13:50:32.041163+0000","last_became_active":"2024-04-25T13:50:31.020177+0000","last_became_peered":"2024-04-25T13:50:31.020177+0000","last_unstale":"2024-04-25T13:50:32.041163+0000","last_undegraded":"2024-04-25T13:50:32.041163+0000","last_fullsized":"2024-04-25T13:50:32.041163+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:48.134078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039521+0000","last_change":"2024-04-25T13:50:31.011107+0000","last_active":"2024-04-25T13:50:32.039521+0000","last_peered":"2024-04-25T13:50:32.039521+0000","last_clean":"2024-04-25T13:50:32.039521+0000","last_became_active":"2024-04-25T13:50:31.006997+0000","last_became_peered":"2024-04-25T13:50:31.006997+0000","last_unstale":"2024-04-25T13:50:32.039521+0000","last_undegraded":"2024-04-25T13:50:32.039521+0000","last_fullsized":"2024-04-25T13:50:32.039521+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.993316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040746+0000","last_change":"2024-04-25T13:50:31.002319+0000","last_active":"2024-04-25T13:50:32.040746+0000","last_peered":"2024-04-25T13:50:32.040746+0000","last_clean":"2024-04-25T13:50:32.040746+0000","last_became_active":"2024-04-25T13:50:31.001625+0000","last_became_peered":"2024-04-25T13:50:31.001625+0000","last_unstale":"2024-04-25T13:50:32.040746+0000","last_undegraded":"2024-04-25T13:50:32.040746+0000","last_fullsized":"2024-04-25T13:50:32.040746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:47.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.439833+0000","last_change":"2024-04-25T13:50:31.019662+0000","last_active":"2024-04-25T13:50:32.439833+0000","last_peered":"2024-04-25T13:50:32.439833+0000","last_clean":"2024-04-25T13:50:32.439833+0000","last_became_active":"2024-04-25T13:50:31.018290+0000","last_became_peered":"2024-04-25T13:50:31.018290+0000","last_unstale":"2024-04-25T13:50:32.439833+0000","last_undegraded":"2024-04-25T13:50:32.439833+0000","last_fullsized":"2024-04-25T13:50:32.439833+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:00.237513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.982929+0000","last_change":"2024-04-25T13:50:31.013869+0000","last_active":"2024-04-25T13:50:31.982929+0000","last_peered":"2024-04-25T13:50:31.982929+0000","last_clean":"2024-04-25T13:50:31.982929+0000","last_became_active":"2024-04-25T13:50:31.012935+0000","last_became_peered":"2024-04-25T13:50:31.012935+0000","last_unstale":"2024-04-25T13:50:31.982929+0000","last_undegraded":"2024-04-25T13:50:31.982929+0000","last_fullsized":"2024-04-25T13:50:31.982929+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:30.592154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041665+0000","last_change":"2024-04-25T13:50:31.000567+0000","last_active":"2024-04-25T13:50:32.041665+0000","last_peered":"2024-04-25T13:50:32.041665+0000","last_clean":"2024-04-25T13:50:32.041665+0000","last_became_active":"2024-04-25T13:50:30.998642+0000","last_became_peered":"2024-04-25T13:50:30.998642+0000","last_unstale":"2024-04-25T13:50:32.041665+0000","last_undegraded":"2024-04-25T13:50:32.041665+0000","last_fullsized":"2024-04-25T13:50:32.041665+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:45.288143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.008240+0000","last_change":"2024-04-25T13:50:31.007892+0000","last_active":"2024-04-25T13:50:31.008240+0000","last_peered":"2024-04-25T13:50:31.008240+0000","last_clean":"2024-04-25T13:50:31.008240+0000","last_became_active":"2024-04-25T13:50:31.007181+0000","last_became_peered":"2024-04-25T13:50:31.007181+0000","last_unstale":"2024-04-25T13:50:31.008240+0000","last_undegraded":"2024-04-25T13:50:31.008240+0000","last_fullsized":"2024-04-25T13:50:31.008240+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:38.400390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041768+0000","last_change":"2024-04-25T13:50:30.999492+0000","last_active":"2024-04-25T13:50:32.041768+0000","last_peered":"2024-04-25T13:50:32.041768+0000","last_clean":"2024-04-25T13:50:32.041768+0000","last_became_active":"2024-04-25T13:50:30.998823+0000","last_became_peered":"2024-04-25T13:50:30.998823+0000","last_unstale":"2024-04-25T13:50:32.041768+0000","last_undegraded":"2024-04-25T13:50:32.041768+0000","last_fullsized":"2024-04-25T13:50:32.041768+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:31.678708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882061+0000","last_change":"2024-04-25T13:50:31.028047+0000","last_active":"2024-04-25T13:50:32.882061+0000","last_peered":"2024-04-25T13:50:32.882061+0000","last_clean":"2024-04-25T13:50:32.882061+0000","last_became_active":"2024-04-25T13:50:31.026346+0000","last_became_peered":"2024-04-25T13:50:31.026346+0000","last_unstale":"2024-04-25T13:50:32.882061+0000","last_undegraded":"2024-04-25T13:50:32.882061+0000","last_fullsized":"2024-04-25T13:50:32.882061+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:15.578715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041049+0000","last_change":"2024-04-25T13:50:30.996109+0000","last_active":"2024-04-25T13:50:32.041049+0000","last_peered":"2024-04-25T13:50:32.041049+0000","last_clean":"2024-04-25T13:50:32.041049+0000","last_became_active":"2024-04-25T13:50:30.995321+0000","last_became_peered":"2024-04-25T13:50:30.995321+0000","last_unstale":"2024-04-25T13:50:32.041049+0000","last_undegraded":"2024-04-25T13:50:32.041049+0000","last_fullsized":"2024-04-25T13:50:32.041049+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.501947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.002059+0000","last_change":"2024-04-25T13:50:31.001713+0000","last_active":"2024-04-25T13:50:31.002059+0000","last_peered":"2024-04-25T13:50:31.002059+0000","last_clean":"2024-04-25T13:50:31.002059+0000","last_became_active":"2024-04-25T13:50:31.001058+0000","last_became_peered":"2024-04-25T13:50:31.001058+0000","last_unstale":"2024-04-25T13:50:31.002059+0000","last_undegraded":"2024-04-25T13:50:31.002059+0000","last_fullsized":"2024-04-25T13:50:31.002059+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:58.049180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041170+0000","last_change":"2024-04-25T13:50:31.020652+0000","last_active":"2024-04-25T13:50:32.041170+0000","last_peered":"2024-04-25T13:50:32.041170+0000","last_clean":"2024-04-25T13:50:32.041170+0000","last_became_active":"2024-04-25T13:50:31.018258+0000","last_became_peered":"2024-04-25T13:50:31.018258+0000","last_unstale":"2024-04-25T13:50:32.041170+0000","last_undegraded":"2024-04-25T13:50:32.041170+0000","last_fullsized":"2024-04-25T13:50:32.041170+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:45:28.863538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.982777+0000","last_change":"2024-04-25T13:50:31.002559+0000","last_active":"2024-04-25T13:50:31.982777+0000","last_peered":"2024-04-25T13:50:31.982777+0000","last_clean":"2024-04-25T13:50:31.982777+0000","last_became_active":"2024-04-25T13:50:31.001461+0000","last_became_peered":"2024-04-25T13:50:31.001461+0000","last_unstale":"2024-04-25T13:50:31.982777+0000","last_undegraded":"2024-04-25T13:50:31.982777+0000","last_fullsized":"2024-04-25T13:50:31.982777+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:27.188073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.981996+0000","last_change":"2024-04-25T13:50:31.012642+0000","last_active":"2024-04-25T13:50:31.981996+0000","last_peered":"2024-04-25T13:50:31.981996+0000","last_clean":"2024-04-25T13:50:31.981996+0000","last_became_active":"2024-04-25T13:50:31.011905+0000","last_became_peered":"2024-04-25T13:50:31.011905+0000","last_unstale":"2024-04-25T13:50:31.981996+0000","last_undegraded":"2024-04-25T13:50:31.981996+0000","last_fullsized":"2024-04-25T13:50:31.981996+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.681981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040793+0000","last_change":"2024-04-25T13:50:31.015893+0000","last_active":"2024-04-25T13:50:32.040793+0000","last_peered":"2024-04-25T13:50:32.040793+0000","last_clean":"2024-04-25T13:50:32.040793+0000","last_became_active":"2024-04-25T13:50:31.014514+0000","last_became_peered":"2024-04-25T13:50:31.014514+0000","last_unstale":"2024-04-25T13:50:32.040793+0000","last_undegraded":"2024-04-25T13:50:32.040793+0000","last_fullsized":"2024-04-25T13:50:32.040793+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:02.938329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.000619+0000","last_change":"2024-04-25T13:50:30.999929+0000","last_active":"2024-04-25T13:50:31.000619+0000","last_peered":"2024-04-25T13:50:31.000619+0000","last_clean":"2024-04-25T13:50:31.000619+0000","last_became_active":"2024-04-25T13:50:30.999229+0000","last_became_peered":"2024-04-25T13:50:30.999229+0000","last_unstale":"2024-04-25T13:50:31.000619+0000","last_undegraded":"2024-04-25T13:50:31.000619+0000","last_fullsized":"2024-04-25T13:50:31.000619+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:39.304701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039933+0000","last_change":"2024-04-25T13:50:31.009348+0000","last_active":"2024-04-25T13:50:32.039933+0000","last_peered":"2024-04-25T13:50:32.039933+0000","last_clean":"2024-04-25T13:50:32.039933+0000","last_became_active":"2024-04-25T13:50:31.005568+0000","last_became_peered":"2024-04-25T13:50:31.005568+0000","last_unstale":"2024-04-25T13:50:32.039933+0000","last_undegraded":"2024-04-25T13:50:32.039933+0000","last_fullsized":"2024-04-25T13:50:32.039933+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:22.902796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.028382+0000","last_change":"2024-04-25T13:50:31.028178+0000","last_active":"2024-04-25T13:50:31.028382+0000","last_peered":"2024-04-25T13:50:31.028382+0000","last_clean":"2024-04-25T13:50:31.028382+0000","last_became_active":"2024-04-25T13:50:31.027822+0000","last_became_peered":"2024-04-25T13:50:31.027822+0000","last_unstale":"2024-04-25T13:50:31.028382+0000","last_undegraded":"2024-04-25T13:50:31.028382+0000","last_fullsized":"2024-04-25T13:50:31.028382+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:23.360248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.030148+0000","last_change":"2024-04-25T13:50:31.029904+0000","last_active":"2024-04-25T13:50:31.030148+0000","last_peered":"2024-04-25T13:50:31.030148+0000","last_clean":"2024-04-25T13:50:31.030148+0000","last_became_active":"2024-04-25T13:50:31.029460+0000","last_became_peered":"2024-04-25T13:50:31.029460+0000","last_unstale":"2024-04-25T13:50:31.030148+0000","last_undegraded":"2024-04-25T13:50:31.030148+0000","last_fullsized":"2024-04-25T13:50:31.030148+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:26:38.848182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039934+0000","last_change":"2024-04-25T13:50:31.022460+0000","last_active":"2024-04-25T13:50:32.039934+0000","last_peered":"2024-04-25T13:50:32.039934+0000","last_clean":"2024-04-25T13:50:32.039934+0000","last_became_active":"2024-04-25T13:50:31.022046+0000","last_became_peered":"2024-04-25T13:50:31.022046+0000","last_unstale":"2024-04-25T13:50:32.039934+0000","last_undegraded":"2024-04-25T13:50:32.039934+0000","last_fullsized":"2024-04-25T13:50:32.039934+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:25.755869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983917+0000","last_change":"2024-04-25T13:50:31.018266+0000","last_active":"2024-04-25T13:50:31.983917+0000","last_peered":"2024-04-25T13:50:31.983917+0000","last_clean":"2024-04-25T13:50:31.983917+0000","last_became_active":"2024-04-25T13:50:31.017118+0000","last_became_peered":"2024-04-25T13:50:31.017118+0000","last_unstale":"2024-04-25T13:50:31.983917+0000","last_undegraded":"2024-04-25T13:50:31.983917+0000","last_fullsized":"2024-04-25T13:50:31.983917+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:07.002502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983708+0000","last_change":"2024-04-25T13:50:31.024842+0000","last_active":"2024-04-25T13:50:31.983708+0000","last_peered":"2024-04-25T13:50:31.983708+0000","last_clean":"2024-04-25T13:50:31.983708+0000","last_became_active":"2024-04-25T13:50:31.024397+0000","last_became_peered":"2024-04-25T13:50:31.024397+0000","last_unstale":"2024-04-25T13:50:31.983708+0000","last_undegraded":"2024-04-25T13:50:31.983708+0000","last_fullsized":"2024-04-25T13:50:31.983708+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:22.191877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.000393+0000","last_change":"2024-04-25T13:50:30.997873+0000","last_active":"2024-04-25T13:50:31.000393+0000","last_peered":"2024-04-25T13:50:31.000393+0000","last_clean":"2024-04-25T13:50:31.000393+0000","last_became_active":"2024-04-25T13:50:30.997166+0000","last_became_peered":"2024-04-25T13:50:30.997166+0000","last_unstale":"2024-04-25T13:50:31.000393+0000","last_undegraded":"2024-04-25T13:50:31.000393+0000","last_fullsized":"2024-04-25T13:50:31.000393+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:13.182795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041277+0000","last_change":"2024-04-25T13:50:31.002850+0000","last_active":"2024-04-25T13:50:32.041277+0000","last_peered":"2024-04-25T13:50:32.041277+0000","last_clean":"2024-04-25T13:50:32.041277+0000","last_became_active":"2024-04-25T13:50:31.002034+0000","last_became_peered":"2024-04-25T13:50:31.002034+0000","last_unstale":"2024-04-25T13:50:32.041277+0000","last_undegraded":"2024-04-25T13:50:32.041277+0000","last_fullsized":"2024-04-25T13:50:32.041277+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:47.411894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"170'4","reported_seq":18,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041672+0000","last_change":"2024-04-25T13:50:29.986919+0000","last_active":"2024-04-25T13:50:32.041672+0000","last_peered":"2024-04-25T13:50:32.041672+0000","last_clean":"2024-04-25T13:50:32.041672+0000","last_became_active":"2024-04-25T13:50:29.985997+0000","last_became_peered":"2024-04-25T13:50:29.985997+0000","last_unstale":"2024-04-25T13:50:32.041672+0000","last_undegraded":"2024-04-25T13:50:32.041672+0000","last_fullsized":"2024-04-25T13:50:32.041672+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.865528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983718+0000","last_change":"2024-04-25T13:50:31.004232+0000","last_active":"2024-04-25T13:50:31.983718+0000","last_peered":"2024-04-25T13:50:31.983718+0000","last_clean":"2024-04-25T13:50:31.983718+0000","last_became_active":"2024-04-25T13:50:31.002546+0000","last_became_peered":"2024-04-25T13:50:31.002546+0000","last_unstale":"2024-04-25T13:50:31.983718+0000","last_undegraded":"2024-04-25T13:50:31.983718+0000","last_fullsized":"2024-04-25T13:50:31.983718+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:37.748528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983371+0000","last_change":"2024-04-25T13:50:29.986327+0000","last_active":"2024-04-25T13:50:31.983371+0000","last_peered":"2024-04-25T13:50:31.983371+0000","last_clean":"2024-04-25T13:50:31.983371+0000","last_became_active":"2024-04-25T13:50:29.985800+0000","last_became_peered":"2024-04-25T13:50:29.985800+0000","last_unstale":"2024-04-25T13:50:31.983371+0000","last_undegraded":"2024-04-25T13:50:31.983371+0000","last_fullsized":"2024-04-25T13:50:31.983371+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:19.502813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040394+0000","last_change":"2024-04-25T13:50:31.017906+0000","last_active":"2024-04-25T13:50:32.040394+0000","last_peered":"2024-04-25T13:50:32.040394+0000","last_clean":"2024-04-25T13:50:32.040394+0000","last_became_active":"2024-04-25T13:50:31.017285+0000","last_became_peered":"2024-04-25T13:50:31.017285+0000","last_unstale":"2024-04-25T13:50:32.040394+0000","last_undegraded":"2024-04-25T13:50:32.040394+0000","last_fullsized":"2024-04-25T13:50:32.040394+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:58.941799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.984240+0000","last_change":"2024-04-25T13:50:29.968012+0000","last_active":"2024-04-25T13:50:31.984240+0000","last_peered":"2024-04-25T13:50:31.984240+0000","last_clean":"2024-04-25T13:50:31.984240+0000","last_became_active":"2024-04-25T13:50:29.967190+0000","last_became_peered":"2024-04-25T13:50:29.967190+0000","last_unstale":"2024-04-25T13:50:31.984240+0000","last_undegraded":"2024-04-25T13:50:31.984240+0000","last_fullsized":"2024-04-25T13:50:31.984240+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:06.734857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042117+0000","last_change":"2024-04-25T13:50:31.005454+0000","last_active":"2024-04-25T13:50:32.042117+0000","last_peered":"2024-04-25T13:50:32.042117+0000","last_clean":"2024-04-25T13:50:32.042117+0000","last_became_active":"2024-04-25T13:50:31.003905+0000","last_became_peered":"2024-04-25T13:50:31.003905+0000","last_unstale":"2024-04-25T13:50:32.042117+0000","last_undegraded":"2024-04-25T13:50:32.042117+0000","last_fullsized":"2024-04-25T13:50:32.042117+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:28.089421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983305+0000","last_change":"2024-04-25T13:50:29.970650+0000","last_active":"2024-04-25T13:50:31.983305+0000","last_peered":"2024-04-25T13:50:31.983305+0000","last_clean":"2024-04-25T13:50:31.983305+0000","last_became_active":"2024-04-25T13:50:29.966219+0000","last_became_peered":"2024-04-25T13:50:29.966219+0000","last_unstale":"2024-04-25T13:50:31.983305+0000","last_undegraded":"2024-04-25T13:50:31.983305+0000","last_fullsized":"2024-04-25T13:50:31.983305+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:23.011996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983882+0000","last_change":"2024-04-25T13:50:31.029728+0000","last_active":"2024-04-25T13:50:31.983882+0000","last_peered":"2024-04-25T13:50:31.983882+0000","last_clean":"2024-04-25T13:50:31.983882+0000","last_became_active":"2024-04-25T13:50:31.029358+0000","last_became_peered":"2024-04-25T13:50:31.029358+0000","last_unstale":"2024-04-25T13:50:31.983882+0000","last_undegraded":"2024-04-25T13:50:31.983882+0000","last_fullsized":"2024-04-25T13:50:31.983882+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:36.260411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.043633+0000","last_change":"2024-04-25T13:50:29.990982+0000","last_active":"2024-04-25T13:50:31.043633+0000","last_peered":"2024-04-25T13:50:31.043633+0000","last_clean":"2024-04-25T13:50:31.043633+0000","last_became_active":"2024-04-25T13:50:29.990575+0000","last_became_peered":"2024-04-25T13:50:29.990575+0000","last_unstale":"2024-04-25T13:50:31.043633+0000","last_undegraded":"2024-04-25T13:50:31.043633+0000","last_fullsized":"2024-04-25T13:50:31.043633+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:19.314109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983247+0000","last_change":"2024-04-25T13:50:31.028780+0000","last_active":"2024-04-25T13:50:31.983247+0000","last_peered":"2024-04-25T13:50:31.983247+0000","last_clean":"2024-04-25T13:50:31.983247+0000","last_became_active":"2024-04-25T13:50:31.028041+0000","last_became_peered":"2024-04-25T13:50:31.028041+0000","last_unstale":"2024-04-25T13:50:31.983247+0000","last_undegraded":"2024-04-25T13:50:31.983247+0000","last_fullsized":"2024-04-25T13:50:31.983247+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:54.366648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.441297+0000","last_change":"2024-04-25T13:50:29.989385+0000","last_active":"2024-04-25T13:50:32.441297+0000","last_peered":"2024-04-25T13:50:32.441297+0000","last_clean":"2024-04-25T13:50:32.441297+0000","last_became_active":"2024-04-25T13:50:29.988079+0000","last_became_peered":"2024-04-25T13:50:29.988079+0000","last_unstale":"2024-04-25T13:50:32.441297+0000","last_undegraded":"2024-04-25T13:50:32.441297+0000","last_fullsized":"2024-04-25T13:50:32.441297+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:06.797342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983712+0000","last_change":"2024-04-25T13:50:31.005945+0000","last_active":"2024-04-25T13:50:31.983712+0000","last_peered":"2024-04-25T13:50:31.983712+0000","last_clean":"2024-04-25T13:50:31.983712+0000","last_became_active":"2024-04-25T13:50:31.004315+0000","last_became_peered":"2024-04-25T13:50:31.004315+0000","last_unstale":"2024-04-25T13:50:31.983712+0000","last_undegraded":"2024-04-25T13:50:31.983712+0000","last_fullsized":"2024-04-25T13:50:31.983712+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:38.544004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:30.987263+0000","last_change":"2024-04-25T13:50:29.981694+0000","last_active":"2024-04-25T13:50:30.987263+0000","last_peered":"2024-04-25T13:50:30.987263+0000","last_clean":"2024-04-25T13:50:30.987263+0000","last_became_active":"2024-04-25T13:50:29.981449+0000","last_became_peered":"2024-04-25T13:50:29.981449+0000","last_unstale":"2024-04-25T13:50:30.987263+0000","last_undegraded":"2024-04-25T13:50:30.987263+0000","last_fullsized":"2024-04-25T13:50:30.987263+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:00.489684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440413+0000","last_change":"2024-04-25T13:50:31.012441+0000","last_active":"2024-04-25T13:50:32.440413+0000","last_peered":"2024-04-25T13:50:32.440413+0000","last_clean":"2024-04-25T13:50:32.440413+0000","last_became_active":"2024-04-25T13:50:31.011713+0000","last_became_peered":"2024-04-25T13:50:31.011713+0000","last_unstale":"2024-04-25T13:50:32.440413+0000","last_undegraded":"2024-04-25T13:50:32.440413+0000","last_fullsized":"2024-04-25T13:50:32.440413+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:02.170602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.984445+0000","last_change":"2024-04-25T13:50:29.986313+0000","last_active":"2024-04-25T13:50:31.984445+0000","last_peered":"2024-04-25T13:50:31.984445+0000","last_clean":"2024-04-25T13:50:31.984445+0000","last_became_active":"2024-04-25T13:50:29.985501+0000","last_became_peered":"2024-04-25T13:50:29.985501+0000","last_unstale":"2024-04-25T13:50:31.984445+0000","last_undegraded":"2024-04-25T13:50:31.984445+0000","last_fullsized":"2024-04-25T13:50:31.984445+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:13.292665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440767+0000","last_change":"2024-04-25T13:50:30.996986+0000","last_active":"2024-04-25T13:50:32.440767+0000","last_peered":"2024-04-25T13:50:32.440767+0000","last_clean":"2024-04-25T13:50:32.440767+0000","last_became_active":"2024-04-25T13:50:30.996107+0000","last_became_peered":"2024-04-25T13:50:30.996107+0000","last_unstale":"2024-04-25T13:50:32.440767+0000","last_undegraded":"2024-04-25T13:50:32.440767+0000","last_fullsized":"2024-04-25T13:50:32.440767+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:03.424841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.984006+0000","last_change":"2024-04-25T13:50:29.965299+0000","last_active":"2024-04-25T13:50:31.984006+0000","last_peered":"2024-04-25T13:50:31.984006+0000","last_clean":"2024-04-25T13:50:31.984006+0000","last_became_active":"2024-04-25T13:50:29.964930+0000","last_became_peered":"2024-04-25T13:50:29.964930+0000","last_unstale":"2024-04-25T13:50:31.984006+0000","last_undegraded":"2024-04-25T13:50:31.984006+0000","last_fullsized":"2024-04-25T13:50:31.984006+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:51.235418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041502+0000","last_change":"2024-04-25T13:50:31.005622+0000","last_active":"2024-04-25T13:50:32.041502+0000","last_peered":"2024-04-25T13:50:32.041502+0000","last_clean":"2024-04-25T13:50:32.041502+0000","last_became_active":"2024-04-25T13:50:31.004854+0000","last_became_peered":"2024-04-25T13:50:31.004854+0000","last_unstale":"2024-04-25T13:50:32.041502+0000","last_undegraded":"2024-04-25T13:50:32.041502+0000","last_fullsized":"2024-04-25T13:50:32.041502+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:16.788603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041293+0000","last_change":"2024-04-25T13:50:29.987794+0000","last_active":"2024-04-25T13:50:32.041293+0000","last_peered":"2024-04-25T13:50:32.041293+0000","last_clean":"2024-04-25T13:50:32.041293+0000","last_became_active":"2024-04-25T13:50:29.986932+0000","last_became_peered":"2024-04-25T13:50:29.986932+0000","last_unstale":"2024-04-25T13:50:32.041293+0000","last_undegraded":"2024-04-25T13:50:32.041293+0000","last_fullsized":"2024-04-25T13:50:32.041293+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:51.991758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.000455+0000","last_change":"2024-04-25T13:50:30.998950+0000","last_active":"2024-04-25T13:50:31.000455+0000","last_peered":"2024-04-25T13:50:31.000455+0000","last_clean":"2024-04-25T13:50:31.000455+0000","last_became_active":"2024-04-25T13:50:30.998222+0000","last_became_peered":"2024-04-25T13:50:30.998222+0000","last_unstale":"2024-04-25T13:50:31.000455+0000","last_undegraded":"2024-04-25T13:50:31.000455+0000","last_fullsized":"2024-04-25T13:50:31.000455+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:20.338398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"170'6","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.046437+0000","last_change":"2024-04-25T13:50:29.975888+0000","last_active":"2024-04-25T13:50:31.046437+0000","last_peered":"2024-04-25T13:50:31.046437+0000","last_clean":"2024-04-25T13:50:31.046437+0000","last_became_active":"2024-04-25T13:50:29.975471+0000","last_became_peered":"2024-04-25T13:50:29.975471+0000","last_unstale":"2024-04-25T13:50:31.046437+0000","last_undegraded":"2024-04-25T13:50:31.046437+0000","last_fullsized":"2024-04-25T13:50:31.046437+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:59.628228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882318+0000","last_change":"2024-04-25T13:50:31.022316+0000","last_active":"2024-04-25T13:50:32.882318+0000","last_peered":"2024-04-25T13:50:32.882318+0000","last_clean":"2024-04-25T13:50:32.882318+0000","last_became_active":"2024-04-25T13:50:31.020578+0000","last_became_peered":"2024-04-25T13:50:31.020578+0000","last_unstale":"2024-04-25T13:50:32.882318+0000","last_undegraded":"2024-04-25T13:50:32.882318+0000","last_fullsized":"2024-04-25T13:50:32.882318+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:21.635891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882058+0000","last_change":"2024-04-25T13:50:29.982794+0000","last_active":"2024-04-25T13:50:32.882058+0000","last_peered":"2024-04-25T13:50:32.882058+0000","last_clean":"2024-04-25T13:50:32.882058+0000","last_became_active":"2024-04-25T13:50:29.982026+0000","last_became_peered":"2024-04-25T13:50:29.982026+0000","last_unstale":"2024-04-25T13:50:32.882058+0000","last_undegraded":"2024-04-25T13:50:32.882058+0000","last_fullsized":"2024-04-25T13:50:32.882058+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:43.804811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983354+0000","last_change":"2024-04-25T13:50:31.010476+0000","last_active":"2024-04-25T13:50:31.983354+0000","last_peered":"2024-04-25T13:50:31.983354+0000","last_clean":"2024-04-25T13:50:31.983354+0000","last_became_active":"2024-04-25T13:50:31.009143+0000","last_became_peered":"2024-04-25T13:50:31.009143+0000","last_unstale":"2024-04-25T13:50:31.983354+0000","last_undegraded":"2024-04-25T13:50:31.983354+0000","last_fullsized":"2024-04-25T13:50:31.983354+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:10.908622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882493+0000","last_change":"2024-04-25T13:50:29.992364+0000","last_active":"2024-04-25T13:50:32.882493+0000","last_peered":"2024-04-25T13:50:32.882493+0000","last_clean":"2024-04-25T13:50:32.882493+0000","last_became_active":"2024-04-25T13:50:29.983593+0000","last_became_peered":"2024-04-25T13:50:29.983593+0000","last_unstale":"2024-04-25T13:50:32.882493+0000","last_undegraded":"2024-04-25T13:50:32.882493+0000","last_fullsized":"2024-04-25T13:50:32.882493+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:07.463410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440212+0000","last_change":"2024-04-25T13:50:31.019097+0000","last_active":"2024-04-25T13:50:32.440212+0000","last_peered":"2024-04-25T13:50:32.440212+0000","last_clean":"2024-04-25T13:50:32.440212+0000","last_became_active":"2024-04-25T13:50:31.017235+0000","last_became_peered":"2024-04-25T13:50:31.017235+0000","last_unstale":"2024-04-25T13:50:32.440212+0000","last_undegraded":"2024-04-25T13:50:32.440212+0000","last_fullsized":"2024-04-25T13:50:32.440212+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:56.729170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039186+0000","last_change":"2024-04-25T13:50:29.967386+0000","last_active":"2024-04-25T13:50:32.039186+0000","last_peered":"2024-04-25T13:50:32.039186+0000","last_clean":"2024-04-25T13:50:32.039186+0000","last_became_active":"2024-04-25T13:50:29.967034+0000","last_became_peered":"2024-04-25T13:50:29.967034+0000","last_unstale":"2024-04-25T13:50:32.039186+0000","last_undegraded":"2024-04-25T13:50:32.039186+0000","last_fullsized":"2024-04-25T13:50:32.039186+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.024447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882316+0000","last_change":"2024-04-25T13:50:31.027384+0000","last_active":"2024-04-25T13:50:32.882316+0000","last_peered":"2024-04-25T13:50:32.882316+0000","last_clean":"2024-04-25T13:50:32.882316+0000","last_became_active":"2024-04-25T13:50:31.026159+0000","last_became_peered":"2024-04-25T13:50:31.026159+0000","last_unstale":"2024-04-25T13:50:32.882316+0000","last_undegraded":"2024-04-25T13:50:32.882316+0000","last_fullsized":"2024-04-25T13:50:32.882316+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:08.706848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"170'8","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.050439+0000","last_change":"2024-04-25T13:50:29.991447+0000","last_active":"2024-04-25T13:50:31.050439+0000","last_peered":"2024-04-25T13:50:31.050439+0000","last_clean":"2024-04-25T13:50:31.050439+0000","last_became_active":"2024-04-25T13:50:29.991066+0000","last_became_peered":"2024-04-25T13:50:29.991066+0000","last_unstale":"2024-04-25T13:50:31.050439+0000","last_undegraded":"2024-04-25T13:50:31.050439+0000","last_fullsized":"2024-04-25T13:50:31.050439+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:35.295582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042513+0000","last_change":"2024-04-25T13:50:31.002918+0000","last_active":"2024-04-25T13:50:32.042513+0000","last_peered":"2024-04-25T13:50:32.042513+0000","last_clean":"2024-04-25T13:50:32.042513+0000","last_became_active":"2024-04-25T13:50:30.999700+0000","last_became_peered":"2024-04-25T13:50:30.999700+0000","last_unstale":"2024-04-25T13:50:32.042513+0000","last_undegraded":"2024-04-25T13:50:32.042513+0000","last_fullsized":"2024-04-25T13:50:32.042513+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:37.680720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440760+0000","last_change":"2024-04-25T13:50:29.987768+0000","last_active":"2024-04-25T13:50:32.440760+0000","last_peered":"2024-04-25T13:50:32.440760+0000","last_clean":"2024-04-25T13:50:32.440760+0000","last_became_active":"2024-04-25T13:50:29.987027+0000","last_became_peered":"2024-04-25T13:50:29.987027+0000","last_unstale":"2024-04-25T13:50:32.440760+0000","last_undegraded":"2024-04-25T13:50:32.440760+0000","last_fullsized":"2024-04-25T13:50:32.440760+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:20.621885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.002636+0000","last_change":"2024-04-25T13:50:31.002138+0000","last_active":"2024-04-25T13:50:31.002636+0000","last_peered":"2024-04-25T13:50:31.002636+0000","last_clean":"2024-04-25T13:50:31.002636+0000","last_became_active":"2024-04-25T13:50:31.001011+0000","last_became_peered":"2024-04-25T13:50:31.001011+0000","last_unstale":"2024-04-25T13:50:31.002636+0000","last_undegraded":"2024-04-25T13:50:31.002636+0000","last_fullsized":"2024-04-25T13:50:31.002636+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:54.925569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983829+0000","last_change":"2024-04-25T13:50:29.983773+0000","last_active":"2024-04-25T13:50:31.983829+0000","last_peered":"2024-04-25T13:50:31.983829+0000","last_clean":"2024-04-25T13:50:31.983829+0000","last_became_active":"2024-04-25T13:50:29.983263+0000","last_became_peered":"2024-04-25T13:50:29.983263+0000","last_unstale":"2024-04-25T13:50:31.983829+0000","last_undegraded":"2024-04-25T13:50:31.983829+0000","last_fullsized":"2024-04-25T13:50:31.983829+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:44.605150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882570+0000","last_change":"2024-04-25T13:50:31.029848+0000","last_active":"2024-04-25T13:50:32.882570+0000","last_peered":"2024-04-25T13:50:32.882570+0000","last_clean":"2024-04-25T13:50:32.882570+0000","last_became_active":"2024-04-25T13:50:31.028328+0000","last_became_peered":"2024-04-25T13:50:31.028328+0000","last_unstale":"2024-04-25T13:50:32.882570+0000","last_undegraded":"2024-04-25T13:50:32.882570+0000","last_fullsized":"2024-04-25T13:50:32.882570+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:46.743026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.043531+0000","last_change":"2024-04-25T13:50:29.988408+0000","last_active":"2024-04-25T13:50:31.043531+0000","last_peered":"2024-04-25T13:50:31.043531+0000","last_clean":"2024-04-25T13:50:31.043531+0000","last_became_active":"2024-04-25T13:50:29.988162+0000","last_became_peered":"2024-04-25T13:50:29.988162+0000","last_unstale":"2024-04-25T13:50:31.043531+0000","last_undegraded":"2024-04-25T13:50:31.043531+0000","last_fullsized":"2024-04-25T13:50:31.043531+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:04.187473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041049+0000","last_change":"2024-04-25T13:50:31.023757+0000","last_active":"2024-04-25T13:50:32.041049+0000","last_peered":"2024-04-25T13:50:32.041049+0000","last_clean":"2024-04-25T13:50:32.041049+0000","last_became_active":"2024-04-25T13:50:31.022570+0000","last_became_peered":"2024-04-25T13:50:31.022570+0000","last_unstale":"2024-04-25T13:50:32.041049+0000","last_undegraded":"2024-04-25T13:50:32.041049+0000","last_fullsized":"2024-04-25T13:50:32.041049+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:49.262749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983529+0000","last_change":"2024-04-25T13:50:29.978206+0000","last_active":"2024-04-25T13:50:31.983529+0000","last_peered":"2024-04-25T13:50:31.983529+0000","last_clean":"2024-04-25T13:50:31.983529+0000","last_became_active":"2024-04-25T13:50:29.977846+0000","last_became_peered":"2024-04-25T13:50:29.977846+0000","last_unstale":"2024-04-25T13:50:31.983529+0000","last_undegraded":"2024-04-25T13:50:31.983529+0000","last_fullsized":"2024-04-25T13:50:31.983529+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:17.408363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983393+0000","last_change":"2024-04-25T13:50:31.011504+0000","last_active":"2024-04-25T13:50:31.983393+0000","last_peered":"2024-04-25T13:50:31.983393+0000","last_clean":"2024-04-25T13:50:31.983393+0000","last_became_active":"2024-04-25T13:50:31.008565+0000","last_became_peered":"2024-04-25T13:50:31.008565+0000","last_unstale":"2024-04-25T13:50:31.983393+0000","last_undegraded":"2024-04-25T13:50:31.983393+0000","last_fullsized":"2024-04-25T13:50:31.983393+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:32.589780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.045694+0000","last_change":"2024-04-25T13:50:29.991210+0000","last_active":"2024-04-25T13:50:31.045694+0000","last_peered":"2024-04-25T13:50:31.045694+0000","last_clean":"2024-04-25T13:50:31.045694+0000","last_became_active":"2024-04-25T13:50:29.990503+0000","last_became_peered":"2024-04-25T13:50:29.990503+0000","last_unstale":"2024-04-25T13:50:31.045694+0000","last_undegraded":"2024-04-25T13:50:31.045694+0000","last_fullsized":"2024-04-25T13:50:31.045694+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:24.340007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.013261+0000","last_change":"2024-04-25T13:50:31.012955+0000","last_active":"2024-04-25T13:50:31.013261+0000","last_peered":"2024-04-25T13:50:31.013261+0000","last_clean":"2024-04-25T13:50:31.013261+0000","last_became_active":"2024-04-25T13:50:31.012291+0000","last_became_peered":"2024-04-25T13:50:31.012291+0000","last_unstale":"2024-04-25T13:50:31.013261+0000","last_undegraded":"2024-04-25T13:50:31.013261+0000","last_fullsized":"2024-04-25T13:50:31.013261+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:43.751344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042365+0000","last_change":"2024-04-25T13:50:29.994977+0000","last_active":"2024-04-25T13:50:32.042365+0000","last_peered":"2024-04-25T13:50:32.042365+0000","last_clean":"2024-04-25T13:50:32.042365+0000","last_became_active":"2024-04-25T13:50:29.992868+0000","last_became_peered":"2024-04-25T13:50:29.992868+0000","last_unstale":"2024-04-25T13:50:32.042365+0000","last_undegraded":"2024-04-25T13:50:32.042365+0000","last_fullsized":"2024-04-25T13:50:32.042365+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:55.413611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.982954+0000","last_change":"2024-04-25T13:50:31.016958+0000","last_active":"2024-04-25T13:50:31.982954+0000","last_peered":"2024-04-25T13:50:31.982954+0000","last_clean":"2024-04-25T13:50:31.982954+0000","last_became_active":"2024-04-25T13:50:31.015494+0000","last_became_peered":"2024-04-25T13:50:31.015494+0000","last_unstale":"2024-04-25T13:50:31.982954+0000","last_undegraded":"2024-04-25T13:50:31.982954+0000","last_fullsized":"2024-04-25T13:50:31.982954+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:46.545434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042921+0000","last_change":"2024-04-25T13:50:29.992141+0000","last_active":"2024-04-25T13:50:32.042921+0000","last_peered":"2024-04-25T13:50:32.042921+0000","last_clean":"2024-04-25T13:50:32.042921+0000","last_became_active":"2024-04-25T13:50:29.989936+0000","last_became_peered":"2024-04-25T13:50:29.989936+0000","last_unstale":"2024-04-25T13:50:32.042921+0000","last_undegraded":"2024-04-25T13:50:32.042921+0000","last_fullsized":"2024-04-25T13:50:32.042921+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:03.260252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.020916+0000","last_change":"2024-04-25T13:50:31.020568+0000","last_active":"2024-04-25T13:50:31.020916+0000","last_peered":"2024-04-25T13:50:31.020916+0000","last_clean":"2024-04-25T13:50:31.020916+0000","last_became_active":"2024-04-25T13:50:31.020030+0000","last_became_peered":"2024-04-25T13:50:31.020030+0000","last_unstale":"2024-04-25T13:50:31.020916+0000","last_undegraded":"2024-04-25T13:50:31.020916+0000","last_fullsized":"2024-04-25T13:50:31.020916+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:23.521776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882654+0000","last_change":"2024-04-25T13:50:29.991731+0000","last_active":"2024-04-25T13:50:32.882654+0000","last_peered":"2024-04-25T13:50:32.882654+0000","last_clean":"2024-04-25T13:50:32.882654+0000","last_became_active":"2024-04-25T13:50:29.982599+0000","last_became_peered":"2024-04-25T13:50:29.982599+0000","last_unstale":"2024-04-25T13:50:32.882654+0000","last_undegraded":"2024-04-25T13:50:32.882654+0000","last_fullsized":"2024-04-25T13:50:32.882654+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:05.962434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440904+0000","last_change":"2024-04-25T13:50:31.015507+0000","last_active":"2024-04-25T13:50:32.440904+0000","last_peered":"2024-04-25T13:50:32.440904+0000","last_clean":"2024-04-25T13:50:32.440904+0000","last_became_active":"2024-04-25T13:50:31.010266+0000","last_became_peered":"2024-04-25T13:50:31.010266+0000","last_unstale":"2024-04-25T13:50:32.440904+0000","last_undegraded":"2024-04-25T13:50:32.440904+0000","last_fullsized":"2024-04-25T13:50:32.440904+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:15.267651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041954+0000","last_change":"2024-04-25T13:50:29.994661+0000","last_active":"2024-04-25T13:50:32.041954+0000","last_peered":"2024-04-25T13:50:32.041954+0000","last_clean":"2024-04-25T13:50:32.041954+0000","last_became_active":"2024-04-25T13:50:29.992959+0000","last_became_peered":"2024-04-25T13:50:29.992959+0000","last_unstale":"2024-04-25T13:50:32.041954+0000","last_undegraded":"2024-04-25T13:50:32.041954+0000","last_fullsized":"2024-04-25T13:50:32.041954+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:27.357299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882815+0000","last_change":"2024-04-25T13:50:31.029421+0000","last_active":"2024-04-25T13:50:32.882815+0000","last_peered":"2024-04-25T13:50:32.882815+0000","last_clean":"2024-04-25T13:50:32.882815+0000","last_became_active":"2024-04-25T13:50:31.027993+0000","last_became_peered":"2024-04-25T13:50:31.027993+0000","last_unstale":"2024-04-25T13:50:32.882815+0000","last_undegraded":"2024-04-25T13:50:32.882815+0000","last_fullsized":"2024-04-25T13:50:32.882815+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:47.268447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"171'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042938+0000","last_change":"2024-04-25T13:50:29.987983+0000","last_active":"2024-04-25T13:50:32.042938+0000","last_peered":"2024-04-25T13:50:32.042938+0000","last_clean":"2024-04-25T13:50:32.042938+0000","last_became_active":"2024-04-25T13:50:29.987451+0000","last_became_peered":"2024-04-25T13:50:29.987451+0000","last_unstale":"2024-04-25T13:50:32.042938+0000","last_undegraded":"2024-04-25T13:50:32.042938+0000","last_fullsized":"2024-04-25T13:50:32.042938+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:10.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983543+0000","last_change":"2024-04-25T13:50:31.020828+0000","last_active":"2024-04-25T13:50:31.983543+0000","last_peered":"2024-04-25T13:50:31.983543+0000","last_clean":"2024-04-25T13:50:31.983543+0000","last_became_active":"2024-04-25T13:50:31.020368+0000","last_became_peered":"2024-04-25T13:50:31.020368+0000","last_unstale":"2024-04-25T13:50:31.983543+0000","last_undegraded":"2024-04-25T13:50:31.983543+0000","last_fullsized":"2024-04-25T13:50:31.983543+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:03.046763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041382+0000","last_change":"2024-04-25T13:50:29.977232+0000","last_active":"2024-04-25T13:50:32.041382+0000","last_peered":"2024-04-25T13:50:32.041382+0000","last_clean":"2024-04-25T13:50:32.041382+0000","last_became_active":"2024-04-25T13:50:29.976199+0000","last_became_peered":"2024-04-25T13:50:29.976199+0000","last_unstale":"2024-04-25T13:50:32.041382+0000","last_undegraded":"2024-04-25T13:50:32.041382+0000","last_fullsized":"2024-04-25T13:50:32.041382+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:25.272511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040391+0000","last_change":"2024-04-25T13:50:31.008847+0000","last_active":"2024-04-25T13:50:32.040391+0000","last_peered":"2024-04-25T13:50:32.040391+0000","last_clean":"2024-04-25T13:50:32.040391+0000","last_became_active":"2024-04-25T13:50:31.007677+0000","last_became_peered":"2024-04-25T13:50:31.007677+0000","last_unstale":"2024-04-25T13:50:32.040391+0000","last_undegraded":"2024-04-25T13:50:32.040391+0000","last_fullsized":"2024-04-25T13:50:32.040391+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:08.616283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040122+0000","last_change":"2024-04-25T13:50:29.976465+0000","last_active":"2024-04-25T13:50:32.040122+0000","last_peered":"2024-04-25T13:50:32.040122+0000","last_clean":"2024-04-25T13:50:32.040122+0000","last_became_active":"2024-04-25T13:50:29.976089+0000","last_became_peered":"2024-04-25T13:50:29.976089+0000","last_unstale":"2024-04-25T13:50:32.040122+0000","last_undegraded":"2024-04-25T13:50:32.040122+0000","last_fullsized":"2024-04-25T13:50:32.040122+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:55.157353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.004550+0000","last_change":"2024-04-25T13:50:31.004037+0000","last_active":"2024-04-25T13:50:31.004550+0000","last_peered":"2024-04-25T13:50:31.004550+0000","last_clean":"2024-04-25T13:50:31.004550+0000","last_became_active":"2024-04-25T13:50:31.003396+0000","last_became_peered":"2024-04-25T13:50:31.003396+0000","last_unstale":"2024-04-25T13:50:31.004550+0000","last_undegraded":"2024-04-25T13:50:31.004550+0000","last_fullsized":"2024-04-25T13:50:31.004550+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:00.687661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:30.990098+0000","last_change":"2024-04-25T13:50:29.977510+0000","last_active":"2024-04-25T13:50:30.990098+0000","last_peered":"2024-04-25T13:50:30.990098+0000","last_clean":"2024-04-25T13:50:30.990098+0000","last_became_active":"2024-04-25T13:50:29.976987+0000","last_became_peered":"2024-04-25T13:50:29.976987+0000","last_unstale":"2024-04-25T13:50:30.990098+0000","last_undegraded":"2024-04-25T13:50:30.990098+0000","last_fullsized":"2024-04-25T13:50:30.990098+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:14.380568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440699+0000","last_change":"2024-04-25T13:50:31.026562+0000","last_active":"2024-04-25T13:50:32.440699+0000","last_peered":"2024-04-25T13:50:32.440699+0000","last_clean":"2024-04-25T13:50:32.440699+0000","last_became_active":"2024-04-25T13:50:31.025896+0000","last_became_peered":"2024-04-25T13:50:31.025896+0000","last_unstale":"2024-04-25T13:50:32.440699+0000","last_undegraded":"2024-04-25T13:50:32.440699+0000","last_fullsized":"2024-04-25T13:50:32.440699+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.706122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042117+0000","last_change":"2024-04-25T13:50:29.995461+0000","last_active":"2024-04-25T13:50:32.042117+0000","last_peered":"2024-04-25T13:50:32.042117+0000","last_clean":"2024-04-25T13:50:32.042117+0000","last_became_active":"2024-04-25T13:50:29.993923+0000","last_became_peered":"2024-04-25T13:50:29.993923+0000","last_unstale":"2024-04-25T13:50:32.042117+0000","last_undegraded":"2024-04-25T13:50:32.042117+0000","last_fullsized":"2024-04-25T13:50:32.042117+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:48.222133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041717+0000","last_change":"2024-04-25T13:50:30.997717+0000","last_active":"2024-04-25T13:50:32.041717+0000","last_peered":"2024-04-25T13:50:32.041717+0000","last_clean":"2024-04-25T13:50:32.041717+0000","last_became_active":"2024-04-25T13:50:30.996912+0000","last_became_peered":"2024-04-25T13:50:30.996912+0000","last_unstale":"2024-04-25T13:50:32.041717+0000","last_undegraded":"2024-04-25T13:50:32.041717+0000","last_fullsized":"2024-04-25T13:50:32.041717+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:38.895955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882686+0000","last_change":"2024-04-25T13:50:29.989447+0000","last_active":"2024-04-25T13:50:32.882686+0000","last_peered":"2024-04-25T13:50:32.882686+0000","last_clean":"2024-04-25T13:50:32.882686+0000","last_became_active":"2024-04-25T13:50:29.988454+0000","last_became_peered":"2024-04-25T13:50:29.988454+0000","last_unstale":"2024-04-25T13:50:32.882686+0000","last_undegraded":"2024-04-25T13:50:32.882686+0000","last_fullsized":"2024-04-25T13:50:32.882686+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:40.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.441133+0000","last_change":"2024-04-25T13:50:31.016387+0000","last_active":"2024-04-25T13:50:32.441133+0000","last_peered":"2024-04-25T13:50:32.441133+0000","last_clean":"2024-04-25T13:50:32.441133+0000","last_became_active":"2024-04-25T13:50:31.013357+0000","last_became_peered":"2024-04-25T13:50:31.013357+0000","last_unstale":"2024-04-25T13:50:32.441133+0000","last_undegraded":"2024-04-25T13:50:32.441133+0000","last_fullsized":"2024-04-25T13:50:32.441133+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:27.151496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983528+0000","last_change":"2024-04-25T13:50:29.986059+0000","last_active":"2024-04-25T13:50:31.983528+0000","last_peered":"2024-04-25T13:50:31.983528+0000","last_clean":"2024-04-25T13:50:31.983528+0000","last_became_active":"2024-04-25T13:50:29.985410+0000","last_became_peered":"2024-04-25T13:50:29.985410+0000","last_unstale":"2024-04-25T13:50:31.983528+0000","last_undegraded":"2024-04-25T13:50:31.983528+0000","last_fullsized":"2024-04-25T13:50:31.983528+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:56.609841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.015869+0000","last_change":"2024-04-25T13:50:31.015549+0000","last_active":"2024-04-25T13:50:31.015869+0000","last_peered":"2024-04-25T13:50:31.015869+0000","last_clean":"2024-04-25T13:50:31.015869+0000","last_became_active":"2024-04-25T13:50:31.014852+0000","last_became_peered":"2024-04-25T13:50:31.014852+0000","last_unstale":"2024-04-25T13:50:31.015869+0000","last_undegraded":"2024-04-25T13:50:31.015869+0000","last_fullsized":"2024-04-25T13:50:31.015869+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:09.410524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"170'3","reported_seq":17,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882529+0000","last_change":"2024-04-25T13:50:29.985942+0000","last_active":"2024-04-25T13:50:32.882529+0000","last_peered":"2024-04-25T13:50:32.882529+0000","last_clean":"2024-04-25T13:50:32.882529+0000","last_became_active":"2024-04-25T13:50:29.984748+0000","last_became_peered":"2024-04-25T13:50:29.984748+0000","last_unstale":"2024-04-25T13:50:32.882529+0000","last_undegraded":"2024-04-25T13:50:32.882529+0000","last_fullsized":"2024-04-25T13:50:32.882529+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:35.011953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":407,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042517+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:50:32.042517+0000","last_peered":"2024-04-25T13:50:32.042517+0000","last_clean":"2024-04-25T13:50:32.042517+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:50:32.042517+0000","last_undegraded":"2024-04-25T13:50:32.042517+0000","last_fullsized":"2024-04-25T13:50:32.042517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1515,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193796,"statfs":{"total":200000000,"available":198447866,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":938,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038740,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":938,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038864,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":938,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":938,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038796,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":934,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194378,"statfs":{"total":200000000,"available":199043504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":939,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574973,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193801,"statfs":{"total":200000000,"available":198453172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:50:35.158 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:50:35.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.348+0000 7ff56c974640 1 Processor -- start 2024-04-25T13:50:35.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.348+0000 7ff56c974640 1 -- start start 2024-04-25T13:50:35.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff56c974640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff568161f90 0x7ff5680cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff56c974640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff5680d0510 0x7ff56815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff56c974640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff5680d0d70 con 0x7ff56815ea70 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff56c974640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff5680d1050 con 0x7ff5680d0510 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff56c974640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff56806dca0 con 0x7ff568161f90 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff565d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff5680d0510 0x7ff56815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff565d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff5680d0510 0x7ff56815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54632/0 (socket says 172.21.15.80:54632) 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff565d9b640 1 -- 172.21.15.80:0/723146910 learned_addr learned my addr 172.21.15.80:0/723146910 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff56659c640 1 --2- 172.21.15.80:0/723146910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff568161f90 0x7ff5680cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff565d9b640 1 -- 172.21.15.80:0/723146910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff568161f90 msgr2=0x7ff5680cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff565d9b640 1 --2- 172.21.15.80:0/723146910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff568161f90 0x7ff5680cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff565d9b640 1 -- 172.21.15.80:0/723146910 >> v1:172.21.15.87:6789/0 conn(0x7ff56815ea70 legacy=0x7ff56815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.352+0000 7ff565d9b640 1 -- 172.21.15.80:0/723146910 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff56806df80 con 0x7ff5680d0510 2024-04-25T13:50:35.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff565d9b640 1 --2- 172.21.15.80:0/723146910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5680d0510 0x7ff56815e530 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009870 tx=0x7ff55c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec629ae049c5295b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56559a640 1 -- 172.21.15.80:0/723146910 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0b1070 con 0x7ff5680d0510 2024-04-25T13:50:35.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56559a640 1 -- 172.21.15.80:0/723146910 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c0ac600 con 0x7ff5680d0510 2024-04-25T13:50:35.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56559a640 1 -- 172.21.15.80:0/723146910 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0ac9d0 con 0x7ff5680d0510 2024-04-25T13:50:35.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 -- 172.21.15.80:0/723146910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5680d0510 msgr2=0x7ff56815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 --2- 172.21.15.80:0/723146910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5680d0510 0x7ff56815e530 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009870 tx=0x7ff55c01c470 comp rx=0 tx=0).stop 2024-04-25T13:50:35.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 -- 172.21.15.80:0/723146910 shutdown_connections 2024-04-25T13:50:35.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 --2- 172.21.15.80:0/723146910 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff5680d0510 0x7ff56815e530 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 --2- 172.21.15.80:0/723146910 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff568161f90 0x7ff5680cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 -- 172.21.15.80:0/723146910 >> 172.21.15.80:0/723146910 conn(0x7ff56806d080 msgr2=0x7ff5680b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 -- 172.21.15.80:0/723146910 shutdown_connections 2024-04-25T13:50:35.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 -- 172.21.15.80:0/723146910 wait complete. 2024-04-25T13:50:35.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 Processor -- start 2024-04-25T13:50:35.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.356+0000 7ff56c974640 1 -- start start 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5680d0510 0x7ff56815d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff56815ea70 0x7ff5681670b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff568161f90 0x7ff5681675f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff56806f630 con 0x7ff56815ea70 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff56806f4b0 con 0x7ff568161f90 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff5680d0920 con 0x7ff5680d0510 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff56815ea70 0x7ff5681670b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff566d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff568161f90 0x7ff5681675f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff56815ea70 0x7ff5681670b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54634/0 (socket says 172.21.15.80:54634) 2024-04-25T13:50:35.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 -- 172.21.15.80:0/2939560391 learned_addr learned my addr 172.21.15.80:0/2939560391 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:35.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56659c640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5680d0510 0x7ff56815d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 -- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5680d0510 msgr2=0x7ff56815d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5680d0510 0x7ff56815d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 -- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff568161f90 msgr2=0x7ff5681675f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff568161f90 0x7ff5681675f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 -- 172.21.15.80:0/2939560391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff568073ea0 con 0x7ff56815ea70 2024-04-25T13:50:35.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff565d9b640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff56815ea70 0x7ff5681670b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff55c01ab80 tx=0x7ff55c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff556ffd640 1 -- 172.21.15.80:0/2939560391 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0b1040 con 0x7ff56815ea70 2024-04-25T13:50:35.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff556ffd640 1 -- 172.21.15.80:0/2939560391 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff55c0bc070 con 0x7ff56815ea70 2024-04-25T13:50:35.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff568145ce0 con 0x7ff56815ea70 2024-04-25T13:50:35.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff556ffd640 1 -- 172.21.15.80:0/2939560391 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c0b7890 con 0x7ff56815ea70 2024-04-25T13:50:35.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff568145a00 con 0x7ff56815ea70 2024-04-25T13:50:35.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff56806a3a0 con 0x7ff56815ea70 2024-04-25T13:50:35.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.360+0000 7ff556ffd640 1 -- 172.21.15.80:0/2939560391 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff55c0ac4c0 con 0x7ff56815ea70 2024-04-25T13:50:35.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.364+0000 7ff556ffd640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff54806f400 0x7ff548071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.364+0000 7ff556ffd640 1 -- 172.21.15.80:0/2939560391 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7ff55c129430 con 0x7ff56815ea70 2024-04-25T13:50:35.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.364+0000 7ff56659c640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff54806f400 0x7ff548071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.364+0000 7ff56659c640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff54806f400 0x7ff548071910 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff56806fa20 tx=0x7ff558008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.368+0000 7ff556ffd640 1 -- 172.21.15.80:0/2939560391 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff55c016150 con 0x7ff56815ea70 2024-04-25T13:50:35.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.524+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff56815f200 con 0x7ff54806f400 2024-04-25T13:50:35.539 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff556ffd640 1 -- 172.21.15.80:0/2939560391 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247967 (secure 0 0 0) 0x7ff56815f200 con 0x7ff54806f400 2024-04-25T13:50:35.541 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:35.543 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:50:35.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff54806f400 msgr2=0x7ff548071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff54806f400 0x7ff548071910 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff56806fa20 tx=0x7ff558008040 comp rx=0 tx=0).stop 2024-04-25T13:50:35.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff56815ea70 msgr2=0x7ff5681670b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff56815ea70 0x7ff5681670b0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7ff55c01ab80 tx=0x7ff55c01ad10 comp rx=0 tx=0).stop 2024-04-25T13:50:35.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 shutdown_connections 2024-04-25T13:50:35.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff54806f400 0x7ff548071910 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff568161f90 0x7ff5681675f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff56815ea70 0x7ff5681670b0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 --2- 172.21.15.80:0/2939560391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff5680d0510 0x7ff56815d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 >> 172.21.15.80:0/2939560391 conn(0x7ff56806d080 msgr2=0x7ff5681682d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 shutdown_connections 2024-04-25T13:50:35.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.544+0000 7ff56c974640 1 -- 172.21.15.80:0/2939560391 wait complete. 2024-04-25T13:50:35.563 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-04-25T13:50:34.324406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":131,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8654,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553845,"statfs":{"total":1600000000,"available":1591138132,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.374096"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041847+0000","last_change":"2024-04-25T13:50:31.012172+0000","last_active":"2024-04-25T13:50:32.041847+0000","last_peered":"2024-04-25T13:50:32.041847+0000","last_clean":"2024-04-25T13:50:32.041847+0000","last_became_active":"2024-04-25T13:50:31.010320+0000","last_became_peered":"2024-04-25T13:50:31.010320+0000","last_unstale":"2024-04-25T13:50:32.041847+0000","last_undegraded":"2024-04-25T13:50:32.041847+0000","last_fullsized":"2024-04-25T13:50:32.041847+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:31.642433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983278+0000","last_change":"2024-04-25T13:50:31.001777+0000","last_active":"2024-04-25T13:50:31.983278+0000","last_peered":"2024-04-25T13:50:31.983278+0000","last_clean":"2024-04-25T13:50:31.983278+0000","last_became_active":"2024-04-25T13:50:31.000850+0000","last_became_peered":"2024-04-25T13:50:31.000850+0000","last_unstale":"2024-04-25T13:50:31.983278+0000","last_undegraded":"2024-04-25T13:50:31.983278+0000","last_fullsized":"2024-04-25T13:50:31.983278+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:44:14.137344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882322+0000","last_change":"2024-04-25T13:50:31.027413+0000","last_active":"2024-04-25T13:50:32.882322+0000","last_peered":"2024-04-25T13:50:32.882322+0000","last_clean":"2024-04-25T13:50:32.882322+0000","last_became_active":"2024-04-25T13:50:31.024726+0000","last_became_peered":"2024-04-25T13:50:31.024726+0000","last_unstale":"2024-04-25T13:50:32.882322+0000","last_undegraded":"2024-04-25T13:50:32.882322+0000","last_fullsized":"2024-04-25T13:50:32.882322+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:04.461651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.982436+0000","last_change":"2024-04-25T13:50:31.024640+0000","last_active":"2024-04-25T13:50:31.982436+0000","last_peered":"2024-04-25T13:50:31.982436+0000","last_clean":"2024-04-25T13:50:31.982436+0000","last_became_active":"2024-04-25T13:50:31.023987+0000","last_became_peered":"2024-04-25T13:50:31.023987+0000","last_unstale":"2024-04-25T13:50:31.982436+0000","last_undegraded":"2024-04-25T13:50:31.982436+0000","last_fullsized":"2024-04-25T13:50:31.982436+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:37.652058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440773+0000","last_change":"2024-04-25T13:50:31.007799+0000","last_active":"2024-04-25T13:50:32.440773+0000","last_peered":"2024-04-25T13:50:32.440773+0000","last_clean":"2024-04-25T13:50:32.440773+0000","last_became_active":"2024-04-25T13:50:31.006933+0000","last_became_peered":"2024-04-25T13:50:31.006933+0000","last_unstale":"2024-04-25T13:50:32.440773+0000","last_undegraded":"2024-04-25T13:50:32.440773+0000","last_fullsized":"2024-04-25T13:50:32.440773+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:30:12.710289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041497+0000","last_change":"2024-04-25T13:50:31.024267+0000","last_active":"2024-04-25T13:50:32.041497+0000","last_peered":"2024-04-25T13:50:32.041497+0000","last_clean":"2024-04-25T13:50:32.041497+0000","last_became_active":"2024-04-25T13:50:31.023289+0000","last_became_peered":"2024-04-25T13:50:31.023289+0000","last_unstale":"2024-04-25T13:50:32.041497+0000","last_undegraded":"2024-04-25T13:50:32.041497+0000","last_fullsized":"2024-04-25T13:50:32.041497+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:51.248739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.015633+0000","last_change":"2024-04-25T13:50:31.015388+0000","last_active":"2024-04-25T13:50:31.015633+0000","last_peered":"2024-04-25T13:50:31.015633+0000","last_clean":"2024-04-25T13:50:31.015633+0000","last_became_active":"2024-04-25T13:50:31.014989+0000","last_became_peered":"2024-04-25T13:50:31.014989+0000","last_unstale":"2024-04-25T13:50:31.015633+0000","last_undegraded":"2024-04-25T13:50:31.015633+0000","last_fullsized":"2024-04-25T13:50:31.015633+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:13:33.621773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039597+0000","last_change":"2024-04-25T13:50:31.013881+0000","last_active":"2024-04-25T13:50:32.039597+0000","last_peered":"2024-04-25T13:50:32.039597+0000","last_clean":"2024-04-25T13:50:32.039597+0000","last_became_active":"2024-04-25T13:50:31.013456+0000","last_became_peered":"2024-04-25T13:50:31.013456+0000","last_unstale":"2024-04-25T13:50:32.039597+0000","last_undegraded":"2024-04-25T13:50:32.039597+0000","last_fullsized":"2024-04-25T13:50:32.039597+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:59.725655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983237+0000","last_change":"2024-04-25T13:50:30.998855+0000","last_active":"2024-04-25T13:50:31.983237+0000","last_peered":"2024-04-25T13:50:31.983237+0000","last_clean":"2024-04-25T13:50:31.983237+0000","last_became_active":"2024-04-25T13:50:30.998104+0000","last_became_peered":"2024-04-25T13:50:30.998104+0000","last_unstale":"2024-04-25T13:50:31.983237+0000","last_undegraded":"2024-04-25T13:50:31.983237+0000","last_fullsized":"2024-04-25T13:50:31.983237+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:26.713936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041163+0000","last_change":"2024-04-25T13:50:31.020776+0000","last_active":"2024-04-25T13:50:32.041163+0000","last_peered":"2024-04-25T13:50:32.041163+0000","last_clean":"2024-04-25T13:50:32.041163+0000","last_became_active":"2024-04-25T13:50:31.020177+0000","last_became_peered":"2024-04-25T13:50:31.020177+0000","last_unstale":"2024-04-25T13:50:32.041163+0000","last_undegraded":"2024-04-25T13:50:32.041163+0000","last_fullsized":"2024-04-25T13:50:32.041163+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:48.134078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039521+0000","last_change":"2024-04-25T13:50:31.011107+0000","last_active":"2024-04-25T13:50:32.039521+0000","last_peered":"2024-04-25T13:50:32.039521+0000","last_clean":"2024-04-25T13:50:32.039521+0000","last_became_active":"2024-04-25T13:50:31.006997+0000","last_became_peered":"2024-04-25T13:50:31.006997+0000","last_unstale":"2024-04-25T13:50:32.039521+0000","last_undegraded":"2024-04-25T13:50:32.039521+0000","last_fullsized":"2024-04-25T13:50:32.039521+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:09.993316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040746+0000","last_change":"2024-04-25T13:50:31.002319+0000","last_active":"2024-04-25T13:50:32.040746+0000","last_peered":"2024-04-25T13:50:32.040746+0000","last_clean":"2024-04-25T13:50:32.040746+0000","last_became_active":"2024-04-25T13:50:31.001625+0000","last_became_peered":"2024-04-25T13:50:31.001625+0000","last_unstale":"2024-04-25T13:50:32.040746+0000","last_undegraded":"2024-04-25T13:50:32.040746+0000","last_fullsized":"2024-04-25T13:50:32.040746+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:22:47.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.439833+0000","last_change":"2024-04-25T13:50:31.019662+0000","last_active":"2024-04-25T13:50:32.439833+0000","last_peered":"2024-04-25T13:50:32.439833+0000","last_clean":"2024-04-25T13:50:32.439833+0000","last_became_active":"2024-04-25T13:50:31.018290+0000","last_became_peered":"2024-04-25T13:50:31.018290+0000","last_unstale":"2024-04-25T13:50:32.439833+0000","last_undegraded":"2024-04-25T13:50:32.439833+0000","last_fullsized":"2024-04-25T13:50:32.439833+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:11:00.237513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.982929+0000","last_change":"2024-04-25T13:50:31.013869+0000","last_active":"2024-04-25T13:50:31.982929+0000","last_peered":"2024-04-25T13:50:31.982929+0000","last_clean":"2024-04-25T13:50:31.982929+0000","last_became_active":"2024-04-25T13:50:31.012935+0000","last_became_peered":"2024-04-25T13:50:31.012935+0000","last_unstale":"2024-04-25T13:50:31.982929+0000","last_undegraded":"2024-04-25T13:50:31.982929+0000","last_fullsized":"2024-04-25T13:50:31.982929+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:30.592154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041665+0000","last_change":"2024-04-25T13:50:31.000567+0000","last_active":"2024-04-25T13:50:32.041665+0000","last_peered":"2024-04-25T13:50:32.041665+0000","last_clean":"2024-04-25T13:50:32.041665+0000","last_became_active":"2024-04-25T13:50:30.998642+0000","last_became_peered":"2024-04-25T13:50:30.998642+0000","last_unstale":"2024-04-25T13:50:32.041665+0000","last_undegraded":"2024-04-25T13:50:32.041665+0000","last_fullsized":"2024-04-25T13:50:32.041665+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:45.288143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.008240+0000","last_change":"2024-04-25T13:50:31.007892+0000","last_active":"2024-04-25T13:50:31.008240+0000","last_peered":"2024-04-25T13:50:31.008240+0000","last_clean":"2024-04-25T13:50:31.008240+0000","last_became_active":"2024-04-25T13:50:31.007181+0000","last_became_peered":"2024-04-25T13:50:31.007181+0000","last_unstale":"2024-04-25T13:50:31.008240+0000","last_undegraded":"2024-04-25T13:50:31.008240+0000","last_fullsized":"2024-04-25T13:50:31.008240+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:21:38.400390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041768+0000","last_change":"2024-04-25T13:50:30.999492+0000","last_active":"2024-04-25T13:50:32.041768+0000","last_peered":"2024-04-25T13:50:32.041768+0000","last_clean":"2024-04-25T13:50:32.041768+0000","last_became_active":"2024-04-25T13:50:30.998823+0000","last_became_peered":"2024-04-25T13:50:30.998823+0000","last_unstale":"2024-04-25T13:50:32.041768+0000","last_undegraded":"2024-04-25T13:50:32.041768+0000","last_fullsized":"2024-04-25T13:50:32.041768+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:58:31.678708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882061+0000","last_change":"2024-04-25T13:50:31.028047+0000","last_active":"2024-04-25T13:50:32.882061+0000","last_peered":"2024-04-25T13:50:32.882061+0000","last_clean":"2024-04-25T13:50:32.882061+0000","last_became_active":"2024-04-25T13:50:31.026346+0000","last_became_peered":"2024-04-25T13:50:31.026346+0000","last_unstale":"2024-04-25T13:50:32.882061+0000","last_undegraded":"2024-04-25T13:50:32.882061+0000","last_fullsized":"2024-04-25T13:50:32.882061+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:24:15.578715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041049+0000","last_change":"2024-04-25T13:50:30.996109+0000","last_active":"2024-04-25T13:50:32.041049+0000","last_peered":"2024-04-25T13:50:32.041049+0000","last_clean":"2024-04-25T13:50:32.041049+0000","last_became_active":"2024-04-25T13:50:30.995321+0000","last_became_peered":"2024-04-25T13:50:30.995321+0000","last_unstale":"2024-04-25T13:50:32.041049+0000","last_undegraded":"2024-04-25T13:50:32.041049+0000","last_fullsized":"2024-04-25T13:50:32.041049+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:56.501947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.002059+0000","last_change":"2024-04-25T13:50:31.001713+0000","last_active":"2024-04-25T13:50:31.002059+0000","last_peered":"2024-04-25T13:50:31.002059+0000","last_clean":"2024-04-25T13:50:31.002059+0000","last_became_active":"2024-04-25T13:50:31.001058+0000","last_became_peered":"2024-04-25T13:50:31.001058+0000","last_unstale":"2024-04-25T13:50:31.002059+0000","last_undegraded":"2024-04-25T13:50:31.002059+0000","last_fullsized":"2024-04-25T13:50:31.002059+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:58.049180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041170+0000","last_change":"2024-04-25T13:50:31.020652+0000","last_active":"2024-04-25T13:50:32.041170+0000","last_peered":"2024-04-25T13:50:32.041170+0000","last_clean":"2024-04-25T13:50:32.041170+0000","last_became_active":"2024-04-25T13:50:31.018258+0000","last_became_peered":"2024-04-25T13:50:31.018258+0000","last_unstale":"2024-04-25T13:50:32.041170+0000","last_undegraded":"2024-04-25T13:50:32.041170+0000","last_fullsized":"2024-04-25T13:50:32.041170+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:45:28.863538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.982777+0000","last_change":"2024-04-25T13:50:31.002559+0000","last_active":"2024-04-25T13:50:31.982777+0000","last_peered":"2024-04-25T13:50:31.982777+0000","last_clean":"2024-04-25T13:50:31.982777+0000","last_became_active":"2024-04-25T13:50:31.001461+0000","last_became_peered":"2024-04-25T13:50:31.001461+0000","last_unstale":"2024-04-25T13:50:31.982777+0000","last_undegraded":"2024-04-25T13:50:31.982777+0000","last_fullsized":"2024-04-25T13:50:31.982777+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:27.188073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.981996+0000","last_change":"2024-04-25T13:50:31.012642+0000","last_active":"2024-04-25T13:50:31.981996+0000","last_peered":"2024-04-25T13:50:31.981996+0000","last_clean":"2024-04-25T13:50:31.981996+0000","last_became_active":"2024-04-25T13:50:31.011905+0000","last_became_peered":"2024-04-25T13:50:31.011905+0000","last_unstale":"2024-04-25T13:50:31.981996+0000","last_undegraded":"2024-04-25T13:50:31.981996+0000","last_fullsized":"2024-04-25T13:50:31.981996+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:18.681981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040793+0000","last_change":"2024-04-25T13:50:31.015893+0000","last_active":"2024-04-25T13:50:32.040793+0000","last_peered":"2024-04-25T13:50:32.040793+0000","last_clean":"2024-04-25T13:50:32.040793+0000","last_became_active":"2024-04-25T13:50:31.014514+0000","last_became_peered":"2024-04-25T13:50:31.014514+0000","last_unstale":"2024-04-25T13:50:32.040793+0000","last_undegraded":"2024-04-25T13:50:32.040793+0000","last_fullsized":"2024-04-25T13:50:32.040793+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:02.938329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.000619+0000","last_change":"2024-04-25T13:50:30.999929+0000","last_active":"2024-04-25T13:50:31.000619+0000","last_peered":"2024-04-25T13:50:31.000619+0000","last_clean":"2024-04-25T13:50:31.000619+0000","last_became_active":"2024-04-25T13:50:30.999229+0000","last_became_peered":"2024-04-25T13:50:30.999229+0000","last_unstale":"2024-04-25T13:50:31.000619+0000","last_undegraded":"2024-04-25T13:50:31.000619+0000","last_fullsized":"2024-04-25T13:50:31.000619+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:39.304701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039933+0000","last_change":"2024-04-25T13:50:31.009348+0000","last_active":"2024-04-25T13:50:32.039933+0000","last_peered":"2024-04-25T13:50:32.039933+0000","last_clean":"2024-04-25T13:50:32.039933+0000","last_became_active":"2024-04-25T13:50:31.005568+0000","last_became_peered":"2024-04-25T13:50:31.005568+0000","last_unstale":"2024-04-25T13:50:32.039933+0000","last_undegraded":"2024-04-25T13:50:32.039933+0000","last_fullsized":"2024-04-25T13:50:32.039933+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:22.902796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.028382+0000","last_change":"2024-04-25T13:50:31.028178+0000","last_active":"2024-04-25T13:50:31.028382+0000","last_peered":"2024-04-25T13:50:31.028382+0000","last_clean":"2024-04-25T13:50:31.028382+0000","last_became_active":"2024-04-25T13:50:31.027822+0000","last_became_peered":"2024-04-25T13:50:31.027822+0000","last_unstale":"2024-04-25T13:50:31.028382+0000","last_undegraded":"2024-04-25T13:50:31.028382+0000","last_fullsized":"2024-04-25T13:50:31.028382+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:23.360248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.030148+0000","last_change":"2024-04-25T13:50:31.029904+0000","last_active":"2024-04-25T13:50:31.030148+0000","last_peered":"2024-04-25T13:50:31.030148+0000","last_clean":"2024-04-25T13:50:31.030148+0000","last_became_active":"2024-04-25T13:50:31.029460+0000","last_became_peered":"2024-04-25T13:50:31.029460+0000","last_unstale":"2024-04-25T13:50:31.030148+0000","last_undegraded":"2024-04-25T13:50:31.030148+0000","last_fullsized":"2024-04-25T13:50:31.030148+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:26:38.848182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039934+0000","last_change":"2024-04-25T13:50:31.022460+0000","last_active":"2024-04-25T13:50:32.039934+0000","last_peered":"2024-04-25T13:50:32.039934+0000","last_clean":"2024-04-25T13:50:32.039934+0000","last_became_active":"2024-04-25T13:50:31.022046+0000","last_became_peered":"2024-04-25T13:50:31.022046+0000","last_unstale":"2024-04-25T13:50:32.039934+0000","last_undegraded":"2024-04-25T13:50:32.039934+0000","last_fullsized":"2024-04-25T13:50:32.039934+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:25.755869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983917+0000","last_change":"2024-04-25T13:50:31.018266+0000","last_active":"2024-04-25T13:50:31.983917+0000","last_peered":"2024-04-25T13:50:31.983917+0000","last_clean":"2024-04-25T13:50:31.983917+0000","last_became_active":"2024-04-25T13:50:31.017118+0000","last_became_peered":"2024-04-25T13:50:31.017118+0000","last_unstale":"2024-04-25T13:50:31.983917+0000","last_undegraded":"2024-04-25T13:50:31.983917+0000","last_fullsized":"2024-04-25T13:50:31.983917+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:46:07.002502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983708+0000","last_change":"2024-04-25T13:50:31.024842+0000","last_active":"2024-04-25T13:50:31.983708+0000","last_peered":"2024-04-25T13:50:31.983708+0000","last_clean":"2024-04-25T13:50:31.983708+0000","last_became_active":"2024-04-25T13:50:31.024397+0000","last_became_peered":"2024-04-25T13:50:31.024397+0000","last_unstale":"2024-04-25T13:50:31.983708+0000","last_undegraded":"2024-04-25T13:50:31.983708+0000","last_fullsized":"2024-04-25T13:50:31.983708+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:22.191877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.000393+0000","last_change":"2024-04-25T13:50:30.997873+0000","last_active":"2024-04-25T13:50:31.000393+0000","last_peered":"2024-04-25T13:50:31.000393+0000","last_clean":"2024-04-25T13:50:31.000393+0000","last_became_active":"2024-04-25T13:50:30.997166+0000","last_became_peered":"2024-04-25T13:50:30.997166+0000","last_unstale":"2024-04-25T13:50:31.000393+0000","last_undegraded":"2024-04-25T13:50:31.000393+0000","last_fullsized":"2024-04-25T13:50:31.000393+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:13.182795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041277+0000","last_change":"2024-04-25T13:50:31.002850+0000","last_active":"2024-04-25T13:50:32.041277+0000","last_peered":"2024-04-25T13:50:32.041277+0000","last_clean":"2024-04-25T13:50:32.041277+0000","last_became_active":"2024-04-25T13:50:31.002034+0000","last_became_peered":"2024-04-25T13:50:31.002034+0000","last_unstale":"2024-04-25T13:50:32.041277+0000","last_undegraded":"2024-04-25T13:50:32.041277+0000","last_fullsized":"2024-04-25T13:50:32.041277+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:47.411894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"170'4","reported_seq":18,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041672+0000","last_change":"2024-04-25T13:50:29.986919+0000","last_active":"2024-04-25T13:50:32.041672+0000","last_peered":"2024-04-25T13:50:32.041672+0000","last_clean":"2024-04-25T13:50:32.041672+0000","last_became_active":"2024-04-25T13:50:29.985997+0000","last_became_peered":"2024-04-25T13:50:29.985997+0000","last_unstale":"2024-04-25T13:50:32.041672+0000","last_undegraded":"2024-04-25T13:50:32.041672+0000","last_fullsized":"2024-04-25T13:50:32.041672+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:09:30.865528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983718+0000","last_change":"2024-04-25T13:50:31.004232+0000","last_active":"2024-04-25T13:50:31.983718+0000","last_peered":"2024-04-25T13:50:31.983718+0000","last_clean":"2024-04-25T13:50:31.983718+0000","last_became_active":"2024-04-25T13:50:31.002546+0000","last_became_peered":"2024-04-25T13:50:31.002546+0000","last_unstale":"2024-04-25T13:50:31.983718+0000","last_undegraded":"2024-04-25T13:50:31.983718+0000","last_fullsized":"2024-04-25T13:50:31.983718+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:37.748528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983371+0000","last_change":"2024-04-25T13:50:29.986327+0000","last_active":"2024-04-25T13:50:31.983371+0000","last_peered":"2024-04-25T13:50:31.983371+0000","last_clean":"2024-04-25T13:50:31.983371+0000","last_became_active":"2024-04-25T13:50:29.985800+0000","last_became_peered":"2024-04-25T13:50:29.985800+0000","last_unstale":"2024-04-25T13:50:31.983371+0000","last_undegraded":"2024-04-25T13:50:31.983371+0000","last_fullsized":"2024-04-25T13:50:31.983371+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:56:19.502813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040394+0000","last_change":"2024-04-25T13:50:31.017906+0000","last_active":"2024-04-25T13:50:32.040394+0000","last_peered":"2024-04-25T13:50:32.040394+0000","last_clean":"2024-04-25T13:50:32.040394+0000","last_became_active":"2024-04-25T13:50:31.017285+0000","last_became_peered":"2024-04-25T13:50:31.017285+0000","last_unstale":"2024-04-25T13:50:32.040394+0000","last_undegraded":"2024-04-25T13:50:32.040394+0000","last_fullsized":"2024-04-25T13:50:32.040394+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:06:58.941799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.984240+0000","last_change":"2024-04-25T13:50:29.968012+0000","last_active":"2024-04-25T13:50:31.984240+0000","last_peered":"2024-04-25T13:50:31.984240+0000","last_clean":"2024-04-25T13:50:31.984240+0000","last_became_active":"2024-04-25T13:50:29.967190+0000","last_became_peered":"2024-04-25T13:50:29.967190+0000","last_unstale":"2024-04-25T13:50:31.984240+0000","last_undegraded":"2024-04-25T13:50:31.984240+0000","last_fullsized":"2024-04-25T13:50:31.984240+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:06.734857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042117+0000","last_change":"2024-04-25T13:50:31.005454+0000","last_active":"2024-04-25T13:50:32.042117+0000","last_peered":"2024-04-25T13:50:32.042117+0000","last_clean":"2024-04-25T13:50:32.042117+0000","last_became_active":"2024-04-25T13:50:31.003905+0000","last_became_peered":"2024-04-25T13:50:31.003905+0000","last_unstale":"2024-04-25T13:50:32.042117+0000","last_undegraded":"2024-04-25T13:50:32.042117+0000","last_fullsized":"2024-04-25T13:50:32.042117+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:28.089421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983305+0000","last_change":"2024-04-25T13:50:29.970650+0000","last_active":"2024-04-25T13:50:31.983305+0000","last_peered":"2024-04-25T13:50:31.983305+0000","last_clean":"2024-04-25T13:50:31.983305+0000","last_became_active":"2024-04-25T13:50:29.966219+0000","last_became_peered":"2024-04-25T13:50:29.966219+0000","last_unstale":"2024-04-25T13:50:31.983305+0000","last_undegraded":"2024-04-25T13:50:31.983305+0000","last_fullsized":"2024-04-25T13:50:31.983305+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:23.011996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983882+0000","last_change":"2024-04-25T13:50:31.029728+0000","last_active":"2024-04-25T13:50:31.983882+0000","last_peered":"2024-04-25T13:50:31.983882+0000","last_clean":"2024-04-25T13:50:31.983882+0000","last_became_active":"2024-04-25T13:50:31.029358+0000","last_became_peered":"2024-04-25T13:50:31.029358+0000","last_unstale":"2024-04-25T13:50:31.983882+0000","last_undegraded":"2024-04-25T13:50:31.983882+0000","last_fullsized":"2024-04-25T13:50:31.983882+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:36.260411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.043633+0000","last_change":"2024-04-25T13:50:29.990982+0000","last_active":"2024-04-25T13:50:31.043633+0000","last_peered":"2024-04-25T13:50:31.043633+0000","last_clean":"2024-04-25T13:50:31.043633+0000","last_became_active":"2024-04-25T13:50:29.990575+0000","last_became_peered":"2024-04-25T13:50:29.990575+0000","last_unstale":"2024-04-25T13:50:31.043633+0000","last_undegraded":"2024-04-25T13:50:31.043633+0000","last_fullsized":"2024-04-25T13:50:31.043633+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:19.314109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983247+0000","last_change":"2024-04-25T13:50:31.028780+0000","last_active":"2024-04-25T13:50:31.983247+0000","last_peered":"2024-04-25T13:50:31.983247+0000","last_clean":"2024-04-25T13:50:31.983247+0000","last_became_active":"2024-04-25T13:50:31.028041+0000","last_became_peered":"2024-04-25T13:50:31.028041+0000","last_unstale":"2024-04-25T13:50:31.983247+0000","last_undegraded":"2024-04-25T13:50:31.983247+0000","last_fullsized":"2024-04-25T13:50:31.983247+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:32:54.366648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.441297+0000","last_change":"2024-04-25T13:50:29.989385+0000","last_active":"2024-04-25T13:50:32.441297+0000","last_peered":"2024-04-25T13:50:32.441297+0000","last_clean":"2024-04-25T13:50:32.441297+0000","last_became_active":"2024-04-25T13:50:29.988079+0000","last_became_peered":"2024-04-25T13:50:29.988079+0000","last_unstale":"2024-04-25T13:50:32.441297+0000","last_undegraded":"2024-04-25T13:50:32.441297+0000","last_fullsized":"2024-04-25T13:50:32.441297+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:06.797342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983712+0000","last_change":"2024-04-25T13:50:31.005945+0000","last_active":"2024-04-25T13:50:31.983712+0000","last_peered":"2024-04-25T13:50:31.983712+0000","last_clean":"2024-04-25T13:50:31.983712+0000","last_became_active":"2024-04-25T13:50:31.004315+0000","last_became_peered":"2024-04-25T13:50:31.004315+0000","last_unstale":"2024-04-25T13:50:31.983712+0000","last_undegraded":"2024-04-25T13:50:31.983712+0000","last_fullsized":"2024-04-25T13:50:31.983712+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:38.544004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:30.987263+0000","last_change":"2024-04-25T13:50:29.981694+0000","last_active":"2024-04-25T13:50:30.987263+0000","last_peered":"2024-04-25T13:50:30.987263+0000","last_clean":"2024-04-25T13:50:30.987263+0000","last_became_active":"2024-04-25T13:50:29.981449+0000","last_became_peered":"2024-04-25T13:50:29.981449+0000","last_unstale":"2024-04-25T13:50:30.987263+0000","last_undegraded":"2024-04-25T13:50:30.987263+0000","last_fullsized":"2024-04-25T13:50:30.987263+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:00.489684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440413+0000","last_change":"2024-04-25T13:50:31.012441+0000","last_active":"2024-04-25T13:50:32.440413+0000","last_peered":"2024-04-25T13:50:32.440413+0000","last_clean":"2024-04-25T13:50:32.440413+0000","last_became_active":"2024-04-25T13:50:31.011713+0000","last_became_peered":"2024-04-25T13:50:31.011713+0000","last_unstale":"2024-04-25T13:50:32.440413+0000","last_undegraded":"2024-04-25T13:50:32.440413+0000","last_fullsized":"2024-04-25T13:50:32.440413+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:02.170602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.984445+0000","last_change":"2024-04-25T13:50:29.986313+0000","last_active":"2024-04-25T13:50:31.984445+0000","last_peered":"2024-04-25T13:50:31.984445+0000","last_clean":"2024-04-25T13:50:31.984445+0000","last_became_active":"2024-04-25T13:50:29.985501+0000","last_became_peered":"2024-04-25T13:50:29.985501+0000","last_unstale":"2024-04-25T13:50:31.984445+0000","last_undegraded":"2024-04-25T13:50:31.984445+0000","last_fullsized":"2024-04-25T13:50:31.984445+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:13.292665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440767+0000","last_change":"2024-04-25T13:50:30.996986+0000","last_active":"2024-04-25T13:50:32.440767+0000","last_peered":"2024-04-25T13:50:32.440767+0000","last_clean":"2024-04-25T13:50:32.440767+0000","last_became_active":"2024-04-25T13:50:30.996107+0000","last_became_peered":"2024-04-25T13:50:30.996107+0000","last_unstale":"2024-04-25T13:50:32.440767+0000","last_undegraded":"2024-04-25T13:50:32.440767+0000","last_fullsized":"2024-04-25T13:50:32.440767+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:03.424841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.984006+0000","last_change":"2024-04-25T13:50:29.965299+0000","last_active":"2024-04-25T13:50:31.984006+0000","last_peered":"2024-04-25T13:50:31.984006+0000","last_clean":"2024-04-25T13:50:31.984006+0000","last_became_active":"2024-04-25T13:50:29.964930+0000","last_became_peered":"2024-04-25T13:50:29.964930+0000","last_unstale":"2024-04-25T13:50:31.984006+0000","last_undegraded":"2024-04-25T13:50:31.984006+0000","last_fullsized":"2024-04-25T13:50:31.984006+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:22:51.235418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041502+0000","last_change":"2024-04-25T13:50:31.005622+0000","last_active":"2024-04-25T13:50:32.041502+0000","last_peered":"2024-04-25T13:50:32.041502+0000","last_clean":"2024-04-25T13:50:32.041502+0000","last_became_active":"2024-04-25T13:50:31.004854+0000","last_became_peered":"2024-04-25T13:50:31.004854+0000","last_unstale":"2024-04-25T13:50:32.041502+0000","last_undegraded":"2024-04-25T13:50:32.041502+0000","last_fullsized":"2024-04-25T13:50:32.041502+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:37:16.788603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041293+0000","last_change":"2024-04-25T13:50:29.987794+0000","last_active":"2024-04-25T13:50:32.041293+0000","last_peered":"2024-04-25T13:50:32.041293+0000","last_clean":"2024-04-25T13:50:32.041293+0000","last_became_active":"2024-04-25T13:50:29.986932+0000","last_became_peered":"2024-04-25T13:50:29.986932+0000","last_unstale":"2024-04-25T13:50:32.041293+0000","last_undegraded":"2024-04-25T13:50:32.041293+0000","last_fullsized":"2024-04-25T13:50:32.041293+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:34:51.991758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.000455+0000","last_change":"2024-04-25T13:50:30.998950+0000","last_active":"2024-04-25T13:50:31.000455+0000","last_peered":"2024-04-25T13:50:31.000455+0000","last_clean":"2024-04-25T13:50:31.000455+0000","last_became_active":"2024-04-25T13:50:30.998222+0000","last_became_peered":"2024-04-25T13:50:30.998222+0000","last_unstale":"2024-04-25T13:50:31.000455+0000","last_undegraded":"2024-04-25T13:50:31.000455+0000","last_fullsized":"2024-04-25T13:50:31.000455+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:20.338398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"170'6","reported_seq":18,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.046437+0000","last_change":"2024-04-25T13:50:29.975888+0000","last_active":"2024-04-25T13:50:31.046437+0000","last_peered":"2024-04-25T13:50:31.046437+0000","last_clean":"2024-04-25T13:50:31.046437+0000","last_became_active":"2024-04-25T13:50:29.975471+0000","last_became_peered":"2024-04-25T13:50:29.975471+0000","last_unstale":"2024-04-25T13:50:31.046437+0000","last_undegraded":"2024-04-25T13:50:31.046437+0000","last_fullsized":"2024-04-25T13:50:31.046437+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:59.628228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882318+0000","last_change":"2024-04-25T13:50:31.022316+0000","last_active":"2024-04-25T13:50:32.882318+0000","last_peered":"2024-04-25T13:50:32.882318+0000","last_clean":"2024-04-25T13:50:32.882318+0000","last_became_active":"2024-04-25T13:50:31.020578+0000","last_became_peered":"2024-04-25T13:50:31.020578+0000","last_unstale":"2024-04-25T13:50:32.882318+0000","last_undegraded":"2024-04-25T13:50:32.882318+0000","last_fullsized":"2024-04-25T13:50:32.882318+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:21.635891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882058+0000","last_change":"2024-04-25T13:50:29.982794+0000","last_active":"2024-04-25T13:50:32.882058+0000","last_peered":"2024-04-25T13:50:32.882058+0000","last_clean":"2024-04-25T13:50:32.882058+0000","last_became_active":"2024-04-25T13:50:29.982026+0000","last_became_peered":"2024-04-25T13:50:29.982026+0000","last_unstale":"2024-04-25T13:50:32.882058+0000","last_undegraded":"2024-04-25T13:50:32.882058+0000","last_fullsized":"2024-04-25T13:50:32.882058+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:11:43.804811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983354+0000","last_change":"2024-04-25T13:50:31.010476+0000","last_active":"2024-04-25T13:50:31.983354+0000","last_peered":"2024-04-25T13:50:31.983354+0000","last_clean":"2024-04-25T13:50:31.983354+0000","last_became_active":"2024-04-25T13:50:31.009143+0000","last_became_peered":"2024-04-25T13:50:31.009143+0000","last_unstale":"2024-04-25T13:50:31.983354+0000","last_undegraded":"2024-04-25T13:50:31.983354+0000","last_fullsized":"2024-04-25T13:50:31.983354+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:10.908622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882493+0000","last_change":"2024-04-25T13:50:29.992364+0000","last_active":"2024-04-25T13:50:32.882493+0000","last_peered":"2024-04-25T13:50:32.882493+0000","last_clean":"2024-04-25T13:50:32.882493+0000","last_became_active":"2024-04-25T13:50:29.983593+0000","last_became_peered":"2024-04-25T13:50:29.983593+0000","last_unstale":"2024-04-25T13:50:32.882493+0000","last_undegraded":"2024-04-25T13:50:32.882493+0000","last_fullsized":"2024-04-25T13:50:32.882493+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:07.463410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440212+0000","last_change":"2024-04-25T13:50:31.019097+0000","last_active":"2024-04-25T13:50:32.440212+0000","last_peered":"2024-04-25T13:50:32.440212+0000","last_clean":"2024-04-25T13:50:32.440212+0000","last_became_active":"2024-04-25T13:50:31.017235+0000","last_became_peered":"2024-04-25T13:50:31.017235+0000","last_unstale":"2024-04-25T13:50:32.440212+0000","last_undegraded":"2024-04-25T13:50:32.440212+0000","last_fullsized":"2024-04-25T13:50:32.440212+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:56.729170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.039186+0000","last_change":"2024-04-25T13:50:29.967386+0000","last_active":"2024-04-25T13:50:32.039186+0000","last_peered":"2024-04-25T13:50:32.039186+0000","last_clean":"2024-04-25T13:50:32.039186+0000","last_became_active":"2024-04-25T13:50:29.967034+0000","last_became_peered":"2024-04-25T13:50:29.967034+0000","last_unstale":"2024-04-25T13:50:32.039186+0000","last_undegraded":"2024-04-25T13:50:32.039186+0000","last_fullsized":"2024-04-25T13:50:32.039186+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:50:47.024447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882316+0000","last_change":"2024-04-25T13:50:31.027384+0000","last_active":"2024-04-25T13:50:32.882316+0000","last_peered":"2024-04-25T13:50:32.882316+0000","last_clean":"2024-04-25T13:50:32.882316+0000","last_became_active":"2024-04-25T13:50:31.026159+0000","last_became_peered":"2024-04-25T13:50:31.026159+0000","last_unstale":"2024-04-25T13:50:32.882316+0000","last_undegraded":"2024-04-25T13:50:32.882316+0000","last_fullsized":"2024-04-25T13:50:32.882316+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:08.706848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"170'8","reported_seq":20,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.050439+0000","last_change":"2024-04-25T13:50:29.991447+0000","last_active":"2024-04-25T13:50:31.050439+0000","last_peered":"2024-04-25T13:50:31.050439+0000","last_clean":"2024-04-25T13:50:31.050439+0000","last_became_active":"2024-04-25T13:50:29.991066+0000","last_became_peered":"2024-04-25T13:50:29.991066+0000","last_unstale":"2024-04-25T13:50:31.050439+0000","last_undegraded":"2024-04-25T13:50:31.050439+0000","last_fullsized":"2024-04-25T13:50:31.050439+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:35.295582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042513+0000","last_change":"2024-04-25T13:50:31.002918+0000","last_active":"2024-04-25T13:50:32.042513+0000","last_peered":"2024-04-25T13:50:32.042513+0000","last_clean":"2024-04-25T13:50:32.042513+0000","last_became_active":"2024-04-25T13:50:30.999700+0000","last_became_peered":"2024-04-25T13:50:30.999700+0000","last_unstale":"2024-04-25T13:50:32.042513+0000","last_undegraded":"2024-04-25T13:50:32.042513+0000","last_fullsized":"2024-04-25T13:50:32.042513+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:37.680720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440760+0000","last_change":"2024-04-25T13:50:29.987768+0000","last_active":"2024-04-25T13:50:32.440760+0000","last_peered":"2024-04-25T13:50:32.440760+0000","last_clean":"2024-04-25T13:50:32.440760+0000","last_became_active":"2024-04-25T13:50:29.987027+0000","last_became_peered":"2024-04-25T13:50:29.987027+0000","last_unstale":"2024-04-25T13:50:32.440760+0000","last_undegraded":"2024-04-25T13:50:32.440760+0000","last_fullsized":"2024-04-25T13:50:32.440760+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:47:20.621885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.002636+0000","last_change":"2024-04-25T13:50:31.002138+0000","last_active":"2024-04-25T13:50:31.002636+0000","last_peered":"2024-04-25T13:50:31.002636+0000","last_clean":"2024-04-25T13:50:31.002636+0000","last_became_active":"2024-04-25T13:50:31.001011+0000","last_became_peered":"2024-04-25T13:50:31.001011+0000","last_unstale":"2024-04-25T13:50:31.002636+0000","last_undegraded":"2024-04-25T13:50:31.002636+0000","last_fullsized":"2024-04-25T13:50:31.002636+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:01:54.925569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983829+0000","last_change":"2024-04-25T13:50:29.983773+0000","last_active":"2024-04-25T13:50:31.983829+0000","last_peered":"2024-04-25T13:50:31.983829+0000","last_clean":"2024-04-25T13:50:31.983829+0000","last_became_active":"2024-04-25T13:50:29.983263+0000","last_became_peered":"2024-04-25T13:50:29.983263+0000","last_unstale":"2024-04-25T13:50:31.983829+0000","last_undegraded":"2024-04-25T13:50:31.983829+0000","last_fullsized":"2024-04-25T13:50:31.983829+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:44.605150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882570+0000","last_change":"2024-04-25T13:50:31.029848+0000","last_active":"2024-04-25T13:50:32.882570+0000","last_peered":"2024-04-25T13:50:32.882570+0000","last_clean":"2024-04-25T13:50:32.882570+0000","last_became_active":"2024-04-25T13:50:31.028328+0000","last_became_peered":"2024-04-25T13:50:31.028328+0000","last_unstale":"2024-04-25T13:50:32.882570+0000","last_undegraded":"2024-04-25T13:50:32.882570+0000","last_fullsized":"2024-04-25T13:50:32.882570+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:46.743026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"170'1","reported_seq":13,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.043531+0000","last_change":"2024-04-25T13:50:29.988408+0000","last_active":"2024-04-25T13:50:31.043531+0000","last_peered":"2024-04-25T13:50:31.043531+0000","last_clean":"2024-04-25T13:50:31.043531+0000","last_became_active":"2024-04-25T13:50:29.988162+0000","last_became_peered":"2024-04-25T13:50:29.988162+0000","last_unstale":"2024-04-25T13:50:31.043531+0000","last_undegraded":"2024-04-25T13:50:31.043531+0000","last_fullsized":"2024-04-25T13:50:31.043531+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:04.187473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041049+0000","last_change":"2024-04-25T13:50:31.023757+0000","last_active":"2024-04-25T13:50:32.041049+0000","last_peered":"2024-04-25T13:50:32.041049+0000","last_clean":"2024-04-25T13:50:32.041049+0000","last_became_active":"2024-04-25T13:50:31.022570+0000","last_became_peered":"2024-04-25T13:50:31.022570+0000","last_unstale":"2024-04-25T13:50:32.041049+0000","last_undegraded":"2024-04-25T13:50:32.041049+0000","last_fullsized":"2024-04-25T13:50:32.041049+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:59:49.262749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983529+0000","last_change":"2024-04-25T13:50:29.978206+0000","last_active":"2024-04-25T13:50:31.983529+0000","last_peered":"2024-04-25T13:50:31.983529+0000","last_clean":"2024-04-25T13:50:31.983529+0000","last_became_active":"2024-04-25T13:50:29.977846+0000","last_became_peered":"2024-04-25T13:50:29.977846+0000","last_unstale":"2024-04-25T13:50:31.983529+0000","last_undegraded":"2024-04-25T13:50:31.983529+0000","last_fullsized":"2024-04-25T13:50:31.983529+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:17.408363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983393+0000","last_change":"2024-04-25T13:50:31.011504+0000","last_active":"2024-04-25T13:50:31.983393+0000","last_peered":"2024-04-25T13:50:31.983393+0000","last_clean":"2024-04-25T13:50:31.983393+0000","last_became_active":"2024-04-25T13:50:31.008565+0000","last_became_peered":"2024-04-25T13:50:31.008565+0000","last_unstale":"2024-04-25T13:50:31.983393+0000","last_undegraded":"2024-04-25T13:50:31.983393+0000","last_fullsized":"2024-04-25T13:50:31.983393+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:53:32.589780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"170'2","reported_seq":14,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.045694+0000","last_change":"2024-04-25T13:50:29.991210+0000","last_active":"2024-04-25T13:50:31.045694+0000","last_peered":"2024-04-25T13:50:31.045694+0000","last_clean":"2024-04-25T13:50:31.045694+0000","last_became_active":"2024-04-25T13:50:29.990503+0000","last_became_peered":"2024-04-25T13:50:29.990503+0000","last_unstale":"2024-04-25T13:50:31.045694+0000","last_undegraded":"2024-04-25T13:50:31.045694+0000","last_fullsized":"2024-04-25T13:50:31.045694+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:55:24.340007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.013261+0000","last_change":"2024-04-25T13:50:31.012955+0000","last_active":"2024-04-25T13:50:31.013261+0000","last_peered":"2024-04-25T13:50:31.013261+0000","last_clean":"2024-04-25T13:50:31.013261+0000","last_became_active":"2024-04-25T13:50:31.012291+0000","last_became_peered":"2024-04-25T13:50:31.012291+0000","last_unstale":"2024-04-25T13:50:31.013261+0000","last_undegraded":"2024-04-25T13:50:31.013261+0000","last_fullsized":"2024-04-25T13:50:31.013261+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:43.751344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042365+0000","last_change":"2024-04-25T13:50:29.994977+0000","last_active":"2024-04-25T13:50:32.042365+0000","last_peered":"2024-04-25T13:50:32.042365+0000","last_clean":"2024-04-25T13:50:32.042365+0000","last_became_active":"2024-04-25T13:50:29.992868+0000","last_became_peered":"2024-04-25T13:50:29.992868+0000","last_unstale":"2024-04-25T13:50:32.042365+0000","last_undegraded":"2024-04-25T13:50:32.042365+0000","last_fullsized":"2024-04-25T13:50:32.042365+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:55.413611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.982954+0000","last_change":"2024-04-25T13:50:31.016958+0000","last_active":"2024-04-25T13:50:31.982954+0000","last_peered":"2024-04-25T13:50:31.982954+0000","last_clean":"2024-04-25T13:50:31.982954+0000","last_became_active":"2024-04-25T13:50:31.015494+0000","last_became_peered":"2024-04-25T13:50:31.015494+0000","last_unstale":"2024-04-25T13:50:31.982954+0000","last_undegraded":"2024-04-25T13:50:31.982954+0000","last_fullsized":"2024-04-25T13:50:31.982954+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:21:46.545434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042921+0000","last_change":"2024-04-25T13:50:29.992141+0000","last_active":"2024-04-25T13:50:32.042921+0000","last_peered":"2024-04-25T13:50:32.042921+0000","last_clean":"2024-04-25T13:50:32.042921+0000","last_became_active":"2024-04-25T13:50:29.989936+0000","last_became_peered":"2024-04-25T13:50:29.989936+0000","last_unstale":"2024-04-25T13:50:32.042921+0000","last_undegraded":"2024-04-25T13:50:32.042921+0000","last_fullsized":"2024-04-25T13:50:32.042921+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:03.260252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.020916+0000","last_change":"2024-04-25T13:50:31.020568+0000","last_active":"2024-04-25T13:50:31.020916+0000","last_peered":"2024-04-25T13:50:31.020916+0000","last_clean":"2024-04-25T13:50:31.020916+0000","last_became_active":"2024-04-25T13:50:31.020030+0000","last_became_peered":"2024-04-25T13:50:31.020030+0000","last_unstale":"2024-04-25T13:50:31.020916+0000","last_undegraded":"2024-04-25T13:50:31.020916+0000","last_fullsized":"2024-04-25T13:50:31.020916+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:23.521776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882654+0000","last_change":"2024-04-25T13:50:29.991731+0000","last_active":"2024-04-25T13:50:32.882654+0000","last_peered":"2024-04-25T13:50:32.882654+0000","last_clean":"2024-04-25T13:50:32.882654+0000","last_became_active":"2024-04-25T13:50:29.982599+0000","last_became_peered":"2024-04-25T13:50:29.982599+0000","last_unstale":"2024-04-25T13:50:32.882654+0000","last_undegraded":"2024-04-25T13:50:32.882654+0000","last_fullsized":"2024-04-25T13:50:32.882654+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:38:05.962434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440904+0000","last_change":"2024-04-25T13:50:31.015507+0000","last_active":"2024-04-25T13:50:32.440904+0000","last_peered":"2024-04-25T13:50:32.440904+0000","last_clean":"2024-04-25T13:50:32.440904+0000","last_became_active":"2024-04-25T13:50:31.010266+0000","last_became_peered":"2024-04-25T13:50:31.010266+0000","last_unstale":"2024-04-25T13:50:32.440904+0000","last_undegraded":"2024-04-25T13:50:32.440904+0000","last_fullsized":"2024-04-25T13:50:32.440904+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:15.267651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041954+0000","last_change":"2024-04-25T13:50:29.994661+0000","last_active":"2024-04-25T13:50:32.041954+0000","last_peered":"2024-04-25T13:50:32.041954+0000","last_clean":"2024-04-25T13:50:32.041954+0000","last_became_active":"2024-04-25T13:50:29.992959+0000","last_became_peered":"2024-04-25T13:50:29.992959+0000","last_unstale":"2024-04-25T13:50:32.041954+0000","last_undegraded":"2024-04-25T13:50:32.041954+0000","last_fullsized":"2024-04-25T13:50:32.041954+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:27.357299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882815+0000","last_change":"2024-04-25T13:50:31.029421+0000","last_active":"2024-04-25T13:50:32.882815+0000","last_peered":"2024-04-25T13:50:32.882815+0000","last_clean":"2024-04-25T13:50:32.882815+0000","last_became_active":"2024-04-25T13:50:31.027993+0000","last_became_peered":"2024-04-25T13:50:31.027993+0000","last_unstale":"2024-04-25T13:50:32.882815+0000","last_undegraded":"2024-04-25T13:50:32.882815+0000","last_fullsized":"2024-04-25T13:50:32.882815+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:21:47.268447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"171'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042938+0000","last_change":"2024-04-25T13:50:29.987983+0000","last_active":"2024-04-25T13:50:32.042938+0000","last_peered":"2024-04-25T13:50:32.042938+0000","last_clean":"2024-04-25T13:50:32.042938+0000","last_became_active":"2024-04-25T13:50:29.987451+0000","last_became_peered":"2024-04-25T13:50:29.987451+0000","last_unstale":"2024-04-25T13:50:32.042938+0000","last_undegraded":"2024-04-25T13:50:32.042938+0000","last_fullsized":"2024-04-25T13:50:32.042938+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:10.021001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983543+0000","last_change":"2024-04-25T13:50:31.020828+0000","last_active":"2024-04-25T13:50:31.983543+0000","last_peered":"2024-04-25T13:50:31.983543+0000","last_clean":"2024-04-25T13:50:31.983543+0000","last_became_active":"2024-04-25T13:50:31.020368+0000","last_became_peered":"2024-04-25T13:50:31.020368+0000","last_unstale":"2024-04-25T13:50:31.983543+0000","last_undegraded":"2024-04-25T13:50:31.983543+0000","last_fullsized":"2024-04-25T13:50:31.983543+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:03.046763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041382+0000","last_change":"2024-04-25T13:50:29.977232+0000","last_active":"2024-04-25T13:50:32.041382+0000","last_peered":"2024-04-25T13:50:32.041382+0000","last_clean":"2024-04-25T13:50:32.041382+0000","last_became_active":"2024-04-25T13:50:29.976199+0000","last_became_peered":"2024-04-25T13:50:29.976199+0000","last_unstale":"2024-04-25T13:50:32.041382+0000","last_undegraded":"2024-04-25T13:50:32.041382+0000","last_fullsized":"2024-04-25T13:50:32.041382+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:25.272511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040391+0000","last_change":"2024-04-25T13:50:31.008847+0000","last_active":"2024-04-25T13:50:32.040391+0000","last_peered":"2024-04-25T13:50:32.040391+0000","last_clean":"2024-04-25T13:50:32.040391+0000","last_became_active":"2024-04-25T13:50:31.007677+0000","last_became_peered":"2024-04-25T13:50:31.007677+0000","last_unstale":"2024-04-25T13:50:32.040391+0000","last_undegraded":"2024-04-25T13:50:32.040391+0000","last_fullsized":"2024-04-25T13:50:32.040391+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:08.616283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.040122+0000","last_change":"2024-04-25T13:50:29.976465+0000","last_active":"2024-04-25T13:50:32.040122+0000","last_peered":"2024-04-25T13:50:32.040122+0000","last_clean":"2024-04-25T13:50:32.040122+0000","last_became_active":"2024-04-25T13:50:29.976089+0000","last_became_peered":"2024-04-25T13:50:29.976089+0000","last_unstale":"2024-04-25T13:50:32.040122+0000","last_undegraded":"2024-04-25T13:50:32.040122+0000","last_fullsized":"2024-04-25T13:50:32.040122+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:55.157353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.004550+0000","last_change":"2024-04-25T13:50:31.004037+0000","last_active":"2024-04-25T13:50:31.004550+0000","last_peered":"2024-04-25T13:50:31.004550+0000","last_clean":"2024-04-25T13:50:31.004550+0000","last_became_active":"2024-04-25T13:50:31.003396+0000","last_became_peered":"2024-04-25T13:50:31.003396+0000","last_unstale":"2024-04-25T13:50:31.004550+0000","last_undegraded":"2024-04-25T13:50:31.004550+0000","last_fullsized":"2024-04-25T13:50:31.004550+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:46:00.687661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":12,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:30.990098+0000","last_change":"2024-04-25T13:50:29.977510+0000","last_active":"2024-04-25T13:50:30.990098+0000","last_peered":"2024-04-25T13:50:30.990098+0000","last_clean":"2024-04-25T13:50:30.990098+0000","last_became_active":"2024-04-25T13:50:29.976987+0000","last_became_peered":"2024-04-25T13:50:29.976987+0000","last_unstale":"2024-04-25T13:50:30.990098+0000","last_undegraded":"2024-04-25T13:50:30.990098+0000","last_fullsized":"2024-04-25T13:50:30.990098+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:14.380568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.440699+0000","last_change":"2024-04-25T13:50:31.026562+0000","last_active":"2024-04-25T13:50:32.440699+0000","last_peered":"2024-04-25T13:50:32.440699+0000","last_clean":"2024-04-25T13:50:32.440699+0000","last_became_active":"2024-04-25T13:50:31.025896+0000","last_became_peered":"2024-04-25T13:50:31.025896+0000","last_unstale":"2024-04-25T13:50:32.440699+0000","last_undegraded":"2024-04-25T13:50:32.440699+0000","last_fullsized":"2024-04-25T13:50:32.440699+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:52.706122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042117+0000","last_change":"2024-04-25T13:50:29.995461+0000","last_active":"2024-04-25T13:50:32.042117+0000","last_peered":"2024-04-25T13:50:32.042117+0000","last_clean":"2024-04-25T13:50:32.042117+0000","last_became_active":"2024-04-25T13:50:29.993923+0000","last_became_peered":"2024-04-25T13:50:29.993923+0000","last_unstale":"2024-04-25T13:50:32.042117+0000","last_undegraded":"2024-04-25T13:50:32.042117+0000","last_fullsized":"2024-04-25T13:50:32.042117+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:44:48.222133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.041717+0000","last_change":"2024-04-25T13:50:30.997717+0000","last_active":"2024-04-25T13:50:32.041717+0000","last_peered":"2024-04-25T13:50:32.041717+0000","last_clean":"2024-04-25T13:50:32.041717+0000","last_became_active":"2024-04-25T13:50:30.996912+0000","last_became_peered":"2024-04-25T13:50:30.996912+0000","last_unstale":"2024-04-25T13:50:32.041717+0000","last_undegraded":"2024-04-25T13:50:32.041717+0000","last_fullsized":"2024-04-25T13:50:32.041717+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:44:38.895955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882686+0000","last_change":"2024-04-25T13:50:29.989447+0000","last_active":"2024-04-25T13:50:32.882686+0000","last_peered":"2024-04-25T13:50:32.882686+0000","last_clean":"2024-04-25T13:50:32.882686+0000","last_became_active":"2024-04-25T13:50:29.988454+0000","last_became_peered":"2024-04-25T13:50:29.988454+0000","last_unstale":"2024-04-25T13:50:32.882686+0000","last_undegraded":"2024-04-25T13:50:32.882686+0000","last_fullsized":"2024-04-25T13:50:32.882686+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:40.770225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.441133+0000","last_change":"2024-04-25T13:50:31.016387+0000","last_active":"2024-04-25T13:50:32.441133+0000","last_peered":"2024-04-25T13:50:32.441133+0000","last_clean":"2024-04-25T13:50:32.441133+0000","last_became_active":"2024-04-25T13:50:31.013357+0000","last_became_peered":"2024-04-25T13:50:31.013357+0000","last_unstale":"2024-04-25T13:50:32.441133+0000","last_undegraded":"2024-04-25T13:50:32.441133+0000","last_fullsized":"2024-04-25T13:50:32.441133+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:27.151496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.983528+0000","last_change":"2024-04-25T13:50:29.986059+0000","last_active":"2024-04-25T13:50:31.983528+0000","last_peered":"2024-04-25T13:50:31.983528+0000","last_clean":"2024-04-25T13:50:31.983528+0000","last_became_active":"2024-04-25T13:50:29.985410+0000","last_became_peered":"2024-04-25T13:50:29.985410+0000","last_unstale":"2024-04-25T13:50:31.983528+0000","last_undegraded":"2024-04-25T13:50:31.983528+0000","last_fullsized":"2024-04-25T13:50:31.983528+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:29:56.609841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":170,"state":"active+clean","last_fresh":"2024-04-25T13:50:31.015869+0000","last_change":"2024-04-25T13:50:31.015549+0000","last_active":"2024-04-25T13:50:31.015869+0000","last_peered":"2024-04-25T13:50:31.015869+0000","last_clean":"2024-04-25T13:50:31.015869+0000","last_became_active":"2024-04-25T13:50:31.014852+0000","last_became_peered":"2024-04-25T13:50:31.014852+0000","last_unstale":"2024-04-25T13:50:31.015869+0000","last_undegraded":"2024-04-25T13:50:31.015869+0000","last_fullsized":"2024-04-25T13:50:31.015869+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:29.950157+0000","last_clean_scrub_stamp":"2024-04-25T13:50:29.950157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:09.410524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"170'3","reported_seq":17,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.882529+0000","last_change":"2024-04-25T13:50:29.985942+0000","last_active":"2024-04-25T13:50:32.882529+0000","last_peered":"2024-04-25T13:50:32.882529+0000","last_clean":"2024-04-25T13:50:32.882529+0000","last_became_active":"2024-04-25T13:50:29.984748+0000","last_became_peered":"2024-04-25T13:50:29.984748+0000","last_unstale":"2024-04-25T13:50:32.882529+0000","last_undegraded":"2024-04-25T13:50:32.882529+0000","last_fullsized":"2024-04-25T13:50:32.882529+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:50:28.938635+0000","last_clean_scrub_stamp":"2024-04-25T13:50:28.938635+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:35.011953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":407,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T13:50:32.042517+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:50:32.042517+0000","last_peered":"2024-04-25T13:50:32.042517+0000","last_clean":"2024-04-25T13:50:32.042517+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:50:32.042517+0000","last_undegraded":"2024-04-25T13:50:32.042517+0000","last_fullsized":"2024-04-25T13:50:32.042517+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1515,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193796,"statfs":{"total":200000000,"available":198447866,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574973,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":938,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038740,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574973,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":938,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038864,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574973,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":938,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574973,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":938,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038796,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574973,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":934,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194378,"statfs":{"total":200000000,"available":199043504,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574973,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":939,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194373,"statfs":{"total":200000000,"available":199038304,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574973,"num_pgs":13,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193801,"statfs":{"total":200000000,"available":198453172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:50:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:50:35.569 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:50:35.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f192a4d5640 1 Processor -- start 2024-04-25T13:50:35.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f192a4d5640 1 -- start start 2024-04-25T13:50:35.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f192a4d5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f192406f5a0 0x7f1924160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f192a4d5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1924161070 0x7f1924161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f192a4d5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f192406fcf0 con 0x7f19240b4bd0 2024-04-25T13:50:35.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f192a4d5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f19241619c0 con 0x7f1924161070 2024-04-25T13:50:35.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f192a4d5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1924073f10 con 0x7f192406f5a0 2024-04-25T13:50:35.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f1928a72640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f19240b4bd0 0x7f19240b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53000/0 (socket says 172.21.15.80:53000) 2024-04-25T13:50:35.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f1928a72640 1 -- 172.21.15.80:0/3671035563 learned_addr learned my addr 172.21.15.80:0/3671035563 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.772+0000 7f19237fe640 1 --2- 172.21.15.80:0/3671035563 >> v2:172.21.15.80:3300/0 conn(0x7f1924161070 0x7f1924161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f1923fff640 1 --2- 172.21.15.80:0/3671035563 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f192406f5a0 0x7f1924160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f1922ffd640 1 -- 172.21.15.80:0/3671035563 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4063022641 0 0) 0x7f192406fcf0 con 0x7f19240b4bd0 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f1922ffd640 1 -- 172.21.15.80:0/3671035563 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1910003530 con 0x7f19240b4bd0 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f19237fe640 1 -- 172.21.15.80:0/3671035563 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f192406f5a0 msgr2=0x7f1924160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f19237fe640 1 --2- 172.21.15.80:0/3671035563 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f192406f5a0 0x7f1924160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f19237fe640 1 -- 172.21.15.80:0/3671035563 >> v1:172.21.15.80:6789/0 conn(0x7f19240b4bd0 legacy=0x7f19240b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f19237fe640 1 -- 172.21.15.80:0/3671035563 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19240b0c60 con 0x7f1924161070 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f19237fe640 1 --2- 172.21.15.80:0/3671035563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924161480 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f190c0097f0 tx=0x7f190c01e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7368b5997133c97f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f1922ffd640 1 -- 172.21.15.80:0/3671035563 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c0ad620 con 0x7f1924161070 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f1922ffd640 1 -- 172.21.15.80:0/3671035563 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f190c0b2070 con 0x7f1924161070 2024-04-25T13:50:35.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f1922ffd640 1 -- 172.21.15.80:0/3671035563 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190c0ad9c0 con 0x7f1924161070 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f192a4d5640 1 -- 172.21.15.80:0/3671035563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 msgr2=0x7f1924161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f192a4d5640 1 --2- 172.21.15.80:0/3671035563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924161480 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f190c0097f0 tx=0x7f190c01e620 comp rx=0 tx=0).stop 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f192a4d5640 1 -- 172.21.15.80:0/3671035563 shutdown_connections 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f192a4d5640 1 --2- 172.21.15.80:0/3671035563 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924161480 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f192a4d5640 1 --2- 172.21.15.80:0/3671035563 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f192406f5a0 0x7f1924160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f192a4d5640 1 -- 172.21.15.80:0/3671035563 >> 172.21.15.80:0/3671035563 conn(0x7f192406d080 msgr2=0x7f19240b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f192a4d5640 1 -- 172.21.15.80:0/3671035563 shutdown_connections 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.776+0000 7f192a4d5640 1 -- 172.21.15.80:0/3671035563 wait complete. 2024-04-25T13:50:35.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 Processor -- start 2024-04-25T13:50:35.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 -- start start 2024-04-25T13:50:35.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f192406f5a0 0x7f1924151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19240b4bd0 0x7f19241555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924142e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f192406ee20 con 0x7f1924161070 2024-04-25T13:50:35.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f192406eca0 con 0x7f192406f5a0 2024-04-25T13:50:35.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f192406efa0 con 0x7f19240b4bd0 2024-04-25T13:50:35.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924142e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924142e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54638/0 (socket says 172.21.15.80:54638) 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 -- 172.21.15.80:0/746505927 learned_addr learned my addr 172.21.15.80:0/746505927 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1923fff640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f192406f5a0 0x7f1924151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f19237fe640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19240b4bd0 0x7f19241555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 -- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19240b4bd0 msgr2=0x7f19241555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19240b4bd0 0x7f19241555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 -- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f192406f5a0 msgr2=0x7f1924151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f192406f5a0 0x7f1924151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 -- 172.21.15.80:0/746505927 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19240b1530 con 0x7f1924161070 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f19237fe640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19240b4bd0 0x7f19241555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1928a72640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924142e80 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f191400cad0 tx=0x7f191400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1920ff9640 1 -- 172.21.15.80:0/746505927 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914017070 con 0x7f1924161070 2024-04-25T13:50:35.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1920ff9640 1 -- 172.21.15.80:0/746505927 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f191400abd0 con 0x7f1924161070 2024-04-25T13:50:35.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1924151690 con 0x7f1924161070 2024-04-25T13:50:35.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f1920ff9640 1 -- 172.21.15.80:0/746505927 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1914012400 con 0x7f1924161070 2024-04-25T13:50:35.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.780+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1924151c40 con 0x7f1924161070 2024-04-25T13:50:35.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.784+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f192406a3a0 con 0x7f1924161070 2024-04-25T13:50:35.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.784+0000 7f1920ff9640 1 -- 172.21.15.80:0/746505927 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f191401a070 con 0x7f1924161070 2024-04-25T13:50:35.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.784+0000 7f1920ff9640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f190406f400 0x7f1904071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:35.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.784+0000 7f1920ff9640 1 -- 172.21.15.80:0/746505927 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f191408f2c0 con 0x7f1924161070 2024-04-25T13:50:35.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.784+0000 7f1923fff640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f190406f400 0x7f1904071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:35.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.784+0000 7f1923fff640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f190406f400 0x7f1904071910 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f19240b6e70 tx=0x7f191c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:35.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.788+0000 7f1920ff9640 1 -- 172.21.15.80:0/746505927 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1914021020 con 0x7f1924161070 2024-04-25T13:50:35.940 INFO:tasks.ceph.mds.b.smithi087.stderr:2024-04-25T13:50:35.941+0000 7fd837c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:50:35.990 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.992+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f19240739c0 con 0x7f1924161070 2024-04-25T13:50:35.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:35.996+0000 7f1920ff9640 1 -- 172.21.15.80:0/746505927 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f191405c3b0 con 0x7f1924161070 2024-04-25T13:50:35.996 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:50:35.996 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:50:35.996 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:50:35.996 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:50:35.996 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:50:35.996 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:50:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f190406f400 msgr2=0x7f1904071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f190406f400 0x7f1904071910 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f19240b6e70 tx=0x7f191c008040 comp rx=0 tx=0).stop 2024-04-25T13:50:35.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 msgr2=0x7f1924142e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924142e80 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f191400cad0 tx=0x7f191400cf90 comp rx=0 tx=0).stop 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 shutdown_connections 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f190406f400 0x7f1904071910 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1924161070 0x7f1924142e80 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f19240b4bd0 0x7f19241555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 --2- 172.21.15.80:0/746505927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f192406f5a0 0x7f1924151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 >> 172.21.15.80:0/746505927 conn(0x7f192406d080 msgr2=0x7f19240cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 shutdown_connections 2024-04-25T13:50:36.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.000+0000 7f192a4d5640 1 -- 172.21.15.80:0/746505927 wait complete. 2024-04-25T13:50:36.018 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:36.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.164+0000 7fa1e789c640 1 Processor -- start 2024-04-25T13:50:36.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.164+0000 7fa1e789c640 1 -- start start 2024-04-25T13:50:36.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e789c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa1e00b1cd0 0x7fa1e0162430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e789c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e006c9b0 con 0x7fa1e015bcc0 2024-04-25T13:50:36.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e789c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1e006f890 con 0x7fa1e00b8420 2024-04-25T13:50:36.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e789c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa1e00b2440 con 0x7fa1e00b1cd0 2024-04-25T13:50:36.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e5638640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa1e015bcc0 0x7fa1e00b7d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53010/0 (socket says 172.21.15.80:53010) 2024-04-25T13:50:36.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e5638640 1 -- 172.21.15.80:0/1837333508 learned_addr learned my addr 172.21.15.80:0/1837333508 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:36.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e5e39640 1 --2- 172.21.15.80:0/1837333508 >> v2:172.21.15.87:3300/0 conn(0x7fa1e00b1cd0 0x7fa1e0162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 344001199 0 0) 0x7fa1d0001120 con 0x7fa1e015bcc0 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa1c8003530 con 0x7fa1e015bcc0 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1960115431 0 0) 0x7fa1c8003530 con 0x7fa1e015bcc0 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 >> v2:172.21.15.87:3300/0 conn(0x7fa1e00b1cd0 msgr2=0x7fa1e0162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 --2- 172.21.15.80:0/1837333508 >> v2:172.21.15.87:3300/0 conn(0x7fa1e00b1cd0 0x7fa1e0162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 >> v1:172.21.15.87:6789/0 conn(0x7fa1e00b8420 legacy=0x7fa1e0161d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e015f600 con 0x7fa1e015bcc0 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fa1d0001850 con 0x7fa1e015bcc0 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 >> v1:172.21.15.80:6789/0 conn(0x7fa1e015bcc0 legacy=0x7fa1e00b7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1c80039c0 0x7fa1c8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c8004810 0x7fa1c8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c80051d0 0x7fa1c800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e00b2440 con 0x7fa1c8004810 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c800c170 con 0x7fa1c80051d0 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c800c370 con 0x7fa1c80039c0 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e4e37640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c8004810 0x7fa1c8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e5638640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1c80039c0 0x7fa1c8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e5e39640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c80051d0 0x7fa1c800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e4e37640 1 -- 172.21.15.80:0/1837333508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1c80039c0 msgr2=0x7fa1c8003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e4e37640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1c80039c0 0x7fa1c8003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e4e37640 1 -- 172.21.15.80:0/1837333508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c80051d0 msgr2=0x7fa1c800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e4e37640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c80051d0 0x7fa1c800b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e5e39640 1 -- 172.21.15.80:0/1837333508 reap_dead start 2024-04-25T13:50:36.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.168+0000 7fa1e4e37640 1 -- 172.21.15.80:0/1837333508 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c8003530 con 0x7fa1c8004810 2024-04-25T13:50:36.171 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e4e37640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c8004810 0x7fa1c8004c20 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4001780 tx=0x7fa1d401c220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=925188c4849077ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d40013a0 con 0x7fa1c8004810 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d40052f0 con 0x7fa1c8004810 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1dffff640 1 -- 172.21.15.80:0/1837333508 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4003840 con 0x7fa1c8004810 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 -- 172.21.15.80:0/1837333508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c8004810 msgr2=0x7fa1c8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c8004810 0x7fa1c8004c20 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4001780 tx=0x7fa1d401c220 comp rx=0 tx=0).stop 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 -- 172.21.15.80:0/1837333508 shutdown_connections 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 --2- 172.21.15.80:0/1837333508 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1c8004810 0x7fa1c8004c20 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 -- 172.21.15.80:0/1837333508 >> 172.21.15.80:0/1837333508 conn(0x7fa1e00cf450 msgr2=0x7fa1e00cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 -- 172.21.15.80:0/1837333508 shutdown_connections 2024-04-25T13:50:36.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 -- 172.21.15.80:0/1837333508 wait complete. 2024-04-25T13:50:36.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 Processor -- start 2024-04-25T13:50:36.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.172+0000 7fa1e789c640 1 -- start start 2024-04-25T13:50:36.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c8004810 0x7fa1e014b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1e01f7650 0x7fa1e01f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1e0159fd0 0x7fa1e015a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e015c0d0 con 0x7fa1e0159fd0 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e015f170 con 0x7fa1c8004810 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e015eb20 con 0x7fa1e01f7650 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1e0159fd0 0x7fa1e015a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1e0159fd0 0x7fa1e015a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54654/0 (socket says 172.21.15.80:54654) 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 -- 172.21.15.80:0/1793822866 learned_addr learned my addr 172.21.15.80:0/1793822866 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e4e37640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1e01f7650 0x7fa1e01f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5638640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c8004810 0x7fa1e014b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 -- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1e01f7650 msgr2=0x7fa1e01f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1e01f7650 0x7fa1e01f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 -- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c8004810 msgr2=0x7fa1e014b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c8004810 0x7fa1e014b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 -- 172.21.15.80:0/1793822866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e01f7fa0 con 0x7fa1e0159fd0 2024-04-25T13:50:36.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e5e39640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1e0159fd0 0x7fa1e015a3e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00d02d0 tx=0x7fa1d800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1ddffb640 1 -- 172.21.15.80:0/1793822866 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8016420 con 0x7fa1e0159fd0 2024-04-25T13:50:36.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1ddffb640 1 -- 172.21.15.80:0/1793822866 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1d801b070 con 0x7fa1e0159fd0 2024-04-25T13:50:36.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e015a920 con 0x7fa1e0159fd0 2024-04-25T13:50:36.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1ddffb640 1 -- 172.21.15.80:0/1793822866 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d8016820 con 0x7fa1e0159fd0 2024-04-25T13:50:36.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e00ad500 con 0x7fa1e0159fd0 2024-04-25T13:50:36.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e006a3a0 con 0x7fa1e0159fd0 2024-04-25T13:50:36.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.176+0000 7fa1ddffb640 1 -- 172.21.15.80:0/1793822866 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa1d80169c0 con 0x7fa1e0159fd0 2024-04-25T13:50:36.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.180+0000 7fa1ddffb640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1cc06f400 0x7fa1cc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.180+0000 7fa1ddffb640 1 -- 172.21.15.80:0/1793822866 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7fa1d8090130 con 0x7fa1e0159fd0 2024-04-25T13:50:36.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.180+0000 7fa1e5638640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1cc06f400 0x7fa1cc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.180+0000 7fa1e5638640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1cc06f400 0x7fa1cc071910 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0004880 tx=0x7fa1d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.184+0000 7fa1ddffb640 1 -- 172.21.15.80:0/1793822866 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa1e015a920 con 0x7fa1e0159fd0 2024-04-25T13:50:36.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1e014b920 con 0x7fa1e0159fd0 2024-04-25T13:50:36.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1ddffb640 1 -- 172.21.15.80:0/1793822866 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5176 (secure 0 0 0) 0x7fa1e01f7fa0 con 0x7fa1e0159fd0 2024-04-25T13:50:36.366 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:36.366 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/29039900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":29039900},{"type":"v1","addr":"172.21.15.87:6836","nonce":29039900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1466426643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1466426643},{"type":"v1","addr":"172.21.15.80:6833","nonce":1466426643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1170719891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1170719891},{"type":"v1","addr":"172.21.15.80:6835","nonce":1170719891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:50:30.371502+0000","modified":"2024-04-25T13:50:33.461986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/986211877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":986211877},{"type":"v1","addr":"172.21.15.87:6837","nonce":986211877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":10}]} 2024-04-25T13:50:36.366 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 81 2024-04-25T13:50:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1cc06f400 msgr2=0x7fa1cc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1cc06f400 0x7fa1cc071910 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0004880 tx=0x7fa1d0009040 comp rx=0 tx=0).stop 2024-04-25T13:50:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1e0159fd0 msgr2=0x7fa1e015a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1e0159fd0 0x7fa1e015a3e0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fa1e00d02d0 tx=0x7fa1d800f520 comp rx=0 tx=0).stop 2024-04-25T13:50:36.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 shutdown_connections 2024-04-25T13:50:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa1cc06f400 0x7fa1cc071910 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa1e0159fd0 0x7fa1e015a3e0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa1e01f7650 0x7fa1e01f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 --2- 172.21.15.80:0/1793822866 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa1c8004810 0x7fa1e014b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 >> 172.21.15.80:0/1793822866 conn(0x7fa1e00cf450 msgr2=0x7fa1e00d1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 shutdown_connections 2024-04-25T13:50:36.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.368+0000 7fa1e789c640 1 -- 172.21.15.80:0/1793822866 wait complete. 2024-04-25T13:50:36.789 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:50:30.371502+0000', 'modified': '2024-04-25T13:50:33.461986+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8814}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8814': {'gid': 8814, 'name': 'b', 'rank': 0, 'incarnation': 78, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.87:6837/986211877', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6835', 'nonce': 986211877}, {'type': 'v1', 'addr': '172.21.15.87:6837', 'nonce': 986211877}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8814, 'qdb_cluster': [8814]} 2024-04-25T13:50:36.789 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:50:36.789 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T13:50:36.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.948+0000 7f8334400640 1 Processor -- start 2024-04-25T13:50:36.947 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.948+0000 7f8334400640 1 -- start start 2024-04-25T13:50:36.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f8334400640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f832c0b0ed0 0x7f832c15c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f8334400640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f832c0ce730 0x7f832c0ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f8334400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 0x7f832c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f8334400640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f832c06ede0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f8334400640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f832c15cda0 con 0x7f832c0ce730 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f8334400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f832c073710 con 0x7f832c06e490 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833219c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f832c0b0ed0 0x7f832c15c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833219c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f832c0b0ed0 0x7f832c15c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54660/0 (socket says 172.21.15.80:54660) 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833219c640 1 -- 172.21.15.80:0/980580496 learned_addr learned my addr 172.21.15.80:0/980580496 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833199b640 1 --2- 172.21.15.80:0/980580496 >> v2:172.21.15.87:3300/0 conn(0x7f832c0ce730 0x7f832c0ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833299d640 1 --2- 172.21.15.80:0/980580496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 0x7f832c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833219c640 1 -- 172.21.15.80:0/980580496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 msgr2=0x7f832c06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833219c640 1 --2- 172.21.15.80:0/980580496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 0x7f832c06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833219c640 1 -- 172.21.15.80:0/980580496 >> v2:172.21.15.87:3300/0 conn(0x7f832c0ce730 msgr2=0x7f832c0ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833219c640 1 --2- 172.21.15.80:0/980580496 >> v2:172.21.15.87:3300/0 conn(0x7f832c0ce730 0x7f832c0ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.951 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.952+0000 7f833219c640 1 -- 172.21.15.80:0/980580496 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c0739f0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f833219c640 1 --2- 172.21.15.80:0/980580496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c15c860 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f832c071c80 tx=0x7f832801ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed021683942391c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f833119a640 1 -- 172.21.15.80:0/980580496 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280b5070 con 0x7f832c0b0ed0 2024-04-25T13:50:36.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f833119a640 1 -- 172.21.15.80:0/980580496 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83280a89d0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f833119a640 1 -- 172.21.15.80:0/980580496 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280a8da0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 -- 172.21.15.80:0/980580496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 msgr2=0x7f832c15c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 --2- 172.21.15.80:0/980580496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c15c860 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f832c071c80 tx=0x7f832801ed10 comp rx=0 tx=0).stop 2024-04-25T13:50:36.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 -- 172.21.15.80:0/980580496 shutdown_connections 2024-04-25T13:50:36.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 --2- 172.21.15.80:0/980580496 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 0x7f832c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 --2- 172.21.15.80:0/980580496 >> v2:172.21.15.87:3300/0 conn(0x7f832c0ce730 0x7f832c0ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 --2- 172.21.15.80:0/980580496 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c15c860 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 -- 172.21.15.80:0/980580496 >> 172.21.15.80:0/980580496 conn(0x7f832c06d080 msgr2=0x7f832c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:36.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 -- 172.21.15.80:0/980580496 shutdown_connections 2024-04-25T13:50:36.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 -- 172.21.15.80:0/980580496 wait complete. 2024-04-25T13:50:36.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 Processor -- start 2024-04-25T13:50:36.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.956+0000 7f8334400640 1 -- start start 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8334400640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 0x7f832c15a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8334400640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c0acf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8334400640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f832c0ce730 0x7f832c1f8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8334400640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f832c0b12e0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8334400640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f832c073fb0 con 0x7f832c0ce730 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8334400640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f832c0733a0 con 0x7f832c06e490 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c0acf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c0acf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54674/0 (socket says 172.21.15.80:54674) 2024-04-25T13:50:36.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 -- 172.21.15.80:0/2757724761 learned_addr learned my addr 172.21.15.80:0/2757724761 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833219c640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 0x7f832c15a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833299d640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f832c0ce730 0x7f832c1f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 -- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 msgr2=0x7f832c15a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 0x7f832c15a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 -- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f832c0ce730 msgr2=0x7f832c1f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f832c0ce730 0x7f832c1f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 -- 172.21.15.80:0/2757724761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c154be0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833299d640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f832c0ce730 0x7f832c1f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f833199b640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c0acf20 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f831c00d5f0 tx=0x7f831c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8322ffd640 1 -- 172.21.15.80:0/2757724761 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c0125f0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8322ffd640 1 -- 172.21.15.80:0/2757724761 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f831c019070 con 0x7f832c0b0ed0 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8334400640 1 -- 172.21.15.80:0/2757724761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c1f8fe0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8322ffd640 1 -- 172.21.15.80:0/2757724761 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c0129b0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.961 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8334400640 1 -- 172.21.15.80:0/2757724761 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c159cf0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.960+0000 7f8322ffd640 1 -- 172.21.15.80:0/2757724761 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f831c012b50 con 0x7f832c0b0ed0 2024-04-25T13:50:36.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f8322ffd640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f831006f430 0x7f8310071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f8322ffd640 1 -- 172.21.15.80:0/2757724761 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f831c08f6f0 con 0x7f832c0b0ed0 2024-04-25T13:50:36.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f831006f430 0x7f8310071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833b75d000 1 Processor -- start 2024-04-25T13:50:36.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833b75d000 1 -- start start 2024-04-25T13:50:36.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f831006f430 0x7f8310071940 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f83280b2000 tx=0x7f8328008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833b75d000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a766ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833b75d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 0x564b5a767420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833b75d000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 0x564b5a767960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833b75d000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x564b5a5d6160 con 0x564b5a608c20 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833b75d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564b5a56be20 con 0x564b5a5f4ea0 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833b75d000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x564b5a524a20 con 0x564b5a1ed280 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a766ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a766ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54690/0 (socket says 172.21.15.80:54690) 2024-04-25T13:50:36.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 -- 172.21.15.80:0/4028030394 learned_addr learned my addr 172.21.15.80:0/4028030394 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833199b640 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 0x564b5a767420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833299d640 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 0x564b5a767960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 -- 172.21.15.80:0/4028030394 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 msgr2=0x564b5a767960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 0x564b5a767960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 -- 172.21.15.80:0/4028030394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 msgr2=0x564b5a767420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 0x564b5a767420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 -- 172.21.15.80:0/4028030394 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b5a529100 con 0x564b5a608c20 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f833219c640 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a766ee0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f8328008010 tx=0x7f8328004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ed6b2808771dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f830ffff640 1 -- 172.21.15.80:0/4028030394 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280cb070 con 0x564b5a608c20 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f830ffff640 1 -- 172.21.15.80:0/4028030394 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83280b0440 con 0x564b5a608c20 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.964+0000 7f830ffff640 1 -- 172.21.15.80:0/4028030394 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280b5040 con 0x564b5a608c20 2024-04-25T13:50:36.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- 172.21.15.80:0/4028030394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 msgr2=0x564b5a766ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a766ee0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f8328008010 tx=0x7f8328004740 comp rx=0 tx=0).stop 2024-04-25T13:50:36.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- 172.21.15.80:0/4028030394 shutdown_connections 2024-04-25T13:50:36.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 0x564b5a767960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 0x564b5a767420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 --2- 172.21.15.80:0/4028030394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a766ee0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- 172.21.15.80:0/4028030394 >> 172.21.15.80:0/4028030394 conn(0x564b5a22cff0 msgr2=0x564b5a71c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:36.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- 172.21.15.80:0/4028030394 shutdown_connections 2024-04-25T13:50:36.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- 172.21.15.80:0/4028030394 wait complete. 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- start start 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 Processor -- start 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a1ed280 0x564b5a76de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a76e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a608c20 0x564b5a7290c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x564b5a579220 con 0x564b5a1ed280 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564b5a50fea0 con 0x564b5a608c20 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x564b5a566b00 con 0x564b5a5f4ea0 2024-04-25T13:50:36.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 10 client.0 _ll_register_callbacks cb 0x564b5a578dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a1ed280 0x564b5a76de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a1ed280 0x564b5a76de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54706/0 (socket says 172.21.15.80:54706) 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 -- 172.21.15.80:0/2964642803 learned_addr learned my addr 172.21.15.80:0/2964642803 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833199b640 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a76e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833299d640 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a608c20 0x564b5a7290c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 -- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 msgr2=0x564b5a76e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a76e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 -- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a608c20 msgr2=0x564b5a7290c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a608c20 0x564b5a7290c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 -- 172.21.15.80:0/2964642803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5a50b500 con 0x564b5a1ed280 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833199b640 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a76e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833219c640 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a1ed280 0x564b5a76de60 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f8328008010 tx=0x7f832801d550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f31cb985f940fe3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f830d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:50:36.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f830d7fa640 1 -- 172.21.15.80:0/2964642803 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83280cb030 con 0x564b5a1ed280 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f830d7fa640 1 -- 172.21.15.80:0/2964642803 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f83280c1040 con 0x564b5a1ed280 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- 172.21.15.80:0/2964642803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83280b0440 con 0x564b5a1ed280 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f830d7fa640 1 -- 172.21.15.80:0/2964642803 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f83280b5040 con 0x564b5a1ed280 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 10 client.8943 fetch_fsmap learned FSMap version 81 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.968+0000 7f833b75d000 1 -- 172.21.15.80:0/2964642803 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x564b5a4f1ea0 con 0x564b5a1ed280 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f830d7fa640 1 -- 172.21.15.80:0/2964642803 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83280b0440 con 0x564b5a1ed280 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f830d7fa640 1 -- 172.21.15.80:0/2964642803 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f83280b8030 con 0x564b5a1ed280 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 10 client.8943 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 10 client.8943 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 20 client.8943 populate_metadata read hostname 'smithi080' 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] conn(0x564b5a2117d0 0x564b5a729e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 4 client.8943 mds_command: new command op to 8814 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:50:36.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 -- 172.21.15.80:0/2964642803 --> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b5a504be0 con 0x564b5a2117d0 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833199b640 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] conn(0x564b5a2117d0 0x564b5a729e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f830d7fa640 10 client.8943 ms_handle_connect on v2:172.21.15.87:6835/986211877 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833199b640 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] conn(0x564b5a2117d0 0x564b5a729e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccd48b9978e34977 server_cookie=c7e56bedae8f18af in_seq=0 out_seq=0 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f830d7fa640 1 -- 172.21.15.80:0/2964642803 <== mds.0 v2:172.21.15.87:6835/986211877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b5a504be0 con 0x564b5a2117d0 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f830d7fa640 10 client.8943 handle_command_reply: tid=0 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 client.8943 shutdown 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 -- 172.21.15.80:0/2964642803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a1ed280 msgr2=0x564b5a76de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a1ed280 0x564b5a76de60 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f8328008010 tx=0x7f832801d550 comp rx=0 tx=0).stop 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 -- 172.21.15.80:0/2964642803 shutdown_connections 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] conn(0x564b5a2117d0 0x564b5a729e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a608c20 0x564b5a7290c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a76e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f830d7fa640 0 client.8943 ms_handle_reset on v2:172.21.15.87:6835/986211877 2024-04-25T13:50:36.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 --2- 172.21.15.80:0/2964642803 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a1ed280 0x564b5a76de60 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 -- 172.21.15.80:0/2964642803 >> 172.21.15.80:0/2964642803 conn(0x564b5a22cff0 msgr2=0x564b5a71cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:36.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 -- 172.21.15.80:0/2964642803 shutdown_connections 2024-04-25T13:50:36.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 1 -- 172.21.15.80:0/2964642803 wait complete. 2024-04-25T13:50:36.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.972+0000 7f833b75d000 20 client.8943 trim_cache size 0 max 16384 2024-04-25T13:50:36.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 Processor -- start 2024-04-25T13:50:36.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- start start 2024-04-25T13:50:36.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a1ed280 0x564b5a7732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a77cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a77d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:36.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f832c15be20 con 0x564b5a608c20 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f832c0abc10 con 0x564b5a1ed280 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f832c15b7d0 con 0x564b5a5f4ea0 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a77d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a77d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54712/0 (socket says 172.21.15.80:54712) 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 -- 172.21.15.80:0/90930646 learned_addr learned my addr 172.21.15.80:0/90930646 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833219c640 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a1ed280 0x564b5a7732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833199b640 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a77cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 -- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 msgr2=0x564b5a77cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a77cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 -- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a1ed280 msgr2=0x564b5a7732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:36.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a1ed280 0x564b5a7732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 -- 172.21.15.80:0/90930646 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b5a720840 con 0x564b5a608c20 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833199b640 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a77cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833219c640 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a1ed280 0x564b5a7732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833299d640 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a77d530 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f8324009e30 tx=0x7f8324007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5492861cecfcf8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f830f7fe640 1 -- 172.21.15.80:0/90930646 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832401b030 con 0x564b5a608c20 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f830f7fe640 1 -- 172.21.15.80:0/90930646 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83240039a0 con 0x564b5a608c20 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f830f7fe640 1 -- 172.21.15.80:0/90930646 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324003b70 con 0x564b5a608c20 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- 172.21.15.80:0/90930646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 msgr2=0x564b5a77d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a77d530 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f8324009e30 tx=0x7f8324007700 comp rx=0 tx=0).stop 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- 172.21.15.80:0/90930646 shutdown_connections 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a77d530 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a5f4ea0 0x564b5a77cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 --2- 172.21.15.80:0/90930646 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a1ed280 0x564b5a7732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:36.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- 172.21.15.80:0/90930646 >> 172.21.15.80:0/90930646 conn(0x564b5a22cff0 msgr2=0x564b5a724050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:36.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- 172.21.15.80:0/90930646 shutdown_connections 2024-04-25T13:50:36.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.992+0000 7f833b75d000 1 -- 172.21.15.80:0/90930646 wait complete. 2024-04-25T13:50:37.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 -- start start 2024-04-25T13:50:37.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 Processor -- start 2024-04-25T13:50:37.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 0x564b5a7d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 0x564b5a7d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a771890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x564b5a56b130 con 0x564b5a608c20 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x564b5a5017d0 con 0x564b5a5f4ea0 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x564b5a570450 con 0x564b5a1ed280 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 10 client.0 _ll_register_callbacks cb 0x7f83280c9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a771890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a771890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54720/0 (socket says 172.21.15.80:54720) 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 -- 172.21.15.80:0/1051747302 learned_addr learned my addr 172.21.15.80:0/1051747302 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833219c640 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 0x564b5a7d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833199b640 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 0x564b5a7d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 -- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 msgr2=0x564b5a7d9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 0x564b5a7d9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 -- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 msgr2=0x564b5a7d9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 0x564b5a7d9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 -- 172.21.15.80:0/1051747302 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b5a729430 con 0x564b5a608c20 2024-04-25T13:50:37.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833299d640 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a771890 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f8324007740 tx=0x7f8324004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33edc2f9ad80c12f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f830d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f830d7fa640 1 -- 172.21.15.80:0/1051747302 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8324017dc0 con 0x564b5a608c20 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f830d7fa640 1 -- 172.21.15.80:0/1051747302 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f8324004020 con 0x564b5a608c20 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 -- 172.21.15.80:0/1051747302 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83240039a0 con 0x564b5a608c20 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f830d7fa640 1 -- 172.21.15.80:0/1051747302 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f8324032050 con 0x564b5a608c20 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 10 client.8949 fetch_fsmap learned FSMap version 81 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 -- 172.21.15.80:0/1051747302 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x564b5a524c70 con 0x564b5a608c20 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f830d7fa640 1 -- 172.21.15.80:0/1051747302 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83240039a0 con 0x564b5a608c20 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f830d7fa640 1 -- 172.21.15.80:0/1051747302 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 81) ==== 3494+0+0 (secure 0 0 0) 0x7f8324035020 con 0x564b5a608c20 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 10 client.8949 fetch_fsmap finished waiting for FSMap version 81 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 10 client.8949 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 20 client.8949 populate_metadata read hostname 'smithi080' 2024-04-25T13:50:37.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] conn(0x564b5a1ff830 0x564b5a772250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 4 client.8949 mds_command: new command op to 8814 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833b75d000 1 -- 172.21.15.80:0/1051747302 --> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] -- command(tid 0: {"prefix": "status"}) -- 0x564b5a579a30 con 0x564b5a1ff830 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:36.996+0000 7f833219c640 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] conn(0x564b5a1ff830 0x564b5a772250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f830d7fa640 10 client.8949 ms_handle_connect on v2:172.21.15.87:6835/986211877 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833219c640 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] conn(0x564b5a1ff830 0x564b5a772250 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6de7c3415f1a4f server_cookie=c763d76eba1d2e74 in_seq=0 out_seq=0 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f830d7fa640 1 -- 172.21.15.80:0/1051747302 <== mds.0 v2:172.21.15.87:6835/986211877 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x564b5a579a30 con 0x564b5a1ff830 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f830d7fa640 10 client.8949 handle_command_reply: tid=0 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 client.8949 shutdown 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 -- 172.21.15.80:0/1051747302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 msgr2=0x564b5a771890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a771890 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f8324007740 tx=0x7f8324004ac0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 -- 172.21.15.80:0/1051747302 shutdown_connections 2024-04-25T13:50:37.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:6835/986211877,v1:172.21.15.87:6837/986211877] conn(0x564b5a1ff830 0x564b5a772250 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x564b5a608c20 0x564b5a771890 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x564b5a5f4ea0 0x564b5a7d9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f830d7fa640 0 client.8949 ms_handle_reset on v2:172.21.15.87:6835/986211877 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 --2- 172.21.15.80:0/1051747302 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x564b5a1ed280 0x564b5a7d9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 -- 172.21.15.80:0/1051747302 >> 172.21.15.80:0/1051747302 conn(0x564b5a22cff0 msgr2=0x564b5a76c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 -- 172.21.15.80:0/1051747302 shutdown_connections 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 1 -- 172.21.15.80:0/1051747302 wait complete. 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f833b75d000 20 client.8949 trim_cache size 0 max 16384 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:50:37.005 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 8814, 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 5.9764620810000002, 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 81, 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 171, 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 171, 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 11.953677645999999 2024-04-25T13:50:37.006 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:50:37.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 -- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f831006f430 msgr2=0x7f8310071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f831006f430 0x7f8310071940 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f83280b2000 tx=0x7f8328008040 comp rx=0 tx=0).stop 2024-04-25T13:50:37.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 -- 172.21.15.80:0/2757724761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 msgr2=0x7f832c0acf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c0acf20 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f831c00d5f0 tx=0x7f831c00dab0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 -- 172.21.15.80:0/2757724761 shutdown_connections 2024-04-25T13:50:37.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f831006f430 0x7f8310071940 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f832c0ce730 0x7f832c1f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f832c0b0ed0 0x7f832c0acf20 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 --2- 172.21.15.80:0/2757724761 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f832c06e490 0x7f832c15a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 -- 172.21.15.80:0/2757724761 >> 172.21.15.80:0/2757724761 conn(0x7f832c06d080 msgr2=0x7f832c0b2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:37.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 -- 172.21.15.80:0/2757724761 shutdown_connections 2024-04-25T13:50:37.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.000+0000 7f8334400640 1 -- 172.21.15.80:0/2757724761 wait complete. 2024-04-25T13:50:37.016 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:50:37.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91caaec640 1 Processor -- start 2024-04-25T13:50:37.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91caaec640 1 -- start start 2024-04-25T13:50:37.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91caaec640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f91c40d14a0 0x7f91c40b4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91caaec640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c40d26d0 0x7f91c40d2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91caaec640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f91c40b9210 con 0x7f91c40b7740 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91caaec640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f91c40d18b0 con 0x7f91c40d14a0 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91caaec640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91c40b5120 con 0x7f91c40d26d0 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c9089640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f91c40b7740 0x7f91c40b7b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53014/0 (socket says 172.21.15.80:53014) 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c9089640 1 -- 172.21.15.80:0/170428295 learned_addr learned my addr 172.21.15.80:0/170428295 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c3fff640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c40d26d0 0x7f91c40d2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c8888640 1 --2- 172.21.15.80:0/170428295 >> v2:172.21.15.87:3300/0 conn(0x7f91c40d14a0 0x7f91c40b4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3757385765 0 0) 0x7f91bc0014a0 con 0x7f91c40b7740 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91ac003530 con 0x7f91c40b7740 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3303182153 0 0) 0x7f91ac003530 con 0x7f91c40b7740 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c40d26d0 msgr2=0x7f91c40d2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:37.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c40d26d0 0x7f91c40d2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 >> v2:172.21.15.87:3300/0 conn(0x7f91c40d14a0 msgr2=0x7f91c40b4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 --2- 172.21.15.80:0/170428295 >> v2:172.21.15.87:3300/0 conn(0x7f91c40d14a0 0x7f91c40b4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c40d56b0 con 0x7f91c40b7740 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f91bc001bd0 con 0x7f91c40b7740 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 >> v1:172.21.15.80:6789/0 conn(0x7f91c40b7740 legacy=0x7f91c40b7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ac0039f0 0x7f91ac003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ac004810 0x7f91ac004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c40d26d0 0x7f91ac005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f91c40b5120 con 0x7f91ac004810 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac00ba80 con 0x7f91ac0039f0 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91ac00bc30 con 0x7f91c40d26d0 2024-04-25T13:50:37.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c3fff640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ac004810 0x7f91ac004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c9089640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c40d26d0 0x7f91ac005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.164+0000 7f91c8888640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ac0039f0 0x7f91ac003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c3fff640 1 -- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c40d26d0 msgr2=0x7f91ac005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:37.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c3fff640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91c40d26d0 0x7f91ac005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c3fff640 1 -- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ac0039f0 msgr2=0x7f91ac003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:37.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c3fff640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91ac0039f0 0x7f91ac003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c3fff640 1 -- 172.21.15.80:0/170428295 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac003530 con 0x7f91ac004810 2024-04-25T13:50:37.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c3fff640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ac004810 0x7f91ac004c20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009a90 tx=0x7f91b801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e5af2d690c65e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:37.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8009690 con 0x7f91ac004810 2024-04-25T13:50:37.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91b80059a0 con 0x7f91ac004810 2024-04-25T13:50:37.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c37fe640 1 -- 172.21.15.80:0/170428295 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b8005d90 con 0x7f91ac004810 2024-04-25T13:50:37.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91caaec640 1 -- 172.21.15.80:0/170428295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ac004810 msgr2=0x7f91ac004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91caaec640 1 --2- 172.21.15.80:0/170428295 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91ac004810 0x7f91ac004c20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f91b8009a90 tx=0x7f91b801ca30 comp rx=0 tx=0).stop 2024-04-25T13:50:37.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91c9089640 1 -- 172.21.15.80:0/170428295 reap_dead start 2024-04-25T13:50:37.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91caaec640 1 -- 172.21.15.80:0/170428295 shutdown_connections 2024-04-25T13:50:37.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91caaec640 1 -- 172.21.15.80:0/170428295 >> 172.21.15.80:0/170428295 conn(0x7f91c4073770 msgr2=0x7f91c406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:37.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91caaec640 1 -- 172.21.15.80:0/170428295 shutdown_connections 2024-04-25T13:50:37.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.168+0000 7f91caaec640 1 -- 172.21.15.80:0/170428295 wait complete. 2024-04-25T13:50:37.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 Processor -- start 2024-04-25T13:50:37.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 -- start start 2024-04-25T13:50:37.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ac004810 0x7f91c41fa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c40d0350 0x7f91c415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c415f2f0 0x7f91c40ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f91c4164d20 con 0x7f91c415f2f0 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f91c4164800 con 0x7f91c40d0350 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91c40b4da0 con 0x7f91ac004810 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c415f2f0 0x7f91c40ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c415f2f0 0x7f91c40ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54736/0 (socket says 172.21.15.80:54736) 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 -- 172.21.15.80:0/474579364 learned_addr learned my addr 172.21.15.80:0/474579364 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c8888640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ac004810 0x7f91c41fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c3fff640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c40d0350 0x7f91c415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 -- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ac004810 msgr2=0x7f91c41fa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ac004810 0x7f91c41fa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 -- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c40d0350 msgr2=0x7f91c415edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c40d0350 0x7f91c415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 -- 172.21.15.80:0/474579364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c40c9370 con 0x7f91c415f2f0 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c8888640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ac004810 0x7f91c41fa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c9089640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c415f2f0 0x7f91c40ca910 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f91bc002240 tx=0x7f91bc00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c17fa640 1 -- 172.21.15.80:0/474579364 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00eb90 con 0x7f91c415f2f0 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c17fa640 1 -- 172.21.15.80:0/474579364 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91bc019070 con 0x7f91c415f2f0 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91c17fa640 1 -- 172.21.15.80:0/474579364 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc013420 con 0x7f91c415f2f0 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c40c9870 con 0x7f91c415f2f0 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.172+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c40c9e30 con 0x7f91c415f2f0 2024-04-25T13:50:37.174 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.176+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91c40c95e0 con 0x7f91c415f2f0 2024-04-25T13:50:37.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.176+0000 7f91c17fa640 1 -- 172.21.15.80:0/474579364 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f91bc0135c0 con 0x7f91c415f2f0 2024-04-25T13:50:37.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.176+0000 7f91c17fa640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91b006f430 0x7f91b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:37.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.176+0000 7f91c17fa640 1 -- 172.21.15.80:0/474579364 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f91bc08ff50 con 0x7f91c415f2f0 2024-04-25T13:50:37.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.176+0000 7f91c8888640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91b006f430 0x7f91b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:37.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.176+0000 7f91c8888640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91b006f430 0x7f91b0071940 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009aa0 tx=0x7f91b4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:37.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.180+0000 7f91c17fa640 1 -- 172.21.15.80:0/474579364 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f91c40b9210 con 0x7f91c415f2f0 2024-04-25T13:50:37.373 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f91c40d4130 con 0x7f91c415f2f0 2024-04-25T13:50:37.374 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91c17fa640 1 -- 172.21.15.80:0/474579364 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f91bc05cfb0 con 0x7f91c415f2f0 2024-04-25T13:50:37.374 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:50:37.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91b006f430 msgr2=0x7f91b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91b006f430 0x7f91b0071940 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f91b4009aa0 tx=0x7f91b4008040 comp rx=0 tx=0).stop 2024-04-25T13:50:37.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c415f2f0 msgr2=0x7f91c40ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:37.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c415f2f0 0x7f91c40ca910 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f91bc002240 tx=0x7f91bc00e530 comp rx=0 tx=0).stop 2024-04-25T13:50:37.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 shutdown_connections 2024-04-25T13:50:37.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91b006f430 0x7f91b0071940 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91c415f2f0 0x7f91c40ca910 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91c40d0350 0x7f91c415edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 --2- 172.21.15.80:0/474579364 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f91ac004810 0x7f91c41fa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:37.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 >> 172.21.15.80:0/474579364 conn(0x7f91c4073770 msgr2=0x7f91c40d4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:37.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 shutdown_connections 2024-04-25T13:50:37.377 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37.376+0000 7f91caaec640 1 -- 172.21.15.80:0/474579364 wait complete. 2024-04-25T13:50:37.396 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:50:37.396 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:50:37.397 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:50:37.397 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:50:37.397 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:50:37.397 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:50:37.397 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:50:37.397 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:50:37.397 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:50:37.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:37.397 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:50:37.403 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:50:37.403 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:50:37.403 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:50:37.403 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:37.403 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:50:37.404 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 41931sec preferred_lft 41931sec 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:37.405 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:37.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:37.407 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:50:37.454 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:50:37.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:37.456 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:50:37.507 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:50:37.507 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:50:37.507 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:50:37.508 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:50:37.508 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:50:37.508 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:50:37.508 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:50:37.508 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:50:37.508 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:50:37.508 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:50:37.508 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:50:37.508 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:50:37.508 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:50:37.508 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:50:37.509 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:50:37.509 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:50:37.509 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:50:37.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:50:37.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:50:37.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:37.627 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:50:37.632 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:50:37.632 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:50:37.632 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:50:37.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:37.633 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:50:37.689 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:50:37.689 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:50:37.689 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:50:37.736 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:50:37.737 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:50:37.737 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:50:37.785 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:50:37.842 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:50:37.890 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:50:37.890 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:50:37.890 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:50:37.890 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:50:37.890 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:50:37.891 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:50:37.892 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:50:37.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:37.893 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:50:37.941 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:50:37.941 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:50:37.986 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:50:38.001 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:50:38.049 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:50:38.049 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:50:38.049 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:50:38.050 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:50:38.051 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:50:38.052 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:50:38.052 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:50:38.052 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:50:38.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:38.052 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:50:38.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:50:38.100+0000 7f1b72421580 -1 init, newargv = 0x7f1b5401b5a0 newargc=15 2024-04-25T13:50:38.099 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[42304]: starting ceph client 2024-04-25T13:50:38.104 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[42304]: starting fuse 2024-04-25T13:50:39.197 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:50:39.214 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:50:39.262 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:50:39.262 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:50:39.262 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:50:39.262 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:50:39.262 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:50:39.263 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:50:39.264 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:50:39.265 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:50:39.265 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:50:39.265 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:50:39.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:39.266 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:50:39.317 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:50:39.318 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:39.318 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:50:39.319 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:50:39.441 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.42304.asok 2024-04-25T13:50:39.445 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:50:39.453 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.42304.asok 2024-04-25T13:50:39.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:39.453 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.42304.asok status 2024-04-25T13:50:39.632 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:50:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:50:39.632 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "42304", 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 8958, 2024-04-25T13:50:39.633 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 8958 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3377968529 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:50:39.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3377968529 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.8958 192.168.0.1:0/3377968529", 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/3377968529", 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 81, 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 171, 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:50:39.635 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:50:39.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:39.649 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:50:39.649 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:50:39.655 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:50:39.656 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:50:39.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:39.656 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:50:39.709 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:50:39.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f8453304640 1 Processor -- start 2024-04-25T13:50:39.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f8453304640 1 -- start start 2024-04-25T13:50:39.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f8453304640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 0x7f844c0d0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f8453304640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f844c0d1320 0x7f844c06d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f8453304640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f844c06db40 0x7f844c160e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f8453304640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f844c1609e0 con 0x7f844c06db40 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f8453304640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f844c06df50 con 0x7f844c0d1320 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f8453304640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f844c0b1e30 con 0x7f844c06f1b0 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84518a1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f844c06db40 0x7f844c160e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84518a1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f844c06db40 0x7f844c160e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54744/0 (socket says 172.21.15.80:54744) 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84518a1640 1 -- 172.21.15.80:0/3985331668 learned_addr learned my addr 172.21.15.80:0/3985331668 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84510a0640 1 --2- 172.21.15.80:0/3985331668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 0x7f844c0d0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f845089f640 1 --2- 172.21.15.80:0/3985331668 >> v2:172.21.15.87:3300/0 conn(0x7f844c0d1320 0x7f844c06d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84518a1640 1 -- 172.21.15.80:0/3985331668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 msgr2=0x7f844c0d0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84518a1640 1 --2- 172.21.15.80:0/3985331668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 0x7f844c0d0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84518a1640 1 -- 172.21.15.80:0/3985331668 >> v2:172.21.15.87:3300/0 conn(0x7f844c0d1320 msgr2=0x7f844c06d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84518a1640 1 --2- 172.21.15.80:0/3985331668 >> v2:172.21.15.87:3300/0 conn(0x7f844c0d1320 0x7f844c06d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.907 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.908+0000 7f84518a1640 1 -- 172.21.15.80:0/3985331668 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c0b2110 con 0x7f844c06db40 2024-04-25T13:50:39.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f84518a1640 1 --2- 172.21.15.80:0/3985331668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c06db40 0x7f844c160e70 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f844c069620 tx=0x7f844801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a42356038748e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8443fff640 1 -- 172.21.15.80:0/3985331668 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480ab5f0 con 0x7f844c06db40 2024-04-25T13:50:39.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8443fff640 1 -- 172.21.15.80:0/3985331668 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84480b0070 con 0x7f844c06db40 2024-04-25T13:50:39.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8443fff640 1 -- 172.21.15.80:0/3985331668 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480ab9b0 con 0x7f844c06db40 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 -- 172.21.15.80:0/3985331668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c06db40 msgr2=0x7f844c160e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 --2- 172.21.15.80:0/3985331668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c06db40 0x7f844c160e70 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f844c069620 tx=0x7f844801c520 comp rx=0 tx=0).stop 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 -- 172.21.15.80:0/3985331668 shutdown_connections 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 --2- 172.21.15.80:0/3985331668 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c06db40 0x7f844c160e70 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 --2- 172.21.15.80:0/3985331668 >> v2:172.21.15.87:3300/0 conn(0x7f844c0d1320 0x7f844c06d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 --2- 172.21.15.80:0/3985331668 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 0x7f844c0d0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 -- 172.21.15.80:0/3985331668 >> 172.21.15.80:0/3985331668 conn(0x7f844c072ed0 msgr2=0x7f844c0732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 -- 172.21.15.80:0/3985331668 shutdown_connections 2024-04-25T13:50:39.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 -- 172.21.15.80:0/3985331668 wait complete. 2024-04-25T13:50:39.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 Processor -- start 2024-04-25T13:50:39.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.912+0000 7f8453304640 1 -- start start 2024-04-25T13:50:39.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844c06db40 0x7f844c151480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 0x7f844c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c0d1320 0x7f844c0b3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f844c15e210 con 0x7f844c0d1320 2024-04-25T13:50:39.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f844c15e090 con 0x7f844c06db40 2024-04-25T13:50:39.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f844c068da0 con 0x7f844c06f1b0 2024-04-25T13:50:39.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c0d1320 0x7f844c0b3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c0d1320 0x7f844c0b3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:54746/0 (socket says 172.21.15.80:54746) 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 -- 172.21.15.80:0/217628829 learned_addr learned my addr 172.21.15.80:0/217628829 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84510a0640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844c06db40 0x7f844c151480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f845089f640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 0x7f844c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 -- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 msgr2=0x7f844c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 0x7f844c0af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 -- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844c06db40 msgr2=0x7f844c151480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844c06db40 0x7f844c151480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 -- 172.21.15.80:0/217628829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f844c151b40 con 0x7f844c0d1320 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f84518a1640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c0d1320 0x7f844c0b3110 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f844801cdc0 tx=0x7f844801ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8441ffb640 1 -- 172.21.15.80:0/217628829 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480be5f0 con 0x7f844c0d1320 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8441ffb640 1 -- 172.21.15.80:0/217628829 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84480be790 con 0x7f844c0d1320 2024-04-25T13:50:39.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8441ffb640 1 -- 172.21.15.80:0/217628829 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84480beb80 con 0x7f844c0d1320 2024-04-25T13:50:39.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f844c0afdf0 con 0x7f844c0d1320 2024-04-25T13:50:39.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f844c15bad0 con 0x7f844c0d1320 2024-04-25T13:50:39.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f844c15bcd0 con 0x7f844c0d1320 2024-04-25T13:50:39.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8441ffb640 1 -- 172.21.15.80:0/217628829 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f84480c83f0 con 0x7f844c0d1320 2024-04-25T13:50:39.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.916+0000 7f8441ffb640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f843006f400 0x7f8430071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:39.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.920+0000 7f8441ffb640 1 -- 172.21.15.80:0/217628829 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f8448129850 con 0x7f844c0d1320 2024-04-25T13:50:39.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.920+0000 7f84510a0640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f843006f400 0x7f8430071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:39.918 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.920+0000 7f84510a0640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f843006f400 0x7f8430071910 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8444003fd0 tx=0x7f84440039a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:39.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:39.920+0000 7f8441ffb640 1 -- 172.21.15.80:0/217628829 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8448016110 con 0x7f844c0d1320 2024-04-25T13:50:40.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.120+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f844c151d10 con 0x7f844c0d1320 2024-04-25T13:50:40.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.120+0000 7f8441ffb640 1 -- 172.21.15.80:0/217628829 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f84480f62f0 con 0x7f844c0d1320 2024-04-25T13:50:40.118 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:50:40.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f843006f400 msgr2=0x7f8430071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:40.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f843006f400 0x7f8430071910 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8444003fd0 tx=0x7f84440039a0 comp rx=0 tx=0).stop 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c0d1320 msgr2=0x7f844c0b3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c0d1320 0x7f844c0b3110 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f844801cdc0 tx=0x7f844801ad60 comp rx=0 tx=0).stop 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 shutdown_connections 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f843006f400 0x7f8430071910 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f844c0d1320 0x7f844c0b3110 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f844c06f1b0 0x7f844c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 --2- 172.21.15.80:0/217628829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f844c06db40 0x7f844c151480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 >> 172.21.15.80:0/217628829 conn(0x7f844c072ed0 msgr2=0x7f844c161f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 shutdown_connections 2024-04-25T13:50:40.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:40.124+0000 7f8453304640 1 -- 172.21.15.80:0/217628829 wait complete. 2024-04-25T13:50:40.140 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:50:40.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:50:40.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:50:40.140 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:50:40.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:50:40.141 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:50:40.141 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:50:40.141 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:50:40.141 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:50:40.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:40.142 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:50:40.148 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:50:40.149 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41929sec preferred_lft 41929sec 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:50:40.150 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:50:40.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:40.152 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:50:40.207 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:50:40.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:40.209 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:50:40.259 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:50:40.260 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:50:40.260 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:50:40.260 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:50:40.260 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:50:40.260 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:50:40.260 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:50:40.260 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:50:40.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:50:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:50:40.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:50:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:50:40.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:40.385 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:50:40.391 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:50:40.391 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:50:40.391 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:50:40.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:40.391 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:50:40.445 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:50:40.445 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:50:40.445 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:50:40.490 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:50:40.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:50:40.490 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:50:40.538 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:50:40.599 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:50:40.647 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:50:40.647 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:50:40.648 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:50:40.649 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:50:40.650 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:50:40.650 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:50:40.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:40.651 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:50:40.699 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:50:40.699 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:50:40.743 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:50:40.758 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:50:40.806 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:50:40.806 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:50:40.806 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:50:40.806 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:50:40.807 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:50:40.808 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:50:40.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:40.809 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:50:40.854 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:50:40.857+0000 7f70308c5580 -1 init, newargv = 0x7f701401b4e0 newargc=15 2024-04-25T13:50:40.854 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25587]: starting ceph client 2024-04-25T13:50:40.859 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25587]: starting fuse 2024-04-25T13:50:41.958 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:50:41.975 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:50:42.023 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:50:42.023 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:50:42.023 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:50:42.023 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:50:42.024 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:50:42.025 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:50:42.026 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:50:42.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:42.027 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:50:42.074 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:50:42.075 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:50:42.075 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:50:42.076 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:50:42.077 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:50:42.199 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.25587.asok 2024-04-25T13:50:42.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:50:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:50:42.211 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.25587.asok 2024-04-25T13:50:42.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:42.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.25587.asok status 2024-04-25T13:50:42.386 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:50:42.386 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:50:42.386 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:50:42.386 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:50:42.386 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "25587", 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 6422, 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 6422 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:50:42.387 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 655339517 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 655339517 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.6422 192.168.0.1:0/655339517", 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/655339517", 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 81, 2024-04-25T13:50:42.388 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 171, 2024-04-25T13:50:42.389 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:50:42.389 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:50:42.389 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:50:42.389 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:50:42.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:42.403 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:50:42.403 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:50:42.410 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:50:42.410 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:50:42.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:50:42.410 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:50:42.466 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:50:42.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb806f4d640 1 Processor -- start 2024-04-25T13:50:42.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb806f4d640 1 -- start start 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb806f4d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8001649d0 0x7fb8000c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb806f4d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8000c7480 con 0x7fb8000b45d0 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb806f4d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb8000d1200 con 0x7fb8000c7070 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb806f4d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8000b9460 con 0x7fb8001649d0 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb8054ea640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb8000b45d0 0x7fb8000d0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42292/0 (socket says 172.21.15.80:42292) 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb8054ea640 1 -- 172.21.15.80:0/2627726548 learned_addr learned my addr 172.21.15.80:0/2627726548 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb804ce9640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8001649d0 0x7fb8000c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4269795127 0 0) 0x7fb7f8001120 con 0x7fb8000b45d0 2024-04-25T13:50:42.628 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.628+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb7e8003530 con 0x7fb8000b45d0 2024-04-25T13:50:42.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2652129380 0 0) 0x7fb7e8003530 con 0x7fb8000b45d0 2024-04-25T13:50:42.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8001649d0 msgr2=0x7fb8000c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:42.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8001649d0 0x7fb8000c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 >> v1:172.21.15.87:6789/0 conn(0x7fb8000c7070 legacy=0x7fb8000d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:42.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8000d3960 con 0x7fb8000b45d0 2024-04-25T13:50:42.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb7f8001850 con 0x7fb8000b45d0 2024-04-25T13:50:42.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 >> v1:172.21.15.80:6789/0 conn(0x7fb8000b45d0 legacy=0x7fb8000d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8001649d0 0x7fb7e80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e8003f70 0x7fb7e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7e8004d50 0x7fb7e8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000b9460 con 0x7fb7e8004d50 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e800bc10 con 0x7fb7e8003f70 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb7e800be10 con 0x7fb8001649d0 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb8054ea640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7e8004d50 0x7fb7e8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb804ce9640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8001649d0 0x7fb7e80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7fffff640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e8003f70 0x7fb7e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb8054ea640 1 -- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8001649d0 msgr2=0x7fb7e80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb8054ea640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb8001649d0 0x7fb7e80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb8054ea640 1 -- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e8003f70 msgr2=0x7fb7e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb8054ea640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7e8003f70 0x7fb7e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb8054ea640 1 -- 172.21.15.80:0/2627726548 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e8003530 con 0x7fb7e8004d50 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb8054ea640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7e8004d50 0x7fb7e8005160 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b5600 tx=0x7fb7f801f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4038a0fb8af9300 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f800c0e0 con 0x7fb7e8004d50 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f8005b00 con 0x7fb7e8004d50 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb7ff7fe640 1 -- 172.21.15.80:0/2627726548 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8007070 con 0x7fb7e8004d50 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb806f4d640 1 -- 172.21.15.80:0/2627726548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7e8004d50 msgr2=0x7fb7e8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb806f4d640 1 --2- 172.21.15.80:0/2627726548 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb7e8004d50 0x7fb7e8005160 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7fb8000b5600 tx=0x7fb7f801f3c0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb8054ea640 1 -- 172.21.15.80:0/2627726548 reap_dead start 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb806f4d640 1 -- 172.21.15.80:0/2627726548 shutdown_connections 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb806f4d640 1 -- 172.21.15.80:0/2627726548 >> 172.21.15.80:0/2627726548 conn(0x7fb8000d3b60 msgr2=0x7fb8000d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb806f4d640 1 -- 172.21.15.80:0/2627726548 shutdown_connections 2024-04-25T13:50:42.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.632+0000 7fb806f4d640 1 -- 172.21.15.80:0/2627726548 wait complete. 2024-04-25T13:50:42.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb806f4d640 1 Processor -- start 2024-04-25T13:50:42.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb806f4d640 1 -- start start 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb806f4d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e8004d50 0x7fb800160450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb806f4d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80015daa0 0x7fb800160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb806f4d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb800161d10 0x7fb800162120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb806f4d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000b90f0 con 0x7fb800161d10 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb806f4d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb8000b8e10 con 0x7fb80015daa0 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb806f4d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb8000d2070 con 0x7fb7e8004d50 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb800161d10 0x7fb800162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb800161d10 0x7fb800162120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44274/0 (socket says 172.21.15.80:44274) 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 -- 172.21.15.80:0/780627346 learned_addr learned my addr 172.21.15.80:0/780627346 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb804ce9640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e8004d50 0x7fb800160450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:42.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb7fffff640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80015daa0 0x7fb800160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 -- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e8004d50 msgr2=0x7fb800160450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e8004d50 0x7fb800160450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 -- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80015daa0 msgr2=0x7fb800160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80015daa0 0x7fb800160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 -- 172.21.15.80:0/780627346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb800162660 con 0x7fb800161d10 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.636+0000 7fb8054ea640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb800161d10 0x7fb800162120 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8026030 tx=0x7fb7e8005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb7fd7fa640 1 -- 172.21.15.80:0/780627346 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8001120 con 0x7fb800161d10 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb7fd7fa640 1 -- 172.21.15.80:0/780627346 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb7f80b6040 con 0x7fb800161d10 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb800162ac0 con 0x7fb800161d10 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb7fd7fa640 1 -- 172.21.15.80:0/780627346 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f8001850 con 0x7fb800161d10 2024-04-25T13:50:42.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80015cce0 con 0x7fb800161d10 2024-04-25T13:50:42.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb800162830 con 0x7fb800161d10 2024-04-25T13:50:42.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb7fd7fa640 1 -- 172.21.15.80:0/780627346 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb7f80c2060 con 0x7fb800161d10 2024-04-25T13:50:42.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb7fd7fa640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7ec06f400 0x7fb7ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:42.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb7fd7fa640 1 -- 172.21.15.80:0/780627346 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7fb7f812c1c0 con 0x7fb800161d10 2024-04-25T13:50:42.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb804ce9640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7ec06f400 0x7fb7ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:42.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.640+0000 7fb804ce9640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7ec06f400 0x7fb7ec071910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009a10 tx=0x7fb7f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:42.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.644+0000 7fb7fd7fa640 1 -- 172.21.15.80:0/780627346 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8000c7480 con 0x7fb800161d10 2024-04-25T13:50:42.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8000d1af0 con 0x7fb800161d10 2024-04-25T13:50:42.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb7fd7fa640 1 -- 172.21.15.80:0/780627346 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5176 (secure 0 0 0) 0x7fb7f80f90b0 con 0x7fb800161d10 2024-04-25T13:50:42.822 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:50:42.822 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/29039900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":29039900},{"type":"v1","addr":"172.21.15.87:6836","nonce":29039900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1466426643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1466426643},{"type":"v1","addr":"172.21.15.80:6833","nonce":1466426643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1170719891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1170719891},{"type":"v1","addr":"172.21.15.80:6835","nonce":1170719891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:50:30.371502+0000","modified":"2024-04-25T13:50:33.461986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/986211877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":986211877},{"type":"v1","addr":"172.21.15.87:6837","nonce":986211877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":10}]} 2024-04-25T13:50:42.822 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 81 2024-04-25T13:50:42.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7ec06f400 msgr2=0x7fb7ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:42.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7ec06f400 0x7fb7ec071910 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb7f0009a10 tx=0x7fb7f0008040 comp rx=0 tx=0).stop 2024-04-25T13:50:42.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb800161d10 msgr2=0x7fb800162120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:42.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb800161d10 0x7fb800162120 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8026030 tx=0x7fb7e8005840 comp rx=0 tx=0).stop 2024-04-25T13:50:42.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 shutdown_connections 2024-04-25T13:50:42.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb7ec06f400 0x7fb7ec071910 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb800161d10 0x7fb800162120 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb80015daa0 0x7fb800160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 --2- 172.21.15.80:0/780627346 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb7e8004d50 0x7fb800160450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:42.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 >> 172.21.15.80:0/780627346 conn(0x7fb8000d3b60 msgr2=0x7fb80006cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:42.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 shutdown_connections 2024-04-25T13:50:42.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:42.824+0000 7fb806f4d640 1 -- 172.21.15.80:0/780627346 wait complete. 2024-04-25T13:50:42.842 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2024-04-25T13:50:43.018 INFO:teuthology.orchestra.run.smithi087.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":8814,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.98142307,"mdsmap_epoch":81,"osdmap_epoch":171,"osdmap_epoch_barrier":171,"uptime":17.958641305} 2024-04-25T13:50:43.018 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 8814, "mdsmap_epoch": 81, "osdmap_epoch": 171, "osdmap_epoch_barrier": 171, "rank_uptime": 11.98142307, "state": "up:active", "uptime": 17.958641305, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:50:43.019 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-25T13:50:43.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.180+0000 7f5eec14e640 1 Processor -- start 2024-04-25T13:50:43.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.180+0000 7f5eec14e640 1 -- start start 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.180+0000 7f5eec14e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40ce2a0 0x7f5ee40ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.180+0000 7f5eec14e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ee4161d10 0x7f5ee4162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.180+0000 7f5eec14e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ee40b1320 con 0x7f5ee40b0f10 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.180+0000 7f5eec14e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5ee40b5820 con 0x7f5ee4161d10 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.180+0000 7f5eec14e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee40d13f0 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5ee9eea640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5ee40b0f10 0x7f5ee40d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:42296/0 (socket says 172.21.15.80:42296) 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eea6eb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5ee4161d10 0x7f5ee4162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5ee9eea640 1 -- 172.21.15.80:0/2104578611 learned_addr learned my addr 172.21.15.80:0/2104578611 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5ee96e9640 1 --2- 172.21.15.80:0/2104578611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40ce2a0 0x7f5ee40ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:43.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eea6eb640 1 -- 172.21.15.80:0/2104578611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40ce2a0 msgr2=0x7f5ee40ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:50:43.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eea6eb640 1 --2- 172.21.15.80:0/2104578611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40ce2a0 0x7f5ee40ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eea6eb640 1 -- 172.21.15.80:0/2104578611 >> v1:172.21.15.80:6789/0 conn(0x7f5ee40b0f10 legacy=0x7f5ee40d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:43.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5ee8ee8640 1 -- 172.21.15.80:0/2104578611 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3866718636 0 0) 0x7f5ed4001120 con 0x7f5ee40b0f10 2024-04-25T13:50:43.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eea6eb640 1 -- 172.21.15.80:0/2104578611 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee40cebf0 con 0x7f5ee4161d10 2024-04-25T13:50:43.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5ee96e9640 1 --2- 172.21.15.80:0/2104578611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40ce2a0 0x7f5ee40ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:43.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eea6eb640 1 --2- 172.21.15.80:0/2104578611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee4161d10 0x7f5ee4162120 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009d90 tx=0x7f5edc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66ac2e33c9a3d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:43.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5ee8ee8640 1 -- 172.21.15.80:0/2104578611 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0b2070 con 0x7f5ee4161d10 2024-04-25T13:50:43.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5ee8ee8640 1 -- 172.21.15.80:0/2104578611 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5edc0a5920 con 0x7f5ee4161d10 2024-04-25T13:50:43.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5ee8ee8640 1 -- 172.21.15.80:0/2104578611 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5edc0a5cf0 con 0x7f5ee4161d10 2024-04-25T13:50:43.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eec14e640 1 -- 172.21.15.80:0/2104578611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee4161d10 msgr2=0x7f5ee4162120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:43.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2104578611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee4161d10 0x7f5ee4162120 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009d90 tx=0x7f5edc01c970 comp rx=0 tx=0).stop 2024-04-25T13:50:43.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eec14e640 1 -- 172.21.15.80:0/2104578611 shutdown_connections 2024-04-25T13:50:43.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2104578611 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee4161d10 0x7f5ee4162120 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2104578611 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40ce2a0 0x7f5ee40ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eec14e640 1 -- 172.21.15.80:0/2104578611 >> 172.21.15.80:0/2104578611 conn(0x7f5ee40b35c0 msgr2=0x7f5ee40b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:43.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eec14e640 1 -- 172.21.15.80:0/2104578611 shutdown_connections 2024-04-25T13:50:43.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.184+0000 7f5eec14e640 1 -- 172.21.15.80:0/2104578611 wait complete. 2024-04-25T13:50:43.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 Processor -- start 2024-04-25T13:50:43.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 -- start start 2024-04-25T13:50:43.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40b0f10 0x7f5ee41540a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:43.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee40ce2a0 0x7f5ee4150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee4161d10 0x7f5ee4158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee415e720 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee40b5300 con 0x7f5ee4161d10 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5ee4162a70 con 0x7f5ee40b0f10 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee40ce2a0 0x7f5ee4150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee40ce2a0 0x7f5ee4150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:44298/0 (socket says 172.21.15.80:44298) 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 -- 172.21.15.80:0/2539645375 learned_addr learned my addr 172.21.15.80:0/2539645375 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee9eea640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40b0f10 0x7f5ee41540a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eea6eb640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee4161d10 0x7f5ee4158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 -- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40b0f10 msgr2=0x7f5ee41540a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40b0f10 0x7f5ee41540a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 -- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee4161d10 msgr2=0x7f5ee4158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee4161d10 0x7f5ee4158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 -- 172.21.15.80:0/2539645375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee4154e10 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee9eea640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40b0f10 0x7f5ee41540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:43.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eea6eb640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee4161d10 0x7f5ee4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:50:43.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee96e9640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee40ce2a0 0x7f5ee4150910 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c7f0 tx=0x7f5ed800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:43.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee27fc640 1 -- 172.21.15.80:0/2539645375 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8015070 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5ee27fc640 1 -- 172.21.15.80:0/2539645375 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed8007820 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.188+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee4154b30 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.192+0000 7f5ee27fc640 1 -- 172.21.15.80:0/2539645375 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed8007c10 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.190 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.192+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee41538a0 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.192+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee406a440 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.192+0000 7f5ee27fc640 1 -- 172.21.15.80:0/2539645375 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ed8018070 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.192+0000 7f5ee27fc640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ed006f400 0x7f5ed0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:50:43.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.192+0000 7f5ee27fc640 1 -- 172.21.15.80:0/2539645375 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5709+0+0 (secure 0 0 0) 0x7f5ed808f460 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.192+0000 7f5ee9eea640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ed006f400 0x7f5ed0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:50:43.191 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.192+0000 7f5ee9eea640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ed006f400 0x7f5ed0071910 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4007fd0 tx=0x7f5ed4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:50:43.195 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.196+0000 7f5ee27fc640 1 -- 172.21.15.80:0/2539645375 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ed805c350 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.356+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f5ee4076a10 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.392+0000 7f5ee27fc640 1 -- 172.21.15.80:0/2539645375 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v172) ==== 181+0+0 (secure 0 0 0) 0x7f5ed805bc50 con 0x7f5ee40ce2a0 2024-04-25T13:50:43.389 INFO:teuthology.orchestra.run.smithi080.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-25T13:50:43.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ed006f400 msgr2=0x7f5ed0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:43.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ed006f400 0x7f5ed0071910 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f5ed4007fd0 tx=0x7f5ed4009040 comp rx=0 tx=0).stop 2024-04-25T13:50:43.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee40ce2a0 msgr2=0x7f5ee4150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:50:43.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee40ce2a0 0x7f5ee4150910 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f5ed800c7f0 tx=0x7f5ed800ccb0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 shutdown_connections 2024-04-25T13:50:43.393 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ed006f400 0x7f5ed0071910 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5ee4161d10 0x7f5ee4158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5ee40ce2a0 0x7f5ee4150910 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 --2- 172.21.15.80:0/2539645375 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5ee40b0f10 0x7f5ee41540a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:50:43.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 >> 172.21.15.80:0/2539645375 conn(0x7f5ee40b35c0 msgr2=0x7f5ee40b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:50:43.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 shutdown_connections 2024-04-25T13:50:43.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:50:43.396+0000 7f5eec14e640 1 -- 172.21.15.80:0/2539645375 wait complete. 2024-04-25T13:50:43.412 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-25T13:50:43.562 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_mon_report_interval":"5"} 2024-04-25T13:50:43.563 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-25T13:50:43.564 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2024-04-25T13:50:43.715 INFO:teuthology.orchestra.run.smithi080.stdout:{"mon_tick_interval":"5"} 2024-04-25T13:50:43.716 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2024-04-25T13:50:43.716 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2024-04-25T13:50:43.716 DEBUG:teuthology.orchestra.run.smithi080:> stdin-killer -- python3 -c ' 2024-04-25T13:50:43.716 DEBUG:teuthology.orchestra.run.smithi080:> import time 2024-04-25T13:50:43.716 DEBUG:teuthology.orchestra.run.smithi080:> import datetime 2024-04-25T13:50:43.716 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:50:43.716 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> # Write some buffered data through before going full, all should be well 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> print("writing some data through which we expect to succeed") 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> bytes = 0 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(f) 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> # writing until we see an error from fsync. As long as we'"'"'re doing 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> # buffered IO, the error should always only appear from fsync and not 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> # from write 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> full = False 2024-04-25T13:50:43.717 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> for n in range(0, int(32 * 1.1)): 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> try: 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> print("wrote bytes via buffered write, moving on to fsync") 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> except OSError as e: 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> if True: 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> print("Unexpected error %s from write() instead of fsync()" % e) 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> raise 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> else: 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> full = True 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> break 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> try: 2024-04-25T13:50:43.718 DEBUG:teuthology.orchestra.run.smithi080:> os.fsync(f) 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> print("fsync'"'"'ed successfully") 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> except OSError as e: 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> full = True 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> break 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> else: 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> if n > 32 * 0.9: 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> # Be cautious in the last region where we expect to hit 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> # the full condition, so that we don'"'"'t overshoot too dramatically 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> time.sleep(15.0) 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> if not full: 2024-04-25T13:50:43.719 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> # close() should not raise an error because we already caught it in 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> # fsync. There shouldn'"'"'t have been any more writeback errors 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> # since then because all IOs got cancelled on the full flag. 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> print("calling close") 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> os.close(f) 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> print("close() did not raise error") 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2024-04-25T13:50:43.720 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:51:00.688 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:51:00.691 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:51:30.444 INFO:teuthology.orchestra.run.smithi080.stdout:writing some data through which we expect to succeed 2024-04-25T13:51:30.444 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed data successfully, will now attempt to fill fs 2024-04-25T13:51:30.444 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 1.00 MB 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 2.00 MB 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 3.00 MB 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 4.00 MB 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.445 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 5.00 MB 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 6.00 MB 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 7.00 MB 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 8.00 MB 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.446 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 9.00 MB 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 10.00 MB 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 11.00 MB 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 12.00 MB 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 13.00 MB 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 14.00 MB 2024-04-25T13:51:30.447 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 15.00 MB 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 16.00 MB 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 17.00 MB 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 18.00 MB 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 19.00 MB 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.448 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 20.00 MB 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 21.00 MB 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 22.00 MB 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 23.00 MB 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 24.00 MB 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.449 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 25.00 MB 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 26.00 MB 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 27.00 MB 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 28.00 MB 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 29.00 MB 2024-04-25T13:51:30.450 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 30.00 MB 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 31.00 MB 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:fsync'ed successfully 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:Not full yet after 32.00 MB 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:wrote bytes via buffered write, moving on to fsync 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:Reached fullness after 33.00 MB 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:calling close 2024-04-25T13:51:30.451 INFO:teuthology.orchestra.run.smithi080.stdout:close() did not raise error 2024-04-25T13:51:30.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:51:30.459 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:51:30.459 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:51:30.506 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:51:30.543 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:51:30.544 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:51:30.544 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:51:30.544 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:30.550 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:51:30.550 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:30.596 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:30.742 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:51:30.744 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:51:30.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde4af91640 1 Processor -- start 2024-04-25T13:51:30.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde4af91640 1 -- start start 2024-04-25T13:51:30.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde4af91640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fde44070490 0x7fde440708a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:30.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde4af91640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde44068f00 con 0x7fde4415f1c0 2024-04-25T13:51:30.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde4af91640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde440691e0 con 0x7fde440cfb60 2024-04-25T13:51:30.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde4af91640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fde44068ca0 con 0x7fde44070490 2024-04-25T13:51:30.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde48d2d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fde4415f1c0 0x7fde44068590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34228/0 (socket says 172.21.15.80:34228) 2024-04-25T13:51:30.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde48d2d640 1 -- 172.21.15.80:0/1179961532 learned_addr learned my addr 172.21.15.80:0/1179961532 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:30.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165369840 0 0) 0x7fde44068f00 con 0x7fde4415f1c0 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fde2c003530 con 0x7fde4415f1c0 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.804+0000 7fde4952e640 1 --2- 172.21.15.80:0/1179961532 >> v2:172.21.15.87:3300/0 conn(0x7fde44070490 0x7fde440708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3569155497 0 0) 0x7fde2c003530 con 0x7fde4415f1c0 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 >> v2:172.21.15.87:3300/0 conn(0x7fde44070490 msgr2=0x7fde440708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 --2- 172.21.15.80:0/1179961532 >> v2:172.21.15.87:3300/0 conn(0x7fde44070490 0x7fde440708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 >> v1:172.21.15.87:6789/0 conn(0x7fde440cfb60 legacy=0x7fde440cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde4415e9c0 con 0x7fde4415f1c0 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fde34001680 con 0x7fde4415f1c0 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 >> v1:172.21.15.80:6789/0 conn(0x7fde4415f1c0 legacy=0x7fde44068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0039c0 0x7fde2c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2c004810 0x7fde2c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde2c0053b0 0x7fde2c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:30.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fde44068ca0 con 0x7fde2c0053b0 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fde2c00c340 con 0x7fde2c0039c0 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fde2c00c540 con 0x7fde2c004810 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4952e640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde2c0053b0 0x7fde2c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde48d2d640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0039c0 0x7fde2c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde43fff640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2c004810 0x7fde2c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4952e640 1 -- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2c004810 msgr2=0x7fde2c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4952e640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2c004810 0x7fde2c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4952e640 1 -- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0039c0 msgr2=0x7fde2c003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4952e640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0039c0 0x7fde2c003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4952e640 1 -- 172.21.15.80:0/1179961532 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde2c003530 con 0x7fde2c0053b0 2024-04-25T13:51:30.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde43fff640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde2c004810 0x7fde2c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:30.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4952e640 1 -- 172.21.15.80:0/1179961532 reap_dead start 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4952e640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde2c0053b0 0x7fde2c00ba60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fde3c007330 tx=0x7fde3c01ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c53fcccaaa55f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c009be0 con 0x7fde2c0053b0 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde3c003ba0 con 0x7fde2c0053b0 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde437fe640 1 -- 172.21.15.80:0/1179961532 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0a5940 con 0x7fde2c0053b0 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4af91640 1 -- 172.21.15.80:0/1179961532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde2c0053b0 msgr2=0x7fde2c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4af91640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde2c0053b0 0x7fde2c00ba60 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fde3c007330 tx=0x7fde3c01ce60 comp rx=0 tx=0).stop 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4af91640 1 -- 172.21.15.80:0/1179961532 shutdown_connections 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4af91640 1 --2- 172.21.15.80:0/1179961532 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde2c0053b0 0x7fde2c00ba60 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4af91640 1 -- 172.21.15.80:0/1179961532 >> 172.21.15.80:0/1179961532 conn(0x7fde440d33a0 msgr2=0x7fde440cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4af91640 1 -- 172.21.15.80:0/1179961532 shutdown_connections 2024-04-25T13:51:30.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.808+0000 7fde4af91640 1 -- 172.21.15.80:0/1179961532 wait complete. 2024-04-25T13:51:30.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 Processor -- start 2024-04-25T13:51:30.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 -- start start 2024-04-25T13:51:30.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0053b0 0x7fde44142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:30.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde441411b0 0x7fde441415c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:30.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde44160bf0 0x7fde44161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:30.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fde440708a0 con 0x7fde441411b0 2024-04-25T13:51:30.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fde4406da30 con 0x7fde2c0053b0 2024-04-25T13:51:30.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fde44070a20 con 0x7fde44160bf0 2024-04-25T13:51:30.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde441411b0 0x7fde441415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde441411b0 0x7fde441415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:48604/0 (socket says 172.21.15.80:48604) 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 -- 172.21.15.80:0/2262147175 learned_addr learned my addr 172.21.15.80:0/2262147175 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde48d2d640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0053b0 0x7fde44142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4952e640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde44160bf0 0x7fde44161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 -- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde44160bf0 msgr2=0x7fde44161000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde44160bf0 0x7fde44161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 -- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0053b0 msgr2=0x7fde44142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0053b0 0x7fde44142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 -- 172.21.15.80:0/2262147175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde44141b00 con 0x7fde441411b0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde43fff640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde441411b0 0x7fde441415c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fde3800d560 tx=0x7fde3800da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde417fa640 1 -- 172.21.15.80:0/2262147175 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38005480 con 0x7fde441411b0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde417fa640 1 -- 172.21.15.80:0/2262147175 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde38017070 con 0x7fde441411b0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde44161540 con 0x7fde441411b0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.812+0000 7fde417fa640 1 -- 172.21.15.80:0/2262147175 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38003860 con 0x7fde441411b0 2024-04-25T13:51:30.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.816+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde441437a0 con 0x7fde441411b0 2024-04-25T13:51:30.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.816+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde4406a3a0 con 0x7fde441411b0 2024-04-25T13:51:30.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.816+0000 7fde417fa640 1 -- 172.21.15.80:0/2262147175 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fde38003a00 con 0x7fde441411b0 2024-04-25T13:51:30.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.816+0000 7fde417fa640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fde3006f400 0x7fde30071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:30.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.816+0000 7fde417fa640 1 -- 172.21.15.80:0/2262147175 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7fde3808f140 con 0x7fde441411b0 2024-04-25T13:51:30.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.816+0000 7fde417fa640 1 -- 172.21.15.80:0/2262147175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7fde30076fe0 con 0x7fde441411b0 2024-04-25T13:51:30.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.816+0000 7fde48d2d640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fde3006f400 0x7fde30071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:30.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.816+0000 7fde48d2d640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fde3006f400 0x7fde30071910 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde340010f0 tx=0x7fde34000f20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:30.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.820+0000 7fde417fa640 1 -- 172.21.15.80:0/2262147175 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fde44161540 con 0x7fde441411b0 2024-04-25T13:51:30.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.980+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fde44071780 con 0x7fde441411b0 2024-04-25T13:51:30.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.984+0000 7fde417fa640 1 -- 172.21.15.80:0/2262147175 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) ==== 74+0+16683 (secure 0 0 0) 0x7fde3805c010 con 0x7fde441411b0 2024-04-25T13:51:30.978 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:30.978 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":175,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:21.279640+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:50:28.108369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T13:50:29.324161+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:30.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.984+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fde3006f400 msgr2=0x7fde30071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:30.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.984+0000 7fde4af91640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fde3006f400 0x7fde30071910 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fde340010f0 tx=0x7fde34000f20 comp rx=0 tx=0).stop 2024-04-25T13:51:30.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde441411b0 msgr2=0x7fde441415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:30.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde441411b0 0x7fde441415c0 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fde3800d560 tx=0x7fde3800da20 comp rx=0 tx=0).stop 2024-04-25T13:51:30.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 shutdown_connections 2024-04-25T13:51:30.980 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fde3006f400 0x7fde30071910 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fde44160bf0 0x7fde44161000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fde441411b0 0x7fde441415c0 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 --2- 172.21.15.80:0/2262147175 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fde2c0053b0 0x7fde44142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:30.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 >> 172.21.15.80:0/2262147175 conn(0x7fde440d33a0 msgr2=0x7fde440b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:30.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 shutdown_connections 2024-04-25T13:51:30.981 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:30.988+0000 7fde4af91640 1 -- 172.21.15.80:0/2262147175 wait complete. 2024-04-25T13:51:31.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:50:28.108369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:50:29.324161+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:51:31.001 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:51:31.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.208+0000 7ff9829b0640 1 Processor -- start 2024-04-25T13:51:31.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.208+0000 7ff9829b0640 1 -- start start 2024-04-25T13:51:31.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.208+0000 7ff9829b0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff97c071e60 0x7ff97c1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.208+0000 7ff9829b0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff97c161170 0x7ff97c0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.208+0000 7ff9829b0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff97c06fbf0 con 0x7ff97c073c40 2024-04-25T13:51:31.204 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.208+0000 7ff9829b0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff97c0b1350 con 0x7ff97c071e60 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.208+0000 7ff9829b0640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff97c06f9a0 con 0x7ff97c161170 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff97c071e60 0x7ff97c1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff97c071e60 0x7ff97c1574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51612/0 (socket says 172.21.15.80:51612) 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97bfff640 1 -- 172.21.15.80:0/2411011060 learned_addr learned my addr 172.21.15.80:0/2411011060 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff980f4d640 1 --2- 172.21.15.80:0/2411011060 >> v2:172.21.15.87:3300/0 conn(0x7ff97c161170 0x7ff97c0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97bfff640 1 -- 172.21.15.80:0/2411011060 >> v2:172.21.15.87:3300/0 conn(0x7ff97c161170 msgr2=0x7ff97c0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97bfff640 1 --2- 172.21.15.80:0/2411011060 >> v2:172.21.15.87:3300/0 conn(0x7ff97c161170 0x7ff97c0b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97bfff640 1 -- 172.21.15.80:0/2411011060 >> v1:172.21.15.87:6789/0 conn(0x7ff97c073c40 legacy=0x7ff97c160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:31.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97bfff640 1 -- 172.21.15.80:0/2411011060 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c0b0a20 con 0x7ff97c071e60 2024-04-25T13:51:31.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97bfff640 1 --2- 172.21.15.80:0/2411011060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c071e60 0x7ff97c1574c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7ff97c161a50 tx=0x7ff96c01ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8867a613f7f4953e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97b7fe640 1 -- 172.21.15.80:0/2411011060 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0b4070 con 0x7ff97c071e60 2024-04-25T13:51:31.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97b7fe640 1 -- 172.21.15.80:0/2411011060 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff96c0a7b70 con 0x7ff97c071e60 2024-04-25T13:51:31.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff97b7fe640 1 -- 172.21.15.80:0/2411011060 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0af440 con 0x7ff97c071e60 2024-04-25T13:51:31.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff9829b0640 1 -- 172.21.15.80:0/2411011060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c071e60 msgr2=0x7ff97c1574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff9829b0640 1 --2- 172.21.15.80:0/2411011060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c071e60 0x7ff97c1574c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7ff97c161a50 tx=0x7ff96c01ebb0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff9829b0640 1 -- 172.21.15.80:0/2411011060 shutdown_connections 2024-04-25T13:51:31.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff9829b0640 1 --2- 172.21.15.80:0/2411011060 >> v2:172.21.15.87:3300/0 conn(0x7ff97c161170 0x7ff97c0b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff9829b0640 1 --2- 172.21.15.80:0/2411011060 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c071e60 0x7ff97c1574c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff9829b0640 1 -- 172.21.15.80:0/2411011060 >> 172.21.15.80:0/2411011060 conn(0x7ff97c0b4160 msgr2=0x7ff97c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:31.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff9829b0640 1 -- 172.21.15.80:0/2411011060 shutdown_connections 2024-04-25T13:51:31.208 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.212+0000 7ff9829b0640 1 -- 172.21.15.80:0/2411011060 wait complete. 2024-04-25T13:51:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 Processor -- start 2024-04-25T13:51:31.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 -- start start 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97c071e60 0x7ff97c146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff97c073c40 0x7ff97c0af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c0cf8a0 0x7ff97c0afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c06e470 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff97c074050 con 0x7ff97c071e60 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff97c06e5f0 con 0x7ff97c073c40 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c0cf8a0 0x7ff97c0afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c0cf8a0 0x7ff97c0afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51622/0 (socket says 172.21.15.80:51622) 2024-04-25T13:51:31.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 -- 172.21.15.80:0/635393058 learned_addr learned my addr 172.21.15.80:0/635393058 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff97bfff640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97c071e60 0x7ff97c146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff973fff640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff97c073c40 0x7ff97c0af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 -- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff97c073c40 msgr2=0x7ff97c0af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff97c073c40 0x7ff97c0af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 -- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97c071e60 msgr2=0x7ff97c146d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97c071e60 0x7ff97c146d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 -- 172.21.15.80:0/635393058 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff97c0d0010 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff980f4d640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c0cf8a0 0x7ff97c0afdf0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff974007e90 tx=0x7ff97400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9797fa640 1 -- 172.21.15.80:0/635393058 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff974014420 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9797fa640 1 -- 172.21.15.80:0/635393058 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff974019070 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff97c0d02f0 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9797fa640 1 -- 172.21.15.80:0/635393058 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff974014800 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.216+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff97c0d08b0 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.220+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff97c06a3a0 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.220+0000 7ff9797fa640 1 -- 172.21.15.80:0/635393058 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff9740149a0 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.220+0000 7ff9797fa640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff95c06f400 0x7ff95c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.220+0000 7ff9797fa640 1 -- 172.21.15.80:0/635393058 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7ff97408fb90 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.220+0000 7ff9797fa640 1 -- 172.21.15.80:0/635393058 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7ff95c077040 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.220+0000 7ff97bfff640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff95c06f400 0x7ff95c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.220+0000 7ff97bfff640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff95c06f400 0x7ff95c071910 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0b1000 tx=0x7ff96c0a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.224+0000 7ff9797fa640 1 -- 172.21.15.80:0/635393058 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff97405c410 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.375 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.380+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7ff97c0b4ec0 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.376 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9797fa640 1 -- 172.21.15.80:0/635393058 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) ==== 93+0+10 (secure 0 0 0) 0x7ff974060710 con 0x7ff97c0cf8a0 2024-04-25T13:51:31.376 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff95c06f400 msgr2=0x7ff95c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff95c06f400 0x7ff95c071910 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff96c0b1000 tx=0x7ff96c0a9070 comp rx=0 tx=0).stop 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c0cf8a0 msgr2=0x7ff97c0afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c0cf8a0 0x7ff97c0afdf0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7ff974007e90 tx=0x7ff97400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 shutdown_connections 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff95c06f400 0x7ff95c071910 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff97c0cf8a0 0x7ff97c0afdf0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff97c073c40 0x7ff97c0af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 --2- 172.21.15.80:0/635393058 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff97c071e60 0x7ff97c146d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 >> 172.21.15.80:0/635393058 conn(0x7ff97c0b4160 msgr2=0x7ff97c0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 shutdown_connections 2024-04-25T13:51:31.379 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.384+0000 7ff9829b0640 1 -- 172.21.15.80:0/635393058 wait complete. 2024-04-25T13:51:31.396 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:51:31.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe1dbc9640 1 Processor -- start 2024-04-25T13:51:31.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe1dbc9640 1 -- start start 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe1dbc9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe1806edb0 0x7efe1806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe1dbc9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe18161b60 0x7efe18161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe1dbc9640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe181624b0 con 0x7efe180b5330 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe1dbc9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efe1806f700 con 0x7efe18161b60 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe1dbc9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efe181629b0 con 0x7efe1806edb0 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe17fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe18161b60 0x7efe18161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe17fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe18161b60 0x7efe18161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51630/0 (socket says 172.21.15.80:51630) 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe17fff640 1 -- 172.21.15.80:0/578366353 learned_addr learned my addr 172.21.15.80:0/578366353 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe16ffd640 1 --2- 172.21.15.80:0/578366353 >> v2:172.21.15.87:3300/0 conn(0x7efe1806edb0 0x7efe1806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe17fff640 1 -- 172.21.15.80:0/578366353 >> v2:172.21.15.87:3300/0 conn(0x7efe1806edb0 msgr2=0x7efe1806f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe17fff640 1 --2- 172.21.15.80:0/578366353 >> v2:172.21.15.87:3300/0 conn(0x7efe1806edb0 0x7efe1806f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe17fff640 1 -- 172.21.15.80:0/578366353 >> v1:172.21.15.87:6789/0 conn(0x7efe180b5330 legacy=0x7efe180d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:31.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.556+0000 7efe17fff640 1 -- 172.21.15.80:0/578366353 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe180b5740 con 0x7efe18161b60 2024-04-25T13:51:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe17fff640 1 --2- 172.21.15.80:0/578366353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe18161b60 0x7efe18161f70 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7efe18074290 tx=0x7efe1001c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1165221747ae34ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe167fc640 1 -- 172.21.15.80:0/578366353 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe100a55d0 con 0x7efe18161b60 2024-04-25T13:51:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe167fc640 1 -- 172.21.15.80:0/578366353 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe100a5770 con 0x7efe18161b60 2024-04-25T13:51:31.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe167fc640 1 -- 172.21.15.80:0/578366353 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe100a5b40 con 0x7efe18161b60 2024-04-25T13:51:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe1dbc9640 1 -- 172.21.15.80:0/578366353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe18161b60 msgr2=0x7efe18161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/578366353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe18161b60 0x7efe18161f70 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7efe18074290 tx=0x7efe1001c710 comp rx=0 tx=0).stop 2024-04-25T13:51:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe1dbc9640 1 -- 172.21.15.80:0/578366353 shutdown_connections 2024-04-25T13:51:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/578366353 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe18161b60 0x7efe18161f70 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/578366353 >> v2:172.21.15.87:3300/0 conn(0x7efe1806edb0 0x7efe1806f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe1dbc9640 1 -- 172.21.15.80:0/578366353 >> 172.21.15.80:0/578366353 conn(0x7efe180b4160 msgr2=0x7efe180d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:31.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe1dbc9640 1 -- 172.21.15.80:0/578366353 shutdown_connections 2024-04-25T13:51:31.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.560+0000 7efe1dbc9640 1 -- 172.21.15.80:0/578366353 wait complete. 2024-04-25T13:51:31.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 Processor -- start 2024-04-25T13:51:31.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 -- start start 2024-04-25T13:51:31.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe1806edb0 0x7efe181489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe180b5330 0x7efe1815ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe18161b60 0x7efe1815b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efe1806e890 con 0x7efe1806edb0 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe1806e5b0 con 0x7efe180b5330 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe180b0cd0 con 0x7efe18161b60 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe1806edb0 0x7efe181489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe1806edb0 0x7efe181489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51642/0 (socket says 172.21.15.80:51642) 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 -- 172.21.15.80:0/2958091801 learned_addr learned my addr 172.21.15.80:0/2958091801 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe17fff640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe18161b60 0x7efe1815b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe16ffd640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe180b5330 0x7efe1815ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 -- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe18161b60 msgr2=0x7efe1815b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe18161b60 0x7efe1815b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 -- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe180b5330 msgr2=0x7efe1815ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe180b5330 0x7efe1815ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 -- 172.21.15.80:0/2958091801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe18148ef0 con 0x7efe1806edb0 2024-04-25T13:51:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe177fe640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe1806edb0 0x7efe181489b0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7efe0800b440 tx=0x7efe0800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efdfffff640 1 -- 172.21.15.80:0/2958091801 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08015020 con 0x7efe1806edb0 2024-04-25T13:51:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efdfffff640 1 -- 172.21.15.80:0/2958091801 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe08011890 con 0x7efe1806edb0 2024-04-25T13:51:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe180b2e30 con 0x7efe1806edb0 2024-04-25T13:51:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efdfffff640 1 -- 172.21.15.80:0/2958091801 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08011c80 con 0x7efe1806edb0 2024-04-25T13:51:31.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.564+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe180b2350 con 0x7efe1806edb0 2024-04-25T13:51:31.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.568+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe1806a3a0 con 0x7efe1806edb0 2024-04-25T13:51:31.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.568+0000 7efdfffff640 1 -- 172.21.15.80:0/2958091801 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe08018020 con 0x7efe1806edb0 2024-04-25T13:51:31.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.568+0000 7efdfffff640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe0406f400 0x7efe04071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.568+0000 7efdfffff640 1 -- 172.21.15.80:0/2958091801 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7efe0808f4d0 con 0x7efe1806edb0 2024-04-25T13:51:31.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.568+0000 7efdfffff640 1 -- 172.21.15.80:0/2958091801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7efe04077040 con 0x7efe1806edb0 2024-04-25T13:51:31.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.568+0000 7efe16ffd640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe0406f400 0x7efe04071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.568+0000 7efe16ffd640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe0406f400 0x7efe04071910 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efe180cedf0 tx=0x7efe0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.572+0000 7efdfffff640 1 -- 172.21.15.80:0/2958091801 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe0801f030 con 0x7efe1806edb0 2024-04-25T13:51:31.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efe18162290 con 0x7efe1806edb0 2024-04-25T13:51:31.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efdfffff640 1 -- 172.21.15.80:0/2958091801 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v175) ==== 104+0+11 (secure 0 0 0) 0x7efe0800d040 con 0x7efe1806edb0 2024-04-25T13:51:31.717 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:51:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe0406f400 msgr2=0x7efe04071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe0406f400 0x7efe04071910 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7efe180cedf0 tx=0x7efe0c008040 comp rx=0 tx=0).stop 2024-04-25T13:51:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe1806edb0 msgr2=0x7efe181489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe1806edb0 0x7efe181489b0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7efe0800b440 tx=0x7efe0800b900 comp rx=0 tx=0).stop 2024-04-25T13:51:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 shutdown_connections 2024-04-25T13:51:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe0406f400 0x7efe04071910 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe18161b60 0x7efe1815b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe180b5330 0x7efe1815ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 --2- 172.21.15.80:0/2958091801 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe1806edb0 0x7efe181489b0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 >> 172.21.15.80:0/2958091801 conn(0x7efe180b4160 msgr2=0x7efe18150e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 shutdown_connections 2024-04-25T13:51:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.724+0000 7efe1dbc9640 1 -- 172.21.15.80:0/2958091801 wait complete. 2024-04-25T13:51:31.737 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:51:31.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.880+0000 7f2039b26640 1 Processor -- start 2024-04-25T13:51:31.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2039b26640 1 -- start start 2024-04-25T13:51:31.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2039b26640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20340b4f10 0x7f20340b69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2039b26640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20340d0390 con 0x7f20340b6f00 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2039b26640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20340d0670 con 0x7f20340b7a20 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2039b26640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20340d0950 con 0x7f20340b4f10 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2032ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f20340b6f00 0x7f20340b7310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41482/0 (socket says 172.21.15.80:41482) 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2032ffd640 1 -- 172.21.15.80:0/3899433118 learned_addr learned my addr 172.21.15.80:0/3899433118 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20337fe640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20340b4f10 0x7f20340b69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300715775 0 0) 0x7f20340d0390 con 0x7f20340b6f00 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f201c003530 con 0x7f20340b6f00 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 219002537 0 0) 0x7f201c003530 con 0x7f20340b6f00 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20340b4f10 msgr2=0x7f20340b69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20340b4f10 0x7f20340b69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 >> v1:172.21.15.87:6789/0 conn(0x7f20340b7a20 legacy=0x7f20340cf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20340d0c30 con 0x7f20340b6f00 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2028001680 con 0x7f20340b6f00 2024-04-25T13:51:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 >> v1:172.21.15.80:6789/0 conn(0x7f20340b6f00 legacy=0x7f20340b7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20340b4f10 0x7f201c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f201c003f50 0x7f201c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f201c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f20340d0950 con 0x7f201c004d50 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f201c00bae0 con 0x7f201c003f50 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f201c00bce0 con 0x7f20340b4f10 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2033fff640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f201c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f20337fe640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20340b4f10 0x7f201c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2032ffd640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f201c003f50 0x7f201c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2033fff640 1 -- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20340b4f10 msgr2=0x7f201c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2033fff640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f20340b4f10 0x7f201c0039f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2033fff640 1 -- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f201c003f50 msgr2=0x7f201c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2033fff640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f201c003f50 0x7f201c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.884+0000 7f2033fff640 1 -- 172.21.15.80:0/3899433118 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201c003530 con 0x7f201c004d50 2024-04-25T13:51:31.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2033fff640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f201c005160 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f20340ce090 tx=0x7f202c01c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66874d6adfbb91b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c0ac070 con 0x7f201c004d50 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202c0a6630 con 0x7f201c004d50 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f20327fc640 1 -- 172.21.15.80:0/3899433118 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202c001b60 con 0x7f201c004d50 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2039b26640 1 -- 172.21.15.80:0/3899433118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 msgr2=0x7f201c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2039b26640 1 --2- 172.21.15.80:0/3899433118 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f201c005160 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f20340ce090 tx=0x7f202c01c840 comp rx=0 tx=0).stop 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2033fff640 1 -- 172.21.15.80:0/3899433118 reap_dead start 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2039b26640 1 -- 172.21.15.80:0/3899433118 shutdown_connections 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2039b26640 1 -- 172.21.15.80:0/3899433118 >> 172.21.15.80:0/3899433118 conn(0x7f20340d3a90 msgr2=0x7f20340d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2039b26640 1 -- 172.21.15.80:0/3899433118 shutdown_connections 2024-04-25T13:51:31.883 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2039b26640 1 -- 172.21.15.80:0/3899433118 wait complete. 2024-04-25T13:51:31.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2039b26640 1 Processor -- start 2024-04-25T13:51:31.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.888+0000 7f2039b26640 1 -- start start 2024-04-25T13:51:31.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f203409d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20341e6f20 0x7f20341e61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f203409f420 0x7f203409f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f20340cff20 con 0x7f201c004d50 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f20340b5320 con 0x7f20341e6f20 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f20340d00a0 con 0x7f203409f420 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f203409d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f203409d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51658/0 (socket says 172.21.15.80:51658) 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 -- 172.21.15.80:0/2796428571 learned_addr learned my addr 172.21.15.80:0/2796428571 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2032ffd640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20341e6f20 0x7f20341e61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2033fff640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f203409f420 0x7f203409f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 -- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f203409f420 msgr2=0x7f203409f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f203409f420 0x7f203409f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 -- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20341e6f20 msgr2=0x7f20341e61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20341e6f20 0x7f20341e61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 -- 172.21.15.80:0/2796428571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20341e8520 con 0x7f201c004d50 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2032ffd640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20341e6f20 0x7f20341e61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f20337fe640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f203409d9e0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f2024010810 tx=0x7f2024010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f201bfff640 1 -- 172.21.15.80:0/2796428571 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202400f040 con 0x7f201c004d50 2024-04-25T13:51:31.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f201bfff640 1 -- 172.21.15.80:0/2796428571 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f202400eb60 con 0x7f201c004d50 2024-04-25T13:51:31.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20341e8800 con 0x7f201c004d50 2024-04-25T13:51:31.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f201bfff640 1 -- 172.21.15.80:0/2796428571 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f202401c400 con 0x7f201c004d50 2024-04-25T13:51:31.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20341e8dc0 con 0x7f201c004d50 2024-04-25T13:51:31.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203406a3a0 con 0x7f201c004d50 2024-04-25T13:51:31.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.892+0000 7f201bfff640 1 -- 172.21.15.80:0/2796428571 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f202401c5a0 con 0x7f201c004d50 2024-04-25T13:51:31.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.896+0000 7f201bfff640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f202006f430 0x7f2020071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:31.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.896+0000 7f201bfff640 1 -- 172.21.15.80:0/2796428571 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f202408fc90 con 0x7f201c004d50 2024-04-25T13:51:31.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.896+0000 7f201bfff640 1 -- 172.21.15.80:0/2796428571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f2020077010 con 0x7f201c004d50 2024-04-25T13:51:31.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.896+0000 7f2032ffd640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f202006f430 0x7f2020071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:31.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.896+0000 7f2032ffd640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f202006f430 0x7f2020071940 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f20341ea0a0 tx=0x7f2028009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:31.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:31.900+0000 7f201bfff640 1 -- 172.21.15.80:0/2796428571 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2024018030 con 0x7f201c004d50 2024-04-25T13:51:32.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.048+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2034071f10 con 0x7f201c004d50 2024-04-25T13:51:32.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.048+0000 7f201bfff640 1 -- 172.21.15.80:0/2796428571 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v175) ==== 100+0+11 (secure 0 0 0) 0x7f202405cc40 con 0x7f201c004d50 2024-04-25T13:51:32.042 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:51:32.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.048+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f202006f430 msgr2=0x7f2020071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.048+0000 7f2039b26640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f202006f430 0x7f2020071940 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f20341ea0a0 tx=0x7f2028009040 comp rx=0 tx=0).stop 2024-04-25T13:51:32.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.048+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 msgr2=0x7f203409d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.048+0000 7f2039b26640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f203409d9e0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f2024010810 tx=0x7f2024010cd0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.052+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 shutdown_connections 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.052+0000 7f2039b26640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f202006f430 0x7f2020071940 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.052+0000 7f2039b26640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f203409f420 0x7f203409f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.052+0000 7f2039b26640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20341e6f20 0x7f20341e61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.052+0000 7f2039b26640 1 --2- 172.21.15.80:0/2796428571 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f201c004d50 0x7f203409d9e0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.052+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 >> 172.21.15.80:0/2796428571 conn(0x7f20340d3a90 msgr2=0x7f2034071250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.052+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 shutdown_connections 2024-04-25T13:51:32.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.052+0000 7f2039b26640 1 -- 172.21.15.80:0/2796428571 wait complete. 2024-04-25T13:51:32.062 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:51:32.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.216+0000 7f2380866640 1 Processor -- start 2024-04-25T13:51:32.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.216+0000 7f2380866640 1 -- start start 2024-04-25T13:51:32.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f2380866640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f237c06eba0 0x7f237c073c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f2380866640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f237c0ce6a0 0x7f237c0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f2380866640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f237c073940 con 0x7f237c0b0e60 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f2380866640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f237c160330 con 0x7f237c0ce6a0 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f2380866640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f237c160580 con 0x7f237c06eba0 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237ad9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f237c0ce6a0 0x7f237c0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237a59c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f237c0b0e60 0x7f237c06e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41484/0 (socket says 172.21.15.80:41484) 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237a59c640 1 -- 172.21.15.80:0/1845936284 learned_addr learned my addr 172.21.15.80:0/1845936284 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f2379d9b640 1 --2- 172.21.15.80:0/1845936284 >> v2:172.21.15.87:3300/0 conn(0x7f237c06eba0 0x7f237c073c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237ad9d640 1 -- 172.21.15.80:0/1845936284 >> v2:172.21.15.87:3300/0 conn(0x7f237c06eba0 msgr2=0x7f237c073c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237ad9d640 1 --2- 172.21.15.80:0/1845936284 >> v2:172.21.15.87:3300/0 conn(0x7f237c06eba0 0x7f237c073c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237ad9d640 1 -- 172.21.15.80:0/1845936284 >> v1:172.21.15.80:6789/0 conn(0x7f237c0b0e60 legacy=0x7f237c06e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237ad9d640 1 -- 172.21.15.80:0/1845936284 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c160860 con 0x7f237c0ce6a0 2024-04-25T13:51:32.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237959a640 1 -- 172.21.15.80:0/1845936284 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3003271921 0 0) 0x7f237c073940 con 0x7f237c0b0e60 2024-04-25T13:51:32.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237ad9d640 1 --2- 172.21.15.80:0/1845936284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c0ceab0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f237400aea0 tx=0x7f237401d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae7c6c001b66fff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237959a640 1 -- 172.21.15.80:0/1845936284 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740a6710 con 0x7f237c0ce6a0 2024-04-25T13:51:32.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237959a640 1 -- 172.21.15.80:0/1845936284 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23740a68b0 con 0x7f237c0ce6a0 2024-04-25T13:51:32.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.220+0000 7f237959a640 1 -- 172.21.15.80:0/1845936284 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740a6c80 con 0x7f237c0ce6a0 2024-04-25T13:51:32.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- 172.21.15.80:0/1845936284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 msgr2=0x7f237c0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 --2- 172.21.15.80:0/1845936284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c0ceab0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f237400aea0 tx=0x7f237401d9a0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- 172.21.15.80:0/1845936284 shutdown_connections 2024-04-25T13:51:32.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 --2- 172.21.15.80:0/1845936284 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c0ceab0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 --2- 172.21.15.80:0/1845936284 >> v2:172.21.15.87:3300/0 conn(0x7f237c06eba0 0x7f237c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- 172.21.15.80:0/1845936284 >> 172.21.15.80:0/1845936284 conn(0x7f237c06d080 msgr2=0x7f237c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- 172.21.15.80:0/1845936284 shutdown_connections 2024-04-25T13:51:32.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- 172.21.15.80:0/1845936284 wait complete. 2024-04-25T13:51:32.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 Processor -- start 2024-04-25T13:51:32.218 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- start start 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f237c0b0e60 0x7f237c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f237c14b700 0x7f237c15bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f237c0cf170 con 0x7f237c0ce6a0 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f237c0ceff0 con 0x7f237c14b700 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.224+0000 7f2380866640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f237c0cf2f0 con 0x7f237c0b0e60 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c15ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51688/0 (socket says 172.21.15.80:51688) 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 -- 172.21.15.80:0/4172744201 learned_addr learned my addr 172.21.15.80:0/4172744201 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:32.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f237ad9d640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f237c14b700 0x7f237c15bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f237a59c640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f237c0b0e60 0x7f237c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 -- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f237c0b0e60 msgr2=0x7f237c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f237c0b0e60 0x7f237c14f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 -- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f237c14b700 msgr2=0x7f237c15bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f237c14b700 0x7f237c15bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 -- 172.21.15.80:0/4172744201 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f237c151df0 con 0x7f237c0ce6a0 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f237ad9d640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f237c14b700 0x7f237c15bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2379d9b640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c15ba20 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f237000d5d0 tx=0x7f237000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f236affd640 1 -- 172.21.15.80:0/4172744201 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23700125f0 con 0x7f237c0ce6a0 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f236affd640 1 -- 172.21.15.80:0/4172744201 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2370019070 con 0x7f237c0ce6a0 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f236affd640 1 -- 172.21.15.80:0/4172744201 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23700129b0 con 0x7f237c0ce6a0 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f237c1520d0 con 0x7f237c0ce6a0 2024-04-25T13:51:32.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f237c152690 con 0x7f237c0ce6a0 2024-04-25T13:51:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f237c06a3a0 con 0x7f237c0ce6a0 2024-04-25T13:51:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f236affd640 1 -- 172.21.15.80:0/4172744201 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2370012b50 con 0x7f237c0ce6a0 2024-04-25T13:51:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f236affd640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f235c06f400 0x7f235c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f236affd640 1 -- 172.21.15.80:0/4172744201 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f237008f5c0 con 0x7f237c0ce6a0 2024-04-25T13:51:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f236affd640 1 -- 172.21.15.80:0/4172744201 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f235c077040 con 0x7f237c0ce6a0 2024-04-25T13:51:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.228+0000 7f237a59c640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f235c06f400 0x7f235c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.232+0000 7f237a59c640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f235c06f400 0x7f235c071910 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f236c000f70 tx=0x7f236c007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.236+0000 7f236affd640 1 -- 172.21.15.80:0/4172744201 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f237005c520 con 0x7f237c0ce6a0 2024-04-25T13:51:32.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f237c06f880 con 0x7f237c0ce6a0 2024-04-25T13:51:32.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f236affd640 1 -- 172.21.15.80:0/4172744201 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v175) ==== 80+0+0 (secure 0 0 0) 0x7f237005be20 con 0x7f237c0ce6a0 2024-04-25T13:51:32.401 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f235c06f400 msgr2=0x7f235c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f235c06f400 0x7f235c071910 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f236c000f70 tx=0x7f236c007c10 comp rx=0 tx=0).stop 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 msgr2=0x7f237c15ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c15ba20 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f237000d5d0 tx=0x7f237000da90 comp rx=0 tx=0).stop 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 shutdown_connections 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f235c06f400 0x7f235c071910 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f237c14b700 0x7f237c15bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f237c0ce6a0 0x7f237c15ba20 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 --2- 172.21.15.80:0/4172744201 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f237c0b0e60 0x7f237c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 >> 172.21.15.80:0/4172744201 conn(0x7f237c06d080 msgr2=0x7f237c158dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.408+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 shutdown_connections 2024-04-25T13:51:32.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.412+0000 7f2380866640 1 -- 172.21.15.80:0/4172744201 wait complete. 2024-04-25T13:51:32.424 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:51:32.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:32.424 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:51:32.432 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[42304]: fuse finished with error 0 and tester_r 0 2024-04-25T13:51:32.434 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:51:32.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:32.434 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:51:32.530 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:51:32.531 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:51:32.531 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:32.537 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:51:32.537 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:32.587 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:32.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.748+0000 7fd9085a7640 1 Processor -- start 2024-04-25T13:51:32.743 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.748+0000 7fd9085a7640 1 -- start start 2024-04-25T13:51:32.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd9085a7640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9000d06e0 0x7fd90006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd9085a7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd90006e9e0 0x7fd90006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd9085a7640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9000d0af0 con 0x7fd900161f90 2024-04-25T13:51:32.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd9085a7640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd9000d0d70 con 0x7fd9000d06e0 2024-04-25T13:51:32.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd9085a7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9000d1050 con 0x7fd90006e9e0 2024-04-25T13:51:32.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd906b44640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd90006e9e0 0x7fd90006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd905b42640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9000d06e0 0x7fd90006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd905b42640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9000d06e0 0x7fd90006e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:56904/0 (socket says 172.21.15.80:56904) 2024-04-25T13:51:32.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd905b42640 1 -- 172.21.15.80:0/2551970813 learned_addr learned my addr 172.21.15.80:0/2551970813 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:32.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd906343640 1 --1- 172.21.15.80:0/2551970813 >> v1:172.21.15.87:6789/0 conn(0x7fd900161f90 0x7fd9000cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:35750/0 (socket says 172.21.15.80:35750) 2024-04-25T13:51:32.746 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd905b42640 1 -- 172.21.15.80:0/2551970813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd90006e9e0 msgr2=0x7fd90006edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:32.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd905b42640 1 --2- 172.21.15.80:0/2551970813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd90006e9e0 0x7fd90006edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd905b42640 1 -- 172.21.15.80:0/2551970813 >> v1:172.21.15.87:6789/0 conn(0x7fd900161f90 legacy=0x7fd9000cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.752+0000 7fd905b42640 1 -- 172.21.15.80:0/2551970813 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90006dca0 con 0x7fd9000d06e0 2024-04-25T13:51:32.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd905b42640 1 --2- 172.21.15.80:0/2551970813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9000d06e0 0x7fd90006e4a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009870 tx=0x7fd8f001c4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56405243d8b2f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd905341640 1 -- 172.21.15.80:0/2551970813 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00b1070 con 0x7fd9000d06e0 2024-04-25T13:51:32.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd905341640 1 -- 172.21.15.80:0/2551970813 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8f00ac410 con 0x7fd9000d06e0 2024-04-25T13:51:32.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd905341640 1 -- 172.21.15.80:0/2551970813 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f00ac7e0 con 0x7fd9000d06e0 2024-04-25T13:51:32.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 -- 172.21.15.80:0/2551970813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9000d06e0 msgr2=0x7fd90006e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2551970813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9000d06e0 0x7fd90006e4a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0009870 tx=0x7fd8f001c4a0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 -- 172.21.15.80:0/2551970813 shutdown_connections 2024-04-25T13:51:32.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2551970813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd90006e9e0 0x7fd90006edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2551970813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9000d06e0 0x7fd90006e4a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 -- 172.21.15.80:0/2551970813 >> 172.21.15.80:0/2551970813 conn(0x7fd90006d080 msgr2=0x7fd9000b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 -- 172.21.15.80:0/2551970813 shutdown_connections 2024-04-25T13:51:32.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 -- 172.21.15.80:0/2551970813 wait complete. 2024-04-25T13:51:32.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 Processor -- start 2024-04-25T13:51:32.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.756+0000 7fd9085a7640 1 -- start start 2024-04-25T13:51:32.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd90006e9e0 0x7fd9001475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9000d06e0 0x7fd90015d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd900161f90 0x7fd90015fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd9001447a0 con 0x7fd90006e9e0 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd900144620 con 0x7fd900161f90 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd90016b510 con 0x7fd9000d06e0 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd90006e9e0 0x7fd9001475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd90006e9e0 0x7fd9001475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51696/0 (socket says 172.21.15.80:51696) 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 -- 172.21.15.80:0/2162492297 learned_addr learned my addr 172.21.15.80:0/2162492297 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906b44640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd900161f90 0x7fd90015fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd905b42640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9000d06e0 0x7fd90015d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 -- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9000d06e0 msgr2=0x7fd90015d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9000d06e0 0x7fd90015d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.754 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 -- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd900161f90 msgr2=0x7fd90015fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd900161f90 0x7fd90015fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 -- 172.21.15.80:0/2162492297 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9001666d0 con 0x7fd90006e9e0 2024-04-25T13:51:32.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd906343640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd90006e9e0 0x7fd9001475a0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00c310 tx=0x7fd8fc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd8f6ffd640 1 -- 172.21.15.80:0/2162492297 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0074e0 con 0x7fd90006e9e0 2024-04-25T13:51:32.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd8f6ffd640 1 -- 172.21.15.80:0/2162492297 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8fc007680 con 0x7fd90006e9e0 2024-04-25T13:51:32.755 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd8f6ffd640 1 -- 172.21.15.80:0/2162492297 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc004a30 con 0x7fd90006e9e0 2024-04-25T13:51:32.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9001669b0 con 0x7fd90006e9e0 2024-04-25T13:51:32.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd900166f70 con 0x7fd90006e9e0 2024-04-25T13:51:32.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd90006a3a0 con 0x7fd90006e9e0 2024-04-25T13:51:32.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.760+0000 7fd8f6ffd640 1 -- 172.21.15.80:0/2162492297 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd8fc01a020 con 0x7fd90006e9e0 2024-04-25T13:51:32.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.764+0000 7fd8f6ffd640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8e406f400 0x7fd8e4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:32.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.764+0000 7fd8f6ffd640 1 -- 172.21.15.80:0/2162492297 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7fd8fc08ea70 con 0x7fd90006e9e0 2024-04-25T13:51:32.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.764+0000 7fd905b42640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8e406f400 0x7fd8e4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:32.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.764+0000 7fd8f6ffd640 1 -- 172.21.15.80:0/2162492297 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7fd8e4077040 con 0x7fd90006e9e0 2024-04-25T13:51:32.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.764+0000 7fd905b42640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8e406f400 0x7fd8e4071910 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00ae000 tx=0x7fd8f0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:32.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.768+0000 7fd8f6ffd640 1 -- 172.21.15.80:0/2162492297 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd9000d0af0 con 0x7fd90006e9e0 2024-04-25T13:51:32.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.928+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9000b69c0 con 0x7fd90006e9e0 2024-04-25T13:51:32.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd8f6ffd640 1 -- 172.21.15.80:0/2162492297 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) ==== 74+0+16683 (secure 0 0 0) 0x7fd8fc05b940 con 0x7fd90006e9e0 2024-04-25T13:51:32.924 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:32.925 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":175,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:21.279640+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:50:28.108369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T13:50:29.324161+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:32.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8e406f400 msgr2=0x7fd8e4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8e406f400 0x7fd8e4071910 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fd8f00ae000 tx=0x7fd8f0008040 comp rx=0 tx=0).stop 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd90006e9e0 msgr2=0x7fd9001475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd90006e9e0 0x7fd9001475a0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc00c310 tx=0x7fd8fc00c7d0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 shutdown_connections 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd8e406f400 0x7fd8e4071910 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd900161f90 0x7fd90015fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9000d06e0 0x7fd90015d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 --2- 172.21.15.80:0/2162492297 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd90006e9e0 0x7fd9001475a0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 >> 172.21.15.80:0/2162492297 conn(0x7fd90006d080 msgr2=0x7fd9000b2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 shutdown_connections 2024-04-25T13:51:32.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:32.932+0000 7fd9085a7640 1 -- 172.21.15.80:0/2162492297 wait complete. 2024-04-25T13:51:32.945 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:50:28.108369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:50:29.324161+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:51:32.946 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:51:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.104+0000 7fbb938cc640 1 Processor -- start 2024-04-25T13:51:33.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.104+0000 7fbb938cc640 1 -- start start 2024-04-25T13:51:33.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb938cc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbb8c0d3ed0 0x7fbb8c0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb938cc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb938cc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb8c0b92a0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb938cc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fbb8c0d4c10 con 0x7fbb8c0d3ed0 2024-04-25T13:51:33.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb938cc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8c0d4ef0 con 0x7fbb8c0d2ad0 2024-04-25T13:51:33.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb91668640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbb8c0d3ed0 0x7fbb8c0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb91668640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fbb8c0d3ed0 0x7fbb8c0d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51702/0 (socket says 172.21.15.80:51702) 2024-04-25T13:51:33.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb91668640 1 -- 172.21.15.80:0/972929920 learned_addr learned my addr 172.21.15.80:0/972929920 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:33.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb91668640 1 -- 172.21.15.80:0/972929920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 msgr2=0x7fbb8c0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:33.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb90e67640 1 --2- 172.21.15.80:0/972929920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb91668640 1 --2- 172.21.15.80:0/972929920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c0b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb91668640 1 -- 172.21.15.80:0/972929920 >> v1:172.21.15.87:6789/0 conn(0x7fbb8c0b7c10 legacy=0x7fbb8c0b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:33.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb91668640 1 -- 172.21.15.80:0/972929920 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c0d51d0 con 0x7fbb8c0d3ed0 2024-04-25T13:51:33.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb90e67640 1 --2- 172.21.15.80:0/972929920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:33.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb91668640 1 --2- 172.21.15.80:0/972929920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0d3ed0 0x7fbb8c0d2590 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009870 tx=0x7fbb8801c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d848077b60b9f27 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb83fff640 1 -- 172.21.15.80:0/972929920 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880b0070 con 0x7fbb8c0d3ed0 2024-04-25T13:51:33.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb83fff640 1 -- 172.21.15.80:0/972929920 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb880ab600 con 0x7fbb8c0d3ed0 2024-04-25T13:51:33.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.108+0000 7fbb83fff640 1 -- 172.21.15.80:0/972929920 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880ab9d0 con 0x7fbb8c0d3ed0 2024-04-25T13:51:33.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- 172.21.15.80:0/972929920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0d3ed0 msgr2=0x7fbb8c0d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 --2- 172.21.15.80:0/972929920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0d3ed0 0x7fbb8c0d2590 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fbb88009870 tx=0x7fbb8801c340 comp rx=0 tx=0).stop 2024-04-25T13:51:33.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- 172.21.15.80:0/972929920 shutdown_connections 2024-04-25T13:51:33.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 --2- 172.21.15.80:0/972929920 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 --2- 172.21.15.80:0/972929920 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0d3ed0 0x7fbb8c0d2590 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- 172.21.15.80:0/972929920 >> 172.21.15.80:0/972929920 conn(0x7fbb8c0737b0 msgr2=0x7fbb8c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:33.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- 172.21.15.80:0/972929920 shutdown_connections 2024-04-25T13:51:33.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- 172.21.15.80:0/972929920 wait complete. 2024-04-25T13:51:33.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 Processor -- start 2024-04-25T13:51:33.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- start start 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0b7c10 0x7fbb8c13eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c15ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0d3ed0 0x7fbb8c159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c0ccae0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8c0cc960 con 0x7fbb8c0d3ed0 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb938cc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fbb8c160620 con 0x7fbb8c0d2ad0 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb91668640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0b7c10 0x7fbb8c13eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb91668640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0b7c10 0x7fbb8c13eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51716/0 (socket says 172.21.15.80:51716) 2024-04-25T13:51:33.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.112+0000 7fbb91668640 1 -- 172.21.15.80:0/3976965156 learned_addr learned my addr 172.21.15.80:0/3976965156 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb91e69640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0d3ed0 0x7fbb8c159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb90e67640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c15ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb91668640 1 -- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 msgr2=0x7fbb8c15ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb91668640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c15ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb91668640 1 -- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0d3ed0 msgr2=0x7fbb8c159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb91668640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0d3ed0 0x7fbb8c159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb91668640 1 -- 172.21.15.80:0/3976965156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb8c1504f0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb91668640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0b7c10 0x7fbb8c13eeb0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fbb880ae040 tx=0x7fbb8801ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb81ffb640 1 -- 172.21.15.80:0/3976965156 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880b0040 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb81ffb640 1 -- 172.21.15.80:0/3976965156 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb880bc070 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb8c1507d0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb81ffb640 1 -- 172.21.15.80:0/3976965156 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb880b6880 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb8c150d90 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8c06a3a0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb81ffb640 1 -- 172.21.15.80:0/3976965156 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fbb880b6a20 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb81ffb640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbb7006f430 0x7fbb70071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb81ffb640 1 -- 172.21.15.80:0/3976965156 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7fbb881296f0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb81ffb640 1 -- 172.21.15.80:0/3976965156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7fbb70077010 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb90e67640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbb7006f430 0x7fbb70071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.116+0000 7fbb90e67640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbb7006f430 0x7fbb70071940 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c006fd0 tx=0x7fbb7c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.115 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.120+0000 7fbb81ffb640 1 -- 172.21.15.80:0/3976965156 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fbb8c0b92a0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb8c1570a0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb81ffb640 1 -- 172.21.15.80:0/3976965156 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) ==== 93+0+10 (secure 0 0 0) 0x7fbb880f66e0 con 0x7fbb8c0b7c10 2024-04-25T13:51:33.278 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbb7006f430 msgr2=0x7fbb70071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbb7006f430 0x7fbb70071940 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c006fd0 tx=0x7fbb7c008040 comp rx=0 tx=0).stop 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0b7c10 msgr2=0x7fbb8c13eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0b7c10 0x7fbb8c13eeb0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fbb880ae040 tx=0x7fbb8801ab80 comp rx=0 tx=0).stop 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 shutdown_connections 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fbb7006f430 0x7fbb70071940 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbb8c0d3ed0 0x7fbb8c159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fbb8c0d2ad0 0x7fbb8c15ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 --2- 172.21.15.80:0/3976965156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fbb8c0b7c10 0x7fbb8c13eeb0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.284+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 >> 172.21.15.80:0/3976965156 conn(0x7fbb8c0737b0 msgr2=0x7fbb8c0caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:33.280 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.288+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 shutdown_connections 2024-04-25T13:51:33.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.288+0000 7fbb938cc640 1 -- 172.21.15.80:0/3976965156 wait complete. 2024-04-25T13:51:33.300 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:51:33.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff1962da640 1 Processor -- start 2024-04-25T13:51:33.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff1962da640 1 -- start start 2024-04-25T13:51:33.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff1962da640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900b5330 0x7ff1900d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff1962da640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff190161b60 0x7ff190161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff1962da640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1901624b0 con 0x7ff1900d0ff0 2024-04-25T13:51:33.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff1962da640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff1901629b0 con 0x7ff190161b60 2024-04-25T13:51:33.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff1962da640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff1900b5740 con 0x7ff1900b5330 2024-04-25T13:51:33.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff194877640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff190161b60 0x7ff190161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff194877640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff190161b60 0x7ff190161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51720/0 (socket says 172.21.15.80:51720) 2024-04-25T13:51:33.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff194877640 1 -- 172.21.15.80:0/3548234557 learned_addr learned my addr 172.21.15.80:0/3548234557 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:33.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff18ffff640 1 --2- 172.21.15.80:0/3548234557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900b5330 0x7ff1900d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff194877640 1 -- 172.21.15.80:0/3548234557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900b5330 msgr2=0x7ff1900d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:33.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff194877640 1 --2- 172.21.15.80:0/3548234557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900b5330 0x7ff1900d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff194877640 1 -- 172.21.15.80:0/3548234557 >> v1:172.21.15.87:6789/0 conn(0x7ff1900d0ff0 legacy=0x7ff19006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:33.452 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.456+0000 7ff194877640 1 -- 172.21.15.80:0/3548234557 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19006dd10 con 0x7ff190161b60 2024-04-25T13:51:33.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff194877640 1 --2- 172.21.15.80:0/3548234557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff190161b60 0x7ff190161f70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ff188009e50 tx=0x7ff18801c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0958d5632a5338 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff18f7fe640 1 -- 172.21.15.80:0/3548234557 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880a6500 con 0x7ff190161b60 2024-04-25T13:51:33.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff18f7fe640 1 -- 172.21.15.80:0/3548234557 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1880a66a0 con 0x7ff190161b60 2024-04-25T13:51:33.453 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff18f7fe640 1 -- 172.21.15.80:0/3548234557 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1880a6a70 con 0x7ff190161b60 2024-04-25T13:51:33.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 -- 172.21.15.80:0/3548234557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff190161b60 msgr2=0x7ff190161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 --2- 172.21.15.80:0/3548234557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff190161b60 0x7ff190161f70 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7ff188009e50 tx=0x7ff18801c780 comp rx=0 tx=0).stop 2024-04-25T13:51:33.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 -- 172.21.15.80:0/3548234557 shutdown_connections 2024-04-25T13:51:33.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 --2- 172.21.15.80:0/3548234557 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff190161b60 0x7ff190161f70 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 --2- 172.21.15.80:0/3548234557 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1900b5330 0x7ff1900d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 -- 172.21.15.80:0/3548234557 >> 172.21.15.80:0/3548234557 conn(0x7ff1900b4160 msgr2=0x7ff1900d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:33.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 -- 172.21.15.80:0/3548234557 shutdown_connections 2024-04-25T13:51:33.454 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 -- 172.21.15.80:0/3548234557 wait complete. 2024-04-25T13:51:33.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 Processor -- start 2024-04-25T13:51:33.455 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.460+0000 7ff1962da640 1 -- start start 2024-04-25T13:51:33.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900b5330 0x7ff19013f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900d0ff0 0x7ff19015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff190161b60 0x7ff1901567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff1900b0cd0 con 0x7ff1900d0ff0 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff19006e890 con 0x7ff1900b5330 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff1900b0e50 con 0x7ff190161b60 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900d0ff0 0x7ff19015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900d0ff0 0x7ff19015b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51722/0 (socket says 172.21.15.80:51722) 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 -- 172.21.15.80:0/1236630261 learned_addr learned my addr 172.21.15.80:0/1236630261 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff194877640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff190161b60 0x7ff1901567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900b5330 0x7ff19013f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 -- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff190161b60 msgr2=0x7ff1901567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff190161b60 0x7ff1901567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 -- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900b5330 msgr2=0x7ff19013f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900b5330 0x7ff19013f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 -- 172.21.15.80:0/1236630261 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19014d460 con 0x7ff1900d0ff0 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900b5330 0x7ff19013f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff187fff640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900d0ff0 0x7ff19015b7c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ff17800b3f0 tx=0x7ff17800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff18d7fa640 1 -- 172.21.15.80:0/1236630261 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178015020 con 0x7ff1900d0ff0 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff18d7fa640 1 -- 172.21.15.80:0/1236630261 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff178011830 con 0x7ff1900d0ff0 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff18d7fa640 1 -- 172.21.15.80:0/1236630261 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff178011c00 con 0x7ff1900d0ff0 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19014d740 con 0x7ff1900d0ff0 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19014dd00 con 0x7ff1900d0ff0 2024-04-25T13:51:33.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19014d1d0 con 0x7ff1900d0ff0 2024-04-25T13:51:33.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff18d7fa640 1 -- 172.21.15.80:0/1236630261 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff178018020 con 0x7ff1900d0ff0 2024-04-25T13:51:33.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff18d7fa640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17006f400 0x7ff170071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.464+0000 7ff18d7fa640 1 -- 172.21.15.80:0/1236630261 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7ff17808ef00 con 0x7ff1900d0ff0 2024-04-25T13:51:33.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.468+0000 7ff18d7fa640 1 -- 172.21.15.80:0/1236630261 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7ff170077040 con 0x7ff1900d0ff0 2024-04-25T13:51:33.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.468+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17006f400 0x7ff170071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.468+0000 7ff18ffff640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17006f400 0x7ff170071910 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff1800097a0 tx=0x7ff180008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.472+0000 7ff18d7fa640 1 -- 172.21.15.80:0/1236630261 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff17801f030 con 0x7ff1900d0ff0 2024-04-25T13:51:33.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.620+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff19006a3a0 con 0x7ff1900d0ff0 2024-04-25T13:51:33.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.620+0000 7ff18d7fa640 1 -- 172.21.15.80:0/1236630261 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v175) ==== 104+0+11 (secure 0 0 0) 0x7ff17805bdd0 con 0x7ff1900d0ff0 2024-04-25T13:51:33.614 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:51:33.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17006f400 msgr2=0x7ff170071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17006f400 0x7ff170071910 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ff1800097a0 tx=0x7ff180008040 comp rx=0 tx=0).stop 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900d0ff0 msgr2=0x7ff19015b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900d0ff0 0x7ff19015b7c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7ff17800b3f0 tx=0x7ff17800b8b0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 shutdown_connections 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff17006f400 0x7ff170071910 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff190161b60 0x7ff1901567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1900d0ff0 0x7ff19015b7c0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 --2- 172.21.15.80:0/1236630261 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1900b5330 0x7ff19013f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 >> 172.21.15.80:0/1236630261 conn(0x7ff1900b4160 msgr2=0x7ff1901510d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 shutdown_connections 2024-04-25T13:51:33.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.624+0000 7ff1962da640 1 -- 172.21.15.80:0/1236630261 wait complete. 2024-04-25T13:51:33.635 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:51:33.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.792+0000 7f40e2d9e640 1 Processor -- start 2024-04-25T13:51:33.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.792+0000 7f40e2d9e640 1 -- start start 2024-04-25T13:51:33.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e2d9e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc071e60 0x7f40dc1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e2d9e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40dc06fbf0 con 0x7f40dc073c40 2024-04-25T13:51:33.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e2d9e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40dc0b0a20 con 0x7f40dc161170 2024-04-25T13:51:33.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e2d9e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40dc0b1350 con 0x7f40dc071e60 2024-04-25T13:51:33.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40dbfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f40dc073c40 0x7f40dc160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41494/0 (socket says 172.21.15.80:41494) 2024-04-25T13:51:33.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40dbfff640 1 -- 172.21.15.80:0/1926339231 learned_addr learned my addr 172.21.15.80:0/1926339231 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:33.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e0b3a640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc071e60 0x7f40dc1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1073666851 0 0) 0x7f40dc06fbf0 con 0x7f40dc073c40 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f40cc003530 con 0x7f40dc073c40 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2658854030 0 0) 0x7f40cc003530 con 0x7f40dc073c40 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc071e60 msgr2=0x7f40dc1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc071e60 0x7f40dc1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 >> v1:172.21.15.87:6789/0 conn(0x7f40dc161170 legacy=0x7f40dc0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc0b0d00 con 0x7f40dc073c40 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f40c8001680 con 0x7f40dc073c40 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 >> v1:172.21.15.80:6789/0 conn(0x7f40dc073c40 legacy=0x7f40dc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc0039f0 0x7f40cc003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc071e60 0x7f40cc004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40cc004d50 0x7f40cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc0b1350 con 0x7f40cc004d50 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f40cc00bb60 con 0x7f40cc0039f0 2024-04-25T13:51:33.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40cc00bd10 con 0x7f40dc071e60 2024-04-25T13:51:33.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e133b640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40cc004d50 0x7f40cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e0b3a640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc0039f0 0x7f40cc003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40dbfff640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc071e60 0x7f40cc004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e133b640 1 -- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc071e60 msgr2=0x7f40cc004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:33.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e133b640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc071e60 0x7f40cc004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e133b640 1 -- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc0039f0 msgr2=0x7f40cc003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:33.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e133b640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc0039f0 0x7f40cc003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.796+0000 7f40e133b640 1 -- 172.21.15.80:0/1926339231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40cc003530 con 0x7f40cc004d50 2024-04-25T13:51:33.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e133b640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40cc004d50 0x7f40cc005180 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f40d4001e70 tx=0x7f40d401c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58589cd761e0fe03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4001a90 con 0x7f40cc004d50 2024-04-25T13:51:33.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d4003b80 con 0x7f40cc004d50 2024-04-25T13:51:33.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40db7fe640 1 -- 172.21.15.80:0/1926339231 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4005080 con 0x7f40cc004d50 2024-04-25T13:51:33.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e2d9e640 1 -- 172.21.15.80:0/1926339231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40cc004d50 msgr2=0x7f40cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e2d9e640 1 --2- 172.21.15.80:0/1926339231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40cc004d50 0x7f40cc005180 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f40d4001e70 tx=0x7f40d401c780 comp rx=0 tx=0).stop 2024-04-25T13:51:33.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e133b640 1 -- 172.21.15.80:0/1926339231 reap_dead start 2024-04-25T13:51:33.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e2d9e640 1 -- 172.21.15.80:0/1926339231 shutdown_connections 2024-04-25T13:51:33.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e2d9e640 1 -- 172.21.15.80:0/1926339231 >> 172.21.15.80:0/1926339231 conn(0x7f40dc0b4160 msgr2=0x7f40dc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:33.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e2d9e640 1 -- 172.21.15.80:0/1926339231 shutdown_connections 2024-04-25T13:51:33.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e2d9e640 1 -- 172.21.15.80:0/1926339231 wait complete. 2024-04-25T13:51:33.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e2d9e640 1 Processor -- start 2024-04-25T13:51:33.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.800+0000 7f40e2d9e640 1 -- start start 2024-04-25T13:51:33.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc004d50 0x7f40dc0cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc15bb30 0x7f40dc15bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40dc14e840 0x7f40dc14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc072270 con 0x7f40dc14e840 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f40dc06f9a0 con 0x7f40cc004d50 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40dc074050 con 0x7f40dc15bb30 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40dc14e840 0x7f40dc14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40dc14e840 0x7f40dc14ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51746/0 (socket says 172.21.15.80:51746) 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 -- 172.21.15.80:0/4214667969 learned_addr learned my addr 172.21.15.80:0/4214667969 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40dbfff640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc15bb30 0x7f40dc15bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e0b3a640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc004d50 0x7f40dc0cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 -- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc15bb30 msgr2=0x7f40dc15bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc15bb30 0x7f40dc15bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 -- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc004d50 msgr2=0x7f40dc0cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc004d50 0x7f40dc0cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 -- 172.21.15.80:0/4214667969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40dc0ce280 con 0x7f40dc14e840 2024-04-25T13:51:33.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e133b640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40dc14e840 0x7f40dc14ec50 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f40d4098190 tx=0x7f40cc005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40d97fa640 1 -- 172.21.15.80:0/4214667969 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d40ba070 con 0x7f40dc14e840 2024-04-25T13:51:33.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40d97fa640 1 -- 172.21.15.80:0/4214667969 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40d4005800 con 0x7f40dc14e840 2024-04-25T13:51:33.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40dc0cf2e0 con 0x7f40dc14e840 2024-04-25T13:51:33.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40d97fa640 1 -- 172.21.15.80:0/4214667969 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4001a90 con 0x7f40dc14e840 2024-04-25T13:51:33.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40dc152ec0 con 0x7f40dc14e840 2024-04-25T13:51:33.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40dc0d0850 con 0x7f40dc14e840 2024-04-25T13:51:33.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.804+0000 7f40d97fa640 1 -- 172.21.15.80:0/4214667969 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f40d40bd070 con 0x7f40dc14e840 2024-04-25T13:51:33.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.808+0000 7f40d97fa640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40c006f430 0x7f40c0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:33.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.808+0000 7f40d97fa640 1 -- 172.21.15.80:0/4214667969 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f40d4128c20 con 0x7f40dc14e840 2024-04-25T13:51:33.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.808+0000 7f40d97fa640 1 -- 172.21.15.80:0/4214667969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f40c0077010 con 0x7f40dc14e840 2024-04-25T13:51:33.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.808+0000 7f40e0b3a640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40c006f430 0x7f40c0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:33.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.808+0000 7f40e0b3a640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40c006f430 0x7f40c0071940 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009a30 tx=0x7f40d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:33.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.812+0000 7f40d97fa640 1 -- 172.21.15.80:0/4214667969 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f40dc0cf2e0 con 0x7f40dc14e840 2024-04-25T13:51:33.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.972+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f40dc06a3a0 con 0x7f40dc14e840 2024-04-25T13:51:33.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.972+0000 7f40d97fa640 1 -- 172.21.15.80:0/4214667969 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v175) ==== 100+0+11 (secure 0 0 0) 0x7f40d40ac020 con 0x7f40dc14e840 2024-04-25T13:51:33.967 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40c006f430 msgr2=0x7f40c0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40c006f430 0x7f40c0071940 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f40d0009a30 tx=0x7f40d0008040 comp rx=0 tx=0).stop 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40dc14e840 msgr2=0x7f40dc14ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40dc14e840 0x7f40dc14ec50 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f40d4098190 tx=0x7f40cc005840 comp rx=0 tx=0).stop 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 shutdown_connections 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40c006f430 0x7f40c0071940 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40dc14e840 0x7f40dc14ec50 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40dc15bb30 0x7f40dc15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.970 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 --2- 172.21.15.80:0/4214667969 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc004d50 0x7f40dc0cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:33.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 >> 172.21.15.80:0/4214667969 conn(0x7f40dc0b4160 msgr2=0x7f40dc15ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:33.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 shutdown_connections 2024-04-25T13:51:33.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:33.976+0000 7f40e2d9e640 1 -- 172.21.15.80:0/4214667969 wait complete. 2024-04-25T13:51:33.992 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:51:34.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f139855f640 1 Processor -- start 2024-04-25T13:51:34.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f139855f640 1 -- start start 2024-04-25T13:51:34.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f139855f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1390149410 0x7f139006f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f139855f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139006caf0 con 0x7f1390068ca0 2024-04-25T13:51:34.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f139855f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f139006cdd0 con 0x7f13900b2020 2024-04-25T13:51:34.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f139855f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f139006d0b0 con 0x7f1390149410 2024-04-25T13:51:34.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f1395afa640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1390068ca0 0x7f139006f2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41504/0 (socket says 172.21.15.80:41504) 2024-04-25T13:51:34.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f1395afa640 1 -- 172.21.15.80:0/1183359598 learned_addr learned my addr 172.21.15.80:0/1183359598 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:34.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f1396afc640 1 --2- 172.21.15.80:0/1183359598 >> v2:172.21.15.87:3300/0 conn(0x7f1390149410 0x7f139006f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1066429203 0 0) 0x7f139006caf0 con 0x7f1390068ca0 2024-04-25T13:51:34.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.148+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f137c003530 con 0x7f1390068ca0 2024-04-25T13:51:34.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2171427889 0 0) 0x7f137c003530 con 0x7f1390068ca0 2024-04-25T13:51:34.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 >> v2:172.21.15.87:3300/0 conn(0x7f1390149410 msgr2=0x7f139006f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 --2- 172.21.15.80:0/1183359598 >> v2:172.21.15.87:3300/0 conn(0x7f1390149410 0x7f139006f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 >> v1:172.21.15.87:6789/0 conn(0x7f13900b2020 legacy=0x7f1390068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139006d390 con 0x7f1390068ca0 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1380001680 con 0x7f1390068ca0 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 >> v1:172.21.15.80:6789/0 conn(0x7f1390068ca0 legacy=0x7f139006f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137c0039c0 0x7f137c003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f137c004810 0x7f137c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f137c0051d0 0x7f137c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f139006d0b0 con 0x7f137c0039c0 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f137c00c170 con 0x7f137c0051d0 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f137c00c370 con 0x7f137c004810 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13962fb640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137c0039c0 0x7f137c003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f1396afc640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f137c0051d0 0x7f137c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13962fb640 1 -- 172.21.15.80:0/1183359598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f137c004810 msgr2=0x7f137c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:34.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13962fb640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f137c004810 0x7f137c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13962fb640 1 -- 172.21.15.80:0/1183359598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f137c0051d0 msgr2=0x7f137c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:34.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13962fb640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f137c0051d0 0x7f137c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f1396afc640 1 -- 172.21.15.80:0/1183359598 reap_dead start 2024-04-25T13:51:34.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13962fb640 1 -- 172.21.15.80:0/1183359598 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137c003530 con 0x7f137c0039c0 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13962fb640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137c0039c0 0x7f137c003e30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f138c001780 tx=0x7f138c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c05d8fc17bc5a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c0013a0 con 0x7f137c0039c0 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138c0052f0 con 0x7f137c0039c0 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f13952f9640 1 -- 172.21.15.80:0/1183359598 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c003840 con 0x7f137c0039c0 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f139855f640 1 -- 172.21.15.80:0/1183359598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137c0039c0 msgr2=0x7f137c003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f139855f640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137c0039c0 0x7f137c003e30 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f138c001780 tx=0x7f138c01c340 comp rx=0 tx=0).stop 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f139855f640 1 -- 172.21.15.80:0/1183359598 shutdown_connections 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f139855f640 1 --2- 172.21.15.80:0/1183359598 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f137c0039c0 0x7f137c003e30 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f139855f640 1 -- 172.21.15.80:0/1183359598 >> 172.21.15.80:0/1183359598 conn(0x7f13900d28f0 msgr2=0x7f13900d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f139855f640 1 -- 172.21.15.80:0/1183359598 shutdown_connections 2024-04-25T13:51:34.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.152+0000 7f139855f640 1 -- 172.21.15.80:0/1183359598 wait complete. 2024-04-25T13:51:34.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f139855f640 1 Processor -- start 2024-04-25T13:51:34.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f139855f640 1 -- start start 2024-04-25T13:51:34.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f139855f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f137c0039c0 0x7f139013ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f139855f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f139015f700 0x7f139015fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f139855f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13900d12e0 0x7f13900d16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f139855f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1390164980 con 0x7f13900d12e0 2024-04-25T13:51:34.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f139855f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1390164800 con 0x7f139015f700 2024-04-25T13:51:34.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f139855f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1390164b00 con 0x7f137c0039c0 2024-04-25T13:51:34.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13900d12e0 0x7f13900d16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13900d12e0 0x7f13900d16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51772/0 (socket says 172.21.15.80:51772) 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 -- 172.21.15.80:0/2164054610 learned_addr learned my addr 172.21.15.80:0/2164054610 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1395afa640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f139015f700 0x7f139015fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f13962fb640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f137c0039c0 0x7f139013ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 -- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f137c0039c0 msgr2=0x7f139013ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f137c0039c0 0x7f139013ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 -- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f139015f700 msgr2=0x7f139015fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f139015f700 0x7f139015fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 -- 172.21.15.80:0/2164054610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13900d1c30 con 0x7f13900d12e0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.156+0000 7f1396afc640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13900d12e0 0x7f13900d16f0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f1390164ee0 tx=0x7f13880115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f1386ffd640 1 -- 172.21.15.80:0/2164054610 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388011db0 con 0x7f13900d12e0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f1386ffd640 1 -- 172.21.15.80:0/2164054610 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f138801b070 con 0x7f13900d12e0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f1386ffd640 1 -- 172.21.15.80:0/2164054610 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13880039a0 con 0x7f13900d12e0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13900cf0a0 con 0x7f13900d12e0 2024-04-25T13:51:34.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13900cf880 con 0x7f13900d12e0 2024-04-25T13:51:34.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13900cf310 con 0x7f13900d12e0 2024-04-25T13:51:34.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f1386ffd640 1 -- 172.21.15.80:0/2164054610 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1388003b40 con 0x7f13900d12e0 2024-04-25T13:51:34.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f1386ffd640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f1386ffd640 1 -- 172.21.15.80:0/2164054610 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f138808fb20 con 0x7f13900d12e0 2024-04-25T13:51:34.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f1386ffd640 1 -- 172.21.15.80:0/2164054610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f1374077040 con 0x7f13900d12e0 2024-04-25T13:51:34.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f13962fb640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.160+0000 7f13962fb640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f138c001750 tx=0x7f138c0011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:34.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.164+0000 7f1386ffd640 1 -- 172.21.15.80:0/2164054610 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13900d1c30 con 0x7f13900d12e0 2024-04-25T13:51:34.328 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f139006a3a0 con 0x7f13900d12e0 2024-04-25T13:51:34.329 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f1386ffd640 1 -- 172.21.15.80:0/2164054610 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v175) ==== 80+0+0 (secure 0 0 0) 0x7f138805ca80 con 0x7f13900d12e0 2024-04-25T13:51:34.329 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 msgr2=0x7f1374071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f138c001750 tx=0x7f138c0011a0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13900d12e0 msgr2=0x7f13900d16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13900d12e0 0x7f13900d16f0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f1390164ee0 tx=0x7f13880115a0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 shutdown_connections 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f137406f400 0x7f1374071910 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13900d12e0 0x7f13900d16f0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f139015f700 0x7f139015fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 --2- 172.21.15.80:0/2164054610 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f137c0039c0 0x7f139013ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 >> 172.21.15.80:0/2164054610 conn(0x7f13900d28f0 msgr2=0x7f13900b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:34.332 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 shutdown_connections 2024-04-25T13:51:34.333 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.336+0000 7f139855f640 1 -- 172.21.15.80:0/2164054610 wait complete. 2024-04-25T13:51:34.350 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:51:34.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:34.351 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:51:34.364 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25587]: fuse finished with error 0 and tester_r 0 2024-04-25T13:51:34.365 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:51:34.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:34.366 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:51:34.489 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:51:34.489 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:51:34.489 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:34.495 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:51:34.495 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:51:34.495 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:51:34.495 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:34.500 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:51:34.501 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:34.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f359b92b640 1 Processor -- start 2024-04-25T13:51:34.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f359b92b640 1 -- start start 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f359b92b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35940b0ed0 0x7f359415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f359b92b640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f359406eba0 0x7f35940ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f359b92b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f359415cda0 con 0x7f35940ce730 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f359b92b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3594073710 con 0x7f35940b0ed0 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f359b92b640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f35940739f0 con 0x7f359406eba0 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f35996c7640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f35940b0ed0 0x7f359415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f3598ec6640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f35940ce730 0x7f359406e490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41520/0 (socket says 172.21.15.80:41520) 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f3598ec6640 1 -- 172.21.15.80:0/3263551845 learned_addr learned my addr 172.21.15.80:0/3263551845 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f3599ec8640 1 --2- 172.21.15.80:0/3263551845 >> v2:172.21.15.87:3300/0 conn(0x7f359406eba0 0x7f35940ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f35996c7640 1 -- 172.21.15.80:0/3263551845 >> v2:172.21.15.87:3300/0 conn(0x7f359406eba0 msgr2=0x7f35940ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f35996c7640 1 --2- 172.21.15.80:0/3263551845 >> v2:172.21.15.87:3300/0 conn(0x7f359406eba0 0x7f35940ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f35996c7640 1 -- 172.21.15.80:0/3263551845 >> v1:172.21.15.80:6789/0 conn(0x7f35940ce730 legacy=0x7f359406e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f35996c7640 1 -- 172.21.15.80:0/3263551845 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3594073cd0 con 0x7f35940b0ed0 2024-04-25T13:51:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.692+0000 7f358bfff640 1 -- 172.21.15.80:0/3263551845 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337425110 0 0) 0x7f359415cda0 con 0x7f35940ce730 2024-04-25T13:51:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f35996c7640 1 --2- 172.21.15.80:0/3263551845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940b0ed0 0x7f359415c860 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f35940b4950 tx=0x7f3590021340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5f4c62eda2f2503 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f358bfff640 1 -- 172.21.15.80:0/3263551845 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900b6070 con 0x7f35940b0ed0 2024-04-25T13:51:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f358bfff640 1 -- 172.21.15.80:0/3263551845 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35900b1600 con 0x7f35940b0ed0 2024-04-25T13:51:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f358bfff640 1 -- 172.21.15.80:0/3263551845 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35900b19d0 con 0x7f35940b0ed0 2024-04-25T13:51:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 -- 172.21.15.80:0/3263551845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940b0ed0 msgr2=0x7f359415c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 --2- 172.21.15.80:0/3263551845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940b0ed0 0x7f359415c860 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f35940b4950 tx=0x7f3590021340 comp rx=0 tx=0).stop 2024-04-25T13:51:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 -- 172.21.15.80:0/3263551845 shutdown_connections 2024-04-25T13:51:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 --2- 172.21.15.80:0/3263551845 >> v2:172.21.15.87:3300/0 conn(0x7f359406eba0 0x7f35940ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 --2- 172.21.15.80:0/3263551845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940b0ed0 0x7f359415c860 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 -- 172.21.15.80:0/3263551845 >> 172.21.15.80:0/3263551845 conn(0x7f359406d080 msgr2=0x7f35940b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 -- 172.21.15.80:0/3263551845 shutdown_connections 2024-04-25T13:51:34.690 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 -- 172.21.15.80:0/3263551845 wait complete. 2024-04-25T13:51:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.696+0000 7f359b92b640 1 Processor -- start 2024-04-25T13:51:34.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 -- start start 2024-04-25T13:51:34.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35940b0ed0 0x7f3594145210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940ce730 0x7f3594145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f359415f760 0x7f35941f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f35940733a0 con 0x7f35940ce730 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f35940b12e0 con 0x7f35940b0ed0 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f359406dc00 con 0x7f359415f760 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940ce730 0x7f3594145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940ce730 0x7f3594145ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51780/0 (socket says 172.21.15.80:51780) 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 -- 172.21.15.80:0/3930344971 learned_addr learned my addr 172.21.15.80:0/3930344971 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f35996c7640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35940b0ed0 0x7f3594145210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3599ec8640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f359415f760 0x7f35941f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 -- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f359415f760 msgr2=0x7f35941f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f359415f760 0x7f35941f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 -- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35940b0ed0 msgr2=0x7f3594145210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35940b0ed0 0x7f3594145210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 -- 172.21.15.80:0/3930344971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359414be30 con 0x7f35940ce730 2024-04-25T13:51:34.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3599ec8640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f359415f760 0x7f35941f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:34.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3598ec6640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940ce730 0x7f3594145ad0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f358400c4b0 tx=0x7f358400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:34.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3589ffb640 1 -- 172.21.15.80:0/3930344971 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584018070 con 0x7f35940ce730 2024-04-25T13:51:34.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3589ffb640 1 -- 172.21.15.80:0/3930344971 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3584007500 con 0x7f35940ce730 2024-04-25T13:51:34.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3589ffb640 1 -- 172.21.15.80:0/3930344971 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584004a30 con 0x7f35940ce730 2024-04-25T13:51:34.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35941fa440 con 0x7f35940ce730 2024-04-25T13:51:34.696 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359415ec80 con 0x7f35940ce730 2024-04-25T13:51:34.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359406a3a0 con 0x7f35940ce730 2024-04-25T13:51:34.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.700+0000 7f3589ffb640 1 -- 172.21.15.80:0/3930344971 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f358401b070 con 0x7f35940ce730 2024-04-25T13:51:34.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.704+0000 7f3589ffb640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357806f430 0x7f3578071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:34.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.704+0000 7f3589ffb640 1 -- 172.21.15.80:0/3930344971 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f358408e580 con 0x7f35940ce730 2024-04-25T13:51:34.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.704+0000 7f3589ffb640 1 -- 172.21.15.80:0/3930344971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f3578077010 con 0x7f35940ce730 2024-04-25T13:51:34.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.704+0000 7f35996c7640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357806f430 0x7f3578071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:34.697 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.704+0000 7f35996c7640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357806f430 0x7f3578071940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f35900b3000 tx=0x7f35900ac000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:34.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.708+0000 7f3589ffb640 1 -- 172.21.15.80:0/3930344971 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f358405b450 con 0x7f35940ce730 2024-04-25T13:51:34.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.888+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f359406f880 con 0x7f35940ce730 2024-04-25T13:51:34.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.888+0000 7f3589ffb640 1 -- 172.21.15.80:0/3930344971 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5176 (secure 0 0 0) 0x7f358405ad50 con 0x7f35940ce730 2024-04-25T13:51:34.883 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:34.883 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/29039900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":29039900},{"type":"v1","addr":"172.21.15.87:6836","nonce":29039900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1466426643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1466426643},{"type":"v1","addr":"172.21.15.80:6833","nonce":1466426643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1170719891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1170719891},{"type":"v1","addr":"172.21.15.80:6835","nonce":1170719891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:50:30.371502+0000","modified":"2024-04-25T13:50:33.461986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/986211877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":986211877},{"type":"v1","addr":"172.21.15.87:6837","nonce":986211877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":10}]} 2024-04-25T13:51:34.883 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 81 2024-04-25T13:51:34.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357806f430 msgr2=0x7f3578071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357806f430 0x7f3578071940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f35900b3000 tx=0x7f35900ac000 comp rx=0 tx=0).stop 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940ce730 msgr2=0x7f3594145ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940ce730 0x7f3594145ad0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f358400c4b0 tx=0x7f358400c970 comp rx=0 tx=0).stop 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 shutdown_connections 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f357806f430 0x7f3578071940 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f359415f760 0x7f35941f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f35940ce730 0x7f3594145ad0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 --2- 172.21.15.80:0/3930344971 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f35940b0ed0 0x7f3594145210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 >> 172.21.15.80:0/3930344971 conn(0x7f359406d080 msgr2=0x7f35941571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 shutdown_connections 2024-04-25T13:51:34.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:34.892+0000 7f359b92b640 1 -- 172.21.15.80:0/3930344971 wait complete. 2024-04-25T13:51:34.904 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:35.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.048+0000 7f4f03ad6640 1 Processor -- start 2024-04-25T13:51:35.042 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.048+0000 7f4f03ad6640 1 -- start start 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4efc0d06e0 0x7f4efc06e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4efc06e9e0 0x7f4efc06edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4efc0d0af0 con 0x7f4efc161f90 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4efc0d0d70 con 0x7f4efc06e9e0 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4efc0d1050 con 0x7f4efc0d06e0 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f02073640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4efc06e9e0 0x7f4efc06edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f02073640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4efc06e9e0 0x7f4efc06edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51782/0 (socket says 172.21.15.80:51782) 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f02073640 1 -- 172.21.15.80:0/3122933859 learned_addr learned my addr 172.21.15.80:0/3122933859 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f01872640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4efc161f90 0x7f4efc0cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41534/0 (socket says 172.21.15.80:41534) 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f01071640 1 --2- 172.21.15.80:0/3122933859 >> v2:172.21.15.87:3300/0 conn(0x7f4efc0d06e0 0x7f4efc06e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f02073640 1 -- 172.21.15.80:0/3122933859 >> v2:172.21.15.87:3300/0 conn(0x7f4efc0d06e0 msgr2=0x7f4efc06e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:35.045 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f02073640 1 --2- 172.21.15.80:0/3122933859 >> v2:172.21.15.87:3300/0 conn(0x7f4efc0d06e0 0x7f4efc06e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f02073640 1 -- 172.21.15.80:0/3122933859 >> v1:172.21.15.80:6789/0 conn(0x7f4efc161f90 legacy=0x7f4efc0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f02073640 1 -- 172.21.15.80:0/3122933859 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc06dca0 con 0x7f4efc06e9e0 2024-04-25T13:51:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f02073640 1 --2- 172.21.15.80:0/3122933859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc06edf0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f4efc074290 tx=0x7f4ef401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3c47b6b81fbd7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f00870640 1 -- 172.21.15.80:0/3122933859 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40b0070 con 0x7f4efc06e9e0 2024-04-25T13:51:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f00870640 1 -- 172.21.15.80:0/3122933859 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4ef40ab600 con 0x7f4efc06e9e0 2024-04-25T13:51:35.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f00870640 1 -- 172.21.15.80:0/3122933859 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef40ab9d0 con 0x7f4efc06e9e0 2024-04-25T13:51:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 -- 172.21.15.80:0/3122933859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 msgr2=0x7f4efc06edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/3122933859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc06edf0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f4efc074290 tx=0x7f4ef401c3d0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 -- 172.21.15.80:0/3122933859 shutdown_connections 2024-04-25T13:51:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/3122933859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc06edf0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.052+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/3122933859 >> v2:172.21.15.87:3300/0 conn(0x7f4efc0d06e0 0x7f4efc06e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 -- 172.21.15.80:0/3122933859 >> 172.21.15.80:0/3122933859 conn(0x7f4efc06d080 msgr2=0x7f4efc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 -- 172.21.15.80:0/3122933859 shutdown_connections 2024-04-25T13:51:35.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 -- 172.21.15.80:0/3122933859 wait complete. 2024-04-25T13:51:35.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 Processor -- start 2024-04-25T13:51:35.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 -- start start 2024-04-25T13:51:35.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4efc0d06e0 0x7f4efc1fa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4efc161f90 0x7f4efc140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc1447a0 con 0x7f4efc06e9e0 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4efc144620 con 0x7f4efc0d06e0 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f03ad6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4efc16b510 con 0x7f4efc161f90 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51786/0 (socket says 172.21.15.80:51786) 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 -- 172.21.15.80:0/884365826 learned_addr learned my addr 172.21.15.80:0/884365826 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01071640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4efc0d06e0 0x7f4efc1fa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f02073640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4efc161f90 0x7f4efc140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 -- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4efc161f90 msgr2=0x7f4efc140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4efc161f90 0x7f4efc140590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 -- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4efc0d06e0 msgr2=0x7f4efc1fa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4efc0d06e0 0x7f4efc1fa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 -- 172.21.15.80:0/884365826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4efc146920 con 0x7f4efc06e9e0 2024-04-25T13:51:35.053 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.056+0000 7f4f01872640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc141480 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00c760 tx=0x7f4eec00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4efa7fc640 1 -- 172.21.15.80:0/884365826 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec017070 con 0x7f4efc06e9e0 2024-04-25T13:51:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4efa7fc640 1 -- 172.21.15.80:0/884365826 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eec00ce10 con 0x7f4efc06e9e0 2024-04-25T13:51:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4efc146640 con 0x7f4efc06e9e0 2024-04-25T13:51:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4efa7fc640 1 -- 172.21.15.80:0/884365826 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eec00a840 con 0x7f4efc06e9e0 2024-04-25T13:51:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4efc163330 con 0x7f4efc06e9e0 2024-04-25T13:51:35.054 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4efc06a3a0 con 0x7f4efc06e9e0 2024-04-25T13:51:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4efa7fc640 1 -- 172.21.15.80:0/884365826 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4eec01a070 con 0x7f4efc06e9e0 2024-04-25T13:51:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4efa7fc640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ee806f400 0x7f4ee8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4efa7fc640 1 -- 172.21.15.80:0/884365826 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f4eec08e6e0 con 0x7f4efc06e9e0 2024-04-25T13:51:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4efa7fc640 1 -- 172.21.15.80:0/884365826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f4ee8077040 con 0x7f4efc06e9e0 2024-04-25T13:51:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4f01071640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ee806f400 0x7f4ee8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.060+0000 7f4f01071640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ee806f400 0x7f4ee8071910 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4efc074640 tx=0x7f4ef0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.064+0000 7f4efa7fc640 1 -- 172.21.15.80:0/884365826 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4efc0d0af0 con 0x7f4efc06e9e0 2024-04-25T13:51:35.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.228+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4efc0b2f00 con 0x7f4efc06e9e0 2024-04-25T13:51:35.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.228+0000 7f4efa7fc640 1 -- 172.21.15.80:0/884365826 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) ==== 74+0+16683 (secure 0 0 0) 0x7f4eec05aff0 con 0x7f4efc06e9e0 2024-04-25T13:51:35.224 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:35.224 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":175,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:21.279640+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:50:28.108369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T13:50:29.324161+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"175","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ee806f400 msgr2=0x7f4ee8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ee806f400 0x7f4ee8071910 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4efc074640 tx=0x7f4ef0008040 comp rx=0 tx=0).stop 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 msgr2=0x7f4efc141480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc141480 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f4eec00c760 tx=0x7f4eec00cc20 comp rx=0 tx=0).stop 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 shutdown_connections 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ee806f400 0x7f4ee8071910 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4efc161f90 0x7f4efc140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4efc0d06e0 0x7f4efc1fa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 --2- 172.21.15.80:0/884365826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4efc06e9e0 0x7f4efc141480 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 >> 172.21.15.80:0/884365826 conn(0x7f4efc06d080 msgr2=0x7f4efc0b27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 shutdown_connections 2024-04-25T13:51:35.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.232+0000 7f4f03ad6640 1 -- 172.21.15.80:0/884365826 wait complete. 2024-04-25T13:51:35.246 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:50:28.108369+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:50:29.324161+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '175', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:51:35.247 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:51:35.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52d0b03640 1 Processor -- start 2024-04-25T13:51:35.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52d0b03640 1 -- start start 2024-04-25T13:51:35.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52d0b03640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f52cc0d2ca0 0x7f52cc0b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52d0b03640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52cc0b92a0 con 0x7f52cc0b9ca0 2024-04-25T13:51:35.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52d0b03640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52cc0d4c10 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52d0b03640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f52cc0d4ef0 con 0x7f52cc0d2ca0 2024-04-25T13:51:35.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52c9d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f52cc0d2ca0 0x7f52cc0b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52c9d9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f52cc0d2ca0 0x7f52cc0b9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51798/0 (socket says 172.21.15.80:51798) 2024-04-25T13:51:35.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52c9d9b640 1 -- 172.21.15.80:0/2527196325 learned_addr learned my addr 172.21.15.80:0/2527196325 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:35.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52cad9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f52cc0b9ca0 0x7f52cc0ba0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41538/0 (socket says 172.21.15.80:41538) 2024-04-25T13:51:35.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52c9d9b640 1 -- 172.21.15.80:0/2527196325 >> v1:172.21.15.87:6789/0 conn(0x7f52cc0d3ed0 legacy=0x7f52cc0d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52c9d9b640 1 -- 172.21.15.80:0/2527196325 >> v1:172.21.15.80:6789/0 conn(0x7f52cc0b9ca0 legacy=0x7f52cc0ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52c9d9b640 1 -- 172.21.15.80:0/2527196325 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc0d51d0 con 0x7f52cc0d2ca0 2024-04-25T13:51:35.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.404+0000 7f52c959a640 1 -- 172.21.15.80:0/2527196325 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3477390830 0 0) 0x7f52c40014a0 con 0x7f52cc0b9ca0 2024-04-25T13:51:35.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52c9d9b640 1 --2- 172.21.15.80:0/2527196325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d2ca0 0x7f52cc0b9760 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009870 tx=0x7f52c001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f386e040e9c1e1b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52c959a640 1 -- 172.21.15.80:0/2527196325 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00aa650 con 0x7f52cc0d2ca0 2024-04-25T13:51:35.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52c959a640 1 -- 172.21.15.80:0/2527196325 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c00af070 con 0x7f52cc0d2ca0 2024-04-25T13:51:35.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52c959a640 1 -- 172.21.15.80:0/2527196325 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c00aaa10 con 0x7f52cc0d2ca0 2024-04-25T13:51:35.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52d0b03640 1 -- 172.21.15.80:0/2527196325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d2ca0 msgr2=0x7f52cc0b9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52d0b03640 1 --2- 172.21.15.80:0/2527196325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d2ca0 0x7f52cc0b9760 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f52c0009870 tx=0x7f52c001c4d0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52d0b03640 1 -- 172.21.15.80:0/2527196325 shutdown_connections 2024-04-25T13:51:35.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52d0b03640 1 --2- 172.21.15.80:0/2527196325 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d2ca0 0x7f52cc0b9760 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52d0b03640 1 -- 172.21.15.80:0/2527196325 >> 172.21.15.80:0/2527196325 conn(0x7f52cc0737b0 msgr2=0x7f52cc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:35.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52d0b03640 1 -- 172.21.15.80:0/2527196325 shutdown_connections 2024-04-25T13:51:35.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.408+0000 7f52d0b03640 1 -- 172.21.15.80:0/2527196325 wait complete. 2024-04-25T13:51:35.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 Processor -- start 2024-04-25T13:51:35.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 -- start start 2024-04-25T13:51:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52cc0b9ca0 0x7f52cc152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52cc0d2ca0 0x7f52cc1fda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d3ed0 0x7f52cc14bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc16e6b0 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc160830 con 0x7f52cc0b9ca0 2024-04-25T13:51:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc16e830 con 0x7f52cc0d2ca0 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d3ed0 0x7f52cc14bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d3ed0 0x7f52cc14bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51802/0 (socket says 172.21.15.80:51802) 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 -- 172.21.15.80:0/2292758934 learned_addr learned my addr 172.21.15.80:0/2292758934 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52ca59c640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52cc0b9ca0 0x7f52cc152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52c9d9b640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52cc0d2ca0 0x7f52cc1fda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 -- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52cc0d2ca0 msgr2=0x7f52cc1fda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52cc0d2ca0 0x7f52cc1fda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 -- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52cc0b9ca0 msgr2=0x7f52cc152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52cc0b9ca0 0x7f52cc152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 -- 172.21.15.80:0/2292758934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc1536c0 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52cad9d640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d3ed0 0x7f52cc14bc40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c700 tx=0x7f52c400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52baffd640 1 -- 172.21.15.80:0/2292758934 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4018070 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52baffd640 1 -- 172.21.15.80:0/2292758934 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52c400ce10 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc1539a0 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52baffd640 1 -- 172.21.15.80:0/2292758934 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c400a880 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.412+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc150d00 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.416+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc06a3a0 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.416+0000 7f52baffd640 1 -- 172.21.15.80:0/2292758934 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f52c401b070 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.416+0000 7f52baffd640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f52ac06f400 0x7f52ac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.416+0000 7f52baffd640 1 -- 172.21.15.80:0/2292758934 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f52c408e810 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.416+0000 7f52baffd640 1 -- 172.21.15.80:0/2292758934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f52ac077040 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.416+0000 7f52ca59c640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f52ac06f400 0x7f52ac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.416+0000 7f52ca59c640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f52ac06f400 0x7f52ac071910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000f70 tx=0x7f52bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.420+0000 7f52baffd640 1 -- 172.21.15.80:0/2292758934 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f52cc0b92a0 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.580+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f52cc0ba150 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.575 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.580+0000 7f52baffd640 1 -- 172.21.15.80:0/2292758934 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v175) ==== 93+0+10 (secure 0 0 0) 0x7f52c405b770 con 0x7f52cc0d3ed0 2024-04-25T13:51:35.576 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:51:35.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f52ac06f400 msgr2=0x7f52ac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f52ac06f400 0x7f52ac071910 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000f70 tx=0x7f52bc009040 comp rx=0 tx=0).stop 2024-04-25T13:51:35.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d3ed0 msgr2=0x7f52cc14bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d3ed0 0x7f52cc14bc40 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f52c400c700 tx=0x7f52c400cbc0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 shutdown_connections 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f52ac06f400 0x7f52ac071910 secure :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f52bc000f70 tx=0x7f52bc009040 comp rx=0 tx=0).stop 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f52cc0d3ed0 0x7f52cc14bc40 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f52cc0d2ca0 0x7f52cc1fda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 --2- 172.21.15.80:0/2292758934 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f52cc0b9ca0 0x7f52cc152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 >> 172.21.15.80:0/2292758934 conn(0x7f52cc0737b0 msgr2=0x7f52cc0c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 shutdown_connections 2024-04-25T13:51:35.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.584+0000 7f52d0b03640 1 -- 172.21.15.80:0/2292758934 wait complete. 2024-04-25T13:51:35.598 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:51:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293d20f640 1 Processor -- start 2024-04-25T13:51:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293d20f640 1 -- start start 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293d20f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f29380d3ed0 0x7f29380d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293d20f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29380b92a0 con 0x7f29380ba770 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293d20f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29380b84f0 con 0x7f29380d2ad0 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293d20f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f29380d4c10 con 0x7f29380d3ed0 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293759e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f29380ba770 0x7f29380b7de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41544/0 (socket says 172.21.15.80:41544) 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293759e640 1 -- 172.21.15.80:0/1196081939 learned_addr learned my addr 172.21.15.80:0/1196081939 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2936d9d640 1 --2- 172.21.15.80:0/1196081939 >> v2:172.21.15.87:3300/0 conn(0x7f29380d3ed0 0x7f29380d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3088261997 0 0) 0x7f2930001120 con 0x7f29380ba770 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2920003530 con 0x7f29380ba770 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1638133496 0 0) 0x7f2920003530 con 0x7f29380ba770 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 >> v2:172.21.15.87:3300/0 conn(0x7f29380d3ed0 msgr2=0x7f29380d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 --2- 172.21.15.80:0/1196081939 >> v2:172.21.15.87:3300/0 conn(0x7f29380d3ed0 0x7f29380d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.756 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 >> v1:172.21.15.87:6789/0 conn(0x7f29380d2ad0 legacy=0x7f29380b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29380d4ef0 con 0x7f29380ba770 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2930001850 con 0x7f29380ba770 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 >> v1:172.21.15.80:6789/0 conn(0x7f29380ba770 legacy=0x7f29380b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29200039c0 0x7f2920003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920004810 0x7f2920004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2920005360 0x7f292000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f29380d4c10 con 0x7f2920005360 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f292000c290 con 0x7f29200039c0 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f292000c490 con 0x7f2920004810 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293759e640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2920005360 0x7f292000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f2936d9d640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29200039c0 0x7f2920003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293659c640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920004810 0x7f2920004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293759e640 1 -- 172.21.15.80:0/1196081939 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920004810 msgr2=0x7f2920004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293759e640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920004810 0x7f2920004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293759e640 1 -- 172.21.15.80:0/1196081939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29200039c0 msgr2=0x7f2920003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293759e640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29200039c0 0x7f2920003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.760+0000 7f293759e640 1 -- 172.21.15.80:0/1196081939 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2920003530 con 0x7f2920005360 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293759e640 1 -- 172.21.15.80:0/1196081939 reap_dead start 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293759e640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2920005360 0x7f292000b9d0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f293000c4c0 tx=0x7f293001cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fd9d0efaae197b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f293000c0e0 con 0x7f2920005360 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f29300a9070 con 0x7f2920005360 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f2935d9b640 1 -- 172.21.15.80:0/1196081939 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29300a5d90 con 0x7f2920005360 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293d20f640 1 -- 172.21.15.80:0/1196081939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2920005360 msgr2=0x7f292000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293d20f640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2920005360 0x7f292000b9d0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f293000c4c0 tx=0x7f293001cc50 comp rx=0 tx=0).stop 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293d20f640 1 -- 172.21.15.80:0/1196081939 shutdown_connections 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293d20f640 1 --2- 172.21.15.80:0/1196081939 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2920005360 0x7f292000b9d0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293d20f640 1 -- 172.21.15.80:0/1196081939 >> 172.21.15.80:0/1196081939 conn(0x7f29380737b0 msgr2=0x7f293806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293d20f640 1 -- 172.21.15.80:0/1196081939 shutdown_connections 2024-04-25T13:51:35.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.764+0000 7f293d20f640 1 -- 172.21.15.80:0/1196081939 wait complete. 2024-04-25T13:51:35.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 Processor -- start 2024-04-25T13:51:35.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 -- start start 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920005360 0x7f29381568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2938151180 0x7f2938160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29381614e0 0x7f2938154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f29380ccae0 con 0x7f2938151180 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f29380cc960 con 0x7f29381614e0 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2938160620 con 0x7f2920005360 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2938151180 0x7f2938160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2938151180 0x7f2938160fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51814/0 (socket says 172.21.15.80:51814) 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 -- 172.21.15.80:0/3782403733 learned_addr learned my addr 172.21.15.80:0/3782403733 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293759e640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29381614e0 0x7f2938154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f2936d9d640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920005360 0x7f29381568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 -- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920005360 msgr2=0x7f29381568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920005360 0x7f29381568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 -- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29381614e0 msgr2=0x7f2938154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29381614e0 0x7f2938154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 -- 172.21.15.80:0/3782403733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29381618f0 con 0x7f2938151180 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293659c640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2938151180 0x7f2938160fa0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f292c00d570 tx=0x7f292c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f291f7fe640 1 -- 172.21.15.80:0/3782403733 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c005480 con 0x7f2938151180 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f291f7fe640 1 -- 172.21.15.80:0/3782403733 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f292c017070 con 0x7f2938151180 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2938157250 con 0x7f2938151180 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f291f7fe640 1 -- 172.21.15.80:0/3782403733 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292c003820 con 0x7f2938151180 2024-04-25T13:51:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f293815c1e0 con 0x7f2938151180 2024-04-25T13:51:35.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.768+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f293806a3a0 con 0x7f2938151180 2024-04-25T13:51:35.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.772+0000 7f291f7fe640 1 -- 172.21.15.80:0/3782403733 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f292c0039c0 con 0x7f2938151180 2024-04-25T13:51:35.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.772+0000 7f291f7fe640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f292406f400 0x7f2924071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:35.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.772+0000 7f291f7fe640 1 -- 172.21.15.80:0/3782403733 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f292c08f020 con 0x7f2938151180 2024-04-25T13:51:35.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.772+0000 7f291f7fe640 1 -- 172.21.15.80:0/3782403733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f2924076fe0 con 0x7f2938151180 2024-04-25T13:51:35.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.772+0000 7f2936d9d640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f292406f400 0x7f2924071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:35.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.772+0000 7f2936d9d640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f292406f400 0x7f2924071910 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2928009fd0 tx=0x7f292800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:35.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.776+0000 7f291f7fe640 1 -- 172.21.15.80:0/3782403733 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29381618f0 con 0x7f2938151180 2024-04-25T13:51:35.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2938156fa0 con 0x7f2938151180 2024-04-25T13:51:35.917 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f291f7fe640 1 -- 172.21.15.80:0/3782403733 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v175) ==== 104+0+11 (secure 0 0 0) 0x7f292c05bf80 con 0x7f2938151180 2024-04-25T13:51:35.917 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f292406f400 msgr2=0x7f2924071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f292406f400 0x7f2924071910 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f2928009fd0 tx=0x7f292800b040 comp rx=0 tx=0).stop 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2938151180 msgr2=0x7f2938160fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2938151180 0x7f2938160fa0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f292c00d570 tx=0x7f292c00da30 comp rx=0 tx=0).stop 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 shutdown_connections 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f292406f400 0x7f2924071910 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29381614e0 0x7f2938154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2938151180 0x7f2938160fa0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 --2- 172.21.15.80:0/3782403733 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2920005360 0x7f29381568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 >> 172.21.15.80:0/3782403733 conn(0x7f29380737b0 msgr2=0x7f29380c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 shutdown_connections 2024-04-25T13:51:35.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:35.924+0000 7f293d20f640 1 -- 172.21.15.80:0/3782403733 wait complete. 2024-04-25T13:51:35.937 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:51:36.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.080+0000 7f2b228f5640 1 Processor -- start 2024-04-25T13:51:36.076 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.080+0000 7f2b228f5640 1 -- start start 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b228f5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b1c0b9e70 0x7f2b1c0ba280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b228f5640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b1c0b92a0 con 0x7f2b1c0d3ed0 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b228f5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b1c0d4c10 con 0x7f2b1c0d2ca0 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b228f5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2b1c0d4ef0 con 0x7f2b1c0b9e70 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2b1c0d3ed0 0x7f2b1c0d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41552/0 (socket says 172.21.15.80:41552) 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1bfff640 1 -- 172.21.15.80:0/378429960 learned_addr learned my addr 172.21.15.80:0/378429960 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b20e92640 1 --2- 172.21.15.80:0/378429960 >> v2:172.21.15.87:3300/0 conn(0x7f2b1c0b9e70 0x7f2b1c0ba280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4294343578 0 0) 0x7f2b0c001120 con 0x7f2b1c0d3ed0 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b04003530 con 0x7f2b1c0d3ed0 2024-04-25T13:51:36.079 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 455202848 0 0) 0x7f2b04003530 con 0x7f2b1c0d3ed0 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 >> v2:172.21.15.87:3300/0 conn(0x7f2b1c0b9e70 msgr2=0x7f2b1c0ba280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 --2- 172.21.15.80:0/378429960 >> v2:172.21.15.87:3300/0 conn(0x7f2b1c0b9e70 0x7f2b1c0ba280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 >> v1:172.21.15.87:6789/0 conn(0x7f2b1c0d2ca0 legacy=0x7f2b1c0b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c0d51d0 con 0x7f2b1c0d3ed0 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2b0c001850 con 0x7f2b1c0d3ed0 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 >> v1:172.21.15.80:6789/0 conn(0x7f2b1c0d3ed0 legacy=0x7f2b1c0d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b040039c0 0x7f2b04003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b04004810 0x7f2b04004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b04005360 0x7f2b0400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c0d4ef0 con 0x7f2b040039c0 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0400c290 con 0x7f2b04005360 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0400c490 con 0x7f2b04004810 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1bfff640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b040039c0 0x7f2b04003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1b7fe640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b04004810 0x7f2b04004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.080 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1bfff640 1 -- 172.21.15.80:0/378429960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b04004810 msgr2=0x7f2b04004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:36.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1bfff640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b04004810 0x7f2b04004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1bfff640 1 -- 172.21.15.80:0/378429960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b04005360 msgr2=0x7f2b0400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:36.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1bfff640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b04005360 0x7f2b0400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b1bfff640 1 -- 172.21.15.80:0/378429960 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b04003530 con 0x7f2b040039c0 2024-04-25T13:51:36.081 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.084+0000 7f2b20e92640 1 -- 172.21.15.80:0/378429960 reap_dead start 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b1bfff640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b040039c0 0x7f2b04003e30 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c4c0 tx=0x7f2b0c01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2629cba46b825b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00c0e0 con 0x7f2b040039c0 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b0c0a9070 con 0x7f2b040039c0 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b1affd640 1 -- 172.21.15.80:0/378429960 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c005d80 con 0x7f2b040039c0 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 -- 172.21.15.80:0/378429960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b040039c0 msgr2=0x7f2b04003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b040039c0 0x7f2b04003e30 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00c4c0 tx=0x7f2b0c01cc50 comp rx=0 tx=0).stop 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 -- 172.21.15.80:0/378429960 shutdown_connections 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 --2- 172.21.15.80:0/378429960 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b040039c0 0x7f2b04003e30 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 -- 172.21.15.80:0/378429960 >> 172.21.15.80:0/378429960 conn(0x7f2b1c0737b0 msgr2=0x7f2b1c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:36.082 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 -- 172.21.15.80:0/378429960 shutdown_connections 2024-04-25T13:51:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 -- 172.21.15.80:0/378429960 wait complete. 2024-04-25T13:51:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 Processor -- start 2024-04-25T13:51:36.083 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.088+0000 7f2b228f5640 1 -- start start 2024-04-25T13:51:36.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b040039c0 0x7f2b1c1f9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b1c155000 0x7f2b1c1544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b1c163cd0 0x7f2b1c1640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c0cc960 con 0x7f2b1c155000 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b1c0baa90 con 0x7f2b040039c0 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b1c0ccae0 con 0x7f2b1c163cd0 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b1c155000 0x7f2b1c1544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b1c155000 0x7f2b1c1544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51828/0 (socket says 172.21.15.80:51828) 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 -- 172.21.15.80:0/1599651959 learned_addr learned my addr 172.21.15.80:0/1599651959 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1bfff640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b040039c0 0x7f2b1c1f9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b20e92640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b1c163cd0 0x7f2b1c1640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 -- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b1c163cd0 msgr2=0x7f2b1c1640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b1c163cd0 0x7f2b1c1640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 -- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b040039c0 msgr2=0x7f2b1c1f9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b040039c0 0x7f2b1c1f9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 -- 172.21.15.80:0/1599651959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b1c164620 con 0x7f2b1c155000 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b20e92640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b1c163cd0 0x7f2b1c1640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b1b7fe640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b1c155000 0x7f2b1c1544c0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000d4a0 tx=0x7f2b1000d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b18ff9640 1 -- 172.21.15.80:0/1599651959 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10005480 con 0x7f2b1c155000 2024-04-25T13:51:36.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b18ff9640 1 -- 172.21.15.80:0/1599651959 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b10017070 con 0x7f2b1c155000 2024-04-25T13:51:36.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1c1607c0 con 0x7f2b1c155000 2024-04-25T13:51:36.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b18ff9640 1 -- 172.21.15.80:0/1599651959 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b10003820 con 0x7f2b1c155000 2024-04-25T13:51:36.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b1c160d80 con 0x7f2b1c155000 2024-04-25T13:51:36.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b1c06a3a0 con 0x7f2b1c155000 2024-04-25T13:51:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.092+0000 7f2b18ff9640 1 -- 172.21.15.80:0/1599651959 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2b100039c0 con 0x7f2b1c155000 2024-04-25T13:51:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.096+0000 7f2b18ff9640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.096+0000 7f2b18ff9640 1 -- 172.21.15.80:0/1599651959 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(175..175 src has 1..175) ==== 5738+0+0 (secure 0 0 0) 0x7f2b1008eff0 con 0x7f2b1c155000 2024-04-25T13:51:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.096+0000 7f2b18ff9640 1 -- 172.21.15.80:0/1599651959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=176}) -- 0x7f2b08077040 con 0x7f2b1c155000 2024-04-25T13:51:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.096+0000 7f2b1bfff640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.089 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.096+0000 7f2b1bfff640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0ab000 tx=0x7f2b0c0b1040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.100+0000 7f2b18ff9640 1 -- 172.21.15.80:0/1599651959 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b1c164620 con 0x7f2b1c155000 2024-04-25T13:51:36.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.252+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2b1c154c30 con 0x7f2b1c155000 2024-04-25T13:51:36.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.252+0000 7f2b18ff9640 1 -- 172.21.15.80:0/1599651959 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v175) ==== 100+0+11 (secure 0 0 0) 0x7f2b1005bec0 con 0x7f2b1c155000 2024-04-25T13:51:36.248 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:51:36.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 msgr2=0x7f2b08071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c0ab000 tx=0x7f2b0c0b1040 comp rx=0 tx=0).stop 2024-04-25T13:51:36.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b1c155000 msgr2=0x7f2b1c1544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b1c155000 0x7f2b1c1544c0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000d4a0 tx=0x7f2b1000d960 comp rx=0 tx=0).stop 2024-04-25T13:51:36.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 shutdown_connections 2024-04-25T13:51:36.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b0806f400 0x7f2b08071910 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b1c163cd0 0x7f2b1c1640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b1c155000 0x7f2b1c1544c0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 --2- 172.21.15.80:0/1599651959 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b040039c0 0x7f2b1c1f9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 >> 172.21.15.80:0/1599651959 conn(0x7f2b1c0737b0 msgr2=0x7f2b1c0cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:36.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 shutdown_connections 2024-04-25T13:51:36.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.256+0000 7f2b228f5640 1 -- 172.21.15.80:0/1599651959 wait complete. 2024-04-25T13:51:36.269 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:36.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2eaa2fe640 1 Processor -- start 2024-04-25T13:51:36.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2eaa2fe640 1 -- start start 2024-04-25T13:51:36.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2eaa2fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea4161f90 0x7f2ea40cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2eaa2fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ea40d0510 0x7f2ea415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2eaa2fe640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2ea40d0d70 con 0x7f2ea415ea70 2024-04-25T13:51:36.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2eaa2fe640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2ea40d1050 con 0x7f2ea40d0510 2024-04-25T13:51:36.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2eaa2fe640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea406dca0 con 0x7f2ea4161f90 2024-04-25T13:51:36.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2ea37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ea40d0510 0x7f2ea415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2ea37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2ea40d0510 0x7f2ea415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51836/0 (socket says 172.21.15.80:51836) 2024-04-25T13:51:36.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2ea37fe640 1 -- 172.21.15.80:0/175605732 learned_addr learned my addr 172.21.15.80:0/175605732 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:36.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2ea3fff640 1 --2- 172.21.15.80:0/175605732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea4161f90 0x7f2ea40cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2ea37fe640 1 -- 172.21.15.80:0/175605732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea4161f90 msgr2=0x7f2ea40cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:36.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2ea37fe640 1 --2- 172.21.15.80:0/175605732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea4161f90 0x7f2ea40cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2ea37fe640 1 -- 172.21.15.80:0/175605732 >> v1:172.21.15.87:6789/0 conn(0x7f2ea415ea70 legacy=0x7f2ea415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:36.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.412+0000 7f2ea37fe640 1 -- 172.21.15.80:0/175605732 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea406df80 con 0x7f2ea40d0510 2024-04-25T13:51:36.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2ea37fe640 1 --2- 172.21.15.80:0/175605732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea40d0510 0x7f2ea415e530 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009870 tx=0x7f2e9801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf7a0bc56d7bb03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2ea2ffd640 1 -- 172.21.15.80:0/175605732 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e980b0070 con 0x7f2ea40d0510 2024-04-25T13:51:36.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2ea2ffd640 1 -- 172.21.15.80:0/175605732 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e980ab600 con 0x7f2ea40d0510 2024-04-25T13:51:36.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2ea2ffd640 1 -- 172.21.15.80:0/175605732 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e980ab9d0 con 0x7f2ea40d0510 2024-04-25T13:51:36.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/175605732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea40d0510 msgr2=0x7f2ea415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/175605732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea40d0510 0x7f2ea415e530 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009870 tx=0x7f2e9801c430 comp rx=0 tx=0).stop 2024-04-25T13:51:36.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/175605732 shutdown_connections 2024-04-25T13:51:36.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/175605732 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea40d0510 0x7f2ea415e530 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/175605732 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea4161f90 0x7f2ea40cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/175605732 >> 172.21.15.80:0/175605732 conn(0x7f2ea406d080 msgr2=0x7f2ea40b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:36.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/175605732 shutdown_connections 2024-04-25T13:51:36.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/175605732 wait complete. 2024-04-25T13:51:36.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 Processor -- start 2024-04-25T13:51:36.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.416+0000 7f2eaa2fe640 1 -- start start 2024-04-25T13:51:36.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea40d0510 0x7f2ea4140570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ea415ea70 0x7f2ea4146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea4161f90 0x7f2ea41600e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea406f630 con 0x7f2ea4161f90 2024-04-25T13:51:36.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea406f4b0 con 0x7f2ea415ea70 2024-04-25T13:51:36.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea40d0920 con 0x7f2ea40d0510 2024-04-25T13:51:36.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea4161f90 0x7f2ea41600e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea4161f90 0x7f2ea41600e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51848/0 (socket says 172.21.15.80:51848) 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 -- 172.21.15.80:0/383571773 learned_addr learned my addr 172.21.15.80:0/383571773 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea37fe640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ea415ea70 0x7f2ea4146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea3fff640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea40d0510 0x7f2ea4140570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 -- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea40d0510 msgr2=0x7f2ea4140570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea40d0510 0x7f2ea4140570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 -- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ea415ea70 msgr2=0x7f2ea4146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ea415ea70 0x7f2ea4146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 -- 172.21.15.80:0/383571773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea4072960 con 0x7f2ea4161f90 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea889b640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea4161f90 0x7f2ea41600e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00b9a0 tx=0x7f2e9c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea0ff9640 1 -- 172.21.15.80:0/383571773 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c019070 con 0x7f2ea4161f90 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea0ff9640 1 -- 172.21.15.80:0/383571773 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2e9c00cba0 con 0x7f2ea4161f90 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea4073350 con 0x7f2ea4161f90 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea0ff9640 1 -- 172.21.15.80:0/383571773 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e9c014440 con 0x7f2ea4161f90 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea4071e80 con 0x7f2ea4161f90 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea406a3a0 con 0x7f2ea4161f90 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.420+0000 7f2ea0ff9640 1 -- 172.21.15.80:0/383571773 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2e9c01c070 con 0x7f2ea4161f90 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.424+0000 7f2ea0ff9640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e9006f400 0x7f2e90071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.424+0000 7f2ea0ff9640 1 -- 172.21.15.80:0/383571773 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 5738+0+0 (secure 0 0 0) 0x7f2e9c08ef90 con 0x7f2ea4161f90 2024-04-25T13:51:36.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.424+0000 7f2ea3fff640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e9006f400 0x7f2e90071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.424+0000 7f2ea3fff640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e9006f400 0x7f2e90071910 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2ea406fa20 tx=0x7f2e94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.428+0000 7f2ea0ff9640 1 -- 172.21.15.80:0/383571773 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2e9c05bfd0 con 0x7f2ea4161f90 2024-04-25T13:51:36.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.608+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ea415fbc0 con 0x7f2ea4161f90 2024-04-25T13:51:36.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.608+0000 7f2ea0ff9640 1 -- 172.21.15.80:0/383571773 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5176 (secure 0 0 0) 0x7f2e9c05b8d0 con 0x7f2ea4161f90 2024-04-25T13:51:36.603 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:36.604 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/29039900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":29039900},{"type":"v1","addr":"172.21.15.87:6836","nonce":29039900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1466426643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1466426643},{"type":"v1","addr":"172.21.15.80:6833","nonce":1466426643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1170719891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1170719891},{"type":"v1","addr":"172.21.15.80:6835","nonce":1170719891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:50:30.371502+0000","modified":"2024-04-25T13:50:33.461986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/986211877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":986211877},{"type":"v1","addr":"172.21.15.87:6837","nonce":986211877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":10}]} 2024-04-25T13:51:36.604 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 81 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e9006f400 msgr2=0x7f2e90071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e9006f400 0x7f2e90071910 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f2ea406fa20 tx=0x7f2e94008040 comp rx=0 tx=0).stop 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea4161f90 msgr2=0x7f2ea41600e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea4161f90 0x7f2ea41600e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f2e9c00b9a0 tx=0x7f2e9c00be60 comp rx=0 tx=0).stop 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 shutdown_connections 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2e9006f400 0x7f2e90071910 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2ea4161f90 0x7f2ea41600e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ea415ea70 0x7f2ea4146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 --2- 172.21.15.80:0/383571773 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2ea40d0510 0x7f2ea4140570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 >> 172.21.15.80:0/383571773 conn(0x7f2ea406d080 msgr2=0x7f2ea41412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:36.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 shutdown_connections 2024-04-25T13:51:36.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.612+0000 7f2eaa2fe640 1 -- 172.21.15.80:0/383571773 wait complete. 2024-04-25T13:51:36.624 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:36.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61ff43640 1 Processor -- start 2024-04-25T13:51:36.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61ff43640 1 -- start start 2024-04-25T13:51:36.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61ff43640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61ff43640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6180b5f20 0x7ff618160e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61ff43640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff6181646e0 con 0x7ff6181642d0 2024-04-25T13:51:36.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61ff43640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff61815acb0 con 0x7ff6180b5f20 2024-04-25T13:51:36.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61ff43640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff61806ea30 con 0x7ff6180731e0 2024-04-25T13:51:36.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61dcdf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61e4e0640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff6180b5f20 0x7ff618160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61dcdf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:45006/0 (socket says 172.21.15.80:45006) 2024-04-25T13:51:36.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61dcdf640 1 -- 172.21.15.80:0/1300704607 learned_addr learned my addr 172.21.15.80:0/1300704607 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:36.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61e4e0640 1 -- 172.21.15.80:0/1300704607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 msgr2=0x7ff618163d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:36.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61e4e0640 1 --2- 172.21.15.80:0/1300704607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618163d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61e4e0640 1 -- 172.21.15.80:0/1300704607 >> v1:172.21.15.87:6789/0 conn(0x7ff6181642d0 legacy=0x7ff618160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.764+0000 7ff61e4e0640 1 -- 172.21.15.80:0/1300704607 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff61806ed10 con 0x7ff6180b5f20 2024-04-25T13:51:36.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61e4e0640 1 --2- 172.21.15.80:0/1300704607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6180b5f20 0x7ff618160e30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff608009d90 tx=0x7ff60801c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1106d050e891fd88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ccdd640 1 -- 172.21.15.80:0/1300704607 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6080b2070 con 0x7ff6180b5f20 2024-04-25T13:51:36.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ccdd640 1 -- 172.21.15.80:0/1300704607 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff6080a5940 con 0x7ff6180b5f20 2024-04-25T13:51:36.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ccdd640 1 -- 172.21.15.80:0/1300704607 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6080a5d10 con 0x7ff6180b5f20 2024-04-25T13:51:36.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ff43640 1 -- 172.21.15.80:0/1300704607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6180b5f20 msgr2=0x7ff618160e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ff43640 1 --2- 172.21.15.80:0/1300704607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6180b5f20 0x7ff618160e30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff608009d90 tx=0x7ff60801c970 comp rx=0 tx=0).stop 2024-04-25T13:51:36.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ff43640 1 -- 172.21.15.80:0/1300704607 shutdown_connections 2024-04-25T13:51:36.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ff43640 1 --2- 172.21.15.80:0/1300704607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6180b5f20 0x7ff618160e30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ff43640 1 --2- 172.21.15.80:0/1300704607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618163d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ff43640 1 -- 172.21.15.80:0/1300704607 >> 172.21.15.80:0/1300704607 conn(0x7ff61806d080 msgr2=0x7ff6180b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:36.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ff43640 1 -- 172.21.15.80:0/1300704607 shutdown_connections 2024-04-25T13:51:36.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.768+0000 7ff61ff43640 1 -- 172.21.15.80:0/1300704607 wait complete. 2024-04-25T13:51:36.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 Processor -- start 2024-04-25T13:51:36.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 -- start start 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6180b5f20 0x7ff618143410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6181642d0 0x7ff6181618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff61806e7d0 con 0x7ff6180b5f20 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff61806e650 con 0x7ff6181642d0 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff61816d940 con 0x7ff6180731e0 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6180b5f20 0x7ff618143410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6180b5f20 0x7ff618143410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51864/0 (socket says 172.21.15.80:51864) 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 -- 172.21.15.80:0/2976376066 learned_addr learned my addr 172.21.15.80:0/2976376066 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61dcdf640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618151eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61e4e0640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6181642d0 0x7ff6181618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 -- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 msgr2=0x7ff618151eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618151eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 -- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6181642d0 msgr2=0x7ff6181618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6181642d0 0x7ff6181618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 -- 172.21.15.80:0/2976376066 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff618154fb0 con 0x7ff6180b5f20 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61d4de640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6180b5f20 0x7ff618143410 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c310 tx=0x7ff60c00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff6167fc640 1 -- 172.21.15.80:0/2976376066 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0052d0 con 0x7ff6180b5f20 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff6167fc640 1 -- 172.21.15.80:0/2976376066 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff60c005470 con 0x7ff6180b5f20 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff618155290 con 0x7ff6180b5f20 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff6167fc640 1 -- 172.21.15.80:0/2976376066 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff60c007260 con 0x7ff6180b5f20 2024-04-25T13:51:36.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff61815b5a0 con 0x7ff6180b5f20 2024-04-25T13:51:36.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.772+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff61806a3a0 con 0x7ff6180b5f20 2024-04-25T13:51:36.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.776+0000 7ff6167fc640 1 -- 172.21.15.80:0/2976376066 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff60c007400 con 0x7ff6180b5f20 2024-04-25T13:51:36.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.776+0000 7ff6167fc640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff60406f400 0x7ff604071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:36.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.776+0000 7ff6167fc640 1 -- 172.21.15.80:0/2976376066 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 5738+0+0 (secure 0 0 0) 0x7ff60c08eb10 con 0x7ff6180b5f20 2024-04-25T13:51:36.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.776+0000 7ff61dcdf640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff60406f400 0x7ff604071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:36.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.776+0000 7ff61dcdf640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff60406f400 0x7ff604071910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff61806e400 tx=0x7ff610008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:36.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.780+0000 7ff6167fc640 1 -- 172.21.15.80:0/2976376066 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff6181646e0 con 0x7ff6180b5f20 2024-04-25T13:51:36.930 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.936+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff61806e430 con 0x7ff6180b5f20 2024-04-25T13:51:36.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff6167fc640 1 -- 172.21.15.80:0/2976376066 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) ==== 74+0+16664 (secure 0 0 0) 0x7ff60c05bb50 con 0x7ff6180b5f20 2024-04-25T13:51:36.933 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:36.933 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":176,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:36.284043+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:50:28.108369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T13:50:29.324161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff60406f400 msgr2=0x7ff604071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff60406f400 0x7ff604071910 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7ff61806e400 tx=0x7ff610008040 comp rx=0 tx=0).stop 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6180b5f20 msgr2=0x7ff618143410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6180b5f20 0x7ff618143410 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00c310 tx=0x7ff60c00c7d0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 shutdown_connections 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff60406f400 0x7ff604071910 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6181642d0 0x7ff6181618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff6180b5f20 0x7ff618143410 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 --2- 172.21.15.80:0/2976376066 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff6180731e0 0x7ff618151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 >> 172.21.15.80:0/2976376066 conn(0x7ff61806d080 msgr2=0x7ff6180b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 shutdown_connections 2024-04-25T13:51:36.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:36.940+0000 7ff61ff43640 1 -- 172.21.15.80:0/2976376066 wait complete. 2024-04-25T13:51:36.955 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:51:36.955 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:37.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f23980af640 1 Processor -- start 2024-04-25T13:51:37.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f23980af640 1 -- start start 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f23980af640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2390161f90 0x7f23900cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f23980af640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f239015ec40 0x7f239015f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f23980af640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23900d0d70 con 0x7f23900d0510 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f23980af640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f23900d1050 con 0x7f2390161f90 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f23980af640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f239006dca0 con 0x7f239015ec40 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f2395e4b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2390161f90 0x7f23900cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f2395e4b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f2390161f90 0x7f23900cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51870/0 (socket says 172.21.15.80:51870) 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f2395e4b640 1 -- 172.21.15.80:0/2589117346 learned_addr learned my addr 172.21.15.80:0/2589117346 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:37.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.104+0000 7f239664c640 1 --2- 172.21.15.80:0/2589117346 >> v2:172.21.15.87:3300/0 conn(0x7f239015ec40 0x7f239015f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f2395e4b640 1 -- 172.21.15.80:0/2589117346 >> v2:172.21.15.87:3300/0 conn(0x7f239015ec40 msgr2=0x7f239015f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:37.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f2395e4b640 1 --2- 172.21.15.80:0/2589117346 >> v2:172.21.15.87:3300/0 conn(0x7f239015ec40 0x7f239015f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f2395e4b640 1 -- 172.21.15.80:0/2589117346 >> v1:172.21.15.87:6789/0 conn(0x7f23900d0510 legacy=0x7f239015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:37.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f2395e4b640 1 -- 172.21.15.80:0/2589117346 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239006df80 con 0x7f2390161f90 2024-04-25T13:51:37.102 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f2395e4b640 1 --2- 172.21.15.80:0/2589117346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f23900cffd0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f23900b4b80 tx=0x7f238c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bbc7619551f162 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f2394e49640 1 -- 172.21.15.80:0/2589117346 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c0b0070 con 0x7f2390161f90 2024-04-25T13:51:37.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f2394e49640 1 -- 172.21.15.80:0/2589117346 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f238c0ab600 con 0x7f2390161f90 2024-04-25T13:51:37.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f2394e49640 1 -- 172.21.15.80:0/2589117346 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c0ab9d0 con 0x7f2390161f90 2024-04-25T13:51:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f23980af640 1 -- 172.21.15.80:0/2589117346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 msgr2=0x7f23900cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f23980af640 1 --2- 172.21.15.80:0/2589117346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f23900cffd0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f23900b4b80 tx=0x7f238c01c3f0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f23980af640 1 -- 172.21.15.80:0/2589117346 shutdown_connections 2024-04-25T13:51:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f23980af640 1 --2- 172.21.15.80:0/2589117346 >> v2:172.21.15.87:3300/0 conn(0x7f239015ec40 0x7f239015f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f23980af640 1 --2- 172.21.15.80:0/2589117346 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f23900cffd0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f23980af640 1 -- 172.21.15.80:0/2589117346 >> 172.21.15.80:0/2589117346 conn(0x7f239006d080 msgr2=0x7f23900b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f23980af640 1 -- 172.21.15.80:0/2589117346 shutdown_connections 2024-04-25T13:51:37.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.108+0000 7f23980af640 1 -- 172.21.15.80:0/2589117346 wait complete. 2024-04-25T13:51:37.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 Processor -- start 2024-04-25T13:51:37.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 -- start start 2024-04-25T13:51:37.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23900d0510 0x7f2390141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239015ec40 0x7f23901595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f2390159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f239006f630 con 0x7f2390161f90 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f239006f4b0 con 0x7f23900d0510 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f23900d0920 con 0x7f239015ec40 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f2390159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f2390159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51886/0 (socket says 172.21.15.80:51886) 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 -- 172.21.15.80:0/4013703001 learned_addr learned my addr 172.21.15.80:0/4013703001 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f2395e4b640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23900d0510 0x7f2390141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239564a640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239015ec40 0x7f23901595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 -- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239015ec40 msgr2=0x7f23901595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239015ec40 0x7f23901595e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.109 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 -- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23900d0510 msgr2=0x7f2390141670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23900d0510 0x7f2390141670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 -- 172.21.15.80:0/4013703001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239015ae30 con 0x7f2390161f90 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239564a640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239015ec40 0x7f23901595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f2395e4b640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23900d0510 0x7f2390141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f239664c640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f2390159b20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f2388007d80 tx=0x7f238800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23867fc640 1 -- 172.21.15.80:0/4013703001 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388014420 con 0x7f2390161f90 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23867fc640 1 -- 172.21.15.80:0/4013703001 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2388019070 con 0x7f2390161f90 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239015b110 con 0x7f2390161f90 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23867fc640 1 -- 172.21.15.80:0/4013703001 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388014800 con 0x7f2390161f90 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.112+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2390157860 con 0x7f2390161f90 2024-04-25T13:51:37.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.116+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239015ab90 con 0x7f2390161f90 2024-04-25T13:51:37.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.116+0000 7f23867fc640 1 -- 172.21.15.80:0/4013703001 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f23880149a0 con 0x7f2390161f90 2024-04-25T13:51:37.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.116+0000 7f23867fc640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237406f430 0x7f2374071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.116+0000 7f23867fc640 1 -- 172.21.15.80:0/4013703001 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 5738+0+0 (secure 0 0 0) 0x7f238808fa30 con 0x7f2390161f90 2024-04-25T13:51:37.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.116+0000 7f2395e4b640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237406f430 0x7f2374071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.116+0000 7f2395e4b640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237406f430 0x7f2374071940 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f238c0ad000 tx=0x7f238c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.120+0000 7f23867fc640 1 -- 172.21.15.80:0/4013703001 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f239015ab90 con 0x7f2390161f90 2024-04-25T13:51:37.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.300+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f239006a3a0 con 0x7f2390161f90 2024-04-25T13:51:37.294 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.300+0000 7f23867fc640 1 -- 172.21.15.80:0/4013703001 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5176 (secure 0 0 0) 0x7f238805c340 con 0x7f2390161f90 2024-04-25T13:51:37.294 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:37.294 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/29039900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":29039900},{"type":"v1","addr":"172.21.15.87:6836","nonce":29039900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1466426643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1466426643},{"type":"v1","addr":"172.21.15.80:6833","nonce":1466426643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1170719891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1170719891},{"type":"v1","addr":"172.21.15.80:6835","nonce":1170719891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:50:30.371502+0000","modified":"2024-04-25T13:50:33.461986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/986211877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":986211877},{"type":"v1","addr":"172.21.15.87:6837","nonce":986211877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":10}]} 2024-04-25T13:51:37.295 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 81 2024-04-25T13:51:37.296 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237406f430 msgr2=0x7f2374071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237406f430 0x7f2374071940 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f238c0ad000 tx=0x7f238c008040 comp rx=0 tx=0).stop 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 msgr2=0x7f2390159b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f2390159b20 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f2388007d80 tx=0x7f238800e4b0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 shutdown_connections 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f237406f430 0x7f2374071940 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2390161f90 0x7f2390159b20 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f239015ec40 0x7f23901595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 --2- 172.21.15.80:0/4013703001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f23900d0510 0x7f2390141670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 >> 172.21.15.80:0/4013703001 conn(0x7f239006d080 msgr2=0x7f23900b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 shutdown_connections 2024-04-25T13:51:37.297 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.304+0000 7f23980af640 1 -- 172.21.15.80:0/4013703001 wait complete. 2024-04-25T13:51:37.315 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:37.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.460+0000 7f772289d640 1 Processor -- start 2024-04-25T13:51:37.456 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.460+0000 7f772289d640 1 -- start start 2024-04-25T13:51:37.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f772289d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f771c161240 0x7f771c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f772289d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771c06fcf0 con 0x7f771c161650 2024-04-25T13:51:37.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f772289d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f771c0b0c60 con 0x7f771c06f5a0 2024-04-25T13:51:37.458 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f772289d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f771c073f10 con 0x7f771c161240 2024-04-25T13:51:37.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f771b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f771c161240 0x7f771c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f7720e3a640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f771c161650 0x7f771c0728e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41558/0 (socket says 172.21.15.80:41558) 2024-04-25T13:51:37.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f7720e3a640 1 -- 172.21.15.80:0/1533911976 learned_addr learned my addr 172.21.15.80:0/1533911976 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:37.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f771b7fe640 1 -- 172.21.15.80:0/1533911976 >> v1:172.21.15.87:6789/0 conn(0x7f771c06f5a0 legacy=0x7f771c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f771b7fe640 1 -- 172.21.15.80:0/1533911976 >> v1:172.21.15.80:6789/0 conn(0x7f771c161650 legacy=0x7f771c0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f771b7fe640 1 -- 172.21.15.80:0/1533911976 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c0b0f40 con 0x7f771c161240 2024-04-25T13:51:37.459 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.464+0000 7f771affd640 1 -- 172.21.15.80:0/1533911976 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975182106 0 0) 0x7f771c06fcf0 con 0x7f771c161650 2024-04-25T13:51:37.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f771b7fe640 1 --2- 172.21.15.80:0/1533911976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161240 0x7f771c0723a0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f77100097c0 tx=0x7f771001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0de0aa41fa3b19b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f771affd640 1 -- 172.21.15.80:0/1533911976 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77100b0070 con 0x7f771c161240 2024-04-25T13:51:37.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f771affd640 1 -- 172.21.15.80:0/1533911976 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77100ab600 con 0x7f771c161240 2024-04-25T13:51:37.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f771affd640 1 -- 172.21.15.80:0/1533911976 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77100ab9d0 con 0x7f771c161240 2024-04-25T13:51:37.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 -- 172.21.15.80:0/1533911976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161240 msgr2=0x7f771c0723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 --2- 172.21.15.80:0/1533911976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161240 0x7f771c0723a0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f77100097c0 tx=0x7f771001c410 comp rx=0 tx=0).stop 2024-04-25T13:51:37.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 -- 172.21.15.80:0/1533911976 shutdown_connections 2024-04-25T13:51:37.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 --2- 172.21.15.80:0/1533911976 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161240 0x7f771c0723a0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 -- 172.21.15.80:0/1533911976 >> 172.21.15.80:0/1533911976 conn(0x7f771c06d080 msgr2=0x7f771c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:37.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 -- 172.21.15.80:0/1533911976 shutdown_connections 2024-04-25T13:51:37.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 -- 172.21.15.80:0/1533911976 wait complete. 2024-04-25T13:51:37.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 Processor -- start 2024-04-25T13:51:37.463 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.468+0000 7f772289d640 1 -- start start 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f771c06f5a0 0x7f771c159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f771c161240 0x7f771c1532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161650 0x7f771c153820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f771c16b270 con 0x7f771c161650 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f771c16b0f0 con 0x7f771c06f5a0 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f771c16b3f0 con 0x7f771c161240 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161650 0x7f771c153820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161650 0x7f771c153820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51906/0 (socket says 172.21.15.80:51906) 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 -- 172.21.15.80:0/1143838983 learned_addr learned my addr 172.21.15.80:0/1143838983 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f771b7fe640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f771c161240 0x7f771c1532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f771bfff640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f771c06f5a0 0x7f771c159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.466 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 -- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f771c161240 msgr2=0x7f771c1532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f771c161240 0x7f771c1532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 -- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f771c06f5a0 msgr2=0x7f771c159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f771c06f5a0 0x7f771c159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 -- 172.21.15.80:0/1143838983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f771c15b220 con 0x7f771c161650 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f771bfff640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f771c06f5a0 0x7f771c159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7720e3a640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161650 0x7f771c153820 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f771400c750 tx=0x7f771400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7718ff9640 1 -- 172.21.15.80:0/1143838983 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771400ce00 con 0x7f771c161650 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7718ff9640 1 -- 172.21.15.80:0/1143838983 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7714016020 con 0x7f771c161650 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771c15b500 con 0x7f771c161650 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7718ff9640 1 -- 172.21.15.80:0/1143838983 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771400a640 con 0x7f771c161650 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f771c15bac0 con 0x7f771c161650 2024-04-25T13:51:37.467 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771c06a3a0 con 0x7f771c161650 2024-04-25T13:51:37.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7718ff9640 1 -- 172.21.15.80:0/1143838983 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f771400a7e0 con 0x7f771c161650 2024-04-25T13:51:37.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7718ff9640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f770806f400 0x7f7708071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f7718ff9640 1 -- 172.21.15.80:0/1143838983 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 5738+0+0 (secure 0 0 0) 0x7f77140913d0 con 0x7f771c161650 2024-04-25T13:51:37.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.472+0000 7f771bfff640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f770806f400 0x7f7708071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.476+0000 7f771bfff640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f770806f400 0x7f7708071910 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f770c000f70 tx=0x7f770c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.476+0000 7f7718ff9640 1 -- 172.21.15.80:0/1143838983 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7714020020 con 0x7f771c161650 2024-04-25T13:51:37.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.660+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f771c15afb0 con 0x7f771c161650 2024-04-25T13:51:37.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f7718ff9640 1 -- 172.21.15.80:0/1143838983 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5176 (secure 0 0 0) 0x7f771405e2a0 con 0x7f771c161650 2024-04-25T13:51:37.657 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:37.657 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":81,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/29039900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":29039900},{"type":"v1","addr":"172.21.15.87:6836","nonce":29039900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1466426643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1466426643},{"type":"v1","addr":"172.21.15.80:6833","nonce":1466426643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1170719891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1170719891},{"type":"v1","addr":"172.21.15.80:6835","nonce":1170719891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:50:30.371502+0000","modified":"2024-04-25T13:50:33.461986+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8814},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8814":{"gid":8814,"name":"b","rank":0,"incarnation":78,"state":"up:active","state_seq":3,"addr":"172.21.15.87:6837/986211877","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":986211877},{"type":"v1","addr":"172.21.15.87:6837","nonce":986211877}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8814,"qdb_cluster":[8814]},"id":10}]} 2024-04-25T13:51:37.657 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 81 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f770806f400 msgr2=0x7f7708071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f770806f400 0x7f7708071910 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f770c000f70 tx=0x7f770c009040 comp rx=0 tx=0).stop 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161650 msgr2=0x7f771c153820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161650 0x7f771c153820 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f771400c750 tx=0x7f771400cc10 comp rx=0 tx=0).stop 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 shutdown_connections 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f770806f400 0x7f7708071910 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f771c161650 0x7f771c153820 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f771c161240 0x7f771c1532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 --2- 172.21.15.80:0/1143838983 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f771c06f5a0 0x7f771c159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 >> 172.21.15.80:0/1143838983 conn(0x7f771c06d080 msgr2=0x7f771c0b2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 shutdown_connections 2024-04-25T13:51:37.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.664+0000 7f772289d640 1 -- 172.21.15.80:0/1143838983 wait complete. 2024-04-25T13:51:37.677 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:37.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6e915640 1 Processor -- start 2024-04-25T13:51:37.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6e915640 1 -- start start 2024-04-25T13:51:37.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6e915640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0a6816d390 0x7f0a6816f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6e915640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a68072120 con 0x7f0a6806eb40 2024-04-25T13:51:37.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6e915640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a6806dd80 con 0x7f0a681622d0 2024-04-25T13:51:37.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6e915640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0a6806e060 con 0x7f0a6816d390 2024-04-25T13:51:37.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6ceb2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0a6816d390 0x7f0a6816f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6ceb2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0a6816d390 0x7f0a6816f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51914/0 (socket says 172.21.15.80:51914) 2024-04-25T13:51:37.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a677fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0a6806eb40 0x7f0a6806ef50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41568/0 (socket says 172.21.15.80:41568) 2024-04-25T13:51:37.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6ceb2640 1 -- 172.21.15.80:0/167658446 learned_addr learned my addr 172.21.15.80:0/167658446 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:37.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6ceb2640 1 -- 172.21.15.80:0/167658446 >> v1:172.21.15.87:6789/0 conn(0x7f0a681622d0 legacy=0x7f0a680725c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:37.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a66ffd640 1 -- 172.21.15.80:0/167658446 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4243630613 0 0) 0x7f0a68072120 con 0x7f0a6806eb40 2024-04-25T13:51:37.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6ceb2640 1 -- 172.21.15.80:0/167658446 >> v1:172.21.15.80:6789/0 conn(0x7f0a6806eb40 legacy=0x7f0a6806ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.824+0000 7f0a6ceb2640 1 -- 172.21.15.80:0/167658446 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a6806e340 con 0x7f0a6816d390 2024-04-25T13:51:37.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a6ceb2640 1 --2- 172.21.15.80:0/167658446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a6816f7b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009e50 tx=0x7f0a6001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cddeb3b0785e160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a66ffd640 1 -- 172.21.15.80:0/167658446 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600a57f0 con 0x7f0a6816d390 2024-04-25T13:51:37.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a66ffd640 1 -- 172.21.15.80:0/167658446 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a600a5990 con 0x7f0a6816d390 2024-04-25T13:51:37.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a66ffd640 1 -- 172.21.15.80:0/167658446 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600a5d60 con 0x7f0a6816d390 2024-04-25T13:51:37.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a6e915640 1 -- 172.21.15.80:0/167658446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 msgr2=0x7f0a6816f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a6e915640 1 --2- 172.21.15.80:0/167658446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a6816f7b0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f0a60009e50 tx=0x7f0a6001c910 comp rx=0 tx=0).stop 2024-04-25T13:51:37.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a6e915640 1 -- 172.21.15.80:0/167658446 shutdown_connections 2024-04-25T13:51:37.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a6e915640 1 --2- 172.21.15.80:0/167658446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a6816f7b0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a6e915640 1 -- 172.21.15.80:0/167658446 >> 172.21.15.80:0/167658446 conn(0x7f0a6806d080 msgr2=0x7f0a680b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:37.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a6e915640 1 -- 172.21.15.80:0/167658446 shutdown_connections 2024-04-25T13:51:37.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.828+0000 7f0a6e915640 1 -- 172.21.15.80:0/167658446 wait complete. 2024-04-25T13:51:37.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 Processor -- start 2024-04-25T13:51:37.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 -- start start 2024-04-25T13:51:37.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6806eb40 0x7f0a68201d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a681622d0 0x7f0a6814a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a680735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0a681728a0 con 0x7f0a6816d390 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0a68172720 con 0x7f0a681622d0 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0a68172a20 con 0x7f0a6806eb40 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a680735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a680735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51916/0 (socket says 172.21.15.80:51916) 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 -- 172.21.15.80:0/613491826 learned_addr learned my addr 172.21.15.80:0/613491826 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a677fe640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a681622d0 0x7f0a6814a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a67fff640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6806eb40 0x7f0a68201d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 -- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6806eb40 msgr2=0x7f0a68201d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6806eb40 0x7f0a68201d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 -- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a681622d0 msgr2=0x7f0a6814a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a681622d0 0x7f0a6814a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 -- 172.21.15.80:0/613491826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a68154410 con 0x7f0a6816d390 2024-04-25T13:51:37.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6ceb2640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a680735f0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0a600af040 tx=0x7f0a600196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a64ff9640 1 -- 172.21.15.80:0/613491826 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600bc030 con 0x7f0a6816d390 2024-04-25T13:51:37.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a64ff9640 1 -- 172.21.15.80:0/613491826 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a600b8470 con 0x7f0a6816d390 2024-04-25T13:51:37.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a681546f0 con 0x7f0a6816d390 2024-04-25T13:51:37.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a64ff9640 1 -- 172.21.15.80:0/613491826 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a600b8840 con 0x7f0a6816d390 2024-04-25T13:51:37.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a68154cb0 con 0x7f0a6816d390 2024-04-25T13:51:37.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.832+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a68154180 con 0x7f0a6816d390 2024-04-25T13:51:37.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.836+0000 7f0a64ff9640 1 -- 172.21.15.80:0/613491826 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0a600bf070 con 0x7f0a6816d390 2024-04-25T13:51:37.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.836+0000 7f0a64ff9640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5406f400 0x7f0a54071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:37.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.836+0000 7f0a64ff9640 1 -- 172.21.15.80:0/613491826 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 5738+0+0 (secure 0 0 0) 0x7f0a60129d70 con 0x7f0a6816d390 2024-04-25T13:51:37.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.836+0000 7f0a67fff640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5406f400 0x7f0a54071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:37.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.836+0000 7f0a67fff640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5406f400 0x7f0a54071910 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0a58000f70 tx=0x7f0a58009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:37.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:37.840+0000 7f0a64ff9640 1 -- 172.21.15.80:0/613491826 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0a60016620 con 0x7f0a6816d390 2024-04-25T13:51:37.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.000+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a6814cda0 con 0x7f0a6816d390 2024-04-25T13:51:37.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.000+0000 7f0a64ff9640 1 -- 172.21.15.80:0/613491826 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) ==== 74+0+16664 (secure 0 0 0) 0x7f0a6814cda0 con 0x7f0a6816d390 2024-04-25T13:51:37.995 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:37.995 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":176,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:36.284043+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:50:28.108369+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T13:50:29.324161+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"176","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.b","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:37.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5406f400 msgr2=0x7f0a54071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5406f400 0x7f0a54071910 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f0a58000f70 tx=0x7f0a58009040 comp rx=0 tx=0).stop 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 msgr2=0x7f0a680735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a680735f0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f0a600af040 tx=0x7f0a600196b0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 shutdown_connections 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0a5406f400 0x7f0a54071910 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0a6816d390 0x7f0a680735f0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0a681622d0 0x7f0a6814a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 --2- 172.21.15.80:0/613491826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0a6806eb40 0x7f0a68201d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 >> 172.21.15.80:0/613491826 conn(0x7f0a6806d080 msgr2=0x7f0a68164160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 shutdown_connections 2024-04-25T13:51:37.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.004+0000 7f0a6e915640 1 -- 172.21.15.80:0/613491826 wait complete. 2024-04-25T13:51:38.017 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:51:38.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2cf074640 1 Processor -- start 2024-04-25T13:51:38.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2cf074640 1 -- start start 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2cf074640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2c80d06e0 0x7fe2c806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2cf074640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fe2c806e9e0 0x7fe2c806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2cf074640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe2c80d0af0 con 0x7fe2c8161f90 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2cf074640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c80d0d70 con 0x7fe2c80d06e0 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2cf074640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fe2c80d1050 con 0x7fe2c806e9e0 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2c7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2c80d06e0 0x7fe2c806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2c7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe2c80d06e0 0x7fe2c806e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51920/0 (socket says 172.21.15.80:51920) 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2c7fff640 1 -- 172.21.15.80:0/2828945278 learned_addr learned my addr 172.21.15.80:0/2828945278 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2cd611640 1 --2- 172.21.15.80:0/2828945278 >> v2:172.21.15.87:3300/0 conn(0x7fe2c806e9e0 0x7fe2c806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2c7fff640 1 -- 172.21.15.80:0/2828945278 >> v2:172.21.15.87:3300/0 conn(0x7fe2c806e9e0 msgr2=0x7fe2c806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2c7fff640 1 --2- 172.21.15.80:0/2828945278 >> v2:172.21.15.87:3300/0 conn(0x7fe2c806e9e0 0x7fe2c806edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2c7fff640 1 -- 172.21.15.80:0/2828945278 >> v1:172.21.15.87:6789/0 conn(0x7fe2c8161f90 legacy=0x7fe2c80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.157 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.160+0000 7fe2c7fff640 1 -- 172.21.15.80:0/2828945278 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c806dca0 con 0x7fe2c80d06e0 2024-04-25T13:51:38.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2c7fff640 1 --2- 172.21.15.80:0/2828945278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c80d06e0 0x7fe2c806e4a0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80b4b80 tx=0x7fe2bc01c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a97e685cb2377 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2c77fe640 1 -- 172.21.15.80:0/2828945278 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0b0070 con 0x7fe2c80d06e0 2024-04-25T13:51:38.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2c77fe640 1 -- 172.21.15.80:0/2828945278 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2bc0ab600 con 0x7fe2c80d06e0 2024-04-25T13:51:38.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2c77fe640 1 -- 172.21.15.80:0/2828945278 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2bc0ab9d0 con 0x7fe2c80d06e0 2024-04-25T13:51:38.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2cf074640 1 -- 172.21.15.80:0/2828945278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c80d06e0 msgr2=0x7fe2c806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2cf074640 1 --2- 172.21.15.80:0/2828945278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c80d06e0 0x7fe2c806e4a0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80b4b80 tx=0x7fe2bc01c1b0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2cf074640 1 -- 172.21.15.80:0/2828945278 shutdown_connections 2024-04-25T13:51:38.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2cf074640 1 --2- 172.21.15.80:0/2828945278 >> v2:172.21.15.87:3300/0 conn(0x7fe2c806e9e0 0x7fe2c806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2cf074640 1 --2- 172.21.15.80:0/2828945278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c80d06e0 0x7fe2c806e4a0 secure :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80b4b80 tx=0x7fe2bc01c1b0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2cf074640 1 -- 172.21.15.80:0/2828945278 >> 172.21.15.80:0/2828945278 conn(0x7fe2c806d080 msgr2=0x7fe2c80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2cf074640 1 -- 172.21.15.80:0/2828945278 shutdown_connections 2024-04-25T13:51:38.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.164+0000 7fe2cf074640 1 -- 172.21.15.80:0/2828945278 wait complete. 2024-04-25T13:51:38.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 Processor -- start 2024-04-25T13:51:38.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 -- start start 2024-04-25T13:51:38.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c806e9e0 0x7fe2c815c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2c8161f90 0x7fe2c81677c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c8167d00 0x7fe2c8162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c81447a0 con 0x7fe2c806e9e0 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8144620 con 0x7fe2c8167d00 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c816b510 con 0x7fe2c8161f90 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c806e9e0 0x7fe2c815c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c806e9e0 0x7fe2c815c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51924/0 (socket says 172.21.15.80:51924) 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 -- 172.21.15.80:0/3697270877 learned_addr learned my addr 172.21.15.80:0/3697270877 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2c7fff640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2c8161f90 0x7fe2c81677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cd611640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c8167d00 0x7fe2c8162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 -- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2c8161f90 msgr2=0x7fe2c81677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2c8161f90 0x7fe2c81677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 -- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c8167d00 msgr2=0x7fe2c8162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c8167d00 0x7fe2c8162ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 -- 172.21.15.80:0/3697270877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c815da90 con 0x7fe2c806e9e0 2024-04-25T13:51:38.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cce10640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c806e9e0 0x7fe2c815c520 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000b9e0 tx=0x7fe2c000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2c57fa640 1 -- 172.21.15.80:0/3697270877 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0019070 con 0x7fe2c806e9e0 2024-04-25T13:51:38.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2c57fa640 1 -- 172.21.15.80:0/3697270877 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2c000cba0 con 0x7fe2c806e9e0 2024-04-25T13:51:38.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2c57fa640 1 -- 172.21.15.80:0/3697270877 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0014440 con 0x7fe2c806e9e0 2024-04-25T13:51:38.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c815dd70 con 0x7fe2c806e9e0 2024-04-25T13:51:38.165 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.168+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c8166fc0 con 0x7fe2c806e9e0 2024-04-25T13:51:38.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.172+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c806a3a0 con 0x7fe2c806e9e0 2024-04-25T13:51:38.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.172+0000 7fe2c57fa640 1 -- 172.21.15.80:0/3697270877 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe2c001c070 con 0x7fe2c806e9e0 2024-04-25T13:51:38.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.172+0000 7fe2c57fa640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2b406f400 0x7fe2b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.172+0000 7fe2c57fa640 1 -- 172.21.15.80:0/3697270877 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(176..176 src has 1..176) ==== 5738+0+0 (secure 0 0 0) 0x7fe2c008eed0 con 0x7fe2c806e9e0 2024-04-25T13:51:38.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.172+0000 7fe2c7fff640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2b406f400 0x7fe2b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.172+0000 7fe2c7fff640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2b406f400 0x7fe2b4071910 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0ad000 tx=0x7fe2bc0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.170 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.176+0000 7fe2c57fa640 1 -- 172.21.15.80:0/3697270877 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe2c005b750 con 0x7fe2c806e9e0 2024-04-25T13:51:38.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.356+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe2c80b6540 con 0x7fe2c806e9e0 2024-04-25T13:51:38.351 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:51:38.356+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:51:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2c57fa640 1 -- 172.21.15.80:0/3697270877 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) ==== 161+0+0 (secure 0 0 0) 0x7fe2c005fa50 con 0x7fe2c806e9e0 2024-04-25T13:51:38.353 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:51:38.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2b406f400 msgr2=0x7fe2b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2b406f400 0x7fe2b4071910 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc0ad000 tx=0x7fe2bc0a6000 comp rx=0 tx=0).stop 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c806e9e0 msgr2=0x7fe2c815c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c806e9e0 0x7fe2c815c520 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000b9e0 tx=0x7fe2c000bea0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 shutdown_connections 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe2b406f400 0x7fe2b4071910 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe2c8167d00 0x7fe2c8162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe2c8161f90 0x7fe2c81677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 --2- 172.21.15.80:0/3697270877 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe2c806e9e0 0x7fe2c815c520 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 >> 172.21.15.80:0/3697270877 conn(0x7fe2c806d080 msgr2=0x7fe2c80b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 shutdown_connections 2024-04-25T13:51:38.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.360+0000 7fe2cf074640 1 -- 172.21.15.80:0/3697270877 wait complete. 2024-04-25T13:51:38.373 INFO:tasks.ceph.mds.b.smithi087.stderr: -8579> 2024-04-25T13:50:35.941+0000 7fd837c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:51:38.377 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:51:38.460 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:51:38.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.524+0000 7f098c992640 1 Processor -- start 2024-04-25T13:51:38.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f098c992640 1 -- start start 2024-04-25T13:51:38.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f098c992640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f09880b5330 0x7f09880d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f098c992640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0988161b60 0x7f0988161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f098c992640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09881624b0 con 0x7f09880d0ff0 2024-04-25T13:51:38.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f098c992640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f09881629b0 con 0x7f0988161b60 2024-04-25T13:51:38.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f098c992640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f09880b5740 con 0x7f09880b5330 2024-04-25T13:51:38.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f0986d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0988161b60 0x7f0988161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f0986d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0988161b60 0x7f0988161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51936/0 (socket says 172.21.15.80:51936) 2024-04-25T13:51:38.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f0986d9d640 1 -- 172.21.15.80:0/1101515662 learned_addr learned my addr 172.21.15.80:0/1101515662 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:38.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f098659c640 1 --2- 172.21.15.80:0/1101515662 >> v2:172.21.15.87:3300/0 conn(0x7f09880b5330 0x7f09880d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f0986d9d640 1 -- 172.21.15.80:0/1101515662 >> v2:172.21.15.87:3300/0 conn(0x7f09880b5330 msgr2=0x7f09880d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:38.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f0986d9d640 1 --2- 172.21.15.80:0/1101515662 >> v2:172.21.15.87:3300/0 conn(0x7f09880b5330 0x7f09880d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f0986d9d640 1 -- 172.21.15.80:0/1101515662 >> v1:172.21.15.87:6789/0 conn(0x7f09880d0ff0 legacy=0x7f098806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.528+0000 7f0986d9d640 1 -- 172.21.15.80:0/1101515662 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f098806dd10 con 0x7f0988161b60 2024-04-25T13:51:38.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f0986d9d640 1 --2- 172.21.15.80:0/1101515662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0988161b60 0x7f0988161f70 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f09880701f0 tx=0x7f098001c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f88363a51210500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098559a640 1 -- 172.21.15.80:0/1101515662 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800a5710 con 0x7f0988161b60 2024-04-25T13:51:38.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098559a640 1 -- 172.21.15.80:0/1101515662 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09800a58b0 con 0x7f0988161b60 2024-04-25T13:51:38.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098559a640 1 -- 172.21.15.80:0/1101515662 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09800a5c80 con 0x7f0988161b60 2024-04-25T13:51:38.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 -- 172.21.15.80:0/1101515662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0988161b60 msgr2=0x7f0988161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 --2- 172.21.15.80:0/1101515662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0988161b60 0x7f0988161f70 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f09880701f0 tx=0x7f098001c960 comp rx=0 tx=0).stop 2024-04-25T13:51:38.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 -- 172.21.15.80:0/1101515662 shutdown_connections 2024-04-25T13:51:38.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 --2- 172.21.15.80:0/1101515662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0988161b60 0x7f0988161f70 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 --2- 172.21.15.80:0/1101515662 >> v2:172.21.15.87:3300/0 conn(0x7f09880b5330 0x7f09880d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 -- 172.21.15.80:0/1101515662 >> 172.21.15.80:0/1101515662 conn(0x7f09880b4160 msgr2=0x7f09880d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:38.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 -- 172.21.15.80:0/1101515662 shutdown_connections 2024-04-25T13:51:38.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 -- 172.21.15.80:0/1101515662 wait complete. 2024-04-25T13:51:38.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 Processor -- start 2024-04-25T13:51:38.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.532+0000 7f098c992640 1 -- start start 2024-04-25T13:51:38.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09880b5330 0x7f0988141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09880d0ff0 0x7f0988158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0988161b60 0x7f0988158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09880b0cd0 con 0x7f09880d0ff0 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f098806e890 con 0x7f0988161b60 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09880b0e50 con 0x7f09880b5330 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09880d0ff0 0x7f0988158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09880d0ff0 0x7f0988158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51950/0 (socket says 172.21.15.80:51950) 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 -- 172.21.15.80:0/4006165398 learned_addr learned my addr 172.21.15.80:0/4006165398 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098659c640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09880b5330 0x7f0988141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0986d9d640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0988161b60 0x7f0988158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 -- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09880b5330 msgr2=0x7f0988141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09880b5330 0x7f0988141240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 -- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0988161b60 msgr2=0x7f0988158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0988161b60 0x7f0988158fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 -- 172.21.15.80:0/4006165398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0988155b50 con 0x7f09880d0ff0 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0985d9b640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09880d0ff0 0x7f0988158a70 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f097c00c370 tx=0x7f097c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0976ffd640 1 -- 172.21.15.80:0/4006165398 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c0052d0 con 0x7f09880d0ff0 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0976ffd640 1 -- 172.21.15.80:0/4006165398 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097c005470 con 0x7f09880d0ff0 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0988141780 con 0x7f09880d0ff0 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f098815bbe0 con 0x7f09880d0ff0 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0976ffd640 1 -- 172.21.15.80:0/4006165398 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f097c007260 con 0x7f09880d0ff0 2024-04-25T13:51:38.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f096c0052a0 con 0x7f09880d0ff0 2024-04-25T13:51:38.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0976ffd640 1 -- 172.21.15.80:0/4006165398 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f097c007400 con 0x7f09880d0ff0 2024-04-25T13:51:38.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0976ffd640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f096806f430 0x7f0968071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:38.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f0976ffd640 1 -- 172.21.15.80:0/4006165398 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 5824+0+0 (secure 0 0 0) 0x7f097c08e620 con 0x7f09880d0ff0 2024-04-25T13:51:38.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.536+0000 7f098659c640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f096806f430 0x7f0968071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:38.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.540+0000 7f098659c640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f096806f430 0x7f0968071940 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0978009ac0 tx=0x7f0978009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:38.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.544+0000 7f0976ffd640 1 -- 172.21.15.80:0/4006165398 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09881624b0 con 0x7f09880d0ff0 2024-04-25T13:51:38.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:38.724+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f096c005860 con 0x7f09880d0ff0 2024-04-25T13:51:39.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.360+0000 7f0976ffd640 1 -- 172.21.15.80:0/4006165398 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) ==== 104+0+0 (secure 0 0 0) 0x7f097c05aee0 con 0x7f09880d0ff0 2024-04-25T13:51:39.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f096806f430 msgr2=0x7f0968071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f096806f430 0x7f0968071940 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0978009ac0 tx=0x7f0978009190 comp rx=0 tx=0).stop 2024-04-25T13:51:39.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09880d0ff0 msgr2=0x7f0988158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09880d0ff0 0x7f0988158a70 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f097c00c370 tx=0x7f097c00c830 comp rx=0 tx=0).stop 2024-04-25T13:51:39.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 shutdown_connections 2024-04-25T13:51:39.360 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f096806f430 0x7f0968071940 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0988161b60 0x7f0988158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09880d0ff0 0x7f0988158a70 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 --2- 172.21.15.80:0/4006165398 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09880b5330 0x7f0988141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 >> 172.21.15.80:0/4006165398 conn(0x7f09880b4160 msgr2=0x7f09881543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 shutdown_connections 2024-04-25T13:51:39.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.368+0000 7f098c992640 1 -- 172.21.15.80:0/4006165398 wait complete. 2024-04-25T13:51:39.386 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:51:39.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8580d80640 1 Processor -- start 2024-04-25T13:51:39.540 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8580d80640 1 -- start start 2024-04-25T13:51:39.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8580d80640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f857c06fb30 0x7f857c0b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8580d80640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f857c0b4a60 con 0x7f857c1609f0 2024-04-25T13:51:39.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8580d80640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f857c160f90 con 0x7f857c0b6640 2024-04-25T13:51:39.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8580d80640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f857c161270 con 0x7f857c06fb30 2024-04-25T13:51:39.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857ad9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f857c1609f0 0x7f857c0d0140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41576/0 (socket says 172.21.15.80:41576) 2024-04-25T13:51:39.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857ad9d640 1 -- 172.21.15.80:0/2281308691 learned_addr learned my addr 172.21.15.80:0/2281308691 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:39.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857a59c640 1 --2- 172.21.15.80:0/2281308691 >> v2:172.21.15.87:3300/0 conn(0x7f857c06fb30 0x7f857c0b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1328070372 0 0) 0x7f857c0b4a60 con 0x7f857c1609f0 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8568004640 con 0x7f857c1609f0 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2942825280 0 0) 0x7f8568004640 con 0x7f857c1609f0 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 >> v2:172.21.15.87:3300/0 conn(0x7f857c06fb30 msgr2=0x7f857c0b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 --2- 172.21.15.80:0/2281308691 >> v2:172.21.15.87:3300/0 conn(0x7f857c06fb30 0x7f857c0b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 >> v1:172.21.15.87:6789/0 conn(0x7f857c0b6640 legacy=0x7f857c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c0b6a50 con 0x7f857c1609f0 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f8574001a00 con 0x7f857c1609f0 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 >> v1:172.21.15.80:6789/0 conn(0x7f857c1609f0 legacy=0x7f857c0d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8568004ad0 0x7f8568004f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8568005920 0x7f8568005d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85680062e0 0x7f856800cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f857c161270 con 0x7f85680062e0 2024-04-25T13:51:39.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f856800d280 con 0x7f8568004ad0 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f856800d480 con 0x7f8568005920 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857ad9d640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85680062e0 0x7f856800cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857a59c640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8568004ad0 0x7f8568004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f8571d9b640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8568005920 0x7f8568005d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857ad9d640 1 -- 172.21.15.80:0/2281308691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8568005920 msgr2=0x7f8568005d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857ad9d640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8568005920 0x7f8568005d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857ad9d640 1 -- 172.21.15.80:0/2281308691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8568004ad0 msgr2=0x7f8568004f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857ad9d640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8568004ad0 0x7f8568004f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.548+0000 7f857ad9d640 1 -- 172.21.15.80:0/2281308691 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8568004640 con 0x7f85680062e0 2024-04-25T13:51:39.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f857ad9d640 1 -- 172.21.15.80:0/2281308691 reap_dead start 2024-04-25T13:51:39.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f857ad9d640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85680062e0 0x7f856800cae0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f857c0ce270 tx=0x7f857401d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97eb9d417bc9274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f857400a4a0 con 0x7f85680062e0 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85740ad650 con 0x7f85680062e0 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8579d9b640 1 -- 172.21.15.80:0/2281308691 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85740ada40 con 0x7f85680062e0 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8580d80640 1 -- 172.21.15.80:0/2281308691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85680062e0 msgr2=0x7f856800cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8580d80640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85680062e0 0x7f856800cae0 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f857c0ce270 tx=0x7f857401d2a0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8580d80640 1 -- 172.21.15.80:0/2281308691 shutdown_connections 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8580d80640 1 --2- 172.21.15.80:0/2281308691 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f85680062e0 0x7f856800cae0 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8580d80640 1 -- 172.21.15.80:0/2281308691 >> 172.21.15.80:0/2281308691 conn(0x7f857c0b35c0 msgr2=0x7f857c0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8580d80640 1 -- 172.21.15.80:0/2281308691 shutdown_connections 2024-04-25T13:51:39.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.552+0000 7f8580d80640 1 -- 172.21.15.80:0/2281308691 wait complete. 2024-04-25T13:51:39.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 Processor -- start 2024-04-25T13:51:39.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 -- start start 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85680062e0 0x7f857c155f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c15da60 0x7f857c1462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f857c157b40 0x7f857c1568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f857c0b0d30 con 0x7f857c157b40 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f857c0b0bb0 con 0x7f85680062e0 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f857c0b0f70 con 0x7f857c15da60 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f857c157b40 0x7f857c1568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f857c157b40 0x7f857c1568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51958/0 (socket says 172.21.15.80:51958) 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 -- 172.21.15.80:0/2529147229 learned_addr learned my addr 172.21.15.80:0/2529147229 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857a59c640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85680062e0 0x7f857c155f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8571d9b640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c15da60 0x7f857c1462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 -- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c15da60 msgr2=0x7f857c1462e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c15da60 0x7f857c1462e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 -- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85680062e0 msgr2=0x7f857c155f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85680062e0 0x7f857c155f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 -- 172.21.15.80:0/2529147229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857c156440 con 0x7f857c157b40 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f857ad9d640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f857c157b40 0x7f857c1568a0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f857401db40 tx=0x7f8568006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f85737fe640 1 -- 172.21.15.80:0/2529147229 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8574004d70 con 0x7f857c157b40 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f85737fe640 1 -- 172.21.15.80:0/2529147229 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85740b4070 con 0x7f857c157b40 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857c1544d0 con 0x7f857c157b40 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f85737fe640 1 -- 172.21.15.80:0/2529147229 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85740019c0 con 0x7f857c157b40 2024-04-25T13:51:39.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f857c154a90 con 0x7f857c157b40 2024-04-25T13:51:39.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.556+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f857c154c60 con 0x7f857c157b40 2024-04-25T13:51:39.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.560+0000 7f85737fe640 1 -- 172.21.15.80:0/2529147229 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f85740b6030 con 0x7f857c157b40 2024-04-25T13:51:39.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.560+0000 7f85737fe640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f855c06f400 0x7f855c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:39.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.560+0000 7f85737fe640 1 -- 172.21.15.80:0/2529147229 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(177..177 src has 1..177) ==== 5824+0+0 (secure 0 0 0) 0x7f85741293c0 con 0x7f857c157b40 2024-04-25T13:51:39.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.560+0000 7f857a59c640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f855c06f400 0x7f855c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:39.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.560+0000 7f857a59c640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f855c06f400 0x7f855c071910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f857c071a20 tx=0x7f856c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:39.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.564+0000 7f85737fe640 1 -- 172.21.15.80:0/2529147229 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f85740be060 con 0x7f857c157b40 2024-04-25T13:51:39.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:39.716+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f857c06a440 con 0x7f857c157b40 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.583+0000 7f85737fe640 1 -- 172.21.15.80:0/2529147229 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v178) ==== 181+0+0 (secure 0 0 0) 0x7f8574016fb0 con 0x7f857c157b40 2024-04-25T13:51:40.579 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:51:40.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f855c06f400 msgr2=0x7f855c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f855c06f400 0x7f855c071910 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f857c071a20 tx=0x7f856c008040 comp rx=0 tx=0).stop 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f857c157b40 msgr2=0x7f857c1568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f857c157b40 0x7f857c1568a0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f857401db40 tx=0x7f8568006950 comp rx=0 tx=0).stop 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 shutdown_connections 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f855c06f400 0x7f855c071910 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f857c157b40 0x7f857c1568a0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f857c15da60 0x7f857c1462e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 --2- 172.21.15.80:0/2529147229 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f85680062e0 0x7f857c155f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 >> 172.21.15.80:0/2529147229 conn(0x7f857c0b35c0 msgr2=0x7f857c071eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 shutdown_connections 2024-04-25T13:51:40.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.587+0000 7f8580d80640 1 -- 172.21.15.80:0/2529147229 wait complete. 2024-04-25T13:51:40.611 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:51:40.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb50ed91640 1 Processor -- start 2024-04-25T13:51:40.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb50ed91640 1 -- start start 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb50ed91640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 0x7fb5080689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb50ed91640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5080707f0 con 0x7fb508161d20 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb50ed91640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb508068ee0 con 0x7fb50806cf60 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb50ed91640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb50806d690 con 0x7fb508068590 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb50d32e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb508161d20 0x7fb508162130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41588/0 (socket says 172.21.15.80:41588) 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb50d32e640 1 -- 172.21.15.80:0/4121938614 learned_addr learned my addr 172.21.15.80:0/4121938614 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb507fff640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 0x7fb5080689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2197345478 0 0) 0x7fb5000014a0 con 0x7fb508161d20 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb4f0003530 con 0x7fb508161d20 2024-04-25T13:51:40.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3212336197 0 0) 0x7fb4f0003530 con 0x7fb508161d20 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 msgr2=0x7fb5080689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb5077fe640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 0x7fb5080689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 >> v1:172.21.15.87:6789/0 conn(0x7fb50806cf60 legacy=0x7fb5081550a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb508073140 con 0x7fb508161d20 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.763+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb500001bd0 con 0x7fb508161d20 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 >> v1:172.21.15.80:6789/0 conn(0x7fb508161d20 legacy=0x7fb508162130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f00039f0 0x7fb4f0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 0x7fb4f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f0004d50 0x7fb4f0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb50806d690 con 0x7fb4f0004d50 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb4f000bb30 con 0x7fb4f00039f0 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb4f000bd30 con 0x7fb508068590 2024-04-25T13:51:40.762 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50d32e640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f0004d50 0x7fb4f0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50cb2d640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f00039f0 0x7fb4f0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb507fff640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 0x7fb4f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50d32e640 1 -- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 msgr2=0x7fb4f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50d32e640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 0x7fb4f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50d32e640 1 -- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f00039f0 msgr2=0x7fb4f0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50d32e640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f00039f0 0x7fb4f0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50d32e640 1 -- 172.21.15.80:0/4121938614 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4f0003530 con 0x7fb4f0004d50 2024-04-25T13:51:40.763 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb507fff640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb508068590 0x7fb4f0004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:40.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50d32e640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f0004d50 0x7fb4f0005160 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb50000aa30 tx=0x7fb50001d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb43342d25237cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50000a650 con 0x7fb4f0004d50 2024-04-25T13:51:40.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5000ad650 con 0x7fb4f0004d50 2024-04-25T13:51:40.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb5077fe640 1 -- 172.21.15.80:0/4121938614 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5000ada40 con 0x7fb4f0004d50 2024-04-25T13:51:40.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50ed91640 1 -- 172.21.15.80:0/4121938614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f0004d50 msgr2=0x7fb4f0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50ed91640 1 --2- 172.21.15.80:0/4121938614 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb4f0004d50 0x7fb4f0005160 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7fb50000aa30 tx=0x7fb50001d380 comp rx=0 tx=0).stop 2024-04-25T13:51:40.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50d32e640 1 -- 172.21.15.80:0/4121938614 reap_dead start 2024-04-25T13:51:40.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50ed91640 1 -- 172.21.15.80:0/4121938614 shutdown_connections 2024-04-25T13:51:40.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50ed91640 1 -- 172.21.15.80:0/4121938614 >> 172.21.15.80:0/4121938614 conn(0x7fb5080b50c0 msgr2=0x7fb5080d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:40.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50ed91640 1 -- 172.21.15.80:0/4121938614 shutdown_connections 2024-04-25T13:51:40.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.767+0000 7fb50ed91640 1 -- 172.21.15.80:0/4121938614 wait complete. 2024-04-25T13:51:40.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 Processor -- start 2024-04-25T13:51:40.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 -- start start 2024-04-25T13:51:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb508157b10 0x7fb508156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5080ab4a0 0x7fb508156da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5081fa450 0x7fb5081fa860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb508072ef0 con 0x7fb508157b10 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb508073700 con 0x7fb5081fa450 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5080739e0 con 0x7fb5080ab4a0 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb508157b10 0x7fb508156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb508157b10 0x7fb508156860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:51968/0 (socket says 172.21.15.80:51968) 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 -- 172.21.15.80:0/3027809242 learned_addr learned my addr 172.21.15.80:0/3027809242 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 -- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5080ab4a0 msgr2=0x7fb508156da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5080ab4a0 0x7fb508156da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 -- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5081fa450 msgr2=0x7fb5081fa860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5081fa450 0x7fb5081fa860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 -- 172.21.15.80:0/3027809242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5080b7af0 con 0x7fb508157b10 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50cb2d640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb508157b10 0x7fb508156860 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800e4b0 tx=0x7fb4f800e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb5057fa640 1 -- 172.21.15.80:0/3027809242 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8019070 con 0x7fb508157b10 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb5057fa640 1 -- 172.21.15.80:0/3027809242 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb4f80052e0 con 0x7fb508157b10 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb508157510 con 0x7fb508157b10 2024-04-25T13:51:40.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb5057fa640 1 -- 172.21.15.80:0/3027809242 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8003820 con 0x7fb508157b10 2024-04-25T13:51:40.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.771+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5080b65b0 con 0x7fb508157b10 2024-04-25T13:51:40.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.775+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb50806a440 con 0x7fb508157b10 2024-04-25T13:51:40.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.775+0000 7fb5057fa640 1 -- 172.21.15.80:0/3027809242 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb4f801c070 con 0x7fb508157b10 2024-04-25T13:51:40.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.775+0000 7fb5057fa640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb4f406f400 0x7fb4f4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:40.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.775+0000 7fb5057fa640 1 -- 172.21.15.80:0/3027809242 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(178..178 src has 1..178) ==== 5386+0+0 (secure 0 0 0) 0x7fb4f808e5d0 con 0x7fb508157b10 2024-04-25T13:51:40.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.775+0000 7fb507fff640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb4f406f400 0x7fb4f4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:40.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.775+0000 7fb507fff640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb4f406f400 0x7fb4f4071910 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb5080b5760 tx=0x7fb4fc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:40.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.779+0000 7fb5057fa640 1 -- 172.21.15.80:0/3027809242 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb508068ee0 con 0x7fb508157b10 2024-04-25T13:51:40.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:40.939+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb508068cc0 con 0x7fb508157b10 2024-04-25T13:51:41.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.587+0000 7fb5057fa640 1 -- 172.21.15.80:0/3027809242 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v179) ==== 169+0+0 (secure 0 0 0) 0x7fb4f805b0d0 con 0x7fb508157b10 2024-04-25T13:51:41.583 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:51:41.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb4f406f400 msgr2=0x7fb4f4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:41.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb4f406f400 0x7fb4f4071910 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fb5080b5760 tx=0x7fb4fc008040 comp rx=0 tx=0).stop 2024-04-25T13:51:41.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb508157b10 msgr2=0x7fb508156860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:41.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb508157b10 0x7fb508156860 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb4f800e4b0 tx=0x7fb4f800e970 comp rx=0 tx=0).stop 2024-04-25T13:51:41.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 shutdown_connections 2024-04-25T13:51:41.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb4f406f400 0x7fb4f4071910 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5081fa450 0x7fb5081fa860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5080ab4a0 0x7fb508156da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 --2- 172.21.15.80:0/3027809242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb508157b10 0x7fb508156860 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 >> 172.21.15.80:0/3027809242 conn(0x7fb5080b50c0 msgr2=0x7fb508160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:41.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 shutdown_connections 2024-04-25T13:51:41.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.591+0000 7fb50ed91640 1 -- 172.21.15.80:0/3027809242 wait complete. 2024-04-25T13:51:41.612 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:51:41.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.771+0000 7f8922eee640 1 Processor -- start 2024-04-25T13:51:41.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.771+0000 7f8922eee640 1 -- start start 2024-04-25T13:51:41.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f8922eee640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f891c0d3ed0 0x7f891c0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f8922eee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 0x7f891c0b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f8922eee640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f891c0b7c10 0x7f891c0b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f8922eee640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f891c0b8560 con 0x7f891c0b7c10 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f8922eee640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f891c0b92a0 con 0x7f891c0d3ed0 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f8922eee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f891c0d4c10 con 0x7f891c0d2ad0 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f891c0b7c10 0x7f891c0b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f891c0b7c10 0x7f891c0b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41302/0 (socket says 172.21.15.80:41302) 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 -- 172.21.15.80:0/2789645826 learned_addr learned my addr 172.21.15.80:0/2789645826 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:41.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f891bfff640 1 --2- 172.21.15.80:0/2789645826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 0x7f891c0b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:41.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f8920c8a640 1 --2- 172.21.15.80:0/2789645826 >> v2:172.21.15.87:3300/0 conn(0x7f891c0d3ed0 0x7f891c0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:41.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 -- 172.21.15.80:0/2789645826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 msgr2=0x7f891c0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:41.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 --2- 172.21.15.80:0/2789645826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 0x7f891c0b76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 -- 172.21.15.80:0/2789645826 >> v2:172.21.15.87:3300/0 conn(0x7f891c0d3ed0 msgr2=0x7f891c0d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:41.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 --2- 172.21.15.80:0/2789645826 >> v2:172.21.15.87:3300/0 conn(0x7f891c0d3ed0 0x7f891c0d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 -- 172.21.15.80:0/2789645826 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c0d4ef0 con 0x7f891c0b7c10 2024-04-25T13:51:41.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f892148b640 1 --2- 172.21.15.80:0/2789645826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0b8020 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f891c08e910 tx=0x7f891401c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8d2ed669c9061f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:41.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f891b7fe640 1 -- 172.21.15.80:0/2789645826 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89140a5870 con 0x7f891c0b7c10 2024-04-25T13:51:41.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f891b7fe640 1 -- 172.21.15.80:0/2789645826 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89140a5a10 con 0x7f891c0b7c10 2024-04-25T13:51:41.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.775+0000 7f891b7fe640 1 -- 172.21.15.80:0/2789645826 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89140a5de0 con 0x7f891c0b7c10 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- 172.21.15.80:0/2789645826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 msgr2=0x7f891c0b8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 --2- 172.21.15.80:0/2789645826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0b8020 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f891c08e910 tx=0x7f891401c9f0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- 172.21.15.80:0/2789645826 shutdown_connections 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 --2- 172.21.15.80:0/2789645826 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0b8020 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 --2- 172.21.15.80:0/2789645826 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 0x7f891c0b76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 --2- 172.21.15.80:0/2789645826 >> v2:172.21.15.87:3300/0 conn(0x7f891c0d3ed0 0x7f891c0d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- 172.21.15.80:0/2789645826 >> 172.21.15.80:0/2789645826 conn(0x7f891c0737b0 msgr2=0x7f891c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- 172.21.15.80:0/2789645826 shutdown_connections 2024-04-25T13:51:41.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- 172.21.15.80:0/2789645826 wait complete. 2024-04-25T13:51:41.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 Processor -- start 2024-04-25T13:51:41.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- start start 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 0x7f891c14d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f891c0d3ed0 0x7f891c0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f891c0ccae0 con 0x7f891c0b7c10 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f891c0cc960 con 0x7f891c0d3ed0 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.779+0000 7f8922eee640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f891c160620 con 0x7f891c0d2ad0 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41316/0 (socket says 172.21.15.80:41316) 2024-04-25T13:51:41.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 -- 172.21.15.80:0/3931354867 learned_addr learned my addr 172.21.15.80:0/3931354867 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f891bfff640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 0x7f891c14d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f892148b640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f891c0d3ed0 0x7f891c0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 -- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 msgr2=0x7f891c14d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 0x7f891c14d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 -- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f891c0d3ed0 msgr2=0x7f891c0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f891c0d3ed0 0x7f891c0cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 -- 172.21.15.80:0/3931354867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891c0cd720 con 0x7f891c0b7c10 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8920c8a640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0ccd40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f890c00d5f0 tx=0x7f890c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f89197fa640 1 -- 172.21.15.80:0/3931354867 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c0125f0 con 0x7f891c0b7c10 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f89197fa640 1 -- 172.21.15.80:0/3931354867 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f890c019070 con 0x7f891c0b7c10 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891c0cda00 con 0x7f891c0b7c10 2024-04-25T13:51:41.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f89197fa640 1 -- 172.21.15.80:0/3931354867 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f890c0129b0 con 0x7f891c0b7c10 2024-04-25T13:51:41.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f891c0cdfc0 con 0x7f891c0b7c10 2024-04-25T13:51:41.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f891c06a3a0 con 0x7f891c0b7c10 2024-04-25T13:51:41.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f89197fa640 1 -- 172.21.15.80:0/3931354867 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f890c012b50 con 0x7f891c0b7c10 2024-04-25T13:51:41.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f89197fa640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f890806f430 0x7f8908071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:41.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f89197fa640 1 -- 172.21.15.80:0/3931354867 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4943+0+0 (secure 0 0 0) 0x7f890c08f270 con 0x7f891c0b7c10 2024-04-25T13:51:41.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.783+0000 7f891bfff640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f890806f430 0x7f8908071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:41.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.787+0000 7f891bfff640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f890806f430 0x7f8908071940 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f891c1571d0 tx=0x7f8910008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:41.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.787+0000 7f89197fa640 1 -- 172.21.15.80:0/3931354867 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f890c05c580 con 0x7f891c0b7c10 2024-04-25T13:51:41.944 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.951+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f891c0cc320 con 0x7f891c0b7c10 2024-04-25T13:51:41.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f89197fa640 1 -- 172.21.15.80:0/3931354867 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7f890c05be80 con 0x7f891c0b7c10 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f890806f430 msgr2=0x7f8908071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f890806f430 0x7f8908071940 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f891c1571d0 tx=0x7f8910008040 comp rx=0 tx=0).stop 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 msgr2=0x7f891c0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0ccd40 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f890c00d5f0 tx=0x7f890c00dab0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 shutdown_connections 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f890806f430 0x7f8908071940 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f891c0d3ed0 0x7f891c0cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f891c0d2ad0 0x7f891c14d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 --2- 172.21.15.80:0/3931354867 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f891c0b7c10 0x7f891c0ccd40 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 >> 172.21.15.80:0/3931354867 conn(0x7f891c0737b0 msgr2=0x7f891c155420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:41.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 shutdown_connections 2024-04-25T13:51:41.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:41.955+0000 7f8922eee640 1 -- 172.21.15.80:0/3931354867 wait complete. 2024-04-25T13:51:41.972 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2024-04-25T13:51:42.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcfdaa8640 1 Processor -- start 2024-04-25T13:51:42.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcfdaa8640 1 -- start start 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcfdaa8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efcf8161f90 0x7efcf80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcfdaa8640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efcf80d0510 0x7efcf815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcfdaa8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf815ea70 0x7efcf815ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcfdaa8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efcf80d0d70 con 0x7efcf8161f90 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcfdaa8640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efcf80d1050 con 0x7efcf80d0510 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcfdaa8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efcf806dca0 con 0x7efcf815ea70 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efcf8161f90 0x7efcf80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf77fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efcf8161f90 0x7efcf80cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41320/0 (socket says 172.21.15.80:41320) 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf77fe640 1 -- 172.21.15.80:0/194939181 learned_addr learned my addr 172.21.15.80:0/194939181 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf7fff640 1 --2- 172.21.15.80:0/194939181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf815ea70 0x7efcf815ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf6ffd640 1 --2- 172.21.15.80:0/194939181 >> v2:172.21.15.87:3300/0 conn(0x7efcf80d0510 0x7efcf815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf77fe640 1 -- 172.21.15.80:0/194939181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf815ea70 msgr2=0x7efcf815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf77fe640 1 --2- 172.21.15.80:0/194939181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf815ea70 0x7efcf815ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf77fe640 1 -- 172.21.15.80:0/194939181 >> v2:172.21.15.87:3300/0 conn(0x7efcf80d0510 msgr2=0x7efcf815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf77fe640 1 --2- 172.21.15.80:0/194939181 >> v2:172.21.15.87:3300/0 conn(0x7efcf80d0510 0x7efcf815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.127+0000 7efcf77fe640 1 -- 172.21.15.80:0/194939181 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf806df80 con 0x7efcf8161f90 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcf77fe640 1 --2- 172.21.15.80:0/194939181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8161f90 0x7efcf80cffd0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7efcf8074290 tx=0x7efce801c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83719aff2154706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcf67fc640 1 -- 172.21.15.80:0/194939181 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce80a5770 con 0x7efcf8161f90 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcf67fc640 1 -- 172.21.15.80:0/194939181 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce80a5910 con 0x7efcf8161f90 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcf67fc640 1 -- 172.21.15.80:0/194939181 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce80a5ce0 con 0x7efcf8161f90 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 -- 172.21.15.80:0/194939181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8161f90 msgr2=0x7efcf80cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/194939181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8161f90 0x7efcf80cffd0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7efcf8074290 tx=0x7efce801c930 comp rx=0 tx=0).stop 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 -- 172.21.15.80:0/194939181 shutdown_connections 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/194939181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf815ea70 0x7efcf815ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/194939181 >> v2:172.21.15.87:3300/0 conn(0x7efcf80d0510 0x7efcf815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/194939181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf8161f90 0x7efcf80cffd0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 -- 172.21.15.80:0/194939181 >> 172.21.15.80:0/194939181 conn(0x7efcf806d080 msgr2=0x7efcf80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:42.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 -- 172.21.15.80:0/194939181 shutdown_connections 2024-04-25T13:51:42.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.131+0000 7efcfdaa8640 1 -- 172.21.15.80:0/194939181 wait complete. 2024-04-25T13:51:42.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 Processor -- start 2024-04-25T13:51:42.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 -- start start 2024-04-25T13:51:42.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf80d0510 0x7efcf81fa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcf815ea70 0x7efcf8165c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf8161f90 0x7efcf8166150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efcf81447a0 con 0x7efcf80d0510 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efcf8144620 con 0x7efcf815ea70 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efcf80d0920 con 0x7efcf8161f90 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf80d0510 0x7efcf81fa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf80d0510 0x7efcf81fa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41322/0 (socket says 172.21.15.80:41322) 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 -- 172.21.15.80:0/4010410694 learned_addr learned my addr 172.21.15.80:0/4010410694 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf7fff640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf8161f90 0x7efcf8166150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf6ffd640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcf815ea70 0x7efcf8165c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 -- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf8161f90 msgr2=0x7efcf8166150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf8161f90 0x7efcf8166150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 -- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcf815ea70 msgr2=0x7efcf8165c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcf815ea70 0x7efcf8165c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 -- 172.21.15.80:0/4010410694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf8163910 con 0x7efcf80d0510 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf7fff640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf8161f90 0x7efcf8166150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf6ffd640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcf815ea70 0x7efcf8165c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcf77fe640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf80d0510 0x7efcf81fa1c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7efce80af040 tx=0x7efce80196d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcdffff640 1 -- 172.21.15.80:0/4010410694 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce80b8670 con 0x7efcf80d0510 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcdffff640 1 -- 172.21.15.80:0/4010410694 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efce80be070 con 0x7efcf80d0510 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf81fa700 con 0x7efcf80d0510 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcdffff640 1 -- 172.21.15.80:0/4010410694 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce80b8a50 con 0x7efcf80d0510 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.135+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf8164710 con 0x7efcf80d0510 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.139+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcf806a3a0 con 0x7efcf80d0510 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.139+0000 7efcdffff640 1 -- 172.21.15.80:0/4010410694 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efce80b8bf0 con 0x7efcf80d0510 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.139+0000 7efcdffff640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce406f400 0x7efce4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.139+0000 7efcdffff640 1 -- 172.21.15.80:0/4010410694 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4943+0+0 (secure 0 0 0) 0x7efce812a120 con 0x7efcf80d0510 2024-04-25T13:51:42.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.139+0000 7efcf6ffd640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce406f400 0x7efce4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.139+0000 7efcf6ffd640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce406f400 0x7efce4071910 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efcf81439a0 tx=0x7efcec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:42.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.143+0000 7efcdffff640 1 -- 172.21.15.80:0/4010410694 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efcf81fa700 con 0x7efcf80d0510 2024-04-25T13:51:42.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.343+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) -- 0x7efcf8165040 con 0x7efcf80d0510 2024-04-25T13:51:42.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.603+0000 7efcdffff640 1 -- 172.21.15.80:0/4010410694 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v545) ==== 131+0+0 (secure 0 0 0) 0x7efcf8163910 con 0x7efcf80d0510 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce406f400 msgr2=0x7efce4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce406f400 0x7efce4071910 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7efcf81439a0 tx=0x7efcec008040 comp rx=0 tx=0).stop 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf80d0510 msgr2=0x7efcf81fa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf80d0510 0x7efcf81fa1c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7efce80af040 tx=0x7efce80196d0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 shutdown_connections 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efce406f400 0x7efce4071910 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efcf8161f90 0x7efcf8166150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efcf815ea70 0x7efcf8165c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 --2- 172.21.15.80:0/4010410694 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efcf80d0510 0x7efcf81fa1c0 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 >> 172.21.15.80:0/4010410694 conn(0x7efcf806d080 msgr2=0x7efcf80b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 shutdown_connections 2024-04-25T13:51:42.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.607+0000 7efcfdaa8640 1 -- 172.21.15.80:0/4010410694 wait complete. 2024-04-25T13:51:42.628 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2024-04-25T13:51:42.628 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2024-04-25T13:51:42.629 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:42.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb90984a640 1 Processor -- start 2024-04-25T13:51:42.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb90984a640 1 -- start start 2024-04-25T13:51:42.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb90984a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9040cc720 0x7fb904073e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb90984a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb9040743d0 0x7fb9040cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb90984a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb9040cd6c0 con 0x7fb9040ce010 2024-04-25T13:51:42.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb90984a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb90415ea50 con 0x7fb9040cc720 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb90984a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb9040729e0 con 0x7fb9040743d0 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb902ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb9040cc720 0x7fb904073e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb9037fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb9040ce010 0x7fb90406ec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48368/0 (socket says 172.21.15.80:48368) 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb9037fe640 1 -- 172.21.15.80:0/137920784 learned_addr learned my addr 172.21.15.80:0/137920784 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb9027fc640 1 --2- 172.21.15.80:0/137920784 >> v2:172.21.15.87:3300/0 conn(0x7fb9040743d0 0x7fb9040cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb902ffd640 1 -- 172.21.15.80:0/137920784 >> v2:172.21.15.87:3300/0 conn(0x7fb9040743d0 msgr2=0x7fb9040cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb902ffd640 1 --2- 172.21.15.80:0/137920784 >> v2:172.21.15.87:3300/0 conn(0x7fb9040743d0 0x7fb9040cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb902ffd640 1 -- 172.21.15.80:0/137920784 >> v1:172.21.15.80:6789/0 conn(0x7fb9040ce010 legacy=0x7fb90406ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.787+0000 7fb902ffd640 1 -- 172.21.15.80:0/137920784 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9040cce20 con 0x7fb9040cc720 2024-04-25T13:51:42.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.791+0000 7fb902ffd640 1 --2- 172.21.15.80:0/137920784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040cc720 0x7fb904073e90 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb9040a01f0 tx=0x7fb8f401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f24fb288d380d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:42.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.791+0000 7fb901ffb640 1 -- 172.21.15.80:0/137920784 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b1070 con 0x7fb9040cc720 2024-04-25T13:51:42.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.791+0000 7fb901ffb640 1 -- 172.21.15.80:0/137920784 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f40ac620 con 0x7fb9040cc720 2024-04-25T13:51:42.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.791+0000 7fb901ffb640 1 -- 172.21.15.80:0/137920784 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40ac9f0 con 0x7fb9040cc720 2024-04-25T13:51:42.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- 172.21.15.80:0/137920784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040cc720 msgr2=0x7fb904073e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 --2- 172.21.15.80:0/137920784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040cc720 0x7fb904073e90 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fb9040a01f0 tx=0x7fb8f401c3c0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- 172.21.15.80:0/137920784 shutdown_connections 2024-04-25T13:51:42.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 --2- 172.21.15.80:0/137920784 >> v2:172.21.15.87:3300/0 conn(0x7fb9040743d0 0x7fb9040cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 --2- 172.21.15.80:0/137920784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040cc720 0x7fb904073e90 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- 172.21.15.80:0/137920784 >> 172.21.15.80:0/137920784 conn(0x7fb9040d20d0 msgr2=0x7fb9040d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:42.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- 172.21.15.80:0/137920784 shutdown_connections 2024-04-25T13:51:42.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- 172.21.15.80:0/137920784 wait complete. 2024-04-25T13:51:42.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 Processor -- start 2024-04-25T13:51:42.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- start start 2024-04-25T13:51:42.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040743d0 0x7fb904153830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb9040cc720 0x7fb9040b62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9040ce010 0x7fb9041503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040ce830 con 0x7fb9040743d0 2024-04-25T13:51:42.792 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040ccb30 con 0x7fb9040ce010 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb90984a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb9040747e0 con 0x7fb9040cc720 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040743d0 0x7fb904153830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb902ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040743d0 0x7fb904153830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41346/0 (socket says 172.21.15.80:41346) 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb902ffd640 1 -- 172.21.15.80:0/1307479096 learned_addr learned my addr 172.21.15.80:0/1307479096 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb9037fe640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9040ce010 0x7fb9041503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb9027fc640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb9040cc720 0x7fb9040b62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb902ffd640 1 -- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb9040cc720 msgr2=0x7fb9040b62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb902ffd640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb9040cc720 0x7fb9040b62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb902ffd640 1 -- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9040ce010 msgr2=0x7fb9041503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb902ffd640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9040ce010 0x7fb9041503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb902ffd640 1 -- 172.21.15.80:0/1307479096 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9040b6f90 con 0x7fb9040743d0 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.795+0000 7fb9037fe640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9040ce010 0x7fb9041503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb902ffd640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040743d0 0x7fb904153830 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4019ff0 tx=0x7fb8f4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:42.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb8ebfff640 1 -- 172.21.15.80:0/1307479096 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b1040 con 0x7fb9040743d0 2024-04-25T13:51:42.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb8ebfff640 1 -- 172.21.15.80:0/1307479096 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb8f40ba070 con 0x7fb9040743d0 2024-04-25T13:51:42.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb904154390 con 0x7fb9040743d0 2024-04-25T13:51:42.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb8ebfff640 1 -- 172.21.15.80:0/1307479096 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b5630 con 0x7fb9040743d0 2024-04-25T13:51:42.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb904154950 con 0x7fb9040743d0 2024-04-25T13:51:42.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90406a3a0 con 0x7fb9040743d0 2024-04-25T13:51:42.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb8ebfff640 1 -- 172.21.15.80:0/1307479096 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb8f40b57d0 con 0x7fb9040743d0 2024-04-25T13:51:42.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb8ebfff640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8f006f430 0x7fb8f0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:42.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb8ebfff640 1 -- 172.21.15.80:0/1307479096 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4943+0+0 (secure 0 0 0) 0x7fb8f4128b60 con 0x7fb9040743d0 2024-04-25T13:51:42.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb9027fc640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8f006f430 0x7fb8f0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:42.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.799+0000 7fb9027fc640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8f006f430 0x7fb8f0071940 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb904152480 tx=0x7fb8f8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:42.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.803+0000 7fb8ebfff640 1 -- 172.21.15.80:0/1307479096 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb8f4016070 con 0x7fb9040743d0 2024-04-25T13:51:42.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.967+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb90406fcf0 con 0x7fb9040743d0 2024-04-25T13:51:42.962 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.967+0000 7fb8ebfff640 1 -- 172.21.15.80:0/1307479096 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) ==== 74+0+12957 (secure 0 0 0) 0x7fb8f40f5d50 con 0x7fb9040743d0 2024-04-25T13:51:42.962 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:42.962 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":179,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:41.587041+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6837/986211877":"2024-04-26T13:51:38.358958+0000","172.21.15.87:6835/986211877":"2024-04-26T13:51:38.358958+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:42.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.967+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8f006f430 msgr2=0x7fb8f0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.967+0000 7fb90984a640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8f006f430 0x7fb8f0071940 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fb904152480 tx=0x7fb8f8008040 comp rx=0 tx=0).stop 2024-04-25T13:51:42.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.967+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040743d0 msgr2=0x7fb904153830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:42.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040743d0 0x7fb904153830 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4019ff0 tx=0x7fb8f4019160 comp rx=0 tx=0).stop 2024-04-25T13:51:42.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 shutdown_connections 2024-04-25T13:51:42.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb8f006f430 0x7fb8f0071940 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb9040ce010 0x7fb9041503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb9040cc720 0x7fb9040b62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 --2- 172.21.15.80:0/1307479096 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb9040743d0 0x7fb904153830 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:42.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 >> 172.21.15.80:0/1307479096 conn(0x7fb9040d20d0 msgr2=0x7fb9040b3360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:42.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 shutdown_connections 2024-04-25T13:51:42.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:42.971+0000 7fb90984a640 1 -- 172.21.15.80:0/1307479096 wait complete. 2024-04-25T13:51:42.986 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:51:42.986 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:51:43.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920fb02640 1 Processor -- start 2024-04-25T13:51:43.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920fb02640 1 -- start start 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920fb02640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92080b5330 0x7f92080d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920fb02640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9208161b60 0x7f9208161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920fb02640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92081624b0 con 0x7f92080d0ff0 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920fb02640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f92081629b0 con 0x7f9208161b60 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920fb02640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92080b5740 con 0x7f92080b5330 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920d09d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f92080d0ff0 0x7f920806edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48376/0 (socket says 172.21.15.80:48376) 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920d09d640 1 -- 172.21.15.80:0/1046830590 learned_addr learned my addr 172.21.15.80:0/1046830590 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920d89e640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92080b5330 0x7f92080d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.137 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920e09f640 1 --2- 172.21.15.80:0/1046830590 >> v2:172.21.15.87:3300/0 conn(0x7f9208161b60 0x7f9208161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3340495507 0 0) 0x7f91f8001120 con 0x7f92080d0ff0 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.139+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f91f4003530 con 0x7f92080d0ff0 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3951178930 0 0) 0x7f91f4003530 con 0x7f92080d0ff0 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92080b5330 msgr2=0x7f92080d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92080b5330 0x7f92080d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 >> v2:172.21.15.87:3300/0 conn(0x7f9208161b60 msgr2=0x7f9208161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 --2- 172.21.15.80:0/1046830590 >> v2:172.21.15.87:3300/0 conn(0x7f9208161b60 0x7f9208161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920806dd10 con 0x7f92080d0ff0 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f91f8001850 con 0x7f92080d0ff0 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 >> v1:172.21.15.80:6789/0 conn(0x7f92080d0ff0 legacy=0x7f920806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92080b5330 0x7f91f40039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91f4003f40 0x7f91f4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4004d50 0x7f91f4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f92081629b0 con 0x7f91f4003f40 2024-04-25T13:51:43.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f92080b5740 con 0x7f91f4004d50 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f91f400bad0 con 0x7f92080b5330 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920d09d640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91f4003f40 0x7f91f4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920e09f640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4004d50 0x7f91f4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920d09d640 1 -- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92080b5330 msgr2=0x7f91f40039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920d09d640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92080b5330 0x7f91f40039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920d09d640 1 -- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4004d50 msgr2=0x7f91f4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920d09d640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4004d50 0x7f91f4005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920d09d640 1 -- 172.21.15.80:0/1046830590 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91f4003530 con 0x7f91f4003f40 2024-04-25T13:51:43.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920e09f640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4004d50 0x7f91f4005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:43.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920d09d640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91f4003f40 0x7f91f4004810 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f9208071350 tx=0x7f91f801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b1319e999e1790 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:43.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f800c0e0 con 0x7f91f4003f40 2024-04-25T13:51:43.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f80a9020 con 0x7f91f4003f40 2024-04-25T13:51:43.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920c89c640 1 -- 172.21.15.80:0/1046830590 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8005b20 con 0x7f91f4003f40 2024-04-25T13:51:43.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920fb02640 1 -- 172.21.15.80:0/1046830590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91f4003f40 msgr2=0x7f91f4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920fb02640 1 --2- 172.21.15.80:0/1046830590 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f91f4003f40 0x7f91f4004810 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f9208071350 tx=0x7f91f801cc40 comp rx=0 tx=0).stop 2024-04-25T13:51:43.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920e09f640 1 -- 172.21.15.80:0/1046830590 reap_dead start 2024-04-25T13:51:43.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920fb02640 1 -- 172.21.15.80:0/1046830590 shutdown_connections 2024-04-25T13:51:43.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920fb02640 1 -- 172.21.15.80:0/1046830590 >> 172.21.15.80:0/1046830590 conn(0x7f92080b4160 msgr2=0x7f92080d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:43.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920fb02640 1 -- 172.21.15.80:0/1046830590 shutdown_connections 2024-04-25T13:51:43.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.143+0000 7f920fb02640 1 -- 172.21.15.80:0/1046830590 wait complete. 2024-04-25T13:51:43.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 Processor -- start 2024-04-25T13:51:43.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 -- start start 2024-04-25T13:51:43.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4003f40 0x7f9208146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9208158c30 0x7f920815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9208159040 0x7f920815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f92080b0cd0 con 0x7f9208158c30 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f920806e890 con 0x7f91f4003f40 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92080b0e50 con 0x7f9208159040 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9208158c30 0x7f920815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9208158c30 0x7f920815b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41368/0 (socket says 172.21.15.80:41368) 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 -- 172.21.15.80:0/3713763925 learned_addr learned my addr 172.21.15.80:0/3713763925 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 -- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9208159040 msgr2=0x7f920815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9208159040 0x7f920815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 -- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4003f40 msgr2=0x7f9208146900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4003f40 0x7f9208146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 -- 172.21.15.80:0/3713763925 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9208146e40 con 0x7f9208158c30 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920d09d640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9208158c30 0x7f920815b3e0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f91f8000c10 tx=0x7f91f4005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:43.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f91fe7fc640 1 -- 172.21.15.80:0/3713763925 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8005900 con 0x7f9208158c30 2024-04-25T13:51:43.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f91fe7fc640 1 -- 172.21.15.80:0/3713763925 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f91f80b2070 con 0x7f9208158c30 2024-04-25T13:51:43.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92081596d0 con 0x7f9208158c30 2024-04-25T13:51:43.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f91fe7fc640 1 -- 172.21.15.80:0/3713763925 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91f8005d20 con 0x7f9208158c30 2024-04-25T13:51:43.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.147+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9208159c90 con 0x7f9208158c30 2024-04-25T13:51:43.145 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.151+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920806a3a0 con 0x7f9208158c30 2024-04-25T13:51:43.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.151+0000 7f91fe7fc640 1 -- 172.21.15.80:0/3713763925 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f91f80a5720 con 0x7f9208158c30 2024-04-25T13:51:43.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.151+0000 7f91fe7fc640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91ec06f400 0x7f91ec071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.151+0000 7f91fe7fc640 1 -- 172.21.15.80:0/3713763925 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4943+0+0 (secure 0 0 0) 0x7f91f8128410 con 0x7f9208158c30 2024-04-25T13:51:43.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.151+0000 7f920d89e640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91ec06f400 0x7f91ec071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.151+0000 7f920d89e640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91ec06f400 0x7f91ec071910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9204006e90 tx=0x7f9204006de0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:43.150 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.155+0000 7f91fe7fc640 1 -- 172.21.15.80:0/3713763925 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9208146e40 con 0x7f9208158c30 2024-04-25T13:51:43.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9208159460 con 0x7f9208158c30 2024-04-25T13:51:43.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f91fe7fc640 1 -- 172.21.15.80:0/3713763925 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v179) ==== 93+0+10 (secure 0 0 0) 0x7f92081624b0 con 0x7f9208158c30 2024-04-25T13:51:43.313 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91ec06f400 msgr2=0x7f91ec071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91ec06f400 0x7f91ec071910 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f9204006e90 tx=0x7f9204006de0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9208158c30 msgr2=0x7f920815b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9208158c30 0x7f920815b3e0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f91f8000c10 tx=0x7f91f4005840 comp rx=0 tx=0).stop 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 shutdown_connections 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f91ec06f400 0x7f91ec071910 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9208159040 0x7f920815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9208158c30 0x7f920815b3e0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 --2- 172.21.15.80:0/3713763925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91f4003f40 0x7f9208146900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 >> 172.21.15.80:0/3713763925 conn(0x7f92080b4160 msgr2=0x7f92080b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:43.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 shutdown_connections 2024-04-25T13:51:43.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.319+0000 7f920fb02640 1 -- 172.21.15.80:0/3713763925 wait complete. 2024-04-25T13:51:43.335 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-04-25T13:51:43.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82edaae640 1 Processor -- start 2024-04-25T13:51:43.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82edaae640 1 -- start start 2024-04-25T13:51:43.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82edaae640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f82e80d59b0 0x7f82e80d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82edaae640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e80d6300 con 0x7f82e80d2e80 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82edaae640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82e8161060 con 0x7f82e80d4330 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82edaae640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f82e8161340 con 0x7f82e80d59b0 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e77fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f82e80d2e80 0x7f82e81607a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48384/0 (socket says 172.21.15.80:48384) 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e77fe640 1 -- 172.21.15.80:0/612742784 learned_addr learned my addr 172.21.15.80:0/612742784 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e6ffd640 1 --2- 172.21.15.80:0/612742784 >> v2:172.21.15.87:3300/0 conn(0x7f82e80d59b0 0x7f82e80d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3887750084 0 0) 0x7f82e00014a0 con 0x7f82e80d2e80 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82d4003530 con 0x7f82e80d2e80 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2099921281 0 0) 0x7f82d4003530 con 0x7f82e80d2e80 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 >> v2:172.21.15.87:3300/0 conn(0x7f82e80d59b0 msgr2=0x7f82e80d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 --2- 172.21.15.80:0/612742784 >> v2:172.21.15.87:3300/0 conn(0x7f82e80d59b0 0x7f82e80d5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 >> v1:172.21.15.87:6789/0 conn(0x7f82e80d4330 legacy=0x7f82e80d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e80d2680 con 0x7f82e80d2e80 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f82e0001bf0 con 0x7f82e80d2e80 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 >> v1:172.21.15.80:6789/0 conn(0x7f82e80d2e80 legacy=0x7f82e81607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d40039c0 0x7f82d4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82d4004810 0x7f82d4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4005260 0x7f82d400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f82e8161340 con 0x7f82d4004810 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82d400c1f0 con 0x7f82d40039c0 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82d400c3f0 con 0x7f82d4005260 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e6ffd640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82d4004810 0x7f82d4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e77fe640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d40039c0 0x7f82d4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e7fff640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4005260 0x7f82d400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e6ffd640 1 -- 172.21.15.80:0/612742784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4005260 msgr2=0x7f82d400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:43.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e6ffd640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4005260 0x7f82d400b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e6ffd640 1 -- 172.21.15.80:0/612742784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d40039c0 msgr2=0x7f82d4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:43.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e6ffd640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82d40039c0 0x7f82d4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e6ffd640 1 -- 172.21.15.80:0/612742784 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d4003530 con 0x7f82d4004810 2024-04-25T13:51:43.487 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.487+0000 7f82e7fff640 1 -- 172.21.15.80:0/612742784 reap_dead start 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82e6ffd640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82d4004810 0x7f82d4004c20 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009ab0 tx=0x7f82d001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32e66ba29fe3876 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00b2070 con 0x7f82d4004810 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d00a5630 con 0x7f82d4004810 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82e67fc640 1 -- 172.21.15.80:0/612742784 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00096b0 con 0x7f82d4004810 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82edaae640 1 -- 172.21.15.80:0/612742784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82d4004810 msgr2=0x7f82d4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82edaae640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82d4004810 0x7f82d4004c20 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f82d0009ab0 tx=0x7f82d001ca40 comp rx=0 tx=0).stop 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82edaae640 1 -- 172.21.15.80:0/612742784 shutdown_connections 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82edaae640 1 --2- 172.21.15.80:0/612742784 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82d4004810 0x7f82d4004c20 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.488 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82edaae640 1 -- 172.21.15.80:0/612742784 >> 172.21.15.80:0/612742784 conn(0x7f82e80737a0 msgr2=0x7f82e806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:43.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82edaae640 1 -- 172.21.15.80:0/612742784 shutdown_connections 2024-04-25T13:51:43.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.491+0000 7f82edaae640 1 -- 172.21.15.80:0/612742784 wait complete. 2024-04-25T13:51:43.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 Processor -- start 2024-04-25T13:51:43.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 -- start start 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4004810 0x7f82e80cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82e81fe100 0x7f82e814ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e81fe510 0x7f82e814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f82e815df60 con 0x7f82e81fe100 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f82e80d34a0 con 0x7f82e81fe510 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f82e815e0e0 con 0x7f82d4004810 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82e81fe100 0x7f82e814ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82e81fe100 0x7f82e814ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41388/0 (socket says 172.21.15.80:41388) 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 -- 172.21.15.80:0/1941609683 learned_addr learned my addr 172.21.15.80:0/1941609683 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e77fe640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4004810 0x7f82e80cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.492 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e7fff640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e81fe510 0x7f82e814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 -- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4004810 msgr2=0x7f82e80cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4004810 0x7f82e80cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 -- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e81fe510 msgr2=0x7f82e814efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e81fe510 0x7f82e814efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 -- 172.21.15.80:0/1941609683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e80cc090 con 0x7f82e81fe100 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e7fff640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e81fe510 0x7f82e814efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82e6ffd640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82e81fe100 0x7f82e814ea80 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f82d00b0040 tx=0x7f82d00197e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82dffff640 1 -- 172.21.15.80:0/1941609683 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00b2040 con 0x7f82e81fe100 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82dffff640 1 -- 172.21.15.80:0/1941609683 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82d000f000 con 0x7f82e81fe100 2024-04-25T13:51:43.493 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e80cc370 con 0x7f82e81fe100 2024-04-25T13:51:43.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82dffff640 1 -- 172.21.15.80:0/1941609683 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d00096b0 con 0x7f82e81fe100 2024-04-25T13:51:43.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.495+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e80ce670 con 0x7f82e81fe100 2024-04-25T13:51:43.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.499+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82e806a3a0 con 0x7f82e81fe100 2024-04-25T13:51:43.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.499+0000 7f82dffff640 1 -- 172.21.15.80:0/1941609683 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f82d00bf030 con 0x7f82e81fe100 2024-04-25T13:51:43.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.499+0000 7f82dffff640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f82c806f400 0x7f82c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:43.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.499+0000 7f82dffff640 1 -- 172.21.15.80:0/1941609683 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4943+0+0 (secure 0 0 0) 0x7f82d0128b10 con 0x7f82e81fe100 2024-04-25T13:51:43.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.499+0000 7f82e77fe640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f82c806f400 0x7f82c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:43.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.499+0000 7f82e77fe640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f82c806f400 0x7f82c8071910 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f82e0000f90 tx=0x7f82e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:43.498 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.503+0000 7f82dffff640 1 -- 172.21.15.80:0/1941609683 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f82d00c6060 con 0x7f82e81fe100 2024-04-25T13:51:43.699 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:43.703+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) -- 0x7f82e80cbe20 con 0x7f82e81fe100 2024-04-25T13:51:44.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.615+0000 7f82dffff640 1 -- 172.21.15.80:0/1941609683 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v547) ==== 138+0+0 (secure 0 0 0) 0x7f82d0016780 con 0x7f82e81fe100 2024-04-25T13:51:44.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f82c806f400 msgr2=0x7f82c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f82c806f400 0x7f82c8071910 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f82e0000f90 tx=0x7f82e0009040 comp rx=0 tx=0).stop 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82e81fe100 msgr2=0x7f82e814ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82e81fe100 0x7f82e814ea80 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f82d00b0040 tx=0x7f82d00197e0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 shutdown_connections 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f82c806f400 0x7f82c8071910 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82e81fe510 0x7f82e814efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f82e81fe100 0x7f82e814ea80 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 --2- 172.21.15.80:0/1941609683 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f82d4004810 0x7f82e80cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 >> 172.21.15.80:0/1941609683 conn(0x7f82e80737a0 msgr2=0x7f82e815f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 shutdown_connections 2024-04-25T13:51:44.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.619+0000 7f82edaae640 1 -- 172.21.15.80:0/1941609683 wait complete. 2024-04-25T13:51:44.640 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T13:51:44.797 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25febf640 1 Processor -- start 2024-04-25T13:51:44.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25febf640 1 -- start start 2024-04-25T13:51:44.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25febf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc25806edb0 0x7fc25806f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:44.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25febf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc258161b60 0x7fc258161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:44.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25febf640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2581624b0 con 0x7fc2580b5330 2024-04-25T13:51:44.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25febf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc25806f700 con 0x7fc25806edb0 2024-04-25T13:51:44.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25febf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc2581629b0 con 0x7fc258161b60 2024-04-25T13:51:44.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25dc5b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc2580b5330 0x7fc2580d0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48398/0 (socket says 172.21.15.80:48398) 2024-04-25T13:51:44.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25dc5b640 1 -- 172.21.15.80:0/1943456396 learned_addr learned my addr 172.21.15.80:0/1943456396 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:44.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.803+0000 7fc25d45a640 1 --2- 172.21.15.80:0/1943456396 >> v2:172.21.15.80:3300/0 conn(0x7fc25806edb0 0x7fc25806f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:44.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25e45c640 1 --2- 172.21.15.80:0/1943456396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc258161b60 0x7fc258161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:44.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25cc59640 1 -- 172.21.15.80:0/1943456396 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778846084 0 0) 0x7fc254001120 con 0x7fc2580b5330 2024-04-25T13:51:44.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25cc59640 1 -- 172.21.15.80:0/1943456396 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc244003530 con 0x7fc2580b5330 2024-04-25T13:51:44.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25d45a640 1 -- 172.21.15.80:0/1943456396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc258161b60 msgr2=0x7fc258161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:44.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25d45a640 1 --2- 172.21.15.80:0/1943456396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc258161b60 0x7fc258161f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25d45a640 1 -- 172.21.15.80:0/1943456396 >> v1:172.21.15.80:6789/0 conn(0x7fc2580b5330 legacy=0x7fc2580d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:44.801 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25d45a640 1 -- 172.21.15.80:0/1943456396 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2580b5740 con 0x7fc25806edb0 2024-04-25T13:51:44.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25d45a640 1 --2- 172.21.15.80:0/1943456396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc25806f1c0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fc2480097c0 tx=0x7fc24801c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554bfee18b666fc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:44.802 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25cc59640 1 -- 172.21.15.80:0/1943456396 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2480b1070 con 0x7fc25806edb0 2024-04-25T13:51:44.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25cc59640 1 -- 172.21.15.80:0/1943456396 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2480a8050 con 0x7fc25806edb0 2024-04-25T13:51:44.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25cc59640 1 -- 172.21.15.80:0/1943456396 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2480ac860 con 0x7fc25806edb0 2024-04-25T13:51:44.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25febf640 1 -- 172.21.15.80:0/1943456396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 msgr2=0x7fc25806f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:44.803 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25febf640 1 --2- 172.21.15.80:0/1943456396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc25806f1c0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fc2480097c0 tx=0x7fc24801c3e0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25febf640 1 -- 172.21.15.80:0/1943456396 shutdown_connections 2024-04-25T13:51:44.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25febf640 1 --2- 172.21.15.80:0/1943456396 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc258161b60 0x7fc258161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25febf640 1 --2- 172.21.15.80:0/1943456396 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc25806f1c0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25febf640 1 -- 172.21.15.80:0/1943456396 >> 172.21.15.80:0/1943456396 conn(0x7fc2580b4160 msgr2=0x7fc2580d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:44.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25febf640 1 -- 172.21.15.80:0/1943456396 shutdown_connections 2024-04-25T13:51:44.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.807+0000 7fc25febf640 1 -- 172.21.15.80:0/1943456396 wait complete. 2024-04-25T13:51:44.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 Processor -- start 2024-04-25T13:51:44.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 -- start start 2024-04-25T13:51:44.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc2580ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:44.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2580b5330 0x7fc258157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:44.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc258161b60 0x7fc258160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:44.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc25806e890 con 0x7fc25806edb0 2024-04-25T13:51:44.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc25806e5b0 con 0x7fc258161b60 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc2580b0cd0 con 0x7fc2580b5330 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc2580ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc2580ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41414/0 (socket says 172.21.15.80:41414) 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 -- 172.21.15.80:0/3284853373 learned_addr learned my addr 172.21.15.80:0/3284853373 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25d45a640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2580b5330 0x7fc258157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 -- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2580b5330 msgr2=0x7fc258157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25e45c640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc258161b60 0x7fc258160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2580b5330 0x7fc258157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 -- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc258161b60 msgr2=0x7fc258160590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc258161b60 0x7fc258160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 -- 172.21.15.80:0/3284853373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc25815f3b0 con 0x7fc25806edb0 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25d45a640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2580b5330 0x7fc258157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25dc5b640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc2580ab4a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fc25400c3c0 tx=0x7fc25400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc24e7fc640 1 -- 172.21.15.80:0/3284853373 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254017070 con 0x7fc25806edb0 2024-04-25T13:51:44.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc24e7fc640 1 -- 172.21.15.80:0/3284853373 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc254005cf0 con 0x7fc25806edb0 2024-04-25T13:51:44.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc24e7fc640 1 -- 172.21.15.80:0/3284853373 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc254007310 con 0x7fc25806edb0 2024-04-25T13:51:44.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc25815b5b0 con 0x7fc25806edb0 2024-04-25T13:51:44.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.811+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc25815bb70 con 0x7fc25806edb0 2024-04-25T13:51:44.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.815+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc25806a3a0 con 0x7fc25806edb0 2024-04-25T13:51:44.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.815+0000 7fc24e7fc640 1 -- 172.21.15.80:0/3284853373 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc25401a070 con 0x7fc25806edb0 2024-04-25T13:51:44.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.815+0000 7fc24e7fc640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23c06f400 0x7fc23c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:44.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.815+0000 7fc24e7fc640 1 -- 172.21.15.80:0/3284853373 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4943+0+0 (secure 0 0 0) 0x7fc25408e3e0 con 0x7fc25806edb0 2024-04-25T13:51:44.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.815+0000 7fc25d45a640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23c06f400 0x7fc23c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:44.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.815+0000 7fc25d45a640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23c06f400 0x7fc23c071910 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc2480ae000 tx=0x7fc248008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:44.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.819+0000 7fc24e7fc640 1 -- 172.21.15.80:0/3284853373 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc2581624b0 con 0x7fc25806edb0 2024-04-25T13:51:44.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.979+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fc2580b48c0 con 0x7fc25806edb0 2024-04-25T13:51:44.982 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc24e7fc640 1 -- 172.21.15.80:0/3284853373 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) ==== 122+0+0 (secure 0 0 0) 0x7fc25405b5d0 con 0x7fc25806edb0 2024-04-25T13:51:44.983 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23c06f400 msgr2=0x7fc23c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23c06f400 0x7fc23c071910 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc2480ae000 tx=0x7fc248008040 comp rx=0 tx=0).stop 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 msgr2=0x7fc2580ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc2580ab4a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fc25400c3c0 tx=0x7fc25400c880 comp rx=0 tx=0).stop 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 shutdown_connections 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc23c06f400 0x7fc23c071910 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc258161b60 0x7fc258160590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc2580b5330 0x7fc258157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 --2- 172.21.15.80:0/3284853373 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc25806edb0 0x7fc2580ab4a0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 >> 172.21.15.80:0/3284853373 conn(0x7fc2580b4160 msgr2=0x7fc2580b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 shutdown_connections 2024-04-25T13:51:44.984 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:44.987+0000 7fc25febf640 1 -- 172.21.15.80:0/3284853373 wait complete. 2024-04-25T13:51:45.003 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:51:45.003 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:51:45.049 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:51:45.091 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:51:45.091 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:51:45.091 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:45.097 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:51:45.097 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:51:45.097 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:45.137 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:51:45.138 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:51:45.138 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:45.144 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:51:45.144 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:51:45.144 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:51:45.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:51:45.188 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T13:51:45.188 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:45.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.339+0000 7f7a55d3c640 1 Processor -- start 2024-04-25T13:51:45.336 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.339+0000 7f7a55d3c640 1 -- start start 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a55d3c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f7a5006f5a0 0x7f7a50160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a55d3c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a50161070 0x7f7a50161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a55d3c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7a5006fcf0 con 0x7f7a500b4bd0 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a55d3c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f7a501619c0 con 0x7f7a5006f5a0 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a55d3c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7a50073f10 con 0x7f7a50161070 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4ffff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7a500b4bd0 0x7f7a500b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48414/0 (socket says 172.21.15.80:48414) 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4ffff640 1 -- 172.21.15.80:0/3251734485 learned_addr learned my addr 172.21.15.80:0/3251734485 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4f7fe640 1 --2- 172.21.15.80:0/3251734485 >> v2:172.21.15.87:3300/0 conn(0x7f7a5006f5a0 0x7f7a50160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.338 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4effd640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a50161070 0x7f7a50161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2002203373 0 0) 0x7f7a5006fcf0 con 0x7f7a500b4bd0 2024-04-25T13:51:45.339 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7a38003530 con 0x7f7a500b4bd0 2024-04-25T13:51:45.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1870100538 0 0) 0x7f7a38003530 con 0x7f7a500b4bd0 2024-04-25T13:51:45.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a50161070 msgr2=0x7f7a50161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:45.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a50161070 0x7f7a50161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 >> v2:172.21.15.87:3300/0 conn(0x7f7a5006f5a0 msgr2=0x7f7a50160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:45.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 --2- 172.21.15.80:0/3251734485 >> v2:172.21.15.87:3300/0 conn(0x7f7a5006f5a0 0x7f7a50160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.340 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a500b0c60 con 0x7f7a500b4bd0 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7a48001a00 con 0x7f7a500b4bd0 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 >> v1:172.21.15.80:6789/0 conn(0x7f7a500b4bd0 legacy=0x7f7a500b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a380039f0 0x7f7a38003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a50161070 0x7f7a38004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a38004d50 0x7f7a38005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7a501619c0 con 0x7f7a380039f0 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50073f10 con 0x7f7a38004d50 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7a3800ba80 con 0x7f7a50161070 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4f7fe640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a380039f0 0x7f7a38003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4f7fe640 1 -- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a50161070 msgr2=0x7f7a38004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4f7fe640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a50161070 0x7f7a38004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4f7fe640 1 -- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a38004d50 msgr2=0x7f7a38005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4f7fe640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a38004d50 0x7f7a38005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.341 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.343+0000 7f7a4f7fe640 1 -- 172.21.15.80:0/3251734485 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a38003530 con 0x7f7a380039f0 2024-04-25T13:51:45.342 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a4f7fe640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a380039f0 0x7f7a38003e60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009670 tx=0x7f7a4001eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4afb19362aaf83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400b4070 con 0x7f7a380039f0 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a400a7690 con 0x7f7a380039f0 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a4e7fc640 1 -- 172.21.15.80:0/3251734485 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400096b0 con 0x7f7a380039f0 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a55d3c640 1 -- 172.21.15.80:0/3251734485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a380039f0 msgr2=0x7f7a38003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a55d3c640 1 --2- 172.21.15.80:0/3251734485 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a380039f0 0x7f7a38003e60 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f7a40009670 tx=0x7f7a4001eca0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a4ffff640 1 -- 172.21.15.80:0/3251734485 reap_dead start 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a55d3c640 1 -- 172.21.15.80:0/3251734485 shutdown_connections 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a55d3c640 1 -- 172.21.15.80:0/3251734485 >> 172.21.15.80:0/3251734485 conn(0x7f7a5006d080 msgr2=0x7f7a500b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a55d3c640 1 -- 172.21.15.80:0/3251734485 shutdown_connections 2024-04-25T13:51:45.343 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.347+0000 7f7a55d3c640 1 -- 172.21.15.80:0/3251734485 wait complete. 2024-04-25T13:51:45.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 Processor -- start 2024-04-25T13:51:45.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 -- start start 2024-04-25T13:51:45.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a50160440 0x7f7a50155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a501480a0 0x7f7a5015aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a501484b0 0x7f7a5015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50161600 con 0x7f7a50160440 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7a50161480 con 0x7f7a501484b0 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7a50161780 con 0x7f7a501480a0 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a50160440 0x7f7a50155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a50160440 0x7f7a50155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41434/0 (socket says 172.21.15.80:41434) 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 -- 172.21.15.80:0/1974540139 learned_addr learned my addr 172.21.15.80:0/1974540139 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4effd640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a501480a0 0x7f7a5015aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4ffff640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a501484b0 0x7f7a5015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 -- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a501480a0 msgr2=0x7f7a5015aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a501480a0 0x7f7a5015aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 -- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a501484b0 msgr2=0x7f7a5015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a501484b0 0x7f7a5015afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 -- 172.21.15.80:0/1974540139 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a50156520 con 0x7f7a50160440 2024-04-25T13:51:45.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a4f7fe640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a50160440 0x7f7a50155fe0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f7a400b2040 tx=0x7f7a4001ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:45.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a37fff640 1 -- 172.21.15.80:0/1974540139 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400b4040 con 0x7f7a50160440 2024-04-25T13:51:45.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a37fff640 1 -- 172.21.15.80:0/1974540139 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a400bf070 con 0x7f7a50160440 2024-04-25T13:51:45.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a37fff640 1 -- 172.21.15.80:0/1974540139 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a400096b0 con 0x7f7a50160440 2024-04-25T13:51:45.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a50160850 con 0x7f7a50160440 2024-04-25T13:51:45.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a50156a80 con 0x7f7a50160440 2024-04-25T13:51:45.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.351+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a5006a3a0 con 0x7f7a50160440 2024-04-25T13:51:45.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.355+0000 7f7a37fff640 1 -- 172.21.15.80:0/1974540139 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7a400ba650 con 0x7f7a50160440 2024-04-25T13:51:45.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.355+0000 7f7a37fff640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7a3c06f400 0x7f7a3c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.355+0000 7f7a37fff640 1 -- 172.21.15.80:0/1974540139 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4943+0+0 (secure 0 0 0) 0x7f7a4012b540 con 0x7f7a50160440 2024-04-25T13:51:45.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.355+0000 7f7a4effd640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7a3c06f400 0x7f7a3c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.355+0000 7f7a4effd640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7a3c06f400 0x7f7a3c071910 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7a5015fa90 tx=0x7f7a44008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:45.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.359+0000 7f7a37fff640 1 -- 172.21.15.80:0/1974540139 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7a50156520 con 0x7f7a50160440 2024-04-25T13:51:45.547 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.551+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a500739c0 con 0x7f7a50160440 2024-04-25T13:51:45.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a37fff640 1 -- 172.21.15.80:0/1974540139 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+3860 (secure 0 0 0) 0x7f7a400189b0 con 0x7f7a50160440 2024-04-25T13:51:45.548 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:45.548 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":83,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6961,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6836/29039900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":29039900},{"type":"v1","addr":"172.21.15.87:6836","nonce":29039900}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":7204,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/870505874","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6835","nonce":870505874},{"type":"v1","addr":"172.21.15.87:6837","nonce":870505874}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83},{"gid":8799,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/1466426643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":1466426643},{"type":"v1","addr":"172.21.15.80:6833","nonce":1466426643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77},{"gid":8802,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/1170719891","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":1170719891},{"type":"v1","addr":"172.21.15.80:6835","nonce":1170719891}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":77}],"filesystems":[]} 2024-04-25T13:51:45.549 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 83 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7a3c06f400 msgr2=0x7f7a3c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7a3c06f400 0x7f7a3c071910 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f7a5015fa90 tx=0x7f7a44008040 comp rx=0 tx=0).stop 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a50160440 msgr2=0x7f7a50155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a50160440 0x7f7a50155fe0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f7a400b2040 tx=0x7f7a4001ba40 comp rx=0 tx=0).stop 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 shutdown_connections 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7a3c06f400 0x7f7a3c071910 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7a501484b0 0x7f7a5015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7a501480a0 0x7f7a5015aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 --2- 172.21.15.80:0/1974540139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7a50160440 0x7f7a50155fe0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 >> 172.21.15.80:0/1974540139 conn(0x7f7a5006d080 msgr2=0x7f7a500b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 shutdown_connections 2024-04-25T13:51:45.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.555+0000 7f7a55d3c640 1 -- 172.21.15.80:0/1974540139 wait complete. 2024-04-25T13:51:45.571 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T13:51:45.571 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T13:51:45.571 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:51:45.571 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:51:45.571 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T13:51:45.571 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T13:51:45.571 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:51:45.572 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:51:45.572 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T13:51:45.572 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T13:51:45.572 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:51:45.572 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:51:45.572 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T13:51:45.572 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T13:51:45.573 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:51:45.573 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:51:45.585 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:51:45.585 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T13:51:45.586 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:51:45.586 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T13:51:45.598 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:51:45.598 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T13:51:45.599 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:51:45.599 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T13:51:45.633 INFO:tasks.ceph.mds.c:Started 2024-04-25T13:51:45.633 DEBUG:teuthology.parallel:result is None 2024-04-25T13:51:45.634 INFO:tasks.ceph.mds.a:Started 2024-04-25T13:51:45.634 DEBUG:teuthology.parallel:result is None 2024-04-25T13:51:45.643 INFO:tasks.ceph.mds.d:Started 2024-04-25T13:51:45.643 DEBUG:teuthology.parallel:result is None 2024-04-25T13:51:45.643 INFO:tasks.ceph.mds.b:Started 2024-04-25T13:51:45.644 DEBUG:teuthology.parallel:result is None 2024-04-25T13:51:45.644 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T13:51:45.718 INFO:tasks.ceph.mds.c.smithi080.stdout:starting mds.c at 2024-04-25T13:51:45.726 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:51:45.743 INFO:tasks.ceph.mds.b.smithi087.stdout:starting mds.b at 2024-04-25T13:51:45.745 INFO:tasks.ceph.mds.d.smithi087.stdout:starting mds.d at 2024-04-25T13:51:45.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.783+0000 7f8d524d6640 1 Processor -- start 2024-04-25T13:51:45.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.783+0000 7f8d524d6640 1 -- start start 2024-04-25T13:51:45.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.783+0000 7f8d524d6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0b5860 0x7f8d4c0b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.783+0000 7f8d524d6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d4c0b61b0 con 0x7f8d4c0cf000 2024-04-25T13:51:45.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.783+0000 7f8d524d6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d4c0700e0 con 0x7f8d4c06c810 2024-04-25T13:51:45.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.783+0000 7f8d524d6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f8d4c0cf410 con 0x7f8d4c0b5860 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.783+0000 7f8d4bfff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8d4c0cf000 0x7f8d4c06f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48440/0 (socket says 172.21.15.80:48440) 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.783+0000 7f8d4bfff640 1 -- 172.21.15.80:0/1932761244 learned_addr learned my addr 172.21.15.80:0/1932761244 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4b7fe640 1 --2- 172.21.15.80:0/1932761244 >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0b5860 0x7f8d4c0b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4007437628 0 0) 0x7f8d3c0014a0 con 0x7f8d4c0cf000 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8d34003530 con 0x7f8d4c0cf000 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1244468536 0 0) 0x7f8d34003530 con 0x7f8d4c0cf000 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0b5860 msgr2=0x7f8d4c0b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 --2- 172.21.15.80:0/1932761244 >> v2:172.21.15.87:3300/0 conn(0x7f8d4c0b5860 0x7f8d4c0b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 >> v1:172.21.15.87:6789/0 conn(0x7f8d4c06c810 legacy=0x7f8d4c06cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c0703c0 con 0x7f8d4c0cf000 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f8d3c001bd0 con 0x7f8d4c0cf000 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 >> v1:172.21.15.80:6789/0 conn(0x7f8d4c0cf000 legacy=0x7f8d4c06f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d340039c0 0x7f8d34003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d34004810 0x7f8d34004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d34005280 0x7f8d3400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c0cf410 con 0x7f8d340039c0 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4bfff640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d340039c0 0x7f8d34003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3400c210 con 0x7f8d34005280 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3400c410 con 0x7f8d34004810 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4b7fe640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d34004810 0x7f8d34004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d50a73640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d34005280 0x7f8d3400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4bfff640 1 -- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d34004810 msgr2=0x7f8d34004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4bfff640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d34004810 0x7f8d34004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4bfff640 1 -- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d34005280 msgr2=0x7f8d3400b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4bfff640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d34005280 0x7f8d3400b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4bfff640 1 -- 172.21.15.80:0/1932761244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d34003530 con 0x7f8d340039c0 2024-04-25T13:51:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d50a73640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d34005280 0x7f8d3400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d50a73640 1 -- 172.21.15.80:0/1932761244 reap_dead start 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4bfff640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d340039c0 0x7f8d34003e50 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00aa20 tx=0x7f8d3c01d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8596fa0edf9c0843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c00a640 con 0x7f8d340039c0 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d3c0ad650 con 0x7f8d340039c0 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d4affd640 1 -- 172.21.15.80:0/1932761244 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0ada40 con 0x7f8d340039c0 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d524d6640 1 -- 172.21.15.80:0/1932761244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d340039c0 msgr2=0x7f8d34003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d524d6640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d340039c0 0x7f8d34003e50 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c00aa20 tx=0x7f8d3c01d330 comp rx=0 tx=0).stop 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d524d6640 1 -- 172.21.15.80:0/1932761244 shutdown_connections 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d524d6640 1 --2- 172.21.15.80:0/1932761244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d340039c0 0x7f8d34003e50 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d524d6640 1 -- 172.21.15.80:0/1932761244 >> 172.21.15.80:0/1932761244 conn(0x7f8d4c0d28f0 msgr2=0x7f8d4c0d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d524d6640 1 -- 172.21.15.80:0/1932761244 shutdown_connections 2024-04-25T13:51:45.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.787+0000 7f8d524d6640 1 -- 172.21.15.80:0/1932761244 wait complete. 2024-04-25T13:51:45.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 Processor -- start 2024-04-25T13:51:45.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 -- start start 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d340039c0 0x7f8d4c15c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c1f9a30 0x7f8d4c1f9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d4c1e88c0 0x7f8d4c1e8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c160bd0 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8d4c160a50 con 0x7f8d340039c0 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8d4c160d50 con 0x7f8d4c1f9a30 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d4c1e88c0 0x7f8d4c1e8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d4c1e88c0 0x7f8d4c1e8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41478/0 (socket says 172.21.15.80:41478) 2024-04-25T13:51:45.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 -- 172.21.15.80:0/1976154667 learned_addr learned my addr 172.21.15.80:0/1976154667 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d4bfff640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d340039c0 0x7f8d4c15c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d4b7fe640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c1f9a30 0x7f8d4c1f9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 -- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c1f9a30 msgr2=0x7f8d4c1f9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c1f9a30 0x7f8d4c1f9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 -- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d340039c0 msgr2=0x7f8d4c15c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d340039c0 0x7f8d4c15c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 -- 172.21.15.80:0/1976154667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d4c1e9210 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d50a73640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d4c1e88c0 0x7f8d4c1e8cd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0b1e90 tx=0x7f8d4400e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d48ff9640 1 -- 172.21.15.80:0/1976154667 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44019070 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d48ff9640 1 -- 172.21.15.80:0/1976154667 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d440050d0 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d4c15cc80 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d48ff9640 1 -- 172.21.15.80:0/1976154667 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d44003600 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.791+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d4c1fad30 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.795+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d4c1fa500 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.795+0000 7f8d48ff9640 1 -- 172.21.15.80:0/1976154667 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8d4401c070 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.795+0000 7f8d48ff9640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8d3806f400 0x7f8d38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:45.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.795+0000 7f8d48ff9640 1 -- 172.21.15.80:0/1976154667 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(179..179 src has 1..179) ==== 4943+0+0 (secure 0 0 0) 0x7f8d4408e140 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.795+0000 7f8d4bfff640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8d3806f400 0x7f8d38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:45.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.795+0000 7f8d4bfff640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8d3806f400 0x7f8d38071910 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0ad000 tx=0x7f8d4c15d460 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:45.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.799+0000 7f8d48ff9640 1 -- 172.21.15.80:0/1976154667 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8d4c1e9210 con 0x7f8d4c1e88c0 2024-04-25T13:51:45.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:45.955+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8d4c146690 con 0x7f8d4c1e88c0 2024-04-25T13:51:46.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d48ff9640 1 -- 172.21.15.80:0/1976154667 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v181) ==== 97+0+0 (secure 0 0 0) 0x7f8d4405ad70 con 0x7f8d4c1e88c0 2024-04-25T13:51:46.637 INFO:teuthology.orchestra.run.smithi080.stderr: removed all blocklist entries 2024-04-25T13:51:46.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8d3806f400 msgr2=0x7f8d38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8d3806f400 0x7f8d38071910 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c0ad000 tx=0x7f8d4c15d460 comp rx=0 tx=0).stop 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d4c1e88c0 msgr2=0x7f8d4c1e8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d4c1e88c0 0x7f8d4c1e8cd0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f8d4c0b1e90 tx=0x7f8d4400e760 comp rx=0 tx=0).stop 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 shutdown_connections 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8d3806f400 0x7f8d38071910 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8d4c1e88c0 0x7f8d4c1e8cd0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8d4c1f9a30 0x7f8d4c1f9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 --2- 172.21.15.80:0/1976154667 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8d340039c0 0x7f8d4c15c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 >> 172.21.15.80:0/1976154667 conn(0x7f8d4c0d28f0 msgr2=0x7f8d4c068590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 shutdown_connections 2024-04-25T13:51:46.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.643+0000 7f8d524d6640 1 -- 172.21.15.80:0/1976154667 wait complete. 2024-04-25T13:51:46.663 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T13:51:46.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.807+0000 7faca1e43640 1 Processor -- start 2024-04-25T13:51:46.804 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.807+0000 7faca1e43640 1 -- start start 2024-04-25T13:51:46.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7faca1e43640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fac9c06f5a0 0x7fac9c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7faca1e43640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fac9c0733b0 0x7fac9c161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7faca1e43640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac9c06fcf0 con 0x7fac9c161070 2024-04-25T13:51:46.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7faca1e43640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c1619c0 con 0x7fac9c06f5a0 2024-04-25T13:51:46.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7faca1e43640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fac9c073f10 con 0x7fac9c0733b0 2024-04-25T13:51:46.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fac9c06f5a0 0x7fac9c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9b7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fac9c06f5a0 0x7fac9c160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41492/0 (socket says 172.21.15.80:41492) 2024-04-25T13:51:46.806 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9b7fe640 1 -- 172.21.15.80:0/116364391 learned_addr learned my addr 172.21.15.80:0/116364391 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:46.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9bfff640 1 --2- 172.21.15.80:0/116364391 >> v2:172.21.15.87:3300/0 conn(0x7fac9c0733b0 0x7fac9c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9b7fe640 1 -- 172.21.15.80:0/116364391 >> v2:172.21.15.87:3300/0 conn(0x7fac9c0733b0 msgr2=0x7fac9c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9b7fe640 1 --2- 172.21.15.80:0/116364391 >> v2:172.21.15.87:3300/0 conn(0x7fac9c0733b0 0x7fac9c161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9b7fe640 1 -- 172.21.15.80:0/116364391 >> v1:172.21.15.87:6789/0 conn(0x7fac9c161070 legacy=0x7fac9c0b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:46.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9b7fe640 1 -- 172.21.15.80:0/116364391 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c0b0c60 con 0x7fac9c06f5a0 2024-04-25T13:51:46.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.811+0000 7fac9bfff640 1 --2- 172.21.15.80:0/116364391 >> v2:172.21.15.87:3300/0 conn(0x7fac9c0733b0 0x7fac9c161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:46.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7fac9b7fe640 1 --2- 172.21.15.80:0/116364391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c06f5a0 0x7fac9c160b30 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fac9c070270 tx=0x7fac9401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d045d297eef60e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7fac9a7fc640 1 -- 172.21.15.80:0/116364391 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940b0070 con 0x7fac9c06f5a0 2024-04-25T13:51:46.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7fac9a7fc640 1 -- 172.21.15.80:0/116364391 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac940ab600 con 0x7fac9c06f5a0 2024-04-25T13:51:46.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7fac9a7fc640 1 -- 172.21.15.80:0/116364391 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac940ab9d0 con 0x7fac9c06f5a0 2024-04-25T13:51:46.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 -- 172.21.15.80:0/116364391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c06f5a0 msgr2=0x7fac9c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 --2- 172.21.15.80:0/116364391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c06f5a0 0x7fac9c160b30 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fac9c070270 tx=0x7fac9401c450 comp rx=0 tx=0).stop 2024-04-25T13:51:46.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 -- 172.21.15.80:0/116364391 shutdown_connections 2024-04-25T13:51:46.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 --2- 172.21.15.80:0/116364391 >> v2:172.21.15.87:3300/0 conn(0x7fac9c0733b0 0x7fac9c161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 --2- 172.21.15.80:0/116364391 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c06f5a0 0x7fac9c160b30 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 -- 172.21.15.80:0/116364391 >> 172.21.15.80:0/116364391 conn(0x7fac9c06d080 msgr2=0x7fac9c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:46.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 -- 172.21.15.80:0/116364391 shutdown_connections 2024-04-25T13:51:46.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 -- 172.21.15.80:0/116364391 wait complete. 2024-04-25T13:51:46.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 Processor -- start 2024-04-25T13:51:46.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.815+0000 7faca1e43640 1 -- start start 2024-04-25T13:51:46.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac9c06f5a0 0x7fac9c147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c0733b0 0x7fac9c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac9c161070 0x7fac9c15bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c16a050 con 0x7fac9c0733b0 2024-04-25T13:51:46.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c169ed0 con 0x7fac9c06f5a0 2024-04-25T13:51:46.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fac9c16a1d0 con 0x7fac9c161070 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c0733b0 0x7fac9c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c0733b0 0x7fac9c15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41496/0 (socket says 172.21.15.80:41496) 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 -- 172.21.15.80:0/2905837274 learned_addr learned my addr 172.21.15.80:0/2905837274 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9b7fe640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac9c06f5a0 0x7fac9c147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 -- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac9c161070 msgr2=0x7fac9c15bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac9c161070 0x7fac9c15bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 -- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac9c06f5a0 msgr2=0x7fac9c147100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac9c06f5a0 0x7fac9c147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 -- 172.21.15.80:0/2905837274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c15c590 con 0x7fac9c0733b0 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9affd640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c0733b0 0x7fac9c15b7f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fac8400b410 tx=0x7fac8400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac93fff640 1 -- 172.21.15.80:0/2905837274 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84015020 con 0x7fac9c0733b0 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac93fff640 1 -- 172.21.15.80:0/2905837274 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac84011830 con 0x7fac9c0733b0 2024-04-25T13:51:46.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c1fad30 con 0x7fac9c0733b0 2024-04-25T13:51:46.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac93fff640 1 -- 172.21.15.80:0/2905837274 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac84011c20 con 0x7fac9c0733b0 2024-04-25T13:51:46.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c1fb2f0 con 0x7fac9c0733b0 2024-04-25T13:51:46.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c15c300 con 0x7fac9c0733b0 2024-04-25T13:51:46.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac93fff640 1 -- 172.21.15.80:0/2905837274 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fac84018020 con 0x7fac9c0733b0 2024-04-25T13:51:46.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac93fff640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7c06f400 0x7fac7c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:46.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac93fff640 1 -- 172.21.15.80:0/2905837274 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 4857+0+0 (secure 0 0 0) 0x7fac8408f3c0 con 0x7fac9c0733b0 2024-04-25T13:51:46.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.819+0000 7fac9b7fe640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7c06f400 0x7fac7c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:46.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.823+0000 7fac9b7fe640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7c06f400 0x7fac7c071910 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fac940ad000 tx=0x7fac94008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:46.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:46.823+0000 7fac93fff640 1 -- 172.21.15.80:0/2905837274 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac8401f020 con 0x7fac9c0733b0 2024-04-25T13:51:47.020 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.027+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fac9c06a3a0 con 0x7fac9c0733b0 2024-04-25T13:51:47.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.027+0000 7fac93fff640 1 -- 172.21.15.80:0/2905837274 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7fac8405c770 con 0x7fac9c0733b0 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: "auth_dump": [ 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.a", 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmzul0GxAAxZcIb1b2ModfCZMBdzuYuQ==", 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:51:47.022 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.b", 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmqxdVKBAA5f/k5hMWbH3o9hjmEL9nmg==", 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.023 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.c", 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpm1Va4IhAAALHGalbff8tTMzNx3Ddowg==", 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mds.d", 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmfBOXLxAALTpXpCSm8+5Bj/KaskFrVQ==", 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:51:47.024 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.0", 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmS1WyBxAA4lqyz5qFVVnI/mMwOX5fhQ==", 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.1", 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmZpRBERAA3enqqxjsDQUZpC/jDibjRA==", 2024-04-25T13:51:47.025 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.2", 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpmcmTKGRAAWOu3rV5HfSaPGqPQocOr2Q==", 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.026 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.3", 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA0XSpm0kvzIRAAHBkNPUWcO317mVTrY/J/Dw==", 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.4", 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpm2M1sIxAA25x0yZQboLTHqkHxr+kmhw==", 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:51:47.027 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.5", 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmEK1oLBAAtL016zCa7rDb9bW/NSA7nw==", 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.6", 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA3XSpmz1RbNRAA5jUnK+750tCCNgP/koiBlQ==", 2024-04-25T13:51:47.028 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "osd.7", 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQA4XSpmF4aBAhAA7fQhQ7rMf3y7HFCEM0eEUQ==", 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow profile osd", 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile osd", 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.029 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.0", 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw==", 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow r", 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.1", 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmFWckAhAA5DtrJSPtZWzxFv6R01Bh8w==", 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:51:47.030 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.2", 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmrZIrORAAgLMg4jufff0IGPqzBrQ2Wg==", 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.3", 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAxXSpmGjtYBhAAxCXy2jqprfJZ3hFOB3MPGQ==", 2024-04-25T13:51:47.031 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow", 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow r", 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow rw", 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow rwx" 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "client.admin", 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAvXSpmcWNzOhAAY+5iHHKimIk/ktG5T83auQ==", 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": "allow *", 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow *", 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.032 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.x", 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmJpaaFhAAJrZQtlWpZ25Esz0wQTyDwQ==", 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "entity": "mgr.y", 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "key": "AQAwXSpmdNLAERAAGffAydQOct3m9P/+ltdzYw==", 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "caps": { 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "mds": "allow *", 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "mon": "allow profile mgr", 2024-04-25T13:51:47.033 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": "allow *" 2024-04-25T13:51:47.034 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.034 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:47.034 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:51:47.034 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7c06f400 msgr2=0x7fac7c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7c06f400 0x7fac7c071910 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fac940ad000 tx=0x7fac94008040 comp rx=0 tx=0).stop 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c0733b0 msgr2=0x7fac9c15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c0733b0 0x7fac9c15b7f0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fac8400b410 tx=0x7fac8400b8d0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 shutdown_connections 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7c06f400 0x7fac7c071910 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac9c161070 0x7fac9c15bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac9c0733b0 0x7fac9c15b7f0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 --2- 172.21.15.80:0/2905837274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac9c06f5a0 0x7fac9c147100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 >> 172.21.15.80:0/2905837274 conn(0x7fac9c06d080 msgr2=0x7fac9c0b4bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 shutdown_connections 2024-04-25T13:51:47.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.031+0000 7faca1e43640 1 -- 172.21.15.80:0/2905837274 wait complete. 2024-04-25T13:51:47.043 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:47.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.179+0000 7f298b9ef640 1 Processor -- start 2024-04-25T13:51:47.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.179+0000 7f298b9ef640 1 -- start start 2024-04-25T13:51:47.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f298b9ef640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f29840ce730 0x7f29840ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f298b9ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f298406e490 0x7f298406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f298b9ef640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f298406ede0 con 0x7f29840b0ed0 2024-04-25T13:51:47.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f298b9ef640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2984073710 con 0x7f29840ce730 2024-04-25T13:51:47.177 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f298b9ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f29840739f0 con 0x7f298406e490 2024-04-25T13:51:47.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2988f8a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f29840ce730 0x7f29840ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2989f8c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f298406e490 0x7f298406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2988f8a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f29840ce730 0x7f29840ceb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:48248/0 (socket says 172.21.15.80:48248) 2024-04-25T13:51:47.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f298978b640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f29840b0ed0 0x7f298415c860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:41620/0 (socket says 172.21.15.80:41620) 2024-04-25T13:51:47.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2988f8a640 1 -- 172.21.15.80:0/2930224165 learned_addr learned my addr 172.21.15.80:0/2930224165 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:47.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2989f8c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f298406e490 0x7f298406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:49542/0 (socket says 172.21.15.80:49542) 2024-04-25T13:51:47.178 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2989f8c640 1 -- 172.21.15.80:0/2930224165 >> v2:172.21.15.87:3300/0 conn(0x7f29840ce730 msgr2=0x7f29840ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2989f8c640 1 --2- 172.21.15.80:0/2930224165 >> v2:172.21.15.87:3300/0 conn(0x7f29840ce730 0x7f29840ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2989f8c640 1 -- 172.21.15.80:0/2930224165 >> v1:172.21.15.87:6789/0 conn(0x7f29840b0ed0 legacy=0x7f298415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2989f8c640 1 -- 172.21.15.80:0/2930224165 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2984073cd0 con 0x7f298406e490 2024-04-25T13:51:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.183+0000 7f2988f8a640 1 --2- 172.21.15.80:0/2930224165 >> v2:172.21.15.87:3300/0 conn(0x7f29840ce730 0x7f29840ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:47.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f2989f8c640 1 --2- 172.21.15.80:0/2930224165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f298406e490 0x7f298406e8a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f297c009d90 tx=0x7f297c01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=633bd6014dd00d52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f297bfff640 1 -- 172.21.15.80:0/2930224165 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0b4070 con 0x7f298406e490 2024-04-25T13:51:47.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f297bfff640 1 -- 172.21.15.80:0/2930224165 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f297c0af600 con 0x7f298406e490 2024-04-25T13:51:47.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f297bfff640 1 -- 172.21.15.80:0/2930224165 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0af9d0 con 0x7f298406e490 2024-04-25T13:51:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 -- 172.21.15.80:0/2930224165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f298406e490 msgr2=0x7f298406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 --2- 172.21.15.80:0/2930224165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f298406e490 0x7f298406e8a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f297c009d90 tx=0x7f297c01f150 comp rx=0 tx=0).stop 2024-04-25T13:51:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 -- 172.21.15.80:0/2930224165 shutdown_connections 2024-04-25T13:51:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 --2- 172.21.15.80:0/2930224165 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f298406e490 0x7f298406e8a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 --2- 172.21.15.80:0/2930224165 >> v2:172.21.15.87:3300/0 conn(0x7f29840ce730 0x7f29840ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 -- 172.21.15.80:0/2930224165 >> 172.21.15.80:0/2930224165 conn(0x7f298406d080 msgr2=0x7f29840b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 -- 172.21.15.80:0/2930224165 shutdown_connections 2024-04-25T13:51:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 -- 172.21.15.80:0/2930224165 wait complete. 2024-04-25T13:51:47.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 Processor -- start 2024-04-25T13:51:47.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.187+0000 7f298b9ef640 1 -- start start 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298406e490 0x7f298414c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29840b0ed0 0x7f2984153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29840ce730 0x7f29841542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f29840733a0 con 0x7f298406e490 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f29840b12e0 con 0x7f29840ce730 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f298406dc00 con 0x7f29840b0ed0 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298406e490 0x7f298414c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298406e490 0x7f298414c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41498/0 (socket says 172.21.15.80:41498) 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 -- 172.21.15.80:0/68034943 learned_addr learned my addr 172.21.15.80:0/68034943 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f2988f8a640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29840b0ed0 0x7f2984153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f2989f8c640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29840ce730 0x7f29841542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 -- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29840b0ed0 msgr2=0x7f2984153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29840b0ed0 0x7f2984153d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 -- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29840ce730 msgr2=0x7f29841542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29840ce730 0x7f29841542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 -- 172.21.15.80:0/68034943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29840cf3f0 con 0x7f298406e490 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298978b640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298406e490 0x7f298414c3a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f298000c3a0 tx=0x7f298000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f2979ffb640 1 -- 172.21.15.80:0/68034943 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29800074e0 con 0x7f298406e490 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f2979ffb640 1 -- 172.21.15.80:0/68034943 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2980007680 con 0x7f298406e490 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298415e6f0 con 0x7f298406e490 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f2979ffb640 1 -- 172.21.15.80:0/68034943 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980004a30 con 0x7f298406e490 2024-04-25T13:51:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f298415ecb0 con 0x7f298406e490 2024-04-25T13:51:47.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f298406a3a0 con 0x7f298406e490 2024-04-25T13:51:47.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f2979ffb640 1 -- 172.21.15.80:0/68034943 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f298001a020 con 0x7f298406e490 2024-04-25T13:51:47.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f2979ffb640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f296806f400 0x7f2968071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.191+0000 7f2979ffb640 1 -- 172.21.15.80:0/68034943 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 4857+0+0 (secure 0 0 0) 0x7f298008e700 con 0x7f298406e490 2024-04-25T13:51:47.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.195+0000 7f2988f8a640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f296806f400 0x7f2968071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.195+0000 7f2988f8a640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f296806f400 0x7f2968071910 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f29840b4b70 tx=0x7f2974008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.199+0000 7f2979ffb640 1 -- 172.21.15.80:0/68034943 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f29840cf3f0 con 0x7f298406e490 2024-04-25T13:51:47.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.347+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f298406f880 con 0x7f298406e490 2024-04-25T13:51:47.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.351+0000 7f2979ffb640 1 -- 172.21.15.80:0/68034943 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) ==== 74+0+12830 (secure 0 0 0) 0x7f298406ede0 con 0x7f298406e490 2024-04-25T13:51:47.346 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:47.346 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":183,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:46.659120+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.351+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f296806f400 msgr2=0x7f2968071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.351+0000 7f298b9ef640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f296806f400 0x7f2968071910 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f29840b4b70 tx=0x7f2974008040 comp rx=0 tx=0).stop 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.351+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298406e490 msgr2=0x7f298414c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.351+0000 7f298b9ef640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298406e490 0x7f298414c3a0 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f298000c3a0 tx=0x7f298000c860 comp rx=0 tx=0).stop 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.355+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 shutdown_connections 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.355+0000 7f298b9ef640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f296806f400 0x7f2968071910 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.355+0000 7f298b9ef640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29840ce730 0x7f29841542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.355+0000 7f298b9ef640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f29840b0ed0 0x7f2984153d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.355+0000 7f298b9ef640 1 --2- 172.21.15.80:0/68034943 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f298406e490 0x7f298414c3a0 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.348 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.355+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 >> 172.21.15.80:0/68034943 conn(0x7f298406d080 msgr2=0x7f29840af8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:47.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.355+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 shutdown_connections 2024-04-25T13:51:47.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.355+0000 7f298b9ef640 1 -- 172.21.15.80:0/68034943 wait complete. 2024-04-25T13:51:47.367 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:51:47.367 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:51:47.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d95e5640 1 Processor -- start 2024-04-25T13:51:47.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d95e5640 1 -- start start 2024-04-25T13:51:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d95e5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d406edb0 0x7fc8d406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d95e5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc8d4161b60 0x7fc8d4161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d95e5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8d41624b0 con 0x7fc8d40b5330 2024-04-25T13:51:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d95e5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc8d406f700 con 0x7fc8d4161b60 2024-04-25T13:51:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d95e5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d41629b0 con 0x7fc8d406edb0 2024-04-25T13:51:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc8d4161b60 0x7fc8d4161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d37fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc8d4161b60 0x7fc8d4161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41502/0 (socket says 172.21.15.80:41502) 2024-04-25T13:51:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d37fe640 1 -- 172.21.15.80:0/1042489139 learned_addr learned my addr 172.21.15.80:0/1042489139 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d27fc640 1 --2- 172.21.15.80:0/1042489139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d406edb0 0x7fc8d406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d37fe640 1 -- 172.21.15.80:0/1042489139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d406edb0 msgr2=0x7fc8d406f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d37fe640 1 --2- 172.21.15.80:0/1042489139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d406edb0 0x7fc8d406f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d37fe640 1 -- 172.21.15.80:0/1042489139 >> v1:172.21.15.87:6789/0 conn(0x7fc8d40b5330 legacy=0x7fc8d40d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.527+0000 7fc8d37fe640 1 -- 172.21.15.80:0/1042489139 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d40b5740 con 0x7fc8d4161b60 2024-04-25T13:51:47.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d37fe640 1 --2- 172.21.15.80:0/1042489139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d4161b60 0x7fc8d4161f70 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009e50 tx=0x7fc8cc01c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98f52a9fabeb543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d1ffb640 1 -- 172.21.15.80:0/1042489139 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0a5830 con 0x7fc8d4161b60 2024-04-25T13:51:47.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d1ffb640 1 -- 172.21.15.80:0/1042489139 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8cc0a59d0 con 0x7fc8d4161b60 2024-04-25T13:51:47.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d1ffb640 1 -- 172.21.15.80:0/1042489139 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0a5da0 con 0x7fc8d4161b60 2024-04-25T13:51:47.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 -- 172.21.15.80:0/1042489139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d4161b60 msgr2=0x7fc8d4161f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/1042489139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d4161b60 0x7fc8d4161f70 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009e50 tx=0x7fc8cc01c970 comp rx=0 tx=0).stop 2024-04-25T13:51:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 -- 172.21.15.80:0/1042489139 shutdown_connections 2024-04-25T13:51:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/1042489139 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d4161b60 0x7fc8d4161f70 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/1042489139 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d406edb0 0x7fc8d406f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 -- 172.21.15.80:0/1042489139 >> 172.21.15.80:0/1042489139 conn(0x7fc8d40b4160 msgr2=0x7fc8d40d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 -- 172.21.15.80:0/1042489139 shutdown_connections 2024-04-25T13:51:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 -- 172.21.15.80:0/1042489139 wait complete. 2024-04-25T13:51:47.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 Processor -- start 2024-04-25T13:51:47.527 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.531+0000 7fc8d95e5640 1 -- start start 2024-04-25T13:51:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d406edb0 0x7fc8d413b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d40b5330 0x7fc8d4140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8d4161b60 0x7fc8d4140590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d406e890 con 0x7fc8d406edb0 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d406e5b0 con 0x7fc8d4161b60 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d40b0cd0 con 0x7fc8d40b5330 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d406edb0 0x7fc8d413b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d406edb0 0x7fc8d413b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41504/0 (socket says 172.21.15.80:41504) 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 -- 172.21.15.80:0/4205782043 learned_addr learned my addr 172.21.15.80:0/4205782043 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d27fc640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d40b5330 0x7fc8d4140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d37fe640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8d4161b60 0x7fc8d4140590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 -- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d40b5330 msgr2=0x7fc8d4140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d40b5330 0x7fc8d4140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 -- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8d4161b60 msgr2=0x7fc8d4140590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8d4161b60 0x7fc8d4140590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 -- 172.21.15.80:0/4205782043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d413b7b0 con 0x7fc8d406edb0 2024-04-25T13:51:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d2ffd640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d406edb0 0x7fc8d413b270 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400c3a0 tx=0x7fc8c400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8cb7fe640 1 -- 172.21.15.80:0/4205782043 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40052d0 con 0x7fc8d406edb0 2024-04-25T13:51:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8cb7fe640 1 -- 172.21.15.80:0/4205782043 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8c4005470 con 0x7fc8d406edb0 2024-04-25T13:51:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8cb7fe640 1 -- 172.21.15.80:0/4205782043 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4007260 con 0x7fc8d406edb0 2024-04-25T13:51:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d4159410 con 0x7fc8d406edb0 2024-04-25T13:51:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d41599d0 con 0x7fc8d406edb0 2024-04-25T13:51:47.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.535+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d406a3a0 con 0x7fc8d406edb0 2024-04-25T13:51:47.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.539+0000 7fc8cb7fe640 1 -- 172.21.15.80:0/4205782043 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc8c4007400 con 0x7fc8d406edb0 2024-04-25T13:51:47.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.539+0000 7fc8cb7fe640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc8b406f430 0x7fc8b4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.539+0000 7fc8cb7fe640 1 -- 172.21.15.80:0/4205782043 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 4857+0+0 (secure 0 0 0) 0x7fc8c408e3b0 con 0x7fc8d406edb0 2024-04-25T13:51:47.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.539+0000 7fc8d27fc640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc8b406f430 0x7fc8b4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.534 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.539+0000 7fc8d27fc640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc8b406f430 0x7fc8b4071940 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4140e70 tx=0x7fc8bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.537 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.543+0000 7fc8cb7fe640 1 -- 172.21.15.80:0/4205782043 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc8d41624b0 con 0x7fc8d406edb0 2024-04-25T13:51:47.700 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc8d40b4600 con 0x7fc8d406edb0 2024-04-25T13:51:47.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8cb7fe640 1 -- 172.21.15.80:0/4205782043 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v183) ==== 93+0+10 (secure 0 0 0) 0x7fc8c405b680 con 0x7fc8d406edb0 2024-04-25T13:51:47.701 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:51:47.703 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc8b406f430 msgr2=0x7fc8b4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc8b406f430 0x7fc8b4071940 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4140e70 tx=0x7fc8bc008040 comp rx=0 tx=0).stop 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d406edb0 msgr2=0x7fc8d413b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d406edb0 0x7fc8d413b270 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400c3a0 tx=0x7fc8c400c860 comp rx=0 tx=0).stop 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 shutdown_connections 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc8b406f430 0x7fc8b4071940 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8d4161b60 0x7fc8d4140590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc8d40b5330 0x7fc8d4140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 --2- 172.21.15.80:0/4205782043 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc8d406edb0 0x7fc8d413b270 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 >> 172.21.15.80:0/4205782043 conn(0x7fc8d40b4160 msgr2=0x7fc8d406cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 shutdown_connections 2024-04-25T13:51:47.704 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.707+0000 7fc8d95e5640 1 -- 172.21.15.80:0/4205782043 wait complete. 2024-04-25T13:51:47.724 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T13:51:47.869 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ef031640 1 Processor -- start 2024-04-25T13:51:47.870 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ef031640 1 -- start start 2024-04-25T13:51:47.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ef031640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e8161630 0x7f01e806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ef031640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f01e80cfb60 0x7f01e80cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ef031640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f01e815d630 0x7f01e815da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ef031640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f01e8143920 con 0x7f01e80cfb60 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ef031640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f01e806f350 con 0x7f01e815d630 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ef031640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f01e80d04b0 con 0x7f01e8161630 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01e7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f01e80cfb60 0x7f01e80cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01e7fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f01e80cfb60 0x7f01e80cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41506/0 (socket says 172.21.15.80:41506) 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01e7fff640 1 -- 172.21.15.80:0/727726105 learned_addr learned my addr 172.21.15.80:0/727726105 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ed5ce640 1 --2- 172.21.15.80:0/727726105 >> v2:172.21.15.87:3300/0 conn(0x7f01e815d630 0x7f01e815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ecdcd640 1 --2- 172.21.15.80:0/727726105 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e8161630 0x7f01e806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01e7fff640 1 -- 172.21.15.80:0/727726105 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e8161630 msgr2=0x7f01e806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01e7fff640 1 --2- 172.21.15.80:0/727726105 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e8161630 0x7f01e806c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01e7fff640 1 -- 172.21.15.80:0/727726105 >> v2:172.21.15.87:3300/0 conn(0x7f01e815d630 msgr2=0x7f01e815da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01e7fff640 1 --2- 172.21.15.80:0/727726105 >> v2:172.21.15.87:3300/0 conn(0x7f01e815d630 0x7f01e815da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01e7fff640 1 -- 172.21.15.80:0/727726105 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e806cf90 con 0x7f01e80cfb60 2024-04-25T13:51:47.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.875+0000 7f01ed5ce640 1 --2- 172.21.15.80:0/727726105 >> v2:172.21.15.87:3300/0 conn(0x7f01e815d630 0x7f01e815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:47.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01e7fff640 1 --2- 172.21.15.80:0/727726105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e80cfb60 0x7f01e80cff70 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f01e80d0be0 tx=0x7f01d001ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b281ba027bb1d084 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01e77fe640 1 -- 172.21.15.80:0/727726105 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d00b5070 con 0x7f01e80cfb60 2024-04-25T13:51:47.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01e77fe640 1 -- 172.21.15.80:0/727726105 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01d00a89d0 con 0x7f01e80cfb60 2024-04-25T13:51:47.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01e77fe640 1 -- 172.21.15.80:0/727726105 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d00a8da0 con 0x7f01e80cfb60 2024-04-25T13:51:47.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 -- 172.21.15.80:0/727726105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e80cfb60 msgr2=0x7f01e80cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 --2- 172.21.15.80:0/727726105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e80cfb60 0x7f01e80cff70 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f01e80d0be0 tx=0x7f01d001ed10 comp rx=0 tx=0).stop 2024-04-25T13:51:47.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 -- 172.21.15.80:0/727726105 shutdown_connections 2024-04-25T13:51:47.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 --2- 172.21.15.80:0/727726105 >> v2:172.21.15.87:3300/0 conn(0x7f01e815d630 0x7f01e815da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 --2- 172.21.15.80:0/727726105 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e80cfb60 0x7f01e80cff70 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 --2- 172.21.15.80:0/727726105 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e8161630 0x7f01e806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 -- 172.21.15.80:0/727726105 >> 172.21.15.80:0/727726105 conn(0x7f01e80d33a0 msgr2=0x7f01e80cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:47.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 -- 172.21.15.80:0/727726105 shutdown_connections 2024-04-25T13:51:47.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.879+0000 7f01ef031640 1 -- 172.21.15.80:0/727726105 wait complete. 2024-04-25T13:51:47.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 Processor -- start 2024-04-25T13:51:47.877 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 -- start start 2024-04-25T13:51:47.879 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e80cfb60 0x7f01e8169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e815d630 0x7f01e813a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e8161630 0x7f01e815b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8161110 con 0x7f01e8161630 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f01e8160e30 con 0x7f01e815d630 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f01e8161a40 con 0x7f01e80cfb60 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e8161630 0x7f01e815b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e8161630 0x7f01e815b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41512/0 (socket says 172.21.15.80:41512) 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 -- 172.21.15.80:0/3756001140 learned_addr learned my addr 172.21.15.80:0/3756001140 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ecdcd640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e80cfb60 0x7f01e8169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01e7fff640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e815d630 0x7f01e813a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 -- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e80cfb60 msgr2=0x7f01e8169c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e80cfb60 0x7f01e8169c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 -- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e815d630 msgr2=0x7f01e813a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e815d630 0x7f01e813a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:47.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 -- 172.21.15.80:0/3756001140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e8070330 con 0x7f01e8161630 2024-04-25T13:51:47.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ed5ce640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e8161630 0x7f01e815b250 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f01e0007e20 tx=0x7f01e000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01e57fa640 1 -- 172.21.15.80:0/3756001140 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e001a070 con 0x7f01e8161630 2024-04-25T13:51:47.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01e57fa640 1 -- 172.21.15.80:0/3756001140 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01e0003ba0 con 0x7f01e8161630 2024-04-25T13:51:47.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e8168e80 con 0x7f01e8161630 2024-04-25T13:51:47.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01e57fa640 1 -- 172.21.15.80:0/3756001140 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01e000de80 con 0x7f01e8161630 2024-04-25T13:51:47.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.883+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e8169440 con 0x7f01e8161630 2024-04-25T13:51:47.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.887+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01e806a3a0 con 0x7f01e8161630 2024-04-25T13:51:47.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.887+0000 7f01e57fa640 1 -- 172.21.15.80:0/3756001140 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f01e001d070 con 0x7f01e8161630 2024-04-25T13:51:47.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.887+0000 7f01e57fa640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01c806f400 0x7f01c8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:47.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.887+0000 7f01e57fa640 1 -- 172.21.15.80:0/3756001140 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 4857+0+0 (secure 0 0 0) 0x7f01e008f1b0 con 0x7f01e8161630 2024-04-25T13:51:47.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.887+0000 7f01ecdcd640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01c806f400 0x7f01c8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:47.882 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.887+0000 7f01ecdcd640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01c806f400 0x7f01c8071910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f01d800d8d0 tx=0x7f01d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:47.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:47.891+0000 7f01e57fa640 1 -- 172.21.15.80:0/3756001140 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f01e0010070 con 0x7f01e8161630 2024-04-25T13:51:48.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.039+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f01e80d0c40 con 0x7f01e8161630 2024-04-25T13:51:48.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01e57fa640 1 -- 172.21.15.80:0/3756001140 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v183) ==== 84+0+65 (secure 0 0 0) 0x7f01e005c480 con 0x7f01e8161630 2024-04-25T13:51:48.037 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:48.037 INFO:teuthology.orchestra.run.smithi080.stdout:[ 2024-04-25T13:51:48.037 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:51:48.037 INFO:teuthology.orchestra.run.smithi080.stdout: "poolnum": 1, 2024-04-25T13:51:48.037 INFO:teuthology.orchestra.run.smithi080.stdout: "poolname": ".mgr" 2024-04-25T13:51:48.037 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:48.037 INFO:teuthology.orchestra.run.smithi080.stdout:] 2024-04-25T13:51:48.039 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01c806f400 msgr2=0x7f01c8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01c806f400 0x7f01c8071910 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f01d800d8d0 tx=0x7f01d8008040 comp rx=0 tx=0).stop 2024-04-25T13:51:48.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e8161630 msgr2=0x7f01e815b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e8161630 0x7f01e815b250 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f01e0007e20 tx=0x7f01e000d4a0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 shutdown_connections 2024-04-25T13:51:48.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f01c806f400 0x7f01c8071910 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f01e8161630 0x7f01e815b250 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f01e815d630 0x7f01e813a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 --2- 172.21.15.80:0/3756001140 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f01e80cfb60 0x7f01e8169c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 >> 172.21.15.80:0/3756001140 conn(0x7f01e80d33a0 msgr2=0x7f01e815ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:48.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 shutdown_connections 2024-04-25T13:51:48.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.043+0000 7f01ef031640 1 -- 172.21.15.80:0/3756001140 wait complete. 2024-04-25T13:51:48.057 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T13:51:48.057 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T13:51:48.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5edb0640 1 Processor -- start 2024-04-25T13:51:48.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5edb0640 1 -- start start 2024-04-25T13:51:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5edb0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580b0f10 0x7fce580d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5edb0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fce58161d10 0x7fce58162120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5edb0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce580b1320 con 0x7fce580ce2a0 2024-04-25T13:51:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5edb0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fce580b5820 con 0x7fce58161d10 2024-04-25T13:51:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5edb0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce580d13f0 con 0x7fce580b0f10 2024-04-25T13:51:48.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5d34d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fce58161d10 0x7fce58162120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5d34d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fce58161d10 0x7fce58162120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41520/0 (socket says 172.21.15.80:41520) 2024-04-25T13:51:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5d34d640 1 -- 172.21.15.80:0/2039357560 learned_addr learned my addr 172.21.15.80:0/2039357560 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5cb4c640 1 --2- 172.21.15.80:0/2039357560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580b0f10 0x7fce580d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5d34d640 1 -- 172.21.15.80:0/2039357560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580b0f10 msgr2=0x7fce580d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5d34d640 1 --2- 172.21.15.80:0/2039357560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580b0f10 0x7fce580d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5d34d640 1 -- 172.21.15.80:0/2039357560 >> v1:172.21.15.87:6789/0 conn(0x7fce580ce2a0 legacy=0x7fce580ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.215+0000 7fce5d34d640 1 -- 172.21.15.80:0/2039357560 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce580d1150 con 0x7fce58161d10 2024-04-25T13:51:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5d34d640 1 --2- 172.21.15.80:0/2039357560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce58162120 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fce48009e50 tx=0x7fce4801ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3a19d4a4053e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:48.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce577fe640 1 -- 172.21.15.80:0/2039357560 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480a57f0 con 0x7fce58161d10 2024-04-25T13:51:48.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce577fe640 1 -- 172.21.15.80:0/2039357560 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce480a5990 con 0x7fce58161d10 2024-04-25T13:51:48.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce577fe640 1 -- 172.21.15.80:0/2039357560 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480a5d60 con 0x7fce58161d10 2024-04-25T13:51:48.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5edb0640 1 -- 172.21.15.80:0/2039357560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 msgr2=0x7fce58162120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5edb0640 1 --2- 172.21.15.80:0/2039357560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce58162120 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fce48009e50 tx=0x7fce4801ca30 comp rx=0 tx=0).stop 2024-04-25T13:51:48.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5edb0640 1 -- 172.21.15.80:0/2039357560 shutdown_connections 2024-04-25T13:51:48.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5edb0640 1 --2- 172.21.15.80:0/2039357560 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce58162120 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5edb0640 1 --2- 172.21.15.80:0/2039357560 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580b0f10 0x7fce580d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5edb0640 1 -- 172.21.15.80:0/2039357560 >> 172.21.15.80:0/2039357560 conn(0x7fce580b35c0 msgr2=0x7fce580b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:48.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5edb0640 1 -- 172.21.15.80:0/2039357560 shutdown_connections 2024-04-25T13:51:48.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.219+0000 7fce5edb0640 1 -- 172.21.15.80:0/2039357560 wait complete. 2024-04-25T13:51:48.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 Processor -- start 2024-04-25T13:51:48.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 -- start start 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce580b0f10 0x7fce581483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580ce2a0 0x7fce580af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce580afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fce5815e720 con 0x7fce58161d10 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fce580b5300 con 0x7fce580b0f10 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fce58162a70 con 0x7fce580ce2a0 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce580afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce580afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41530/0 (socket says 172.21.15.80:41530) 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 -- 172.21.15.80:0/4043750936 learned_addr learned my addr 172.21.15.80:0/4043750936 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5cb4c640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce580b0f10 0x7fce581483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce57fff640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580ce2a0 0x7fce580af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 -- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580ce2a0 msgr2=0x7fce580af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580ce2a0 0x7fce580af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 -- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce580b0f10 msgr2=0x7fce581483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce580b0f10 0x7fce581483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 -- 172.21.15.80:0/4043750936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5806f570 con 0x7fce58161d10 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5d34d640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce580afdf0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fce48009e50 tx=0x7fce480197d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce557fa640 1 -- 172.21.15.80:0/4043750936 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480b8640 con 0x7fce58161d10 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce557fa640 1 -- 172.21.15.80:0/4043750936 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fce480bf070 con 0x7fce58161d10 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5806f850 con 0x7fce58161d10 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce557fa640 1 -- 172.21.15.80:0/4043750936 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce480b8a20 con 0x7fce58161d10 2024-04-25T13:51:48.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce580b06b0 con 0x7fce58161d10 2024-04-25T13:51:48.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.223+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5806a440 con 0x7fce58161d10 2024-04-25T13:51:48.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.227+0000 7fce557fa640 1 -- 172.21.15.80:0/4043750936 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fce480b8bc0 con 0x7fce58161d10 2024-04-25T13:51:48.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.227+0000 7fce557fa640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3806f430 0x7fce38071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.227+0000 7fce557fa640 1 -- 172.21.15.80:0/4043750936 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(183..183 src has 1..183) ==== 4857+0+0 (secure 0 0 0) 0x7fce481299c0 con 0x7fce58161d10 2024-04-25T13:51:48.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.227+0000 7fce5cb4c640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3806f430 0x7fce38071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.227+0000 7fce5cb4c640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3806f430 0x7fce38071940 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fce580d28d0 tx=0x7fce50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:48.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.231+0000 7fce557fa640 1 -- 172.21.15.80:0/4043750936 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fce4812be20 con 0x7fce58161d10 2024-04-25T13:51:48.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.391+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fce5806f300 con 0x7fce58161d10 2024-04-25T13:51:48.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.651+0000 7fce557fa640 1 -- 172.21.15.80:0/4043750936 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v184) ==== 138+0+0 (secure 0 0 0) 0x7fce48016620 con 0x7fce58161d10 2024-04-25T13:51:48.645 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' created 2024-04-25T13:51:48.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3806f430 msgr2=0x7fce38071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3806f430 0x7fce38071940 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fce580d28d0 tx=0x7fce50008040 comp rx=0 tx=0).stop 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 msgr2=0x7fce580afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce580afdf0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fce48009e50 tx=0x7fce480197d0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 shutdown_connections 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fce3806f430 0x7fce38071940 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fce58161d10 0x7fce580afdf0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fce580ce2a0 0x7fce580af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 --2- 172.21.15.80:0/4043750936 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fce580b0f10 0x7fce581483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 >> 172.21.15.80:0/4043750936 conn(0x7fce580b35c0 msgr2=0x7fce580757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 shutdown_connections 2024-04-25T13:51:48.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.655+0000 7fce5edb0640 1 -- 172.21.15.80:0/4043750936 wait complete. 2024-04-25T13:51:48.678 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T13:51:48.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.843+0000 7f1683170640 1 Processor -- start 2024-04-25T13:51:48.839 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.843+0000 7f1683170640 1 -- start start 2024-04-25T13:51:48.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1683170640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c06f5a0 0x7f167c160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.841 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1683170640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f167c161070 0x7f167c161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1683170640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f167c06fcf0 con 0x7f167c0b4bd0 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1683170640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f167c1619c0 con 0x7f167c161070 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1683170640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f167c073f10 con 0x7f167c06f5a0 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1680f0c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c06f5a0 0x7f167c160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f167bfff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f167c161070 0x7f167c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f168170d640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f167c0b4bd0 0x7f167c0b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:41650/0 (socket says 172.21.15.80:41650) 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f167bfff640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f167c161070 0x7f167c161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:48296/0 (socket says 172.21.15.80:48296) 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f168170d640 1 -- 172.21.15.80:0/3001259509 learned_addr learned my addr 172.21.15.80:0/3001259509 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1680f0c640 1 -- 172.21.15.80:0/3001259509 >> v2:172.21.15.87:3300/0 conn(0x7f167c161070 msgr2=0x7f167c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1680f0c640 1 --2- 172.21.15.80:0/3001259509 >> v2:172.21.15.87:3300/0 conn(0x7f167c161070 0x7f167c161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.842 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1680f0c640 1 -- 172.21.15.80:0/3001259509 >> v1:172.21.15.87:6789/0 conn(0x7f167c0b4bd0 legacy=0x7f167c0b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.843 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.847+0000 7f1680f0c640 1 -- 172.21.15.80:0/3001259509 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c0b0c60 con 0x7f167c06f5a0 2024-04-25T13:51:48.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1680f0c640 1 --2- 172.21.15.80:0/3001259509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c06f5a0 0x7f167c160b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f166c0097c0 tx=0x7f166c01c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a5a25c8bf79661 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:48.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f167b7fe640 1 -- 172.21.15.80:0/3001259509 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c0b0070 con 0x7f167c06f5a0 2024-04-25T13:51:48.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f167b7fe640 1 -- 172.21.15.80:0/3001259509 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c0ab600 con 0x7f167c06f5a0 2024-04-25T13:51:48.845 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f167b7fe640 1 -- 172.21.15.80:0/3001259509 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c0ab9d0 con 0x7f167c06f5a0 2024-04-25T13:51:48.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1683170640 1 -- 172.21.15.80:0/3001259509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c06f5a0 msgr2=0x7f167c160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1683170640 1 --2- 172.21.15.80:0/3001259509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c06f5a0 0x7f167c160b30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f166c0097c0 tx=0x7f166c01c410 comp rx=0 tx=0).stop 2024-04-25T13:51:48.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1683170640 1 -- 172.21.15.80:0/3001259509 shutdown_connections 2024-04-25T13:51:48.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1683170640 1 --2- 172.21.15.80:0/3001259509 >> v2:172.21.15.87:3300/0 conn(0x7f167c161070 0x7f167c161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1683170640 1 --2- 172.21.15.80:0/3001259509 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c06f5a0 0x7f167c160b30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1683170640 1 -- 172.21.15.80:0/3001259509 >> 172.21.15.80:0/3001259509 conn(0x7f167c06d080 msgr2=0x7f167c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:48.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1683170640 1 -- 172.21.15.80:0/3001259509 shutdown_connections 2024-04-25T13:51:48.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.851+0000 7f1683170640 1 -- 172.21.15.80:0/3001259509 wait complete. 2024-04-25T13:51:48.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 Processor -- start 2024-04-25T13:51:48.848 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 -- start start 2024-04-25T13:51:48.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f167c06f5a0 0x7f167c15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f167c0b4bd0 0x7f167c15af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c161070 0x7f167c15b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f167c168850 con 0x7f167c06f5a0 2024-04-25T13:51:48.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f167c1686d0 con 0x7f167c0b4bd0 2024-04-25T13:51:48.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f167c1689d0 con 0x7f167c161070 2024-04-25T13:51:48.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f167c06f5a0 0x7f167c15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.851 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f167c06f5a0 0x7f167c15a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41540/0 (socket says 172.21.15.80:41540) 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 -- 172.21.15.80:0/2513264550 learned_addr learned my addr 172.21.15.80:0/2513264550 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f167bfff640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f167c0b4bd0 0x7f167c15af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 -- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c161070 msgr2=0x7f167c15b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f168170d640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c161070 0x7f167c15b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c161070 0x7f167c15b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 -- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f167c0b4bd0 msgr2=0x7f167c15af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f167c0b4bd0 0x7f167c15af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 -- 172.21.15.80:0/2513264550 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f167c150250 con 0x7f167c06f5a0 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1680f0c640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f167c06f5a0 0x7f167c15a9e0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f166c01a040 tx=0x7f166c01ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f16797fa640 1 -- 172.21.15.80:0/2513264550 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c0b0050 con 0x7f167c06f5a0 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f16797fa640 1 -- 172.21.15.80:0/2513264550 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f166c0bc070 con 0x7f167c06f5a0 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f167c150530 con 0x7f167c06f5a0 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f16797fa640 1 -- 172.21.15.80:0/2513264550 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166c0b5830 con 0x7f167c06f5a0 2024-04-25T13:51:48.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f167c150810 con 0x7f167c06f5a0 2024-04-25T13:51:48.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.855+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f167c06a3a0 con 0x7f167c06f5a0 2024-04-25T13:51:48.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.859+0000 7f16797fa640 1 -- 172.21.15.80:0/2513264550 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f166c0b59d0 con 0x7f167c06f5a0 2024-04-25T13:51:48.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.859+0000 7f16797fa640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f166806f430 0x7f1668071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:48.853 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.859+0000 7f16797fa640 1 -- 172.21.15.80:0/2513264550 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(184..184 src has 1..184) ==== 5227+0+0 (secure 0 0 0) 0x7f166c1298a0 con 0x7f167c06f5a0 2024-04-25T13:51:48.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.859+0000 7f167bfff640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f166806f430 0x7f1668071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:48.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.859+0000 7f167bfff640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f166806f430 0x7f1668071940 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f167c159e30 tx=0x7f1670008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:48.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:48.863+0000 7f16797fa640 1 -- 172.21.15.80:0/2513264550 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f166c0c5020 con 0x7f167c06f5a0 2024-04-25T13:51:49.021 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.027+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f167c0739c0 con 0x7f167c06f5a0 2024-04-25T13:51:49.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.663+0000 7f16797fa640 1 -- 172.21.15.80:0/2513264550 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v185) ==== 170+0+0 (secure 0 0 0) 0x7f166c016090 con 0x7f167c06f5a0 2024-04-25T13:51:49.657 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' created 2024-04-25T13:51:49.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f166806f430 msgr2=0x7f1668071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:49.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f166806f430 0x7f1668071940 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f167c159e30 tx=0x7f1670008040 comp rx=0 tx=0).stop 2024-04-25T13:51:49.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f167c06f5a0 msgr2=0x7f167c15a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:49.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f167c06f5a0 0x7f167c15a9e0 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f166c01a040 tx=0x7f166c01ac50 comp rx=0 tx=0).stop 2024-04-25T13:51:49.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 shutdown_connections 2024-04-25T13:51:49.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f166806f430 0x7f1668071940 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f167c161070 0x7f167c15b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f167c0b4bd0 0x7f167c15af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 --2- 172.21.15.80:0/2513264550 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f167c06f5a0 0x7f167c15a9e0 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 >> 172.21.15.80:0/2513264550 conn(0x7f167c06d080 msgr2=0x7f167c0d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:49.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 shutdown_connections 2024-04-25T13:51:49.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.667+0000 7f1683170640 1 -- 172.21.15.80:0/2513264550 wait complete. 2024-04-25T13:51:49.695 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T13:51:49.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.847+0000 7f40c448d640 1 Processor -- start 2024-04-25T13:51:49.844 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c448d640 1 -- start start 2024-04-25T13:51:49.846 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c448d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c448d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f40bc15c710 0x7f40bc071e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c448d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40bc0737f0 con 0x7f40bc0723a0 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c448d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f40bc0b53d0 con 0x7f40bc15c710 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c448d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc160b20 con 0x7f40bc073080 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c1a28640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f40bc15c710 0x7f40bc071e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c1a28640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f40bc15c710 0x7f40bc071e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41556/0 (socket says 172.21.15.80:41556) 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c1a28640 1 -- 172.21.15.80:0/3492770712 learned_addr learned my addr 172.21.15.80:0/3492770712 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c2229640 1 --2- 172.21.15.80:0/3492770712 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c1a28640 1 -- 172.21.15.80:0/3492770712 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 msgr2=0x7f40bc073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c1a28640 1 --2- 172.21.15.80:0/3492770712 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c1a28640 1 -- 172.21.15.80:0/3492770712 >> v1:172.21.15.87:6789/0 conn(0x7f40bc0723a0 legacy=0x7f40bc06dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:49.847 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.851+0000 7f40c1a28640 1 -- 172.21.15.80:0/3492770712 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc1604d0 con 0x7f40bc15c710 2024-04-25T13:51:49.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c1a28640 1 --2- 172.21.15.80:0/3492770712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc071e60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009870 tx=0x7f40ac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0406ce66a5d36b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:49.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c1227640 1 -- 172.21.15.80:0/3492770712 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0b1070 con 0x7f40bc15c710 2024-04-25T13:51:49.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c1227640 1 -- 172.21.15.80:0/3492770712 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40ac0a8050 con 0x7f40bc15c710 2024-04-25T13:51:49.849 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c1227640 1 -- 172.21.15.80:0/3492770712 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0ac860 con 0x7f40bc15c710 2024-04-25T13:51:49.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c448d640 1 -- 172.21.15.80:0/3492770712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 msgr2=0x7f40bc071e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:49.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c448d640 1 --2- 172.21.15.80:0/3492770712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc071e60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009870 tx=0x7f40ac01c4d0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c448d640 1 -- 172.21.15.80:0/3492770712 shutdown_connections 2024-04-25T13:51:49.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c448d640 1 --2- 172.21.15.80:0/3492770712 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc071e60 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c448d640 1 --2- 172.21.15.80:0/3492770712 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c448d640 1 -- 172.21.15.80:0/3492770712 >> 172.21.15.80:0/3492770712 conn(0x7f40bc0b4160 msgr2=0x7f40bc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:49.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c448d640 1 -- 172.21.15.80:0/3492770712 shutdown_connections 2024-04-25T13:51:49.850 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.855+0000 7f40c448d640 1 -- 172.21.15.80:0/3492770712 wait complete. 2024-04-25T13:51:49.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 Processor -- start 2024-04-25T13:51:49.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 -- start start 2024-04-25T13:51:49.854 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc0723a0 0x7f40bc14f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc15cb20 con 0x7f40bc15c710 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f40bc06eb00 con 0x7f40bc0723a0 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f40bc14b610 con 0x7f40bc073080 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2a2a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c1a28640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2229640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc0723a0 0x7f40bc14f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c1a28640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:49620/0 (socket says 172.21.15.80:49620) 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c1a28640 1 -- 172.21.15.80:0/1418420854 learned_addr learned my addr 172.21.15.80:0/1418420854 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2a2a640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41570/0 (socket says 172.21.15.80:41570) 2024-04-25T13:51:49.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2a2a640 1 -- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 msgr2=0x7f40bc147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2a2a640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc147f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2a2a640 1 -- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc0723a0 msgr2=0x7f40bc14f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2a2a640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc0723a0 0x7f40bc14f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2a2a640 1 -- 172.21.15.80:0/1418420854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40bc149020 con 0x7f40bc15c710 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c2a2a640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc155030 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f40b400e910 tx=0x7f40b400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40b2ffd640 1 -- 172.21.15.80:0/1418420854 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4001f60 con 0x7f40bc15c710 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40b2ffd640 1 -- 172.21.15.80:0/1418420854 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40b4003980 con 0x7f40bc15c710 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40bc149300 con 0x7f40bc15c710 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40b2ffd640 1 -- 172.21.15.80:0/1418420854 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b4003d70 con 0x7f40bc15c710 2024-04-25T13:51:49.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40bc1498c0 con 0x7f40bc15c710 2024-04-25T13:51:49.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.859+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40bc06a3a0 con 0x7f40bc15c710 2024-04-25T13:51:49.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.863+0000 7f40b2ffd640 1 -- 172.21.15.80:0/1418420854 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f40b40050d0 con 0x7f40bc15c710 2024-04-25T13:51:49.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.863+0000 7f40b2ffd640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40a006f430 0x7f40a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:49.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.863+0000 7f40b2ffd640 1 -- 172.21.15.80:0/1418420854 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(185..185 src has 1..185) ==== 5609+0+0 (secure 0 0 0) 0x7f40b4091980 con 0x7f40bc15c710 2024-04-25T13:51:49.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.863+0000 7f40c2229640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40a006f430 0x7f40a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:49.858 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.863+0000 7f40c2229640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40a006f430 0x7f40a0071940 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f40bc160290 tx=0x7f40b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:49.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:49.867+0000 7f40b2ffd640 1 -- 172.21.15.80:0/1418420854 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f40b4016020 con 0x7f40bc15c710 2024-04-25T13:51:50.058 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.063+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f40bc148db0 con 0x7f40bc15c710 2024-04-25T13:51:50.701 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.707+0000 7f40b2ffd640 1 -- 172.21.15.80:0/1418420854 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v86) ==== 174+0+0 (secure 0 0 0) 0x7f40b405e3f0 con 0x7f40bc15c710 2024-04-25T13:51:50.701 INFO:teuthology.orchestra.run.smithi080.stderr:new fs with metadata pool 22 and data pool 23 2024-04-25T13:51:50.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40a006f430 msgr2=0x7f40a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:50.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40a006f430 0x7f40a0071940 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f40bc160290 tx=0x7f40b8008040 comp rx=0 tx=0).stop 2024-04-25T13:51:50.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 msgr2=0x7f40bc155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:50.707 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc155030 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f40b400e910 tx=0x7f40b400edd0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 shutdown_connections 2024-04-25T13:51:50.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f40a006f430 0x7f40a0071940 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f40bc15c710 0x7f40bc155030 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f40bc073080 0x7f40bc147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 --2- 172.21.15.80:0/1418420854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40bc0723a0 0x7f40bc14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 >> 172.21.15.80:0/1418420854 conn(0x7f40bc0b4160 msgr2=0x7f40bc15d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:50.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.711+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 shutdown_connections 2024-04-25T13:51:50.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.715+0000 7f40c448d640 1 -- 172.21.15.80:0/1418420854 wait complete. 2024-04-25T13:51:50.736 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:50.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f2593305640 1 Processor -- start 2024-04-25T13:51:50.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f2593305640 1 -- start start 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f2593305640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f258c0b9240 0x7f258c0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f2593305640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 0x7f258c0d08f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f2593305640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f258c0d0e30 con 0x7f258c0d1a60 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f2593305640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f258c0b6920 con 0x7f258c0b9240 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f2593305640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f258c0cfb20 con 0x7f258c0d04e0 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f25918a2640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f258c0d1a60 0x7f258c0d1e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:48448/0 (socket says 172.21.15.80:48448) 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f25918a2640 1 -- 172.21.15.80:0/3318286574 learned_addr learned my addr 172.21.15.80:0/3318286574 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f25910a1640 1 --2- 172.21.15.80:0/3318286574 >> v2:172.21.15.87:3300/0 conn(0x7f258c0b9240 0x7f258c0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f25908a0640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 0x7f258c0d08f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1795719228 0 0) 0x7f2584001120 con 0x7f258c0d1a60 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2574003530 con 0x7f258c0d1a60 2024-04-25T13:51:50.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4142289369 0 0) 0x7f2574003530 con 0x7f258c0d1a60 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 msgr2=0x7f258c0d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 0x7f258c0d08f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 >> v2:172.21.15.87:3300/0 conn(0x7f258c0b9240 msgr2=0x7f258c0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 --2- 172.21.15.80:0/3318286574 >> v2:172.21.15.87:3300/0 conn(0x7f258c0b9240 0x7f258c0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c0b6b70 con 0x7f258c0d1a60 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f25910a1640 1 --2- 172.21.15.80:0/3318286574 >> v2:172.21.15.87:3300/0 conn(0x7f258c0b9240 0x7f258c0cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2584001850 con 0x7f258c0d1a60 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 >> v1:172.21.15.80:6789/0 conn(0x7f258c0d1a60 legacy=0x7f258c0d1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 0x7f25740039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f2574004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2574004d50 0x7f2574005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0b6920 con 0x7f2574003f40 2024-04-25T13:51:50.886 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0cfb20 con 0x7f2574004d50 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f257400bad0 con 0x7f258c0d04e0 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.887+0000 7f25908a0640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f2574004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25918a2640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2574004d50 0x7f2574005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25910a1640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 0x7f25740039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25908a0640 1 -- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 msgr2=0x7f25740039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25908a0640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 0x7f25740039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25908a0640 1 -- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2574004d50 msgr2=0x7f2574005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25908a0640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2574004d50 0x7f2574005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25908a0640 1 -- 172.21.15.80:0/3318286574 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2574003530 con 0x7f2574003f40 2024-04-25T13:51:50.887 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25910a1640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c0d04e0 0x7f25740039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:50.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25908a0640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f2574004810 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f2580009ae0 tx=0x7f258001ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabe74555097dbe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:50.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800b2070 con 0x7f2574003f40 2024-04-25T13:51:50.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25800a5630 con 0x7f2574003f40 2024-04-25T13:51:50.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f258bfff640 1 -- 172.21.15.80:0/3318286574 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25800096b0 con 0x7f2574003f40 2024-04-25T13:51:50.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f2593305640 1 -- 172.21.15.80:0/3318286574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 msgr2=0x7f2574004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:50.888 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f2593305640 1 --2- 172.21.15.80:0/3318286574 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f2574004810 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f2580009ae0 tx=0x7f258001ca60 comp rx=0 tx=0).stop 2024-04-25T13:51:50.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f25918a2640 1 -- 172.21.15.80:0/3318286574 reap_dead start 2024-04-25T13:51:50.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f2593305640 1 -- 172.21.15.80:0/3318286574 shutdown_connections 2024-04-25T13:51:50.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f2593305640 1 -- 172.21.15.80:0/3318286574 >> 172.21.15.80:0/3318286574 conn(0x7f258c0d3ad0 msgr2=0x7f258c0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:50.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f2593305640 1 -- 172.21.15.80:0/3318286574 shutdown_connections 2024-04-25T13:51:50.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.891+0000 7f2593305640 1 -- 172.21.15.80:0/3318286574 wait complete. 2024-04-25T13:51:50.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 Processor -- start 2024-04-25T13:51:50.889 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 -- start start 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f258c159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c15a150 0x7f258c1fb800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f258c1fbd40 0x7f258c158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0d08f0 con 0x7f2574003f40 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f258c0cf900 con 0x7f258c1fbd40 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f258c0d0a70 con 0x7f258c15a150 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f258c159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f258c159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:41586/0 (socket says 172.21.15.80:41586) 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 -- 172.21.15.80:0/3850519388 learned_addr learned my addr 172.21.15.80:0/3850519388 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25918a2640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f258c1fbd40 0x7f258c158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25908a0640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c15a150 0x7f258c1fb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 -- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c15a150 msgr2=0x7f258c1fb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:50.892 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c15a150 0x7f258c1fb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 -- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f258c1fbd40 msgr2=0x7f258c158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f258c1fbd40 0x7f258c158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 -- 172.21.15.80:0/3850519388 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258c15a560 con 0x7f2574003f40 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f25910a1640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f258c159c10 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f257c00f400 tx=0x7f257c00f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2589ffb640 1 -- 172.21.15.80:0/3850519388 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c0147e0 con 0x7f2574003f40 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2589ffb640 1 -- 172.21.15.80:0/3850519388 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f257c01b070 con 0x7f2574003f40 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258c163b40 con 0x7f2574003f40 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2589ffb640 1 -- 172.21.15.80:0/3850519388 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257c014b80 con 0x7f2574003f40 2024-04-25T13:51:50.893 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.895+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258c164100 con 0x7f2574003f40 2024-04-25T13:51:50.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.899+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f258c06a3a0 con 0x7f2574003f40 2024-04-25T13:51:50.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.899+0000 7f2589ffb640 1 -- 172.21.15.80:0/3850519388 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f257c020460 con 0x7f2574003f40 2024-04-25T13:51:50.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.899+0000 7f2589ffb640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f257806f400 0x7f2578071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:50.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.899+0000 7f2589ffb640 1 -- 172.21.15.80:0/3850519388 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f257c091010 con 0x7f2574003f40 2024-04-25T13:51:50.894 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.899+0000 7f25908a0640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f257806f400 0x7f2578071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:50.895 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.899+0000 7f25908a0640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f257806f400 0x7f2578071910 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f25800af000 tx=0x7f25800046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:50.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:50.903+0000 7f2589ffb640 1 -- 172.21.15.80:0/3850519388 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f258c15a560 con 0x7f2574003f40 2024-04-25T13:51:51.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.063+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f258c074c30 con 0x7f2574003f40 2024-04-25T13:51:51.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.067+0000 7f2589ffb640 1 -- 172.21.15.80:0/3850519388 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) ==== 74+0+16613 (secure 0 0 0) 0x7f257c05dfe0 con 0x7f2574003f40 2024-04-25T13:51:51.063 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:51.063 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":186,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:50.067310+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:51:48.404349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T13:51:49.039512+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:51.065 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f257806f400 msgr2=0x7f2578071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f257806f400 0x7f2578071910 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f25800af000 tx=0x7f25800046d0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 msgr2=0x7f258c159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f258c159c10 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f257c00f400 tx=0x7f257c00f8c0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 shutdown_connections 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f257806f400 0x7f2578071910 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f258c1fbd40 0x7f258c158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f258c15a150 0x7f258c1fb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 --2- 172.21.15.80:0/3850519388 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2574003f40 0x7f258c159c10 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 >> 172.21.15.80:0/3850519388 conn(0x7f258c0d3ad0 msgr2=0x7f258c070c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 shutdown_connections 2024-04-25T13:51:51.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.071+0000 7f2593305640 1 -- 172.21.15.80:0/3850519388 wait complete. 2024-04-25T13:51:51.086 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:51.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.239+0000 7f9e43ab6640 1 Processor -- start 2024-04-25T13:51:51.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.239+0000 7f9e43ab6640 1 -- start start 2024-04-25T13:51:51.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e43ab6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9e3c161f90 0x7f9e3c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e43ab6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15ec40 0x7f9e3c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e43ab6640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e3c0d0d70 con 0x7f9e3c0d0510 2024-04-25T13:51:51.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e43ab6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9e3c0d1050 con 0x7f9e3c161f90 2024-04-25T13:51:51.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e43ab6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3c06dca0 con 0x7f9e3c15ec40 2024-04-25T13:51:51.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e41051640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9e3c0d0510 0x7f9e3c15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41214/0 (socket says 172.21.15.80:41214) 2024-04-25T13:51:51.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e41051640 1 -- 172.21.15.80:0/1440712391 learned_addr learned my addr 172.21.15.80:0/1440712391 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:51.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e41852640 1 --2- 172.21.15.80:0/1440712391 >> v2:172.21.15.87:3300/0 conn(0x7f9e3c161f90 0x7f9e3c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e42053640 1 --2- 172.21.15.80:0/1440712391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15ec40 0x7f9e3c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.241 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e40850640 1 -- 172.21.15.80:0/1440712391 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1251919790 0 0) 0x7f9e3c0d0d70 con 0x7f9e3c0d0510 2024-04-25T13:51:51.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e40850640 1 -- 172.21.15.80:0/1440712391 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9e28003530 con 0x7f9e3c0d0510 2024-04-25T13:51:51.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e40850640 1 -- 172.21.15.80:0/1440712391 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1096303688 0 0) 0x7f9e28003530 con 0x7f9e3c0d0510 2024-04-25T13:51:51.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e41852640 1 -- 172.21.15.80:0/1440712391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15ec40 msgr2=0x7f9e3c15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:51.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e41852640 1 --2- 172.21.15.80:0/1440712391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15ec40 0x7f9e3c15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e41852640 1 -- 172.21.15.80:0/1440712391 >> v1:172.21.15.80:6789/0 conn(0x7f9e3c0d0510 legacy=0x7f9e3c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.243+0000 7f9e41852640 1 -- 172.21.15.80:0/1440712391 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3c06df80 con 0x7f9e3c161f90 2024-04-25T13:51:51.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e41852640 1 --2- 172.21.15.80:0/1440712391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c161f90 0x7f9e3c0cffd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9e380097c0 tx=0x7f9e3801c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=134dc98a629e76fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:51.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e40850640 1 -- 172.21.15.80:0/1440712391 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e380b0070 con 0x7f9e3c161f90 2024-04-25T13:51:51.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e40850640 1 -- 172.21.15.80:0/1440712391 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e380ab400 con 0x7f9e3c161f90 2024-04-25T13:51:51.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e40850640 1 -- 172.21.15.80:0/1440712391 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e30001680 con 0x7f9e3c161f90 2024-04-25T13:51:51.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1440712391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c161f90 msgr2=0x7f9e3c0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1440712391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c161f90 0x7f9e3c0cffd0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9e380097c0 tx=0x7f9e3801c2d0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1440712391 shutdown_connections 2024-04-25T13:51:51.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1440712391 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15ec40 0x7f9e3c15f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1440712391 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c161f90 0x7f9e3c0cffd0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1440712391 >> 172.21.15.80:0/1440712391 conn(0x7f9e3c06d080 msgr2=0x7f9e3c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:51.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1440712391 shutdown_connections 2024-04-25T13:51:51.246 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.251+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1440712391 wait complete. 2024-04-25T13:51:51.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 Processor -- start 2024-04-25T13:51:51.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 -- start start 2024-04-25T13:51:51.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c0d0510 0x7f9e3c0b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.251 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c15ec40 0x7f9e3c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15bfe0 0x7f9e3c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c06f630 con 0x7f9e3c0d0510 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9e3c06f4b0 con 0x7f9e3c15ec40 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9e3c0d0920 con 0x7f9e3c15bfe0 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c0d0510 0x7f9e3c0b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c0d0510 0x7f9e3c0b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58450/0 (socket says 172.21.15.80:58450) 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 -- 172.21.15.80:0/1843184696 learned_addr learned my addr 172.21.15.80:0/1843184696 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41051640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c15ec40 0x7f9e3c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e42053640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15bfe0 0x7f9e3c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 -- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15bfe0 msgr2=0x7f9e3c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15bfe0 0x7f9e3c15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 -- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c15ec40 msgr2=0x7f9e3c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c15ec40 0x7f9e3c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 -- 172.21.15.80:0/1843184696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e3c0b0a10 con 0x7f9e3c0d0510 2024-04-25T13:51:51.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e41852640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c0d0510 0x7f9e3c0b04d0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9e38019f60 tx=0x7f9e380190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:51.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e2e7fc640 1 -- 172.21.15.80:0/1843184696 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e380b0040 con 0x7f9e3c0d0510 2024-04-25T13:51:51.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e2e7fc640 1 -- 172.21.15.80:0/1843184696 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e380bc070 con 0x7f9e3c0d0510 2024-04-25T13:51:51.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e3c0aba10 con 0x7f9e3c0d0510 2024-04-25T13:51:51.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e2e7fc640 1 -- 172.21.15.80:0/1843184696 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e380b5480 con 0x7f9e3c0d0510 2024-04-25T13:51:51.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.255+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e3c14bc10 con 0x7f9e3c0d0510 2024-04-25T13:51:51.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.259+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e3c06a3a0 con 0x7f9e3c0d0510 2024-04-25T13:51:51.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.259+0000 7f9e2e7fc640 1 -- 172.21.15.80:0/1843184696 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9e380b3070 con 0x7f9e3c0d0510 2024-04-25T13:51:51.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.259+0000 7f9e2e7fc640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2006f400 0x7f9e20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.259+0000 7f9e2e7fc640 1 -- 172.21.15.80:0/1843184696 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f9e38128b70 con 0x7f9e3c0d0510 2024-04-25T13:51:51.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.259+0000 7f9e41051640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2006f400 0x7f9e20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.254 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.259+0000 7f9e41051640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2006f400 0x7f9e20071910 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9e30005890 tx=0x7f9e30009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:51.258 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.263+0000 7f9e2e7fc640 1 -- 172.21.15.80:0/1843184696 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9e38015fb0 con 0x7f9e3c0d0510 2024-04-25T13:51:51.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.427+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9e3c0cf5a0 con 0x7f9e3c0d0510 2024-04-25T13:51:51.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e2e7fc640 1 -- 172.21.15.80:0/1843184696 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v186) ==== 74+0+16613 (secure 0 0 0) 0x7f9e380f5a60 con 0x7f9e3c0d0510 2024-04-25T13:51:51.425 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:51.426 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":186,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:50.067310+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:51:48.404349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T13:51:49.039512+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2006f400 msgr2=0x7f9e20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2006f400 0x7f9e20071910 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f9e30005890 tx=0x7f9e30009040 comp rx=0 tx=0).stop 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c0d0510 msgr2=0x7f9e3c0b04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c0d0510 0x7f9e3c0b04d0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f9e38019f60 tx=0x7f9e380190d0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 shutdown_connections 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9e2006f400 0x7f9e20071910 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9e3c15bfe0 0x7f9e3c15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9e3c15ec40 0x7f9e3c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 --2- 172.21.15.80:0/1843184696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9e3c0d0510 0x7f9e3c0b04d0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 >> 172.21.15.80:0/1843184696 conn(0x7f9e3c06d080 msgr2=0x7f9e3c0b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:51.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 shutdown_connections 2024-04-25T13:51:51.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.431+0000 7f9e43ab6640 1 -- 172.21.15.80:0/1843184696 wait complete. 2024-04-25T13:51:51.448 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T13:51:51.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.607+0000 7f4ec17bc640 1 Processor -- start 2024-04-25T13:51:51.604 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.607+0000 7f4ec17bc640 1 -- start start 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ec17bc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4ebc071e60 0x7f4ebc1574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ec17bc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc161170 0x7f4ebc0b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ec17bc640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4ebc06fbf0 con 0x7f4ebc073c40 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ec17bc640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4ebc0b1350 con 0x7f4ebc071e60 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ec17bc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc06f9a0 con 0x7f4ebc161170 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eba7fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4ebc073c40 0x7f4ebc160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41222/0 (socket says 172.21.15.80:41222) 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eba7fc640 1 -- 172.21.15.80:0/3682459338 learned_addr learned my addr 172.21.15.80:0/3682459338 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebaffd640 1 --2- 172.21.15.80:0/3682459338 >> v2:172.21.15.87:3300/0 conn(0x7f4ebc071e60 0x7f4ebc1574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebb7fe640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc161170 0x7f4ebc0b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 482258653 0 0) 0x7f4ebc06fbf0 con 0x7f4ebc073c40 2024-04-25T13:51:51.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4ea4003530 con 0x7f4ebc073c40 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3877346452 0 0) 0x7f4ea4003530 con 0x7f4ebc073c40 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc161170 msgr2=0x7f4ebc0b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc161170 0x7f4ebc0b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 >> v2:172.21.15.87:3300/0 conn(0x7f4ebc071e60 msgr2=0x7f4ebc1574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 --2- 172.21.15.80:0/3682459338 >> v2:172.21.15.87:3300/0 conn(0x7f4ebc071e60 0x7f4ebc1574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc0b0a20 con 0x7f4ebc073c40 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f4eb0001680 con 0x7f4ebc073c40 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 >> v1:172.21.15.80:6789/0 conn(0x7f4ebc073c40 legacy=0x7f4ebc160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea40039c0 0x7f4ea4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea4004810 0x7f4ea4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea4005160 0x7f4ea400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc0b1350 con 0x7f4ea4005160 2024-04-25T13:51:51.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc06f9a0 con 0x7f4ea40039c0 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea400bf10 con 0x7f4ea4004810 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebb7fe640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea4005160 0x7f4ea400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebaffd640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea40039c0 0x7f4ea4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eba7fc640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea4004810 0x7f4ea4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebb7fe640 1 -- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea4004810 msgr2=0x7f4ea4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebb7fe640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea4004810 0x7f4ea4004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebb7fe640 1 -- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea40039c0 msgr2=0x7f4ea4003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebb7fe640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ea40039c0 0x7f4ea4003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4ebb7fe640 1 -- 172.21.15.80:0/3682459338 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea4003530 con 0x7f4ea4005160 2024-04-25T13:51:51.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.611+0000 7f4eba7fc640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ea4004810 0x7f4ea4004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4ebb7fe640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea4005160 0x7f4ea400b9d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0d21b0 tx=0x7f4eb401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b1b9746548265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb40b4070 con 0x7f4ea4005160 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eb40a7c10 con 0x7f4ea4005160 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4eb9ffb640 1 -- 172.21.15.80:0/3682459338 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb400bc10 con 0x7f4ea4005160 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4ec17bc640 1 -- 172.21.15.80:0/3682459338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea4005160 msgr2=0x7f4ea400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4ec17bc640 1 --2- 172.21.15.80:0/3682459338 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ea4005160 0x7f4ea400b9d0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc0d21b0 tx=0x7f4eb401ec00 comp rx=0 tx=0).stop 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4ebb7fe640 1 -- 172.21.15.80:0/3682459338 reap_dead start 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4ec17bc640 1 -- 172.21.15.80:0/3682459338 shutdown_connections 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4ec17bc640 1 -- 172.21.15.80:0/3682459338 >> 172.21.15.80:0/3682459338 conn(0x7f4ebc0b4160 msgr2=0x7f4ebc0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4ec17bc640 1 -- 172.21.15.80:0/3682459338 shutdown_connections 2024-04-25T13:51:51.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.615+0000 7f4ec17bc640 1 -- 172.21.15.80:0/3682459338 wait complete. 2024-04-25T13:51:51.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 Processor -- start 2024-04-25T13:51:51.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 -- start start 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0b29e0 0x7f4ebc0af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc1481d0 0x7f4ebc15ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc15f190 0x7f4ebc0b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc072270 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4ebc0b0fe0 con 0x7f4ebc15f190 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4ebc074050 con 0x7f4ebc1481d0 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0b29e0 0x7f4ebc0af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0b29e0 0x7f4ebc0af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58460/0 (socket says 172.21.15.80:58460) 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 -- 172.21.15.80:0/1057281154 learned_addr learned my addr 172.21.15.80:0/1057281154 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebb7fe640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc15f190 0x7f4ebc0b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4eba7fc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc1481d0 0x7f4ebc15ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 -- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc1481d0 msgr2=0x7f4ebc15ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc1481d0 0x7f4ebc15ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 -- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc15f190 msgr2=0x7f4ebc0b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc15f190 0x7f4ebc0b1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 -- 172.21.15.80:0/1057281154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ebc15f5a0 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ebaffd640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0b29e0 0x7f4ebc0af040 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f4eac011a00 tx=0x7f4eac011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ea37fe640 1 -- 172.21.15.80:0/1057281154 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac00faf0 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ea37fe640 1 -- 172.21.15.80:0/1057281154 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4eac00fc90 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ebc0af7d0 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ea37fe640 1 -- 172.21.15.80:0/1057281154 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eac014440 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.619+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ebc0afd90 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.623+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ebc06a3a0 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.623+0000 7f4ea37fe640 1 -- 172.21.15.80:0/1057281154 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4eac012040 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.623+0000 7f4ea37fe640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ea806f430 0x7f4ea8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:51.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.623+0000 7f4ea37fe640 1 -- 172.21.15.80:0/1057281154 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(186..186 src has 1..186) ==== 5709+0+0 (secure 0 0 0) 0x7f4eac08f870 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.623+0000 7f4eba7fc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ea806f430 0x7f4ea8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:51.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.623+0000 7f4eba7fc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ea806f430 0x7f4ea8071940 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001ec0 tx=0x7f4eb0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:51.622 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.627+0000 7f4ea37fe640 1 -- 172.21.15.80:0/1057281154 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4eac019030 con 0x7f4ebc0b29e0 2024-04-25T13:51:51.641 INFO:tasks.ceph.mds.a.smithi080.stderr:2024-04-25T13:51:51.647+0000 7fd99e42d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:51:51.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:51.831+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4ebc15ddc0 con 0x7f4ebc0b29e0 2024-04-25T13:51:52.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.723+0000 7f4ea37fe640 1 -- 172.21.15.80:0/1057281154 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) ==== 118+0+0 (secure 0 0 0) 0x7f4eac05c8b0 con 0x7f4ebc0b29e0 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ea806f430 msgr2=0x7f4ea8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ea806f430 0x7f4ea8071940 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001ec0 tx=0x7f4eb0009040 comp rx=0 tx=0).stop 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0b29e0 msgr2=0x7f4ebc0af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0b29e0 0x7f4ebc0af040 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f4eac011a00 tx=0x7f4eac011ec0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 shutdown_connections 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4ea806f430 0x7f4ea8071940 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4ebc15f190 0x7f4ebc0b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4ebc1481d0 0x7f4ebc15ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 --2- 172.21.15.80:0/1057281154 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4ebc0b29e0 0x7f4ebc0af040 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 >> 172.21.15.80:0/1057281154 conn(0x7f4ebc0b4160 msgr2=0x7f4ebc153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 shutdown_connections 2024-04-25T13:51:52.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.727+0000 7f4ec17bc640 1 -- 172.21.15.80:0/1057281154 wait complete. 2024-04-25T13:51:52.748 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T13:51:52.749 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T13:51:52.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b37edc640 1 Processor -- start 2024-04-25T13:51:52.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b37edc640 1 -- start start 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b37edc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b30071e60 0x7f5b301574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b37edc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b30161170 0x7f5b300b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b37edc640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3006fbf0 con 0x7f5b30073c40 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b37edc640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5b300b1350 con 0x7f5b30071e60 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b37edc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3006f9a0 con 0x7f5b30161170 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b35c78640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b30071e60 0x7f5b301574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b35c78640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5b30071e60 0x7f5b301574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58466/0 (socket says 172.21.15.80:58466) 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b35c78640 1 -- 172.21.15.80:0/2200385077 learned_addr learned my addr 172.21.15.80:0/2200385077 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b36479640 1 --2- 172.21.15.80:0/2200385077 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b30161170 0x7f5b300b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b35c78640 1 -- 172.21.15.80:0/2200385077 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b30161170 msgr2=0x7f5b300b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b35c78640 1 --2- 172.21.15.80:0/2200385077 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b30161170 0x7f5b300b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b35c78640 1 -- 172.21.15.80:0/2200385077 >> v1:172.21.15.87:6789/0 conn(0x7f5b30073c40 legacy=0x7f5b30160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:52.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.915+0000 7f5b35c78640 1 -- 172.21.15.80:0/2200385077 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b300b0a20 con 0x7f5b30071e60 2024-04-25T13:51:52.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b35c78640 1 --2- 172.21.15.80:0/2200385077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b301574c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009870 tx=0x7f5b2c01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5908e2b3c89505f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:52.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b34c76640 1 -- 172.21.15.80:0/2200385077 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0b1070 con 0x7f5b30071e60 2024-04-25T13:51:52.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b34c76640 1 -- 172.21.15.80:0/2200385077 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2c0ac630 con 0x7f5b30071e60 2024-04-25T13:51:52.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b34c76640 1 -- 172.21.15.80:0/2200385077 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0a8050 con 0x7f5b30071e60 2024-04-25T13:51:52.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b37edc640 1 -- 172.21.15.80:0/2200385077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 msgr2=0x7f5b301574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:52.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b37edc640 1 --2- 172.21.15.80:0/2200385077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b301574c0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c009870 tx=0x7f5b2c01c520 comp rx=0 tx=0).stop 2024-04-25T13:51:52.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b37edc640 1 -- 172.21.15.80:0/2200385077 shutdown_connections 2024-04-25T13:51:52.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b37edc640 1 --2- 172.21.15.80:0/2200385077 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b30161170 0x7f5b300b4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b37edc640 1 --2- 172.21.15.80:0/2200385077 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b301574c0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b37edc640 1 -- 172.21.15.80:0/2200385077 >> 172.21.15.80:0/2200385077 conn(0x7f5b300b4160 msgr2=0x7f5b300d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:52.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b37edc640 1 -- 172.21.15.80:0/2200385077 shutdown_connections 2024-04-25T13:51:52.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.919+0000 7f5b37edc640 1 -- 172.21.15.80:0/2200385077 wait complete. 2024-04-25T13:51:52.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 Processor -- start 2024-04-25T13:51:52.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 -- start start 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b30164040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b30073c40 0x7f5b300afa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b300b3100 0x7f5b300b1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30169bb0 con 0x7f5b30071e60 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5b30074050 con 0x7f5b30073c40 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b30169d30 con 0x7f5b300b3100 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b30164040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b30164040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58474/0 (socket says 172.21.15.80:58474) 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 -- 172.21.15.80:0/3753950570 learned_addr learned my addr 172.21.15.80:0/3753950570 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b36479640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b300b3100 0x7f5b300b1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35477640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b30073c40 0x7f5b300afa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:52.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 -- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b300b3100 msgr2=0x7f5b300b1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b300b3100 0x7f5b300b1500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 -- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b30073c40 msgr2=0x7f5b300afa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b30073c40 0x7f5b300afa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 -- 172.21.15.80:0/3753950570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b30161d10 con 0x7f5b30071e60 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35477640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b30073c40 0x7f5b300afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b35c78640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b30164040 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0af040 tx=0x7f5b2c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b267fc640 1 -- 172.21.15.80:0/3753950570 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0bc070 con 0x7f5b30071e60 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b267fc640 1 -- 172.21.15.80:0/3753950570 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b2c0ab5e0 con 0x7f5b30071e60 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b30161ff0 con 0x7f5b30071e60 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b267fc640 1 -- 172.21.15.80:0/3753950570 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c0b1070 con 0x7f5b30071e60 2024-04-25T13:51:52.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b301625b0 con 0x7f5b30071e60 2024-04-25T13:51:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.923+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b3006a3a0 con 0x7f5b30071e60 2024-04-25T13:51:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.927+0000 7f5b267fc640 1 -- 172.21.15.80:0/3753950570 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5b2c0bf070 con 0x7f5b30071e60 2024-04-25T13:51:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.927+0000 7f5b267fc640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1406f400 0x7f5b14071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.927+0000 7f5b267fc640 1 -- 172.21.15.80:0/3753950570 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f5b2c128db0 con 0x7f5b30071e60 2024-04-25T13:51:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.927+0000 7f5b35477640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1406f400 0x7f5b14071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:52.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.927+0000 7f5b35477640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1406f400 0x7f5b14071910 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5b300b4b70 tx=0x7f5b20009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:52.925 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:52.931+0000 7f5b267fc640 1 -- 172.21.15.80:0/3753950570 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b2c016110 con 0x7f5b30071e60 2024-04-25T13:51:53.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.123+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f5b3015d5f0 con 0x7f5b30071e60 2024-04-25T13:51:53.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.723+0000 7f5b267fc640 1 -- 172.21.15.80:0/3753950570 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) ==== 122+0+0 (secure 0 0 0) 0x7f5b2c0f5d30 con 0x7f5b30071e60 2024-04-25T13:51:53.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1406f400 msgr2=0x7f5b14071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:53.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1406f400 0x7f5b14071910 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f5b300b4b70 tx=0x7f5b20009040 comp rx=0 tx=0).stop 2024-04-25T13:51:53.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 msgr2=0x7f5b30164040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:53.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b30164040 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0af040 tx=0x7f5b2c0192c0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 shutdown_connections 2024-04-25T13:51:53.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1406f400 0x7f5b14071910 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b300b3100 0x7f5b300b1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.723 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b30073c40 0x7f5b300afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 --2- 172.21.15.80:0/3753950570 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b30071e60 0x7f5b30164040 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 >> 172.21.15.80:0/3753950570 conn(0x7f5b300b4160 msgr2=0x7f5b301607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:53.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 shutdown_connections 2024-04-25T13:51:53.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.727+0000 7f5b37edc640 1 -- 172.21.15.80:0/3753950570 wait complete. 2024-04-25T13:51:53.746 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T13:51:53.746 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:53.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b8920640 1 Processor -- start 2024-04-25T13:51:53.897 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b8920640 1 -- start start 2024-04-25T13:51:53.898 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b8920640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b4068590 0x7fd5b40689a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:53.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b8920640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b4068ee0 con 0x7fd5b40c7d40 2024-04-25T13:51:53.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b8920640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5b40b1e40 con 0x7fd5b40b1a30 2024-04-25T13:51:53.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b8920640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b40d2520 con 0x7fd5b4068590 2024-04-25T13:51:53.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b2d9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fd5b40c7d40 0x7fd5b40c8150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41224/0 (socket says 172.21.15.80:41224) 2024-04-25T13:51:53.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b2d9d640 1 -- 172.21.15.80:0/2490780305 learned_addr learned my addr 172.21.15.80:0/2490780305 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:53.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b1d9b640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b4068590 0x7fd5b40689a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:53.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3331832052 0 0) 0x7fd5a40014a0 con 0x7fd5b40c7d40 2024-04-25T13:51:53.899 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5a0003530 con 0x7fd5b40c7d40 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 492071228 0 0) 0x7fd5a0003530 con 0x7fd5b40c7d40 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b4068590 msgr2=0x7fd5b40689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b4068590 0x7fd5b40689a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 >> v1:172.21.15.87:6789/0 conn(0x7fd5b40b1a30 legacy=0x7fd5b415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b41627d0 con 0x7fd5b40c7d40 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fd5a4001bd0 con 0x7fd5b40c7d40 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 >> v1:172.21.15.80:6789/0 conn(0x7fd5b40c7d40 legacy=0x7fd5b40c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5a00039f0 0x7fd5a0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b4068590 0x7fd5a0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a0004d50 0x7fd5a0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:53.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b40d2520 con 0x7fd5a00039f0 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a000bbc0 con 0x7fd5a0004d50 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a000bdc0 con 0x7fd5b4068590 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b259c640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5a00039f0 0x7fd5a0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b2d9d640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a0004d50 0x7fd5a0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.903+0000 7fd5b1d9b640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b4068590 0x7fd5a0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b259c640 1 -- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b4068590 msgr2=0x7fd5a0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b259c640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b4068590 0x7fd5a0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b259c640 1 -- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a0004d50 msgr2=0x7fd5a0005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b259c640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a0004d50 0x7fd5a0005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b259c640 1 -- 172.21.15.80:0/2490780305 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a0003530 con 0x7fd5a00039f0 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b259c640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5a00039f0 0x7fd5a0003e60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac001750 tx=0x7fd5ac01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18445137794f3b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac0013a0 con 0x7fd5a00039f0 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5ac0052f0 con 0x7fd5a00039f0 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b159a640 1 -- 172.21.15.80:0/2490780305 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac003840 con 0x7fd5a00039f0 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b8920640 1 -- 172.21.15.80:0/2490780305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5a00039f0 msgr2=0x7fd5a0003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b8920640 1 --2- 172.21.15.80:0/2490780305 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5a00039f0 0x7fd5a0003e60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac001750 tx=0x7fd5ac01c3b0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b2d9d640 1 -- 172.21.15.80:0/2490780305 reap_dead start 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b8920640 1 -- 172.21.15.80:0/2490780305 shutdown_connections 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b8920640 1 -- 172.21.15.80:0/2490780305 >> 172.21.15.80:0/2490780305 conn(0x7fd5b40cf390 msgr2=0x7fd5b40cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b8920640 1 -- 172.21.15.80:0/2490780305 shutdown_connections 2024-04-25T13:51:53.904 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.907+0000 7fd5b8920640 1 -- 172.21.15.80:0/2490780305 wait complete. 2024-04-25T13:51:53.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 Processor -- start 2024-04-25T13:51:53.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 -- start start 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a00039f0 0x7fd5b4161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b415e780 0x7fd5b414b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5b415f260 0x7fd5b415f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b416a030 con 0x7fd5b415f260 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd5b40d0aa0 con 0x7fd5a00039f0 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd5b416a1b0 con 0x7fd5b415e780 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5b415f260 0x7fd5b415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5b415f260 0x7fd5b415f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58498/0 (socket says 172.21.15.80:58498) 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 -- 172.21.15.80:0/1213134115 learned_addr learned my addr 172.21.15.80:0/1213134115 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:53.908 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b259c640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a00039f0 0x7fd5b4161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b1d9b640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b415e780 0x7fd5b414b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 -- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b415e780 msgr2=0x7fd5b414b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b415e780 0x7fd5b414b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 -- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a00039f0 msgr2=0x7fd5b4161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a00039f0 0x7fd5b4161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 -- 172.21.15.80:0/1213134115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5b415fbb0 con 0x7fd5b415f260 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b1d9b640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b415e780 0x7fd5b414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b2d9d640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5b415f260 0x7fd5b415f670 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40b17f0 tx=0x7fd5a400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5aaffd640 1 -- 172.21.15.80:0/1213134115 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4019070 con 0x7fd5b415f260 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5aaffd640 1 -- 172.21.15.80:0/1213134115 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a400ec40 con 0x7fd5b415f260 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5b4161a20 con 0x7fd5b415f260 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5aaffd640 1 -- 172.21.15.80:0/1213134115 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4014480 con 0x7fd5b415f260 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.911+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5b406f260 con 0x7fd5b415f260 2024-04-25T13:51:53.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.915+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5b406a3a0 con 0x7fd5b415f260 2024-04-25T13:51:53.910 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.915+0000 7fd5aaffd640 1 -- 172.21.15.80:0/1213134115 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd5a401c070 con 0x7fd5b415f260 2024-04-25T13:51:53.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.915+0000 7fd5aaffd640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd59406f400 0x7fd594071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:53.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.915+0000 7fd5aaffd640 1 -- 172.21.15.80:0/1213134115 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7fd5a408efd0 con 0x7fd5b415f260 2024-04-25T13:51:53.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.915+0000 7fd5b259c640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd59406f400 0x7fd594071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:53.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.915+0000 7fd5b259c640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd59406f400 0x7fd594071910 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4072b20 tx=0x7fd5ac0011a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:53.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:53.919+0000 7fd5aaffd640 1 -- 172.21.15.80:0/1213134115 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd5b415fbb0 con 0x7fd5b415f260 2024-04-25T13:51:54.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.103+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5b406da50 con 0x7fd5b415f260 2024-04-25T13:51:54.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5aaffd640 1 -- 172.21.15.80:0/1213134115 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7fd5b4068ee0 con 0x7fd5b415f260 2024-04-25T13:51:54.101 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:54.101 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:51:54.101 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:51:54.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd59406f400 msgr2=0x7fd594071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd59406f400 0x7fd594071910 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4072b20 tx=0x7fd5ac0011a0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5b415f260 msgr2=0x7fd5b415f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5b415f260 0x7fd5b415f670 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fd5b40b17f0 tx=0x7fd5a400e530 comp rx=0 tx=0).stop 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 shutdown_connections 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd59406f400 0x7fd594071910 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd5b415f260 0x7fd5b415f670 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd5b415e780 0x7fd5b414b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 --2- 172.21.15.80:0/1213134115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5a00039f0 0x7fd5b4161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 >> 172.21.15.80:0/1213134115 conn(0x7fd5b40cf390 msgr2=0x7fd5b4162db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 shutdown_connections 2024-04-25T13:51:54.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.107+0000 7fd5b8920640 1 -- 172.21.15.80:0/1213134115 wait complete. 2024-04-25T13:51:54.121 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:51:54.271 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.275+0000 7f5ee19e3640 1 Processor -- start 2024-04-25T13:51:54.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.275+0000 7f5ee19e3640 1 -- start start 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5ee19e3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5edc161f90 0x7f5edc0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5ee19e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc0d0510 0x7f5edc15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5ee19e3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5edc0d0d70 con 0x7f5edc15ea70 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5ee19e3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f5edc0d1050 con 0x7f5edc161f90 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5ee19e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc06dca0 con 0x7f5edc0d0510 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5edaffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f5edc161f90 0x7f5edc0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5edb7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5edc15ea70 0x7f5edc15ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41230/0 (socket says 172.21.15.80:41230) 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5edb7fe640 1 -- 172.21.15.80:0/3993404362 learned_addr learned my addr 172.21.15.80:0/3993404362 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:54.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5eda7fc640 1 --2- 172.21.15.80:0/3993404362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc0d0510 0x7f5edc15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5edaffd640 1 -- 172.21.15.80:0/3993404362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc0d0510 msgr2=0x7f5edc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:54.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5edaffd640 1 --2- 172.21.15.80:0/3993404362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc0d0510 0x7f5edc15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5edaffd640 1 -- 172.21.15.80:0/3993404362 >> v1:172.21.15.80:6789/0 conn(0x7f5edc15ea70 legacy=0x7f5edc15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5edaffd640 1 -- 172.21.15.80:0/3993404362 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc06df80 con 0x7f5edc161f90 2024-04-25T13:51:54.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.279+0000 7f5ed9ffb640 1 -- 172.21.15.80:0/3993404362 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1935058985 0 0) 0x7f5edc0d0d70 con 0x7f5edc15ea70 2024-04-25T13:51:54.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5edaffd640 1 --2- 172.21.15.80:0/3993404362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc0cffd0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40097c0 tx=0x7f5ed401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89605f7386b60a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:54.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ed9ffb640 1 -- 172.21.15.80:0/3993404362 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40b0070 con 0x7f5edc161f90 2024-04-25T13:51:54.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ed9ffb640 1 -- 172.21.15.80:0/3993404362 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ed40ab600 con 0x7f5edc161f90 2024-04-25T13:51:54.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ed9ffb640 1 -- 172.21.15.80:0/3993404362 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed40ab9d0 con 0x7f5edc161f90 2024-04-25T13:51:54.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 -- 172.21.15.80:0/3993404362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 msgr2=0x7f5edc0cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/3993404362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc0cffd0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40097c0 tx=0x7f5ed401c3b0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 -- 172.21.15.80:0/3993404362 shutdown_connections 2024-04-25T13:51:54.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/3993404362 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc0d0510 0x7f5edc15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.277 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/3993404362 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc0cffd0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 -- 172.21.15.80:0/3993404362 >> 172.21.15.80:0/3993404362 conn(0x7f5edc06d080 msgr2=0x7f5edc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:54.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 -- 172.21.15.80:0/3993404362 shutdown_connections 2024-04-25T13:51:54.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 -- 172.21.15.80:0/3993404362 wait complete. 2024-04-25T13:51:54.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 Processor -- start 2024-04-25T13:51:54.279 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.283+0000 7f5ee19e3640 1 -- start start 2024-04-25T13:51:54.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5edc0d0510 0x7f5edc13fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc15ea70 0x7f5edc159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc06f4b0 con 0x7f5edc161f90 2024-04-25T13:51:54.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc06f630 con 0x7f5edc0d0510 2024-04-25T13:51:54.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc0d0920 con 0x7f5edc15ea70 2024-04-25T13:51:54.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.281 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc15a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58522/0 (socket says 172.21.15.80:58522) 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 -- 172.21.15.80:0/1104613953 learned_addr learned my addr 172.21.15.80:0/1104613953 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5eda7fc640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc15ea70 0x7f5edc159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edaffd640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5edc0d0510 0x7f5edc13fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 -- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc15ea70 msgr2=0x7f5edc159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc15ea70 0x7f5edc159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 -- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5edc0d0510 msgr2=0x7f5edc13fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5edc0d0510 0x7f5edc13fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 -- 172.21.15.80:0/1104613953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc160dd0 con 0x7f5edc161f90 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edaffd640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5edc0d0510 0x7f5edc13fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edb7fe640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc15a170 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c7b0 tx=0x7f5ecc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ed3fff640 1 -- 172.21.15.80:0/1104613953 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc018070 con 0x7f5edc161f90 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ed3fff640 1 -- 172.21.15.80:0/1104613953 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ecc00a640 con 0x7f5edc161f90 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5edc1610b0 con 0x7f5edc161f90 2024-04-25T13:51:54.282 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ed3fff640 1 -- 172.21.15.80:0/1104613953 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc00ce60 con 0x7f5edc161f90 2024-04-25T13:51:54.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5edc161670 con 0x7f5edc161f90 2024-04-25T13:51:54.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5edc160b40 con 0x7f5edc161f90 2024-04-25T13:51:54.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ed3fff640 1 -- 172.21.15.80:0/1104613953 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5ecc01b070 con 0x7f5edc161f90 2024-04-25T13:51:54.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ed3fff640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ebc06f400 0x7f5ebc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.283 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5ed3fff640 1 -- 172.21.15.80:0/1104613953 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f5ecc08e760 con 0x7f5edc161f90 2024-04-25T13:51:54.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.287+0000 7f5edaffd640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ebc06f400 0x7f5ebc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.284 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.291+0000 7f5edaffd640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ebc06f400 0x7f5ebc071910 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ad000 tx=0x7f5ed4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:54.288 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.295+0000 7f5ed3fff640 1 -- 172.21.15.80:0/1104613953 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5ecc05b770 con 0x7f5edc161f90 2024-04-25T13:51:54.444 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.451+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5edc155860 con 0x7f5edc161f90 2024-04-25T13:51:54.447 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.451+0000 7f5ed3fff640 1 -- 172.21.15.80:0/1104613953 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) ==== 74+0+16600 (secure 0 0 0) 0x7f5ecc05b070 con 0x7f5edc161f90 2024-04-25T13:51:54.447 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:54.448 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":187,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:51:51.668047+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:51:48.404349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T13:51:49.039512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"187","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ebc06f400 msgr2=0x7f5ebc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ebc06f400 0x7f5ebc071910 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5ed40ad000 tx=0x7f5ed4008040 comp rx=0 tx=0).stop 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 msgr2=0x7f5edc15a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc15a170 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00c7b0 tx=0x7f5ecc00cc70 comp rx=0 tx=0).stop 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 shutdown_connections 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5ebc06f400 0x7f5ebc071910 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5edc161f90 0x7f5edc15a170 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5edc15ea70 0x7f5edc159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 --2- 172.21.15.80:0/1104613953 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5edc0d0510 0x7f5edc13fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 >> 172.21.15.80:0/1104613953 conn(0x7f5edc06d080 msgr2=0x7f5edc0b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:54.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 shutdown_connections 2024-04-25T13:51:54.451 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.455+0000 7f5ee19e3640 1 -- 172.21.15.80:0/1104613953 wait complete. 2024-04-25T13:51:54.472 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T13:51:54.472 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:51:54.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f147b9ad640 1 Processor -- start 2024-04-25T13:51:54.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f147b9ad640 1 -- start start 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f147b9ad640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f147406f5a0 0x7f1474160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f147b9ad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161070 0x7f1474161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f147b9ad640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f147406fcf0 con 0x7f14740b4bd0 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f147b9ad640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f14741619c0 con 0x7f147406f5a0 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f147b9ad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1474073f10 con 0x7f1474161070 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1479f4a640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f14740b4bd0 0x7f14740b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41232/0 (socket says 172.21.15.80:41232) 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1479f4a640 1 -- 172.21.15.80:0/3087829289 learned_addr learned my addr 172.21.15.80:0/3087829289 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1479749640 1 --2- 172.21.15.80:0/3087829289 >> v2:172.21.15.87:3300/0 conn(0x7f147406f5a0 0x7f1474160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1478f48640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161070 0x7f1474161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 905401892 0 0) 0x7f147406fcf0 con 0x7f14740b4bd0 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f145c003530 con 0x7f14740b4bd0 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1172632168 0 0) 0x7f145c003530 con 0x7f14740b4bd0 2024-04-25T13:51:54.617 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161070 msgr2=0x7f1474161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161070 0x7f1474161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 >> v2:172.21.15.87:3300/0 conn(0x7f147406f5a0 msgr2=0x7f1474160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 --2- 172.21.15.80:0/3087829289 >> v2:172.21.15.87:3300/0 conn(0x7f147406f5a0 0x7f1474160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14740b0c60 con 0x7f14740b4bd0 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1464001a00 con 0x7f14740b4bd0 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 >> v1:172.21.15.80:6789/0 conn(0x7f14740b4bd0 legacy=0x7f14740b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.619+0000 7f1473fff640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161070 0x7f145c0039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1473fff640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f145c003f70 0x7f145c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1473fff640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f145c004d50 0x7f145c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f14741619c0 con 0x7f145c003f70 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1474073f10 con 0x7f145c004d50 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f145c00bb00 con 0x7f1474161070 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1478f48640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f145c003f70 0x7f145c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1479749640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161070 0x7f145c0039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1479f4a640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f145c004d50 0x7f145c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1478f48640 1 -- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161070 msgr2=0x7f145c0039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:54.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1478f48640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161070 0x7f145c0039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1478f48640 1 -- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f145c004d50 msgr2=0x7f145c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:54.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1478f48640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f145c004d50 0x7f145c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1478f48640 1 -- 172.21.15.80:0/3087829289 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145c003530 con 0x7f145c003f70 2024-04-25T13:51:54.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1478f48640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f145c003f70 0x7f145c004810 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f1468009ae0 tx=0x7f146801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b922040e6beb62c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:54.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680b2070 con 0x7f145c003f70 2024-04-25T13:51:54.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14680a5550 con 0x7f145c003f70 2024-04-25T13:51:54.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1473fff640 1 -- 172.21.15.80:0/3087829289 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680096b0 con 0x7f145c003f70 2024-04-25T13:51:54.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f147b9ad640 1 -- 172.21.15.80:0/3087829289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f145c003f70 msgr2=0x7f145c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f147b9ad640 1 --2- 172.21.15.80:0/3087829289 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f145c003f70 0x7f145c004810 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f1468009ae0 tx=0x7f146801ca90 comp rx=0 tx=0).stop 2024-04-25T13:51:54.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f1479f4a640 1 -- 172.21.15.80:0/3087829289 reap_dead start 2024-04-25T13:51:54.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f147b9ad640 1 -- 172.21.15.80:0/3087829289 shutdown_connections 2024-04-25T13:51:54.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f147b9ad640 1 -- 172.21.15.80:0/3087829289 >> 172.21.15.80:0/3087829289 conn(0x7f147406d080 msgr2=0x7f14740b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:54.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f147b9ad640 1 -- 172.21.15.80:0/3087829289 shutdown_connections 2024-04-25T13:51:54.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.623+0000 7f147b9ad640 1 -- 172.21.15.80:0/3087829289 wait complete. 2024-04-25T13:51:54.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 Processor -- start 2024-04-25T13:51:54.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 -- start start 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f145c003f70 0x7f14740d1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0450 0x7f14740d2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1474154fa0 0x7f1474157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1474161600 con 0x7f1474154fa0 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1474161480 con 0x7f14740d0450 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1474161780 con 0x7f145c003f70 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1474154fa0 0x7f1474157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1474154fa0 0x7f1474157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58540/0 (socket says 172.21.15.80:58540) 2024-04-25T13:51:54.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 -- 172.21.15.80:0/3192120252 learned_addr learned my addr 172.21.15.80:0/3192120252 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1478f48640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0450 0x7f14740d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479749640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f145c003f70 0x7f14740d1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 -- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f145c003f70 msgr2=0x7f14740d1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f145c003f70 0x7f14740d1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 -- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0450 msgr2=0x7f14740d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0450 0x7f14740d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 -- 172.21.15.80:0/3192120252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14740cfa20 con 0x7f1474154fa0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1478f48640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0450 0x7f14740d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1479f4a640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1474154fa0 0x7f1474157f20 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f1464002080 tx=0x7f1464007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1471ffb640 1 -- 172.21.15.80:0/3192120252 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146400aca0 con 0x7f1474154fa0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1471ffb640 1 -- 172.21.15.80:0/3192120252 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1464019070 con 0x7f1474154fa0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f147415ab60 con 0x7f1474154fa0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f1471ffb640 1 -- 172.21.15.80:0/3192120252 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640135f0 con 0x7f1474154fa0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.627+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14741566f0 con 0x7f1474154fa0 2024-04-25T13:51:54.625 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.631+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f147406a3a0 con 0x7f1474154fa0 2024-04-25T13:51:54.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.631+0000 7f1471ffb640 1 -- 172.21.15.80:0/3192120252 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1464013790 con 0x7f1474154fa0 2024-04-25T13:51:54.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.631+0000 7f1471ffb640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f3a0 0x7f14600718b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.626 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.631+0000 7f1471ffb640 1 -- 172.21.15.80:0/3192120252 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f146408f860 con 0x7f1474154fa0 2024-04-25T13:51:54.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.631+0000 7f1479749640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f3a0 0x7f14600718b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.627 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.631+0000 7f1479749640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f3a0 0x7f14600718b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f14740b6e70 tx=0x7f146c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:54.630 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.635+0000 7f1471ffb640 1 -- 172.21.15.80:0/3192120252 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f146405c7e0 con 0x7f1474154fa0 2024-04-25T13:51:54.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.783+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f147415ae40 con 0x7f146006f3a0 2024-04-25T13:51:54.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.799+0000 7f1471ffb640 1 -- 172.21.15.80:0/3192120252 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247988 (secure 0 0 0) 0x7f147415ae40 con 0x7f146006f3a0 2024-04-25T13:51:54.796 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:54.798 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:51:54.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f3a0 msgr2=0x7f14600718b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f3a0 0x7f14600718b0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f14740b6e70 tx=0x7f146c008040 comp rx=0 tx=0).stop 2024-04-25T13:51:54.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1474154fa0 msgr2=0x7f1474157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1474154fa0 0x7f1474157f20 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f1464002080 tx=0x7f1464007b20 comp rx=0 tx=0).stop 2024-04-25T13:51:54.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 shutdown_connections 2024-04-25T13:51:54.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f3a0 0x7f14600718b0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1474154fa0 0x7f1474157f20 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0450 0x7f14740d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 --2- 172.21.15.80:0/3192120252 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f145c003f70 0x7f14740d1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 >> 172.21.15.80:0/3192120252 conn(0x7f147406d080 msgr2=0x7f14740b4fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:54.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 shutdown_connections 2024-04-25T13:51:54.800 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.803+0000 7f147b9ad640 1 -- 172.21.15.80:0/3192120252 wait complete. 2024-04-25T13:51:54.815 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2024-04-25T13:51:54.355029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9401,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553098,"statfs":{"total":1600000000,"available":1590372364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.699844"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785413+0000","last_change":"2024-04-25T13:51:50.697229+0000","last_active":"2024-04-25T13:51:51.785413+0000","last_peered":"2024-04-25T13:51:51.785413+0000","last_clean":"2024-04-25T13:51:51.785413+0000","last_became_active":"2024-04-25T13:51:50.688184+0000","last_became_peered":"2024-04-25T13:51:50.688184+0000","last_unstale":"2024-04-25T13:51:51.785413+0000","last_undegraded":"2024-04-25T13:51:51.785413+0000","last_fullsized":"2024-04-25T13:51:51.785413+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:41.511822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782652+0000","last_change":"2024-04-25T13:51:50.702744+0000","last_active":"2024-04-25T13:51:51.782652+0000","last_peered":"2024-04-25T13:51:51.782652+0000","last_clean":"2024-04-25T13:51:51.782652+0000","last_became_active":"2024-04-25T13:51:50.697026+0000","last_became_peered":"2024-04-25T13:51:50.697026+0000","last_unstale":"2024-04-25T13:51:51.782652+0000","last_undegraded":"2024-04-25T13:51:51.782652+0000","last_fullsized":"2024-04-25T13:51:51.782652+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:25.676539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719778+0000","last_change":"2024-04-25T13:51:50.711083+0000","last_active":"2024-04-25T13:51:51.719778+0000","last_peered":"2024-04-25T13:51:51.719778+0000","last_clean":"2024-04-25T13:51:51.719778+0000","last_became_active":"2024-04-25T13:51:50.709923+0000","last_became_peered":"2024-04-25T13:51:50.709923+0000","last_unstale":"2024-04-25T13:51:51.719778+0000","last_undegraded":"2024-04-25T13:51:51.719778+0000","last_fullsized":"2024-04-25T13:51:51.719778+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:19.593245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782805+0000","last_change":"2024-04-25T13:51:50.709572+0000","last_active":"2024-04-25T13:51:51.782805+0000","last_peered":"2024-04-25T13:51:51.782805+0000","last_clean":"2024-04-25T13:51:51.782805+0000","last_became_active":"2024-04-25T13:51:50.706315+0000","last_became_peered":"2024-04-25T13:51:50.706315+0000","last_unstale":"2024-04-25T13:51:51.782805+0000","last_undegraded":"2024-04-25T13:51:51.782805+0000","last_fullsized":"2024-04-25T13:51:51.782805+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:59.728167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783079+0000","last_change":"2024-04-25T13:51:50.702171+0000","last_active":"2024-04-25T13:51:51.783079+0000","last_peered":"2024-04-25T13:51:51.783079+0000","last_clean":"2024-04-25T13:51:51.783079+0000","last_became_active":"2024-04-25T13:51:50.701004+0000","last_became_peered":"2024-04-25T13:51:50.701004+0000","last_unstale":"2024-04-25T13:51:51.783079+0000","last_undegraded":"2024-04-25T13:51:51.783079+0000","last_fullsized":"2024-04-25T13:51:51.783079+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:04.887335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783544+0000","last_change":"2024-04-25T13:51:50.694741+0000","last_active":"2024-04-25T13:51:51.783544+0000","last_peered":"2024-04-25T13:51:51.783544+0000","last_clean":"2024-04-25T13:51:51.783544+0000","last_became_active":"2024-04-25T13:51:50.694020+0000","last_became_peered":"2024-04-25T13:51:50.694020+0000","last_unstale":"2024-04-25T13:51:51.783544+0000","last_undegraded":"2024-04-25T13:51:51.783544+0000","last_fullsized":"2024-04-25T13:51:51.783544+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:00.521742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786256+0000","last_change":"2024-04-25T13:51:50.712643+0000","last_active":"2024-04-25T13:51:51.786256+0000","last_peered":"2024-04-25T13:51:51.786256+0000","last_clean":"2024-04-25T13:51:51.786256+0000","last_became_active":"2024-04-25T13:51:50.704921+0000","last_became_peered":"2024-04-25T13:51:50.704921+0000","last_unstale":"2024-04-25T13:51:51.786256+0000","last_undegraded":"2024-04-25T13:51:51.786256+0000","last_fullsized":"2024-04-25T13:51:51.786256+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:33.236236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783163+0000","last_change":"2024-04-25T13:51:50.712752+0000","last_active":"2024-04-25T13:51:51.783163+0000","last_peered":"2024-04-25T13:51:51.783163+0000","last_clean":"2024-04-25T13:51:51.783163+0000","last_became_active":"2024-04-25T13:51:50.711972+0000","last_became_peered":"2024-04-25T13:51:50.711972+0000","last_unstale":"2024-04-25T13:51:51.783163+0000","last_undegraded":"2024-04-25T13:51:51.783163+0000","last_fullsized":"2024-04-25T13:51:51.783163+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:34.577388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783246+0000","last_change":"2024-04-25T13:51:50.695646+0000","last_active":"2024-04-25T13:51:51.783246+0000","last_peered":"2024-04-25T13:51:51.783246+0000","last_clean":"2024-04-25T13:51:51.783246+0000","last_became_active":"2024-04-25T13:51:50.692814+0000","last_became_peered":"2024-04-25T13:51:50.692814+0000","last_unstale":"2024-04-25T13:51:51.783246+0000","last_undegraded":"2024-04-25T13:51:51.783246+0000","last_fullsized":"2024-04-25T13:51:51.783246+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:36.237846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783573+0000","last_change":"2024-04-25T13:51:50.698102+0000","last_active":"2024-04-25T13:51:51.783573+0000","last_peered":"2024-04-25T13:51:51.783573+0000","last_clean":"2024-04-25T13:51:51.783573+0000","last_became_active":"2024-04-25T13:51:50.692066+0000","last_became_peered":"2024-04-25T13:51:50.692066+0000","last_unstale":"2024-04-25T13:51:51.783573+0000","last_undegraded":"2024-04-25T13:51:51.783573+0000","last_fullsized":"2024-04-25T13:51:51.783573+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:18.999599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783014+0000","last_change":"2024-04-25T13:51:50.704691+0000","last_active":"2024-04-25T13:51:51.783014+0000","last_peered":"2024-04-25T13:51:51.783014+0000","last_clean":"2024-04-25T13:51:51.783014+0000","last_became_active":"2024-04-25T13:51:50.703945+0000","last_became_peered":"2024-04-25T13:51:50.703945+0000","last_unstale":"2024-04-25T13:51:51.783014+0000","last_undegraded":"2024-04-25T13:51:51.783014+0000","last_fullsized":"2024-04-25T13:51:51.783014+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:59.914623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784037+0000","last_change":"2024-04-25T13:51:50.701268+0000","last_active":"2024-04-25T13:51:51.784037+0000","last_peered":"2024-04-25T13:51:51.784037+0000","last_clean":"2024-04-25T13:51:51.784037+0000","last_became_active":"2024-04-25T13:51:50.694246+0000","last_became_peered":"2024-04-25T13:51:50.694246+0000","last_unstale":"2024-04-25T13:51:51.784037+0000","last_undegraded":"2024-04-25T13:51:51.784037+0000","last_fullsized":"2024-04-25T13:51:51.784037+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:43.634430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718663+0000","last_change":"2024-04-25T13:51:50.707493+0000","last_active":"2024-04-25T13:51:51.718663+0000","last_peered":"2024-04-25T13:51:51.718663+0000","last_clean":"2024-04-25T13:51:51.718663+0000","last_became_active":"2024-04-25T13:51:50.706316+0000","last_became_peered":"2024-04-25T13:51:50.706316+0000","last_unstale":"2024-04-25T13:51:51.718663+0000","last_undegraded":"2024-04-25T13:51:51.718663+0000","last_fullsized":"2024-04-25T13:51:51.718663+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:12.987095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718367+0000","last_change":"2024-04-25T13:51:50.703773+0000","last_active":"2024-04-25T13:51:51.718367+0000","last_peered":"2024-04-25T13:51:51.718367+0000","last_clean":"2024-04-25T13:51:51.718367+0000","last_became_active":"2024-04-25T13:51:50.697623+0000","last_became_peered":"2024-04-25T13:51:50.697623+0000","last_unstale":"2024-04-25T13:51:51.718367+0000","last_undegraded":"2024-04-25T13:51:51.718367+0000","last_fullsized":"2024-04-25T13:51:51.718367+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:20.821609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783952+0000","last_change":"2024-04-25T13:51:50.687296+0000","last_active":"2024-04-25T13:51:51.783952+0000","last_peered":"2024-04-25T13:51:51.783952+0000","last_clean":"2024-04-25T13:51:51.783952+0000","last_became_active":"2024-04-25T13:51:50.686370+0000","last_became_peered":"2024-04-25T13:51:50.686370+0000","last_unstale":"2024-04-25T13:51:51.783952+0000","last_undegraded":"2024-04-25T13:51:51.783952+0000","last_fullsized":"2024-04-25T13:51:51.783952+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:50.850144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786453+0000","last_change":"2024-04-25T13:51:50.703700+0000","last_active":"2024-04-25T13:51:51.786453+0000","last_peered":"2024-04-25T13:51:51.786453+0000","last_clean":"2024-04-25T13:51:51.786453+0000","last_became_active":"2024-04-25T13:51:50.701557+0000","last_became_peered":"2024-04-25T13:51:50.701557+0000","last_unstale":"2024-04-25T13:51:51.786453+0000","last_undegraded":"2024-04-25T13:51:51.786453+0000","last_fullsized":"2024-04-25T13:51:51.786453+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:31.264908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783333+0000","last_change":"2024-04-25T13:51:50.702423+0000","last_active":"2024-04-25T13:51:51.783333+0000","last_peered":"2024-04-25T13:51:51.783333+0000","last_clean":"2024-04-25T13:51:51.783333+0000","last_became_active":"2024-04-25T13:51:50.701373+0000","last_became_peered":"2024-04-25T13:51:50.701373+0000","last_unstale":"2024-04-25T13:51:51.783333+0000","last_undegraded":"2024-04-25T13:51:51.783333+0000","last_fullsized":"2024-04-25T13:51:51.783333+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:11.722868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.677156+0000","last_change":"2024-04-25T13:51:50.706454+0000","last_active":"2024-04-25T13:51:51.677156+0000","last_peered":"2024-04-25T13:51:51.677156+0000","last_clean":"2024-04-25T13:51:51.677156+0000","last_became_active":"2024-04-25T13:51:50.702993+0000","last_became_peered":"2024-04-25T13:51:50.702993+0000","last_unstale":"2024-04-25T13:51:51.677156+0000","last_undegraded":"2024-04-25T13:51:51.677156+0000","last_fullsized":"2024-04-25T13:51:51.677156+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:05.767580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783182+0000","last_change":"2024-04-25T13:51:50.717854+0000","last_active":"2024-04-25T13:51:51.783182+0000","last_peered":"2024-04-25T13:51:51.783182+0000","last_clean":"2024-04-25T13:51:51.783182+0000","last_became_active":"2024-04-25T13:51:50.717193+0000","last_became_peered":"2024-04-25T13:51:50.717193+0000","last_unstale":"2024-04-25T13:51:51.783182+0000","last_undegraded":"2024-04-25T13:51:51.783182+0000","last_fullsized":"2024-04-25T13:51:51.783182+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:14.615894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718496+0000","last_change":"2024-04-25T13:51:50.694370+0000","last_active":"2024-04-25T13:51:51.718496+0000","last_peered":"2024-04-25T13:51:51.718496+0000","last_clean":"2024-04-25T13:51:51.718496+0000","last_became_active":"2024-04-25T13:51:50.693039+0000","last_became_peered":"2024-04-25T13:51:50.693039+0000","last_unstale":"2024-04-25T13:51:51.718496+0000","last_undegraded":"2024-04-25T13:51:51.718496+0000","last_fullsized":"2024-04-25T13:51:51.718496+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:10.571985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783707+0000","last_change":"2024-04-25T13:51:50.702590+0000","last_active":"2024-04-25T13:51:51.783707+0000","last_peered":"2024-04-25T13:51:51.783707+0000","last_clean":"2024-04-25T13:51:51.783707+0000","last_became_active":"2024-04-25T13:51:50.701912+0000","last_became_peered":"2024-04-25T13:51:50.701912+0000","last_unstale":"2024-04-25T13:51:51.783707+0000","last_undegraded":"2024-04-25T13:51:51.783707+0000","last_fullsized":"2024-04-25T13:51:51.783707+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:41.589275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784336+0000","last_change":"2024-04-25T13:51:50.705759+0000","last_active":"2024-04-25T13:51:51.784336+0000","last_peered":"2024-04-25T13:51:51.784336+0000","last_clean":"2024-04-25T13:51:51.784336+0000","last_became_active":"2024-04-25T13:51:50.700029+0000","last_became_peered":"2024-04-25T13:51:50.700029+0000","last_unstale":"2024-04-25T13:51:51.784336+0000","last_undegraded":"2024-04-25T13:51:51.784336+0000","last_fullsized":"2024-04-25T13:51:51.784336+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:51.708194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785262+0000","last_change":"2024-04-25T13:51:50.705583+0000","last_active":"2024-04-25T13:51:51.785262+0000","last_peered":"2024-04-25T13:51:51.785262+0000","last_clean":"2024-04-25T13:51:51.785262+0000","last_became_active":"2024-04-25T13:51:50.704862+0000","last_became_peered":"2024-04-25T13:51:50.704862+0000","last_unstale":"2024-04-25T13:51:51.785262+0000","last_undegraded":"2024-04-25T13:51:51.785262+0000","last_fullsized":"2024-04-25T13:51:51.785262+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:17.953128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782443+0000","last_change":"2024-04-25T13:51:50.696549+0000","last_active":"2024-04-25T13:51:51.782443+0000","last_peered":"2024-04-25T13:51:51.782443+0000","last_clean":"2024-04-25T13:51:51.782443+0000","last_became_active":"2024-04-25T13:51:50.694930+0000","last_became_peered":"2024-04-25T13:51:50.694930+0000","last_unstale":"2024-04-25T13:51:51.782443+0000","last_undegraded":"2024-04-25T13:51:51.782443+0000","last_fullsized":"2024-04-25T13:51:51.782443+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:54.128665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718772+0000","last_change":"2024-04-25T13:51:50.693802+0000","last_active":"2024-04-25T13:51:51.718772+0000","last_peered":"2024-04-25T13:51:51.718772+0000","last_clean":"2024-04-25T13:51:51.718772+0000","last_became_active":"2024-04-25T13:51:50.691885+0000","last_became_peered":"2024-04-25T13:51:50.691885+0000","last_unstale":"2024-04-25T13:51:51.718772+0000","last_undegraded":"2024-04-25T13:51:51.718772+0000","last_fullsized":"2024-04-25T13:51:51.718772+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:32.191434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786227+0000","last_change":"2024-04-25T13:51:50.694647+0000","last_active":"2024-04-25T13:51:51.786227+0000","last_peered":"2024-04-25T13:51:51.786227+0000","last_clean":"2024-04-25T13:51:51.786227+0000","last_became_active":"2024-04-25T13:51:50.693917+0000","last_became_peered":"2024-04-25T13:51:50.693917+0000","last_unstale":"2024-04-25T13:51:51.786227+0000","last_undegraded":"2024-04-25T13:51:51.786227+0000","last_fullsized":"2024-04-25T13:51:51.786227+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.742703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718603+0000","last_change":"2024-04-25T13:51:50.706471+0000","last_active":"2024-04-25T13:51:51.718603+0000","last_peered":"2024-04-25T13:51:51.718603+0000","last_clean":"2024-04-25T13:51:51.718603+0000","last_became_active":"2024-04-25T13:51:50.705492+0000","last_became_peered":"2024-04-25T13:51:50.705492+0000","last_unstale":"2024-04-25T13:51:51.718603+0000","last_undegraded":"2024-04-25T13:51:51.718603+0000","last_fullsized":"2024-04-25T13:51:51.718603+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:27.832656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783555+0000","last_change":"2024-04-25T13:51:50.713534+0000","last_active":"2024-04-25T13:51:51.783555+0000","last_peered":"2024-04-25T13:51:51.783555+0000","last_clean":"2024-04-25T13:51:51.783555+0000","last_became_active":"2024-04-25T13:51:50.713177+0000","last_became_peered":"2024-04-25T13:51:50.713177+0000","last_unstale":"2024-04-25T13:51:51.783555+0000","last_undegraded":"2024-04-25T13:51:51.783555+0000","last_fullsized":"2024-04-25T13:51:51.783555+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:56.471714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718720+0000","last_change":"2024-04-25T13:51:50.705709+0000","last_active":"2024-04-25T13:51:51.718720+0000","last_peered":"2024-04-25T13:51:51.718720+0000","last_clean":"2024-04-25T13:51:51.718720+0000","last_became_active":"2024-04-25T13:51:50.701186+0000","last_became_peered":"2024-04-25T13:51:50.701186+0000","last_unstale":"2024-04-25T13:51:51.718720+0000","last_undegraded":"2024-04-25T13:51:51.718720+0000","last_fullsized":"2024-04-25T13:51:51.718720+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:07.009066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674561+0000","last_change":"2024-04-25T13:51:50.706020+0000","last_active":"2024-04-25T13:51:51.674561+0000","last_peered":"2024-04-25T13:51:51.674561+0000","last_clean":"2024-04-25T13:51:51.674561+0000","last_became_active":"2024-04-25T13:51:50.705117+0000","last_became_peered":"2024-04-25T13:51:50.705117+0000","last_unstale":"2024-04-25T13:51:51.674561+0000","last_undegraded":"2024-04-25T13:51:51.674561+0000","last_fullsized":"2024-04-25T13:51:51.674561+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.661713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674162+0000","last_change":"2024-04-25T13:51:50.714789+0000","last_active":"2024-04-25T13:51:51.674162+0000","last_peered":"2024-04-25T13:51:51.674162+0000","last_clean":"2024-04-25T13:51:51.674162+0000","last_became_active":"2024-04-25T13:51:50.713827+0000","last_became_peered":"2024-04-25T13:51:50.713827+0000","last_unstale":"2024-04-25T13:51:51.674162+0000","last_undegraded":"2024-04-25T13:51:51.674162+0000","last_fullsized":"2024-04-25T13:51:51.674162+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:23.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676128+0000","last_change":"2024-04-25T13:51:50.701827+0000","last_active":"2024-04-25T13:51:51.676128+0000","last_peered":"2024-04-25T13:51:51.676128+0000","last_clean":"2024-04-25T13:51:51.676128+0000","last_became_active":"2024-04-25T13:51:50.698840+0000","last_became_peered":"2024-04-25T13:51:50.698840+0000","last_unstale":"2024-04-25T13:51:51.676128+0000","last_undegraded":"2024-04-25T13:51:51.676128+0000","last_fullsized":"2024-04-25T13:51:51.676128+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:22.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784052+0000","last_change":"2024-04-25T13:51:50.694293+0000","last_active":"2024-04-25T13:51:51.784052+0000","last_peered":"2024-04-25T13:51:51.784052+0000","last_clean":"2024-04-25T13:51:51.784052+0000","last_became_active":"2024-04-25T13:51:50.693570+0000","last_became_peered":"2024-04-25T13:51:50.693570+0000","last_unstale":"2024-04-25T13:51:51.784052+0000","last_undegraded":"2024-04-25T13:51:51.784052+0000","last_fullsized":"2024-04-25T13:51:51.784052+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:42.023479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"186'4","reported_seq":18,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782306+0000","last_change":"2024-04-25T13:51:49.702768+0000","last_active":"2024-04-25T13:51:51.782306+0000","last_peered":"2024-04-25T13:51:51.782306+0000","last_clean":"2024-04-25T13:51:51.782306+0000","last_became_active":"2024-04-25T13:51:49.702087+0000","last_became_peered":"2024-04-25T13:51:49.702087+0000","last_unstale":"2024-04-25T13:51:51.782306+0000","last_undegraded":"2024-04-25T13:51:51.782306+0000","last_fullsized":"2024-04-25T13:51:51.782306+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:07.971890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674339+0000","last_change":"2024-04-25T13:51:50.713802+0000","last_active":"2024-04-25T13:51:51.674339+0000","last_peered":"2024-04-25T13:51:51.674339+0000","last_clean":"2024-04-25T13:51:51.674339+0000","last_became_active":"2024-04-25T13:51:50.709321+0000","last_became_peered":"2024-04-25T13:51:50.709321+0000","last_unstale":"2024-04-25T13:51:51.674339+0000","last_undegraded":"2024-04-25T13:51:51.674339+0000","last_fullsized":"2024-04-25T13:51:51.674339+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:03.244199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.673585+0000","last_change":"2024-04-25T13:51:49.705814+0000","last_active":"2024-04-25T13:51:51.673585+0000","last_peered":"2024-04-25T13:51:51.673585+0000","last_clean":"2024-04-25T13:51:51.673585+0000","last_became_active":"2024-04-25T13:51:49.703717+0000","last_became_peered":"2024-04-25T13:51:49.703717+0000","last_unstale":"2024-04-25T13:51:51.673585+0000","last_undegraded":"2024-04-25T13:51:51.673585+0000","last_fullsized":"2024-04-25T13:51:51.673585+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:50.544404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783872+0000","last_change":"2024-04-25T13:51:50.707481+0000","last_active":"2024-04-25T13:51:51.783872+0000","last_peered":"2024-04-25T13:51:51.783872+0000","last_clean":"2024-04-25T13:51:51.783872+0000","last_became_active":"2024-04-25T13:51:50.706771+0000","last_became_peered":"2024-04-25T13:51:50.706771+0000","last_unstale":"2024-04-25T13:51:51.783872+0000","last_undegraded":"2024-04-25T13:51:51.783872+0000","last_fullsized":"2024-04-25T13:51:51.783872+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:58.272633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785674+0000","last_change":"2024-04-25T13:51:49.692055+0000","last_active":"2024-04-25T13:51:51.785674+0000","last_peered":"2024-04-25T13:51:51.785674+0000","last_clean":"2024-04-25T13:51:51.785674+0000","last_became_active":"2024-04-25T13:51:49.691889+0000","last_became_peered":"2024-04-25T13:51:49.691889+0000","last_unstale":"2024-04-25T13:51:51.785674+0000","last_undegraded":"2024-04-25T13:51:51.785674+0000","last_fullsized":"2024-04-25T13:51:51.785674+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:26.600957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719049+0000","last_change":"2024-04-25T13:51:50.701062+0000","last_active":"2024-04-25T13:51:51.719049+0000","last_peered":"2024-04-25T13:51:51.719049+0000","last_clean":"2024-04-25T13:51:51.719049+0000","last_became_active":"2024-04-25T13:51:50.699625+0000","last_became_peered":"2024-04-25T13:51:50.699625+0000","last_unstale":"2024-04-25T13:51:51.719049+0000","last_undegraded":"2024-04-25T13:51:51.719049+0000","last_fullsized":"2024-04-25T13:51:51.719049+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:18.288634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783754+0000","last_change":"2024-04-25T13:51:49.687991+0000","last_active":"2024-04-25T13:51:51.783754+0000","last_peered":"2024-04-25T13:51:51.783754+0000","last_clean":"2024-04-25T13:51:51.783754+0000","last_became_active":"2024-04-25T13:51:49.687693+0000","last_became_peered":"2024-04-25T13:51:49.687693+0000","last_unstale":"2024-04-25T13:51:51.783754+0000","last_undegraded":"2024-04-25T13:51:51.783754+0000","last_fullsized":"2024-04-25T13:51:51.783754+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:28:08.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784756+0000","last_change":"2024-04-25T13:51:50.684961+0000","last_active":"2024-04-25T13:51:51.784756+0000","last_peered":"2024-04-25T13:51:51.784756+0000","last_clean":"2024-04-25T13:51:51.784756+0000","last_became_active":"2024-04-25T13:51:50.684168+0000","last_became_peered":"2024-04-25T13:51:50.684168+0000","last_unstale":"2024-04-25T13:51:51.784756+0000","last_undegraded":"2024-04-25T13:51:51.784756+0000","last_fullsized":"2024-04-25T13:51:51.784756+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:30.192436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719607+0000","last_change":"2024-04-25T13:51:49.694022+0000","last_active":"2024-04-25T13:51:51.719607+0000","last_peered":"2024-04-25T13:51:51.719607+0000","last_clean":"2024-04-25T13:51:51.719607+0000","last_became_active":"2024-04-25T13:51:49.693294+0000","last_became_peered":"2024-04-25T13:51:49.693294+0000","last_unstale":"2024-04-25T13:51:51.719607+0000","last_undegraded":"2024-04-25T13:51:51.719607+0000","last_fullsized":"2024-04-25T13:51:51.719607+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:59.266242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784857+0000","last_change":"2024-04-25T13:51:50.709506+0000","last_active":"2024-04-25T13:51:51.784857+0000","last_peered":"2024-04-25T13:51:51.784857+0000","last_clean":"2024-04-25T13:51:51.784857+0000","last_became_active":"2024-04-25T13:51:50.708831+0000","last_became_peered":"2024-04-25T13:51:50.708831+0000","last_unstale":"2024-04-25T13:51:51.784857+0000","last_undegraded":"2024-04-25T13:51:51.784857+0000","last_fullsized":"2024-04-25T13:51:51.784857+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:52.663346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783861+0000","last_change":"2024-04-25T13:51:49.679494+0000","last_active":"2024-04-25T13:51:51.783861+0000","last_peered":"2024-04-25T13:51:51.783861+0000","last_clean":"2024-04-25T13:51:51.783861+0000","last_became_active":"2024-04-25T13:51:49.679191+0000","last_became_peered":"2024-04-25T13:51:49.679191+0000","last_unstale":"2024-04-25T13:51:51.783861+0000","last_undegraded":"2024-04-25T13:51:51.783861+0000","last_fullsized":"2024-04-25T13:51:51.783861+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.523506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784699+0000","last_change":"2024-04-25T13:51:50.688984+0000","last_active":"2024-04-25T13:51:51.784699+0000","last_peered":"2024-04-25T13:51:51.784699+0000","last_clean":"2024-04-25T13:51:51.784699+0000","last_became_active":"2024-04-25T13:51:50.688028+0000","last_became_peered":"2024-04-25T13:51:50.688028+0000","last_unstale":"2024-04-25T13:51:51.784699+0000","last_undegraded":"2024-04-25T13:51:51.784699+0000","last_fullsized":"2024-04-25T13:51:51.784699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:57.622221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782758+0000","last_change":"2024-04-25T13:51:49.689427+0000","last_active":"2024-04-25T13:51:51.782758+0000","last_peered":"2024-04-25T13:51:51.782758+0000","last_clean":"2024-04-25T13:51:51.782758+0000","last_became_active":"2024-04-25T13:51:49.687499+0000","last_became_peered":"2024-04-25T13:51:49.687499+0000","last_unstale":"2024-04-25T13:51:51.782758+0000","last_undegraded":"2024-04-25T13:51:51.782758+0000","last_fullsized":"2024-04-25T13:51:51.782758+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:04.777830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783361+0000","last_change":"2024-04-25T13:51:50.692946+0000","last_active":"2024-04-25T13:51:51.783361+0000","last_peered":"2024-04-25T13:51:51.783361+0000","last_clean":"2024-04-25T13:51:51.783361+0000","last_became_active":"2024-04-25T13:51:50.692186+0000","last_became_peered":"2024-04-25T13:51:50.692186+0000","last_unstale":"2024-04-25T13:51:51.783361+0000","last_undegraded":"2024-04-25T13:51:51.783361+0000","last_fullsized":"2024-04-25T13:51:51.783361+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:04.185988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785586+0000","last_change":"2024-04-25T13:51:49.676972+0000","last_active":"2024-04-25T13:51:51.785586+0000","last_peered":"2024-04-25T13:51:51.785586+0000","last_clean":"2024-04-25T13:51:51.785586+0000","last_became_active":"2024-04-25T13:51:49.676721+0000","last_became_peered":"2024-04-25T13:51:49.676721+0000","last_unstale":"2024-04-25T13:51:51.785586+0000","last_undegraded":"2024-04-25T13:51:51.785586+0000","last_fullsized":"2024-04-25T13:51:51.785586+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.544234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676120+0000","last_change":"2024-04-25T13:51:50.699774+0000","last_active":"2024-04-25T13:51:51.676120+0000","last_peered":"2024-04-25T13:51:51.676120+0000","last_clean":"2024-04-25T13:51:51.676120+0000","last_became_active":"2024-04-25T13:51:50.698918+0000","last_became_peered":"2024-04-25T13:51:50.698918+0000","last_unstale":"2024-04-25T13:51:51.676120+0000","last_undegraded":"2024-04-25T13:51:51.676120+0000","last_fullsized":"2024-04-25T13:51:51.676120+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:57.556654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784051+0000","last_change":"2024-04-25T13:51:49.695156+0000","last_active":"2024-04-25T13:51:51.784051+0000","last_peered":"2024-04-25T13:51:51.784051+0000","last_clean":"2024-04-25T13:51:51.784051+0000","last_became_active":"2024-04-25T13:51:49.694878+0000","last_became_peered":"2024-04-25T13:51:49.694878+0000","last_unstale":"2024-04-25T13:51:51.784051+0000","last_undegraded":"2024-04-25T13:51:51.784051+0000","last_fullsized":"2024-04-25T13:51:51.784051+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:00.199207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676056+0000","last_change":"2024-04-25T13:51:50.705533+0000","last_active":"2024-04-25T13:51:51.676056+0000","last_peered":"2024-04-25T13:51:51.676056+0000","last_clean":"2024-04-25T13:51:51.676056+0000","last_became_active":"2024-04-25T13:51:50.703454+0000","last_became_peered":"2024-04-25T13:51:50.703454+0000","last_unstale":"2024-04-25T13:51:51.676056+0000","last_undegraded":"2024-04-25T13:51:51.676056+0000","last_fullsized":"2024-04-25T13:51:51.676056+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:31.247845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"186'3","reported_seq":17,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785595+0000","last_change":"2024-04-25T13:51:49.688388+0000","last_active":"2024-04-25T13:51:51.785595+0000","last_peered":"2024-04-25T13:51:51.785595+0000","last_clean":"2024-04-25T13:51:51.785595+0000","last_became_active":"2024-04-25T13:51:49.688155+0000","last_became_peered":"2024-04-25T13:51:49.688155+0000","last_unstale":"2024-04-25T13:51:51.785595+0000","last_undegraded":"2024-04-25T13:51:51.785595+0000","last_fullsized":"2024-04-25T13:51:51.785595+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:21.369037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674791+0000","last_change":"2024-04-25T13:51:50.711409+0000","last_active":"2024-04-25T13:51:51.674791+0000","last_peered":"2024-04-25T13:51:51.674791+0000","last_clean":"2024-04-25T13:51:51.674791+0000","last_became_active":"2024-04-25T13:51:50.705564+0000","last_became_peered":"2024-04-25T13:51:50.705564+0000","last_unstale":"2024-04-25T13:51:51.674791+0000","last_undegraded":"2024-04-25T13:51:51.674791+0000","last_fullsized":"2024-04-25T13:51:51.674791+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:52.917202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"186'6","reported_seq":20,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.673940+0000","last_change":"2024-04-25T13:51:49.704400+0000","last_active":"2024-04-25T13:51:51.673940+0000","last_peered":"2024-04-25T13:51:51.673940+0000","last_clean":"2024-04-25T13:51:51.673940+0000","last_became_active":"2024-04-25T13:51:49.702643+0000","last_became_peered":"2024-04-25T13:51:49.702643+0000","last_unstale":"2024-04-25T13:51:51.673940+0000","last_undegraded":"2024-04-25T13:51:51.673940+0000","last_fullsized":"2024-04-25T13:51:51.673940+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:57.646098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676464+0000","last_change":"2024-04-25T13:51:50.690953+0000","last_active":"2024-04-25T13:51:51.676464+0000","last_peered":"2024-04-25T13:51:51.676464+0000","last_clean":"2024-04-25T13:51:51.676464+0000","last_became_active":"2024-04-25T13:51:50.690213+0000","last_became_peered":"2024-04-25T13:51:50.690213+0000","last_unstale":"2024-04-25T13:51:51.676464+0000","last_undegraded":"2024-04-25T13:51:51.676464+0000","last_fullsized":"2024-04-25T13:51:51.676464+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:14.979862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785980+0000","last_change":"2024-04-25T13:51:49.680836+0000","last_active":"2024-04-25T13:51:51.785980+0000","last_peered":"2024-04-25T13:51:51.785980+0000","last_clean":"2024-04-25T13:51:51.785980+0000","last_became_active":"2024-04-25T13:51:49.680303+0000","last_became_peered":"2024-04-25T13:51:49.680303+0000","last_unstale":"2024-04-25T13:51:51.785980+0000","last_undegraded":"2024-04-25T13:51:51.785980+0000","last_fullsized":"2024-04-25T13:51:51.785980+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:13.903872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785023+0000","last_change":"2024-04-25T13:51:50.698034+0000","last_active":"2024-04-25T13:51:51.785023+0000","last_peered":"2024-04-25T13:51:51.785023+0000","last_clean":"2024-04-25T13:51:51.785023+0000","last_became_active":"2024-04-25T13:51:50.697297+0000","last_became_peered":"2024-04-25T13:51:50.697297+0000","last_unstale":"2024-04-25T13:51:51.785023+0000","last_undegraded":"2024-04-25T13:51:51.785023+0000","last_fullsized":"2024-04-25T13:51:51.785023+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:30.747047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676861+0000","last_change":"2024-04-25T13:51:49.675554+0000","last_active":"2024-04-25T13:51:51.676861+0000","last_peered":"2024-04-25T13:51:51.676861+0000","last_clean":"2024-04-25T13:51:51.676861+0000","last_became_active":"2024-04-25T13:51:49.675179+0000","last_became_peered":"2024-04-25T13:51:49.675179+0000","last_unstale":"2024-04-25T13:51:51.676861+0000","last_undegraded":"2024-04-25T13:51:51.676861+0000","last_fullsized":"2024-04-25T13:51:51.676861+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.539861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782921+0000","last_change":"2024-04-25T13:51:50.708874+0000","last_active":"2024-04-25T13:51:51.782921+0000","last_peered":"2024-04-25T13:51:51.782921+0000","last_clean":"2024-04-25T13:51:51.782921+0000","last_became_active":"2024-04-25T13:51:50.707578+0000","last_became_peered":"2024-04-25T13:51:50.707578+0000","last_unstale":"2024-04-25T13:51:51.782921+0000","last_undegraded":"2024-04-25T13:51:51.782921+0000","last_fullsized":"2024-04-25T13:51:51.782921+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:08.365733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785217+0000","last_change":"2024-04-25T13:51:49.695717+0000","last_active":"2024-04-25T13:51:51.785217+0000","last_peered":"2024-04-25T13:51:51.785217+0000","last_clean":"2024-04-25T13:51:51.785217+0000","last_became_active":"2024-04-25T13:51:49.693639+0000","last_became_peered":"2024-04-25T13:51:49.693639+0000","last_unstale":"2024-04-25T13:51:51.785217+0000","last_undegraded":"2024-04-25T13:51:51.785217+0000","last_fullsized":"2024-04-25T13:51:51.785217+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:42.217857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719105+0000","last_change":"2024-04-25T13:51:50.688074+0000","last_active":"2024-04-25T13:51:51.719105+0000","last_peered":"2024-04-25T13:51:51.719105+0000","last_clean":"2024-04-25T13:51:51.719105+0000","last_became_active":"2024-04-25T13:51:50.684820+0000","last_became_peered":"2024-04-25T13:51:50.684820+0000","last_unstale":"2024-04-25T13:51:51.719105+0000","last_undegraded":"2024-04-25T13:51:51.719105+0000","last_fullsized":"2024-04-25T13:51:51.719105+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:10.885680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"186'8","reported_seq":22,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.675125+0000","last_change":"2024-04-25T13:51:49.693899+0000","last_active":"2024-04-25T13:51:51.675125+0000","last_peered":"2024-04-25T13:51:51.675125+0000","last_clean":"2024-04-25T13:51:51.675125+0000","last_became_active":"2024-04-25T13:51:49.693082+0000","last_became_peered":"2024-04-25T13:51:49.693082+0000","last_unstale":"2024-04-25T13:51:51.675125+0000","last_undegraded":"2024-04-25T13:51:51.675125+0000","last_fullsized":"2024-04-25T13:51:51.675125+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:32.260557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785366+0000","last_change":"2024-04-25T13:51:50.693691+0000","last_active":"2024-04-25T13:51:51.785366+0000","last_peered":"2024-04-25T13:51:51.785366+0000","last_clean":"2024-04-25T13:51:51.785366+0000","last_became_active":"2024-04-25T13:51:50.692526+0000","last_became_peered":"2024-04-25T13:51:50.692526+0000","last_unstale":"2024-04-25T13:51:51.785366+0000","last_undegraded":"2024-04-25T13:51:51.785366+0000","last_fullsized":"2024-04-25T13:51:51.785366+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:39.227639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719202+0000","last_change":"2024-04-25T13:51:49.699624+0000","last_active":"2024-04-25T13:51:51.719202+0000","last_peered":"2024-04-25T13:51:51.719202+0000","last_clean":"2024-04-25T13:51:51.719202+0000","last_became_active":"2024-04-25T13:51:49.698758+0000","last_became_peered":"2024-04-25T13:51:49.698758+0000","last_unstale":"2024-04-25T13:51:51.719202+0000","last_undegraded":"2024-04-25T13:51:51.719202+0000","last_fullsized":"2024-04-25T13:51:51.719202+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:29.538938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676437+0000","last_change":"2024-04-25T13:51:50.716893+0000","last_active":"2024-04-25T13:51:51.676437+0000","last_peered":"2024-04-25T13:51:51.676437+0000","last_clean":"2024-04-25T13:51:51.676437+0000","last_became_active":"2024-04-25T13:51:50.716517+0000","last_became_peered":"2024-04-25T13:51:50.716517+0000","last_unstale":"2024-04-25T13:51:51.676437+0000","last_undegraded":"2024-04-25T13:51:51.676437+0000","last_fullsized":"2024-04-25T13:51:51.676437+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:59.473791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719454+0000","last_change":"2024-04-25T13:51:49.701634+0000","last_active":"2024-04-25T13:51:51.719454+0000","last_peered":"2024-04-25T13:51:51.719454+0000","last_clean":"2024-04-25T13:51:51.719454+0000","last_became_active":"2024-04-25T13:51:49.700041+0000","last_became_peered":"2024-04-25T13:51:49.700041+0000","last_unstale":"2024-04-25T13:51:51.719454+0000","last_undegraded":"2024-04-25T13:51:51.719454+0000","last_fullsized":"2024-04-25T13:51:51.719454+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:27.236152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785025+0000","last_change":"2024-04-25T13:51:50.706400+0000","last_active":"2024-04-25T13:51:51.785025+0000","last_peered":"2024-04-25T13:51:51.785025+0000","last_clean":"2024-04-25T13:51:51.785025+0000","last_became_active":"2024-04-25T13:51:50.705628+0000","last_became_peered":"2024-04-25T13:51:50.705628+0000","last_unstale":"2024-04-25T13:51:51.785025+0000","last_undegraded":"2024-04-25T13:51:51.785025+0000","last_fullsized":"2024-04-25T13:51:51.785025+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:15.340959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785769+0000","last_change":"2024-04-25T13:51:49.704674+0000","last_active":"2024-04-25T13:51:51.785769+0000","last_peered":"2024-04-25T13:51:51.785769+0000","last_clean":"2024-04-25T13:51:51.785769+0000","last_became_active":"2024-04-25T13:51:49.702709+0000","last_became_peered":"2024-04-25T13:51:49.702709+0000","last_unstale":"2024-04-25T13:51:51.785769+0000","last_undegraded":"2024-04-25T13:51:51.785769+0000","last_fullsized":"2024-04-25T13:51:51.785769+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:30.768066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786668+0000","last_change":"2024-04-25T13:51:50.712343+0000","last_active":"2024-04-25T13:51:51.786668+0000","last_peered":"2024-04-25T13:51:51.786668+0000","last_clean":"2024-04-25T13:51:51.786668+0000","last_became_active":"2024-04-25T13:51:50.711810+0000","last_became_peered":"2024-04-25T13:51:50.711810+0000","last_unstale":"2024-04-25T13:51:51.786668+0000","last_undegraded":"2024-04-25T13:51:51.786668+0000","last_fullsized":"2024-04-25T13:51:51.786668+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:50.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782092+0000","last_change":"2024-04-25T13:51:49.686462+0000","last_active":"2024-04-25T13:51:51.782092+0000","last_peered":"2024-04-25T13:51:51.782092+0000","last_clean":"2024-04-25T13:51:51.782092+0000","last_became_active":"2024-04-25T13:51:49.685685+0000","last_became_peered":"2024-04-25T13:51:49.685685+0000","last_unstale":"2024-04-25T13:51:51.782092+0000","last_undegraded":"2024-04-25T13:51:51.782092+0000","last_fullsized":"2024-04-25T13:51:51.782092+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.929105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718993+0000","last_change":"2024-04-25T13:51:50.702149+0000","last_active":"2024-04-25T13:51:51.718993+0000","last_peered":"2024-04-25T13:51:51.718993+0000","last_clean":"2024-04-25T13:51:51.718993+0000","last_became_active":"2024-04-25T13:51:50.695831+0000","last_became_peered":"2024-04-25T13:51:50.695831+0000","last_unstale":"2024-04-25T13:51:51.718993+0000","last_undegraded":"2024-04-25T13:51:51.718993+0000","last_fullsized":"2024-04-25T13:51:51.718993+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:44.413262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784175+0000","last_change":"2024-04-25T13:51:49.693961+0000","last_active":"2024-04-25T13:51:51.784175+0000","last_peered":"2024-04-25T13:51:51.784175+0000","last_clean":"2024-04-25T13:51:51.784175+0000","last_became_active":"2024-04-25T13:51:49.693592+0000","last_became_peered":"2024-04-25T13:51:49.693592+0000","last_unstale":"2024-04-25T13:51:51.784175+0000","last_undegraded":"2024-04-25T13:51:51.784175+0000","last_fullsized":"2024-04-25T13:51:51.784175+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:20.767319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785174+0000","last_change":"2024-04-25T13:51:50.691990+0000","last_active":"2024-04-25T13:51:51.785174+0000","last_peered":"2024-04-25T13:51:51.785174+0000","last_clean":"2024-04-25T13:51:51.785174+0000","last_became_active":"2024-04-25T13:51:50.691085+0000","last_became_peered":"2024-04-25T13:51:50.691085+0000","last_unstale":"2024-04-25T13:51:51.785174+0000","last_undegraded":"2024-04-25T13:51:51.785174+0000","last_fullsized":"2024-04-25T13:51:51.785174+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:44.429056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782451+0000","last_change":"2024-04-25T13:51:49.689929+0000","last_active":"2024-04-25T13:51:51.782451+0000","last_peered":"2024-04-25T13:51:51.782451+0000","last_clean":"2024-04-25T13:51:51.782451+0000","last_became_active":"2024-04-25T13:51:49.688012+0000","last_became_peered":"2024-04-25T13:51:49.688012+0000","last_unstale":"2024-04-25T13:51:51.782451+0000","last_undegraded":"2024-04-25T13:51:51.782451+0000","last_fullsized":"2024-04-25T13:51:51.782451+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:47.582175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784924+0000","last_change":"2024-04-25T13:51:50.698400+0000","last_active":"2024-04-25T13:51:51.784924+0000","last_peered":"2024-04-25T13:51:51.784924+0000","last_clean":"2024-04-25T13:51:51.784924+0000","last_became_active":"2024-04-25T13:51:50.690952+0000","last_became_peered":"2024-04-25T13:51:50.690952+0000","last_unstale":"2024-04-25T13:51:51.784924+0000","last_undegraded":"2024-04-25T13:51:51.784924+0000","last_fullsized":"2024-04-25T13:51:51.784924+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:56.607704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786287+0000","last_change":"2024-04-25T13:51:49.701674+0000","last_active":"2024-04-25T13:51:51.786287+0000","last_peered":"2024-04-25T13:51:51.786287+0000","last_clean":"2024-04-25T13:51:51.786287+0000","last_became_active":"2024-04-25T13:51:49.701208+0000","last_became_peered":"2024-04-25T13:51:49.701208+0000","last_unstale":"2024-04-25T13:51:51.786287+0000","last_undegraded":"2024-04-25T13:51:51.786287+0000","last_fullsized":"2024-04-25T13:51:51.786287+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:52.785448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784284+0000","last_change":"2024-04-25T13:51:50.709025+0000","last_active":"2024-04-25T13:51:51.784284+0000","last_peered":"2024-04-25T13:51:51.784284+0000","last_clean":"2024-04-25T13:51:51.784284+0000","last_became_active":"2024-04-25T13:51:50.708219+0000","last_became_peered":"2024-04-25T13:51:50.708219+0000","last_unstale":"2024-04-25T13:51:51.784284+0000","last_undegraded":"2024-04-25T13:51:51.784284+0000","last_fullsized":"2024-04-25T13:51:51.784284+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:13.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786087+0000","last_change":"2024-04-25T13:51:49.705597+0000","last_active":"2024-04-25T13:51:51.786087+0000","last_peered":"2024-04-25T13:51:51.786087+0000","last_clean":"2024-04-25T13:51:51.786087+0000","last_became_active":"2024-04-25T13:51:49.703751+0000","last_became_peered":"2024-04-25T13:51:49.703751+0000","last_unstale":"2024-04-25T13:51:51.786087+0000","last_undegraded":"2024-04-25T13:51:51.786087+0000","last_fullsized":"2024-04-25T13:51:51.786087+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:56.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.677671+0000","last_change":"2024-04-25T13:51:50.703736+0000","last_active":"2024-04-25T13:51:51.677671+0000","last_peered":"2024-04-25T13:51:51.677671+0000","last_clean":"2024-04-25T13:51:51.677671+0000","last_became_active":"2024-04-25T13:51:50.701333+0000","last_became_peered":"2024-04-25T13:51:50.701333+0000","last_unstale":"2024-04-25T13:51:51.677671+0000","last_undegraded":"2024-04-25T13:51:51.677671+0000","last_fullsized":"2024-04-25T13:51:51.677671+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:22.799998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.677535+0000","last_change":"2024-04-25T13:51:49.695107+0000","last_active":"2024-04-25T13:51:51.677535+0000","last_peered":"2024-04-25T13:51:51.677535+0000","last_clean":"2024-04-25T13:51:51.677535+0000","last_became_active":"2024-04-25T13:51:49.694745+0000","last_became_peered":"2024-04-25T13:51:49.694745+0000","last_unstale":"2024-04-25T13:51:51.677535+0000","last_undegraded":"2024-04-25T13:51:51.677535+0000","last_fullsized":"2024-04-25T13:51:51.677535+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:27.752486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786554+0000","last_change":"2024-04-25T13:51:50.713613+0000","last_active":"2024-04-25T13:51:51.786554+0000","last_peered":"2024-04-25T13:51:51.786554+0000","last_clean":"2024-04-25T13:51:51.786554+0000","last_became_active":"2024-04-25T13:51:50.711814+0000","last_became_peered":"2024-04-25T13:51:50.711814+0000","last_unstale":"2024-04-25T13:51:51.786554+0000","last_undegraded":"2024-04-25T13:51:51.786554+0000","last_fullsized":"2024-04-25T13:51:51.786554+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:03.001620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"187'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.721883+0000","last_change":"2024-04-25T13:51:49.704754+0000","last_active":"2024-04-25T13:51:51.721883+0000","last_peered":"2024-04-25T13:51:51.721883+0000","last_clean":"2024-04-25T13:51:51.721883+0000","last_became_active":"2024-04-25T13:51:49.699641+0000","last_became_peered":"2024-04-25T13:51:49.699641+0000","last_unstale":"2024-04-25T13:51:51.721883+0000","last_undegraded":"2024-04-25T13:51:51.721883+0000","last_fullsized":"2024-04-25T13:51:51.721883+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:14.302772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784444+0000","last_change":"2024-04-25T13:51:50.704674+0000","last_active":"2024-04-25T13:51:51.784444+0000","last_peered":"2024-04-25T13:51:51.784444+0000","last_clean":"2024-04-25T13:51:51.784444+0000","last_became_active":"2024-04-25T13:51:50.703719+0000","last_became_peered":"2024-04-25T13:51:50.703719+0000","last_unstale":"2024-04-25T13:51:51.784444+0000","last_undegraded":"2024-04-25T13:51:51.784444+0000","last_fullsized":"2024-04-25T13:51:51.784444+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:35.891863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783581+0000","last_change":"2024-04-25T13:51:49.683005+0000","last_active":"2024-04-25T13:51:51.783581+0000","last_peered":"2024-04-25T13:51:51.783581+0000","last_clean":"2024-04-25T13:51:51.783581+0000","last_became_active":"2024-04-25T13:51:49.681865+0000","last_became_peered":"2024-04-25T13:51:49.681865+0000","last_unstale":"2024-04-25T13:51:51.783581+0000","last_undegraded":"2024-04-25T13:51:51.783581+0000","last_fullsized":"2024-04-25T13:51:51.783581+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:14.401844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676876+0000","last_change":"2024-04-25T13:51:50.687988+0000","last_active":"2024-04-25T13:51:51.676876+0000","last_peered":"2024-04-25T13:51:51.676876+0000","last_clean":"2024-04-25T13:51:51.676876+0000","last_became_active":"2024-04-25T13:51:50.676039+0000","last_became_peered":"2024-04-25T13:51:50.676039+0000","last_unstale":"2024-04-25T13:51:51.676876+0000","last_undegraded":"2024-04-25T13:51:51.676876+0000","last_fullsized":"2024-04-25T13:51:51.676876+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:39.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676857+0000","last_change":"2024-04-25T13:51:49.701480+0000","last_active":"2024-04-25T13:51:51.676857+0000","last_peered":"2024-04-25T13:51:51.676857+0000","last_clean":"2024-04-25T13:51:51.676857+0000","last_became_active":"2024-04-25T13:51:49.701109+0000","last_became_peered":"2024-04-25T13:51:49.701109+0000","last_unstale":"2024-04-25T13:51:51.676857+0000","last_undegraded":"2024-04-25T13:51:51.676857+0000","last_fullsized":"2024-04-25T13:51:51.676857+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:07.990989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.677139+0000","last_change":"2024-04-25T13:51:50.700574+0000","last_active":"2024-04-25T13:51:51.677139+0000","last_peered":"2024-04-25T13:51:51.677139+0000","last_clean":"2024-04-25T13:51:51.677139+0000","last_became_active":"2024-04-25T13:51:50.697338+0000","last_became_peered":"2024-04-25T13:51:50.697338+0000","last_unstale":"2024-04-25T13:51:51.677139+0000","last_undegraded":"2024-04-25T13:51:51.677139+0000","last_fullsized":"2024-04-25T13:51:51.677139+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.115243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.675498+0000","last_change":"2024-04-25T13:51:49.686869+0000","last_active":"2024-04-25T13:51:51.675498+0000","last_peered":"2024-04-25T13:51:51.675498+0000","last_clean":"2024-04-25T13:51:51.675498+0000","last_became_active":"2024-04-25T13:51:49.684310+0000","last_became_peered":"2024-04-25T13:51:49.684310+0000","last_unstale":"2024-04-25T13:51:51.675498+0000","last_undegraded":"2024-04-25T13:51:51.675498+0000","last_fullsized":"2024-04-25T13:51:51.675498+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:06.439754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674659+0000","last_change":"2024-04-25T13:51:50.710605+0000","last_active":"2024-04-25T13:51:51.674659+0000","last_peered":"2024-04-25T13:51:51.674659+0000","last_clean":"2024-04-25T13:51:51.674659+0000","last_became_active":"2024-04-25T13:51:50.709309+0000","last_became_peered":"2024-04-25T13:51:50.709309+0000","last_unstale":"2024-04-25T13:51:51.674659+0000","last_undegraded":"2024-04-25T13:51:51.674659+0000","last_fullsized":"2024-04-25T13:51:51.674659+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:15.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785326+0000","last_change":"2024-04-25T13:51:49.696135+0000","last_active":"2024-04-25T13:51:51.785326+0000","last_peered":"2024-04-25T13:51:51.785326+0000","last_clean":"2024-04-25T13:51:51.785326+0000","last_became_active":"2024-04-25T13:51:49.694223+0000","last_became_peered":"2024-04-25T13:51:49.694223+0000","last_unstale":"2024-04-25T13:51:51.785326+0000","last_undegraded":"2024-04-25T13:51:51.785326+0000","last_fullsized":"2024-04-25T13:51:51.785326+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:21.904303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784479+0000","last_change":"2024-04-25T13:51:50.704009+0000","last_active":"2024-04-25T13:51:51.784479+0000","last_peered":"2024-04-25T13:51:51.784479+0000","last_clean":"2024-04-25T13:51:51.784479+0000","last_became_active":"2024-04-25T13:51:50.702487+0000","last_became_peered":"2024-04-25T13:51:50.702487+0000","last_unstale":"2024-04-25T13:51:51.784479+0000","last_undegraded":"2024-04-25T13:51:51.784479+0000","last_fullsized":"2024-04-25T13:51:51.784479+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:41.466337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783692+0000","last_change":"2024-04-25T13:51:49.681283+0000","last_active":"2024-04-25T13:51:51.783692+0000","last_peered":"2024-04-25T13:51:51.783692+0000","last_clean":"2024-04-25T13:51:51.783692+0000","last_became_active":"2024-04-25T13:51:49.678905+0000","last_became_peered":"2024-04-25T13:51:49.678905+0000","last_unstale":"2024-04-25T13:51:51.783692+0000","last_undegraded":"2024-04-25T13:51:51.783692+0000","last_fullsized":"2024-04-25T13:51:51.783692+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:18.529412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719313+0000","last_change":"2024-04-25T13:51:50.697270+0000","last_active":"2024-04-25T13:51:51.719313+0000","last_peered":"2024-04-25T13:51:51.719313+0000","last_clean":"2024-04-25T13:51:51.719313+0000","last_became_active":"2024-04-25T13:51:50.696465+0000","last_became_peered":"2024-04-25T13:51:50.696465+0000","last_unstale":"2024-04-25T13:51:51.719313+0000","last_undegraded":"2024-04-25T13:51:51.719313+0000","last_fullsized":"2024-04-25T13:51:51.719313+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:37.951762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674258+0000","last_change":"2024-04-25T13:51:49.691466+0000","last_active":"2024-04-25T13:51:51.674258+0000","last_peered":"2024-04-25T13:51:51.674258+0000","last_clean":"2024-04-25T13:51:51.674258+0000","last_became_active":"2024-04-25T13:51:49.690640+0000","last_became_peered":"2024-04-25T13:51:49.690640+0000","last_unstale":"2024-04-25T13:51:51.674258+0000","last_undegraded":"2024-04-25T13:51:51.674258+0000","last_fullsized":"2024-04-25T13:51:51.674258+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.058511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676555+0000","last_change":"2024-04-25T13:51:50.703020+0000","last_active":"2024-04-25T13:51:51.676555+0000","last_peered":"2024-04-25T13:51:51.676555+0000","last_clean":"2024-04-25T13:51:51.676555+0000","last_became_active":"2024-04-25T13:51:50.699795+0000","last_became_peered":"2024-04-25T13:51:50.699795+0000","last_unstale":"2024-04-25T13:51:51.676555+0000","last_undegraded":"2024-04-25T13:51:51.676555+0000","last_fullsized":"2024-04-25T13:51:51.676555+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:34.494669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.675171+0000","last_change":"2024-04-25T13:51:49.702153+0000","last_active":"2024-04-25T13:51:51.675171+0000","last_peered":"2024-04-25T13:51:51.675171+0000","last_clean":"2024-04-25T13:51:51.675171+0000","last_became_active":"2024-04-25T13:51:49.701399+0000","last_became_peered":"2024-04-25T13:51:49.701399+0000","last_unstale":"2024-04-25T13:51:51.675171+0000","last_undegraded":"2024-04-25T13:51:51.675171+0000","last_fullsized":"2024-04-25T13:51:51.675171+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:39.850383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":435,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782610+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:51:51.782610+0000","last_peered":"2024-04-25T13:51:51.782610+0000","last_clean":"2024-04-25T13:51:51.782610+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:51:51.782610+0000","last_undegraded":"2024-04-25T13:51:51.782610+0000","last_fullsized":"2024-04-25T13:51:51.782610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193703,"statfs":{"total":200000000,"available":198352018,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944718,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1031,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1032,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194280,"statfs":{"total":200000000,"available":198943058,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1607,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193705,"statfs":{"total":200000000,"available":198354372,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:51:54.820 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:51:54.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e57ae8640 1 Processor -- start 2024-04-25T13:51:54.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e57ae8640 1 -- start start 2024-04-25T13:51:54.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e57ae8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e500d06e0 0x7f1e5006e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e57ae8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1e5006e9e0 0x7f1e5006edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:54.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e57ae8640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1e500d0af0 con 0x7f1e50161f90 2024-04-25T13:51:54.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e57ae8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1e500d0d70 con 0x7f1e5006e9e0 2024-04-25T13:51:54.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e57ae8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e500d1050 con 0x7f1e500d06e0 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e56085640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1e5006e9e0 0x7f1e5006edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e56085640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1e5006e9e0 0x7f1e5006edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58546/0 (socket says 172.21.15.80:58546) 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e56085640 1 -- 172.21.15.80:0/309736626 learned_addr learned my addr 172.21.15.80:0/309736626 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e55884640 1 --1- 172.21.15.80:0/309736626 >> v1:172.21.15.80:6789/0 conn(0x7f1e50161f90 0x7f1e500cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41238/0 (socket says 172.21.15.80:41238) 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e55083640 1 --2- 172.21.15.80:0/309736626 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e500d06e0 0x7f1e5006e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e56085640 1 -- 172.21.15.80:0/309736626 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e500d06e0 msgr2=0x7f1e5006e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e56085640 1 --2- 172.21.15.80:0/309736626 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e500d06e0 0x7f1e5006e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e56085640 1 -- 172.21.15.80:0/309736626 >> v1:172.21.15.80:6789/0 conn(0x7f1e50161f90 legacy=0x7f1e500cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:54.999+0000 7f1e56085640 1 -- 172.21.15.80:0/309736626 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e5006dca0 con 0x7f1e5006e9e0 2024-04-25T13:51:54.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e56085640 1 --2- 172.21.15.80:0/309736626 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e5006e9e0 0x7f1e5006edf0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009e50 tx=0x7f1e4801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5129ca01587482c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:54.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e54882640 1 -- 172.21.15.80:0/309736626 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480a5740 con 0x7f1e5006e9e0 2024-04-25T13:51:54.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e54882640 1 -- 172.21.15.80:0/309736626 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e480a58e0 con 0x7f1e5006e9e0 2024-04-25T13:51:54.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e54882640 1 -- 172.21.15.80:0/309736626 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e480a5cb0 con 0x7f1e5006e9e0 2024-04-25T13:51:54.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 -- 172.21.15.80:0/309736626 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e5006e9e0 msgr2=0x7f1e5006edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:54.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/309736626 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e5006e9e0 0x7f1e5006edf0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009e50 tx=0x7f1e4801c970 comp rx=0 tx=0).stop 2024-04-25T13:51:54.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 -- 172.21.15.80:0/309736626 shutdown_connections 2024-04-25T13:51:54.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/309736626 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e5006e9e0 0x7f1e5006edf0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/309736626 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e500d06e0 0x7f1e5006e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:54.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 -- 172.21.15.80:0/309736626 >> 172.21.15.80:0/309736626 conn(0x7f1e5006d080 msgr2=0x7f1e500b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:54.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 -- 172.21.15.80:0/309736626 shutdown_connections 2024-04-25T13:51:54.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 -- 172.21.15.80:0/309736626 wait complete. 2024-04-25T13:51:54.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 Processor -- start 2024-04-25T13:51:54.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.003+0000 7f1e57ae8640 1 -- start start 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e5006e9e0 0x7f1e50147430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e500d06e0 0x7f1e50145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e50161f90 0x7f1e50131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1e501447a0 con 0x7f1e500d06e0 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1e50144620 con 0x7f1e50161f90 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1e5016b510 con 0x7f1e5006e9e0 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e500d06e0 0x7f1e50145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e500d06e0 0x7f1e50145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58554/0 (socket says 172.21.15.80:58554) 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 -- 172.21.15.80:0/1086321773 learned_addr learned my addr 172.21.15.80:0/1086321773 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 -- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e5006e9e0 msgr2=0x7f1e50147430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e5006e9e0 0x7f1e50147430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 -- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e50161f90 msgr2=0x7f1e50131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e50161f90 0x7f1e50131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 -- 172.21.15.80:0/1086321773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e50144de0 con 0x7f1e500d06e0 2024-04-25T13:51:55.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e55083640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e500d06e0 0x7f1e50145e80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00c820 tx=0x7f1e4c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:55.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e467fc640 1 -- 172.21.15.80:0/1086321773 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c015070 con 0x7f1e500d06e0 2024-04-25T13:51:55.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e467fc640 1 -- 172.21.15.80:0/1086321773 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1e4c007a30 con 0x7f1e500d06e0 2024-04-25T13:51:55.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e467fc640 1 -- 172.21.15.80:0/1086321773 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e4c007e20 con 0x7f1e500d06e0 2024-04-25T13:51:55.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e501450c0 con 0x7f1e500d06e0 2024-04-25T13:51:55.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e50145680 con 0x7f1e500d06e0 2024-04-25T13:51:55.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e50144b50 con 0x7f1e500d06e0 2024-04-25T13:51:55.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e467fc640 1 -- 172.21.15.80:0/1086321773 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1e4c018070 con 0x7f1e500d06e0 2024-04-25T13:51:55.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e467fc640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e3406f430 0x7f1e34071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.007+0000 7f1e467fc640 1 -- 172.21.15.80:0/1086321773 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f1e4c08f150 con 0x7f1e500d06e0 2024-04-25T13:51:55.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.011+0000 7f1e55884640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e3406f430 0x7f1e34071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.011+0000 7f1e55884640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e3406f430 0x7f1e34071940 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1e40000f70 tx=0x7f1e40007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:55.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.015+0000 7f1e467fc640 1 -- 172.21.15.80:0/1086321773 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1e4c05c0d0 con 0x7f1e500d06e0 2024-04-25T13:51:55.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.163+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1e5015e120 con 0x7f1e3406f430 2024-04-25T13:51:55.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.179+0000 7f1e467fc640 1 -- 172.21.15.80:0/1086321773 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247988 (secure 0 0 0) 0x7f1e5015e120 con 0x7f1e3406f430 2024-04-25T13:51:55.178 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:55.179 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:51:55.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e3406f430 msgr2=0x7f1e34071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e3406f430 0x7f1e34071940 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1e40000f70 tx=0x7f1e40007a10 comp rx=0 tx=0).stop 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e500d06e0 msgr2=0x7f1e50145e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e500d06e0 0x7f1e50145e80 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f1e4c00c820 tx=0x7f1e4c00cce0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 shutdown_connections 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1e3406f430 0x7f1e34071940 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1e50161f90 0x7f1e50131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1e500d06e0 0x7f1e50145e80 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 --2- 172.21.15.80:0/1086321773 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1e5006e9e0 0x7f1e50147430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 >> 172.21.15.80:0/1086321773 conn(0x7f1e5006d080 msgr2=0x7f1e500b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 shutdown_connections 2024-04-25T13:51:55.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.183+0000 7f1e57ae8640 1 -- 172.21.15.80:0/1086321773 wait complete. 2024-04-25T13:51:55.197 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":527,"stamp":"2024-04-25T13:51:54.355029+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9401,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553098,"statfs":{"total":1600000000,"available":1590372364,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.699844"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785413+0000","last_change":"2024-04-25T13:51:50.697229+0000","last_active":"2024-04-25T13:51:51.785413+0000","last_peered":"2024-04-25T13:51:51.785413+0000","last_clean":"2024-04-25T13:51:51.785413+0000","last_became_active":"2024-04-25T13:51:50.688184+0000","last_became_peered":"2024-04-25T13:51:50.688184+0000","last_unstale":"2024-04-25T13:51:51.785413+0000","last_undegraded":"2024-04-25T13:51:51.785413+0000","last_fullsized":"2024-04-25T13:51:51.785413+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:04:41.511822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782652+0000","last_change":"2024-04-25T13:51:50.702744+0000","last_active":"2024-04-25T13:51:51.782652+0000","last_peered":"2024-04-25T13:51:51.782652+0000","last_clean":"2024-04-25T13:51:51.782652+0000","last_became_active":"2024-04-25T13:51:50.697026+0000","last_became_peered":"2024-04-25T13:51:50.697026+0000","last_unstale":"2024-04-25T13:51:51.782652+0000","last_undegraded":"2024-04-25T13:51:51.782652+0000","last_fullsized":"2024-04-25T13:51:51.782652+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:25.676539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719778+0000","last_change":"2024-04-25T13:51:50.711083+0000","last_active":"2024-04-25T13:51:51.719778+0000","last_peered":"2024-04-25T13:51:51.719778+0000","last_clean":"2024-04-25T13:51:51.719778+0000","last_became_active":"2024-04-25T13:51:50.709923+0000","last_became_peered":"2024-04-25T13:51:50.709923+0000","last_unstale":"2024-04-25T13:51:51.719778+0000","last_undegraded":"2024-04-25T13:51:51.719778+0000","last_fullsized":"2024-04-25T13:51:51.719778+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:19.593245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782805+0000","last_change":"2024-04-25T13:51:50.709572+0000","last_active":"2024-04-25T13:51:51.782805+0000","last_peered":"2024-04-25T13:51:51.782805+0000","last_clean":"2024-04-25T13:51:51.782805+0000","last_became_active":"2024-04-25T13:51:50.706315+0000","last_became_peered":"2024-04-25T13:51:50.706315+0000","last_unstale":"2024-04-25T13:51:51.782805+0000","last_undegraded":"2024-04-25T13:51:51.782805+0000","last_fullsized":"2024-04-25T13:51:51.782805+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:59.728167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783079+0000","last_change":"2024-04-25T13:51:50.702171+0000","last_active":"2024-04-25T13:51:51.783079+0000","last_peered":"2024-04-25T13:51:51.783079+0000","last_clean":"2024-04-25T13:51:51.783079+0000","last_became_active":"2024-04-25T13:51:50.701004+0000","last_became_peered":"2024-04-25T13:51:50.701004+0000","last_unstale":"2024-04-25T13:51:51.783079+0000","last_undegraded":"2024-04-25T13:51:51.783079+0000","last_fullsized":"2024-04-25T13:51:51.783079+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:25:04.887335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783544+0000","last_change":"2024-04-25T13:51:50.694741+0000","last_active":"2024-04-25T13:51:51.783544+0000","last_peered":"2024-04-25T13:51:51.783544+0000","last_clean":"2024-04-25T13:51:51.783544+0000","last_became_active":"2024-04-25T13:51:50.694020+0000","last_became_peered":"2024-04-25T13:51:50.694020+0000","last_unstale":"2024-04-25T13:51:51.783544+0000","last_undegraded":"2024-04-25T13:51:51.783544+0000","last_fullsized":"2024-04-25T13:51:51.783544+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:00.521742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786256+0000","last_change":"2024-04-25T13:51:50.712643+0000","last_active":"2024-04-25T13:51:51.786256+0000","last_peered":"2024-04-25T13:51:51.786256+0000","last_clean":"2024-04-25T13:51:51.786256+0000","last_became_active":"2024-04-25T13:51:50.704921+0000","last_became_peered":"2024-04-25T13:51:50.704921+0000","last_unstale":"2024-04-25T13:51:51.786256+0000","last_undegraded":"2024-04-25T13:51:51.786256+0000","last_fullsized":"2024-04-25T13:51:51.786256+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:35:33.236236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783163+0000","last_change":"2024-04-25T13:51:50.712752+0000","last_active":"2024-04-25T13:51:51.783163+0000","last_peered":"2024-04-25T13:51:51.783163+0000","last_clean":"2024-04-25T13:51:51.783163+0000","last_became_active":"2024-04-25T13:51:50.711972+0000","last_became_peered":"2024-04-25T13:51:50.711972+0000","last_unstale":"2024-04-25T13:51:51.783163+0000","last_undegraded":"2024-04-25T13:51:51.783163+0000","last_fullsized":"2024-04-25T13:51:51.783163+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:34.577388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783246+0000","last_change":"2024-04-25T13:51:50.695646+0000","last_active":"2024-04-25T13:51:51.783246+0000","last_peered":"2024-04-25T13:51:51.783246+0000","last_clean":"2024-04-25T13:51:51.783246+0000","last_became_active":"2024-04-25T13:51:50.692814+0000","last_became_peered":"2024-04-25T13:51:50.692814+0000","last_unstale":"2024-04-25T13:51:51.783246+0000","last_undegraded":"2024-04-25T13:51:51.783246+0000","last_fullsized":"2024-04-25T13:51:51.783246+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:36.237846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783573+0000","last_change":"2024-04-25T13:51:50.698102+0000","last_active":"2024-04-25T13:51:51.783573+0000","last_peered":"2024-04-25T13:51:51.783573+0000","last_clean":"2024-04-25T13:51:51.783573+0000","last_became_active":"2024-04-25T13:51:50.692066+0000","last_became_peered":"2024-04-25T13:51:50.692066+0000","last_unstale":"2024-04-25T13:51:51.783573+0000","last_undegraded":"2024-04-25T13:51:51.783573+0000","last_fullsized":"2024-04-25T13:51:51.783573+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:17:18.999599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783014+0000","last_change":"2024-04-25T13:51:50.704691+0000","last_active":"2024-04-25T13:51:51.783014+0000","last_peered":"2024-04-25T13:51:51.783014+0000","last_clean":"2024-04-25T13:51:51.783014+0000","last_became_active":"2024-04-25T13:51:50.703945+0000","last_became_peered":"2024-04-25T13:51:50.703945+0000","last_unstale":"2024-04-25T13:51:51.783014+0000","last_undegraded":"2024-04-25T13:51:51.783014+0000","last_fullsized":"2024-04-25T13:51:51.783014+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:59.914623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784037+0000","last_change":"2024-04-25T13:51:50.701268+0000","last_active":"2024-04-25T13:51:51.784037+0000","last_peered":"2024-04-25T13:51:51.784037+0000","last_clean":"2024-04-25T13:51:51.784037+0000","last_became_active":"2024-04-25T13:51:50.694246+0000","last_became_peered":"2024-04-25T13:51:50.694246+0000","last_unstale":"2024-04-25T13:51:51.784037+0000","last_undegraded":"2024-04-25T13:51:51.784037+0000","last_fullsized":"2024-04-25T13:51:51.784037+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:38:43.634430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718663+0000","last_change":"2024-04-25T13:51:50.707493+0000","last_active":"2024-04-25T13:51:51.718663+0000","last_peered":"2024-04-25T13:51:51.718663+0000","last_clean":"2024-04-25T13:51:51.718663+0000","last_became_active":"2024-04-25T13:51:50.706316+0000","last_became_peered":"2024-04-25T13:51:50.706316+0000","last_unstale":"2024-04-25T13:51:51.718663+0000","last_undegraded":"2024-04-25T13:51:51.718663+0000","last_fullsized":"2024-04-25T13:51:51.718663+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:35:12.987095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718367+0000","last_change":"2024-04-25T13:51:50.703773+0000","last_active":"2024-04-25T13:51:51.718367+0000","last_peered":"2024-04-25T13:51:51.718367+0000","last_clean":"2024-04-25T13:51:51.718367+0000","last_became_active":"2024-04-25T13:51:50.697623+0000","last_became_peered":"2024-04-25T13:51:50.697623+0000","last_unstale":"2024-04-25T13:51:51.718367+0000","last_undegraded":"2024-04-25T13:51:51.718367+0000","last_fullsized":"2024-04-25T13:51:51.718367+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:20.821609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783952+0000","last_change":"2024-04-25T13:51:50.687296+0000","last_active":"2024-04-25T13:51:51.783952+0000","last_peered":"2024-04-25T13:51:51.783952+0000","last_clean":"2024-04-25T13:51:51.783952+0000","last_became_active":"2024-04-25T13:51:50.686370+0000","last_became_peered":"2024-04-25T13:51:50.686370+0000","last_unstale":"2024-04-25T13:51:51.783952+0000","last_undegraded":"2024-04-25T13:51:51.783952+0000","last_fullsized":"2024-04-25T13:51:51.783952+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:50.850144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786453+0000","last_change":"2024-04-25T13:51:50.703700+0000","last_active":"2024-04-25T13:51:51.786453+0000","last_peered":"2024-04-25T13:51:51.786453+0000","last_clean":"2024-04-25T13:51:51.786453+0000","last_became_active":"2024-04-25T13:51:50.701557+0000","last_became_peered":"2024-04-25T13:51:50.701557+0000","last_unstale":"2024-04-25T13:51:51.786453+0000","last_undegraded":"2024-04-25T13:51:51.786453+0000","last_fullsized":"2024-04-25T13:51:51.786453+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:00:31.264908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783333+0000","last_change":"2024-04-25T13:51:50.702423+0000","last_active":"2024-04-25T13:51:51.783333+0000","last_peered":"2024-04-25T13:51:51.783333+0000","last_clean":"2024-04-25T13:51:51.783333+0000","last_became_active":"2024-04-25T13:51:50.701373+0000","last_became_peered":"2024-04-25T13:51:50.701373+0000","last_unstale":"2024-04-25T13:51:51.783333+0000","last_undegraded":"2024-04-25T13:51:51.783333+0000","last_fullsized":"2024-04-25T13:51:51.783333+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:42:11.722868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.677156+0000","last_change":"2024-04-25T13:51:50.706454+0000","last_active":"2024-04-25T13:51:51.677156+0000","last_peered":"2024-04-25T13:51:51.677156+0000","last_clean":"2024-04-25T13:51:51.677156+0000","last_became_active":"2024-04-25T13:51:50.702993+0000","last_became_peered":"2024-04-25T13:51:50.702993+0000","last_unstale":"2024-04-25T13:51:51.677156+0000","last_undegraded":"2024-04-25T13:51:51.677156+0000","last_fullsized":"2024-04-25T13:51:51.677156+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:05.767580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783182+0000","last_change":"2024-04-25T13:51:50.717854+0000","last_active":"2024-04-25T13:51:51.783182+0000","last_peered":"2024-04-25T13:51:51.783182+0000","last_clean":"2024-04-25T13:51:51.783182+0000","last_became_active":"2024-04-25T13:51:50.717193+0000","last_became_peered":"2024-04-25T13:51:50.717193+0000","last_unstale":"2024-04-25T13:51:51.783182+0000","last_undegraded":"2024-04-25T13:51:51.783182+0000","last_fullsized":"2024-04-25T13:51:51.783182+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:14.615894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718496+0000","last_change":"2024-04-25T13:51:50.694370+0000","last_active":"2024-04-25T13:51:51.718496+0000","last_peered":"2024-04-25T13:51:51.718496+0000","last_clean":"2024-04-25T13:51:51.718496+0000","last_became_active":"2024-04-25T13:51:50.693039+0000","last_became_peered":"2024-04-25T13:51:50.693039+0000","last_unstale":"2024-04-25T13:51:51.718496+0000","last_undegraded":"2024-04-25T13:51:51.718496+0000","last_fullsized":"2024-04-25T13:51:51.718496+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:10.571985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783707+0000","last_change":"2024-04-25T13:51:50.702590+0000","last_active":"2024-04-25T13:51:51.783707+0000","last_peered":"2024-04-25T13:51:51.783707+0000","last_clean":"2024-04-25T13:51:51.783707+0000","last_became_active":"2024-04-25T13:51:50.701912+0000","last_became_peered":"2024-04-25T13:51:50.701912+0000","last_unstale":"2024-04-25T13:51:51.783707+0000","last_undegraded":"2024-04-25T13:51:51.783707+0000","last_fullsized":"2024-04-25T13:51:51.783707+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:41.589275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784336+0000","last_change":"2024-04-25T13:51:50.705759+0000","last_active":"2024-04-25T13:51:51.784336+0000","last_peered":"2024-04-25T13:51:51.784336+0000","last_clean":"2024-04-25T13:51:51.784336+0000","last_became_active":"2024-04-25T13:51:50.700029+0000","last_became_peered":"2024-04-25T13:51:50.700029+0000","last_unstale":"2024-04-25T13:51:51.784336+0000","last_undegraded":"2024-04-25T13:51:51.784336+0000","last_fullsized":"2024-04-25T13:51:51.784336+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:51.708194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785262+0000","last_change":"2024-04-25T13:51:50.705583+0000","last_active":"2024-04-25T13:51:51.785262+0000","last_peered":"2024-04-25T13:51:51.785262+0000","last_clean":"2024-04-25T13:51:51.785262+0000","last_became_active":"2024-04-25T13:51:50.704862+0000","last_became_peered":"2024-04-25T13:51:50.704862+0000","last_unstale":"2024-04-25T13:51:51.785262+0000","last_undegraded":"2024-04-25T13:51:51.785262+0000","last_fullsized":"2024-04-25T13:51:51.785262+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:01:17.953128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782443+0000","last_change":"2024-04-25T13:51:50.696549+0000","last_active":"2024-04-25T13:51:51.782443+0000","last_peered":"2024-04-25T13:51:51.782443+0000","last_clean":"2024-04-25T13:51:51.782443+0000","last_became_active":"2024-04-25T13:51:50.694930+0000","last_became_peered":"2024-04-25T13:51:50.694930+0000","last_unstale":"2024-04-25T13:51:51.782443+0000","last_undegraded":"2024-04-25T13:51:51.782443+0000","last_fullsized":"2024-04-25T13:51:51.782443+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:54.128665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718772+0000","last_change":"2024-04-25T13:51:50.693802+0000","last_active":"2024-04-25T13:51:51.718772+0000","last_peered":"2024-04-25T13:51:51.718772+0000","last_clean":"2024-04-25T13:51:51.718772+0000","last_became_active":"2024-04-25T13:51:50.691885+0000","last_became_peered":"2024-04-25T13:51:50.691885+0000","last_unstale":"2024-04-25T13:51:51.718772+0000","last_undegraded":"2024-04-25T13:51:51.718772+0000","last_fullsized":"2024-04-25T13:51:51.718772+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:32.191434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786227+0000","last_change":"2024-04-25T13:51:50.694647+0000","last_active":"2024-04-25T13:51:51.786227+0000","last_peered":"2024-04-25T13:51:51.786227+0000","last_clean":"2024-04-25T13:51:51.786227+0000","last_became_active":"2024-04-25T13:51:50.693917+0000","last_became_peered":"2024-04-25T13:51:50.693917+0000","last_unstale":"2024-04-25T13:51:51.786227+0000","last_undegraded":"2024-04-25T13:51:51.786227+0000","last_fullsized":"2024-04-25T13:51:51.786227+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:29:56.742703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718603+0000","last_change":"2024-04-25T13:51:50.706471+0000","last_active":"2024-04-25T13:51:51.718603+0000","last_peered":"2024-04-25T13:51:51.718603+0000","last_clean":"2024-04-25T13:51:51.718603+0000","last_became_active":"2024-04-25T13:51:50.705492+0000","last_became_peered":"2024-04-25T13:51:50.705492+0000","last_unstale":"2024-04-25T13:51:51.718603+0000","last_undegraded":"2024-04-25T13:51:51.718603+0000","last_fullsized":"2024-04-25T13:51:51.718603+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:27.832656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783555+0000","last_change":"2024-04-25T13:51:50.713534+0000","last_active":"2024-04-25T13:51:51.783555+0000","last_peered":"2024-04-25T13:51:51.783555+0000","last_clean":"2024-04-25T13:51:51.783555+0000","last_became_active":"2024-04-25T13:51:50.713177+0000","last_became_peered":"2024-04-25T13:51:50.713177+0000","last_unstale":"2024-04-25T13:51:51.783555+0000","last_undegraded":"2024-04-25T13:51:51.783555+0000","last_fullsized":"2024-04-25T13:51:51.783555+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:56.471714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718720+0000","last_change":"2024-04-25T13:51:50.705709+0000","last_active":"2024-04-25T13:51:51.718720+0000","last_peered":"2024-04-25T13:51:51.718720+0000","last_clean":"2024-04-25T13:51:51.718720+0000","last_became_active":"2024-04-25T13:51:50.701186+0000","last_became_peered":"2024-04-25T13:51:50.701186+0000","last_unstale":"2024-04-25T13:51:51.718720+0000","last_undegraded":"2024-04-25T13:51:51.718720+0000","last_fullsized":"2024-04-25T13:51:51.718720+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:07.009066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674561+0000","last_change":"2024-04-25T13:51:50.706020+0000","last_active":"2024-04-25T13:51:51.674561+0000","last_peered":"2024-04-25T13:51:51.674561+0000","last_clean":"2024-04-25T13:51:51.674561+0000","last_became_active":"2024-04-25T13:51:50.705117+0000","last_became_peered":"2024-04-25T13:51:50.705117+0000","last_unstale":"2024-04-25T13:51:51.674561+0000","last_undegraded":"2024-04-25T13:51:51.674561+0000","last_fullsized":"2024-04-25T13:51:51.674561+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:04.661713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674162+0000","last_change":"2024-04-25T13:51:50.714789+0000","last_active":"2024-04-25T13:51:51.674162+0000","last_peered":"2024-04-25T13:51:51.674162+0000","last_clean":"2024-04-25T13:51:51.674162+0000","last_became_active":"2024-04-25T13:51:50.713827+0000","last_became_peered":"2024-04-25T13:51:50.713827+0000","last_unstale":"2024-04-25T13:51:51.674162+0000","last_undegraded":"2024-04-25T13:51:51.674162+0000","last_fullsized":"2024-04-25T13:51:51.674162+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:38:23.597177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676128+0000","last_change":"2024-04-25T13:51:50.701827+0000","last_active":"2024-04-25T13:51:51.676128+0000","last_peered":"2024-04-25T13:51:51.676128+0000","last_clean":"2024-04-25T13:51:51.676128+0000","last_became_active":"2024-04-25T13:51:50.698840+0000","last_became_peered":"2024-04-25T13:51:50.698840+0000","last_unstale":"2024-04-25T13:51:51.676128+0000","last_undegraded":"2024-04-25T13:51:51.676128+0000","last_fullsized":"2024-04-25T13:51:51.676128+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:22.162435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784052+0000","last_change":"2024-04-25T13:51:50.694293+0000","last_active":"2024-04-25T13:51:51.784052+0000","last_peered":"2024-04-25T13:51:51.784052+0000","last_clean":"2024-04-25T13:51:51.784052+0000","last_became_active":"2024-04-25T13:51:50.693570+0000","last_became_peered":"2024-04-25T13:51:50.693570+0000","last_unstale":"2024-04-25T13:51:51.784052+0000","last_undegraded":"2024-04-25T13:51:51.784052+0000","last_fullsized":"2024-04-25T13:51:51.784052+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:55:42.023479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"186'4","reported_seq":18,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782306+0000","last_change":"2024-04-25T13:51:49.702768+0000","last_active":"2024-04-25T13:51:51.782306+0000","last_peered":"2024-04-25T13:51:51.782306+0000","last_clean":"2024-04-25T13:51:51.782306+0000","last_became_active":"2024-04-25T13:51:49.702087+0000","last_became_peered":"2024-04-25T13:51:49.702087+0000","last_unstale":"2024-04-25T13:51:51.782306+0000","last_undegraded":"2024-04-25T13:51:51.782306+0000","last_fullsized":"2024-04-25T13:51:51.782306+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:34:07.971890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674339+0000","last_change":"2024-04-25T13:51:50.713802+0000","last_active":"2024-04-25T13:51:51.674339+0000","last_peered":"2024-04-25T13:51:51.674339+0000","last_clean":"2024-04-25T13:51:51.674339+0000","last_became_active":"2024-04-25T13:51:50.709321+0000","last_became_peered":"2024-04-25T13:51:50.709321+0000","last_unstale":"2024-04-25T13:51:51.674339+0000","last_undegraded":"2024-04-25T13:51:51.674339+0000","last_fullsized":"2024-04-25T13:51:51.674339+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:03.244199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.673585+0000","last_change":"2024-04-25T13:51:49.705814+0000","last_active":"2024-04-25T13:51:51.673585+0000","last_peered":"2024-04-25T13:51:51.673585+0000","last_clean":"2024-04-25T13:51:51.673585+0000","last_became_active":"2024-04-25T13:51:49.703717+0000","last_became_peered":"2024-04-25T13:51:49.703717+0000","last_unstale":"2024-04-25T13:51:51.673585+0000","last_undegraded":"2024-04-25T13:51:51.673585+0000","last_fullsized":"2024-04-25T13:51:51.673585+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:29:50.544404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783872+0000","last_change":"2024-04-25T13:51:50.707481+0000","last_active":"2024-04-25T13:51:51.783872+0000","last_peered":"2024-04-25T13:51:51.783872+0000","last_clean":"2024-04-25T13:51:51.783872+0000","last_became_active":"2024-04-25T13:51:50.706771+0000","last_became_peered":"2024-04-25T13:51:50.706771+0000","last_unstale":"2024-04-25T13:51:51.783872+0000","last_undegraded":"2024-04-25T13:51:51.783872+0000","last_fullsized":"2024-04-25T13:51:51.783872+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:58.272633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785674+0000","last_change":"2024-04-25T13:51:49.692055+0000","last_active":"2024-04-25T13:51:51.785674+0000","last_peered":"2024-04-25T13:51:51.785674+0000","last_clean":"2024-04-25T13:51:51.785674+0000","last_became_active":"2024-04-25T13:51:49.691889+0000","last_became_peered":"2024-04-25T13:51:49.691889+0000","last_unstale":"2024-04-25T13:51:51.785674+0000","last_undegraded":"2024-04-25T13:51:51.785674+0000","last_fullsized":"2024-04-25T13:51:51.785674+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:26.600957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719049+0000","last_change":"2024-04-25T13:51:50.701062+0000","last_active":"2024-04-25T13:51:51.719049+0000","last_peered":"2024-04-25T13:51:51.719049+0000","last_clean":"2024-04-25T13:51:51.719049+0000","last_became_active":"2024-04-25T13:51:50.699625+0000","last_became_peered":"2024-04-25T13:51:50.699625+0000","last_unstale":"2024-04-25T13:51:51.719049+0000","last_undegraded":"2024-04-25T13:51:51.719049+0000","last_fullsized":"2024-04-25T13:51:51.719049+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:52:18.288634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783754+0000","last_change":"2024-04-25T13:51:49.687991+0000","last_active":"2024-04-25T13:51:51.783754+0000","last_peered":"2024-04-25T13:51:51.783754+0000","last_clean":"2024-04-25T13:51:51.783754+0000","last_became_active":"2024-04-25T13:51:49.687693+0000","last_became_peered":"2024-04-25T13:51:49.687693+0000","last_unstale":"2024-04-25T13:51:51.783754+0000","last_undegraded":"2024-04-25T13:51:51.783754+0000","last_fullsized":"2024-04-25T13:51:51.783754+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:28:08.107707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784756+0000","last_change":"2024-04-25T13:51:50.684961+0000","last_active":"2024-04-25T13:51:51.784756+0000","last_peered":"2024-04-25T13:51:51.784756+0000","last_clean":"2024-04-25T13:51:51.784756+0000","last_became_active":"2024-04-25T13:51:50.684168+0000","last_became_peered":"2024-04-25T13:51:50.684168+0000","last_unstale":"2024-04-25T13:51:51.784756+0000","last_undegraded":"2024-04-25T13:51:51.784756+0000","last_fullsized":"2024-04-25T13:51:51.784756+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:31:30.192436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719607+0000","last_change":"2024-04-25T13:51:49.694022+0000","last_active":"2024-04-25T13:51:51.719607+0000","last_peered":"2024-04-25T13:51:51.719607+0000","last_clean":"2024-04-25T13:51:51.719607+0000","last_became_active":"2024-04-25T13:51:49.693294+0000","last_became_peered":"2024-04-25T13:51:49.693294+0000","last_unstale":"2024-04-25T13:51:51.719607+0000","last_undegraded":"2024-04-25T13:51:51.719607+0000","last_fullsized":"2024-04-25T13:51:51.719607+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:59.266242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784857+0000","last_change":"2024-04-25T13:51:50.709506+0000","last_active":"2024-04-25T13:51:51.784857+0000","last_peered":"2024-04-25T13:51:51.784857+0000","last_clean":"2024-04-25T13:51:51.784857+0000","last_became_active":"2024-04-25T13:51:50.708831+0000","last_became_peered":"2024-04-25T13:51:50.708831+0000","last_unstale":"2024-04-25T13:51:51.784857+0000","last_undegraded":"2024-04-25T13:51:51.784857+0000","last_fullsized":"2024-04-25T13:51:51.784857+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:52.663346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783861+0000","last_change":"2024-04-25T13:51:49.679494+0000","last_active":"2024-04-25T13:51:51.783861+0000","last_peered":"2024-04-25T13:51:51.783861+0000","last_clean":"2024-04-25T13:51:51.783861+0000","last_became_active":"2024-04-25T13:51:49.679191+0000","last_became_peered":"2024-04-25T13:51:49.679191+0000","last_unstale":"2024-04-25T13:51:51.783861+0000","last_undegraded":"2024-04-25T13:51:51.783861+0000","last_fullsized":"2024-04-25T13:51:51.783861+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:47.523506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784699+0000","last_change":"2024-04-25T13:51:50.688984+0000","last_active":"2024-04-25T13:51:51.784699+0000","last_peered":"2024-04-25T13:51:51.784699+0000","last_clean":"2024-04-25T13:51:51.784699+0000","last_became_active":"2024-04-25T13:51:50.688028+0000","last_became_peered":"2024-04-25T13:51:50.688028+0000","last_unstale":"2024-04-25T13:51:51.784699+0000","last_undegraded":"2024-04-25T13:51:51.784699+0000","last_fullsized":"2024-04-25T13:51:51.784699+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:57.622221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782758+0000","last_change":"2024-04-25T13:51:49.689427+0000","last_active":"2024-04-25T13:51:51.782758+0000","last_peered":"2024-04-25T13:51:51.782758+0000","last_clean":"2024-04-25T13:51:51.782758+0000","last_became_active":"2024-04-25T13:51:49.687499+0000","last_became_peered":"2024-04-25T13:51:49.687499+0000","last_unstale":"2024-04-25T13:51:51.782758+0000","last_undegraded":"2024-04-25T13:51:51.782758+0000","last_fullsized":"2024-04-25T13:51:51.782758+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:04.777830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783361+0000","last_change":"2024-04-25T13:51:50.692946+0000","last_active":"2024-04-25T13:51:51.783361+0000","last_peered":"2024-04-25T13:51:51.783361+0000","last_clean":"2024-04-25T13:51:51.783361+0000","last_became_active":"2024-04-25T13:51:50.692186+0000","last_became_peered":"2024-04-25T13:51:50.692186+0000","last_unstale":"2024-04-25T13:51:51.783361+0000","last_undegraded":"2024-04-25T13:51:51.783361+0000","last_fullsized":"2024-04-25T13:51:51.783361+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:00:04.185988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785586+0000","last_change":"2024-04-25T13:51:49.676972+0000","last_active":"2024-04-25T13:51:51.785586+0000","last_peered":"2024-04-25T13:51:51.785586+0000","last_clean":"2024-04-25T13:51:51.785586+0000","last_became_active":"2024-04-25T13:51:49.676721+0000","last_became_peered":"2024-04-25T13:51:49.676721+0000","last_unstale":"2024-04-25T13:51:51.785586+0000","last_undegraded":"2024-04-25T13:51:51.785586+0000","last_fullsized":"2024-04-25T13:51:51.785586+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:09.544234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676120+0000","last_change":"2024-04-25T13:51:50.699774+0000","last_active":"2024-04-25T13:51:51.676120+0000","last_peered":"2024-04-25T13:51:51.676120+0000","last_clean":"2024-04-25T13:51:51.676120+0000","last_became_active":"2024-04-25T13:51:50.698918+0000","last_became_peered":"2024-04-25T13:51:50.698918+0000","last_unstale":"2024-04-25T13:51:51.676120+0000","last_undegraded":"2024-04-25T13:51:51.676120+0000","last_fullsized":"2024-04-25T13:51:51.676120+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:57.556654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784051+0000","last_change":"2024-04-25T13:51:49.695156+0000","last_active":"2024-04-25T13:51:51.784051+0000","last_peered":"2024-04-25T13:51:51.784051+0000","last_clean":"2024-04-25T13:51:51.784051+0000","last_became_active":"2024-04-25T13:51:49.694878+0000","last_became_peered":"2024-04-25T13:51:49.694878+0000","last_unstale":"2024-04-25T13:51:51.784051+0000","last_undegraded":"2024-04-25T13:51:51.784051+0000","last_fullsized":"2024-04-25T13:51:51.784051+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:00.199207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676056+0000","last_change":"2024-04-25T13:51:50.705533+0000","last_active":"2024-04-25T13:51:51.676056+0000","last_peered":"2024-04-25T13:51:51.676056+0000","last_clean":"2024-04-25T13:51:51.676056+0000","last_became_active":"2024-04-25T13:51:50.703454+0000","last_became_peered":"2024-04-25T13:51:50.703454+0000","last_unstale":"2024-04-25T13:51:51.676056+0000","last_undegraded":"2024-04-25T13:51:51.676056+0000","last_fullsized":"2024-04-25T13:51:51.676056+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:31.247845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"186'3","reported_seq":17,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785595+0000","last_change":"2024-04-25T13:51:49.688388+0000","last_active":"2024-04-25T13:51:51.785595+0000","last_peered":"2024-04-25T13:51:51.785595+0000","last_clean":"2024-04-25T13:51:51.785595+0000","last_became_active":"2024-04-25T13:51:49.688155+0000","last_became_peered":"2024-04-25T13:51:49.688155+0000","last_unstale":"2024-04-25T13:51:51.785595+0000","last_undegraded":"2024-04-25T13:51:51.785595+0000","last_fullsized":"2024-04-25T13:51:51.785595+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:58:21.369037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674791+0000","last_change":"2024-04-25T13:51:50.711409+0000","last_active":"2024-04-25T13:51:51.674791+0000","last_peered":"2024-04-25T13:51:51.674791+0000","last_clean":"2024-04-25T13:51:51.674791+0000","last_became_active":"2024-04-25T13:51:50.705564+0000","last_became_peered":"2024-04-25T13:51:50.705564+0000","last_unstale":"2024-04-25T13:51:51.674791+0000","last_undegraded":"2024-04-25T13:51:51.674791+0000","last_fullsized":"2024-04-25T13:51:51.674791+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:28:52.917202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"186'6","reported_seq":20,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.673940+0000","last_change":"2024-04-25T13:51:49.704400+0000","last_active":"2024-04-25T13:51:51.673940+0000","last_peered":"2024-04-25T13:51:51.673940+0000","last_clean":"2024-04-25T13:51:51.673940+0000","last_became_active":"2024-04-25T13:51:49.702643+0000","last_became_peered":"2024-04-25T13:51:49.702643+0000","last_unstale":"2024-04-25T13:51:51.673940+0000","last_undegraded":"2024-04-25T13:51:51.673940+0000","last_fullsized":"2024-04-25T13:51:51.673940+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:52:57.646098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676464+0000","last_change":"2024-04-25T13:51:50.690953+0000","last_active":"2024-04-25T13:51:51.676464+0000","last_peered":"2024-04-25T13:51:51.676464+0000","last_clean":"2024-04-25T13:51:51.676464+0000","last_became_active":"2024-04-25T13:51:50.690213+0000","last_became_peered":"2024-04-25T13:51:50.690213+0000","last_unstale":"2024-04-25T13:51:51.676464+0000","last_undegraded":"2024-04-25T13:51:51.676464+0000","last_fullsized":"2024-04-25T13:51:51.676464+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:14.979862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785980+0000","last_change":"2024-04-25T13:51:49.680836+0000","last_active":"2024-04-25T13:51:51.785980+0000","last_peered":"2024-04-25T13:51:51.785980+0000","last_clean":"2024-04-25T13:51:51.785980+0000","last_became_active":"2024-04-25T13:51:49.680303+0000","last_became_peered":"2024-04-25T13:51:49.680303+0000","last_unstale":"2024-04-25T13:51:51.785980+0000","last_undegraded":"2024-04-25T13:51:51.785980+0000","last_fullsized":"2024-04-25T13:51:51.785980+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:13.903872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785023+0000","last_change":"2024-04-25T13:51:50.698034+0000","last_active":"2024-04-25T13:51:51.785023+0000","last_peered":"2024-04-25T13:51:51.785023+0000","last_clean":"2024-04-25T13:51:51.785023+0000","last_became_active":"2024-04-25T13:51:50.697297+0000","last_became_peered":"2024-04-25T13:51:50.697297+0000","last_unstale":"2024-04-25T13:51:51.785023+0000","last_undegraded":"2024-04-25T13:51:51.785023+0000","last_fullsized":"2024-04-25T13:51:51.785023+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:51:30.747047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676861+0000","last_change":"2024-04-25T13:51:49.675554+0000","last_active":"2024-04-25T13:51:51.676861+0000","last_peered":"2024-04-25T13:51:51.676861+0000","last_clean":"2024-04-25T13:51:51.676861+0000","last_became_active":"2024-04-25T13:51:49.675179+0000","last_became_peered":"2024-04-25T13:51:49.675179+0000","last_unstale":"2024-04-25T13:51:51.676861+0000","last_undegraded":"2024-04-25T13:51:51.676861+0000","last_fullsized":"2024-04-25T13:51:51.676861+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:48.539861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782921+0000","last_change":"2024-04-25T13:51:50.708874+0000","last_active":"2024-04-25T13:51:51.782921+0000","last_peered":"2024-04-25T13:51:51.782921+0000","last_clean":"2024-04-25T13:51:51.782921+0000","last_became_active":"2024-04-25T13:51:50.707578+0000","last_became_peered":"2024-04-25T13:51:50.707578+0000","last_unstale":"2024-04-25T13:51:51.782921+0000","last_undegraded":"2024-04-25T13:51:51.782921+0000","last_fullsized":"2024-04-25T13:51:51.782921+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:08.365733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785217+0000","last_change":"2024-04-25T13:51:49.695717+0000","last_active":"2024-04-25T13:51:51.785217+0000","last_peered":"2024-04-25T13:51:51.785217+0000","last_clean":"2024-04-25T13:51:51.785217+0000","last_became_active":"2024-04-25T13:51:49.693639+0000","last_became_peered":"2024-04-25T13:51:49.693639+0000","last_unstale":"2024-04-25T13:51:51.785217+0000","last_undegraded":"2024-04-25T13:51:51.785217+0000","last_fullsized":"2024-04-25T13:51:51.785217+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:42.217857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719105+0000","last_change":"2024-04-25T13:51:50.688074+0000","last_active":"2024-04-25T13:51:51.719105+0000","last_peered":"2024-04-25T13:51:51.719105+0000","last_clean":"2024-04-25T13:51:51.719105+0000","last_became_active":"2024-04-25T13:51:50.684820+0000","last_became_peered":"2024-04-25T13:51:50.684820+0000","last_unstale":"2024-04-25T13:51:51.719105+0000","last_undegraded":"2024-04-25T13:51:51.719105+0000","last_fullsized":"2024-04-25T13:51:51.719105+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:10.885680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"186'8","reported_seq":22,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.675125+0000","last_change":"2024-04-25T13:51:49.693899+0000","last_active":"2024-04-25T13:51:51.675125+0000","last_peered":"2024-04-25T13:51:51.675125+0000","last_clean":"2024-04-25T13:51:51.675125+0000","last_became_active":"2024-04-25T13:51:49.693082+0000","last_became_peered":"2024-04-25T13:51:49.693082+0000","last_unstale":"2024-04-25T13:51:51.675125+0000","last_undegraded":"2024-04-25T13:51:51.675125+0000","last_fullsized":"2024-04-25T13:51:51.675125+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:50:32.260557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785366+0000","last_change":"2024-04-25T13:51:50.693691+0000","last_active":"2024-04-25T13:51:51.785366+0000","last_peered":"2024-04-25T13:51:51.785366+0000","last_clean":"2024-04-25T13:51:51.785366+0000","last_became_active":"2024-04-25T13:51:50.692526+0000","last_became_peered":"2024-04-25T13:51:50.692526+0000","last_unstale":"2024-04-25T13:51:51.785366+0000","last_undegraded":"2024-04-25T13:51:51.785366+0000","last_fullsized":"2024-04-25T13:51:51.785366+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:39.227639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719202+0000","last_change":"2024-04-25T13:51:49.699624+0000","last_active":"2024-04-25T13:51:51.719202+0000","last_peered":"2024-04-25T13:51:51.719202+0000","last_clean":"2024-04-25T13:51:51.719202+0000","last_became_active":"2024-04-25T13:51:49.698758+0000","last_became_peered":"2024-04-25T13:51:49.698758+0000","last_unstale":"2024-04-25T13:51:51.719202+0000","last_undegraded":"2024-04-25T13:51:51.719202+0000","last_fullsized":"2024-04-25T13:51:51.719202+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:29.538938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676437+0000","last_change":"2024-04-25T13:51:50.716893+0000","last_active":"2024-04-25T13:51:51.676437+0000","last_peered":"2024-04-25T13:51:51.676437+0000","last_clean":"2024-04-25T13:51:51.676437+0000","last_became_active":"2024-04-25T13:51:50.716517+0000","last_became_peered":"2024-04-25T13:51:50.716517+0000","last_unstale":"2024-04-25T13:51:51.676437+0000","last_undegraded":"2024-04-25T13:51:51.676437+0000","last_fullsized":"2024-04-25T13:51:51.676437+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:59.473791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719454+0000","last_change":"2024-04-25T13:51:49.701634+0000","last_active":"2024-04-25T13:51:51.719454+0000","last_peered":"2024-04-25T13:51:51.719454+0000","last_clean":"2024-04-25T13:51:51.719454+0000","last_became_active":"2024-04-25T13:51:49.700041+0000","last_became_peered":"2024-04-25T13:51:49.700041+0000","last_unstale":"2024-04-25T13:51:51.719454+0000","last_undegraded":"2024-04-25T13:51:51.719454+0000","last_fullsized":"2024-04-25T13:51:51.719454+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:36:27.236152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785025+0000","last_change":"2024-04-25T13:51:50.706400+0000","last_active":"2024-04-25T13:51:51.785025+0000","last_peered":"2024-04-25T13:51:51.785025+0000","last_clean":"2024-04-25T13:51:51.785025+0000","last_became_active":"2024-04-25T13:51:50.705628+0000","last_became_peered":"2024-04-25T13:51:50.705628+0000","last_unstale":"2024-04-25T13:51:51.785025+0000","last_undegraded":"2024-04-25T13:51:51.785025+0000","last_fullsized":"2024-04-25T13:51:51.785025+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:15.340959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"186'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785769+0000","last_change":"2024-04-25T13:51:49.704674+0000","last_active":"2024-04-25T13:51:51.785769+0000","last_peered":"2024-04-25T13:51:51.785769+0000","last_clean":"2024-04-25T13:51:51.785769+0000","last_became_active":"2024-04-25T13:51:49.702709+0000","last_became_peered":"2024-04-25T13:51:49.702709+0000","last_unstale":"2024-04-25T13:51:51.785769+0000","last_undegraded":"2024-04-25T13:51:51.785769+0000","last_fullsized":"2024-04-25T13:51:51.785769+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:30.768066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786668+0000","last_change":"2024-04-25T13:51:50.712343+0000","last_active":"2024-04-25T13:51:51.786668+0000","last_peered":"2024-04-25T13:51:51.786668+0000","last_clean":"2024-04-25T13:51:51.786668+0000","last_became_active":"2024-04-25T13:51:50.711810+0000","last_became_peered":"2024-04-25T13:51:50.711810+0000","last_unstale":"2024-04-25T13:51:51.786668+0000","last_undegraded":"2024-04-25T13:51:51.786668+0000","last_fullsized":"2024-04-25T13:51:51.786668+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:24:50.645837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782092+0000","last_change":"2024-04-25T13:51:49.686462+0000","last_active":"2024-04-25T13:51:51.782092+0000","last_peered":"2024-04-25T13:51:51.782092+0000","last_clean":"2024-04-25T13:51:51.782092+0000","last_became_active":"2024-04-25T13:51:49.685685+0000","last_became_peered":"2024-04-25T13:51:49.685685+0000","last_unstale":"2024-04-25T13:51:51.782092+0000","last_undegraded":"2024-04-25T13:51:51.782092+0000","last_fullsized":"2024-04-25T13:51:51.782092+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:35.929105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.718993+0000","last_change":"2024-04-25T13:51:50.702149+0000","last_active":"2024-04-25T13:51:51.718993+0000","last_peered":"2024-04-25T13:51:51.718993+0000","last_clean":"2024-04-25T13:51:51.718993+0000","last_became_active":"2024-04-25T13:51:50.695831+0000","last_became_peered":"2024-04-25T13:51:50.695831+0000","last_unstale":"2024-04-25T13:51:51.718993+0000","last_undegraded":"2024-04-25T13:51:51.718993+0000","last_fullsized":"2024-04-25T13:51:51.718993+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:44.413262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784175+0000","last_change":"2024-04-25T13:51:49.693961+0000","last_active":"2024-04-25T13:51:51.784175+0000","last_peered":"2024-04-25T13:51:51.784175+0000","last_clean":"2024-04-25T13:51:51.784175+0000","last_became_active":"2024-04-25T13:51:49.693592+0000","last_became_peered":"2024-04-25T13:51:49.693592+0000","last_unstale":"2024-04-25T13:51:51.784175+0000","last_undegraded":"2024-04-25T13:51:51.784175+0000","last_fullsized":"2024-04-25T13:51:51.784175+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:20.767319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785174+0000","last_change":"2024-04-25T13:51:50.691990+0000","last_active":"2024-04-25T13:51:51.785174+0000","last_peered":"2024-04-25T13:51:51.785174+0000","last_clean":"2024-04-25T13:51:51.785174+0000","last_became_active":"2024-04-25T13:51:50.691085+0000","last_became_peered":"2024-04-25T13:51:50.691085+0000","last_unstale":"2024-04-25T13:51:51.785174+0000","last_undegraded":"2024-04-25T13:51:51.785174+0000","last_fullsized":"2024-04-25T13:51:51.785174+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:23:44.429056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782451+0000","last_change":"2024-04-25T13:51:49.689929+0000","last_active":"2024-04-25T13:51:51.782451+0000","last_peered":"2024-04-25T13:51:51.782451+0000","last_clean":"2024-04-25T13:51:51.782451+0000","last_became_active":"2024-04-25T13:51:49.688012+0000","last_became_peered":"2024-04-25T13:51:49.688012+0000","last_unstale":"2024-04-25T13:51:51.782451+0000","last_undegraded":"2024-04-25T13:51:51.782451+0000","last_fullsized":"2024-04-25T13:51:51.782451+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:47.582175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784924+0000","last_change":"2024-04-25T13:51:50.698400+0000","last_active":"2024-04-25T13:51:51.784924+0000","last_peered":"2024-04-25T13:51:51.784924+0000","last_clean":"2024-04-25T13:51:51.784924+0000","last_became_active":"2024-04-25T13:51:50.690952+0000","last_became_peered":"2024-04-25T13:51:50.690952+0000","last_unstale":"2024-04-25T13:51:51.784924+0000","last_undegraded":"2024-04-25T13:51:51.784924+0000","last_fullsized":"2024-04-25T13:51:51.784924+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:44:56.607704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786287+0000","last_change":"2024-04-25T13:51:49.701674+0000","last_active":"2024-04-25T13:51:51.786287+0000","last_peered":"2024-04-25T13:51:51.786287+0000","last_clean":"2024-04-25T13:51:51.786287+0000","last_became_active":"2024-04-25T13:51:49.701208+0000","last_became_peered":"2024-04-25T13:51:49.701208+0000","last_unstale":"2024-04-25T13:51:51.786287+0000","last_undegraded":"2024-04-25T13:51:51.786287+0000","last_fullsized":"2024-04-25T13:51:51.786287+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:52.785448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784284+0000","last_change":"2024-04-25T13:51:50.709025+0000","last_active":"2024-04-25T13:51:51.784284+0000","last_peered":"2024-04-25T13:51:51.784284+0000","last_clean":"2024-04-25T13:51:51.784284+0000","last_became_active":"2024-04-25T13:51:50.708219+0000","last_became_peered":"2024-04-25T13:51:50.708219+0000","last_unstale":"2024-04-25T13:51:51.784284+0000","last_undegraded":"2024-04-25T13:51:51.784284+0000","last_fullsized":"2024-04-25T13:51:51.784284+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:57:13.198741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786087+0000","last_change":"2024-04-25T13:51:49.705597+0000","last_active":"2024-04-25T13:51:51.786087+0000","last_peered":"2024-04-25T13:51:51.786087+0000","last_clean":"2024-04-25T13:51:51.786087+0000","last_became_active":"2024-04-25T13:51:49.703751+0000","last_became_peered":"2024-04-25T13:51:49.703751+0000","last_unstale":"2024-04-25T13:51:51.786087+0000","last_undegraded":"2024-04-25T13:51:51.786087+0000","last_fullsized":"2024-04-25T13:51:51.786087+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:56.960252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.677671+0000","last_change":"2024-04-25T13:51:50.703736+0000","last_active":"2024-04-25T13:51:51.677671+0000","last_peered":"2024-04-25T13:51:51.677671+0000","last_clean":"2024-04-25T13:51:51.677671+0000","last_became_active":"2024-04-25T13:51:50.701333+0000","last_became_peered":"2024-04-25T13:51:50.701333+0000","last_unstale":"2024-04-25T13:51:51.677671+0000","last_undegraded":"2024-04-25T13:51:51.677671+0000","last_fullsized":"2024-04-25T13:51:51.677671+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:22.799998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"186'2","reported_seq":16,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.677535+0000","last_change":"2024-04-25T13:51:49.695107+0000","last_active":"2024-04-25T13:51:51.677535+0000","last_peered":"2024-04-25T13:51:51.677535+0000","last_clean":"2024-04-25T13:51:51.677535+0000","last_became_active":"2024-04-25T13:51:49.694745+0000","last_became_peered":"2024-04-25T13:51:49.694745+0000","last_unstale":"2024-04-25T13:51:51.677535+0000","last_undegraded":"2024-04-25T13:51:51.677535+0000","last_fullsized":"2024-04-25T13:51:51.677535+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:27:27.752486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.786554+0000","last_change":"2024-04-25T13:51:50.713613+0000","last_active":"2024-04-25T13:51:51.786554+0000","last_peered":"2024-04-25T13:51:51.786554+0000","last_clean":"2024-04-25T13:51:51.786554+0000","last_became_active":"2024-04-25T13:51:50.711814+0000","last_became_peered":"2024-04-25T13:51:50.711814+0000","last_unstale":"2024-04-25T13:51:51.786554+0000","last_undegraded":"2024-04-25T13:51:51.786554+0000","last_fullsized":"2024-04-25T13:51:51.786554+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:32:03.001620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"187'1","reported_seq":15,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.721883+0000","last_change":"2024-04-25T13:51:49.704754+0000","last_active":"2024-04-25T13:51:51.721883+0000","last_peered":"2024-04-25T13:51:51.721883+0000","last_clean":"2024-04-25T13:51:51.721883+0000","last_became_active":"2024-04-25T13:51:49.699641+0000","last_became_peered":"2024-04-25T13:51:49.699641+0000","last_unstale":"2024-04-25T13:51:51.721883+0000","last_undegraded":"2024-04-25T13:51:51.721883+0000","last_fullsized":"2024-04-25T13:51:51.721883+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:14.302772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784444+0000","last_change":"2024-04-25T13:51:50.704674+0000","last_active":"2024-04-25T13:51:51.784444+0000","last_peered":"2024-04-25T13:51:51.784444+0000","last_clean":"2024-04-25T13:51:51.784444+0000","last_became_active":"2024-04-25T13:51:50.703719+0000","last_became_peered":"2024-04-25T13:51:50.703719+0000","last_unstale":"2024-04-25T13:51:51.784444+0000","last_undegraded":"2024-04-25T13:51:51.784444+0000","last_fullsized":"2024-04-25T13:51:51.784444+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:42:35.891863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783581+0000","last_change":"2024-04-25T13:51:49.683005+0000","last_active":"2024-04-25T13:51:51.783581+0000","last_peered":"2024-04-25T13:51:51.783581+0000","last_clean":"2024-04-25T13:51:51.783581+0000","last_became_active":"2024-04-25T13:51:49.681865+0000","last_became_peered":"2024-04-25T13:51:49.681865+0000","last_unstale":"2024-04-25T13:51:51.783581+0000","last_undegraded":"2024-04-25T13:51:51.783581+0000","last_fullsized":"2024-04-25T13:51:51.783581+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:35:14.401844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676876+0000","last_change":"2024-04-25T13:51:50.687988+0000","last_active":"2024-04-25T13:51:51.676876+0000","last_peered":"2024-04-25T13:51:51.676876+0000","last_clean":"2024-04-25T13:51:51.676876+0000","last_became_active":"2024-04-25T13:51:50.676039+0000","last_became_peered":"2024-04-25T13:51:50.676039+0000","last_unstale":"2024-04-25T13:51:51.676876+0000","last_undegraded":"2024-04-25T13:51:51.676876+0000","last_fullsized":"2024-04-25T13:51:51.676876+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:17:39.803115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676857+0000","last_change":"2024-04-25T13:51:49.701480+0000","last_active":"2024-04-25T13:51:51.676857+0000","last_peered":"2024-04-25T13:51:51.676857+0000","last_clean":"2024-04-25T13:51:51.676857+0000","last_became_active":"2024-04-25T13:51:49.701109+0000","last_became_peered":"2024-04-25T13:51:49.701109+0000","last_unstale":"2024-04-25T13:51:51.676857+0000","last_undegraded":"2024-04-25T13:51:51.676857+0000","last_fullsized":"2024-04-25T13:51:51.676857+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:07.990989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.677139+0000","last_change":"2024-04-25T13:51:50.700574+0000","last_active":"2024-04-25T13:51:51.677139+0000","last_peered":"2024-04-25T13:51:51.677139+0000","last_clean":"2024-04-25T13:51:51.677139+0000","last_became_active":"2024-04-25T13:51:50.697338+0000","last_became_peered":"2024-04-25T13:51:50.697338+0000","last_unstale":"2024-04-25T13:51:51.677139+0000","last_undegraded":"2024-04-25T13:51:51.677139+0000","last_fullsized":"2024-04-25T13:51:51.677139+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:16.115243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.675498+0000","last_change":"2024-04-25T13:51:49.686869+0000","last_active":"2024-04-25T13:51:51.675498+0000","last_peered":"2024-04-25T13:51:51.675498+0000","last_clean":"2024-04-25T13:51:51.675498+0000","last_became_active":"2024-04-25T13:51:49.684310+0000","last_became_peered":"2024-04-25T13:51:49.684310+0000","last_unstale":"2024-04-25T13:51:51.675498+0000","last_undegraded":"2024-04-25T13:51:51.675498+0000","last_fullsized":"2024-04-25T13:51:51.675498+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:06.439754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674659+0000","last_change":"2024-04-25T13:51:50.710605+0000","last_active":"2024-04-25T13:51:51.674659+0000","last_peered":"2024-04-25T13:51:51.674659+0000","last_clean":"2024-04-25T13:51:51.674659+0000","last_became_active":"2024-04-25T13:51:50.709309+0000","last_became_peered":"2024-04-25T13:51:50.709309+0000","last_unstale":"2024-04-25T13:51:51.674659+0000","last_undegraded":"2024-04-25T13:51:51.674659+0000","last_fullsized":"2024-04-25T13:51:51.674659+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:15.307555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.785326+0000","last_change":"2024-04-25T13:51:49.696135+0000","last_active":"2024-04-25T13:51:51.785326+0000","last_peered":"2024-04-25T13:51:51.785326+0000","last_clean":"2024-04-25T13:51:51.785326+0000","last_became_active":"2024-04-25T13:51:49.694223+0000","last_became_peered":"2024-04-25T13:51:49.694223+0000","last_unstale":"2024-04-25T13:51:51.785326+0000","last_undegraded":"2024-04-25T13:51:51.785326+0000","last_fullsized":"2024-04-25T13:51:51.785326+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:21.904303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.784479+0000","last_change":"2024-04-25T13:51:50.704009+0000","last_active":"2024-04-25T13:51:51.784479+0000","last_peered":"2024-04-25T13:51:51.784479+0000","last_clean":"2024-04-25T13:51:51.784479+0000","last_became_active":"2024-04-25T13:51:50.702487+0000","last_became_peered":"2024-04-25T13:51:50.702487+0000","last_unstale":"2024-04-25T13:51:51.784479+0000","last_undegraded":"2024-04-25T13:51:51.784479+0000","last_fullsized":"2024-04-25T13:51:51.784479+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:41.466337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.783692+0000","last_change":"2024-04-25T13:51:49.681283+0000","last_active":"2024-04-25T13:51:51.783692+0000","last_peered":"2024-04-25T13:51:51.783692+0000","last_clean":"2024-04-25T13:51:51.783692+0000","last_became_active":"2024-04-25T13:51:49.678905+0000","last_became_peered":"2024-04-25T13:51:49.678905+0000","last_unstale":"2024-04-25T13:51:51.783692+0000","last_undegraded":"2024-04-25T13:51:51.783692+0000","last_fullsized":"2024-04-25T13:51:51.783692+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:00:18.529412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.719313+0000","last_change":"2024-04-25T13:51:50.697270+0000","last_active":"2024-04-25T13:51:51.719313+0000","last_peered":"2024-04-25T13:51:51.719313+0000","last_clean":"2024-04-25T13:51:51.719313+0000","last_became_active":"2024-04-25T13:51:50.696465+0000","last_became_peered":"2024-04-25T13:51:50.696465+0000","last_unstale":"2024-04-25T13:51:51.719313+0000","last_undegraded":"2024-04-25T13:51:51.719313+0000","last_fullsized":"2024-04-25T13:51:51.719313+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:37.951762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.674258+0000","last_change":"2024-04-25T13:51:49.691466+0000","last_active":"2024-04-25T13:51:51.674258+0000","last_peered":"2024-04-25T13:51:51.674258+0000","last_clean":"2024-04-25T13:51:51.674258+0000","last_became_active":"2024-04-25T13:51:49.690640+0000","last_became_peered":"2024-04-25T13:51:49.690640+0000","last_unstale":"2024-04-25T13:51:51.674258+0000","last_undegraded":"2024-04-25T13:51:51.674258+0000","last_fullsized":"2024-04-25T13:51:51.674258+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:39:22.058511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":12,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.676555+0000","last_change":"2024-04-25T13:51:50.703020+0000","last_active":"2024-04-25T13:51:51.676555+0000","last_peered":"2024-04-25T13:51:51.676555+0000","last_clean":"2024-04-25T13:51:51.676555+0000","last_became_active":"2024-04-25T13:51:50.699795+0000","last_became_peered":"2024-04-25T13:51:50.699795+0000","last_unstale":"2024-04-25T13:51:51.676555+0000","last_undegraded":"2024-04-25T13:51:51.676555+0000","last_fullsized":"2024-04-25T13:51:51.676555+0000","mapping_epoch":185,"log_start":"0'0","ondisk_log_start":"0'0","created":185,"last_epoch_clean":186,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:49.653290+0000","last_clean_scrub_stamp":"2024-04-25T13:51:49.653290+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:34.494669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":14,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.675171+0000","last_change":"2024-04-25T13:51:49.702153+0000","last_active":"2024-04-25T13:51:51.675171+0000","last_peered":"2024-04-25T13:51:51.675171+0000","last_clean":"2024-04-25T13:51:51.675171+0000","last_became_active":"2024-04-25T13:51:49.701399+0000","last_became_peered":"2024-04-25T13:51:49.701399+0000","last_unstale":"2024-04-25T13:51:51.675171+0000","last_undegraded":"2024-04-25T13:51:51.675171+0000","last_fullsized":"2024-04-25T13:51:51.675171+0000","mapping_epoch":184,"log_start":"0'0","ondisk_log_start":"0'0","created":184,"last_epoch_clean":185,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:51:48.639166+0000","last_clean_scrub_stamp":"2024-04-25T13:51:48.639166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:54:39.850383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"15'72","reported_seq":435,"reported_epoch":187,"state":"active+clean","last_fresh":"2024-04-25T13:51:51.782610+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:51:51.782610+0000","last_peered":"2024-04-25T13:51:51.782610+0000","last_clean":"2024-04-25T13:51:51.782610+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:51:51.782610+0000","last_undegraded":"2024-04-25T13:51:51.782610+0000","last_fullsized":"2024-04-25T13:51:51.782610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574989,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1609,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193703,"statfs":{"total":200000000,"available":198352018,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574989,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944718,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574989,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1031,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944136,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574989,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1032,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194280,"statfs":{"total":200000000,"available":198943058,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574989,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574989,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944650,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574989,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1030,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194281,"statfs":{"total":200000000,"available":198944706,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574989,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1607,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193705,"statfs":{"total":200000000,"available":198354372,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:51:55.201 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T13:51:55.201 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T13:51:55.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.399+0000 7f4b84940640 1 Processor -- start 2024-04-25T13:51:55.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.399+0000 7f4b84940640 1 -- start start 2024-04-25T13:51:55.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4b80161240 0x7f4b800723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f4b80161650 0x7f4b800728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b8006fcf0 con 0x7f4b8006f5a0 2024-04-25T13:51:55.401 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4b80073f10 con 0x7f4b80161240 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f4b800b0c60 con 0x7f4b80161650 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7dd9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4b80161240 0x7f4b800723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7dd9b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4b80161240 0x7f4b800723a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58564/0 (socket says 172.21.15.80:58564) 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7dd9b640 1 -- 172.21.15.80:0/2415314424 learned_addr learned my addr 172.21.15.80:0/2415314424 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7ed9d640 1 --2- 172.21.15.80:0/2415314424 >> v2:172.21.15.87:3300/0 conn(0x7f4b80161650 0x7f4b800728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7dd9b640 1 -- 172.21.15.80:0/2415314424 >> v2:172.21.15.87:3300/0 conn(0x7f4b80161650 msgr2=0x7f4b800728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7dd9b640 1 --2- 172.21.15.80:0/2415314424 >> v2:172.21.15.87:3300/0 conn(0x7f4b80161650 0x7f4b800728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7dd9b640 1 -- 172.21.15.80:0/2415314424 >> v1:172.21.15.87:6789/0 conn(0x7f4b8006f5a0 legacy=0x7f4b80160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7dd9b640 1 -- 172.21.15.80:0/2415314424 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b800b0f40 con 0x7f4b80161240 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7ed9d640 1 --2- 172.21.15.80:0/2415314424 >> v2:172.21.15.87:3300/0 conn(0x7f4b80161650 0x7f4b800728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7dd9b640 1 --2- 172.21.15.80:0/2415314424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161240 0x7f4b800723a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f4b80070270 tx=0x7f4b7401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb4dd55ad90e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7d59a640 1 -- 172.21.15.80:0/2415314424 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740b0070 con 0x7f4b80161240 2024-04-25T13:51:55.402 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7d59a640 1 -- 172.21.15.80:0/2415314424 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b740ab600 con 0x7f4b80161240 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b7d59a640 1 -- 172.21.15.80:0/2415314424 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b740ab9d0 con 0x7f4b80161240 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 -- 172.21.15.80:0/2415314424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161240 msgr2=0x7f4b800723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 --2- 172.21.15.80:0/2415314424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161240 0x7f4b800723a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f4b80070270 tx=0x7f4b7401c450 comp rx=0 tx=0).stop 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 -- 172.21.15.80:0/2415314424 shutdown_connections 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 --2- 172.21.15.80:0/2415314424 >> v2:172.21.15.87:3300/0 conn(0x7f4b80161650 0x7f4b800728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 --2- 172.21.15.80:0/2415314424 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161240 0x7f4b800723a0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.403+0000 7f4b84940640 1 -- 172.21.15.80:0/2415314424 >> 172.21.15.80:0/2415314424 conn(0x7f4b8006d080 msgr2=0x7f4b800b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 -- 172.21.15.80:0/2415314424 shutdown_connections 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 -- 172.21.15.80:0/2415314424 wait complete. 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 Processor -- start 2024-04-25T13:51:55.403 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 -- start start 2024-04-25T13:51:55.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b8006f5a0 0x7f4b80147330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b80161240 0x7f4b800b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161650 0x7f4b8015b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8016a260 con 0x7f4b80161650 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8016a0e0 con 0x7f4b80161240 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8016a3e0 con 0x7f4b8006f5a0 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161650 0x7f4b8015b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161650 0x7f4b8015b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58570/0 (socket says 172.21.15.80:58570) 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 -- 172.21.15.80:0/32639964 learned_addr learned my addr 172.21.15.80:0/32639964 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7e59c640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b8006f5a0 0x7f4b80147330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7dd9b640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b80161240 0x7f4b800b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 -- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b8006f5a0 msgr2=0x7f4b80147330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.406 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b8006f5a0 0x7f4b80147330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 -- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b80161240 msgr2=0x7f4b800b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b80161240 0x7f4b800b4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 -- 172.21.15.80:0/32639964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b80155a10 con 0x7f4b80161650 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b7ed9d640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161650 0x7f4b8015b4a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c5f0 tx=0x7f4b7000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b6effd640 1 -- 172.21.15.80:0/32639964 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70015070 con 0x7f4b80161650 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b6effd640 1 -- 172.21.15.80:0/32639964 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b70004220 con 0x7f4b80161650 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b800b57b0 con 0x7f4b80161650 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b6effd640 1 -- 172.21.15.80:0/32639964 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b70003620 con 0x7f4b80161650 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.407+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b800ab5d0 con 0x7f4b80161650 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.411+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b800b5520 con 0x7f4b80161650 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.411+0000 7f4b6effd640 1 -- 172.21.15.80:0/32639964 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4b700037c0 con 0x7f4b80161650 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.411+0000 7f4b6effd640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b6006f430 0x7f4b60071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.411+0000 7f4b6effd640 1 -- 172.21.15.80:0/32639964 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f4b7008efd0 con 0x7f4b80161650 2024-04-25T13:51:55.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.411+0000 7f4b7e59c640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b6006f430 0x7f4b60071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.411+0000 7f4b7e59c640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b6006f430 0x7f4b60071940 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4b800b6e70 tx=0x7f4b78007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:55.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.415+0000 7f4b6effd640 1 -- 172.21.15.80:0/32639964 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4b7005bf60 con 0x7f4b80161650 2024-04-25T13:51:55.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.639+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f4b8006a3a0 con 0x7f4b80161650 2024-04-25T13:51:55.639 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.643+0000 7f4b6effd640 1 -- 172.21.15.80:0/32639964 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f4b70062130 con 0x7f4b80161650 2024-04-25T13:51:55.639 INFO:teuthology.orchestra.run.smithi080.stdout:[client.0] 2024-04-25T13:51:55.639 INFO:teuthology.orchestra.run.smithi080.stdout: key = AQAwXSpmHqOpNBAAST1ykpzMag+cKhNYTvk6rw== 2024-04-25T13:51:55.639 INFO:teuthology.orchestra.run.smithi080.stdout: caps mds = "allow" 2024-04-25T13:51:55.639 INFO:teuthology.orchestra.run.smithi080.stdout: caps mon = "allow r" 2024-04-25T13:51:55.639 INFO:teuthology.orchestra.run.smithi080.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T13:51:55.640 INFO:teuthology.orchestra.run.smithi080.stderr:updated caps for client.0 2024-04-25T13:51:55.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b6006f430 msgr2=0x7f4b60071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b6006f430 0x7f4b60071940 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4b800b6e70 tx=0x7f4b78007240 comp rx=0 tx=0).stop 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161650 msgr2=0x7f4b8015b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161650 0x7f4b8015b4a0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000c5f0 tx=0x7f4b7000cab0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 shutdown_connections 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4b6006f430 0x7f4b60071940 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4b80161650 0x7f4b8015b4a0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4b80161240 0x7f4b800b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 --2- 172.21.15.80:0/32639964 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4b8006f5a0 0x7f4b80147330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 >> 172.21.15.80:0/32639964 conn(0x7f4b8006d080 msgr2=0x7f4b800b15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 shutdown_connections 2024-04-25T13:51:55.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.647+0000 7f4b84940640 1 -- 172.21.15.80:0/32639964 wait complete. 2024-04-25T13:51:55.661 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:51:55.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd049518640 1 Processor -- start 2024-04-25T13:51:55.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd049518640 1 -- start start 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd049518640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0440731e0 0x7fd044163d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd049518640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd0440b5f20 0x7fd044160e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd049518640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd0441646e0 con 0x7fd0441642d0 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd049518640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd04415acb0 con 0x7fd0440731e0 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd049518640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd04406ea30 con 0x7fd0440b5f20 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd042ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0440731e0 0x7fd044163d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd042ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0440731e0 0x7fd044163d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58576/0 (socket says 172.21.15.80:58576) 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd042ffd640 1 -- 172.21.15.80:0/3771116224 learned_addr learned my addr 172.21.15.80:0/3771116224 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:55.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd042ffd640 1 -- 172.21.15.80:0/3771116224 >> v2:172.21.15.87:3300/0 conn(0x7fd0440b5f20 msgr2=0x7fd044160e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:55.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd0437fe640 1 --2- 172.21.15.80:0/3771116224 >> v2:172.21.15.87:3300/0 conn(0x7fd0440b5f20 0x7fd044160e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd042ffd640 1 --2- 172.21.15.80:0/3771116224 >> v2:172.21.15.87:3300/0 conn(0x7fd0440b5f20 0x7fd044160e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd042ffd640 1 -- 172.21.15.80:0/3771116224 >> v1:172.21.15.87:6789/0 conn(0x7fd0441642d0 legacy=0x7fd044160720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:55.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.811+0000 7fd042ffd640 1 -- 172.21.15.80:0/3771116224 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04406ed10 con 0x7fd0440731e0 2024-04-25T13:51:55.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd042ffd640 1 --2- 172.21.15.80:0/3771116224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440731e0 0x7fd044163d90 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fd0441618b0 tx=0x7fd03401c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbed0c1f9fbdb5cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:55.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd041ffb640 1 -- 172.21.15.80:0/3771116224 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0340a55a0 con 0x7fd0440731e0 2024-04-25T13:51:55.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd041ffb640 1 -- 172.21.15.80:0/3771116224 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0340a5740 con 0x7fd0440731e0 2024-04-25T13:51:55.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd041ffb640 1 -- 172.21.15.80:0/3771116224 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0340a5b10 con 0x7fd0440731e0 2024-04-25T13:51:55.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd049518640 1 -- 172.21.15.80:0/3771116224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440731e0 msgr2=0x7fd044163d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd049518640 1 --2- 172.21.15.80:0/3771116224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440731e0 0x7fd044163d90 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fd0441618b0 tx=0x7fd03401c7d0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd049518640 1 -- 172.21.15.80:0/3771116224 shutdown_connections 2024-04-25T13:51:55.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd049518640 1 --2- 172.21.15.80:0/3771116224 >> v2:172.21.15.87:3300/0 conn(0x7fd0440b5f20 0x7fd044160e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd049518640 1 --2- 172.21.15.80:0/3771116224 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440731e0 0x7fd044163d90 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd049518640 1 -- 172.21.15.80:0/3771116224 >> 172.21.15.80:0/3771116224 conn(0x7fd04406d080 msgr2=0x7fd0440b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:55.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd049518640 1 -- 172.21.15.80:0/3771116224 shutdown_connections 2024-04-25T13:51:55.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.815+0000 7fd049518640 1 -- 172.21.15.80:0/3771116224 wait complete. 2024-04-25T13:51:55.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 Processor -- start 2024-04-25T13:51:55.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 -- start start 2024-04-25T13:51:55.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440b5f20 0x7fd044146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0441642d0 0x7fd044159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.814 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0441586a0 0x7fd044158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd04406e7d0 con 0x7fd0440b5f20 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd04406e650 con 0x7fd0441642d0 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd04416d940 con 0x7fd0441586a0 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440b5f20 0x7fd044146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440b5f20 0x7fd044146f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58588/0 (socket says 172.21.15.80:58588) 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 -- 172.21.15.80:0/3109590280 learned_addr learned my addr 172.21.15.80:0/3109590280 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 -- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0441586a0 msgr2=0x7fd044158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0441586a0 0x7fd044158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd0427fc640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0441642d0 0x7fd044159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 -- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0441642d0 msgr2=0x7fd044159470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0441642d0 0x7fd044159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 -- 172.21.15.80:0/3109590280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd044158ff0 con 0x7fd0440b5f20 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd042ffd640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440b5f20 0x7fd044146f00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fd03401ae80 tx=0x7fd034019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:55.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd03b7fe640 1 -- 172.21.15.80:0/3109590280 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0340b8670 con 0x7fd0440b5f20 2024-04-25T13:51:55.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd03b7fe640 1 -- 172.21.15.80:0/3109590280 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0340be070 con 0x7fd0440b5f20 2024-04-25T13:51:55.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd03b7fe640 1 -- 172.21.15.80:0/3109590280 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0340b8a70 con 0x7fd0440b5f20 2024-04-25T13:51:55.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd044162960 con 0x7fd0440b5f20 2024-04-25T13:51:55.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd044162f20 con 0x7fd0440b5f20 2024-04-25T13:51:55.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.819+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04406a3a0 con 0x7fd0440b5f20 2024-04-25T13:51:55.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.823+0000 7fd03b7fe640 1 -- 172.21.15.80:0/3109590280 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd0340b8c10 con 0x7fd0440b5f20 2024-04-25T13:51:55.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.823+0000 7fd03b7fe640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd02406f400 0x7fd024071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:55.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.823+0000 7fd03b7fe640 1 -- 172.21.15.80:0/3109590280 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7fd03412a1c0 con 0x7fd0440b5f20 2024-04-25T13:51:55.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.823+0000 7fd0427fc640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd02406f400 0x7fd024071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:55.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.823+0000 7fd0427fc640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd02406f400 0x7fd024071910 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd04415b1c0 tx=0x7fd02c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:55.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:55.827+0000 7fd03b7fe640 1 -- 172.21.15.80:0/3109590280 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd044158ff0 con 0x7fd0440b5f20 2024-04-25T13:51:55.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.003+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd04406e430 con 0x7fd0440b5f20 2024-04-25T13:51:56.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.007+0000 7fd03b7fe640 1 -- 172.21.15.80:0/3109590280 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7fd0441646e0 con 0x7fd0440b5f20 2024-04-25T13:51:56.001 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:51:56.001 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:51:56.001 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:51:56.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd02406f400 msgr2=0x7fd024071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd02406f400 0x7fd024071910 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd04415b1c0 tx=0x7fd02c008040 comp rx=0 tx=0).stop 2024-04-25T13:51:56.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440b5f20 msgr2=0x7fd044146f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440b5f20 0x7fd044146f00 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fd03401ae80 tx=0x7fd034019570 comp rx=0 tx=0).stop 2024-04-25T13:51:56.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 shutdown_connections 2024-04-25T13:51:56.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd02406f400 0x7fd024071910 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0441586a0 0x7fd044158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0441642d0 0x7fd044159470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 --2- 172.21.15.80:0/3109590280 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0440b5f20 0x7fd044146f00 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 >> 172.21.15.80:0/3109590280 conn(0x7fd04406d080 msgr2=0x7fd0440d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 shutdown_connections 2024-04-25T13:51:56.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.011+0000 7fd049518640 1 -- 172.21.15.80:0/3109590280 wait complete. 2024-04-25T13:51:56.426 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T13:51:50.067203+0000', 'modified': '2024-04-25T13:51:53.127795+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9213}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9213': {'gid': 9213, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.80:6835/3609922017', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.80:6834', 'nonce': 3609922017}, {'type': 'v1', 'addr': '172.21.15.80:6835', 'nonce': 3609922017}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9213, 'qdb_cluster': [9213]} 2024-04-25T13:51:56.426 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T13:51:56.426 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T13:51:56.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f389b946640 1 Processor -- start 2024-04-25T13:51:56.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f389b946640 1 -- start start 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f389b946640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3894144680 0x7f389415f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f389b946640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38940d0270 0x7f3894070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f389b946640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3894068e90 con 0x7f3894068590 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f389b946640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3894069170 con 0x7f38940d0270 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f389b946640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f389415e9c0 con 0x7f3894144680 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f3899ee3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38940d0270 0x7f3894070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f3899ee3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f38940d0270 0x7f3894070490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58598/0 (socket says 172.21.15.80:58598) 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f3899ee3640 1 -- 172.21.15.80:0/4154939622 learned_addr learned my addr 172.21.15.80:0/4154939622 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f38996e2640 1 --2- 172.21.15.80:0/4154939622 >> v2:172.21.15.87:3300/0 conn(0x7f3894144680 0x7f389415f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f3899ee3640 1 -- 172.21.15.80:0/4154939622 >> v2:172.21.15.87:3300/0 conn(0x7f3894144680 msgr2=0x7f389415f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f3899ee3640 1 --2- 172.21.15.80:0/4154939622 >> v2:172.21.15.87:3300/0 conn(0x7f3894144680 0x7f389415f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f3899ee3640 1 -- 172.21.15.80:0/4154939622 >> v1:172.21.15.87:6789/0 conn(0x7f3894068590 legacy=0x7f38940cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.595+0000 7f3899ee3640 1 -- 172.21.15.80:0/4154939622 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389415eca0 con 0x7f38940d0270 2024-04-25T13:51:56.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f3899ee3640 1 --2- 172.21.15.80:0/4154939622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940d0270 0x7f3894070490 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f3894072740 tx=0x7f388c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2dd64a31e81d87d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f3893fff640 1 -- 172.21.15.80:0/4154939622 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0b0070 con 0x7f38940d0270 2024-04-25T13:51:56.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f3893fff640 1 -- 172.21.15.80:0/4154939622 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f388c0ab600 con 0x7f38940d0270 2024-04-25T13:51:56.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f3893fff640 1 -- 172.21.15.80:0/4154939622 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0ab9d0 con 0x7f38940d0270 2024-04-25T13:51:56.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f389b946640 1 -- 172.21.15.80:0/4154939622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940d0270 msgr2=0x7f3894070490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f389b946640 1 --2- 172.21.15.80:0/4154939622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940d0270 0x7f3894070490 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f3894072740 tx=0x7f388c01c450 comp rx=0 tx=0).stop 2024-04-25T13:51:56.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f389b946640 1 -- 172.21.15.80:0/4154939622 shutdown_connections 2024-04-25T13:51:56.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f389b946640 1 --2- 172.21.15.80:0/4154939622 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f38940d0270 0x7f3894070490 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f389b946640 1 --2- 172.21.15.80:0/4154939622 >> v2:172.21.15.87:3300/0 conn(0x7f3894144680 0x7f389415f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f389b946640 1 -- 172.21.15.80:0/4154939622 >> 172.21.15.80:0/4154939622 conn(0x7f38940d33a0 msgr2=0x7f38940cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f389b946640 1 -- 172.21.15.80:0/4154939622 shutdown_connections 2024-04-25T13:51:56.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.599+0000 7f389b946640 1 -- 172.21.15.80:0/4154939622 wait complete. 2024-04-25T13:51:56.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 Processor -- start 2024-04-25T13:51:56.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 -- start start 2024-04-25T13:51:56.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3894068590 0x7f38940b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3894144680 0x7f389415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3894159ec0 0x7f38940add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.599 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f389416b180 con 0x7f3894068590 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f389416b000 con 0x7f3894144680 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f389416b300 con 0x7f3894159ec0 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3894068590 0x7f38940b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3894068590 0x7f38940b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58614/0 (socket says 172.21.15.80:58614) 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 -- 172.21.15.80:0/3730988688 learned_addr learned my addr 172.21.15.80:0/3730988688 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f3898ee1640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3894144680 0x7f389415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f3899ee3640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3894159ec0 0x7f38940add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 -- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3894159ec0 msgr2=0x7f38940add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3894159ec0 0x7f38940add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 -- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3894144680 msgr2=0x7f389415ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3894144680 0x7f389415ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 -- 172.21.15.80:0/3730988688 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38940b1020 con 0x7f3894068590 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f3898ee1640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3894144680 0x7f389415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f38996e2640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3894068590 0x7f38940b20c0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f3884007e60 tx=0x7f388400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.600 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f3891ffb640 1 -- 172.21.15.80:0/3730988688 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388401a070 con 0x7f3894068590 2024-04-25T13:51:56.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f3891ffb640 1 -- 172.21.15.80:0/3730988688 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3884003ba0 con 0x7f3894068590 2024-04-25T13:51:56.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 -- 172.21.15.80:0/3730988688 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38940b1300 con 0x7f3894068590 2024-04-25T13:51:56.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f3891ffb640 1 -- 172.21.15.80:0/3730988688 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884004030 con 0x7f3894068590 2024-04-25T13:51:56.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.603+0000 7f389b946640 1 -- 172.21.15.80:0/3730988688 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38940b18c0 con 0x7f3894068590 2024-04-25T13:51:56.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.607+0000 7f3891ffb640 1 -- 172.21.15.80:0/3730988688 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f388401d070 con 0x7f3894068590 2024-04-25T13:51:56.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.607+0000 7f3891ffb640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f388006f400 0x7f3880071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.601 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.607+0000 7f3891ffb640 1 -- 172.21.15.80:0/3730988688 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f388408f3e0 con 0x7f3894068590 2024-04-25T13:51:56.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.607+0000 7f3898ee1640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f388006f400 0x7f3880071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.602 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.607+0000 7f38a2ca3000 1 Processor -- start 2024-04-25T13:51:56.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.607+0000 7f38a2ca3000 1 -- start start 2024-04-25T13:51:56.603 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.607+0000 7f3898ee1640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f388006f400 0x7f3880071910 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f38940d00b0 tx=0x7f3888008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d0c8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 0x55e52d0c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e52cf37160 con 0x55e52cf69c20 2024-04-25T13:51:56.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e52cecce20 con 0x55e52cf55ea0 2024-04-25T13:51:56.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55e52ce85a20 con 0x55e52cb4e280 2024-04-25T13:51:56.606 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58630/0 (socket says 172.21.15.80:58630) 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 -- 172.21.15.80:0/172409556 learned_addr learned my addr 172.21.15.80:0/172409556 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f3898ee1640 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d0c8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 -- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 msgr2=0x55e52d0c8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f3899ee3640 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 0x55e52d0c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 0x55e52d0c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 -- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 msgr2=0x55e52d0c8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d0c8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 -- 172.21.15.80:0/172409556 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52ce8a100 con 0x55e52cf69c20 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f3898ee1640 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38996e2640 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0c7ee0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f3884057d90 tx=0x7f38840583e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a5a42658e12e796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f387affd640 1 -- 172.21.15.80:0/172409556 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884075020 con 0x55e52cf69c20 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f387affd640 1 -- 172.21.15.80:0/172409556 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3884060010 con 0x55e52cf69c20 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f387affd640 1 -- 172.21.15.80:0/172409556 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38840751c0 con 0x55e52cf69c20 2024-04-25T13:51:56.607 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- 172.21.15.80:0/172409556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 msgr2=0x55e52d0c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0c7ee0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f3884057d90 tx=0x7f38840583e0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- 172.21.15.80:0/172409556 shutdown_connections 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 0x55e52d0c8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d0c8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/172409556 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0c7ee0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- 172.21.15.80:0/172409556 >> 172.21.15.80:0/172409556 conn(0x55e52cb8dff0 msgr2=0x55e52d07d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- 172.21.15.80:0/172409556 shutdown_connections 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- 172.21.15.80:0/172409556 wait complete. 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 -- start start 2024-04-25T13:51:56.608 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.611+0000 7f38a2ca3000 1 Processor -- start 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf55ea0 0x55e52d0cf3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf69c20 0x55e52d08a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e52ceda220 con 0x55e52cb4e280 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e52ce70ea0 con 0x55e52cf69c20 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55e52cec7b00 con 0x55e52cf55ea0 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 10 client.0 _ll_register_callbacks cb 0x55e52ced9dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58638/0 (socket says 172.21.15.80:58638) 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 -- 172.21.15.80:0/2862286494 learned_addr learned my addr 172.21.15.80:0/2862286494 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 -- 172.21.15.80:0/2862286494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf55ea0 msgr2=0x55e52d0cf3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3898ee1640 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf55ea0 0x55e52d0cf3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3899ee3640 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf69c20 0x55e52d08a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf55ea0 0x55e52d0cf3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 -- 172.21.15.80:0/2862286494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf69c20 msgr2=0x55e52d08a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf69c20 0x55e52d08a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 -- 172.21.15.80:0/2862286494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52ce6c500 con 0x55e52cb4e280 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38996e2640 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cee60 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f3884058f80 tx=0x7f388408f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bfb1ed2d75ea969 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 1 -- 172.21.15.80:0/2862286494 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388407b060 con 0x55e52cb4e280 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 1 -- 172.21.15.80:0/2862286494 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f38840647a0 con 0x55e52cb4e280 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2862286494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3884060010 con 0x55e52cb4e280 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 1 -- 172.21.15.80:0/2862286494 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f3884081050 con 0x55e52cb4e280 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 10 client.9351 fetch_fsmap learned FSMap version 89 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2862286494 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55e52ce52ea0 con 0x55e52cb4e280 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 1 -- 172.21.15.80:0/2862286494 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3884060010 con 0x55e52cb4e280 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 1 -- 172.21.15.80:0/2862286494 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f388405af70 con 0x55e52cb4e280 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 10 client.9351 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 10 client.9351 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:51:56.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 20 client.9351 populate_metadata read hostname 'smithi080' 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] conn(0x55e52cb727d0 0x55e52d08ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 4 client.9351 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2862286494 --> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e52ce65be0 con 0x55e52cb727d0 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3898ee1640 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] conn(0x55e52cb727d0 0x55e52d08ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3898ee1640 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] conn(0x55e52cb727d0 0x55e52d08ae30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580eac48f6688906 server_cookie=d15ac0bb74df0fef in_seq=0 out_seq=0 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 10 client.9351 ms_handle_connect on v2:172.21.15.80:6834/3609922017 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 1 -- 172.21.15.80:0/2862286494 <== mds.0 v2:172.21.15.80:6834/3609922017 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e52ce65be0 con 0x55e52cb727d0 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 10 client.9351 handle_command_reply: tid=0 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 client.9351 shutdown 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2862286494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 msgr2=0x55e52d0cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cee60 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f3884058f80 tx=0x7f388408f070 comp rx=0 tx=0).stop 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2862286494 shutdown_connections 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] conn(0x55e52cb727d0 0x55e52d08ae30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf69c20 0x55e52d08a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f3873fff640 0 client.9351 ms_handle_reset on v2:172.21.15.80:6834/3609922017 2024-04-25T13:51:56.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf55ea0 0x55e52d0cf3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2862286494 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cee60 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2862286494 >> 172.21.15.80:0/2862286494 conn(0x55e52cb8dff0 msgr2=0x55e52d07dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2862286494 shutdown_connections 2024-04-25T13:51:56.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2862286494 wait complete. 2024-04-25T13:51:56.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.615+0000 7f38a2ca3000 20 client.9351 trim_cache size 0 max 16384 2024-04-25T13:51:56.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38a2ca3000 1 Processor -- start 2024-04-25T13:51:56.629 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38a2ca3000 1 -- start start 2024-04-25T13:51:56.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d0cd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.631 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf69c20 0x55e52d0cdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3894203290 con 0x55e52cb4e280 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f38941406e0 con 0x55e52cf55ea0 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f38941fb910 con 0x55e52cf69c20 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58650/0 (socket says 172.21.15.80:58650) 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 -- 172.21.15.80:0/2152103736 learned_addr learned my addr 172.21.15.80:0/2152103736 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 -- 172.21.15.80:0/2152103736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf69c20 msgr2=0x55e52d0cdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 --2- 172.21.15.80:0/2152103736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf69c20 0x55e52d0cdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 -- 172.21.15.80:0/2152103736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 msgr2=0x55e52d0cd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 --2- 172.21.15.80:0/2152103736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d0cd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 -- 172.21.15.80:0/2152103736 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e52d081840 con 0x55e52cb4e280 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f38996e2640 1 --2- 172.21.15.80:0/2152103736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cd040 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f3884058300 tx=0x7f3884089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62987836859850a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f387a7fc640 1 -- 172.21.15.80:0/2152103736 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884075040 con 0x55e52cb4e280 2024-04-25T13:51:56.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f387a7fc640 1 -- 172.21.15.80:0/2152103736 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3884060010 con 0x55e52cb4e280 2024-04-25T13:51:56.633 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.635+0000 7f387a7fc640 1 -- 172.21.15.80:0/2152103736 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884058360 con 0x55e52cb4e280 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2152103736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 msgr2=0x55e52d0cd040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2152103736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cd040 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f3884058300 tx=0x7f3884089040 comp rx=0 tx=0).stop 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2152103736 shutdown_connections 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2152103736 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cf69c20 0x55e52d0cdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2152103736 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d0cd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/2152103736 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cb4e280 0x55e52d0cd040 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2152103736 >> 172.21.15.80:0/2152103736 conn(0x55e52cb8dff0 msgr2=0x55e52d085000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2152103736 shutdown_connections 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- 172.21.15.80:0/2152103736 wait complete. 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- start start 2024-04-25T13:51:56.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 Processor -- start 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 0x55e52d13ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d13e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0f9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x55e52cecc130 con 0x55e52cf69c20 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x55e52ce627d0 con 0x55e52cf55ea0 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x55e52ced1450 con 0x55e52cb4e280 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 10 client.0 _ll_register_callbacks cb 0x7f388405b6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0f9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0f9970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58662/0 (socket says 172.21.15.80:58662) 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 -- 172.21.15.80:0/700517938 learned_addr learned my addr 172.21.15.80:0/700517938 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38996e2640 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 0x55e52d13ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.640 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3898ee1640 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d13e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 -- 172.21.15.80:0/700517938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 msgr2=0x55e52d13ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 0x55e52d13ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 -- 172.21.15.80:0/700517938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 msgr2=0x55e52d13e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d13e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 -- 172.21.15.80:0/700517938 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e52d08a430 con 0x55e52cf69c20 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3899ee3640 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0f9970 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f388c0ae040 tx=0x7f388c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4890b62507a64e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3873fff640 10 client.0 ms_handle_connect on v2:172.21.15.80:3300/0 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3873fff640 1 -- 172.21.15.80:0/700517938 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0b0040 con 0x55e52cf69c20 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3873fff640 1 -- 172.21.15.80:0/700517938 <== mon.0 v2:172.21.15.80:3300/0 2 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f388c0abbb0 con 0x55e52cf69c20 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- 172.21.15.80:0/700517938 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3884060010 con 0x55e52cf69c20 2024-04-25T13:51:56.641 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3873fff640 1 -- 172.21.15.80:0/700517938 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f388c0c3050 con 0x55e52cf69c20 2024-04-25T13:51:56.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 10 client.9357 fetch_fsmap learned FSMap version 89 2024-04-25T13:51:56.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f38a2ca3000 1 -- 172.21.15.80:0/700517938 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55e52ce85c70 con 0x55e52cf69c20 2024-04-25T13:51:56.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3873fff640 1 -- 172.21.15.80:0/700517938 <== mon.0 v2:172.21.15.80:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3884060010 con 0x55e52cf69c20 2024-04-25T13:51:56.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.639+0000 7f3873fff640 1 -- 172.21.15.80:0/700517938 <== mon.0 v2:172.21.15.80:3300/0 5 ==== fsmap(e 89) ==== 3494+0+0 (secure 0 0 0) 0x7f388c0c6020 con 0x55e52cf69c20 2024-04-25T13:51:56.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 10 client.9357 fetch_fsmap finished waiting for FSMap version 89 2024-04-25T13:51:56.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 10 client.9357 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T13:51:56.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 20 client.9357 populate_metadata read hostname 'smithi080' 2024-04-25T13:51:56.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] conn(0x55e52cb60830 0x55e52d0fa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 4 client.9357 mds_command: new command op to 9213 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 -- 172.21.15.80:0/700517938 --> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] -- command(tid 0: {"prefix": "status"}) -- 0x55e52cedaa30 con 0x55e52cb60830 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38996e2640 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] conn(0x55e52cb60830 0x55e52d0fa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f3873fff640 10 client.9357 ms_handle_connect on v2:172.21.15.80:6834/3609922017 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38996e2640 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] conn(0x55e52cb60830 0x55e52d0fa5b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eace1781559dd2ec server_cookie=c73ab42c212df591 in_seq=0 out_seq=0 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f3873fff640 1 -- 172.21.15.80:0/700517938 <== mds.0 v2:172.21.15.80:6834/3609922017 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55e52cedaa30 con 0x55e52cb60830 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f3873fff640 10 client.9357 handle_command_reply: tid=0 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 client.9357 shutdown 2024-04-25T13:51:56.643 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 -- 172.21.15.80:0/700517938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 msgr2=0x55e52d0f9970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0f9970 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f388c0ae040 tx=0x7f388c01ac90 comp rx=0 tx=0).stop 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 -- 172.21.15.80:0/700517938 shutdown_connections 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.80:6834/3609922017,v1:172.21.15.80:6835/3609922017] conn(0x55e52cb60830 0x55e52d0fa5b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x55e52cf69c20 0x55e52d0f9970 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55e52cf55ea0 0x55e52d13e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f3873fff640 0 client.9357 ms_handle_reset on v2:172.21.15.80:6834/3609922017 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 --2- 172.21.15.80:0/700517938 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x55e52cb4e280 0x55e52d13ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 -- 172.21.15.80:0/700517938 >> 172.21.15.80:0/700517938 conn(0x55e52cb8dff0 msgr2=0x55e52d0d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 -- 172.21.15.80:0/700517938 shutdown_connections 2024-04-25T13:51:56.644 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 1 -- 172.21.15.80:0/700517938 wait complete. 2024-04-25T13:51:56.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f38a2ca3000 20 client.9357 trim_cache size 0 max 16384 2024-04-25T13:51:56.645 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:51:56.645 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:51:56.645 INFO:teuthology.orchestra.run.smithi080.stdout: "whoami": 0, 2024-04-25T13:51:56.645 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 9213, 2024-04-25T13:51:56.645 INFO:teuthology.orchestra.run.smithi080.stdout: "want_state": "up:active", 2024-04-25T13:51:56.645 INFO:teuthology.orchestra.run.smithi080.stdout: "state": "up:active", 2024-04-25T13:51:56.645 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs", 2024-04-25T13:51:56.646 INFO:teuthology.orchestra.run.smithi080.stdout: "rank_uptime": 5.934649029, 2024-04-25T13:51:56.646 INFO:teuthology.orchestra.run.smithi080.stdout: "mdsmap_epoch": 89, 2024-04-25T13:51:56.646 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch": 187, 2024-04-25T13:51:56.646 INFO:teuthology.orchestra.run.smithi080.stdout: "osdmap_epoch_barrier": 187, 2024-04-25T13:51:56.646 INFO:teuthology.orchestra.run.smithi080.stdout: "uptime": 10.909443467999999 2024-04-25T13:51:56.646 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 -- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f388006f400 msgr2=0x7f3880071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f388006f400 0x7f3880071910 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f38940d00b0 tx=0x7f3888008040 comp rx=0 tx=0).stop 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 -- 172.21.15.80:0/3730988688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3894068590 msgr2=0x7f38940b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3894068590 0x7f38940b20c0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f3884007e60 tx=0x7f388400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 -- 172.21.15.80:0/3730988688 shutdown_connections 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f388006f400 0x7f3880071910 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3894159ec0 0x7f38940add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3894144680 0x7f389415ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 --2- 172.21.15.80:0/3730988688 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3894068590 0x7f38940b20c0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 -- 172.21.15.80:0/3730988688 >> 172.21.15.80:0/3730988688 conn(0x7f38940d33a0 msgr2=0x7f38940b3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 -- 172.21.15.80:0/3730988688 shutdown_connections 2024-04-25T13:51:56.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.643+0000 7f389b946640 1 -- 172.21.15.80:0/3730988688 wait complete. 2024-04-25T13:51:56.657 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:51:56.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb06be88640 1 Processor -- start 2024-04-25T13:51:56.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb06be88640 1 -- start start 2024-04-25T13:51:56.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb06be88640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0640d59b0 0x7fb0640d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb06be88640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb0640d6300 con 0x7fb0640d4330 2024-04-25T13:51:56.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb06be88640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb064161060 con 0x7fb0640d2e80 2024-04-25T13:51:56.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb06be88640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb064161340 con 0x7fb0640d59b0 2024-04-25T13:51:56.786 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb069423640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb0640d59b0 0x7fb0640d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb06a425640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb0640d4330 0x7fb0640d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41252/0 (socket says 172.21.15.80:41252) 2024-04-25T13:51:56.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb06a425640 1 -- 172.21.15.80:0/1384532697 learned_addr learned my addr 172.21.15.80:0/1384532697 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:51:56.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb069423640 1 -- 172.21.15.80:0/1384532697 >> v1:172.21.15.87:6789/0 conn(0x7fb0640d2e80 legacy=0x7fb0641607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:56.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb069423640 1 -- 172.21.15.80:0/1384532697 >> v1:172.21.15.80:6789/0 conn(0x7fb0640d4330 legacy=0x7fb0640d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb068c22640 1 -- 172.21.15.80:0/1384532697 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 100406390 0 0) 0x7fb060001120 con 0x7fb0640d4330 2024-04-25T13:51:56.787 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.791+0000 7fb069423640 1 -- 172.21.15.80:0/1384532697 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0640d2680 con 0x7fb0640d59b0 2024-04-25T13:51:56.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb069423640 1 --2- 172.21.15.80:0/1384532697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb0640d5dc0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fb0540097c0 tx=0x7fb05401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee309b5db99b46a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb068c22640 1 -- 172.21.15.80:0/1384532697 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540b0070 con 0x7fb0640d59b0 2024-04-25T13:51:56.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb068c22640 1 -- 172.21.15.80:0/1384532697 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb0540ab600 con 0x7fb0640d59b0 2024-04-25T13:51:56.788 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb068c22640 1 -- 172.21.15.80:0/1384532697 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0540a4e60 con 0x7fb0640d59b0 2024-04-25T13:51:56.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 -- 172.21.15.80:0/1384532697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 msgr2=0x7fb0640d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 --2- 172.21.15.80:0/1384532697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb0640d5dc0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fb0540097c0 tx=0x7fb05401c3d0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 -- 172.21.15.80:0/1384532697 shutdown_connections 2024-04-25T13:51:56.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 --2- 172.21.15.80:0/1384532697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb0640d5dc0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 -- 172.21.15.80:0/1384532697 >> 172.21.15.80:0/1384532697 conn(0x7fb0640737a0 msgr2=0x7fb06406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 -- 172.21.15.80:0/1384532697 shutdown_connections 2024-04-25T13:51:56.789 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 -- 172.21.15.80:0/1384532697 wait complete. 2024-04-25T13:51:56.790 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 Processor -- start 2024-04-25T13:51:56.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.795+0000 7fb06be88640 1 -- start start 2024-04-25T13:51:56.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0640d2e80 0x7fb0640cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.793 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0640d4330 0x7fb0641560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb064162460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb06415e0e0 con 0x7fb0640d59b0 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb06415df60 con 0x7fb0640d2e80 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb06416e950 con 0x7fb0640d4330 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb064162460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb064162460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58668/0 (socket says 172.21.15.80:58668) 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 -- 172.21.15.80:0/59633061 learned_addr learned my addr 172.21.15.80:0/59633061 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 -- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0640d4330 msgr2=0x7fb0641560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0640d4330 0x7fb0641560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 -- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0640d2e80 msgr2=0x7fb0640cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb069c24640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0640d2e80 0x7fb0640cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.794 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0640d2e80 0x7fb0640cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 -- 172.21.15.80:0/59633061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0641fdd00 con 0x7fb0640d59b0 2024-04-25T13:51:56.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06a425640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb064162460 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb06000c3e0 tx=0x7fb06000c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb05a7fc640 1 -- 172.21.15.80:0/59633061 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060017070 con 0x7fb0640d59b0 2024-04-25T13:51:56.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb05a7fc640 1 -- 172.21.15.80:0/59633061 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb060005cf0 con 0x7fb0640d59b0 2024-04-25T13:51:56.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0641fdfe0 con 0x7fb0640d59b0 2024-04-25T13:51:56.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb05a7fc640 1 -- 172.21.15.80:0/59633061 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb060007310 con 0x7fb0640d59b0 2024-04-25T13:51:56.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0640c9990 con 0x7fb0640d59b0 2024-04-25T13:51:56.795 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0641fda70 con 0x7fb0640d59b0 2024-04-25T13:51:56.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb05a7fc640 1 -- 172.21.15.80:0/59633061 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb06001a070 con 0x7fb0640d59b0 2024-04-25T13:51:56.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb05a7fc640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb04806f430 0x7fb048071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:56.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb05a7fc640 1 -- 172.21.15.80:0/59633061 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7fb06008e6a0 con 0x7fb0640d59b0 2024-04-25T13:51:56.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.799+0000 7fb069c24640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb04806f430 0x7fb048071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:56.796 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.803+0000 7fb069c24640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb04806f430 0x7fb048071940 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb0640b8fd0 tx=0x7fb05c007240 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:56.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.803+0000 7fb05a7fc640 1 -- 172.21.15.80:0/59633061 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb0640d6300 con 0x7fb0640d59b0 2024-04-25T13:51:56.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb06406a3a0 con 0x7fb0640d59b0 2024-04-25T13:51:56.989 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb05a7fc640 1 -- 172.21.15.80:0/59633061 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) ==== 53+0+73 (secure 0 0 0) 0x7fb06005b510 con 0x7fb0640d59b0 2024-04-25T13:51:56.990 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:51:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb04806f430 msgr2=0x7fb048071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb04806f430 0x7fb048071940 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb0640b8fd0 tx=0x7fb05c007240 comp rx=0 tx=0).stop 2024-04-25T13:51:56.992 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 msgr2=0x7fb064162460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb064162460 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fb06000c3e0 tx=0x7fb06000c7e0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 shutdown_connections 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb04806f430 0x7fb048071940 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb0640d59b0 0x7fb064162460 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb0640d4330 0x7fb0641560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 --2- 172.21.15.80:0/59633061 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb0640d2e80 0x7fb0640cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 >> 172.21.15.80:0/59633061 conn(0x7fb0640737a0 msgr2=0x7fb064163010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.995+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 shutdown_connections 2024-04-25T13:51:56.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:56.999+0000 7fb06be88640 1 -- 172.21.15.80:0/59633061 wait complete. 2024-04-25T13:51:57.010 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:51:57.010 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:51:57.010 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi080.front.sepia.ceph.com 2024-04-25T13:51:57.010 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T13:51:57.010 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T13:51:57.010 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:51:57.010 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:51:57.010 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:51:57.011 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:51:57.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:57.011 DEBUG:teuthology.orchestra.run.smithi080:> ip addr 2024-04-25T13:51:57.016 INFO:teuthology.orchestra.run.smithi080.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:51:57.016 INFO:teuthology.orchestra.run.smithi080.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 ::1/128 scope host 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b2 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f0 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:d8:d9:b3 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: altname enp7s0f1 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:57.017 INFO:teuthology.orchestra.run.smithi080.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0c:c4:7a:8f:cb:a9 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.21.15.80/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft 41851sec preferred_lft 41851sec 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::ec4:7aff:fe8f:cba9/64 scope link 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 02:42:29:15:50:3f brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:51:57.018 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:51:57.019 INFO:teuthology.orchestra.run.smithi080.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:57.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:57.020 DEBUG:teuthology.orchestra.run.smithi080:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:51:57.068 INFO:teuthology.orchestra.run.smithi080.stdout:1 2024-04-25T13:51:57.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:57.070 DEBUG:teuthology.orchestra.run.smithi080:> route 2024-04-25T13:51:57.119 INFO:teuthology.orchestra.run.smithi080.stdout:Kernel IP routing table 2024-04-25T13:51:57.119 INFO:teuthology.orchestra.run.smithi080.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:51:57.119 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:51:57.120 INFO:teuthology.orchestra.run.smithi080.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:51:57.120 INFO:teuthology.orchestra.run.smithi080.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:51:57.120 INFO:teuthology.orchestra.run.smithi080.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:51:57.120 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:51:57.120 INFO:teuthology.orchestra.run.smithi080.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:51:57.120 INFO:teuthology.orchestra.run.smithi080.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:51:57.120 INFO:teuthology.orchestra.run.smithi080.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:51:57.120 INFO:teuthology.orchestra.run.smithi080.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:51:57.120 DEBUG:teuthology.orchestra.run.smithi080:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:51:57.120 DEBUG:teuthology.orchestra.run.smithi080:> set -e 2024-04-25T13:51:57.120 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:51:57.121 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:51:57.121 DEBUG:teuthology.orchestra.run.smithi080:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:51:57.121 DEBUG:teuthology.orchestra.run.smithi080:> ') 2024-04-25T13:51:57.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:51:57.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:51:57.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:57.235 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:51:57.284 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:51:57.284 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:51:57.285 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:51:57.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:57.285 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link set brx.0 up 2024-04-25T13:51:57.338 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T13:51:57.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:51:57.338 DEBUG:teuthology.orchestra.run.smithi080:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T13:51:57.384 INFO:teuthology.orchestra.run.smithi080.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T13:51:57.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:51:57.384 DEBUG:teuthology.orchestra.run.smithi080:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:51:57.432 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:51:57.489 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:51:57.537 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:51:57.537 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:51:57.537 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:51:57.538 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:51:57.539 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:51:57.540 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:51:57.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:57.540 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:51:57.589 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:51:57.589 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T13:51:57.633 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:51:57.649 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:51:57.697 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:51:57.698 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:51:57.699 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:51:57.699 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:51:57.699 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:51:57.699 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:51:57.699 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:51:57.699 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:51:57.699 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:51:57.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:57.700 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:51:57.748 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:2024-04-25T13:51:57.755+0000 7f5be2f65580 -1 init, newargv = 0x7f5bc4005d30 newargc=15 2024-04-25T13:51:57.748 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[43916]: starting ceph client 2024-04-25T13:51:57.754 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[43916]: starting fuse 2024-04-25T13:51:58.849 DEBUG:teuthology.orchestra.run.smithi080:> sudo modprobe fuse 2024-04-25T13:51:58.865 DEBUG:teuthology.orchestra.run.smithi080:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:51:58.915 INFO:teuthology.orchestra.run.smithi080.stdout:/sys 2024-04-25T13:51:58.915 INFO:teuthology.orchestra.run.smithi080.stdout:/proc 2024-04-25T13:51:58.915 INFO:teuthology.orchestra.run.smithi080.stdout:/dev 2024-04-25T13:51:58.915 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/pts 2024-04-25T13:51:58.915 INFO:teuthology.orchestra.run.smithi080.stdout:/run 2024-04-25T13:51:58.915 INFO:teuthology.orchestra.run.smithi080.stdout:/ 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/security 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/shm 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/run/lock 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/cgroup 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/pstore 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/bpf 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/hugepages 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/dev/mqueue 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/debug 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/tracing 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/proc/fs/nfsd 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/fs/fuse/connections 2024-04-25T13:51:58.916 INFO:teuthology.orchestra.run.smithi080.stdout:/sys/kernel/config 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/run/rpc_pipefs 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/run/user/1000 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:51:58.917 INFO:teuthology.orchestra.run.smithi080.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T13:51:58.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:58.918 DEBUG:teuthology.orchestra.run.smithi080:> ls /sys/fs/fuse/connections 2024-04-25T13:51:58.964 INFO:teuthology.orchestra.run.smithi080.stdout:45 2024-04-25T13:51:58.964 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> import glob 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> import re 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> import os 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> import subprocess 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> def _find_admin_socket(client_name): 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> files = glob.glob(asok_path) 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> # Given a non-glob path, it better be there 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> if "*" not in asok_path: 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> assert(len(files) == 1) 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> return files[0] 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:51:58.965 DEBUG:teuthology.orchestra.run.smithi080:> for f in files: 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> contents = proc_f.read() 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> if mountpoint in contents: 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> return f 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> print(_find_admin_socket("client.0")) 2024-04-25T13:51:58.966 DEBUG:teuthology.orchestra.run.smithi080:> ' 2024-04-25T13:51:59.087 INFO:teuthology.orchestra.run.smithi080.stdout:/var/run/ceph/ceph-client.0.43916.asok 2024-04-25T13:51:59.091 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:51:59.098 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43916.asok 2024-04-25T13:51:59.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:59.099 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43916.asok status 2024-04-25T13:51:59.273 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:51:59.273 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": { 2024-04-25T13:51:59.273 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:51:59.273 INFO:teuthology.orchestra.run.smithi080.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:51:59.273 INFO:teuthology.orchestra.run.smithi080.stdout: "entity_id": "0", 2024-04-25T13:51:59.273 INFO:teuthology.orchestra.run.smithi080.stdout: "hostname": "smithi080", 2024-04-25T13:51:59.273 INFO:teuthology.orchestra.run.smithi080.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "pid": "43916", 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "root": "/" 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_count": 0, 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "dentry_pinned_count": 0, 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "id": 9369, 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "inst": { 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "name": { 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "client", 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "num": 9369 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:51:59.274 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3790325526 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": { 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3790325526 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "inst_str": "client.9369 192.168.0.1:0/3790325526", 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "addr_str": "192.168.0.1:0/3790325526", 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "inode_count": 1, 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "mds_epoch": 89, 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch": 187, 2024-04-25T13:51:59.275 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:51:59.276 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklisted": false, 2024-04-25T13:51:59.276 INFO:teuthology.orchestra.run.smithi080.stdout: "fs_name": "cephfs" 2024-04-25T13:51:59.276 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:51:59.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:59.290 DEBUG:teuthology.orchestra.run.smithi080:> stat --file-system '--printf=%T 2024-04-25T13:51:59.290 DEBUG:teuthology.orchestra.run.smithi080:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T13:51:59.296 INFO:teuthology.orchestra.run.smithi080.stdout:fuseblk 2024-04-25T13:51:59.296 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T13:51:59.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:59.296 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T13:51:59.358 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T13:51:59.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f588df9d640 1 Processor -- start 2024-04-25T13:51:59.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f588df9d640 1 -- start start 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f588df9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f588806f5a0 0x7f5888160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f588df9d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 0x7f5888161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f588df9d640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f588806fcf0 con 0x7f58880b4bd0 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f588df9d640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f58881619c0 con 0x7f588806f5a0 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f588df9d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5888073f10 con 0x7f5888161070 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f58877fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f588806f5a0 0x7f5888160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f58877fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f588806f5a0 0x7f5888160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58672/0 (socket says 172.21.15.80:58672) 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f58877fe640 1 -- 172.21.15.80:0/870943546 learned_addr learned my addr 172.21.15.80:0/870943546 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f587ffff640 1 --2- 172.21.15.80:0/870943546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 0x7f5888161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f58877fe640 1 -- 172.21.15.80:0/870943546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 msgr2=0x7f5888161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f58877fe640 1 --2- 172.21.15.80:0/870943546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 0x7f5888161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f58877fe640 1 -- 172.21.15.80:0/870943546 >> v1:172.21.15.87:6789/0 conn(0x7f58880b4bd0 legacy=0x7f58880b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:51:59.559 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.563+0000 7f58877fe640 1 -- 172.21.15.80:0/870943546 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58880b0c60 con 0x7f588806f5a0 2024-04-25T13:51:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f58877fe640 1 --2- 172.21.15.80:0/870943546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f588806f5a0 0x7f5888160b30 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f587800bb40 tx=0x7f587801eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a12cdc728ee2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f5886ffd640 1 -- 172.21.15.80:0/870943546 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780b4070 con 0x7f588806f5a0 2024-04-25T13:51:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f5886ffd640 1 -- 172.21.15.80:0/870943546 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58780a79c0 con 0x7f588806f5a0 2024-04-25T13:51:59.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f5886ffd640 1 -- 172.21.15.80:0/870943546 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780a7d90 con 0x7f588806f5a0 2024-04-25T13:51:59.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f588df9d640 1 -- 172.21.15.80:0/870943546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f588806f5a0 msgr2=0x7f5888160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f588df9d640 1 --2- 172.21.15.80:0/870943546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f588806f5a0 0x7f5888160b30 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f587800bb40 tx=0x7f587801eb40 comp rx=0 tx=0).stop 2024-04-25T13:51:59.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f588df9d640 1 -- 172.21.15.80:0/870943546 shutdown_connections 2024-04-25T13:51:59.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f588df9d640 1 --2- 172.21.15.80:0/870943546 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 0x7f5888161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f588df9d640 1 --2- 172.21.15.80:0/870943546 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f588806f5a0 0x7f5888160b30 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f588df9d640 1 -- 172.21.15.80:0/870943546 >> 172.21.15.80:0/870943546 conn(0x7f588806d080 msgr2=0x7f58880b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:59.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f588df9d640 1 -- 172.21.15.80:0/870943546 shutdown_connections 2024-04-25T13:51:59.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.567+0000 7f588df9d640 1 -- 172.21.15.80:0/870943546 wait complete. 2024-04-25T13:51:59.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 Processor -- start 2024-04-25T13:51:59.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 -- start start 2024-04-25T13:51:59.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f588806f5a0 0x7f588815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58880b4bd0 0x7f588815e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 0x7f5888159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f588806ee20 con 0x7f58880b4bd0 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f588806eca0 con 0x7f588806f5a0 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f588806efa0 con 0x7f5888161070 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58880b4bd0 0x7f588815e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58880b4bd0 0x7f588815e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:58674/0 (socket says 172.21.15.80:58674) 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 -- 172.21.15.80:0/1324366383 learned_addr learned my addr 172.21.15.80:0/1324366383 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f58877fe640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f588806f5a0 0x7f588815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f5887fff640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 0x7f5888159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 -- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 msgr2=0x7f5888159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 0x7f5888159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 -- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f588806f5a0 msgr2=0x7f588815e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f588806f5a0 0x7f588815e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 -- 172.21.15.80:0/1324366383 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58881456d0 con 0x7f58880b4bd0 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f587ffff640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58880b4bd0 0x7f588815e9e0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f587000c7f0 tx=0x7f587000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f5884ff9640 1 -- 172.21.15.80:0/1324366383 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870015070 con 0x7f58880b4bd0 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f5884ff9640 1 -- 172.21.15.80:0/1324366383 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5870007820 con 0x7f58880b4bd0 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f588815d6e0 con 0x7f58880b4bd0 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f5884ff9640 1 -- 172.21.15.80:0/1324366383 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870007c10 con 0x7f58880b4bd0 2024-04-25T13:51:59.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.571+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f588815dca0 con 0x7f58880b4bd0 2024-04-25T13:51:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.575+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f588806a3a0 con 0x7f58880b4bd0 2024-04-25T13:51:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.575+0000 7f5884ff9640 1 -- 172.21.15.80:0/1324366383 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5870018070 con 0x7f58880b4bd0 2024-04-25T13:51:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.575+0000 7f5884ff9640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f586c06f400 0x7f586c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:51:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.575+0000 7f5884ff9640 1 -- 172.21.15.80:0/1324366383 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f587008f210 con 0x7f58880b4bd0 2024-04-25T13:51:59.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.575+0000 7f58877fe640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f586c06f400 0x7f586c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:51:59.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.575+0000 7f58877fe640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f586c06f400 0x7f586c071910 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f58780b1000 tx=0x7f58780a9070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:51:59.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.579+0000 7f5884ff9640 1 -- 172.21.15.80:0/1324366383 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f587005bb30 con 0x7f58880b4bd0 2024-04-25T13:51:59.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.779+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5888160260 con 0x7f58880b4bd0 2024-04-25T13:51:59.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.779+0000 7f5884ff9640 1 -- 172.21.15.80:0/1324366383 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) ==== 53+0+73 (secure 0 0 0) 0x7f587005fe30 con 0x7f58880b4bd0 2024-04-25T13:51:59.775 INFO:teuthology.orchestra.run.smithi080.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T13:51:59.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f586c06f400 msgr2=0x7f586c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f586c06f400 0x7f586c071910 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f58780b1000 tx=0x7f58780a9070 comp rx=0 tx=0).stop 2024-04-25T13:51:59.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58880b4bd0 msgr2=0x7f588815e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:51:59.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58880b4bd0 0x7f588815e9e0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f587000c7f0 tx=0x7f587000ccb0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 shutdown_connections 2024-04-25T13:51:59.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f586c06f400 0x7f586c071910 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5888161070 0x7f5888159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f58880b4bd0 0x7f588815e9e0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 --2- 172.21.15.80:0/1324366383 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f588806f5a0 0x7f588815e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:51:59.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 >> 172.21.15.80:0/1324366383 conn(0x7f588806d080 msgr2=0x7f58880b1530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:51:59.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 shutdown_connections 2024-04-25T13:51:59.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:51:59.783+0000 7f588df9d640 1 -- 172.21.15.80:0/1324366383 wait complete. 2024-04-25T13:51:59.797 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T13:51:59.797 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T13:51:59.797 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-04-25T13:51:59.798 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T13:51:59.798 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T13:51:59.798 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T13:51:59.798 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T13:51:59.798 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T13:51:59.798 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:51:59.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:59.798 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-04-25T13:51:59.805 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T13:51:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T13:51:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T13:51:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:59.805 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 41849sec preferred_lft 41849sec 2024-04-25T13:51:59.806 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 02:42:47:66:6e:b8 brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-04-25T13:51:59.807 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:59.808 INFO:teuthology.orchestra.run.smithi087.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T13:51:59.808 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:51:59.808 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-04-25T13:51:59.808 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-04-25T13:51:59.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:59.808 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T13:51:59.861 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-04-25T13:51:59.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:51:59.863 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-04-25T13:51:59.914 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-04-25T13:51:59.914 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T13:51:59.914 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-04-25T13:51:59.915 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T13:51:59.915 INFO:teuthology.orchestra.run.smithi087.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-04-25T13:51:59.915 INFO:teuthology.orchestra.run.smithi087.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-04-25T13:51:59.915 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T13:51:59.915 INFO:teuthology.orchestra.run.smithi087.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:51:59.915 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:51:59.915 INFO:teuthology.orchestra.run.smithi087.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-04-25T13:51:59.915 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T13:51:59.915 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T13:51:59.915 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-04-25T13:51:59.915 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T13:51:59.915 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T13:51:59.916 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T13:51:59.916 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-04-25T13:52:00.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:52:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T13:52:00.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:52:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:52:00.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:00.038 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:52:00.043 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:52:00.043 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:52:00.043 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:52:00.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:00.043 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-04-25T13:52:00.095 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T13:52:00.096 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:52:00.096 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T13:52:00.141 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T13:52:00.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T13:52:00.141 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:52:00.190 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:52:00.247 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:52:00.296 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:52:00.296 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:52:00.296 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:52:00.296 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:52:00.296 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:52:00.296 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:52:00.297 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:52:00.298 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:52:00.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:00.299 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:52:00.346 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T13:52:00.346 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T13:52:00.391 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:52:00.405 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:52:00.454 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:52:00.455 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:52:00.456 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:52:00.457 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:52:00.457 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:52:00.457 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:52:00.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:00.457 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:52:00.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:2024-04-25T13:52:00.500+0000 7fb53ecb9580 -1 init, newargv = 0x7fb52c001f10 newargc=15 2024-04-25T13:52:00.499 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25767]: starting ceph client 2024-04-25T13:52:00.505 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25767]: starting fuse 2024-04-25T13:52:00.858 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:52:00.861 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:52:01.603 DEBUG:teuthology.orchestra.run.smithi087:> sudo modprobe fuse 2024-04-25T13:52:01.619 DEBUG:teuthology.orchestra.run.smithi087:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/sys 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/proc 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/dev 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/pts 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/run 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/ 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/security 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/shm 2024-04-25T13:52:01.667 INFO:teuthology.orchestra.run.smithi087.stdout:/run/lock 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/cgroup 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/pstore 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/bpf 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/hugepages 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/dev/mqueue 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/debug 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/tracing 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/proc/fs/nfsd 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/fs/fuse/connections 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/sys/kernel/config 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/run/rpc_pipefs 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/run/user/1000 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph 2024-04-25T13:52:01.668 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:52:01.669 INFO:teuthology.orchestra.run.smithi087.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T13:52:01.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:01.670 DEBUG:teuthology.orchestra.run.smithi087:> ls /sys/fs/fuse/connections 2024-04-25T13:52:01.718 INFO:teuthology.orchestra.run.smithi087.stdout:45 2024-04-25T13:52:01.718 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> sudo stdin-killer -- python3 -c ' 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> import glob 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> import re 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> import os 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> import subprocess 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> def _find_admin_socket(client_name): 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> files = glob.glob(asok_path) 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> # Given a non-glob path, it better be there 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> if "*" not in asok_path: 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> assert(len(files) == 1) 2024-04-25T13:52:01.719 DEBUG:teuthology.orchestra.run.smithi087:> return files[0] 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> for f in files: 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> contents = proc_f.read() 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> if mountpoint in contents: 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> return f 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> print(_find_admin_socket("client.1")) 2024-04-25T13:52:01.720 DEBUG:teuthology.orchestra.run.smithi087:> ' 2024-04-25T13:52:01.840 INFO:teuthology.orchestra.run.smithi087.stdout:/var/run/ceph/ceph-client.1.25767.asok 2024-04-25T13:52:01.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-04-25T13:52:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T13:52:01.852 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.25767.asok 2024-04-25T13:52:01.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:01.852 DEBUG:teuthology.orchestra.run.smithi087:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.25767.asok status 2024-04-25T13:52:02.024 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-04-25T13:52:02.024 INFO:teuthology.orchestra.run.smithi087.stdout: "metadata": { 2024-04-25T13:52:02.024 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_sha1": "994e2f20102a20fa9a27966c14b625cc01884df8", 2024-04-25T13:52:02.024 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-3291-g994e2f20 (994e2f20102a20fa9a27966c14b625cc01884df8) squid (dev)", 2024-04-25T13:52:02.024 INFO:teuthology.orchestra.run.smithi087.stdout: "entity_id": "1", 2024-04-25T13:52:02.024 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": "smithi087", 2024-04-25T13:52:02.024 INFO:teuthology.orchestra.run.smithi087.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "pid": "25767", 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "root": "/" 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_count": 0, 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "dentry_pinned_count": 0, 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 7354, 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "inst": { 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "name": { 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "client", 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "num": 7354 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 309514532 2024-04-25T13:52:02.025 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": { 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "192.168.0.1:0", 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 309514532 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "inst_str": "client.7354 192.168.0.1:0/309514532", 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "addr_str": "192.168.0.1:0/309514532", 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "inode_count": 1, 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_epoch": 89, 2024-04-25T13:52:02.026 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch": 187, 2024-04-25T13:52:02.027 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_epoch_barrier": 0, 2024-04-25T13:52:02.027 INFO:teuthology.orchestra.run.smithi087.stdout: "blocklisted": false, 2024-04-25T13:52:02.027 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs" 2024-04-25T13:52:02.027 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-04-25T13:52:02.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:02.040 DEBUG:teuthology.orchestra.run.smithi087:> stat --file-system '--printf=%T 2024-04-25T13:52:02.040 DEBUG:teuthology.orchestra.run.smithi087:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T13:52:02.046 INFO:teuthology.orchestra.run.smithi087.stdout:fuseblk 2024-04-25T13:52:02.046 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T13:52:02.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:02.047 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T13:52:02.100 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:02.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb601682640 1 Processor -- start 2024-04-25T13:52:02.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb601682640 1 -- start start 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb601682640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc161240 0x7fb5fc0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb601682640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5fc06fcf0 con 0x7fb5fc161650 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb601682640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5fc0b0c60 con 0x7fb5fc06f5a0 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb601682640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5fc073f10 con 0x7fb5fc161240 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5fb7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb5fc161650 0x7fb5fc0728e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53536/0 (socket says 172.21.15.80:53536) 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5fb7fe640 1 -- 172.21.15.80:0/1371123511 learned_addr learned my addr 172.21.15.80:0/1371123511 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5fa7fc640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc161240 0x7fb5fc0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2680966173 0 0) 0x7fb5fc06fcf0 con 0x7fb5fc161650 2024-04-25T13:52:02.263 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5e8003530 con 0x7fb5fc161650 2024-04-25T13:52:02.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1207981115 0 0) 0x7fb5e8003530 con 0x7fb5fc161650 2024-04-25T13:52:02.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc161240 msgr2=0x7fb5fc0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:02.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc161240 0x7fb5fc0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 >> v1:172.21.15.87:6789/0 conn(0x7fb5fc06f5a0 legacy=0x7fb5fc160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5fc0b0f40 con 0x7fb5fc161650 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb5f4001680 con 0x7fb5fc161650 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 >> v1:172.21.15.80:6789/0 conn(0x7fb5fc161650 legacy=0x7fb5fc0728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc161240 0x7fb5e80039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003f70 0x7fb5e8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5e8004d50 0x7fb5e8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc073f10 con 0x7fb5e8004d50 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.267+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e800bc10 con 0x7fb5e8003f70 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e800be10 con 0x7fb5fc161240 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5fb7fe640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5e8004d50 0x7fb5e8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5fa7fc640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003f70 0x7fb5e8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5faffd640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc161240 0x7fb5e80039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5fb7fe640 1 -- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc161240 msgr2=0x7fb5e80039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:02.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5fb7fe640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc161240 0x7fb5e80039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5fb7fe640 1 -- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003f70 msgr2=0x7fb5e8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5fb7fe640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5e8003f70 0x7fb5e8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5fb7fe640 1 -- 172.21.15.80:0/1371123511 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e8003530 con 0x7fb5e8004d50 2024-04-25T13:52:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5fb7fe640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5e8004d50 0x7fb5e8005160 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc070290 tx=0x7fb5f401f410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d2a67adc1889b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4007d60 con 0x7fb5e8004d50 2024-04-25T13:52:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5f4005ae0 con 0x7fb5e8004d50 2024-04-25T13:52:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.271+0000 7fb5f9ffb640 1 -- 172.21.15.80:0/1371123511 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5f4007070 con 0x7fb5e8004d50 2024-04-25T13:52:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- 172.21.15.80:0/1371123511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5e8004d50 msgr2=0x7fb5e8005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 --2- 172.21.15.80:0/1371123511 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5e8004d50 0x7fb5e8005160 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc070290 tx=0x7fb5f401f410 comp rx=0 tx=0).stop 2024-04-25T13:52:02.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb5fb7fe640 1 -- 172.21.15.80:0/1371123511 reap_dead start 2024-04-25T13:52:02.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- 172.21.15.80:0/1371123511 shutdown_connections 2024-04-25T13:52:02.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- 172.21.15.80:0/1371123511 >> 172.21.15.80:0/1371123511 conn(0x7fb5fc06d080 msgr2=0x7fb5fc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:02.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- 172.21.15.80:0/1371123511 shutdown_connections 2024-04-25T13:52:02.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- 172.21.15.80:0/1371123511 wait complete. 2024-04-25T13:52:02.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 Processor -- start 2024-04-25T13:52:02.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- start start 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5fc15e980 0x7fb5fc1f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc15fe30 0x7fb5fc14bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5fc160240 0x7fb5fc14c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc0723a0 con 0x7fb5fc15e980 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb5fc06f9b0 con 0x7fb5fc160240 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb601682640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb5fc072520 con 0x7fb5fc15fe30 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb5faffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5fc15e980 0x7fb5fc1f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb5faffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5fc15e980 0x7fb5fc1f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42878/0 (socket says 172.21.15.80:42878) 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb5faffd640 1 -- 172.21.15.80:0/3837831468 learned_addr learned my addr 172.21.15.80:0/3837831468 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb5fa7fc640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc15fe30 0x7fb5fc14bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.275+0000 7fb5fb7fe640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5fc160240 0x7fb5fc14c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5faffd640 1 -- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc15fe30 msgr2=0x7fb5fc14bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5faffd640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc15fe30 0x7fb5fc14bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5faffd640 1 -- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5fc160240 msgr2=0x7fb5fc14c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5faffd640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5fc160240 0x7fb5fc14c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5faffd640 1 -- 172.21.15.80:0/3837831468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5fc15d8e0 con 0x7fb5fc15e980 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5fa7fc640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc15fe30 0x7fb5fc14bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5faffd640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5fc15e980 0x7fb5fc1f5ff0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e820 tx=0x7fb5ec00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5f37fe640 1 -- 172.21.15.80:0/3837831468 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec003b90 con 0x7fb5fc15e980 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5f37fe640 1 -- 172.21.15.80:0/3837831468 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5ec018070 con 0x7fb5fc15e980 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5fc15dbc0 con 0x7fb5fc15e980 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5f37fe640 1 -- 172.21.15.80:0/3837831468 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec0050d0 con 0x7fb5fc15e980 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5fc15e180 con 0x7fb5fc15e980 2024-04-25T13:52:02.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5fc06a3a0 con 0x7fb5fc15e980 2024-04-25T13:52:02.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5f37fe640 1 -- 172.21.15.80:0/3837831468 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb5ec005270 con 0x7fb5fc15e980 2024-04-25T13:52:02.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5f37fe640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5dc06f430 0x7fb5dc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5f37fe640 1 -- 172.21.15.80:0/3837831468 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7fb5ec08fc30 con 0x7fb5fc15e980 2024-04-25T13:52:02.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.279+0000 7fb5fa7fc640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5dc06f430 0x7fb5dc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.276 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.283+0000 7fb5fa7fc640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5dc06f430 0x7fb5dc071940 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009af0 tx=0x7fb5e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:02.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.283+0000 7fb5f37fe640 1 -- 172.21.15.80:0/3837831468 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb5ec05cd20 con 0x7fb5fc15e980 2024-04-25T13:52:02.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.475+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5fc0739c0 con 0x7fb5fc15e980 2024-04-25T13:52:02.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.475+0000 7fb5f37fe640 1 -- 172.21.15.80:0/3837831468 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7fb5ec05c620 con 0x7fb5fc15e980 2024-04-25T13:52:02.470 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:02.470 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:52:02.470 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:52:02.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5dc06f430 msgr2=0x7fb5dc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5dc06f430 0x7fb5dc071940 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fb5e4009af0 tx=0x7fb5e4008040 comp rx=0 tx=0).stop 2024-04-25T13:52:02.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5fc15e980 msgr2=0x7fb5fc1f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5fc15e980 0x7fb5fc1f5ff0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e820 tx=0x7fb5ec00ece0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 shutdown_connections 2024-04-25T13:52:02.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb5dc06f430 0x7fb5dc071940 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb5fc160240 0x7fb5fc14c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb5fc15fe30 0x7fb5fc14bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 --2- 172.21.15.80:0/3837831468 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb5fc15e980 0x7fb5fc1f5ff0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 >> 172.21.15.80:0/3837831468 conn(0x7fb5fc06d080 msgr2=0x7fb5fc0b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:02.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 shutdown_connections 2024-04-25T13:52:02.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.479+0000 7fb601682640 1 -- 172.21.15.80:0/3837831468 wait complete. 2024-04-25T13:52:02.491 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:52:02.650 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":9213,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.931629185,"mdsmap_epoch":89,"osdmap_epoch":187,"osdmap_epoch_barrier":187,"uptime":16.906427672} 2024-04-25T13:52:02.651 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 9213, "mdsmap_epoch": 89, "osdmap_epoch": 187, "osdmap_epoch_barrier": 187, "rank_uptime": 11.931629185, "state": "up:active", "uptime": 16.906427672, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:52:02.651 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2024-04-25T13:52:02.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d863c1640 1 Processor -- start 2024-04-25T13:52:02.805 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d863c1640 1 -- start start 2024-04-25T13:52:02.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d863c1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9d800af310 0x7f9d800b50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d863c1640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d8015cc80 con 0x7f9d800b3960 2024-04-25T13:52:02.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d863c1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d800b5c00 con 0x7f9d800b2e40 2024-04-25T13:52:02.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d863c1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f9d800b5ee0 con 0x7f9d800af310 2024-04-25T13:52:02.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d7ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f9d800af310 0x7f9d800b50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.807 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d8495e640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9d800b3960 0x7f9d800d1f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53548/0 (socket says 172.21.15.80:53548) 2024-04-25T13:52:02.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d8495e640 1 -- 172.21.15.80:0/720624166 learned_addr learned my addr 172.21.15.80:0/720624166 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:02.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d7effd640 1 -- 172.21.15.80:0/720624166 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2996028644 0 0) 0x7f9d70001120 con 0x7f9d800b3960 2024-04-25T13:52:02.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d7effd640 1 -- 172.21.15.80:0/720624166 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d68003450 con 0x7f9d800b3960 2024-04-25T13:52:02.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d7ffff640 1 -- 172.21.15.80:0/720624166 >> v1:172.21.15.87:6789/0 conn(0x7f9d800b2e40 legacy=0x7f9d800b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d7ffff640 1 -- 172.21.15.80:0/720624166 >> v1:172.21.15.80:6789/0 conn(0x7f9d800b3960 legacy=0x7f9d800d1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.808 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.811+0000 7f9d7ffff640 1 -- 172.21.15.80:0/720624166 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d800b61c0 con 0x7f9d800af310 2024-04-25T13:52:02.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d7ffff640 1 --2- 172.21.15.80:0/720624166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800af310 0x7f9d800b50c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009d90 tx=0x7f9d7801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d9e7a72a88c5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:02.809 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d7effd640 1 -- 172.21.15.80:0/720624166 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d780b2070 con 0x7f9d800af310 2024-04-25T13:52:02.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d7effd640 1 -- 172.21.15.80:0/720624166 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d780a5980 con 0x7f9d800af310 2024-04-25T13:52:02.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d7effd640 1 -- 172.21.15.80:0/720624166 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d780a5d50 con 0x7f9d800af310 2024-04-25T13:52:02.810 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d863c1640 1 -- 172.21.15.80:0/720624166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800af310 msgr2=0x7f9d800b50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d863c1640 1 --2- 172.21.15.80:0/720624166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800af310 0x7f9d800b50c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009d90 tx=0x7f9d7801c940 comp rx=0 tx=0).stop 2024-04-25T13:52:02.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d863c1640 1 -- 172.21.15.80:0/720624166 shutdown_connections 2024-04-25T13:52:02.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d863c1640 1 --2- 172.21.15.80:0/720624166 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800af310 0x7f9d800b50c0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d863c1640 1 -- 172.21.15.80:0/720624166 >> 172.21.15.80:0/720624166 conn(0x7f9d80073a70 msgr2=0x7f9d80073e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:02.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d863c1640 1 -- 172.21.15.80:0/720624166 shutdown_connections 2024-04-25T13:52:02.811 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.815+0000 7f9d863c1640 1 -- 172.21.15.80:0/720624166 wait complete. 2024-04-25T13:52:02.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 Processor -- start 2024-04-25T13:52:02.812 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 -- start start 2024-04-25T13:52:02.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d800af310 0x7f9d801463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800b2e40 0x7f9d801585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800b3960 0x7f9d80099250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9d800d2680 con 0x7f9d800b2e40 2024-04-25T13:52:02.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9d800b5600 con 0x7f9d800b3960 2024-04-25T13:52:02.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9d800d2800 con 0x7f9d800af310 2024-04-25T13:52:02.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800b2e40 0x7f9d801585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800b2e40 0x7f9d801585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42902/0 (socket says 172.21.15.80:42902) 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 -- 172.21.15.80:0/1333042886 learned_addr learned my addr 172.21.15.80:0/1333042886 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7ffff640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d800af310 0x7f9d801463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d8495e640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800b3960 0x7f9d80099250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 -- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d800af310 msgr2=0x7f9d801463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d800af310 0x7f9d801463d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 -- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800b3960 msgr2=0x7f9d80099250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800b3960 0x7f9d80099250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 -- 172.21.15.80:0/1333042886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d800d2e40 con 0x7f9d800b2e40 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d8495e640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800b3960 0x7f9d80099250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7f7fe640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800b2e40 0x7f9d801585a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400c370 tx=0x7f9d7400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7cff9640 1 -- 172.21.15.80:0/1333042886 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d740052d0 con 0x7f9d800b2e40 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7cff9640 1 -- 172.21.15.80:0/1333042886 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d74005470 con 0x7f9d800b2e40 2024-04-25T13:52:02.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d80145330 con 0x7f9d800b2e40 2024-04-25T13:52:02.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d7cff9640 1 -- 172.21.15.80:0/1333042886 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74007260 con 0x7f9d800b2e40 2024-04-25T13:52:02.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.819+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d801458f0 con 0x7f9d800b2e40 2024-04-25T13:52:02.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.823+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d8006a440 con 0x7f9d800b2e40 2024-04-25T13:52:02.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.823+0000 7f9d7cff9640 1 -- 172.21.15.80:0/1333042886 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9d74007400 con 0x7f9d800b2e40 2024-04-25T13:52:02.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.823+0000 7f9d7cff9640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9d6c06f400 0x7f9d6c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:02.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.823+0000 7f9d7cff9640 1 -- 172.21.15.80:0/1333042886 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(187..187 src has 1..187) ==== 5709+0+0 (secure 0 0 0) 0x7f9d7408e750 con 0x7f9d800b2e40 2024-04-25T13:52:02.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.823+0000 7f9d7ffff640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9d6c06f400 0x7f9d6c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:02.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.823+0000 7f9d7ffff640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9d6c06f400 0x7f9d6c071910 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9d780af000 tx=0x7f9d780a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:02.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.827+0000 7f9d7cff9640 1 -- 172.21.15.80:0/1333042886 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9d7405b640 con 0x7f9d800b2e40 2024-04-25T13:52:02.985 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:02.991+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) -- 0x7f9d801450c0 con 0x7f9d800b2e40 2024-04-25T13:52:03.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.415+0000 7f9d7cff9640 1 -- 172.21.15.80:0/1333042886 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v188) ==== 181+0+0 (secure 0 0 0) 0x7f9d7405af40 con 0x7f9d800b2e40 2024-04-25T13:52:03.409 INFO:teuthology.orchestra.run.smithi080.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2024-04-25T13:52:03.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9d6c06f400 msgr2=0x7f9d6c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:03.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9d6c06f400 0x7f9d6c071910 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9d780af000 tx=0x7f9d780a7070 comp rx=0 tx=0).stop 2024-04-25T13:52:03.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800b2e40 msgr2=0x7f9d801585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:03.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800b2e40 0x7f9d801585a0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400c370 tx=0x7f9d7400c830 comp rx=0 tx=0).stop 2024-04-25T13:52:03.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 shutdown_connections 2024-04-25T13:52:03.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9d6c06f400 0x7f9d6c071910 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:03.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9d800b3960 0x7f9d80099250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:03.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9d800b2e40 0x7f9d801585a0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:03.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 --2- 172.21.15.80:0/1333042886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9d800af310 0x7f9d801463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:03.414 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 >> 172.21.15.80:0/1333042886 conn(0x7f9d80073a70 msgr2=0x7f9d80158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:03.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 shutdown_connections 2024-04-25T13:52:03.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:03.419+0000 7f9d863c1640 1 -- 172.21.15.80:0/1333042886 wait complete. 2024-04-25T13:52:03.443 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2024-04-25T13:52:03.611 INFO:teuthology.orchestra.run.smithi080.stdout:{"osd_mon_report_interval":"5"} 2024-04-25T13:52:03.611 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2024-04-25T13:52:03.611 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2024-04-25T13:52:03.611 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:03.611 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:03.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:03.617 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2024-04-25T13:52:03.809 INFO:teuthology.orchestra.run.smithi080.stderr:16+0 records in 2024-04-25T13:52:03.809 INFO:teuthology.orchestra.run.smithi080.stderr:16+0 records out 2024-04-25T13:52:03.809 INFO:teuthology.orchestra.run.smithi080.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.135444 s, 124 MB/s 2024-04-25T13:52:03.810 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:03.810 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:03.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:03.816 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2024-04-25T13:52:03.983 INFO:teuthology.orchestra.run.smithi080.stderr:17+0 records in 2024-04-25T13:52:03.983 INFO:teuthology.orchestra.run.smithi080.stderr:17+0 records out 2024-04-25T13:52:03.984 INFO:teuthology.orchestra.run.smithi080.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.114689 s, 155 MB/s 2024-04-25T13:52:03.985 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2024-04-25T13:52:03.985 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:52:04.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b4a72e640 1 Processor -- start 2024-04-25T13:52:04.136 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b4a72e640 1 -- start start 2024-04-25T13:52:04.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b4a72e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3b4406f5a0 0x7f3b44160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:04.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b4a72e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3b440733b0 0x7f3b44161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:04.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b4a72e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b4406fcf0 con 0x7f3b44161070 2024-04-25T13:52:04.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b4a72e640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3b441619c0 con 0x7f3b440733b0 2024-04-25T13:52:04.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b4a72e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3b44073f10 con 0x7f3b4406f5a0 2024-04-25T13:52:04.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b48ccb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3b440733b0 0x7f3b44161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:04.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b48ccb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3b440733b0 0x7f3b44161480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42916/0 (socket says 172.21.15.80:42916) 2024-04-25T13:52:04.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b48ccb640 1 -- 172.21.15.80:0/507795845 learned_addr learned my addr 172.21.15.80:0/507795845 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:04.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b43fff640 1 --2- 172.21.15.80:0/507795845 >> v2:172.21.15.87:3300/0 conn(0x7f3b4406f5a0 0x7f3b44160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:04.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b48ccb640 1 -- 172.21.15.80:0/507795845 >> v2:172.21.15.87:3300/0 conn(0x7f3b4406f5a0 msgr2=0x7f3b44160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:04.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b48ccb640 1 --2- 172.21.15.80:0/507795845 >> v2:172.21.15.87:3300/0 conn(0x7f3b4406f5a0 0x7f3b44160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b48ccb640 1 -- 172.21.15.80:0/507795845 >> v1:172.21.15.87:6789/0 conn(0x7f3b44161070 legacy=0x7f3b440b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:04.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.143+0000 7f3b48ccb640 1 -- 172.21.15.80:0/507795845 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b440b0c60 con 0x7f3b440733b0 2024-04-25T13:52:04.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b48ccb640 1 --2- 172.21.15.80:0/507795845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b440733b0 0x7f3b44161480 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f3b44071e30 tx=0x7f3b3401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ce8095217f0e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:04.140 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b42ffd640 1 -- 172.21.15.80:0/507795845 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340a55f0 con 0x7f3b440733b0 2024-04-25T13:52:04.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b42ffd640 1 -- 172.21.15.80:0/507795845 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b340a5790 con 0x7f3b440733b0 2024-04-25T13:52:04.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b42ffd640 1 -- 172.21.15.80:0/507795845 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340a5b60 con 0x7f3b440733b0 2024-04-25T13:52:04.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 -- 172.21.15.80:0/507795845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b440733b0 msgr2=0x7f3b44161480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:04.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/507795845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b440733b0 0x7f3b44161480 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f3b44071e30 tx=0x7f3b3401c920 comp rx=0 tx=0).stop 2024-04-25T13:52:04.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 -- 172.21.15.80:0/507795845 shutdown_connections 2024-04-25T13:52:04.141 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/507795845 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b440733b0 0x7f3b44161480 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/507795845 >> v2:172.21.15.87:3300/0 conn(0x7f3b4406f5a0 0x7f3b44160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 -- 172.21.15.80:0/507795845 >> 172.21.15.80:0/507795845 conn(0x7f3b4406d080 msgr2=0x7f3b440b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:04.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 -- 172.21.15.80:0/507795845 shutdown_connections 2024-04-25T13:52:04.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 -- 172.21.15.80:0/507795845 wait complete. 2024-04-25T13:52:04.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 Processor -- start 2024-04-25T13:52:04.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.147+0000 7f3b4a72e640 1 -- start start 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4406f5a0 0x7f3b44141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b440733b0 0x7f3b4415fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b44161070 0x7f3b440b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4416a050 con 0x7f3b44161070 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3b44169ed0 con 0x7f3b440733b0 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3b4416a1d0 con 0x7f3b4406f5a0 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b44161070 0x7f3b440b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b44161070 0x7f3b440b00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42920/0 (socket says 172.21.15.80:42920) 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 -- 172.21.15.80:0/3348600266 learned_addr learned my addr 172.21.15.80:0/3348600266 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 -- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4406f5a0 msgr2=0x7f3b44141a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4406f5a0 0x7f3b44141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 -- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b440733b0 msgr2=0x7f3b4415fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b437fe640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b440733b0 0x7f3b4415fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:04.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b440733b0 0x7f3b4415fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 -- 172.21.15.80:0/3348600266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b4413f090 con 0x7f3b44161070 2024-04-25T13:52:04.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b48ccb640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b44161070 0x7f3b440b00b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f3b340af040 tx=0x7f3b340196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:04.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b40ff9640 1 -- 172.21.15.80:0/3348600266 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34009510 con 0x7f3b44161070 2024-04-25T13:52:04.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b40ff9640 1 -- 172.21.15.80:0/3348600266 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b340be070 con 0x7f3b44161070 2024-04-25T13:52:04.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b4413f370 con 0x7f3b44161070 2024-04-25T13:52:04.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b40ff9640 1 -- 172.21.15.80:0/3348600266 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340b8870 con 0x7f3b44161070 2024-04-25T13:52:04.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b4413f930 con 0x7f3b44161070 2024-04-25T13:52:04.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4406a3a0 con 0x7f3b44161070 2024-04-25T13:52:04.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.151+0000 7f3b40ff9640 1 -- 172.21.15.80:0/3348600266 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3b340b8a10 con 0x7f3b44161070 2024-04-25T13:52:04.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.155+0000 7f3b40ff9640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b3006f430 0x7f3b30071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:04.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.155+0000 7f3b40ff9640 1 -- 172.21.15.80:0/3348600266 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 5709+0+0 (secure 0 0 0) 0x7f3b3412a690 con 0x7f3b44161070 2024-04-25T13:52:04.148 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.155+0000 7f3b43fff640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b3006f430 0x7f3b30071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:04.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.155+0000 7f3b43fff640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b3006f430 0x7f3b30071940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3b440b6e70 tx=0x7f3b3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:04.151 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.159+0000 7f3b40ff9640 1 -- 172.21.15.80:0/3348600266 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3b340c7020 con 0x7f3b44161070 2024-04-25T13:52:04.312 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.319+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f3b440739c0 con 0x7f3b44161070 2024-04-25T13:52:04.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.319+0000 7f3b40ff9640 1 -- 172.21.15.80:0/3348600266 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v188) ==== 81+0+34096 (secure 0 0 0) 0x7f3b340165a0 con 0x7f3b44161070 2024-04-25T13:52:04.315 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:04.315 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:04.315 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 188, 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:52:03.414703+0000", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:52:04.316 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 23, 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:52:04.317 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:04.318 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:04.319 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:04.320 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:04.321 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:04.322 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:04.323 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 22, 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:48.404349+0000", 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:04.324 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:04.325 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "186", 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:04.326 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:04.327 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:04.328 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.329 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 23, 2024-04-25T13:52:04.330 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:49.039512+0000", 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:52:04.331 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "188", 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:04.332 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:04.333 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:04.334 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:04.335 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.336 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.337 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.338 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:52:04.339 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.340 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.341 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.342 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.343 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.344 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.345 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:52:04.346 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:04.347 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.348 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.349 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.350 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.351 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.352 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:04.353 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.354 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.355 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:52:04.356 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:04.357 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:04.358 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:04.359 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:52:04.360 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:52:04.361 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.362 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.363 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:04.364 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:04.365 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:04.366 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:04.367 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.368 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:04.369 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:04.370 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:04.371 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:52:04.372 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:52:04.373 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:04.374 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:04.375 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:04.376 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:04.377 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:04.378 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:04.379 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:52:04.380 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:04.381 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:04.382 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.383 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:04.384 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:04.385 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:04.386 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:52:04.387 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:52:04.388 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:04.389 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b3006f430 msgr2=0x7f3b30071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b3006f430 0x7f3b30071940 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f3b440b6e70 tx=0x7f3b3c008040 comp rx=0 tx=0).stop 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b44161070 msgr2=0x7f3b440b00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b44161070 0x7f3b440b00b0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f3b340af040 tx=0x7f3b340196c0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 shutdown_connections 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3b3006f430 0x7f3b30071940 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3b44161070 0x7f3b440b00b0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b440733b0 0x7f3b4415fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 --2- 172.21.15.80:0/3348600266 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3b4406f5a0 0x7f3b44141a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:04.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 >> 172.21.15.80:0/3348600266 conn(0x7f3b4406d080 msgr2=0x7f3b440b2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:04.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 shutdown_connections 2024-04-25T13:52:04.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:04.323+0000 7f3b4a72e640 1 -- 172.21.15.80:0/3348600266 wait complete. 2024-04-25T13:52:04.391 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:52:09.392 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:52:09.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.555+0000 7f3833d3f640 1 Processor -- start 2024-04-25T13:52:09.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.555+0000 7f3833d3f640 1 -- start start 2024-04-25T13:52:09.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3833d3f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f382c161f90 0x7f382c0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3833d3f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c15ec40 0x7f382c15f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3833d3f640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f382c0d0d70 con 0x7f382c0d0510 2024-04-25T13:52:09.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3833d3f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f382c0d1050 con 0x7f382c161f90 2024-04-25T13:52:09.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3833d3f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f382c06dca0 con 0x7f382c15ec40 2024-04-25T13:52:09.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38312da640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f382c0d0510 0x7f382c15e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:53560/0 (socket says 172.21.15.80:53560) 2024-04-25T13:52:09.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38312da640 1 -- 172.21.15.80:0/837967244 learned_addr learned my addr 172.21.15.80:0/837967244 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:09.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38322dc640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c15ec40 0x7f382c15f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3831adb640 1 --2- 172.21.15.80:0/837967244 >> v2:172.21.15.87:3300/0 conn(0x7f382c161f90 0x7f382c0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3744443606 0 0) 0x7f382c0d0d70 con 0x7f382c0d0510 2024-04-25T13:52:09.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3818003530 con 0x7f382c0d0510 2024-04-25T13:52:09.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3175198924 0 0) 0x7f3818003530 con 0x7f382c0d0510 2024-04-25T13:52:09.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c15ec40 msgr2=0x7f382c15f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c15ec40 0x7f382c15f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 >> v2:172.21.15.87:3300/0 conn(0x7f382c161f90 msgr2=0x7f382c0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 --2- 172.21.15.80:0/837967244 >> v2:172.21.15.87:3300/0 conn(0x7f382c161f90 0x7f382c0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c06df80 con 0x7f382c0d0510 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f381c001680 con 0x7f382c0d0510 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 >> v1:172.21.15.80:6789/0 conn(0x7f382c0d0510 legacy=0x7f382c15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c15ec40 0x7f38180039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3818003f40 0x7f3818004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3818004d50 0x7f3818005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f382c0d1050 con 0x7f3818003f40 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f382c06dca0 con 0x7f3818004d50 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f381800bad0 con 0x7f382c15ec40 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38312da640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3818003f40 0x7f3818004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f3831adb640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c15ec40 0x7f38180039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38322dc640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3818004d50 0x7f3818005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38312da640 1 -- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c15ec40 msgr2=0x7f38180039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:09.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38312da640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c15ec40 0x7f38180039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38312da640 1 -- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3818004d50 msgr2=0x7f3818005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:09.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38312da640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3818004d50 0x7f3818005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38312da640 1 -- 172.21.15.80:0/837967244 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3818003530 con 0x7f3818003f40 2024-04-25T13:52:09.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.559+0000 7f38322dc640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3818004d50 0x7f3818005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f38312da640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3818003f40 0x7f3818004810 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c240 tx=0x7f381c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da66def3c5fc2fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c007d40 con 0x7f3818003f40 2024-04-25T13:52:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f381c0a9020 con 0x7f3818003f40 2024-04-25T13:52:09.561 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3830ad9640 1 -- 172.21.15.80:0/837967244 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f381c005b00 con 0x7f3818003f40 2024-04-25T13:52:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3833d3f640 1 -- 172.21.15.80:0/837967244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3818003f40 msgr2=0x7f3818004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3833d3f640 1 --2- 172.21.15.80:0/837967244 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3818003f40 0x7f3818004810 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f381c00c240 tx=0x7f381c01cc40 comp rx=0 tx=0).stop 2024-04-25T13:52:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f38322dc640 1 -- 172.21.15.80:0/837967244 reap_dead start 2024-04-25T13:52:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3833d3f640 1 -- 172.21.15.80:0/837967244 shutdown_connections 2024-04-25T13:52:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3833d3f640 1 -- 172.21.15.80:0/837967244 >> 172.21.15.80:0/837967244 conn(0x7f382c06d080 msgr2=0x7f382c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3833d3f640 1 -- 172.21.15.80:0/837967244 shutdown_connections 2024-04-25T13:52:09.562 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.567+0000 7f3833d3f640 1 -- 172.21.15.80:0/837967244 wait complete. 2024-04-25T13:52:09.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 Processor -- start 2024-04-25T13:52:09.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 -- start start 2024-04-25T13:52:09.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c13d030 0x7f382c0739b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c0b0240 0x7f382c15d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c1608a0 0x7f382c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f382c06f630 con 0x7f382c13d030 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f382c06f4b0 con 0x7f382c0b0240 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f382c0d0920 con 0x7f382c1608a0 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c13d030 0x7f382c0739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c13d030 0x7f382c0739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:42950/0 (socket says 172.21.15.80:42950) 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 -- 172.21.15.80:0/1317432714 learned_addr learned my addr 172.21.15.80:0/1317432714 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f38322dc640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c1608a0 0x7f382c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f38312da640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c0b0240 0x7f382c15d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 -- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c1608a0 msgr2=0x7f382c160cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c1608a0 0x7f382c160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 -- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c0b0240 msgr2=0x7f382c15d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c0b0240 0x7f382c15d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 -- 172.21.15.80:0/1317432714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f382c1611f0 con 0x7f382c13d030 2024-04-25T13:52:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3831adb640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c13d030 0x7f382c0739b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f38280119b0 tx=0x7f3828011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f38227fc640 1 -- 172.21.15.80:0/1317432714 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f382800faf0 con 0x7f382c13d030 2024-04-25T13:52:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f38227fc640 1 -- 172.21.15.80:0/1317432714 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f382800fc90 con 0x7f382c13d030 2024-04-25T13:52:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f38227fc640 1 -- 172.21.15.80:0/1317432714 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3828014440 con 0x7f382c13d030 2024-04-25T13:52:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f382c15de30 con 0x7f382c13d030 2024-04-25T13:52:09.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f382c072bf0 con 0x7f382c13d030 2024-04-25T13:52:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.571+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f382c06a3a0 con 0x7f382c13d030 2024-04-25T13:52:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.575+0000 7f38227fc640 1 -- 172.21.15.80:0/1317432714 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3828012040 con 0x7f382c13d030 2024-04-25T13:52:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.575+0000 7f38227fc640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f381006f400 0x7f3810071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.575+0000 7f38227fc640 1 -- 172.21.15.80:0/1317432714 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(188..188 src has 1..188) ==== 5709+0+0 (secure 0 0 0) 0x7f382808f930 con 0x7f382c13d030 2024-04-25T13:52:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.575+0000 7f38312da640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f381006f400 0x7f3810071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:09.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.575+0000 7f38312da640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f381006f400 0x7f3810071910 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f381c0aa000 tx=0x7f382c15f590 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:09.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.579+0000 7f38227fc640 1 -- 172.21.15.80:0/1317432714 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f382c1611f0 con 0x7f382c13d030 2024-04-25T13:52:09.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.731+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f382c06f120 con 0x7f382c13d030 2024-04-25T13:52:09.728 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f38227fc640 1 -- 172.21.15.80:0/1317432714 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v188) ==== 81+0+34096 (secure 0 0 0) 0x7f3828019030 con 0x7f382c13d030 2024-04-25T13:52:09.729 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:09.729 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:09.729 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 188, 2024-04-25T13:52:09.729 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:52:03.414703+0000", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:52:09.730 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 23, 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:09.731 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:52:09.732 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:09.733 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:09.734 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:09.735 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:09.736 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 22, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:48.404349+0000", 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:09.737 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:09.738 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "186", 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:09.739 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:09.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:52:09.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:09.742 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 23, 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:49.039512+0000", 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:09.743 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:52:09.744 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "188", 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:09.745 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:09.746 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:09.747 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:09.748 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:09.749 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.750 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:52:09.751 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.754 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.755 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:52:09.756 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.757 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.758 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.759 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.760 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:52:09.761 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:09.762 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.763 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.764 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.765 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:52:09.766 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.767 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.768 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:09.769 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.770 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.771 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:52:09.772 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.773 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.774 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:09.775 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.776 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.777 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:52:09.778 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:09.779 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:09.780 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:09.781 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.782 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.783 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:09.784 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:09.785 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.786 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.787 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:09.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:09.789 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.790 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:09.791 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:09.792 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:52:09.793 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:09.794 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:52:09.795 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:52:09.796 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:52:09.797 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:52:09.797 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:52:09.797 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:09.797 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:09.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f381006f400 msgr2=0x7f3810071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f381006f400 0x7f3810071910 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f381c0aa000 tx=0x7f382c15f590 comp rx=0 tx=0).stop 2024-04-25T13:52:09.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c13d030 msgr2=0x7f382c0739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:09.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c13d030 0x7f382c0739b0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f38280119b0 tx=0x7f3828011e70 comp rx=0 tx=0).stop 2024-04-25T13:52:09.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 shutdown_connections 2024-04-25T13:52:09.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f381006f400 0x7f3810071910 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.798 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f382c1608a0 0x7f382c160cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f382c0b0240 0x7f382c15d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 --2- 172.21.15.80:0/1317432714 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f382c13d030 0x7f382c0739b0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:09.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 >> 172.21.15.80:0/1317432714 conn(0x7f382c06d080 msgr2=0x7f382c1412f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:09.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 shutdown_connections 2024-04-25T13:52:09.799 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:09.735+0000 7f3833d3f640 1 -- 172.21.15.80:0/1317432714 wait complete. 2024-04-25T13:52:09.799 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:52:14.800 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:52:14.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.963+0000 7fa77d23c640 1 Processor -- start 2024-04-25T13:52:14.960 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.963+0000 7fa77d23c640 1 -- start start 2024-04-25T13:52:14.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77d23c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fa778161650 0x7fa778072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:14.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77d23c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa77806fcf0 con 0x7fa77806f5a0 2024-04-25T13:52:14.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77d23c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7780b0c60 con 0x7fa778161240 2024-04-25T13:52:14.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77d23c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fa778073f10 con 0x7fa778161650 2024-04-25T13:52:14.963 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa776d9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa77806f5a0 0x7fa778160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34696/0 (socket says 172.21.15.80:34696) 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa776d9d640 1 -- 172.21.15.80:0/303857446 learned_addr learned my addr 172.21.15.80:0/303857446 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77759e640 1 --2- 172.21.15.80:0/303857446 >> v2:172.21.15.87:3300/0 conn(0x7fa778161650 0x7fa778072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3175076693 0 0) 0x7fa77806fcf0 con 0x7fa77806f5a0 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa760003530 con 0x7fa77806f5a0 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 202818545 0 0) 0x7fa760003530 con 0x7fa77806f5a0 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 >> v2:172.21.15.87:3300/0 conn(0x7fa778161650 msgr2=0x7fa778072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 --2- 172.21.15.80:0/303857446 >> v2:172.21.15.87:3300/0 conn(0x7fa778161650 0x7fa778072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 >> v1:172.21.15.87:6789/0 conn(0x7fa778161240 legacy=0x7fa7780723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7780b0f40 con 0x7fa77806f5a0 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fa768001680 con 0x7fa77806f5a0 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 >> v1:172.21.15.80:6789/0 conn(0x7fa77806f5a0 legacy=0x7fa778160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7600039c0 0x7fa760003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa760004810 0x7fa760004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760005260 0x7fa76000b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:14.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa778073f10 con 0x7fa760004810 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa76000c1f0 con 0x7fa760005260 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa76000c3f0 con 0x7fa7600039c0 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77659c640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa760004810 0x7fa760004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77759e640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760005260 0x7fa76000b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa776d9d640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7600039c0 0x7fa760003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77659c640 1 -- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7600039c0 msgr2=0x7fa760003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77659c640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7600039c0 0x7fa760003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77659c640 1 -- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760005260 msgr2=0x7fa76000b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77659c640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760005260 0x7fa76000b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77759e640 1 -- 172.21.15.80:0/303857446 reap_dead start 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa77659c640 1 -- 172.21.15.80:0/303857446 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa760003530 con 0x7fa760004810 2024-04-25T13:52:14.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.967+0000 7fa776d9d640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa7600039c0 0x7fa760003e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:14.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa77659c640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa760004810 0x7fa760004c20 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa76c001780 tx=0x7fa76c01c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4a7db6d5dec4a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:14.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0013a0 con 0x7fa760004810 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa76c0052f0 con 0x7fa760004810 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa775d9b640 1 -- 172.21.15.80:0/303857446 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa76c003840 con 0x7fa760004810 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa77d23c640 1 -- 172.21.15.80:0/303857446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa760004810 msgr2=0x7fa760004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa77d23c640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa760004810 0x7fa760004c20 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fa76c001780 tx=0x7fa76c01c400 comp rx=0 tx=0).stop 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa77d23c640 1 -- 172.21.15.80:0/303857446 shutdown_connections 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa77d23c640 1 --2- 172.21.15.80:0/303857446 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa760004810 0x7fa760004c20 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa77d23c640 1 -- 172.21.15.80:0/303857446 >> 172.21.15.80:0/303857446 conn(0x7fa77806d080 msgr2=0x7fa7780b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa77d23c640 1 -- 172.21.15.80:0/303857446 shutdown_connections 2024-04-25T13:52:14.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.971+0000 7fa77d23c640 1 -- 172.21.15.80:0/303857446 wait complete. 2024-04-25T13:52:14.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 Processor -- start 2024-04-25T13:52:14.968 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 -- start start 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760004810 0x7fa7781fa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa77815fe80 0x7fa7780af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7780afdf0 0x7fa7781fd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780b5560 con 0x7fa7780afdf0 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa7780b53e0 con 0x7fa760004810 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa7780b56e0 con 0x7fa77815fe80 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7780afdf0 0x7fa7781fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7780afdf0 0x7fa7781fd070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40718/0 (socket says 172.21.15.80:40718) 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 -- 172.21.15.80:0/3501653741 learned_addr learned my addr 172.21.15.80:0/3501653741 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa776d9d640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760004810 0x7fa7781fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77659c640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa77815fe80 0x7fa7780af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 -- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa77815fe80 msgr2=0x7fa7780af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa77815fe80 0x7fa7780af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 -- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760004810 msgr2=0x7fa7781fa570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760004810 0x7fa7781fa570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 -- 172.21.15.80:0/3501653741 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7780b0200 con 0x7fa7780afdf0 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77759e640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7780afdf0 0x7fa7781fd070 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa7780701f0 tx=0x7fa7700115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa75f7fe640 1 -- 172.21.15.80:0/3501653741 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770011e10 con 0x7fa7780afdf0 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa75f7fe640 1 -- 172.21.15.80:0/3501653741 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa77001b070 con 0x7fa7780afdf0 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7781e9240 con 0x7fa7780afdf0 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa75f7fe640 1 -- 172.21.15.80:0/3501653741 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700039a0 con 0x7fa7780afdf0 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7781e9800 con 0x7fa7780afdf0 2024-04-25T13:52:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.975+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7781e8fb0 con 0x7fa7780afdf0 2024-04-25T13:52:14.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.979+0000 7fa75f7fe640 1 -- 172.21.15.80:0/3501653741 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa770003b40 con 0x7fa7780afdf0 2024-04-25T13:52:14.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.979+0000 7fa75f7fe640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:14.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.979+0000 7fa75f7fe640 1 -- 172.21.15.80:0/3501653741 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5709+0+0 (secure 0 0 0) 0x7fa77008fad0 con 0x7fa7780afdf0 2024-04-25T13:52:14.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.979+0000 7fa75f7fe640 1 -- 172.21.15.80:0/3501653741 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7fa764076fb0 con 0x7fa7780afdf0 2024-04-25T13:52:14.973 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.979+0000 7fa776d9d640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:14.974 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.979+0000 7fa776d9d640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa76406f430 0x7fa764071940 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa77806b930 tx=0x7fa768009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:14.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:14.983+0000 7fa75f7fe640 1 -- 172.21.15.80:0/3501653741 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa7780b0200 con 0x7fa7780afdf0 2024-04-25T13:52:15.139 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.143+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fa77806a3a0 con 0x7fa7780afdf0 2024-04-25T13:52:15.143 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.147+0000 7fa75f7fe640 1 -- 172.21.15.80:0/3501653741 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) ==== 81+0+34115 (secure 0 0 0) 0x7fa77005c9c0 con 0x7fa7780afdf0 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 189, 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:52:11.293030+0000", 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:52:15.144 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:52:15.145 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 23, 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:15.146 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:15.147 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:15.148 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:15.149 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:15.150 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:52:15.151 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 22, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:48.404349+0000", 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:15.152 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:15.153 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "186", 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:15.154 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:15.155 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:52:15.156 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:15.157 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 23, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:49.039512+0000", 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1027, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:15.158 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:15.159 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "189", 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:15.160 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:15.161 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.162 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:15.163 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.164 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.165 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.166 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.167 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.168 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.169 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:15.170 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.170 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.170 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.170 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.170 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.170 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:52:15.171 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.172 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.173 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:52:15.174 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:15.175 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:52:15.176 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.177 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.178 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.179 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:15.180 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.181 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.182 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.183 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.184 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:52:15.185 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.186 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.187 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.188 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.189 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.191 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.192 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.193 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.194 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.195 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:52:15.196 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.197 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.198 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.199 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:52:15.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:15.202 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:52:15.203 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:52:15.205 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.207 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.209 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:15.210 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.211 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:15.212 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:52:15.213 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:52:15.214 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.215 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa76406f430 msgr2=0x7fa764071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa76406f430 0x7fa764071940 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fa77806b930 tx=0x7fa768009040 comp rx=0 tx=0).stop 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7780afdf0 msgr2=0x7fa7781fd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7780afdf0 0x7fa7781fd070 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7fa7780701f0 tx=0x7fa7700115a0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 shutdown_connections 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa76406f430 0x7fa764071940 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa7780afdf0 0x7fa7781fd070 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa77815fe80 0x7fa7780af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 --2- 172.21.15.80:0/3501653741 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa760004810 0x7fa7781fa570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 >> 172.21.15.80:0/3501653741 conn(0x7fa77806d080 msgr2=0x7fa7780b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 shutdown_connections 2024-04-25T13:52:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.151+0000 7fa77d23c640 1 -- 172.21.15.80:0/3501653741 wait complete. 2024-04-25T13:52:15.218 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-25T13:52:15.218 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:15.218 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:15.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:15.225 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2024-04-25T13:52:15.346 INFO:teuthology.orchestra.run.smithi080.stderr:dd: error writing 'large_file_b': No space left on device 2024-04-25T13:52:15.347 INFO:teuthology.orchestra.run.smithi080.stderr:dd: closing output file 'large_file_b': No space left on device 2024-04-25T13:52:15.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:52:15.348 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:52:15.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f080dd31640 1 Processor -- start 2024-04-25T13:52:15.505 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f080dd31640 1 -- start start 2024-04-25T13:52:15.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f080dd31640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f080815d630 0x7f080815da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f080dd31640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080806cd00 con 0x7f08080cfb60 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f080dd31640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f080806cf90 con 0x7f0808161630 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f080dd31640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0808143920 con 0x7f080815d630 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f0807fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f080815d630 0x7f080815da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f0807fff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f080815d630 0x7f080815da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40730/0 (socket says 172.21.15.80:40730) 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f0807fff640 1 -- 172.21.15.80:0/319054849 learned_addr learned my addr 172.21.15.80:0/319054849 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f0806ffd640 1 --1- 172.21.15.80:0/319054849 >> v1:172.21.15.80:6789/0 conn(0x7f08080cfb60 0x7f08080cff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34702/0 (socket says 172.21.15.80:34702) 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f0807fff640 1 -- 172.21.15.80:0/319054849 >> v1:172.21.15.87:6789/0 conn(0x7f0808161630 legacy=0x7f080806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f0807fff640 1 -- 172.21.15.80:0/319054849 >> v1:172.21.15.80:6789/0 conn(0x7f08080cfb60 legacy=0x7f08080cff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f0807fff640 1 -- 172.21.15.80:0/319054849 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080806d270 con 0x7f080815d630 2024-04-25T13:52:15.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.511+0000 7f08067fc640 1 -- 172.21.15.80:0/319054849 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3798192118 0 0) 0x7f080806cd00 con 0x7f08080cfb60 2024-04-25T13:52:15.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f0807fff640 1 --2- 172.21.15.80:0/319054849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080815d630 0x7f080815da40 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009e50 tx=0x7f07f801f1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05b027c8c98362b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:15.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f08067fc640 1 -- 172.21.15.80:0/319054849 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f80b4070 con 0x7f080815d630 2024-04-25T13:52:15.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f08067fc640 1 -- 172.21.15.80:0/319054849 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07f80af400 con 0x7f080815d630 2024-04-25T13:52:15.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f08067fc640 1 -- 172.21.15.80:0/319054849 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f80af7d0 con 0x7f080815d630 2024-04-25T13:52:15.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f080dd31640 1 -- 172.21.15.80:0/319054849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080815d630 msgr2=0x7f080815da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f080dd31640 1 --2- 172.21.15.80:0/319054849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080815d630 0x7f080815da40 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f07f8009e50 tx=0x7f07f801f1e0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f080dd31640 1 -- 172.21.15.80:0/319054849 shutdown_connections 2024-04-25T13:52:15.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f080dd31640 1 --2- 172.21.15.80:0/319054849 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f080815d630 0x7f080815da40 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f080dd31640 1 -- 172.21.15.80:0/319054849 >> 172.21.15.80:0/319054849 conn(0x7f08080d33a0 msgr2=0x7f08080cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:15.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f080dd31640 1 -- 172.21.15.80:0/319054849 shutdown_connections 2024-04-25T13:52:15.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.515+0000 7f080dd31640 1 -- 172.21.15.80:0/319054849 wait complete. 2024-04-25T13:52:15.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 Processor -- start 2024-04-25T13:52:15.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 -- start start 2024-04-25T13:52:15.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08080cfb60 0x7f08080cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080815d630 0x7f08080ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0808161630 0x7f080815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0808161110 con 0x7f08080cfb60 2024-04-25T13:52:15.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0808161a40 con 0x7f0808161630 2024-04-25T13:52:15.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08080cbb40 con 0x7f080815d630 2024-04-25T13:52:15.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08080cfb60 0x7f08080cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08080cfb60 0x7f08080cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40746/0 (socket says 172.21.15.80:40746) 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 -- 172.21.15.80:0/4191715355 learned_addr learned my addr 172.21.15.80:0/4191715355 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f0807fff640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0808161630 0x7f080815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f0806ffd640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080815d630 0x7f08080ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 -- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080815d630 msgr2=0x7f08080ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080815d630 0x7f08080ca090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 -- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0808161630 msgr2=0x7f080815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0808161630 0x7f080815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 -- 172.21.15.80:0/4191715355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08080ccc40 con 0x7f08080cfb60 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f0806ffd640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080815d630 0x7f08080ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f08077fe640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08080cfb60 0x7f08080cc700 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f080000b410 tx=0x7f080000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f07fffff640 1 -- 172.21.15.80:0/4191715355 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800015030 con 0x7f08080cfb60 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f07fffff640 1 -- 172.21.15.80:0/4191715355 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0800004020 con 0x7f08080cfb60 2024-04-25T13:52:15.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f07fffff640 1 -- 172.21.15.80:0/4191715355 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800004410 con 0x7f08080cfb60 2024-04-25T13:52:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08080c8af0 con 0x7f08080cfb60 2024-04-25T13:52:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.519+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08080c92d0 con 0x7f08080cfb60 2024-04-25T13:52:15.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.523+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080806a3a0 con 0x7f08080cfb60 2024-04-25T13:52:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.523+0000 7f07fffff640 1 -- 172.21.15.80:0/4191715355 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0800018070 con 0x7f08080cfb60 2024-04-25T13:52:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.523+0000 7f07fffff640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07e806f400 0x7f07e8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.523+0000 7f07fffff640 1 -- 172.21.15.80:0/4191715355 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5709+0+0 (secure 0 0 0) 0x7f080008ed70 con 0x7f08080cfb60 2024-04-25T13:52:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.523+0000 7f07fffff640 1 -- 172.21.15.80:0/4191715355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7f07e8076fe0 con 0x7f08080cfb60 2024-04-25T13:52:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.523+0000 7f0806ffd640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07e806f400 0x7f07e8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.523+0000 7f0806ffd640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07e806f400 0x7f07e8071910 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f08080cb610 tx=0x7f07f0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:15.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.527+0000 7f07fffff640 1 -- 172.21.15.80:0/4191715355 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f080005bcf0 con 0x7f08080cfb60 2024-04-25T13:52:15.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.687+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f08080d0c40 con 0x7f08080cfb60 2024-04-25T13:52:15.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f07fffff640 1 -- 172.21.15.80:0/4191715355 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) ==== 81+0+34115 (secure 0 0 0) 0x7f080005b5f0 con 0x7f08080cfb60 2024-04-25T13:52:15.684 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 189, 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:52:11.293030+0000", 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:52:15.685 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 23, 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:52:15.686 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:15.687 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:15.688 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:15.689 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:15.690 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:15.691 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:52:15.692 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 22, 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:48.404349+0000", 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:15.693 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:52:15.694 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "186", 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:15.695 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:15.696 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:15.697 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.698 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 23, 2024-04-25T13:52:15.699 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:49.039512+0000", 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1027, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:15.700 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "189", 2024-04-25T13:52:15.701 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:15.702 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:15.703 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.704 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:15.705 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.706 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:52:15.707 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:52:15.708 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:52:15.709 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:52:15.710 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:15.711 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.712 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.712 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.712 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.712 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.712 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.712 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.712 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.713 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.713 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.713 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.713 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.713 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.713 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.713 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.713 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.714 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.715 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.716 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.717 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.718 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.719 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.720 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.721 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:52:15.722 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.723 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:52:15.724 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.725 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.726 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.727 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.728 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.729 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.730 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.731 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.732 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.733 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.734 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.735 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:52:15.736 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.737 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.738 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:52:15.739 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:52:15.740 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.741 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:52:15.742 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.743 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:52:15.744 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.745 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.746 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.747 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.748 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.749 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:15.750 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.751 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.752 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.753 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:15.754 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:15.755 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.756 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.757 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.758 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.759 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.760 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:15.761 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:52:15.762 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:15.763 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:15.764 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:15.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07e806f400 msgr2=0x7f07e8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07e806f400 0x7f07e8071910 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f08080cb610 tx=0x7f07f0009040 comp rx=0 tx=0).stop 2024-04-25T13:52:15.766 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08080cfb60 msgr2=0x7f08080cc700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08080cfb60 0x7f08080cc700 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f080000b410 tx=0x7f080000b8d0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 shutdown_connections 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f07e806f400 0x7f07e8071910 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0808161630 0x7f080815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f080815d630 0x7f08080ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 --2- 172.21.15.80:0/4191715355 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08080cfb60 0x7f08080cc700 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 >> 172.21.15.80:0/4191715355 conn(0x7f08080d33a0 msgr2=0x7f0808142820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 shutdown_connections 2024-04-25T13:52:15.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.691+0000 7f080dd31640 1 -- 172.21.15.80:0/4191715355 wait complete. 2024-04-25T13:52:15.768 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:15.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1af961640 1 Processor -- start 2024-04-25T13:52:15.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1af961640 1 -- start start 2024-04-25T13:52:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1af961640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1a80d06e0 0x7ff1a806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1af961640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7ff1a806e9e0 0x7ff1a806edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1af961640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a80d0af0 con 0x7ff1a8161f90 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1af961640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a80d0d70 con 0x7ff1a80d06e0 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1af961640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a80d1050 con 0x7ff1a806e9e0 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1acefc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1a80d06e0 0x7ff1a806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1acefc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7ff1a80d06e0 0x7ff1a806e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40762/0 (socket says 172.21.15.80:40762) 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1acefc640 1 -- 172.21.15.80:0/2906540722 learned_addr learned my addr 172.21.15.80:0/2906540722 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1adefe640 1 --2- 172.21.15.80:0/2906540722 >> v2:172.21.15.87:3300/0 conn(0x7ff1a806e9e0 0x7ff1a806edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1acefc640 1 -- 172.21.15.80:0/2906540722 >> v2:172.21.15.87:3300/0 conn(0x7ff1a806e9e0 msgr2=0x7ff1a806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1acefc640 1 --2- 172.21.15.80:0/2906540722 >> v2:172.21.15.87:3300/0 conn(0x7ff1a806e9e0 0x7ff1a806edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1acefc640 1 -- 172.21.15.80:0/2906540722 >> v1:172.21.15.87:6789/0 conn(0x7ff1a8161f90 legacy=0x7ff1a80cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:15.924 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.927+0000 7ff1acefc640 1 -- 172.21.15.80:0/2906540722 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a806dca0 con 0x7ff1a80d06e0 2024-04-25T13:52:15.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1acefc640 1 --2- 172.21.15.80:0/2906540722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a806e4a0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80b4b80 tx=0x7ff19801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5009e322358aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:15.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff19ffff640 1 -- 172.21.15.80:0/2906540722 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1980b0070 con 0x7ff1a80d06e0 2024-04-25T13:52:15.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff19ffff640 1 -- 172.21.15.80:0/2906540722 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1980ab600 con 0x7ff1a80d06e0 2024-04-25T13:52:15.926 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff19ffff640 1 -- 172.21.15.80:0/2906540722 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1980ab9d0 con 0x7ff1a80d06e0 2024-04-25T13:52:15.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1af961640 1 -- 172.21.15.80:0/2906540722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 msgr2=0x7ff1a806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1af961640 1 --2- 172.21.15.80:0/2906540722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a806e4a0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7ff1a80b4b80 tx=0x7ff19801c3d0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1af961640 1 -- 172.21.15.80:0/2906540722 shutdown_connections 2024-04-25T13:52:15.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1af961640 1 --2- 172.21.15.80:0/2906540722 >> v2:172.21.15.87:3300/0 conn(0x7ff1a806e9e0 0x7ff1a806edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1af961640 1 --2- 172.21.15.80:0/2906540722 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a806e4a0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1af961640 1 -- 172.21.15.80:0/2906540722 >> 172.21.15.80:0/2906540722 conn(0x7ff1a806d080 msgr2=0x7ff1a80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:15.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1af961640 1 -- 172.21.15.80:0/2906540722 shutdown_connections 2024-04-25T13:52:15.927 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.931+0000 7ff1af961640 1 -- 172.21.15.80:0/2906540722 wait complete. 2024-04-25T13:52:15.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 Processor -- start 2024-04-25T13:52:15.928 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 -- start start 2024-04-25T13:52:15.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1a806e9e0 0x7ff1a8145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a8163380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a8161f90 0x7ff1a8147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a81447a0 con 0x7ff1a80d06e0 2024-04-25T13:52:15.931 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a8144620 con 0x7ff1a8161f90 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a816b510 con 0x7ff1a806e9e0 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a8163380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a8163380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40778/0 (socket says 172.21.15.80:40778) 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 -- 172.21.15.80:0/303482378 learned_addr learned my addr 172.21.15.80:0/303482378 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1adefe640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a8161f90 0x7ff1a8147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1ad6fd640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1a806e9e0 0x7ff1a8145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 -- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1a806e9e0 msgr2=0x7ff1a8145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1a806e9e0 0x7ff1a8145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 -- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a8161f90 msgr2=0x7ff1a8147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a8161f90 0x7ff1a8147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 -- 172.21.15.80:0/303482378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a8145b70 con 0x7ff1a80d06e0 2024-04-25T13:52:15.932 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1ad6fd640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1a806e9e0 0x7ff1a8145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:15.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1acefc640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a8163380 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7ff19801a000 tx=0x7ff198019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:15.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff19dffb640 1 -- 172.21.15.80:0/303482378 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1980abe60 con 0x7ff1a80d06e0 2024-04-25T13:52:15.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff19dffb640 1 -- 172.21.15.80:0/303482378 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1980b4640 con 0x7ff1a80d06e0 2024-04-25T13:52:15.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff19dffb640 1 -- 172.21.15.80:0/303482378 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1980b0040 con 0x7ff1a80d06e0 2024-04-25T13:52:15.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a8145ff0 con 0x7ff1a80d06e0 2024-04-25T13:52:15.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.935+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a81465b0 con 0x7ff1a80d06e0 2024-04-25T13:52:15.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.939+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a8145d40 con 0x7ff1a80d06e0 2024-04-25T13:52:15.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.939+0000 7ff19dffb640 1 -- 172.21.15.80:0/303482378 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7ff1980ab4c0 con 0x7ff1a80d06e0 2024-04-25T13:52:15.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.939+0000 7ff19dffb640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff18c06f430 0x7ff18c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:15.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.939+0000 7ff19dffb640 1 -- 172.21.15.80:0/303482378 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5709+0+0 (secure 0 0 0) 0x7ff198129740 con 0x7ff1a80d06e0 2024-04-25T13:52:15.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.939+0000 7ff19dffb640 1 -- 172.21.15.80:0/303482378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7ff18c076fb0 con 0x7ff1a80d06e0 2024-04-25T13:52:15.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.939+0000 7ff1ad6fd640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff18c06f430 0x7ff18c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:15.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.939+0000 7ff1ad6fd640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff18c06f430 0x7ff18c071940 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806fa20 tx=0x7ff1a4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:15.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:15.943+0000 7ff19dffb640 1 -- 172.21.15.80:0/303482378 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7ff198016110 con 0x7ff1a80d06e0 2024-04-25T13:52:16.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.131+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1a806a3a0 con 0x7ff1a80d06e0 2024-04-25T13:52:16.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.131+0000 7ff19dffb640 1 -- 172.21.15.80:0/303482378 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7ff1980f67a0 con 0x7ff1a80d06e0 2024-04-25T13:52:16.127 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:16.127 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:52:16.127 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:52:16.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff18c06f430 msgr2=0x7ff18c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff18c06f430 0x7ff18c071940 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806fa20 tx=0x7ff1a4009040 comp rx=0 tx=0).stop 2024-04-25T13:52:16.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 msgr2=0x7ff1a8163380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a8163380 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7ff19801a000 tx=0x7ff198019170 comp rx=0 tx=0).stop 2024-04-25T13:52:16.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 shutdown_connections 2024-04-25T13:52:16.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7ff18c06f430 0x7ff18c071940 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1a8161f90 0x7ff1a8147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7ff1a80d06e0 0x7ff1a8163380 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 --2- 172.21.15.80:0/303482378 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7ff1a806e9e0 0x7ff1a8145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 >> 172.21.15.80:0/303482378 conn(0x7ff1a806d080 msgr2=0x7ff1a80b5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:16.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 shutdown_connections 2024-04-25T13:52:16.130 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.135+0000 7ff1af961640 1 -- 172.21.15.80:0/303482378 wait complete. 2024-04-25T13:52:16.147 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:52:16.318 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":9213,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":25.598564761999999,"mdsmap_epoch":89,"osdmap_epoch":189,"osdmap_epoch_barrier":187,"uptime":30.573373257} 2024-04-25T13:52:16.318 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 9213, "mdsmap_epoch": 89, "osdmap_epoch": 189, "osdmap_epoch_barrier": 187, "rank_uptime": 25.598564762, "state": "up:active", "uptime": 30.573373257, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:52:16.318 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:52:16.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:16.319 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2024-04-25T13:52:16.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:16.327 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2024-04-25T13:52:16.385 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:52:16.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f3139640 1 Processor -- start 2024-04-25T13:52:16.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f3139640 1 -- start start 2024-04-25T13:52:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f3139640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec0b8660 0x7f13ec168f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f3139640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13ec0cfb20 con 0x7f13ec0b9240 2024-04-25T13:52:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f3139640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13ec0b6b70 con 0x7f13ec0d04e0 2024-04-25T13:52:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f3139640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13ec0b6920 con 0x7f13ec0b8660 2024-04-25T13:52:16.583 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f0ed5640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f13ec0b9240 0x7f13ec0cf1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:34712/0 (socket says 172.21.15.80:34712) 2024-04-25T13:52:16.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f0ed5640 1 -- 172.21.15.80:0/3928675777 learned_addr learned my addr 172.21.15.80:0/3928675777 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:16.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13f16d6640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec0b8660 0x7f13ec168f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:16.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2535677351 0 0) 0x7f13ec0cfb20 con 0x7f13ec0b9240 2024-04-25T13:52:16.584 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.587+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13d4003530 con 0x7f13ec0b9240 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1126256593 0 0) 0x7f13d4003530 con 0x7f13ec0b9240 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec0b8660 msgr2=0x7f13ec168f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec0b8660 0x7f13ec168f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 >> v1:172.21.15.87:6789/0 conn(0x7f13ec0d04e0 legacy=0x7f13ec0d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec0b6e50 con 0x7f13ec0b9240 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f13dc001800 con 0x7f13ec0b9240 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 >> v1:172.21.15.80:6789/0 conn(0x7f13ec0b9240 legacy=0x7f13ec0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d40039f0 0x7f13d4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4004810 0x7f13d4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec0b8660 0x7f13d40052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:16.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec0b6920 con 0x7f13d4004810 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13d400bb90 con 0x7f13d40039f0 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13d400bd90 con 0x7f13ec0b8660 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13ebfff640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4004810 0x7f13d4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f0ed5640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d40039f0 0x7f13d4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13ebfff640 1 -- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec0b8660 msgr2=0x7f13d40052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13ebfff640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec0b8660 0x7f13d40052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13ebfff640 1 -- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d40039f0 msgr2=0x7f13d4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13ebfff640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d40039f0 0x7f13d4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13ebfff640 1 -- 172.21.15.80:0/3928675777 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d4003530 con 0x7f13d4004810 2024-04-25T13:52:16.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f0ed5640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d40039f0 0x7f13d4003e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:16.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13ebfff640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4004810 0x7f13d4004c20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f13e0001780 tx=0x7f13e001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d104da6feb02f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:16.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00013a0 con 0x7f13d4004810 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e00052f0 con 0x7f13d4004810 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13eb7fe640 1 -- 172.21.15.80:0/3928675777 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e0003840 con 0x7f13d4004810 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f3139640 1 -- 172.21.15.80:0/3928675777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4004810 msgr2=0x7f13d4004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f3139640 1 --2- 172.21.15.80:0/3928675777 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13d4004810 0x7f13d4004c20 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f13e0001780 tx=0x7f13e001c420 comp rx=0 tx=0).stop 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f16d6640 1 -- 172.21.15.80:0/3928675777 reap_dead start 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f3139640 1 -- 172.21.15.80:0/3928675777 shutdown_connections 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f3139640 1 -- 172.21.15.80:0/3928675777 >> 172.21.15.80:0/3928675777 conn(0x7f13ec0d3ad0 msgr2=0x7f13ec0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f3139640 1 -- 172.21.15.80:0/3928675777 shutdown_connections 2024-04-25T13:52:16.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.591+0000 7f13f3139640 1 -- 172.21.15.80:0/3928675777 wait complete. 2024-04-25T13:52:16.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f3139640 1 Processor -- start 2024-04-25T13:52:16.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f3139640 1 -- start start 2024-04-25T13:52:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f3139640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec1573e0 0x7f13ec06cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f3139640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13ec06d310 0x7f13ec157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f3139640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13ec158000 0x7f13ec09d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f3139640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec169b00 con 0x7f13ec158000 2024-04-25T13:52:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f3139640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec169980 con 0x7f13ec06d310 2024-04-25T13:52:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f3139640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f13ec169c80 con 0x7f13ec1573e0 2024-04-25T13:52:16.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13ec158000 0x7f13ec09d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13ec158000 0x7f13ec09d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:40806/0 (socket says 172.21.15.80:40806) 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 -- 172.21.15.80:0/2555983052 learned_addr learned my addr 172.21.15.80:0/2555983052 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13ebfff640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13ec06d310 0x7f13ec157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f0ed5640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec1573e0 0x7f13ec06cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 -- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec1573e0 msgr2=0x7f13ec06cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec1573e0 0x7f13ec06cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 -- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13ec06d310 msgr2=0x7f13ec157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13ec06d310 0x7f13ec157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 -- 172.21.15.80:0/2555983052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec06c5f0 con 0x7f13ec158000 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13f16d6640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13ec158000 0x7f13ec09d870 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f13e4000f90 tx=0x7f13e400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.595+0000 7f13e97fa640 1 -- 172.21.15.80:0/2555983052 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e400bae0 con 0x7f13ec158000 2024-04-25T13:52:16.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13e97fa640 1 -- 172.21.15.80:0/2555983052 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13e4019070 con 0x7f13ec158000 2024-04-25T13:52:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ec06dc60 con 0x7f13ec158000 2024-04-25T13:52:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13e97fa640 1 -- 172.21.15.80:0/2555983052 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e40039a0 con 0x7f13ec158000 2024-04-25T13:52:16.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ec06e220 con 0x7f13ec158000 2024-04-25T13:52:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13ec06a3a0 con 0x7f13ec158000 2024-04-25T13:52:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13e97fa640 1 -- 172.21.15.80:0/2555983052 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f13e4013070 con 0x7f13ec158000 2024-04-25T13:52:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13e97fa640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13d806f400 0x7f13d8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13e97fa640 1 -- 172.21.15.80:0/2555983052 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5709+0+0 (secure 0 0 0) 0x7f13e408f120 con 0x7f13ec158000 2024-04-25T13:52:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13e97fa640 1 -- 172.21.15.80:0/2555983052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7f13d8076fe0 con 0x7f13ec158000 2024-04-25T13:52:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13f0ed5640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13d806f400 0x7f13d8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:16.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.599+0000 7f13f0ed5640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13d806f400 0x7f13d8071910 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f13dc000c10 tx=0x7f13dc000f80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:16.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.603+0000 7f13e97fa640 1 -- 172.21.15.80:0/2555983052 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f13e405c130 con 0x7f13ec158000 2024-04-25T13:52:16.761 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.767+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f13ec075ad0 con 0x7f13ec158000 2024-04-25T13:52:16.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13e97fa640 1 -- 172.21.15.80:0/2555983052 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) ==== 81+0+34115 (secure 0 0 0) 0x7f13e4064120 con 0x7f13ec158000 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 189, 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:52:11.293030+0000", 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:52:16.765 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 23, 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:52:16.766 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:16.767 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:16.768 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:16.769 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.770 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:52:16.771 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.772 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 22, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:48.404349+0000", 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:16.773 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.774 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "186", 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:16.775 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:16.776 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:16.777 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.778 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 23, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:49.039512+0000", 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1027, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:16.779 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.780 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "189", 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:16.781 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:16.782 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:52:16.783 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:16.784 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:16.785 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.786 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:52:16.787 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.788 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.789 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.790 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.791 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.792 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:16.793 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:16.794 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:16.795 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.796 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.797 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.798 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:52:16.799 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:16.800 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.801 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.802 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:16.803 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:52:16.804 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:16.805 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:52:16.806 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:16.807 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.808 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.809 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:16.810 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.811 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:52:16.812 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:16.813 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:16.814 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.815 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:16.816 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.817 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.818 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:16.819 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:16.820 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:16.821 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:16.822 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.823 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:52:16.824 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:16.825 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:16.826 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:16.827 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:16.828 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:52:16.829 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:16.830 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:52:16.831 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:52:16.832 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:52:16.833 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:52:16.833 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:52:16.833 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:16.833 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:16.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13d806f400 msgr2=0x7f13d8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13d806f400 0x7f13d8071910 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f13dc000c10 tx=0x7f13dc000f80 comp rx=0 tx=0).stop 2024-04-25T13:52:16.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13ec158000 msgr2=0x7f13ec09d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:16.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13ec158000 0x7f13ec09d870 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f13e4000f90 tx=0x7f13e400b480 comp rx=0 tx=0).stop 2024-04-25T13:52:16.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 shutdown_connections 2024-04-25T13:52:16.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f13d806f400 0x7f13d8071910 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f13ec158000 0x7f13ec09d870 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13ec06d310 0x7f13ec157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 --2- 172.21.15.80:0/2555983052 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f13ec1573e0 0x7f13ec06cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:16.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 >> 172.21.15.80:0/2555983052 conn(0x7f13ec0d3ad0 msgr2=0x7f13ec070f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:16.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 shutdown_connections 2024-04-25T13:52:16.835 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:16.771+0000 7f13f3139640 1 -- 172.21.15.80:0/2555983052 wait complete. 2024-04-25T13:52:16.836 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:52:21.836 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:52:21.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c3a076640 1 Processor -- start 2024-04-25T13:52:21.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c3a076640 1 -- start start 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c3a076640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c34161f90 0x7f0c340cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c3a076640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c340d0510 0x7f0c3415e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c3a076640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c3415ea70 0x7f0c3415ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c3a076640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0c340d0d70 con 0x7f0c340d0510 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c3a076640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0c340d1050 con 0x7f0c34161f90 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c3a076640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c3406dca0 con 0x7f0c3415ea70 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c32ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c340d0510 0x7f0c3415e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c32ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c340d0510 0x7f0c3415e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49604/0 (socket says 172.21.15.80:49604) 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c32ffd640 1 -- 172.21.15.80:0/3852758181 learned_addr learned my addr 172.21.15.80:0/3852758181 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c33fff640 1 --2- 172.21.15.80:0/3852758181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c3415ea70 0x7f0c3415ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c337fe640 1 --2- 172.21.15.80:0/3852758181 >> v2:172.21.15.87:3300/0 conn(0x7f0c34161f90 0x7f0c340cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c32ffd640 1 -- 172.21.15.80:0/3852758181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c3415ea70 msgr2=0x7f0c3415ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:22.000 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c32ffd640 1 --2- 172.21.15.80:0/3852758181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c3415ea70 0x7f0c3415ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c32ffd640 1 -- 172.21.15.80:0/3852758181 >> v2:172.21.15.87:3300/0 conn(0x7f0c34161f90 msgr2=0x7f0c340cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:22.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c32ffd640 1 --2- 172.21.15.80:0/3852758181 >> v2:172.21.15.87:3300/0 conn(0x7f0c34161f90 0x7f0c340cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.003+0000 7f0c32ffd640 1 -- 172.21.15.80:0/3852758181 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3406df80 con 0x7f0c340d0510 2024-04-25T13:52:22.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c32ffd640 1 --2- 172.21.15.80:0/3852758181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c340d0510 0x7f0c3415e530 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f0c340d17b0 tx=0x7f0c2801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba34eac3e770f4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:22.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c327fc640 1 -- 172.21.15.80:0/3852758181 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c280b1070 con 0x7f0c340d0510 2024-04-25T13:52:22.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c327fc640 1 -- 172.21.15.80:0/3852758181 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c280a8050 con 0x7f0c340d0510 2024-04-25T13:52:22.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c327fc640 1 -- 172.21.15.80:0/3852758181 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c280ac860 con 0x7f0c340d0510 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 -- 172.21.15.80:0/3852758181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c340d0510 msgr2=0x7f0c3415e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 --2- 172.21.15.80:0/3852758181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c340d0510 0x7f0c3415e530 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f0c340d17b0 tx=0x7f0c2801c460 comp rx=0 tx=0).stop 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 -- 172.21.15.80:0/3852758181 shutdown_connections 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 --2- 172.21.15.80:0/3852758181 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c3415ea70 0x7f0c3415ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 --2- 172.21.15.80:0/3852758181 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c340d0510 0x7f0c3415e530 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 --2- 172.21.15.80:0/3852758181 >> v2:172.21.15.87:3300/0 conn(0x7f0c34161f90 0x7f0c340cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 -- 172.21.15.80:0/3852758181 >> 172.21.15.80:0/3852758181 conn(0x7f0c3406d080 msgr2=0x7f0c340b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 -- 172.21.15.80:0/3852758181 shutdown_connections 2024-04-25T13:52:22.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.007+0000 7f0c3a076640 1 -- 172.21.15.80:0/3852758181 wait complete. 2024-04-25T13:52:22.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 Processor -- start 2024-04-25T13:52:22.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 -- start start 2024-04-25T13:52:22.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c340d0510 0x7f0c34163320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:22.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c3415ea70 0x7f0c341efa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:22.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c34161f90 0x7f0c341eff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:22.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341447a0 con 0x7f0c34161f90 2024-04-25T13:52:22.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34144620 con 0x7f0c3415ea70 2024-04-25T13:52:22.007 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c340d0920 con 0x7f0c340d0510 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c34161f90 0x7f0c341eff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c34161f90 0x7f0c341eff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49612/0 (socket says 172.21.15.80:49612) 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 -- 172.21.15.80:0/773041107 learned_addr learned my addr 172.21.15.80:0/773041107 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c337fe640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c340d0510 0x7f0c34163320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c32ffd640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c3415ea70 0x7f0c341efa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 -- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c340d0510 msgr2=0x7f0c34163320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c340d0510 0x7f0c34163320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 -- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c3415ea70 msgr2=0x7f0c341efa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c3415ea70 0x7f0c341efa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 -- 172.21.15.80:0/773041107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c34142d70 con 0x7f0c34161f90 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c33fff640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c34161f90 0x7f0c341eff50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c007d10 tx=0x7f0c2c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c1bfff640 1 -- 172.21.15.80:0/773041107 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c014420 con 0x7f0c34161f90 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c1bfff640 1 -- 172.21.15.80:0/773041107 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c2c019070 con 0x7f0c34161f90 2024-04-25T13:52:22.008 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c34143050 con 0x7f0c34161f90 2024-04-25T13:52:22.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c1bfff640 1 -- 172.21.15.80:0/773041107 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c014800 con 0x7f0c34161f90 2024-04-25T13:52:22.009 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.011+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341f0550 con 0x7f0c34161f90 2024-04-25T13:52:22.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.015+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c34163a00 con 0x7f0c34161f90 2024-04-25T13:52:22.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.015+0000 7f0c1bfff640 1 -- 172.21.15.80:0/773041107 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0c2c0149a0 con 0x7f0c34161f90 2024-04-25T13:52:22.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.015+0000 7f0c1bfff640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c2006f400 0x7f0c20071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:22.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.015+0000 7f0c1bfff640 1 -- 172.21.15.80:0/773041107 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(189..189 src has 1..189) ==== 5709+0+0 (secure 0 0 0) 0x7f0c2c0903b0 con 0x7f0c34161f90 2024-04-25T13:52:22.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.015+0000 7f0c1bfff640 1 -- 172.21.15.80:0/773041107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=190}) -- 0x7f0c20076fe0 con 0x7f0c34161f90 2024-04-25T13:52:22.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.015+0000 7f0c337fe640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c2006f400 0x7f0c20071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:22.010 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.015+0000 7f0c337fe640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c2006f400 0x7f0c20071910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0c3406fa20 tx=0x7f0c24008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:22.013 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.019+0000 7f0c1bfff640 1 -- 172.21.15.80:0/773041107 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c2c05d4a0 con 0x7f0c34161f90 2024-04-25T13:52:22.172 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.179+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7f0c3406a3a0 con 0x7f0c34161f90 2024-04-25T13:52:22.175 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c1bfff640 1 -- 172.21.15.80:0/773041107 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v189) ==== 81+0+34115 (secure 0 0 0) 0x7f0c2c05cda0 con 0x7f0c34161f90 2024-04-25T13:52:22.176 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:22.176 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 189, 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:52:11.293030+0000", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:52:22.177 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 23, 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:52:22.178 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:22.179 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:22.180 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:22.181 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:22.182 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:22.183 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:22.184 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 22, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:48.404349+0000", 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:22.185 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:52:22.186 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "186", 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:22.187 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:22.188 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:22.189 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.190 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 23, 2024-04-25T13:52:22.191 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:49.039512+0000", 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1027, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool,full,full_quota", 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:52:22.192 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "189", 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:22.193 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:22.194 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:22.195 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:22.196 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:52:22.197 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:22.198 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.199 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.200 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:52:22.201 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:52:22.202 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:22.203 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.204 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:22.205 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.206 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.206 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:52:22.206 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:22.206 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.206 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.207 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.208 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.209 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.210 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.211 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.212 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.213 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:52:22.214 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:22.215 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:52:22.216 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:52:22.217 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:52:22.218 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:52:22.219 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:22.220 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.221 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.222 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.223 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.224 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:22.225 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:22.226 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.227 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.228 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:52:22.229 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.230 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.231 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.232 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.233 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.234 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.235 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:52:22.236 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:22.237 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:22.238 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:22.239 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.240 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.241 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.242 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:22.243 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.244 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:22.245 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:22.246 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:22.247 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:52:22.248 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:52:22.249 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:52:22.250 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:52:22.250 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:52:22.250 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:52:22.250 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:52:22.250 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:52:22.250 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:22.250 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c2006f400 msgr2=0x7f0c20071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c2006f400 0x7f0c20071910 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f0c3406fa20 tx=0x7f0c24008040 comp rx=0 tx=0).stop 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c34161f90 msgr2=0x7f0c341eff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c34161f90 0x7f0c341eff50 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c007d10 tx=0x7f0c2c00e4b0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 shutdown_connections 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c2006f400 0x7f0c20071910 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c34161f90 0x7f0c341eff50 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c3415ea70 0x7f0c341efa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 --2- 172.21.15.80:0/773041107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c340d0510 0x7f0c34163320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 >> 172.21.15.80:0/773041107 conn(0x7f0c3406d080 msgr2=0x7f0c340b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 shutdown_connections 2024-04-25T13:52:22.252 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:22.183+0000 7f0c3a076640 1 -- 172.21.15.80:0/773041107 wait complete. 2024-04-25T13:52:22.253 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2024-04-25T13:52:27.254 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:52:27.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd353cf640 1 Processor -- start 2024-04-25T13:52:27.413 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd353cf640 1 -- start start 2024-04-25T13:52:27.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd353cf640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdd300b4850 0x7fdd300d0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd353cf640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fdd300d0940 0x7fdd300d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd353cf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300d1d80 0x7fdd300d2190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd353cf640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fdd300b4c60 con 0x7fdd300b4850 2024-04-25T13:52:27.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd353cf640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fdd300d26d0 con 0x7fdd300d0940 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd353cf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdd300b93e0 con 0x7fdd300d1d80 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2effd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdd300b4850 0x7fdd300d0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2effd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fdd300b4850 0x7fdd300d0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49626/0 (socket says 172.21.15.80:49626) 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2effd640 1 -- 172.21.15.80:0/1503885749 learned_addr learned my addr 172.21.15.80:0/1503885749 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2f7fe640 1 --2- 172.21.15.80:0/1503885749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300d1d80 0x7fdd300d2190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2e7fc640 1 --2- 172.21.15.80:0/1503885749 >> v2:172.21.15.87:3300/0 conn(0x7fdd300d0940 0x7fdd300d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2effd640 1 -- 172.21.15.80:0/1503885749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300d1d80 msgr2=0x7fdd300d2190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2effd640 1 --2- 172.21.15.80:0/1503885749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300d1d80 0x7fdd300d2190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2effd640 1 -- 172.21.15.80:0/1503885749 >> v2:172.21.15.87:3300/0 conn(0x7fdd300d0940 msgr2=0x7fdd300d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2effd640 1 --2- 172.21.15.80:0/1503885749 >> v2:172.21.15.87:3300/0 conn(0x7fdd300d0940 0x7fdd300d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.419+0000 7fdd2effd640 1 -- 172.21.15.80:0/1503885749 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd300cf220 con 0x7fdd300b4850 2024-04-25T13:52:27.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd2effd640 1 --2- 172.21.15.80:0/1503885749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd300b4850 0x7fdd300d0400 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fdd30072ac0 tx=0x7fdd2001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e8d9d81ecae2ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:27.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd2dffb640 1 -- 172.21.15.80:0/1503885749 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd200b0070 con 0x7fdd300b4850 2024-04-25T13:52:27.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd2dffb640 1 -- 172.21.15.80:0/1503885749 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd200ab600 con 0x7fdd300b4850 2024-04-25T13:52:27.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd2dffb640 1 -- 172.21.15.80:0/1503885749 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd200ab9d0 con 0x7fdd300b4850 2024-04-25T13:52:27.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 -- 172.21.15.80:0/1503885749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd300b4850 msgr2=0x7fdd300d0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 --2- 172.21.15.80:0/1503885749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd300b4850 0x7fdd300d0400 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fdd30072ac0 tx=0x7fdd2001c500 comp rx=0 tx=0).stop 2024-04-25T13:52:27.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 -- 172.21.15.80:0/1503885749 shutdown_connections 2024-04-25T13:52:27.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 --2- 172.21.15.80:0/1503885749 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300d1d80 0x7fdd300d2190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 --2- 172.21.15.80:0/1503885749 >> v2:172.21.15.87:3300/0 conn(0x7fdd300d0940 0x7fdd300d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 --2- 172.21.15.80:0/1503885749 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd300b4850 0x7fdd300d0400 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 -- 172.21.15.80:0/1503885749 >> 172.21.15.80:0/1503885749 conn(0x7fdd300d3a60 msgr2=0x7fdd300d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:27.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 -- 172.21.15.80:0/1503885749 shutdown_connections 2024-04-25T13:52:27.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.423+0000 7fdd353cf640 1 -- 172.21.15.80:0/1503885749 wait complete. 2024-04-25T13:52:27.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 Processor -- start 2024-04-25T13:52:27.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 -- start start 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300b4850 0x7fdd301f7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd300d1d80 0x7fdd301594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd30159d40 0x7fdd30162c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fdd300b3ad0 con 0x7fdd30159d40 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fdd300b3950 con 0x7fdd300d1d80 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fdd300b3c50 con 0x7fdd300b4850 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd30159d40 0x7fdd30162c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd30159d40 0x7fdd30162c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49628/0 (socket says 172.21.15.80:49628) 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 -- 172.21.15.80:0/3545168888 learned_addr learned my addr 172.21.15.80:0/3545168888 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2effd640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300b4850 0x7fdd301f7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 -- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300b4850 msgr2=0x7fdd301f7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300b4850 0x7fdd301f7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 -- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd300d1d80 msgr2=0x7fdd301594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2e7fc640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd300d1d80 0x7fdd301594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd300d1d80 0x7fdd301594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 -- 172.21.15.80:0/3545168888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd30161390 con 0x7fdd30159d40 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2effd640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300b4850 0x7fdd301f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd2f7fe640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd30159d40 0x7fdd30162c70 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fdd28007cf0 tx=0x7fdd2800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd177fe640 1 -- 172.21.15.80:0/3545168888 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd28014420 con 0x7fdd30159d40 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd177fe640 1 -- 172.21.15.80:0/3545168888 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd28019070 con 0x7fdd30159d40 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd30161ed0 con 0x7fdd30159d40 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd177fe640 1 -- 172.21.15.80:0/3545168888 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd28014820 con 0x7fdd30159d40 2024-04-25T13:52:27.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd30162490 con 0x7fdd30159d40 2024-04-25T13:52:27.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.427+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd301f7b30 con 0x7fdd30159d40 2024-04-25T13:52:27.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.431+0000 7fdd177fe640 1 -- 172.21.15.80:0/3545168888 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fdd280149c0 con 0x7fdd30159d40 2024-04-25T13:52:27.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.431+0000 7fdd177fe640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdd1c06f430 0x7fdd1c071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.431+0000 7fdd177fe640 1 -- 172.21.15.80:0/3545168888 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7fdd2808fa90 con 0x7fdd30159d40 2024-04-25T13:52:27.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.431+0000 7fdd2effd640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdd1c06f430 0x7fdd1c071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.431+0000 7fdd2effd640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdd1c06f430 0x7fdd1c071940 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdd200ad000 tx=0x7fdd200a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:27.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.435+0000 7fdd177fe640 1 -- 172.21.15.80:0/3545168888 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fdd301f7b30 con 0x7fdd30159d40 2024-04-25T13:52:27.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.595+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fdd3006a3a0 con 0x7fdd30159d40 2024-04-25T13:52:27.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.599+0000 7fdd177fe640 1 -- 172.21.15.80:0/3545168888 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v190) ==== 81+0+34096 (secure 0 0 0) 0x7fdd2805c980 con 0x7fdd30159d40 2024-04-25T13:52:27.594 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 190, 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:52:26.300379+0000", 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:52:27.595 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 23, 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.596 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:52:27.597 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:27.598 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:27.599 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:27.600 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:27.601 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 22, 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:52:27.602 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:48.404349+0000", 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:52:27.603 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "186", 2024-04-25T13:52:27.604 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:27.605 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.606 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:52:27.607 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:27.608 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 23, 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:49.039512+0000", 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:27.609 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:52:27.610 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "190", 2024-04-25T13:52:27.611 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:27.612 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:27.613 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:27.614 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.615 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:27.616 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.617 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:52:27.618 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:27.619 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:27.620 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:27.621 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.622 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:52:27.623 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.624 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.625 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.626 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:27.627 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.628 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:27.629 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:27.630 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.631 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:52:27.632 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.633 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.634 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:27.635 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.636 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.637 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.638 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.639 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.640 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:27.641 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.642 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.643 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.644 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.645 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:27.646 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:27.647 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:27.648 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.649 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.650 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:27.651 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:27.652 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.653 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.654 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.655 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:27.656 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:27.657 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:27.658 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.659 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:27.660 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:27.661 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:52:27.662 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:52:27.663 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:52:27.664 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:52:27.664 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:52:27.664 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:52:27.664 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:52:27.664 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:27.664 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:27.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdd1c06f430 msgr2=0x7fdd1c071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdd1c06f430 0x7fdd1c071940 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fdd200ad000 tx=0x7fdd200a6000 comp rx=0 tx=0).stop 2024-04-25T13:52:27.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd30159d40 msgr2=0x7fdd30162c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd30159d40 0x7fdd30162c70 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fdd28007cf0 tx=0x7fdd2800e4b0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 shutdown_connections 2024-04-25T13:52:27.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fdd1c06f430 0x7fdd1c071940 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fdd30159d40 0x7fdd30162c70 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdd300d1d80 0x7fdd301594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 --2- 172.21.15.80:0/3545168888 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fdd300b4850 0x7fdd301f7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 >> 172.21.15.80:0/3545168888 conn(0x7fdd300d3a60 msgr2=0x7fdd300b1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:27.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 shutdown_connections 2024-04-25T13:52:27.666 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.603+0000 7fdd353cf640 1 -- 172.21.15.80:0/3545168888 wait complete. 2024-04-25T13:52:27.666 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2024-04-25T13:52:27.667 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2024-04-25T13:52:27.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2b138d640 1 Processor -- start 2024-04-25T13:52:27.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2b138d640 1 -- start start 2024-04-25T13:52:27.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2b138d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 0x7fb2ac0cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2b138d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fb2ac0d0510 0x7fb2ac15e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2b138d640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2ac0d0d70 con 0x7fb2ac15ea70 2024-04-25T13:52:27.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2b138d640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fb2ac0d1050 con 0x7fb2ac0d0510 2024-04-25T13:52:27.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2b138d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ac06dca0 con 0x7fb2ac161f90 2024-04-25T13:52:27.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2ab7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb2ac15ea70 0x7fb2ac15ee80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55612/0 (socket says 172.21.15.80:55612) 2024-04-25T13:52:27.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2ab7fe640 1 -- 172.21.15.80:0/2740624901 learned_addr learned my addr 172.21.15.80:0/2740624901 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2aaffd640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 0x7fb2ac0cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2aa7fc640 1 --2- 172.21.15.80:0/2740624901 >> v2:172.21.15.87:3300/0 conn(0x7fb2ac0d0510 0x7fb2ac15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 104018934 0 0) 0x7fb2ac0d0d70 con 0x7fb2ac15ea70 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb294003530 con 0x7fb2ac15ea70 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3024775322 0 0) 0x7fb294003530 con 0x7fb2ac15ea70 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 msgr2=0x7fb2ac0cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 0x7fb2ac0cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 >> v2:172.21.15.87:3300/0 conn(0x7fb2ac0d0510 msgr2=0x7fb2ac15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 --2- 172.21.15.80:0/2740624901 >> v2:172.21.15.87:3300/0 conn(0x7fb2ac0d0510 0x7fb2ac15e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ac06df80 con 0x7fb2ac15ea70 2024-04-25T13:52:27.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fb2a4001a00 con 0x7fb2ac15ea70 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 >> v1:172.21.15.80:6789/0 conn(0x7fb2ac15ea70 legacy=0x7fb2ac15ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2940039c0 0x7fb294003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 0x7fb294004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb294004d50 0x7fb294005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac0d1050 con 0x7fb294004d50 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac06dca0 con 0x7fb2940039c0 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb29400ba50 con 0x7fb2ac161f90 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2ab7fe640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb294004d50 0x7fb294005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2aaffd640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2940039c0 0x7fb294003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2aa7fc640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 0x7fb294004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2ab7fe640 1 -- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 msgr2=0x7fb294004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2ab7fe640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 0x7fb294004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2ab7fe640 1 -- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2940039c0 msgr2=0x7fb294003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:27.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2ab7fe640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2940039c0 0x7fb294003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2ab7fe640 1 -- 172.21.15.80:0/2740624901 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb294003530 con 0x7fb294004d50 2024-04-25T13:52:27.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.827+0000 7fb2aa7fc640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb2ac161f90 0x7fb294004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2ab7fe640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb294004d50 0x7fb294005160 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400a880 tx=0x7fb2a401fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e7d4302a980ab44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a400a4a0 con 0x7fb294004d50 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb2a40ad020 con 0x7fb294004d50 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2a9ffb640 1 -- 172.21.15.80:0/2740624901 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a40a9740 con 0x7fb294004d50 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2b138d640 1 -- 172.21.15.80:0/2740624901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb294004d50 msgr2=0x7fb294005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2b138d640 1 --2- 172.21.15.80:0/2740624901 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb294004d50 0x7fb294005160 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400a880 tx=0x7fb2a401fb00 comp rx=0 tx=0).stop 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2ab7fe640 1 -- 172.21.15.80:0/2740624901 reap_dead start 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2b138d640 1 -- 172.21.15.80:0/2740624901 shutdown_connections 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2b138d640 1 -- 172.21.15.80:0/2740624901 >> 172.21.15.80:0/2740624901 conn(0x7fb2ac06d080 msgr2=0x7fb2ac0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2b138d640 1 -- 172.21.15.80:0/2740624901 shutdown_connections 2024-04-25T13:52:27.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.831+0000 7fb2b138d640 1 -- 172.21.15.80:0/2740624901 wait complete. 2024-04-25T13:52:27.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 Processor -- start 2024-04-25T13:52:27.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 -- start start 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb294004d50 0x7fb2ac0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2ac146730 0x7fb2ac1f4330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac146f00 0x7fb2ac147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac162520 con 0x7fb2ac146730 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ac1623a0 con 0x7fb2ac146f00 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ac144620 con 0x7fb294004d50 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2ac146730 0x7fb2ac1f4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2ac146730 0x7fb2ac1f4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49650/0 (socket says 172.21.15.80:49650) 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 -- 172.21.15.80:0/2247553791 learned_addr learned my addr 172.21.15.80:0/2247553791 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aaffd640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb294004d50 0x7fb2ac0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2ab7fe640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac146f00 0x7fb2ac147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 -- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb294004d50 msgr2=0x7fb2ac0b2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb294004d50 0x7fb2ac0b2e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 -- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac146f00 msgr2=0x7fb2ac147310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac146f00 0x7fb2ac147310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 -- 172.21.15.80:0/2247553791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ac147850 con 0x7fb2ac146730 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2aa7fc640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2ac146730 0x7fb2ac1f4330 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb29c010890 tx=0x7fb29c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2937fe640 1 -- 172.21.15.80:0/2247553791 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c00f040 con 0x7fb2ac146730 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2937fe640 1 -- 172.21.15.80:0/2247553791 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb29c004220 con 0x7fb2ac146730 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ac13de80 con 0x7fb2ac146730 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2937fe640 1 -- 172.21.15.80:0/2247553791 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb29c010e80 con 0x7fb2ac146730 2024-04-25T13:52:27.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ac13e660 con 0x7fb2ac146730 2024-04-25T13:52:27.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.835+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2ac13e0f0 con 0x7fb2ac146730 2024-04-25T13:52:27.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.839+0000 7fb2937fe640 1 -- 172.21.15.80:0/2247553791 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb29c00e960 con 0x7fb2ac146730 2024-04-25T13:52:27.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.839+0000 7fb2937fe640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb29806f400 0x7fb298071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:27.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.839+0000 7fb2937fe640 1 -- 172.21.15.80:0/2247553791 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7fb29c08f900 con 0x7fb2ac146730 2024-04-25T13:52:27.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.839+0000 7fb2aaffd640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb29806f400 0x7fb298071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:27.834 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.839+0000 7fb2aaffd640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb29806f400 0x7fb298071910 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac06fa20 tx=0x7fb2a0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:27.837 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.843+0000 7fb2937fe640 1 -- 172.21.15.80:0/2247553791 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb2ac147850 con 0x7fb2ac146730 2024-04-25T13:52:27.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:27.999+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) -- 0x7fb2ac06a3a0 con 0x7fb2ac146730 2024-04-25T13:52:27.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.003+0000 7fb2937fe640 1 -- 172.21.15.80:0/2247553791 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v190) ==== 81+0+34096 (secure 0 0 0) 0x7fb29c018030 con 0x7fb2ac146730 2024-04-25T13:52:27.998 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:27.998 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:27.998 INFO:teuthology.orchestra.run.smithi080.stdout: "epoch": 190, 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "created": "2024-04-25T13:40:11.435899+0000", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "modified": "2024-04-25T13:52:26.300379+0000", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "last_up_change": "2024-04-25T13:40:17.485337+0000", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "last_in_change": "2024-04-25T13:40:14.377362+0000", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_num": 5799936, 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_set": [ 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "pglog_hardlimit", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "purged_snapdirs", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_deletes", 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: "sortbitwise" 2024-04-25T13:52:27.999 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_version": 4, 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "full_ratio": 0.69999998807907104, 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "backfillfull_ratio": 0.60000002384185791, 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "nearfull_ratio": 0.60000002384185791, 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_snapshot": "", 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_max": 23, 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "max_osd": 8, 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "require_min_compat_client": "luminous", 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "min_compat_client": "jewel", 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "require_osd_release": "squid", 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "allow_crimson": false, 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "pools": [ 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.000 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 1, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": ".mgr", 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:40:18.139319+0000", 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:28.001 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 1, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 1, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 1, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 1, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 1, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "110", 2024-04-25T13:52:28.002 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:28.003 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:28.004 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_max": 32, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 1 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.005 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "mgr": {} 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 8, 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 8, 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 0.25, 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.006 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 22, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_metadata", 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:48.404349+0000", 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:28.007 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 32, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 32, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 32, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 32, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 32, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:28.008 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "186", 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 0, 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:28.009 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:28.010 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_bias": 4, 2024-04-25T13:52:28.011 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 16, 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "recovery_priority": 5 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "metadata": "cephfs" 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 2, 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 2, 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 2, 2024-04-25T13:52:28.012 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 2, 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "pool": 23, 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_name": "cephfs_data", 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "create_time": "2024-04-25T13:51:49.039512+0000", 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "flags": 1, 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "flags_names": "hashpspool", 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "type": 1, 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "size": 2, 2024-04-25T13:52:28.013 INFO:teuthology.orchestra.run.smithi080.stdout: "min_size": 1, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_rule": 0, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_count": 0, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_target": 0, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_barrier": 0, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "object_hash": 2, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_autoscale_mode": "off", 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num": 64, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num": 64, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_placement_num_target": 64, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_target": 64, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_pending": 64, 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "last_pg_merge_meta": { 2024-04-25T13:52:28.014 INFO:teuthology.orchestra.run.smithi080.stdout: "source_pgid": "0.0", 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "ready_epoch": 0, 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_started": 0, 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "last_epoch_clean": 0, 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "source_version": "0'0", 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "target_version": "0'0" 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "last_change": "190", 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend": "0", 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "last_force_op_resend_preluminous": "0", 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "auid": 0, 2024-04-25T13:52:28.015 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_mode": "selfmanaged", 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_seq": 0, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "snap_epoch": 0, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "pool_snaps": [], 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps": "[]", 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_bytes": 33554432, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "quota_max_objects": 0, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "tiers": [], 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "tier_of": -1, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "read_tier": -1, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "write_tier": -1, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_mode": "none", 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_bytes": 0, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "target_max_objects": 0, 2024-04-25T13:52:28.016 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_flush_age": 0, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "cache_min_evict_age": 0, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profile": "", 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_params": { 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "none" 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_period": 0, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_count": 0, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "use_gmt_hitset": true, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "min_read_recency_for_promote": 0, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "min_write_recency_for_promote": 0, 2024-04-25T13:52:28.017 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_grade_decay_rate": 0, 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "hit_set_search_last_n": 0, 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "grade_table": [], 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "stripe_width": 0, 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "expected_num_objects": 0, 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "fast_read": false, 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "options": { 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_num_min": 64, 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "target_size_ratio": 0.89999997615814209 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.018 INFO:teuthology.orchestra.run.smithi080.stdout: "application_metadata": { 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "cephfs": { 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "data": "cephfs" 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "read_balance": { 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "score_type": "Fair distribution", 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "score_acting": 1.3799999952316284, 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "score_stable": 1.3799999952316284, 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "optimal_score": 1, 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_acting": 1.3799999952316284, 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "raw_score_stable": 1.3799999952316284, 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity_weighted": 1, 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity": 1, 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: "average_primary_affinity_weighted": 1 2024-04-25T13:52:28.019 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "osds": [ 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:28.020 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6808", 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.021 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6810", 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6812", 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.022 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6814", 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6819", 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.023 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6821", 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6816", 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.024 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6817", 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2104747008 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6810/2104747008", 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6814/2104747008", 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6821/2104747008", 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6817/2104747008", 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.025 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:28.026 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6800", 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6801", 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.027 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6802", 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6803", 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:28.028 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6806", 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6807", 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.029 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6804", 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6805", 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1560454941 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6801/1560454941", 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6803/1560454941", 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6807/1560454941", 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6805/1560454941", 2024-04-25T13:52:28.030 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:28.031 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6809", 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6811", 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.032 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6813", 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6815", 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.033 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6822", 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6827", 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.034 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6818", 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6820", 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 968203588 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6811/968203588", 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6815/968203588", 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6827/968203588", 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6820/968203588", 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:28.035 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:28.036 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6823", 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6824", 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.037 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6825", 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6826", 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.038 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6830", 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6831", 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:28.039 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6828", 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.80:6829", 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 3781596558 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.80:6824/3781596558", 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.80:6826/3781596558", 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.80:6831/3781596558", 2024-04-25T13:52:28.040 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.80:6829/3781596558", 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:28.041 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6816", 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6817", 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.042 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6818", 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6819", 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.043 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6822", 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6823", 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.044 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6820", 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6821", 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 2235735313 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.045 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6817/2235735313", 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6819/2235735313", 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6823/2235735313", 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6821/2235735313", 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:28.046 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6800", 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.047 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6801", 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6802", 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6803", 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:28.048 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6806", 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6807", 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.049 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6804", 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6805", 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 149615972 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6801/149615972", 2024-04-25T13:52:28.050 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6803/149615972", 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6807/149615972", 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6805/149615972", 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:28.051 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6824", 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6825", 2024-04-25T13:52:28.052 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6826", 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6827", 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:28.053 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6830", 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6831", 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:28.054 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6828", 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6829", 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 1131074035 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.055 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6825/1131074035", 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6827/1131074035", 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6831/1131074035", 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6829/1131074035", 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2024-04-25T13:52:28.056 INFO:teuthology.orchestra.run.smithi080.stdout: "up": 1, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "in": 1, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "weight": 1, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_affinity": 1, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_begin": 0, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "last_clean_end": 0, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "up_from": 13, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "up_thru": 185, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "down_at": 0, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "lost_at": 0, 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addrs": { 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.057 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6808", 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6809", 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addrs": { 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.058 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6810", 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6811", 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addrs": { 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6814", 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.059 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6815", 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addrs": { 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "addrvec": [ 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v2", 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6812", 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.060 INFO:teuthology.orchestra.run.smithi080.stdout: "type": "v1", 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "addr": "172.21.15.87:6813", 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "nonce": 710369794 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "public_addr": "172.21.15.87:6809/710369794", 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "cluster_addr": "172.21.15.87:6811/710369794", 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_back_addr": "172.21.15.87:6815/710369794", 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "heartbeat_front_addr": "172.21.15.87:6813/710369794", 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "state": [ 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "exists", 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "up" 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: ] 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: "osd_xinfo": [ 2024-04-25T13:52:28.061 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 0, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.507445+0000", 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 1, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:28.062 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.510377+0000", 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 2, 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.576727+0000", 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.063 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 3, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.545819+0000", 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 4, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:28.064 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.564232+0000", 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 5, 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.472321+0000", 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 6, 2024-04-25T13:52:28.065 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.534056+0000", 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: { 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "osd": 7, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "down_stamp": "0.000000", 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_probability": 0, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "laggy_interval": 0, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "features": 4540701547738038271, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "old_weight": 0, 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "last_purged_snaps_scrub": "2024-04-25T13:40:15.538847+0000", 2024-04-25T13:52:28.066 INFO:teuthology.orchestra.run.smithi080.stdout: "dead_epoch": 0 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: ], 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap": [], 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_items": [], 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_upmap_primaries": [], 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "pg_temp": [], 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "primary_temp": [], 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "blocklist": {}, 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "range_blocklist": {}, 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "erasure_code_profiles": { 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "default": { 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "crush-failure-domain": "osd", 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "k": "2", 2024-04-25T13:52:28.067 INFO:teuthology.orchestra.run.smithi080.stdout: "m": "1", 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "plugin": "jerasure", 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "technique": "reed_sol_van" 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: }, 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "removed_snaps_queue": [], 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "new_removed_snaps": [], 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "new_purged_snaps": [], 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "crush_node_flags": {}, 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "device_class_flags": {}, 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode": { 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_enabled": false, 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_bucket_count": 0, 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "degraded_stretch_mode": 0, 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "recovering_stretch_mode": 0, 2024-04-25T13:52:28.068 INFO:teuthology.orchestra.run.smithi080.stdout: "stretch_mode_bucket": 0 2024-04-25T13:52:28.069 INFO:teuthology.orchestra.run.smithi080.stdout: } 2024-04-25T13:52:28.069 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:28.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb29806f400 msgr2=0x7fb298071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb29806f400 0x7fb298071910 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac06fa20 tx=0x7fb2a0008040 comp rx=0 tx=0).stop 2024-04-25T13:52:28.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2ac146730 msgr2=0x7fb2ac1f4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2ac146730 0x7fb2ac1f4330 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb29c010890 tx=0x7fb29c010d50 comp rx=0 tx=0).stop 2024-04-25T13:52:28.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 shutdown_connections 2024-04-25T13:52:28.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb29806f400 0x7fb298071910 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb2ac146f00 0x7fb2ac147310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb2ac146730 0x7fb2ac1f4330 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 --2- 172.21.15.80:0/2247553791 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb294004d50 0x7fb2ac0b2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 >> 172.21.15.80:0/2247553791 conn(0x7fb2ac06d080 msgr2=0x7fb2ac15fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:28.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 shutdown_connections 2024-04-25T13:52:28.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.007+0000 7fb2b138d640 1 -- 172.21.15.80:0/2247553791 wait complete. 2024-04-25T13:52:28.072 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:28.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.227+0000 7f77b3e2c640 1 Processor -- start 2024-04-25T13:52:28.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b3e2c640 1 -- start start 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b3e2c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f77ac161240 0x7f77ac0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b3e2c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f77ac161650 0x7f77ac0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b3e2c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77ac06fcf0 con 0x7f77ac06f5a0 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b3e2c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f77ac073f10 con 0x7f77ac161650 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b3e2c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f77ac0b0c60 con 0x7f77ac161240 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b23c9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f77ac161650 0x7f77ac0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b23c9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f77ac161650 0x7f77ac0728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49656/0 (socket says 172.21.15.80:49656) 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b23c9640 1 -- 172.21.15.80:0/1748797616 learned_addr learned my addr 172.21.15.80:0/1748797616 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b13c7640 1 --2- 172.21.15.80:0/1748797616 >> v2:172.21.15.87:3300/0 conn(0x7f77ac161240 0x7f77ac0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:28.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b23c9640 1 -- 172.21.15.80:0/1748797616 >> v2:172.21.15.87:3300/0 conn(0x7f77ac161240 msgr2=0x7f77ac0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:28.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b23c9640 1 --2- 172.21.15.80:0/1748797616 >> v2:172.21.15.87:3300/0 conn(0x7f77ac161240 0x7f77ac0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b23c9640 1 -- 172.21.15.80:0/1748797616 >> v1:172.21.15.87:6789/0 conn(0x7f77ac06f5a0 legacy=0x7f77ac160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:28.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.231+0000 7f77b23c9640 1 -- 172.21.15.80:0/1748797616 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac0b0f40 con 0x7f77ac161650 2024-04-25T13:52:28.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b23c9640 1 --2- 172.21.15.80:0/1748797616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0728e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f77ac06ff70 tx=0x7f779c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec00f3b034fd4517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:28.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b0bc6640 1 -- 172.21.15.80:0/1748797616 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0b0070 con 0x7f77ac161650 2024-04-25T13:52:28.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b0bc6640 1 -- 172.21.15.80:0/1748797616 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779c0ab600 con 0x7f77ac161650 2024-04-25T13:52:28.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b0bc6640 1 -- 172.21.15.80:0/1748797616 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0ab9d0 con 0x7f77ac161650 2024-04-25T13:52:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1748797616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 msgr2=0x7f77ac0728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1748797616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0728e0 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f77ac06ff70 tx=0x7f779c01c450 comp rx=0 tx=0).stop 2024-04-25T13:52:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1748797616 shutdown_connections 2024-04-25T13:52:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1748797616 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0728e0 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1748797616 >> v2:172.21.15.87:3300/0 conn(0x7f77ac161240 0x7f77ac0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1748797616 >> 172.21.15.80:0/1748797616 conn(0x7f77ac06d080 msgr2=0x7f77ac0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1748797616 shutdown_connections 2024-04-25T13:52:28.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1748797616 wait complete. 2024-04-25T13:52:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 Processor -- start 2024-04-25T13:52:28.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.235+0000 7f77b3e2c640 1 -- start start 2024-04-25T13:52:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77ac06f5a0 0x7f77ac14b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:28.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77ac161240 0x7f77ac162490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac16a260 con 0x7f77ac161650 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f77ac16a0e0 con 0x7f77ac06f5a0 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f77ac16a3e0 con 0x7f77ac161240 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49660/0 (socket says 172.21.15.80:49660) 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 -- 172.21.15.80:0/1737030669 learned_addr learned my addr 172.21.15.80:0/1737030669 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b1bc8640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77ac06f5a0 0x7f77ac14b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b13c7640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77ac161240 0x7f77ac162490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 -- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77ac161240 msgr2=0x7f77ac162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77ac161240 0x7f77ac162490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 -- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77ac06f5a0 msgr2=0x7f77ac14b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77ac06f5a0 0x7f77ac14b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 -- 172.21.15.80:0/1737030669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77ac15f440 con 0x7f77ac161650 2024-04-25T13:52:28.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b23c9640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0cef70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ae040 tx=0x7f779c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:28.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77aa7fc640 1 -- 172.21.15.80:0/1737030669 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0b0040 con 0x7f77ac161650 2024-04-25T13:52:28.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77aa7fc640 1 -- 172.21.15.80:0/1737030669 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f779c0bc070 con 0x7f77ac161650 2024-04-25T13:52:28.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77ac15f720 con 0x7f77ac161650 2024-04-25T13:52:28.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77aa7fc640 1 -- 172.21.15.80:0/1737030669 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f779c0b6640 con 0x7f77ac161650 2024-04-25T13:52:28.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77ac15fce0 con 0x7f77ac161650 2024-04-25T13:52:28.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77ac06a3a0 con 0x7f77ac161650 2024-04-25T13:52:28.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77aa7fc640 1 -- 172.21.15.80:0/1737030669 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f779c0b67e0 con 0x7f77ac161650 2024-04-25T13:52:28.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77aa7fc640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f779806f400 0x7f7798071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:28.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.239+0000 7f77aa7fc640 1 -- 172.21.15.80:0/1737030669 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f779c129400 con 0x7f77ac161650 2024-04-25T13:52:28.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.243+0000 7f77b1bc8640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f779806f400 0x7f7798071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:28.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.243+0000 7f77b1bc8640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f779806f400 0x7f7798071910 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0b6e70 tx=0x7f77a40094d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:28.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.247+0000 7f77aa7fc640 1 -- 172.21.15.80:0/1737030669 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f779c0c7060 con 0x7f77ac161650 2024-04-25T13:52:28.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77ac0739c0 con 0x7f77ac161650 2024-04-25T13:52:28.433 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77aa7fc640 1 -- 172.21.15.80:0/1737030669 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7f779c0160d0 con 0x7f77ac161650 2024-04-25T13:52:28.433 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:28.434 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:52:28.434 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:52:28.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f779806f400 msgr2=0x7f7798071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f779806f400 0x7f7798071910 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f77ac0b6e70 tx=0x7f77a40094d0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 msgr2=0x7f77ac0cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0cef70 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f779c0ae040 tx=0x7f779c01ac90 comp rx=0 tx=0).stop 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 shutdown_connections 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f779806f400 0x7f7798071910 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f77ac161650 0x7f77ac0cef70 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f77ac161240 0x7f77ac162490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 --2- 172.21.15.80:0/1737030669 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77ac06f5a0 0x7f77ac14b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 >> 172.21.15.80:0/1737030669 conn(0x7f77ac06d080 msgr2=0x7f77ac0757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 shutdown_connections 2024-04-25T13:52:28.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:28.439+0000 7f77b3e2c640 1 -- 172.21.15.80:0/1737030669 wait complete. 2024-04-25T13:52:28.454 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2024-04-25T13:52:28.622 INFO:teuthology.orchestra.run.smithi080.stdout:{"cluster_fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","whoami":0,"id":9213,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":37.902918816000003,"mdsmap_epoch":89,"osdmap_epoch":190,"osdmap_epoch_barrier":189,"uptime":42.877739087999998} 2024-04-25T13:52:28.622 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "3a629b7a-ce3b-47b8-98ac-844ad072aa94", "fs_name": "cephfs", "id": 9213, "mdsmap_epoch": 89, "osdmap_epoch": 190, "osdmap_epoch_barrier": 189, "rank_uptime": 37.902918816, "state": "up:active", "uptime": 42.877739088, "want_state": "up:active", "whoami": 0 } 2024-04-25T13:52:28.622 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-25T13:52:28.622 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:28.623 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:28.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:28.628 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2024-04-25T13:52:29.076 INFO:teuthology.orchestra.run.smithi080.stderr:50+0 records in 2024-04-25T13:52:29.076 INFO:teuthology.orchestra.run.smithi080.stderr:50+0 records out 2024-04-25T13:52:29.076 INFO:teuthology.orchestra.run.smithi080.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.397427 s, 132 MB/s 2024-04-25T13:52:29.077 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T13:52:29.077 DEBUG:teuthology.orchestra.run.smithi080:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:52:29.113 DEBUG:teuthology.orchestra.run.smithi087:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T13:52:29.155 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T13:52:29.155 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:52:29.155 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:29.155 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:29.165 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:29.165 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:29.212 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:52:29.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafccd93640 1 Processor -- start 2024-04-25T13:52:29.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafccd93640 1 -- start start 2024-04-25T13:52:29.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafccd93640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fafc80b5a00 0x7fafc8141100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafccd93640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8162710 0x7fafc806d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafccd93640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fafc806df50 con 0x7fafc806db40 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafccd93640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fafc80b62a0 con 0x7fafc80b5a00 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafccd93640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fafc80b5fc0 con 0x7fafc8162710 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc6d9d640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fafc806db40 0x7fafc8068590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55628/0 (socket says 172.21.15.80:55628) 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc6d9d640 1 -- 172.21.15.80:0/3618647582 learned_addr learned my addr 172.21.15.80:0/3618647582 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc659c640 1 --2- 172.21.15.80:0/3618647582 >> v2:172.21.15.87:3300/0 conn(0x7fafc80b5a00 0x7fafc8141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc5d9b640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8162710 0x7fafc806d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358794992 0 0) 0x7fafc0001120 con 0x7fafc806db40 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fafb4003530 con 0x7fafc806db40 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1272552700 0 0) 0x7fafb4003530 con 0x7fafc806db40 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8162710 msgr2=0x7fafc806d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8162710 0x7fafc806d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 >> v2:172.21.15.87:3300/0 conn(0x7fafc80b5a00 msgr2=0x7fafc8141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:29.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 --2- 172.21.15.80:0/3618647582 >> v2:172.21.15.87:3300/0 conn(0x7fafc80b5a00 0x7fafc8141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc815e900 con 0x7fafc806db40 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fafc0001850 con 0x7fafc806db40 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 >> v1:172.21.15.80:6789/0 conn(0x7fafc806db40 legacy=0x7fafc8068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafb4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb4004810 0x7fafb4004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8162710 0x7fafb4005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80b62a0 con 0x7fafb40039c0 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fafc80b5fc0 con 0x7fafb4004810 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fafb400ba00 con 0x7fafc8162710 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc659c640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafb4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc6d9d640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8162710 0x7fafb4005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc659c640 1 -- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8162710 msgr2=0x7fafb4005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc5d9b640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb4004810 0x7fafb4004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc659c640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8162710 0x7fafb4005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc659c640 1 -- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb4004810 msgr2=0x7fafb4004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:29.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc659c640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafb4004810 0x7fafb4004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.419+0000 7fafc659c640 1 -- 172.21.15.80:0/3618647582 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafb4003530 con 0x7fafb40039c0 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafc659c640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafb4003e30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009f90 tx=0x7fafb801cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9e40140a896376 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80b2070 con 0x7fafb40039c0 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb80a5a70 con 0x7fafb40039c0 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafc559a640 1 -- 172.21.15.80:0/3618647582 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8009c10 con 0x7fafb40039c0 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafccd93640 1 -- 172.21.15.80:0/3618647582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 msgr2=0x7fafb4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafccd93640 1 --2- 172.21.15.80:0/3618647582 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafb4003e30 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009f90 tx=0x7fafb801cc80 comp rx=0 tx=0).stop 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafc6d9d640 1 -- 172.21.15.80:0/3618647582 reap_dead start 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafccd93640 1 -- 172.21.15.80:0/3618647582 shutdown_connections 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafccd93640 1 -- 172.21.15.80:0/3618647582 >> 172.21.15.80:0/3618647582 conn(0x7fafc8072ed0 msgr2=0x7fafc80732e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafccd93640 1 -- 172.21.15.80:0/3618647582 shutdown_connections 2024-04-25T13:52:29.419 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.423+0000 7fafccd93640 1 -- 172.21.15.80:0/3618647582 wait complete. 2024-04-25T13:52:29.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 Processor -- start 2024-04-25T13:52:29.420 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 -- start start 2024-04-25T13:52:29.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafc81f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc8158830 0x7fafc81676f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8167c30 0x7fafc81f9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8169210 con 0x7fafb40039c0 2024-04-25T13:52:29.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fafc8169090 con 0x7fafc8158830 2024-04-25T13:52:29.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fafc8169390 con 0x7fafc8167c30 2024-04-25T13:52:29.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafc81f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.423 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafc81f94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49684/0 (socket says 172.21.15.80:49684) 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 -- 172.21.15.80:0/1660666473 learned_addr learned my addr 172.21.15.80:0/1660666473 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc6d9d640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8167c30 0x7fafc81f9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc5d9b640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc8158830 0x7fafc81676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 -- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8167c30 msgr2=0x7fafc81f9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8167c30 0x7fafc81f9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 -- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc8158830 msgr2=0x7fafc81676f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc8158830 0x7fafc81676f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 -- 172.21.15.80:0/1660666473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafc815dd80 con 0x7fafb40039c0 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafc659c640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafc81f94a0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fafb801cab0 tx=0x7fafb8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafbeffd640 1 -- 172.21.15.80:0/1660666473 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80b2040 con 0x7fafb40039c0 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafbeffd640 1 -- 172.21.15.80:0/1660666473 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafb80bd070 con 0x7fafb40039c0 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafbeffd640 1 -- 172.21.15.80:0/1660666473 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb80adc80 con 0x7fafb40039c0 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafc81f89c0 con 0x7fafb40039c0 2024-04-25T13:52:29.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.427+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafc81f8f80 con 0x7fafb40039c0 2024-04-25T13:52:29.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.431+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafc806a3a0 con 0x7fafb40039c0 2024-04-25T13:52:29.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.431+0000 7fafbeffd640 1 -- 172.21.15.80:0/1660666473 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fafb80c2460 con 0x7fafb40039c0 2024-04-25T13:52:29.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.431+0000 7fafbeffd640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa806f430 0x7fafa8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.425 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.431+0000 7fafbeffd640 1 -- 172.21.15.80:0/1660666473 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7fafb812a7b0 con 0x7fafb40039c0 2024-04-25T13:52:29.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.431+0000 7fafc5d9b640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa806f430 0x7fafa8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.431+0000 7fafc5d9b640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa806f430 0x7fafa8071940 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fafc8076060 tx=0x7fafb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:29.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.435+0000 7fafbeffd640 1 -- 172.21.15.80:0/1660666473 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fafb8016990 con 0x7fafb40039c0 2024-04-25T13:52:29.592 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fafc80d1a40 con 0x7fafb40039c0 2024-04-25T13:52:29.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafbeffd640 1 -- 172.21.15.80:0/1660666473 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) ==== 74+0+16607 (secure 0 0 0) 0x7fafb80ade20 con 0x7fafb40039c0 2024-04-25T13:52:29.593 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:29.593 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":190,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:52:26.300379+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:51:48.404349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T13:51:49.039512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:52:29.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa806f430 msgr2=0x7fafa8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa806f430 0x7fafa8071940 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fafc8076060 tx=0x7fafb0008040 comp rx=0 tx=0).stop 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 msgr2=0x7fafc81f94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafc81f94a0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fafb801cab0 tx=0x7fafb8009750 comp rx=0 tx=0).stop 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 shutdown_connections 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fafa806f430 0x7fafa8071940 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fafc8167c30 0x7fafc81f9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fafc8158830 0x7fafc81676f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 --2- 172.21.15.80:0/1660666473 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fafb40039c0 0x7fafc81f94a0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 >> 172.21.15.80:0/1660666473 conn(0x7fafc8072ed0 msgr2=0x7fafc806cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 shutdown_connections 2024-04-25T13:52:29.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.599+0000 7fafccd93640 1 -- 172.21.15.80:0/1660666473 wait complete. 2024-04-25T13:52:29.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:51:48.404349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:51:49.039512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:52:29.615 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:52:29.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0ac690b640 1 Processor -- start 2024-04-25T13:52:29.765 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0ac690b640 1 -- start start 2024-04-25T13:52:29.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0ac690b640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ac00b5330 0x7f0ac00d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0ac690b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac0161b60 0x7f0ac0161f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0ac690b640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ac01624b0 con 0x7f0ac00d0ff0 2024-04-25T13:52:29.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0ac690b640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0ac01629b0 con 0x7f0ac00b5330 2024-04-25T13:52:29.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0ac690b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac00b5740 con 0x7f0ac0161b60 2024-04-25T13:52:29.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0abffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ac00b5330 0x7f0ac00d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0abf7fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0ac00d0ff0 0x7f0ac006edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55630/0 (socket says 172.21.15.80:55630) 2024-04-25T13:52:29.767 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0abf7fe640 1 -- 172.21.15.80:0/495441591 learned_addr learned my addr 172.21.15.80:0/495441591 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:29.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0ac4ea8640 1 --2- 172.21.15.80:0/495441591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac0161b60 0x7f0ac0161f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0abffff640 1 -- 172.21.15.80:0/495441591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac0161b60 msgr2=0x7f0ac0161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:29.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0abffff640 1 --2- 172.21.15.80:0/495441591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac0161b60 0x7f0ac0161f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0abffff640 1 -- 172.21.15.80:0/495441591 >> v1:172.21.15.80:6789/0 conn(0x7f0ac00d0ff0 legacy=0x7f0ac006edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0abeffd640 1 -- 172.21.15.80:0/495441591 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 342101813 0 0) 0x7f0ab4001120 con 0x7f0ac00d0ff0 2024-04-25T13:52:29.768 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.771+0000 7f0abffff640 1 -- 172.21.15.80:0/495441591 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac006dd10 con 0x7f0ac00b5330 2024-04-25T13:52:29.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.775+0000 7f0abffff640 1 --2- 172.21.15.80:0/495441591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac00b5330 0x7f0ac00d0ab0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00097c0 tx=0x7f0ab001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8435f6bdd50f57b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:29.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.775+0000 7f0abeffd640 1 -- 172.21.15.80:0/495441591 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00b0070 con 0x7f0ac00b5330 2024-04-25T13:52:29.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.775+0000 7f0abeffd640 1 -- 172.21.15.80:0/495441591 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab00ab600 con 0x7f0ac00b5330 2024-04-25T13:52:29.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.775+0000 7f0abeffd640 1 -- 172.21.15.80:0/495441591 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab00ab9d0 con 0x7f0ac00b5330 2024-04-25T13:52:29.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- 172.21.15.80:0/495441591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac00b5330 msgr2=0x7f0ac00d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 --2- 172.21.15.80:0/495441591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac00b5330 0x7f0ac00d0ab0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00097c0 tx=0x7f0ab001c410 comp rx=0 tx=0).stop 2024-04-25T13:52:29.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- 172.21.15.80:0/495441591 shutdown_connections 2024-04-25T13:52:29.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 --2- 172.21.15.80:0/495441591 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac0161b60 0x7f0ac0161f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 --2- 172.21.15.80:0/495441591 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac00b5330 0x7f0ac00d0ab0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- 172.21.15.80:0/495441591 >> 172.21.15.80:0/495441591 conn(0x7f0ac00b4160 msgr2=0x7f0ac00d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:29.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- 172.21.15.80:0/495441591 shutdown_connections 2024-04-25T13:52:29.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- 172.21.15.80:0/495441591 wait complete. 2024-04-25T13:52:29.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 Processor -- start 2024-04-25T13:52:29.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- start start 2024-04-25T13:52:29.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ac00b5330 0x7f0ac0168380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac00d0ff0 0x7f0ac00af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac0161b60 0x7f0ac00afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac00b0cd0 con 0x7f0ac0161b60 2024-04-25T13:52:29.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac006e890 con 0x7f0ac00b5330 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac690b640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0ac00b0e50 con 0x7f0ac00d0ff0 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac4ea8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac0161b60 0x7f0ac00afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac4ea8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac0161b60 0x7f0ac00afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49702/0 (socket says 172.21.15.80:49702) 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0ac4ea8640 1 -- 172.21.15.80:0/3141216562 learned_addr learned my addr 172.21.15.80:0/3141216562 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0abf7fe640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac00d0ff0 0x7f0ac00af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.779+0000 7f0abffff640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ac00b5330 0x7f0ac0168380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac4ea8640 1 -- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac00d0ff0 msgr2=0x7f0ac00af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac4ea8640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac00d0ff0 0x7f0ac00af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac4ea8640 1 -- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ac00b5330 msgr2=0x7f0ac0168380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac4ea8640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ac00b5330 0x7f0ac0168380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac4ea8640 1 -- 172.21.15.80:0/3141216562 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac0072a10 con 0x7f0ac0161b60 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abffff640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ac00b5330 0x7f0ac0168380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac4ea8640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac0161b60 0x7f0ac00afdf0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800cce0 tx=0x7f0ab800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:29.777 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abcff9640 1 -- 172.21.15.80:0/3141216562 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab800ce90 con 0x7f0ac0161b60 2024-04-25T13:52:29.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abcff9640 1 -- 172.21.15.80:0/3141216562 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ab80039a0 con 0x7f0ac0161b60 2024-04-25T13:52:29.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac0072cf0 con 0x7f0ac0161b60 2024-04-25T13:52:29.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abcff9640 1 -- 172.21.15.80:0/3141216562 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8003d70 con 0x7f0ac0161b60 2024-04-25T13:52:29.778 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac00732b0 con 0x7f0ac0161b60 2024-04-25T13:52:29.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac006a3a0 con 0x7f0ac0161b60 2024-04-25T13:52:29.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abcff9640 1 -- 172.21.15.80:0/3141216562 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0ab8004000 con 0x7f0ac0161b60 2024-04-25T13:52:29.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abcff9640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0aac06f5d0 0x7f0aac071ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:29.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abcff9640 1 -- 172.21.15.80:0/3141216562 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f0ab808f940 con 0x7f0ac0161b60 2024-04-25T13:52:29.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abffff640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0aac06f5d0 0x7f0aac071ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:29.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.783+0000 7f0abffff640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0aac06f5d0 0x7f0aac071ae0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00ad000 tx=0x7f0ab0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:29.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.787+0000 7f0abcff9640 1 -- 172.21.15.80:0/3141216562 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0ab8014030 con 0x7f0ac0161b60 2024-04-25T13:52:29.945 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.951+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ac006fd00 con 0x7f0ac0161b60 2024-04-25T13:52:29.946 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.951+0000 7f0abcff9640 1 -- 172.21.15.80:0/3141216562 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) ==== 93+0+10 (secure 0 0 0) 0x7f0ab805ca30 con 0x7f0ac0161b60 2024-04-25T13:52:29.946 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:52:29.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0aac06f5d0 msgr2=0x7f0aac071ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0aac06f5d0 0x7f0aac071ae0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f0ab00ad000 tx=0x7f0ab0008040 comp rx=0 tx=0).stop 2024-04-25T13:52:29.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac0161b60 msgr2=0x7f0ac00afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:29.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac0161b60 0x7f0ac00afdf0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800cce0 tx=0x7f0ab800a5c0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.948 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 shutdown_connections 2024-04-25T13:52:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0aac06f5d0 0x7f0aac071ae0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ac0161b60 0x7f0ac00afdf0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ac00d0ff0 0x7f0ac00af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 --2- 172.21.15.80:0/3141216562 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ac00b5330 0x7f0ac0168380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 >> 172.21.15.80:0/3141216562 conn(0x7f0ac00b4160 msgr2=0x7f0ac00b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 shutdown_connections 2024-04-25T13:52:29.949 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:29.955+0000 7f0ac690b640 1 -- 172.21.15.80:0/3141216562 wait complete. 2024-04-25T13:52:29.967 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:52:30.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.127+0000 7f8baf9be640 1 Processor -- start 2024-04-25T13:52:30.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.127+0000 7f8baf9be640 1 -- start start 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.127+0000 7f8baf9be640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8ba80b5330 0x7f8ba80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.127+0000 7f8baf9be640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba80b5740 con 0x7f8ba80d0ff0 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.127+0000 7f8baf9be640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba806dd10 con 0x7f8ba8161b60 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.127+0000 7f8baf9be640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba81629b0 con 0x7f8ba80b5330 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8bad75a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8ba80b5330 0x7f8ba80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8bacf59640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f8ba80d0ff0 0x7f8ba806edb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55642/0 (socket says 172.21.15.80:55642) 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8bad75a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f8ba80b5330 0x7f8ba80d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49710/0 (socket says 172.21.15.80:49710) 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8bad75a640 1 -- 172.21.15.80:0/717245479 learned_addr learned my addr 172.21.15.80:0/717245479 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:30.124 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8bad75a640 1 -- 172.21.15.80:0/717245479 >> v1:172.21.15.87:6789/0 conn(0x7f8ba8161b60 legacy=0x7f8ba8161f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:30.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8bad75a640 1 -- 172.21.15.80:0/717245479 >> v1:172.21.15.80:6789/0 conn(0x7f8ba80d0ff0 legacy=0x7f8ba806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8bad75a640 1 -- 172.21.15.80:0/717245479 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba806dff0 con 0x7f8ba80b5330 2024-04-25T13:52:30.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8bad75a640 1 --2- 172.21.15.80:0/717245479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba80d0ab0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009870 tx=0x7f8b9801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41abd9da6c28d0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8b9ffff640 1 -- 172.21.15.80:0/717245479 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980b2070 con 0x7f8ba80b5330 2024-04-25T13:52:30.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8b9ffff640 1 -- 172.21.15.80:0/717245479 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b980ad680 con 0x7f8ba80b5330 2024-04-25T13:52:30.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8b9ffff640 1 -- 172.21.15.80:0/717245479 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980ada50 con 0x7f8ba80b5330 2024-04-25T13:52:30.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8baf9be640 1 -- 172.21.15.80:0/717245479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 msgr2=0x7f8ba80d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8baf9be640 1 --2- 172.21.15.80:0/717245479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba80d0ab0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009870 tx=0x7f8b9801c490 comp rx=0 tx=0).stop 2024-04-25T13:52:30.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8baf9be640 1 -- 172.21.15.80:0/717245479 shutdown_connections 2024-04-25T13:52:30.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8baf9be640 1 --2- 172.21.15.80:0/717245479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba80d0ab0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8baf9be640 1 -- 172.21.15.80:0/717245479 >> 172.21.15.80:0/717245479 conn(0x7f8ba80b4160 msgr2=0x7f8ba80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:30.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8baf9be640 1 -- 172.21.15.80:0/717245479 shutdown_connections 2024-04-25T13:52:30.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.131+0000 7f8baf9be640 1 -- 172.21.15.80:0/717245479 wait complete. 2024-04-25T13:52:30.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 Processor -- start 2024-04-25T13:52:30.129 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 -- start start 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba814ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ba80d0ff0 0x7f8ba815e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ba8161b60 0x7f8ba81525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80b0e50 con 0x7f8ba80b5330 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80b0cd0 con 0x7f8ba80d0ff0 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba806f4c0 con 0x7f8ba8161b60 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba814ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba814ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49726/0 (socket says 172.21.15.80:49726) 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 -- 172.21.15.80:0/4168410498 learned_addr learned my addr 172.21.15.80:0/4168410498 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 -- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ba8161b60 msgr2=0x7f8ba81525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.132 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ba8161b60 0x7f8ba81525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 -- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ba80d0ff0 msgr2=0x7f8ba815e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ba80d0ff0 0x7f8ba815e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 -- 172.21.15.80:0/4168410498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba8159050 con 0x7f8ba80b5330 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8bad75a640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba814ae20 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8b9801a0c0 tx=0x7f8b9801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8b9dffb640 1 -- 172.21.15.80:0/4168410498 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980b2040 con 0x7f8ba80b5330 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8b9dffb640 1 -- 172.21.15.80:0/4168410498 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b980be070 con 0x7f8ba80b5330 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba8159330 con 0x7f8ba80b5330 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8b9dffb640 1 -- 172.21.15.80:0/4168410498 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980b8660 con 0x7f8ba80b5330 2024-04-25T13:52:30.133 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.135+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81598f0 con 0x7f8ba80b5330 2024-04-25T13:52:30.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.139+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba806a3a0 con 0x7f8ba80b5330 2024-04-25T13:52:30.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.139+0000 7f8b9dffb640 1 -- 172.21.15.80:0/4168410498 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f8b980b8800 con 0x7f8ba80b5330 2024-04-25T13:52:30.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.139+0000 7f8b9dffb640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b8c06f400 0x7f8b8c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.139+0000 7f8b9dffb640 1 -- 172.21.15.80:0/4168410498 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f8b981295a0 con 0x7f8ba80b5330 2024-04-25T13:52:30.134 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.139+0000 7f8bacf59640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b8c06f400 0x7f8b8c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.135 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.139+0000 7f8bacf59640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b8c06f400 0x7f8b8c071910 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4000f70 tx=0x7f8ba4007a10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.138 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.143+0000 7f8b9dffb640 1 -- 172.21.15.80:0/4168410498 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f8b98016110 con 0x7f8ba80b5330 2024-04-25T13:52:30.299 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.303+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8ba8076330 con 0x7f8ba80b5330 2024-04-25T13:52:30.300 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8b9dffb640 1 -- 172.21.15.80:0/4168410498 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) ==== 104+0+11 (secure 0 0 0) 0x7f8b980ad650 con 0x7f8ba80b5330 2024-04-25T13:52:30.300 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b8c06f400 msgr2=0x7f8b8c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b8c06f400 0x7f8b8c071910 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4000f70 tx=0x7f8ba4007a10 comp rx=0 tx=0).stop 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 msgr2=0x7f8ba814ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba814ae20 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f8b9801a0c0 tx=0x7f8b9801acd0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 shutdown_connections 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f8b8c06f400 0x7f8b8c071910 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f8ba8161b60 0x7f8ba81525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8ba80d0ff0 0x7f8ba815e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 --2- 172.21.15.80:0/4168410498 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f8ba80b5330 0x7f8ba814ae20 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.307+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 >> 172.21.15.80:0/4168410498 conn(0x7f8ba80b4160 msgr2=0x7f8ba8160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.311+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 shutdown_connections 2024-04-25T13:52:30.304 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.311+0000 7f8baf9be640 1 -- 172.21.15.80:0/4168410498 wait complete. 2024-04-25T13:52:30.322 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:52:30.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec367d6640 1 Processor -- start 2024-04-25T13:52:30.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec367d6640 1 -- start start 2024-04-25T13:52:30.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec367d6640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fec300d1a80 0x7fec300d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec367d6640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fec300cf8c0 0x7fec300d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec367d6640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fec300d4a30 con 0x7fec300cf4b0 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec367d6640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fec300d2b40 con 0x7fec300cf8c0 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec367d6640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fec300cfcd0 con 0x7fec300d1a80 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec34d73640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fec300cf8c0 0x7fec300d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec34d73640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fec300cf8c0 0x7fec300d01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49734/0 (socket says 172.21.15.80:49734) 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec34d73640 1 -- 172.21.15.80:0/4039786061 learned_addr learned my addr 172.21.15.80:0/4039786061 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec2ffff640 1 --2- 172.21.15.80:0/4039786061 >> v2:172.21.15.87:3300/0 conn(0x7fec300d1a80 0x7fec300d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec34d73640 1 -- 172.21.15.80:0/4039786061 >> v2:172.21.15.87:3300/0 conn(0x7fec300d1a80 msgr2=0x7fec300d1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec34d73640 1 --2- 172.21.15.80:0/4039786061 >> v2:172.21.15.87:3300/0 conn(0x7fec300d1a80 0x7fec300d1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec34d73640 1 -- 172.21.15.80:0/4039786061 >> v1:172.21.15.87:6789/0 conn(0x7fec300cf4b0 legacy=0x7fec300b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.479+0000 7fec34d73640 1 -- 172.21.15.80:0/4039786061 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300b4490 con 0x7fec300cf8c0 2024-04-25T13:52:30.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec34d73640 1 --2- 172.21.15.80:0/4039786061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300d01f0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fec30073110 tx=0x7fec2801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43e227c659e7390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec2f7fe640 1 -- 172.21.15.80:0/4039786061 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec280a5710 con 0x7fec300cf8c0 2024-04-25T13:52:30.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec2f7fe640 1 -- 172.21.15.80:0/4039786061 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec280a58b0 con 0x7fec300cf8c0 2024-04-25T13:52:30.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec2f7fe640 1 -- 172.21.15.80:0/4039786061 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec280a5c80 con 0x7fec300cf8c0 2024-04-25T13:52:30.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 -- 172.21.15.80:0/4039786061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 msgr2=0x7fec300d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 --2- 172.21.15.80:0/4039786061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300d01f0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fec30073110 tx=0x7fec2801c960 comp rx=0 tx=0).stop 2024-04-25T13:52:30.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 -- 172.21.15.80:0/4039786061 shutdown_connections 2024-04-25T13:52:30.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 --2- 172.21.15.80:0/4039786061 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300d01f0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 --2- 172.21.15.80:0/4039786061 >> v2:172.21.15.87:3300/0 conn(0x7fec300d1a80 0x7fec300d1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 -- 172.21.15.80:0/4039786061 >> 172.21.15.80:0/4039786061 conn(0x7fec300d3ad0 msgr2=0x7fec300d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:30.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 -- 172.21.15.80:0/4039786061 shutdown_connections 2024-04-25T13:52:30.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 -- 172.21.15.80:0/4039786061 wait complete. 2024-04-25T13:52:30.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 Processor -- start 2024-04-25T13:52:30.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.483+0000 7fec367d6640 1 -- start start 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec300cf4b0 0x7fec301508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec300d1a80 0x7fec3014dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fec300d2760 con 0x7fec300cf8c0 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fec300d2480 con 0x7fec300cf4b0 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fec300b7030 con 0x7fec300d1a80 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49744/0 (socket says 172.21.15.80:49744) 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 -- 172.21.15.80:0/4148233823 learned_addr learned my addr 172.21.15.80:0/4148233823 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec2ffff640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec300cf4b0 0x7fec301508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec34d73640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec300d1a80 0x7fec3014dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 -- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec300d1a80 msgr2=0x7fec3014dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec300d1a80 0x7fec3014dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 -- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec300cf4b0 msgr2=0x7fec301508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec300cf4b0 0x7fec301508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 -- 172.21.15.80:0/4148233823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec300cb660 con 0x7fec300cf8c0 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec34d73640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec300d1a80 0x7fec3014dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec27fff640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300cae30 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c370 tx=0x7fec1800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec2d7fa640 1 -- 172.21.15.80:0/4148233823 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec180052d0 con 0x7fec300cf8c0 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec2d7fa640 1 -- 172.21.15.80:0/4148233823 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fec18005470 con 0x7fec300cf8c0 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec2d7fa640 1 -- 172.21.15.80:0/4148233823 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18007260 con 0x7fec300cf8c0 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec300cb940 con 0x7fec300cf8c0 2024-04-25T13:52:30.483 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec300cbf00 con 0x7fec300cf8c0 2024-04-25T13:52:30.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3006a3a0 con 0x7fec300cf8c0 2024-04-25T13:52:30.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.487+0000 7fec2d7fa640 1 -- 172.21.15.80:0/4148233823 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fec18007400 con 0x7fec300cf8c0 2024-04-25T13:52:30.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.491+0000 7fec2d7fa640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec1006f400 0x7fec10071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.491+0000 7fec2d7fa640 1 -- 172.21.15.80:0/4148233823 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7fec1808e740 con 0x7fec300cf8c0 2024-04-25T13:52:30.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.491+0000 7fec2ffff640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec1006f400 0x7fec10071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.491+0000 7fec2ffff640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec1006f400 0x7fec10071910 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fec300b4bf0 tx=0x7fec20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.489 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.495+0000 7fec2d7fa640 1 -- 172.21.15.80:0/4148233823 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fec1805b630 con 0x7fec300cf8c0 2024-04-25T13:52:30.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.659+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fec300cb3f0 con 0x7fec300cf8c0 2024-04-25T13:52:30.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.659+0000 7fec2d7fa640 1 -- 172.21.15.80:0/4148233823 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) ==== 100+0+11 (secure 0 0 0) 0x7fec1805af30 con 0x7fec300cf8c0 2024-04-25T13:52:30.654 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:52:30.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec1006f400 msgr2=0x7fec10071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec1006f400 0x7fec10071910 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fec300b4bf0 tx=0x7fec20008040 comp rx=0 tx=0).stop 2024-04-25T13:52:30.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 msgr2=0x7fec300cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300cae30 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fec1800c370 tx=0x7fec1800c830 comp rx=0 tx=0).stop 2024-04-25T13:52:30.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 shutdown_connections 2024-04-25T13:52:30.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fec1006f400 0x7fec10071910 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fec300d1a80 0x7fec3014dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fec300cf8c0 0x7fec300cae30 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 --2- 172.21.15.80:0/4148233823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec300cf4b0 0x7fec301508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 >> 172.21.15.80:0/4148233823 conn(0x7fec300d3ad0 msgr2=0x7fec300b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:30.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 shutdown_connections 2024-04-25T13:52:30.657 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.663+0000 7fec367d6640 1 -- 172.21.15.80:0/4148233823 wait complete. 2024-04-25T13:52:30.675 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:52:30.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3207090640 1 Processor -- start 2024-04-25T13:52:30.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3207090640 1 -- start start 2024-04-25T13:52:30.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3207090640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32000d45b0 0x7f32000b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3207090640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d0160 0x7f32000d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3207090640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f32000d0ab0 con 0x7f32000d1870 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3207090640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f32000d49c0 con 0x7f32000d45b0 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3207090640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f32000cf980 con 0x7f32000d0160 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3204e2c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32000d45b0 0x7f32000b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3204e2c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f32000d45b0 0x7f32000b4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49756/0 (socket says 172.21.15.80:49756) 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3204e2c640 1 -- 172.21.15.80:0/2595446465 learned_addr learned my addr 172.21.15.80:0/2595446465 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f320562d640 1 --1- 172.21.15.80:0/2595446465 >> v1:172.21.15.80:6789/0 conn(0x7f32000d1870 0x7f32000d1c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55650/0 (socket says 172.21.15.80:55650) 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f31fffff640 1 --2- 172.21.15.80:0/2595446465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d0160 0x7f32000d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3204e2c640 1 -- 172.21.15.80:0/2595446465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d0160 msgr2=0x7f32000d0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3204e2c640 1 --2- 172.21.15.80:0/2595446465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d0160 0x7f32000d0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.823+0000 7f3204e2c640 1 -- 172.21.15.80:0/2595446465 >> v1:172.21.15.80:6789/0 conn(0x7f32000d1870 legacy=0x7f32000d1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3204e2c640 1 -- 172.21.15.80:0/2595446465 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32000cf3c0 con 0x7f32000d45b0 2024-04-25T13:52:30.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f31fffff640 1 --2- 172.21.15.80:0/2595446465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d0160 0x7f32000d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:30.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3204e2c640 1 --2- 172.21.15.80:0/2595446465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d45b0 0x7f32000b4fd0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009870 tx=0x7f31f001c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f825e0e71d25651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f31ff7fe640 1 -- 172.21.15.80:0/2595446465 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00b0070 con 0x7f32000d45b0 2024-04-25T13:52:30.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f31ff7fe640 1 -- 172.21.15.80:0/2595446465 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f00ab600 con 0x7f32000d45b0 2024-04-25T13:52:30.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f31ff7fe640 1 -- 172.21.15.80:0/2595446465 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00ab9d0 con 0x7f32000d45b0 2024-04-25T13:52:30.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3207090640 1 -- 172.21.15.80:0/2595446465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d45b0 msgr2=0x7f32000b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3207090640 1 --2- 172.21.15.80:0/2595446465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d45b0 0x7f32000b4fd0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009870 tx=0x7f31f001c360 comp rx=0 tx=0).stop 2024-04-25T13:52:30.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3207090640 1 -- 172.21.15.80:0/2595446465 shutdown_connections 2024-04-25T13:52:30.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3207090640 1 --2- 172.21.15.80:0/2595446465 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d0160 0x7f32000d0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3207090640 1 --2- 172.21.15.80:0/2595446465 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d45b0 0x7f32000b4fd0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3207090640 1 -- 172.21.15.80:0/2595446465 >> 172.21.15.80:0/2595446465 conn(0x7f32000d3a90 msgr2=0x7f32000d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:30.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3207090640 1 -- 172.21.15.80:0/2595446465 shutdown_connections 2024-04-25T13:52:30.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.827+0000 7f3207090640 1 -- 172.21.15.80:0/2595446465 wait complete. 2024-04-25T13:52:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 Processor -- start 2024-04-25T13:52:30.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 -- start start 2024-04-25T13:52:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d0160 0x7f32000ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32000d1870 0x7f32001f7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d45b0 0x7f32001f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f32000b6a10 con 0x7f32000d0160 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f32000b6d80 con 0x7f32000d1870 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f32000cf070 con 0x7f32000d45b0 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d0160 0x7f32000ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d0160 0x7f32000ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:49768/0 (socket says 172.21.15.80:49768) 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 -- 172.21.15.80:0/294307864 learned_addr learned my addr 172.21.15.80:0/294307864 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f320562d640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d45b0 0x7f32001f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f31fffff640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32000d1870 0x7f32001f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 -- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d45b0 msgr2=0x7f32001f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d45b0 0x7f32001f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 -- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32000d1870 msgr2=0x7f32001f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32000d1870 0x7f32001f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 -- 172.21.15.80:0/294307864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32001f7a50 con 0x7f32000d0160 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f320562d640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d45b0 0x7f32001f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:30.828 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3204e2c640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d0160 0x7f32000ca040 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f31f001aa10 tx=0x7f31f0019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f31fd7fa640 1 -- 172.21.15.80:0/294307864 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00b0040 con 0x7f32000d0160 2024-04-25T13:52:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f31fd7fa640 1 -- 172.21.15.80:0/294307864 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31f00aa950 con 0x7f32000d0160 2024-04-25T13:52:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f320009f960 con 0x7f32000d0160 2024-04-25T13:52:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f31fd7fa640 1 -- 172.21.15.80:0/294307864 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31f00aad40 con 0x7f32000d0160 2024-04-25T13:52:30.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.831+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32000c7a40 con 0x7f32000d0160 2024-04-25T13:52:30.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.835+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32000c7560 con 0x7f32000d0160 2024-04-25T13:52:30.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.835+0000 7f31fd7fa640 1 -- 172.21.15.80:0/294307864 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f31f00ab4c0 con 0x7f32000d0160 2024-04-25T13:52:30.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.835+0000 7f31fd7fa640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31e006f400 0x7f31e0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:30.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.835+0000 7f31fd7fa640 1 -- 172.21.15.80:0/294307864 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f31f0129c90 con 0x7f32000d0160 2024-04-25T13:52:30.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.835+0000 7f31fffff640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31e006f400 0x7f31e0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:30.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.835+0000 7f31fffff640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31e006f400 0x7f31e0071910 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f32000d0d90 tx=0x7f31e8008450 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:30.833 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:30.839+0000 7f31fd7fa640 1 -- 172.21.15.80:0/294307864 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f31f0016070 con 0x7f32000d0160 2024-04-25T13:52:31.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.007+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f320006a3a0 con 0x7f32000d0160 2024-04-25T13:52:31.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.007+0000 7f31fd7fa640 1 -- 172.21.15.80:0/294307864 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v190) ==== 80+0+0 (secure 0 0 0) 0x7f31f00f6d80 con 0x7f32000d0160 2024-04-25T13:52:31.003 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:52:31.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31e006f400 msgr2=0x7f31e0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31e006f400 0x7f31e0071910 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f32000d0d90 tx=0x7f31e8008450 comp rx=0 tx=0).stop 2024-04-25T13:52:31.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d0160 msgr2=0x7f32000ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d0160 0x7f32000ca040 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f31f001aa10 tx=0x7f31f0019100 comp rx=0 tx=0).stop 2024-04-25T13:52:31.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 shutdown_connections 2024-04-25T13:52:31.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f31e006f400 0x7f31e0071910 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f32000d45b0 0x7f32001f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32000d1870 0x7f32001f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 --2- 172.21.15.80:0/294307864 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f32000d0160 0x7f32000ca040 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 >> 172.21.15.80:0/294307864 conn(0x7f32000d3a90 msgr2=0x7f32000b5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:31.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 shutdown_connections 2024-04-25T13:52:31.006 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.011+0000 7f3207090640 1 -- 172.21.15.80:0/294307864 wait complete. 2024-04-25T13:52:31.010 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:52:31.013 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:52:31.024 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi080.front.sepia.ceph.com... 2024-04-25T13:52:31.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:31.024 DEBUG:teuthology.orchestra.run.smithi080:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T13:52:31.076 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi080.stderr:ceph-fuse[43916]: fuse finished with error 0 and tester_r 0 2024-04-25T13:52:31.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:52:31.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:31.083 DEBUG:teuthology.orchestra.run.smithi080:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T13:52:31.147 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:52:31.148 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:52:31.148 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:31.154 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:52:31.154 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:31.202 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:52:31.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.355+0000 7f512961c640 1 Processor -- start 2024-04-25T13:52:31.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.355+0000 7f512961c640 1 -- start start 2024-04-25T13:52:31.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f512961c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5124161f90 0x7f51240cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f512961c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f512415ec40 0x7f512415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f512961c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51240d0d70 con 0x7f51240d0510 2024-04-25T13:52:31.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f512961c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f51240d1050 con 0x7f512415ec40 2024-04-25T13:52:31.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f512961c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f512406dca0 con 0x7f5124161f90 2024-04-25T13:52:31.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f51237fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f512415ec40 0x7f512415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f51237fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f512415ec40 0x7f512415f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37548/0 (socket says 172.21.15.80:37548) 2024-04-25T13:52:31.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f51237fe640 1 -- 172.21.15.80:0/2376518394 learned_addr learned my addr 172.21.15.80:0/2376518394 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:31.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f5122ffd640 1 --2- 172.21.15.80:0/2376518394 >> v2:172.21.15.87:3300/0 conn(0x7f5124161f90 0x7f51240cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f51237fe640 1 -- 172.21.15.80:0/2376518394 >> v2:172.21.15.87:3300/0 conn(0x7f5124161f90 msgr2=0x7f51240cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:31.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f51237fe640 1 --2- 172.21.15.80:0/2376518394 >> v2:172.21.15.87:3300/0 conn(0x7f5124161f90 0x7f51240cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f51237fe640 1 -- 172.21.15.80:0/2376518394 >> v1:172.21.15.87:6789/0 conn(0x7f51240d0510 legacy=0x7f512415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:31.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.359+0000 7f51237fe640 1 -- 172.21.15.80:0/2376518394 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512406df80 con 0x7f512415ec40 2024-04-25T13:52:31.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f51237fe640 1 --2- 172.21.15.80:0/2376518394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f512415ec40 0x7f512415f050 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f51240d17b0 tx=0x7f511c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d653c734d3500e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:31.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f5121ffb640 1 -- 172.21.15.80:0/2376518394 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0a56b0 con 0x7f512415ec40 2024-04-25T13:52:31.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f5121ffb640 1 -- 172.21.15.80:0/2376518394 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c0a5850 con 0x7f512415ec40 2024-04-25T13:52:31.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f5121ffb640 1 -- 172.21.15.80:0/2376518394 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0a5c20 con 0x7f512415ec40 2024-04-25T13:52:31.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 -- 172.21.15.80:0/2376518394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f512415ec40 msgr2=0x7f512415f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 --2- 172.21.15.80:0/2376518394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f512415ec40 0x7f512415f050 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f51240d17b0 tx=0x7f511c01c8c0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 -- 172.21.15.80:0/2376518394 shutdown_connections 2024-04-25T13:52:31.365 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 --2- 172.21.15.80:0/2376518394 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f512415ec40 0x7f512415f050 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 --2- 172.21.15.80:0/2376518394 >> v2:172.21.15.87:3300/0 conn(0x7f5124161f90 0x7f51240cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 -- 172.21.15.80:0/2376518394 >> 172.21.15.80:0/2376518394 conn(0x7f512406d080 msgr2=0x7f51240b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:31.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 -- 172.21.15.80:0/2376518394 shutdown_connections 2024-04-25T13:52:31.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 -- 172.21.15.80:0/2376518394 wait complete. 2024-04-25T13:52:31.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 Processor -- start 2024-04-25T13:52:31.366 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.363+0000 7f512961c640 1 -- start start 2024-04-25T13:52:31.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51240d0510 0x7f512413dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512415ec40 0x7f51240b2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5124161f90 0x7f5124072730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f512406f630 con 0x7f5124161f90 2024-04-25T13:52:31.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f512406f4b0 con 0x7f51240d0510 2024-04-25T13:52:31.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f51240d0920 con 0x7f512415ec40 2024-04-25T13:52:31.367 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5124161f90 0x7f5124072730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5124161f90 0x7f5124072730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37550/0 (socket says 172.21.15.80:37550) 2024-04-25T13:52:31.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 -- 172.21.15.80:0/2769356859 learned_addr learned my addr 172.21.15.80:0/2769356859 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:31.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f5122ffd640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51240d0510 0x7f512413dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51227fc640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512415ec40 0x7f51240b2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 -- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512415ec40 msgr2=0x7f51240b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.368 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512415ec40 0x7f51240b2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 -- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51240d0510 msgr2=0x7f512413dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51240d0510 0x7f512413dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 -- 172.21.15.80:0/2769356859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51241618c0 con 0x7f5124161f90 2024-04-25T13:52:31.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f51237fe640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5124161f90 0x7f5124072730 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f511c0af040 tx=0x7f511c0196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:31.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f510b7fe640 1 -- 172.21.15.80:0/2769356859 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c009510 con 0x7f5124161f90 2024-04-25T13:52:31.369 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f510b7fe640 1 -- 172.21.15.80:0/2769356859 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f511c0be070 con 0x7f5124161f90 2024-04-25T13:52:31.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f510b7fe640 1 -- 172.21.15.80:0/2769356859 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f511c0b8890 con 0x7f5124161f90 2024-04-25T13:52:31.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512413cc30 con 0x7f5124161f90 2024-04-25T13:52:31.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512413d1f0 con 0x7f5124161f90 2024-04-25T13:52:31.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.367+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f512413c980 con 0x7f5124161f90 2024-04-25T13:52:31.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.371+0000 7f510b7fe640 1 -- 172.21.15.80:0/2769356859 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f511c0b8a30 con 0x7f5124161f90 2024-04-25T13:52:31.370 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.371+0000 7f510b7fe640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f400 0x7f5110071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.371+0000 7f510b7fe640 1 -- 172.21.15.80:0/2769356859 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f511c12ac70 con 0x7f5124161f90 2024-04-25T13:52:31.371 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.371+0000 7f5122ffd640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f400 0x7f5110071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.371+0000 7f5122ffd640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f400 0x7f5110071910 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f512406fa20 tx=0x7f5114008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:31.372 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.375+0000 7f510b7fe640 1 -- 172.21.15.80:0/2769356859 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f511c0165a0 con 0x7f5124161f90 2024-04-25T13:52:31.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.535+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f512406a3a0 con 0x7f5124161f90 2024-04-25T13:52:31.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f510b7fe640 1 -- 172.21.15.80:0/2769356859 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) ==== 74+0+16607 (secure 0 0 0) 0x7f511c0f7710 con 0x7f5124161f90 2024-04-25T13:52:31.532 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:31.533 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":190,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:52:26.300379+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:51:48.404349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T13:51:49.039512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:52:31.535 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f400 msgr2=0x7f5110071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f400 0x7f5110071910 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f512406fa20 tx=0x7f5114008040 comp rx=0 tx=0).stop 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5124161f90 msgr2=0x7f5124072730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5124161f90 0x7f5124072730 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f511c0af040 tx=0x7f511c0196c0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 shutdown_connections 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f511006f400 0x7f5110071910 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5124161f90 0x7f5124072730 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f512415ec40 0x7f51240b2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 --2- 172.21.15.80:0/2769356859 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f51240d0510 0x7f512413dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 >> 172.21.15.80:0/2769356859 conn(0x7f512406d080 msgr2=0x7f51240d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 shutdown_connections 2024-04-25T13:52:31.536 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.539+0000 7f512961c640 1 -- 172.21.15.80:0/2769356859 wait complete. 2024-04-25T13:52:31.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:51:48.404349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:51:49.039512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:52:31.556 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:52:31.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b3d3ae640 1 Processor -- start 2024-04-25T13:52:31.708 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b3d3ae640 1 -- start start 2024-04-25T13:52:31.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b3d3ae640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f5b38161170 0x7f5b381574c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b3d3ae640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b38073c40 0x7f5b380b4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b3d3ae640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b3806fbf0 con 0x7f5b38071e60 2024-04-25T13:52:31.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b3d3ae640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f5b380b1350 con 0x7f5b38161170 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b3d3ae640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b3806f9a0 con 0x7f5b38073c40 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b36ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f5b38071e60 0x7f5b38160a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41562/0 (socket says 172.21.15.80:41562) 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b36ffd640 1 -- 172.21.15.80:0/796372813 learned_addr learned my addr 172.21.15.80:0/796372813 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/796372813 >> v2:172.21.15.87:3300/0 conn(0x7f5b38161170 0x7f5b381574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b377fe640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b38073c40 0x7f5b380b4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1902937749 0 0) 0x7f5b3806fbf0 con 0x7f5b38071e60 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5b24005740 con 0x7f5b38071e60 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1959007390 0 0) 0x7f5b24005740 con 0x7f5b38071e60 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b38073c40 msgr2=0x7f5b380b4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b38073c40 0x7f5b380b4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 >> v2:172.21.15.87:3300/0 conn(0x7f5b38161170 msgr2=0x7f5b381574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 --2- 172.21.15.80:0/796372813 >> v2:172.21.15.87:3300/0 conn(0x7f5b38161170 0x7f5b381574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b380b0a20 con 0x7f5b38071e60 2024-04-25T13:52:31.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f5b28001680 con 0x7f5b38071e60 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 >> v1:172.21.15.80:6789/0 conn(0x7f5b38071e60 legacy=0x7f5b38160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b24005bd0 0x7f5b24006040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b24006e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b38073c40 0x7f5b24007370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5b380b1350 con 0x7f5b24006a20 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3806f9a0 con 0x7f5b24005bd0 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b2400dbe0 con 0x7f5b38073c40 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b24006e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b36ffd640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b24005bd0 0x7f5b24006040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b2ffff640 1 -- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b38073c40 msgr2=0x7f5b24007370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b38073c40 0x7f5b24007370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b2ffff640 1 -- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b24005bd0 msgr2=0x7f5b24006040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b24005bd0 0x7f5b24006040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.715+0000 7f5b2ffff640 1 -- 172.21.15.80:0/796372813 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b24005740 con 0x7f5b24006a20 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b24006e30 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009ae0 tx=0x7f5b2001cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e313b303b8504492 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200b2070 con 0x7f5b24006a20 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b200a56a0 con 0x7f5b24006a20 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b367fc640 1 -- 172.21.15.80:0/796372813 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200096b0 con 0x7f5b24006a20 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/796372813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 msgr2=0x7f5b24006e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b3d3ae640 1 --2- 172.21.15.80:0/796372813 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b24006e30 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009ae0 tx=0x7f5b2001cab0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b377fe640 1 -- 172.21.15.80:0/796372813 reap_dead start 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/796372813 shutdown_connections 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/796372813 >> 172.21.15.80:0/796372813 conn(0x7f5b380b4160 msgr2=0x7f5b380d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/796372813 shutdown_connections 2024-04-25T13:52:31.715 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.719+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/796372813 wait complete. 2024-04-25T13:52:31.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 Processor -- start 2024-04-25T13:52:31.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 -- start start 2024-04-25T13:52:31.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b38145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b381f9b40 0x7f5b38142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b380cf0e0 0x7f5b3815b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38072270 con 0x7f5b24006a20 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f5b380b0fe0 con 0x7f5b381f9b40 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f5b38074050 con 0x7f5b380cf0e0 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b38145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b38145e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37570/0 (socket says 172.21.15.80:37570) 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 -- 172.21.15.80:0/3149571607 learned_addr learned my addr 172.21.15.80:0/3149571607 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b377fe640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b380cf0e0 0x7f5b3815b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b381f9b40 0x7f5b38142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 -- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b380cf0e0 msgr2=0x7f5b3815b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b380cf0e0 0x7f5b3815b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 -- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b381f9b40 msgr2=0x7f5b38142b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b381f9b40 0x7f5b38142b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 -- 172.21.15.80:0/3149571607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b381f6400 con 0x7f5b24006a20 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b381f9b40 0x7f5b38142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b377fe640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b380cf0e0 0x7f5b3815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b36ffd640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b38145e90 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f5b28010c60 tx=0x7f5b2800e4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b2f7fe640 1 -- 172.21.15.80:0/3149571607 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2801b070 con 0x7f5b24006a20 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b2f7fe640 1 -- 172.21.15.80:0/3149571607 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5b28011040 con 0x7f5b24006a20 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b381fa270 con 0x7f5b24006a20 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b2f7fe640 1 -- 172.21.15.80:0/3149571607 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b280145e0 con 0x7f5b24006a20 2024-04-25T13:52:31.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b381fa830 con 0x7f5b24006a20 2024-04-25T13:52:31.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.723+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b381faa70 con 0x7f5b24006a20 2024-04-25T13:52:31.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.727+0000 7f5b2f7fe640 1 -- 172.21.15.80:0/3149571607 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f5b2801e070 con 0x7f5b24006a20 2024-04-25T13:52:31.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.727+0000 7f5b2f7fe640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1806f430 0x7f5b18071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:31.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.727+0000 7f5b2f7fe640 1 -- 172.21.15.80:0/3149571607 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f5b28091f20 con 0x7f5b24006a20 2024-04-25T13:52:31.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.727+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1806f430 0x7f5b18071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:31.722 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.727+0000 7f5b2ffff640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1806f430 0x7f5b18071940 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5b200af000 tx=0x7f5b200046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:31.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.731+0000 7f5b2f7fe640 1 -- 172.21.15.80:0/3149571607 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f5b2805ee10 con 0x7f5b24006a20 2024-04-25T13:52:31.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.879+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5b3806a3a0 con 0x7f5b24006a20 2024-04-25T13:52:31.876 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.883+0000 7f5b2f7fe640 1 -- 172.21.15.80:0/3149571607 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) ==== 93+0+10 (secure 0 0 0) 0x7f5b2805e710 con 0x7f5b24006a20 2024-04-25T13:52:31.876 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:52:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1806f430 msgr2=0x7f5b18071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1806f430 0x7f5b18071940 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5b200af000 tx=0x7f5b200046d0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 msgr2=0x7f5b38145e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b38145e90 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f5b28010c60 tx=0x7f5b2800e4d0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 shutdown_connections 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f5b1806f430 0x7f5b18071940 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f5b380cf0e0 0x7f5b3815b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5b381f9b40 0x7f5b38142b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 --2- 172.21.15.80:0/3149571607 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5b24006a20 0x7f5b38145e90 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 >> 172.21.15.80:0/3149571607 conn(0x7f5b380b4160 msgr2=0x7f5b38072480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 shutdown_connections 2024-04-25T13:52:31.881 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:31.887+0000 7f5b3d3ae640 1 -- 172.21.15.80:0/3149571607 wait complete. 2024-04-25T13:52:31.899 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:52:32.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd309f640 1 Processor -- start 2024-04-25T13:52:32.044 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd309f640 1 -- start start 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd309f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc0d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd309f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcbcc0ba770 0x7fcbcc0b7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd309f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcbcc0b92a0 con 0x7fcbcc0d2ad0 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd309f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fcbcc0d4c10 con 0x7fcbcc0ba770 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd309f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcbcc0d4ef0 con 0x7fcbcc0d3ed0 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd0e3b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc0d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd163c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fcbcc0ba770 0x7fcbcc0b7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd0e3b640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc0d2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:50002/0 (socket says 172.21.15.80:50002) 2024-04-25T13:52:32.046 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd0e3b640 1 -- 172.21.15.80:0/3240710194 learned_addr learned my addr 172.21.15.80:0/3240710194 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:32.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd0e3b640 1 -- 172.21.15.80:0/3240710194 >> v2:172.21.15.87:3300/0 conn(0x7fcbcc0ba770 msgr2=0x7fcbcc0b7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:32.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd0e3b640 1 --2- 172.21.15.80:0/3240710194 >> v2:172.21.15.87:3300/0 conn(0x7fcbcc0ba770 0x7fcbcc0b7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd0e3b640 1 -- 172.21.15.80:0/3240710194 >> v1:172.21.15.87:6789/0 conn(0x7fcbcc0d2ad0 legacy=0x7fcbcc0b76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.047 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.051+0000 7fcbd0e3b640 1 -- 172.21.15.80:0/3240710194 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc0d51d0 con 0x7fcbcc0d3ed0 2024-04-25T13:52:32.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd0e3b640 1 --2- 172.21.15.80:0/3240710194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc0d2590 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009870 tx=0x7fcbc801c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37a54e102cbbbea6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbc37fe640 1 -- 172.21.15.80:0/3240710194 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80aa650 con 0x7fcbcc0d3ed0 2024-04-25T13:52:32.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbc37fe640 1 -- 172.21.15.80:0/3240710194 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc80af070 con 0x7fcbcc0d3ed0 2024-04-25T13:52:32.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbc37fe640 1 -- 172.21.15.80:0/3240710194 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80aaa10 con 0x7fcbcc0d3ed0 2024-04-25T13:52:32.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd309f640 1 -- 172.21.15.80:0/3240710194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 msgr2=0x7fcbcc0d2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd309f640 1 --2- 172.21.15.80:0/3240710194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc0d2590 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009870 tx=0x7fcbc801c4d0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd309f640 1 -- 172.21.15.80:0/3240710194 shutdown_connections 2024-04-25T13:52:32.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd309f640 1 --2- 172.21.15.80:0/3240710194 >> v2:172.21.15.87:3300/0 conn(0x7fcbcc0ba770 0x7fcbcc0b7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd309f640 1 --2- 172.21.15.80:0/3240710194 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc0d2590 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd309f640 1 -- 172.21.15.80:0/3240710194 >> 172.21.15.80:0/3240710194 conn(0x7fcbcc0737b0 msgr2=0x7fcbcc06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:32.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd309f640 1 -- 172.21.15.80:0/3240710194 shutdown_connections 2024-04-25T13:52:32.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.055+0000 7fcbd309f640 1 -- 172.21.15.80:0/3240710194 wait complete. 2024-04-25T13:52:32.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 Processor -- start 2024-04-25T13:52:32.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 -- start start 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbcc0ba770 0x7fcbcc0b7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbcc0d2ad0 0x7fcbcc14d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc0b8320 con 0x7fcbcc0ba770 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fcbcc0b84a0 con 0x7fcbcc0d2ad0 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fcbcc160620 con 0x7fcbcc0d3ed0 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbcc0ba770 0x7fcbcc0b7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbcc0ba770 0x7fcbcc0b7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37578/0 (socket says 172.21.15.80:37578) 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 -- 172.21.15.80:0/1530153539 learned_addr learned my addr 172.21.15.80:0/1530153539 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbc3fff640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbcc0d2ad0 0x7fcbcc14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd163c640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 -- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 msgr2=0x7fcbcc159a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.055 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc159a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 -- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbcc0d2ad0 msgr2=0x7fcbcc14d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbcc0d2ad0 0x7fcbcc14d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 -- 172.21.15.80:0/1530153539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbcc153240 con 0x7fcbcc0ba770 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd0e3b640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbcc0ba770 0x7fcbcc0b7ca0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fcbc801a100 tx=0x7fcbc801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbc17fa640 1 -- 172.21.15.80:0/1530153539 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80bd5f0 con 0x7fcbcc0ba770 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbc17fa640 1 -- 172.21.15.80:0/1530153539 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbc80a6040 con 0x7fcbcc0ba770 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbcc153520 con 0x7fcbcc0ba770 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbc17fa640 1 -- 172.21.15.80:0/1530153539 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbc80bd9f0 con 0x7fcbcc0ba770 2024-04-25T13:52:32.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbcc153ae0 con 0x7fcbcc0ba770 2024-04-25T13:52:32.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.059+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbcc06a3a0 con 0x7fcbcc0ba770 2024-04-25T13:52:32.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.063+0000 7fcbc17fa640 1 -- 172.21.15.80:0/1530153539 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fcbc80b33f0 con 0x7fcbcc0ba770 2024-04-25T13:52:32.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.063+0000 7fcbc17fa640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcbb806f430 0x7fcbb8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.063+0000 7fcbc17fa640 1 -- 172.21.15.80:0/1530153539 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7fcbc81297a0 con 0x7fcbcc0ba770 2024-04-25T13:52:32.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.063+0000 7fcbc3fff640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcbb806f430 0x7fcbb8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.057 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.063+0000 7fcbc3fff640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcbb806f430 0x7fcbb8071940 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc15e2d0 tx=0x7fcbbc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.060 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.067+0000 7fcbc17fa640 1 -- 172.21.15.80:0/1530153539 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fcbc80c3060 con 0x7fcbcc0ba770 2024-04-25T13:52:32.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fcbcc0c9740 con 0x7fcbcc0ba770 2024-04-25T13:52:32.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbc17fa640 1 -- 172.21.15.80:0/1530153539 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) ==== 104+0+11 (secure 0 0 0) 0x7fcbc8016150 con 0x7fcbcc0ba770 2024-04-25T13:52:32.221 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:52:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcbb806f430 msgr2=0x7fcbb8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcbb806f430 0x7fcbb8071940 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc15e2d0 tx=0x7fcbbc009040 comp rx=0 tx=0).stop 2024-04-25T13:52:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbcc0ba770 msgr2=0x7fcbcc0b7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbcc0ba770 0x7fcbcc0b7ca0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fcbc801a100 tx=0x7fcbc801ad10 comp rx=0 tx=0).stop 2024-04-25T13:52:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 shutdown_connections 2024-04-25T13:52:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fcbb806f430 0x7fcbb8071940 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fcbcc0d3ed0 0x7fcbcc159a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcbcc0d2ad0 0x7fcbcc14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 --2- 172.21.15.80:0/1530153539 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fcbcc0ba770 0x7fcbcc0b7ca0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 >> 172.21.15.80:0/1530153539 conn(0x7fcbcc0737b0 msgr2=0x7fcbcc15b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 shutdown_connections 2024-04-25T13:52:32.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.227+0000 7fcbd309f640 1 -- 172.21.15.80:0/1530153539 wait complete. 2024-04-25T13:52:32.242 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:52:32.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd9279f8640 1 Processor -- start 2024-04-25T13:52:32.384 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd9279f8640 1 -- start start 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd9279f8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200d3ed0 0x7fd9200d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd9279f8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd9200d2ad0 0x7fd9200b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd9279f8640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd9200b7c10 0x7fd9200b8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd9279f8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd9200b8560 con 0x7fd9200d2ad0 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd9279f8640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd9200b92a0 con 0x7fd9200b7c10 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd9279f8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9200d4c10 con 0x7fd9200d3ed0 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd924f93640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd9200d2ad0 0x7fd9200b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd924f93640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd9200d2ad0 0x7fd9200b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37586/0 (socket says 172.21.15.80:37586) 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd924f93640 1 -- 172.21.15.80:0/4092491886 learned_addr learned my addr 172.21.15.80:0/4092491886 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd925794640 1 --2- 172.21.15.80:0/4092491886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200d3ed0 0x7fd9200d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.387 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd925f95640 1 --2- 172.21.15.80:0/4092491886 >> v2:172.21.15.87:3300/0 conn(0x7fd9200b7c10 0x7fd9200b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd924f93640 1 -- 172.21.15.80:0/4092491886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200d3ed0 msgr2=0x7fd9200d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd924f93640 1 --2- 172.21.15.80:0/4092491886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200d3ed0 0x7fd9200d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd924f93640 1 -- 172.21.15.80:0/4092491886 >> v2:172.21.15.87:3300/0 conn(0x7fd9200b7c10 msgr2=0x7fd9200b8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd924f93640 1 --2- 172.21.15.80:0/4092491886 >> v2:172.21.15.87:3300/0 conn(0x7fd9200b7c10 0x7fd9200b8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.388 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.391+0000 7fd924f93640 1 -- 172.21.15.80:0/4092491886 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9200d4ef0 con 0x7fd9200d2ad0 2024-04-25T13:52:32.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd924f93640 1 --2- 172.21.15.80:0/4092491886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d2ad0 0x7fd9200b76d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fd9200b8dc0 tx=0x7fd91001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8637f0976fa053 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd917fff640 1 -- 172.21.15.80:0/4092491886 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9100ab620 con 0x7fd9200d2ad0 2024-04-25T13:52:32.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd917fff640 1 -- 172.21.15.80:0/4092491886 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9100b0070 con 0x7fd9200d2ad0 2024-04-25T13:52:32.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd917fff640 1 -- 172.21.15.80:0/4092491886 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9100ab9e0 con 0x7fd9200d2ad0 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 -- 172.21.15.80:0/4092491886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d2ad0 msgr2=0x7fd9200b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 --2- 172.21.15.80:0/4092491886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d2ad0 0x7fd9200b76d0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fd9200b8dc0 tx=0x7fd91001c460 comp rx=0 tx=0).stop 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 -- 172.21.15.80:0/4092491886 shutdown_connections 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 --2- 172.21.15.80:0/4092491886 >> v2:172.21.15.87:3300/0 conn(0x7fd9200b7c10 0x7fd9200b8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 --2- 172.21.15.80:0/4092491886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d2ad0 0x7fd9200b76d0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 --2- 172.21.15.80:0/4092491886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200d3ed0 0x7fd9200d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 -- 172.21.15.80:0/4092491886 >> 172.21.15.80:0/4092491886 conn(0x7fd9200737b0 msgr2=0x7fd92006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 -- 172.21.15.80:0/4092491886 shutdown_connections 2024-04-25T13:52:32.390 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.395+0000 7fd9279f8640 1 -- 172.21.15.80:0/4092491886 wait complete. 2024-04-25T13:52:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 Processor -- start 2024-04-25T13:52:32.391 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 -- start start 2024-04-25T13:52:32.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200b7c10 0x7fd9201530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9200d2ad0 0x7fd9201537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d3ed0 0x7fd92014f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd9200d2f90 con 0x7fd9200d3ed0 2024-04-25T13:52:32.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd920160620 con 0x7fd9200d2ad0 2024-04-25T13:52:32.394 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd9201607a0 con 0x7fd9200b7c10 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d3ed0 0x7fd92014f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d3ed0 0x7fd92014f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37596/0 (socket says 172.21.15.80:37596) 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 -- 172.21.15.80:0/3891996153 learned_addr learned my addr 172.21.15.80:0/3891996153 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925794640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200b7c10 0x7fd9201530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd924f93640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9200d2ad0 0x7fd9201537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 -- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200b7c10 msgr2=0x7fd9201530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200b7c10 0x7fd9201530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 -- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9200d2ad0 msgr2=0x7fd9201537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9200d2ad0 0x7fd9201537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 -- 172.21.15.80:0/3891996153 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd920152330 con 0x7fd9200d3ed0 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd925f95640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d3ed0 0x7fd92014f300 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fd918007d20 tx=0x7fd91800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd915ffb640 1 -- 172.21.15.80:0/3891996153 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918014420 con 0x7fd9200d3ed0 2024-04-25T13:52:32.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd915ffb640 1 -- 172.21.15.80:0/3891996153 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd918019070 con 0x7fd9200d3ed0 2024-04-25T13:52:32.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9201525d0 con 0x7fd9200d3ed0 2024-04-25T13:52:32.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd915ffb640 1 -- 172.21.15.80:0/3891996153 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd918014820 con 0x7fd9200d3ed0 2024-04-25T13:52:32.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9201fa4f0 con 0x7fd9200d3ed0 2024-04-25T13:52:32.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.399+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd920158620 con 0x7fd9200d3ed0 2024-04-25T13:52:32.396 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.403+0000 7fd915ffb640 1 -- 172.21.15.80:0/3891996153 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd9180149c0 con 0x7fd9200d3ed0 2024-04-25T13:52:32.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.403+0000 7fd915ffb640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd90406f400 0x7fd904071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.403+0000 7fd915ffb640 1 -- 172.21.15.80:0/3891996153 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7fd91808fa00 con 0x7fd9200d3ed0 2024-04-25T13:52:32.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.403+0000 7fd925794640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd90406f400 0x7fd904071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.397 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.403+0000 7fd925794640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd90406f400 0x7fd904071910 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b930 tx=0x7fd91c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.400 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.407+0000 7fd915ffb640 1 -- 172.21.15.80:0/3891996153 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd91805ca10 con 0x7fd9200d3ed0 2024-04-25T13:52:32.548 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd92006a3a0 con 0x7fd9200d3ed0 2024-04-25T13:52:32.549 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd915ffb640 1 -- 172.21.15.80:0/3891996153 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) ==== 100+0+11 (secure 0 0 0) 0x7fd91805c310 con 0x7fd9200d3ed0 2024-04-25T13:52:32.549 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd90406f400 msgr2=0x7fd904071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd90406f400 0x7fd904071910 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00b930 tx=0x7fd91c008040 comp rx=0 tx=0).stop 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d3ed0 msgr2=0x7fd92014f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d3ed0 0x7fd92014f300 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fd918007d20 tx=0x7fd91800e4b0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 shutdown_connections 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd90406f400 0x7fd904071910 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd9200d3ed0 0x7fd92014f300 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd9200d2ad0 0x7fd9201537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 --2- 172.21.15.80:0/3891996153 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd9200b7c10 0x7fd9201530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 >> 172.21.15.80:0/3891996153 conn(0x7fd9200737b0 msgr2=0x7fd9201573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 shutdown_connections 2024-04-25T13:52:32.552 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.555+0000 7fd9279f8640 1 -- 172.21.15.80:0/3891996153 wait complete. 2024-04-25T13:52:32.570 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T13:52:32.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d4045640 1 Processor -- start 2024-04-25T13:52:32.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d4045640 1 -- start start 2024-04-25T13:52:32.716 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d4045640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc0d0270 0x7f46cc070490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d4045640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46cc068e90 con 0x7f46cc068590 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d4045640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46cc069170 con 0x7f46cc144680 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d4045640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc15e9c0 con 0x7f46cc0d0270 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d15e0640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f46cc068590 0x7f46cc0cfb60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41570/0 (socket says 172.21.15.80:41570) 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d15e0640 1 -- 172.21.15.80:0/2052983993 learned_addr learned my addr 172.21.15.80:0/2052983993 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d25e2640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc0d0270 0x7f46cc070490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2678321540 0 0) 0x7f46cc068e90 con 0x7f46cc068590 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.719+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f46b8003530 con 0x7f46cc068590 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 301434327 0 0) 0x7f46b8003530 con 0x7f46cc068590 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc0d0270 msgr2=0x7f46cc070490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc0d0270 0x7f46cc070490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 >> v1:172.21.15.87:6789/0 conn(0x7f46cc144680 legacy=0x7f46cc15f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc15eca0 con 0x7f46cc068590 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f46c80016a0 con 0x7f46cc068590 2024-04-25T13:52:32.717 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 >> v1:172.21.15.80:6789/0 conn(0x7f46cc068590 legacy=0x7f46cc0cfb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b80039f0 0x7f46b8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46b8004810 0x7f46b8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8005160 0x7f46b800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc15e9c0 con 0x7f46b8005160 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f46b800bff0 con 0x7f46b80039f0 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f46b800c1f0 con 0x7f46b8004810 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d25e2640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8005160 0x7f46b800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d15e0640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46b8004810 0x7f46b8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d1de1640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b80039f0 0x7f46b8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d25e2640 1 -- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46b8004810 msgr2=0x7f46b8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d25e2640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46b8004810 0x7f46b8004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d25e2640 1 -- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b80039f0 msgr2=0x7f46b8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:32.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d25e2640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b80039f0 0x7f46b8003e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d25e2640 1 -- 172.21.15.80:0/2052983993 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b8003530 con 0x7f46b8005160 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d25e2640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8005160 0x7f46b800b9d0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009fc0 tx=0x7f46c401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3805747cfe8d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40b2070 con 0x7f46b8005160 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c40a5ba0 con 0x7f46b8005160 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d0ddf640 1 -- 172.21.15.80:0/2052983993 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4009c10 con 0x7f46b8005160 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d4045640 1 -- 172.21.15.80:0/2052983993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8005160 msgr2=0x7f46b800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d4045640 1 --2- 172.21.15.80:0/2052983993 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46b8005160 0x7f46b800b9d0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f46c4009fc0 tx=0x7f46c401cca0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d25e2640 1 -- 172.21.15.80:0/2052983993 reap_dead start 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d4045640 1 -- 172.21.15.80:0/2052983993 shutdown_connections 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d4045640 1 -- 172.21.15.80:0/2052983993 >> 172.21.15.80:0/2052983993 conn(0x7f46cc0d33a0 msgr2=0x7f46cc0cf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d4045640 1 -- 172.21.15.80:0/2052983993 shutdown_connections 2024-04-25T13:52:32.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.723+0000 7f46d4045640 1 -- 172.21.15.80:0/2052983993 wait complete. 2024-04-25T13:52:32.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 Processor -- start 2024-04-25T13:52:32.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 -- start start 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b8005160 0x7f46cc15b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46cc13e720 0x7f46cc146b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc15b630 0x7f46cc147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0689a0 con 0x7f46cc13e720 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0d0480 con 0x7f46b8005160 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc068b20 con 0x7f46cc15b630 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46cc13e720 0x7f46cc146b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46cc13e720 0x7f46cc146b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37620/0 (socket says 172.21.15.80:37620) 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 -- 172.21.15.80:0/1961804341 learned_addr learned my addr 172.21.15.80:0/1961804341 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d1de1640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b8005160 0x7f46cc15b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d25e2640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc15b630 0x7f46cc147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.724 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 -- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc15b630 msgr2=0x7f46cc147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc15b630 0x7f46cc147080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 -- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b8005160 msgr2=0x7f46cc15b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b8005160 0x7f46cc15b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 -- 172.21.15.80:0/1961804341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc1f5c70 con 0x7f46cc13e720 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d1de1640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b8005160 0x7f46cc15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d15e0640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46cc13e720 0x7f46cc146b40 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f46c80117c0 tx=0x7f46c8011c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46be7fc640 1 -- 172.21.15.80:0/1961804341 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8010040 con 0x7f46cc13e720 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46be7fc640 1 -- 172.21.15.80:0/1961804341 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46c801a070 con 0x7f46cc13e720 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc1f5f50 con 0x7f46cc13e720 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46be7fc640 1 -- 172.21.15.80:0/1961804341 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c800fde0 con 0x7f46cc13e720 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.727+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc1f6510 con 0x7f46cc13e720 2024-04-25T13:52:32.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.731+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc1f59e0 con 0x7f46cc13e720 2024-04-25T13:52:32.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.731+0000 7f46be7fc640 1 -- 172.21.15.80:0/1961804341 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f46c801f460 con 0x7f46cc13e720 2024-04-25T13:52:32.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.731+0000 7f46be7fc640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f46b006f430 0x7f46b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:32.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.731+0000 7f46be7fc640 1 -- 172.21.15.80:0/1961804341 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f46c80922e0 con 0x7f46cc13e720 2024-04-25T13:52:32.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.731+0000 7f46d1de1640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f46b006f430 0x7f46b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:32.726 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.731+0000 7f46d1de1640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f46b006f430 0x7f46b0071940 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0695c0 tx=0x7f46c0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:32.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.735+0000 7f46be7fc640 1 -- 172.21.15.80:0/1961804341 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f46c805f2f0 con 0x7f46cc13e720 2024-04-25T13:52:32.901 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.907+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f46cc06a3a0 con 0x7f46cc13e720 2024-04-25T13:52:32.902 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.907+0000 7f46be7fc640 1 -- 172.21.15.80:0/1961804341 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v190) ==== 80+0+0 (secure 0 0 0) 0x7f46c805ebf0 con 0x7f46cc13e720 2024-04-25T13:52:32.902 INFO:teuthology.orchestra.run.smithi080.stderr:listed 0 entries 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f46b006f430 msgr2=0x7f46b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f46b006f430 0x7f46b0071940 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f46cc0695c0 tx=0x7f46c0009040 comp rx=0 tx=0).stop 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46cc13e720 msgr2=0x7f46cc146b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46cc13e720 0x7f46cc146b40 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f46c80117c0 tx=0x7f46c8011c80 comp rx=0 tx=0).stop 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 shutdown_connections 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f46b006f430 0x7f46b0071940 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f46cc15b630 0x7f46cc147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46cc13e720 0x7f46cc146b40 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 --2- 172.21.15.80:0/1961804341 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46b8005160 0x7f46cc15b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:32.905 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 >> 172.21.15.80:0/1961804341 conn(0x7f46cc0d33a0 msgr2=0x7f46cc143730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:32.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 shutdown_connections 2024-04-25T13:52:32.906 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:32.911+0000 7f46d4045640 1 -- 172.21.15.80:0/1961804341 wait complete. 2024-04-25T13:52:32.926 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi087.front.sepia.ceph.com... 2024-04-25T13:52:32.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:32.926 DEBUG:teuthology.orchestra.run.smithi087:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T13:52:32.939 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi087.stderr:ceph-fuse[25767]: fuse finished with error 0 and tester_r 0 2024-04-25T13:52:32.941 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:52:32.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:32.941 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T13:52:33.108 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:52:33.109 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:33.109 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:33.115 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:52:33.115 INFO:tasks.cephfs.mount:Terminating background process 2024-04-25T13:52:33.115 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:52:33.115 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:33.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:52:33.158 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:33.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe888b8640 1 Processor -- start 2024-04-25T13:52:33.313 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe888b8640 1 -- start start 2024-04-25T13:52:33.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe888b8640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7efe840b0ed0 0x7efe8415c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe888b8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe840ce730 0x7efe840ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe888b8640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe8406e490 0x7efe8406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe888b8640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7efe8406ede0 con 0x7efe8406e490 2024-04-25T13:52:33.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe888b8640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7efe8415cda0 con 0x7efe840b0ed0 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe888b8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe84073710 con 0x7efe840ce730 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe82d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe8406e490 0x7efe8406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe82d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7efe8406e490 0x7efe8406e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37634/0 (socket says 172.21.15.80:37634) 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe82d9d640 1 -- 172.21.15.80:0/935284009 learned_addr learned my addr 172.21.15.80:0/935284009 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe81d9b640 1 --2- 172.21.15.80:0/935284009 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe840ce730 0x7efe840ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe8259c640 1 --2- 172.21.15.80:0/935284009 >> v2:172.21.15.87:3300/0 conn(0x7efe840b0ed0 0x7efe8415c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe82d9d640 1 -- 172.21.15.80:0/935284009 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe840ce730 msgr2=0x7efe840ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe82d9d640 1 --2- 172.21.15.80:0/935284009 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe840ce730 0x7efe840ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe82d9d640 1 -- 172.21.15.80:0/935284009 >> v2:172.21.15.87:3300/0 conn(0x7efe840b0ed0 msgr2=0x7efe8415c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe82d9d640 1 --2- 172.21.15.80:0/935284009 >> v2:172.21.15.87:3300/0 conn(0x7efe840b0ed0 0x7efe8415c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.319+0000 7efe82d9d640 1 -- 172.21.15.80:0/935284009 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe840739f0 con 0x7efe8406e490 2024-04-25T13:52:33.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.323+0000 7efe82d9d640 1 --2- 172.21.15.80:0/935284009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe8406e490 0x7efe8406e8a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7efe84071650 tx=0x7efe6c01ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1e1f88fedb9c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:33.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.323+0000 7efe8159a640 1 -- 172.21.15.80:0/935284009 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0b5070 con 0x7efe8406e490 2024-04-25T13:52:33.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.323+0000 7efe8159a640 1 -- 172.21.15.80:0/935284009 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c0a89d0 con 0x7efe8406e490 2024-04-25T13:52:33.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.323+0000 7efe8159a640 1 -- 172.21.15.80:0/935284009 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0a8da0 con 0x7efe8406e490 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.323+0000 7efe888b8640 1 -- 172.21.15.80:0/935284009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe8406e490 msgr2=0x7efe8406e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.323+0000 7efe888b8640 1 --2- 172.21.15.80:0/935284009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe8406e490 0x7efe8406e8a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7efe84071650 tx=0x7efe6c01ed10 comp rx=0 tx=0).stop 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 -- 172.21.15.80:0/935284009 shutdown_connections 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 --2- 172.21.15.80:0/935284009 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe8406e490 0x7efe8406e8a0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 --2- 172.21.15.80:0/935284009 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe840ce730 0x7efe840ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 --2- 172.21.15.80:0/935284009 >> v2:172.21.15.87:3300/0 conn(0x7efe840b0ed0 0x7efe8415c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 -- 172.21.15.80:0/935284009 >> 172.21.15.80:0/935284009 conn(0x7efe8406d080 msgr2=0x7efe840b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 -- 172.21.15.80:0/935284009 shutdown_connections 2024-04-25T13:52:33.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 -- 172.21.15.80:0/935284009 wait complete. 2024-04-25T13:52:33.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 Processor -- start 2024-04-25T13:52:33.321 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 -- start start 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe8406e490 0x7efe84158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe840b0ed0 0x7efe8415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe840ce730 0x7efe841efc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7efe840b12e0 con 0x7efe840ce730 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7efe84073fb0 con 0x7efe840b0ed0 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe888b8640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7efe840733a0 con 0x7efe8406e490 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe82d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe840ce730 0x7efe841efc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe82d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe840ce730 0x7efe841efc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37650/0 (socket says 172.21.15.80:37650) 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe82d9d640 1 -- 172.21.15.80:0/4168991080 learned_addr learned my addr 172.21.15.80:0/4168991080 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe8259c640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe8406e490 0x7efe84158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.324 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe81d9b640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe840b0ed0 0x7efe8415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe82d9d640 1 -- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe8406e490 msgr2=0x7efe84158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe82d9d640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe8406e490 0x7efe84158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe82d9d640 1 -- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe840b0ed0 msgr2=0x7efe8415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe82d9d640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe840b0ed0 0x7efe8415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.327+0000 7efe82d9d640 1 -- 172.21.15.80:0/4168991080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8415f2e0 con 0x7efe840ce730 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe82d9d640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe840ce730 0x7efe841efc00 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0b3040 tx=0x7efe6c01bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe7affd640 1 -- 172.21.15.80:0/4168991080 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0b5070 con 0x7efe840ce730 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe7affd640 1 -- 172.21.15.80:0/4168991080 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe6c0b09d0 con 0x7efe840ce730 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8415f5c0 con 0x7efe840ce730 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe7affd640 1 -- 172.21.15.80:0/4168991080 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c0b0d80 con 0x7efe840ce730 2024-04-25T13:52:33.325 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe8415a9d0 con 0x7efe840ce730 2024-04-25T13:52:33.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe8406a3a0 con 0x7efe840ce730 2024-04-25T13:52:33.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe7affd640 1 -- 172.21.15.80:0/4168991080 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7efe6c0c2060 con 0x7efe840ce730 2024-04-25T13:52:33.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe7affd640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6406f400 0x7efe64071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe7affd640 1 -- 172.21.15.80:0/4168991080 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7efe6c12c7f0 con 0x7efe840ce730 2024-04-25T13:52:33.326 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe8259c640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6406f400 0x7efe64071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.327 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.331+0000 7efe8259c640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6406f400 0x7efe64071910 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7efe7c004470 tx=0x7efe7c003a40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:33.330 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.335+0000 7efe7affd640 1 -- 172.21.15.80:0/4168991080 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7efe6c018870 con 0x7efe840ce730 2024-04-25T13:52:33.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.515+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe8406f880 con 0x7efe840ce730 2024-04-25T13:52:33.512 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe7affd640 1 -- 172.21.15.80:0/4168991080 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7efe6c0f97c0 con 0x7efe840ce730 2024-04-25T13:52:33.512 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:33.512 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:52:33.513 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:52:33.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6406f400 msgr2=0x7efe64071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.514 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6406f400 0x7efe64071910 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7efe7c004470 tx=0x7efe7c003a40 comp rx=0 tx=0).stop 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe840ce730 msgr2=0x7efe841efc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe840ce730 0x7efe841efc00 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0b3040 tx=0x7efe6c01bab0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 shutdown_connections 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7efe6406f400 0x7efe64071910 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7efe840ce730 0x7efe841efc00 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7efe840b0ed0 0x7efe8415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 --2- 172.21.15.80:0/4168991080 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7efe8406e490 0x7efe84158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 >> 172.21.15.80:0/4168991080 conn(0x7efe8406d080 msgr2=0x7efe841576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 shutdown_connections 2024-04-25T13:52:33.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.519+0000 7efe888b8640 1 -- 172.21.15.80:0/4168991080 wait complete. 2024-04-25T13:52:33.533 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:52:33.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f25502e9640 1 Processor -- start 2024-04-25T13:52:33.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f25502e9640 1 -- start start 2024-04-25T13:52:33.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f25502e9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f25480b0ed0 0x7f254815c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f25502e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25480ce730 0x7f25480ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f25502e9640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f254815cda0 con 0x7f25481608f0 2024-04-25T13:52:33.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f25502e9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f2548161410 con 0x7f25480b0ed0 2024-04-25T13:52:33.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f25502e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2548073710 con 0x7f25480ce730 2024-04-25T13:52:33.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254e085640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f25480b0ed0 0x7f254815c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254e085640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f25480b0ed0 0x7f254815c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37662/0 (socket says 172.21.15.80:37662) 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254e085640 1 -- 172.21.15.80:0/2218688662 learned_addr learned my addr 172.21.15.80:0/2218688662 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254e886640 1 --1- 172.21.15.80:0/2218688662 >> v1:172.21.15.80:6789/0 conn(0x7f25481608f0 0x7f2548160d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41578/0 (socket says 172.21.15.80:41578) 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254d884640 1 --2- 172.21.15.80:0/2218688662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25480ce730 0x7f25480ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254e085640 1 -- 172.21.15.80:0/2218688662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25480ce730 msgr2=0x7f25480ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254e085640 1 --2- 172.21.15.80:0/2218688662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25480ce730 0x7f25480ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254e085640 1 -- 172.21.15.80:0/2218688662 >> v1:172.21.15.80:6789/0 conn(0x7f25481608f0 legacy=0x7f2548160d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254d083640 1 -- 172.21.15.80:0/2218688662 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3621638831 0 0) 0x7f254815cda0 con 0x7f25481608f0 2024-04-25T13:52:33.684 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.687+0000 7f254e085640 1 -- 172.21.15.80:0/2218688662 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25480739f0 con 0x7f25480b0ed0 2024-04-25T13:52:33.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.691+0000 7f254e085640 1 --2- 172.21.15.80:0/2218688662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254815c860 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f2544009870 tx=0x7f254401ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5a3d363d69fda9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:33.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.691+0000 7f254d083640 1 -- 172.21.15.80:0/2218688662 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440b5070 con 0x7f25480b0ed0 2024-04-25T13:52:33.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.691+0000 7f254d083640 1 -- 172.21.15.80:0/2218688662 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25440a8a70 con 0x7f25480b0ed0 2024-04-25T13:52:33.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.691+0000 7f254d083640 1 -- 172.21.15.80:0/2218688662 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440a8e40 con 0x7f25480b0ed0 2024-04-25T13:52:33.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.691+0000 7f25502e9640 1 -- 172.21.15.80:0/2218688662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 msgr2=0x7f254815c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.691+0000 7f25502e9640 1 --2- 172.21.15.80:0/2218688662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254815c860 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f2544009870 tx=0x7f254401ed70 comp rx=0 tx=0).stop 2024-04-25T13:52:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 -- 172.21.15.80:0/2218688662 shutdown_connections 2024-04-25T13:52:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 --2- 172.21.15.80:0/2218688662 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25480ce730 0x7f25480ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 --2- 172.21.15.80:0/2218688662 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254815c860 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 -- 172.21.15.80:0/2218688662 >> 172.21.15.80:0/2218688662 conn(0x7f254806d080 msgr2=0x7f25480b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 -- 172.21.15.80:0/2218688662 shutdown_connections 2024-04-25T13:52:33.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 -- 172.21.15.80:0/2218688662 wait complete. 2024-04-25T13:52:33.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 Processor -- start 2024-04-25T13:52:33.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 -- start start 2024-04-25T13:52:33.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254814d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25480ce730 0x7f2548158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481608f0 0x7f25480af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f25480b12e0 con 0x7f25480b0ed0 2024-04-25T13:52:33.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2548073fb0 con 0x7f25480ce730 2024-04-25T13:52:33.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f25502e9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f25480733a0 con 0x7f25481608f0 2024-04-25T13:52:33.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e085640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254814d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e085640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254814d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37664/0 (socket says 172.21.15.80:37664) 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e085640 1 -- 172.21.15.80:0/4223879886 learned_addr learned my addr 172.21.15.80:0/4223879886 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254d884640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25480ce730 0x7f2548158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e886640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481608f0 0x7f25480af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e085640 1 -- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481608f0 msgr2=0x7f25480af8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e085640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481608f0 0x7f25480af8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e085640 1 -- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25480ce730 msgr2=0x7f2548158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e085640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25480ce730 0x7f2548158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e085640 1 -- 172.21.15.80:0/4223879886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f254815afa0 con 0x7f25480b0ed0 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.695+0000 7f254e886640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481608f0 0x7f25480af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f254e085640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254814d300 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f254401c9a0 tx=0x7f254401bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f253effd640 1 -- 172.21.15.80:0/4223879886 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440c0070 con 0x7f25480b0ed0 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f253effd640 1 -- 172.21.15.80:0/4223879886 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25440ae550 con 0x7f25480b0ed0 2024-04-25T13:52:33.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f254815b280 con 0x7f25480b0ed0 2024-04-25T13:52:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f253effd640 1 -- 172.21.15.80:0/4223879886 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25440b5070 con 0x7f25480b0ed0 2024-04-25T13:52:33.694 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f254815b840 con 0x7f25480b0ed0 2024-04-25T13:52:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f254806a3a0 con 0x7f25480b0ed0 2024-04-25T13:52:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f253effd640 1 -- 172.21.15.80:0/4223879886 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f25440c3070 con 0x7f25480b0ed0 2024-04-25T13:52:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f253effd640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f252c06f400 0x7f252c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f253effd640 1 -- 172.21.15.80:0/4223879886 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f254412cb00 con 0x7f25480b0ed0 2024-04-25T13:52:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f254d884640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f252c06f400 0x7f252c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:33.695 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.699+0000 7f254d884640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f252c06f400 0x7f252c071910 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2548158960 tx=0x7f2538008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:33.698 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.703+0000 7f253effd640 1 -- 172.21.15.80:0/4223879886 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f25440188d0 con 0x7f25480b0ed0 2024-04-25T13:52:33.857 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.863+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f254806f880 con 0x7f25480b0ed0 2024-04-25T13:52:33.859 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.863+0000 7f253effd640 1 -- 172.21.15.80:0/4223879886 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) ==== 74+0+16607 (secure 0 0 0) 0x7f25440f99f0 con 0x7f25480b0ed0 2024-04-25T13:52:33.859 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:33.859 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":190,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:52:26.300379+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:51:48.404349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T13:51:49.039512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:52:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f252c06f400 msgr2=0x7f252c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f252c06f400 0x7f252c071910 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f2548158960 tx=0x7f2538008040 comp rx=0 tx=0).stop 2024-04-25T13:52:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 msgr2=0x7f254814d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254814d300 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f254401c9a0 tx=0x7f254401bb10 comp rx=0 tx=0).stop 2024-04-25T13:52:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 shutdown_connections 2024-04-25T13:52:33.861 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f252c06f400 0x7f252c071910 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f25481608f0 0x7f25480af8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f25480ce730 0x7f2548158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 --2- 172.21.15.80:0/4223879886 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f25480b0ed0 0x7f254814d300 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:33.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 >> 172.21.15.80:0/4223879886 conn(0x7f254806d080 msgr2=0x7f25480cff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:33.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 shutdown_connections 2024-04-25T13:52:33.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:33.867+0000 7f25502e9640 1 -- 172.21.15.80:0/4223879886 wait complete. 2024-04-25T13:52:33.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T13:40:18.139319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T13:51:48.404349+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '186', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T13:51:49.039512+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '190', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T13:52:33.881 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T13:52:34.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c563b1640 1 Processor -- start 2024-04-25T13:52:34.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c563b1640 1 -- start start 2024-04-25T13:52:34.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c563b1640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1c500d3ed0 0x7f1c500d2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c563b1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c500d2ad0 0x7f1c500b76d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c563b1640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c500b92a0 con 0x7f1c500b7c10 2024-04-25T13:52:34.022 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c563b1640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1c500d4c30 con 0x7f1c500d2ad0 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c563b1640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1c500d4ef0 con 0x7f1c500d3ed0 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c4f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c500d2ad0 0x7f1c500b76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c4f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1c500d2ad0 0x7f1c500b76d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37670/0 (socket says 172.21.15.80:37670) 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.023+0000 7f1c4f7fe640 1 -- 172.21.15.80:0/1833323298 learned_addr learned my addr 172.21.15.80:0/1833323298 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4ffff640 1 --2- 172.21.15.80:0/1833323298 >> v2:172.21.15.87:3300/0 conn(0x7f1c500d3ed0 0x7f1c500d2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4f7fe640 1 -- 172.21.15.80:0/1833323298 >> v2:172.21.15.87:3300/0 conn(0x7f1c500d3ed0 msgr2=0x7f1c500d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4f7fe640 1 --2- 172.21.15.80:0/1833323298 >> v2:172.21.15.87:3300/0 conn(0x7f1c500d3ed0 0x7f1c500d2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4f7fe640 1 -- 172.21.15.80:0/1833323298 >> v1:172.21.15.87:6789/0 conn(0x7f1c500b7c10 legacy=0x7f1c500b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4f7fe640 1 -- 172.21.15.80:0/1833323298 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c500d51d0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4f7fe640 1 --2- 172.21.15.80:0/1833323298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c500b76d0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f1c5008e910 tx=0x7f1c3801c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396c644d90a9651a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4effd640 1 -- 172.21.15.80:0/1833323298 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c380b0070 con 0x7f1c500d2ad0 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4effd640 1 -- 172.21.15.80:0/1833323298 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c380ab600 con 0x7f1c500d2ad0 2024-04-25T13:52:34.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c4effd640 1 -- 172.21.15.80:0/1833323298 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c380a4e60 con 0x7f1c500d2ad0 2024-04-25T13:52:34.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c563b1640 1 -- 172.21.15.80:0/1833323298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 msgr2=0x7f1c500b76d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c563b1640 1 --2- 172.21.15.80:0/1833323298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c500b76d0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f1c5008e910 tx=0x7f1c3801c1b0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c563b1640 1 -- 172.21.15.80:0/1833323298 shutdown_connections 2024-04-25T13:52:34.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c563b1640 1 --2- 172.21.15.80:0/1833323298 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c500b76d0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c563b1640 1 --2- 172.21.15.80:0/1833323298 >> v2:172.21.15.87:3300/0 conn(0x7f1c500d3ed0 0x7f1c500d2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.024 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c563b1640 1 -- 172.21.15.80:0/1833323298 >> 172.21.15.80:0/1833323298 conn(0x7f1c500737b0 msgr2=0x7f1c5006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:34.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c563b1640 1 -- 172.21.15.80:0/1833323298 shutdown_connections 2024-04-25T13:52:34.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.027+0000 7f1c563b1640 1 -- 172.21.15.80:0/1833323298 wait complete. 2024-04-25T13:52:34.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c563b1640 1 Processor -- start 2024-04-25T13:52:34.025 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c563b1640 1 -- start start 2024-04-25T13:52:34.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c563b1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c500b7c10 0x7f1c50158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c563b1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c50156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c563b1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c500d3ed0 0x7f1c5014e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c563b1640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1c500ccae0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c563b1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1c500cc960 con 0x7f1c500b7c10 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c563b1640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1c50160620 con 0x7f1c500d3ed0 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c50156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c50156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37672/0 (socket says 172.21.15.80:37672) 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4f7fe640 1 -- 172.21.15.80:0/2016210310 learned_addr learned my addr 172.21.15.80:0/2016210310 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4ffff640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c500b7c10 0x7f1c50158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4f7fe640 1 -- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c500d3ed0 msgr2=0x7f1c5014e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c5494e640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c500d3ed0 0x7f1c5014e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4f7fe640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c500d3ed0 0x7f1c5014e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4f7fe640 1 -- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c500b7c10 msgr2=0x7f1c50158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4f7fe640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c500b7c10 0x7f1c50158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.031+0000 7f1c4f7fe640 1 -- 172.21.15.80:0/2016210310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c501595c0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4f7fe640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c50156820 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f1c380ae040 tx=0x7f1c38018f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4cff9640 1 -- 172.21.15.80:0/2016210310 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c380b0040 con 0x7f1c500d2ad0 2024-04-25T13:52:34.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4cff9640 1 -- 172.21.15.80:0/2016210310 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c380b9070 con 0x7f1c500d2ad0 2024-04-25T13:52:34.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c501598a0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4cff9640 1 -- 172.21.15.80:0/2016210310 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c380b4440 con 0x7f1c500d2ad0 2024-04-25T13:52:34.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c500cf0f0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c5006a3a0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4cff9640 1 -- 172.21.15.80:0/2016210310 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1c380b45e0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4cff9640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c3006f400 0x7f1c30071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4cff9640 1 -- 172.21.15.80:0/2016210310 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f1c38128d80 con 0x7f1c500d2ad0 2024-04-25T13:52:34.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4ffff640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c3006f400 0x7f1c30071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.035+0000 7f1c4ffff640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c3006f400 0x7f1c30071910 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000b930 tx=0x7f1c40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.039+0000 7f1c4cff9640 1 -- 172.21.15.80:0/2016210310 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1c500b92a0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.187+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1c50156de0 con 0x7f1c500d2ad0 2024-04-25T13:52:34.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c4cff9640 1 -- 172.21.15.80:0/2016210310 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v190) ==== 93+0+10 (secure 0 0 0) 0x7f1c380f5d90 con 0x7f1c500d2ad0 2024-04-25T13:52:34.184 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 1 2024-04-25T13:52:34.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c3006f400 msgr2=0x7f1c30071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c3006f400 0x7f1c30071910 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000b930 tx=0x7f1c40008040 comp rx=0 tx=0).stop 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 msgr2=0x7f1c50156820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c50156820 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f1c380ae040 tx=0x7f1c38018f50 comp rx=0 tx=0).stop 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 shutdown_connections 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1c3006f400 0x7f1c30071910 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1c500d3ed0 0x7f1c5014e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1c500d2ad0 0x7f1c50156820 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 --2- 172.21.15.80:0/2016210310 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1c500b7c10 0x7f1c50158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 >> 172.21.15.80:0/2016210310 conn(0x7f1c500737b0 msgr2=0x7f1c500ca390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 shutdown_connections 2024-04-25T13:52:34.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.191+0000 7f1c563b1640 1 -- 172.21.15.80:0/2016210310 wait complete. 2024-04-25T13:52:34.205 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T13:52:34.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3b003f2640 1 Processor -- start 2024-04-25T13:52:34.344 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3b003f2640 1 -- start start 2024-04-25T13:52:34.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3b003f2640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3af80b5330 0x7f3af80d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3b003f2640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f3af80d0ff0 0x7f3af806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3b003f2640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3af80b5740 con 0x7f3af806f2f0 2024-04-25T13:52:34.346 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3b003f2640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f3af806f700 con 0x7f3af80b5330 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3b003f2640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f3af81629b0 con 0x7f3af80d0ff0 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3afe18e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3af80b5330 0x7f3af80d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3afe18e640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f3af80b5330 0x7f3af80d0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37682/0 (socket says 172.21.15.80:37682) 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3afe18e640 1 -- 172.21.15.80:0/2384128034 learned_addr learned my addr 172.21.15.80:0/2384128034 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3afd98d640 1 --2- 172.21.15.80:0/2384128034 >> v2:172.21.15.87:3300/0 conn(0x7f3af80d0ff0 0x7f3af806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3afe18e640 1 -- 172.21.15.80:0/2384128034 >> v2:172.21.15.87:3300/0 conn(0x7f3af80d0ff0 msgr2=0x7f3af806edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3afe18e640 1 --2- 172.21.15.80:0/2384128034 >> v2:172.21.15.87:3300/0 conn(0x7f3af80d0ff0 0x7f3af806edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3afe18e640 1 -- 172.21.15.80:0/2384128034 >> v1:172.21.15.87:6789/0 conn(0x7f3af806f2f0 legacy=0x7f3af8161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:34.347 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.351+0000 7f3afe18e640 1 -- 172.21.15.80:0/2384128034 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af806dd10 con 0x7f3af80b5330 2024-04-25T13:52:34.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3afe18e640 1 --2- 172.21.15.80:0/2384128034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af80b5330 0x7f3af80d0ab0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f3af8074290 tx=0x7f3af401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67737f9381ad0d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3afd18c640 1 -- 172.21.15.80:0/2384128034 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40b0070 con 0x7f3af80b5330 2024-04-25T13:52:34.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3afd18c640 1 -- 172.21.15.80:0/2384128034 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af40ab600 con 0x7f3af80b5330 2024-04-25T13:52:34.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3afd18c640 1 -- 172.21.15.80:0/2384128034 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40ab9d0 con 0x7f3af80b5330 2024-04-25T13:52:34.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3b003f2640 1 -- 172.21.15.80:0/2384128034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af80b5330 msgr2=0x7f3af80d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3b003f2640 1 --2- 172.21.15.80:0/2384128034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af80b5330 0x7f3af80d0ab0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f3af8074290 tx=0x7f3af401c450 comp rx=0 tx=0).stop 2024-04-25T13:52:34.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3b003f2640 1 -- 172.21.15.80:0/2384128034 shutdown_connections 2024-04-25T13:52:34.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3b003f2640 1 --2- 172.21.15.80:0/2384128034 >> v2:172.21.15.87:3300/0 conn(0x7f3af80d0ff0 0x7f3af806edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3b003f2640 1 --2- 172.21.15.80:0/2384128034 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af80b5330 0x7f3af80d0ab0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3b003f2640 1 -- 172.21.15.80:0/2384128034 >> 172.21.15.80:0/2384128034 conn(0x7f3af80b4160 msgr2=0x7f3af80d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:34.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3b003f2640 1 -- 172.21.15.80:0/2384128034 shutdown_connections 2024-04-25T13:52:34.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.355+0000 7f3b003f2640 1 -- 172.21.15.80:0/2384128034 wait complete. 2024-04-25T13:52:34.351 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 Processor -- start 2024-04-25T13:52:34.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 -- start start 2024-04-25T13:52:34.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af806f2f0 0x7f3af815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.354 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3af80b5330 0x7f3af8140f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3af80d0ff0 0x7f3af8143910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f3af80b0cd0 con 0x7f3af806f2f0 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f3af806e890 con 0x7f3af80d0ff0 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f3af80b0e50 con 0x7f3af80b5330 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af806f2f0 0x7f3af815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af806f2f0 0x7f3af815bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37688/0 (socket says 172.21.15.80:37688) 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 -- 172.21.15.80:0/1564931278 learned_addr learned my addr 172.21.15.80:0/1564931278 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe98f640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3af80d0ff0 0x7f3af8143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afd98d640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3af80b5330 0x7f3af8140f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 -- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3af80b5330 msgr2=0x7f3af8140f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3af80b5330 0x7f3af8140f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 -- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3af80d0ff0 msgr2=0x7f3af8143910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3af80d0ff0 0x7f3af8143910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.355 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 -- 172.21.15.80:0/1564931278 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3af8141470 con 0x7f3af806f2f0 2024-04-25T13:52:34.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afd98d640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3af80b5330 0x7f3af8140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:34.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3afe18e640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af806f2f0 0x7f3af815bb10 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3af4000c10 tx=0x7f3af40191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3aeeffd640 1 -- 172.21.15.80:0/1564931278 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40bb070 con 0x7f3af806f2f0 2024-04-25T13:52:34.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3aeeffd640 1 -- 172.21.15.80:0/1564931278 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3af40b6440 con 0x7f3af806f2f0 2024-04-25T13:52:34.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3af8073050 con 0x7f3af806f2f0 2024-04-25T13:52:34.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3aeeffd640 1 -- 172.21.15.80:0/1564931278 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40b0040 con 0x7f3af806f2f0 2024-04-25T13:52:34.356 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3af8073610 con 0x7f3af806f2f0 2024-04-25T13:52:34.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3af806a3a0 con 0x7f3af806f2f0 2024-04-25T13:52:34.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.359+0000 7f3aeeffd640 1 -- 172.21.15.80:0/1564931278 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f3af40be070 con 0x7f3af806f2f0 2024-04-25T13:52:34.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.363+0000 7f3aeeffd640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3adc06f430 0x7f3adc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.363+0000 7f3aeeffd640 1 -- 172.21.15.80:0/1564931278 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f3af4128c40 con 0x7f3af806f2f0 2024-04-25T13:52:34.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.363+0000 7f3afd98d640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3adc06f430 0x7f3adc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.357 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.363+0000 7f3afd98d640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3adc06f430 0x7f3adc071940 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3af80b63b0 tx=0x7f3ae8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.361 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.367+0000 7f3aeeffd640 1 -- 172.21.15.80:0/1564931278 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f3af40160d0 con 0x7f3af806f2f0 2024-04-25T13:52:34.507 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.511+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3af80ce8b0 con 0x7f3af806f2f0 2024-04-25T13:52:34.508 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3aeeffd640 1 -- 172.21.15.80:0/1564931278 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v190) ==== 104+0+11 (secure 0 0 0) 0x7f3af40f5c50 con 0x7f3af806f2f0 2024-04-25T13:52:34.508 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 32 2024-04-25T13:52:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3adc06f430 msgr2=0x7f3adc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3adc06f430 0x7f3adc071940 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f3af80b63b0 tx=0x7f3ae8008040 comp rx=0 tx=0).stop 2024-04-25T13:52:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af806f2f0 msgr2=0x7f3af815bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af806f2f0 0x7f3af815bb10 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f3af4000c10 tx=0x7f3af40191f0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 shutdown_connections 2024-04-25T13:52:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f3adc06f430 0x7f3adc071940 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.510 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3af80d0ff0 0x7f3af8143910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f3af80b5330 0x7f3af8140f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 --2- 172.21.15.80:0/1564931278 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f3af806f2f0 0x7f3af815bb10 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 >> 172.21.15.80:0/1564931278 conn(0x7f3af80b4160 msgr2=0x7f3af8146cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:34.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 shutdown_connections 2024-04-25T13:52:34.511 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.515+0000 7f3b003f2640 1 -- 172.21.15.80:0/1564931278 wait complete. 2024-04-25T13:52:34.529 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T13:52:34.676 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c23a3a640 1 Processor -- start 2024-04-25T13:52:34.677 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c23a3a640 1 -- start start 2024-04-25T13:52:34.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c23a3a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f0c1c073080 0x7f0c1c073ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c23a3a640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c1c15c710 0x7f0c1c06e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c23a3a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c1c06edc0 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.679 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c23a3a640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1c0b53d0 con 0x7f0c1c15c710 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c23a3a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0c1c0cf640 con 0x7f0c1c073080 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c20fd5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c1c15c710 0x7f0c1c06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c20fd5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0c1c15c710 0x7f0c1c06e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37702/0 (socket says 172.21.15.80:37702) 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c20fd5640 1 -- 172.21.15.80:0/2240528120 learned_addr learned my addr 172.21.15.80:0/2240528120 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c217d6640 1 --2- 172.21.15.80:0/2240528120 >> v2:172.21.15.87:3300/0 conn(0x7f0c1c073080 0x7f0c1c073ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c20fd5640 1 -- 172.21.15.80:0/2240528120 >> v2:172.21.15.87:3300/0 conn(0x7f0c1c073080 msgr2=0x7f0c1c073ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c20fd5640 1 --2- 172.21.15.80:0/2240528120 >> v2:172.21.15.87:3300/0 conn(0x7f0c1c073080 0x7f0c1c073ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c20fd5640 1 -- 172.21.15.80:0/2240528120 >> v1:172.21.15.87:6789/0 conn(0x7f0c1c06e9b0 legacy=0x7f0c1c071e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:34.680 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.683+0000 7f0c20fd5640 1 -- 172.21.15.80:0/2240528120 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c0737f0 con 0x7f0c1c15c710 2024-04-25T13:52:34.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c20fd5640 1 --2- 172.21.15.80:0/2240528120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c15c710 0x7f0c1c06e470 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c071630 tx=0x7f0c0c01c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ca3752e4c37be81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c13fff640 1 -- 172.21.15.80:0/2240528120 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0b0070 con 0x7f0c1c15c710 2024-04-25T13:52:34.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c13fff640 1 -- 172.21.15.80:0/2240528120 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c0c0ab600 con 0x7f0c1c15c710 2024-04-25T13:52:34.681 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c13fff640 1 -- 172.21.15.80:0/2240528120 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0ab9d0 con 0x7f0c1c15c710 2024-04-25T13:52:34.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c23a3a640 1 -- 172.21.15.80:0/2240528120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c15c710 msgr2=0x7f0c1c06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/2240528120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c15c710 0x7f0c1c06e470 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c071630 tx=0x7f0c0c01c270 comp rx=0 tx=0).stop 2024-04-25T13:52:34.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c23a3a640 1 -- 172.21.15.80:0/2240528120 shutdown_connections 2024-04-25T13:52:34.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/2240528120 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c15c710 0x7f0c1c06e470 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/2240528120 >> v2:172.21.15.87:3300/0 conn(0x7f0c1c073080 0x7f0c1c073ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c23a3a640 1 -- 172.21.15.80:0/2240528120 >> 172.21.15.80:0/2240528120 conn(0x7f0c1c0b4160 msgr2=0x7f0c1c0d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:34.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c23a3a640 1 -- 172.21.15.80:0/2240528120 shutdown_connections 2024-04-25T13:52:34.682 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.687+0000 7f0c23a3a640 1 -- 172.21.15.80:0/2240528120 wait complete. 2024-04-25T13:52:34.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 Processor -- start 2024-04-25T13:52:34.683 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 -- start start 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c06e9b0 0x7f0c1c155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1c073080 0x7f0c1c13d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c1c15c710 0x7f0c1c151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c15cb20 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0c1c072570 con 0x7f0c1c073080 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0c1c14fe00 con 0x7f0c1c15c710 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c06e9b0 0x7f0c1c155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c06e9b0 0x7f0c1c155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37718/0 (socket says 172.21.15.80:37718) 2024-04-25T13:52:34.686 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 -- 172.21.15.80:0/3721194247 learned_addr learned my addr 172.21.15.80:0/3721194247 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c20fd5640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1c073080 0x7f0c1c13d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 -- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c1c15c710 msgr2=0x7f0c1c151570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c1c15c710 0x7f0c1c151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 -- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1c073080 msgr2=0x7f0c1c13d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1c073080 0x7f0c1c13d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 -- 172.21.15.80:0/3721194247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c1c156460 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c20fd5640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1c073080 0x7f0c1c13d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c217d6640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c06e9b0 0x7f0c1c155f20 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800d5d0 tx=0x7f0c1800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c11ffb640 1 -- 172.21.15.80:0/3721194247 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18004240 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c11ffb640 1 -- 172.21.15.80:0/3721194247 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c180043e0 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c11ffb640 1 -- 172.21.15.80:0/3721194247 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c18003840 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c1c156970 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c1c156f30 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.691+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c1c06a3a0 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.695+0000 7f0c11ffb640 1 -- 172.21.15.80:0/3721194247 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0c180039e0 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.695+0000 7f0c11ffb640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c0006f400 0x7f0c00071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:34.688 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.695+0000 7f0c11ffb640 1 -- 172.21.15.80:0/3721194247 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f0c1808f570 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.695+0000 7f0c20fd5640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c0006f400 0x7f0c00071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:34.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.695+0000 7f0c20fd5640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c0006f400 0x7f0c00071910 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0ad000 tx=0x7f0c0c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:34.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.699+0000 7f0c11ffb640 1 -- 172.21.15.80:0/3721194247 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0c1805c460 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0c1c06f840 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.852 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c11ffb640 1 -- 172.21.15.80:0/3721194247 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v190) ==== 100+0+11 (secure 0 0 0) 0x7f0c1805bd60 con 0x7f0c1c06e9b0 2024-04-25T13:52:34.852 INFO:teuthology.orchestra.run.smithi080.stdout:pg_num: 64 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c0006f400 msgr2=0x7f0c00071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c0006f400 0x7f0c00071910 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c0ad000 tx=0x7f0c0c008040 comp rx=0 tx=0).stop 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c06e9b0 msgr2=0x7f0c1c155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c06e9b0 0x7f0c1c155f20 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800d5d0 tx=0x7f0c1800da90 comp rx=0 tx=0).stop 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 shutdown_connections 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0c0006f400 0x7f0c00071910 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0c1c15c710 0x7f0c1c151570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c1c073080 0x7f0c1c13d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 --2- 172.21.15.80:0/3721194247 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0c1c06e9b0 0x7f0c1c155f20 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 >> 172.21.15.80:0/3721194247 conn(0x7f0c1c0b4160 msgr2=0x7f0c1c0cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:34.855 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 shutdown_connections 2024-04-25T13:52:34.856 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:34.859+0000 7f0c23a3a640 1 -- 172.21.15.80:0/3721194247 wait complete. 2024-04-25T13:52:34.873 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:35.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.027+0000 7f92fec2f640 1 Processor -- start 2024-04-25T13:52:35.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.027+0000 7f92fec2f640 1 -- start start 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fec2f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92f8161f90 0x7f92f80cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fec2f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f92f80d0510 0x7f92f815e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fec2f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92f80d0d70 con 0x7f92f815ea70 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fec2f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f92f80d1050 con 0x7f92f8161f90 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fec2f640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f92f806dca0 con 0x7f92f80d0510 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fc9cb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92f8161f90 0x7f92f80cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fc9cb640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f92f8161f90 0x7f92f80cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37734/0 (socket says 172.21.15.80:37734) 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fc9cb640 1 -- 172.21.15.80:0/872475583 learned_addr learned my addr 172.21.15.80:0/872475583 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92f7fff640 1 --2- 172.21.15.80:0/872475583 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d0510 0x7f92f815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fc9cb640 1 -- 172.21.15.80:0/872475583 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d0510 msgr2=0x7f92f815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fc9cb640 1 --2- 172.21.15.80:0/872475583 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d0510 0x7f92f815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fc9cb640 1 -- 172.21.15.80:0/872475583 >> v1:172.21.15.87:6789/0 conn(0x7f92f815ea70 legacy=0x7f92f815ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:35.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92fc9cb640 1 -- 172.21.15.80:0/872475583 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f806df80 con 0x7f92f8161f90 2024-04-25T13:52:35.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.031+0000 7f92f7fff640 1 --2- 172.21.15.80:0/872475583 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d0510 0x7f92f815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fc9cb640 1 --2- 172.21.15.80:0/872475583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80cffd0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f92f8074290 tx=0x7f92e801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f67e0e49c5a4ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92f77fe640 1 -- 172.21.15.80:0/872475583 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80b0070 con 0x7f92f8161f90 2024-04-25T13:52:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92f77fe640 1 -- 172.21.15.80:0/872475583 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92e80ab600 con 0x7f92f8161f90 2024-04-25T13:52:35.028 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92f77fe640 1 -- 172.21.15.80:0/872475583 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e80ab9d0 con 0x7f92f8161f90 2024-04-25T13:52:35.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 -- 172.21.15.80:0/872475583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 msgr2=0x7f92f80cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 --2- 172.21.15.80:0/872475583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80cffd0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f92f8074290 tx=0x7f92e801c3f0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 -- 172.21.15.80:0/872475583 shutdown_connections 2024-04-25T13:52:35.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 --2- 172.21.15.80:0/872475583 >> v2:172.21.15.87:3300/0 conn(0x7f92f80d0510 0x7f92f815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 --2- 172.21.15.80:0/872475583 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80cffd0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 -- 172.21.15.80:0/872475583 >> 172.21.15.80:0/872475583 conn(0x7f92f806d080 msgr2=0x7f92f80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:35.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 -- 172.21.15.80:0/872475583 shutdown_connections 2024-04-25T13:52:35.029 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 -- 172.21.15.80:0/872475583 wait complete. 2024-04-25T13:52:35.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 Processor -- start 2024-04-25T13:52:35.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.035+0000 7f92fec2f640 1 -- start start 2024-04-25T13:52:35.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80d0510 0x7f92f81fc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f815ea70 0x7f92f81630f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f92f806f4b0 con 0x7f92f8161f90 2024-04-25T13:52:35.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f92f806f630 con 0x7f92f815ea70 2024-04-25T13:52:35.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f92f80d0920 con 0x7f92f80d0510 2024-04-25T13:52:35.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37742/0 (socket says 172.21.15.80:37742) 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 -- 172.21.15.80:0/476715274 learned_addr learned my addr 172.21.15.80:0/476715274 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92f7fff640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f815ea70 0x7f92f81630f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fc9cb640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80d0510 0x7f92f81fc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 -- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80d0510 msgr2=0x7f92f81fc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80d0510 0x7f92f81fc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 -- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f815ea70 msgr2=0x7f92f81630f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f815ea70 0x7f92f81630f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 -- 172.21.15.80:0/476715274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f8072d20 con 0x7f92f8161f90 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fc9cb640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80d0510 0x7f92f81fc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fd1cc640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80af8b0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f92f000ba10 tx=0x7f92f000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92f57fa640 1 -- 172.21.15.80:0/476715274 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0019070 con 0x7f92f8161f90 2024-04-25T13:52:35.034 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92f57fa640 1 -- 172.21.15.80:0/476715274 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92f000cb40 con 0x7f92f8161f90 2024-04-25T13:52:35.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f8073000 con 0x7f92f8161f90 2024-04-25T13:52:35.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92f57fa640 1 -- 172.21.15.80:0/476715274 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f0014440 con 0x7f92f8161f90 2024-04-25T13:52:35.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f80735c0 con 0x7f92f8161f90 2024-04-25T13:52:35.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f806a3a0 con 0x7f92f8161f90 2024-04-25T13:52:35.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92f57fa640 1 -- 172.21.15.80:0/476715274 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f92f001c070 con 0x7f92f8161f90 2024-04-25T13:52:35.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92f57fa640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92d806f430 0x7f92d8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92f57fa640 1 -- 172.21.15.80:0/476715274 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f92f008eea0 con 0x7f92f8161f90 2024-04-25T13:52:35.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.039+0000 7f92fc9cb640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92d806f430 0x7f92d8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.036 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.043+0000 7f92fc9cb640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92d806f430 0x7f92d8071940 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f92e80ad000 tx=0x7f92e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.041 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.047+0000 7f92f57fa640 1 -- 172.21.15.80:0/476715274 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f92f005bd90 con 0x7f92f8161f90 2024-04-25T13:52:35.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92f8072ab0 con 0x7f92f8161f90 2024-04-25T13:52:35.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92f57fa640 1 -- 172.21.15.80:0/476715274 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7f92f005b690 con 0x7f92f8161f90 2024-04-25T13:52:35.237 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:35.237 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:52:35.238 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92d806f430 msgr2=0x7f92d8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92d806f430 0x7f92d8071940 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f92e80ad000 tx=0x7f92e8008040 comp rx=0 tx=0).stop 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 msgr2=0x7f92f80af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80af8b0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f92f000ba10 tx=0x7f92f000bed0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 shutdown_connections 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f92d806f430 0x7f92d8071940 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f92f8161f90 0x7f92f80af8b0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92f815ea70 0x7f92f81630f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 --2- 172.21.15.80:0/476715274 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f92f80d0510 0x7f92f81fc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.243+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 >> 172.21.15.80:0/476715274 conn(0x7f92f806d080 msgr2=0x7f92f80b4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.247+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 shutdown_connections 2024-04-25T13:52:35.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.247+0000 7f92fec2f640 1 -- 172.21.15.80:0/476715274 wait complete. 2024-04-25T13:52:35.260 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:52:35.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa420b85640 1 Processor -- start 2024-04-25T13:52:35.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa420b85640 1 -- start start 2024-04-25T13:52:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa420b85640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c1685d0 0x7fa41c1689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa420b85640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fa41c0b60d0 0x7fa41c0b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa420b85640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa41c168f20 con 0x7fa41c0768c0 2024-04-25T13:52:35.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa420b85640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fa41c0b50c0 con 0x7fa41c0b60d0 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa420b85640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c0b5c40 con 0x7fa41c1685d0 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41a59c640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fa41c0768c0 0x7fa41c164800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41594/0 (socket says 172.21.15.80:41594) 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41a59c640 1 -- 172.21.15.80:0/3638784906 learned_addr learned my addr 172.21.15.80:0/3638784906 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41ad9d640 1 --2- 172.21.15.80:0/3638784906 >> v2:172.21.15.80:3300/0 conn(0x7fa41c0b60d0 0x7fa41c0b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa419d9b640 1 --2- 172.21.15.80:0/3638784906 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c1685d0 0x7fa41c1689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41959a640 1 -- 172.21.15.80:0/3638784906 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 578855607 0 0) 0x7fa40c001120 con 0x7fa41c0768c0 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41959a640 1 -- 172.21.15.80:0/3638784906 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa408003530 con 0x7fa41c0768c0 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41ad9d640 1 -- 172.21.15.80:0/3638784906 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c1685d0 msgr2=0x7fa41c1689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41ad9d640 1 --2- 172.21.15.80:0/3638784906 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c1685d0 0x7fa41c1689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41ad9d640 1 -- 172.21.15.80:0/3638784906 >> v1:172.21.15.80:6789/0 conn(0x7fa41c0768c0 legacy=0x7fa41c164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.411+0000 7fa41ad9d640 1 -- 172.21.15.80:0/3638784906 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c0b5310 con 0x7fa41c0b60d0 2024-04-25T13:52:35.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa41ad9d640 1 --2- 172.21.15.80:0/3638784906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c0b64e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fa414009d90 tx=0x7fa41401c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a79e527079c061 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa41959a640 1 -- 172.21.15.80:0/3638784906 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140b2070 con 0x7fa41c0b60d0 2024-04-25T13:52:35.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa41959a640 1 -- 172.21.15.80:0/3638784906 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4140a5730 con 0x7fa41c0b60d0 2024-04-25T13:52:35.409 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa41959a640 1 -- 172.21.15.80:0/3638784906 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4140a5b00 con 0x7fa41c0b60d0 2024-04-25T13:52:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa420b85640 1 -- 172.21.15.80:0/3638784906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 msgr2=0x7fa41c0b64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa420b85640 1 --2- 172.21.15.80:0/3638784906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c0b64e0 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fa414009d90 tx=0x7fa41401c970 comp rx=0 tx=0).stop 2024-04-25T13:52:35.410 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa420b85640 1 -- 172.21.15.80:0/3638784906 shutdown_connections 2024-04-25T13:52:35.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa420b85640 1 --2- 172.21.15.80:0/3638784906 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c0b64e0 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa420b85640 1 --2- 172.21.15.80:0/3638784906 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c1685d0 0x7fa41c1689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa420b85640 1 -- 172.21.15.80:0/3638784906 >> 172.21.15.80:0/3638784906 conn(0x7fa41c073af0 msgr2=0x7fa41c073f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:35.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa420b85640 1 -- 172.21.15.80:0/3638784906 shutdown_connections 2024-04-25T13:52:35.411 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.415+0000 7fa420b85640 1 -- 172.21.15.80:0/3638784906 wait complete. 2024-04-25T13:52:35.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 Processor -- start 2024-04-25T13:52:35.412 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 -- start start 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c0768c0 0x7fa41c1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c15b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa41c1685d0 0x7fa41c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0d0e20 con 0x7fa41c0b60d0 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fa41c0d0ca0 con 0x7fa41c1685d0 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fa41c171910 con 0x7fa41c0768c0 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c15b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c15b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37760/0 (socket says 172.21.15.80:37760) 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 -- 172.21.15.80:0/1590212658 learned_addr learned my addr 172.21.15.80:0/1590212658 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:35.415 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa41ad9d640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa41c1685d0 0x7fa41c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa41a59c640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c0768c0 0x7fa41c1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 -- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c0768c0 msgr2=0x7fa41c1573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c0768c0 0x7fa41c1573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 -- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa41c1685d0 msgr2=0x7fa41c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa41c1685d0 0x7fa41c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 -- 172.21.15.80:0/1590212658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41c157bf0 con 0x7fa41c0b60d0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa419d9b640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c15b0d0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa40400d5f0 tx=0x7fa40400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa412ffd640 1 -- 172.21.15.80:0/1590212658 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040125f0 con 0x7fa41c0b60d0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa412ffd640 1 -- 172.21.15.80:0/1590212658 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa404019070 con 0x7fa41c0b60d0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa41c0af190 con 0x7fa41c0b60d0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa412ffd640 1 -- 172.21.15.80:0/1590212658 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040129b0 con 0x7fa41c0b60d0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa41c0af750 con 0x7fa41c0b60d0 2024-04-25T13:52:35.416 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.419+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa41c157960 con 0x7fa41c0b60d0 2024-04-25T13:52:35.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.423+0000 7fa412ffd640 1 -- 172.21.15.80:0/1590212658 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fa404012b50 con 0x7fa41c0b60d0 2024-04-25T13:52:35.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.423+0000 7fa412ffd640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa3fc06f400 0x7fa3fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.423+0000 7fa412ffd640 1 -- 172.21.15.80:0/1590212658 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7fa40408f500 con 0x7fa41c0b60d0 2024-04-25T13:52:35.417 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.423+0000 7fa41a59c640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa3fc06f400 0x7fa3fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.418 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.423+0000 7fa41a59c640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa3fc06f400 0x7fa3fc071910 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa40c005890 tx=0x7fa40c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.421 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.427+0000 7fa412ffd640 1 -- 172.21.15.80:0/1590212658 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fa40405be30 con 0x7fa41c0b60d0 2024-04-25T13:52:35.581 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.587+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa41c06a3a0 con 0x7fa41c0b60d0 2024-04-25T13:52:35.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.587+0000 7fa412ffd640 1 -- 172.21.15.80:0/1590212658 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) ==== 74+0+16607 (secure 0 0 0) 0x7fa404060130 con 0x7fa41c0b60d0 2024-04-25T13:52:35.583 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:35.583 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":190,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:52:26.300379+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:51:48.404349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T13:51:49.039512+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:52:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa3fc06f400 msgr2=0x7fa3fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa3fc06f400 0x7fa3fc071910 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa40c005890 tx=0x7fa40c009040 comp rx=0 tx=0).stop 2024-04-25T13:52:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 msgr2=0x7fa41c15b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c15b0d0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fa40400d5f0 tx=0x7fa40400dab0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 shutdown_connections 2024-04-25T13:52:35.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fa3fc06f400 0x7fa3fc071910 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa41c1685d0 0x7fa41c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fa41c0b60d0 0x7fa41c15b0d0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 --2- 172.21.15.80:0/1590212658 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fa41c0768c0 0x7fa41c1573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 >> 172.21.15.80:0/1590212658 conn(0x7fa41c073af0 msgr2=0x7fa41c164800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:35.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 shutdown_connections 2024-04-25T13:52:35.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.591+0000 7fa420b85640 1 -- 172.21.15.80:0/1590212658 wait complete. 2024-04-25T13:52:35.605 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:52:35.605 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f8148404640 1 Processor -- start 2024-04-25T13:52:35.745 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f8148404640 1 -- start start 2024-04-25T13:52:35.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f8148404640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81400b5860 0x7f81400b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f8148404640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f814006cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f8148404640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f81400b61b0 con 0x7f81400cf000 2024-04-25T13:52:35.747 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f8148404640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f81400cf410 con 0x7f81400b5860 2024-04-25T13:52:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f8148404640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f81400700e0 con 0x7f814006c810 2024-04-25T13:52:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f81469a1640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f814006cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f814599f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81400b5860 0x7f81400b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f814599f640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f81400b5860 0x7f81400b5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:56000/0 (socket says 172.21.15.80:56000) 2024-04-25T13:52:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f81461a0640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f81400cf000 0x7f814006f800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:37460/0 (socket says 172.21.15.80:37460) 2024-04-25T13:52:35.748 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.751+0000 7f814599f640 1 -- 172.21.15.80:0/3781474065 learned_addr learned my addr 172.21.15.80:0/3781474065 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f81469a1640 1 -- 172.21.15.80:0/3781474065 >> v2:172.21.15.87:3300/0 conn(0x7f81400b5860 msgr2=0x7f81400b5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f81469a1640 1 --2- 172.21.15.80:0/3781474065 >> v2:172.21.15.87:3300/0 conn(0x7f81400b5860 0x7f81400b5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f81469a1640 1 -- 172.21.15.80:0/3781474065 >> v1:172.21.15.87:6789/0 conn(0x7f81400cf000 legacy=0x7f814006f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f81469a1640 1 -- 172.21.15.80:0/3781474065 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81400703c0 con 0x7f814006c810 2024-04-25T13:52:35.749 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f814599f640 1 --2- 172.21.15.80:0/3781474065 >> v2:172.21.15.87:3300/0 conn(0x7f81400b5860 0x7f81400b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:35.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f81469a1640 1 --2- 172.21.15.80:0/3781474065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f814006cc20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8138009d90 tx=0x7f813801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ecef91b114dd82d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f814519e640 1 -- 172.21.15.80:0/3781474065 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380b2070 con 0x7f814006c810 2024-04-25T13:52:35.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f814519e640 1 -- 172.21.15.80:0/3781474065 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81380a5890 con 0x7f814006c810 2024-04-25T13:52:35.750 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f814519e640 1 -- 172.21.15.80:0/3781474065 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380a5c60 con 0x7f814006c810 2024-04-25T13:52:35.751 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f8148404640 1 -- 172.21.15.80:0/3781474065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 msgr2=0x7f814006cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f8148404640 1 --2- 172.21.15.80:0/3781474065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f814006cc20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f8138009d90 tx=0x7f813801c8e0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f8148404640 1 -- 172.21.15.80:0/3781474065 shutdown_connections 2024-04-25T13:52:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f8148404640 1 --2- 172.21.15.80:0/3781474065 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f814006cc20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f8148404640 1 --2- 172.21.15.80:0/3781474065 >> v2:172.21.15.87:3300/0 conn(0x7f81400b5860 0x7f81400b5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f8148404640 1 -- 172.21.15.80:0/3781474065 >> 172.21.15.80:0/3781474065 conn(0x7f81400d28f0 msgr2=0x7f81400d2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f8148404640 1 -- 172.21.15.80:0/3781474065 shutdown_connections 2024-04-25T13:52:35.752 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.755+0000 7f8148404640 1 -- 172.21.15.80:0/3781474065 wait complete. 2024-04-25T13:52:35.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.759+0000 7f8148404640 1 Processor -- start 2024-04-25T13:52:35.753 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.759+0000 7f8148404640 1 -- start start 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.759+0000 7f8148404640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f8140150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.759+0000 7f8148404640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81400b5860 0x7f814015e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.759+0000 7f8148404640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81400cf000 0x7f8140157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.759+0000 7f8148404640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f8140169d60 con 0x7f81400cf000 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.759+0000 7f8148404640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f8140169be0 con 0x7f81400b5860 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.759+0000 7f8148404640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f8140169ee0 con 0x7f814006c810 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81400cf000 0x7f8140157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81400cf000 0x7f8140157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37772/0 (socket says 172.21.15.80:37772) 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 -- 172.21.15.80:0/4272786062 learned_addr learned my addr 172.21.15.80:0/4272786062 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f814599f640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81400b5860 0x7f814015e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81461a0640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f8140150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.757 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 -- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 msgr2=0x7f8140150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f8140150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 -- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81400b5860 msgr2=0x7f814015e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81400b5860 0x7f814015e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 -- 172.21.15.80:0/4272786062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814015f370 con 0x7f81400cf000 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f814599f640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81400b5860 0x7f814015e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81469a1640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81400cf000 0x7f8140157380 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f81380b0040 tx=0x7f8138019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8136ffd640 1 -- 172.21.15.80:0/4272786062 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380b2050 con 0x7f81400cf000 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81401fb130 con 0x7f81400cf000 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8136ffd640 1 -- 172.21.15.80:0/4272786062 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81380be070 con 0x7f81400cf000 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8136ffd640 1 -- 172.21.15.80:0/4272786062 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81380ad850 con 0x7f81400cf000 2024-04-25T13:52:35.758 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81401fb6f0 con 0x7f81400cf000 2024-04-25T13:52:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814006a3a0 con 0x7f81400cf000 2024-04-25T13:52:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8136ffd640 1 -- 172.21.15.80:0/4272786062 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f81380ad9f0 con 0x7f81400cf000 2024-04-25T13:52:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8136ffd640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f812406f430 0x7f8124071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f8136ffd640 1 -- 172.21.15.80:0/4272786062 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f8138129e20 con 0x7f81400cf000 2024-04-25T13:52:35.759 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.763+0000 7f81461a0640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f812406f430 0x7f8124071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:35.760 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.767+0000 7f81461a0640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f812406f430 0x7f8124071940 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8130000f70 tx=0x7f8130007c90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:35.764 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.771+0000 7f8136ffd640 1 -- 172.21.15.80:0/4272786062 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f81380c7020 con 0x7f81400cf000 2024-04-25T13:52:35.952 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8140150560 con 0x7f81400cf000 2024-04-25T13:52:35.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8136ffd640 1 -- 172.21.15.80:0/4272786062 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7f8138016560 con 0x7f81400cf000 2024-04-25T13:52:35.953 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:35.953 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:52:35.954 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:52:35.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f812406f430 msgr2=0x7f8124071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f812406f430 0x7f8124071940 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8130000f70 tx=0x7f8130007c90 comp rx=0 tx=0).stop 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81400cf000 msgr2=0x7f8140157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81400cf000 0x7f8140157380 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f81380b0040 tx=0x7f8138019680 comp rx=0 tx=0).stop 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 shutdown_connections 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f812406f430 0x7f8124071940 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f81400cf000 0x7f8140157380 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f81400b5860 0x7f814015e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 --2- 172.21.15.80:0/4272786062 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f814006c810 0x7f8140150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 >> 172.21.15.80:0/4272786062 conn(0x7f81400d28f0 msgr2=0x7f81400b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 shutdown_connections 2024-04-25T13:52:35.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:35.959+0000 7f8148404640 1 -- 172.21.15.80:0/4272786062 wait complete. 2024-04-25T13:52:35.974 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:52:36.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f0356a14640 1 Processor -- start 2024-04-25T13:52:36.116 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f0356a14640 1 -- start start 2024-04-25T13:52:36.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f0356a14640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f03500d2e80 0x7f03501607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.118 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f0356a14640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f03500d59b0 0x7f03500d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f0356a14640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03500d6300 con 0x7f03500d4330 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f0356a14640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0350161060 con 0x7f03500d2e80 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f0356a14640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0350161340 con 0x7f03500d59b0 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f034ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f03500d2e80 0x7f03501607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f034ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f03500d2e80 0x7f03501607a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37788/0 (socket says 172.21.15.80:37788) 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f034ffff640 1 -- 172.21.15.80:0/285389697 learned_addr learned my addr 172.21.15.80:0/285389697 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f034f7fe640 1 --2- 172.21.15.80:0/285389697 >> v2:172.21.15.87:3300/0 conn(0x7f03500d59b0 0x7f03500d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f034ffff640 1 -- 172.21.15.80:0/285389697 >> v2:172.21.15.87:3300/0 conn(0x7f03500d59b0 msgr2=0x7f03500d5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f034ffff640 1 --2- 172.21.15.80:0/285389697 >> v2:172.21.15.87:3300/0 conn(0x7f03500d59b0 0x7f03500d5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f034ffff640 1 -- 172.21.15.80:0/285389697 >> v1:172.21.15.87:6789/0 conn(0x7f03500d4330 legacy=0x7f03500d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:36.119 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.123+0000 7f034ffff640 1 -- 172.21.15.80:0/285389697 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03500d2680 con 0x7f03500d2e80 2024-04-25T13:52:36.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f034ffff640 1 --2- 172.21.15.80:0/285389697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03501607a0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f035008ec00 tx=0x7f034001c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2bc7bd97e1779d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.120 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f034effd640 1 -- 172.21.15.80:0/285389697 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400b0070 con 0x7f03500d2e80 2024-04-25T13:52:36.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f034effd640 1 -- 172.21.15.80:0/285389697 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03400ab600 con 0x7f03500d2e80 2024-04-25T13:52:36.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f034effd640 1 -- 172.21.15.80:0/285389697 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400ab9d0 con 0x7f03500d2e80 2024-04-25T13:52:36.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 -- 172.21.15.80:0/285389697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 msgr2=0x7f03501607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.121 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 --2- 172.21.15.80:0/285389697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03501607a0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f035008ec00 tx=0x7f034001c270 comp rx=0 tx=0).stop 2024-04-25T13:52:36.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 -- 172.21.15.80:0/285389697 shutdown_connections 2024-04-25T13:52:36.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 --2- 172.21.15.80:0/285389697 >> v2:172.21.15.87:3300/0 conn(0x7f03500d59b0 0x7f03500d5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 --2- 172.21.15.80:0/285389697 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03501607a0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 -- 172.21.15.80:0/285389697 >> 172.21.15.80:0/285389697 conn(0x7f03500737a0 msgr2=0x7f035006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:36.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 -- 172.21.15.80:0/285389697 shutdown_connections 2024-04-25T13:52:36.122 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 -- 172.21.15.80:0/285389697 wait complete. 2024-04-25T13:52:36.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 Processor -- start 2024-04-25T13:52:36.123 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.127+0000 7f0356a14640 1 -- start start 2024-04-25T13:52:36.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03500cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.125 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03500d4330 0x7f0350155dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03500d59b0 0x7f035015f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f03500d3410 con 0x7f03500d2e80 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f03500d3290 con 0x7f03500d59b0 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f035015df60 con 0x7f03500d4330 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03500cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03500cccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37798/0 (socket says 172.21.15.80:37798) 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 -- 172.21.15.80:0/1473199439 learned_addr learned my addr 172.21.15.80:0/1473199439 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 -- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03500d4330 msgr2=0x7f0350155dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03500d4330 0x7f0350155dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.126 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 -- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03500d59b0 msgr2=0x7f035015f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03500d59b0 0x7f035015f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 -- 172.21.15.80:0/1473199439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03500cd740 con 0x7f03500d2e80 2024-04-25T13:52:36.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034ffff640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03500cccc0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f03400ae040 tx=0x7f034001aab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034cff9640 1 -- 172.21.15.80:0/1473199439 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400b0040 con 0x7f03500d2e80 2024-04-25T13:52:36.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034cff9640 1 -- 172.21.15.80:0/1473199439 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03400bc070 con 0x7f03500d2e80 2024-04-25T13:52:36.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03500cda20 con 0x7f03500d2e80 2024-04-25T13:52:36.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034cff9640 1 -- 172.21.15.80:0/1473199439 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400b6860 con 0x7f03500d2e80 2024-04-25T13:52:36.127 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03500cdfe0 con 0x7f03500d2e80 2024-04-25T13:52:36.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035006a3a0 con 0x7f03500d2e80 2024-04-25T13:52:36.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034cff9640 1 -- 172.21.15.80:0/1473199439 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f03400b6a00 con 0x7f03500d2e80 2024-04-25T13:52:36.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034cff9640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f033c06f400 0x7f033c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.131+0000 7f034cff9640 1 -- 172.21.15.80:0/1473199439 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(190..190 src has 1..190) ==== 5709+0+0 (secure 0 0 0) 0x7f0340129740 con 0x7f03500d2e80 2024-04-25T13:52:36.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.135+0000 7f034f7fe640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f033c06f400 0x7f033c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.128 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.135+0000 7f034f7fe640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f033c06f400 0x7f033c071910 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f03500d5320 tx=0x7f0344008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.131 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.139+0000 7f034cff9640 1 -- 172.21.15.80:0/1473199439 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f03500d6300 con 0x7f03500d2e80 2024-04-25T13:52:36.316 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03500bbb90 con 0x7f03500d2e80 2024-04-25T13:52:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f034cff9640 1 -- 172.21.15.80:0/1473199439 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5179 (secure 0 0 0) 0x7f03400f6630 con 0x7f03500d2e80 2024-04-25T13:52:36.317 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:36.317 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":89,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T13:51:50.067203+0000","modified":"2024-04-25T13:51:53.127795+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9213},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9213":{"gid":9213,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.80:6835/3609922017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3609922017},{"type":"v1","addr":"172.21.15.80:6835","nonce":3609922017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9213,"qdb_cluster":[9213]},"id":11}]} 2024-04-25T13:52:36.317 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 89 2024-04-25T13:52:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f033c06f400 msgr2=0x7f033c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f033c06f400 0x7f033c071910 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f03500d5320 tx=0x7f0344008040 comp rx=0 tx=0).stop 2024-04-25T13:52:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 msgr2=0x7f03500cccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.319 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03500cccc0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f03400ae040 tx=0x7f034001aab0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 shutdown_connections 2024-04-25T13:52:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f033c06f400 0x7f033c071910 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03500d59b0 0x7f035015f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f03500d4330 0x7f0350155dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 --2- 172.21.15.80:0/1473199439 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f03500d2e80 0x7f03500cccc0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 >> 172.21.15.80:0/1473199439 conn(0x7f03500737a0 msgr2=0x7f0350156860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 shutdown_connections 2024-04-25T13:52:36.320 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.323+0000 7f0356a14640 1 -- 172.21.15.80:0/1473199439 wait complete. 2024-04-25T13:52:36.337 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:52:36.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f1831535640 1 Processor -- start 2024-04-25T13:52:36.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f1831535640 1 -- start start 2024-04-25T13:52:36.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f1831535640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d2e80 0x7f182c1607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f1831535640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f182c0d59b0 0x7f182c0d5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f1831535640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f182c0d6300 con 0x7f182c0d4330 2024-04-25T13:52:36.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f1831535640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f182c161060 con 0x7f182c0d59b0 2024-04-25T13:52:36.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f1831535640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f182c161340 con 0x7f182c0d2e80 2024-04-25T13:52:36.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182a7fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f182c0d59b0 0x7f182c0d5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182b7fe640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f182c0d4330 0x7f182c0d4740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:37490/0 (socket says 172.21.15.80:37490) 2024-04-25T13:52:36.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182b7fe640 1 -- 172.21.15.80:0/1713754858 learned_addr learned my addr 172.21.15.80:0/1713754858 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:36.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182affd640 1 --2- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d2e80 0x7f182c1607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182a7fc640 1 -- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d2e80 msgr2=0x7f182c1607a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:36.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182a7fc640 1 --2- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d2e80 0x7f182c1607a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182a7fc640 1 -- 172.21.15.80:0/1713754858 >> v1:172.21.15.87:6789/0 conn(0x7f182c0d4330 legacy=0x7f182c0d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182a7fc640 1 -- 172.21.15.80:0/1713754858 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182c0d2680 con 0x7f182c0d59b0 2024-04-25T13:52:36.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.475+0000 7f182affd640 1 --2- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d2e80 0x7f182c1607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:36.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f182a7fc640 1 --2- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d59b0 0x7f182c0d5dc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f18200097c0 tx=0x7f182001c410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f18eb53fda1cce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1829ffb640 1 -- 172.21.15.80:0/1713754858 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18200b0070 con 0x7f182c0d59b0 2024-04-25T13:52:36.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1829ffb640 1 -- 172.21.15.80:0/1713754858 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18200ab600 con 0x7f182c0d59b0 2024-04-25T13:52:36.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1829ffb640 1 -- 172.21.15.80:0/1713754858 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18200ab9d0 con 0x7f182c0d59b0 2024-04-25T13:52:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1831535640 1 -- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d59b0 msgr2=0x7f182c0d5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1831535640 1 --2- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d59b0 0x7f182c0d5dc0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f18200097c0 tx=0x7f182001c410 comp rx=0 tx=0).stop 2024-04-25T13:52:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1831535640 1 -- 172.21.15.80:0/1713754858 shutdown_connections 2024-04-25T13:52:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1831535640 1 --2- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d59b0 0x7f182c0d5dc0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1831535640 1 --2- 172.21.15.80:0/1713754858 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d2e80 0x7f182c1607a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1831535640 1 -- 172.21.15.80:0/1713754858 >> 172.21.15.80:0/1713754858 conn(0x7f182c0737a0 msgr2=0x7f182c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1831535640 1 -- 172.21.15.80:0/1713754858 shutdown_connections 2024-04-25T13:52:36.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.479+0000 7f1831535640 1 -- 172.21.15.80:0/1713754858 wait complete. 2024-04-25T13:52:36.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 Processor -- start 2024-04-25T13:52:36.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 -- start start 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d2e80 0x7f182c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0d4330 0x7f182c153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d59b0 0x7f182c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f182c0d3410 con 0x7f182c0d4330 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f182c0d3290 con 0x7f182c0d2e80 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f182c15df60 con 0x7f182c0d59b0 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0d4330 0x7f182c153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0d4330 0x7f182c153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37806/0 (socket says 172.21.15.80:37806) 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 -- 172.21.15.80:0/2332897805 learned_addr learned my addr 172.21.15.80:0/2332897805 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182affd640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d2e80 0x7f182c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182b7fe640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d59b0 0x7f182c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 -- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d59b0 msgr2=0x7f182c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d59b0 0x7f182c158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.479 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 -- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d2e80 msgr2=0x7f182c157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d2e80 0x7f182c157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 -- 172.21.15.80:0/2332897805 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f182c157a60 con 0x7f182c0d4330 2024-04-25T13:52:36.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f182a7fc640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0d4330 0x7f182c153a60 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f18200ae040 tx=0x7f18200191b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f18137fe640 1 -- 172.21.15.80:0/2332897805 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18200b0040 con 0x7f182c0d4330 2024-04-25T13:52:36.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f18137fe640 1 -- 172.21.15.80:0/2332897805 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f18200ab400 con 0x7f182c0d4330 2024-04-25T13:52:36.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f182c155ee0 con 0x7f182c0d4330 2024-04-25T13:52:36.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f18137fe640 1 -- 172.21.15.80:0/2332897805 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1820006e20 con 0x7f182c0d4330 2024-04-25T13:52:36.480 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f182c1564a0 con 0x7f182c0d4330 2024-04-25T13:52:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.483+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f182c06a3a0 con 0x7f182c0d4330 2024-04-25T13:52:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.487+0000 7f18137fe640 1 -- 172.21.15.80:0/2332897805 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f18200bc020 con 0x7f182c0d4330 2024-04-25T13:52:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.487+0000 7f18137fe640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f181806f430 0x7f1818071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.487+0000 7f18137fe640 1 -- 172.21.15.80:0/2332897805 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5709+0+0 (secure 0 0 0) 0x7f1820128520 con 0x7f182c0d4330 2024-04-25T13:52:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.487+0000 7f18137fe640 1 -- 172.21.15.80:0/2332897805 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=192}) -- 0x7f1818077010 con 0x7f182c0d4330 2024-04-25T13:52:36.481 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.487+0000 7f182affd640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f181806f430 0x7f1818071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.482 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.487+0000 7f182affd640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f181806f430 0x7f1818071940 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f182c0b8fd0 tx=0x7f181c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.491+0000 7f18137fe640 1 -- 172.21.15.80:0/2332897805 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1820016090 con 0x7f182c0d4330 2024-04-25T13:52:36.642 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.647+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f182c155c50 con 0x7f182c0d4330 2024-04-25T13:52:36.645 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f18137fe640 1 -- 172.21.15.80:0/2332897805 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) ==== 74+0+16626 (secure 0 0 0) 0x7f18200f5420 con 0x7f182c0d4330 2024-04-25T13:52:36.645 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:36.645 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":191,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:52:36.303114+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-04-25T13:51:48.404349+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"186","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-04-25T13:51:49.039512+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:52:36.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f181806f430 msgr2=0x7f1818071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f181806f430 0x7f1818071940 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f182c0b8fd0 tx=0x7f181c008040 comp rx=0 tx=0).stop 2024-04-25T13:52:36.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0d4330 msgr2=0x7f182c153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0d4330 0x7f182c153a60 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7f18200ae040 tx=0x7f18200191b0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 shutdown_connections 2024-04-25T13:52:36.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f181806f430 0x7f1818071940 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.647 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f182c0d59b0 0x7f182c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f182c0d4330 0x7f182c153a60 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 --2- 172.21.15.80:0/2332897805 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f182c0d2e80 0x7f182c157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 >> 172.21.15.80:0/2332897805 conn(0x7f182c0737a0 msgr2=0x7f182c0d5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:36.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 shutdown_connections 2024-04-25T13:52:36.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.651+0000 7f1831535640 1 -- 172.21.15.80:0/2332897805 wait complete. 2024-04-25T13:52:36.666 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T13:52:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80cd246640 1 Processor -- start 2024-04-25T13:52:36.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80cd246640 1 -- start start 2024-04-25T13:52:36.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80cd246640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80c80b2d30 0x7f80c80d1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80cd246640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80cd246640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80c80d30b0 con 0x7f80c80d0350 2024-04-25T13:52:36.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80cd246640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f80c80d1120 con 0x7f80c80b2d30 2024-04-25T13:52:36.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80cd246640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f80c80ce1c0 con 0x7f80c80ceca0 2024-04-25T13:52:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80c759e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80c6d9d640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f80c80b2d30 0x7f80c80d1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80c759e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:50166/0 (socket says 172.21.15.80:50166) 2024-04-25T13:52:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.823+0000 7f80c759e640 1 -- 172.21.15.80:0/3666544162 learned_addr learned my addr 172.21.15.80:0/3666544162 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.827+0000 7f80c759e640 1 -- 172.21.15.80:0/3666544162 >> v2:172.21.15.87:3300/0 conn(0x7f80c80b2d30 msgr2=0x7f80c80d1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.827+0000 7f80c759e640 1 --2- 172.21.15.80:0/3666544162 >> v2:172.21.15.87:3300/0 conn(0x7f80c80b2d30 0x7f80c80d1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.827+0000 7f80c759e640 1 -- 172.21.15.80:0/3666544162 >> v1:172.21.15.87:6789/0 conn(0x7f80c80d0350 legacy=0x7f80c80d0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.827+0000 7f80c759e640 1 -- 172.21.15.80:0/3666544162 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c80ce4a0 con 0x7f80c80ceca0 2024-04-25T13:52:36.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.827+0000 7f80c759e640 1 --2- 172.21.15.80:0/3666544162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80cf0b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009e50 tx=0x7f80c001ca30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96dbc63f11fbf17f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.827+0000 7f80c5d9b640 1 -- 172.21.15.80:0/3666544162 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00a57d0 con 0x7f80c80ceca0 2024-04-25T13:52:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.827+0000 7f80c5d9b640 1 -- 172.21.15.80:0/3666544162 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80c00a5970 con 0x7f80c80ceca0 2024-04-25T13:52:36.823 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.827+0000 7f80c5d9b640 1 -- 172.21.15.80:0/3666544162 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80c00a5d40 con 0x7f80c80ceca0 2024-04-25T13:52:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 -- 172.21.15.80:0/3666544162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 msgr2=0x7f80c80cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 --2- 172.21.15.80:0/3666544162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80cf0b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f80c0009e50 tx=0x7f80c001ca30 comp rx=0 tx=0).stop 2024-04-25T13:52:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 -- 172.21.15.80:0/3666544162 shutdown_connections 2024-04-25T13:52:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 --2- 172.21.15.80:0/3666544162 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80cf0b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 --2- 172.21.15.80:0/3666544162 >> v2:172.21.15.87:3300/0 conn(0x7f80c80b2d30 0x7f80c80d1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 -- 172.21.15.80:0/3666544162 >> 172.21.15.80:0/3666544162 conn(0x7f80c80d3ad0 msgr2=0x7f80c80d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:36.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 -- 172.21.15.80:0/3666544162 shutdown_connections 2024-04-25T13:52:36.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 -- 172.21.15.80:0/3666544162 wait complete. 2024-04-25T13:52:36.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 Processor -- start 2024-04-25T13:52:36.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.831+0000 7f80cd246640 1 -- start start 2024-04-25T13:52:36.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80c80b2d30 0x7f80c81590b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80b8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c80d0350 0x7f80c815f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.829 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f80c816d2f0 con 0x7f80c80b2d30 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f80c816d170 con 0x7f80c80d0350 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f80c816d470 con 0x7f80c80ceca0 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80c80b2d30 0x7f80c81590b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80c80b2d30 0x7f80c81590b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37812/0 (socket says 172.21.15.80:37812) 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 -- 172.21.15.80:0/4096084755 learned_addr learned my addr 172.21.15.80:0/4096084755 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c759e640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c80d0350 0x7f80c815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 -- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 msgr2=0x7f80c80b8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c659c640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80b8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80b8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 -- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c80d0350 msgr2=0x7f80c815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c80d0350 0x7f80c815f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 -- 172.21.15.80:0/4096084755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80c80b0900 con 0x7f80c80b2d30 2024-04-25T13:52:36.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c759e640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c80d0350 0x7f80c815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:36.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80c6d9d640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80c80b2d30 0x7f80c81590b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f80b800e840 tx=0x7f80b800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80af7fe640 1 -- 172.21.15.80:0/4096084755 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b8017070 con 0x7f80c80b2d30 2024-04-25T13:52:36.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80af7fe640 1 -- 172.21.15.80:0/4096084755 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80b800c940 con 0x7f80c80b2d30 2024-04-25T13:52:36.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80af7fe640 1 -- 172.21.15.80:0/4096084755 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80b800cd10 con 0x7f80c80b2d30 2024-04-25T13:52:36.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80c80b0be0 con 0x7f80c80b2d30 2024-04-25T13:52:36.831 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80c80b11a0 con 0x7f80c80b2d30 2024-04-25T13:52:36.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80c806a3a0 con 0x7f80c80b2d30 2024-04-25T13:52:36.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.835+0000 7f80af7fe640 1 -- 172.21.15.80:0/4096084755 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f80b801a070 con 0x7f80c80b2d30 2024-04-25T13:52:36.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.839+0000 7f80af7fe640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80b406f400 0x7f80b4071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:36.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.839+0000 7f80af7fe640 1 -- 172.21.15.80:0/4096084755 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(191..191 src has 1..191) ==== 5709+0+0 (secure 0 0 0) 0x7f80b808f5c0 con 0x7f80c80b2d30 2024-04-25T13:52:36.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.839+0000 7f80af7fe640 1 -- 172.21.15.80:0/4096084755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=192}) -- 0x7f80b4076f80 con 0x7f80c80b2d30 2024-04-25T13:52:36.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.839+0000 7f80c659c640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80b406f400 0x7f80b4071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:36.832 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.839+0000 7f80c659c640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80b406f400 0x7f80b4071910 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f80c80b79e0 tx=0x7f80bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:36.836 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:36.843+0000 7f80af7fe640 1 -- 172.21.15.80:0/4096084755 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f80b805c540 con 0x7f80c80b2d30 2024-04-25T13:52:37.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.023+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f80c8074c30 con 0x7f80c80b2d30 2024-04-25T13:52:37.019 INFO:tasks.ceph.mon.a.smithi080.stderr:2024-04-25T13:52:37.026+0000 7fa226fdc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T13:52:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.026+0000 7f80af7fe640 1 -- 172.21.15.80:0/4096084755 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v90) ==== 161+0+0 (secure 0 0 0) 0x7f80b805be40 con 0x7f80c80b2d30 2024-04-25T13:52:37.023 INFO:teuthology.orchestra.run.smithi080.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T13:52:37.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80b406f400 msgr2=0x7f80b4071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:37.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80b406f400 0x7f80b4071910 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f80c80b79e0 tx=0x7f80bc009040 comp rx=0 tx=0).stop 2024-04-25T13:52:37.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80c80b2d30 msgr2=0x7f80c81590b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:37.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80c80b2d30 0x7f80c81590b0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f80b800e840 tx=0x7f80b800ed00 comp rx=0 tx=0).stop 2024-04-25T13:52:37.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 shutdown_connections 2024-04-25T13:52:37.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80b406f400 0x7f80b4071910 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80c80d0350 0x7f80c815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.026 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80c80ceca0 0x7f80c80b8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 --2- 172.21.15.80:0/4096084755 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80c80b2d30 0x7f80c81590b0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.030+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 >> 172.21.15.80:0/4096084755 conn(0x7f80c80d3ad0 msgr2=0x7f80c80b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:37.030 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.034+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 shutdown_connections 2024-04-25T13:52:37.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.034+0000 7f80cd246640 1 -- 172.21.15.80:0/4096084755 wait complete. 2024-04-25T13:52:37.031 INFO:tasks.ceph.mds.a.smithi080.stderr: -8554> 2024-04-25T13:51:51.647+0000 7fd99e42d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T13:52:37.052 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T13:52:37.095 INFO:tasks.ceph.mds.a.smithi080.stdout:starting mds.a at 2024-04-25T13:52:37.199 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.202+0000 7f0e84998640 1 Processor -- start 2024-04-25T13:52:37.200 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e84998640 1 -- start start 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e84998640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80161270 0x7f0e800724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e84998640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0e800b08c0 0x7f0e80072e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e84998640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e80161680 con 0x7f0e80072a00 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e84998640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0e800d12e0 con 0x7f0e800b08c0 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e84998640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e80073350 con 0x7f0e80161270 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e7ed9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0e800b08c0 0x7f0e80072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e75d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0e80072a00 0x7f0e8006ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41606/0 (socket says 172.21.15.80:41606) 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e75d9b640 1 -- 172.21.15.80:0/3213304304 learned_addr learned my addr 172.21.15.80:0/3213304304 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e7e59c640 1 --2- 172.21.15.80:0/3213304304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80161270 0x7f0e800724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:37.202 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e7ed9d640 1 -- 172.21.15.80:0/3213304304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80161270 msgr2=0x7f0e800724c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e7ed9d640 1 --2- 172.21.15.80:0/3213304304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80161270 0x7f0e800724c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e7dd9b640 1 -- 172.21.15.80:0/3213304304 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3787775148 0 0) 0x7f0e80161680 con 0x7f0e80072a00 2024-04-25T13:52:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e7ed9d640 1 -- 172.21.15.80:0/3213304304 >> v1:172.21.15.80:6789/0 conn(0x7f0e80072a00 legacy=0x7f0e8006ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e7ed9d640 1 -- 172.21.15.80:0/3213304304 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e80163de0 con 0x7f0e800b08c0 2024-04-25T13:52:37.203 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.206+0000 7f0e7e59c640 1 --2- 172.21.15.80:0/3213304304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80161270 0x7f0e800724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e7ed9d640 1 --2- 172.21.15.80:0/3213304304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e800b08c0 0x7f0e80072e10 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009dc0 tx=0x7f0e7801c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fa58d2d3d40e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e7dd9b640 1 -- 172.21.15.80:0/3213304304 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780b2070 con 0x7f0e800b08c0 2024-04-25T13:52:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e7dd9b640 1 -- 172.21.15.80:0/3213304304 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e780a5830 con 0x7f0e800b08c0 2024-04-25T13:52:37.205 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e7dd9b640 1 -- 172.21.15.80:0/3213304304 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780a5c00 con 0x7f0e800b08c0 2024-04-25T13:52:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 -- 172.21.15.80:0/3213304304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e800b08c0 msgr2=0x7f0e80072e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 --2- 172.21.15.80:0/3213304304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e800b08c0 0x7f0e80072e10 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f0e78009dc0 tx=0x7f0e7801c970 comp rx=0 tx=0).stop 2024-04-25T13:52:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 -- 172.21.15.80:0/3213304304 shutdown_connections 2024-04-25T13:52:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 --2- 172.21.15.80:0/3213304304 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e800b08c0 0x7f0e80072e10 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 --2- 172.21.15.80:0/3213304304 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80161270 0x7f0e800724c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 -- 172.21.15.80:0/3213304304 >> 172.21.15.80:0/3213304304 conn(0x7f0e800b4160 msgr2=0x7f0e800d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 -- 172.21.15.80:0/3213304304 shutdown_connections 2024-04-25T13:52:37.206 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 -- 172.21.15.80:0/3213304304 wait complete. 2024-04-25T13:52:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 Processor -- start 2024-04-25T13:52:37.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.210+0000 7f0e84998640 1 -- start start 2024-04-25T13:52:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80072a00 0x7f0e8014a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e80161270 0x7f0e800b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e801527d0 0x7f0e800b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8016d300 con 0x7f0e801527d0 2024-04-25T13:52:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8016d180 con 0x7f0e80161270 2024-04-25T13:52:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8016d480 con 0x7f0e80072a00 2024-04-25T13:52:37.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e801527d0 0x7f0e800b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e801527d0 0x7f0e800b5550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37834/0 (socket says 172.21.15.80:37834) 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 -- 172.21.15.80:0/121047672 learned_addr learned my addr 172.21.15.80:0/121047672 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e75d9b640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e80161270 0x7f0e800b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 -- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80072a00 msgr2=0x7f0e8014a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80072a00 0x7f0e8014a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 -- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e80161270 msgr2=0x7f0e800b5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e80161270 0x7f0e800b5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 -- 172.21.15.80:0/121047672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e80157a40 con 0x7f0e801527d0 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e75d9b640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e80161270 0x7f0e800b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e7ed9d640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e801527d0 0x7f0e800b5550 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f0e780b0040 tx=0x7f0e78019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e777fe640 1 -- 172.21.15.80:0/121047672 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780b2050 con 0x7f0e801527d0 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e777fe640 1 -- 172.21.15.80:0/121047672 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e780b3040 con 0x7f0e801527d0 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e777fe640 1 -- 172.21.15.80:0/121047672 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e780ad850 con 0x7f0e801527d0 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e800b1600 con 0x7f0e801527d0 2024-04-25T13:52:37.211 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e800b1bc0 con 0x7f0e801527d0 2024-04-25T13:52:37.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e80157c10 con 0x7f0e801527d0 2024-04-25T13:52:37.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.214+0000 7f0e777fe640 1 -- 172.21.15.80:0/121047672 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0e780c0020 con 0x7f0e801527d0 2024-04-25T13:52:37.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.218+0000 7f0e777fe640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0e6406f400 0x7f0e64071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:37.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.218+0000 7f0e777fe640 1 -- 172.21.15.80:0/121047672 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 5795+0+0 (secure 0 0 0) 0x7f0e78129c80 con 0x7f0e801527d0 2024-04-25T13:52:37.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.218+0000 7f0e777fe640 1 -- 172.21.15.80:0/121047672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=193}) -- 0x7f0e64077080 con 0x7f0e801527d0 2024-04-25T13:52:37.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.218+0000 7f0e7e59c640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0e6406f400 0x7f0e64071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:37.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.218+0000 7f0e7e59c640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0e6406f400 0x7f0e64071910 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800a890 tx=0x7f0e68008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:37.216 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.222+0000 7f0e777fe640 1 -- 172.21.15.80:0/121047672 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0e78016560 con 0x7f0e801527d0 2024-04-25T13:52:37.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:37.402+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0e8006a3a0 con 0x7f0e801527d0 2024-04-25T13:52:38.027 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.030+0000 7f0e777fe640 1 -- 172.21.15.80:0/121047672 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v91) ==== 104+0+0 (secure 0 0 0) 0x7f0e780f6ba0 con 0x7f0e801527d0 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0e6406f400 msgr2=0x7f0e64071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0e6406f400 0x7f0e64071910 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f0e6800a890 tx=0x7f0e68008040 comp rx=0 tx=0).stop 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e801527d0 msgr2=0x7f0e800b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e801527d0 0x7f0e800b5550 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f0e780b0040 tx=0x7f0e78019710 comp rx=0 tx=0).stop 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 shutdown_connections 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0e6406f400 0x7f0e64071910 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0e801527d0 0x7f0e800b5550 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0e80161270 0x7f0e800b5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 --2- 172.21.15.80:0/121047672 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0e80072a00 0x7f0e8014a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 >> 172.21.15.80:0/121047672 conn(0x7f0e800b4160 msgr2=0x7f0e800757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 shutdown_connections 2024-04-25T13:52:38.031 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.034+0000 7f0e84998640 1 -- 172.21.15.80:0/121047672 wait complete. 2024-04-25T13:52:38.059 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T13:52:38.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc59dd86640 1 Processor -- start 2024-04-25T13:52:38.261 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc59dd86640 1 -- start start 2024-04-25T13:52:38.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc59dd86640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5980ce2a0 0x7fc5980ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:38.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc59dd86640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5980b1320 con 0x7fc5980b0f10 2024-04-25T13:52:38.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc59dd86640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5980cebf0 con 0x7fc598161d10 2024-04-25T13:52:38.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc59dd86640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc5980b5820 con 0x7fc5980ce2a0 2024-04-25T13:52:38.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5977fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fc5980b0f10 0x7fc5980d0c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41614/0 (socket says 172.21.15.80:41614) 2024-04-25T13:52:38.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5977fe640 1 -- 172.21.15.80:0/3038045990 learned_addr learned my addr 172.21.15.80:0/3038045990 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:38.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3462000402 0 0) 0x7fc588001120 con 0x7fc5980b0f10 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc580003530 con 0x7fc5980b0f10 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc596ffd640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5980ce2a0 0x7fc5980ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 267012748 0 0) 0x7fc580003530 con 0x7fc5980b0f10 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5980ce2a0 msgr2=0x7fc5980ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5980ce2a0 0x7fc5980ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 >> v1:172.21.15.87:6789/0 conn(0x7fc598161d10 legacy=0x7fc598162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5980b4a60 con 0x7fc5980b0f10 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fc588001850 con 0x7fc5980b0f10 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 >> v1:172.21.15.80:6789/0 conn(0x7fc5980b0f10 legacy=0x7fc5980d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5980ce2a0 0x7fc5800039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc580004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:38.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc580004d50 0x7fc580005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc5980b5820 con 0x7fc580003f70 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc58000ff70 con 0x7fc580004d50 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.266+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc580010170 con 0x7fc5980ce2a0 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc596ffd640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc580004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc597fff640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc580004d50 0x7fc580005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc596ffd640 1 -- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5980ce2a0 msgr2=0x7fc5800039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc596ffd640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc5980ce2a0 0x7fc5800039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc596ffd640 1 -- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc580004d50 msgr2=0x7fc580005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc596ffd640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc580004d50 0x7fc580005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.266 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc596ffd640 1 -- 172.21.15.80:0/3038045990 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc580003530 con 0x7fc580003f70 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc596ffd640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc580004810 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009ae0 tx=0x7fc58c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c878b397ba2923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0b2070 con 0x7fc580003f70 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58c0a55e0 con 0x7fc580003f70 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc5967fc640 1 -- 172.21.15.80:0/3038045990 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0096b0 con 0x7fc580003f70 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc59dd86640 1 -- 172.21.15.80:0/3038045990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 msgr2=0x7fc580004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc59dd86640 1 --2- 172.21.15.80:0/3038045990 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc580004810 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009ae0 tx=0x7fc58c01c9f0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc597fff640 1 -- 172.21.15.80:0/3038045990 reap_dead start 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc59dd86640 1 -- 172.21.15.80:0/3038045990 shutdown_connections 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc59dd86640 1 -- 172.21.15.80:0/3038045990 >> 172.21.15.80:0/3038045990 conn(0x7fc5980b35c0 msgr2=0x7fc5980b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:38.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc59dd86640 1 -- 172.21.15.80:0/3038045990 shutdown_connections 2024-04-25T13:52:38.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.270+0000 7fc59dd86640 1 -- 172.21.15.80:0/3038045990 wait complete. 2024-04-25T13:52:38.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 Processor -- start 2024-04-25T13:52:38.269 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 -- start start 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc59815c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc59815cdd0 0x7fc5981e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5981e7370 0x7fc5981e7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc5980b5020 con 0x7fc580003f70 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc5980b5300 con 0x7fc5981e7370 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc59815e720 con 0x7fc59815cdd0 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc59815c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc59815c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37852/0 (socket says 172.21.15.80:37852) 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 -- 172.21.15.80:0/3973838457 learned_addr learned my addr 172.21.15.80:0/3973838457 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:38.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc597fff640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5981e7370 0x7fc5981e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc596ffd640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc59815cdd0 0x7fc5981e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 -- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc59815cdd0 msgr2=0x7fc5981e6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc59815cdd0 0x7fc5981e6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 -- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5981e7370 msgr2=0x7fc5981e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5981e7370 0x7fc5981e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 -- 172.21.15.80:0/3973838457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc59815d1e0 con 0x7fc580003f70 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc5977fe640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc59815c890 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fc58800f310 tx=0x7fc58800f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc588007d00 con 0x7fc580003f70 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc58800a0d0 con 0x7fc580003f70 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc598148930 con 0x7fc580003f70 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58800a4c0 con 0x7fc580003f70 2024-04-25T13:52:38.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.274+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5981590f0 con 0x7fc580003f70 2024-04-25T13:52:38.274 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.278+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc59806a440 con 0x7fc580003f70 2024-04-25T13:52:38.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.278+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc588014070 con 0x7fc580003f70 2024-04-25T13:52:38.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.278+0000 7fc57ffff640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc58406f430 0x7fc584071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:38.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.278+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(192..192 src has 1..192) ==== 5795+0+0 (secure 0 0 0) 0x7fc588091c30 con 0x7fc580003f70 2024-04-25T13:52:38.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.278+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=193}) -- 0x7fc5840770c0 con 0x7fc580003f70 2024-04-25T13:52:38.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.278+0000 7fc596ffd640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc58406f430 0x7fc584071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:38.275 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.278+0000 7fc596ffd640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc58406f430 0x7fc584071940 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009ae0 tx=0x7fc58c0046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:38.278 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.282+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc59815d1e0 con 0x7fc580003f70 2024-04-25T13:52:38.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:38.430+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc598069c90 con 0x7fc580003f70 2024-04-25T13:52:39.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.042+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v193) ==== 181+0+0 (secure 0 0 0) 0x7fc5980b1320 con 0x7fc580003f70 2024-04-25T13:52:39.038 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_metadata' removed 2024-04-25T13:52:39.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.046+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 <== mon.0 v2:172.21.15.80:3300/0 8 ==== osd_map(193..193 src has 1..193) ==== 296+0+0 (secure 0 0 0) 0x7fc588090210 con 0x7fc580003f70 2024-04-25T13:52:39.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.046+0000 7fc57ffff640 1 -- 172.21.15.80:0/3973838457 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=194}) -- 0x7fc5840754f0 con 0x7fc580003f70 2024-04-25T13:52:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc58406f430 msgr2=0x7fc584071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc58406f430 0x7fc584071940 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009ae0 tx=0x7fc58c0046d0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.048 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 msgr2=0x7fc59815c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc59815c890 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fc58800f310 tx=0x7fc58800f7d0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 shutdown_connections 2024-04-25T13:52:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc58406f430 0x7fc584071940 secure :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc58c009ae0 tx=0x7fc58c0046d0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5981e7370 0x7fc5981e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc59815cdd0 0x7fc5981e6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 --2- 172.21.15.80:0/3973838457 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc580003f70 0x7fc59815c890 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.049 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 >> 172.21.15.80:0/3973838457 conn(0x7fc5980b35c0 msgr2=0x7fc598160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:39.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 shutdown_connections 2024-04-25T13:52:39.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.054+0000 7fc59dd86640 1 -- 172.21.15.80:0/3973838457 wait complete. 2024-04-25T13:52:39.072 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T13:52:39.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d2ac9640 1 Processor -- start 2024-04-25T13:52:39.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d2ac9640 1 -- start start 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d2ac9640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36cc0b0f10 0x7f36cc0d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d2ac9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f36cc0ce2a0 0x7f36cc0ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d2ac9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 0x7f36cc0cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d2ac9640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f36cc0b5820 con 0x7f36cc0b0f10 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d2ac9640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f36cc0d13f0 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d2ac9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc0b1320 con 0x7f36cc0cf730 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d0865640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36cc0b0f10 0x7f36cc0d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d0865640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f36cc0b0f10 0x7f36cc0d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37858/0 (socket says 172.21.15.80:37858) 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d0865640 1 -- 172.21.15.80:0/2789018156 learned_addr learned my addr 172.21.15.80:0/2789018156 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36cbfff640 1 --2- 172.21.15.80:0/2789018156 >> v2:172.21.15.87:3300/0 conn(0x7f36cc0ce2a0 0x7f36cc0ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.222+0000 7f36d1066640 1 --2- 172.21.15.80:0/2789018156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 0x7f36cc0cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d0865640 1 -- 172.21.15.80:0/2789018156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 msgr2=0x7f36cc0cfb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d0865640 1 --2- 172.21.15.80:0/2789018156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 0x7f36cc0cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d0865640 1 -- 172.21.15.80:0/2789018156 >> v2:172.21.15.87:3300/0 conn(0x7f36cc0ce2a0 msgr2=0x7f36cc0ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:39.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d0865640 1 --2- 172.21.15.80:0/2789018156 >> v2:172.21.15.87:3300/0 conn(0x7f36cc0ce2a0 0x7f36cc0ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d0865640 1 -- 172.21.15.80:0/2789018156 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc0cebf0 con 0x7f36cc0b0f10 2024-04-25T13:52:39.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d0865640 1 --2- 172.21.15.80:0/2789018156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0b0f10 0x7f36cc0d0c10 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0b14d0 tx=0x7f36bc01c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71f87dbd1a646bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36cb7fe640 1 -- 172.21.15.80:0/2789018156 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0b0070 con 0x7f36cc0b0f10 2024-04-25T13:52:39.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36cb7fe640 1 -- 172.21.15.80:0/2789018156 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36bc0ab600 con 0x7f36cc0b0f10 2024-04-25T13:52:39.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36cb7fe640 1 -- 172.21.15.80:0/2789018156 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0ab9d0 con 0x7f36cc0b0f10 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 -- 172.21.15.80:0/2789018156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0b0f10 msgr2=0x7f36cc0d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/2789018156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0b0f10 0x7f36cc0d0c10 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f36cc0b14d0 tx=0x7f36bc01c520 comp rx=0 tx=0).stop 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 -- 172.21.15.80:0/2789018156 shutdown_connections 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/2789018156 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 0x7f36cc0cfb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/2789018156 >> v2:172.21.15.87:3300/0 conn(0x7f36cc0ce2a0 0x7f36cc0ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/2789018156 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0b0f10 0x7f36cc0d0c10 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 -- 172.21.15.80:0/2789018156 >> 172.21.15.80:0/2789018156 conn(0x7f36cc0b35c0 msgr2=0x7f36cc0b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 -- 172.21.15.80:0/2789018156 shutdown_connections 2024-04-25T13:52:39.223 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.226+0000 7f36d2ac9640 1 -- 172.21.15.80:0/2789018156 wait complete. 2024-04-25T13:52:39.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 Processor -- start 2024-04-25T13:52:39.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 -- start start 2024-04-25T13:52:39.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36cc0b0f10 0x7f36cc150ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0ce2a0 0x7f36cc153520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 0x7f36cc1f8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc15e720 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f36cc0b5300 con 0x7f36cc0b0f10 2024-04-25T13:52:39.227 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f36cc0d0490 con 0x7f36cc0cf730 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0ce2a0 0x7f36cc153520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0ce2a0 0x7f36cc153520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37872/0 (socket says 172.21.15.80:37872) 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 -- 172.21.15.80:0/3948246422 learned_addr learned my addr 172.21.15.80:0/3948246422 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d0865640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36cc0b0f10 0x7f36cc150ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 -- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 msgr2=0x7f36cc1f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 0x7f36cc1f8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 -- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36cc0b0f10 msgr2=0x7f36cc150ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36cc0b0f10 0x7f36cc150ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 -- 172.21.15.80:0/3948246422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36cc0b3ee0 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d0865640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36cc0b0f10 0x7f36cc150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36cbfff640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0ce2a0 0x7f36cc153520 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f36c000d620 tx=0x7f36c000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c000b620 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c0019070 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36cc152480 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c000b9e0 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36cc152a40 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.230+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36cc1521f0 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.234+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f36c000bb80 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.234+0000 7f36c97fa640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f36b806f400 0x7f36b8071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:39.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.234+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(193..193 src has 1..193) ==== 5357+0+0 (secure 0 0 0) 0x7f36c008f530 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.234+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=194}) -- 0x7f36b8076280 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.234+0000 7f36d0865640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f36b806f400 0x7f36b8071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:39.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.234+0000 7f36d0865640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f36b806f400 0x7f36b8071910 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ad000 tx=0x7f36bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:39.233 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.238+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f36c005c610 con 0x7f36cc0ce2a0 2024-04-25T13:52:39.392 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:39.398+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f36cc06a440 con 0x7f36cc0ce2a0 2024-04-25T13:52:40.040 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.046+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v194) ==== 169+0+0 (secure 0 0 0) 0x7f36c005bf10 con 0x7f36cc0ce2a0 2024-04-25T13:52:40.041 INFO:teuthology.orchestra.run.smithi080.stderr:pool 'cephfs_data' removed 2024-04-25T13:52:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.046+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 <== mon.0 v2:172.21.15.80:3300/0 8 ==== osd_map(194..194 src has 1..194) ==== 296+0+0 (secure 0 0 0) 0x7f36c00c1e70 con 0x7f36cc0ce2a0 2024-04-25T13:52:40.050 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.046+0000 7f36c97fa640 1 -- 172.21.15.80:0/3948246422 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=195}) -- 0x7f36b8075010 con 0x7f36cc0ce2a0 2024-04-25T13:52:40.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f36b806f400 msgr2=0x7f36b8071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f36b806f400 0x7f36b8071910 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ad000 tx=0x7f36bc008040 comp rx=0 tx=0).stop 2024-04-25T13:52:40.051 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0ce2a0 msgr2=0x7f36cc153520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0ce2a0 0x7f36cc153520 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f36c000d620 tx=0x7f36c000dae0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 shutdown_connections 2024-04-25T13:52:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f36b806f400 0x7f36b8071910 secure :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0ad000 tx=0x7f36bc008040 comp rx=0 tx=0).stop 2024-04-25T13:52:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f36cc0cf730 0x7f36cc1f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f36cc0ce2a0 0x7f36cc153520 secure :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f36c000d620 tx=0x7f36c000dae0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 --2- 172.21.15.80:0/3948246422 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f36cc0b0f10 0x7f36cc150ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.052 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.054+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 >> 172.21.15.80:0/3948246422 conn(0x7f36cc0b35c0 msgr2=0x7f36cc160c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:40.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.062+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 shutdown_connections 2024-04-25T13:52:40.056 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.062+0000 7f36d2ac9640 1 -- 172.21.15.80:0/3948246422 wait complete. 2024-04-25T13:52:40.083 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T13:52:40.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe567c63640 1 Processor -- start 2024-04-25T13:52:40.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe567c63640 1 -- start start 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe567c63640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600d2ca0 0x7fe5600b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe567c63640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5600b92a0 con 0x7fe5600d3ed0 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe567c63640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5600d4c10 con 0x7fe5600b9ca0 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe567c63640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe5600d4ef0 con 0x7fe5600d2ca0 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5659ff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe5600d3ed0 0x7fe5600d2590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41626/0 (socket says 172.21.15.80:41626) 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5659ff640 1 -- 172.21.15.80:0/2056167330 learned_addr learned my addr 172.21.15.80:0/2056167330 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3062692209 0 0) 0x7fe55c001120 con 0x7fe5600d3ed0 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe54c003530 con 0x7fe5600d3ed0 2024-04-25T13:52:40.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5651fe640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600d2ca0 0x7fe5600b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3223498602 0 0) 0x7fe54c003530 con 0x7fe5600d3ed0 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600d2ca0 msgr2=0x7fe5600b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600d2ca0 0x7fe5600b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 >> v1:172.21.15.87:6789/0 conn(0x7fe5600b9ca0 legacy=0x7fe5600ba0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5600d51d0 con 0x7fe5600d3ed0 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fe55c001850 con 0x7fe5600d3ed0 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 >> v1:172.21.15.80:6789/0 conn(0x7fe5600d3ed0 legacy=0x7fe5600d2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54c0039f0 0x7fe54c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54c004810 0x7fe54c004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600d2ca0 0x7fe54c00b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe5600d4ef0 con 0x7fe54c0039f0 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c0055d0 con 0x7fe54c004810 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe54c00bf10 con 0x7fe5600d2ca0 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5659ff640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54c0039f0 0x7fe54c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.225 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5651fe640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54c004810 0x7fe54c004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe566200640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600d2ca0 0x7fe54c00b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5659ff640 1 -- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600d2ca0 msgr2=0x7fe54c00b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:40.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5659ff640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600d2ca0 0x7fe54c00b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5659ff640 1 -- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54c004810 msgr2=0x7fe54c004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:40.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5659ff640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54c004810 0x7fe54c004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5659ff640 1 -- 172.21.15.80:0/2056167330 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c003530 con 0x7fe54c0039f0 2024-04-25T13:52:40.226 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.226+0000 7fe5651fe640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe54c004810 0x7fe54c004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:40.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe5659ff640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54c0039f0 0x7fe54c003e60 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c4c0 tx=0x7fe55c01cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1757429069716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:40.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c00c0e0 con 0x7fe54c0039f0 2024-04-25T13:52:40.228 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe55c0a9070 con 0x7fe54c0039f0 2024-04-25T13:52:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe5649fd640 1 -- 172.21.15.80:0/2056167330 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c0a5de0 con 0x7fe54c0039f0 2024-04-25T13:52:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe567c63640 1 -- 172.21.15.80:0/2056167330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54c0039f0 msgr2=0x7fe54c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe567c63640 1 --2- 172.21.15.80:0/2056167330 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe54c0039f0 0x7fe54c003e60 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fe55c00c4c0 tx=0x7fe55c01cc80 comp rx=0 tx=0).stop 2024-04-25T13:52:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe566200640 1 -- 172.21.15.80:0/2056167330 reap_dead start 2024-04-25T13:52:40.229 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe567c63640 1 -- 172.21.15.80:0/2056167330 shutdown_connections 2024-04-25T13:52:40.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe567c63640 1 -- 172.21.15.80:0/2056167330 >> 172.21.15.80:0/2056167330 conn(0x7fe5600737b0 msgr2=0x7fe56006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:40.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe567c63640 1 -- 172.21.15.80:0/2056167330 shutdown_connections 2024-04-25T13:52:40.230 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe567c63640 1 -- 172.21.15.80:0/2056167330 wait complete. 2024-04-25T13:52:40.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe567c63640 1 Processor -- start 2024-04-25T13:52:40.231 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.234+0000 7fe567c63640 1 -- start start 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600c7eb0 0x7fe56016a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe56016adb0 0x7fe5600ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5600ce720 0x7fe56014c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe56016c290 con 0x7fe5600ce720 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe560160830 con 0x7fe56016adb0 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe56016c410 con 0x7fe5600c7eb0 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5600ce720 0x7fe56014c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5600ce720 0x7fe56014c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37878/0 (socket says 172.21.15.80:37878) 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 -- 172.21.15.80:0/1250509696 learned_addr learned my addr 172.21.15.80:0/1250509696 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe5651fe640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe56016adb0 0x7fe5600ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.234 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe5659ff640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600c7eb0 0x7fe56016a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 -- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600c7eb0 msgr2=0x7fe56016a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600c7eb0 0x7fe56016a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 -- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe56016adb0 msgr2=0x7fe5600ce1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe56016adb0 0x7fe5600ce1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 -- 172.21.15.80:0/1250509696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5600ceb30 con 0x7fe5600ce720 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe566200640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5600ce720 0x7fe56014c710 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fe56008ebf0 tx=0x7fe55800eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe5567fc640 1 -- 172.21.15.80:0/1250509696 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558003980 con 0x7fe5600ce720 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe5567fc640 1 -- 172.21.15.80:0/1250509696 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe558018070 con 0x7fe5600ce720 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe5567fc640 1 -- 172.21.15.80:0/1250509696 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe558003d60 con 0x7fe5600ce720 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56016ab60 con 0x7fe5600ce720 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe560151460 con 0x7fe5600ce720 2024-04-25T13:52:40.235 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56006a3a0 con 0x7fe5600ce720 2024-04-25T13:52:40.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe5567fc640 1 -- 172.21.15.80:0/1250509696 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe5580050d0 con 0x7fe5600ce720 2024-04-25T13:52:40.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.238+0000 7fe5567fc640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe54406f400 0x7fe544071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.242+0000 7fe5567fc640 1 -- 172.21.15.80:0/1250509696 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fe55808eb60 con 0x7fe5600ce720 2024-04-25T13:52:40.236 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.242+0000 7fe5659ff640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe54406f400 0x7fe544071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.242+0000 7fe5659ff640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe54406f400 0x7fe544071910 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0ab000 tx=0x7fe55c0b0040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:40.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.246+0000 7fe5567fc640 1 -- 172.21.15.80:0/1250509696 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe5600ceb30 con 0x7fe5600ce720 2024-04-25T13:52:40.389 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.394+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe56006c970 con 0x7fe5600ce720 2024-04-25T13:52:40.395 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe5567fc640 1 -- 172.21.15.80:0/1250509696 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) ==== 104+0+0 (secure 0 0 0) 0x7fe55805be70 con 0x7fe5600ce720 2024-04-25T13:52:40.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe54406f400 msgr2=0x7fe544071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe54406f400 0x7fe544071910 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fe55c0ab000 tx=0x7fe55c0b0040 comp rx=0 tx=0).stop 2024-04-25T13:52:40.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5600ce720 msgr2=0x7fe56014c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5600ce720 0x7fe56014c710 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fe56008ebf0 tx=0x7fe55800eb90 comp rx=0 tx=0).stop 2024-04-25T13:52:40.398 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 shutdown_connections 2024-04-25T13:52:40.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe54406f400 0x7fe544071910 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe5600ce720 0x7fe56014c710 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe56016adb0 0x7fe5600ce1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 --2- 172.21.15.80:0/1250509696 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe5600c7eb0 0x7fe56016a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 >> 172.21.15.80:0/1250509696 conn(0x7fe5600737b0 msgr2=0x7fe5601550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:40.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 shutdown_connections 2024-04-25T13:52:40.399 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.402+0000 7fe567c63640 1 -- 172.21.15.80:0/1250509696 wait complete. 2024-04-25T13:52:40.417 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2024-04-25T13:52:40.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc061d88640 1 Processor -- start 2024-04-25T13:52:40.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc061d88640 1 -- start start 2024-04-25T13:52:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc061d88640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc05c161240 0x7fc05c0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc061d88640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc05c161650 0x7fc05c0728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc061d88640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc05c06fcf0 con 0x7fc05c06f5a0 2024-04-25T13:52:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc061d88640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fc05c073f10 con 0x7fc05c161650 2024-04-25T13:52:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc061d88640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc05c0b0c60 con 0x7fc05c161240 2024-04-25T13:52:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc05bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc05c161650 0x7fc05c0728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc05bfff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fc05c161650 0x7fc05c0728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37890/0 (socket says 172.21.15.80:37890) 2024-04-25T13:52:40.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc05bfff640 1 -- 172.21.15.80:0/1250521441 learned_addr learned my addr 172.21.15.80:0/1250521441 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:40.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc05b7fe640 1 --1- 172.21.15.80:0/1250521441 >> v1:172.21.15.80:6789/0 conn(0x7fc05c06f5a0 0x7fc05c160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:41638/0 (socket says 172.21.15.80:41638) 2024-04-25T13:52:40.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc053fff640 1 --2- 172.21.15.80:0/1250521441 >> v2:172.21.15.87:3300/0 conn(0x7fc05c161240 0x7fc05c0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc05bfff640 1 -- 172.21.15.80:0/1250521441 >> v2:172.21.15.87:3300/0 conn(0x7fc05c161240 msgr2=0x7fc05c0723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:40.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc05bfff640 1 --2- 172.21.15.80:0/1250521441 >> v2:172.21.15.87:3300/0 conn(0x7fc05c161240 0x7fc05c0723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc05bfff640 1 -- 172.21.15.80:0/1250521441 >> v1:172.21.15.80:6789/0 conn(0x7fc05c06f5a0 legacy=0x7fc05c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.567 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.570+0000 7fc05bfff640 1 -- 172.21.15.80:0/1250521441 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c0b0f40 con 0x7fc05c161650 2024-04-25T13:52:40.568 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc05bfff640 1 --2- 172.21.15.80:0/1250521441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c161650 0x7fc05c0728e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc05c06ff70 tx=0x7fc05401c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10c7891cf50f720 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:40.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc05affd640 1 -- 172.21.15.80:0/1250521441 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0540a5620 con 0x7fc05c161650 2024-04-25T13:52:40.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc05affd640 1 -- 172.21.15.80:0/1250521441 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0540a57c0 con 0x7fc05c161650 2024-04-25T13:52:40.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc05affd640 1 -- 172.21.15.80:0/1250521441 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0540a5b90 con 0x7fc05c161650 2024-04-25T13:52:40.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc061d88640 1 -- 172.21.15.80:0/1250521441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c161650 msgr2=0x7fc05c0728e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.571 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc061d88640 1 --2- 172.21.15.80:0/1250521441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c161650 0x7fc05c0728e0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc05c06ff70 tx=0x7fc05401c960 comp rx=0 tx=0).stop 2024-04-25T13:52:40.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc061d88640 1 -- 172.21.15.80:0/1250521441 shutdown_connections 2024-04-25T13:52:40.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc061d88640 1 --2- 172.21.15.80:0/1250521441 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c161650 0x7fc05c0728e0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc061d88640 1 --2- 172.21.15.80:0/1250521441 >> v2:172.21.15.87:3300/0 conn(0x7fc05c161240 0x7fc05c0723a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc061d88640 1 -- 172.21.15.80:0/1250521441 >> 172.21.15.80:0/1250521441 conn(0x7fc05c06d080 msgr2=0x7fc05c0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:40.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc061d88640 1 -- 172.21.15.80:0/1250521441 shutdown_connections 2024-04-25T13:52:40.572 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.574+0000 7fc061d88640 1 -- 172.21.15.80:0/1250521441 wait complete. 2024-04-25T13:52:40.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 Processor -- start 2024-04-25T13:52:40.573 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 -- start start 2024-04-25T13:52:40.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c06f5a0 0x7fc05c15fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc05c161240 0x7fc05c141140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c161650 0x7fc05c1fd1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c06e820 con 0x7fc05c06f5a0 2024-04-25T13:52:40.576 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc05c06e6a0 con 0x7fc05c161650 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc05c06e9a0 con 0x7fc05c161240 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c06f5a0 0x7fc05c15fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c06f5a0 0x7fc05c15fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37892/0 (socket says 172.21.15.80:37892) 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 -- 172.21.15.80:0/1632361862 learned_addr learned my addr 172.21.15.80:0/1632361862 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05bfff640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c161650 0x7fc05c1fd1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc053fff640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc05c161240 0x7fc05c141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 -- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc05c161240 msgr2=0x7fc05c141140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc05c161240 0x7fc05c141140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 -- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c161650 msgr2=0x7fc05c1fd1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c161650 0x7fc05c1fd1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 -- 172.21.15.80:0/1632361862 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc05c06e220 con 0x7fc05c06f5a0 2024-04-25T13:52:40.577 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05bfff640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c161650 0x7fc05c1fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:40.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc053fff640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc05c161240 0x7fc05c141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:40.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc05b7fe640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c06f5a0 0x7fc05c15fc20 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00c370 tx=0x7fc04c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:40.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc058ff9640 1 -- 172.21.15.80:0/1632361862 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c0074e0 con 0x7fc05c06f5a0 2024-04-25T13:52:40.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc058ff9640 1 -- 172.21.15.80:0/1632361862 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc04c007680 con 0x7fc05c06f5a0 2024-04-25T13:52:40.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc058ff9640 1 -- 172.21.15.80:0/1632361862 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc04c004a30 con 0x7fc05c06f5a0 2024-04-25T13:52:40.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc05c141680 con 0x7fc05c06f5a0 2024-04-25T13:52:40.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.578+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc05c14fda0 con 0x7fc05c06f5a0 2024-04-25T13:52:40.578 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.582+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc05c06a3a0 con 0x7fc05c06f5a0 2024-04-25T13:52:40.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.582+0000 7fc058ff9640 1 -- 172.21.15.80:0/1632361862 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc04c01a020 con 0x7fc05c06f5a0 2024-04-25T13:52:40.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.582+0000 7fc058ff9640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc03c06f400 0x7fc03c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:40.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.582+0000 7fc058ff9640 1 -- 172.21.15.80:0/1632361862 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fc04c08e6f0 con 0x7fc05c06f5a0 2024-04-25T13:52:40.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.582+0000 7fc053fff640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc03c06f400 0x7fc03c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:40.579 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.582+0000 7fc053fff640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc03c06f400 0x7fc03c071910 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc05c141f40 tx=0x7fc044008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:40.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.586+0000 7fc058ff9640 1 -- 172.21.15.80:0/1632361862 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc04c05b970 con 0x7fc05c06f5a0 2024-04-25T13:52:40.791 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:40.794+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) -- 0x7fc05c0739c0 con 0x7fc05c06f5a0 2024-04-25T13:52:41.570 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.574+0000 7fc058ff9640 1 -- 172.21.15.80:0/1632361862 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v595) ==== 135+0+0 (secure 0 0 0) 0x7fc04c05b270 con 0x7fc05c06f5a0 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc03c06f400 msgr2=0x7fc03c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc03c06f400 0x7fc03c071910 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc05c141f40 tx=0x7fc044008040 comp rx=0 tx=0).stop 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c06f5a0 msgr2=0x7fc05c15fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c06f5a0 0x7fc05c15fc20 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7fc04c00c370 tx=0x7fc04c00c830 comp rx=0 tx=0).stop 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 shutdown_connections 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc03c06f400 0x7fc03c071910 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc05c161650 0x7fc05c1fd1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc05c161240 0x7fc05c141140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 --2- 172.21.15.80:0/1632361862 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc05c06f5a0 0x7fc05c15fc20 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 >> 172.21.15.80:0/1632361862 conn(0x7fc05c06d080 msgr2=0x7fc05c154b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 shutdown_connections 2024-04-25T13:52:41.574 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:41.578+0000 7fc061d88640 1 -- 172.21.15.80:0/1632361862 wait complete. 2024-04-25T13:52:41.600 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2024-04-25T13:52:41.600 INFO:tasks.cephfs_test_runner: 2024-04-25T13:52:41.600 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T13:52:41.600 INFO:tasks.cephfs_test_runner:Ran 10 tests in 700.376s 2024-04-25T13:52:41.600 INFO:tasks.cephfs_test_runner: 2024-04-25T13:52:41.600 INFO:tasks.cephfs_test_runner:OK 2024-04-25T13:52:41.601 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T13:52:41.612 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T13:52:41.620 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T13:52:41.620 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:41.621 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:41.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:52:41.627 DEBUG:teuthology.orchestra.run.smithi080:> set -ex 2024-04-25T13:52:41.627 DEBUG:teuthology.orchestra.run.smithi080:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:41.673 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336464k,nr_inodes=4084116,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=16927 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274676k,nr_inodes=818669,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274680k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T13:52:41.673 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:52:41.673 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:41.680 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:52:41.680 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-04-25T13:52:41.680 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T13:52:41.726 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16336444k,nr_inodes=4084111,mode=755,inode64 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17685 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274672k,nr_inodes=818668,mode=700,uid=1000,gid=1271,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274676k,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T13:52:41.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:41.726 DEBUG:teuthology.orchestra.run.smithi080:> ip netns list 2024-04-25T13:52:41.732 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T13:52:41.732 INFO:teuthology.orchestra.run.smithi080.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T13:52:41.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:41.733 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T13:52:41.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:41.790 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T13:52:41.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:41.869 DEBUG:teuthology.orchestra.run.smithi080:> sudo ip link delete ceph-brx 2024-04-25T13:52:42.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:42.021 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-04-25T13:52:42.028 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T13:52:42.028 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T13:52:42.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:42.028 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T13:52:42.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:42.083 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T13:52:42.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T13:52:42.170 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-04-25T13:52:42.343 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T13:52:42.353 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T13:52:42.353 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:52:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd0268c5640 1 Processor -- start 2024-04-25T13:52:42.513 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd0268c5640 1 -- start start 2024-04-25T13:52:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd0268c5640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0200b0eb0 0x7fd0200d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd0268c5640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd0200d1570 0x7fd020160a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd0268c5640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd02015af50 con 0x7fd02015ab40 2024-04-25T13:52:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd0268c5640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd02006fb10 con 0x7fd0200b0eb0 2024-04-25T13:52:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd0268c5640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd02006fdf0 con 0x7fd0200d1570 2024-04-25T13:52:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd01f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0200b0eb0 0x7fd0200d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.515 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd01f7fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd0200b0eb0 0x7fd0200d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36860/0 (socket says 172.21.15.80:36860) 2024-04-25T13:52:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd01f7fe640 1 -- 172.21.15.80:0/370874816 learned_addr learned my addr 172.21.15.80:0/370874816 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd024e62640 1 --2- 172.21.15.80:0/370874816 >> v2:172.21.15.87:3300/0 conn(0x7fd0200d1570 0x7fd020160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd01f7fe640 1 -- 172.21.15.80:0/370874816 >> v2:172.21.15.87:3300/0 conn(0x7fd0200d1570 msgr2=0x7fd020160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd01f7fe640 1 --2- 172.21.15.80:0/370874816 >> v2:172.21.15.87:3300/0 conn(0x7fd0200d1570 0x7fd020160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd01f7fe640 1 -- 172.21.15.80:0/370874816 >> v1:172.21.15.87:6789/0 conn(0x7fd02015ab40 legacy=0x7fd0200b0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:42.516 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.518+0000 7fd01f7fe640 1 -- 172.21.15.80:0/370874816 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0200700d0 con 0x7fd0200b0eb0 2024-04-25T13:52:42.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd01f7fe640 1 --2- 172.21.15.80:0/370874816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200b0eb0 0x7fd0200d1030 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fd0200ce3c0 tx=0x7fd01401c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=481cf037274ae577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:42.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd01effd640 1 -- 172.21.15.80:0/370874816 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0140a5610 con 0x7fd0200b0eb0 2024-04-25T13:52:42.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd01effd640 1 -- 172.21.15.80:0/370874816 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0140a57b0 con 0x7fd0200b0eb0 2024-04-25T13:52:42.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd01effd640 1 -- 172.21.15.80:0/370874816 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0140a5b80 con 0x7fd0200b0eb0 2024-04-25T13:52:42.518 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd0268c5640 1 -- 172.21.15.80:0/370874816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200b0eb0 msgr2=0x7fd0200d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd0268c5640 1 --2- 172.21.15.80:0/370874816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200b0eb0 0x7fd0200d1030 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fd0200ce3c0 tx=0x7fd01401c7d0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd0268c5640 1 -- 172.21.15.80:0/370874816 shutdown_connections 2024-04-25T13:52:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd0268c5640 1 --2- 172.21.15.80:0/370874816 >> v2:172.21.15.87:3300/0 conn(0x7fd0200d1570 0x7fd020160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd0268c5640 1 --2- 172.21.15.80:0/370874816 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200b0eb0 0x7fd0200d1030 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd0268c5640 1 -- 172.21.15.80:0/370874816 >> 172.21.15.80:0/370874816 conn(0x7fd0200d1d10 msgr2=0x7fd0200d2120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd0268c5640 1 -- 172.21.15.80:0/370874816 shutdown_connections 2024-04-25T13:52:42.519 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.522+0000 7fd0268c5640 1 -- 172.21.15.80:0/370874816 wait complete. 2024-04-25T13:52:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 Processor -- start 2024-04-25T13:52:42.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 -- start start 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0200b0eb0 0x7fd020072cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200d1570 0x7fd020151330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd02015ab40 0x7fd0200b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd0200b1440 con 0x7fd0200d1570 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd0200b12c0 con 0x7fd02015ab40 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd02016d400 con 0x7fd0200b0eb0 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200d1570 0x7fd020151330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200d1570 0x7fd020151330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36872/0 (socket says 172.21.15.80:36872) 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 -- 172.21.15.80:0/2275639146 learned_addr learned my addr 172.21.15.80:0/2275639146 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 -- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0200b0eb0 msgr2=0x7fd020072cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0200b0eb0 0x7fd020072cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 -- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd02015ab40 msgr2=0x7fd0200b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd024e62640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd02015ab40 0x7fd0200b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd02015ab40 0x7fd0200b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 -- 172.21.15.80:0/2275639146 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd020073610 con 0x7fd0200d1570 2024-04-25T13:52:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01f7fe640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200d1570 0x7fd020151330 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fd0140af040 tx=0x7fd014019570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01cff9640 1 -- 172.21.15.80:0/2275639146 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd014009510 con 0x7fd0200d1570 2024-04-25T13:52:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01cff9640 1 -- 172.21.15.80:0/2275639146 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd0140be070 con 0x7fd0200d1570 2024-04-25T13:52:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd020158300 con 0x7fd0200d1570 2024-04-25T13:52:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd01cff9640 1 -- 172.21.15.80:0/2275639146 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0140b88d0 con 0x7fd0200d1570 2024-04-25T13:52:42.524 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0201588c0 con 0x7fd0200d1570 2024-04-25T13:52:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.526+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd02006a3a0 con 0x7fd0200d1570 2024-04-25T13:52:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.530+0000 7fd01cff9640 1 -- 172.21.15.80:0/2275639146 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd0140b8a70 con 0x7fd0200d1570 2024-04-25T13:52:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.530+0000 7fd01cff9640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd00c06f400 0x7fd00c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.530+0000 7fd01cff9640 1 -- 172.21.15.80:0/2275639146 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fd01412a9a0 con 0x7fd0200d1570 2024-04-25T13:52:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.530+0000 7fd01ffff640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd00c06f400 0x7fd00c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.530+0000 7fd01ffff640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd00c06f400 0x7fd00c071910 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd02006dbc0 tx=0x7fd018006bd0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:42.528 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.534+0000 7fd01cff9640 1 -- 172.21.15.80:0/2275639146 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fd02015af50 con 0x7fd0200d1570 2024-04-25T13:52:42.687 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.690+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd0201595a0 con 0x7fd00c06f400 2024-04-25T13:52:42.689 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.694+0000 7fd01cff9640 1 -- 172.21.15.80:0/2275639146 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12586 (secure 0 0 0) 0x7fd0201595a0 con 0x7fd00c06f400 2024-04-25T13:52:42.689 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:42.690 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:52:42.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.694+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd00c06f400 msgr2=0x7fd00c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.694+0000 7fd0268c5640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd00c06f400 0x7fd00c071910 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd02006dbc0 tx=0x7fd018006bd0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.694+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200d1570 msgr2=0x7fd020151330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.694+0000 7fd0268c5640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200d1570 0x7fd020151330 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fd0140af040 tx=0x7fd014019570 comp rx=0 tx=0).stop 2024-04-25T13:52:42.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.698+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 shutdown_connections 2024-04-25T13:52:42.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.698+0000 7fd0268c5640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd00c06f400 0x7fd00c071910 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.698+0000 7fd0268c5640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd02015ab40 0x7fd0200b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.692 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.698+0000 7fd0268c5640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd0200d1570 0x7fd020151330 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.698+0000 7fd0268c5640 1 --2- 172.21.15.80:0/2275639146 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd0200b0eb0 0x7fd020072cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.698+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 >> 172.21.15.80:0/2275639146 conn(0x7fd0200d1d10 msgr2=0x7fd0200b2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:42.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.698+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 shutdown_connections 2024-04-25T13:52:42.693 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.698+0000 7fd0268c5640 1 -- 172.21.15.80:0/2275639146 wait complete. 2024-04-25T13:52:42.712 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":558,"stamp":"2024-04-25T13:52:42.374439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450347,"statfs":{"total":1600000000,"available":1485155902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485517,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-136,"num_write_kb":-85086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002252"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":446,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-04-25T13:52:37.036175+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:52:37.036175+0000","last_peered":"2024-04-25T13:52:37.036175+0000","last_clean":"2024-04-25T13:52:37.036175+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:52:37.036175+0000","last_undegraded":"2024-04-25T13:52:37.036175+0000","last_fullsized":"2024-04-25T13:52:37.036175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9884,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185427,"statfs":{"total":200000000,"available":189878024,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175819,"statfs":{"total":200000000,"available":180038867,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21541,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173770,"statfs":{"total":200000000,"available":177941335,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7259,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188052,"statfs":{"total":200000000,"available":192566216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9253,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186059,"statfs":{"total":200000000,"available":190524615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21541,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173771,"statfs":{"total":200000000,"available":177941849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13349,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181963,"statfs":{"total":200000000,"available":186330513,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9829,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185482,"statfs":{"total":200000000,"available":189934483,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:52:42.712 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:52:42.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c4755e640 1 Processor -- start 2024-04-25T13:52:42.860 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c4755e640 1 -- start start 2024-04-25T13:52:42.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c4755e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c400b1a30 0x7f7c4015dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c4755e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c4015e4f0 con 0x7f7c400c7d40 2024-04-25T13:52:42.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c4755e640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7c400b1e40 con 0x7f7c40068590 2024-04-25T13:52:42.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c4755e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c400d2520 con 0x7f7c400b1a30 2024-04-25T13:52:42.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c45afb640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f7c400c7d40 0x7f7c400c8150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55350/0 (socket says 172.21.15.80:55350) 2024-04-25T13:52:42.862 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c45afb640 1 -- 172.21.15.80:0/4025976715 learned_addr learned my addr 172.21.15.80:0/4025976715 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:42.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3832920418 0 0) 0x7f7c4015e4f0 con 0x7f7c400c7d40 2024-04-25T13:52:42.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c452fa640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c400b1a30 0x7f7c4015dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7c2c003530 con 0x7f7c400c7d40 2024-04-25T13:52:42.864 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2973717186 0 0) 0x7f7c2c003530 con 0x7f7c400c7d40 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c400b1a30 msgr2=0x7f7c4015dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c400b1a30 0x7f7c4015dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 >> v1:172.21.15.87:6789/0 conn(0x7f7c40068590 legacy=0x7f7c400689a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c401627d0 con 0x7f7c400c7d40 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7c38001a20 con 0x7f7c400c7d40 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 >> v1:172.21.15.80:6789/0 conn(0x7f7c400c7d40 legacy=0x7f7c400c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0039f0 0x7f7c2c003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c400b1a30 0x7f7c2c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c004d50 0x7f7c2c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7c400d2520 con 0x7f7c2c004d50 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c2c00bb90 con 0x7f7c2c0039f0 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c2c00bd90 con 0x7f7c400b1a30 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c45afb640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c004d50 0x7f7c2c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.865 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c452fa640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0039f0 0x7f7c2c003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c44af9640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c400b1a30 0x7f7c2c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c45afb640 1 -- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c400b1a30 msgr2=0x7f7c2c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:42.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c45afb640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c400b1a30 0x7f7c2c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c45afb640 1 -- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0039f0 msgr2=0x7f7c2c003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:42.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c45afb640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c2c0039f0 0x7f7c2c003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.866 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.866+0000 7f7c45afb640 1 -- 172.21.15.80:0/4025976715 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c2c003530 con 0x7f7c2c004d50 2024-04-25T13:52:42.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c45afb640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c004d50 0x7f7c2c005160 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800a850 tx=0x7f7c3801fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6e24cf6aae88ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:42.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c3800a4a0 con 0x7f7c2c004d50 2024-04-25T13:52:42.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c380aa710 con 0x7f7c2c004d50 2024-04-25T13:52:42.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c37fff640 1 -- 172.21.15.80:0/4025976715 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c380aab00 con 0x7f7c2c004d50 2024-04-25T13:52:42.867 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c4755e640 1 -- 172.21.15.80:0/4025976715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c004d50 msgr2=0x7f7c2c005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c4755e640 1 --2- 172.21.15.80:0/4025976715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c2c004d50 0x7f7c2c005160 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f7c3800a850 tx=0x7f7c3801fac0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c45afb640 1 -- 172.21.15.80:0/4025976715 reap_dead start 2024-04-25T13:52:42.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c4755e640 1 -- 172.21.15.80:0/4025976715 shutdown_connections 2024-04-25T13:52:42.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c4755e640 1 -- 172.21.15.80:0/4025976715 >> 172.21.15.80:0/4025976715 conn(0x7f7c400cf390 msgr2=0x7f7c400cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:42.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c4755e640 1 -- 172.21.15.80:0/4025976715 shutdown_connections 2024-04-25T13:52:42.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.870+0000 7f7c4755e640 1 -- 172.21.15.80:0/4025976715 wait complete. 2024-04-25T13:52:42.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 Processor -- start 2024-04-25T13:52:42.868 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 -- start start 2024-04-25T13:52:42.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c004d50 0x7f7c40160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.871 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c40160fc0 0x7f7c4014ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c4014b340 0x7f7c401f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f7c400d07c0 con 0x7f7c40160fc0 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f7c400d04e0 con 0x7f7c4014b340 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f7c400d0aa0 con 0x7f7c2c004d50 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c40160fc0 0x7f7c4014ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c40160fc0 0x7f7c4014ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36888/0 (socket says 172.21.15.80:36888) 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 -- 172.21.15.80:0/3956596644 learned_addr learned my addr 172.21.15.80:0/3956596644 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c452fa640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c004d50 0x7f7c40160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c45afb640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c4014b340 0x7f7c401f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 -- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c004d50 msgr2=0x7f7c40160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c004d50 0x7f7c40160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 -- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c4014b340 msgr2=0x7f7c401f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:42.872 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c4014b340 0x7f7c401f7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:42.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 -- 172.21.15.80:0/3956596644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c4014b750 con 0x7f7c40160fc0 2024-04-25T13:52:42.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c45afb640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c4014b340 0x7f7c401f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:42.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c44af9640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c40160fc0 0x7f7c4014ae00 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000e4b0 tx=0x7f7c3000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:42.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c35ffb640 1 -- 172.21.15.80:0/3956596644 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c300052e0 con 0x7f7c40160fc0 2024-04-25T13:52:42.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c35ffb640 1 -- 172.21.15.80:0/3956596644 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7c30018070 con 0x7f7c40160fc0 2024-04-25T13:52:42.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c35ffb640 1 -- 172.21.15.80:0/3956596644 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c30003820 con 0x7f7c40160fc0 2024-04-25T13:52:42.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c4014bba0 con 0x7f7c40160fc0 2024-04-25T13:52:42.873 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.874+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c40146f10 con 0x7f7c40160fc0 2024-04-25T13:52:42.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.878+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c40147340 con 0x7f7c40160fc0 2024-04-25T13:52:42.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.878+0000 7f7c35ffb640 1 -- 172.21.15.80:0/3956596644 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f7c300039c0 con 0x7f7c40160fc0 2024-04-25T13:52:42.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.878+0000 7f7c35ffb640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2406f400 0x7f7c24071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:42.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.878+0000 7f7c35ffb640 1 -- 172.21.15.80:0/3956596644 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f7c3008ecd0 con 0x7f7c40160fc0 2024-04-25T13:52:42.874 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.878+0000 7f7c452fa640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2406f400 0x7f7c24071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:42.875 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.878+0000 7f7c452fa640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2406f400 0x7f7c24071910 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7c40072b20 tx=0x7f7c3c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:42.878 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:42.882+0000 7f7c35ffb640 1 -- 172.21.15.80:0/3956596644 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f7c4014b750 con 0x7f7c40160fc0 2024-04-25T13:52:43.033 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.038+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7c4014b930 con 0x7f7c2406f400 2024-04-25T13:52:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.038+0000 7f7c35ffb640 1 -- 172.21.15.80:0/3956596644 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12586 (secure 0 0 0) 0x7f7c4014b930 con 0x7f7c2406f400 2024-04-25T13:52:43.035 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:43.035 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2406f400 msgr2=0x7f7c24071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2406f400 0x7f7c24071910 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f7c40072b20 tx=0x7f7c3c008040 comp rx=0 tx=0).stop 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c40160fc0 msgr2=0x7f7c4014ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c40160fc0 0x7f7c4014ae00 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000e4b0 tx=0x7f7c3000e970 comp rx=0 tx=0).stop 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 shutdown_connections 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f7c2406f400 0x7f7c24071910 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7c4014b340 0x7f7c401f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7c40160fc0 0x7f7c4014ae00 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 --2- 172.21.15.80:0/3956596644 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f7c2c004d50 0x7f7c40160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.037 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 >> 172.21.15.80:0/3956596644 conn(0x7f7c400cf390 msgr2=0x7f7c4006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 shutdown_connections 2024-04-25T13:52:43.038 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.042+0000 7f7c4755e640 1 -- 172.21.15.80:0/3956596644 wait complete. 2024-04-25T13:52:43.057 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":558,"stamp":"2024-04-25T13:52:42.374439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450347,"statfs":{"total":1600000000,"available":1485155902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485517,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-136,"num_write_kb":-85086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002252"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":446,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-04-25T13:52:37.036175+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:52:37.036175+0000","last_peered":"2024-04-25T13:52:37.036175+0000","last_clean":"2024-04-25T13:52:37.036175+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:52:37.036175+0000","last_undegraded":"2024-04-25T13:52:37.036175+0000","last_fullsized":"2024-04-25T13:52:37.036175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9884,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185427,"statfs":{"total":200000000,"available":189878024,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175819,"statfs":{"total":200000000,"available":180038867,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21541,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173770,"statfs":{"total":200000000,"available":177941335,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7259,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188052,"statfs":{"total":200000000,"available":192566216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9253,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186059,"statfs":{"total":200000000,"available":190524615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21541,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173771,"statfs":{"total":200000000,"available":177941849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13349,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181963,"statfs":{"total":200000000,"available":186330513,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9829,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185482,"statfs":{"total":200000000,"available":189934483,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:52:43.058 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T13:52:43.058 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:52:43.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.210+0000 7fac8d924640 1 Processor -- start 2024-04-25T13:52:43.207 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.210+0000 7fac8d924640 1 -- start start 2024-04-25T13:52:43.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac8d924640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac88161240 0x7fac880723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac8d924640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fac88161650 0x7fac880728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.209 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac8d924640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fac8806fcf0 con 0x7fac8806f5a0 2024-04-25T13:52:43.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac8d924640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fac88073f10 con 0x7fac88161650 2024-04-25T13:52:43.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac8d924640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fac880b0c60 con 0x7fac88161240 2024-04-25T13:52:43.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac86ffd640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fac8806f5a0 0x7fac88160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55366/0 (socket says 172.21.15.80:55366) 2024-04-25T13:52:43.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac86ffd640 1 -- 172.21.15.80:0/396412902 learned_addr learned my addr 172.21.15.80:0/396412902 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:43.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac877fe640 1 --2- 172.21.15.80:0/396412902 >> v2:172.21.15.87:3300/0 conn(0x7fac88161650 0x7fac880728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2519406547 0 0) 0x7fac8806fcf0 con 0x7fac8806f5a0 2024-04-25T13:52:43.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac867fc640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac88161240 0x7fac880723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.210 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fac70003530 con 0x7fac8806f5a0 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2974340876 0 0) 0x7fac70003530 con 0x7fac8806f5a0 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac88161240 msgr2=0x7fac880723a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac88161240 0x7fac880723a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 >> v2:172.21.15.87:3300/0 conn(0x7fac88161650 msgr2=0x7fac880728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 --2- 172.21.15.80:0/396412902 >> v2:172.21.15.87:3300/0 conn(0x7fac88161650 0x7fac880728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac880b0f40 con 0x7fac8806f5a0 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7fac78001680 con 0x7fac8806f5a0 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 >> v1:172.21.15.80:6789/0 conn(0x7fac8806f5a0 legacy=0x7fac88160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac700039c0 0x7fac70003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac70004810 0x7fac70004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac88161240 0x7fac70005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.212 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fac88073f10 con 0x7fac70004810 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fac880b0c60 con 0x7fac700039c0 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fac7000ba80 con 0x7fac88161240 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac867fc640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac70004810 0x7fac70004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac86ffd640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac700039c0 0x7fac70003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac877fe640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac88161240 0x7fac70005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac867fc640 1 -- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac88161240 msgr2=0x7fac70005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac867fc640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac88161240 0x7fac70005160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac867fc640 1 -- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac700039c0 msgr2=0x7fac70003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac867fc640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac700039c0 0x7fac70003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.213 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.214+0000 7fac867fc640 1 -- 172.21.15.80:0/396412902 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac70003530 con 0x7fac70004810 2024-04-25T13:52:43.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac867fc640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac70004810 0x7fac70004c20 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009ae0 tx=0x7fac7c01ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f9a6725258c3d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0096b0 con 0x7fac70004810 2024-04-25T13:52:43.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac7c0a6470 con 0x7fac70004810 2024-04-25T13:52:43.214 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac85ffb640 1 -- 172.21.15.80:0/396412902 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0a6860 con 0x7fac70004810 2024-04-25T13:52:43.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac8d924640 1 -- 172.21.15.80:0/396412902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac70004810 msgr2=0x7fac70004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac8d924640 1 --2- 172.21.15.80:0/396412902 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac70004810 0x7fac70004c20 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009ae0 tx=0x7fac7c01ca90 comp rx=0 tx=0).stop 2024-04-25T13:52:43.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac877fe640 1 -- 172.21.15.80:0/396412902 reap_dead start 2024-04-25T13:52:43.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac8d924640 1 -- 172.21.15.80:0/396412902 shutdown_connections 2024-04-25T13:52:43.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac8d924640 1 -- 172.21.15.80:0/396412902 >> 172.21.15.80:0/396412902 conn(0x7fac8806d080 msgr2=0x7fac880b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac8d924640 1 -- 172.21.15.80:0/396412902 shutdown_connections 2024-04-25T13:52:43.215 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.218+0000 7fac8d924640 1 -- 172.21.15.80:0/396412902 wait complete. 2024-04-25T13:52:43.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 Processor -- start 2024-04-25T13:52:43.217 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 -- start start 2024-04-25T13:52:43.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac70004810 0x7fac88157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8814ea80 0x7fac88157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac880cef70 0x7fac881583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fac88168250 con 0x7fac880cef70 2024-04-25T13:52:43.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fac881680d0 con 0x7fac70004810 2024-04-25T13:52:43.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fac881683d0 con 0x7fac8814ea80 2024-04-25T13:52:43.219 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac880cef70 0x7fac881583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac880cef70 0x7fac881583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36904/0 (socket says 172.21.15.80:36904) 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 -- 172.21.15.80:0/1535684771 learned_addr learned my addr 172.21.15.80:0/1535684771 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac86ffd640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac70004810 0x7fac88157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac867fc640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8814ea80 0x7fac88157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 -- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8814ea80 msgr2=0x7fac88157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8814ea80 0x7fac88157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 -- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac70004810 msgr2=0x7fac88157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac70004810 0x7fac88157510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 -- 172.21.15.80:0/1535684771 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac880cf6c0 con 0x7fac880cef70 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac867fc640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8814ea80 0x7fac88157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac877fe640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac880cef70 0x7fac881583b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fac80009fd0 tx=0x7fac8000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac6ffff640 1 -- 172.21.15.80:0/1535684771 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80016620 con 0x7fac880cef70 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac6ffff640 1 -- 172.21.15.80:0/1535684771 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fac8001b070 con 0x7fac880cef70 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac880cf9a0 con 0x7fac880cef70 2024-04-25T13:52:43.220 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac6ffff640 1 -- 172.21.15.80:0/1535684771 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac80016a00 con 0x7fac880cef70 2024-04-25T13:52:43.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac880cff60 con 0x7fac880cef70 2024-04-25T13:52:43.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.222+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac8806a3a0 con 0x7fac880cef70 2024-04-25T13:52:43.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.226+0000 7fac6ffff640 1 -- 172.21.15.80:0/1535684771 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fac80016ba0 con 0x7fac880cef70 2024-04-25T13:52:43.221 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.226+0000 7fac6ffff640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7406f430 0x7fac74071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.226+0000 7fac6ffff640 1 -- 172.21.15.80:0/1535684771 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fac8008fc90 con 0x7fac880cef70 2024-04-25T13:52:43.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.226+0000 7fac86ffd640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7406f430 0x7fac74071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.222 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.226+0000 7fac86ffd640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7406f430 0x7fac74071940 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fac78001210 tx=0x7fac78007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.224 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.230+0000 7fac6ffff640 1 -- 172.21.15.80:0/1535684771 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fac80025020 con 0x7fac880cef70 2024-04-25T13:52:43.381 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.386+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac880cf450 con 0x7fac7406f430 2024-04-25T13:52:43.383 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.386+0000 7fac6ffff640 1 -- 172.21.15.80:0/1535684771 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12586 (secure 0 0 0) 0x7fac880cf450 con 0x7fac7406f430 2024-04-25T13:52:43.383 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:43.384 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:52:43.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7406f430 msgr2=0x7fac74071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.385 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7406f430 0x7fac74071940 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fac78001210 tx=0x7fac78007040 comp rx=0 tx=0).stop 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac880cef70 msgr2=0x7fac881583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac880cef70 0x7fac881583b0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fac80009fd0 tx=0x7fac8000f520 comp rx=0 tx=0).stop 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 shutdown_connections 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fac7406f430 0x7fac74071940 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fac880cef70 0x7fac881583b0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fac8814ea80 0x7fac88157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 --2- 172.21.15.80:0/1535684771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fac70004810 0x7fac88157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 >> 172.21.15.80:0/1535684771 conn(0x7fac8806d080 msgr2=0x7fac8815c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 shutdown_connections 2024-04-25T13:52:43.386 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.390+0000 7fac8d924640 1 -- 172.21.15.80:0/1535684771 wait complete. 2024-04-25T13:52:43.406 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":558,"stamp":"2024-04-25T13:52:42.374439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":112152,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1450347,"statfs":{"total":1600000000,"available":1485155902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485517,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-136,"num_write_kb":-85086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002252"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":446,"reported_epoch":192,"state":"active+clean","last_fresh":"2024-04-25T13:52:37.036175+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:52:37.036175+0000","last_peered":"2024-04-25T13:52:37.036175+0000","last_clean":"2024-04-25T13:52:37.036175+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:52:37.036175+0000","last_undegraded":"2024-04-25T13:52:37.036175+0000","last_fullsized":"2024-04-25T13:52:37.036175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9884,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185427,"statfs":{"total":200000000,"available":189878024,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":19493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":175819,"statfs":{"total":200000000,"available":180038867,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21541,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173770,"statfs":{"total":200000000,"available":177941335,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":7259,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":188052,"statfs":{"total":200000000,"available":192566216,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9253,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":186059,"statfs":{"total":200000000,"available":190524615,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574998,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":21541,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":173771,"statfs":{"total":200000000,"available":177941849,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":13349,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":181963,"statfs":{"total":200000000,"available":186330513,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":9829,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":185482,"statfs":{"total":200000000,"available":189934483,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:52:43.406 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T13:52:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.554+0000 7f09ef41c640 1 Processor -- start 2024-04-25T13:52:43.551 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.554+0000 7f09ef41c640 1 -- start start 2024-04-25T13:52:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ef41c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e80723a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ef41c640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f09e8161650 0x7f09e80728e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ef41c640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09e806fcf0 con 0x7f09e806f5a0 2024-04-25T13:52:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ef41c640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f09e8073f10 con 0x7f09e8161650 2024-04-25T13:52:43.553 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ef41c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09e80b0c60 con 0x7f09e8161240 2024-04-25T13:52:43.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ed9b9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f09e8161650 0x7f09e80728e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ec9b7640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e80723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ed9b9640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f09e8161650 0x7f09e80728e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:41822/0 (socket says 172.21.15.80:41822) 2024-04-25T13:52:43.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ed1b8640 1 --1- >> v1:172.21.15.87:6789/0 conn(0x7f09e806f5a0 0x7f09e8160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:43250/0 (socket says 172.21.15.80:43250) 2024-04-25T13:52:43.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ed9b9640 1 -- 172.21.15.80:0/3392481898 learned_addr learned my addr 172.21.15.80:0/3392481898 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:43.554 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ec9b7640 1 -- 172.21.15.80:0/3392481898 >> v2:172.21.15.87:3300/0 conn(0x7f09e8161650 msgr2=0x7f09e80728e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:43.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ec9b7640 1 --2- 172.21.15.80:0/3392481898 >> v2:172.21.15.87:3300/0 conn(0x7f09e8161650 0x7f09e80728e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ec9b7640 1 -- 172.21.15.80:0/3392481898 >> v1:172.21.15.87:6789/0 conn(0x7f09e806f5a0 legacy=0x7f09e8160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.555 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.558+0000 7f09ec9b7640 1 -- 172.21.15.80:0/3392481898 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e80b0f40 con 0x7f09e8161240 2024-04-25T13:52:43.556 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ec9b7640 1 --2- 172.21.15.80:0/3392481898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e80723a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097c0 tx=0x7f09d801c410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe138e1d736403c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09dffff640 1 -- 172.21.15.80:0/3392481898 <== mon.2 v2:172.21.15.87:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80b0070 con 0x7f09e8161240 2024-04-25T13:52:43.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09dffff640 1 -- 172.21.15.80:0/3392481898 <== mon.2 v2:172.21.15.87:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09d80ab600 con 0x7f09e8161240 2024-04-25T13:52:43.557 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09dffff640 1 -- 172.21.15.80:0/3392481898 <== mon.2 v2:172.21.15.87:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d80ab9d0 con 0x7f09e8161240 2024-04-25T13:52:43.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ef41c640 1 -- 172.21.15.80:0/3392481898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 msgr2=0x7f09e80723a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ef41c640 1 --2- 172.21.15.80:0/3392481898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e80723a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f09d80097c0 tx=0x7f09d801c410 comp rx=0 tx=0).stop 2024-04-25T13:52:43.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ef41c640 1 -- 172.21.15.80:0/3392481898 shutdown_connections 2024-04-25T13:52:43.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ef41c640 1 --2- 172.21.15.80:0/3392481898 >> v2:172.21.15.87:3300/0 conn(0x7f09e8161650 0x7f09e80728e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ef41c640 1 --2- 172.21.15.80:0/3392481898 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e80723a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ef41c640 1 -- 172.21.15.80:0/3392481898 >> 172.21.15.80:0/3392481898 conn(0x7f09e806d080 msgr2=0x7f09e80b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ef41c640 1 -- 172.21.15.80:0/3392481898 shutdown_connections 2024-04-25T13:52:43.558 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.562+0000 7f09ef41c640 1 -- 172.21.15.80:0/3392481898 wait complete. 2024-04-25T13:52:43.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 Processor -- start 2024-04-25T13:52:43.560 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 -- start start 2024-04-25T13:52:43.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09e806f5a0 0x7f09e81fad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e814f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09e8161650 0x7f09e814ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f09e816a260 con 0x7f09e8161650 2024-04-25T13:52:43.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f09e816a0e0 con 0x7f09e806f5a0 2024-04-25T13:52:43.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f09e816a3e0 con 0x7f09e8161240 2024-04-25T13:52:43.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09e8161650 0x7f09e814ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.563 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09e8161650 0x7f09e814ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36908/0 (socket says 172.21.15.80:36908) 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 -- 172.21.15.80:0/97658787 learned_addr learned my addr 172.21.15.80:0/97658787 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ec9b7640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e814f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed1b8640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09e806f5a0 0x7f09e81fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 -- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 msgr2=0x7f09e814f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e814f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 -- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09e806f5a0 msgr2=0x7f09e81fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09e806f5a0 0x7f09e81fad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 -- 172.21.15.80:0/97658787 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e8153d40 con 0x7f09e8161650 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ec9b7640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed1b8640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09e806f5a0 0x7f09e81fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ed9b9640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09e8161650 0x7f09e814ac50 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f09e000ec10 tx=0x7f09e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ddffb640 1 -- 172.21.15.80:0/97658787 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e000cb20 con 0x7f09e8161650 2024-04-25T13:52:43.564 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ddffb640 1 -- 172.21.15.80:0/97658787 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09e000cce0 con 0x7f09e8161650 2024-04-25T13:52:43.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e8154020 con 0x7f09e8161650 2024-04-25T13:52:43.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ddffb640 1 -- 172.21.15.80:0/97658787 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09e0003be0 con 0x7f09e8161650 2024-04-25T13:52:43.565 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.566+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e81545e0 con 0x7f09e8161650 2024-04-25T13:52:43.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.570+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e806a3a0 con 0x7f09e8161650 2024-04-25T13:52:43.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.570+0000 7f09ddffb640 1 -- 172.21.15.80:0/97658787 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f09e0012050 con 0x7f09e8161650 2024-04-25T13:52:43.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.570+0000 7f09ddffb640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09cc06f430 0x7f09cc071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:43.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.570+0000 7f09ddffb640 1 -- 172.21.15.80:0/97658787 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f09e008f670 con 0x7f09e8161650 2024-04-25T13:52:43.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.570+0000 7f09ed1b8640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09cc06f430 0x7f09cc071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:43.566 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.570+0000 7f09ed1b8640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09cc06f430 0x7f09cc071940 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007fd0 tx=0x7f09e4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:43.569 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.574+0000 7f09ddffb640 1 -- 172.21.15.80:0/97658787 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f09e0022020 con 0x7f09e8161650 2024-04-25T13:52:43.725 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.730+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f09e8153ad0 con 0x7f09e8161650 2024-04-25T13:52:43.727 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.730+0000 7f09ddffb640 1 -- 172.21.15.80:0/97658787 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) ==== 74+0+12959 (secure 0 0 0) 0x7f09e0010040 con 0x7f09e8161650 2024-04-25T13:52:43.727 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:43.727 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":194,"fsid":"3a629b7a-ce3b-47b8-98ac-844ad072aa94","created":"2024-04-25T13:40:11.435899+0000","modified":"2024-04-25T13:52:40.046150+0000","last_up_change":"2024-04-25T13:40:17.485337+0000","last_in_change":"2024-04-25T13:40:14.377362+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T13:40:18.139319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6808","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6810","nonce":2104747008}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6812","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6814","nonce":2104747008}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6819","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6821","nonce":2104747008}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6816","nonce":2104747008},{"type":"v1","addr":"172.21.15.80:6817","nonce":2104747008}]},"public_addr":"172.21.15.80:6810/2104747008","cluster_addr":"172.21.15.80:6814/2104747008","heartbeat_back_addr":"172.21.15.80:6821/2104747008","heartbeat_front_addr":"172.21.15.80:6817/2104747008","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6800","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6801","nonce":1560454941}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6802","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6803","nonce":1560454941}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6806","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6807","nonce":1560454941}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6804","nonce":1560454941},{"type":"v1","addr":"172.21.15.80:6805","nonce":1560454941}]},"public_addr":"172.21.15.80:6801/1560454941","cluster_addr":"172.21.15.80:6803/1560454941","heartbeat_back_addr":"172.21.15.80:6807/1560454941","heartbeat_front_addr":"172.21.15.80:6805/1560454941","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6809","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6811","nonce":968203588}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6813","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6815","nonce":968203588}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6822","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6827","nonce":968203588}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6818","nonce":968203588},{"type":"v1","addr":"172.21.15.80:6820","nonce":968203588}]},"public_addr":"172.21.15.80:6811/968203588","cluster_addr":"172.21.15.80:6815/968203588","heartbeat_back_addr":"172.21.15.80:6827/968203588","heartbeat_front_addr":"172.21.15.80:6820/968203588","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6823","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6824","nonce":3781596558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6825","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6826","nonce":3781596558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6830","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6831","nonce":3781596558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6828","nonce":3781596558},{"type":"v1","addr":"172.21.15.80:6829","nonce":3781596558}]},"public_addr":"172.21.15.80:6824/3781596558","cluster_addr":"172.21.15.80:6826/3781596558","heartbeat_back_addr":"172.21.15.80:6831/3781596558","heartbeat_front_addr":"172.21.15.80:6829/3781596558","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6817","nonce":2235735313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6819","nonce":2235735313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6823","nonce":2235735313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":2235735313},{"type":"v1","addr":"172.21.15.87:6821","nonce":2235735313}]},"public_addr":"172.21.15.87:6817/2235735313","cluster_addr":"172.21.15.87:6819/2235735313","heartbeat_back_addr":"172.21.15.87:6823/2235735313","heartbeat_front_addr":"172.21.15.87:6821/2235735313","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6801","nonce":149615972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6803","nonce":149615972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6807","nonce":149615972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":149615972},{"type":"v1","addr":"172.21.15.87:6805","nonce":149615972}]},"public_addr":"172.21.15.87:6801/149615972","cluster_addr":"172.21.15.87:6803/149615972","heartbeat_back_addr":"172.21.15.87:6807/149615972","heartbeat_front_addr":"172.21.15.87:6805/149615972","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6825","nonce":1131074035}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6827","nonce":1131074035}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6831","nonce":1131074035}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":1131074035},{"type":"v1","addr":"172.21.15.87:6829","nonce":1131074035}]},"public_addr":"172.21.15.87:6825/1131074035","cluster_addr":"172.21.15.87:6827/1131074035","heartbeat_back_addr":"172.21.15.87:6831/1131074035","heartbeat_front_addr":"172.21.15.87:6829/1131074035","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":185,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6809","nonce":710369794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6811","nonce":710369794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6815","nonce":710369794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":710369794},{"type":"v1","addr":"172.21.15.87:6813","nonce":710369794}]},"public_addr":"172.21.15.87:6809/710369794","cluster_addr":"172.21.15.87:6811/710369794","heartbeat_back_addr":"172.21.15.87:6815/710369794","heartbeat_front_addr":"172.21.15.87:6813/710369794","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.507445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.510377+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.576727+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.545819+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.564232+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.472321+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.534056+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T13:40:15.538847+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.80:6835/3609922017":"2024-04-26T13:52:37.028258+0000","172.21.15.80:6834/3609922017":"2024-04-26T13:52:37.028258+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09cc06f430 msgr2=0x7f09cc071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09cc06f430 0x7f09cc071940 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f09e4007fd0 tx=0x7f09e4009040 comp rx=0 tx=0).stop 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09e8161650 msgr2=0x7f09e814ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09e8161650 0x7f09e814ac50 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f09e000ec10 tx=0x7f09e000c490 comp rx=0 tx=0).stop 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 shutdown_connections 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f09cc06f430 0x7f09cc071940 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f09e8161650 0x7f09e814ac50 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09e8161240 0x7f09e814f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 --2- 172.21.15.80:0/97658787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f09e806f5a0 0x7f09e81fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 >> 172.21.15.80:0/97658787 conn(0x7f09e806d080 msgr2=0x7f09e80b5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:43.730 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 shutdown_connections 2024-04-25T13:52:43.731 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:43.734+0000 7f09ef41c640 1 -- 172.21.15.80:0/97658787 wait complete. 2024-04-25T13:52:44.748 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T13:52:44.749 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T13:52:44.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ae0595640 1 Processor -- start 2024-04-25T13:52:44.909 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ae0595640 1 -- start start 2024-04-25T13:52:44.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ae0595640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ad80b0f10 0x7f0ad80d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ae0595640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad80b1320 con 0x7f0ad8161d10 2024-04-25T13:52:44.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ae0595640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0ad80d1150 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ae0595640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0ad80b5820 con 0x7f0ad80b0f10 2024-04-25T13:52:44.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ade331640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ad80b0f10 0x7f0ad80d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0adeb32640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0ad8161d10 0x7f0ad8162120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55370/0 (socket says 172.21.15.80:55370) 2024-04-25T13:52:44.911 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0adeb32640 1 -- 172.21.15.80:0/3153035067 learned_addr learned my addr 172.21.15.80:0/3153035067 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:44.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ade331640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0ad80b0f10 0x7f0ad80d0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36918/0 (socket says 172.21.15.80:36918) 2024-04-25T13:52:44.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0add32f640 1 -- 172.21.15.80:0/3153035067 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3637756420 0 0) 0x7f0ad00014a0 con 0x7f0ad8161d10 2024-04-25T13:52:44.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0add32f640 1 -- 172.21.15.80:0/3153035067 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0ac4003530 con 0x7f0ad8161d10 2024-04-25T13:52:44.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ade331640 1 -- 172.21.15.80:0/3153035067 >> v1:172.21.15.87:6789/0 conn(0x7f0ad80ce2a0 legacy=0x7f0ad80ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ade331640 1 -- 172.21.15.80:0/3153035067 >> v1:172.21.15.80:6789/0 conn(0x7f0ad8161d10 legacy=0x7f0ad8162120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.912 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.914+0000 7f0ade331640 1 -- 172.21.15.80:0/3153035067 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad80b4a60 con 0x7f0ad80b0f10 2024-04-25T13:52:44.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0ade331640 1 --2- 172.21.15.80:0/3153035067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80b0f10 0x7f0ad80d0c10 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40097c0 tx=0x7f0ad401c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56712620e3733c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0add32f640 1 -- 172.21.15.80:0/3153035067 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40b0070 con 0x7f0ad80b0f10 2024-04-25T13:52:44.913 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0add32f640 1 -- 172.21.15.80:0/3153035067 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ad40ab600 con 0x7f0ad80b0f10 2024-04-25T13:52:44.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0add32f640 1 -- 172.21.15.80:0/3153035067 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad40ab9d0 con 0x7f0ad80b0f10 2024-04-25T13:52:44.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0ae0595640 1 -- 172.21.15.80:0/3153035067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80b0f10 msgr2=0x7f0ad80d0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0ae0595640 1 --2- 172.21.15.80:0/3153035067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80b0f10 0x7f0ad80d0c10 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40097c0 tx=0x7f0ad401c4a0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0ae0595640 1 -- 172.21.15.80:0/3153035067 shutdown_connections 2024-04-25T13:52:44.914 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0ae0595640 1 --2- 172.21.15.80:0/3153035067 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80b0f10 0x7f0ad80d0c10 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0ae0595640 1 -- 172.21.15.80:0/3153035067 >> 172.21.15.80:0/3153035067 conn(0x7f0ad80b35c0 msgr2=0x7f0ad80b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0ae0595640 1 -- 172.21.15.80:0/3153035067 shutdown_connections 2024-04-25T13:52:44.915 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.918+0000 7f0ae0595640 1 -- 172.21.15.80:0/3153035067 wait complete. 2024-04-25T13:52:44.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 Processor -- start 2024-04-25T13:52:44.916 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 -- start start 2024-04-25T13:52:44.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ad80b0f10 0x7f0ad8155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80ce2a0 0x7f0ad8157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.919 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ad8161d10 0x7f0ad81583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad815e720 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad8162c40 con 0x7f0ad8161d10 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad816c8d0 con 0x7f0ad80b0f10 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80ce2a0 0x7f0ad8157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80ce2a0 0x7f0ad8157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36920/0 (socket says 172.21.15.80:36920) 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 -- 172.21.15.80:0/1343271021 learned_addr learned my addr 172.21.15.80:0/1343271021 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0adeb32640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ad8161d10 0x7f0ad81583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ade331640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ad80b0f10 0x7f0ad8155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 -- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ad80b0f10 msgr2=0x7f0ad8155c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0ad80b0f10 0x7f0ad8155c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 -- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ad8161d10 msgr2=0x7f0ad81583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.920 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ad8161d10 0x7f0ad81583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 -- 172.21.15.80:0/1343271021 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad81570d0 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0adeb32640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ad8161d10 0x7f0ad81583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0addb30640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80ce2a0 0x7f0ad8157e90 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800c370 tx=0x7f0ac800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac80052d0 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ac8005470 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad81573b0 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac8007260 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad814efd0 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.921 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.922+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0ad8156e80 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0ac8007400 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0aceffd640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0abc06f400 0x7f0abc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f0ac808ecf0 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0aceffd640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7f0abc075310 0x7f0abc077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 --> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ac8010e50 con 0x7f0abc075310 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f0ac808f180 con 0x7f0ad80ce2a0 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0ade331640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0abc06f400 0x7f0abc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0adeb32640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7f0abc075310 0x7f0abc077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0adeb32640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7f0abc075310 0x7f0abc077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0ade331640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0abc06f400 0x7f0abc071910 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40ad000 tx=0x7f0ad40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:44.923 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.926+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 <== osd.0 v2:172.21.15.80:6808/2104747008 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f0ac8010e50 con 0x7f0abc075310 2024-04-25T13:52:44.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 --> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0ad8158910 con 0x7f0abc075310 2024-04-25T13:52:44.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0aceffd640 1 -- 172.21.15.80:0/1343271021 <== osd.0 v2:172.21.15.80:6808/2104747008 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f0ad00072b0 con 0x7f0abc075310 2024-04-25T13:52:44.934 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:44.934 INFO:teuthology.orchestra.run.smithi080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-25T13:52:44.934 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:44.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7f0abc075310 msgr2=0x7f0abc077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.80:6808/2104747008,v1:172.21.15.80:6810/2104747008] conn(0x7f0abc075310 0x7f0abc077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:44.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0abc06f400 msgr2=0x7f0abc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0abc06f400 0x7f0abc071910 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f0ad40ad000 tx=0x7f0ad40a6000 comp rx=0 tx=0).stop 2024-04-25T13:52:44.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80ce2a0 msgr2=0x7f0ad8157e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:44.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 --2- 172.21.15.80:0/1343271021 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0ad80ce2a0 0x7f0ad8157e90 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f0ac800c370 tx=0x7f0ac800c830 comp rx=0 tx=0).stop 2024-04-25T13:52:44.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0adeb32640 1 -- 172.21.15.80:0/1343271021 reap_dead start 2024-04-25T13:52:44.935 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 shutdown_connections 2024-04-25T13:52:44.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 >> 172.21.15.80:0/1343271021 conn(0x7f0ad80b35c0 msgr2=0x7f0ad80757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:44.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 shutdown_connections 2024-04-25T13:52:44.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:44.938+0000 7f0ae0595640 1 -- 172.21.15.80:0/1343271021 wait complete. 2024-04-25T13:52:44.953 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T13:52:45.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a5d846640 1 Processor -- start 2024-04-25T13:52:45.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a5d846640 1 -- start start 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a5d846640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a58161f90 0x7f4a580cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a5d846640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4a5815ec40 0x7f4a5815f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a5d846640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a580d0d70 con 0x7f4a580d0510 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a5d846640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4a580d1050 con 0x7f4a5815ec40 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a5d846640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a5806dca0 con 0x7f4a58161f90 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a577fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f4a5815ec40 0x7f4a5815f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a567fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f4a580d0510 0x7f4a5815e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55374/0 (socket says 172.21.15.80:55374) 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a567fc640 1 -- 172.21.15.80:0/2744473479 learned_addr learned my addr 172.21.15.80:0/2744473479 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a56ffd640 1 --2- 172.21.15.80:0/2744473479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a58161f90 0x7f4a580cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a577fe640 1 -- 172.21.15.80:0/2744473479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a58161f90 msgr2=0x7f4a580cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a577fe640 1 --2- 172.21.15.80:0/2744473479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a58161f90 0x7f4a580cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.103 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a577fe640 1 -- 172.21.15.80:0/2744473479 >> v1:172.21.15.80:6789/0 conn(0x7f4a580d0510 legacy=0x7f4a5815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a55ffb640 1 -- 172.21.15.80:0/2744473479 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 123998588 0 0) 0x7f4a580d0d70 con 0x7f4a580d0510 2024-04-25T13:52:45.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.106+0000 7f4a577fe640 1 -- 172.21.15.80:0/2744473479 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5806df80 con 0x7f4a5815ec40 2024-04-25T13:52:45.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a577fe640 1 --2- 172.21.15.80:0/2744473479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a5815ec40 0x7f4a5815f050 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f4a500097c0 tx=0x7f4a5001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c79f2374233e069 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a55ffb640 1 -- 172.21.15.80:0/2744473479 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500b0070 con 0x7f4a5815ec40 2024-04-25T13:52:45.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a55ffb640 1 -- 172.21.15.80:0/2744473479 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a500ab600 con 0x7f4a5815ec40 2024-04-25T13:52:45.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a55ffb640 1 -- 172.21.15.80:0/2744473479 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a500ab9d0 con 0x7f4a5815ec40 2024-04-25T13:52:45.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a5d846640 1 -- 172.21.15.80:0/2744473479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a5815ec40 msgr2=0x7f4a5815f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a5d846640 1 --2- 172.21.15.80:0/2744473479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a5815ec40 0x7f4a5815f050 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f4a500097c0 tx=0x7f4a5001c3d0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a5d846640 1 -- 172.21.15.80:0/2744473479 shutdown_connections 2024-04-25T13:52:45.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a5d846640 1 --2- 172.21.15.80:0/2744473479 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a5815ec40 0x7f4a5815f050 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a5d846640 1 --2- 172.21.15.80:0/2744473479 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a58161f90 0x7f4a580cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a5d846640 1 -- 172.21.15.80:0/2744473479 >> 172.21.15.80:0/2744473479 conn(0x7f4a5806d080 msgr2=0x7f4a580b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a5d846640 1 -- 172.21.15.80:0/2744473479 shutdown_connections 2024-04-25T13:52:45.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.110+0000 7f4a5d846640 1 -- 172.21.15.80:0/2744473479 wait complete. 2024-04-25T13:52:45.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 Processor -- start 2024-04-25T13:52:45.108 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 -- start start 2024-04-25T13:52:45.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a580d0510 0x7f4a5813ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a5815ec40 0x7f4a5813a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a58161f90 0x7f4a581f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5806f630 con 0x7f4a580d0510 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4a5806f4b0 con 0x7f4a58161f90 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4a580d0920 con 0x7f4a5815ec40 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a580d0510 0x7f4a5813ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a580d0510 0x7f4a5813ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36930/0 (socket says 172.21.15.80:36930) 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 -- 172.21.15.80:0/775653045 learned_addr learned my addr 172.21.15.80:0/775653045 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a577fe640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a58161f90 0x7f4a581f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a567fc640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a5815ec40 0x7f4a5813a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.111 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 -- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a5815ec40 msgr2=0x7f4a5813a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a5815ec40 0x7f4a5813a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 -- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a58161f90 msgr2=0x7f4a581f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a58161f90 0x7f4a581f76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 -- 172.21.15.80:0/775653045 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581460f0 con 0x7f4a580d0510 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a56ffd640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a580d0510 0x7f4a5813ae70 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f4a48007e90 tx=0x7f4a4800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a3ffff640 1 -- 172.21.15.80:0/775653045 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48014420 con 0x7f4a580d0510 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a3ffff640 1 -- 172.21.15.80:0/775653045 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a48019070 con 0x7f4a580d0510 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a581463d0 con 0x7f4a580d0510 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a3ffff640 1 -- 172.21.15.80:0/775653045 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a48014820 con 0x7f4a580d0510 2024-04-25T13:52:45.112 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a58146990 con 0x7f4a580d0510 2024-04-25T13:52:45.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.114+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a5806a3a0 con 0x7f4a580d0510 2024-04-25T13:52:45.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.118+0000 7f4a3ffff640 1 -- 172.21.15.80:0/775653045 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f4a480149c0 con 0x7f4a580d0510 2024-04-25T13:52:45.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.118+0000 7f4a3ffff640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.118+0000 7f4a3ffff640 1 -- 172.21.15.80:0/775653045 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f4a4808f590 con 0x7f4a580d0510 2024-04-25T13:52:45.113 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.118+0000 7f4a567fc640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.114 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.118+0000 7f4a567fc640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000f70 tx=0x7f4a4c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.117 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.122+0000 7f4a3ffff640 1 -- 172.21.15.80:0/775653045 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f4a48023020 con 0x7f4a580d0510 2024-04-25T13:52:45.264 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f4a58143cc0 con 0x7f4a4406f400 2024-04-25T13:52:45.265 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a3ffff640 1 -- 172.21.15.80:0/775653045 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4a58143cc0 con 0x7f4a4406f400 2024-04-25T13:52:45.265 INFO:teuthology.orchestra.run.smithi080.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T13:52:45.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 msgr2=0x7f4a44071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c000f70 tx=0x7f4a4c009040 comp rx=0 tx=0).stop 2024-04-25T13:52:45.267 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a580d0510 msgr2=0x7f4a5813ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a580d0510 0x7f4a5813ae70 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f4a48007e90 tx=0x7f4a4800e4b0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 shutdown_connections 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f4a4406f400 0x7f4a44071910 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4a58161f90 0x7f4a581f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4a5815ec40 0x7f4a5813a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 --2- 172.21.15.80:0/775653045 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f4a580d0510 0x7f4a5813ae70 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 >> 172.21.15.80:0/775653045 conn(0x7f4a5806d080 msgr2=0x7f4a580b63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 shutdown_connections 2024-04-25T13:52:45.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.270+0000 7f4a5d846640 1 -- 172.21.15.80:0/775653045 wait complete. 2024-04-25T13:52:45.288 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T13:52:45.289 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T13:52:45.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f463d966640 1 Processor -- start 2024-04-25T13:52:45.424 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f463d966640 1 -- start start 2024-04-25T13:52:45.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f463d966640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f46380b09a0 0x7f46381602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.426 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f463d966640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4638160820 0x7f46380d0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f463d966640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f46380b4a60 con 0x7f463806fb30 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f463d966640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f4638161270 con 0x7f46380b09a0 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f463d966640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f46380b0db0 con 0x7f4638160820 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f46367fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f46380b09a0 0x7f46381602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f46367fc640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f46380b09a0 0x7f46381602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36938/0 (socket says 172.21.15.80:36938) 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f46367fc640 1 -- 172.21.15.80:0/1660184585 learned_addr learned my addr 172.21.15.80:0/1660184585 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f46377fe640 1 --2- 172.21.15.80:0/1660184585 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4638160820 0x7f46380d0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f46367fc640 1 -- 172.21.15.80:0/1660184585 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4638160820 msgr2=0x7f46380d0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f46367fc640 1 --2- 172.21.15.80:0/1660184585 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4638160820 0x7f46380d0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f46367fc640 1 -- 172.21.15.80:0/1660184585 >> v1:172.21.15.87:6789/0 conn(0x7f463806fb30 legacy=0x7f46380b6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.427 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.430+0000 7f46367fc640 1 -- 172.21.15.80:0/1660184585 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4638160f90 con 0x7f46380b09a0 2024-04-25T13:52:45.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f46367fc640 1 --2- 172.21.15.80:0/1660184585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 0x7f46381602e0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f462c009870 tx=0x7f462c01c560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abef9d07d0a139de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.428 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f4635ffb640 1 -- 172.21.15.80:0/1660184585 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0ab430 con 0x7f46380b09a0 2024-04-25T13:52:45.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f4635ffb640 1 -- 172.21.15.80:0/1660184585 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462c0b0070 con 0x7f46380b09a0 2024-04-25T13:52:45.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f4635ffb640 1 -- 172.21.15.80:0/1660184585 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0ab7f0 con 0x7f46380b09a0 2024-04-25T13:52:45.429 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 -- 172.21.15.80:0/1660184585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 msgr2=0x7f46381602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 --2- 172.21.15.80:0/1660184585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 0x7f46381602e0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f462c009870 tx=0x7f462c01c560 comp rx=0 tx=0).stop 2024-04-25T13:52:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 -- 172.21.15.80:0/1660184585 shutdown_connections 2024-04-25T13:52:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 --2- 172.21.15.80:0/1660184585 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f4638160820 0x7f46380d0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 --2- 172.21.15.80:0/1660184585 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 0x7f46381602e0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 -- 172.21.15.80:0/1660184585 >> 172.21.15.80:0/1660184585 conn(0x7f46380b35c0 msgr2=0x7f46380b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 -- 172.21.15.80:0/1660184585 shutdown_connections 2024-04-25T13:52:45.430 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 -- 172.21.15.80:0/1660184585 wait complete. 2024-04-25T13:52:45.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 Processor -- start 2024-04-25T13:52:45.431 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.434+0000 7f463d966640 1 -- start start 2024-04-25T13:52:45.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f463806fb30 0x7f4638150760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 0x7f4638151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4638160820 0x7f46381520b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f46381697e0 con 0x7f46380b09a0 2024-04-25T13:52:45.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f4638169660 con 0x7f4638160820 2024-04-25T13:52:45.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f4638169960 con 0x7f463806fb30 2024-04-25T13:52:45.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 0x7f4638151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.434 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 0x7f4638151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36940/0 (socket says 172.21.15.80:36940) 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 -- 172.21.15.80:0/2067844807 learned_addr learned my addr 172.21.15.80:0/2067844807 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46377fe640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4638160820 0x7f46381520b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f4636ffd640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f463806fb30 0x7f4638150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 -- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f463806fb30 msgr2=0x7f4638150760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f463806fb30 0x7f4638150760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 -- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4638160820 msgr2=0x7f46381520b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4638160820 0x7f46381520b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 -- 172.21.15.80:0/2067844807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f463815daa0 con 0x7f46380b09a0 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f4636ffd640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f463806fb30 0x7f4638150760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f46367fc640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 0x7f4638151b70 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f462c0ad040 tx=0x7f462c019300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0b4420 con 0x7f46380b09a0 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f462c0b45e0 con 0x7f46380b09a0 2024-04-25T13:52:45.435 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f462c0b47d0 con 0x7f46380b09a0 2024-04-25T13:52:45.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46380d2000 con 0x7f46380b09a0 2024-04-25T13:52:45.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f463815e000 con 0x7f46380b09a0 2024-04-25T13:52:45.436 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f463806a440 con 0x7f46380b09a0 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.438+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f462c0bd020 con 0x7f46380b09a0 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f461ffff640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f462406f400 0x7f4624071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f462c129130 con 0x7f46380b09a0 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f461ffff640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7f4624075310 0x7f4624077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 --> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f462c0b7e50 con 0x7f4624075310 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f462c1295c0 con 0x7f46380b09a0 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f4636ffd640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f462406f400 0x7f4624071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f46377fe640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7f4624075310 0x7f4624077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.437 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f46377fe640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7f4624075310 0x7f4624077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f4636ffd640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f462406f400 0x7f4624071910 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4628007fd0 tx=0x7f4628009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.438 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.442+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 <== osd.1 v2:172.21.15.80:6800/1560454941 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f462c0b7e50 con 0x7f4624075310 2024-04-25T13:52:45.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 --> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4638158c80 con 0x7f4624075310 2024-04-25T13:52:45.448 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f461ffff640 1 -- 172.21.15.80:0/2067844807 <== osd.1 v2:172.21.15.80:6800/1560454941 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f4638158c80 con 0x7f4624075310 2024-04-25T13:52:45.448 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:45.449 INFO:teuthology.orchestra.run.smithi080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-25T13:52:45.449 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7f4624075310 msgr2=0x7f4624077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.80:6800/1560454941,v1:172.21.15.80:6801/1560454941] conn(0x7f4624075310 0x7f4624077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f462406f400 msgr2=0x7f4624071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f462406f400 0x7f4624071910 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4628007fd0 tx=0x7f4628009040 comp rx=0 tx=0).stop 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 msgr2=0x7f4638151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 --2- 172.21.15.80:0/2067844807 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f46380b09a0 0x7f4638151b70 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f462c0ad040 tx=0x7f462c019300 comp rx=0 tx=0).stop 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f46377fe640 1 -- 172.21.15.80:0/2067844807 reap_dead start 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 shutdown_connections 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 >> 172.21.15.80:0/2067844807 conn(0x7f46380b35c0 msgr2=0x7f4638156290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 shutdown_connections 2024-04-25T13:52:45.450 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.454+0000 7f463d966640 1 -- 172.21.15.80:0/2067844807 wait complete. 2024-04-25T13:52:45.466 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T13:52:45.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63bae3640 1 Processor -- start 2024-04-25T13:52:45.609 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63bae3640 1 -- start start 2024-04-25T13:52:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63bae3640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fb6340d06e0 0x7fb63406e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63bae3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb63406e9e0 0x7fb63406edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63bae3640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb6340d0af0 con 0x7fb634161f90 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63bae3640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fb6340d0d70 con 0x7fb6340d06e0 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63bae3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb6340d1050 con 0x7fb63406e9e0 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63987f640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fb634161f90 0x7fb6340cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55378/0 (socket says 172.21.15.80:55378) 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63987f640 1 -- 172.21.15.80:0/3953172489 learned_addr learned my addr 172.21.15.80:0/3953172489 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63907e640 1 --2- 172.21.15.80:0/3953172489 >> v2:172.21.15.80:3300/0 conn(0x7fb6340d06e0 0x7fb63406e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63a080640 1 --2- 172.21.15.80:0/3953172489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb63406e9e0 0x7fb63406edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63907e640 1 -- 172.21.15.80:0/3953172489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb63406e9e0 msgr2=0x7fb63406edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63907e640 1 --2- 172.21.15.80:0/3953172489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb63406e9e0 0x7fb63406edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63887d640 1 -- 172.21.15.80:0/3953172489 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1502500948 0 0) 0x7fb630001120 con 0x7fb634161f90 2024-04-25T13:52:45.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63907e640 1 -- 172.21.15.80:0/3953172489 >> v1:172.21.15.80:6789/0 conn(0x7fb634161f90 legacy=0x7fb6340cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63907e640 1 -- 172.21.15.80:0/3953172489 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb63406dca0 con 0x7fb6340d06e0 2024-04-25T13:52:45.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.614+0000 7fb63a080640 1 --2- 172.21.15.80:0/3953172489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb63406e9e0 0x7fb63406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:45.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63907e640 1 --2- 172.21.15.80:0/3953172489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6340d06e0 0x7fb63406e4a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fb6240097c0 tx=0x7fb62401c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1626a1222bda8f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63887d640 1 -- 172.21.15.80:0/3953172489 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240b0070 con 0x7fb6340d06e0 2024-04-25T13:52:45.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63887d640 1 -- 172.21.15.80:0/3953172489 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6240ab600 con 0x7fb6340d06e0 2024-04-25T13:52:45.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63887d640 1 -- 172.21.15.80:0/3953172489 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6240ab9d0 con 0x7fb6340d06e0 2024-04-25T13:52:45.614 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63bae3640 1 -- 172.21.15.80:0/3953172489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6340d06e0 msgr2=0x7fb63406e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63bae3640 1 --2- 172.21.15.80:0/3953172489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6340d06e0 0x7fb63406e4a0 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fb6240097c0 tx=0x7fb62401c3e0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63bae3640 1 -- 172.21.15.80:0/3953172489 shutdown_connections 2024-04-25T13:52:45.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63bae3640 1 --2- 172.21.15.80:0/3953172489 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb63406e9e0 0x7fb63406edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63bae3640 1 --2- 172.21.15.80:0/3953172489 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb6340d06e0 0x7fb63406e4a0 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63bae3640 1 -- 172.21.15.80:0/3953172489 >> 172.21.15.80:0/3953172489 conn(0x7fb63406d080 msgr2=0x7fb6340b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63bae3640 1 -- 172.21.15.80:0/3953172489 shutdown_connections 2024-04-25T13:52:45.615 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.618+0000 7fb63bae3640 1 -- 172.21.15.80:0/3953172489 wait complete. 2024-04-25T13:52:45.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 Processor -- start 2024-04-25T13:52:45.616 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 -- start start 2024-04-25T13:52:45.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63406e9e0 0x7fb634165310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.618 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6340d06e0 0x7fb634167070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb634161f90 0x7fb6341675b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fb6341447a0 con 0x7fb63406e9e0 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fb634144620 con 0x7fb634161f90 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fb63416b510 con 0x7fb6340d06e0 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63406e9e0 0x7fb634165310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63406e9e0 0x7fb634165310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36966/0 (socket says 172.21.15.80:36966) 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 -- 172.21.15.80:0/4148354095 learned_addr learned my addr 172.21.15.80:0/4148354095 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63a080640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb634161f90 0x7fb6341675b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63907e640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6340d06e0 0x7fb634167070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 -- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6340d06e0 msgr2=0x7fb634167070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6340d06e0 0x7fb634167070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 -- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb634161f90 msgr2=0x7fb6341675b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb634161f90 0x7fb6341675b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.619 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 -- 172.21.15.80:0/4148354095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb634145500 con 0x7fb63406e9e0 2024-04-25T13:52:45.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63987f640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63406e9e0 0x7fb634165310 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c3e0 tx=0x7fb63000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb62a7fc640 1 -- 172.21.15.80:0/4148354095 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6300074e0 con 0x7fb63406e9e0 2024-04-25T13:52:45.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb62a7fc640 1 -- 172.21.15.80:0/4148354095 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6300047f0 con 0x7fb63406e9e0 2024-04-25T13:52:45.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb634147980 con 0x7fb63406e9e0 2024-04-25T13:52:45.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb62a7fc640 1 -- 172.21.15.80:0/4148354095 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb630004be0 con 0x7fb63406e9e0 2024-04-25T13:52:45.620 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb634145a80 con 0x7fb63406e9e0 2024-04-25T13:52:45.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.622+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb63406a3a0 con 0x7fb63406e9e0 2024-04-25T13:52:45.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.626+0000 7fb62a7fc640 1 -- 172.21.15.80:0/4148354095 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fb630011070 con 0x7fb63406e9e0 2024-04-25T13:52:45.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.626+0000 7fb62a7fc640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb61806f400 0x7fb618071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.626+0000 7fb62a7fc640 1 -- 172.21.15.80:0/4148354095 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fb63008f0e0 con 0x7fb63406e9e0 2024-04-25T13:52:45.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.626+0000 7fb63907e640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb61806f400 0x7fb618071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.621 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.626+0000 7fb63907e640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb61806f400 0x7fb618071910 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb6240ad000 tx=0x7fb6240a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.624 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.630+0000 7fb62a7fc640 1 -- 172.21.15.80:0/4148354095 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fb6340d0af0 con 0x7fb63406e9e0 2024-04-25T13:52:45.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.782+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fb63406e5d0 con 0x7fb61806f400 2024-04-25T13:52:45.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb62a7fc640 1 -- 172.21.15.80:0/4148354095 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fb63406e5d0 con 0x7fb61806f400 2024-04-25T13:52:45.780 INFO:teuthology.orchestra.run.smithi080.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T13:52:45.783 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb61806f400 msgr2=0x7fb618071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb61806f400 0x7fb618071910 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb6240ad000 tx=0x7fb6240a6000 comp rx=0 tx=0).stop 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63406e9e0 msgr2=0x7fb634165310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63406e9e0 0x7fb634165310 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fb63000c3e0 tx=0x7fb63000c8a0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 shutdown_connections 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fb61806f400 0x7fb618071910 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb634161f90 0x7fb6341675b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fb6340d06e0 0x7fb634167070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 --2- 172.21.15.80:0/4148354095 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fb63406e9e0 0x7fb634165310 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 >> 172.21.15.80:0/4148354095 conn(0x7fb63406d080 msgr2=0x7fb6340d1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.786+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 shutdown_connections 2024-04-25T13:52:45.784 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.790+0000 7fb63bae3640 1 -- 172.21.15.80:0/4148354095 wait complete. 2024-04-25T13:52:45.802 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T13:52:45.802 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T13:52:45.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.954+0000 7f1f5e158640 1 Processor -- start 2024-04-25T13:52:45.950 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.954+0000 7f1f5e158640 1 -- start start 2024-04-25T13:52:45.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.954+0000 7f1f5e158640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1f580d24a0 0x7f1f580d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f5e158640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f58164290 con 0x7f1f580d4820 2024-04-25T13:52:45.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f5e158640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1f58164570 con 0x7f1f580d0ed0 2024-04-25T13:52:45.953 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f5e158640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1f58164040 con 0x7f1f580d24a0 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f57fff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f1f580d4820 0x7f1f580d4c30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55386/0 (socket says 172.21.15.80:55386) 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f57fff640 1 -- 172.21.15.80:0/3349508107 learned_addr learned my addr 172.21.15.80:0/3349508107 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f4effd640 1 --2- 172.21.15.80:0/3349508107 >> v2:172.21.15.87:3300/0 conn(0x7f1f580d24a0 0x7f1f580d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3696150436 0 0) 0x7f1f58164290 con 0x7f1f580d4820 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1f44003530 con 0x7f1f580d4820 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2864568245 0 0) 0x7f1f44003530 con 0x7f1f580d4820 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 >> v2:172.21.15.87:3300/0 conn(0x7f1f580d24a0 msgr2=0x7f1f580d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 --2- 172.21.15.80:0/3349508107 >> v2:172.21.15.87:3300/0 conn(0x7f1f580d24a0 0x7f1f580d28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 >> v1:172.21.15.87:6789/0 conn(0x7f1f580d0ed0 legacy=0x7f1f580d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f58164850 con 0x7f1f580d4820 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f1f50001a00 con 0x7f1f580d4820 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 >> v1:172.21.15.80:6789/0 conn(0x7f1f580d4820 legacy=0x7f1f580d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f440039c0 0x7f1f44003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.954 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f44004810 0x7f1f44004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f440051d0 0x7f1f4400b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58164040 con 0x7f1f440051d0 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1f4400c170 con 0x7f1f44004810 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1f4400c370 con 0x7f1f440039c0 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f57fff640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f440051d0 0x7f1f4400b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f4effd640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f44004810 0x7f1f44004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f577fe640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f440039c0 0x7f1f44003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f57fff640 1 -- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f440039c0 msgr2=0x7f1f44003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f57fff640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f440039c0 0x7f1f44003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f57fff640 1 -- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f44004810 msgr2=0x7f1f44004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f57fff640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f44004810 0x7f1f44004c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.955 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f57fff640 1 -- 172.21.15.80:0/3349508107 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f44003530 con 0x7f1f440051d0 2024-04-25T13:52:45.956 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.958+0000 7f1f4effd640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f44004810 0x7f1f44004c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:45.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f57fff640 1 -- 172.21.15.80:0/3349508107 reap_dead start 2024-04-25T13:52:45.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f57fff640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f440051d0 0x7f1f4400b9d0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000a810 tx=0x7f1f5001d280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a79d88d2130aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5000a460 con 0x7f1f440051d0 2024-04-25T13:52:45.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f500ac640 con 0x7f1f440051d0 2024-04-25T13:52:45.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f56ffd640 1 -- 172.21.15.80:0/3349508107 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500a5e60 con 0x7f1f440051d0 2024-04-25T13:52:45.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 -- 172.21.15.80:0/3349508107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f440051d0 msgr2=0x7f1f4400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f440051d0 0x7f1f4400b9d0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000a810 tx=0x7f1f5001d280 comp rx=0 tx=0).stop 2024-04-25T13:52:45.957 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 -- 172.21.15.80:0/3349508107 shutdown_connections 2024-04-25T13:52:45.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 --2- 172.21.15.80:0/3349508107 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f440051d0 0x7f1f4400b9d0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 -- 172.21.15.80:0/3349508107 >> 172.21.15.80:0/3349508107 conn(0x7f1f5806c5f0 msgr2=0x7f1f5806ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 -- 172.21.15.80:0/3349508107 shutdown_connections 2024-04-25T13:52:45.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 -- 172.21.15.80:0/3349508107 wait complete. 2024-04-25T13:52:45.958 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 Processor -- start 2024-04-25T13:52:45.959 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.962+0000 7f1f5e158640 1 -- start start 2024-04-25T13:52:45.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f440051d0 0x7f1f580cd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f58156960 0x7f1f58159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f5815dc60 0x7f1f58159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1f580d12e0 con 0x7f1f5815dc60 2024-04-25T13:52:45.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1f58164bc0 con 0x7f1f58156960 2024-04-25T13:52:45.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1f580d1460 con 0x7f1f440051d0 2024-04-25T13:52:45.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f5815dc60 0x7f1f58159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.964 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f5815dc60 0x7f1f58159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36982/0 (socket says 172.21.15.80:36982) 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 -- 172.21.15.80:0/2780831724 learned_addr learned my addr 172.21.15.80:0/2780831724 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f577fe640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f440051d0 0x7f1f580cd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f4effd640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f58156960 0x7f1f58159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 -- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f440051d0 msgr2=0x7f1f580cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1f440051d0 0x7f1f580cd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 -- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f58156960 msgr2=0x7f1f58159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f58156960 0x7f1f58159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 -- 172.21.15.80:0/2780831724 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f58159dd0 con 0x7f1f5815dc60 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f57fff640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f5815dc60 0x7f1f58159740 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1f50000f90 tx=0x7f1f44005840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500019c0 con 0x7f1f5815dc60 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1f500a5e60 con 0x7f1f5815dc60 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f5815a0b0 con 0x7f1f5815dc60 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f500ac9f0 con 0x7f1f5815dc60 2024-04-25T13:52:45.965 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f5815a670 con 0x7f1f5815dc60 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1f58157cd0 con 0x7f1f5815dc60 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1f500070c0 con 0x7f1f5815dc60 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f54ff9640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1f3806f400 0x7f1f38071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.966+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f1f50129250 con 0x7f1f5815dc60 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.970+0000 7f1f54ff9640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f1f38075310 0x7f1f38077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.970+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 --> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1f500ac640 con 0x7f1f38075310 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.970+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f1f501296e0 con 0x7f1f5815dc60 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.970+0000 7f1f577fe640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1f3806f400 0x7f1f38071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.970+0000 7f1f4effd640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f1f38075310 0x7f1f38077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:45.966 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.970+0000 7f1f4effd640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f1f38075310 0x7f1f38077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.970+0000 7f1f577fe640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1f3806f400 0x7f1f38071910 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1f48000c10 tx=0x7f1f48009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:45.967 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.970+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 <== osd.2 v2:172.21.15.80:6809/968203588 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f1f500ac640 con 0x7f1f38075310 2024-04-25T13:52:45.975 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 --> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f1f58156ef0 con 0x7f1f38075310 2024-04-25T13:52:45.976 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f54ff9640 1 -- 172.21.15.80:0/2780831724 <== osd.2 v2:172.21.15.80:6809/968203588 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f1f58156ef0 con 0x7f1f38075310 2024-04-25T13:52:45.976 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:45.976 INFO:teuthology.orchestra.run.smithi080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-25T13:52:45.976 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:45.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f1f38075310 msgr2=0x7f1f38077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.977 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.80:6809/968203588,v1:172.21.15.80:6811/968203588] conn(0x7f1f38075310 0x7f1f38077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1f3806f400 msgr2=0x7f1f38071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f1f3806f400 0x7f1f38071910 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1f48000c10 tx=0x7f1f48009040 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f5815dc60 msgr2=0x7f1f58159740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 --2- 172.21.15.80:0/2780831724 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1f5815dc60 0x7f1f58159740 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f1f50000f90 tx=0x7f1f44005840 comp rx=0 tx=0).stop 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f57fff640 1 -- 172.21.15.80:0/2780831724 reap_dead start 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 shutdown_connections 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 >> 172.21.15.80:0/2780831724 conn(0x7f1f5806c5f0 msgr2=0x7f1f58151c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 shutdown_connections 2024-04-25T13:52:45.978 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:45.982+0000 7f1f5e158640 1 -- 172.21.15.80:0/2780831724 wait complete. 2024-04-25T13:52:45.995 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.146+0000 7f072d1b0640 1 Processor -- start 2024-04-25T13:52:46.142 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.146+0000 7f072d1b0640 1 -- start start 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.146+0000 7f072d1b0640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f07280d06e0 0x7f072806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.146+0000 7f072d1b0640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07280d0af0 con 0x7f072806e9e0 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.146+0000 7f072d1b0640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07280d0d70 con 0x7f0728161f90 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.146+0000 7f072d1b0640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f07280d1050 con 0x7f07280d06e0 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f071ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f07280d06e0 0x7f072806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f071ffff640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f07280d06e0 0x7f072806e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36984/0 (socket says 172.21.15.80:36984) 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f071ffff640 1 -- 172.21.15.80:0/3131828693 learned_addr learned my addr 172.21.15.80:0/3131828693 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072759e640 1 --1- 172.21.15.80:0/3131828693 >> v1:172.21.15.80:6789/0 conn(0x7f072806e9e0 0x7f072806edf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55388/0 (socket says 172.21.15.80:55388) 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f071ffff640 1 -- 172.21.15.80:0/3131828693 >> v1:172.21.15.87:6789/0 conn(0x7f0728161f90 legacy=0x7f07280cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f071ffff640 1 -- 172.21.15.80:0/3131828693 >> v1:172.21.15.80:6789/0 conn(0x7f072806e9e0 legacy=0x7f072806edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.144 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f071ffff640 1 -- 172.21.15.80:0/3131828693 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072806dca0 con 0x7f07280d06e0 2024-04-25T13:52:46.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f071ffff640 1 --2- 172.21.15.80:0/3131828693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07280d06e0 0x7f072806e4a0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f0710009870 tx=0x7f071001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ab49d9004daa2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072659c640 1 -- 172.21.15.80:0/3131828693 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100b1070 con 0x7f07280d06e0 2024-04-25T13:52:46.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072659c640 1 -- 172.21.15.80:0/3131828693 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f07100ac630 con 0x7f07280d06e0 2024-04-25T13:52:46.146 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072659c640 1 -- 172.21.15.80:0/3131828693 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07100a8050 con 0x7f07280d06e0 2024-04-25T13:52:46.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072d1b0640 1 -- 172.21.15.80:0/3131828693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07280d06e0 msgr2=0x7f072806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072d1b0640 1 --2- 172.21.15.80:0/3131828693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07280d06e0 0x7f072806e4a0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f0710009870 tx=0x7f071001c410 comp rx=0 tx=0).stop 2024-04-25T13:52:46.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072d1b0640 1 -- 172.21.15.80:0/3131828693 shutdown_connections 2024-04-25T13:52:46.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072d1b0640 1 --2- 172.21.15.80:0/3131828693 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f07280d06e0 0x7f072806e4a0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072d1b0640 1 -- 172.21.15.80:0/3131828693 >> 172.21.15.80:0/3131828693 conn(0x7f072806d080 msgr2=0x7f07280b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072d1b0640 1 -- 172.21.15.80:0/3131828693 shutdown_connections 2024-04-25T13:52:46.147 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.150+0000 7f072d1b0640 1 -- 172.21.15.80:0/3131828693 wait complete. 2024-04-25T13:52:46.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 Processor -- start 2024-04-25T13:52:46.149 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 -- start start 2024-04-25T13:52:46.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f072806e9e0 0x7f07281414d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07280d06e0 0x7f072815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0728161f90 0x7f0728139aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f072806f680 con 0x7f0728161f90 2024-04-25T13:52:46.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f072806f500 con 0x7f07280d06e0 2024-04-25T13:52:46.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0728144620 con 0x7f072806e9e0 2024-04-25T13:52:46.152 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0728161f90 0x7f0728139aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0728161f90 0x7f0728139aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36994/0 (socket says 172.21.15.80:36994) 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 -- 172.21.15.80:0/471314513 learned_addr learned my addr 172.21.15.80:0/471314513 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f071ffff640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07280d06e0 0x7f072815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f0726d9d640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f072806e9e0 0x7f07281414d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 -- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f072806e9e0 msgr2=0x7f07281414d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f072806e9e0 0x7f07281414d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 -- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07280d06e0 msgr2=0x7f072815b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07280d06e0 0x7f072815b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 -- 172.21.15.80:0/471314513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072813ffc0 con 0x7f0728161f90 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072759e640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0728161f90 0x7f0728139aa0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f072000c6f0 tx=0x7f072000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f071f7fe640 1 -- 172.21.15.80:0/471314513 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0720017070 con 0x7f0728161f90 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f071f7fe640 1 -- 172.21.15.80:0/471314513 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0720005db0 con 0x7f0728161f90 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0728140430 con 0x7f0728161f90 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f071f7fe640 1 -- 172.21.15.80:0/471314513 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072000a840 con 0x7f0728161f90 2024-04-25T13:52:46.153 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.154+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07281409f0 con 0x7f0728161f90 2024-04-25T13:52:46.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.158+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0728140190 con 0x7f0728161f90 2024-04-25T13:52:46.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.158+0000 7f071f7fe640 1 -- 172.21.15.80:0/471314513 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f072001a070 con 0x7f0728161f90 2024-04-25T13:52:46.154 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.158+0000 7f071f7fe640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f070806f400 0x7f0708071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.158+0000 7f071f7fe640 1 -- 172.21.15.80:0/471314513 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f072008e4f0 con 0x7f0728161f90 2024-04-25T13:52:46.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.158+0000 7f0726d9d640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f070806f400 0x7f0708071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.158+0000 7f0726d9d640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f070806f400 0x7f0708071910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f072806fa20 tx=0x7f0718009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.158 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.162+0000 7f071f7fe640 1 -- 172.21.15.80:0/471314513 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f0728140190 con 0x7f0728161f90 2024-04-25T13:52:46.314 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.318+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f072806a3a0 con 0x7f070806f400 2024-04-25T13:52:46.315 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.318+0000 7f071f7fe640 1 -- 172.21.15.80:0/471314513 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f072806a3a0 con 0x7f070806f400 2024-04-25T13:52:46.315 INFO:teuthology.orchestra.run.smithi080.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T13:52:46.317 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f070806f400 msgr2=0x7f0708071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f070806f400 0x7f0708071910 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f072806fa20 tx=0x7f0718009040 comp rx=0 tx=0).stop 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0728161f90 msgr2=0x7f0728139aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0728161f90 0x7f0728139aa0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f072000c6f0 tx=0x7f072000cbb0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 shutdown_connections 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f070806f400 0x7f0708071910 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0728161f90 0x7f0728139aa0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f07280d06e0 0x7f072815b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 --2- 172.21.15.80:0/471314513 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f072806e9e0 0x7f07281414d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 >> 172.21.15.80:0/471314513 conn(0x7f072806d080 msgr2=0x7f07280757a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 shutdown_connections 2024-04-25T13:52:46.318 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.322+0000 7f072d1b0640 1 -- 172.21.15.80:0/471314513 wait complete. 2024-04-25T13:52:46.335 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T13:52:46.336 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T13:52:46.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.474+0000 7f7521576640 1 Processor -- start 2024-04-25T13:52:46.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.474+0000 7f7521576640 1 -- start start 2024-04-25T13:52:46.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f7521576640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f751c0d24a0 0x7f751c0d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f7521576640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c0d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f7521576640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f751c164290 con 0x7f751c0d0ed0 2024-04-25T13:52:46.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f7521576640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f751c164040 con 0x7f751c0d24a0 2024-04-25T13:52:46.472 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f7521576640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f751c164bc0 con 0x7f751c0d4820 2024-04-25T13:52:46.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751b7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c0d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.473 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751a7fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f751c0d24a0 0x7f751c0d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751b7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c0d4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:36088/0 (socket says 172.21.15.80:36088) 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751a7fc640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f751c0d24a0 0x7f751c0d28b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:41888/0 (socket says 172.21.15.80:41888) 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751b7fe640 1 -- 172.21.15.80:0/302117571 learned_addr learned my addr 172.21.15.80:0/302117571 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751affd640 1 --1- 172.21.15.80:0/302117571 >> v1:172.21.15.87:6789/0 conn(0x7f751c0d0ed0 0x7f751c0d12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.87:6789/0 says I am v1:172.21.15.80:43292/0 (socket says 172.21.15.80:43292) 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751a7fc640 1 -- 172.21.15.80:0/302117571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 msgr2=0x7f751c0d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751a7fc640 1 --2- 172.21.15.80:0/302117571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c0d4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751a7fc640 1 -- 172.21.15.80:0/302117571 >> v1:172.21.15.87:6789/0 conn(0x7f751c0d0ed0 legacy=0x7f751c0d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751a7fc640 1 -- 172.21.15.80:0/302117571 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c164570 con 0x7f751c0d24a0 2024-04-25T13:52:46.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751b7fe640 1 --2- 172.21.15.80:0/302117571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c0d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:46.475 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.478+0000 7f751a7fc640 1 --2- 172.21.15.80:0/302117571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f751c0d24a0 0x7f751c0d28b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75100097c0 tx=0x7f751001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e10cb3748d88509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7519ffb640 1 -- 172.21.15.80:0/302117571 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75100aa5f0 con 0x7f751c0d24a0 2024-04-25T13:52:46.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7519ffb640 1 -- 172.21.15.80:0/302117571 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75100af070 con 0x7f751c0d24a0 2024-04-25T13:52:46.476 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7519ffb640 1 -- 172.21.15.80:0/302117571 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75100aa9b0 con 0x7f751c0d24a0 2024-04-25T13:52:46.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 -- 172.21.15.80:0/302117571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f751c0d24a0 msgr2=0x7f751c0d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 --2- 172.21.15.80:0/302117571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f751c0d24a0 0x7f751c0d28b0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f75100097c0 tx=0x7f751001c470 comp rx=0 tx=0).stop 2024-04-25T13:52:46.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 -- 172.21.15.80:0/302117571 shutdown_connections 2024-04-25T13:52:46.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 --2- 172.21.15.80:0/302117571 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c0d4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 --2- 172.21.15.80:0/302117571 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f751c0d24a0 0x7f751c0d28b0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 -- 172.21.15.80:0/302117571 >> 172.21.15.80:0/302117571 conn(0x7f751c06c5f0 msgr2=0x7f751c06ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 -- 172.21.15.80:0/302117571 shutdown_connections 2024-04-25T13:52:46.477 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 -- 172.21.15.80:0/302117571 wait complete. 2024-04-25T13:52:46.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 Processor -- start 2024-04-25T13:52:46.478 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.482+0000 7f7521576640 1 -- start start 2024-04-25T13:52:46.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f751c0d0ed0 0x7f751c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f751c0d24a0 0x7f751c14bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f751c16d3e0 con 0x7f751c0d0ed0 2024-04-25T13:52:46.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f751c0d3040 con 0x7f751c0d24a0 2024-04-25T13:52:46.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f751c16d560 con 0x7f751c0d4820 2024-04-25T13:52:46.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f751c0d0ed0 0x7f751c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.484 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f751c0d0ed0 0x7f751c0c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:36998/0 (socket says 172.21.15.80:36998) 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 -- 172.21.15.80:0/4290159765 learned_addr learned my addr 172.21.15.80:0/4290159765 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751a7fc640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f751c0d24a0 0x7f751c14bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751b7fe640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 -- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 msgr2=0x7f751c14c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f751c0d4820 0x7f751c14c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 -- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f751c0d24a0 msgr2=0x7f751c14bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f751c0d24a0 0x7f751c14bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 -- 172.21.15.80:0/4290159765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c14d130 con 0x7f751c0d0ed0 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751affd640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f751c0d0ed0 0x7f751c0c8810 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f750c00c370 tx=0x7f750c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c0074e0 con 0x7f751c0d0ed0 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f750c007680 con 0x7f751c0d0ed0 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c15eff0 con 0x7f751c0d0ed0 2024-04-25T13:52:46.485 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c004a30 con 0x7f751c0d0ed0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c159950 con 0x7f751c0d0ed0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f751c06a440 con 0x7f751c0d0ed0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f750c01a020 con 0x7f751c0d0ed0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f750806f400 0x7f7508071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f750c08f190 con 0x7f751c0d0ed0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7f7508075310 0x7f7508077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 --> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7508077e30 con 0x7f7508075310 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f750c08f620 con 0x7f751c0d0ed0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751b7fe640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7f7508075310 0x7f7508077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.486+0000 7f751a7fc640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f750806f400 0x7f7508071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.490+0000 7f751b7fe640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7f7508075310 0x7f7508077760 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.490+0000 7f751a7fc640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f750806f400 0x7f7508071910 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f75100ae000 tx=0x7f75100a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.486 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.490+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 <== osd.3 v2:172.21.15.80:6823/3781596558 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f7508077e30 con 0x7f7508075310 2024-04-25T13:52:46.494 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.498+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 --> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f751c1584f0 con 0x7f7508075310 2024-04-25T13:52:46.495 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.498+0000 7f75037fe640 1 -- 172.21.15.80:0/4290159765 <== osd.3 v2:172.21.15.80:6823/3781596558 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f751c1584f0 con 0x7f7508075310 2024-04-25T13:52:46.495 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:46.495 INFO:teuthology.orchestra.run.smithi080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-25T13:52:46.495 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:46.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7f7508075310 msgr2=0x7f7508077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.80:6823/3781596558,v1:172.21.15.80:6824/3781596558] conn(0x7f7508075310 0x7f7508077760 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.496 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f750806f400 msgr2=0x7f7508071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f750806f400 0x7f7508071910 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f75100ae000 tx=0x7f75100a7000 comp rx=0 tx=0).stop 2024-04-25T13:52:46.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f751c0d0ed0 msgr2=0x7f751c0c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 --2- 172.21.15.80:0/4290159765 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f751c0d0ed0 0x7f751c0c8810 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f750c00c370 tx=0x7f750c00c830 comp rx=0 tx=0).stop 2024-04-25T13:52:46.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f751b7fe640 1 -- 172.21.15.80:0/4290159765 reap_dead start 2024-04-25T13:52:46.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 shutdown_connections 2024-04-25T13:52:46.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 >> 172.21.15.80:0/4290159765 conn(0x7f751c06c5f0 msgr2=0x7f751c0bada0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 shutdown_connections 2024-04-25T13:52:46.497 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.502+0000 7f7521576640 1 -- 172.21.15.80:0/4290159765 wait complete. 2024-04-25T13:52:46.514 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T13:52:46.648 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b9bca640 1 Processor -- start 2024-04-25T13:52:46.649 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b9bca640 1 -- start start 2024-04-25T13:52:46.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b9bca640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe6b4073f00 0x7fe6b4074310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b9bca640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b4157ce0 0x7fe6b41580f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b9bca640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe6b40cd5f0 con 0x7fe6b415e960 2024-04-25T13:52:46.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b9bca640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fe6b406e9f0 con 0x7fe6b4073f00 2024-04-25T13:52:46.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b9bca640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b415eeb0 con 0x7fe6b4157ce0 2024-04-25T13:52:46.651 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b2ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe6b4073f00 0x7fe6b4074310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b2ffd640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fe6b4073f00 0x7fe6b4074310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37002/0 (socket says 172.21.15.80:37002) 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b37fe640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7fe6b415e960 0x7fe6b40ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55398/0 (socket says 172.21.15.80:55398) 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b2ffd640 1 -- 172.21.15.80:0/2718260946 learned_addr learned my addr 172.21.15.80:0/2718260946 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b3fff640 1 --2- 172.21.15.80:0/2718260946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b4157ce0 0x7fe6b41580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b2ffd640 1 -- 172.21.15.80:0/2718260946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b4157ce0 msgr2=0x7fe6b41580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b2ffd640 1 --2- 172.21.15.80:0/2718260946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b4157ce0 0x7fe6b41580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b2ffd640 1 -- 172.21.15.80:0/2718260946 >> v1:172.21.15.80:6789/0 conn(0x7fe6b415e960 legacy=0x7fe6b40ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b2ffd640 1 -- 172.21.15.80:0/2718260946 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b415f190 con 0x7fe6b4073f00 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b27fc640 1 -- 172.21.15.80:0/2718260946 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2134862868 0 0) 0x7fe6a4001120 con 0x7fe6b415e960 2024-04-25T13:52:46.652 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.654+0000 7fe6b3fff640 1 --2- 172.21.15.80:0/2718260946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b4157ce0 0x7fe6b41580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:46.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b2ffd640 1 --2- 172.21.15.80:0/2718260946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4073f00 0x7fe6b4074310 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe6b406eb70 tx=0x7fe6a801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139ccb5f6764fde0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b27fc640 1 -- 172.21.15.80:0/2718260946 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80ab620 con 0x7fe6b4073f00 2024-04-25T13:52:46.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b27fc640 1 -- 172.21.15.80:0/2718260946 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a80b0070 con 0x7fe6b4073f00 2024-04-25T13:52:46.653 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b27fc640 1 -- 172.21.15.80:0/2718260946 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80ab9e0 con 0x7fe6b4073f00 2024-04-25T13:52:46.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2718260946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4073f00 msgr2=0x7fe6b4074310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2718260946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4073f00 0x7fe6b4074310 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7fe6b406eb70 tx=0x7fe6a801c510 comp rx=0 tx=0).stop 2024-04-25T13:52:46.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2718260946 shutdown_connections 2024-04-25T13:52:46.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2718260946 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b4157ce0 0x7fe6b41580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.654 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2718260946 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4073f00 0x7fe6b4074310 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2718260946 >> 172.21.15.80:0/2718260946 conn(0x7fe6b40d20d0 msgr2=0x7fe6b40d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2718260946 shutdown_connections 2024-04-25T13:52:46.655 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.658+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2718260946 wait complete. 2024-04-25T13:52:46.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 Processor -- start 2024-04-25T13:52:46.656 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 -- start start 2024-04-25T13:52:46.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6b4073f00 0x7fe6b40c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.659 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4157ce0 0x7fe6b40c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b415e960 0x7fe6b40c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b406f810 con 0x7fe6b4157ce0 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b406f690 con 0x7fe6b4073f00 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b406f990 con 0x7fe6b415e960 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4157ce0 0x7fe6b40c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4157ce0 0x7fe6b40c76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37012/0 (socket says 172.21.15.80:37012) 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 -- 172.21.15.80:0/2864810425 learned_addr learned my addr 172.21.15.80:0/2864810425 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b37fe640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6b4073f00 0x7fe6b40c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b3fff640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b415e960 0x7fe6b40c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 -- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b415e960 msgr2=0x7fe6b40c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b415e960 0x7fe6b40c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 -- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6b4073f00 msgr2=0x7fe6b40c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6b4073f00 0x7fe6b40c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.660 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 -- 172.21.15.80:0/2864810425 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b40ce6e0 con 0x7fe6b4157ce0 2024-04-25T13:52:46.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b2ffd640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4157ce0 0x7fe6b40c76b0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80ad040 tx=0x7fe6a801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe69bfff640 1 -- 172.21.15.80:0/2864810425 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a80ba070 con 0x7fe6b4157ce0 2024-04-25T13:52:46.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe69bfff640 1 -- 172.21.15.80:0/2864810425 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe6a80abbe0 con 0x7fe6b4157ce0 2024-04-25T13:52:46.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe69bfff640 1 -- 172.21.15.80:0/2864810425 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a8004020 con 0x7fe6b4157ce0 2024-04-25T13:52:46.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b4151080 con 0x7fe6b4157ce0 2024-04-25T13:52:46.661 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.662+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b4151320 con 0x7fe6b4157ce0 2024-04-25T13:52:46.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.666+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b406a3a0 con 0x7fe6b4157ce0 2024-04-25T13:52:46.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.666+0000 7fe69bfff640 1 -- 172.21.15.80:0/2864810425 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fe6a80041c0 con 0x7fe6b4157ce0 2024-04-25T13:52:46.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.666+0000 7fe69bfff640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe6a006f430 0x7fe6a0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.666+0000 7fe69bfff640 1 -- 172.21.15.80:0/2864810425 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fe6a8129150 con 0x7fe6b4157ce0 2024-04-25T13:52:46.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.666+0000 7fe6b37fe640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe6a006f430 0x7fe6a0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.662 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.666+0000 7fe6b37fe640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe6a006f430 0x7fe6a0071940 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4000f70 tx=0x7fe6a4007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.665 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.670+0000 7fe69bfff640 1 -- 172.21.15.80:0/2864810425 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fe6a8016130 con 0x7fe6b4157ce0 2024-04-25T13:52:46.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.822+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe6b40c8250 con 0x7fe6a006f430 2024-04-25T13:52:46.817 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.822+0000 7fe69bfff640 1 -- 172.21.15.80:0/2864810425 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe6b40c8250 con 0x7fe6a006f430 2024-04-25T13:52:46.817 INFO:teuthology.orchestra.run.smithi080.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T13:52:46.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe6a006f430 msgr2=0x7fe6a0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe6a006f430 0x7fe6a0071940 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4000f70 tx=0x7fe6a4007c10 comp rx=0 tx=0).stop 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4157ce0 msgr2=0x7fe6b40c76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4157ce0 0x7fe6b40c76b0 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7fe6a80ad040 tx=0x7fe6a801ad50 comp rx=0 tx=0).stop 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 shutdown_connections 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fe6a006f430 0x7fe6a0071940 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fe6b415e960 0x7fe6b40c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fe6b4157ce0 0x7fe6b40c76b0 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 --2- 172.21.15.80:0/2864810425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe6b4073f00 0x7fe6b40c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 >> 172.21.15.80:0/2864810425 conn(0x7fe6b40d20d0 msgr2=0x7fe6b40d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 shutdown_connections 2024-04-25T13:52:46.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.826+0000 7fe6b9bca640 1 -- 172.21.15.80:0/2864810425 wait complete. 2024-04-25T13:52:46.839 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T13:52:46.840 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T13:52:46.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.994+0000 7f0fcf852640 1 Processor -- start 2024-04-25T13:52:46.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.994+0000 7f0fcf852640 1 -- start start 2024-04-25T13:52:46.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fcf852640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0fc80b76e0 0x7f0fc80b7af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:46.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fcf852640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc80b8030 con 0x7f0fc80b92b0 2024-04-25T13:52:46.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fcf852640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0fc8164040 con 0x7f0fc8164b00 2024-04-25T13:52:46.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fcf852640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f0fc80d1900 con 0x7f0fc80b76e0 2024-04-25T13:52:46.993 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fccded640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f0fc80b76e0 0x7f0fc80b7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:46.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fcddef640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f0fc80b92b0 0x7f0fc80b96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55410/0 (socket says 172.21.15.80:55410) 2024-04-25T13:52:46.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fcddef640 1 -- 172.21.15.80:0/2160667715 learned_addr learned my addr 172.21.15.80:0/2160667715 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:46.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fccded640 1 -- 172.21.15.80:0/2160667715 >> v1:172.21.15.87:6789/0 conn(0x7f0fc8164b00 legacy=0x7f0fc80d0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fccded640 1 -- 172.21.15.80:0/2160667715 >> v1:172.21.15.80:6789/0 conn(0x7f0fc80b92b0 legacy=0x7f0fc80b96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fccded640 1 -- 172.21.15.80:0/2160667715 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc80d1620 con 0x7f0fc80b76e0 2024-04-25T13:52:46.994 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fbbfff640 1 -- 172.21.15.80:0/2160667715 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1773886292 0 0) 0x7f0fc00014a0 con 0x7f0fc80b92b0 2024-04-25T13:52:46.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fccded640 1 --2- 172.21.15.80:0/2160667715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc80b76e0 0x7f0fc80b7af0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0097c0 tx=0x7f0fbc01c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fbc95122cccf7a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:46.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fbbfff640 1 -- 172.21.15.80:0/2160667715 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0b0070 con 0x7f0fc80b76e0 2024-04-25T13:52:46.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fbbfff640 1 -- 172.21.15.80:0/2160667715 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fbc0ab600 con 0x7f0fc80b76e0 2024-04-25T13:52:46.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:46.998+0000 7f0fbbfff640 1 -- 172.21.15.80:0/2160667715 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0ab9d0 con 0x7f0fc80b76e0 2024-04-25T13:52:46.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- 172.21.15.80:0/2160667715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc80b76e0 msgr2=0x7f0fc80b7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:46.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 --2- 172.21.15.80:0/2160667715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc80b76e0 0x7f0fc80b7af0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc0097c0 tx=0x7f0fbc01c4a0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- 172.21.15.80:0/2160667715 shutdown_connections 2024-04-25T13:52:46.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 --2- 172.21.15.80:0/2160667715 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc80b76e0 0x7f0fc80b7af0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:46.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- 172.21.15.80:0/2160667715 >> 172.21.15.80:0/2160667715 conn(0x7f0fc8073620 msgr2=0x7f0fc80d26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:46.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- 172.21.15.80:0/2160667715 shutdown_connections 2024-04-25T13:52:46.997 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- 172.21.15.80:0/2160667715 wait complete. 2024-04-25T13:52:46.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 Processor -- start 2024-04-25T13:52:46.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- start start 2024-04-25T13:52:47.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc80b76e0 0x7f0fc8157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc80b92b0 0x7f0fc8152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc8164b00 0x7f0fc81554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc816ea80 con 0x7f0fc8164b00 2024-04-25T13:52:47.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc80d55b0 con 0x7f0fc80b92b0 2024-04-25T13:52:47.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcf852640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0fc816ec00 con 0x7f0fc80b76e0 2024-04-25T13:52:47.001 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.002+0000 7f0fcddef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc8164b00 0x7f0fc81554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcddef640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc8164b00 0x7f0fc81554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37032/0 (socket says 172.21.15.80:37032) 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcddef640 1 -- 172.21.15.80:0/891955832 learned_addr learned my addr 172.21.15.80:0/891955832 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fccded640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc80b92b0 0x7f0fc8152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcd5ee640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc80b76e0 0x7f0fc8157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcddef640 1 -- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc80b76e0 msgr2=0x7f0fc8157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcddef640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc80b76e0 0x7f0fc8157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcddef640 1 -- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc80b92b0 msgr2=0x7f0fc8152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcddef640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc80b92b0 0x7f0fc8152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcddef640 1 -- 172.21.15.80:0/891955832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc80ca8f0 con 0x7f0fc8164b00 2024-04-25T13:52:47.002 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fccded640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0fc80b92b0 0x7f0fc8152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcd5ee640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f0fc80b76e0 0x7f0fc8157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcddef640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc8164b00 0x7f0fc81554a0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c700 tx=0x7f0fc000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc0018070 con 0x7f0fc8164b00 2024-04-25T13:52:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fc000ce10 con 0x7f0fc8164b00 2024-04-25T13:52:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc81588c0 con 0x7f0fc8164b00 2024-04-25T13:52:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fc000a880 con 0x7f0fc8164b00 2024-04-25T13:52:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc80cae90 con 0x7f0fc8164b00 2024-04-25T13:52:47.003 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0fc806a440 con 0x7f0fc8164b00 2024-04-25T13:52:47.004 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f0fc001b070 con 0x7f0fc8164b00 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0fac06f400 0x7f0fac071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f0fc008eac0 con 0x7f0fc8164b00 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f0fac075310 0x7f0fac077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 --> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0fac077e30 con 0x7f0fac075310 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f0fc008ef50 con 0x7f0fc8164b00 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcd5ee640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0fac06f400 0x7f0fac071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fccded640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f0fac075310 0x7f0fac077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.006+0000 7f0fcd5ee640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0fac06f400 0x7f0fac071910 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4000f70 tx=0x7f0fc4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.010+0000 7f0fccded640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f0fac075310 0x7f0fac077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.005 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.010+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 <== osd.4 v2:172.21.15.87:6816/2235735313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f0fac077e30 con 0x7f0fac075310 2024-04-25T13:52:47.016 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 --> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0fc8158670 con 0x7f0fac075310 2024-04-25T13:52:47.017 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fb9ffb640 1 -- 172.21.15.80:0/891955832 <== osd.4 v2:172.21.15.87:6816/2235735313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f0fc8158670 con 0x7f0fac075310 2024-04-25T13:52:47.017 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:47.017 INFO:teuthology.orchestra.run.smithi080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-25T13:52:47.018 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:47.018 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f0fac075310 msgr2=0x7f0fac077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6816/2235735313,v1:172.21.15.87:6817/2235735313] conn(0x7f0fac075310 0x7f0fac077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0fac06f400 msgr2=0x7f0fac071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f0fac06f400 0x7f0fac071910 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4000f70 tx=0x7f0fc4009040 comp rx=0 tx=0).stop 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc8164b00 msgr2=0x7f0fc81554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 --2- 172.21.15.80:0/891955832 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f0fc8164b00 0x7f0fc81554a0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f0fc000c700 tx=0x7f0fc000cbc0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcddef640 1 -- 172.21.15.80:0/891955832 reap_dead start 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 shutdown_connections 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 >> 172.21.15.80:0/891955832 conn(0x7f0fc8073620 msgr2=0x7f0fc80d0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 shutdown_connections 2024-04-25T13:52:47.019 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.022+0000 7f0fcf852640 1 -- 172.21.15.80:0/891955832 wait complete. 2024-04-25T13:52:47.036 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T13:52:47.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f147927c640 1 Processor -- start 2024-04-25T13:52:47.176 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f147927c640 1 -- start start 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f147927c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 0x7f14740cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f147927c640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f147415ec40 0x7f147415f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f147927c640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f14740d0d70 con 0x7f14740d0510 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f147927c640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f14740d1050 con 0x7f147415ec40 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f147927c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f147406dca0 con 0x7f1474161f90 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f14737fe640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f147415ec40 0x7f147415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f14727fc640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f14740d0510 0x7f147415e530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55424/0 (socket says 172.21.15.80:55424) 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f14727fc640 1 -- 172.21.15.80:0/3910170072 learned_addr learned my addr 172.21.15.80:0/3910170072 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f1472ffd640 1 --2- 172.21.15.80:0/3910170072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 0x7f14740cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.179 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f14737fe640 1 -- 172.21.15.80:0/3910170072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 msgr2=0x7f14740cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f14737fe640 1 --2- 172.21.15.80:0/3910170072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 0x7f14740cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f14737fe640 1 -- 172.21.15.80:0/3910170072 >> v1:172.21.15.80:6789/0 conn(0x7f14740d0510 legacy=0x7f147415e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f1471ffb640 1 -- 172.21.15.80:0/3910170072 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2334087360 0 0) 0x7f14740d0d70 con 0x7f14740d0510 2024-04-25T13:52:47.180 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.182+0000 7f14737fe640 1 -- 172.21.15.80:0/3910170072 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f147406df80 con 0x7f147415ec40 2024-04-25T13:52:47.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f14737fe640 1 --2- 172.21.15.80:0/3910170072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415f050 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f14640097c0 tx=0x7f146401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f934bf0c09c202 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f1471ffb640 1 -- 172.21.15.80:0/3910170072 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640b0070 con 0x7f147415ec40 2024-04-25T13:52:47.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f1471ffb640 1 -- 172.21.15.80:0/3910170072 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f14640ab600 con 0x7f147415ec40 2024-04-25T13:52:47.181 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f1471ffb640 1 -- 172.21.15.80:0/3910170072 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14640ab9d0 con 0x7f147415ec40 2024-04-25T13:52:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f147927c640 1 -- 172.21.15.80:0/3910170072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 msgr2=0x7f147415f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f147927c640 1 --2- 172.21.15.80:0/3910170072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415f050 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f14640097c0 tx=0x7f146401c3d0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f147927c640 1 -- 172.21.15.80:0/3910170072 shutdown_connections 2024-04-25T13:52:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f147927c640 1 --2- 172.21.15.80:0/3910170072 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415f050 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f147927c640 1 --2- 172.21.15.80:0/3910170072 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 0x7f14740cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f147927c640 1 -- 172.21.15.80:0/3910170072 >> 172.21.15.80:0/3910170072 conn(0x7f147406d080 msgr2=0x7f14740b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f147927c640 1 -- 172.21.15.80:0/3910170072 shutdown_connections 2024-04-25T13:52:47.182 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.186+0000 7f147927c640 1 -- 172.21.15.80:0/3910170072 wait complete. 2024-04-25T13:52:47.183 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 Processor -- start 2024-04-25T13:52:47.184 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 -- start start 2024-04-25T13:52:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0510 0x7f1474166480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 0x7f1474166fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f147406f630 con 0x7f147415ec40 2024-04-25T13:52:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f147406f4b0 con 0x7f14740d0510 2024-04-25T13:52:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f14740d0920 con 0x7f1474161f90 2024-04-25T13:52:47.186 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37064/0 (socket says 172.21.15.80:37064) 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 -- 172.21.15.80:0/2150170415 learned_addr learned my addr 172.21.15.80:0/2150170415 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f1472ffd640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0510 0x7f1474166480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14737fe640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 0x7f1474166fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 -- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 msgr2=0x7f1474166fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 0x7f1474166fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 -- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0510 msgr2=0x7f1474166480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0510 0x7f1474166480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 -- 172.21.15.80:0/2150170415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14741653e0 con 0x7f147415ec40 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f1472ffd640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0510 0x7f1474166480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f14727fc640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415cec0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f146800c410 tx=0x7f1468001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f145b7fe640 1 -- 172.21.15.80:0/2150170415 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14680074e0 con 0x7f147415ec40 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f145b7fe640 1 -- 172.21.15.80:0/2150170415 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1468007680 con 0x7f147415ec40 2024-04-25T13:52:47.187 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f145b7fe640 1 -- 172.21.15.80:0/2150170415 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004a10 con 0x7f147415ec40 2024-04-25T13:52:47.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14741656c0 con 0x7f147415ec40 2024-04-25T13:52:47.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1474165c80 con 0x7f147415ec40 2024-04-25T13:52:47.188 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.190+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f147406a3a0 con 0x7f147415ec40 2024-04-25T13:52:47.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.194+0000 7f145b7fe640 1 -- 172.21.15.80:0/2150170415 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1468011070 con 0x7f147415ec40 2024-04-25T13:52:47.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.194+0000 7f145b7fe640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f400 0x7f1460071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.194+0000 7f145b7fe640 1 -- 172.21.15.80:0/2150170415 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f146808f170 con 0x7f147415ec40 2024-04-25T13:52:47.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.194+0000 7f1472ffd640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f400 0x7f1460071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.189 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.194+0000 7f1472ffd640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f400 0x7f1460071910 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f147406fa20 tx=0x7f146c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.192 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.198+0000 7f145b7fe640 1 -- 172.21.15.80:0/2150170415 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f146805c440 con 0x7f147415ec40 2024-04-25T13:52:47.349 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.354+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f1474165170 con 0x7f146006f400 2024-04-25T13:52:47.350 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.354+0000 7f145b7fe640 1 -- 172.21.15.80:0/2150170415 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1474165170 con 0x7f146006f400 2024-04-25T13:52:47.350 INFO:teuthology.orchestra.run.smithi080.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T13:52:47.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f400 msgr2=0x7f1460071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f400 0x7f1460071910 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f147406fa20 tx=0x7f146c00b040 comp rx=0 tx=0).stop 2024-04-25T13:52:47.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 msgr2=0x7f147415cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415cec0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7f146800c410 tx=0x7f1468001e80 comp rx=0 tx=0).stop 2024-04-25T13:52:47.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 shutdown_connections 2024-04-25T13:52:47.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f146006f400 0x7f1460071910 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1474161f90 0x7f1474166fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.352 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f147415ec40 0x7f147415cec0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 --2- 172.21.15.80:0/2150170415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f14740d0510 0x7f1474166480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 >> 172.21.15.80:0/2150170415 conn(0x7f147406d080 msgr2=0x7f14740b6840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:47.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 shutdown_connections 2024-04-25T13:52:47.353 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.358+0000 7f147927c640 1 -- 172.21.15.80:0/2150170415 wait complete. 2024-04-25T13:52:47.371 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T13:52:47.371 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T13:52:47.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090cf87640 1 Processor -- start 2024-04-25T13:52:47.517 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090cf87640 1 -- start start 2024-04-25T13:52:47.520 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090cf87640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f09080d0ed0 0x7f09080d12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090cf87640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09080d4820 0x7f09080d4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090cf87640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09080d1820 con 0x7f09080d24a0 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090cf87640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f0908164040 con 0x7f09080d0ed0 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090cf87640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0908164bc0 con 0x7f09080d4820 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f0905d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f09080d24a0 0x7f09080d28b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55432/0 (socket says 172.21.15.80:55432) 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f0905d9b640 1 -- 172.21.15.80:0/1718057337 learned_addr learned my addr 172.21.15.80:0/1718057337 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090659c640 1 --2- 172.21.15.80:0/1718057337 >> v2:172.21.15.87:3300/0 conn(0x7f09080d0ed0 0x7f09080d12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09080d4820 0x7f09080d4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1558802081 0 0) 0x7f08f8001120 con 0x7f09080d24a0 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f08f0003530 con 0x7f09080d24a0 2024-04-25T13:52:47.521 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2731330413 0 0) 0x7f08f0003530 con 0x7f09080d24a0 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09080d4820 msgr2=0x7f09080d4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.522+0000 7f090559a640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09080d4820 0x7f09080d4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 >> v2:172.21.15.87:3300/0 conn(0x7f09080d0ed0 msgr2=0x7f09080d12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 --2- 172.21.15.80:0/1718057337 >> v2:172.21.15.87:3300/0 conn(0x7f09080d0ed0 0x7f09080d12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09080d2fc0 con 0x7f09080d24a0 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f08f8001850 con 0x7f09080d24a0 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 >> v1:172.21.15.80:6789/0 conn(0x7f09080d24a0 legacy=0x7f09080d28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f00039c0 0x7f08f0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09080d4820 0x7f08f0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08f0004d50 0x7f08f0005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0908164040 con 0x7f08f0004d50 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0908164bc0 con 0x7f08f00039c0 2024-04-25T13:52:47.522 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f08f000ba00 con 0x7f09080d4820 2024-04-25T13:52:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08f0004d50 0x7f08f0005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0905d9b640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09080d4820 0x7f08f0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0906d9d640 1 -- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09080d4820 msgr2=0x7f08f0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f09080d4820 0x7f08f0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0906d9d640 1 -- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f00039c0 msgr2=0x7f08f0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f08f00039c0 0x7f08f0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.523 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0906d9d640 1 -- 172.21.15.80:0/1718057337 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08f0003530 con 0x7f08f0004d50 2024-04-25T13:52:47.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08f0004d50 0x7f08f0005160 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f090808e760 tx=0x7f090001cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3477baf472b10f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09000b2070 con 0x7f08f0004d50 2024-04-25T13:52:47.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09000a5c50 con 0x7f08f0004d50 2024-04-25T13:52:47.525 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090559a640 1 -- 172.21.15.80:0/1718057337 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0900009c10 con 0x7f08f0004d50 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090cf87640 1 -- 172.21.15.80:0/1718057337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08f0004d50 msgr2=0x7f08f0005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090cf87640 1 --2- 172.21.15.80:0/1718057337 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f08f0004d50 0x7f08f0005160 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f090808e760 tx=0x7f090001cbe0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f0906d9d640 1 -- 172.21.15.80:0/1718057337 reap_dead start 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090cf87640 1 -- 172.21.15.80:0/1718057337 shutdown_connections 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090cf87640 1 -- 172.21.15.80:0/1718057337 >> 172.21.15.80:0/1718057337 conn(0x7f090806c5f0 msgr2=0x7f090806ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090cf87640 1 -- 172.21.15.80:0/1718057337 shutdown_connections 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.526+0000 7f090cf87640 1 -- 172.21.15.80:0/1718057337 wait complete. 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 Processor -- start 2024-04-25T13:52:47.526 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 -- start start 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f090815c060 0x7f09080c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f090815d630 0x7f09080ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908162f00 0x7f0908163310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f0908164850 con 0x7f090815c060 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f0908164570 con 0x7f0908162f00 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f0908157820 con 0x7f090815d630 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f090815c060 0x7f09080c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f090815c060 0x7f09080c9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37082/0 (socket says 172.21.15.80:37082) 2024-04-25T13:52:47.529 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 -- 172.21.15.80:0/1151386844 learned_addr learned my addr 172.21.15.80:0/1151386844 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908162f00 0x7f0908163310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f0905d9b640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f090815d630 0x7f09080ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 -- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f090815d630 msgr2=0x7f09080ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f090815d630 0x7f09080ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 -- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908162f00 msgr2=0x7f0908163310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908162f00 0x7f0908163310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 -- 172.21.15.80:0/1151386844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0908163850 con 0x7f090815c060 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0908162f00 0x7f0908163310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090659c640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f090815c060 0x7f09080c9170 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00e7c0 tx=0x7f08fc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc017070 con 0x7f090815c060 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f08fc00ee70 con 0x7f090815c060 2024-04-25T13:52:47.530 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00cb60 con 0x7f090815c060 2024-04-25T13:52:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0908155a00 con 0x7f090815c060 2024-04-25T13:52:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.530+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090815c870 con 0x7f090815c060 2024-04-25T13:52:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f090806a440 con 0x7f090815c060 2024-04-25T13:52:47.531 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f08fc01a070 con 0x7f090815c060 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f08f6ffd640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08e806f430 0x7f08e8071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f08fc08f180 con 0x7f090815c060 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f08f6ffd640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f08e8075340 0x7f08e8077790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 --> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08fc012e50 con 0x7f08e8075340 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f08fc08f610 con 0x7f090815c060 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f0905d9b640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08e806f430 0x7f08e8071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f08e8075340 0x7f08e8077790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f0905d9b640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08e806f430 0x7f08e8071940 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f08f8004850 tx=0x7f08f8001280 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.532 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.534+0000 7f0906d9d640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f08e8075340 0x7f08e8077790 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.533 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.538+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 <== osd.5 v2:172.21.15.87:6800/149615972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f08fc012e50 con 0x7f08e8075340 2024-04-25T13:52:47.543 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.546+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 --> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f09080d43d0 con 0x7f08e8075340 2024-04-25T13:52:47.544 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f08f6ffd640 1 -- 172.21.15.80:0/1151386844 <== osd.5 v2:172.21.15.87:6800/149615972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f09080d43d0 con 0x7f08e8075340 2024-04-25T13:52:47.544 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:47.544 INFO:teuthology.orchestra.run.smithi080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-25T13:52:47.544 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:47.545 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f08e8075340 msgr2=0x7f08e8077790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6800/149615972,v1:172.21.15.87:6801/149615972] conn(0x7f08e8075340 0x7f08e8077790 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08e806f430 msgr2=0x7f08e8071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f08e806f430 0x7f08e8071940 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f08f8004850 tx=0x7f08f8001280 comp rx=0 tx=0).stop 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f090815c060 msgr2=0x7f09080c9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 --2- 172.21.15.80:0/1151386844 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f090815c060 0x7f09080c9170 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00e7c0 tx=0x7f08fc00ec80 comp rx=0 tx=0).stop 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f0906d9d640 1 -- 172.21.15.80:0/1151386844 reap_dead start 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 shutdown_connections 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 >> 172.21.15.80:0/1151386844 conn(0x7f090806c5f0 msgr2=0x7f0908157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 shutdown_connections 2024-04-25T13:52:47.546 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.550+0000 7f090cf87640 1 -- 172.21.15.80:0/1151386844 wait complete. 2024-04-25T13:52:47.562 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T13:52:47.702 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.706+0000 7f2b6c923640 1 Processor -- start 2024-04-25T13:52:47.706 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.706+0000 7f2b6c923640 1 -- start start 2024-04-25T13:52:47.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6c923640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f2b6806f5a0 0x7f2b68160b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6c923640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680733b0 0x7f2b68161480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.710 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6c923640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b6806fcf0 con 0x7f2b68161070 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6c923640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f2b681619c0 con 0x7f2b6806f5a0 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6c923640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b68073f10 con 0x7f2b680733b0 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b65d9b640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f2b68161070 0x7f2b680b4bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55448/0 (socket says 172.21.15.80:55448) 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b65d9b640 1 -- 172.21.15.80:0/1421939996 learned_addr learned my addr 172.21.15.80:0/1421939996 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6659c640 1 --2- 172.21.15.80:0/1421939996 >> v2:172.21.15.87:3300/0 conn(0x7f2b6806f5a0 0x7f2b68160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b66d9d640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680733b0 0x7f2b68161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3103418681 0 0) 0x7f2b6806fcf0 con 0x7f2b68161070 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2b50003530 con 0x7f2b68161070 2024-04-25T13:52:47.711 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3307338458 0 0) 0x7f2b50003530 con 0x7f2b68161070 2024-04-25T13:52:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680733b0 msgr2=0x7f2b68161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680733b0 0x7f2b68161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 >> v2:172.21.15.87:3300/0 conn(0x7f2b6806f5a0 msgr2=0x7f2b68160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 --2- 172.21.15.80:0/1421939996 >> v2:172.21.15.87:3300/0 conn(0x7f2b6806f5a0 0x7f2b68160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b680b0c60 con 0x7f2b68161070 2024-04-25T13:52:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f2b5c001680 con 0x7f2b68161070 2024-04-25T13:52:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 >> v1:172.21.15.80:6789/0 conn(0x7f2b68161070 legacy=0x7f2b680b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.712 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680733b0 0x7f2b500039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b50003f70 0x7f2b50004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b50004d50 0x7f2b50005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b681619c0 con 0x7f2b50004d50 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b68073f10 con 0x7f2b50003f70 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b5000bb00 con 0x7f2b680733b0 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b66d9d640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b50004d50 0x7f2b50005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b6659c640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680733b0 0x7f2b500039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b65d9b640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b50003f70 0x7f2b50004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b66d9d640 1 -- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680733b0 msgr2=0x7f2b500039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b66d9d640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680733b0 0x7f2b500039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b66d9d640 1 -- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b50003f70 msgr2=0x7f2b50004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:47.713 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b66d9d640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b50003f70 0x7f2b50004810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.710+0000 7f2b66d9d640 1 -- 172.21.15.80:0/1421939996 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b50003530 con 0x7f2b50004d50 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b66d9d640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b50004d50 0x7f2b50005160 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f2b68070290 tx=0x7f2b6001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae9d1846dcec1a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600b2070 con 0x7f2b50004d50 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b600a5a10 con 0x7f2b50004d50 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6559a640 1 -- 172.21.15.80:0/1421939996 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60009c10 con 0x7f2b50004d50 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6c923640 1 -- 172.21.15.80:0/1421939996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b50004d50 msgr2=0x7f2b50005160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6c923640 1 --2- 172.21.15.80:0/1421939996 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b50004d50 0x7f2b50005160 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f2b68070290 tx=0x7f2b6001cc00 comp rx=0 tx=0).stop 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b66d9d640 1 -- 172.21.15.80:0/1421939996 reap_dead start 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6c923640 1 -- 172.21.15.80:0/1421939996 shutdown_connections 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6c923640 1 -- 172.21.15.80:0/1421939996 >> 172.21.15.80:0/1421939996 conn(0x7f2b6806d080 msgr2=0x7f2b680b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6c923640 1 -- 172.21.15.80:0/1421939996 shutdown_connections 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6c923640 1 -- 172.21.15.80:0/1421939996 wait complete. 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6c923640 1 Processor -- start 2024-04-25T13:52:47.714 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.714+0000 7f2b6c923640 1 -- start start 2024-04-25T13:52:47.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b681f8690 0x7f2b681f3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680d0520 0x7f2b680d0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b68153080 0x7f2b68153490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f2b6806f9b0 con 0x7f2b68153080 2024-04-25T13:52:47.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f2b680b1220 con 0x7f2b681f8690 2024-04-25T13:52:47.718 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f2b68161480 con 0x7f2b680d0520 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b68153080 0x7f2b68153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b68153080 0x7f2b68153490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37102/0 (socket says 172.21.15.80:37102) 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 -- 172.21.15.80:0/3190096464 learned_addr learned my addr 172.21.15.80:0/3190096464 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b65d9b640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680d0520 0x7f2b680d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6659c640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b681f8690 0x7f2b681f3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 -- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680d0520 msgr2=0x7f2b680d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680d0520 0x7f2b680d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 -- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b681f8690 msgr2=0x7f2b681f3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b681f8690 0x7f2b681f3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 -- 172.21.15.80:0/3190096464 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b680d0e70 con 0x7f2b68153080 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b66d9d640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b68153080 0x7f2b68153490 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f2b6001d4a0 tx=0x7f2b60009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b56ffd640 1 -- 172.21.15.80:0/3190096464 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b600b2040 con 0x7f2b68153080 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b56ffd640 1 -- 172.21.15.80:0/3190096464 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b600bc070 con 0x7f2b68153080 2024-04-25T13:52:47.719 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b68155e50 con 0x7f2b68153080 2024-04-25T13:52:47.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b56ffd640 1 -- 172.21.15.80:0/3190096464 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60009c10 con 0x7f2b68153080 2024-04-25T13:52:47.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b68155580 con 0x7f2b68153080 2024-04-25T13:52:47.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b681557c0 con 0x7f2b68153080 2024-04-25T13:52:47.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b56ffd640 1 -- 172.21.15.80:0/3190096464 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f2b600ad840 con 0x7f2b68153080 2024-04-25T13:52:47.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b56ffd640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4806f400 0x7f2b48071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:47.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b56ffd640 1 -- 172.21.15.80:0/3190096464 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f2b601294b0 con 0x7f2b68153080 2024-04-25T13:52:47.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.718+0000 7f2b6659c640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4806f400 0x7f2b48071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:47.720 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.722+0000 7f2b6659c640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4806f400 0x7f2b48071910 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2b68152da0 tx=0x7f2b58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:47.721 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.722+0000 7f2b56ffd640 1 -- 172.21.15.80:0/3190096464 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f2b681557c0 con 0x7f2b68153080 2024-04-25T13:52:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.886+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f2b68156130 con 0x7f2b4806f400 2024-04-25T13:52:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.886+0000 7f2b56ffd640 1 -- 172.21.15.80:0/3190096464 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2b68156130 con 0x7f2b4806f400 2024-04-25T13:52:47.880 INFO:teuthology.orchestra.run.smithi080.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T13:52:47.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4806f400 msgr2=0x7f2b48071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4806f400 0x7f2b48071910 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2b68152da0 tx=0x7f2b58008040 comp rx=0 tx=0).stop 2024-04-25T13:52:47.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b68153080 msgr2=0x7f2b68153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:47.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b68153080 0x7f2b68153490 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f2b6001d4a0 tx=0x7f2b60009750 comp rx=0 tx=0).stop 2024-04-25T13:52:47.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 shutdown_connections 2024-04-25T13:52:47.884 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f2b4806f400 0x7f2b48071910 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f2b68153080 0x7f2b68153490 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f2b680d0520 0x7f2b680d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 --2- 172.21.15.80:0/3190096464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b681f8690 0x7f2b681f3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:47.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 >> 172.21.15.80:0/3190096464 conn(0x7f2b6806d080 msgr2=0x7f2b680d1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:47.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 shutdown_connections 2024-04-25T13:52:47.885 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:47.890+0000 7f2b6c923640 1 -- 172.21.15.80:0/3190096464 wait complete. 2024-04-25T13:52:47.903 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T13:52:47.903 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T13:52:48.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.062+0000 7f763022a640 1 Processor -- start 2024-04-25T13:52:48.059 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.062+0000 7f763022a640 1 -- start start 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f763022a640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f76280d0f50 0x7f7628161600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f763022a640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76280b0e10 con 0x7f76280b48c0 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f763022a640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f76280b10f0 con 0x7f762815d520 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f763022a640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f76280b0bc0 con 0x7f76280d0f50 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762dfc6640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f76280b48c0 0x7f76280d0840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:55464/0 (socket says 172.21.15.80:55464) 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762dfc6640 1 -- 172.21.15.80:0/3196715589 learned_addr learned my addr 172.21.15.80:0/3196715589 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762d7c5640 1 --2- 172.21.15.80:0/3196715589 >> v2:172.21.15.87:3300/0 conn(0x7f76280d0f50 0x7f7628161600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 133030875 0 0) 0x7f76280b0e10 con 0x7f76280b48c0 2024-04-25T13:52:48.061 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7614003530 con 0x7f76280b48c0 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2897655936 0 0) 0x7f7614003530 con 0x7f76280b48c0 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 >> v2:172.21.15.87:3300/0 conn(0x7f76280d0f50 msgr2=0x7f7628161600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 --2- 172.21.15.80:0/3196715589 >> v2:172.21.15.87:3300/0 conn(0x7f76280d0f50 0x7f7628161600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 >> v1:172.21.15.87:6789/0 conn(0x7f762815d520 legacy=0x7f762815d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762806f710 con 0x7f76280b48c0 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f7620001a00 con 0x7f76280b48c0 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 >> v1:172.21.15.80:6789/0 conn(0x7f76280b48c0 legacy=0x7f76280d0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76140039c0 0x7f7614003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.063 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7614004810 0x7f7614004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7614005260 0x7f761400dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f76280b0bc0 con 0x7f7614004810 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f761400e400 con 0x7f7614005260 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f761400e600 con 0x7f76140039c0 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762d7c5640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7614004810 0x7f7614004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762e7c7640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7614005260 0x7f761400dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762dfc6640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76140039c0 0x7f7614003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762d7c5640 1 -- 172.21.15.80:0/3196715589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76140039c0 msgr2=0x7f7614003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762d7c5640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76140039c0 0x7f7614003e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762d7c5640 1 -- 172.21.15.80:0/3196715589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7614005260 msgr2=0x7f761400dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762d7c5640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7614005260 0x7f761400dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762d7c5640 1 -- 172.21.15.80:0/3196715589 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7614003530 con 0x7f7614004810 2024-04-25T13:52:48.064 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.066+0000 7f762e7c7640 1 -- 172.21.15.80:0/3196715589 reap_dead start 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f762d7c5640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7614004810 0x7f7614004c20 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f7618009ae0 tx=0x7f761801d4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4844a716855a20bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180b2070 con 0x7f7614004810 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76180a5550 con 0x7f7614004810 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f762cfc4640 1 -- 172.21.15.80:0/3196715589 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180096b0 con 0x7f7614004810 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 -- 172.21.15.80:0/3196715589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7614004810 msgr2=0x7f7614004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7614004810 0x7f7614004c20 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f7618009ae0 tx=0x7f761801d4d0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 -- 172.21.15.80:0/3196715589 shutdown_connections 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 --2- 172.21.15.80:0/3196715589 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7614004810 0x7f7614004c20 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 -- 172.21.15.80:0/3196715589 >> 172.21.15.80:0/3196715589 conn(0x7f76280b35c0 msgr2=0x7f76280b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 -- 172.21.15.80:0/3196715589 shutdown_connections 2024-04-25T13:52:48.066 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 -- 172.21.15.80:0/3196715589 wait complete. 2024-04-25T13:52:48.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 Processor -- start 2024-04-25T13:52:48.067 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.070+0000 7f763022a640 1 -- start start 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7614004810 0x7f762814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7628150380 0x7f76280af8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76280afdf0 0x7f762815b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f762815e2c0 con 0x7f7628150380 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f762815e140 con 0x7f7614004810 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f762815e440 con 0x7f76280afdf0 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7628150380 0x7f76280af8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7628150380 0x7f76280af8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37124/0 (socket says 172.21.15.80:37124) 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 -- 172.21.15.80:0/2177024242 learned_addr learned my addr 172.21.15.80:0/2177024242 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762dfc6640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7614004810 0x7f762814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762e7c7640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76280afdf0 0x7f762815b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 -- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76280afdf0 msgr2=0x7f762815b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.070 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76280afdf0 0x7f762815b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 -- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7614004810 msgr2=0x7f762814fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7614004810 0x7f762814fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 -- 172.21.15.80:0/2177024242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7628073640 con 0x7f7628150380 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762e7c7640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f76280afdf0 0x7f762815b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f762d7c5640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7628150380 0x7f76280af8b0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f761801c8a0 tx=0x7f7618019810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180b2040 con 0x7f7628150380 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76180ad440 con 0x7f7628150380 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7628150160 con 0x7f7628150380 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76180ad810 con 0x7f7628150380 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76280b0200 con 0x7f7628150380 2024-04-25T13:52:48.071 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7628073220 con 0x7f7628150380 2024-04-25T13:52:48.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f76180bf060 con 0x7f7628150380 2024-04-25T13:52:48.072 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f761e7fc640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f760c06f400 0x7f760c071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.074+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f7618128ca0 con 0x7f7628150380 2024-04-25T13:52:48.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.078+0000 7f761e7fc640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f760c075310 0x7f760c077760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.078+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 --> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f76180b3e50 con 0x7f760c075310 2024-04-25T13:52:48.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.078+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7f7618129130 con 0x7f7628150380 2024-04-25T13:52:48.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.078+0000 7f762dfc6640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f760c06f400 0x7f760c071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.078+0000 7f762e7c7640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f760c075310 0x7f760c077760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.078+0000 7f762dfc6640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f760c06f400 0x7f760c071910 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f76200012f0 tx=0x7f7620009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.073 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.078+0000 7f762e7c7640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f760c075310 0x7f760c077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.074 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.078+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 <== osd.6 v2:172.21.15.87:6824/1131074035 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7f76180b3e50 con 0x7f760c075310 2024-04-25T13:52:48.085 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 --> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7628073a50 con 0x7f760c075310 2024-04-25T13:52:48.086 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f761e7fc640 1 -- 172.21.15.80:0/2177024242 <== osd.6 v2:172.21.15.87:6824/1131074035 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7f7628073a50 con 0x7f760c075310 2024-04-25T13:52:48.086 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:48.086 INFO:teuthology.orchestra.run.smithi080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-25T13:52:48.086 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:48.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f760c075310 msgr2=0x7f760c077760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6824/1131074035,v1:172.21.15.87:6825/1131074035] conn(0x7f760c075310 0x7f760c077760 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f760c06f400 msgr2=0x7f760c071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.087 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f760c06f400 0x7f760c071910 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f76200012f0 tx=0x7f7620009040 comp rx=0 tx=0).stop 2024-04-25T13:52:48.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7628150380 msgr2=0x7f76280af8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 --2- 172.21.15.80:0/2177024242 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f7628150380 0x7f76280af8b0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f761801c8a0 tx=0x7f7618019810 comp rx=0 tx=0).stop 2024-04-25T13:52:48.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f762e7c7640 1 -- 172.21.15.80:0/2177024242 reap_dead start 2024-04-25T13:52:48.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 shutdown_connections 2024-04-25T13:52:48.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 >> 172.21.15.80:0/2177024242 conn(0x7f76280b35c0 msgr2=0x7f7628153290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:48.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 shutdown_connections 2024-04-25T13:52:48.088 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.090+0000 7f763022a640 1 -- 172.21.15.80:0/2177024242 wait complete. 2024-04-25T13:52:48.106 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T13:52:48.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5697604640 1 Processor -- start 2024-04-25T13:52:48.237 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5697604640 1 -- start start 2024-04-25T13:52:48.239 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5697604640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f56900b8cd0 0x7f56900b5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5697604640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56900d0b00 0x7f56900b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5697604640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f569015e8c0 con 0x7f56900cf690 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5697604640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f56900ce8d0 con 0x7f56900d0b00 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5697604640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f56900cebb0 con 0x7f56900b8cd0 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5695ba1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56900d0b00 0x7f56900b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5695ba1640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f56900d0b00 0x7f56900b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37140/0 (socket says 172.21.15.80:37140) 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.242+0000 7f5695ba1640 1 -- 172.21.15.80:0/1186071137 learned_addr learned my addr 172.21.15.80:0/1186071137 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f56953a0640 1 --2- 172.21.15.80:0/1186071137 >> v2:172.21.15.87:3300/0 conn(0x7f56900b8cd0 0x7f56900b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5695ba1640 1 -- 172.21.15.80:0/1186071137 >> v2:172.21.15.87:3300/0 conn(0x7f56900b8cd0 msgr2=0x7f56900b5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5695ba1640 1 --2- 172.21.15.80:0/1186071137 >> v2:172.21.15.87:3300/0 conn(0x7f56900b8cd0 0x7f56900b5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5695ba1640 1 -- 172.21.15.80:0/1186071137 >> v1:172.21.15.87:6789/0 conn(0x7f56900cf690 legacy=0x7f56900d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:48.240 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5695ba1640 1 -- 172.21.15.80:0/1186071137 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56900cee90 con 0x7f56900d0b00 2024-04-25T13:52:48.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5695ba1640 1 --2- 172.21.15.80:0/1186071137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56900d0b00 0x7f56900b6920 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f56900b6660 tx=0x7f568801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1404c31c396b186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5687fff640 1 -- 172.21.15.80:0/1186071137 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880b2070 con 0x7f56900d0b00 2024-04-25T13:52:48.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5687fff640 1 -- 172.21.15.80:0/1186071137 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56880a9070 con 0x7f56900d0b00 2024-04-25T13:52:48.242 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5687fff640 1 -- 172.21.15.80:0/1186071137 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880ad880 con 0x7f56900d0b00 2024-04-25T13:52:48.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5697604640 1 -- 172.21.15.80:0/1186071137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56900d0b00 msgr2=0x7f56900b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5697604640 1 --2- 172.21.15.80:0/1186071137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56900d0b00 0x7f56900b6920 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f56900b6660 tx=0x7f568801c450 comp rx=0 tx=0).stop 2024-04-25T13:52:48.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5697604640 1 -- 172.21.15.80:0/1186071137 shutdown_connections 2024-04-25T13:52:48.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5697604640 1 --2- 172.21.15.80:0/1186071137 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f56900d0b00 0x7f56900b6920 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5697604640 1 --2- 172.21.15.80:0/1186071137 >> v2:172.21.15.87:3300/0 conn(0x7f56900b8cd0 0x7f56900b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5697604640 1 -- 172.21.15.80:0/1186071137 >> 172.21.15.80:0/1186071137 conn(0x7f56900d3ad0 msgr2=0x7f56900d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:48.243 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5697604640 1 -- 172.21.15.80:0/1186071137 shutdown_connections 2024-04-25T13:52:48.244 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.246+0000 7f5697604640 1 -- 172.21.15.80:0/1186071137 wait complete. 2024-04-25T13:52:48.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 Processor -- start 2024-04-25T13:52:48.245 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 -- start start 2024-04-25T13:52:48.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56900b8cd0 0x7f56900cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56900cf690 0x7f56901556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.247 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5690159390 0x7f569009f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f56900d2c20 con 0x7f5690159390 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f56900d2aa0 con 0x7f56900b8cd0 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f569016bb00 con 0x7f56900cf690 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5690159390 0x7f569009f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5690159390 0x7f569009f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37146/0 (socket says 172.21.15.80:37146) 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 -- 172.21.15.80:0/3953705913 learned_addr learned my addr 172.21.15.80:0/3953705913 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f56953a0640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56900b8cd0 0x7f56900cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 -- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56900cf690 msgr2=0x7f56901556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56900cf690 0x7f56901556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 -- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56900b8cd0 msgr2=0x7f56900cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56900b8cd0 0x7f56900cd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 -- 172.21.15.80:0/3953705913 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56900cdea0 con 0x7f5690159390 2024-04-25T13:52:48.248 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f56953a0640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56900b8cd0 0x7f56900cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:48.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5695ba1640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5690159390 0x7f569009f380 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f56880b0040 tx=0x7f56880191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5685ffb640 1 -- 172.21.15.80:0/3953705913 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880bb070 con 0x7f5690159390 2024-04-25T13:52:48.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5685ffb640 1 -- 172.21.15.80:0/3953705913 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56880add30 con 0x7f5690159390 2024-04-25T13:52:48.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5685ffb640 1 -- 172.21.15.80:0/3953705913 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56880b2040 con 0x7f5690159390 2024-04-25T13:52:48.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f569006c790 con 0x7f5690159390 2024-04-25T13:52:48.249 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.250+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f569006ca30 con 0x7f5690159390 2024-04-25T13:52:48.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.254+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56900cc9a0 con 0x7f5690159390 2024-04-25T13:52:48.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.254+0000 7f5685ffb640 1 -- 172.21.15.80:0/3953705913 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f56880be070 con 0x7f5690159390 2024-04-25T13:52:48.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.254+0000 7f5685ffb640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f567406f430 0x7f5674071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.254+0000 7f5685ffb640 1 -- 172.21.15.80:0/3953705913 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f5688127f10 con 0x7f5690159390 2024-04-25T13:52:48.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.254+0000 7f56953a0640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f567406f430 0x7f5674071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.250 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.254+0000 7f56953a0640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f567406f430 0x7f5674071940 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f568c009f50 tx=0x7f568c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.253 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.258+0000 7f5685ffb640 1 -- 172.21.15.80:0/3953705913 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f56900cdea0 con 0x7f5690159390 2024-04-25T13:52:48.404 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f569006a3a0 con 0x7f567406f430 2024-04-25T13:52:48.405 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5685ffb640 1 -- 172.21.15.80:0/3953705913 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f569006a3a0 con 0x7f567406f430 2024-04-25T13:52:48.405 INFO:teuthology.orchestra.run.smithi080.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T13:52:48.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f567406f430 msgr2=0x7f5674071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f567406f430 0x7f5674071940 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f568c009f50 tx=0x7f568c00b040 comp rx=0 tx=0).stop 2024-04-25T13:52:48.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5690159390 msgr2=0x7f569009f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5690159390 0x7f569009f380 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f56880b0040 tx=0x7f56880191f0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 shutdown_connections 2024-04-25T13:52:48.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f567406f430 0x7f5674071940 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.407 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f5690159390 0x7f569009f380 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f56900cf690 0x7f56901556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 --2- 172.21.15.80:0/3953705913 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56900b8cd0 0x7f56900cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 >> 172.21.15.80:0/3953705913 conn(0x7f56900d3ad0 msgr2=0x7f56900b7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:48.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 shutdown_connections 2024-04-25T13:52:48.408 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.410+0000 7f5697604640 1 -- 172.21.15.80:0/3953705913 wait complete. 2024-04-25T13:52:48.425 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T13:52:48.426 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T13:52:48.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.586+0000 7fd778af4640 1 Processor -- start 2024-04-25T13:52:48.582 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.586+0000 7fd778af4640 1 -- start start 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd778af4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740b0f10 0x7fd7740d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd778af4640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fd7740ce2a0 0x7fd7740ce6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd778af4640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7740cf730 0x7fd7740cfb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd778af4640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7fd7740b5820 con 0x7fd7740cf730 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd778af4640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fd7740d13f0 con 0x7fd7740ce2a0 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd778af4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd7740b1320 con 0x7fd7740b0f10 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd772d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7740cf730 0x7fd7740cfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd772d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7fd7740cf730 0x7fd7740cfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37160/0 (socket says 172.21.15.80:37160) 2024-04-25T13:52:48.585 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd772d9d640 1 -- 172.21.15.80:0/4226207469 learned_addr learned my addr 172.21.15.80:0/4226207469 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:48.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd77259c640 1 --2- 172.21.15.80:0/4226207469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740b0f10 0x7fd7740d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd769d9b640 1 --2- 172.21.15.80:0/4226207469 >> v2:172.21.15.87:3300/0 conn(0x7fd7740ce2a0 0x7fd7740ce6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd772d9d640 1 -- 172.21.15.80:0/4226207469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740b0f10 msgr2=0x7fd7740d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:48.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd772d9d640 1 --2- 172.21.15.80:0/4226207469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740b0f10 0x7fd7740d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd772d9d640 1 -- 172.21.15.80:0/4226207469 >> v2:172.21.15.87:3300/0 conn(0x7fd7740ce2a0 msgr2=0x7fd7740ce6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:48.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd772d9d640 1 --2- 172.21.15.80:0/4226207469 >> v2:172.21.15.87:3300/0 conn(0x7fd7740ce2a0 0x7fd7740ce6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd772d9d640 1 -- 172.21.15.80:0/4226207469 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7740cebf0 con 0x7fd7740cf730 2024-04-25T13:52:48.586 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.590+0000 7fd769d9b640 1 --2- 172.21.15.80:0/4226207469 >> v2:172.21.15.87:3300/0 conn(0x7fd7740ce2a0 0x7fd7740ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:48.587 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd772d9d640 1 --2- 172.21.15.80:0/4226207469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740cf730 0x7fd7740cfb40 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fd774071aa0 tx=0x7fd76c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fd7efbeb4510bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd771d9b640 1 -- 172.21.15.80:0/4226207469 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0a5670 con 0x7fd7740cf730 2024-04-25T13:52:48.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd771d9b640 1 -- 172.21.15.80:0/4226207469 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd76c0a5810 con 0x7fd7740cf730 2024-04-25T13:52:48.588 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd771d9b640 1 -- 172.21.15.80:0/4226207469 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0a5be0 con 0x7fd7740cf730 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 -- 172.21.15.80:0/4226207469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740cf730 msgr2=0x7fd7740cfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 --2- 172.21.15.80:0/4226207469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740cf730 0x7fd7740cfb40 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7fd774071aa0 tx=0x7fd76c01c9f0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 -- 172.21.15.80:0/4226207469 shutdown_connections 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 --2- 172.21.15.80:0/4226207469 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740cf730 0x7fd7740cfb40 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 --2- 172.21.15.80:0/4226207469 >> v2:172.21.15.87:3300/0 conn(0x7fd7740ce2a0 0x7fd7740ce6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 --2- 172.21.15.80:0/4226207469 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740b0f10 0x7fd7740d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 -- 172.21.15.80:0/4226207469 >> 172.21.15.80:0/4226207469 conn(0x7fd7740b35c0 msgr2=0x7fd7740b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 -- 172.21.15.80:0/4226207469 shutdown_connections 2024-04-25T13:52:48.589 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 -- 172.21.15.80:0/4226207469 wait complete. 2024-04-25T13:52:48.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 Processor -- start 2024-04-25T13:52:48.590 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.594+0000 7fd778af4640 1 -- start start 2024-04-25T13:52:48.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740b0f10 0x7fd774151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7740ce2a0 0x7fd774158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740cf730 0x7fd774157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fd77415e720 con 0x7fd7740b0f10 2024-04-25T13:52:48.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fd7740b5300 con 0x7fd7740ce2a0 2024-04-25T13:52:48.593 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fd7740d0490 con 0x7fd7740cf730 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740b0f10 0x7fd774151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740b0f10 0x7fd774151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37174/0 (socket says 172.21.15.80:37174) 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 -- 172.21.15.80:0/449977358 learned_addr learned my addr 172.21.15.80:0/449977358 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd769d9b640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7740ce2a0 0x7fd774158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd772d9d640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740cf730 0x7fd774157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 -- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740cf730 msgr2=0x7fd774157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fd7740cf730 0x7fd774157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 -- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7740ce2a0 msgr2=0x7fd774158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7740ce2a0 0x7fd774158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 -- 172.21.15.80:0/449977358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7740d2610 con 0x7fd7740b0f10 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd769d9b640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd7740ce2a0 0x7fd774158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd77259c640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740b0f10 0x7fd774151390 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fd76400d5c0 tx=0x7fd76400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764004240 con 0x7fd7740b0f10 2024-04-25T13:52:48.594 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7640043e0 con 0x7fd7740b0f10 2024-04-25T13:52:48.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd774151c90 con 0x7fd7740b0f10 2024-04-25T13:52:48.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764003840 con 0x7fd7740b0f10 2024-04-25T13:52:48.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd774152250 con 0x7fd7740b0f10 2024-04-25T13:52:48.595 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd77406a440 con 0x7fd7740b0f10 2024-04-25T13:52:48.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fd7640039e0 con 0x7fd7740b0f10 2024-04-25T13:52:48.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd76b7fe640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd75406f430 0x7fd754071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fd76408fc20 con 0x7fd7740b0f10 2024-04-25T13:52:48.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd76b7fe640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7fd754075340 0x7fd754077790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.598+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 --> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd754077e60 con 0x7fd754075340 2024-04-25T13:52:48.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.602+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_get_version_reply(handle=1 version=194) ==== 24+0+0 (secure 0 0 0) 0x7fd7640900b0 con 0x7fd7740b0f10 2024-04-25T13:52:48.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.602+0000 7fd769d9b640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd75406f430 0x7fd754071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.596 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.602+0000 7fd772d9d640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7fd754075340 0x7fd754077790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.602+0000 7fd769d9b640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd75406f430 0x7fd754071940 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd7740b06e0 tx=0x7fd75c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.597 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.602+0000 7fd772d9d640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7fd754075340 0x7fd754077790 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.598 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.602+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 <== osd.7 v2:172.21.15.87:6808/710369794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+25033 (crc 0 0 0) 0x7fd754077e60 con 0x7fd754075340 2024-04-25T13:52:48.610 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.614+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 --> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd7740cf390 con 0x7fd754075340 2024-04-25T13:52:48.611 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.614+0000 7fd76b7fe640 1 -- 172.21.15.80:0/449977358 <== osd.7 v2:172.21.15.87:6808/710369794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+565 (crc 0 0 0) 0x7fd7740cf390 con 0x7fd754075340 2024-04-25T13:52:48.611 INFO:teuthology.orchestra.run.smithi080.stdout:{ 2024-04-25T13:52:48.611 INFO:teuthology.orchestra.run.smithi080.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-04-25T13:52:48.611 INFO:teuthology.orchestra.run.smithi080.stdout:} 2024-04-25T13:52:48.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7fd754075340 msgr2=0x7fd754077790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.612 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6808/710369794,v1:172.21.15.87:6809/710369794] conn(0x7fd754075340 0x7fd754077790 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd75406f430 msgr2=0x7fd754071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fd75406f430 0x7fd754071940 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fd7740b06e0 tx=0x7fd75c008040 comp rx=0 tx=0).stop 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740b0f10 msgr2=0x7fd774151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 --2- 172.21.15.80:0/449977358 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fd7740b0f10 0x7fd774151390 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7fd76400d5c0 tx=0x7fd76400da80 comp rx=0 tx=0).stop 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd772d9d640 1 -- 172.21.15.80:0/449977358 reap_dead start 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 shutdown_connections 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 >> 172.21.15.80:0/449977358 conn(0x7fd7740b35c0 msgr2=0x7fd7740b39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 shutdown_connections 2024-04-25T13:52:48.613 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.618+0000 7fd778af4640 1 -- 172.21.15.80:0/449977358 wait complete. 2024-04-25T13:52:48.631 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T13:52:48.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7d6026640 1 Processor -- start 2024-04-25T13:52:48.769 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7d6026640 1 -- start start 2024-04-25T13:52:48.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7d6026640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d0161f90 0x7fc7d00cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7d6026640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7d00d0510 0x7fc7d015e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7d6026640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7d00d0d70 con 0x7fc7d015ea70 2024-04-25T13:52:48.770 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7d6026640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7fc7d00d1050 con 0x7fc7d00d0510 2024-04-25T13:52:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7d6026640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d006dca0 con 0x7fc7d0161f90 2024-04-25T13:52:48.771 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7ceffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7d00d0510 0x7fc7d015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7cf7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d0161f90 0x7fc7d00cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7cf7fe640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d0161f90 0x7fc7d00cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3301/0 says I am v2:172.21.15.80:36204/0 (socket says 172.21.15.80:36204) 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7ceffd640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7fc7d00d0510 0x7fc7d015e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.80:42024/0 (socket says 172.21.15.80:42024) 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.774+0000 7fc7cf7fe640 1 -- 172.21.15.80:0/1452031678 learned_addr learned my addr 172.21.15.80:0/1452031678 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7ceffd640 1 -- 172.21.15.80:0/1452031678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d0161f90 msgr2=0x7fc7d00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7ceffd640 1 --2- 172.21.15.80:0/1452031678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d0161f90 0x7fc7d00cffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7ceffd640 1 -- 172.21.15.80:0/1452031678 >> v1:172.21.15.87:6789/0 conn(0x7fc7d015ea70 legacy=0x7fc7d015ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7ceffd640 1 -- 172.21.15.80:0/1452031678 --> v2:172.21.15.87:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d006df80 con 0x7fc7d00d0510 2024-04-25T13:52:48.772 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7cf7fe640 1 --2- 172.21.15.80:0/1452031678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d0161f90 0x7fc7d00cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:52:48.773 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7ceffd640 1 --2- 172.21.15.80:0/1452031678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d00d0510 0x7fc7d015e530 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80097c0 tx=0x7fc7b801c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a00e46d9926be89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7ce7fc640 1 -- 172.21.15.80:0/1452031678 <== mon.1 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80b0070 con 0x7fc7d00d0510 2024-04-25T13:52:48.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7ce7fc640 1 -- 172.21.15.80:0/1452031678 <== mon.1 v2:172.21.15.87:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7b80ab600 con 0x7fc7d00d0510 2024-04-25T13:52:48.774 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7ce7fc640 1 -- 172.21.15.80:0/1452031678 <== mon.1 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b80ab9d0 con 0x7fc7d00d0510 2024-04-25T13:52:48.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7d6026640 1 -- 172.21.15.80:0/1452031678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d00d0510 msgr2=0x7fc7d015e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7d6026640 1 --2- 172.21.15.80:0/1452031678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d00d0510 0x7fc7d015e530 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80097c0 tx=0x7fc7b801c3b0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7d6026640 1 -- 172.21.15.80:0/1452031678 shutdown_connections 2024-04-25T13:52:48.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7d6026640 1 --2- 172.21.15.80:0/1452031678 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d00d0510 0x7fc7d015e530 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7d6026640 1 --2- 172.21.15.80:0/1452031678 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d0161f90 0x7fc7d00cffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7d6026640 1 -- 172.21.15.80:0/1452031678 >> 172.21.15.80:0/1452031678 conn(0x7fc7d006d080 msgr2=0x7fc7d00b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:48.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7d6026640 1 -- 172.21.15.80:0/1452031678 shutdown_connections 2024-04-25T13:52:48.775 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.778+0000 7fc7d6026640 1 -- 172.21.15.80:0/1452031678 wait complete. 2024-04-25T13:52:48.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 Processor -- start 2024-04-25T13:52:48.776 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 -- start start 2024-04-25T13:52:48.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d00d0510 0x7fc7d015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d015ea70 0x7fc7d015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.779 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d0161f90 0x7fc7d013a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d006f630 con 0x7fc7d0161f90 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7fc7d006f4b0 con 0x7fc7d015ea70 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7fc7d00d0920 con 0x7fc7d00d0510 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d0161f90 0x7fc7d013a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d0161f90 0x7fc7d013a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37188/0 (socket says 172.21.15.80:37188) 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 -- 172.21.15.80:0/2822030231 learned_addr learned my addr 172.21.15.80:0/2822030231 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7ceffd640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d015ea70 0x7fc7d015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cf7fe640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d00d0510 0x7fc7d015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 -- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d00d0510 msgr2=0x7fc7d015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d00d0510 0x7fc7d015beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 -- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d015ea70 msgr2=0x7fc7d015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d015ea70 0x7fc7d015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 -- 172.21.15.80:0/2822030231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7d015cc50 con 0x7fc7d0161f90 2024-04-25T13:52:48.780 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cf7fe640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d00d0510 0x7fc7d015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:52:48.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7cffff640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d0161f90 0x7fc7d013a650 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c760 tx=0x7fc7c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7c7fff640 1 -- 172.21.15.80:0/2822030231 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0003b80 con 0x7fc7d0161f90 2024-04-25T13:52:48.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7c7fff640 1 -- 172.21.15.80:0/2822030231 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7c0003d20 con 0x7fc7d0161f90 2024-04-25T13:52:48.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7d015cf30 con 0x7fc7d0161f90 2024-04-25T13:52:48.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7c7fff640 1 -- 172.21.15.80:0/2822030231 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7c000aa30 con 0x7fc7d0161f90 2024-04-25T13:52:48.781 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.782+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7d015d4f0 con 0x7fc7d0161f90 2024-04-25T13:52:48.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.786+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7d006a3a0 con 0x7fc7d0161f90 2024-04-25T13:52:48.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.786+0000 7fc7c7fff640 1 -- 172.21.15.80:0/2822030231 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7fc7c000abd0 con 0x7fc7d0161f90 2024-04-25T13:52:48.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.786+0000 7fc7c7fff640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7b006f430 0x7fc7b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:48.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.786+0000 7fc7c7fff640 1 -- 172.21.15.80:0/2822030231 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7fc7c008eea0 con 0x7fc7d0161f90 2024-04-25T13:52:48.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.786+0000 7fc7cf7fe640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7b006f430 0x7fc7b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:48.782 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.786+0000 7fc7cf7fe640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7b006f430 0x7fc7b0071940 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc7d006fa20 tx=0x7fc7c8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:48.785 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.790+0000 7fc7c7fff640 1 -- 172.21.15.80:0/2822030231 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7fc7c005c080 con 0x7fc7d0161f90 2024-04-25T13:52:48.933 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.938+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fc7d015c9e0 con 0x7fc7b006f430 2024-04-25T13:52:48.934 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.938+0000 7fc7c7fff640 1 -- 172.21.15.80:0/2822030231 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc7d015c9e0 con 0x7fc7b006f430 2024-04-25T13:52:48.934 INFO:teuthology.orchestra.run.smithi080.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T13:52:48.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7b006f430 msgr2=0x7fc7b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7b006f430 0x7fc7b0071940 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fc7d006fa20 tx=0x7fc7c8008040 comp rx=0 tx=0).stop 2024-04-25T13:52:48.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d0161f90 msgr2=0x7fc7d013a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:48.936 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d0161f90 0x7fc7d013a650 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c760 tx=0x7fc7c000cc20 comp rx=0 tx=0).stop 2024-04-25T13:52:48.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 shutdown_connections 2024-04-25T13:52:48.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7fc7b006f430 0x7fc7b0071940 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7fc7d0161f90 0x7fc7d013a650 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc7d015ea70 0x7fc7d015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 --2- 172.21.15.80:0/2822030231 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7fc7d00d0510 0x7fc7d015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:48.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 >> 172.21.15.80:0/2822030231 conn(0x7fc7d006d080 msgr2=0x7fc7d00b6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:48.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 shutdown_connections 2024-04-25T13:52:48.937 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:48.942+0000 7fc7d6026640 1 -- 172.21.15.80:0/2822030231 wait complete. 2024-04-25T13:52:48.955 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:52:49.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80fd02f640 1 Processor -- start 2024-04-25T13:52:49.093 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80fd02f640 1 -- start start 2024-04-25T13:52:49.095 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80fd02f640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f80f80d5490 0x7f80f80d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80fd02f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80b92a0 0x7f80f80b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80fd02f640 1 -- --> v1:172.21.15.87:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f80f80d43f0 con 0x7f80f80d2560 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80fd02f640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f80f80d46d0 con 0x7f80f80d5490 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80fd02f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f80f80d49b0 con 0x7f80f80b92a0 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80f6d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f80f80d5490 0x7f80f80d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80f6d9d640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f80f80d5490 0x7f80f80d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37192/0 (socket says 172.21.15.80:37192) 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80f6d9d640 1 -- 172.21.15.80:0/2825612 learned_addr learned my addr 172.21.15.80:0/2825612 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80f759e640 1 --2- 172.21.15.80:0/2825612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80b92a0 0x7f80f80b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80f6d9d640 1 -- 172.21.15.80:0/2825612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80b92a0 msgr2=0x7f80f80b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80f6d9d640 1 --2- 172.21.15.80:0/2825612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80b92a0 0x7f80f80b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.098+0000 7f80f6d9d640 1 -- 172.21.15.80:0/2825612 >> v1:172.21.15.87:6789/0 conn(0x7f80f80d2560 legacy=0x7f80f80d2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:49.096 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.102+0000 7f80f6d9d640 1 -- 172.21.15.80:0/2825612 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f80d4c90 con 0x7f80f80d5490 2024-04-25T13:52:49.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.102+0000 7f80f6d9d640 1 --2- 172.21.15.80:0/2825612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80d5490 0x7f80f80d14a0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f80e8006fd0 tx=0x7f80e801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34c021e1910ab5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:49.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.102+0000 7f80f5d9b640 1 -- 172.21.15.80:0/2825612 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80a95f0 con 0x7f80f80d5490 2024-04-25T13:52:49.098 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.102+0000 7f80f5d9b640 1 -- 172.21.15.80:0/2825612 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e80a9790 con 0x7f80f80d5490 2024-04-25T13:52:49.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.102+0000 7f80f5d9b640 1 -- 172.21.15.80:0/2825612 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80a9b60 con 0x7f80f80d5490 2024-04-25T13:52:49.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.102+0000 7f80fd02f640 1 -- 172.21.15.80:0/2825612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80d5490 msgr2=0x7f80f80d14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:49.099 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.102+0000 7f80fd02f640 1 --2- 172.21.15.80:0/2825612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80d5490 0x7f80f80d14a0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f80e8006fd0 tx=0x7f80e801c510 comp rx=0 tx=0).stop 2024-04-25T13:52:49.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 -- 172.21.15.80:0/2825612 shutdown_connections 2024-04-25T13:52:49.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 --2- 172.21.15.80:0/2825612 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80b92a0 0x7f80f80b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 --2- 172.21.15.80:0/2825612 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80d5490 0x7f80f80d14a0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 -- 172.21.15.80:0/2825612 >> 172.21.15.80:0/2825612 conn(0x7f80f8073740 msgr2=0x7f80f806c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:49.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 -- 172.21.15.80:0/2825612 shutdown_connections 2024-04-25T13:52:49.100 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 -- 172.21.15.80:0/2825612 wait complete. 2024-04-25T13:52:49.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 Processor -- start 2024-04-25T13:52:49.101 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 -- start start 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80b92a0 0x7f80f81523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80d2560 0x7f80f80cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f80d5490 0x7f80f80cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f80f80ba000 con 0x7f80f80b92a0 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f80f80b5140 con 0x7f80f80d5490 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80fd02f640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f80f80ba180 con 0x7f80f80d2560 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80b92a0 0x7f80f81523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f6d9d640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80b92a0 0x7f80f81523a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:37206/0 (socket says 172.21.15.80:37206) 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f6d9d640 1 -- 172.21.15.80:0/3527862443 learned_addr learned my addr 172.21.15.80:0/3527862443 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f659c640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80d2560 0x7f80f80cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:49.104 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f759e640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f80d5490 0x7f80f80cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f6d9d640 1 -- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80d2560 msgr2=0x7f80f80cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f6d9d640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80d2560 0x7f80f80cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f6d9d640 1 -- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f80d5490 msgr2=0x7f80f80cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f6d9d640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f80d5490 0x7f80f80cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f6d9d640 1 -- 172.21.15.80:0/3527862443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f81528e0 con 0x7f80f80b92a0 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.106+0000 7f80f659c640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80d2560 0x7f80f80cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80f6d9d640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80b92a0 0x7f80f81523a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f80e801cdb0 tx=0x7f80e80192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80df7fe640 1 -- 172.21.15.80:0/3527862443 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e80a95f0 con 0x7f80f80b92a0 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80df7fe640 1 -- 172.21.15.80:0/3527862443 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f80e80a9d70 con 0x7f80f80b92a0 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80df7fe640 1 -- 172.21.15.80:0/3527862443 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e8006280 con 0x7f80f80b92a0 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f80b5d90 con 0x7f80f80b92a0 2024-04-25T13:52:49.105 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f80cc4e0 con 0x7f80f80b92a0 2024-04-25T13:52:49.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f806a3a0 con 0x7f80f80b92a0 2024-04-25T13:52:49.106 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80df7fe640 1 -- 172.21.15.80:0/3527862443 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f80e8006420 con 0x7f80f80b92a0 2024-04-25T13:52:49.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80df7fe640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80e406f430 0x7f80e4071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:52:49.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80df7fe640 1 -- 172.21.15.80:0/3527862443 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f80e81296e0 con 0x7f80f80b92a0 2024-04-25T13:52:49.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80f659c640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80e406f430 0x7f80e4071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:52:49.107 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.110+0000 7f80f659c640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80e406f430 0x7f80e4071940 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f80f80d1320 tx=0x7f80ec009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:52:49.110 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.114+0000 7f80df7fe640 1 -- 172.21.15.80:0/3527862443 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f80e80ad020 con 0x7f80f80b92a0 2024-04-25T13:52:49.268 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.274+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f80f80d2d80 con 0x7f80e406f430 2024-04-25T13:52:49.270 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.274+0000 7f80df7fe640 1 -- 172.21.15.80:0/3527862443 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12565 (secure 0 0 0) 0x7f80f80d2d80 con 0x7f80e406f430 2024-04-25T13:52:49.270 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:52:49.271 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:52:49.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80e406f430 msgr2=0x7f80e4071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:49.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80e406f430 0x7f80e4071940 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f80f80d1320 tx=0x7f80ec009040 comp rx=0 tx=0).stop 2024-04-25T13:52:49.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80b92a0 msgr2=0x7f80f81523a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:52:49.272 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80b92a0 0x7f80f81523a0 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7f80e801cdb0 tx=0x7f80e80192b0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 shutdown_connections 2024-04-25T13:52:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f80e406f430 0x7f80e4071940 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f80f80d5490 0x7f80f80cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f80f80d2560 0x7f80f80cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 --2- 172.21.15.80:0/3527862443 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f80f80b92a0 0x7f80f81523a0 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:52:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 >> 172.21.15.80:0/3527862443 conn(0x7f80f8073740 msgr2=0x7f80f8153c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:52:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 shutdown_connections 2024-04-25T13:52:49.273 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:52:49.278+0000 7f80fd02f640 1 -- 172.21.15.80:0/3527862443 wait complete. 2024-04-25T13:52:49.292 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":561,"stamp":"2024-04-25T13:52:48.376279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9732,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552767,"statfs":{"total":1600000000,"available":1590034392,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52428800,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-32,"num_write_kb":-84992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.325366"},"pg_stats":[{"pgid":"1.0","version":"15'72","reported_seq":450,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-25T13:52:40.057855+0000","last_change":"2024-04-25T13:40:19.607916+0000","last_active":"2024-04-25T13:52:40.057855+0000","last_peered":"2024-04-25T13:52:40.057855+0000","last_clean":"2024-04-25T13:52:40.057855+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:52:40.057855+0000","last_undegraded":"2024-04-25T13:52:40.057855+0000","last_fullsized":"2024-04-25T13:52:40.057855+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T13:40:18.501576+0000","last_clean_scrub_stamp":"2024-04-25T13:40:18.501576+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:14.786337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193663,"statfs":{"total":200000000,"available":198311523,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193663,"statfs":{"total":200000000,"available":198311523,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:52:49.295 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T13:40:18.501576+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=13, tm_min=40, tm_sec=18, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=13, tm_min=52, tm_sec=43, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T13:52:49.296 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T13:53:01.165 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:53:01.171 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:53:09.296 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T13:53:09.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d417e640 1 Processor -- start 2024-04-25T13:53:09.457 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d417e640 1 -- start start 2024-04-25T13:53:09.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d417e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89cc0b9240 0x7f89cc0cf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d417e640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f89cc0b7650 0x7f89cc0b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d417e640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89cc0cfb20 con 0x7f89cc0d04e0 2024-04-25T13:53:09.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d417e640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f89cc0b6920 con 0x7f89cc0b7650 2024-04-25T13:53:09.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d417e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc0b6b70 con 0x7f89cc0b9240 2024-04-25T13:53:09.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d1719640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f89cc0d04e0 0x7f89cc0d08f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54090/0 (socket says 172.21.15.80:54090) 2024-04-25T13:53:09.460 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d1719640 1 -- 172.21.15.80:0/3906442036 learned_addr learned my addr 172.21.15.80:0/3906442036 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d271b640 1 --2- 172.21.15.80:0/3906442036 >> v2:172.21.15.87:3300/0 conn(0x7f89cc0b7650 0x7f89cc0b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3026348571 0 0) 0x7f89cc0cfb20 con 0x7f89cc0d04e0 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89b8003530 con 0x7f89cc0d04e0 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d1f1a640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89cc0b9240 0x7f89cc0cf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 671422092 0 0) 0x7f89b8003530 con 0x7f89cc0d04e0 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89cc0b9240 msgr2=0x7f89cc0cf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89cc0b9240 0x7f89cc0cf1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 >> v2:172.21.15.87:3300/0 conn(0x7f89cc0b7650 msgr2=0x7f89cc0b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 --2- 172.21.15.80:0/3906442036 >> v2:172.21.15.87:3300/0 conn(0x7f89cc0b7650 0x7f89cc0b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc0b6e50 con 0x7f89cc0d04e0 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f89c8001a00 con 0x7f89cc0d04e0 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 >> v1:172.21.15.80:6789/0 conn(0x7f89cc0d04e0 legacy=0x7f89cc0d08f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89cc0b9240 0x7f89b80039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.461 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b8003f40 0x7f89b8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89b8004d50 0x7f89b8005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc0b6920 con 0x7f89b8003f40 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc0b6b70 con 0x7f89b8004d50 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.462+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89b800bb60 con 0x7f89cc0b9240 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d1719640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b8003f40 0x7f89b8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d271b640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89b8004d50 0x7f89b8005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d1f1a640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89cc0b9240 0x7f89b80039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d1719640 1 -- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89cc0b9240 msgr2=0x7f89b80039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d1719640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89cc0b9240 0x7f89b80039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d1719640 1 -- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89b8004d50 msgr2=0x7f89b8005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d1719640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89b8004d50 0x7f89b8005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d1719640 1 -- 172.21.15.80:0/3906442036 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b8003530 con 0x7f89b8003f40 2024-04-25T13:53:09.462 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d271b640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89b8004d50 0x7f89b8005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d1719640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b8003f40 0x7f89b8004810 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0c75d0 tx=0x7f89c801d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2f0513b53218f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c800a500 con 0x7f89b8003f40 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c80a5e50 con 0x7f89b8003f40 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d0f18640 1 -- 172.21.15.80:0/3906442036 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c80074f0 con 0x7f89b8003f40 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d417e640 1 -- 172.21.15.80:0/3906442036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b8003f40 msgr2=0x7f89b8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d417e640 1 --2- 172.21.15.80:0/3906442036 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89b8003f40 0x7f89b8004810 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0c75d0 tx=0x7f89c801d270 comp rx=0 tx=0).stop 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d271b640 1 -- 172.21.15.80:0/3906442036 reap_dead start 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d417e640 1 -- 172.21.15.80:0/3906442036 shutdown_connections 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d417e640 1 -- 172.21.15.80:0/3906442036 >> 172.21.15.80:0/3906442036 conn(0x7f89cc0d3ad0 msgr2=0x7f89cc0d3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d417e640 1 -- 172.21.15.80:0/3906442036 shutdown_connections 2024-04-25T13:53:09.464 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.466+0000 7f89d417e640 1 -- 172.21.15.80:0/3906442036 wait complete. 2024-04-25T13:53:09.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d417e640 1 Processor -- start 2024-04-25T13:53:09.465 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d417e640 1 -- start start 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d417e640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89b8003f40 0x7f89cc1fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d417e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc15ec60 0x7f89cc1fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d417e640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89cc06d520 0x7f89cc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d417e640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc1690e0 con 0x7f89cc06d520 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d417e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f89cc168f60 con 0x7f89cc15ec60 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d417e640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f89cc169260 con 0x7f89b8003f40 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89cc06d520 0x7f89cc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89cc06d520 0x7f89cc06d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38792/0 (socket says 172.21.15.80:38792) 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 -- 172.21.15.80:0/4017713191 learned_addr learned my addr 172.21.15.80:0/4017713191 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d1f1a640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89b8003f40 0x7f89cc1fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d1719640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc15ec60 0x7f89cc1fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 -- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89b8003f40 msgr2=0x7f89cc1fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.468 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89b8003f40 0x7f89cc1fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 -- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc15ec60 msgr2=0x7f89cc1fa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc15ec60 0x7f89cc1fa3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 -- 172.21.15.80:0/4017713191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89cc06c760 con 0x7f89cc06d520 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.470+0000 7f89d271b640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89cc06d520 0x7f89cc06d930 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0d0100 tx=0x7f89c4011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89c27fc640 1 -- 172.21.15.80:0/4017713191 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c401d070 con 0x7f89cc06d520 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89c27fc640 1 -- 172.21.15.80:0/4017713191 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89c4004250 con 0x7f89cc06d520 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89cc06ca40 con 0x7f89cc06d520 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89c27fc640 1 -- 172.21.15.80:0/4017713191 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89c4015730 con 0x7f89cc06d520 2024-04-25T13:53:09.469 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89cc06d000 con 0x7f89cc06d520 2024-04-25T13:53:09.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89cc15f7b0 con 0x7f89cc06d520 2024-04-25T13:53:09.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89c27fc640 1 -- 172.21.15.80:0/4017713191 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f89c4020070 con 0x7f89cc06d520 2024-04-25T13:53:09.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89c27fc640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89b006f430 0x7f89b0071940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89c27fc640 1 -- 172.21.15.80:0/4017713191 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f89c408e800 con 0x7f89cc06d520 2024-04-25T13:53:09.470 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89d1f1a640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89b006f430 0x7f89b0071940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.471 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.474+0000 7f89d1f1a640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89b006f430 0x7f89b0071940 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f89bc006fd0 tx=0x7f89bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:09.474 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.478+0000 7f89c27fc640 1 -- 172.21.15.80:0/4017713191 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f89c405ba80 con 0x7f89cc06d520 2024-04-25T13:53:09.632 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.638+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 --> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f89cc06a3a0 con 0x7f89b006f430 2024-04-25T13:53:09.634 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.638+0000 7f89c27fc640 1 -- 172.21.15.80:0/4017713191 <== mgr.4104 v2:172.21.15.87:6832/1633598483 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+12552 (secure 0 0 0) 0x7f89cc06a3a0 con 0x7f89b006f430 2024-04-25T13:53:09.635 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:53:09.635 INFO:teuthology.orchestra.run.smithi080.stderr:dumped all 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89b006f430 msgr2=0x7f89b0071940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89b006f430 0x7f89b0071940 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f89bc006fd0 tx=0x7f89bc008040 comp rx=0 tx=0).stop 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89cc06d520 msgr2=0x7f89cc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89cc06d520 0x7f89cc06d930 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f89cc0d0100 tx=0x7f89c4011920 comp rx=0 tx=0).stop 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 shutdown_connections 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f89b006f430 0x7f89b0071940 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f89cc06d520 0x7f89cc06d930 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89cc15ec60 0x7f89cc1fa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 --2- 172.21.15.80:0/4017713191 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f89b8003f40 0x7f89cc1fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 >> 172.21.15.80:0/4017713191 conn(0x7f89cc0d3ad0 msgr2=0x7f89cc15dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:09.637 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 shutdown_connections 2024-04-25T13:53:09.638 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.642+0000 7f89d417e640 1 -- 172.21.15.80:0/4017713191 wait complete. 2024-04-25T13:53:09.657 INFO:teuthology.orchestra.run.smithi080.stdout:{"pg_ready":true,"pg_map":{"version":571,"stamp":"2024-04-25T13:53:08.382135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9732,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552767,"statfs":{"total":1600000000,"available":1590034392,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003496"},"pg_stats":[{"pgid":"1.0","version":"194'74","reported_seq":461,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-25T13:52:49.224987+0000","last_change":"2024-04-25T13:52:49.225065+0000","last_active":"2024-04-25T13:52:49.224987+0000","last_peered":"2024-04-25T13:52:49.224987+0000","last_clean":"2024-04-25T13:52:49.224987+0000","last_became_active":"2024-04-25T13:40:19.607417+0000","last_became_peered":"2024-04-25T13:40:19.607417+0000","last_unstale":"2024-04-25T13:52:49.224987+0000","last_undegraded":"2024-04-25T13:52:49.224987+0000","last_fullsized":"2024-04-25T13:52:49.224987+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"194'74","last_scrub_stamp":"2024-04-25T13:52:49.224856+0000","last_deep_scrub":"194'74","last_deep_scrub_stamp":"2024-04-25T13:52:49.224856+0000","last_clean_scrub_stamp":"2024-04-25T13:52:49.224856+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:58.647605+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193663,"statfs":{"total":200000000,"available":198311523,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575004,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1072,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194240,"statfs":{"total":200000000,"available":198901891,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575004,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1648,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193663,"statfs":{"total":200000000,"available":198311523,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[]}} 2024-04-25T13:53:09.658 DEBUG:teuthology.orchestra.run.smithi080:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T13:53:09.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a17aba640 1 Processor -- start 2024-04-25T13:53:09.813 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a17aba640 1 -- start start 2024-04-25T13:53:09.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a17aba640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a100d04d0 0x7f1a100d08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a17aba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a1006c920 0x7f1a1006cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a17aba640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f1a1006d270 0x7f1a10169770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a17aba640 1 -- --> v2:172.21.15.80:3300/0 -- mon_getmap magic: 0 -- 0x7f1a100d4880 con 0x7f1a100d04d0 2024-04-25T13:53:09.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a17aba640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f1a100d2700 con 0x7f1a1006d270 2024-04-25T13:53:09.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a17aba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1a100d45a0 con 0x7f1a1006c920 2024-04-25T13:53:09.815 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15856640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a100d04d0 0x7f1a100d08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15856640 1 --2- >> v2:172.21.15.80:3300/0 conn(0x7f1a100d04d0 0x7f1a100d08e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38800/0 (socket says 172.21.15.80:38800) 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15856640 1 -- 172.21.15.80:0/2723353275 learned_addr learned my addr 172.21.15.80:0/2723353275 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a16057640 1 --2- 172.21.15.80:0/2723353275 >> v2:172.21.15.87:3300/0 conn(0x7f1a1006d270 0x7f1a10169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15055640 1 --2- 172.21.15.80:0/2723353275 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a1006c920 0x7f1a1006cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15856640 1 -- 172.21.15.80:0/2723353275 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a1006c920 msgr2=0x7f1a1006cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15856640 1 --2- 172.21.15.80:0/2723353275 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a1006c920 0x7f1a1006cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15856640 1 -- 172.21.15.80:0/2723353275 >> v2:172.21.15.87:3300/0 conn(0x7f1a1006d270 msgr2=0x7f1a10169770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15856640 1 --2- 172.21.15.80:0/2723353275 >> v2:172.21.15.87:3300/0 conn(0x7f1a1006d270 0x7f1a10169770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.816 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.818+0000 7f1a15856640 1 -- 172.21.15.80:0/2723353275 --> v2:172.21.15.80:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a100b5180 con 0x7f1a100d04d0 2024-04-25T13:53:09.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a15856640 1 --2- 172.21.15.80:0/2723353275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a100d04d0 0x7f1a100d08e0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f1a10073ae0 tx=0x7f1a0001c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94be3eba5919f587 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:09.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a14854640 1 -- 172.21.15.80:0/2723353275 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000b2070 con 0x7f1a100d04d0 2024-04-25T13:53:09.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a14854640 1 -- 172.21.15.80:0/2723353275 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a000ad640 con 0x7f1a100d04d0 2024-04-25T13:53:09.818 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a14854640 1 -- 172.21.15.80:0/2723353275 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a000ada10 con 0x7f1a100d04d0 2024-04-25T13:53:09.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 -- 172.21.15.80:0/2723353275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a100d04d0 msgr2=0x7f1a100d08e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 --2- 172.21.15.80:0/2723353275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a100d04d0 0x7f1a100d08e0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7f1a10073ae0 tx=0x7f1a0001c3d0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 -- 172.21.15.80:0/2723353275 shutdown_connections 2024-04-25T13:53:09.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 --2- 172.21.15.80:0/2723353275 >> v2:172.21.15.87:3300/0 conn(0x7f1a1006d270 0x7f1a10169770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 --2- 172.21.15.80:0/2723353275 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a1006c920 0x7f1a1006cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.819 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 --2- 172.21.15.80:0/2723353275 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a100d04d0 0x7f1a100d08e0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 -- 172.21.15.80:0/2723353275 >> 172.21.15.80:0/2723353275 conn(0x7f1a100d3a80 msgr2=0x7f1a100d3e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:09.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 -- 172.21.15.80:0/2723353275 shutdown_connections 2024-04-25T13:53:09.820 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.822+0000 7f1a17aba640 1 -- 172.21.15.80:0/2723353275 wait complete. 2024-04-25T13:53:09.821 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a17aba640 1 Processor -- start 2024-04-25T13:53:09.822 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a17aba640 1 -- start start 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a17aba640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a1006c920 0x7f1a100cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a17aba640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a1006d270 0x7f1a101f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a17aba640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a100d04d0 0x7f1a10154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a17aba640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f1a100b4610 con 0x7f1a1006d270 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a17aba640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1006d680 con 0x7f1a1006c920 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a17aba640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f1a100b4790 con 0x7f1a100d04d0 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a1006d270 0x7f1a101f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a1006d270 0x7f1a101f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38812/0 (socket says 172.21.15.80:38812) 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 -- 172.21.15.80:0/3259958933 learned_addr learned my addr 172.21.15.80:0/3259958933 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15856640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a1006c920 0x7f1a100cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a16057640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a100d04d0 0x7f1a10154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.824 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 -- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a100d04d0 msgr2=0x7f1a10154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a100d04d0 0x7f1a10154e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 -- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a1006c920 msgr2=0x7f1a100cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a1006c920 0x7f1a100cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 -- 172.21.15.80:0/3259958933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a100cb3e0 con 0x7f1a1006d270 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.826+0000 7f1a15055640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a1006d270 0x7f1a101f9720 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f1a04007e90 tx=0x7f1a0400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a0e7fc640 1 -- 172.21.15.80:0/3259958933 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04014420 con 0x7f1a1006d270 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a0e7fc640 1 -- 172.21.15.80:0/3259958933 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a04019070 con 0x7f1a1006d270 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a100cb680 con 0x7f1a1006d270 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a0e7fc640 1 -- 172.21.15.80:0/3259958933 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04014800 con 0x7f1a1006d270 2024-04-25T13:53:09.825 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a100cbc40 con 0x7f1a1006d270 2024-04-25T13:53:09.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a100cdb40 con 0x7f1a1006d270 2024-04-25T13:53:09.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a0e7fc640 1 -- 172.21.15.80:0/3259958933 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f1a040149a0 con 0x7f1a1006d270 2024-04-25T13:53:09.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a0e7fc640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19fc06f400 0x7f19fc071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:09.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a0e7fc640 1 -- 172.21.15.80:0/3259958933 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f1a0408f880 con 0x7f1a1006d270 2024-04-25T13:53:09.826 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a15856640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19fc06f400 0x7f19fc071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:09.827 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.830+0000 7f1a15856640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19fc06f400 0x7f19fc071910 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1a000af000 tx=0x7f1a00008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:09.830 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.834+0000 7f1a0e7fc640 1 -- 172.21.15.80:0/3259958933 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f1a0405ca70 con 0x7f1a1006d270 2024-04-25T13:53:09.991 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.994+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f1a1006a3a0 con 0x7f1a1006d270 2024-04-25T13:53:09.995 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:09.998+0000 7f1a0e7fc640 1 -- 172.21.15.80:0/3259958933 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7f1a0405c370 con 0x7f1a1006d270 2024-04-25T13:53:09.996 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a0e7fc640 1 -- 172.21.15.80:0/3259958933 <== mon.0 v2:172.21.15.80:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f1a0405be80 con 0x7f1a1006d270 2024-04-25T13:53:09.998 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19fc06f400 msgr2=0x7f19fc071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19fc06f400 0x7f19fc071910 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1a000af000 tx=0x7f1a00008040 comp rx=0 tx=0).stop 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a1006d270 msgr2=0x7f1a101f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a1006d270 0x7f1a101f9720 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f1a04007e90 tx=0x7f1a0400e4b0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 shutdown_connections 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f19fc06f400 0x7f19fc071910 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f1a100d04d0 0x7f1a10154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f1a1006d270 0x7f1a101f9720 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 --2- 172.21.15.80:0/3259958933 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1a1006c920 0x7f1a100cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 >> 172.21.15.80:0/3259958933 conn(0x7f1a100d3a80 msgr2=0x7f1a100b86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 shutdown_connections 2024-04-25T13:53:09.999 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.002+0000 7f1a17aba640 1 -- 172.21.15.80:0/3259958933 wait complete. 2024-04-25T13:53:10.018 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T13:53:10.019 DEBUG:teuthology.orchestra.run.smithi080:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T13:53:10.155 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef4763640 1 Processor -- start 2024-04-25T13:53:10.156 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef4763640 1 -- start start 2024-04-25T13:53:10.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef4763640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec0745a0 0x7f9eec06ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.159 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef4763640 1 --2- >> v2:172.21.15.87:3300/0 conn(0x7f9eec06f170 0x7f9eec06f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef4763640 1 -- --> v1:172.21.15.80:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9eec0cd6c0 con 0x7f9eec0cc720 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef4763640 1 -- --> v2:172.21.15.87:3300/0 -- mon_getmap magic: 0 -- 0x7f9eec15ea50 con 0x7f9eec06f170 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef4763640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9eec0729e0 con 0x7f9eec0745a0 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef24ff640 1 --1- >> v1:172.21.15.80:6789/0 conn(0x7f9eec0cc720 0x7f9eec073e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.80:6789/0 says I am v1:172.21.15.80:54098/0 (socket says 172.21.15.80:54098) 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef24ff640 1 -- 172.21.15.80:0/1718630572 learned_addr learned my addr 172.21.15.80:0/1718630572 (peer_addr_for_me v1:172.21.15.80:0/0) 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef2d00640 1 --2- 172.21.15.80:0/1718630572 >> v2:172.21.15.87:3300/0 conn(0x7f9eec06f170 0x7f9eec06f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef1cfe640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec0745a0 0x7f9eec06ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 <== mon.0 v1:172.21.15.80:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2297189246 0 0) 0x7f9eec0cd6c0 con 0x7f9eec0cc720 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 --> v1:172.21.15.80:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9ed8003530 con 0x7f9eec0cc720 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 <== mon.0 v1:172.21.15.80:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2352040793 0 0) 0x7f9ed8003530 con 0x7f9eec0cc720 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec0745a0 msgr2=0x7f9eec06ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec0745a0 0x7f9eec06ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.160 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 >> v2:172.21.15.87:3300/0 conn(0x7f9eec06f170 msgr2=0x7f9eec06f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 --2- 172.21.15.80:0/1718630572 >> v2:172.21.15.87:3300/0 conn(0x7f9eec06f170 0x7f9eec06f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 --> v1:172.21.15.80:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec0cce20 con 0x7f9eec0cc720 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 <== mon.0 v1:172.21.15.80:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 2374588931 0 0) 0x7f9ee8001680 con 0x7f9eec0cc720 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 >> v1:172.21.15.80:6789/0 conn(0x7f9eec0cc720 legacy=0x7f9eec073e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ed80039c0 0x7f9ed8003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed8004810 0x7f9ed8004c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ed8005160 0x7f9ed800b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec15ea50 con 0x7f9ed80039c0 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec0729e0 con 0x7f9ed8004810 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9ed800c010 con 0x7f9ed8005160 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef1cfe640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec0745a0 0x7f9eec06ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef24ff640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ed80039c0 0x7f9ed8003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef2d00640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ed8005160 0x7f9ed800b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.161 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef24ff640 1 -- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ed8005160 msgr2=0x7f9ed800b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:10.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef24ff640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9ed8005160 0x7f9ed800b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef24ff640 1 -- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed8004810 msgr2=0x7f9ed8004c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T13:53:10.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef1cfe640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed8004810 0x7f9ed8004c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef24ff640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed8004810 0x7f9ed8004c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.162 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.162+0000 7f9ef24ff640 1 -- 172.21.15.80:0/1718630572 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ed8003530 con 0x7f9ed80039c0 2024-04-25T13:53:10.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef24ff640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ed80039c0 0x7f9ed8003e30 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f9eec06df30 tx=0x7f9ee801cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5e644aea9e1db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:10.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8007d60 con 0x7f9ed80039c0 2024-04-25T13:53:10.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9ee80a9020 con 0x7f9ed80039c0 2024-04-25T13:53:10.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef14fd640 1 -- 172.21.15.80:0/1718630572 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee8005b00 con 0x7f9ed80039c0 2024-04-25T13:53:10.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef4763640 1 -- 172.21.15.80:0/1718630572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ed80039c0 msgr2=0x7f9ed8003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:10.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef4763640 1 --2- 172.21.15.80:0/1718630572 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9ed80039c0 0x7f9ed8003e30 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f9eec06df30 tx=0x7f9ee801cce0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.163 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef2d00640 1 -- 172.21.15.80:0/1718630572 reap_dead start 2024-04-25T13:53:10.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef4763640 1 -- 172.21.15.80:0/1718630572 shutdown_connections 2024-04-25T13:53:10.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef4763640 1 -- 172.21.15.80:0/1718630572 >> 172.21.15.80:0/1718630572 conn(0x7f9eec0d20d0 msgr2=0x7f9eec0d24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:10.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef4763640 1 -- 172.21.15.80:0/1718630572 shutdown_connections 2024-04-25T13:53:10.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.166+0000 7f9ef4763640 1 -- 172.21.15.80:0/1718630572 wait complete. 2024-04-25T13:53:10.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 Processor -- start 2024-04-25T13:53:10.164 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 -- start start 2024-04-25T13:53:10.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80039c0 0x7f9eec15ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9eec0b1680 0x7f9eec0b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 --2- >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec163cd0 0x7f9eec1640e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.166 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 -- --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec0ccb30 con 0x7f9eec0b1680 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 -- 0x7f9eec06e710 con 0x7f9ed80039c0 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 -- --> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] -- mon_getmap magic: 0 -- 0x7f9eec15f490 con 0x7f9eec163cd0 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9eec0b1680 0x7f9eec0b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 --2- >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9eec0b1680 0x7f9eec0b1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.80:3300/0 says I am v2:172.21.15.80:38816/0 (socket says 172.21.15.80:38816) 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 -- 172.21.15.80:0/622440312 learned_addr learned my addr 172.21.15.80:0/622440312 (peer_addr_for_me v2:172.21.15.80:0/0) 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef2d00640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec163cd0 0x7f9eec1640e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef24ff640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80039c0 0x7f9eec15ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 -- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec163cd0 msgr2=0x7f9eec1640e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec163cd0 0x7f9eec1640e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 -- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80039c0 msgr2=0x7f9eec15ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80039c0 0x7f9eec15ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 -- 172.21.15.80:0/622440312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9eec164620 con 0x7f9eec0b1680 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef2d00640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec163cd0 0x7f9eec1640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:53:10.167 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef24ff640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80039c0 0x7f9eec15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T13:53:10.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef1cfe640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9eec0b1680 0x7f9eec0b1a90 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0107c0 tx=0x7f9edc010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:10.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ee2ffd640 1 -- 172.21.15.80:0/622440312 <== mon.0 v2:172.21.15.80:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00f040 con 0x7f9eec0b1680 2024-04-25T13:53:10.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ee2ffd640 1 -- 172.21.15.80:0/622440312 <== mon.0 v2:172.21.15.80:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f9edc00eae0 con 0x7f9eec0b1680 2024-04-25T13:53:10.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ee2ffd640 1 -- 172.21.15.80:0/622440312 <== mon.0 v2:172.21.15.80:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9edc01c400 con 0x7f9eec0b1680 2024-04-25T13:53:10.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9eec0b2400 con 0x7f9eec0b1680 2024-04-25T13:53:10.168 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9eec0caca0 con 0x7f9eec0b1680 2024-04-25T13:53:10.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.170+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9eec0b2170 con 0x7f9eec0b1680 2024-04-25T13:53:10.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.174+0000 7f9ee2ffd640 1 -- 172.21.15.80:0/622440312 <== mon.0 v2:172.21.15.80:3300/0 4 ==== mgrmap(e 5) ==== 93207+0+0 (secure 0 0 0) 0x7f9edc01c5a0 con 0x7f9eec0b1680 2024-04-25T13:53:10.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.174+0000 7f9ee2ffd640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ed006f400 0x7f9ed0071910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T13:53:10.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.174+0000 7f9ee2ffd640 1 -- 172.21.15.80:0/622440312 <== mon.0 v2:172.21.15.80:3300/0 5 ==== osd_map(194..194 src has 1..194) ==== 4943+0+0 (secure 0 0 0) 0x7f9edc08f5a0 con 0x7f9eec0b1680 2024-04-25T13:53:10.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.174+0000 7f9ef24ff640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ed006f400 0x7f9ed0071910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T13:53:10.169 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.174+0000 7f9ef24ff640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ed006f400 0x7f9ed0071910 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80aa000 tx=0x7f9ee8001a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T13:53:10.173 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.178+0000 7f9ee2ffd640 1 -- 172.21.15.80:0/622440312 <== mon.0 v2:172.21.15.80:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+179799 (secure 0 0 0) 0x7f9edc018030 con 0x7f9eec0b1680 2024-04-25T13:53:10.358 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.362+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 --> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9eec0ca7b0 con 0x7f9eec0b1680 2024-04-25T13:53:10.359 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.362+0000 7f9ee2ffd640 1 -- 172.21.15.80:0/622440312 <== mon.0 v2:172.21.15.80:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+3863 (secure 0 0 0) 0x7f9edc05c790 con 0x7f9eec0b1680 2024-04-25T13:53:10.359 INFO:teuthology.orchestra.run.smithi080.stdout: 2024-04-25T13:53:10.360 INFO:teuthology.orchestra.run.smithi080.stdout:{"epoch":91,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/66879732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":66879732},{"type":"v1","addr":"172.21.15.87:6835","nonce":66879732}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7264,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/2209525025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":2209525025},{"type":"v1","addr":"172.21.15.87:6837","nonce":2209525025}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9204,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6833/4143719384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6832","nonce":4143719384},{"type":"v1","addr":"172.21.15.80:6833","nonce":4143719384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":9594,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.80:6835/3475103033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.80:6834","nonce":3475103033},{"type":"v1","addr":"172.21.15.80:6835","nonce":3475103033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":91}],"filesystems":[]} 2024-04-25T13:53:10.360 INFO:teuthology.orchestra.run.smithi080.stderr:dumped fsmap epoch 91 2024-04-25T13:53:10.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ed006f400 msgr2=0x7f9ed0071910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:10.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ed006f400 0x7f9ed0071910 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9ee80aa000 tx=0x7f9ee8001a50 comp rx=0 tx=0).stop 2024-04-25T13:53:10.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9eec0b1680 msgr2=0x7f9eec0b1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T13:53:10.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9eec0b1680 0x7f9eec0b1a90 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f9edc0107c0 tx=0x7f9edc010c80 comp rx=0 tx=0).stop 2024-04-25T13:53:10.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 shutdown_connections 2024-04-25T13:53:10.362 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:6832/1633598483,v1:172.21.15.87:6833/1633598483] conn(0x7f9ed006f400 0x7f9ed0071910 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3301/0,v1:172.21.15.87:6790/0] conn(0x7f9eec163cd0 0x7f9eec1640e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.80:3300/0,v1:172.21.15.80:6789/0] conn(0x7f9eec0b1680 0x7f9eec0b1a90 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 --2- 172.21.15.80:0/622440312 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9ed80039c0 0x7f9eec15ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T13:53:10.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 >> 172.21.15.80:0/622440312 conn(0x7f9eec0d20d0 msgr2=0x7f9eec0b2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T13:53:10.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 shutdown_connections 2024-04-25T13:53:10.363 INFO:teuthology.orchestra.run.smithi080.stderr:2024-04-25T13:53:10.366+0000 7f9ef4763640 1 -- 172.21.15.80:0/622440312 wait complete. 2024-04-25T13:53:10.382 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T13:53:10.382 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T13:53:10.382 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T13:53:10.382 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.401 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T13:53:10.401 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T13:53:10.401 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.461 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T13:53:10.462 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T13:53:10.462 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.483 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T13:53:10.483 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T13:53:10.483 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.539 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T13:53:10.539 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T13:53:10.540 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T13:53:10.540 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.582 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T13:53:10.582 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T13:53:10.582 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.631 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T13:53:10.631 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T13:53:10.631 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.666 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T13:53:10.666 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T13:53:10.666 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.732 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T13:53:10.733 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T13:53:10.733 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.777 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T13:53:10.778 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T13:53:10.778 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.803 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T13:53:10.803 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T13:53:10.804 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.869 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T13:53:10.869 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T13:53:10.869 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.933 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T13:53:10.933 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T13:53:10.934 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T13:53:10.934 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:10.962 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T13:53:10.962 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T13:53:10.962 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:11.007 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T13:53:11.008 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T13:53:11.008 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T13:53:11.008 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:11.042 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T13:53:11.042 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T13:53:11.042 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:11.064 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T13:53:11.064 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T13:53:11.064 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T13:53:11.124 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T13:53:11.125 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T13:53:11.125 DEBUG:teuthology.orchestra.run.smithi080:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2024-04-25T13:53:11.146 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:53:11.146 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T13:53:11.572 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:53:11.573 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T13:53:11.838 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:53:11.838 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T13:53:12.022 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:53:12.022 DEBUG:teuthology.orchestra.run.smithi080:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T13:53:12.222 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:53:12.222 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T13:53:12.484 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:53:12.484 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T13:53:12.698 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:53:12.698 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T13:53:12.882 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:53:12.882 DEBUG:teuthology.orchestra.run.smithi087:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T13:53:13.078 INFO:tasks.ceph:Archiving mon data... 2024-04-25T13:53:13.080 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/data/mon.a.tgz 2024-04-25T13:53:13.080 DEBUG:teuthology.orchestra.run.smithi080:> mktemp 2024-04-25T13:53:13.086 INFO:teuthology.orchestra.run.smithi080.stdout:/tmp/tmp.q3St44zGKx 2024-04-25T13:53:13.086 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.q3St44zGKx 2024-04-25T13:53:13.238 DEBUG:teuthology.orchestra.run.smithi080:> sudo chmod 0666 /tmp/tmp.q3St44zGKx 2024-04-25T13:53:13.299 DEBUG:teuthology.orchestra.remote:smithi080:/tmp/tmp.q3St44zGKx is 484KB 2024-04-25T13:53:13.397 DEBUG:teuthology.orchestra.run.smithi080:> rm -fr /tmp/tmp.q3St44zGKx 2024-04-25T13:53:13.403 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/data/mon.b.tgz 2024-04-25T13:53:13.403 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-04-25T13:53:13.409 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.3fXp5Y4prQ 2024-04-25T13:53:13.409 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.3fXp5Y4prQ 2024-04-25T13:53:13.569 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.3fXp5Y4prQ 2024-04-25T13:53:13.633 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.3fXp5Y4prQ is 505KB 2024-04-25T13:53:13.699 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.3fXp5Y4prQ 2024-04-25T13:53:13.704 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/data/mon.c.tgz 2024-04-25T13:53:13.704 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-04-25T13:53:13.753 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.0a61ERDlmF 2024-04-25T13:53:13.754 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.0a61ERDlmF 2024-04-25T13:53:13.923 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.0a61ERDlmF 2024-04-25T13:53:13.984 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.0a61ERDlmF is 505KB 2024-04-25T13:53:14.047 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.0a61ERDlmF 2024-04-25T13:53:14.055 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T13:53:14.056 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T13:53:14.058 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T13:53:14.116 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T13:53:14.157 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T13:53:14.157 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T13:53:14.159 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/remote/smithi080/crash 2024-04-25T13:53:14.159 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T13:53:14.173 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/remote/smithi087/crash 2024-04-25T13:53:14.174 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T13:53:14.213 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T13:53:14.213 DEBUG:teuthology.orchestra.run.smithi080:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:53:14.221 DEBUG:teuthology.orchestra.run.smithi087:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T13:53:14.296 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T13:53:14.315 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T13:53:14.352 INFO:tasks.ceph:Compressing logs... 2024-04-25T13:53:14.352 DEBUG:teuthology.orchestra.run.smithi080:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:53:14.361 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T13:53:14.389 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36882.log 2024-04-25T13:53:14.390 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28654.log 2024-04-25T13:53:14.391 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26418.log 2024-04-25T13:53:14.391 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36882.log: /var/log/ceph/ceph-client.admin.28654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31282.log 2024-04-25T13:53:14.394 INFO:teuthology.orchestra.run.smithi080.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28654.log.gz 2024-04-25T13:53:14.394 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41391.log 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26418.log: /var/log/ceph/ceph-client.admin.31282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36674.log 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.26418.log.gz 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.36882.log.gz 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6%/var/log/ceph/ceph-client.admin.41391.log: -- replaced with /var/log/ceph/ceph-client.admin.31282.log.gz 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34529.log 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41391.log.gz 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35619.log 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34529.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36674.log.gz 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37678.log 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43141.log 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34529.log.gz 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35619.log.gz 2024-04-25T13:53:14.395 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24921.log 2024-04-25T13:53:14.396 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.37678.log.gz 2024-04-25T13:53:14.396 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39299.log 2024-04-25T13:53:14.396 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43141.log.gz 2024-04-25T13:53:14.396 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25829.log 2024-04-25T13:53:14.400 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42829.log 2024-04-25T13:53:14.400 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34810.log/var/log/ceph/ceph-client.admin.39299.log: 2024-04-25T13:53:14.400 INFO:teuthology.orchestra.run.smithi080.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.24921.log.gz 2024-04-25T13:53:14.400 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25829.log: /var/log/ceph/ceph-client.admin.42829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23719.log 2024-04-25T13:53:14.400 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39299.log.gz 2024-04-25T13:53:14.400 INFO:teuthology.orchestra.run.smithi080.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.25829.log.gz 2024-04-25T13:53:14.400 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42829.log.gz 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33395.log 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23719.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34810.log.gz 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43375.log 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.23719.log.gz 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35432.log 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43375.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33395.log.gz 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41828.log 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35432.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43375.log.gz 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35432.log.gz 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28414.log 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28090.log 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41828.log.gz 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34005.log 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35314.log 2024-04-25T13:53:14.401 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28090.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28414.log.gz 2024-04-25T13:53:14.402 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34005.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T13:53:14.402 INFO:teuthology.orchestra.run.smithi080.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.28090.log.gz 2024-04-25T13:53:14.402 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34005.log.gz 2024-04-25T13:53:14.402 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41854.log 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35314.log.gz 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26972.log 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25154.log 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37068.log 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41854.log.gz 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25154.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26972.log.gz 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37386.log 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.25154.log.gz 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35100.log 2024-04-25T13:53:14.415 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34477.log 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37068.log.gz 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35100.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.37386.log.gz 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27284.log 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35100.log.gz 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43089.log 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43401.log 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34477.log.gz 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27284.log.gz 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33215.log 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43089.log.gz 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43510.log 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr:gzip/var/log/ceph/ceph-client.admin.33215.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.33687.log 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43401.log.gz 2024-04-25T13:53:14.416 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43510.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33215.log.gz 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44091.log 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43510.log.gz 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29693.log 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27128.log 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33687.log.gz 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29693.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44091.log.gz 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27258.log 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29693.log.gz 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26681.log 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23829.log 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27128.log.gz 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26681.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27258.log.gz 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27024.log 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26681.log.gz 2024-04-25T13:53:14.417 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23803.log 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36081.log 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.23829.log.gz 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23803.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27024.log.gz 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30035.log 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.23803.log.gz 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39273.log 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35340.log 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36081.log.gz 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39273.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30035.log.gz 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44776.log 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39273.log.gz 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38416.log 2024-04-25T13:53:14.418 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41313.log 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35340.log.gz 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38416.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44776.log.gz 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42511.log 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38416.log.gz 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41776.log 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43784.log 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41313.log.gz 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41776.log: 94.5% 88.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42511.log.gz 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40753.log 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41776.log.gz 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27566.log 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40753.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43784.log.gz 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27538.log 2024-04-25T13:53:14.419 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40753.log.gz 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37183.log 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27566.log: /var/log/ceph/ceph-client.admin.27538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40673.log 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27566.log.gz 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37183.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27538.log.gz 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37417.log 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37183.log.gz 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.40673.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45103.log 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37417.log: /var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39977.log 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37417.log.gz 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28969.log 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45103.log.gz 2024-04-25T13:53:14.420 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32301.log 2024-04-25T13:53:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23393.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23290.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.25409.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.23393.log: /var/log/ceph/ceph-client.admin.23290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.24943.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.23290.log.gz 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.25409.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.23393.log.gz 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.24943.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23218.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23338.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.1.24943.log.gz 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.23218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.25767.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.23928.log 2024-04-25T13:53:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.23338.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.23218.log.gz 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.25767.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.23338.log.gz 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.23928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23314.log 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr: 94.9% -- replaced with /var/log/ceph/ceph-client.1.25409.log.gz 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.24588.log 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.1.23928.log.gz 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.1.25767.log.gz 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.23314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.24047.log 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr: 88.3%/var/log/ceph/ceph-client.1.24588.log: -- replaced with /var/log/ceph/ceph-client.admin.23314.log.gz 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.3.24047.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23242.log 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr: 90.8%gzip -- replaced with /var/log/ceph/ceph-client.3.24047.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.23369.log 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-04-25T13:53:14.444 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.23242.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.23242.log.gz 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr: 93.3% -- replaced with /var/log/ceph/ceph-client.1.24588.log.gz 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.24238.log 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.23369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.25251.log 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.23369.log.gz 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.24238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.25587.log 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.25251.log: /var/log/ceph/ceph-client.1.25587.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mon.c.log: 89.2% -- replaced with /var/log/ceph/ceph-client.1.24238.log.gz 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph.audit.log: 91.8% -- replaced with /var/log/ceph/ceph-client.1.25251.log.gz 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr: 93.1% -- replaced with /var/log/ceph/ceph-client.1.25587.log.gz 2024-04-25T13:53:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.24433.log 2024-04-25T13:53:14.450 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.25098.log 2024-04-25T13:53:14.450 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.24433.log: 94.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 91.8% -- replaced with /var/log/ceph/ceph-client.1.24433.log.gz 2024-04-25T13:53:14.450 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-04-25T13:53:14.450 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T13:53:14.451 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.25098.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T13:53:14.451 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mds.b.log: 89.2% -- replaced with /var/log/ceph/ceph-client.1.25098.log.gz 2024-04-25T13:53:14.451 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23266.log 2024-04-25T13:53:14.472 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T13:53:14.472 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.admin.23266.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.23266.log.gz 2024-04-25T13:53:14.478 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.24761.log 2024-04-25T13:53:14.479 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-osd.4.log: 91.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T13:53:14.495 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/ceph-client.1.24761.log: 91.4% -- replaced with /var/log/ceph/ceph-client.1.24761.log.gz 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36700.log 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28969.log: /var/log/ceph/ceph-client.admin.32301.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39977.log.gz 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30966.log 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28969.log.gz 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32301.log.gz 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29771.log 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38272.log 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30966.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36700.log.gz 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29771.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30966.log.gz 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39540.log 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29771.log.gz 2024-04-25T13:53:14.554 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45597.log 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45311.log 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39540.log: /var/log/ceph/ceph-client.admin.45597.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38272.log.gz 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26366.log 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39540.log.gz 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45597.log.gz 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35196.log 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27856.log 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26366.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45311.log.gz 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38524.log 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.26366.log.gz 2024-04-25T13:53:14.555 INFO:teuthology.orchestra.run.smithi080.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35196.log.gz 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38962.log 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.27856.log.gz 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23208.log 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38524.log: /var/log/ceph/ceph-client.admin.38962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30713.log 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38962.log.gz 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38524.log.gz 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37730.log 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.23208.log.gz 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39644.log/var/log/ceph/ceph-client.admin.30713.log: 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26764.log 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30713.log.gz 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37730.log.gz 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25489.log 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26764.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39644.log.gz 2024-04-25T13:53:14.556 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45233.log 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.26764.log.gz 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30659.log 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25489.log: /var/log/ceph/ceph-client.admin.45233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41592.log 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.25489.log.gz 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30659.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45233.log.gz 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39592.log 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30659.log.gz 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31586.log 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41592.log: /var/log/ceph/ceph-client.admin.39592.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41592.log.gz 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39103.log 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39592.log.gz 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41540.log 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31586.log: /var/log/ceph/ceph-client.admin.39103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27433.log 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31586.log.gz 2024-04-25T13:53:14.557 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39103.log.gz 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41339.log 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27433.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41540.log.gz 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27433.log.gz 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44567.log 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40974.log 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41339.log: /var/log/ceph/ceph-client.admin.44567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36988.log 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40974.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41339.log.gz 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44567.log.gz 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37600.log 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40974.log.gz 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.36988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24739.log 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41436.log 2024-04-25T13:53:14.558 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24739.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37600.log.gz 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28519.log 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.24739.log.gz 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34676.log 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29416.log 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28519.log: /var/log/ceph/ceph-client.admin.34676.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41436.log.gz 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24583.log 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28519.log.gz 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34676.log.gz 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29416.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29416.log.gz 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.28248.log 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.36988.log.gz 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41026.log 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.24583.log.gz 2024-04-25T13:53:14.559 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.28248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40727.log 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33163.log 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41026.log.gz 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36271.log 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40727.log: /var/log/ceph/ceph-client.admin.33163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28571.log 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40727.log.gz 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33163.log.gz 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34784.log 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36271.log.gz 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33875.log 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34784.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28571.log.gz 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30992.log 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.34784.log.gz 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.28248.log.gz 2024-04-25T13:53:14.560 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42358.log 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30992.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33875.log.gz 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25944.log 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42358.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30992.log.gz 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29548.log 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42358.log.gz 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30104.log 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25944.log.gz 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35484.log 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29548.log: /var/log/ceph/ceph-client.admin.30104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41078.log 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.30104.log.gz 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29548.log.gz 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35672.log 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41078.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35484.log.gz 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40783.log 2024-04-25T13:53:14.561 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41078.log.gz 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5/var/log/ceph/ceph-client.admin.35672.log: --verbose -- /var/log/ceph/ceph-client.admin.32801.log 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:gzip/var/log/ceph/ceph-client.admin.40783.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40617.log 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35672.log.gz 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32801.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40783.log.gz 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38193.log 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32801.log.gz 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43167.log 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40617.log.gz 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44646.log 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38193.log.gz 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43167.log: /var/log/ceph/ceph-client.admin.44646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25541.log 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44646.log.gz 2024-04-25T13:53:14.562 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43167.log.gz 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35366.log 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25541.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23959.log 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25541.log.gz 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr:gzip/var/log/ceph/ceph-client.admin.35366.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25411.log 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35458.log 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35366.log.gz 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25411.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.23959.log.gz 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24136.log 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.25411.log.gz 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28192.log 2024-04-25T13:53:14.563 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35458.log.gz 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40010.log 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24136.log.gz 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26920.log 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28192.log: /var/log/ceph/ceph-client.admin.40010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24976.log 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40010.log.gz 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28192.log.gz 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33319.log 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26920.log.gz 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44146.log 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30248.log 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.24976.log.gz 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44146.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.33319.log.gz 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33111.log 2024-04-25T13:53:14.564 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30248.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44146.log.gz 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27076.log 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30248.log.gz 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45285.log 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38011.log 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33111.log.gz 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45285.log: 94.1% -- replaced with /var/log/ceph/ceph-client.0.40673.log.gz 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27076.log.gz 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42620.log 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45285.log.gz 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32327.log 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38011.log.gz 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26392.log 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32327.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42620.log.gz 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37360.log 2024-04-25T13:53:14.565 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32327.log.gz 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30130.log 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26392.log.gz 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24869.log 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30130.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.37360.log.gzgzip 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.27752.log 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30130.log.gz 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24338.log 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27752.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.24869.log.gz 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30445.log 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27752.log.gz 2024-04-25T13:53:14.566 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43732.log 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30445.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.38446.log 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.24338.log.gz 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43732.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30445.log.gz 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43732.log.gz 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43916.log 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.2.log: 89.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.38446.log.gz 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.34611.log 2024-04-25T13:53:14.567 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.43916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33635.log 2024-04-25T13:53:14.568 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37652.log 2024-04-25T13:53:14.568 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34611.log.gz 2024-04-25T13:53:14.568 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39899.log/var/log/ceph/ceph-client.admin.33635.log: 2024-04-25T13:53:14.568 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27724.log 2024-04-25T13:53:14.568 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33635.log.gz 2024-04-25T13:53:14.568 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39899.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.22758.log 2024-04-25T13:53:14.568 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37652.log.gz 2024-04-25T13:53:14.568 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27724.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39899.log.gz 2024-04-25T13:53:14.569 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26707.log 2024-04-25T13:53:14.569 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.27724.log.gz 2024-04-25T13:53:14.569 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.tmp-client.admin.22758.log: 0.0%gzip -- replaced with /var/log/ceph/ceph.tmp-client.admin.22758.log.gz 2024-04-25T13:53:14.569 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33901.log 2024-04-25T13:53:14.569 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24557.log 2024-04-25T13:53:14.569 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33901.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26707.log.gz 2024-04-25T13:53:14.569 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44802.log 2024-04-25T13:53:14.569 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33901.log.gz 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34940.log 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24557.log: /var/log/ceph/ceph-client.admin.44802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35958.log 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.24557.log.gz 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34940.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44802.log.gz 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39357.log 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45003.log 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35958.log.gz 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-04-25T13:53:14.570 INFO:teuthology.orchestra.run.smithi080.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.34940.log.gz 2024-04-25T13:53:14.571 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30367.log 2024-04-25T13:53:14.571 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39357.log.gz 2024-04-25T13:53:14.571 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45003.log.gz 2024-04-25T13:53:14.571 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29203.log 2024-04-25T13:53:14.571 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-04-25T13:53:14.571 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42751.log 2024-04-25T13:53:14.571 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1%/var/log/ceph/ceph-client.admin.29203.log: -- replaced with /var/log/ceph/ceph-client.admin.30367.log.gz 2024-04-25T13:53:14.571 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41157.log 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42751.log: gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29203.log.gz -5 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40541.log 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42751.log.gz 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36856.log 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41157.log.gz 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32144.log 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40541.log: /var/log/ceph/ceph-client.admin.36856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33953.log 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40541.log.gz 2024-04-25T13:53:14.572 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32144.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36856.log.gz 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28839.log 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2%/var/log/ceph/ceph-client.admin.33953.log: -- replaced with /var/log/ceph/ceph-client.admin.32144.log.gz 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24141.log 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33953.log.gz 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32509.log 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28839.log.gz 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29719.log 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.24141.log.gz 2024-04-25T13:53:14.573 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43628.log 2024-04-25T13:53:14.574 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32509.log.gz 2024-04-25T13:53:14.574 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28142.log 2024-04-25T13:53:14.574 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29719.log: /var/log/ceph/ceph-client.admin.43628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26472.log 2024-04-25T13:53:14.574 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29719.log.gz 2024-04-25T13:53:14.574 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43628.log.gz 2024-04-25T13:53:14.574 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27696.log 2024-04-25T13:53:14.574 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40268.log 2024-04-25T13:53:14.574 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27696.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.28142.log.gz 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26472.log.gz 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.27696.log.gz 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29285.log 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40268.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28388.log 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40268.log.gz 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28473.log 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29285.log.gz 2024-04-25T13:53:14.575 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45337.log 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr:gzip 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28388.log.gz 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28473.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.24140.log 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45337.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34161.log 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.28473.log.gz 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45337.log.gz 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32925.log 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24140.log: /var/log/ceph/ceph-client.admin.34161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32736.log 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24140.log.gz 2024-04-25T13:53:14.576 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.34161.log.gz 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23458.log 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45650.log 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32925.log.gz 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.32736.log.gz 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37808.log 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45650.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26894.log 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.23458.log.gz 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45650.log.gz 2024-04-25T13:53:14.577 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42777.log 2024-04-25T13:53:14.578 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39748.log 2024-04-25T13:53:14.578 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5%/var/log/ceph/ceph-client.admin.26894.log: -- replaced with /var/log/ceph/ceph-client.admin.37808.log.gz 2024-04-25T13:53:14.578 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31612.log 2024-04-25T13:53:14.578 INFO:teuthology.orchestra.run.smithi080.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42777.log.gz 2024-04-25T13:53:14.578 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.26894.log.gz 2024-04-25T13:53:14.578 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32061.log 2024-04-25T13:53:14.578 INFO:teuthology.orchestra.run.smithi080.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39748.log.gz 2024-04-25T13:53:14.578 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41684.log 2024-04-25T13:53:14.579 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31612.log.gz 2024-04-25T13:53:14.579 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45467.log 2024-04-25T13:53:14.579 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5/var/log/ceph/ceph-client.admin.41684.log: --verbose -- /var/log/ceph/ceph-client.admin.39462.log 2024-04-25T13:53:14.579 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32061.log.gz 2024-04-25T13:53:14.579 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45467.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37626.log 2024-04-25T13:53:14.579 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41684.log.gz 2024-04-25T13:53:14.579 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45467.log.gz 2024-04-25T13:53:14.579 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32457.log 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39462.log.gz 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44828.log 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37626.log: /var/log/ceph/ceph-client.admin.32457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43545.log 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37626.log.gz 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32457.log.gz 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25463.log 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44828.log.gz 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31217.log 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- 89.3% /var/log/ceph/ceph-client.admin.42563.log 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43545.log.gz 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25463.log: /var/log/ceph/ceph-client.admin.31217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31902.log 2024-04-25T13:53:14.580 INFO:teuthology.orchestra.run.smithi080.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.25463.log.gz 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31217.log.gz 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29151.log 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42563.log.gz 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33003.log 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31902.log.gz 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5/var/log/ceph/ceph-client.admin.29151.log: --verbose -- /var/log/ceph/ceph-client.admin.27154.log 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34557.log 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29151.log.gz 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33003.log.gz 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42646.log 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27154.log.gz 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34135.log 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32379.log 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34557.log.gz 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42646.log: /var/log/ceph/ceph-client.admin.34135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38991.log 2024-04-25T13:53:14.581 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42646.log.gz 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34135.log.gz 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36449.log 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32379.log.gz 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41566.log 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38991.log.gz 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41365.log 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41566.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36449.log.gz 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34503.log 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41566.log.gz 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23775.log 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41365.log.gz 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32977.log 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38576.log 2024-04-25T13:53:14.582 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34503.log.gz 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24346.log 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.23775.log.gz 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32977.log.gz 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34265.log 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24346.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34213.log 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38576.log.gz 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.24346.log.gz 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23667.log 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34265.log.gz 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23641.log 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34213.log: /var/log/ceph/ceph-client.admin.23667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38550.log 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34213.log.gz 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28623.log 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.23667.log.gz 2024-04-25T13:53:14.583 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38298.log 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.23641.log.gz 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38550.log: /var/log/ceph/ceph-client.admin.28623.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38550.log.gz 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28623.log.gz 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31478.log 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41802.log 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38298.log.gz 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31122.log 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37307.log 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31478.log.gz 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31122.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41802.log.gz 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40042.log 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.31122.log.gz 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43115.log 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37307.log: /var/log/ceph/ceph-client.admin.40042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38602.log 2024-04-25T13:53:14.584 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40042.log.gz 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37307.log.gz 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31185.log 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43115.log: 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44672.log 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43115.log.gz 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27512.log 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.38602.log.gz 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24817.log 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44672.log: -- replaced with /var/log/ceph/ceph-client.admin.31185.log.gz 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:gzip/var/log/ceph/ceph-client.admin.27512.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39017.log 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44672.log.gz 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27830.log 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27512.log.gz 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.24817.log.gz 2024-04-25T13:53:14.585 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34057.log 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39017.log.gz 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44698.log 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27830.log.gz 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31981.log 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34057.log.gz 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32118.log 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44698.log.gz 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30327.log 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31981.log.gz 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36596.log 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32118.log.gz 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43252.log 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36596.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30327.log.gz 2024-04-25T13:53:14.586 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26842.log 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36596.log.gz 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39435.log 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26842.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35698.log 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43252.log.gz 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39435.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26842.log.gz 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37157.log 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39435.log.gz 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41652.log 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35698.log: /var/log/ceph/ceph-client.admin.37157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39873.log 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35698.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26127.log 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37157.log.gz 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41652.log: /var/log/ceph/ceph-client.admin.39873.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41652.log.gz 2024-04-25T13:53:14.587 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39873.log.gz 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29177.log 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23881.log 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26127.log: /var/log/ceph/ceph-client.admin.29177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28813.log 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26127.log.gz 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28680.log 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29177.log.gz 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34291.log 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.23881.log.gz 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42725.log 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28813.log.gz 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28680.log: /var/log/ceph/ceph-client.admin.34291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24791.log 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28680.log.gz 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41462.log 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34291.log.gz 2024-04-25T13:53:14.588 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44515.log 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42725.log.gz 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24791.log: /var/log/ceph/ceph-client.admin.41462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43428.log 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.24791.log.gz 88.6% 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -- replaced with /var/log/ceph/ceph-client.admin.41462.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.27592.log 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44515.log: /var/log/ceph/ceph-client.admin.43428.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44515.log.gz 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43428.log.gz 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43037.log 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28760.log 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8%/var/log/ceph/ceph-client.admin.43037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34862.log 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27592.log.gz 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34347.log 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43037.log.gz 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34862.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28760.log.gz 2024-04-25T13:53:14.589 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38128.log 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34862.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36343.log 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34347.log: /var/log/ceph/ceph-client.admin.38128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36395.log 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.34347.log.gz 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24351.log 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38128.log.gz/var/log/ceph/ceph-client.admin.36343.log: 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45626.log 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36343.log.gz 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39129.log 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36395.log.gz 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24351.log: /var/log/ceph/ceph-client.admin.45626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26262.log 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33796.log 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% 87.9% -- replaced with /var/log/ceph/ceph-client.admin.24351.log.gz 2024-04-25T13:53:14.590 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45626.log.gz 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26262.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39129.log.gz 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.43916.log.gz 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35802.log 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26262.log.gz 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45259.log 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35802.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33796.log.gz 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.33451.log 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45259.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35802.log.gz 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38327.log 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45259.log.gz 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31443.log 2024-04-25T13:53:14.591 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.33451.log: /var/log/ceph/ceph-client.admin.38327.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31443.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38327.log.gz 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24132.log 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44065.log 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31443.log.gz 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37131.log 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44065.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.24132.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32768.log 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44065.log.gz 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37131.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37131.log.gz 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39383.log 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26603.log 2024-04-25T13:53:14.592 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39383.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.32768.log.gz 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42855.log 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41105.log 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39383.log.gz 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26603.log: /var/log/ceph/ceph-client.admin.42855.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.26603.log.gz 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42855.log.gz 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29021.log 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38934.log 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4%/var/log/ceph/ceph-client.admin.29021.log: -- replaced with /var/log/ceph/ceph-client.admin.41105.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40303.log 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.29021.log.gz 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38934.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38934.log.gz 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33031.log 2024-04-25T13:53:14.593 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33293.log 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33031.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40303.log.gz 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26236.log 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39670.log 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33031.log.gz 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33293.log: /var/log/ceph/ceph-client.admin.26236.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33293.log.gz 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26236.log.gz 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40567.log 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35645.log 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36107.log 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40567.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39670.log.gz 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39330.log 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% 91.0% -- replaced with /var/log/ceph/ceph-client.admin.40567.log.gz 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35645.log.gz 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36107.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36107.log.gz 2024-04-25T13:53:14.594 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43454.log 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30685.log 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43454.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39330.log.gz 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36622.log 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.43454.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.44311.log 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30685.log: /var/log/ceph/ceph-client.admin.36622.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.30685.log.gz 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36622.log.gz 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29099.log 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35406.log 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44311.log: /var/log/ceph/ceph-client.admin.29099.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27778.log 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44311.log.gz 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29099.log.gz 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35406.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35406.log.gz 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40329.log 2024-04-25T13:53:14.595 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31690.log 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8%/var/log/ceph/ceph-client.admin.40329.log: -- replaced with /var/log/ceph/ceph-client.admin.27778.log.gz 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37574.log 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40329.log.gz 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31690.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31690.log.gz 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23745.log 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26210.log 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23745.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38871.log 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37574.log.gz 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.23745.log.gz 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26210.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.26210.log.gz 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41235.log 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27618.log 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6%/var/log/ceph/ceph-client.admin.41235.log: -- replaced with /var/log/ceph/ceph-client.admin.38871.log.gzgzip 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42410.log 2024-04-25T13:53:14.596 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41235.log.gz 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27618.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27618.log.gz 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29255.log 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34031.log 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29255.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42410.log.gz 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45493.log 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33770.log 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29255.log.gz 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34031.log: /var/log/ceph/ceph-client.admin.45493.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34031.log.gz 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45493.log.gz 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35932.log 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34641.log 2024-04-25T13:53:14.597 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1%/var/log/ceph/ceph-client.admin.35932.log: -- replaced with /var/log/ceph/ceph-client.admin.33770.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26101.log 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35932.log.gz 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34641.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34641.log.gz 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42068.log 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28440.log 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3%/var/log/ceph/ceph-client.admin.42068.log: -- replaced with /var/log/ceph/ceph-client.admin.26101.log.gz 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37548.log 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42068.log.gz 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28440.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28440.log.gz 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44430.log 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29468.log 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6%/var/log/ceph/ceph-client.admin.44430.log: -- replaced with /var/log/ceph/ceph-client.admin.37548.log.gz 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24347.log 2024-04-25T13:53:14.598 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44430.log.gz 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29468.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29468.log.gz 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42933.log 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5%/var/log/ceph/ceph-client.admin.42933.log: -- replaced with /var/log/ceph/ceph-client.admin.24347.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24335.log 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42933.log.gz 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38355.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26314.log 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30765.log 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr: 88.5%/var/log/ceph/ceph-client.admin.26314.log: -- replaced with /var/log/ceph/ceph-client.admin.24335.log.gz 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37704.log 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.26314.log.gz 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30765.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30765.log.gz 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29611.log 2024-04-25T13:53:14.599 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.25104.log 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29611.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.37704.log.gz 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41910.log 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38908.log 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.29611.log.gz 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41910.log: /var/log/ceph/ceph-client.0.25104.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41910.log.gz 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.25104.log.gz 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42172.log 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38498.log 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr: 89.4%/var/log/ceph/ceph-client.admin.42172.log: -- replaced with /var/log/ceph/ceph-client.admin.38908.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27050.log 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42172.log.gz 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38498.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38498.log.gz 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T13:53:14.600 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28012.log 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-osd.1.log: 88.1%gzip -- replaced with /var/log/ceph/ceph-client.admin.27050.log.gz 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.45077.log 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28012.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28012.log.gz 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31792.log 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27804.log 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45077.log: /var/log/ceph/ceph-client.admin.31792.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45077.log.gz 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31792.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36726.log 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27804.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27804.log.gz 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26153.log 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38096.log 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36726.log: /var/log/ceph/ceph-client.admin.26153.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.36726.log.gz 2024-04-25T13:53:14.601 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26738.log 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.26153.log.gz 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38096.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38096.log.gz 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32535.log 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34583.log 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26738.log: /var/log/ceph/ceph-client.admin.32535.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.26738.log.gz 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32535.log.gz 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37985.log 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34583.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34583.log.gz 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37495.log 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42304.log 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37985.log: /var/log/ceph/ceph-client.admin.37495.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37985.log.gz 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37782.log 2024-04-25T13:53:14.602 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37495.log.gz 2024-04-25T13:53:14.610 INFO:teuthology.orchestra.run.smithi087.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.42304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43706.log 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37782.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37782.log.gz 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33849.log 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43706.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43706.log.gz 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25673.log 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33849.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.33849.log.gz 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35046.log 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25673.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.25673.log.gz 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36932.log 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.35046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26790.log 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36932.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36932.log.gz 2024-04-25T13:53:14.615 INFO:teuthology.orchestra.run.smithi080.stderr: 93.7% -- replaced with /var/log/ceph/ceph-client.0.42304.log.gz 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31096.log 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26790.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.26790.log.gz 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40385.log 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31096.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.31096.log.gz 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24895.log 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40385.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40385.log.gz 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25437.log 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24895.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.24895.log.gz 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25803.log 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25437.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.25437.log.gz 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36029.log 2024-04-25T13:53:14.616 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25803.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25803.log.gz 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27485.log 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36029.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36029.log.gz 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31308.log 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27485.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27485.log.gz 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43349.log 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31308.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.31308.log.gz 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44028.log 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43349.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43349.log.gz 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29494.log 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44028.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44028.log.gz 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45155.log 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29494.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.29494.log.gz 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24609.log 2024-04-25T13:53:14.617 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45155.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45155.log.gz 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45129.log 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24609.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.24609.log.gz 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36477.log 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45129.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45129.log.gz 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43284.log 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36477.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36477.log.gz 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23490.log 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43284.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43284.log.gz 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34381.log 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23490.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.23490.log.gz 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35126.log 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34381.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34381.log.gz 2024-04-25T13:53:14.618 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36369.log 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35126.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35126.log.gz 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31044.log 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36369.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36369.log.gz 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24635.log 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31044.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.31044.log.gz 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27908.log 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24635.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.24635.log.gz 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24063.log 2024-04-25T13:53:14.621 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27908.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27908.log.gz 2024-04-25T13:53:14.622 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42803.log 2024-04-25T13:53:14.622 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24063.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.24063.log.gz 2024-04-25T13:53:14.622 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41882.log 2024-04-25T13:53:14.623 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42803.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42803.log.gz 2024-04-25T13:53:14.624 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24687.log 2024-04-25T13:53:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41882.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41882.log.gz 2024-04-25T13:53:14.625 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29229.log 2024-04-25T13:53:14.626 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24687.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.24687.log.gz 2024-04-25T13:53:14.626 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35286.log 2024-04-25T13:53:14.627 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29229.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.29229.log.gz 2024-04-25T13:53:14.628 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41514.log 2024-04-25T13:53:14.629 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35286.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35286.log.gz 2024-04-25T13:53:14.629 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32196.log 2024-04-25T13:53:14.630 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41514.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41514.log.gz 2024-04-25T13:53:14.630 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43011.log 2024-04-25T13:53:14.632 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32196.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32196.log.gz 2024-04-25T13:53:14.632 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39722.log 2024-04-25T13:53:14.633 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43011.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43011.log.gz 2024-04-25T13:53:14.633 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28116.log 2024-04-25T13:53:14.634 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39722.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39722.log.gz 2024-04-25T13:53:14.634 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45207.log 2024-04-25T13:53:14.636 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28116.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.28116.log.gz 2024-04-25T13:53:14.636 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45181.log 2024-04-25T13:53:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45207.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45207.log.gz 2024-04-25T13:53:14.637 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44854.log 2024-04-25T13:53:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45181.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45181.log.gz 2024-04-25T13:53:14.638 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42907.log 2024-04-25T13:53:14.639 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44854.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44854.log.gz 2024-04-25T13:53:14.640 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30606.log 2024-04-25T13:53:14.641 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42907.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42907.log.gz 2024-04-25T13:53:14.641 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45415.log 2024-04-25T13:53:14.642 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30606.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30606.log.gz 2024-04-25T13:53:14.642 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27986.log 2024-04-25T13:53:14.643 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45415.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45415.log.gz 2024-04-25T13:53:14.644 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30061.log 2024-04-25T13:53:14.644 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27986.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27986.log.gz 2024-04-25T13:53:14.645 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32658.log 2024-04-25T13:53:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30061.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30061.log.gz 2024-04-25T13:53:14.646 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36133.log 2024-04-25T13:53:14.647 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32658.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32658.log.gz 2024-04-25T13:53:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25725.log 2024-04-25T13:53:14.648 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36133.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36133.log.gz 2024-04-25T13:53:14.649 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30185.log 2024-04-25T13:53:14.650 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25725.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25725.log.gz 2024-04-25T13:53:14.650 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42120.log 2024-04-25T13:53:14.651 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30185.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30185.log.gz 2024-04-25T13:53:14.651 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36648.log 2024-04-25T13:53:14.652 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42120.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42120.log.gz 2024-04-25T13:53:14.653 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25299.log 2024-04-25T13:53:14.654 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36648.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36648.log.gz 2024-04-25T13:53:14.654 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40212.log 2024-04-25T13:53:14.655 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25299.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.25299.log.gz 2024-04-25T13:53:14.655 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36159.log 2024-04-25T13:53:14.656 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40212.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40212.log.gz 2024-04-25T13:53:14.657 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39951.log 2024-04-25T13:53:14.658 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36159.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36159.log.gz 2024-04-25T13:53:14.658 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32087.log 2024-04-25T13:53:14.659 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39951.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39951.log.gz 2024-04-25T13:53:14.659 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25855.log 2024-04-25T13:53:14.660 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32087.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.32087.log.gz 2024-04-25T13:53:14.661 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41052.log 2024-04-25T13:53:14.662 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25855.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25855.log.gz 2024-04-25T13:53:14.666 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26048.log 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41052.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41052.log.gz 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28734.log 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26048.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26048.log.gz 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28734.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28734.log.gz 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29047.log 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40489.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38841.log 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29047.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.29047.log.gz 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38219.log 2024-04-25T13:53:14.670 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38841.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38841.log.gz 2024-04-25T13:53:14.671 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44489.log 2024-04-25T13:53:14.671 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38219.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38219.log.gz 2024-04-25T13:53:14.672 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-04-25T13:53:14.672 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44489.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44489.log.gz 2024-04-25T13:53:14.673 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43193.log 2024-04-25T13:53:14.674 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42457.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-04-25T13:53:14.674 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24142.log 2024-04-25T13:53:14.675 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43193.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43193.log.gz 2024-04-25T13:53:14.676 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30843.log 2024-04-25T13:53:14.677 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24142.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24142.log.gz 2024-04-25T13:53:14.677 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31716.log 2024-04-25T13:53:14.680 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30843.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30843.log.gz 2024-04-25T13:53:14.680 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23523.log 2024-04-25T13:53:14.680 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31716.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31716.log.gz 2024-04-25T13:53:14.680 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28597.log 2024-04-25T13:53:14.684 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23523.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.23523.log.gz 2024-04-25T13:53:14.684 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39051.log 2024-04-25T13:53:14.684 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28597.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28597.log.gz 2024-04-25T13:53:14.684 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37469.log 2024-04-25T13:53:14.685 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39051.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39051.log.gz 2024-04-25T13:53:14.685 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23588.log 2024-04-25T13:53:14.688 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37469.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37469.log.gz 2024-04-25T13:53:14.688 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33137.log 2024-04-25T13:53:14.688 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.23588.log.gz 2024-04-25T13:53:14.688 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33505.log 2024-04-25T13:53:14.688 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33137.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33137.log.gz 2024-04-25T13:53:14.689 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34109.log 2024-04-25T13:53:14.690 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33505.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33505.log.gz 2024-04-25T13:53:14.690 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44352.log 2024-04-25T13:53:14.691 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34109.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34109.log.gz 2024-04-25T13:53:14.691 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26629.log 2024-04-25T13:53:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44352.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44352.log.gz 2024-04-25T13:53:14.692 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24089.log 2024-04-25T13:53:14.693 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26629.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.26629.log.gz 2024-04-25T13:53:14.693 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45676.log 2024-04-25T13:53:14.694 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24089.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.24089.log.gz 2024-04-25T13:53:14.694 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36421.log 2024-04-25T13:53:14.695 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45676.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45676.log.gz 2024-04-25T13:53:14.695 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T13:53:14.696 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36421.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36421.log.gz 2024-04-25T13:53:14.697 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28943.log 2024-04-25T13:53:14.700 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.log: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T13:53:14.701 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43482.log 2024-04-25T13:53:14.701 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28943.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28943.log.gz 2024-04-25T13:53:14.702 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29576.log 2024-04-25T13:53:14.703 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43482.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43482.log.gz 2024-04-25T13:53:14.703 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39819.log 2024-04-25T13:53:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29576.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.29576.log.gz 2024-04-25T13:53:14.704 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-04-25T13:53:14.705 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39819.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39819.log.gz 2024-04-25T13:53:14.706 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28545.log 2024-04-25T13:53:14.707 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34732.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz 2024-04-25T13:53:14.707 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26550.log 2024-04-25T13:53:14.708 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28545.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28545.log.gz 2024-04-25T13:53:14.708 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25385.log 2024-04-25T13:53:14.709 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26550.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.26550.log.gz 2024-04-25T13:53:14.710 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36219.log 2024-04-25T13:53:14.711 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25385.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25385.log.gz 2024-04-25T13:53:14.711 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33267.log 2024-04-25T13:53:14.712 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36219.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36219.log.gz 2024-04-25T13:53:14.712 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41990.log 2024-04-25T13:53:14.713 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33267.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33267.log.gz 2024-04-25T13:53:14.714 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42384.log 2024-04-25T13:53:14.715 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41990.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41990.log.gz 2024-04-25T13:53:14.715 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24345.log 2024-04-25T13:53:14.716 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42384.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42384.log.gz 2024-04-25T13:53:14.716 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23092.log 2024-04-25T13:53:14.717 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24345.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.24345.log.gz 2024-04-25T13:53:14.718 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31387.log 2024-04-25T13:53:14.719 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23092.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.23092.log.gz 2024-04-25T13:53:14.719 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33583.log 2024-04-25T13:53:14.720 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31387.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31387.log.gz 2024-04-25T13:53:14.720 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24661.log 2024-04-25T13:53:14.721 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33583.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33583.log.gz 2024-04-25T13:53:14.721 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33661.log 2024-04-25T13:53:14.722 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24661.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.24661.log.gz 2024-04-25T13:53:14.723 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29520.log 2024-04-25T13:53:14.724 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33661.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33661.log.gz 2024-04-25T13:53:14.724 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25595.log 2024-04-25T13:53:14.725 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29520.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29520.log.gz 2024-04-25T13:53:14.725 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34758.log 2024-04-25T13:53:14.726 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25595.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.25595.log.gz 2024-04-25T13:53:14.727 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38037.log 2024-04-25T13:53:14.728 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34758.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34758.log.gz 2024-04-25T13:53:14.728 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39618.log 2024-04-25T13:53:14.729 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38037.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38037.log.gz 2024-04-25T13:53:14.729 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31560.log 2024-04-25T13:53:14.730 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39618.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39618.log.gz 2024-04-25T13:53:14.731 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33718.log 2024-04-25T13:53:14.731 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31560.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.31560.log.gz 2024-04-25T13:53:14.732 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44285.log 2024-04-25T13:53:14.733 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33718.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.33718.log.gz 2024-04-25T13:53:14.733 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30739.log 2024-04-25T13:53:14.734 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44285.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44285.log.gz 2024-04-25T13:53:14.734 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24843.log 2024-04-25T13:53:14.736 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30739.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30739.log.gz 2024-04-25T13:53:14.736 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23933.log 2024-04-25T13:53:14.739 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24843.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.24843.log.gz 2024-04-25T13:53:14.739 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40463.log 2024-04-25T13:53:14.739 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23933.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.23933.log.gz 2024-04-25T13:53:14.739 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27882.log 2024-04-25T13:53:14.742 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40463.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40463.log.gz 2024-04-25T13:53:14.743 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32431.log 2024-04-25T13:53:14.743 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27882.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27882.log.gz 2024-04-25T13:53:14.743 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28995.log 2024-04-25T13:53:14.743 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32431.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32431.log.gz 2024-04-25T13:53:14.743 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41131.log 2024-04-25T13:53:14.745 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28995.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28995.log.gz 2024-04-25T13:53:14.745 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40437.log 2024-04-25T13:53:14.745 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41131.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41131.log.gz 2024-04-25T13:53:14.745 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33609.log 2024-04-25T13:53:14.747 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40437.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40437.log.gz 2024-04-25T13:53:14.748 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25515.log 2024-04-25T13:53:14.748 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33609.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33609.log.gz 2024-04-25T13:53:14.748 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44259.log 2024-04-25T13:53:14.749 INFO:teuthology.orchestra.run.smithi087.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T13:53:14.754 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25515.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.25515.log.gz 2024-04-25T13:53:14.754 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44724.log 2024-04-25T13:53:14.754 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44259.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44259.log.gz 2024-04-25T13:53:14.755 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24531.log 2024-04-25T13:53:14.755 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44724.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44724.log.gz 2024-04-25T13:53:14.755 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30471.log 2024-04-25T13:53:14.755 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24531.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.24531.log.gz 2024-04-25T13:53:14.755 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44620.log 2024-04-25T13:53:14.756 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30471.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30471.log.gz 2024-04-25T13:53:14.756 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32632.log 2024-04-25T13:53:14.760 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44620.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44620.log.gz 2024-04-25T13:53:14.760 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25911.log 2024-04-25T13:53:14.761 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32632.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32632.log.gz 2024-04-25T13:53:14.761 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26655.log 2024-04-25T13:53:14.761 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25911.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.25911.log.gz 2024-04-25T13:53:14.761 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26288.log 2024-04-25T13:53:14.761 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26655.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.26655.log.gz 2024-04-25T13:53:14.761 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27934.log 2024-04-25T13:53:14.761 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26288.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.26288.log.gz 2024-04-25T13:53:14.762 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28787.log 2024-04-25T13:53:14.763 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27934.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27934.log.gz 2024-04-25T13:53:14.763 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29663.log 2024-04-25T13:53:14.763 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28787.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28787.log.gz 2024-04-25T13:53:14.764 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27206.log 2024-04-25T13:53:14.764 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29663.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29663.log.gz 2024-04-25T13:53:14.764 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30791.log 2024-04-25T13:53:14.765 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27206.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27206.log.gz 2024-04-25T13:53:14.765 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30895.log 2024-04-25T13:53:14.766 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30791.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.30791.log.gz 2024-04-25T13:53:14.766 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39409.log 2024-04-25T13:53:14.767 INFO:teuthology.orchestra.run.smithi087.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T13:53:14.767 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30895.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30895.log.gz 2024-04-25T13:53:14.767 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41964.log 2024-04-25T13:53:14.768 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39409.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39409.log.gz 2024-04-25T13:53:14.769 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39925.log 2024-04-25T13:53:14.769 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41964.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41964.log.gz 2024-04-25T13:53:14.770 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30275.log 2024-04-25T13:53:14.771 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39925.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39925.log.gz 2024-04-25T13:53:14.771 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24765.log 2024-04-25T13:53:14.772 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30275.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.30275.log.gz 2024-04-25T13:53:14.772 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44541.log 2024-04-25T13:53:14.773 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24765.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.24765.log.gz 2024-04-25T13:53:14.773 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34239.log 2024-04-25T13:53:14.774 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44541.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44541.log.gz 2024-04-25T13:53:14.775 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33345.log 2024-04-25T13:53:14.776 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34239.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34239.log.gz 2024-04-25T13:53:14.776 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38760.log 2024-04-25T13:53:14.777 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33345.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.33345.log.gz 2024-04-25T13:53:14.778 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39077.log 2024-04-25T13:53:14.791 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.38760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34317.log 2024-04-25T13:53:14.793 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39077.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39077.log.gz 2024-04-25T13:53:14.811 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33744.log 2024-04-25T13:53:14.812 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34317.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34317.log.gz 2024-04-25T13:53:14.828 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24139.log 2024-04-25T13:53:14.828 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33744.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33744.log.gz 2024-04-25T13:53:14.835 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29925.log 2024-04-25T13:53:14.836 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24139.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.24139.log.gz 2024-04-25T13:53:14.837 INFO:teuthology.orchestra.run.smithi080.stderr: 95.1% -- replaced with /var/log/ceph/ceph-client.0.38760.log.gz 2024-04-25T13:53:14.837 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27644.log 2024-04-25T13:53:14.838 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29925.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29925.log.gz 2024-04-25T13:53:14.839 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25569.log 2024-04-25T13:53:14.839 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27644.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27644.log.gz 2024-04-25T13:53:14.840 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35776.log 2024-04-25T13:53:14.841 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25569.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.25569.log.gz 2024-04-25T13:53:14.841 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34990.log 2024-04-25T13:53:14.842 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35776.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35776.log.gz 2024-04-25T13:53:14.842 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31018.log 2024-04-25T13:53:14.843 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34990.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34990.log.gz 2024-04-25T13:53:14.843 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30921.log 2024-04-25T13:53:14.844 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31018.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.31018.log.gz 2024-04-25T13:53:14.845 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40943.log 2024-04-25T13:53:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30921.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30921.log.gz 2024-04-25T13:53:14.846 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26946.log 2024-04-25T13:53:14.847 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40943.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40943.log.gz 2024-04-25T13:53:14.847 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.29981.log 2024-04-25T13:53:14.848 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26946.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.26946.log.gz 2024-04-25T13:53:14.848 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25621.log 2024-04-25T13:53:14.867 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.29981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44750.log 2024-04-25T13:53:14.868 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25621.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.25621.log.gz 2024-04-25T13:53:14.882 INFO:teuthology.orchestra.run.smithi087.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T13:53:14.899 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44404.log 2024-04-25T13:53:14.900 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44750.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44750.log.gz 2024-04-25T13:53:14.907 INFO:teuthology.orchestra.run.smithi080.stderr: 95.3% -- replaced with /var/log/ceph/ceph-client.0.35046.log.gz 2024-04-25T13:53:14.907 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37881.log 2024-04-25T13:53:14.908 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44404.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44404.log.gz 2024-04-25T13:53:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37756.log 2024-04-25T13:53:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37881.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37881.log.gz 2024-04-25T13:53:14.909 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30393.log 2024-04-25T13:53:14.910 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37756.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37756.log.gz 2024-04-25T13:53:14.910 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29637.log 2024-04-25T13:53:14.911 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30393.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.30393.log.gz 2024-04-25T13:53:14.912 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37281.log 2024-04-25T13:53:14.912 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29637.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29637.log.gz 2024-04-25T13:53:14.912 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28302.log 2024-04-25T13:53:14.913 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37281.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.37281.log.gz 2024-04-25T13:53:14.914 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43602.log 2024-04-25T13:53:14.914 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28302.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.28302.log.gz 2024-04-25T13:53:14.915 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24333.log 2024-04-25T13:53:14.915 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43602.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43602.log.gz 2024-04-25T13:53:14.916 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31638.log 2024-04-25T13:53:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24333.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.24333.log.gz 2024-04-25T13:53:14.917 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40186.log 2024-04-25T13:53:14.918 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31638.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.31638.log.gz 2024-04-25T13:53:14.918 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25751.log 2024-04-25T13:53:14.919 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40186.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40186.log.gz 2024-04-25T13:53:14.919 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39488.log 2024-04-25T13:53:14.920 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25751.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.25751.log.gz 2024-04-25T13:53:14.920 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30301.log 2024-04-25T13:53:14.921 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39488.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39488.log.gz 2024-04-25T13:53:14.921 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34836.log 2024-04-25T13:53:14.922 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30301.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30301.log.gz 2024-04-25T13:53:14.922 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26868.log 2024-04-25T13:53:14.923 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34836.log: 93.9% -- replaced with /var/log/ceph/ceph-client.0.33451.log.gz 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34836.log.gz 2024-04-25T13:53:14.923 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.924 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37042.log 2024-04-25T13:53:14.924 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31504.log 2024-04-25T13:53:14.924 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.26868.log.gz 2024-04-25T13:53:14.925 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37042.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37042.log.gz 2024-04-25T13:53:14.925 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44378.log 2024-04-25T13:53:14.925 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32710.log 2024-04-25T13:53:14.926 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31504.log: /var/log/ceph/ceph-client.admin.44378.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31504.log.gz 2024-04-25T13:53:14.926 INFO:teuthology.orchestra.run.smithi080.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26075.log 2024-04-25T13:53:14.926 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44378.log.gz 2024-04-25T13:53:14.927 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32710.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32710.log.gz 2024-04-25T13:53:14.928 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37907.log 2024-04-25T13:53:14.928 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31152.log 2024-04-25T13:53:14.928 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.26075.log.gz 2024-04-25T13:53:14.929 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37907.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.37907.log.gz --verbose 2024-04-25T13:53:14.929 INFO:teuthology.orchestra.run.smithi080.stderr: -- /var/log/ceph/ceph-client.admin.33822.log 2024-04-25T13:53:14.929 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31152.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31152.log.gz 2024-04-25T13:53:14.929 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44880.log 2024-04-25T13:53:14.930 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24131.log 2024-04-25T13:53:14.931 INFO:teuthology.orchestra.run.smithi080.stderr: 87.9%/var/log/ceph/ceph-client.admin.44880.log: -- replaced with /var/log/ceph/ceph-client.admin.33822.log.gz 2024-04-25T13:53:14.931 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44880.log.gzgzip 2024-04-25T13:53:14.931 INFO:teuthology.orchestra.run.smithi080.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37521.log 2024-04-25T13:53:14.931 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24131.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.24131.log.gz 2024-04-25T13:53:14.932 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32899.log 2024-04-25T13:53:14.932 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27336.log 2024-04-25T13:53:14.932 INFO:teuthology.orchestra.run.smithi080.stderr: 89.1%/var/log/ceph/ceph-client.admin.32899.log: -- replaced with /var/log/ceph/ceph-client.admin.37521.log.gz 2024-04-25T13:53:14.933 INFO:teuthology.orchestra.run.smithi080.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32899.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40865.log 2024-04-25T13:53:14.933 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:14.934 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27336.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.27336.log.gz 2024-04-25T13:53:14.934 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25326.log 2024-04-25T13:53:14.934 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31530.log 2024-04-25T13:53:14.935 INFO:teuthology.orchestra.run.smithi080.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40865.log.gz 2024-04-25T13:53:14.935 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35750.log 2024-04-25T13:53:14.935 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.25326.log.gz 2024-04-25T13:53:14.936 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31530.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.31530.log.gz 2024-04-25T13:53:14.937 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39793.log 2024-04-25T13:53:14.937 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31334.log 2024-04-25T13:53:14.938 INFO:teuthology.orchestra.run.smithi080.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35750.log.gz 2024-04-25T13:53:14.939 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24011.log 2024-04-25T13:53:14.939 INFO:teuthology.orchestra.run.smithi080.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39793.log.gz 2024-04-25T13:53:14.939 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35906.log 2024-04-25T13:53:14.939 INFO:teuthology.orchestra.run.smithi080.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31334.log.gz 2024-04-25T13:53:14.939 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24011.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.24011.log.gz 2024-04-25T13:53:14.939 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29745.log 2024-04-25T13:53:14.940 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29823.log 2024-04-25T13:53:14.940 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35906.log: /var/log/ceph/ceph-client.admin.29745.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.35906.log.gz 2024-04-25T13:53:14.940 INFO:teuthology.orchestra.run.smithi080.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29745.log.gz 2024-04-25T13:53:14.941 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38063.log 2024-04-25T13:53:14.941 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39155.log 2024-04-25T13:53:14.942 INFO:teuthology.orchestra.run.smithi080.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29823.log.gz/var/log/ceph/ceph-client.admin.38063.log: 2024-04-25T13:53:14.942 INFO:teuthology.orchestra.run.smithi080.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38063.log.gz 2024-04-25T13:53:14.942 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25881.log 2024-04-25T13:53:14.943 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30633.log 2024-04-25T13:53:14.943 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39155.log: /var/log/ceph/ceph-client.admin.25881.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39155.log.gz 2024-04-25T13:53:14.943 INFO:teuthology.orchestra.run.smithi080.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.25881.log.gz 2024-04-25T13:53:14.944 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45519.log 2024-04-25T13:53:14.945 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25699.log 2024-04-25T13:53:14.945 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30633.log: /var/log/ceph/ceph-client.admin.45519.log: 88.2% 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30633.log.gz 2024-04-25T13:53:14.945 INFO:teuthology.orchestra.run.smithi080.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45519.log.gz 2024-04-25T13:53:14.945 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29125.log 2024-04-25T13:53:14.946 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T13:53:14.946 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25699.log: /var/log/ceph/ceph-client.admin.29125.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.25699.log.gz 2024-04-25T13:53:14.946 INFO:teuthology.orchestra.run.smithi080.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29125.log.gz 2024-04-25T13:53:14.946 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31361.log 2024-04-25T13:53:14.947 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23985.log 2024-04-25T13:53:14.947 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.31361.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.31361.log.gz 2024-04-25T13:53:14.950 INFO:teuthology.orchestra.run.smithi080.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T13:53:14.950 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24342.log 2024-04-25T13:53:14.951 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T13:53:14.952 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23985.log: /var/log/ceph/ceph-client.admin.24342.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.24342.log.gz 2024-04-25T13:53:14.952 INFO:teuthology.orchestra.run.smithi080.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.23985.log.gz 2024-04-25T13:53:14.952 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40134.log 2024-04-25T13:53:14.953 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45571.log 2024-04-25T13:53:14.953 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.40134.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40134.log.gz 2024-04-25T13:53:14.968 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32007.log 2024-04-25T13:53:14.968 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45571.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45571.log.gz 2024-04-25T13:53:14.969 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32684.log 2024-04-25T13:53:14.969 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32007.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.32007.log.gz 2024-04-25T13:53:14.970 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35259.log 2024-04-25T13:53:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32684.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32684.log.gz 2024-04-25T13:53:14.971 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33979.log 2024-04-25T13:53:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35259.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35259.log.gz 2024-04-25T13:53:14.972 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29073.log 2024-04-25T13:53:14.973 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33979.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33979.log.gz 2024-04-25T13:53:14.974 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2024-04-25T13:53:14.975 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29073.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.29073.log.gz 2024-04-25T13:53:14.975 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35541.log 2024-04-25T13:53:14.976 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42016.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2024-04-25T13:53:14.976 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25970.log 2024-04-25T13:53:14.977 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35541.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35541.log.gz 2024-04-25T13:53:14.977 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35828.log 2024-04-25T13:53:14.978 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25970.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.25970.log.gz 2024-04-25T13:53:14.978 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T13:53:14.979 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35828.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35828.log.gz 2024-04-25T13:53:14.980 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23907.log 2024-04-25T13:53:14.987 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30159.log 2024-04-25T13:53:14.988 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23907.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.23907.log.gz 2024-04-25T13:53:15.003 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42698.log 2024-04-25T13:53:15.004 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30159.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30159.log.gz 2024-04-25T13:53:15.005 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38390.log 2024-04-25T13:53:15.006 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42698.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42698.log.gz 2024-04-25T13:53:15.023 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43654.log 2024-04-25T13:53:15.024 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38390.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38390.log.gz 2024-04-25T13:53:15.027 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28706.log 2024-04-25T13:53:15.028 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43654.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43654.log.gz 2024-04-25T13:53:15.035 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25996.log 2024-04-25T13:53:15.036 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28706.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28706.log.gz 2024-04-25T13:53:15.040 INFO:teuthology.orchestra.run.smithi080.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T13:53:15.043 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43860.log 2024-04-25T13:53:15.044 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25996.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.25996.log.gz 2024-04-25T13:53:15.045 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42537.log 2024-04-25T13:53:15.045 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43860.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43860.log.gz 2024-04-25T13:53:15.046 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40108.log 2024-04-25T13:53:15.046 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42537.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42537.log.gz 2024-04-25T13:53:15.047 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37098.log 2024-04-25T13:53:15.048 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40108.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40108.log.gz 2024-04-25T13:53:15.049 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45389.log 2024-04-25T13:53:15.049 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37098.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37098.log.gz 2024-04-25T13:53:15.049 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43680.log 2024-04-25T13:53:15.049 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45389.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45389.log.gz 2024-04-25T13:53:15.050 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33189.log 2024-04-25T13:53:15.050 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43680.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43680.log.gz 2024-04-25T13:53:15.051 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28917.log 2024-04-25T13:53:15.051 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33189.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.33189.log.gz 2024-04-25T13:53:15.052 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37834.log 2024-04-25T13:53:15.052 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28917.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.28917.log.gz 2024-04-25T13:53:15.054 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42198.log 2024-04-25T13:53:15.054 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37834.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37834.log.gz 2024-04-25T13:53:15.054 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37216.log 2024-04-25T13:53:15.056 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42198.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.42198.log.gz 2024-04-25T13:53:15.056 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28064.log 2024-04-25T13:53:15.056 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37216.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37216.log.gz 2024-04-25T13:53:15.057 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29875.log 2024-04-25T13:53:15.057 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28064.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.28064.log.gz 2024-04-25T13:53:15.058 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28891.log 2024-04-25T13:53:15.059 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29875.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.29875.log.gz 2024-04-25T13:53:15.059 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34702.log 2024-04-25T13:53:15.060 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28891.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28891.log.gz 2024-04-25T13:53:15.060 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23614.log 2024-04-25T13:53:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34702.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34702.log.gz 2024-04-25T13:53:15.061 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39696.log 2024-04-25T13:53:15.062 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23614.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.23614.log.gz 2024-04-25T13:53:15.062 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29849.log 2024-04-25T13:53:15.063 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39696.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39696.log.gz 2024-04-25T13:53:15.063 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28328.log 2024-04-25T13:53:15.064 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29849.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29849.log.gz 2024-04-25T13:53:15.064 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42094.log 2024-04-25T13:53:15.065 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28328.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28328.log.gz 2024-04-25T13:53:15.065 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40917.log 2024-04-25T13:53:15.066 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42094.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42094.log.gz 2024-04-25T13:53:15.067 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34888.log 2024-04-25T13:53:15.067 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40917.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40917.log.gz 2024-04-25T13:53:15.068 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27310.log 2024-04-25T13:53:15.068 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34888.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34888.log.gz 2024-04-25T13:53:15.070 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41750.log 2024-04-25T13:53:15.070 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27310.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.27310.log.gz 2024-04-25T13:53:15.070 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32035.log 2024-04-25T13:53:15.071 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41750.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41750.log.gz 2024-04-25T13:53:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43063.log 2024-04-25T13:53:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32035.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32035.log.gz 2024-04-25T13:53:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42881.log 2024-04-25T13:53:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43063.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43063.log.gz 2024-04-25T13:53:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26498.log 2024-04-25T13:53:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42881.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42881.log.gz 2024-04-25T13:53:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45029.log 2024-04-25T13:53:15.076 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26498.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26498.log.gz 2024-04-25T13:53:15.077 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35510.log 2024-04-25T13:53:15.077 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45029.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45029.log.gz 2024-04-25T13:53:15.077 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36317.log 2024-04-25T13:53:15.078 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35510.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35510.log.gz 2024-04-25T13:53:15.078 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34914.log 2024-04-25T13:53:15.079 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36317.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36317.log.gz 2024-04-25T13:53:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37933.log 2024-04-25T13:53:15.080 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34914.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34914.log.gz 2024-04-25T13:53:15.081 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28038.log 2024-04-25T13:53:15.082 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37933.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.37933.log.gz 2024-04-25T13:53:15.082 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.25249.log 2024-04-25T13:53:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28038.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28038.log.gz 2024-04-25T13:53:15.083 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30869.log 2024-04-25T13:53:15.085 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.2.25249.log: 91.4% -- replaced with /var/log/ceph/ceph-client.2.25249.log.gz 2024-04-25T13:53:15.085 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43219.log 2024-04-25T13:53:15.086 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30869.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.30869.log.gz 2024-04-25T13:53:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23693.log 2024-04-25T13:53:15.087 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43219.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43219.log.gz 2024-04-25T13:53:15.088 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36185.log 2024-04-25T13:53:15.089 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23693.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.23693.log.gz 2024-04-25T13:53:15.089 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31742.log 2024-04-25T13:53:15.090 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36185.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36185.log.gz 2024-04-25T13:53:15.090 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29350.log 2024-04-25T13:53:15.092 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31742.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.31742.log.gz 2024-04-25T13:53:15.092 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38814.log 2024-04-25T13:53:15.093 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29350.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29350.log.gz 2024-04-25T13:53:15.093 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37334.log 2024-04-25T13:53:15.094 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38814.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38814.log.gz 2024-04-25T13:53:15.094 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39847.log 2024-04-25T13:53:15.095 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37334.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37334.log.gz 2024-04-25T13:53:15.096 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36512.log 2024-04-25T13:53:15.096 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39847.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39847.log.gz 2024-04-25T13:53:15.097 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27960.log 2024-04-25T13:53:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36512.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36512.log.gz 2024-04-25T13:53:15.098 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43571.log 2024-04-25T13:53:15.099 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27960.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.27960.log.gz 2024-04-25T13:53:15.099 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35724.log 2024-04-25T13:53:15.100 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43571.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43571.log.gz 2024-04-25T13:53:15.100 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36003.log 2024-04-25T13:53:15.101 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35724.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35724.log.gz 2024-04-25T13:53:15.102 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27368.log 2024-04-25T13:53:15.102 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36003.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36003.log.gz 2024-04-25T13:53:15.103 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36778.log 2024-04-25T13:53:15.104 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27368.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27368.log.gz 2024-04-25T13:53:15.104 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2024-04-25T13:53:15.105 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36778.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36778.log.gz 2024-04-25T13:53:15.105 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30817.log 2024-04-25T13:53:15.106 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42985.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2024-04-25T13:53:15.107 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34083.log 2024-04-25T13:53:15.107 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30817.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.30817.log.gz 2024-04-25T13:53:15.108 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26444.log 2024-04-25T13:53:15.109 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34083.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.34083.log.gz 2024-04-25T13:53:15.109 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41209.log 2024-04-25T13:53:15.110 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26444.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26444.log.gz 2024-04-25T13:53:15.110 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43758.log 2024-04-25T13:53:15.111 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41209.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.41209.log.gz 2024-04-25T13:53:15.111 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39514.log 2024-04-25T13:53:15.112 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43758.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43758.log.gz 2024-04-25T13:53:15.113 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31848.log 2024-04-25T13:53:15.114 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39514.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.39514.log.gz 2024-04-25T13:53:15.114 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43810.log 2024-04-25T13:53:15.131 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.0.31848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30528.log 2024-04-25T13:53:15.133 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43810.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.43810.log.gz 2024-04-25T13:53:15.152 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32222.log 2024-04-25T13:53:15.152 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30528.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30528.log.gz 2024-04-25T13:53:15.159 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31928.log 2024-04-25T13:53:15.160 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32222.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32222.log.gz 2024-04-25T13:53:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44977.log 2024-04-25T13:53:15.168 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31928.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.31928.log.gz 2024-04-25T13:53:15.172 INFO:teuthology.orchestra.run.smithi080.stderr: 95.4% -- replaced with /var/log/ceph/ceph-client.0.29981.log.gz 2024-04-25T13:53:15.173 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30580.log 2024-04-25T13:53:15.173 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44977.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44977.log.gz 2024-04-25T13:53:15.174 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32951.log 2024-04-25T13:53:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30580.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30580.log.gz 2024-04-25T13:53:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39195.log 2024-04-25T13:53:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32951.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32951.log.gz 2024-04-25T13:53:15.176 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42594.log 2024-04-25T13:53:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39195.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.39195.log.gz 2024-04-25T13:53:15.177 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33241.log 2024-04-25T13:53:15.178 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42594.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42594.log.gz 2024-04-25T13:53:15.178 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41938.log 2024-04-25T13:53:15.179 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33241.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33241.log.gz 2024-04-25T13:53:15.179 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40160.log 2024-04-25T13:53:15.180 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41938.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41938.log.gz 2024-04-25T13:53:15.180 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38628.log 2024-04-25T13:53:15.181 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40160.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40160.log.gz 2024-04-25T13:53:15.181 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25777.log 2024-04-25T13:53:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38628.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38628.log.gz 2024-04-25T13:53:15.182 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34415.log 2024-04-25T13:53:15.183 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25777.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.25777.log.gz 2024-04-25T13:53:15.183 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35222.log 2024-04-25T13:53:15.184 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34415.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.34415.log.gz 2024-04-25T13:53:15.184 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45545.log 2024-04-25T13:53:15.185 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35222.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35222.log.gz 2024-04-25T13:53:15.185 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43970.log 2024-04-25T13:53:15.186 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45545.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45545.log.gz 2024-04-25T13:53:15.186 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33059.log 2024-04-25T13:53:15.187 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43970.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43970.log.gz 2024-04-25T13:53:15.187 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31664.log 2024-04-25T13:53:15.188 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33059.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33059.log.gz 2024-04-25T13:53:15.188 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25647.log 2024-04-25T13:53:15.189 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31664.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.31664.log.gz 2024-04-25T13:53:15.189 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39247.log 2024-04-25T13:53:15.190 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25647.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.25647.log.gz 2024-04-25T13:53:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36752.log 2024-04-25T13:53:15.191 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39247.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39247.log.gz 2024-04-25T13:53:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29442.log 2024-04-25T13:53:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36752.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36752.log.gz 2024-04-25T13:53:15.192 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29318.log 2024-04-25T13:53:15.193 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29442.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29442.log.gz 2024-04-25T13:53:15.194 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43998.log 2024-04-25T13:53:15.195 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29318.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29318.log.gz 2024-04-25T13:53:15.195 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44456.log 2024-04-25T13:53:15.196 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.43998.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43998.log.gz 2024-04-25T13:53:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41261.log 2024-04-25T13:53:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44456.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44456.log.gz 2024-04-25T13:53:15.197 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36830.log 2024-04-25T13:53:15.198 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41261.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41261.log.gz 2024-04-25T13:53:15.199 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38654.log 2024-04-25T13:53:15.200 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36830.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36830.log.gz 2024-04-25T13:53:15.200 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23855.log 2024-04-25T13:53:15.201 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38654.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38654.log.gz 2024-04-25T13:53:15.202 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35166.log 2024-04-25T13:53:15.202 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.23855.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.23855.log.gz 2024-04-25T13:53:15.203 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32249.log 2024-04-25T13:53:15.204 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35166.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.35166.log.gz 2024-04-25T13:53:15.204 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25359.log 2024-04-25T13:53:15.205 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32249.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.32249.log.gz 2024-04-25T13:53:15.205 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38704.log 2024-04-25T13:53:15.206 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.25359.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.25359.log.gz 2024-04-25T13:53:15.206 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44951.log 2024-04-25T13:53:15.207 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38704.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38704.log.gz 2024-04-25T13:53:15.208 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42248.log 2024-04-25T13:53:15.209 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44951.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44951.log.gz 2024-04-25T13:53:15.209 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42042.log 2024-04-25T13:53:15.210 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42248.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-04-25T13:53:15.210 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35567.log 2024-04-25T13:53:15.211 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42042.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42042.log.gz 2024-04-25T13:53:15.211 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42146.log 2024-04-25T13:53:15.212 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35567.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.35567.log.gz 2024-04-25T13:53:15.213 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27232.log 2024-04-25T13:53:15.214 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42146.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42146.log.gz 2024-04-25T13:53:15.214 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41183.log 2024-04-25T13:53:15.215 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27232.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.27232.log.gz 2024-04-25T13:53:15.215 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26340.log 2024-04-25T13:53:15.216 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41183.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41183.log.gz 2024-04-25T13:53:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30222.log 2024-04-25T13:53:15.217 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26340.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.26340.log.gz 2024-04-25T13:53:15.218 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30554.log 2024-04-25T13:53:15.219 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30222.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30222.log.gz 2024-04-25T13:53:15.219 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29797.log 2024-04-25T13:53:15.220 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30554.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30554.log.gz 2024-04-25T13:53:15.220 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26816.log 2024-04-25T13:53:15.221 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.29797.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29797.log.gz 2024-04-25T13:53:15.222 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38245.log 2024-04-25T13:53:15.223 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26816.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.26816.log.gz 2024-04-25T13:53:15.223 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44117.log 2024-04-25T13:53:15.224 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38245.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38245.log.gz 2024-04-25T13:53:15.225 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26576.log 2024-04-25T13:53:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44117.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44117.log.gz 2024-04-25T13:53:15.226 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41000.log 2024-04-25T13:53:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26576.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26576.log.gz 2024-04-25T13:53:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32483.log 2024-04-25T13:53:15.228 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41000.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41000.log.gz 2024-04-25T13:53:15.229 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40411.log 2024-04-25T13:53:15.229 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32483.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32483.log.gz 2024-04-25T13:53:15.229 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26524.log 2024-04-25T13:53:15.231 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40411.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40411.log.gz 2024-04-25T13:53:15.231 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40515.log 2024-04-25T13:53:15.232 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26524.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.26524.log.gz 2024-04-25T13:53:15.233 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33927.log 2024-04-25T13:53:15.233 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40515.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40515.log.gz 2024-04-25T13:53:15.233 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35854.log 2024-04-25T13:53:15.234 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33927.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33927.log.gz 2024-04-25T13:53:15.235 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44906.log 2024-04-25T13:53:15.236 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35854.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35854.log.gz 2024-04-25T13:53:15.236 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27180.log 2024-04-25T13:53:15.237 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44906.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44906.log.gz 2024-04-25T13:53:15.237 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37443.log 2024-04-25T13:53:15.237 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27180.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.27180.log.gz 2024-04-25T13:53:15.238 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-04-25T13:53:15.239 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37443.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.37443.log.gz 2024-04-25T13:53:15.239 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28865.log 2024-04-25T13:53:15.240 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40891.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-04-25T13:53:15.240 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26022.log 2024-04-25T13:53:15.241 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28865.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28865.log.gz 2024-04-25T13:53:15.241 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30419.log 2024-04-25T13:53:15.242 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26022.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.26022.log.gz 2024-04-25T13:53:15.243 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32833.log 2024-04-25T13:53:15.244 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30419.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30419.log.gz 2024-04-25T13:53:15.244 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36055.log 2024-04-25T13:53:15.245 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32833.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32833.log.gz 2024-04-25T13:53:15.245 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44172.log 2024-04-25T13:53:15.246 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36055.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36055.log.gz 2024-04-25T13:53:15.246 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39566.log 2024-04-25T13:53:15.247 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44172.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44172.log.gz 2024-04-25T13:53:15.248 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33531.log 2024-04-25T13:53:15.248 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.39566.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.39566.log.gz 2024-04-25T13:53:15.249 INFO:teuthology.orchestra.run.smithi087.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T13:53:15.249 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45363.log 2024-04-25T13:53:15.250 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33531.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33531.log.gz 2024-04-25T13:53:15.250 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27459.log 2024-04-25T13:53:15.252 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45363.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45363.log.gz 2024-04-25T13:53:15.252 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27102.log 2024-04-25T13:53:15.253 INFO:teuthology.orchestra.run.smithi087.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T13:53:15.254 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27459.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.27459.log.gz 2024-04-25T13:53:15.254 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32580.log 2024-04-25T13:53:15.254 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27102.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27102.log.gz 2024-04-25T13:53:15.254 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32353.log 2024-04-25T13:53:15.255 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32580.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32580.log.gz 2024-04-25T13:53:15.255 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32405.log 2024-04-25T13:53:15.256 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32353.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.32353.log.gz 2024-04-25T13:53:15.256 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41287.log 2024-04-25T13:53:15.257 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32405.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32405.log.gz 2024-04-25T13:53:15.258 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40839.log 2024-04-25T13:53:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41287.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41287.log.gz 2024-04-25T13:53:15.259 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31070.log 2024-04-25T13:53:15.260 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40839.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40839.log.gz 2024-04-25T13:53:15.260 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36566.log 2024-04-25T13:53:15.261 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31070.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.31070.log.gz 2024-04-25T13:53:15.261 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36540.log 2024-04-25T13:53:15.262 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36566.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.36566.log.gz 2024-04-25T13:53:15.263 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35593.log 2024-04-25T13:53:15.264 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36540.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36540.log.gz 2024-04-25T13:53:15.264 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26998.log 2024-04-25T13:53:15.265 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35593.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35593.log.gz 2024-04-25T13:53:15.265 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42959.log 2024-04-25T13:53:15.266 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26998.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.26998.log.gz 2024-04-25T13:53:15.267 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30502.log 2024-04-25T13:53:15.268 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42959.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42959.log.gz 2024-04-25T13:53:15.268 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24713.log 2024-04-25T13:53:15.269 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.30502.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.30502.log.gz 2024-04-25T13:53:15.269 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26179.log 2024-04-25T13:53:15.271 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24713.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.24713.log.gz 2024-04-25T13:53:15.271 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32170.log 2024-04-25T13:53:15.273 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.26179.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26179.log.gz 2024-04-25T13:53:15.273 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40359.log 2024-04-25T13:53:15.274 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32170.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32170.log.gz 2024-04-25T13:53:15.274 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31415.log 2024-04-25T13:53:15.288 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40359.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40359.log.gz 2024-04-25T13:53:15.289 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36804.log 2024-04-25T13:53:15.289 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.31415.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.31415.log.gz 2024-04-25T13:53:15.289 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33085.log 2024-04-25T13:53:15.289 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.36804.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36804.log.gz 2024-04-25T13:53:15.289 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45441.log 2024-04-25T13:53:15.289 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.33085.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33085.log.gz 2024-04-25T13:53:15.290 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24037.log 2024-04-25T13:53:15.290 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.45441.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45441.log.gz 2024-04-25T13:53:15.290 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32275.log 2024-04-25T13:53:15.290 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24037.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.24037.log.gz 2024-04-25T13:53:15.290 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41618.log 2024-04-25T13:53:15.290 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32275.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32275.log.gz 2024-04-25T13:53:15.291 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34187.log 2024-04-25T13:53:15.291 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41618.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41618.log.gz 2024-04-25T13:53:15.291 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41488.log 2024-04-25T13:53:15.291 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.34187.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.34187.log.gz 2024-04-25T13:53:15.291 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28354.log 2024-04-25T13:53:15.291 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.41488.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41488.log.gz 2024-04-25T13:53:15.292 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44593.log 2024-04-25T13:53:15.292 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.28354.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28354.log.gz 2024-04-25T13:53:15.292 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38472.log 2024-04-25T13:53:15.292 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44593.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44593.log.gz 2024-04-25T13:53:15.292 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42672.log 2024-04-25T13:53:15.293 INFO:teuthology.orchestra.run.smithi087.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T13:53:15.300 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.38472.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38472.log.gz 2024-04-25T13:53:15.301 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27670.log 2024-04-25T13:53:15.301 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.42672.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42672.log.gz 2024-04-25T13:53:15.301 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24143.log 2024-04-25T13:53:15.301 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.27670.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.27670.log.gz 2024-04-25T13:53:15.301 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44233.log 2024-04-25T13:53:15.301 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.24143.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.24143.log.gz 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40240.log 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.44233.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44233.log.gz 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32606.log 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.40240.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40240.log.gz 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37959.log 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.32606.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32606.log.gz 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35880.log 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.37959.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37959.log.gz 2024-04-25T13:53:15.302 INFO:teuthology.orchestra.run.smithi080.stderr:/var/log/ceph/ceph-client.admin.35880.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35880.log.gz 2024-04-25T13:53:15.325 INFO:teuthology.orchestra.run.smithi087.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T13:53:15.326 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-04-25T13:53:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:real 0m0.920s 2024-04-25T13:53:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:user 0m4.495s 2024-04-25T13:53:15.326 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m0.199s 2024-04-25T13:53:15.341 INFO:teuthology.orchestra.run.smithi080.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T13:53:15.408 INFO:teuthology.orchestra.run.smithi080.stderr: 94.6% -- replaced with /var/log/ceph/ceph-client.0.31848.log.gz 2024-04-25T13:53:15.478 INFO:teuthology.orchestra.run.smithi080.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T13:53:15.531 INFO:teuthology.orchestra.run.smithi080.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T13:53:15.533 INFO:teuthology.orchestra.run.smithi080.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T13:53:15.586 INFO:teuthology.orchestra.run.smithi080.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T13:53:15.648 INFO:teuthology.orchestra.run.smithi080.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T13:53:15.649 INFO:teuthology.orchestra.run.smithi080.stderr: 2024-04-25T13:53:15.649 INFO:teuthology.orchestra.run.smithi080.stderr:real 0m1.287s 2024-04-25T13:53:15.649 INFO:teuthology.orchestra.run.smithi080.stderr:user 0m7.775s 2024-04-25T13:53:15.650 INFO:teuthology.orchestra.run.smithi080.stderr:sys 0m0.522s 2024-04-25T13:53:15.650 INFO:tasks.ceph:Archiving logs... 2024-04-25T13:53:15.651 DEBUG:teuthology.misc:Transferring archived files from smithi080:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/remote/smithi080/log 2024-04-25T13:53:15.652 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:53:17.049 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/remote/smithi087/log 2024-04-25T13:53:17.050 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T13:53:17.549 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T13:53:17.559 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T13:53:17.559 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:53:17.562 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T13:53:17.640 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T13:53:17.650 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T13:53:17.651 DEBUG:teuthology.orchestra.run.smithi080:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:53:17.653 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T13:53:17.661 INFO:teuthology.orchestra.run.smithi080.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T13:53:17.661 INFO:teuthology.orchestra.run.smithi080.stdout:============================================================================== 2024-04-25T13:53:17.662 INFO:teuthology.orchestra.run.smithi080.stdout:+hv01.front.sepi 67.205.162.81 3 u 12 64 377 0.096 -3.404 3.453 2024-04-25T13:53:17.662 INFO:teuthology.orchestra.run.smithi080.stdout:+hv02.front.sepi 63.231.80.2 3 u 6 64 377 0.100 -0.768 3.432 2024-04-25T13:53:17.662 INFO:teuthology.orchestra.run.smithi080.stdout:*hv03.front.sepi 74.6.168.72 3 u 6 64 377 0.126 -4.034 2.717 2024-04-25T13:53:17.662 INFO:teuthology.orchestra.run.smithi080.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T13:53:17.665 INFO:teuthology.orchestra.run.smithi087.stdout: remote refid st t when poll reach delay offset jitter 2024-04-25T13:53:17.665 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================== 2024-04-25T13:53:17.666 INFO:teuthology.orchestra.run.smithi087.stdout:+hv01.front.sepi 67.205.162.81 3 u 66 64 377 0.106 -3.025 2.022 2024-04-25T13:53:17.666 INFO:teuthology.orchestra.run.smithi087.stdout:+hv02.front.sepi 63.231.80.2 3 u 5 64 377 0.074 -0.283 2.434 2024-04-25T13:53:17.666 INFO:teuthology.orchestra.run.smithi087.stdout:*hv03.front.sepi 74.6.168.72 3 u 7 64 377 0.088 -1.897 3.123 2024-04-25T13:53:17.666 INFO:teuthology.orchestra.run.smithi087.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-04-25T13:53:17.667 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T13:53:17.677 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T13:53:17.677 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T13:53:17.687 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T13:53:17.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T13:53:17.707 INFO:teuthology.task.internal:Duration was 1420.337700 seconds 2024-04-25T13:53:17.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T13:53:17.717 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T13:53:17.717 DEBUG:teuthology.orchestra.run.smithi080:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:53:17.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T13:53:17.755 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T13:53:17.756 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi080.front.sepia.ceph.com 2024-04-25T13:53:17.756 DEBUG:teuthology.orchestra.run.smithi080:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:53:17.805 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-04-25T13:53:17.805 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T13:53:17.814 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T13:53:17.815 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:53:17.849 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T13:53:17.887 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T13:53:17.887 DEBUG:teuthology.orchestra.run.smithi080:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:53:17.921 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T13:53:17.993 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T13:53:18.003 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T13:53:18.003 DEBUG:teuthology.orchestra.run.smithi080:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:53:18.016 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T13:53:18.028 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T13:53:18.039 DEBUG:teuthology.orchestra.run.smithi080:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:53:18.060 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:53:18.068 INFO:teuthology.orchestra.run.smithi080.stdout:kernel.core_pattern = core 2024-04-25T13:53:18.080 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-04-25T13:53:18.092 DEBUG:teuthology.orchestra.run.smithi080:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:53:18.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:53:18.127 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T13:53:18.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T13:53:18.140 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T13:53:18.153 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T13:53:18.154 DEBUG:teuthology.misc:Transferring archived files from smithi080:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/remote/smithi080 2024-04-25T13:53:18.154 DEBUG:teuthology.orchestra.run.smithi080:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:53:18.193 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-04-25_10:00:23-fs-wip-pdonnell-testing-20240425.015853-debug-distro-default-smithi/7673174/remote/smithi087 2024-04-25T13:53:18.193 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T13:53:18.213 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T13:53:18.213 DEBUG:teuthology.orchestra.run.smithi080:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:53:18.233 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T13:53:18.258 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T13:53:18.269 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T13:53:18.269 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T13:53:18.280 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T13:53:18.280 DEBUG:teuthology.orchestra.run.smithi080:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:53:18.283 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T13:53:18.286 INFO:teuthology.orchestra.run.smithi080.stdout: 658639 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:53 /home/ubuntu/cephtest 2024-04-25T13:53:18.309 INFO:teuthology.orchestra.run.smithi087.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 13:53 /home/ubuntu/cephtest 2024-04-25T13:53:18.310 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T13:53:18.322 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T13:53:18.351 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/mds-full} duration: 1420.3376998901367 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-04-25T13:53:18.352 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T13:53:18.428 INFO:teuthology.run:pass